The Y-2011 bug

Because of accidentally typing "2010" on a few posts that were written in advance, today's post and a few others won't show up correctly in Reader or Twitter. Some people will have seen these posts, but some not.  Today's I think is particularly worth reading, so check it out here.

So maybe reading *should* be harder

Some weeks back I chided Jonah Lehrer for his assertion that he'd

love [e-readers] to include a feature that allows us to undo their ease, to make the act of reading just a little bit more difficult. Perhaps we need to alter the fonts, or reduce the contrast, or invert the monochrome color scheme. Our eyes will need to struggle, and we’ll certainly read slower, but that’s the point: Only then will we process the text a little less unconsciously, with less reliance on the ventral pathway. We won’t just scan the words – we will contemplate their meaning.

This sounded like a bunch of neuro-babble to me, partly because the research he cited seemed to be about something else entirely.

Obviously, the ventral pathway is the problem.

Spoke too soon


To the rescue come Diemand-Yauman, Oppenheimer & Vaughan, who just published a new paper in my favorite journal, Cognition. The abstract says it all:

Previous research has shown that disfluency -- the subjective experience of difficulty associated with cognitive operations -- leads to deeper processing. Two studies explore the extent to which this deeper processing engendered by disfluency interventions can lead to improved memory performance. Study 1 found that information in hard-to-read fonts was better remembered that easier to read information in a controlled laboratory setting. Study 2 extended this finding to high school classrooms. The results suggest that superficial changes to learning materials could yield significant improvements in educational outcomes.

The first experiment involved remembering 21 pieces of information over a 15-minute interval, which while promising, has it's limitations. Here are the authors:

There are a number of reasons why this result might not generalize to actual classroom environments. First, while the effects persisted for 15 min, the time between learning and testing is typically much longer in school settings. Moreover, there are a large number of other substantive differences between the lab and actual classrooms, including the nature of materials, the learning strategies adopted, and the presence of distractions in the environment... Another concern is that because disfluent reading is, by definition, perceived as more difficult, less motivated students may become frustrated. While paid laboratory participants are willing to persist in the face of challenging fonts for 90 s, the increase in perceived difficulty may provide motivational barriers for actual students.

Or it could just make the students bored.

In a second, truly heroic study, the researchers talked a bunch of teachers at a public high school into sending them all their classroom worksheets and powerpoint slides. The researchers recreated two versions of these materials: one in an easy-to-read font and one in a difficult-to-read font. Each of the teachers taught at least two sections of the same course, so they were able to use one set of materials with one group of students and the other set with the another group. The classes included English, Physics, Chemistry and History.

Once again, the researchers found better learning with the hard-to-read fonts.

Notes and Caveats


The researchers seem open to a number of possibilities as to why hard-to-read fonts would lead to better learning:

It is worth noting that it is not the difficulty, per se, that leads to improvements in learning but rather the fact that the intervention engages processes that support learning.

Moreover, unlike Lehrer, they don't recommend making everything harder to read, learn or do:

Not all difficulties are desirable, and presumably interventions that engage more elaborative processes without also increasing difficulty would be even more effective at improving educational outcomes.

There is one obvious concern one might have about their Experiment 2: the teachers were blind to hypothesis, but not to condition. The authors attempt to wave this away but asserting that the teachers would likely make the wrong hypothesis (that learning should be worse when the font is hard), and thus any "experimenter" bias would be in the wrong direction. However, we have no way of knowing whether the teachers attempted to compensate for the hard-to-read materials by explaining thing better. In fact, the authors had no way of testing whether the teachers behaved similarly in both conditions.

That's not at all saying I think it was a bad study or shouldn't have been published. I think it's a fantastic study. I don't know how they roped those teachers into the project, but this is the kind of go-get-it science people should be practicing. The study isn't perfect or conclusive, but no studies are. The goal is simply to have results that are clear enough that they generate more research and new hypotheses.


-------
Connor Diemand-Yauman, Daniel M. Oppenheimer, and Erikka B. Vaughan (2011). Fortune favors the bold (and the italicized): Effects of disfluency on educational outcomes Cognition, 118, 111-115 : doi:10.1016/j.cognition.2010.09.012

missing 2

One of the formulas in the last post was missing a 2. Everything has now been recalculated. Some numbers changed. The basic result is that some of the numbers are not quite as dire as I had stated: the original example experiment, which had 15 participants per condition and an effect significant at p=.05 has a 51% chance of replicating (in the sense of producing another significant p-value when re-run exactly), again assuming the effect was real and the effect size is as measured in the first experiment.

Revetro responds

Earlier this month I blogged about a study supposedly produced by Revetro on texting during sex. The main point of the post was that researchers have to be careful about ensuring data quality (e.g., are the participants actually paying attention to the questions?). I also remarked that I had been unable to find the original article.

Jennifer Jacobson at Revetro very kindly emailed me several days ago in order to point me to the original study. The survey question under discussion can be found in the "We interrupt this dinner" section.

So at least the study exists. I'm hoping to find out more about the methods they used.

A Poorly-edited Editors' Handbook

Most psychology journals require that papers follow the American Psychological Association's style guide. This guidebook covers everything from the structure of the paper to the right way of formatting section headings, and it is updated every so often.

The sixth edition was released over the summer, and it seems it had to be recalled due to "errors and inconsistencies."

I haven't actually seen the 6th edition myself (I just bought the 5th edition a couple years ago and am not in a hurry to buy the new one). On the whole, it's a good manual and the rules make sense. However, reviewers will sometimes thank you for breaking the more frustrating rules , like the rule that charts and tables should be appended to the end of the manuscript -- not included in the document itself. This probably made sense in the day of type-written manuscripts, but makes modern electronic manuscripts very hard to read. Electronic documents are wonderful for many things, but the ease of flipping back and forth from one section to another is not one of them.

Hopefully the 6th edition fixed some of those out-dated rules. But I'll wait to find out once the fixed version appears.