The verbal memory hegemony

One fact about the world is that the most famous memory researchers did most of their work on verbal memory. Alan Baddeley and George Miller both come to mind -- and I doubt anybody can think of more famous memory researchers in the last 50 years.

Another fact about the world is that many researchers -- not necessarily Baddeley or Miller -- have assumed that anything discovered using memory tests involving words should apply to other forms of memory as well. To pick unfairly on one person, Cowan notes in his masterful paper "The magical number 4 in short-term memory" that out of several related experiments, one has results that diverge from the others. Cowan attempts an explanation but basically throws up his hands. He doesn't notice that of all the experiments discussed in that section, the divergent one was the only one to use visual rather than verbal stimuli.

Similarly, a reviewer of my paper which just came out complained that the results reported in the paper only "told us things we already knew." As evidence, the reviewer cited a number of other papers, all of which had investigated verbal rather than visual short-term memory.

As it happens, the results in this case were very similar to what had been reported previously for verbal memory. But it could have come out differently. That was the point of doing the experiment.

Partly because of this bias in favor of verbal materials, not enough is known about visual memory, though this has been changing in recent years, thanks in part to folks like Steve Luck, George Alvarez, Yuhong Jiang, Edward Vogel and several others.

----------

Cowan, N. (2001). The magical number 4 in short-term memory: a reconsideration of mental storage capacity. Behavioral and brain sciences, 24, 87-185.

Hartshorne, J.K. (2008). Visual working memory capacity and proactive interference. Public Library of Science One

Miller, G.A. (1956). The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychological Review, 63, 81-97.

A scientist at work: Street corner surveying

As much as I love Web-based experiments, they aren't ideal in all situations. Currently, I have a series of short surveys, each of which requires 20 or so participants. When I say "series of short surveys," that doesn't mean I have them all in advance. The results of each survey dictate what the next survey will be.

This is hard to run online, because it means I need time between surveys to analyze the results and think up a new experiment. On the Web, it's hard to take a timeout.

Instead, I took on a new research assistant, made a sign, and bought a bunch of candy. Then I set up shop outside the Harvard Science Center and began giving away candy in exchange for participation in the surveys.

At first, it was awful. We got lots of tight-lipped smiles, but nobody stopped or even really made eye-contact. I thought, "Why did I think this was a good idea? I hate this sort of thing." Within about 10 minutes, though, we got into a groove and have been collecting data at a pretty good clip every time we go out.

It turns out that, other than feeling like a canvaser, it's a fun way of collecting data. You get to be outdoors and away from the computer. You get to actually interact with people. And the pace of the research is if anything even faster than Web-based research. We typically average 30 or so participants per hour. The Moral Sense Test gets that kind of traffic; The Cognition and Language Lab, unfortunately, does not. This is probably not unrelated to the 392 appearances of "Marc Hauser" in the New York Times archives, compared with the single appearance for "Joshua Hartshorne." (Journalists: if you are reading this, call me!)

Street corner surveying is an old method. Many people seem to believe it is more reliable than Web-based surveying. Why is beyond me. We are stopping busy people with other things on their minds. Many just want candy. We are in a busy, noisy area with tours passing by, camera bulbs flashing and the occasional demonstration. And on Tuesdays, there is a farmer's market in the same area.

Sometimes the responses are hard to explain. One control question reads along the lines of: "John has two children. How likely do you think it is that he has two children? How likely do you think it is he has three children?" More than a few people agree that it is more likely that John has three children than that he has two. One person carefully corrected the grammar on one page, which was a neighborly thing to do, except that the grammar on that page was actually right, and the "corrections" made it wrong.

When collecting data from humans, there is always noise.

Should we trust experiments on the Web?

When I first started doing Web-based experiments, a number of people in my own lab were skeptical as to whether I would get anything valuable out of them. Part of this was due to worries about method (How do you know the participants are paying attention? How do you know they are telling the truth?), but I think part of it was also a suspicion of the Internet in general, which, as we all know, is full of an awful lot of crap.

For this reason, I expected some difficulties getting my Web-based studies published. However, the first of these studies was accepted without much drama, and what concerns the reviewers did raise had nothing to do with the Web (granted that only one of the experiments in that paper was run online). Similarly, while the second study (run in collaboration with Tal Makovski) has run into some significant hurdles in getting published, none of them involved the fact that the experiments were all run online.

Until now. After major revisions and some new experiments, we submitted the paper to a new journal where we thought it would be well-received. Unfortunately, it was not, and many of the concerns involved the Web. Two of the reviewers clearly articulated that they just don't trust Web-based experiments. One went so far as to say that Web-based experiments should never be run unless there is absolutely no way to do the experiment in the lab.

(I would use direct quotes, but the reviewers certainly did not expect their comments to show up on a blog, anonymously or not. So you will have to take my word for it.)

Obviously, I trust Web-based experiments. I have written enough posts about why I think concerns are misguided, so I won't rehash that here. I am more interested in why exactly people have trouble with Web-based experiments as opposed to other methodologies.

Is it because the Web-based method is relatively new? Is it because the Internet is full of porn? Or is it simply the case that for any given method, there are a certain number of people who just don't trust it?

I have been doing street-corner surveying lately (a well-established method), and I can tell you that although it ultimately gives decent results, some very odd things happen along the way. But I suppose if, as a reviewer, I tried to reject a paper because I "just don't trust surveys," the action editor would override me.

Results from an Experiment: The Time Course of Visual Short-Term Memory

The first experiment I ran on the Web has finally made it into print. Rather fittingly, it has been published in a Web-based journal: The Public Library of Science One.

Visual Memory is a Scrawny Creature

That experiment, The Time Course of Visual Short-Term Memory, was part of a larger study probing a fundamental question about memory: why is visual working (short-term) memory so lousy? In recent years, visual memory folk like Edward Vogel and George Alvarez have debated whether we can store as many as four items in visual memory, while on the other hand researchers looking more at verbal memory, such as Nelson Cowan, have been arguing over whether verbal memory can store only four items. There are memory tricks that can allow you to keep a hundred words in short-term memory; nobody has reported any similar tricks for visual memory.

There are many other ways in which visual memory is piddly compared to verbal memory, and I go into them in depth in the paper. Interestingly, previous researchers have not made much out of this difference, possibly because people seem to work on either visual memory or verbal memory, but not both.

Does Verbal Memory Explain the Differences between Humans and Apes?

One possibility that occurred to me is that if verbal memory in fact is considerably more robust and more useful than visual memory, that would endow verbal animals (i.e., adult humans) with significant advantages over non-verbal animals (e.g., human infants and all other animals). Just as writing has allowed some human cultures to supplement our limited memory capacity -- try doing a complicated math problem in your head; the real limitation is memory -- language could allow us to supplement limited non-verbal memory systems.

In fact, I found that many of the differences between adult humans on the one side and young children and apes on the other are found in tasks with large working memory demands. More examples are given in the paper, but this includes theory of mind tasks.

Is Verbal Memory Really Better?

Of course, this is fruitless speculation unless visual working memory is really inferior. The problem is that visual and verbal memory capacity is tested in somewhat different ways. The easiest way to test verbal memory capacity is to give people a list of words to remember and then ask them to repeat that list back (this forms an important part of many IQ tests).

This is obviously impossible with visual memory tests.

In a visual memory test, the participant is usually shown several images to remember. Then, after a delay, they are shown another image and asked if that is the same as one of the original images. Notice that you can be right 50% of the time just by guessing. Thus, to get a good measure, you need to do this many times.

Proactive Interference

This brings up the specter of proactive interference. I have written about proactive interference recently and won't belabor it here. The basic intuition is that if you do many trials of a memory test, it becomes hard to remember which stimuli were on which trial. So if you have been asked to remember circles of different colors, and then you are asked if the last trial contained a blue circle, you might remember that you have seen a blue circle recently but not remember if it was on the last trial or not.

So if visual working memory capacity tasks require many trials and verbal working memory tasks do not, one possible reason for the poor performance observed for visual working memory might be greater proactive interference.

Nope -- not proactive interference

The short version of the results of the published paper is that proactive interference does decrease measured capacity for visual working memory, but not by very much (about 15%). So it cannot account for the differences between visual and verbal working memory. The search must go on.

I hope to describe how the Web-based experiment contributed to this result in a future post. But interested readers can also read the paper itself. It is fairly short and reasonably non-technical.

Participate in current Web-based experiments at my lab by clicking here.

-------
Hartshorne, J.K. (2008). Visual working memory capacity and proactive interference. Public Library of Science One

CogLangLab's first published paper!

The first paper to contain data collected at my website (technically, at the old website) has just been published. The experiment in question was The Time Course of Visual Short-Term Memory.

This is the first of hopefully two papers using that data. The second paper will look at individual differences and aging. That paper is still in preparation and will hopefully be submitted in August.

I will explain the results and import of the just-published paper in an upcoming post.

--------

Hartshorne, J.K. (2008). Visual working memory capacity and proactive interference. Public Library of Science One

Cats, instincts, and evolution

Cats are very strange animals. One of their odder behaviors is scratching around their food bowl. They look like they are trying to kick dirt over the food in order to bury it. Presumably that is what they do in the wild. But the "dirt" house cats are kicking over the food is imaginary.

This suggests that wild cats do not know they are burying their food. That is, it is not goal-directed behavior. If it were goal-directed, house cats would either not bother with the scratching, or they would be very frustrated by their lack of success.

This is not to suggest at all that cats are all dumb instinct. On the one hand, much of their behavior seems very human-like. On the other hand, plenty of human behavior is instinct masquerading as goal-directed behavior. Sex, for instance, has a clear purpose but that's not usually why we do it. (Even if you believe sex has purposes other than procreation, such as pair bonding, it is hard to explain prostitution -- which, while not as common as cats "burying" their food, is still common enough to require explanation.

So why do cats scratch around their bowl? I suspect it just feels good. Evolution does not require that we know why we do what we do -- just that we do it.

Publishing scientific results: a timeline

A couple months ago, I talked about the slow rate of publication. I find the sloth-speed process irritating not so much because I am impatient -- though I am -- but because I would like to release the results of studies to my participants while they still remember they were in the study.

Just for fun, I thought I would outline the chronology of my upcoming paper in PLoS One.

Winter 2006-2007 Began data collection.

June 2007 Data collection complete

September 11, 2007 Paper submitted to Experimental Psychology

October 11, 2007 Paper rejected by Experimental Psychology

...Several months spent thinking about how to improve the paper...

March 22, 2008 New paper submitted to PLoS One. Told to expect a reply within a few weeks.

June 3, 2008 PLoS One asks for some minor revisions

June 16, 2008 Revision sumbitted

June 20, 2008 Paper accepted

July 23, 2008 Paper will be published online

................

It is worth noting that both Experimental Psychology and PLoS One are extraordinarily fast. That's part of why I submitted to those two journals. I submitted another paper in late January to a more traditional journal. I am still waiting for a reply.

    

How Much do you Know?

Mahesh Srinivasan, another member of the Laboratory for Developmental Studies, has just started his inaugural Web-based study.

It's a quick 1-2 minute experiment assessing people's general knowledge about the world. Participants will answer two simple questions and be asked to do a simple task. At the end of the experiment, you will find out what it was all about.

Participate by clicking here.

Children are very, very strange

I mentioned previously that I have a summer intern working with me on a language acquisition project. As part of one of our projects, she is analyzing transcripts of children interacting with their parents. Here are some observations she made recently:

I have come to learn that interacting with small children must be incredibly painful and that parents and their children have weird conversations.

The "painful" observation was left without further information, but she included the following excerpt from a transcript to illustrate the latter point:

Father: Marky's sexy?

Child: Yeah.

Father: Why is he sexy?

Child: Because he doesn't have any clothes on.

Father: You're right.

Father: Marky's sexy because he doesn't have any clothes on. 

Father: Are you sexy now?

Here is another data point she included:

Child: But we think G-d is mean.

Father: Why?

Child: Because he spanks me.

Interference in Memory

I wrote recently about interference processes that cause memory failure. As I wrote before, retroactive interference is when learning new information causes you to forget what you learned previously. In proactive interference, old information makes it hard to learn new information.

It turns out that there are (maybe) two types of proactive interference, and this may tell us a great deal about how memory works.

How Specific?

Half a century ago, Keppel & Underwood found that people quickly get worse at memory tasks. A basic task works like this: Remember the following letters: "etnmwo"

Now look away from the screen. After a few seconds, ask yourself what the letters were. How many could you remember?

Keppel & Underwood task was slightly different, but this gives you the basic idea. Again, what they found was that as people play this game, they actually do best on the first trial, worse on the 2nd, even worse on the 3rd, etc. (People bottom out fairly quickly, as we'll see in a future post.)

Keppel & Underwood suggested that this was due to proactive interference, which now seems pretty well established.

Later researchers discovered a curious thing. If the memory task is done with letters for a while, and then the experimenter switches to numbers, the participants suddenly get better. It doesn't have to just be letters and numbers. Switching from one type of item (say, names of car manufacturers) to another type (say, names of animal species) typically leads to an improvement in performance.

This has been called "release from proactive interference." But it is not the only kind.

More Specific

The type of proactive interference discussed above has been called "item-nonspecific" proactive interference. Learning information about one item made it harder to remember information about similar items.

This can be contrasted with "item-specific" proactive interference. As an example, go back to the sample memory test above. You were asked to remember "etnmwo." Suppose in the next trial, I asked you to remember "oaqzp" for a few seconds, after which I asked you:

"Is one of the letters are are supposed to remember an E?"

There is a decent chance you would incorrectly say "yes." This is because, although E was not one of the letters on this trial, it was one of the letters on the previous one. If I had instead asked about the letter C, which was not in either trial, you would be more likely to respond correctly and say "No."

This effect was discovered by Monsell using what is called the "Recent Probes Paradigm" -- which is basically what I just described.

Two Types or One?

One could legitimately wonder if these are really two different phenomena. That is, maybe item-specific proactive interference is simply a stronger version of item-nonspecific proactive interference.

It is hard to answer that question using behavioral experiments. Luckily, this is one of those places where neuroimaging can be helpful in understanding behavior. Recent neuroimaging results have found a strong overlap between the brain regions involved in the two types of proactive interference.

What Does this Say about Models of Memory?

Jonides and colleagues have been developing a model of memory that may both describe and predict the data on proactive interference.

In the model, to the extent that I understand it, you perform a short-term memory task like the ones described above by activating representations of the items. That is, to remember "aort," you would activate your long-term memory representations of A, O, R & T. But you do not actually hold those representations in consciousness; it is more that you make them easy to retrieve.

Now, suppose I ask you to repeat back those letters. You have to retrieve each of the four letters into consciousness so that you can give me your answer. You do this via something vaguely akin to a keyword search. That is, you search your memory for relevant features (e.g., a letter, recently encountered, seen on a blog, etc.). Since A, O, R & T all match those features and are all activated in memory, you retrieve them successfully.

Suppose on the next trial, though, you have to remember W, Z, P & E. So you activate those representations in memory. But A, O, R & T also remain somewhat active. And they also match most of the features (i.e., "keywords"). So you might accidentally retrieve one of them (item-specific proactive interference). In addition, since memories overlap, the still-active A, O, R & T representations make it harder to activate and maintain the representations of W, Z, P & E, since the compete for use of some of the same neurons. This might just make you fail to activate or retrieve anything at all.

Notice that if on the next trial, I ask you to remember 9, 3, 5, & 2, these items share fewer features with the letters on the previous trials, making the "keyword search" easier. Also, the representations of 9, 3, 5 & 2 in the brain are more distinct from the representations of the letters in trials one and two than either were from each other. Thus, you get release from item-nonspecific proactive interference.

Monsell, S. (1978). Recency, immediate recognition memory, and reaction time. Cognitive Psychology, 10(4), 465-501.

Keppel, G., Underwood, B.J. (1962). Proactive inhibition in short-term retention of single items. Journal of Verbal Learning & Verbal Behavior, 1, 153-161.

Jonides, J., Lewis, R.L., Nee, D.E., Lustig, C.A., Berman, M.G. (2008). The mind and brain of short-term memory. Annual Review of Psychology, 59, 193-224.

Neuroscience that matters

Science, like any other human activity, is subject to trends and fashions. Some are brief fads; others are slow waves that wash through society. For the last decade or two, cognitive neuroscience has been hot -- particularly neuroimaging.

A pretty typical example of cognitive neuroscience appears in this recent piece by the New York Times about research into the brain basis of sarcasm, which I read because I've been considering starting some work on sarcasm.

I generally don't like the media coverage of cognitive neuroscience, since it often acts surprised that human behavior is the result of activity in our brain. This particular article did not have that problem, but it still suffered from failing to answer the most important question any article has to answer.

The Right Parahippocampal Gyrus detects Sarcasm. So What?

The punch line of the article was that a neuroimaging study found the right parahippocampal gyrus to be active in sarcasm detection. Why this is important is left to the reader to decide.

So why is it?

In a lecture last spring, Randy Buckner distinguished between two types of cognitive neuroscience. 

In one, neuroscience techniques (patient studies, fMRI, single-cell recording, etc.) are used as behavioral measures. The goal of that type of research is to better understand human behavior. For instance, you might use fMRI to see if different brain regions are used in interpreting sarcasm and irony, which would suggest that the two phenomena are truly distinct.

The other kind of cognitive neuroscience uses the techniques of neuroscience to better understand how the brain produces the behavior in question. For instance, what computations to the neurons perform such that a person can perceive sarcasm? 

I am sympathetic to both types of cognitive neuroscience, though I tend to feel that there are very few human behaviors we understand well enough to seriously explore their neural instantiations (the basic phenomena of sensory perception are the only clear candidates I can think of, though basic memory processes might also make that list). You can't reverse-engineer a product if you don't know what it does. 

Interpreting Cognitive Neuroscience

In terms of the sarcasm article, it wasn't clear what this study adds to our understanding of what sarcasm is.  So I don't think it counts as the first type of cognitive neuroscience. 

Is it the second type? Some part of the brain must be involved in detecting sarcasm, so discovering which part of the brain it is in and of itself doesn't tell us much about implementation. Finding out that Sprint's national HQ is in Overland Park, KS, doesn't, by itself, tell you very much about Sprint, other than that it has an HQ, which you already probably guessed.

That doesn't  mean it's without information. Based on what you know about Overland Park, KS -- its tax regulations, local worker pool, lines of transportation and communication, etc. -- you might derive a great deal of information about how Spring works. But, unfortunately, the Times article didn't tell us much useful. I certainly don't know enough about the right  parahippocampal gyrus to really tell much of a story.

This is not a criticism of the journal article, which I haven't yet read. I'm actually pretty happy somebody is working on this issue. I just wish the Times had told me something useful about their work.