The verbal memory hegemony

One fact about the world is that the most famous memory researchers did most of their work on verbal memory. Alan Baddeley and George Miller both come to mind -- and I doubt anybody can think of more famous memory researchers in the last 50 years.

Another fact about the world is that many researchers -- not necessarily Baddeley or Miller -- have assumed that anything discovered using memory tests involving words should apply to other forms of memory as well. To pick unfairly on one person, Cowan notes in his masterful paper "The magical number 4 in short-term memory" that out of several related experiments, one has results that diverge from the others. Cowan attempts an explanation but basically throws up his hands. He doesn't notice that of all the experiments discussed in that section, the divergent one was the only one to use visual rather than verbal stimuli.

Similarly, a reviewer of my paper which just came out complained that the results reported in the paper only "told us things we already knew." As evidence, the reviewer cited a number of other papers, all of which had investigated verbal rather than visual short-term memory.

As it happens, the results in this case were very similar to what had been reported previously for verbal memory. But it could have come out differently. That was the point of doing the experiment.

Partly because of this bias in favor of verbal materials, not enough is known about visual memory, though this has been changing in recent years, thanks in part to folks like Steve Luck, George Alvarez, Yuhong Jiang, Edward Vogel and several others.

----------

Cowan, N. (2001). The magical number 4 in short-term memory: a reconsideration of mental storage capacity. Behavioral and brain sciences, 24, 87-185.

Hartshorne, J.K. (2008). Visual working memory capacity and proactive interference. Public Library of Science One

Miller, G.A. (1956). The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychological Review, 63, 81-97.

A scientist at work: Street corner surveying

As much as I love Web-based experiments, they aren't ideal in all situations. Currently, I have a series of short surveys, each of which requires 20 or so participants. When I say "series of short surveys," that doesn't mean I have them all in advance. The results of each survey dictate what the next survey will be.

This is hard to run online, because it means I need time between surveys to analyze the results and think up a new experiment. On the Web, it's hard to take a timeout.

Instead, I took on a new research assistant, made a sign, and bought a bunch of candy. Then I set up shop outside the Harvard Science Center and began giving away candy in exchange for participation in the surveys.

At first, it was awful. We got lots of tight-lipped smiles, but nobody stopped or even really made eye-contact. I thought, "Why did I think this was a good idea? I hate this sort of thing." Within about 10 minutes, though, we got into a groove and have been collecting data at a pretty good clip every time we go out.

It turns out that, other than feeling like a canvaser, it's a fun way of collecting data. You get to be outdoors and away from the computer. You get to actually interact with people. And the pace of the research is if anything even faster than Web-based research. We typically average 30 or so participants per hour. The Moral Sense Test gets that kind of traffic; The Cognition and Language Lab, unfortunately, does not. This is probably not unrelated to the 392 appearances of "Marc Hauser" in the New York Times archives, compared with the single appearance for "Joshua Hartshorne." (Journalists: if you are reading this, call me!)

Street corner surveying is an old method. Many people seem to believe it is more reliable than Web-based surveying. Why is beyond me. We are stopping busy people with other things on their minds. Many just want candy. We are in a busy, noisy area with tours passing by, camera bulbs flashing and the occasional demonstration. And on Tuesdays, there is a farmer's market in the same area.

Sometimes the responses are hard to explain. One control question reads along the lines of: "John has two children. How likely do you think it is that he has two children? How likely do you think it is he has three children?" More than a few people agree that it is more likely that John has three children than that he has two. One person carefully corrected the grammar on one page, which was a neighborly thing to do, except that the grammar on that page was actually right, and the "corrections" made it wrong.

When collecting data from humans, there is always noise.

Should we trust experiments on the Web?

When I first started doing Web-based experiments, a number of people in my own lab were skeptical as to whether I would get anything valuable out of them. Part of this was due to worries about method (How do you know the participants are paying attention? How do you know they are telling the truth?), but I think part of it was also a suspicion of the Internet in general, which, as we all know, is full of an awful lot of crap.

For this reason, I expected some difficulties getting my Web-based studies published. However, the first of these studies was accepted without much drama, and what concerns the reviewers did raise had nothing to do with the Web (granted that only one of the experiments in that paper was run online). Similarly, while the second study (run in collaboration with Tal Makovski) has run into some significant hurdles in getting published, none of them involved the fact that the experiments were all run online.

Until now. After major revisions and some new experiments, we submitted the paper to a new journal where we thought it would be well-received. Unfortunately, it was not, and many of the concerns involved the Web. Two of the reviewers clearly articulated that they just don't trust Web-based experiments. One went so far as to say that Web-based experiments should never be run unless there is absolutely no way to do the experiment in the lab.

(I would use direct quotes, but the reviewers certainly did not expect their comments to show up on a blog, anonymously or not. So you will have to take my word for it.)

Obviously, I trust Web-based experiments. I have written enough posts about why I think concerns are misguided, so I won't rehash that here. I am more interested in why exactly people have trouble with Web-based experiments as opposed to other methodologies.

Is it because the Web-based method is relatively new? Is it because the Internet is full of porn? Or is it simply the case that for any given method, there are a certain number of people who just don't trust it?

I have been doing street-corner surveying lately (a well-established method), and I can tell you that although it ultimately gives decent results, some very odd things happen along the way. But I suppose if, as a reviewer, I tried to reject a paper because I "just don't trust surveys," the action editor would override me.

Results from an Experiment: The Time Course of Visual Short-Term Memory

The first experiment I ran on the Web has finally made it into print. Rather fittingly, it has been published in a Web-based journal: The Public Library of Science One.

Visual Memory is a Scrawny Creature

That experiment, The Time Course of Visual Short-Term Memory, was part of a larger study probing a fundamental question about memory: why is visual working (short-term) memory so lousy? In recent years, visual memory folk like Edward Vogel and George Alvarez have debated whether we can store as many as four items in visual memory, while on the other hand researchers looking more at verbal memory, such as Nelson Cowan, have been arguing over whether verbal memory can store only four items. There are memory tricks that can allow you to keep a hundred words in short-term memory; nobody has reported any similar tricks for visual memory.

There are many other ways in which visual memory is piddly compared to verbal memory, and I go into them in depth in the paper. Interestingly, previous researchers have not made much out of this difference, possibly because people seem to work on either visual memory or verbal memory, but not both.

Does Verbal Memory Explain the Differences between Humans and Apes?

One possibility that occurred to me is that if verbal memory in fact is considerably more robust and more useful than visual memory, that would endow verbal animals (i.e., adult humans) with significant advantages over non-verbal animals (e.g., human infants and all other animals). Just as writing has allowed some human cultures to supplement our limited memory capacity -- try doing a complicated math problem in your head; the real limitation is memory -- language could allow us to supplement limited non-verbal memory systems.

In fact, I found that many of the differences between adult humans on the one side and young children and apes on the other are found in tasks with large working memory demands. More examples are given in the paper, but this includes theory of mind tasks.

Is Verbal Memory Really Better?

Of course, this is fruitless speculation unless visual working memory is really inferior. The problem is that visual and verbal memory capacity is tested in somewhat different ways. The easiest way to test verbal memory capacity is to give people a list of words to remember and then ask them to repeat that list back (this forms an important part of many IQ tests).

This is obviously impossible with visual memory tests.

In a visual memory test, the participant is usually shown several images to remember. Then, after a delay, they are shown another image and asked if that is the same as one of the original images. Notice that you can be right 50% of the time just by guessing. Thus, to get a good measure, you need to do this many times.

Proactive Interference

This brings up the specter of proactive interference. I have written about proactive interference recently and won't belabor it here. The basic intuition is that if you do many trials of a memory test, it becomes hard to remember which stimuli were on which trial. So if you have been asked to remember circles of different colors, and then you are asked if the last trial contained a blue circle, you might remember that you have seen a blue circle recently but not remember if it was on the last trial or not.

So if visual working memory capacity tasks require many trials and verbal working memory tasks do not, one possible reason for the poor performance observed for visual working memory might be greater proactive interference.

Nope -- not proactive interference

The short version of the results of the published paper is that proactive interference does decrease measured capacity for visual working memory, but not by very much (about 15%). So it cannot account for the differences between visual and verbal working memory. The search must go on.

I hope to describe how the Web-based experiment contributed to this result in a future post. But interested readers can also read the paper itself. It is fairly short and reasonably non-technical.

Participate in current Web-based experiments at my lab by clicking here.

-------
Hartshorne, J.K. (2008). Visual working memory capacity and proactive interference. Public Library of Science One

CogLangLab's first published paper!

The first paper to contain data collected at my website (technically, at the old website) has just been published. The experiment in question was The Time Course of Visual Short-Term Memory.

This is the first of hopefully two papers using that data. The second paper will look at individual differences and aging. That paper is still in preparation and will hopefully be submitted in August.

I will explain the results and import of the just-published paper in an upcoming post.

--------

Hartshorne, J.K. (2008). Visual working memory capacity and proactive interference. Public Library of Science One

Cats, instincts, and evolution

Cats are very strange animals. One of their odder behaviors is scratching around their food bowl. They look like they are trying to kick dirt over the food in order to bury it. Presumably that is what they do in the wild. But the "dirt" house cats are kicking over the food is imaginary.

This suggests that wild cats do not know they are burying their food. That is, it is not goal-directed behavior. If it were goal-directed, house cats would either not bother with the scratching, or they would be very frustrated by their lack of success.

This is not to suggest at all that cats are all dumb instinct. On the one hand, much of their behavior seems very human-like. On the other hand, plenty of human behavior is instinct masquerading as goal-directed behavior. Sex, for instance, has a clear purpose but that's not usually why we do it. (Even if you believe sex has purposes other than procreation, such as pair bonding, it is hard to explain prostitution -- which, while not as common as cats "burying" their food, is still common enough to require explanation.

So why do cats scratch around their bowl? I suspect it just feels good. Evolution does not require that we know why we do what we do -- just that we do it.

Publishing scientific results: a timeline

A couple months ago, I talked about the slow rate of publication. I find the sloth-speed process irritating not so much because I am impatient -- though I am -- but because I would like to release the results of studies to my participants while they still remember they were in the study.

Just for fun, I thought I would outline the chronology of my upcoming paper in PLoS One.

Winter 2006-2007 Began data collection.

June 2007 Data collection complete

September 11, 2007 Paper submitted to Experimental Psychology

October 11, 2007 Paper rejected by Experimental Psychology

...Several months spent thinking about how to improve the paper...

March 22, 2008 New paper submitted to PLoS One. Told to expect a reply within a few weeks.

June 3, 2008 PLoS One asks for some minor revisions

June 16, 2008 Revision sumbitted

June 20, 2008 Paper accepted

July 23, 2008 Paper will be published online

................

It is worth noting that both Experimental Psychology and PLoS One are extraordinarily fast. That's part of why I submitted to those two journals. I submitted another paper in late January to a more traditional journal. I am still waiting for a reply.

    

How Much do you Know?

Mahesh Srinivasan, another member of the Laboratory for Developmental Studies, has just started his inaugural Web-based study.

It's a quick 1-2 minute experiment assessing people's general knowledge about the world. Participants will answer two simple questions and be asked to do a simple task. At the end of the experiment, you will find out what it was all about.

Participate by clicking here.

Children are very, very strange

I mentioned previously that I have a summer intern working with me on a language acquisition project. As part of one of our projects, she is analyzing transcripts of children interacting with their parents. Here are some observations she made recently:

I have come to learn that interacting with small children must be incredibly painful and that parents and their children have weird conversations.

The "painful" observation was left without further information, but she included the following excerpt from a transcript to illustrate the latter point:

Father: Marky's sexy?

Child: Yeah.

Father: Why is he sexy?

Child: Because he doesn't have any clothes on.

Father: You're right.

Father: Marky's sexy because he doesn't have any clothes on. 

Father: Are you sexy now?

Here is another data point she included:

Child: But we think G-d is mean.

Father: Why?

Child: Because he spanks me.

Interference in Memory

I wrote recently about interference processes that cause memory failure. As I wrote before, retroactive interference is when learning new information causes you to forget what you learned previously. In proactive interference, old information makes it hard to learn new information.

It turns out that there are (maybe) two types of proactive interference, and this may tell us a great deal about how memory works.

How Specific?

Half a century ago, Keppel & Underwood found that people quickly get worse at memory tasks. A basic task works like this: Remember the following letters: "etnmwo"

Now look away from the screen. After a few seconds, ask yourself what the letters were. How many could you remember?

Keppel & Underwood task was slightly different, but this gives you the basic idea. Again, what they found was that as people play this game, they actually do best on the first trial, worse on the 2nd, even worse on the 3rd, etc. (People bottom out fairly quickly, as we'll see in a future post.)

Keppel & Underwood suggested that this was due to proactive interference, which now seems pretty well established.

Later researchers discovered a curious thing. If the memory task is done with letters for a while, and then the experimenter switches to numbers, the participants suddenly get better. It doesn't have to just be letters and numbers. Switching from one type of item (say, names of car manufacturers) to another type (say, names of animal species) typically leads to an improvement in performance.

This has been called "release from proactive interference." But it is not the only kind.

More Specific

The type of proactive interference discussed above has been called "item-nonspecific" proactive interference. Learning information about one item made it harder to remember information about similar items.

This can be contrasted with "item-specific" proactive interference. As an example, go back to the sample memory test above. You were asked to remember "etnmwo." Suppose in the next trial, I asked you to remember "oaqzp" for a few seconds, after which I asked you:

"Is one of the letters are are supposed to remember an E?"

There is a decent chance you would incorrectly say "yes." This is because, although E was not one of the letters on this trial, it was one of the letters on the previous one. If I had instead asked about the letter C, which was not in either trial, you would be more likely to respond correctly and say "No."

This effect was discovered by Monsell using what is called the "Recent Probes Paradigm" -- which is basically what I just described.

Two Types or One?

One could legitimately wonder if these are really two different phenomena. That is, maybe item-specific proactive interference is simply a stronger version of item-nonspecific proactive interference.

It is hard to answer that question using behavioral experiments. Luckily, this is one of those places where neuroimaging can be helpful in understanding behavior. Recent neuroimaging results have found a strong overlap between the brain regions involved in the two types of proactive interference.

What Does this Say about Models of Memory?

Jonides and colleagues have been developing a model of memory that may both describe and predict the data on proactive interference.

In the model, to the extent that I understand it, you perform a short-term memory task like the ones described above by activating representations of the items. That is, to remember "aort," you would activate your long-term memory representations of A, O, R & T. But you do not actually hold those representations in consciousness; it is more that you make them easy to retrieve.

Now, suppose I ask you to repeat back those letters. You have to retrieve each of the four letters into consciousness so that you can give me your answer. You do this via something vaguely akin to a keyword search. That is, you search your memory for relevant features (e.g., a letter, recently encountered, seen on a blog, etc.). Since A, O, R & T all match those features and are all activated in memory, you retrieve them successfully.

Suppose on the next trial, though, you have to remember W, Z, P & E. So you activate those representations in memory. But A, O, R & T also remain somewhat active. And they also match most of the features (i.e., "keywords"). So you might accidentally retrieve one of them (item-specific proactive interference). In addition, since memories overlap, the still-active A, O, R & T representations make it harder to activate and maintain the representations of W, Z, P & E, since the compete for use of some of the same neurons. This might just make you fail to activate or retrieve anything at all.

Notice that if on the next trial, I ask you to remember 9, 3, 5, & 2, these items share fewer features with the letters on the previous trials, making the "keyword search" easier. Also, the representations of 9, 3, 5 & 2 in the brain are more distinct from the representations of the letters in trials one and two than either were from each other. Thus, you get release from item-nonspecific proactive interference.

Monsell, S. (1978). Recency, immediate recognition memory, and reaction time. Cognitive Psychology, 10(4), 465-501.

Keppel, G., Underwood, B.J. (1962). Proactive inhibition in short-term retention of single items. Journal of Verbal Learning & Verbal Behavior, 1, 153-161.

Jonides, J., Lewis, R.L., Nee, D.E., Lustig, C.A., Berman, M.G. (2008). The mind and brain of short-term memory. Annual Review of Psychology, 59, 193-224.

Neuroscience that matters

Science, like any other human activity, is subject to trends and fashions. Some are brief fads; others are slow waves that wash through society. For the last decade or two, cognitive neuroscience has been hot -- particularly neuroimaging.

A pretty typical example of cognitive neuroscience appears in this recent piece by the New York Times about research into the brain basis of sarcasm, which I read because I've been considering starting some work on sarcasm.

I generally don't like the media coverage of cognitive neuroscience, since it often acts surprised that human behavior is the result of activity in our brain. This particular article did not have that problem, but it still suffered from failing to answer the most important question any article has to answer.

The Right Parahippocampal Gyrus detects Sarcasm. So What?

The punch line of the article was that a neuroimaging study found the right parahippocampal gyrus to be active in sarcasm detection. Why this is important is left to the reader to decide.

So why is it?

In a lecture last spring, Randy Buckner distinguished between two types of cognitive neuroscience. 

In one, neuroscience techniques (patient studies, fMRI, single-cell recording, etc.) are used as behavioral measures. The goal of that type of research is to better understand human behavior. For instance, you might use fMRI to see if different brain regions are used in interpreting sarcasm and irony, which would suggest that the two phenomena are truly distinct.

The other kind of cognitive neuroscience uses the techniques of neuroscience to better understand how the brain produces the behavior in question. For instance, what computations to the neurons perform such that a person can perceive sarcasm? 

I am sympathetic to both types of cognitive neuroscience, though I tend to feel that there are very few human behaviors we understand well enough to seriously explore their neural instantiations (the basic phenomena of sensory perception are the only clear candidates I can think of, though basic memory processes might also make that list). You can't reverse-engineer a product if you don't know what it does. 

Interpreting Cognitive Neuroscience

In terms of the sarcasm article, it wasn't clear what this study adds to our understanding of what sarcasm is.  So I don't think it counts as the first type of cognitive neuroscience. 

Is it the second type? Some part of the brain must be involved in detecting sarcasm, so discovering which part of the brain it is in and of itself doesn't tell us much about implementation. Finding out that Sprint's national HQ is in Overland Park, KS, doesn't, by itself, tell you very much about Sprint, other than that it has an HQ, which you already probably guessed.

That doesn't  mean it's without information. Based on what you know about Overland Park, KS -- its tax regulations, local worker pool, lines of transportation and communication, etc. -- you might derive a great deal of information about how Spring works. But, unfortunately, the Times article didn't tell us much useful. I certainly don't know enough about the right  parahippocampal gyrus to really tell much of a story.

This is not a criticism of the journal article, which I haven't yet read. I'm actually pretty happy somebody is working on this issue. I just wish the Times had told me something useful about their work.

Forgetting what you haven't yet learned

More than one student has complained that the space in their head is limited, and new information is simply pushing the old information out. In the terms of memory research, this is retroactive interference: learning new information causes you to forget old information.

The way this is typically studied in the laboratory is to have the participant learn something -- often a paired associate (think "Concentration") -- then learn something else, and then finally be tested on the original memory item(s). This way, one can vary that middle task in order to study how different activities cause different amounts/types of retroactive interference.

The is another type of interference: proactive interference. This is the effect that learning one piece of information has on future learning. That is, the books a student has already read make it harder to learn new information.

Just like retroactive interference, proactive interference is seen in both short-term and long-term memory. 

Memory Systems: How Does Memory Work?

The existence of interference tells us a lot about how memory works, because there is nothing necessary about it.

Consider a computer. We don't expect each new file we add to our computer to cause the computer to lose other files, short of copying over those original files. Similarly, the file I added today should not affect a file I add tomorrow, short of causing me to run out of disk space.

So why is human memory affected this way?

Overlapping Memories

There are a couple reasons it could be. One is that memory is probably overlapping. A computer -- at least, in its basic forms -- saves each file in a unique place in memory. The human brain, however, probably reuses the same units for different memories. Memories are overlapping.

How exactly this works is still very much a matter of research and debate, but it makes a certain amount of sense. Suppose you have several different memories about your mother. It would make sense for your mental representation of your mother to show up in each of those memories. For one thing, that should make it easier to relate those memories to one another.

Searching for Memories

Another way interference might appear in memory is in how it effects memory retrieval. The more files you put on your computer, the harder it is to find the files you want. This is especially true if you keep them all in one directory and use keyword searches.

Human memory retrieval probably does not work like a keyword search, but nonetheless it is reasonable to assume that the more memories you have, the more similar memories you have. Thus, finding the right memory is harder, because you have to distinguish it from similar memories.

How exactly this plays out depends on your model of memory. I will talk about one I particularly like in a future post.

Upcoming Posts

Although  my main research is in semantics and pragmatics -- aspects of language -- I have also worked on working memory. I have a paper coming out shortly based partly on an experiment I ran at my Web-based lab. Over the next week or two, I plan to write about some of the fundamental questions about memory addressed in that paper, as well as write about the paper and lay out its results.

It appears I am in the right field

You Should Get a PhD in Science (like chemistry, math, or engineering)

You're both smart and innovative when it comes to ideas. Maybe you'll find a cure for cancer - or develop the latest underground drug.

What Advanced Degree Should You Get?

Motivations for Science

Where do cognitive scientists get subjects for their studies?

There is a certain amount of variation, but the workhorse of cognitive science is the Psych 100 student. At many universities, introductory psychology students are required to participate in studies (though I believe there is often an option for people who strongly object).

This is billed as an educational experience, and more or less effort is put into making it educational (I've been very impressed with both Harvard and MIT on this point), but it is also part of the machinery that makes the science possible.

The other option typically is to pay participants. Currently, the going rate at Harvard is $10/hour. This is supposed to be compensation for time, travel, etc., but certainly lots of undergraduates who are not currently enrolled in a psych class use it to generate pocket cash.

Conflicting Goals

One potential drawback of this system is that the motivations of the participants and of the researchers are not always aligned. The researcher typically wants to get good data; the participant may just want their $10 or course credit.

The truth is the vast majority of participants give the experiment a good faith effort, but there are always some (I'd say about 5-10%, in my experience) who just answer randomly and quickly in order to get out as soon as possible.

There are ways to help realign the participants' and researchers' interests. One is to the program the experiment such that if you get all the answers right, you finish sooner than if you guess randomly. That makes guessing a bit less tempting a strategy. (An easy way of doing this in a computerized experiment is to have the computer respond with a long error message every time a question is answered incorrectly, with the effect that participants who make many errors take longer to finish.)

What Motivations do Parents Have?

Prior to working in a developmental lab, I wondered what motivations parents have for bringing in their children for developmental studies. It takes them more time, since unlike our "adult" subjects, they typically do not live on campus, and they get compensated less (we give our participants a cheap toy plus $5 for gas -- and $5 means a lot less to a parent than a college student).

I had heard it rumored that many are hoping the "affiliation" with Harvard will help their kids in the future or that they are very interested in having a scientist study their kid and discover what a genius the kid is. This frankly made me a bit uncomfortable.

Now I've actually interacted with a lot of parents and kids, and if those are their motivations, I don't see it. The main motivation seems to be that the kids really enjoy coming to the lab. We have a big bin full of toys, and we usually play with them for a while when they first come in. And then, the experiments are tailored such that kids really find them entertaining. Finally, many seem to really enjoy collecting the stuffed animals we give them as prizes at the end.

Parents are always looking for ways to keep their children entertained. It never occurred to me that taking the kids to a developmental lab would be one of those ways, but it appears that it is.

(Some parents are also definitely motivated by participating in science. At the end of the experiment, I always describe the experiment to them. Some are clearly not overly interested, but others may stay an extra 10-15 minutes to talk about the study.)

Does your child have philosophical potential?

Having just written about the difficulty in defining the word know, the following passage from Sperber & Wilson's Relevance:

Suppose, for example, that a child has not yet realised that X knows that P implies P, and so uses know interchangeably with believe. We would say that he had not yet mastered the concept. On the other hand, if he has grasped this logical pint but is unable to think of a single instance of something he is prepared to call knowledge, we would regard this as a failure of memory or experience (or a mark of philosophical potential) rather than of understanding.

Cheekiness aside, it actually takes children a while to fully understand the difference between know and believe. According to Bartsch & Wellman's classic corpus study, Children Talk about the Mind, children begin understanding those two verbs in their third year of life, but they don't appear to have truly mastered the concepts until around the age of 4. This is in contrast with verbs of desire like want, which kids know by the time they are 2.

Why are kids slow to understand know and believe? Some of the difficulty appears to be in understanding false beliefs -- that is, the fact that the contents of a person's mind may conflict with the actual state of the world (John believes Algeria is in South America, but it's not). Until a child has mastered that concept, there isn't any substantive difference between know and believe.