The New and Improved Web-based Cognition and Language Laboratory

Followers of this blog know that it is tied to a Web-based cognitive science lab. Dedicated followers also know that the old website was fuggly, to say the least.

It took longer than I care to admit, but the new and improved laboratory is now open for visitors. Important improvements, beyond the esthetics, include actual results listed on the results page, a bit more background into our twin topics, cognition and language, and other added content.

As with all new releases, this one is bound to be buggy. If you notice any mistakes or problems, please send me a discreet email at coglanglab_AT_coglanglab.org or bare my laundry for all the world to see by leaving a comment here.

What is the relationship between short-term and long-term memory?

In a textbook, you may see a description of memory in terms of stages. The first stage is iconic memory, which lasts just a few seconds, during which you can to some degree revive the perceptual experience you are trying to remember. Think of this almost like the afterglow of a bright flash of light.

Then comes short-term memory, which may or may not be also described as working memory (they aren't necessarily the same thing), which allows you to remember something for a short period of time by actively maintaining it. Anteriograde amnesics (like the guy in Memento) have intact short-term memory. What they don't have is long-term memory, which is basically recalling to mind something you haven't thought about in a while.

There are many aspects of the relationship between short-term memory and long-term memory that are still not clear. Over the last several months, Tal Makovski and I have been running a study trying to clarify part of this relationship.

We thought we had concluded it last week. I took the experiment offline, analyzed the results, wrote up a report and sent it to Tal. He wrote back with conclusions based on the data completely different from those that I had. Bascially, the results of two conditions are numerically different, but statistically there is no difference. He believes that if we had more participants, the difference would become statistically significant. I don't.

It's up to you, dear readers, to prove one of us wrong. The experiment is back online (click here for info; here to go straight to the test). It involves a quick visual short-term memory test, then you watch a video, after which you'll be quizzed on your memory for the video. It's by far the most entertaining of the experiments I've put online, mainly because the video is fantastic. It is Bill et John: Episode II, which was profiled in Slate. I've watched it easily a hundred times in the course of designing this study, and it's still fall-out-of-your-chair funny. Which is good, because it's nearly 10 minutes long, making the entire experiment take about 15 minutes.

Once again, you can find the experiment here. Once the results are available, I will post them on this blog and on the website.

Why girls say "holded" but boys say "held"

The most remarkable aspect of the human language faculty is that children learn so quickly and with such ease. Prior to school, they are already learning new words at the rate of one every two hours, counting the hours that they are asleep. Not only does this happen without intense schooling -- the method by which we learn most difficult skills -- but explicit instruction seems to be useless. Here is a conversation between a small child and father, reproduced in The Language Instinct:

Child: Want other one spoon, Daddy.
Father: You mean, you want THE OTHER SPOON.
Child: Yes, I want other one spoon, please, Daddy.
Father: Can you say "the other spoon"?
Child: Other . . . one . . . spoon.
Father: Say ... "other."
Child: Other.
Father: "Spoon."
Child: Spoon.
Father: "Other ... Spoon."
Child: Other ... spoon. Now give me other one spoon?

All that said, language learning still takes the little geniuses a few years, and along the way they make mistakes. Mistakes are interesting because they provide a window into how and what the child is learning. One of the most interesting and probably best-studied mistakes is grammatical over-regularizations of the sort "I holded the mouses" or "I runned with the wolfes." Notice that the child has probably never heard anybody say "holded" or "mouses," so this proves children aren't simply repeating what they've heard. Here, the child has taken irregular verbs (held, ran) and nouns (mice, wolves) and made them regular. The standard explanation for this -- though not the only one -- is that over-regularization occurs because we have a default rule to add the regular affix (walk+ed, dog+s) unless we have memorized an irregular form. If the child has not yet learned the irregular form or momentarily forgets it, an over-regularization results.

A few years ago, I was working with Michael Ullman at Georgetown University on a project investigating possible differences in how men and women perform particular linguistic computations. The bigger project continues, and if I get enough requests, I may eventually post something about it. The basic idea was that a number of studies have suggested that women perform better on declarative memory tasks than men do. Since Dr. Ullman's model of language processing ascribes certain processes to declarative memory (to be exact, "declarative memory-related brain structures) and others to procedural memory (careful: different researchers use this word to mean different things), that predicted differences between how men and women would perform certain linguistic functions. This was a way of testing Ullman's model and also perhaps learn something that could be useful in medical treatments of patients with language problems.

One day, it occured to me to explore childhood over-regularization. If women/girls have better memory for words, then they should have better memory for irregular forms and make fewer over-regularizations. We tested this and found a very surprising result: girls were actually three times more likely to over-regularize than boys.

This had us stumped for a while, but we eventually found an explanation. The "other model" mentioned above argues that over-regularization happens not through over-applying a grammatical rule but by analogy to regular forms (the difference is subtle, but it has big implications for how our minds and brains work -- for that reason this was one of the hot controversies throughout the 80s and 90s). Ullman's and similar models had always argued this was impossible because regular forms (walked, houses) are (mostly) not stored in memory. However, our ongoing research had suggested that women in fact do store a reasonable number of regular forms in memory after all, presumably because of superior declarative memory function. When we investigated the over-regularization data more carefully, we found evidence that the girls' over-regularizations -- but not the boys' -- were indeed a result of analogical reasoning, not rule-use. For whatever reason -- this is still not well understood -- the over-regularization-by-analogy process led to more "holded"s than the over-regularization-by-rule process.

And that is why girls say "holded" while boys say "held". You can read the journal article here.

I have been thinking about running a related experiment at my Web-based lab some day in the future. In the meantime, I've been investigating other topics (please participate by clicking here).

Update on cat cognition

For those who were interested in how cats see the world, check out this new study. I'm not sure it isn't just saying that cats have motor memory (as do humans), but it's interesting nonetheless.

The Best Language Site on the Web

News junkies might start up their web browser day any number of ways. There are those who prefer the Post to the Times. Those with a business mind might start with the Journal. On the West Coast, there are those who swear by LA's daily. I myself start with Slate.

However, I can state with little fear of correction, that the website of record for die-hard language buffs is the Language Log. The Language Log, I admit, is not for the faint of heart. The bloggers are linguists, and they like nothing better than parsing syntax. This is not William Safire.

What makes the Language Log great is that the writers really know what they are talking about. Growing up, I went to a chamber music camp several summers in a row (I played viola). One of my friends who attended the camp was a singer. One year, a violinist in her ensemble decided that, rather than play the violin part, she wanted do the voice part for one movement of a piece. I never heard them perform, but I am assured she was awful. My friend complained:

"If you haven't studied the violin, you wouldn't try to perform a difficult piece for an audience of trained musicians. You'd barely be able to get a single note in tune, and you'd know it. Everybody can open their mouths and make sound come out, which means they think they can sing."

The world of language is afflicted with a similar problem. Everybody speaks a language, and many people believe they are experts in language (here, here, here). A great deal of what is written about language is embarrasing. To make matters worse, the field is packed with urban legends about all the (they have less than a half-dozen, approximately the same number as we have in English). Here is an urban legend the Language Log uncovered about the Irish not having a word for sex.

Language is one of the most complicated things in existence, and even the professionals understand remarkably little. The bloggers at the Language Log do a great job of giving even the casual reader a feel for what language is really about. They also spend a considerable portion of their time debunking fallacies and myths. If you read only one blog about language, LL would be my choice. If you read two, then you might consider reading my blog as well:)

Subliminal messaging

When I was a small child, I thought the idea of subliminal messaging was way cool. Learn languages in your sleep! Control people's minds by inserting inaudible dialogue into the background! Wicked!

To the best of my knowledge, that type of subliminal messaging -- a hidden, language-based message -- doesn't exist (but if you have evidence of one, please comment!). Influencing another's actions turns out to be pretty easy. There are many well-documented ways to manipulate others. I will focus here on getting the answers you want. Basically, response management comes down to how you phrase the question.

In a classic study by Tversky and Kahneman, participants were given two options for combating a plague that was projected to kill 600 people. Plan A was sure to save 200 people. Plan B had a 1/3 probability of saving 600 and a 2/3 probability of saving nobody. 78% of participants took the safe option: A. Rephrasing the question in terms of deaths (400 guaranteed under Plan A; 1/3 probably of 0 and 2/3 probability of 600 under Plan B) reversed the result: 78% of participants chose plan B. This is because humans are risk-prone when dealing with losses ("let's hope for the best") but risk-averse when dealing with gains ("let's keep what we have").

In another study by Tversky and colleagues, they found that if you offer a shopper a "one time only" sale on a piece of merchandice (e.g., a Sony CD player), most (66%) will buy it, happy to avoid further shopping. If you offer them two different products (one by Sony, one by Aiwa), both on sale, nearly half (46%) will continue shopping rather than buy either. The addition of choices makes people less likely to choose.

In a different study (Strack & Mussweiler; pdf) asked one set of participants "Did Gandhi live to the age of 140?" The participants presumably all responded, "No." The second question was to estimate how long Gandhi lived. The average estimate was 67. The second group of participants was first asked "Did Gandhi live past the age of 9." Again, presumably everybody replied correctly. On the second question, they estimated on average that Gandhi lived to 50.

There are many other examples. This is why experts will tell you that polls are next to meaningless unless you know the exact wording of the question. It's not subliminal mind control like in the movies, but manipulating people's decisions (or, at least their answers to surveys) is fairly easy.

(BTW, Gandhi lived to 78.)

One of my favorite illusions

Even if you've seen this beore, it's worth seeing again. The following link is to a video of two teams (white and black) playing a ball game. Your task is to watch the white team and carefully count how many times they pass the ball (concentration is important):

http://viscog.beckman.uiuc.edu/grafs/demos/15.html

Now, watch it again, paying attention to everything on the screen. Most people will notice something they did not see the first time when they were just watching the white team. (This is hard to do in a blog, without spoilers!)

This gets back to my previous posts about how attention affects how you perceive what you see. The professor that designed this, by the way, is Daniel Simons of the University of Illinois. He's produced a great deal of fantastic research; this is just one example. In a random small-world coincidence, he was the previous tenent of the office suite I worked in last year at Harvard.

Results from an experiment!!

In this post, I am very pleased to release the results of one of my very first Web-based experiments. This was the experiment on visual attention alluded to a few days ago. Web-based experimenting -- that is, what we do at the Cognition and Language Lab -- has a number of advantages over traditional lab-based experiments. The primary one, as far as I'm concerned, is the ability to easily test large numbers of participants. Typical cognition experiments require 8-20 participants. Testing some hypotheses, however, may require hundreds or even thousands of participants.

Recruiting 1000+ participants, explaining the experiment to them and recording their data is time-consuming and thus expensive. Recruiting a thousand participants on the Internet is not exactly easy, but it is easier than the alternatives. Testing a thousand participants through the Internet is a snap. Once the experiment is loaded on the Web page, there's nothing more for me to do.

Why might an experiment require large numbers of subjects? There are many reasons. The experiment I am about describe is one example.

Early this year, as I was setting up my first Web-based lab (the Visual Cognition Online Lab, now closed), Tal Makovski, then a post-doc at the same lab as me at Harvard, came to me with an idea.

One of Tal's research interests is visual attention. As I mentioned in that previous post, you don't pay attention to everything you see, and this can affect what parts of your environment you are aware of. Not long ago, Tal and his PhD advisor suggested that in some instances, trying to ignore an area in your visual field actually causes you to pay more attention to it (Tsal & Makovski, 2006). They called this the "White Bear Hypothesis." This name comes from the following "experiment":

Quick! Don't think about a white bear!

If you immediately thought about a white bear, you get the idea.

In the original study, participants were supposed to identify a quickly-presented stimulus, while ignoring a distractor. The stimulus always appeared in the same position, and the distractor always appeared in the same position. The participants knew where to look and what to ignore. It was particularly important to ignore the distractor, because it would otherwise throw the participant off. In one version of this type of task, the stimulus is either an H or a K, and your job is to say which one you see. The distractor is also either an H or a K. It requires some concentration to not accidentally identify the distractor instead of the stimulus. Again, the location is what sets the stimulus and distractor apart.

The subjects did this task over and over. On some trials, instead of the normal task, two dots would appear -- one in the distractor's location and one in another location that had not been important so far. Although these dots appeared simultaneously, participants said that the one in the distractor location appeared first. This is the result you would expect if participants were paying particular attention to the distractor location (despite the fact that they were supposed to be ignoring that location).

One problem with this experiment is this: The appearance of the dots was called a "suprise trial," but it happened many times during the course of the experiment. The first time, the participants might have been surprised, but after that, they knew that occasionally two dots would appear -- one in the distractor location -- and that they would need to report the order in which the dots appeared. This might encourage them to pay attention to the distractor location after all.

Why not do just one surprise trial per participant? The reason Tsal & Makovski repeated the surprise trials was to get statistical robustness. There is a reason that standardized tests like the SATs have more than one question; this produces a more stable and more nuanced result. The worry with Tsal & Makovski's study was that perhaps they had the equivalent of an SAT test with one question repeated a hundred times. The authors used a number of controls to try to eliminate this possibility, but the doubt still lingered.

Now with our new Web-based lab, Tal reasoned that we could "surprise" each participant only one time, and make up for the reduced amount of data by having many participants. That is exactly what we did.

There were 7 versions of the experiment (more about that below). A little over 500 people completed the final version. The participant was briefly presented with a letter in the middle of the screen. They were to press one key if the letter was an H or a K, and a different key for S or C. A distractor letter appeared near to the target, which either matched the target category (congruent trials) or did not not (incongruent trials). This was repeated a number of times (16 in the final version of the experiment). Not surprisingly, participants were significantly more accurate and significantly faster in the congruent condition than the incongruent condition. Scientifically, this was expected, but it was exciting nonetheless. By putting the experiment on the Web, we lost a lot of control over the timing of the display. Similarly, we can't get faithful reports of the participant's speed in responding. Many people had been skeptical that our program would have enough percision to successfully show this effect.

On the 17th trial, either a P or a Q flashed on the screen, either in the same position as the distractors had appeared or in a different position. Participants were then asked, "Did you see a P or a Q?"

Unfortunately for us, there was no significant difference in accuracy when the P or Q appeared in the distractor location (73.1%) or the new location (70.7%). The numbers go in the direction of the hypothesis, but statistically the two results are equivalent. With over 500 participants already tested, it is unlikely that testing more will make this difference significant.

What does that mean? There are two possibilities. One possibility is that Tsal & Makovski's original result was in fact due to the repetetive nature of their task.

Another possibility was that our new experiment wasn't sensitive enough. There are many ways this would be possible. The early versions of the task were either too fast (people couldn't see the P or Q regardless of its position) or too slow (everybody go the P and Q correct regardless of its position). If the distractor was too easy to ignore, that could mean we would not get an effect, so we adjusted the difficulty of that task. Etc. Perhaps, in the end, it was still too easy. Perhaps the two-dot suprise trial would have shown the effect, but the P/Q task does not. The possibilities are endless.

The story would have been better if we had made a major discovery. Unfortunately, this is a more typical: an inconclusive result. Still, I'm pretty happy with the outcome. In terms of the technical aspects, this was by far the most ambitious Web experiment I have run. Most Web experiments are surveys. I wasn't sure that they dynamic aspects of this experiment -- especially recording response time -- would even be possible. The fact that the distractor task (H/K vs. S/C) worked as expected is very encouraging.

This is the first of what I hope will be many posts describing results from the Web-based lab. This was one of the most abstract experiments we've run. The rest should be more lay-person-friendly. Completed experiments included tests of visual memory and writing abilities. Our ongoing experiments look at how people interpret language, how birth order affects personality, and how parents and children communicate.

Cat Cognition

How do cats remember where they are?

Last Friday my wife and adopted two kittens. Their litter box was originally kept in the pantry. This was done in a sanitary manner, but it still wasn't ideal. Yesterday, we were able to clear a space in the mud room for the litter box and moved it there. The next step was to explain to Moshe and Noah where to find their litter.

When we brought them home, we taught them the location by picking them up every hour or two and carrying them to the litter box. It took most of the day, but they eventually caught on.

We tried the same method for teaching them the new location. Noam caught on reasonably well. Over the next day, I would see him march purposefully into the pantry, get to the wall, then stop and look puzzled. Finally, he'd begin to mew. I would pick him up and carry him to the mud room, where he found the litter box and proceeded to do his business. Through the day, he seemed more and more confident, but we still found him searching through the pantry.

Moshe had even more difficulty adjusting. I would find him in the pantry looking for the litter box. I would carry him from the pantry to the mud room and place him in the litter box. Sometimes he left immediately. Sometimes he'd scratch at the litter, or even lie down in it for a few minutes. He'd do anything but his business. Then he would go straight back to the pantry.

After a full day of him refusing to use the litter box, or go anywhere else, we moved it back. The new plan is to move it little by little each day from the pantry to the mud room (they are next to each other).

This says something interesting about the cat's conception of where to go to the bathroom. As humans, we see it as the litter box, but the cat may not see it that way at all. When given conflicting signals between the learned physical object (the box with litter) and the learned location (the pantry), Moshe chose the pantry. Even Noah, the more flexible of the two, perserverated on the pantry.

Humans, I admit, perservate on locations as well. You may get up in the middle of the night and expect to find your glasses on the shelf, only to remember you just through out that shelf yesterday and the glasses are on the new nightstand. Where we differ is you would not refuse to use your glasses just because they are on the nightstand, not on the shelf.

I'll keep an eye out for research on cat cognition, and if I find anything interesting, I'll post it here. I'm still on vacation, but when I get back to the lab in a couple weeks, I'll be posting more actual research rather than observations. Feel free to comment with links to cat cognition research.

This whole matter might have been simplified if cats could talk. Please help us study human language by participating in the Find the Dax experiment, which takes about 5 minutes. If you have already participated, thank you!

Do you see what you see?

Descartes famously asked just how much one should trust one's own senses. Is it not possible that an extremely powerful and malicious demon could trick you into thinking you see things that you do not? (Even more subtly: could a demon trick you into thinking you think things that you do not?)

As it so happens, demons are not necessary, because we are deceived by our senses all the time. Daniel Schacter argued in the Seven Sins of Memory that human memory should not be understood as a function that is meant to keep a perfect record of what has happened, but as a tool that is meant to be used to promote survival. As such, the "seven sins of memory" (common memory "faults" like forgetting things) are often byproducts of useful features or are even useful features in and of themselves. For instance, having a memory junked full of useless details is not efficient; it is better to remember only that which is important. It cuts down on time speant searching for an important memory. Anyone who has dealt with a file cabinet full of documents nobody will ever need should recognize this insight. (Full disclosure: Dr. Schacter is a professor in the Harvard Psychology Department, where I am a graduate student.)

The same argument applies to the senses. The sense did not evolve in order to give us perfectly accurate representations of the world. They evolved in order to help us cope with the world. Often, the two goals are one and the same, but they are not always. Visual illusions are examples where our eyes do not give us faithful representations of the world, usually because of tricks our brains employ to make our vision more useful, not less.

Other aspects of our senses may not be inaccurate per se, but they are not the only solutions to the problems. The typical human eye, for instance, has three different types of cones, which optimally respond to three different wavelenghts of light: 565 nm, 535, nm and 440 nm, which our brain perceives as red, green and blue. It is important to note that "red," "green" and "blue" are perceptions that exist in our minds. As I understand it, our cones could preferentially respond to 566 nm, 536 nm, and 441 nm without our perception of red, green and blue being altered. We could only have two kinds of cones (as red-green colorblind people do), and thus have a more impoverished perception of color. We could have 4 types of cones, as some people do, and have a richer perception of color.

The point being made here is that our brains do not passively view the world, and our eyes do not just take photographs. What we "see" is a representation of the world that is faithful in some respects and unfaithful in others. Additionally, it is constantly touched-up. When a fashion photographer takes a photograph, she not only manipulates the lighting, the clothing, the angle, etc., all before the image reaches the camera, but after the picture is taken, the photograph is altered. This alteration -- removing of blemishes, fading this, enhancing that -- is not done randomly; it is done to improve the usefulness of the image -- usefulness in terms of selling product or magazines or whatever. The eye and brain are similar.

The same argument The New York Times just ran a fantastic piece touching on just this issue. It is perfectly possible for something to be square in your field of view, but for you to completely fail to see it. I have this happen with my keys all the time. This is at least partly due to the fact that you don't pay equal attention to everything in your line of sight. Your attention focuses in a particular place (or possibly places -- this is actually a very complicated area of research). Magicians use this fact, manipulating your attention so that you "see" what they want you to see. Thus, magicians and perception researchers often have much to say to each other. I had heard of symposiums like the one described in the Times article before. Without spoiling any of the great stories in it, I recommend you read the article before it gets archived and starts costing money.

We recently completed an experiment probing the allocation of attention. This was one of my first Web-based experiments. The results are in, and as soon as I have a chance, I'll share them here on this site. These will be the first results from our Web-based experiments we'll be sharing on line.

Does birth order affect your personality?

I was more than a little shocked when Steven Pinker stated point blank that birth order does not affect personality. It was a little more than a page or two in his long The Blank Slate, but it was the part that most stuck with me. Not long after reading that, I came across a chapter collected by Gary Marcus making the same argument. In essence, the argument was that study after study had failed to find any empirical evidence of birth order effects.

(Full disclosure: I have known Gary Marcus for some time, and his early work was a spring-board for my first published paper. I have met Pinker only a few times, but he was the PhD mentor of my first cognitive science mentor, Michael Ullman.)

I didn't believe a word of it. But, as they say, "them's the results." There has been some success in finding a birth order effect on IQ, with IQ dropping a little with each child. A recent mammoth study replicating these results got a great deal of press. Still, that's not really an effect on personality, which is what interested me.

This is not as esoteric a point as it may seem. There is a lot riding on it. The reason Pinker brought up birth order in his "The Blank Slate," was that it is a test case in the Nature vs. Nurture debate. Specifically, what affect do parents and the familiiy environment have on children as adults? That's a hard question to test, because it's hard to quantify "family environment." But birth order is easy to quantify. Pinker argues that family environment and parent child-rearing strategies are fairly irrelevant. The lack of birth order effects on personality lends strong support.

As I read the literature on birth order effects, I started to wonder if the problem was with quantifying "personality." These studies all hinge on having a good measure for "personality," otherwise you can't say that people of a similar birth order are similar in personality. I began to think I had a way around this problem, and I developed an experiment. I ran two versions of the experiment on undergraduate psychology students at Central Michigan University, and it showed weak but significant birth order effects. Undergraduates at a single school is a fairly limited sample, so I'm running a larger version of the same experiment through the Web.

You can participate by clicking here: Birth Order Study.

In a few weeks, I hope to do a more detailed post on this topic.

Mirroring

For a while I've also been writing a blog about CLL for scienceblogs.com, so as to reach a more targeted audience. That's a lot of work, so from here on out, I'll probably just mirror the two blogs.

Again, why have two blogs with the same material? A lot of the traffic that a blog at blogspot or scienceblogs gets is via the general site feed. At blogspot, the readers are a broad spectrum of people interested in a wide variety of things. At scienceblogs, it's limited to people who would be interested in a bunch of science blogs. There are advantages to both types of audiences.

Please feel free to leave comments in you have anything to say about this strategy.

Find the Dax

Steven Pinker includes this entertaining clip from a "movie script" in one of his books:

Woman: I'm leaving you.
Man: Who is he?

This is one of my favorite examples of the truism that a lot more is going on in language than meets the eye. To understand this dialog, one has to make a number of assumptions. More broadly, to understand it, one has to know a great deal about human behavior. Imagine a computer programmed to understand language but which knows nothing about humans. How would it interpret this scene? And yet we bring all this background knowledge to bear effortlessly, and make inferences without even necessarily noticing it.

This phenomenon -- the background machinery of language comprehension, if you will -- is at the heart of what the Cognition and Language Laboratory. That said, only one of our current experiments really explores that issue. Please check it out and participate. It takes about five minutes, and at the end you will learn a little bit more about this line of research:

Find the Dax (http://coglanglab.org/DaxStory)

Getting Technorati set up...

Technorati Profile

Status Report

The laboratory's first experiments went live on our site about two months ago. The Birth Order experiment has done a reasonably good job of attracting participants, with about 500 so far. This is about 1/3 - 1/2 the number that are necessary, but hopefully by early fall we will be done.

The other new experiment, Find the Dax, is not doing as well. When I checked two days ago, only 74 people had completed the experiment. This experiment doesn't need as many participants as the Birth Order study, but at the minimum we need about 250. I'm not quite sure why fewer people have been interested in participating.

This is the second language experiment I have run. The first was back on my old site. That one also attracted participants at an abnormally slow rate. Perhaps people aren't as interested in experiments that test language processing. I hope this is not the case, as we plan to do primarily language experiments in the future!

If anybody has any ideas on how to attract more people to participate in language-related experiments, I'd love to hear them. Post your comments here or email coglanglab_AT_gmail.com.