Games with Words: New Web lab launched

The new Lab is launched (finally). I was a long ways from the first to start running experiments on the Web. Nonetheless, when I got started in late 2006, the Web had mostly been used for surveys, and there were only a few examples of really successful Web laboratories (like the Moral Sense Test, FaceResearch and Project Implicit). There were many examples of failed attempts. So I wasn't really sure what a Web laboratory should look like, how it could best be utilized, or what would make it attractive and useful for participants.

I put together a website known as Visual Cognition Online for the lab I was working at. I was intrigued by the possibility of running one-trial experiments. Testing people involves a lot of noise, so we usually try to get many measurements (sometimes hundreds) from each participant, in order to get a good estimate of what we're trying to measure. Sometimes this isn't practical.
The best analogy that comes to mind is football. A lot of luck and random variation goes into each game, so ideally, we'd like each team to play each other several times (like happens in baseball). However, the physics of football makes this impractical (it'd kill the players).

Running a study on the Web makes it possible to test more participants, which means we don't need as many trials from each. A few studies worked well enough, and I got other good data along the way (like this project), so when the lab moved to MN and I moved to graduate school, I started the Cognition and Language Lab along the same model.

Web Research blooms

In the last two years, Web research has really taken off, and we've all gotten a better sense of what it was useful for. The projects that make me most excited are those run by the likes of TestMyBrain.org, Games with a Purpose, and Phrase Detectives. These sites harness the massive size of the Internet to do work that wasn't just impossible before -- it was frankly inconceivable.

As I understand it, the folks behind Games with a Purpose are mainly interested in machine learning. They train computer programs to do things, like tag photographs according to content. To train their computer programs, they need a whole bunch of photographs tagged for content; you can't test a computer -- or a person -- if you don't know what the correct answer is. Their games are focused around doing things like tagging photographs. Phrase Detectives does something similar, but with language.

The most exciting results from TestMyBrain.org (full disclosure: the owner is a friend of mine, a classmate at Harvard, and also a collaborator) have focused on the development and aging of various skills. Normally, when we look at development, we test a few different age groups. An extraordinarily ambitious project would test some 5 year olds, some 20 year olds, some 50 year olds, and some 80 year olds. By testing on the Web, they have been able to look at development and aging from the early teenage years through retirement age (I'll blog about some of my own similar work in the near future).

Enter: GamesWithWords.org

This Fall, I started renovating coglanglab.org in order to incorporate some of the things I liked about those other sites. The project quickly grew, and in the end I decided that the old name (Cognition and Language Lab) just didn't fit anymore. GamesWithWords.org was born.

I've incorporated many aspects of the other sites that I like. One is simply to make the site more engaging (reflected, I hope, in the new name). It's always been my goal to make the Lab interesting and fun for participants (the primary goal of this blog is to explain the research and disseminate results), and I've tried to adopt the best ideas I've seen elsewhere.

Ultimately, of course, the purpose of any experiment is not just to produce data, but to produce good data that tests hypotheses and furthers theory. This sometimes limits what I can do with experiments (for instance, while I'd love to give individualized feedback to each participant for every experiment, sometimes the design just doesn't lend itself to feedback. Of the two experiments that are currently like, one offers feedback, one doesn't.

I'll be writing more about the new experiments over the upcoming days.

Renovations at the Cognition and Language Lab

I am in the processing of doing a complete overhaul of the Web-based laboratory. The site has been due for some editing for a while; the page about me still lists me as an "incoming graduate student," though I just started my third year.

More importantly, though, I want to make the website more interesting. Though I've collected some very good data, leading to two publications already with several more on their way, the experiments I'm currently interested in running require more participants. Right now I get about 30-40 participants a day. For the new experiments to work, I need closer to 100 per day.

Here is where you, the reader, comes in. What do you think would make the site more interesting and the experiments more compelling? I am doing a few things already. First, you may have noticed there are lately more pictures on the website. The new experiments are all going to be game-like. Participants will get back scores and, in some cases, know how they did compared to others. This has worked very well for folks like Games with a Purpose or TestMyBrain.org. I also admit that some of the experiments I've posted over the last few years have been pretty dry.

One last thing I'm considering doing is changing the name of the site to reflect the new brand. I had planned on LanguageGames.org, but someone just snagged that domain. I could still go with LanguageGames.com, but there is always the risk of confusion. What else might be a catchy name?

If you have any ideas about the domain name or any other aspect of the website, please leave a comment here or email me at coglanglab@coglanglab.org.

And now, on the radio

The radio show I discussed a couple weeks ago finally aired. I would have posted earlier, but I wasn't aware it had happened. My appearance is brief, but it was fun to do. The journalist (Michelle Elliot) does a very nice job of discussing birth order effects, so it is definitely worth listening to.

Does birth order affect who you are friends with? Results from a new study

In 1874, in preparing a demographic survey of English scientists, Francis Galton noticed a funny thing: nearly half of all English scientists were oldest or only sons. In the following 135 years, the notion that your birth order affects personality, intelligence, success in the world, and just about anything and everything else has become a mainstay in popular culture. A search on Amazon turned up over 10,000 books, including such titles as The Birth Order Book: Why You are the Way You Are and The Birth Order Connection: Finding and Keeping the Love of Your Life.

Are Birth Order Effects Real?

It may be surprising, then, that the scientific evidence for birth order effects is heavily controversial. Perhaps more studies that failed to find any effect of birth order as studies that do (see Ernst & Angst's Birth Order: Its Influence on Personality), and many studies that do show such effects are hopelessly confounded. Take Galton's original study. In the 1800s, many scientists were men of independent wealth. Generally, the bulk of inherited wealth went to the oldest son. So there were just more oldest and only sons available to be scientists. There were probably also more younger sons in the military, but this shouldn't be taken as evidence that younger children are more violent.

Modern day family size tends to be related to ethnicity and class. Poor families have more children than rich families, so being a middle child actually correlates with being poor. So if poor educational achievement correlates with being a middle child (which it probably does), that may just be that more middle children go to crumbling schools than do oldest or youngest children.

So if there are no birth order effects, what does that mean? Researchers like Steven Pinker (The Blank Slate) and Judith Rich Harris (The Nurture Assumption: Why Children Turn out the Way They Do) point out that if birth order effects are not real then this calls into question whether your home environment has any effect on how you turned out. If the existence or non-existence of siblings doesn't effect your personality, what chance is there that whether your parents read to you had an effect either? (They both go on to marshall additional evidence that the home environment has no lasting effects.)

Seriously?

Harris and Pinker's claims fly in the fact of much of what we hear. Everybody knows that children who are read to do better in school, right? Children who were beaten by their parents beat their own children. Unfortunately, data like these run up against a very basic problem in science: correlation does not equal cause and effect.

For instance, it is almost certainly the case that the height of the youngest child in the family is correlated with the height of the oldest child of the family. I'm tall and my younger brothers are tall, too. But I can't take credit for making them tall. We happen to share the same growth-related genes.

Similarly, maybe the genes that lead a particular parent to be the sort of parent who reads to her child are also genes that make children do well in school. If this sounds implausible ("genes for reading to one's child?"), consider that there is a personality difference between people who like to read and those who don't. The former might read to their children more. And those children, inheriting the same book-worm genes, do well in school.

(If you don't believe behavior is partly inheritable, The Blank Slate does a good job of summarizing the mountain of research that it is. You might also speculate on why cats act like cats, snakes like snakes, and humans like humans. It can't all be learned. My cats were raised by humans from birth, but they have yet to learn to talk, tie their shoes or clean their own litter box.)

Love, Marriage and Birth Order

One of the difficulties with the existing literature is that it tends to test specific claims about birth order (e.g., older children are more likely to be scientists). So if the experiment fails (the hypothesis is disproven), you only know that your specific idea about birth order is wrong. It doesn't mean that birth order doesn't have some other effect.

In trying to think of a very general test of birth order effects, I realized that a century of research has shown that like marries like. Spouses correlate on just about everything measurable, from physique to personality. Certainly, spouses aren't identical twins, but they tend to be more alike one another than you'd expect if people married random strangers off the street. So if birth order affects personality, and people choose spouses based on having similar personalities, spouses should (on average) have similar birth orders.

In a paper to be published by the Journal of Individual Psychology, my co-authors and I found not only do spouses have similar birth orders, best friends do, too. The effect is not huge, but it probably shouldn't be. Nobody thinks birth order rigidly determines personality, and neither do people rigidly marry others with identical personalities. What we expected was a weak correlation.

Unlike much previous research, this effect can't be due to socio-economic factors. Although middle children are more common in poorer families or in families of certain ethnic groups, there is no ethnic group -- can be no ethnic group -- in which older children are more common than younger children. And yet oldest-oldest and youngest-youngest pairings were more common that oldest-youngest pairings.

Personality or Intelligence?

Afficionados will notice there is another possible explanation. There is now convincing evidence that birth order is related to intelligence. Since spouse's tend to IQs correlate, it could be that IQ drove this effect. Whether this is a counter-hypothesis or not depends on whether you think intelligence is completely unrelated to personality, which seems to be more of a definitional question than a scientific one. But it does raise the question of which aspects of behavior are affected by birth order.

Either way, I think these results maintain some hope for the idea that we are at least partly shaped by how we grew up. It's important to recognize that the effects we found in this study were not large, though of course it would be surprising if it were large. Nobody thinks birth order rigidly determines personality, and neither do people always marry clones of themselves. So the correlation was never likely to be large.

Many Thanks

I want to conclude this post by thanking the participants in this study, many of whom may be among the readers. One of the surveys included in this paper was conducted through the Cognition and Language Laboratory on the Web. And of course, I invite anyone who is interesting in helping out in future research to participate in the new experiments we have running on the site.

Calling all 12 year olds

I've been analyzing data from the Memory Test. The response to that experiment has been fantastic, so I'm able to look at performance based on age, from about 14 years old to about 84 years old. Interestingly, by 14 years old, people are performing at adult levels. I have a few kids in the 10-13 range, but not quite enough. It would be nice to know at what age people hit adult competency.

So...if you or someone you know is in that age range, I'd like a few more participants in the near future. I should actually be able to put up a description of the results relatively quickly in this case, should I get enough participants.

New Experiment: Learning Verbs

I run a graduate student research workshop for students in the linguistics and psychology departments here at Harvard. We did a segment on word-learning. One of our guest-speakers started her presentation by pointing out that the field often things of word learning as learning nouns, despite the fact that there are many other types of words that probably have to be learned differently.

The experiments at the Cognition & Language Laboratory have been guilty of this oversight (this one for example). I'm happy to say that this flaw has been rectified. Late last week I posted the GorpTest, in which you can try to learn new verbs.

I'll be writing more on this topic in the near future. In the meantime, please participate.