Games with Words: New Web lab launched

The new Lab is launched (finally). I was a long ways from the first to start running experiments on the Web. Nonetheless, when I got started in late 2006, the Web had mostly been used for surveys, and there were only a few examples of really successful Web laboratories (like the Moral Sense Test, FaceResearch and Project Implicit). There were many examples of failed attempts. So I wasn't really sure what a Web laboratory should look like, how it could best be utilized, or what would make it attractive and useful for participants.

I put together a website known as Visual Cognition Online for the lab I was working at. I was intrigued by the possibility of running one-trial experiments. Testing people involves a lot of noise, so we usually try to get many measurements (sometimes hundreds) from each participant, in order to get a good estimate of what we're trying to measure. Sometimes this isn't practical.
The best analogy that comes to mind is football. A lot of luck and random variation goes into each game, so ideally, we'd like each team to play each other several times (like happens in baseball). However, the physics of football makes this impractical (it'd kill the players).

Running a study on the Web makes it possible to test more participants, which means we don't need as many trials from each. A few studies worked well enough, and I got other good data along the way (like this project), so when the lab moved to MN and I moved to graduate school, I started the Cognition and Language Lab along the same model.

Web Research blooms

In the last two years, Web research has really taken off, and we've all gotten a better sense of what it was useful for. The projects that make me most excited are those run by the likes of TestMyBrain.org, Games with a Purpose, and Phrase Detectives. These sites harness the massive size of the Internet to do work that wasn't just impossible before -- it was frankly inconceivable.

As I understand it, the folks behind Games with a Purpose are mainly interested in machine learning. They train computer programs to do things, like tag photographs according to content. To train their computer programs, they need a whole bunch of photographs tagged for content; you can't test a computer -- or a person -- if you don't know what the correct answer is. Their games are focused around doing things like tagging photographs. Phrase Detectives does something similar, but with language.

The most exciting results from TestMyBrain.org (full disclosure: the owner is a friend of mine, a classmate at Harvard, and also a collaborator) have focused on the development and aging of various skills. Normally, when we look at development, we test a few different age groups. An extraordinarily ambitious project would test some 5 year olds, some 20 year olds, some 50 year olds, and some 80 year olds. By testing on the Web, they have been able to look at development and aging from the early teenage years through retirement age (I'll blog about some of my own similar work in the near future).

Enter: GamesWithWords.org

This Fall, I started renovating coglanglab.org in order to incorporate some of the things I liked about those other sites. The project quickly grew, and in the end I decided that the old name (Cognition and Language Lab) just didn't fit anymore. GamesWithWords.org was born.

I've incorporated many aspects of the other sites that I like. One is simply to make the site more engaging (reflected, I hope, in the new name). It's always been my goal to make the Lab interesting and fun for participants (the primary goal of this blog is to explain the research and disseminate results), and I've tried to adopt the best ideas I've seen elsewhere.

Ultimately, of course, the purpose of any experiment is not just to produce data, but to produce good data that tests hypotheses and furthers theory. This sometimes limits what I can do with experiments (for instance, while I'd love to give individualized feedback to each participant for every experiment, sometimes the design just doesn't lend itself to feedback. Of the two experiments that are currently like, one offers feedback, one doesn't.

I'll be writing more about the new experiments over the upcoming days.