Science in the classroom

Science Magazine recently profiled a new website that links up scientists and classroom teachers in order to improve science education. It looks like a great project, and definitely something needed.

Pronoun Sleuth

George Washington always refers to George Washington. The pronoun he, on the other hand, can refer to George Washington, Thomas Jefferson, John Adams, or anyone else who is male (not just presidents). So, unlike proper names which have the same meaning regardless of context, pronouns have almost no meaning without context.

Just saying that we figure out who he and she refer to based on context begs the question of how we do it. What aspects of context matter? The fact that today is Tuesday? Whether it is sunny or rainy? (This isn't a straw man -- both of these things can matter in the right circumstance; I leave it to the reader as an exercise to come up with examples.)

In one of our newest experiments, we took sentences with pronouns and systematically obscured aspects of the context to see if people could still figure out who the pronoun refers to. If they can, then that aspect of the context didn't matter. Play Pronoun Sleuth by clicking here.

Making science relevant

One thing about being a scientist interested in how people think and how different groups of people think differently is that you constantly notice differences in how scientists and non-scientists think differently.

For instance, scientists like evidence. You think how you parent your children affects how they turn out? Maybe. It's a testable question. (For my position on the question and on the evidence, read here.) One mark of a great researcher is the ability to spot untested assumptions (one of my favorite examples being Marc Hauser's work on language evolution). Of course, some scientists are less confined by evidence than others, and my avowedly non-scientist wife is as empirical-minded as they come. But it seems to be generally true (though I admit I don't know any well-controlled studies).

Where am I going with this?

Does making science relevant help science education?

In the last issue of Science Magazine, Hulleman and Harackiewicz point out that

Many educators and funding agencies share the belief that making education relevant to students’ lives will increase engagement and learning. However, little empirical evidence supports the specific role of relevance in promoting optimal educational outcomes, and most evidence that does exist is anecdotal or correlational.

I smell an experiment. It was surprisingly simple: students in a high school system in the Midwest were randomly assigned to write essays (as many as 8 during the semester, with an average of 4.7 essays/student) that either just summarized what they were learning in science or tried to apply what they learned to their own lives. The students who wrote essays relating science to their lives got better science grades and reported more interest in science at the end.

Motivation or depth of processing?

The authors discuss this in terms of motivation: students who see the relevance of science are more interested in it. (They also seem to imply that it improves confidence.) I'm interested in understanding the mechanism better (a professor in my department complains that Science articles are necessarily too short to give necessary experimental detail and theoretical motivation): were these students more interested because of the personal relevance per se, or was it simply that thinking about relevance required investing in and reinterpreting the material. After all, science isn't just a list of facts (or shouldn't be, anyway). Facts are boring; interpretations are what make science science.

But that is, as they say, academic. As the authors point out, this was a relatively simple method that improved grades and interest in science. Assuming, of course, that it replicates, this is a valuable contribution.

And it suggests that making science relevant improves education outcomes. If that seemed obvious from the get-go, it's worth remembering how many obvious truths have turned out to be wrong. Occasionally proving the obvious is an occupation hazard, but still worth the effort.

Games with Words: New Web lab launched

The new Lab is launched (finally). I was a long ways from the first to start running experiments on the Web. Nonetheless, when I got started in late 2006, the Web had mostly been used for surveys, and there were only a few examples of really successful Web laboratories (like the Moral Sense Test, FaceResearch and Project Implicit). There were many examples of failed attempts. So I wasn't really sure what a Web laboratory should look like, how it could best be utilized, or what would make it attractive and useful for participants.

I put together a website known as Visual Cognition Online for the lab I was working at. I was intrigued by the possibility of running one-trial experiments. Testing people involves a lot of noise, so we usually try to get many measurements (sometimes hundreds) from each participant, in order to get a good estimate of what we're trying to measure. Sometimes this isn't practical.
The best analogy that comes to mind is football. A lot of luck and random variation goes into each game, so ideally, we'd like each team to play each other several times (like happens in baseball). However, the physics of football makes this impractical (it'd kill the players).

Running a study on the Web makes it possible to test more participants, which means we don't need as many trials from each. A few studies worked well enough, and I got other good data along the way (like this project), so when the lab moved to MN and I moved to graduate school, I started the Cognition and Language Lab along the same model.

Web Research blooms

In the last two years, Web research has really taken off, and we've all gotten a better sense of what it was useful for. The projects that make me most excited are those run by the likes of TestMyBrain.org, Games with a Purpose, and Phrase Detectives. These sites harness the massive size of the Internet to do work that wasn't just impossible before -- it was frankly inconceivable.

As I understand it, the folks behind Games with a Purpose are mainly interested in machine learning. They train computer programs to do things, like tag photographs according to content. To train their computer programs, they need a whole bunch of photographs tagged for content; you can't test a computer -- or a person -- if you don't know what the correct answer is. Their games are focused around doing things like tagging photographs. Phrase Detectives does something similar, but with language.

The most exciting results from TestMyBrain.org (full disclosure: the owner is a friend of mine, a classmate at Harvard, and also a collaborator) have focused on the development and aging of various skills. Normally, when we look at development, we test a few different age groups. An extraordinarily ambitious project would test some 5 year olds, some 20 year olds, some 50 year olds, and some 80 year olds. By testing on the Web, they have been able to look at development and aging from the early teenage years through retirement age (I'll blog about some of my own similar work in the near future).

Enter: GamesWithWords.org

This Fall, I started renovating coglanglab.org in order to incorporate some of the things I liked about those other sites. The project quickly grew, and in the end I decided that the old name (Cognition and Language Lab) just didn't fit anymore. GamesWithWords.org was born.

I've incorporated many aspects of the other sites that I like. One is simply to make the site more engaging (reflected, I hope, in the new name). It's always been my goal to make the Lab interesting and fun for participants (the primary goal of this blog is to explain the research and disseminate results), and I've tried to adopt the best ideas I've seen elsewhere.

Ultimately, of course, the purpose of any experiment is not just to produce data, but to produce good data that tests hypotheses and furthers theory. This sometimes limits what I can do with experiments (for instance, while I'd love to give individualized feedback to each participant for every experiment, sometimes the design just doesn't lend itself to feedback. Of the two experiments that are currently like, one offers feedback, one doesn't.

I'll be writing more about the new experiments over the upcoming days.

How scientists feel about peer review...

I just had a manuscript rejected. I like to believe I handled it better.

Obama & I

Geoff Nunberg has a fantastic Fresh Air commentary posted on his website about the political misuse of linguistic information. Pundits frequently use statistical information about language -- the frequency of the word I in a politicians speeches, for instance -- to editorialize about the politician's outlook or personality.

That is to say, pundits frequently misuse statistical information. Most of what they say on the topic is nonsense. Nunberg has the details, so I won't repeat them here. There is one segment worth quoting in full, though:

To Liberman, those misperceptions suggest that Will and Fish are suffering from what psychologists call confirmation bias. If you're convinced that Obama is uppity or arrogant, you're going to fix on every pronoun that seems to confirm that opinion.

Watch this Space

GamesWithWords.org, the successor to CogLangLab.org will (hopefully) be launched within the next week or so. The blog name itself has changed in advance. And, in fact, you will find that the gameswithwords.org URL already works, though it just takes you to the old site.

Why a new website? Among reasons, I've been overhauling the website to make it more engaging and more fun. The old name didn't really fit anymore. Plus it was always hard to say, which is particularly egregious for a language-themed website.

More to come soon...

A Poorly-edited Editors' Handbook

Most psychology journals require that papers follow the American Psychological Association's style guide. This guidebook covers everything from the structure of the paper to the right way of formatting section headings, and it is updated every so often.

The sixth edition was released over the summer, and it seems it had to be recalled due to "errors and inconsistencies."

I haven't actually seen the 6th edition myself (I just bought the 5th edition a couple years ago and am not in a hurry to buy the new one). On the whole, it's a good manual and the rules make sense. However, reviewers will sometimes thank you for breaking the more frustrating rules , like the rule that charts and tables should be appended to the end of the manuscript -- not included in the document itself. This probably made sense in the day of type-written manuscripts, but makes modern electronic manuscripts very hard to read. Electronic documents are wonderful for many things, but the ease of flipping back and forth from one section to another is not one of them.

Hopefully the 6th edition fixed some of those out-dated rules. But I'll wait to find out once the fixed version appears.

Changes in this blog

As I've mentioned in a previous post, I'm in the process of renovating the lab website. There will also be significant structural changes to this blog (probably a regular schedule for posting, for instance).

All this renovation is taking a considerable amount of time, and you may have noticed the lack of frequent posting. This will continue until the new site is launched, hopefully in the next month.

Magic babies

There's an interesting article today over at Slate (Why Babies Crave Magic) that features work from one of my favorite local labs.

Making Super-babies

Parenting advice is no doubt as old as time itself. There is good advice, and then there are myths.

The Walt Disney Company is, in a roundabout fashion, owning up to one myth, which is that their Baby Einstein videos make babies smarter. This has been a well-known myth in scientific circles -- the American Academy of Pediatrics recommends no videos of any type for children under 2. Controlled experiments are tough, since it's hard to assign children to either watch or not watch TV (this tends to correlate with parental factors), but a quick search found a conference paper showing that toddlers have difficulty learning words presented on television, which fits with what I hear from other language development people that young children do not learn vocabulary from television (this isn't a literature I know well -- the youngest kids I study are 4 years old).

This brings up a myth about bilingualism. Many parents believe that raising a child bilingual makes them smarter. Some do this by having their children watch Spanish-language programming like Diego. This is likely a waste of time for two reasons: first, children typically do not learn a language if it makes up less than 20% of what they hear during a day. So a television program or two isn't going to do much good (again, citing other language researchers; I didn't see an obvious paper relating to this).

Second, though, the evidence that bilingualism makes a baby smarter is weak. The problem, again, is that controlled experiments are impossible. There is no way of randomly assigning toddlers to be bilingual or not. And bilingualism correlates with family (e.g., cultural and genetic) factors. As anyone who has spent time with a bilingual family knows, raising a child bilingual is a lot of work, and many parents don't bother. The parents who do are, by definition, not randomly distributed.

That said, there is a good reason to raise your children bilingual, even if it doesn't make them smarter: your children will be able to speak two languages! And that's pretty useful.

But if you want to make smarter babies, the best option I know of is to play with them more.

Vaccination and the Assault on Health

I had always though that refusal to get a flu vaccination was relatively harmless masochism. Refusal to vaccinate one's own children, on the other hand, should probably be prosecuted as child abuse, but at the least the negative consequences stay close to home.

Yesterday, however, I read two articles on vaccination. One in Slate looks at the risks the unvaccinated pose to people with immunity problems (she's unable to get childcare for her child, who is undergoing cancer treatment, because the risk of being around unvaccinated children is too high). If that seems like a parochial problem ("my kid doesn't have cancer; why should I worry about vaccination rates?"), the other article, appearing in Wired, is feature-length, and focuses on the anti-vaccine movement and the dangers it poses to the health of everyone.

Both note the rise in non-vaccination and the concomitant rise in outbreaks of the scourges of yesteryear. And they were scourges:

Just 60 years ago, polio paralyzed 16,000 Americans every year, while rubella caused birth defects and mental retardation in as many as 20,000 newborns. Measles infected 4 million children, killing 3,000 annually, and a bacterium called Haemophilus influenzae type b caused Hib meningitis in mor ehtan 15,000 children, leaving many with permanent brain damage...

But refusing to vaccinate is more than just a convenient way of decreasing the probability you'll have to pay for college (and that your neighbor's kid with leukemia will survive). This is because the un-vaccinated put the vaccinated at risk.

The Risk to Us All

As told in the Wired article, an unvaccinated 17-year-old Indiana girl picked up measles on a 2005 trip to Bucharest. When she returned, she went to a church gathering of 500 people. Of the 50 attendees who had not been vaccinated, 32 developed measles. Any adults who got measles had at least made the choice to take on that risk, but the children had not.

Even worse are the two people who had been vaccinated but nonetheless got sick. They had been responsible and protected themselves, but this reckless 17-year-old and her parents endangered their lives. First, though, three cheers for vaccines. Of the unvaccinated, 64% got sick. Of the vaccinated and those with natural immunity, only 0.8% got sick.

But still, vaccines don't always work. Sometimes they don't take. Sometimes your immune response may have weakened (for instance, through aging). Or you might just have bad luck. A 2002 study in The Journal of Infectious Diseases determined that you were safer as an unvaccinated person in a well-vaccinated country than as a vaccinated person in a largely un-vaccinated country.

People who refuse vaccines aren't just risking themselves, and parents who refuse vaccines for their children aren't just risking their children, they are risking you and me.

Baby-Killers

What makes this even worse is that every baby is initially unvaccinated. Children have to reach a certain age in order to get vaccines. What protects babies is that everyone older is healthy (i.e., vaccinated). So adult vaccine-refuseniks made it through infancy partly thanks to everyone else getting vaccinated. But they aren't willing to give other babies the same chance.

Do people have the right to choose for themselves whether they want vaccines? Sure -- as long as they live on top of a mountain or on a deserted island away from contact with anyone else. Mandatory vaccination**, and now!

(**With medical exceptions, of course)

Why do so many homophones have two pronunciations?

An interest in puns has led me to start reading the literature on homophones. Interestingly, in appears that in the scientific literature "homophone" and "homograph" mean the same thing, which explains why there are so many papers about mispronouncing homophones. Here's a representative quote:

"...reports a failure to use context in reading, by people with autism, such that homophones are mispronounced (eg: 'there was a tear in her eye' might be misread so as to sound like 'there was a tear in her dress").'

Sticklers will note that "tear in her eye" actually does involve a homophone (tier), but I don't think that's what the authors meant.

Readers of this blog know that I'm not a prescriptivist -- that is, I believe words mean whatever most speakers of a language think the words mean. So I'm not going to claim that these authors are misusing the word, since there seem to be so many of them. That said, it would be convenient to have a term for two words that have the same pronunciation which is distinct from the term for two words with distinct pronunciations but are written in the same way.

Recruiting Laboratory Participants

I am in the process of revamping the Internet laboratory, as I'm trying to increase the number of participants. Some very successful websites recruit ~500/day. I have been averaging about 30/day -- still respectable, but it limits what I can do.

In this context, I read recent reports from the folks behind Phrase Detectives with interest. Phrase Detectives, it appears, gets a slightly greater amount of traffic than I do. What I focused on was their method of advertising and how well it works. They noted that their traffic comes in the following forms:

direct: 46%

website link: 29%

search: 12%

Facebook advertisement: 13%

Then they looked at the bounce rate (the number of visitors who arrive at the home page then scoot away) for each of these sources:

direct: 33%

link: 29%

search: 44%

Facebook advertisement: 90%

It appears that paid advertisements -- the only one of these sources that actually costs money -- isn't worth much. In the end, only 4% of visitors who didn't bounce came through the paid advertisement.

Renovations at the Cognition and Language Lab

I am in the processing of doing a complete overhaul of the Web-based laboratory. The site has been due for some editing for a while; the page about me still lists me as an "incoming graduate student," though I just started my third year.

More importantly, though, I want to make the website more interesting. Though I've collected some very good data, leading to two publications already with several more on their way, the experiments I'm currently interested in running require more participants. Right now I get about 30-40 participants a day. For the new experiments to work, I need closer to 100 per day.

Here is where you, the reader, comes in. What do you think would make the site more interesting and the experiments more compelling? I am doing a few things already. First, you may have noticed there are lately more pictures on the website. The new experiments are all going to be game-like. Participants will get back scores and, in some cases, know how they did compared to others. This has worked very well for folks like Games with a Purpose or TestMyBrain.org. I also admit that some of the experiments I've posted over the last few years have been pretty dry.

One last thing I'm considering doing is changing the name of the site to reflect the new brand. I had planned on LanguageGames.org, but someone just snagged that domain. I could still go with LanguageGames.com, but there is always the risk of confusion. What else might be a catchy name?

If you have any ideas about the domain name or any other aspect of the website, please leave a comment here or email me at coglanglab@coglanglab.org.