Field of Science

Who knows more words? Americans, Canadians, the British, or Australians?

I have been hard at work on preliminary analyses of data from the the Vocab Quiz, which is a difficult 32 word vocabulary test. Over 2,000 people from around the world have participated so far, so I was curious to see which of the English-speaking nationalities was doing best.

Since the test was made by an American (me), you might expect Americans to do best (maybe I chose words or definitions of words that are less familiar to those in other countries). Instead, Americans (78.4% correct) are near the bottom of the heap, behind the British (79.8%), New Zealanders (82.2%), the Irish (80.1%), South Africans (83.9%), and Australians (78.6% -- OK that one is close). At least we're beating the Canadians (77.4%).


A fluke?

Maybe that was just bad luck. Plus, some of those samples are small -- there are fewer than 10 folks from New Zealand so far. So I pulled down data from the Mind Reading Quotient, which also includes a (different) vocabulary test. Since the Mind Reading Quotient has been running longer, there are more participants (around 3,000). The situation was no better: This time, we weren't even beating the Canadians. 

Maybe this poor showing was due to immigrants in America who don't know English well? Sorry -- the above results only include people whose native language is English. 

I also considered the possibility  that maybe Americans are performing poorly because I designed the tests to be hard, inadvertently including worse that are rare in America but common elsewhere. But the consistency of results across other countries makes that seem unlikely: What do the British, New Zealanders, Irish, South Africans and Australians all know that we don't? This hypothesis suggests that the poor showing by Americans is due to one or two items in particular. Right now there isn't enough data to do item-by-item analyses, but once we have more. Which brings me to...

Data collection continues

If you want to check how good your vocabulary is compared to everyone else who has taken the test -- and if you haven't done so already -- you can take the Vocab Quiz here. At the Mind Reading Quotient, you can test your ability to understand other people -- to read between the lines.

Update:

Phytophactor asks whether these results are significant. In the MRQ data, all the comparisons are significant, with the exception of US v. Canada (which went the other direction in the Vocab Quiz data anyway). The comparison with Australia is a trend (p=.06). See comments below for additional details. I did not run the stats for Vocab Quiz.

5 comments:

The Phytophactor said...

Let's see some stats. Those percentages just aren't different at all when you take into account variation around the mean.

GamesWithWords said...

@Phytophactor: I tend to trust replication more than stats. Which would you rather have: A nonsignificant result that you can replicate or a significant result that doesn't replicate?

But since I still had the MRQ data open, I checked: US scores were significantly worse than New Zealand (t(24.9)=2.8, p=.01), the UK (t(466.1)=2.8, p=.005), South Africa (t(5.1)=4.1, p=.009), Ireland (t(21.7)=3.0, p=.006), a trend for Australia (t(99.6)=1.9, p=.06), and no sig. difference for Canada (t(193.2)=1.1, p=.3), which you'll recall didn't replicate anyway.

Anonymous said...

I tried the exploding game. It was long and boring and there was no way to tell if something had been 'touched' by reading the nonsense words and names. Touch means contact. Most Q's, there was just no way to say yes or no, who, what or how many.

There's also no way to tell by your stats if you're 'getting it' or not. Gobbledygook. Or maybe gobbledygeek, as the case might be here :)

Also -- there's no way to leave a comment in the comments section there. Maybe that's why there aren't any comments there.

All in all, I'd say the exploding game is tedious and has too much ridiculousness in it. Something has to make a modicum of sense, or allow a best guess, in order to be useful, at least to the average internet participant. After a while I just started clicking random boxes. Hope that helps your study.

Anonymous said...

Additionally, I registered but the site is not recognizing me. No email was sent to confirm the account. This website is dense and not user friendly or intuitive.

GamesWithWords said...

I'm not sure what you mean by not being able to leave a comment. We do get comments. Just to double-check, I just left a comment on that task myself, and it worked fine.

As far as ID & password recovery, I also checked that, and it seems to work fine. If you want to send me an email (gameswithwords@gameswithwords.org) with the email address you think you used and your username (if you remember it), I can look to see what might be going on.