Field of Science

Showing posts with label language. Show all posts
Showing posts with label language. Show all posts

Updated results on the relationship between English dialects

I've updated the interactive visualization of the relationships between the Englishes of the world to include a couple dozen additional native languages. Check it out.

Which English: The Science, Part 1

I've gotten a number of questions about the science behind our WhichEnglish quiz. Actually, I had intended to post a more detailed discussion days ago, but I got distracted by other matters.

In this project, we are looking at three interrelated issues:

1. How does the age at which you start learning a language affect how well you learn that language?
2. How is learning a foreign language affected by the language you already know?
3. How are the grammars of different English dialects related?

And of course, we train an algorithm to predict participants' native language and dialect of English based on their answers. I return to that at the end.

Age of Acquisition

Although WhichEnglish has a few scientific targets, age-of-acquisition effects were the original inspiration. Everybody knows that the older you are when you start learning a foreign language, the harder it is to learn. One possibility is that there is a critical period: Up to some age, you can learn a language like a native. After that age, you will never learn it perfectly. The other possibility is that there is no specific period for language learning; rather, language-learning simply gets a little harder every day.

The evidence is unclear. Ideally, you would compare people who started learning some language (say, English) from birth with people who started as 1 year-olds and people who started as 2 year-olds, etc. Or maybe you would want something even finer-grained. The problem is that you need a decent number of people at each age (50 would be a good number), and it quickly becomes infeasible.

One study that came close to this ideal used census data. The authors -- led by Kenji Hakuta -- realized that the US census asks foreign-born residents to rate their own English ability. The authors compared this measure of English ability with the year of immigration (an approximation for the age at which the person started learning English). Their results showed a steady decline, rather than a critical period.

We are trying to build on this work in a few ways. For one, it would be nice to confirm (or disconfirm) the previous results with a more sensitive measure of English ability. So rather than just ask people how good their English is, we have them take a test. Also, we are getting more precise information about when the participant started learning English and in what contexts.

Also, there is good reason to suspect that age-of-acquisition affects different aspects of language differently. Studies have shown that even people who began learning a language as toddlers have detectable -- if very subtle -- accents. However, people who start learning foreign languages as adults usually report that learning vocabulary isn't so hard. Grammar seems to be somewhere in between. The Hakuta study didn't distinguish these different aspects of language.

WhichEnglish focuses on grammar. We also have a vocabulary quiz to look at vocabulary. A pronunciation test is in the works.

First language effects

When we started thinking about studying age-of-acquisition effects, we quickly realized a problem. We needed questions that would be difficult for someone who learned English as a second language. But which aspects of grammar are difficult seems to depend on your first language. I personally have difficulty with aspect in Russian because the English aspect system is much less complex. However, dealing with tense in Russian is relatively straightforward, since the Russian tense system is much less complex that English's.

Since we didn't know for sure what the language backgrounds of our participants would be, we wanted a range of questions that covered the different kinds of problems people with different backgrounds might have.

As we combed the literature, we realized that it was pretty fragmented. One study might say that grammar rule x is difficult for Japanese-speakers and grammar rule y is difficult for German-speakers, but there would be no information on how Japanese-speakers fare with grammar rule y and how German-speakers manage with grammar rule x. This actually makes sense: Most studies look at speakers of one or at most a handful of language backgrounds. This is partly a matter of research interest (the researchers are usually interested in some particular language) and partly a matter of feasibility (in a lab setting, you can only test so many participants). We realized that our study, by virtue of being on the Internet and recruiting people from a wide array of backgrounds, would provide an opportunity to get more systematic data across a large number of languages.

This is pretty exploratory. We don't have strong hypotheses. But as data comes in, we will be analyzing to see what we get, and we will report it here.

The Grammars of English

In designing our age-of-acquisition study, we realized a second problem. Correct English grammar varies across different dialects. In Newfoundland, you can say "Throw me down the stairs the hammer," but most places, you can't. (I have heard that this is said in parts of Rhode Island, too, but only anecdotally.) We don't want to count a late-learner of English who says "Throw me down the stairs the hammer" as not knowing English if in fact she lives in Newfoundland!

So what we really wanted were questions for which the correct answer is the same in all English dialects. But we didn't know what those were. Again, the literature was only partly helpful here. For obvious reasons, researchers tend to be interested in understanding peculiar constructions specific to certain dialects, rather than recording what is the same everywhere (boring).

We picked out a lot of grammar rules that we at least had no reason to believe varied across dialect. But we also realized that there was an opportunity here to study differences across dialects. So we included a subset of items that we thought probably would be different across dialects so that we can explore relationships across dialects.

The algorithm

When you take the quiz, at the end we give you our best guess as to what your native language is and what dialect of English you speak. How is that related to the three issues I just discussed?

It's deeply related. The best way of proving that you understand how people's understanding of grammar is affected by the age at which they started learning, their first language (if any), and the dialect of English they speak, is to show that you can actually distinguish people based on their grammar. In fact, training an algorithm to make just that distinction is a common way of analyzing and exploring data.

There are also obvious practical applications for an algorithm that can guess someone's language background based on their grammar (for education, localization of websites, and so on).

But an important reason we included the algorithm's predictions in the quiz itself was to present the results of the study to participants in the study as the study goes on. Certainly, you can read this and other blog posts I've written about the project as well. But it probably took you as long to read this post as to do the quiz. The algorithm and its predictions boil down the essence of the study in a compelling way. Based on the (numerous) emails I have gotten, it has inspired a lot of people to think more about language. Which is great. The best Web-based studies are a two-way street, where the participants get something out of the experience, too.

We chose the particular algorithm we use because it runs quickly and could be trained on very little data. You can read more about it by clicking on "how it works" in our data visualization. We are testing out more sophisticated algorithms as well, which are likely to do much better. Algorithms for detecting underlying patterns is actually a specialty of my laboratory, and this will be a fantastic dataset to work with. These algorithms mostly run too slowly to use as part of the quiz (nobody wants to wait 10 minutes for their results), but the plan is to describe those results in future posts and/or in future data visualizations.

In conclusion

If you have any questions about this work, please ask in the comments below or shoot me an email at gameswithwords@gmail.com.

Findings: Which English -- updated dialect chart

I have updated the dialect chart based on the results for the first few days. Since the new version shows up automatically in the frame in the previous post, I haven't added it in here. And you can get a better look at it on the website.

The biggest difference is that also added several "dialects" for non-native speakers of English. That is, I added five new dialects, one each for people whose first language was Spanish, German, Portuguese, Dutch, or Finnish. I'll be adding more of these dialects in the future, but those just happen to be the groups for which we have a decent number of respondents.

As you can see, the algorithm finds that American & Canadian speakers are more likely one another than they are like anyone else. Similarly, English, Irish, Scottish, and Australian speakers are more likely one another than anyone else. And the non-native English speakers also form a group. I'll leave you to explore the more fine-grained groupings on your own.

If you are wondering why New Zealanders are off by themselves, that's mostly because we don't have very many of them, and the algorithm has difficulty classifying dialects for which there isn't much data. Same for Welsh English, South African English, and Black Vernacular English. So if you know people who speak any of those dialects...

The English Grammars of the World

It's widely observed that not everybody speaks English the same way. Depending on where you grew up, you might say y'all, you guys, or just you. You might pronounce grocery as if it were "groshery" or "grossery." There have been some excellent, fine-grained studies of how these aspects of English vary across the United States and elsewhere, such as this one.

But vocabulary and pronunciation aren't the only things that vary across different dialects of English. We are in the midst of a soft launch of a new project which will, among things, help map out the differences in English grammar around the world.

I put together a visualization of early results below (you may want to load it in its own page -- depending on your browser, the embedded version below may not work). You can use this graphic to explore the similarities among nine English dialects (American, Canadian, English English, Irish, New Zealandish,  Northern Irish, Scottish, and South African).

As more results come in (about other dialects like Ebonics and Welsh, about specific parts of America or Canada, etc.), I'll be updating this graphic. So please take the survey and then check back in soon.



Load the graphic directly here.

What makes interdisciplinary work difficult

I just read "When physicists do linguistics." Yes, I'm late to the party. In my defense, it only just appeared in my twitter feed. This article by Ben Zimmer describes work published earlier this year, in which a group of physicists applied the mathematics of gas expansion to vocabulary change. This paper was not well received. Among the experts discussed, Josef Fruehwald, a University of Pennsylvania graduate student, compares the physicists to Intro to Linguistics students (not favorably).

Part of the problem is that the physicists seem to have not understood the dataset they were working with and were in any case confused about what a word is, which is a problem if you are studying words! Influential linguist Mark Liberman wrote "The paper's quantitative results clearly will not hold for anything that a linguist, lexicographer, or psychologist would want to call 'words.'"

Zimmer concludes that
Tensions over [the paper] may really boil down to something simple: The need for better communication between disciplines that previously had little to do with each other. As new data models allow mathematicians and physicists to make their own contributions about language, scientific journals need to make sure that their work is on a firm footing by involving linguists in the review process. That way, culturomics can benefit from an older kind of scholarship -- namely, what linguists already know about humans shape words and words shape humans.
Beyond pointing out that linguists and other non-physicists don't already apply sophisticated mathematical models to language -- there are several entire fields that already do this work, such as computational linguistics and natural language processing -- I respectfully suggest that involving linguists at the review process is way too late. If the goal is to improve the quality of the science, bringing in linguists to point out that a project is wrong-headed after the project is already completed doesn't really do anyone much good. I guess it's good not to publish something that is wrong, but it would be even better to publish something that is right. For that, you need to make sure you are doing the right project to begin with.

This brings me to the difficulty with interdisciplinary research. The typical newly-minted professor -- that is, someone just starting to do research on his/her own without regular guidance from a mentor/advisor -- has studied that field for several years as an undergraduate, 5+ years as a graduate student, and several more years as a post-doc. In fact, in some fields even newly-minted professors aren't considered ready to release into the wild and are still working with a mentor. What this tells me is that it takes as much as 10 years of training and guidance before you are ready to be fully on your own. (This will vary somewhat across disciplines.)

Now maybe someone who has already mastered one scientific field can master the second one more quickly. I'm frankly not sure that's true, but it is an empirical question. But it seems very unlikely that anyone, no matter how smart nor how well trained in their first field, is ready to tackle big questions in a new field without at least a few years of training and guidance from an experienced researcher in that field.

This is not a happy conclusion. I'm getting a taste of this now, as I cross-train in computational modeling (my background is pure experimental). It is not fun to go from being regarded as an expert in your field to suddenly being the least knowledgeable person in your laboratory. (After a year of training, it's possible I'm finally a more competent computational modeler than at least the incoming graduate students, though it's a tough call -- they, at least, typically have several years of relevant undergraduate coursework.) And I'm not even moving disciplines, just sub-disciplines within cognitive science!

So it's not surprising that some choose the "shortcut" of reading a few papers, diving in, and hoping for the best, especially since the demands of the career mean that nobody really has time to take a few years off to learn a new discipline. But it's not clear that this is a particularly effective strategy. All the best interdisciplinary work I have seen -- or been involved in -- involved an interdisciplinary team of researchers. This makes sense. It's hard enough to be an expert in one field. Why try to be an expert in two fields when you could just collaborate with someone who has already done the hard work of becoming an expert in that discipline? Just sayin'.

Citizen Science: Rinse & Repeat

One of the funny things about language is that everybody has their own. There is no "English" out there, existing independently of all its speakers. Instead, there are about one billion people out there, all of whom speak their own idiolect. Most likely, no two people share exactly the same vocabulary (I know some words you might not, possibly including idiolect, and you know some words I don't). Reasonable people can disagree about grammar rules, particularly if one is from Florida and the other from Northern Ireland.

This is one of the reasons we decided to ask people to create usernames in order to contribute to VerbCorner. Suppose two people answer the same question on VerbCorner but disagree. One possibility is that one of them made a mistake (which happens!). But another possibility is that they actually speak different dialects of English, and both are correct (for their dialect). It's hard to tell these possibilities apart by looking at just one question, but by looking at their answers to a set of questions, we can start to get a handle on whether this was a mistake or a real disagreement. The more answers we get from the same person -- particularly across different tasks -- the easier it is to do these analyses.

If we didn't have usernames, it would be hard to figure out which answers all belong to the same person. This is particularly true if the same person comes back to the website from time to time.

People are coming back. At last check, we have ten folks who have answered over 500 questions and four who have answered over 1000. (You can see this by clicking "more" on the leader-board on the main page).

Still, it would be great if we had even more folks who have answered large numbers of questions. Our goal is to have everyone in the top 20 to have answered at least 500 questions by the end of the month.

What makes a sentence ungrammatical?

This is the latest in a series of posts explaining the scientific motivations for the VerbCorner project.


There are many sentences that are grammatical but don't make much sense, including Chomsky's famous “colorless green ideas sleep furiously,” and sentences which seemed perfectly interpretable but are grammatical, such as “John fell the vase” or “Sally laughed Mary” (where the first sentence means that John caused the vase to fall, and the second sentence means that Sally made Mary laugh). You can hit at a window or kick at a window but not shatter at a window or break at a window (unless you are the one shattering or breaking!).

Sentence frames

Notice that these are not agreement errors (“Sally laughed”) or other word-ending errors ("Sally runned to the store"), but instead have something to do with the structure of the sentence as a whole. Linguists often refer to these sentence structures as "frames". There is the transitive frame (NOUN VERB NOUN), the intransitive frame (NOUN VERB), the 'at' frame (NOUN VERB at NOUN), etc. And it seems that certain verbs can go in some frames but not others.

There are many sentence frames (there is disagreement about exactly how to count them, but there are at least a few dozen), and most verbs can appear in somewhere around a half dozen of them. For instance, "thump" can appear in at least eight frames:


NOUN VERB NOUN:                                                  John thumped the door.
NOUN VERB NOUN with NOUN:                             John thumped the door with a stick.
NOUN VERB NOUNs together:                                   John thumped the sticks together.
NOUN VERB NOUN ADJECTIVE:                           John thumped the door open.
NOUN VERB NOUN ADJECTIVE with NOUN:       John thumped the door open with a stick.
NOUN VERB NOUN to [STATE]:                              John thumped the door to pieces.
NOUN VERB NOUN to [STATE] with NOUN:         John thumped the door to pieces with a stick.
NOUN VERB NOUN against NOUN:                         John thumped the stick against the door.

But there are a large number of frames "thump" can't appear in (at least, not without a lot of straining), such as:

NOUN VERB NOUN that SENTENCE:                    John thumped that Mary was angry.
NOUN VERB NOUN NOUN:                                    John thumped Mary the book.
NOUN VERB easily:                                                   Books thump easily.
There VERB NOUN out of [LOCATION]:               There thumped John out of the house.
NOUN VERB what INFINITIVE:                             John thumped what to do.
NOUN VERB INFINITIVE:                                      John thumped to sing


Explaining language

Perhaps these are just funny facts that we must learn about the language we speak, with no rhyme or reason. This is probably true for some aspects of grammar, like which verbs are irregular (that the past tense of “sleep” is “slept” is a historical accident). But a lot of researchers have suspected that there is a reason why language is the way it is and why certain verbs can go into certain frames but not others.

Going back several decades, researchers noticed that when you sort sentences based on the kind of sentence frames they can fit into, you do not get incoherent jumbles of verbs, but rather groups of verbs that all seem to share something in common. So “shatter” and “break” can be used with the object that is shattering or breaking as the direct object ("John shattered/broke the vase") or as the subject ("The vase shattered/broke"). All the verbs that can do this seem to describe some caused change of state (the vase is changing). Verbs that do not describe some kind of caused change cannot appear in both of these forms (you can say “John hit/kicked the vase" but not "The vase hit/kicked" -- at least not without a very special vase!).

Causality might also explain why you can hit at a window or kick at a window but not shatter or break at a window: the addition of the preposition "at" suggests that the action was ineffectual (you tried hitting the window without doing much damage) which is simply nonsensical with words that by their very definition require success. How do you ineffectually shatter a window? You either shatter it or you don't.

So maybe which verbs can go in which frames is not so mysterious after all. Maybe it is a simple function of meaning. Certain verbs have the right meanings for certain sentence frames. No more explanation necessary.

The VerbCorner Contribution


When you group verbs based on the frames they can appear in, you get several hundred groups of verbs in English. Of these, only a handful have been studied in any detail. While it does look like those groups can be explained in terms of their meaning, you might wonder if perhaps these are unusual cases, and if researchers looked at the rest, we would find something different. In fact, a number of researchers have wondered just that.

The difficulty has always been that there are a lot of verbs and a lot of groups. Studying just one group can take a research team years. Studying all of them would take lifetimes.

This is why we decided to crowd-source the problem. Rather than have a few people spend a lifetime, if a lots of people each contribute just a little, we can finish the project in a couple years, if not sooner.

Contribute to the VerbCorner project at gameswithwords.org/VerbCorner/

A Critical Period for Learning Language?

If you bring adults and children into the lab and try teaching them a new language, adults will learn much more of the language much more rapidly than the children. This is odd, because probably one of the most famous facts about learning languages -- something known by just about everyone whether you are a scientist who studies language or not -- is that adults have a lot less success at learning language than children. So whatever it is that children do better, it's something that operates on a timescale too slow to see in the lab. 

This makes studying the differences between adult and child language learners tricky, and a lot less is known that we'd like. Even the shape of the change in language learning ability is not well-known: is the drop-off in language learning ability gradual, or is there a sudden plummet at a particular age? Many researchers favor the latter possibility, but it has been hard to demonstrate simply because of the problem of collecting data. The perhaps most comprehensive study comes from Kenji Hakuta, Ellen Bialystok and Edward Wiley, who used U.S.A. Census data from 2,016,317 Spanish-speaking immigrants and 324,444 Chinese-speaking* immigrants, to study English proficiency as a function of when the person began learning the language. 

Their graph shows a very gradual decline in English proficiency as a function of when the person moved to the U.S.



Unfortunately, the measure of English proficiency wasn't very sophisticated. The Census simply asks people to say how well they speak English: "not at all", "not well", "well", "very well", and "speak only English". This is better than nothing, and the authors show that it correlates with a more sophisticated test of English proficiency, but it's possible that the reason the lines in the graphs look so smooth is that this five-point scale is simply too coarse to show anything more. The measure also collapses over vocabulary, grammar, accent, etc., and we know that these behave differently (your ability to learn a native-like accent goes first).

A New Test

This was something we had in mind when devising The Vocab Quiz. If we get enough non-native Speakers of English, we could track English proficiency as a function of age ... at least as measured by vocabulary (we also have a grammar test in the works, but that's more difficult to put together and so may take us a while yet). I don't think we'll get two million participants, but even just a few thousand would be enough. If English is your second (or third or fourth, etc.) language, please participate. In addition to helping us with our research and helping advance the science of language in general, you will also be able to see how your vocabulary compares with the typical native English speaker who participates in the experiment.

--------
Hakuta, K., Bialystok, E., & Wiley, E. (2003). Critical Evidence: A Test of the Critical-Period Hypothesis for Second-Language Acquisition Psychological Science, 14 (1), 31-38 DOI: 10.1111/1467-9280.01415



*Yes, I know: Chinese is a family of languages, not a single language. But the paper does not report a by-language breakdown for this group.

Do You Speak Korean?


Learning new languages is hard for many reasons. One of those reasons is that the meaning of an individual word can have a lot of nuances, and the degree to which those nuances match up with the nuances of similar words in your first language can make learning the new language easier; the degree to which the nuances diverge can make learning the
new language harder.

In a new experiment, we are looking at English-speakers learning Korean and Korean-speakers learning English. In particular, we are studying a specific set of words that previous research has suggested give foreign language learners a great deal of difficulty.

We are hoping that we will be able to track how knowledge of these words develops as you move from being a novice to a fluent speaker. For this, we will need to find a lots of people who are learning Korean, as well as Korean-speakers who are learning English. If you are one, please participate.

The experiment is called "Trials of the Heart". You can find it here.

We do also need monolingual English speakers (people whose first and essentially only language is English) for comparison, so if you that's you, you are welcome to participate, too!

Image credit

Findings: The Role of World Knowledge in Pronoun Interpretation

A few months ago, I posted the results of That Kind of Person. This was the final experiment in a paper on pronoun interpretation, a paper which is now in press. You can find a PDF of the accepted version here.

How it Began

Isaac Asimov famously observed that "the most exciting phrase to hear in science, the one that heralds new discoveries, is not 'Eureka!' but 'That's funny...'" That quote describes this project fairly well. The project grew out of a norming study. Norming studies aren't really even real experiments -- they are mini experiments used to choose stimuli.

I was designing an ERP ("brain wave") study of pronoun processing. A group in Europe had published a paper using ERPs to look at a well-known phenomenon in pronoun interpretation, one which has been discussed a lot on this blog, in which pronoun interpretation clearly depends on context:

(1) Sally frightens Mary because she...
(2) Sally likes Mary because she...

Most people think that "she" refers to Sally in (1) but Mary in (2). This seems to be a function of the verbs in (1-2), since that's all that's different between the sentences, and in fact other verbs also affect pronoun interpretation. We wanted to follow up some of the previous ERP work, and we were just choosing sentences. You get nice big ERP effects (that is, big changes in the brain waves) when something is surprising, so people often compare sentences with unexpected words to those with expected words, which is what this previous group had done:

(3) Sally frightens Bill because she...
(4) Bill frightens Sally because she...

You should get the sense that the pronoun "she" is a bit more surprising in (4) than in (3). Comparing these sentences to (1-2) should make it clear why this is.

The Twist

A number of authors argued that what is going on is that these sentences (1-4) introduce an explanation ("because..."). As you are reading or listening to the sentence, you think through typical causes of the event in question (frightening, liking, etc.) and so come up with a guess as to who is going to be mentioned in the explanation. More good explanations of an instance of frightening involve the frightener than the frightenee, and more good explanations of an instance of liking involve the like-ee than the liker.

The authors supported the argument by pointing to studies showing that what you know about the participants in the event matters. In general, you might think that in any given event involving a king and a butler, kings are more likely to be responsible for the event simply because kings have more power. So in the following sentence, you might interpret the pronoun as referring to the king even though it goes against the "typical" pattern for frighten (preferring explanations involve the frightener).

(5) The butler frightened the king because...

What got people particularly excited about this is that it all has to happen very fast. Studies have shown that you can interpret the pronoun in such sentences in a fraction of a second. If you can do this based on a complex inference about who is likely to do what, that's very impressive and puts strong constraints on our theory of language.

The Problem

I was in the process of designing an ERP experiment to follow up a previous one in Dutch that I wanted to replicate in English. I had created a number of sentences, and we were running a simple experiment in which people rate how "natural" the sentences sound. We were doing this just to make sure none of our sentences were weird, since that -- as already mentioned -- can have been effects on the brain waves, which could swamp any effects of the pronoun. Again, we expected people to rate (4) as less natural than (3); what we wanted to make sure was that people didn't rate both (3) and (4) as pretty odd. We tested a couple hundred such sentences, from which we would pick the best for the study.

I was worried, though, because a number of previous studies had suggested that gender itself might matter. This follows from the claim that who the event participants are matters (e.g., kings vs. butlers). Specifically, a few studies had reported that in a story about a man and a woman, people expect the man to be talked about more than the woman, analogous to expecting references to the king rather than the butler in (5). Was this a confound?

I ran the study anyway, because we would be able to see in the data just how bad the problem was. To my surprise, there was no effect of gender at all. I started looking at the literature more carefully and noticed that several people had similarly failed to find such effects. One paper had found an effect, but it seemed to be present in only a small handful of sentences out of the large number they had tested. I looked into studies that had investigated sentences like (5) and discovered ... that they didn't exist! Rather, the studies researchers had been citing weren't about pronoun interpretation at all but something else. To be fair, some researchers had suggested that there might be a relationship between this other phenomenon and pronoun interpretation, but it had never been shown. I followed up with some experiments seeing whether the king/butler manipulation would affect pronoun interpretation, and it didn't. (For good measure, I also showed that there is little if any relationship between that other phenomenon and pronouns.)

A Different Problem

So it looked like the data upon which much recent work on pronouns is built was either un-replicable or apocryphal. However, the associated theory had become so entrenched, that this was a difficult dataset to publish. I ultimately had to run around a dozen separate experiments in order to convince reviewers that these effects really don't exist (or mostly don't exist -- there do seem to be a tiny percentage of sentences, around 5%, where you can get reliable if very small effects of gender). (A typical paper has 1-4 experiments, so a dozen is a lot. Just in order to keep the paper from growing to an unmanageable length, I combined various experiments together and reported each one as a separate condition of a larger experiment.)

Most of these experiments were run on Amazon Mechanical Turk, but the final one was run at GamesWithWords.org and was announced on this blog (read the results of that specific experiment here). The paper is now in press at Language & Cognitive Processes. You can read the final submitted version here.

Conclusion

So what does all this mean? In many ways, it's a correction to the literature. A lot of theoretical work was built around findings that turned out to be wrong or nonexistent. In particular, the idea that pronoun interpretation involves a lot of very rapid inferences based on your general knowledge about the world. That's not quite the same thing as having a new theory, but we've been exploring some possibilities that no doubt will be talked about more here in the future.
----

Joshua K. Hartshorne (2014). What is implicit causality? Language and Cognitive Processes

New Experiment: The Language & Memory Test

There is a close relationship between language and memory, since of course whenever you use words and grammar, you have to access your memory for those words and that grammar. If you couldn't remember anything, you couldn't learn language to begin with.

The relationship between language and memory is not well understood, partly because they tend to be studied by different people, though there are a few labs squarely interested in the relationship between language and memory, such as the Brain and Language Lab at Georgetown University.

This week, I posted a new experiment, "The Language & Memory Test", which explores the relationship between memory and language. The experiment consists of two components. One is a memory test. At the end, you will see your score and how it compares with other people who took the test. This test is surprisingly hard for how simple it seems.

In the other part, you will try to learn to use some new words. We'll be studying the relationship between different aspects of your memory performance and how you learn these new words. As always, there will be a bit more explanation at the end of the experiment. When the experiment is done and the results are known, there will be a full description of them and what we learned here at the blog and at GamesWithWords.org.

Try the Language & Memory test here.

New Experiment: Collecting Fancy Art

Over the last few years, we've run a lot of experiments online at GamesWithWords.org, resulting so far in four publications, with a number of others currently under review at various journals. Most of these have experiments have focused on how people process and interpret language. I just posted a new experiment (Collecting Fancy Art) that is more squarely focused on learning language. Language learning experiments are somewhat tricky to do online, since they tend to take longer than the 5-10 minute format of online experiments, but they are important.

One of the most salient truths about language is that language has to be learned. This is clearly pretty hard, or other animals would be able to do it and we'd already have computers that were pretty good at language. But just how the learning process happens is a bit of a mystery, partly because language is a complex, interconnected system. When you learn one word, it affects how you use other words.

In this experiment, you will simultaneously learn the meanings of three different words. We're interested in seeing how your understanding of these words develops. As always, you'll learn more about the experiment at the end. And check back here in the future: After the experiment is completed, the results will be posted here.

The experiment is called "Collecting Fancy Art". You can find it here.

Laying to rest an old myth about Chinese

I just got back from my second research trip to Taiwan in three years (with another planned soon!) and fourth trip overall. As always, I had a great time and ate as much beef noodle soup as I could manage.


As always, I spent a couple months beforehand brushing up my reading and writing. This isn't something I have to do before trips to Spain or Russia. A few hours spent learning Spanish or Russian orthography, and you are set for life. As soon as I blink, I forget how to read and write Chinese. This is because, as is well known, rather than a couple dozen phonetic symbols, Chinese employs thousands of easily-confusable characters which, if you don't use for a while, you end up confusing.

This isn't just a problem for foreigners. Students in Taiwan (and China or Japan, I assume) continue investing significant amounts of time into learning to read and write additional characters well through secondary school. This raises the question of why Chinese-speakers don't just adopt a phonetic writing system?

Problems with a Chinese phonetic writing system

The argument one often hears is that Chinese has so many homophones (words that sounds like), that if you wrote them all the same way, there would be so much ambiguity that it would be impossible to read. The character system solves this by having different characters for different words, even ones that sound alike.

In the last century, when switching to a phonetic system was proposed, a scholar illustrated this problem with the following poem, which reads something like this:
Shi shi shi shi shi shi, shi shi, shi shi shi shi. Shi shi shi shi shi shi shi shi shi, shi shi shi shi shi, shi shi, shi shi shi shi shi. Shi shi shi shi shi, shi shi shi, shi shi shi shi shi shi. Shi shi shi shi shi shi, shi shi shi. Shi shi shi, shi shi shi shi shi shi. Shi shi shi, shi shi shi shi shi shi shi shi. Shi shi shi shi shi shi shi shi shi shi shi shi shi. Shi shi shi shi.
As written, this is incomprehensible. Only if you write it in characters
the meaning becomes clear:
A poet named Shi lived in a stone house and liked to eat lion flesh and he vowed to eat ten of them. He used to go to the market in search of lions and one day chanced to see ten of them there. Shi killed the lions with arrows and picked up their bodies carrying them back to his stone house. His house was dripping with water so he requested that his servants proceed to dry it. Then he began to try to eat the bodies of the ten lions. It was only then he realized that these were in fact ten lions made of stone. Try to explain the riddle.
Problems with this argument

This argument sounds compelling until you realize that what is being claimed is that you can't understand a Chinese sentence based on its sound alone. This means that not only is it impossible to understand phonetically-written Chinese, it is impossible to understand spoken Chinese (which, like phonetically-written Chinese, doesn't have any characters to help disambiguate similar-sounding words). Since a billion people speak Mandarin Chinese every day, there must be a problem with this argument!

There are a few. First of all, I wrote the poem phonetically ignoring the five Chinese tones. Like many languages, Chinese uses intonation phonetically -- an 'i' with a rising tone is different from an 'i' with a falling tone. Writing a tonal language without tones is like writing English without vowels -- much harder to read. Similarly, the phonetic writing above does not have any breaks between words, making it much harder to read (imaginewritingEnglishwithoutspacesbetweenwords). True, written Chinese doesn't mark word boundaries, but then it has all the extra information encoded in the characters to help with any ambiguity.

Second, this poem uses very archaic Chinese (different vocabulary and different grammar than modern Mandarin). It's not clear how many people would understand the poem spoken aloud. Wikipedia gives a nice translation of the poem into modern Mandarin, which involves many different sounds, not just 'shi'.

The most important problem is that there actually is a perfectly good phonetic system for writing Chinese. Actually, there are several, but the most common is pinyin. People can and do write entire texts in pinyin.

Why care? 

Why go to the effort of debunking this myth? This often comes up in arguments over whether the Chinese should adopt a new writing system, but that's not really my concern. Very often, there is a tendency to believe that different cultures and languages are much more different from one another than they are. One hears about strange aspects of other languages without even pausing to think about the fact that your own language has many of those same features. The writing systems of English and Chinese are actually alike in many ways (both are partially phonetic and partially semantic -- a topic for a different post). I can only speak for myself, but the more I learn about a given language, usually the less foreign it seems. Which is a fact worth thinking about.

Small World of Words

A group of researchers in Belgium is putting together a very large word association network by asking volunteers to say which words are related to which other words. They are hoping to recruit around 300,000 participants, which makes it my kind of study! (Technically, I've never tried 300,000 participants -- I think we've never gone beyond about 50,000, though we have some new things in the pipeline...)

It looks interesting. To participate, go to www.smallworldofwords.com. You can read more about the project here.

I say "uncle", you say "DaJiu"

Kinship terms (mother, uncle, niece, etc.) are socially important and generally learned early in acquisition. Interestingly, different languages have different sets of terms. Mandarin, for instance, divides "uncle" into "father's older brother", "father's younger brother", and "mother's brother".
Stranger things (to an anglophone, anyway) happen, too: In Northern Paiute, the kin terms for grandparents and grandchildren are self-reciprocal: you would use the same word to refer to your grandmother (if you are female) that she uses to refer to you. (See my previous post on "mommy" across languages.)






































Kinship terms in English and Northern Paiute. Ignore all the logical terms for now.
(Figure taken from Kemp & Regier, 2012)

Even so, there are a lot of similarities across languages. Disjunctions are relatively rare; that is, it's unusual to see a word that means "father or cousin". Usually there are more words to distinguish varieties of closely-related relatives (sister, brother) than distant relatives (cousin). How come? One obvious answer is that maybe the kinship systems we have are just better than the alternatives (ones with words like "facousin" = "father or cousin"), but it would be nice to show this.

Optimal Kinship Terms

In a paper earlier this year, Charles Kemp and Terry Regier did just that.
We show that major aspects of kin classification follow directly from two general principles: Categories tend to be simple, which minimizes cognitive load, and to be informative, which maximizes communicative efficiency ... The principles of simplicity and informativeness trade off against each other... A system with a single category that includes all possible relatives would be simple but uninformative because this category does not help to pick out specific relatives. A system with a different name for each relative would be complex but highly informative because it picks out individual relatives perfectly. 
That seems intuitively reasonable, but these are computational folk, so they formalized this with math. The details are in the paper, but roughly: They formalize the notion of complexity by using minimum description length in a representational language based on primitives like FEMALE and PARENT. The descriptions of the various terms in English and Northern Paiute are shown in parts C and D of the figure above. Communicativeness is formalized by measuring how ambiguous each term is (how many people it could potentially refer to).

A language is considered "better" than another if it out-scores the other on one dimension (e.g., simplicity) and no worse on the other (informativeness). A language is near-optimal if it there is hardly any possible language that is better. They looked at a number of different existing kinship systems (English, Northern Paiute, and a bunch of others) and found that all of them were near-optimal.

Evolution, Culture, or Development?

There are generally three ways of explaining any given behavior: evolution (we evolved to behave that way), culture (culture -- possibly through cultural evolution -- made us that way), or development (we learned to behave that way). For instance, it's rare to find people who chiefly eat arsenic. This could be because of evolution (we evolved to avoid arsenic because the arsenic-eaters don't have children and pass on their genes), cultural evolution (cultures that prized arsenic-eating all died out, leaving the non-arsenic cultures as the only game in town), or development (we learned as children, through trial and error, that eating arsenic is a bad idea). If I remember my Psych 101, food preferences actually involve all three.

What about kinship terms? If they are optimal, who do we credit with their optimality? Probably not development (we don't each individually create optimal kinship terms in childhood). Kemp and Regier seem to favor cultural evolution: over time, more useful kinship terms stuck in the lexicon of a given language and useless ones like "facousin" died out. It would be nice to show, however, that it is not actually genetic. This wouldn't have to be genes for kinship terms, but it could be genes that bias you to learn naming systems that are near-optimal (kinship naming systems or otherwise). One would need to show that these arose for language and not just cognition in general.

------
ResearchBlogging.org Kemp, C., and Regier, T. (2012). Kinship Categories Across Languages Reflect General Communicative Principles Science, 336 (6084), 1049-1054 DOI: 10.1126/science.1218811

Still testing...

I was hoping to post the results of That Kind of Person today. When I announced the study two weeks ago, I estimated that it would take about two weeks to get enough data. For some reason, traffic on the site plummeted late last week.

So maybe one more week. As soon as I know the results, you will, and since this is (please let it be) the last experiment (#8!) for a paper, I am checking the numbers constantly. Many thanks to those who have already participated (those who haven't, you can find the experiment here; it shouldn't take more than 5 minutes).

Findings: Linguistic Universals in Pronoun Resolution - Episode II

A new paper, based on data collected through GamesWithWords.org, is now in press (click here for the accepted draft). Below is an overview of the paper.

Many of the experiments at GamesWithWords.org have to do with pronouns. I find pronouns interesting because, unlike many other words, the meaning of a pronoun is almost entirely dependent on context. So while "Jane Austen" refers to Jane Austen no matter who says it or when, "I" refers to a different person, depending mostly on who says it (but not entirely: an actor playing a part uses "I" to refer not to himself but to the character he's playing). Things get even hairier when we start looking at other pronouns like "he" and "she". This means that pronouns are a good laboratory animal for investigating how people use context to help interpret language.

Mice make lousy laboratory animals for studying the role of context in language.
Pronouns are better.

I have spent a lot of time looking at one particular contextual effect, originally discovered by Garvey and Caramazza in the mid-70s:

(1) Sally frightens Mary because she...
(2) Sally loves Mary because she...

Although the pronoun is ambiguous, most people guess that she refers to Sally in (1) but Mary in (2). That is, the verb used (frightens, loves) seems to affect pronoun resolution. Replace "frightens" and "loves" with other verbs, and what happens to the pronoun depends on the verb: some verbs lead to subject resolutions like frightens, some to object resolutions like loves, and some leave people unsure (that is, they think that either interpretation of the pronoun is equally reasonable).

The question is why. One possibility is that this is some idiosyncratic fact about the verb. Just as you learn that the past tense of walk is walked but the past tense of run is ran, you learn that some verbs lead you to resolve pronouns to the verbs' subject and some the verbs' object (and some verbs have no preference). This was what was tentatively suggested in the original Garvey and Caramazza paper.

Does the meaning of the verb matter?

One of the predictions of this account is that there's nothing necessary about the fact that frightens leads to subject resolutions whereas loves leads to object resolutions, just as there is no deep reason that run's past tense is ran. English could have been different.

Many researchers have suspected that the pronoun effects we see are not accidental; the pronoun effects arise from some fundamental aspect of the meanings of frightens and loves. Even Garvey & Caramazza suspected this, but all the hypotheses they considered they were able to rule out. Recently, using data from GamesWithWords.org, we presented some evidence that this is right. Interestingly, while researchers studying pronouns were busy trying to come up with some theory of verb meaning that would explain the pronoun effects, many semanticists were independently busy trying to explain verb meaning for entirely different reasons. Usually, they are interested in explaining things like verb alternations. So, for instance, they might notice that verbs for which the subject experiences an emotion about the object:

(3) Mary likes/loves/hates/fears John.

can take "that" complements:

(4) Mary likes/loves/hates/fears that John climbs mountains.

However, verbs for which the object experiences an emotion caused by the subject do not:

(5) Mary pleases/delights/angers/frightens John.
(6) *Mary pleases/delights/angers/frightens that John climbs mountains.

[The asterisk means that the sentence is ill-formed in English.]

Linguists working on these problems have put together lists of verbs, all of which have similar meanings and which can be used in the same way. (VerbNet is the most comprehensive of these.) Notice that in this particular work, "please" and "frighten" end up in the same group as each other and a different group from "like" and "fear" are in a different one: Even though "frighten" and "fear" are similar in terms of the emotion they describe, they have a very different structure in terms of who -- the subject or the object -- feels the emotion.

We took one such list of verb classes and showed that it explained the pronoun effect quite well: Verbs that were in the same meaning class had the same pronoun effect. This suggests that meaning is what is driving the pronoun effect.

Or does it?

If the pronoun effect is driven by the meaning of a verb, then it shouldn't matter what language that verb is in. If you have two verbs in two languages with the same meaning, they should both show the same pronoun effect.

We aren't the first people to have thought of this. As early as 1983, Brown and Fish compared English and Mandarin. The most comprehensive study so far is probably Goikoetxea, Pascual and Ancha's mammoth study of Spanish verbs. The problem was determining identifying cross-linguistic synonyms. Does the Spanish word asustar mean frighten, scare, or terrify?
Is this orangutan scared, frightened or terrified? Does it matter?

Once we showed that frighten, scare and terrify all have the same pronoun effect in English, the problem disappeared. It no longer mattered what the exact translation of asustar or any other word was: Given that entire classes of verbs in English have the same pronoun effect, all we needed to do was find verbs in other languages that fit into the same class.

We focused on transitive verbs of emotion. These are the two classes already introduced: those where the subject experiences the emotion (like/love/hate/fear) and those where the object does (please/delight/anger/frighten) (note that there are quite a few of both types of verbs). We collected new data in Japanese, Mandarin and Russian (the Japanese and Russian studies were run at GamesWithWords.org and/or its predecessor, CogLangLab.org) and re-analyzed published data from English, Dutch, Italian, Spanish, and Finnish.

Results for English verbs (above). "Experiencer-Subject" verbs are the ones like "fear" and "Experiencer-Object" are the ones like "frighten". You can see that people were consistently more likely to think that the pronoun in sentences like (1-2) referred to the subject of Experiencer-Object verbs than Experiencer-Subject verbs.

The results are the same for Mandarin (above). There aren't as many dots because we didn't test as many of the verbs in Mandarin, but the pattern is striking.

The Dutch results (above). The pattern is again the same. Again, Dutch has more of these verb, but the study we re-analyzed had only tested a few of them.

You can read the paper and see the rest of the graphs here. In the future, we would like to test more different kinds of verbs and more languages, but the results so far are striking, and suggest that the pronoun effect is caused by what verbs mean, not some idiosyncratic grammatical feature of the language. There is still a lot to be worked out, though. For instance, we're now pretty sure that some component of meaning is relevant to the pronoun effect, but which component and why?

------------
Hartshorne, J., and Snedeker, J. (2012). Verb argument structure predicts implicit causality: The advantages of finer-grained semantics Language and Cognitive Processes, 1-35 DOI: 10.1080/01690965.2012.689305

Goikoetxea, E., Pascual, G., and Acha, J. (2008). Normative study of the implicit causality of 100 interpersonal verbs in Spanish Behavior Research Methods, 40 (3), 760-772 DOI: 10.3758/BRM.40.3.760

Garvery, C., and Caramazza, A. (1974). Implicit causality in verbs Linguistic Inquiry, 5 (3), 459-464

Roger Brown and Deborah Fish (1983). Are there universal schemas of psychological causality? Archives de Psychologie, 51, 145-153

Is Dragon Dictate a believer?

I've been using Dictate to take notes on Talmy's Toward a Cognitive Semantics. One of the example sentences is as follows:
I aimed my gun into the living room. (p. 109)
I cannot by any means convince Dictate to print this. It prefers to convert "my gun" to "my God". For example, on my third try, it wrote:
I aim to my God into the living room.
Dictate offers a number of alternatives in case its initial transcription is incorrect. Right now, it is suggesting, as an alternative to "aim to my God":
aimed to my God
aim to my God and
aim to my god
aim to my gun
aimed to my God and
aim to my garden
aimed to my god
aimed to my gun
aim to my guide
aim to my God in
aimed to my God in 
Perhaps Nuance has a religious bent, but I suspect that this is a simple N-gram error. Like many natural language processing systems, Nuance figures out what word you are saying in part by reference to the surrounding words. So in general, it thinks that common bigrams (2-word sequences) are more likely than uncommon bigrams.

According to Google, "my god" appears on the Web 133,000,000 times, whereas "my gun" appears only 8,770,000 times. So "my god" is just much more likely. Similarly, "aim to" is fairly common (215,000,000) hits. So even though "aim to my God" is gibberish, the two components -- "aim to" and "my god" -- are fairly common, whereas the correct phrase -- "aimed my gun" -- is fairly rare (138,000 hits). (The bigram "aimed my" is also infrequent: 474,000 hits).

N-gram systems work better than most everything else, which is why Nuance, Google, and many other companies use them. But examples like this show their deep limitations, in that they make many obvious errors -- obvious to humans, anyway. In this case, because Nuance doesn't know what sentences mean, and doesn't even know basic grammar, it can't tell that "aimed to my god" is both grammatically incorrect and meaningless.

Perspective in language

Language often indicates perspective:
(1) Give me that.
(2) *Give me this.
The reason that (2) is weird -- by convention, an asterisk marks a bad sentence -- is that the word this suggests that whatever is being requested is close to the speaker. Consider also:
(3) Jane came home.
(4) Jane went home.
If we were currently at Jane's home, it would be more natural to say (3) than (4). Of course, we could say (4), but we would be shifting our perspective, treating wherever Jane was as the reference point, rather than where we are now (this is particularly common in story-telling).

A less prosaic example

That is all fairly familiar, so when I turned to section 6.1 of Chapter 1 of Leonard Talmy's Toward a Cognitive Semantics, titled "Perspectival Location", I wasn't expecting anything particularly new. Then I read these examples (p. 69):
(5) The lunchroom door slowly opened and two men walked in.
(6) Two men slowly opened the lunchroom door and walked in.
These sentences describe the same event, but place the reader in a very different position. As Talmy points out, when reading (5), one gets the sense that you are in the lunchroom, whereas in (6), you get the sense that you outside of the lunchroom ... either that, or the door to the lunchroom is transparent glass.

Implied movement


Talmy gives another great pair of examples on page 71:
(7) There are some houses in the valley.
(8) There is a house every now and then through the valley.
The first sentence implies a static point of view, far from the houses, allowing you to see all the houses at once (Talmy calls this "stationary distal perspective point with global scope of attention"), whereas (8) gives the sense of moving through the valley and among the houses, with only a few within view at any given time ("moving proximal perspective point with local scope of attention")


Writing

Talmy's purpose is to put together a taxonomy of linguistic devices, and most of this chapter is trying to lay out all the different factors along which language can vary (for instance, the different types of perspective one can take). And that is of course why I'm reading it.

But it's also interesting to think about as a writer. One flaw in bad writing is using sentences that adopt the wrong perspective (telling a story about Jennifer, who is in the lunchroom, and then using (6)). This example from Talmy shows just how complicated the issues are ... and the tools available to a good writer for subtly guiding the reader through the story.

The toughest cop "alive"

At little while back, while walking down the street and minding my own business, I saw the following advertisement:

This looks like a scare quote; the implication is that Ace Ticket is nowhere near the best, that it is absurd to suggest that it is the best, which, I assume, is not what they were trying to convey.

One of my colleagues -- yes, I did send this around -- suggested that perhaps Ace Ticket hoped we would read this as a direct quote: somebody has called them the best, and they are quoting. The BBC, apparently, does this regularly. For instance, here's a recent article headline: Families tell of their fears for Syria's 'disappeared'.

I'm not a prescriptivist. But this is confusing. I can't tell, from the article, who the BBC is quoting, especially since the term "disappeared", used to mean "abducted or killed", is now standard usage.

It seems I'm not the only one confused. Buzzfeed has a list of "the worst of unnecessary quotation marks", most of which are unintended scare quotes (the tagline: These aren't just "unnecessary," they're "incomprehensible."). For example:
and:

You can find the rest of the Buzzfeed list here.

Someone in the comments brought the World's Most Interesting Man into the mix (I can only hope this starts a meme):