Even Experts Don't Know what Brain Scans Mean

For some reason, many people find neuroscience more compelling than psychology. That is, if you tell them that men seem to like video games more than women, they are unconvinced, but if you say that brain scans of men and women playing video games showing that the pleasure centers of their brains respond to video games, suddenly it all seems more compelling.

More flavors is more fun, and the world can accept variation in what types of evidence people find compelling -- and we're probably the better for it. In this case, though, there is a problem in that neuroscientific data is very hard to interpret. Jerome Kagan said it perfectly in his latest book, so I'll leave it to him:

A more persuasive example is seen in the reactions to pictures that are symbolic of unpleasant (snakes, bloodied bodies), pleasant (children playing, couples kissing), or neutral (tables, chairs) emotional situations.The unpleasant scenes typically induce the largest eyeblink startle response to a loud sound due to recruitment of the amygdala. However, there is greater blood flow to temporal and parietal areas to the pleasant than to the unpleasant pictures, and, making matters more ambiguous, the amplitudes of the event-related waveform eight-tenths of a second after the appearance of the photographs are equivalent to the pleasant and unpleasant scenes. A scientist who wanted to know whether unpleasant or pleasant scenes were more arousing could arrive at three different conclusions depending on the evidence selected.

Daniel Engber in Slate has more excellent discussion of this problem.

Similarly, many posts ago, I noted that another Harvard psychologist, Dan Gilbert, prefers to simply ask people if they are happy rather than use a physiological measure because the only reason we think a particular physiological measure indicates happiness is because it correlates with people's self-reports of being happy. In other words, using any physiological measure (including brain scans) as indication of a mental state is circular.


----
Kagan (2007) What Is Emotion, pp. 81-82.

----
PS Since I've been writing about Russian lately, I wanted to mention an English-language Russian news aggregator that I came across. This site is from the writer behind the well-known Siberian Light Russia blog.

Fractured Consciousness

The modern scientific consensus is that the 'mind' is just a word we use to describe our experience of our own brains in action. That is, mind and brain are more or less the same thing, just described at different levels (this gets stuck in the semantics because the brain monitors some nonconscious things such as heart rate, activities not normally thought of as in the domain of the mind).

That said, some in the scientific community and many in the general community have difficulty buying this 'astonishing hypothesis' (check out the comments to my last post on the subject).

Different people arrive at the hypothesis by their own paths. To me, the most compelling evidence is the range of bizarre consequences of brain damage. For instance, check out this late-December New York Times profile of a recent case of blind-sight, a phenomenon in which a person, due to brain damage, believes herself to be blind, but is clearly able to see. Oliver Sacks books are full of such cases, such as hemispheric neglect, in which people lose their awareness of half the world, to the extent that they eat from only one side of their plate, shave only one side of their face, and may even only be able to turn in one direction. A recent obituary of a famous amnesic noted how work with amnesics has shown that losing one's ability to form memories is in essence losing part of oneself.

Data like these make it hard to save dualism. If there is a non-material soul, it is not responsible for memory, for having a sense of left or right, or probably even for consciousness itself. That doesn't seem to leave much for the non-material soul to do. This conclusion may be disheartening, but it seems inescapable.

How good is your memory?

The average 20-29 year old scores a 2.5 on my Memory Test. How well can you do?

There are, of course, different types of memory. Most people think of 'memory' as an ability to recall facts and events from days or even years ago. This is what was destroyed in the famous amnesic H. M. However, H. M. was still able to remember new information for at least a few seconds; that is, his short-term ("working") memory was spared. There are also other types of memory, such as iconic memory, also knows as "sensory" memory. Moreover, memory for facts seems to dissociate from memory for skills ("know-how").

The Memory Test tests visual working memory.

Before you take the test, please do me one favor. If you want to test yourself multiple times, feel free to do so. But please check off the "have you done this experiment before" box. Failing to do this can screw up the data, so it's important.

What Does the Test Involve?

You try to remember four simple shapes for one second. Afterwards, you are shown a single shape. You have to decide if it is one of the four you were to remember. There are 40 trials, plus some practice trials.

A note about the practice: The practice trials are really, really hard. That is to get you warmed up, just like a runner tying weights to her ankles during her warm-up. The actual test is easier.

How is the Score Calculated?

On any given trial, you get the answer either right or wrong. We could just calculate what percentage you get right, but that would mean getting a score like "80%," which isn't very satisfying. 80% of what?

A formula developed by Nelson Cowan can be used to estimate how many of the shapes, on average, actually make it into your short-term memory store. The formula is this:

(% hits + % correct rejections - 1) / (Total number of objects)

A 'hit' means answering 'yes, this is one of the four objects,' when in fact that is the correct answer. A 'correct rejection' is saying 'no, this is not one of the four object,' when in fact it is not.

From the math, the score can run from -1/4, if you get every question wrong, to 4, if you get every question right (which has happened, but rarely). If you guessed at random, you should get half the questions right, in which case your score should be 0.

Keep in mind that this depends completely on the shapes. If the shapes are really hard to remember (as the practice shapes are), scores will be lower. If they are very easy, scores will be higher. What makes a shape easy is not just how complex it is, but how similar it is to the other shapes (how easy the shapes are to confuse with one another).

What Does the Score Mean?

You could have a higher or lower score for a number of reasons. For one thing, you might have guessed abnormally well or abnormally poorly. All tests are subject to a guessing effect. On average, guessing cancels itself out, but if the test is short enough and enough people take is, somebody is likely to get everything right (or wrong) just by chance.

Luck aside, a good score could mean that you have more "room" in your short-term memory. It might also mean you are better at avoiding interference. There are several types of interference in memory, and so you could be better at avoiding any one of them. You might also be better at paying attention, or you might have developed a useful strategy for success on this task. (That said, visual short-term memory does appear to be anywhere near as susceptible to strategies as verbal short-term memory.)

Remember one thing. This is not a clinical test. Though clinical tests for verbal short-term memory exist, I'm not sure there even are clinical tests for visual short-term memory. This is just for fun. Enjoy it.

Wait. How Do you Know What the Average Score Is?

The Memory Test is nearly identical to an experiment I ran previously. I used the data from that version to estimate what the scores will be on this version.

(Photo served from the National Geographic website)

Androids Run Amok at the New York Times?

I have been reading Steve Pinker's excellent essay in the New York Times about the advent of personal genetics. Reading it, though, I noticed something odd. The Times includes hyperlinks in most of its articles, usually linking to searches for key terms within its own archive. I used to think this linking was done by hand, as I do in my own posts. Lately, I think it's done by an android (and not a very smart one).

Often the links are helpful in the obvious way. Pinker mentions Kareem Abdul-Jabbar, and the Times helpfully links to a list of recent articles that mention him. Presumably this is for the people who don't know who he is (though a link to the Abdul-Jabbar Wikipedia entry might be more useful).

Some links are less obvious. In a sentence that begins "Though health and nutrition can affect stature..." the Time sticks in a hyperlink for articles related to nutrition. I guess that's in case the word stirs me into wondering what else the Times has written about nutrition. That can't explain the following sentence though:

Another kind of headache for geneticists comes from gene variants that do have large effects but that are unique to you or to some tiny fraction of humanity.

There is just no way any human thought that readers would want a list of articles from the medical section about headaches. This suggests that the Times simply has a list of keywords that are automatically tagged in every article...or perhaps it is slightly more sophisticated and the keywords vary based on the section of the paper.

I'm not sure how useful this is even in the best of circumstances. Has anyone ever actually clicked on one of these links and read any of the articles listed? If so, comment away!

(picture from Weeklyreader.com)

Skeptical of the Skeptics

I have complained -- more than once -- that the media and the public believe a psychological fact (some people are addicted to computer games) if a neuroimaging study is somehow involved, even if the study itself is irrelevant (after all, the definition of addiction does not require a certain pattern of brain activity -- it requires a certain pattern of physical activity).

Not surprisingly, I and like-minded researchers were pleased when a study came out last year quantifying this apparent fact. That is, the researchers actually found that people rated an explanation of a psychological phenomenon as better if it contained an irrelevant neuroscience fact.

Neuroskeptic has written a very provocative piece urging us to be skeptical of this paper:

This kind of research - which claims to provide hard, scientific evidence for the existence of a commonly believed in psychological phenomenon, usually some annoyingly irrational human quirk - is dangerous; it should always be read with extra care. The danger is that the results can seem so obviously true ("Well of course!") and so important ("How many times have I complained about this?") that the methodological strengths and weaknesses of the study go unnoticed.

Read the rest of the post here.

Is language just statistics?

Many years ago, I attended a talk in which a researcher (in restrospect, probably a graduate student) was talking about some work she was doing on modeling learning. She mentioned that a colleague was very proud of a model he had put together in which he had a model world populated by model creatures which learned to avoid predators and find food.

She reported that he said, "Look, they are able to learn this without *any* input from the programmer. It's all nurture, not nature." She argued with him at length to point out that he had programmed into his model creatures the structures that allowed to them to learn. Change any of those parameters, and they ceased to learn.

There are a number of researchers in the field of language who, impressed by the success of statistical-learning models, argue that much or all of language learning can be accomplished by simply noticing statistical patterns in language. For instance, there is a class of words in English that tend to follow the word "the." A traditional grammarian might call these "nouns," but this becomes unnecessary when using statistics.

There are many variants of this approach, some more successful than others. Some are more careful in their claims (one paper, I recall, stated strongly that the described model did away with not only grammatical rules, but words themselves).

While I am impressed by much of the work that has come out of this approach, I don't think it can ever do away with complex (possibly innate) structure. The anecdote above is an argument by analogy. Here is a great extended quote from Language Learnability and Language Development, Steven Pinker's original, 1984 foray into book writing:

As I argued in Pinker (1979), in most distributional learning procedures there are vast numbers of properties that a learner could record, and since the child is looking for correlations among these properties, he or she faces a combinatorial explosion of possibilities. For example, he or she could record of a given word that it occurs int eh first (or second, or third, or nth) position in a sentence, that it is to the left (or right) of word X or word Y or ..., or that it is to the left of the word sequence WXYZ, or that it occurs n the same sentence with word X (or words X, Y, Z, or some subset of them), and so on. Adding semantic and inflectional information to the space of possibilities only makes the explosion more explosive. To be sure, the inappropriate properties will correlate with no others and hence will eventually be ignored, leaving only the appropriate grammatical properties, but only after astronomical amounts of memory space, computation, or both.

In any case, most of these properties should be eliminated by an astute learner as being inappropriate to learning a human language in the first place. For example, there is no linguistic phenomenon in any language that is contingent upon a word's occupying the third serial position in a sentence, so why bother testing for one? Testing for correlations among irrelevant properties is not only wasteful but potentially dangerous, since many spurious correlations will arise in local samples of the input. For example, the child could hear the sentences John eats meat, John eats slowly, and the meat is good and then conclude that the slowly is good is a possible English sentence.

Ultimately, a pure-statistics model still has to decide what regularities to keep track of and what to ignore, and that requires at least some innate structure. It probably also requires fairly complex grammatical structures, whether learned or innate.

The Purpose of Language

A book I'm currently reading quotes the well-known linguist Charles Fillmore as writing

the language of face-to-face conversation is the basic and primary use of language, all others being best described in terms of their manner of deviation from that base... I assume that this position is neither particularly controversial nor in need of explanation.

If only it were so. Uber-linguist Noam Chomsky said in a talk I attended that language is not "for communication." I've never been quite sure what he meant by this, so I decided this was a good time to find out.

Googling turned up this interview, in which his statement is much more mild. He seems to simply state that to the extent language is used socially, it isn't always for the purpose of communication. I can get on board with that.

This other interview, however, makes a stronger claim. Here is a representative quote:

If human language has a function at all it's for expression of thought. So if you just think about your own use of language, a rather small part is used for communication. Much of human language is just used to establish social relations. Suppose you go to a bar in Kyoto and you spend an evening talking to your friends. You're not 'communicating'. You're rarely communicating. You're not presenting them with any information that changes their belief systems. You're simply engaged in a kind of social play.

Perhaps. I'm still with Fillmore that this seems to be derivative on communication, but I'm not even sure what kind of evidence could be found that would favor one position or the other.

Cutting Down Trees to Save the Forest

It took me a while to understand the concept of ecotourism. When I signed up to spend a year with the just-emergent Great Baikal Trail organization from 2003-2004, I honestly picked it for reasons unrelated to its mission: building nature trails.

Baikal is the world's largest (by volume) and deepest lake, with 20% of the world's fresh water and a dizzying array of species found nowhere else (some of them very tasty, I have to admit). It's an incredible place to visit, and I feel lucky to have spent so long there (for reasons, read this and this). It is also one of the world's more pristine habitats, by virtue largely of being in the middle of Siberia. However, the region will eventually develop, and the question is how.

GBT operates on a If-You-Build-It-They-Will-Come principle: namely, if the right infrastructure is put in place, an industry built on tourism will develop, displacing the most likely alternative possibility, which is logging and paper mills (both of which are necessary activities, but would be a shame to see in the Baikal area).

More importantly, if the tourism economy is based on the local natural wonders, there is strong economic pressure to maintain Baikal in its pristine state. It might stay more pristine if all the humans moved elsewhere, but that's not likely to happen. For one thing, if nobody lived there and nobody visited, who has the motivation to maintain the Lake?

The Great Baikal Trail runs summer trail-building programs. An eco-trail is not just any rut in the ground, but requires sophisticated engineering (to prevent, among things, erosion). GBT has been running for over half a decade now and has built up considerable expertise. If anybody is interested in volunteering on a two-week trail project, look them up. Since the tourism economy is still nascent, this also represents one of the few opportunities for non-Russian speakers to travel extensively in the region.

P.S. If there are any Russian speakers reading this, please check out my new 5-minute experiment in Russian.

(for once, photographs are my own)

Building a Better Spell-Checker

Today Slate carried an interesting piece about spell-checker technology by Chris Wilson. A spell-checker typically works in the obvious way: a word you type in is compared to a dictionary. The question is where the dictionary comes from. If you use a lot of proper nouns -- or, in my case, a lot of technical jargon -- you risk the red-squiggly wrath of Microsoft Word.

It's been clear to me for a while that search engines work from much larger lexicons than do word processors. The article fills in some detail as to how they do this (not surprisingly, it involves some of the sophisticated statistics that has become so important in computer approaches to language). Read the article here.


(image borrowed from eduscapes.com)

Science in the New Administration

During the Fall, I wrote a number of posts (starting with this one) arguing that science policy in the US was in poor shape, and that Obama looked like the more likely of the candidates to turn that around.

While I focused more on support for basic science, I was certainly concerned about how policy-makers use science to support their policy decisions. So far, all signs from the not-yet-nascent Obama administration continue to be promising.

Children giving orders to Mom and Dad

During the last month, I have been studying requests. Requests are interesting because there are many ways of making them, including commands ("Give me that"), requests ("Please give me that"), indirect requests ("Could you give me that?"), and hints ("Wouldn't it be nice if I had one of those?").

I just ran across a description of a fairly old line of research that is worth quoting directly:

Studies of role playing (Andersen (1978, 1989), Corsaro (1985), Mitchell-Kernan and Kernan (1977) have made it very clear that children make use of the symbolic value of characters' control acts types and forms. In Andersen's study, children of four and five were assigned specific roles through puppets, and she played a complementary role. This allowed her to see, within each child, the representation of contrasting roles, such as Father and Mother and Child, Doctor and Nurse and Patient. Fathers received fewer orders but gave them more, and received few imperatives, but gave them. Doctors were the same. The Child addressed six times as many imperatives to Mothers as to Fathers, and eight times as many 'let's' forms to Fathers as to the Mothers.

It would appear that children not only think fathers outrank mothers on the dominance hierarchy but that they seem to think they themselves outrank their mothers. Why this is I leave to others to speculate on.

Ervin-Tripp, S, Guo, J., Lampert, M. (1990). Politeness and persuasion in children’s control acts. Journal of Pragmatics, 14, 307-331

(Photo borrowed from http://www.radconsultancy.com/).

Mind and Brain

In periodic posts, I've been trying to lay out the modern scientific consensus on the mind/brain problem, with mixed success. If I had come across the following passage, from Ray Jackendoff's Language, Consciousness, Culture, a bit earlier, I might have saved some trouble, since I feel it is one of the clearest, most concise statements on the topic I have seen:

The predominant view is a strict materialism, in which consciousness is taken to be an emergent property of brains that are undergoing certain sorts of activity.

Although the distinction is not usually made explicit, one could assert the materialist position in either of two ways. The first would be 'methodological materialism': let's see how far we can get toward explaining consciousness under materialist assumptions, while potentially leaving open the possibility of an inexplicable residue. The second would be 'dogmatic materialism,' which would leave no room for anything but materialist explanation. Since we have no scientific tools for any sort of nonmaterialist explanation, the two positions are in practice indistinguishable, and they lead to the same research...

Of course, materialism goes strongly against folk intuition about the mind, which concurs with Descartes in thinking of the conscious mind as associated with a nonmaterial 'soul' or the like... The soul is taken to be capable of existence independently of the body. It potentially survives the death of the body and makes its way in the world as a ghost or a spirit or ensconced in another body through reincarnation... Needless to say, most people cherish the idea of being able to survive the death of their bodies, so materialism is more than an 'astonishing hypothesis,' to use Crick's (1994) term: it is a truly distressing and alienating one. Nevertheless, by now it does seem the only reasonable way to approach consciousness scientifically.

CogLangLab по-русски

One of the advantages of posting experiments on the Web rather than running them in a lab is that it makes it easier to recruit participants who don't happen to live near the lab.

Two years ago, I was testing an idea in the literature about the differences between reading an alphabetic script like English vs. reading a character-based script like Chinese. Although there are a fair number of Chinese-speakers living in Cambridge, it was still a lot of work to recruit enough participants. When I finally do the follow-up study, I'll post it on the Web.

In the meantime, I have a new experiment in Russian. Because America produces the bulk of the world's scientific research (for now), much of the work on language has focused on English. Periodically, it's a good idea to check other languages and make sure what we know about English generalizes (or doesn't). And so was born

Угадай кто сликтопоз

It takes about 5 minutes to complete. Participate in it by clicking here.

More amazing birds

I've been hearing for some time that starlings are remarkable vocal learners. For a brief time, there was a starling lab in our building.

I came across this video on grrlscientist's blog. I'm not sure if "amazing" or "creepy" is the right response.

Are elders better scientists?

A recent paper, discussed in a recent issue of Nature, found that across disciplines, professors in their 50s and 60s published about twice the number of papers each year as professors in their 30s. This is taken in the article as evidence that older professors can be very productive.

Nature allows readers to common on news briefs, and the comments raised the same issues I had. Here are the first two, for instance:

They don't seem to consider that older professors have larger research groups, i.e. more underlings to actually write the papers. Perhaps a better photo to illustrate the story would be the aged professor in their office wielding a red pen over their students' manuscripts.

Well, the older professors are also more established and have more connections, and therefore can participate in both small and large collaborative projects. No offense, but this survey only seems to prove an already obvious point.

Basically, older faculty tend to not only have more graduate students and post-docs, they also tend to have broad collaboration networks. This is not to say that older researchers are not productive, or that even less-productive older researchers aren't valuable members of the community, just that these data seem hard to interpret.