Field psychology

Most psychology experiments are performed in a laboratory setting. This leads critics to wonder about their ecological validity: that is, just because somebody acts one way in the lab, do you know that is how they would act in real life.

There is another problem, summarized very nicely in a recent paper by Sugiyama and colleagues. In a footnote explaining some "oddities" in the results of their study of one of the world's most remote civilizations (the Shiwiar of the Amazon), they note:

Experimentation under field conditions injects higher levels of error variance into results than are obtainable under well-controlled laboratory conditions. More significant than factors such as added distractions, interruptions, and language difficulties is the extreme cultural strangeness of experimental testing itself, with its unfamiliar necessity of adhering to formal, abstract, and seemingly arbitrary behavioral and communicative constraints. Shiwiar subjects had no prior experience with experimental test-taking situations. This situation introduces confusion into the communicative pragmatics inherent in the task situation, and error variance into results. Restricting one's responses to the question explicitly asked, and ignoring information (such as who may be exhibiting generosity to whom) that is relevant to real life but not to a test problem, is a skill one learns in classrooms and courtrooms.

I have run into this in less exotic locales than the Amazon. As part of my ongoing study of reading, I have tested a number of native Chinese speakers who reside in the US -- mostly graduate students at Harvard. Even though these were smart, well-educated people, a number of them had great difficulty understanding how to do the experiment. Colleagues of mine who study visual perception have had similar difficulties when dealing with Chinese participants. The fact is that psychology experiments are relatively new and relatively rare in Asia, and so fewer people are familiar with what to do. No doubt we American scientists also design our experiments in ways that are culturally familiar to us (and thus not to the Chinese).


Sugiyama, L.S., Tooby, J., Cosmides, L. (2002). Cross-cultural evidence of cognitive adaptations for social exchange among the Shiwiar of Ecuadorian Amazonia. Proceedings of the National Academy of Sciences of the United States of America, 99(17), 11537-11542.