Thinking outside the circle

I'm currently at the CUNY Human Sentence Processing conference. I'll start blogging the most interesting reports soon, but at the moment I'm too busy conferencing to actually write about the conference. In the meantime, in honor of the conference, I give you this slide from GraphJam:

Why Aren't More Kids Studying Behavioral Sciences?

The New York Times has a profile of a finalist in the Intel Science Talent Search (for those of you who missed the change, that's the renamed Westinghouse competition). Newspaper articles -- by style less than by design -- are often cryptic, and this one notes in a single-sentence paragraph towards the end that the profiled student's project is "the only behavioral science project among the 40 finalists."

The article has no more to say on the subject, so we are left to guess what the author found so remarkable about this. It probably does not directly reflect a bias of the judges. I haven't been to a science fair recently, but I suspect that now, as always, they are dominated by chemistry and biology. When our nations students are steered towards science, the behavioral sciences are an afterthought.

This is unfortunate. Certainly, we need more talented people entering medical research or developing new energy sources, but recent events also highlight the need for advancements in market forecasting (predict human behavior) and regulatory schema (control human behavior) -- just to name a few vexing behavioral science problems.

Are Web-Based Experiments Reliable? The Data Say 'Yes.'

After a few months, I'm back to the task of getting the Video Test experiments published. As I mentioned last year, the paper had run aground partly due to reviewers' skepticism about Web-based experiments.

I sat down to improve the section of the paper that justifies using Web-based experiments. That required looking for other published experiments. I've done this haphazardly over the years, but this time I was much more systematic. I knew there were a fair number of published Web surveys, but I was surprised to discover there are many, many more published Web-based experiments than I thought. I also turned up a fairly large number of studies in which researchers directly compared Web-based and lab-based studies, typically finding the former to be as reliable as the latter.

In fact, I found so much I almost felt silly writing the justification. It seems strange to be justifying what has become essentially a well-established method. In fact, many researchers who use write up Web-based experiments don't even bother to do so.

The Data
Without further ado, here is a draft of that justification:

Internet-based experiments have become increasingly popular in recent years, with at least 21% of APA journals having published at least one paper relying on Internet-based methods (Skitka & Sargis, 2006). In the cognitive and perceptual research, domains in which the methodology has been particularly productive include face perception (inter alia, Bestelmeyer, Jones, DeBruine, Little & Welling, in press; Boothroyd, Jones, Burt, Cornwell, Little, Tiddeman & Perrett, 2005; Feinberg, DeBruine, Jones & Little, 2008; Feinberg, Jones, DeBruine, Moore, Smith, Cornwell, Tiddeman, Boothroyd & Perrett, 2005; Fessler & Navarrete, 2003; Little, Burriss, Jones, DeBruine & Caldwell, 2008; Little, Jones & Burriss, 2007; Little, Jones, Burt & Berrett, 2007; Little, Jones & DeBruine, 2008; Little, Jones, DeBruine & Feinberg, 2008; Smith, Jones DeBruine & Little, in press; Welling, Jones & DeBruine, 2008; Wilson & Daly, 2004) and reaction-time based studies of implicit social biases (inter alia, Bar-Anan, Nosek & Vianello, in press; Graham, Haidt & Nosek, in press; Lindner & Nosek, 2009; Nosek & Hansen, 2008; Ranganath & Nosek, 2008; Schwartz, Vartanian, Nosek & Brownell, 2006).

A number of researchers have directly compared the results of Internet-based and laboratory-based studies, finding that the former are highly reliable and the two methods produce similar results, both within and between subjects (Buchanan, T., & Smith, J. L., 2000; Gosling, Vazire, Srivastava & John, 2004; Linnman, Carlbring, Ahman, Anderesson & Andersson, 2004; McGraw, Tew, & Williams, 2000; Meyerson & Tryon, 2003; Ollesch, Heineken & Schulte, 2006; Srivastava, John, Gosling & Potter, 2003). Importantly for the present work, a recent study of VWM found converging results from Internet-based and Laboratory-based methods (Hartshorne, 2008).




Bar-Anan, Y., Nosek, B. A., & Vianello, M. (in press). The sorting paired features task: A measure of association strengths. Experimental Psychology.

Bestelmeyer, P. E. G., Jones, B. C., DeBruine, L. M., Little, A. C., & Welling, L. L. M. (in press). Face aftereffects demonstrate interdependent processing of expressions and the invariant characteristics of sex and race. Visual Cognition.

Boothroyd, L. G., Jones, B. C., Burt, D. M., Cornwell, R. E., Little, A. C., Tiddeman, B. P., & Perrett, D. I. (2005). Facial masculinity is related to perceived age but not perceived health. Evolution and Human Behavior, 26, 417-431.

Buchanan, T., & Smith, J. L. (1999). Using the Internet for psychological research: Personality testing on the World Wide Web. British Journal of Psychology, 90, 125-144.

Feinberg, D. R., DeBruine, L. M., Jones, B. C., & Little, A. C. (2008). Correlated preferences for men’s facial and vocal masculinity. Evolution and Human Behavior, 29, 233-241.

Feinberg, D. R., Jones, B. C., DeBruine, L. M., Moore, F. R., Smith, M. J. L., Cornwell, R. E., Tiddeman, B. P., Boothroyd, L. G., Perrett. (2005). The voice and face of woman: One ornament that signals quality? Evolution and Human Behavior, 26, 398-408.

Fessler, D. M. T., & Navarrete, C. D. (2003). Domain-specific variation in disgust sensitivity across the menstrual cycle. Evolution and Human Behavior, 24, 406 – 417.

Gosling, S. D., Vazire, S., Srivastava, S. & John, O. P. (2004). Should we trust web-based studies? A comparitive analysis of six preconceptions about Internet questionnaires. American Psychologist, 49, 93-104.

Graham, J., Haidt, J., & Nosek, B. A. (in press). Liberals and conservatives rely on different sets of moral foundations. Journal of Personality and Social Psychology.

Hartshorne, J. K. (2008). Visual working memory capacity and proactive interference. PloS ONE 3(7): e2716.
Lindner, N. M., & Nosek, B. A. (2009). Alienable speech: Ideological variations in the application of free-speech principles. Political Psychology, 30 67-92.

Linnman, C., Carlbring, P., Ahman, A., Andersson, H., & Andersson, G. (2004). The Stroop effect on the Internet. Computers in Human Behavior, 22, 448-455.

Little, A. C., Burriss, R. P., Jones, B. C., DeBruine, L. M., & Caldwell, C. C. (2008). Social influence in human face preference: men and women are influenced for long-term but not short-term attractiveness decisions. Evolution and Human Behavior, 29, 140-146.

Little, A. C., Jones, B. C., & Burriss, R. P. (2007). Preferences for masculinity in male bodies change across the menstrual cycle. Hormones and Behavior, 52, 633-639.

Little, A. C., Jones, B. C., Burt, D. M., & Perrett, D. I. (2007). Preferences for symmetry in faces change across the menstrual cycle. Biological Psychology, 76, 209-216.

Little, A. C., Jones, B. C., & DeBruine, L. M. (2008). Preferences for variation in masculinity in real male faces change across the menstrual cycle. Personality and Individual Differences, 45: 478-482.

Little, A. C., Jones, B. C., DeBruine, L. M., & Feinberg, D. R. (2008). Symmetry and sexual-dimorphism in human faces: Interrelated preferences suggest both signal quality. Behavioral Ecology, 19: 902-908.

Meyerson, P. & Tryon, W. W. (2003). Validating Internet research: a test of the psychometric equivalence of Internet and in-person samples. Behavior Research Methods, Instruments & Computers, 35, 614-620.

Nosek, B. A., & Hansen, J. J. (2008). The associations in our heads belong to us: Searching for attitudes and knowledge in implicit evaluation. Cognition and Emotion, 22, 553-594.

Heike Ollesch, Edgar Heineken, Frank P. Schulte (2006). Physical or virtual presence of the experimenter: Psychological online-experiments in different settings International Journal of Internet Science, 1 (1), 71-81

Ranganath, K. A., & Nosek, B. A. (2008). Implicit attitude generalization occurs immediately, explicit attitude generalization takes time. Psychological Science, 19, 249-254.

Schwartz, M. B., Vartanian, L. R., Nosek, B. A., & Brownell, K. D. (2006). The influence of one's own body weight on implicit and explicit anti-fat bias. Obesity, 14(3), 440-447

Smith, F. G., Jones, B. C., DeBruine, L. M., & Little, A. C. (in press). Interactions between masculinity-femininity and apparent health in face preferences. Behavioral Ecology.

Srivistava, S., John, O. P., Gosling, S. D., & Potter, J. (2003). Development of personality in early and middle adulthood: Set like plaster or persistent change? Journal of Personality and Social Psychology, 84, 1041-1053.

Welling, L. L. M., Jones, B. C., & DeBruine, L. M. (2008). Sex drive is positively associated with women’s preferences for sexual dimorphism in men’s and women’s faces. Personality and Individual Differences, 44(1): 161-170.

Wilson, M., & Daly, M. (2004). Do pretty women inspire men to discount the future? Proceedings of the Royal Society of London B: Biological Sciences, 271, S177-S179.

Alzheimer's, Autism & the NCAA: Science News for 3/17

Do vaccines give Somalis autism? Can diabetes give you Alzheimer's? Does losing make you win? Anyone scanning the science news articles this week would know the answers to these questions.

First, Freakonomics has a discussion of a recent paper showing that NCAA basketball teams are more likely to win if they are 1 point behind at halftime than if they are 1 point ahead. It seems that when people are slightly behind in a game at halftime, they work even harder in the second half relative to people who are way behind, slightly ahead or way ahead.

Second, the New York Times (byline: Donald McNeil Jr.) discusses the abnormally high rate of autism among Somali immigrants in Minneapolis. The article gives several explanatory hypotheses (including a statistical fluke), but a lot of time is spent on the "possibility" that these cases of autism are caused by vaccinations. The fact that the article doesn't mention that this is simply absurd is glaring (though it does mention "some children" had autistic tendancies before being vaccinated). More interesting is that many of these kids appear to have had seizures, something which is mentioned only in passing.

Finally, Amanda Schaffer at Slate discusses the possible relationship between insulin and Alzheimer's (Diabetes of the Brain: Is Alzheimer's disease actually a form of diabetes?).

The Scientist as Parent

A few weeks ago, the New York Times carried an alarmist and basically silly article about cognitive scientists studying their own children. Alarmist because it suggested this is an ethically grey area. For the types of experiments they were talking about, there are no ethical issues. Their opening story was about Pawan Sinha spending a few hours videotaping his newborn baby's environment. 

Silly, because observant parents have always been interested in what their children are thinking about how their babies develop from infants to walking, talking human beings. Muscian parents, I imagine, spend extra time singing to their babies. Artist parents no doubt are interested in their newborn's sense of creativity. The parental fascination of scientist parents often gets channeled into...well, science.

Renaisauce has just written a much more insightful and ultimately sweet essay about parental love from the perspective of a neuroscientist who is also a new parent. I won't quote from it, because I recommend you read it in its entirety.

(Picture of Darius Sinha borrowed from the New York Times)

The Academic Job Market Tanks

"This is a year of no jobs." Ph.D.s are stacked up "like planes hovering over La Guardia. -- Catherine Stimpson, dean of the Graduate School of Arts and Sciences at New York University.

The above quote is taken from a recent article in the New York Times. Although people usually flock to graduate school in a down economy, the down economy means fewer spots in graduate school. This is just as well, it seems, if there are fewer jobs for graduating Ph.D.s.

The article is based mostly on anecdote, but the anecdotes match what I have seen as well. A graduate student from UT-Austin frets that more and more job searches have been pulled as universities announce hiring freezes. Two colleagues of mine who were on the market this year also reported jobs they had applied for disappearing. One has managed to find a post-doc position; the future of the other is uncertain.

For those who want numbers, there are a few in the article. It reports 15% drop in history department job searches and a 25% drop in the length of the American Mathematic Association's largest list of job postings.

In addition to the problems faced by people on the market, this is problematic for a country that wants to increase its intellectual output. Ph.D.s are long and hard and not worth it if there is no job at the end. Discouraging employment figures are not going to help the president's goal of increasing our nation's supply of scientists and engineers. To the extent that the work of historians and area-studies researchers informs policy, it seems we'd want to make sure there are employment prospects for humanities students as well.

Again, the Times has no numbers, but the article quoted a few discouraged undergraduates who are putting off graduate study (though frankly I don't think going straight from undergraduate to graduate programs is a good idea, anyway). Moreover, they point to Thomas Benton, a columnist for The Chronicle of Higher Eduction -- academia's trade journal -- who has been actively discouraging students from going into the humanities, arguing that it makes no sense unless you are wealthy or well-connected. I'm not sure undergraduates read the Chronicle, but the existence of that sentiment is troubling.

Ours is a knowledge-driven economy. Everybody seems to recognize that in the push to get more Americans to go to college. Hopefully, there will be professors there to teach them.

Will There be a Neuroscience Culture War?

Martha Farah (Neuroscientist, UPenn) and Nancey Murphy (Theologian, Fuller Theological Seminary), writing in a recent issue of Science, argue that "neuroscience will post a far more fundamental challenge than evolutionary biology to many religions."

The reason is straightforward:

Most religions endorse the idea of a soul (or spirit) that is distinct from the physical body ... However, as neuroscience begins to reveal the mechanisms underlying personality, love, morality, and spirituality, the idea of a ghost in the machine becomes strained. Brain imaging indicates that all of these traits have physical correlates in brain function. Furthermore, pharmacologic influences on these traits, as well as the effects of localized stimulation or damage, demonstrate that the brain processes in question are not mere correlates but are the physical bases of these central aspects of our personhood. If these aspects of the person are all features of the machine, why have a ghost at all? [Emphasis mine.]

While not at all detracting from their point, it's interesting that neuroscience does not yet seem to be a major target of religious conservatives. The authors argue that such a backlash is a brewin' ("'Nonmaterialist neuroscience' has joined 'intelligent design' as an alternative interpretation of scientific data"), but the evidence is a recently published book. The term gets a paltry number of Google hits, the first few of which, at least, are people attacking the concept.

They make one further interesting point: dualism is a relatively new concept, which came into existence about a century later than Jesus. By implication, those who insist on a strict interpretation of the Bible actually should support materialism. If the culture war comes, this is unlikely to make a compelling argument, but it does say something very interesting about human nature.

Yes, HR 801 is about Open-Access Journals

Stevan Harnad has questioned some of the claims in my last post. I hate to say this, as a part-time semanticist, but Harnad's criticisms of the above post are mostly semantics. He did more accurately quote the letter of the NIH and Harvard policies (and I apologize for my sloppy wording), but I believe my formulation got much closer to the intent and impact of these policies.

Harnad points out that the NIH and Harvard policies do not mandate publishing in open-access journals; they simply mandate making publications open-access after a certain period of time. As far as I can tell, pretty much everybody -- including Conyers, the bill's sponsor, and probably Harnad -- understands that these policies hurt subscription-based journals. After all, it is very hard to sell access to something which is free.

It might not have been clear from his comment, but Harnad supports free access to peer-reviewed research. His focus is not on open-access journals, however, so he has some stake in pointing out that there are other open-access models. I appreciate this point.

Still, I think the key issue is that these papers still need to go through peer-review and publication, something Harnad goes to pains to point out elsewhere. Currently, we have two models: subscription-based journals, which pass on the costs of review and publication to the reader, and open-access journals, which pass on the costs to the author or a private foundation. If these policies make the subscription-based journals less profitable, then the open-access journals presumably become more competitive.

Currently, open-access journals have several things working against them. For one thing, they (typically) cost the authors money. I have more than once wanted to submit a paper to the Journal of Vision, but I can't accord the $85/page publication costs. Also, the open-access journals are newer and less prestigious, and in academics, prestige of the journal can count for a great deal. If the open-access policies force subscription-based publishers to raise their own publication fees or go out of business, this presumably should help open-access journals .

I used "presumably" a few times, and if there are good reasons to believe that policies like those of NIH and Harvard harm open-access journals and subscription journals alike, then I'd like to know about them.

--
One minor point about the distinction between for-profit and non-profit journals: Subscription-based journals, whether for-profit or non-profit, sell subscriptions and thus are paid for their role in publication. Thus, I'm not really sure what Harnad was getting at in pointing out that some non-profit journals also support Conyers' bill, which is intended to support the subscription model.

Congress Considers Killing Open-Access Journals

It's often repeated that science thrives on the free exchange of ideas. Thus, the fact that it actually costs a lot of money to get access to scientific papers ($146/year for Science alone) has struck more than one person as odd.

A recent movement has led to the creation of open-access journals, which do not charge access fees. This movement has gained traction at universities (e.g., Harvard) and also at government agencies. NIH recently required the researchers they fund to publish in journals which are either open-access or make their papers open-access within a year of publication.

Fortunately for the for-profit journal system, Congress is considering H.R. 801, which would forbid NIH and other government agencies from implementing such policies. The conceit of the bill is that NIH is requiring researchers to give up their copyrights, though of course researchers hardly ever -- and, as far as I know, never -- retain the copyrights to their works. Publishers require the transfer of the copyright as a condition of publication.

Talking Robots

Farhad Manjoo has an article in Slate on text-to-voice technology. We've come a long way in making talking robots in recent years (does anybody remember the old Mac OS's terrible talking voice?), but the technology has a long way to go yet. The article has a good historical overview and some great sound clips.

Can Peer Review Solve Conflicts of Interest?

As I wrote recently, Stephen Quake has been writing about conflicts of interest in research over at The Wild Side blog. He proposes solving these problems with peer review. I like the article, and he has many thoughtful things to say on the topic, but I don't really understand this proposal.

He doesn't give a lot of details as to how this would work. For instance:

When this bureaucracy asked me for a plan to manage conflicts in my own research, I wrote one that described all of the steps involved in peer review – and the COI committee sent it back as “too much.” In their view the process that scientific publications go through was more rigorous than necessary.

This reads as if the committee thought it would be too much work for him, but it sounds like too much work for the peer reviewers. Peer reviewers are not professional peer reviewers: they are typically volunteers from within the community who review papers partly out of a sense of social responsibility. Any proposal for expanding peer review has to keep in mind that the reviewers might not want the extra responsibilities.

I have not seen his proposal, but I'm not sure if it would work even in principle. As I wrote previously, nobody really knows what the data are but the researcher. Analyses can always be run in many different ways, potentially giving different results -- a problem whether your concern is conscious bias or unconscious bias. It's just not clear what peer review is supposed to do about that.

Scientists vs. Engineers

What is the difference between a scientist and an engineer? Cowbirds in Love boils it down to its essence:

Experimental Pragmatics & Visa Problems

Experimental pragmatics is the science of language use. It's also a great conference held in Europe every year and a half or so. I'll be attending this spring to present a poster. Well, the poster is an excuse; mainly it is simply the conference for my field, and just about everybody who works on problems similar to the ones I work on will be there.

I am excited about going to Europe for the conference (the other two I'm going to this spring are in Denver and in Davis, California), which is a good thing, since we may all be doing more of that in the future.

This morning, the New York Times reports that conference organizers are increasingly reluctant to hold their conferences in the United States. The reason is a resurgence of visa problems for foreign researchers trying to enter America. Many will remember that this was a major problem post-9/11 and contributed to the precipitous fall of America's standing as a place for higher education. These issues improved somewhat in recent years but have mysteriously cropped up again in the last six months.

The article uses no hard numbers to support its claims about an increase in visa processing times or the decrease in American-based conferences. Still, the decrease in US-based conferences makes sense; American visas are hard to get, and their availability has been unpredictable in recent years.