After a few months, I'm back to the task of getting the Video Test experiments published. As I mentioned last year, the paper had run aground partly due to reviewers' skepticism about Web-based experiments.
I sat down to improve the section of the paper that justifies using Web-based experiments. That required looking for other published experiments. I've done this haphazardly over the years, but this time I was much more systematic. I knew there were a fair number of published Web surveys, but I was surprised to discover there are many, many more published Web-based experiments than I thought. I also turned up a fairly large number of studies in which researchers directly compared Web-based and lab-based studies, typically finding the former to be as reliable as the latter.
In fact, I found so much I almost felt silly writing the justification. It seems strange to be justifying what has become essentially a well-established method. In fact, many researchers who use write up Web-based experiments don't even bother to do so.
The Data
Without further ado, here is a draft of that justification:
Internet-based experiments have become increasingly popular in recent years, with at least 21% of APA journals having published at least one paper relying on Internet-based methods (Skitka & Sargis, 2006). In the cognitive and perceptual research, domains in which the methodology has been particularly productive include face perception (inter alia, Bestelmeyer, Jones, DeBruine, Little & Welling, in press; Boothroyd, Jones, Burt, Cornwell, Little, Tiddeman & Perrett, 2005; Feinberg, DeBruine, Jones & Little, 2008; Feinberg, Jones, DeBruine, Moore, Smith, Cornwell, Tiddeman, Boothroyd & Perrett, 2005; Fessler & Navarrete, 2003; Little, Burriss, Jones, DeBruine & Caldwell, 2008; Little, Jones & Burriss, 2007; Little, Jones, Burt & Berrett, 2007; Little, Jones & DeBruine, 2008; Little, Jones, DeBruine & Feinberg, 2008; Smith, Jones DeBruine & Little, in press; Welling, Jones & DeBruine, 2008; Wilson & Daly, 2004) and reaction-time based studies of implicit social biases (inter alia, Bar-Anan, Nosek & Vianello, in press; Graham, Haidt & Nosek, in press; Lindner & Nosek, 2009; Nosek & Hansen, 2008; Ranganath & Nosek, 2008; Schwartz, Vartanian, Nosek & Brownell, 2006).
A number of researchers have directly compared the results of Internet-based and laboratory-based studies, finding that the former are highly reliable and the two methods produce similar results, both within and between subjects (Buchanan, T., & Smith, J. L., 2000; Gosling, Vazire, Srivastava & John, 2004; Linnman, Carlbring, Ahman, Anderesson & Andersson, 2004; McGraw, Tew, & Williams, 2000; Meyerson & Tryon, 2003; Ollesch, Heineken & Schulte, 2006; Srivastava, John, Gosling & Potter, 2003). Importantly for the present work, a recent study of VWM found converging results from Internet-based and Laboratory-based methods (Hartshorne, 2008).
Bar-Anan, Y., Nosek, B. A., & Vianello, M. (in press). The sorting paired features task: A measure of association strengths. Experimental Psychology.
Bestelmeyer, P. E. G., Jones, B. C., DeBruine, L. M., Little, A. C., & Welling, L. L. M. (in press). Face aftereffects demonstrate interdependent processing of expressions and the invariant characteristics of sex and race. Visual Cognition.
Boothroyd, L. G., Jones, B. C., Burt, D. M., Cornwell, R. E., Little, A. C., Tiddeman, B. P., & Perrett, D. I. (2005). Facial masculinity is related to perceived age but not perceived health. Evolution and Human Behavior, 26, 417-431.
Buchanan, T., & Smith, J. L. (1999). Using the Internet for psychological research: Personality testing on the World Wide Web. British Journal of Psychology, 90, 125-144.
Feinberg, D. R., DeBruine, L. M., Jones, B. C., & Little, A. C. (2008). Correlated preferences for men’s facial and vocal masculinity. Evolution and Human Behavior, 29, 233-241.
Feinberg, D. R., Jones, B. C., DeBruine, L. M., Moore, F. R., Smith, M. J. L., Cornwell, R. E., Tiddeman, B. P., Boothroyd, L. G., Perrett. (2005). The voice and face of woman: One ornament that signals quality? Evolution and Human Behavior, 26, 398-408.
Fessler, D. M. T., & Navarrete, C. D. (2003). Domain-specific variation in disgust sensitivity across the menstrual cycle. Evolution and Human Behavior, 24, 406 – 417.
Gosling, S. D., Vazire, S., Srivastava, S. & John, O. P. (2004). Should we trust web-based studies? A comparitive analysis of six preconceptions about Internet questionnaires. American Psychologist, 49, 93-104.
Graham, J., Haidt, J., & Nosek, B. A. (in press). Liberals and conservatives rely on different sets of moral foundations. Journal of Personality and Social Psychology.
Hartshorne, J. K. (2008). Visual working memory capacity and proactive interference. PloS ONE 3(7): e2716.
Lindner, N. M., & Nosek, B. A. (2009). Alienable speech: Ideological variations in the application of free-speech principles. Political Psychology, 30 67-92.
Linnman, C., Carlbring, P., Ahman, A., Andersson, H., & Andersson, G. (2004). The Stroop effect on the Internet. Computers in Human Behavior, 22, 448-455.
Little, A. C., Burriss, R. P., Jones, B. C., DeBruine, L. M., & Caldwell, C. C. (2008). Social influence in human face preference: men and women are influenced for long-term but not short-term attractiveness decisions. Evolution and Human Behavior, 29, 140-146.
Little, A. C., Jones, B. C., & Burriss, R. P. (2007). Preferences for masculinity in male bodies change across the menstrual cycle. Hormones and Behavior, 52, 633-639.
Little, A. C., Jones, B. C., Burt, D. M., & Perrett, D. I. (2007). Preferences for symmetry in faces change across the menstrual cycle. Biological Psychology, 76, 209-216.
Little, A. C., Jones, B. C., & DeBruine, L. M. (2008). Preferences for variation in masculinity in real male faces change across the menstrual cycle. Personality and Individual Differences, 45: 478-482.
Little, A. C., Jones, B. C., DeBruine, L. M., & Feinberg, D. R. (2008). Symmetry and sexual-dimorphism in human faces: Interrelated preferences suggest both signal quality. Behavioral Ecology, 19: 902-908.
Meyerson, P. & Tryon, W. W. (2003). Validating Internet research: a test of the psychometric equivalence of Internet and in-person samples. Behavior Research Methods, Instruments & Computers, 35, 614-620.
Nosek, B. A., & Hansen, J. J. (2008). The associations in our heads belong to us: Searching for attitudes and knowledge in implicit evaluation. Cognition and Emotion, 22, 553-594.
Heike Ollesch, Edgar Heineken, Frank P. Schulte (2006). Physical or virtual presence of the experimenter: Psychological online-experiments in different settings International Journal of Internet Science, 1 (1), 71-81
Ranganath, K. A., & Nosek, B. A. (2008). Implicit attitude generalization occurs immediately, explicit attitude generalization takes time. Psychological Science, 19, 249-254.
Schwartz, M. B., Vartanian, L. R., Nosek, B. A., & Brownell, K. D. (2006). The influence of one's own body weight on implicit and explicit anti-fat bias. Obesity, 14(3), 440-447
Smith, F. G., Jones, B. C., DeBruine, L. M., & Little, A. C. (in press). Interactions between masculinity-femininity and apparent health in face preferences. Behavioral Ecology.
Srivistava, S., John, O. P., Gosling, S. D., & Potter, J. (2003). Development of personality in early and middle adulthood: Set like plaster or persistent change? Journal of Personality and Social Psychology, 84, 1041-1053.
Welling, L. L. M., Jones, B. C., & DeBruine, L. M. (2008). Sex drive is positively associated with women’s preferences for sexual dimorphism in men’s and women’s faces. Personality and Individual Differences, 44(1): 161-170.
Wilson, M., & Daly, M. (2004). Do pretty women inspire men to discount the future? Proceedings of the Royal Society of London B: Biological Sciences, 271, S177-S179.
- Home
- Angry by Choice
- Catalogue of Organisms
- Chinleana
- Doc Madhattan
- Games with Words
- Genomics, Medicine, and Pseudoscience
- History of Geology
- Moss Plants and More
- Pleiotropy
- Plektix
- RRResearch
- Skeptic Wonder
- The Culture of Chemistry
- The Curious Wavefunction
- The Phytophactor
- The View from a Microbiologist
- Variety of Life
Field of Science
-
-
Don't tell me they found Tyrannosaurus rex meat again!2 weeks ago in Genomics, Medicine, and Pseudoscience
-
-
-
Course Corrections4 months ago in Angry by Choice
-
-
The Site is Dead, Long Live the Site2 years ago in Catalogue of Organisms
-
The Site is Dead, Long Live the Site2 years ago in Variety of Life
-
Does mathematics carry human biases?4 years ago in PLEKTIX
-
-
-
-
A New Placodont from the Late Triassic of China5 years ago in Chinleana
-
Posted: July 22, 2018 at 03:03PM6 years ago in Field Notes
-
Bryophyte Herbarium Survey7 years ago in Moss Plants and More
-
Harnessing innate immunity to cure HIV8 years ago in Rule of 6ix
-
WE MOVED!8 years ago in Games with Words
-
-
-
-
post doc job opportunity on ribosome biochemistry!9 years ago in Protein Evolution and Other Musings
-
Growing the kidney: re-blogged from Science Bitez9 years ago in The View from a Microbiologist
-
Blogging Microbes- Communicating Microbiology to Netizens10 years ago in Memoirs of a Defective Brain
-
-
-
The Lure of the Obscure? Guest Post by Frank Stahl12 years ago in Sex, Genes & Evolution
-
-
Lab Rat Moving House13 years ago in Life of a Lab Rat
-
Goodbye FoS, thanks for all the laughs13 years ago in Disease Prone
-
-
Slideshow of NASA's Stardust-NExT Mission Comet Tempel 1 Flyby13 years ago in The Large Picture Blog
-
in The Biology Files
No comments:
Post a Comment