tag:blogger.com,1999:blog-7701757403364514168.post914163009502640942..comments2023-10-23T11:13:35.712-04:00Comments on Games with Words: Garbage in, Garbage outEdwardhttp://www.blogger.com/profile/04295927435118827266noreply@blogger.comBlogger2125tag:blogger.com,1999:blog-7701757403364514168.post-49989670583511651392010-07-20T20:49:30.171-04:002010-07-20T20:49:30.171-04:00We haven't really been optimistic about attent...We haven't really been optimistic about attention provided by workers on AMT.<br /><br />As I mentioned in a comment on the blog post, in my studies I found highly variant failure rates on AMT. This obviously depends on a number of factors, as you correctly pointed out. In our post, we have been tacit about whether AMT workers are generally "good" or "bad"; we just reported how they compared to other subject pools in the only comparative study we ran and knew about. For these kinds of data, it's really the more the merrier - experimenters need to know when it's ok to use AMT and when it's not.<br /><br />Regarding the novelty of the procedure suggested in the Oppenheimer et al. (2009), all I can say is that detecting actual attention is hardly a new thing (nor does our post implies that we were ignorant about "catch trials" and the like...). What those authors provided is a theoretical and empirical argument in favor of its use.<br /><br />Thanks for *your* attention :-) I'm glad we involuntarily stimulated some discussion on research methods.Gabriele Paolaccihttp://experimentalturk.wordpress.comnoreply@blogger.comtag:blogger.com,1999:blog-7701757403364514168.post-8471768661882140602010-07-20T14:27:35.188-04:002010-07-20T14:27:35.188-04:00I'm not surprised that exp soc psych hadn'...I'm not surprised that exp soc psych hadn't heard of this, it was a timely reminder for me.<br /><br />Catch trials were mentioned in my first year undergrad, but only for surveys and they were called Lie scales. <br /><br />Thanks for the post, very interesting.Disgruntled PhDhttps://www.blogger.com/profile/00926204336056169207noreply@blogger.com