QRTEngine: An easy solution for running online reaction time experiments using Qualtrics

详细信息    查看全文

• 作者：Jonathan S. Barnhoorn ; Erwin Haasnoot ; Bruno R. Bocanegra…
• 关键词：Online experiments ; Qualtrics ; JavaScript ; Amazon Mechanical Turk ; Open ; source
• 刊名：Behavior Research Methods
• 出版年：2015
• 出版时间：December 2015
• 年：2015
• 卷：47
• 期：4
• 页码：918-929
• 全文大小：902 KB
• 参考文献：Asendorpf, J. B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J. J. A., Fiedler, K., & Wicherts, J. M. (2013). Recommendations for increasing replicability in psychology. European Journal of Personality, 27, 108鈥?19. doi:10.鈥?002/鈥媝er.鈥?919 CrossRef
  Behrend, T. S., Sharek, D. J., Meade, A. W., & Wiebe, E. N. (2011). The viability of crowdsourcing for survey research. Behavior Research Methods, 43, 800鈥?13. doi:10.鈥?758/鈥媠13428-011-0081-0 CrossRef PubMed
  Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon鈥檚 Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6, 3鈥?. doi:10.鈥?177/鈥?745691610393980鈥?/span> CrossRef PubMed
  Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013). Evaluating Amazon鈥檚 Mechanical Turk as a tool for experimental behavioral research. PLoS ONE, 8, e57410. doi:10.鈥?371/鈥媕ournal.鈥媝one.鈥?057410 PubMedCentral CrossRef PubMed
  de Leeuw, J. R. (2014). jsPsych: A JavaScript library for creating behavioral experiments in a Web browser. Behavior Research Methods. doi:10.鈥?758/鈥媠13428-014-0458-y
  Eimer, M., & Schlaghecken, F. (2002). Links between conscious awareness and response inhibition: Evidence from masked priming. Psychonomic Bulletin & Review, 9, 514鈥?20. doi:10.鈥?758/鈥婤F03196307 CrossRef
  Germine, L., Nakayama, K., Duchaine, B. C., Chabris, C. F., Chatterjee, G., & Wilmer, J. B. (2012). Is the Web as good as the lab? Comparable performance from Web and lab in cognitive/perceptual experiments. Psychonomic Bulletin & Review, 19, 847鈥?57. doi:10.鈥?758/鈥媠13423-012-0296-9 CrossRef
  Keller, F., Gunasekharan, S., Mayo, N., & Corley, M. (2009). Timing accuracy of Web experiments: A case study using the WebExp software package. Behavior Research Methods, 41, 1鈥?2. doi:10.鈥?758/鈥婤RM.鈥?1.鈥?.鈥?2 CrossRef PubMed
  MacLeod, C. M. (1991). Half a century of research on the Stroop effect: An integrative review. Psychological Bulletin, 109, 163鈥?03. doi:10.鈥?037/鈥?033-2909.鈥?09.鈥?.鈥?63 CrossRef PubMed
  Mann, J. (2012). High Resolution Time. W3C recommendation. Retrieved July 1, 2014, from www.鈥媤3.鈥媜rg/鈥婽R/鈥媓r-time/鈥?/span>
  Mason, W., & Suri, S. (2012). Conducting behavioral research on Amazon鈥檚 Mechanical Turk. Behavior Research Methods, 44, 1鈥?3. doi:10.鈥?758/鈥媠13428-011-0124-6 CrossRef PubMed
  Math么t, S., Schreij, D., & Theeuwes, J. (2012). OpenSesame: An open-source, graphical experiment builder for the social sciences. Behavior Research Methods, 44, 314鈥?24. doi:10.鈥?758/鈥媠13428-011-0168-7 PubMedCentral CrossRef PubMed
  Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010). Running experiments on Amazon Mechanical Turk. Judgment and Decision Making, 5, 411鈥?19.
  Raymond, J. E., Shapiro, K. L., & Arnell, K. M. (1992). Temporary suppression of visual processing in an RSVP task: An attentional blink? Journal of Experimental Psychology: Human Perception and Performance, 18, 849鈥?60. doi:10.鈥?037/鈥?096-1523.鈥?8.鈥?.鈥?49 PubMed
  Schneider, W., Eschman, A., & Zuccolotto, A. (2002). E-Prime [Computer software]. Pittsburgh: Psychology Software Tools.
  Schubert, T. W., Murteira, C., Collins, E. C., & Lopes, D. (2013). ScriptingRT: A software library for collecting response latencies in online studies of cognition. PLoS ONE, 8, e67769. doi:10.鈥?371/鈥媕ournal.鈥媝one.鈥?067769 PubMedCentral CrossRef PubMed
  Simcox, T., & Fiez, J. A. (2014). Collecting response times using Amazon Mechanical Turk and Adobe Flash. Behavior Research Methods, 46, 95鈥?11. doi:10.鈥?758/鈥媠13428-013-0345-y CrossRef PubMed
  Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18, 643鈥?62. doi:10.鈥?037/鈥?096-3445.鈥?21.鈥?.鈥?5 CrossRef
  Ulrich, R., & Giray, M. (1989). Time resolution of clocks: Effects on reaction time measurement鈥擥ood news for bad clocks. British Journal of Mathematical and Statistical Psychology, 42, 1鈥?2. doi:10.鈥?111/鈥媕.鈥?044-8317.鈥?989.鈥媡b01111.鈥媥 CrossRef
  von Bastian, C. C., Locher, A., & Ruflin, M. (2013). Tatool: A Java-based open-source programming framework for psychological studies. Behavior Research Methods, 45, 108鈥?15. doi:10.鈥?758/鈥媠13428-012-0224-y CrossRef
  Woltman, G. (2012). Prime95Webpage. Retrieved from www.鈥媘ersenne.鈥媜rg/鈥媐reesoft/鈥?/span>
  
• 作者单位：Jonathan S. Barnhoorn (1) (2) (5)
  Erwin Haasnoot (1) (3)
  Bruno R. Bocanegra (1) (4)
  Henk van Steenbergen (1) (4)
  
  1. Institute of Psychology, Leiden University, Wassenaarseweg 52, Leiden, 2333, AK, The Netherlands
  2. Department of Cognitive Psychology and Ergonomics, Faculty of Behavioral, Management and Social Sciences, University of Twente, Enschede, The Netherlands
  5. MIRA Research Institute, University of Twente, Enschede, The Netherlands
  3. University of Sheffield, Sheffield, UK
  4. Leiden Institute for Brain and Cognition, Leiden University, Leiden, The Netherlands
  
• 刊物主题：Cognitive Psychology;
• 出版者：Springer US
• ISSN：1554-3528

文摘

Performing online behavioral research is gaining increased popularity among researchers in psychological and cognitive science. However, the currently available methods for conducting online reaction time experiments are often complicated and typically require advanced technical skills. In this article, we introduce the Qualtrics Reaction Time Engine (QRTEngine), an open-source JavaScript engine that can be embedded in the online survey development environment Qualtrics. The QRTEngine can be used to easily develop browser-based online reaction time experiments with accurate timing within current browser capabilities, and it requires only minimal programming skills. After introducing the QRTEngine, we briefly discuss how to create and distribute a Stroop task. Next, we describe a study in which we investigated the timing accuracy of the engine under different processor loads using external chronometry. Finally, we show that the QRTEngine can be used to reproduce classic behavioral effects in three reaction time paradigms: a Stroop task, an attentional blink task, and a masked-priming task. These findings demonstrate that QRTEngine can be used as a tool for conducting online behavioral research even when this requires accurate stimulus presentation times. Keywords Online experiments Qualtrics JavaScript Amazon Mechanical Turk Open-source