Many Labs 3: Evaluating participant pool quality across the academic semester via replication

详细信息    查看全文

• 作者：Charles R. Ebersole^a ; ^{cebersole@virginia.edu" class="auth_mail" title="E-mail the corresponding author} ; Olivia E. Atherton^b ; ^{oeatherton@ucdavis.edu" class="auth_mail" title="E-mail the corresponding author} ; Aimee L. Belanger^c ; ^{belangal@miamioh.edu" class="auth_mail" title="E-mail the corresponding author} ; Hayley M. Skulborstad^c ; ^{skulbohm@MiamiOH.edu" class="auth_mail" title="E-mail the corresponding author} ; Jill M. Allen^d ; ^{jill.allen1@montana.edu" class="auth_mail" title="E-mail the corresponding author} ; Jonathan B. Banks^e ; ^{jb2676@nova.edu" class="auth_mail" title="E-mail the corresponding author} ; Erica Baranski^f ; ^{ericanbaranski@gmail.com" class="auth_mail" title="E-mail the corresponding author} ; Michael J. Bernstein^g ; ^{mjb70@psu.edu" class="auth_mail" title="E-mail the corresponding author} ; Diane B.V. Bonfiglio^h ; ^{dbonfig@ashland.edu" class="auth_mail" title="E-mail the corresponding author} ; Leanne Boucher^e ; ^{leanne.boucher@nova.edu" class="auth_mail" title="E-mail the corresponding author} ; Elizabeth R. Brownⁱ ; ^{elizabeth.r.brown@unf.edu" class="auth_mail" title="E-mail the corresponding author} ; Nancy I. Budiman^b ; ^{nibudiman@ucdavis.edu" class="auth_mail" title="E-mail the corresponding author} ; Athena H. Cairo^j ; ^{cairoah@vcu.edu" class="auth_mail" title="E-mail the corresponding author} ; Colin A. Capaldi^k ; ^{colin_capaldi@carleton.ca" class="auth_mail" title="E-mail the corresponding author} ; Christopher R. Chartier^h ; ^{cchartie@ashland.edu" class="auth_mail" title="E-mail the corresponding author} ; Joanne M. Chung^b ; ^{jmhchung@ucdavis.edu" class="auth_mail" title="E-mail the corresponding author} ; David C. Cicero^l ; ^{dcicero@hawaii.edu" class="auth_mail" title="E-mail the corresponding author} ; Jennifer A. Coleman^j ; ^{colemanj3@vcu.edu" class="auth_mail" title="E-mail the corresponding author} ; John G. Conway^m ; ^{john.conway@ufl.edu" class="auth_mail" title="E-mail the corresponding author} ; William E. Davisⁿ ; ^{dbillium@gmail.com" class="auth_mail" title="E-mail the corresponding author} ; Thierry Devos^o ; ^{tdevos@mail.sdsu.edu" class="auth_mail" title="E-mail the corresponding author} ; Melody M. Fletcher^p ; ^{melody.fletcher@eagles.usm.edu" class="auth_mail" title="E-mail the corresponding author} ; Komi German^b ; ^{ktgerman@ucdavis.edu" class="auth_mail" title="E-mail the corresponding author} ; Jon E. Grahe^q ; ^{graheje@plu.edu" class="auth_mail" title="E-mail the corresponding author} ; Anthony D. Hermann^r ; ^{ahermann@bradley.edu" class="auth_mail" title="E-mail the corresponding author} ; Joshua A. Hicksⁿ ; ^{joshua.hicks@gmail.com" class="auth_mail" title="E-mail the corresponding author} ; Nathan Honeycutt^o ; ^{nhoneycutt@mail.sdsu.edu" class="auth_mail" title="E-mail the corresponding author} ; Brandon Humphrey^c ; ^{humphrbt@miamioh.edu" class="auth_mail" title="E-mail the corresponding author} ; Matthew Janus^a ; ^{mj3at@virginia.edu" class="auth_mail" title="E-mail the corresponding author} ; David J. Johnson^s ; ^{djjohnson@smcm.edu" class="auth_mail" title="E-mail the corresponding author} ; Jennifer A. Joy-Gaba^j ; ^{jjoygaba@vcu.edu" class="auth_mail" title="E-mail the corresponding author} ; Hannah Juzeler^q ; ^{juzelehj@plu.edu" class="auth_mail" title="E-mail the corresponding author} ; Ashley Keres^h ; ^{akeres@ashland.edu" class="auth_mail" title="E-mail the corresponding author} ; Diana Kinney^a ; ^{dmk2wa@virginia.edu" class="auth_mail" title="E-mail the corresponding author} ; Jacqeline Kirshenbaum^r ; ^{jkirshenbaum@mail.bradley.edu" class="auth_mail" title="E-mail the corresponding author} ; Richard A. Klein^m ; ^{raklein@ufl.edu" class="auth_mail" title="E-mail the corresponding author} ; Richard E. Lucas^s ; ^{lucasri@msu.edu" class="auth_mail" title="E-mail the corresponding author} ; Christopher J.N. Lustgraaf^p ; ^{cjn.lustgraaf@gmail.com" class="auth_mail" title="E-mail the corresponding author} ; Daniel Martin^a ; ^{dpmartin42@gmail.com" class="auth_mail" title="E-mail the corresponding author} ; Madhavi Menon^e ; ^{madhavi@nova.edu" class="auth_mail" title="E-mail the corresponding author} ; Mitchell Metzger^h ; ^{mmetzger@ashland.edu" class="auth_mail" title="E-mail the corresponding author} ; Jaclyn M. Moloney^j ; ^{moloneyjm@vcu.edu" class="auth_mail" title="E-mail the corresponding author} ; Patrick J. Morse^f ; ^{pmors001@ucr.edu" class="auth_mail" title="E-mail the corresponding author} ; Radmila Prislin^o ; ^{rprislin@mail.sdsu.edu" class="auth_mail" title="E-mail the corresponding author} ; Timothy Razza^e ; ^{razzatim@nova.edu" class="auth_mail" title="E-mail the corresponding author} ; Daniel E. Re^t ; ^{dan.re@utoronto.ca" class="auth_mail" title="E-mail the corresponding author} ; Nicholas O. Rule^t ; ^{rule@psych.utoronto.ca" class="auth_mail" title="E-mail the corresponding author} ; Donald F. Sacco^p ; ^{Donald.Sacco@usm.edu" class="auth_mail" title="E-mail the corresponding author} ; Kyle Sauerberger^f ; ^{kylesauerberger@gmail.com" class="auth_mail" title="E-mail the corresponding author} ; Emily Shrider^h ; ^{eshrider@ashland.edu" class="auth_mail" title="E-mail the corresponding author}
• 关键词：Social psychology ; Cognitive psychology ; Replication ; Participant pool ; Individual differences ; Sampling effects ; Situational effects
• 刊名：Journal of Experimental Social Psychology
• 出版年：2016
• 出版时间：November 2016
• 年：2016
• 卷：67
• 期：Complete
• 页码：68-82
• 全文大小：650 K

文摘

The university participant pool is a key resource for behavioral research, and data quality is believed to vary over the course of the academic semester. This crowdsourced project examined time of semester variation in 10 known effects, 10 individual differences, and 3 data quality indicators over the course of the academic semester in 20 participant pools (N = 2696) and with an online sample (N = 737). Weak time of semester effects were observed on data quality indicators, participant sex, and a few individual differences—conscientiousness, mood, and stress. However, there was little evidence for time of semester qualifying experimental or correlational effects. The generality of this evidence is unknown because only a subset of the tested effects demonstrated evidence for the original result in the whole sample. Mean characteristics of pool samples change slightly during the semester, but these data suggest that those changes are mostly irrelevant for detecting effects.