Many Labs 3: Evaluating participant pool quality across the academic semester via replication


Charles R Ebersole, Olivia E Atherton, Aimee L Belanger, Hayley M Skulborstad, Jill M Allen, Jonathan B Banks, Erica Baranski, Michael J Bernstein, Diane B V Bonfiglio, Leanne Boucher, Elizabeth R Brown, Nancy I Budiman, Athena H Cairo, Colin A Capaldi, Christopher R Chartier, Joanne M Chung, David C Cicero, Jennifer A Coleman, John G Conway, William E Davis, Thierry Devos, Melody M Fletcher, Komi German, Jon E Grahe, Anthony D Hermann, Joshua A Hicks, Nathan Honeycutt, Brandon Humphrey, Matthew Janus, David J Johnson, Jennifer A Joy-Gaba, Hannah Juzeler, Ashley Keres, Diana Kinney, Jacqeline Kirshenbaum, Richard A Klein, Richard E Lucas, Christopher J N Lustgraaf, Daniel Martin, Madhavi Menon, Mitchell Metzger, Jaclyn M Moloney, Patrick J Morse, Radmila Prislin, Timothy Razza, Daniel E Re, Nicholas O Rule, Tdonald F Sacco, Kyle Sauerberger, Emily Shrider, Megan Shultz, Courtney Siemsen, Karin Sobocko, R. Weylin Sternglanz, Amy Summerville, Konstantin O Tskhay, Zack van Allen, Leigh Ann Vaughn, Ryan J Walker, Ashley Weinberg, John Paul Wilson, James H Wirth, Jessica Wortman, Brian A Nosek


Journal of Experimental Social Psychology


The university participant pool is a key resource for behavioral research, and data quality is believed to vary over the course of the academic semester. This crowdsourced project examined time of semester variation in 10 known effects, 10 individual differences, and 3 data quality indicators over the course of the academic semester in 20 participant pools (N = 2696) and with an online sample (N = 737). Weak time of semester effects were observed on data quality indicators, participant sex, and a few individual differences conscientiousness, mood, and stress. However, there was little evidence for time of semester qualifying experimental or correlational effects. The generality of this evidence is unknown because only a subset of the tested effects demonstrated evidence for the original result in the whole sample. Mean characteristics of pool samples change slightly during the semester, but these data suggest that those changes are mostly irrelevant for detecting effects. (C) 2015 Elsevier Inc. All rights reserved.



How is this information collected?

This collection of Montana State authored publications is collected by the Library to highlight the achievements of Montana State researchers and more fully understand the research output of the University. They use a number of resources to pull together as complete a list as possible and understand that there may be publications that are missed. If you note the omission of a current publication or want to know more about the collection and display of this information email Leila Sterman.