Scientific Utopia III: Crowdsourcing Science Uhlmann, Eric Luis; Ebersole, Charles R.; Chartier, Christopher R. ...
Perspectives on psychological science,
09/2019, Letnik:
14, Številka:
5
Journal Article
Recenzirano
Odprti dostop
Most scientific research is conducted by small teams of investigators who together formulate hypotheses, collect data, conduct analyses, and report novel findings. These teams operate independently ...as vertically integrated silos. Here we argue that scientific research that is horizontally distributed can provide substantial complementary value, aiming to maximize available resources, promote inclusiveness and transparency, and increase rigor and reliability. This alternative approach enables researchers to tackle ambitious projects that would not be possible under the standard model. Crowdsourced scientific initiatives vary in the degree of communication between project members from largely independent work curated by a coordination team to crowd collaboration on shared activities. The potential benefits and challenges of large-scale collaboration span the entire research process: ideation, study design, data collection, data analysis, reporting, and peer review. Complementing traditional small science with crowdsourced approaches can accelerate the progress of science and improve the quality of scientific research.
The Action-sentence Compatibility Effect (ACE) is a well-known demonstration of the role of motor activity in the comprehension of language. Participants are asked to make sensibility judgments on ...sentences by producing movements toward the body or away from the body. The ACE is the finding that movements are faster when the direction of the movement (e.g.,
toward
) matches the direction of the action in the to-be-judged sentence (e.g.,
Art gave you the pen
describes action toward you). We report on a pre-registered, multi-lab replication of one version of the ACE. The results show that none of the 18 labs involved in the study observed a reliable ACE, and that the meta-analytic estimate of the size of the ACE was essentially zero.
Psychological researchers have begun to utilize Amazon’s Mechanical Turk (MTurk) marketplace as a participant pool. Although past work has established that MTurk is well suited to examining ...individual behavior, pseudo-dyadic interactions, in which participants falsely believe they are interacting with a partner, are a key element of social and cognitive psychology. The ability to conduct such interdependent research on MTurk would increase the utility of this online population for a broad range of psychologists. The present research therefore attempts to qualitatively replicate well-established pseudo-dyadic tasks on MTurk in order to establish the utility of this platform as a tool for researchers. We find that participants do behave as if a partner is real, even when doing so incurs a financial cost, and that they are sensitive to subtle information about the partner in a minimal-groups paradigm, supporting the use of MTurk for pseudo-dyadic research.
•The tacit coordination performance of interacting groups is investigated.•Groups demonstrate far better coordination performance than individuals.•Majority and demonstrative group processes drive ...the intergroup coordination advantage.
Tacit coordination between individuals has received considerable research attention (Mehta, Starmer, & Sugden, 1994; Abele & Stasser, 2008). However, groups often must coordinate tacitly with other groups, and such intergroup coordination has been rarely studied. In three experiments, we found that interacting groups are more successful at coordinating tacitly than are individuals. This advantage is driven by two types of coordination salience that are uniquely derived from groups deliberating and making collective responses. Consensual salience occurs when groups select a response because a majority of members support it. Majorities efficiently identify popular response tendencies (i.e., focal points) and thereby increase the chances of matching other groups’ responses. Disjunctive salience occurs when at least one member of a group suggests a focal point. We propose that focal points are often demonstratively evident when mentioned, and if proposed by any group member, are likely to be adopted as the group response.
In the past 10 years, the field of relationship science—like many other fields—has been exposed to dramatic changes in how scientists approach the research process. Relationship science has been at ...the forefront of many recent changes in the field, whether it be high profile replication attempts or broader discussions about how to increase rigor and reproducibility. A major goal of this special issue was to provide an opportunity for relationship scientists to engage with these issues and reforms. The first four articles in this special issue represent a sampling of different approaches relationship researchers have used to enhance the credibility of their work.
Tacit coordination is crucial for many social interactions, including those among couples. Two different coordination requirements have been distinguished. Matching problems require interdependent ...actors to choose the same action; mismatching problems require the choice of different actions. We tested the performance of romantic couples relative to complete strangers in both matching and mismatching coordination. Social focal point theory (SFPT) posits that knowledge of social similarity in a domain pertinent to the coordination task will enhance matching coordination while knowledge of divergence will enhance mismatching coordination. Hence, we predicted that couples are more likely to detect and use social focal points in matching but not mismatching tasks, due to their interpersonal similarity and their wealth of mutual social knowledge. Indeed, couples were significantly better than strangers at matching but not at mismatching. Further analyses suggest that interplay between social knowledge and task demands, as delineated in SFPT, determines coordination success.
Background: The Psychological Science Accelerator (PSA) recently completed a large-scale moral psychology study using translated versions of the Oxford Utilitarianism Scale (OUS). However, the ...translated versions have no validity evidence. Objective: The study investigated the structural validity evidence of the OUS across 15 translated versions and produced version-specific validity reports. Methods: We analyzed OUS data from the PSA, which was collected internationally on a centralized online questionnaire. We also collected qualitative feedback from experts for each translated version. Results: For each version, we produced version-specific psychometric reports which include the following: (1) descriptive item and demographics analyses, (2) factor structure evidence using confirmatory factor analyses, (3) measurement invariance testing across languages using multiple-group confirmatory factor analyses and alignment optimization, and (4) reliability analyses using coefficients α and ω.
Interpreting a failure to replicate is complicated by the fact that the failure could be due to the original finding being a false positive, unrecognized moderating influences between the original ...and replication procedures, or faulty implementation of the procedures in the replication. One strategy to maximize replication quality is involving the original authors in study design. We (N = 17 Labs and N = 1,550 participants, after exclusions) experimentally tested whether original author involvement improved replicability of a classic finding from Terror Management Theory (Greenberg et al., 1994). Our results were non-diagnostic of whether original author involvement improves replicability because we were unable to replicate the finding under any conditions. This suggests that the original finding was either a false positive or the conditions necessary to obtain it are not fully understood or no longer exist. Data, materials, analysis code, preregistration, and supplementary documents can be found on the OSF page: https://osf.io/8ccnw/
“Crowdsourcing” is a methodological approach in which several researchers coordinate their resources to achieve research goals that would otherwise be difficult to attain individually. This article ...introduces a Nexus—a collection of empirical and theoretical articles that will be published in Collabra: Psychology—that is intended to encourage more crowdsourced research in psychological science by providing a specific outlet for such projects and by assisting researchers in developing and executing their projects. We describe how individuals can propose and lead a crowdsourced research project, how individuals can contribute to other ongoing projects, and other ways to contribute to this Nexus. Ultimately, we hope this Nexus will contain a set of highly-informative articles that demonstrate the flexibility and range of the types of research questions that can be addressed with crowdsourced research methods.
PDF
