Evidence of low replicability, poor reproducibility, and questionable research practices has prompted a broad effort toward transparency and openness in science, termed the open science movement. We ...examined articles published in HFES’s flagship journal, Human Factors, to gauge the prevalence of open science practices. Empirical articles published in the journal between the years 2017 and 2022 were assessed for evidence of study preregistration, materials sharing, data sharing, and analytic code sharing. The majority of articles made no mention of any open science practices. The most common practices adopted were data and code sharing, though these were reported in no more than 17% of the articles in any year. Journal policies may be worthwhile to better incentivize openness and transparency.
We issue a call for the design and conduct of experimental trials to test
the effects of researchers' adoption of Open Science (OS) research
practices. OS emerged to address lapses in the ...transparency, quality, integrity,
and reproducibility of research by proposing that investigators institute
practices, such as preregistering study hypotheses, procedures and statistical
analyses, prior to launching their research. These practices have been greeted
with enthusiasm by some parts of the scientific community, but empirical
evidence of their effects relies mainly on observational studies; furthermore,
questions remain about the time and effort required by these practices and their
ultimate benefit to science. To assess the outcomes of OS research practices, we
propose they be viewed as behavioral interventions for scientists and tested in
randomized controlled trials (RCTs), to identify potential benefits and
(unintended) harms. As this is a call to action rather than an action plan
per se
, we sketch out four potential trial designs to
encourage further deliberation and planning. Experimental tests to document the
outcomes of OS practices can provide evidence to optimize how scientists,
funders, policymakers, and institutions utilize these strategies to advance
scientific practice.
Full text
Available for:
CEKLJ, FFLJ, NUK, ODKLJ, PEFLJ
Computer code plays a vital role in modern science, from the conception and design of experiments through to final data analyses. Open sharing of code has been widely discussed as being advantageous ...to the scientific process, allowing experiments to be more easily replicated, helping with error detection, and reducing wasted effort and resources. In the case of psychology, the code used to present stimuli is a fundamental component of many experiments. It is not known, however, the degree to which researchers are sharing this type of code. To estimate this, we conducted a survey of 400 psychology papers published between 2016 and 2021, identifying those working with the open-source tools Psychtoolbox and PsychoPy that openly share stimulus presentation code. For those that did, we established if it would run following download and also appraised the code's usability in terms of style and documentation. It was found that only 8.4% of papers shared stimulus code, compared to 17.9% sharing analysis code and 31.7% sharing data. Of shared code, 70% ran directly or after minor corrections. For code that did not run, the main error was missing dependencies (66.7%). The usability of the code was moderate, with low levels of code annotation and minimal documentation provided. These results suggest that stimulus presentation code sharing lags behind other forms of code and data sharing, potentially due to less emphasis on such code in open-science discussions and in journal policies. The results also highlight a need for improved documentation to maximize code utility.
Purpose/Significance Open Science, as a revolutionary system engineering, is faced up with many challenges in its evolution, for which it needs to break through the limitations of multiple barriers ...between nations, expertise, narrow interests, and among others. Method/Process China had made continuous efforts in innovating the scientific-research incentive system, creating an open scientific environment, and strengthening organizational and platform construction support. In the process of accelerating the full implementation of the values of open science, namely "freedom, openness, cooperation and sharing", and continuously deepening the all-round cooperation with international organizations including UNESCO by establishing and operating Category II centers, through which it had accumulated experience in cooperating with international organizations to promote the development open science. Result/Conclusion Its successful practice proves the effectiveness of open science as a non-market incentive mechani
Pre‐registration: Why and How Simmons, Joseph; Nelson, Leif; Simonsohn, Uri
Journal of consumer psychology,
January 2021, 2021-01-00, Volume:
31, Issue:
1
Journal Article
Peer reviewed
In this article, we (1) discuss the reasons why pre‐registration is a good idea, both for the field and individual researchers, (2) respond to arguments against pre‐registration, (3) describe how to ...best write and review a pre‐registration, and (4) comment on pre‐registration’s rapidly accelerating popularity. Along the way, we describe the (big) problem that pre‐registration can solve (i.e., false positives caused by p‐hacking), while also offering viable solutions to the problems that pre‐registration cannot solve (e.g., hidden confounds or fraud). Pre‐registration does not guarantee that every published finding will be true, but without it you can safely bet that many more will be false. It is time for our field to embrace pre‐registration, while taking steps to ensure that it is done right.
This article is part of a Research Dialogue:Krishna (2021): https://doi.org/10.1002/jcpy.1211Pham & Oh (2021): https://doi.org/10.1002/jcpy.1209Simmons et al. (2021): https://doi.org/10.1002/jcpy.1207Pham & Oh (2021): https://doi.org/10.1002/jcpy.1213
This research explored the readiness towards the implementation of open science initiatives in the Malaysian Comprehensive Public Universities; both quantitative and qualitative research methods were ...implemented. The survey questionnaire was distributed to the academic researchers and library professionals from the selected universities and a semi-structured interview was conducted with the heads of the libraries together with their deputies to complement the first method. The findings indicate that the public universities in the country were partially involved with open science, even though the term is still new to most of the participants. The Institutional Repository is the commonly known means through which these universities promote and practice open science, although only 10% to 30% of the contents were made available to the general public. However, these institutions are currently working on implementing a policy on how to operate and fully engage other aspects of the open science practices, such as open data, open collaboration, open innovation, and the likes.
An Agenda for Open Science in Communication Dienlin, Tobias; Johannes, Niklas; Bowman, Nicholas David ...
Journal of communication,
02/2021, Volume:
71, Issue:
1
Journal Article
Peer reviewed
Open access
Abstract
In the last 10 years, many canonical findings in the social sciences appear unreliable. This so-called “replication crisis” has spurred calls for open science practices, which aim to ...increase the reproducibility, replicability, and generalizability of findings. Communication research is subject to many of the same challenges that have caused low replicability in other fields. As a result, we propose an agenda for adopting open science practices in Communication, which includes the following seven suggestions: (1) publish materials, data, and code; (2) preregister studies and submit registered reports; (3) conduct replications; (4) collaborate; (5) foster open science skills; (6) implement Transparency and Openness Promotion Guidelines; and (7) incentivize open science practices. Although in our agenda we focus mostly on quantitative research, we also reflect on open science practices relevant to qualitative research. We conclude by discussing potential objections and concerns associated with open science practices.
Data Sharing and Cardiology Dey, Pranammya; Ross, Joseph S.; Ritchie, Jessica D. ...
Journal of the American College of Cardiology,
12/2017, Volume:
70, Issue:
24
Journal Article
Peer reviewed
Open access
Sharing deidentified patient-level research data presents immense opportunities to all stakeholders involved in cardiology research and practice. Sharing data encourages the use of existing data for ...knowledge generation to improve practice, while also allowing for validation of disseminated research. In this review, we discuss key initiatives and platforms that have helped to accelerate progress toward greater sharing of data. These efforts are being prompted by government, universities, philanthropic sponsors of research, major industry players, and collaborations among some of these entities. As data sharing becomes a more common expectation, policy changes will be required to encourage and assist data generators with the process of sharing the data they create. Patients also will need access to their own data and to be empowered to share those data with researchers. Although medicine still lags behind other fields in achieving data sharing’s full potential, cardiology research has the potential to lead the way.
Display omitted
Psychology endeavors to develop theories of human capacities and behaviors on the basis of a variety of methodologies and dependent measures. We argue that one of the most divisive factors in ...psychological science is whether researchers choose to use computational modeling of theories (over and above data) during the scientific-inference process. Modeling is undervalued yet holds promise for advancing psychological science. The inherent demands of computational modeling guide us toward better science by forcing us to conceptually analyze, specify, and formalize intuitions that otherwise remain unexamined—what we dub open theory. Constraining our inference process through modeling enables us to build explanatory and predictive theories. Here, we present scientific inference in psychology as a path function in which each step shapes the next. Computational modeling can constrain these steps, thus advancing scientific inference over and above the stewardship of experimental practice (e.g., preregistration). If psychology continues to eschew computational modeling, we predict more replicability crises and persistent failure at coherent theory building. This is because without formal modeling we lack open and transparent theorizing. We also explain how to formalize, specify, and implement a computational model, emphasizing that the advantages of modeling can be achieved by anyone with benefit to all.
Despite psychological scientists' increasing interest in replicability, open science, research transparency, and the improvement of methods and practices, the clinical psychology community has been ...slow to engage. This has been shifting more recently, and with this review, we hope to facilitate this emerging dialogue. We begin by examining some potential areas of weakness in clinical psychology in terms of methods, practices, and evidentiary base. We then discuss a select overview of solutions, tools, and current concerns of the reform movement from a clinical psychological science perspective. We examine areas of clinical science expertise (e.g., implementation science) that should be leveraged to inform open science and reform efforts. Finally, we reiterate the call to clinical psychologists to increase their efforts toward reform that can further improve the credibility of clinical psychological science.
PDF
