Improving the reproducibility of biomedical research is a major challenge. Transparent and accurate reporting is vital to this process; it allows readers to assess the reliability of the findings and ...repeat or build upon the work of other researchers. The ARRIVE guidelines (Animal Research: Reporting In Vivo Experiments) were developed in 2010 to help authors and journals identify the minimum information necessary to report in publications describing in vivo experiments. Despite widespread endorsement by the scientific community, the impact of ARRIVE on the transparency of reporting in animal research publications has been limited. We have revised the ARRIVE guidelines to update them and facilitate their use in practice. The revised guidelines are published alongside this paper. This explanation and elaboration document was developed as part of the revision. It provides further information about each of the 21 items in ARRIVE 2.0, including the rationale and supporting evidence for their inclusion in the guidelines, elaboration of details to report, and examples of good reporting from the published literature. This document also covers advice and best practice in the design and conduct of animal studies to support researchers in improving standards from the start of the experimental design process through to publication.
Survey research methodology is widely used in marketing, and it is important for both the field and individual researchers to follow stringent guidelines to ensure that meaningful insights are ...attained. To assess the extent to which marketing researchers are utilizing best practices in designing, administering, and analyzing surveys, we review the prevalence of published empirical survey work during the 2006–2015 period in three top marketing journals—
Journal of the Academy of Marketing Science
(
JAMS
),
Journal of Marketing
(
JM
), and
Journal of Marketing Research
(
JMR
)—and then conduct an in-depth analysis of 202 survey-based studies published in
JAMS
. We focus on key issues in two broad areas of survey research (issues related to the choice of the object of measurement and selection of raters, and issues related to the measurement of the constructs of interest), and we describe conceptual considerations related to each specific issue, review how marketing researchers have attended to these issues in their published work, and identify appropriate best practices.
With the broader and highly successful usage of machine learning (ML) in industry and the sciences, there has been a growing demand for explainable artificial intelligence (XAI). Interpretability and ...explanation methods for gaining a better understanding of the problem-solving abilities and strategies of nonlinear ML, in particular, deep neural networks, are, therefore, receiving increased attention. In this work, we aim to: 1) provide a timely overview of this active emerging field, with a focus on " post hoc " explanations, and explain its theoretical foundations; 2) put interpretability algorithms to a test both from a theory and comparative evaluation perspective using extensive simulations; 3) outline best practice aspects, i.e., how to best include interpretation methods into the standard usage of ML; and 4) demonstrate successful usage of XAI in a representative selection of application scenarios. Finally, we discuss challenges and possible future directions of this exciting foundational field of ML.
Mass spectrometry-based metabolomics approaches can enable detection and quantification of many thousands of metabolite features simultaneously. However, compound identification and reliable ...quantification are greatly complicated owing to the chemical complexity and dynamic range of the metabolome. Simultaneous quantification of many metabolites within complex mixtures can additionally be complicated by ion suppression, fragmentation and the presence of isomers. Here we present guidelines covering sample preparation, replication and randomization, quantification, recovery and recombination, ion suppression and peak misidentification, as a means to enable high-quality reporting of liquid chromatography- and gas chromatography-mass spectrometry-based metabolomics-derived data.
Despite broad scientific interest in harnessing the power of Earth's microbiomes, knowledge gaps hinder their efficient use for addressing urgent societal and environmental challenges. We argue that ...structuring research and technology developments around a design-build-test-learn (DBTL) cycle will advance microbiome engineering and spur new discoveries of the basic scientific principles governing microbiome function. In this Review, we present key elements of an iterative DBTL cycle for microbiome engineering, focusing on generalizable approaches, including top-down and bottom-up design processes, synthetic and self-assembled construction methods, and emerging tools to analyse microbiome function. These approaches can be used to harness microbiomes for broad applications related to medicine, agriculture, energy and the environment. We also discuss key challenges and opportunities of each approach and synthesize them into best practice guidelines for engineering microbiomes. We anticipate that adoption of a DBTL framework will rapidly advance microbiome-based biotechnologies aimed at improving human and animal health, agriculture and enabling the bioeconomy.
Public relative performance feedback (RPF) on an individual worker’s productivity metrics is used in various organizations with the hopes of improving worker productivity, but its effects are not ...well understood. We examine whether public RPF could be leveraged to facilitate adoption of best practices in an organization by enabling the validation of best practices shared by identifiable top performers. We use data from two emergency departments, both of which shared best practices for improving productivity and one of which changed from privately to publicly disclosing RPF to physicians. The public disclosure of RPF allowed workers to identify their top-performing coworkers, which in turn enabled the identification and validation of best practices within the work group. We find that the intervention is associated with a 10.9% improvement in physician productivity. We also find evidence for a significant reduction in variation in productivity across providers, which stems from bottom-ranked workers exhibiting differentially large improvements in productivity. These effects hold without sacrificing system-level performance, service quality, or worker attrition. Our results suggest that public disclosure of RPF, along with the validation of the best practices being shared, can improve worker productivity.
The online supplement is available at
https://doi.org/10.1287/mnsc.2017.2745
.
This paper was accepted by Serguei Netessine, operations management.
•Summary of development of contact tracing applications around the world.•Demonstrates divergence of “data-first” and “privacy-first” approaches.•Contact tracing faces heightened public consciousness ...of online privacy.•Major research on personal and national attitudes to privacy is needed.•Calls for best practice guidelines to reassure citizens on data collection.
The implementation of digital contact tracing applications around the world to help reduce the spread of the COVID-19 pandemic represents one of the most ambitious uses of massive-scale citizen data ever attempted. There is major divergence among nations, however, between a “privacy-first” approach which protects citizens’ data at the cost of extremely limited access for public health authorities and researchers, and a “data-first” approach which stores large amounts of data which, while of immeasurable value to epidemiologists and other researchers, may significantly intrude upon citizens’ privacy. The lack of a consensus on privacy protection in the contact tracing process creates risks of non-compliance or deliberate obfuscation from citizens who fear revealing private aspects of their lives – a factor greatly exacerbated by recent major scandals over online privacy and the illicit use of citizens’ digital information, which have heightened public consciousness of these issues and created significant new challenges for any collection of large-scale public data. While digital contact tracing for COVID-19 remains in its infancy, the lack of consensus around best practices for its implementation and for reassuring citizens of the protection of their privacy may already have impeded its capacity to contribute to the pandemic response.
Abstract
Statistical practice in psychological science is undergoing reform which is reflected in part by strong recommendations for reporting and interpreting effect sizes and their confidence ...intervals. We present principles and recommendations for research reporting and emphasize the variety of ways effect sizes can be reported. Additionally, we emphasize interpreting and reporting unstandardized effect sizes because of common misconceptions regarding standardized effect sizes which we elucidate. Effect sizes should directly answer their motivating research questions, be comprehensible to the average reader, and be based on meaningful metrics of their constituent variables. We illustrate our recommendations with empirical examples involving a One-way ANOVA, a categorical variable analysis, an interaction effect in linear regression, and a simple mediation model, emphasizing the interpretation of effect sizes.
Translational Abstract
We present general principles of good research reporting, elucidate common misconceptions about standardized effect sizes, and provide recommendations for good research reporting. Effect sizes should directly answer their motivating research questions, be comprehensible to the average reader, and be based on meaningful metrics of their constituent variables. We illustrate our recommendations with four different empirical examples involving popular statistical methods such as ANOVA, categorical variable analysis, multiple linear regression, and simple mediation; these examples serve as a tutorial to enhance practice in the research reporting of effect sizes.
According to mindset theory, students who believe their personal characteristics can change-that is, those who hold a growth mindset-will achieve more than students who believe their characteristics ...are fixed. Proponents of the theory have developed interventions to influence students' mindsets, claiming that these interventions lead to large gains in academic achievement. Despite their popularity, the evidence for growth mindset intervention benefits has not been systematically evaluated considering both the quantity and quality of the evidence. Here, we provide such a review by (a) evaluating empirical studies' adherence to a set of best practices essential for drawing causal conclusions and (b) conducting three meta-analyses. When examining all studies (63 studies, N = 97,672), we found major shortcomings in study design, analysis, and reporting, and suggestions of researcher and publication bias: Authors with a financial incentive to report positive findings published significantly larger effects than authors without this incentive. Across all studies, we observed a small overall effect: d¯ = 0.05, 95% CI = 0.02, 0.09, which was nonsignificant after correcting for potential publication bias. No theoretically meaningful moderators were significant. When examining only studies demonstrating the intervention influenced students' mindsets as intended (13 studies, N = 18,355), the effect was nonsignificant: d¯ = 0.04, 95% CI = −0.01, 0.10. When examining the highest-quality evidence (6 studies, N = 13,571), the effect was nonsignificant: d¯ = 0.02, 95% CI = −0.06, 0.10. We conclude that apparent effects of growth mindset interventions on academic achievement are likely attributable to inadequate study design, reporting flaws, and bias.
Public Significance Statement
This systematic review and meta-analysis suggest that, despite the popularity of growth mindset interventions in schools, positive results are rare and possibly spurious due to inadequately designed interventions, reporting flaws, and bias.
PDF
