Promoting Transparency in Social Science Research Miguel, E.; Camerer, C.; Casey, K. ...
Science (American Association for the Advancement of Science),
01/2014, Letnik:
343, Številka:
6166
Journal Article
Recenzirano
Odprti dostop
Social scientists should adopt higher transparency standards to improve the quality and credibility of research.
There is growing appreciation for the advantages of experimentation in the social ...sciences. Policy-relevant claims that in the past were backed by theoretical arguments and inconclusive correlations are now being investigated using more credible methods. Changes have been particularly pronounced in development economics, where hundreds of randomized trials have been carried out over the last decade. When experimentation is difficult or impossible, researchers are using quasi-experimental designs. Governments and advocacy groups display a growing appetite for evidence-based policy-making. In 2005, Mexico established an independent government agency to rigorously evaluate social programs, and in 2012, the U.S. Office of Management and Budget advised federal agencies to present evidence from randomized program evaluations in budget requests (
1
,
2
).
Qualitative research networks (QRNs) bring together researchers from diverse contexts working on multi-country studies. The networks may themselves form a consortium or may contribute to a wider ...research agenda within a consortium with colleagues from other disciplines. The purpose of a QRN is to ensure robust methods and processes that enable comparisons across contexts. Under the Self-Testing Africa (STAR) initiative and the REACHOUT project on community health systems, QRNs were established, bringing together researchers across countries to coordinate multi-country qualitative research and to ensure robust methods and processes allowing comparisons across contexts. QRNs face both practical challenges in facilitating this iterative exchange process across sites and conceptual challenges interpreting findings between contexts. This paper distils key lessons and reflections from both QRN experiences on how to conduct trustworthy qualitative research across different contexts with examples from Bangladesh, Ethiopia, Kenya, Indonesia, Malawi, Mozambique, Zambia and Zimbabwe.
The process of generating evidence for this paper followed a thematic analysis method: themes initially identified were refined during several rounds of discussions in an iterative process until final themes were agreed upon in a joint learning process.
Four guiding principles emerged from our analysis: a) explicit communication strategies that sustain dialogue and build trust and collective reflexivity; b) translation of contextually embedded concepts; c) setting parameters for contextualizing, and d) supporting empirical and conceptual generalisability. Under each guiding principle, we describe how credibility, dependability, confirmability and transferability can be enhanced and share good practices to be considered by other researchers.
Qualitative research is often context-specific with tools designed to explore local experiences and understandings. Without efforts to synthesise and systematically share findings, common understandings, experiences and lessons are missed. The logistical and conceptual challenges of qualitative research across multiple partners and contexts must be actively managed, including a shared commitment to continuous 'joint learning' by partners. Clarity and agreement on concepts and common methods and timelines at an early stage is critical to ensure alignment and focus in intercountry qualitative research and analysis processes. Building good relationships and trust among network participants enhance the quality of qualitative research findings.
Objectives
Data sharing is well established in biological research, but evidence on sharing of clinical trial or public health research study data remains limited, in particular studies of research ...participants’ perspectives of data sharing. This study systematically reviewed international evidence of research participants’ attitudes towards the sharing of data for secondary research use.
Methods
Systematic search of seven databases, and author-, citation- and bibliography-follow up to identify studies examining research participants’ attitudes towards data sharing. Studies were thematically analysed using NVivo v10 to identify recurring themes.
Results
Nine studies were eligible for inclusion. Thematic analysis identified four key themes: (1) benefits of data sharing, including benefit to participants or immediate community, benefits to the public and benefits to science or research; (2) fears and harms, such as fear of exploitation, stigmatization or repercussions, alongside concerns about confidentiality and misuse of data; (3) data sharing processes, in particular the role of consent in the process; and (4) the relationship between participants and research such as trust in different types of research or organization and the relationship with the original research team.
Conclusions
The available literature on attitudes towards sharing data from clinical trials or public health interventions remains scant. This study has identified four themes regarding research participants’ attitudes and preferences, which should be considered by policy makers, and explored with further research.
The 11th edition of Research Methodology in Strategy and Management focuses on understanding cutting edge methodological approaches to the study of organizations, managers, and strategy. To this end, ...the chapters are intended to give readers a sense of new approaches as well as retrospective accounts by Kathy Eisenhardt and Denny Gioia.
Systematic reviews (SRs) are cornerstones of evidence-based medicine and have contributed significantly to breakthroughs since the 1980's. However, preclinical SRs remain relatively rare despite ...their many advantages. Since 2011 the Dutch health funding organisation (ZonMw) has run a grant scheme dedicated to promoting the training, coaching and conduct of preclinical SRs. Our study focuses on this funding scheme to investigate the relevance, effects and benefits of conducting preclinical SRs on researchers and their research.
We recruited researchers who attended funded preclinical SR workshops and who conducted, are still conducting, or prematurely stopped a SR with funded coaching. We gathered data using online questionnaires followed by semi-structured interviews. Both aimed to explore the impact of conducting a SR on researchers' subsequent work, attitudes, and views about their research field. Data-analysis was performed using Excel and ATLAS.ti.
Conducting preclinical SRs had two distinct types of impact. First, the researchers acquired new skills and insights, leading to a change in mindset regarding the quality of animal research. This was mainly seen in the way participants planned, conducted and reported their subsequent animal studies, which were more transparent and of a higher quality than their previous work. Second, participants were eager to share their newly acquired knowledge within their laboratories and to advocate for change within their research teams and fields of interest. In particular, they emphasised the need for preclinical SRs and improved experimental design within preclinical research, promoting these through education and published opinion papers.
Being trained and coached in the conduct of preclinical SRs appears to be a contributing factor to many beneficial changes which will impact the quality of preclinical research in the long-term. Our findings suggest that this ZonMw funding scheme is helpful in improving the quality and transparency of preclinical research. Similar funding schemes should be encouraged, preferably by a broader group of funders or financers, in the future.
In an effort to better utilize published evidence obtained from animal experiments, systematic reviews of preclinical studies are increasingly more common-along with the methods and tools to appraise ...them (e.g., SYstematic Review Center for Laboratory animal Experimentation SYRCLE's risk of bias tool). We performed a cross-sectional study of a sample of recent preclinical systematic reviews (2015-2018) and examined a range of epidemiological characteristics and used a 46-item checklist to assess reporting details. We identified 442 reviews published across 43 countries in 23 different disease domains that used 26 animal species. Reporting of key details to ensure transparency and reproducibility was inconsistent across reviews and within article sections. Items were most completely reported in the title, introduction, and results sections of the reviews, while least reported in the methods and discussion sections. Less than half of reviews reported that a risk of bias assessment for internal and external validity was undertaken, and none reported methods for evaluating construct validity. Our results demonstrate that a considerable number of preclinical systematic reviews investigating diverse topics have been conducted; however, their quality of reporting is inconsistent. Our study provides the justification and evidence to inform the development of guidelines for conducting and reporting preclinical systematic reviews.
The use of numerical/quantitative data in qualitative research studies and reports has been controversial. Prominent qualitative researchers such as Howard Becker and Martyn Hammersley have supported ...the inclusion of what Becker called “quasi-statistics”: simple counts of things to make statements such as “some,” “usually,” and “most” more precise. However, others have resisted such uses, particularly when they are requested by reviewers for journals. This paper presents both the advantages of integrating quantitative information in qualitative data collection, analysis, and reporting, and the potential problems created by such uses and how these can be dealt with. It also addresses the definition of mixed methods research, arguing that the use of numbers by itself doesn't make a study “mixed methods.”
ObjectivesParticipant-led research (PLR) is a rapidly developing form of citizen science in which individuals can create personal and generalisable knowledge. Although PLR lacks a formal framework ...for ethical review, participants should not be excused from considering the ethical implications of their work. Therefore, a PLR cohort consisting of 24 self-trackers aimed to: (1) substitute research ethics board procedures with engagement in ethical reflection before and throughout the study and (2) draft principles to encourage further development of the governance and ethical review of PLR.MethodsA qualitative case study method was used to analyse the ethical reflection process. Participants discussed study risks, risk management strategies and benefits pre-project, during a series of weekly webinars, via individual meetings with the participant-organisers, and during semi-structured interviews at project completion. Themes arising from discussions and interviews were used to draft prospective principles to guide PLR.ResultsData control, aggregation and identifiability were the most common risks identified. These were addressed by a commitment to transparency among all participants and by establishing participant control via self-collection and self-management of data. Group discussions and resources (eg, assistance with experimental design and data analysis) were the most commonly referenced benefits of participation. Additional benefits included greater understanding of one’s physiology and greater ability to structure an experiment. Nine principles were constructed to encourage further development of ethical PLR practices. All participants expressed interest in participating in future PLR.ConclusionsProjects involving a small number of participants can sustain engagement in ethical reflection among participants and participant-organisers. PLR that prioritises transparency, participant control of data and ongoing risk-to-benefit evaluation is compatible with the principles that underlie traditional ethical review of health research, while being appropriate for a context in which citizen scientists play the central role.
Economists studying innovation and technological change have made significant progress toward understanding firms' profit incentives as drivers of innovation. However, innovative performance in firms ...should also depend heavily on the pecuniary and nonpecuniary motives of the employees actually working in research and development. Using data on more than 1,700 Ph.D. scientists and engineers, we examine the relationships between individuals' motives (e.g., desire for intellectual challenge, income, or responsibility) and their innovative performance. We find that motives matter, but different motives have very different effects: Motives regarding intellectual challenge, independence, and money have a strong positive relationship with innovative output, whereas motives regarding job security and responsibility tend to have a negative relationship. We also explore possible mechanisms underlying the observed relationships between motives and performance. Although hours worked (
quantity of effort
) have a strong positive effect on performance, motives appear to affect innovative performance primarily via other dimensions of effort (
character of effort
). Finally, we find some evidence that the role of motives differs in upstream research versus downstream development.
PDF
