The power of language to modify the reader's perception of interpreting biomedical results cannot be underestimated. Misreporting and misinterpretation are pressing problems in randomized controlled ...trials (RCT) output. This may be partially related to the statistical significance paradigm used in clinical trials centered around a P value below 0.05 cutoff. Strict use of this P value may lead to strategies of clinical researchers to describe their clinical results with P values approaching but not reaching the threshold to be "almost significant." The question is how phrases expressing nonsignificant results have been reported in RCTs over the past 30 years. To this end, we conducted a quantitative analysis of English full texts containing 567,758 RCTs recorded in PubMed between 1990 and 2020 (81.5% of all published RCTs in PubMed). We determined the exact presence of 505 predefined phrases denoting results that approach but do not cross the line of formal statistical significance (P < 0.05). We modeled temporal trends in phrase data with Bayesian linear regression. Evidence for temporal change was obtained through Bayes factor (BF) analysis. In a randomly sampled subset, the associated P values were manually extracted. We identified 61,741 phrases in 49,134 RCTs indicating almost significant results (8.65%; 95% confidence interval (CI): 8.58% to 8.73%). The overall prevalence of these phrases remained stable over time, with the most prevalent phrases being "marginally significant" (in 7,735 RCTs), "all but significant" (7,015), "a nonsignificant trend" (3,442), "failed to reach statistical significance" (2,578), and "a strong trend" (1,700). The strongest evidence for an increased temporal prevalence was found for "a numerical trend," "a positive trend," "an increasing trend," and "nominally significant." In contrast, the phrases "all but significant," "approaches statistical significance," "did not quite reach statistical significance," "difference was apparent," "failed to reach statistical significance," and "not quite significant" decreased over time. In a random sampled subset of 29,000 phrases, the manually identified and corresponding 11,926 P values, 68,1% ranged between 0.05 and 0.15 (CI: 67. to 69.0; median 0.06). Our results show that RCT reports regularly contain specific phrases describing marginally nonsignificant results to report P values close to but above the dominant 0.05 cutoff. The fact that the prevalence of the phrases remained stable over time indicates that this practice of broadly interpreting P values close to a predefined threshold remains prevalent. To enhance responsible and transparent interpretation of RCT results, researchers, clinicians, reviewers, and editors may reduce the focus on formal statistical significance thresholds and stimulate reporting of P values with corresponding effect sizes and CIs and focus on the clinical relevance of the statistical difference found in RCTs.
Celotno besedilo
Dostopno za:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, SIK, UILJ, UKNU, UL, UM, UPUK
Objective To investigate whether language used in science abstracts can skew towards the use of strikingly positive and negative words over time.Design Retrospective analysis of all scientific ...abstracts in PubMed between 1974 and 2014.Methods The yearly frequencies of positive, negative, and neutral words (25 preselected words in each category), plus 100 randomly selected words were normalised for the total number of abstracts. Subanalyses included pattern quantification of individual words, specificity for selected high impact journals, and comparison between author affiliations within or outside countries with English as the official majority language. Frequency patterns were compared with 4% of all books ever printed and digitised by use of Google Books Ngram Viewer. Main outcome measures Frequencies of positive and negative words in abstracts compared with frequencies of words with a neutral and random connotation, expressed as relative change since 1980.Results The absolute frequency of positive words increased from 2.0% (1974-80) to 17.5% (2014), a relative increase of 880% over four decades. All 25 individual positive words contributed to the increase, particularly the words “robust,” “novel,” “innovative,” and “unprecedented,” which increased in relative frequency up to 15 000%. Comparable but less pronounced results were obtained when restricting the analysis to selected journals with high impact factors. Authors affiliated to an institute in a non-English speaking country used significantly more positive words. Negative word frequencies increased from 1.3% (1974-80) to 3.2% (2014), a relative increase of 257%. Over the same time period, no apparent increase was found in neutral or random word use, or in the frequency of positive word use in published books.Conclusions Our lexicographic analysis indicates that scientific abstracts are currently written with more positive and negative words, and provides an insight into the evolution of scientific writing. Apparently scientists look on the bright side of research results. But whether this perception fits reality should be questioned.
Many randomized controlled trials (RCTs) are biased and difficult to reproduce due to methodological flaws and poor reporting. There is increasing attention for responsible research practices and ...implementation of reporting guidelines, but whether these efforts have improved the methodological quality of RCTs (e.g., lower risk of bias) is unknown. We, therefore, mapped risk-of-bias trends over time in RCT publications in relation to journal and author characteristics. Meta-information of 176,620 RCTs published between 1966 and 2018 was extracted. The risk-of-bias probability (random sequence generation, allocation concealment, blinding of patients/personnel, and blinding of outcome assessment) was assessed using a risk-of-bias machine learning tool. This tool was simultaneously validated using 63,327 human risk-of-bias assessments obtained from 17,394 RCTs evaluated in the Cochrane Database of Systematic Reviews (CDSR). Moreover, RCT registration and CONSORT Statement reporting were assessed using automated searches. Publication characteristics included the number of authors, journal impact factor (JIF), and medical discipline. The annual number of published RCTs substantially increased over 4 decades, accompanied by increases in authors (5.2 to 7.8) and institutions (2.9 to 4.8). The risk of bias remained present in most RCTs but decreased over time for allocation concealment (63% to 51%), random sequence generation (57% to 36%), and blinding of outcome assessment (58% to 52%). Trial registration (37% to 47%) and the use of the CONSORT Statement (1% to 20%) also rapidly increased. In journals with a higher impact factor (>10), the risk of bias was consistently lower with higher levels of RCT registration and the use of the CONSORT Statement. Automated risk-of-bias predictions had accuracies above 70% for allocation concealment (70.7%), random sequence generation (72.1%), and blinding of patients/personnel (79.8%), but not for blinding of outcome assessment (62.7%). In conclusion, the likelihood of bias in RCTs has generally decreased over the last decades. This optimistic trend may be driven by increased knowledge augmented by mandatory trial registration and more stringent reporting guidelines and journal requirements. Nevertheless, relatively high probabilities of bias remain, particularly in journals with lower impact factors. This emphasizes that further improvement of RCT registration, conduct, and reporting is still urgently needed.
Celotno besedilo
Dostopno za:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, SIK, UILJ, UKNU, UL, UM, UPUK
Introduction The focus on quantitative indicators-number of publications and grants, journal impact factors, Hirsch-index-has become pervasive in research management, funding systems, and research ...and publication practices (SES). Accountability through performance measurement has become the gold standard to increase productivity and (cost-) efficiency in academia. Scientific careers are strongly shaped by the push to produce more in a veritable 'publish or perish' culture. To this end, we investigated the perspectives of biomedical researchers on responsible assessment criteria that foster responsible conduct of research. Methods We performed a qualitative focus group study among 3 University medical centers in the Netherlands. In these centers, we performed 2 randomly selected groups of early career researchers (PhD and postdoc level & senior researchers (associate and full professors) from these 3 institutions and explored how relational responsibilities relate to responsible conduct of research and inquired how potential (formal) assessment criteria could correspond with these responsibilities. Results In this study we highlighted what is considered responsible research among junior and senior researchers in the Netherlands and how this can be assessed in formal assessment criteria. The participants reflected on responsible research and highlighted several academic responsibilities (such as supervision, collaboration and teaching) that are often overlooked and that are considered a crucial prerequisite for responsible research. As these responsibilities pertain to intercollegiate relations, we henceforth refer to them as relational. After our systematic analysis of these relational responsibilities, participants suggested some ideas to improve current assessment criteria. We focused on how these duties can be reflected in multidimensional, concrete and sustainable assessment criteria. Focus group participants emphasized the importance of assessing team science (both individual as collective), suggested the use of a narrative in researcher assessment and valued the use of 360 degrees assessment of researchers. Participants believed that these alternative assessments, centered on relational responsibilities, could help in fostering responsible research practices. However, participants stressed that unclarity about the new assessment criteria would only cause more publication stress and insecurity about evaluation of their performance. Conclusion Our study suggests that relational responsibilities should ideally play a more prominent role in future assessment criteria as they correspond with and aspire the practice of responsible research. Our participants gave several suggestions how to make these skills quantifiable and assessable in future assessment criteria. However, the development of these criteria is still in its infancy, implementation can cause uncertainties among those assessed and consequently, future research should focus on how to make these criteria more tangible, concrete and applicable in daily practice to make them applicable to measure and assess responsible research practices in institutions. Trial registration Open Science Framework
Celotno besedilo
Dostopno za:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, SIK, UILJ, UKNU, UL, UM, UPUK
Breaches of research integrity have shocked the academic community. Initially explanations were sought at the level of individual researchers but over time increased recognition emerged of the ...important role that the research integrity climate may play in influencing researchers' (mis)behavior. In this study we aim to assess whether researchers from different academic ranks and disciplinary fields experience the research integrity climate differently. We sent an online questionnaire to academic researchers in Amsterdam using the Survey of Organizational Research Climate. Bonferroni corrected mean differences showed that junior researchers (PhD students, postdocs and assistant professors) perceive the research integrity climate more negatively than senior researchers (associate and full professors). Junior researchers note that their supervisors are less committed to talk about key research integrity principles compared to senior researchers (MD = -.39, CI = -.55, -.24). PhD students perceive more competition and suspicion among colleagues (MD = -.19, CI = -.35, -.05) than associate and full professors. We found that researchers from the natural sciences overall express a more positive perception of the research integrity climate. Researchers from social sciences as well as from the humanities perceive less fairness of their departments' expectations in terms of publishing and acquiring funding compared to natural sciences and biomedical sciences (MD = -.44, CI = -.74, -.15; MD = -.36, CI = -.61, -.11). Results suggest that department leaders in the humanities and social sciences should do more to set fairer expectations for their researchers and that senior scientists should ensure junior researchers are socialized into research integrity practices and foster a climate in their group where suspicion among colleagues has no place.
Celotno besedilo
Dostopno za:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, SIK, UILJ, UKNU, UL, UM, UPUK
Personality influences decision making and ethical considerations. Its influence on the occurrence of research misbehavior has never been studied. This study aims to determine the association between ...personality traits and self-reported questionable research practices and research misconduct. We hypothesized that narcissistic, Machiavellianistic and psychopathic traits as well as self-esteem are associated with research misbehavior.
Included in this cross-sectional study design were 535 Dutch biomedical scientists (response rate 65%) from all hierarchical layers of 4 university medical centers in the Netherlands. We used validated personality questionnaires such as the Dark Triad (narcissism, psychopathy, and Machiavellianism), Rosenberg's Self-Esteem Scale, the Publication Pressure Questionnaire (PPQ), and also demographic and job-specific characteristics to investigate the association of personality traits with a composite research misbehavior severity score.
Machiavellianism was positively associated (beta 1.28, CI 1.06-1.53) with self-reported research misbehavior, while narcissism, psychopathy and self-esteem were not. Exploratory analysis revealed that narcissism and research misconduct were more severe among persons in higher academic ranks (i.e., professors) (p<0.01 and p<0.001, respectively), and self-esteem scores and publication pressure were lower (p<0.001 and p<0.01, respectively) as compared to postgraduate PhD fellows.
Machiavellianism may be a risk factor for research misbehaviour. Narcissism and research misbehaviour were more prevalent among biomedical scientists in higher academic positions. These results suggest that personality has an impact on research behavior and should be taken into account in fostering responsible conduct of research.
Celotno besedilo
Dostopno za:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, SIK, UILJ, UKNU, UL, UM, UPUK
Publication of scientific research papers is important for professionals working in academic medical centres. Quantitative measures of scientific output determine status and prestige, and serve to ...rank universities as well as individuals. The pressure to generate maximum scientific output is high, and quantitative aspects may tend to dominate over qualitative ones. How this pressure influences professionals' perception of science and their personal well-being is unknown.
We performed an online survey inviting all medical professors (n = 1206) of the 8 academic medical centres in The Netherlands to participate. They were asked to fill out 2 questionnaires; a validated Publication Pressure Questionnaire and the Maslach Burnout Inventory. In total, 437 professors completed the questionnaires. among them, 54% judge that publication pressure 'has become excessive', 39% believe that publication pressure 'affects the credibility of medical research' and 26% judge that publication pressure has a 'sickening effect on medical science'. The burn out questionnaire indicates that 24% of medical professors have signs of burn out. The number of years of professorship was significantly related with experiencing less publication pressure. Significant and strong associations between burn out symptoms and the level of perceived publication pressure were found. The main limitation is the possibility of response bias.
A substantial proportion of medical professors believe that publication pressure has become excessive, and have a cynical view on the validity of medical science. These perceptions are statistically correlated to burn out symptoms. Further research should address the effects of publication pressure in more detail and identify alternative ways to stimulate the quality of medical science.
Celotno besedilo
Dostopno za:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, SIK, UILJ, UKNU, UL, UM, UPUK
Various stakeholders in science have put research integrity high on their agenda. Among them, research funders are prominently placed to foster research integrity by requiring that the organizations ...and individual researchers they support make an explicit commitment to research integrity. Moreover, funders need to adopt appropriate research integrity practices themselves. To facilitate this, we recommend that funders develop and implement a Research Integrity Promotion Plan (RIPP). This Consensus View offers a range of examples of how funders are already promoting research integrity, distills 6 core topics that funders should cover in a RIPP, and provides guidelines on how to develop and implement a RIPP. We believe that the 6 core topics we put forward will guide funders towards strengthening research integrity policy in their organization and guide the researchers and research organizations they fund.
Celotno besedilo
Dostopno za:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, SIK, UILJ, UKNU, UL, UM, UPUK
PDF
