The design of randomised controlled trials (RCTs) should incorporate characteristics (such as concealment of randomised allocation and blinding of participants and personnel) that avoid biases ...resulting from lack of comparability of the intervention and control groups. Empirical evidence suggests that the absence of such characteristics leads to biased intervention effect estimates, but the findings of different studies are not consistent.
To examine the influence of unclear or inadequate random sequence generation and allocation concealment, and unclear or absent double blinding, on intervention effect estimates and between-trial heterogeneity, and whether or not these influences vary with type of clinical area, intervention, comparison and outcome measure.
Data were combined from seven contributing meta-epidemiological studies (collections of meta-analyses in which trial characteristics are assessed and results recorded). The resulting database was used to identify and remove overlapping meta-analyses. Outcomes were coded such that odds ratios < 1 correspond to beneficial intervention effects. Outcome measures were classified as mortality, other objective or subjective. We examined agreement between assessments of trial characteristics in trials assessed in more than one contributing study. We used hierarchical Bayesian bias models to estimate the effect of trial characteristics on average bias quantified as ratios of odds ratios (RORs) with 95% credible intervals (CrIs) comparing trials with and without a characteristic and in increasing between-trial heterogeneity.
The analysis data set contained 1973 trials included in 234 meta-analyses. Median kappa statistics for agreement between assessments of trial characteristics were: sequence generation 0.60, allocation concealment 0.58 and blinding 0.87. Intervention effect estimates were exaggerated by an average 11% in trials with inadequate or unclear (compared with adequate) sequence generation (ROR 0.89, 95% CrI 0.82 to 0.96); between-trial heterogeneity was higher among such trials. Bias associated with inadequate or unclear sequence generation was greatest for subjective outcomes (ROR 0.83, 95% CrI 0.74 to 0.94) and the increase in heterogeneity was greatest for such outcomes standard deviation (SD) 0.20, 95% CrI 0.03 to 0.32. The effect of inadequate or unclear (compared with adequate) allocation concealment was greatest among meta-analyses with a subjectively assessed outcome intervention effect (ROR 0.85, 95% CrI 0.75 to 0.95), and the increase in between-trial heterogeneity was also greatest for such outcomes (SD 0.20, 95% CrI 0.02 to 0.33). Lack of, or unclear, double blinding (compared with double blinding) was associated with an average 13% exaggeration of intervention effects (ROR 0.87, 95% CrI 0.79 to 0.96), and between-trial heterogeneity was increased for such studies (SD 0.14, 95% CrI 0.02 to 0.30). Average bias (ROR 0.78, 95% CrI 0.65 to 0.92) and between-trial heterogeneity (SD 0.37, 95% CrI 0.19 to 0.53) were greatest for meta-analyses assessing subjective outcomes. Among meta-analyses with subjectively assessed outcomes, the effect of lack of blinding appeared greater than the effect of inadequate or unclear sequence generation or allocation concealment.
Bias associated with specific reported study design characteristics leads to exaggeration of beneficial intervention effect estimates and increases in between-trial heterogeneity. For each of the three characteristics assessed, these effects were greatest for subjectively assessed outcomes. Assessments of the risk of bias in RCTs should account for these findings. Further research is needed to understand the effects of attrition bias, as well as the relative importance of blinding of patients, care-givers and outcome assessors, and thus separate the effects of performance and detection bias.
National Institute for Health Research Health Technology Assessment programme.
To develop recommendations for the diagnosis, prevention and treatment of neuropsychiatric systemic lupus erythematosus (NPSLE) manifestations.
The authors compiled questions on prevalence and risk ...factors, diagnosis and monitoring, therapy and prognosis of NPSLE. A systematic literature search was performed and evidence was categorised based on sample size and study design.
Systemic lupus erythematosus (SLE) patients are at increased risk of several neuropsychiatric manifestations. Common (cumulative incidence > 5%) manifestations include cerebrovascular disease (CVD) and seizures; relatively uncommon (1-5%) are severe cognitive dysfunction, major depression, acute confusional state (ACS), peripheral nervous disorders psychosis. Strong risk factors (at least fivefold increased risk) are previous or concurrent severe NPSLE (for cognitive dysfunction, seizures) and antiphospholipid antibodies (for CVD, seizures, chorea). The diagnostic work-up of suspected NPSLE is comparable to that in patients without SLE who present with the same manifestations, and aims to exclude causes unrelated to SLE. Investigations include cerebrospinal fluid analysis (to exclude central nervous system infection), EEG (to diagnose seizure disorder), neuropsychological tests (to assess cognitive dysfunction), nerve conduction studies (for peripheral neuropathy) and MRI (T1/T2, fluid-attenuating inversion recovery, diffusion-weighted imaging, enhanced T1 sequence). Glucocorticoids and immunosuppressive therapy are indicated when NPSLE is thought to reflect an inflammatory process (optic neuritis, transverse myelitis, peripheral neuropathy, refractory seizures, psychosis, ACS) and in the presence of generalised lupus activity. Antiplatelet/anticoagulation therapy is indicated when manifestations are related to antiphospholipid antibodies, particularly thrombotic CVD.
Neuropsychiatric manifestations in SLE patients should be first evaluated and treated as in patients without SLE, and secondarily attributed to SLE and treated accordingly.
In response to overwhelming evidence and the consequences of poor-quality reporting of randomized, controlled trials (RCTs), many medical journals and editorial groups have now endorsed the CONSORT ...(Consolidated Standards of Reporting Trials) statement, a 22-item checklist and flow diagram. Because CONSORT primarily aimed at improving the quality of reporting of efficacy, only 1 checklist item specifically addressed the reporting of safety. Considerable evidence suggests that reporting of harms-related data from RCTs also needs improvement. Members of the CONSORT Group, including journal editors and scientists, met in Montebello, Quebec, Canada, in May 2003 to address this problem. The result is the following document: the standard CONSORT checklist with 10 new recommendations about reporting harms-related issues, accompanying explanation, and examples to highlight specific aspects of proper reporting. We hope that this document, in conjunction with other CONSORT-related materials (http://www.consort-statement.org), will help authors improve their reporting of harms-related data from RCTs. Better reporting will help readers critically appraise and interpret trial results. Journals can support this goal by revising Instructions to Authors so that they refer authors to this document.
Systemic lupus erythematosus (SLE) is a complex disease with variable presentations, course and prognosis. We sought to develop evidence-based recommendations addressing the major issues in the ...management of SLE.
The EULAR Task Force on SLE comprised 19 specialists and a clinical epidemiologist. Key questions for the management of SLE were compiled using the Delphi technique. A systematic search of PubMed and Cochrane Library Reports was performed using McMaster/Hedges clinical queries' strategies for questions related to the diagnosis, prognosis, monitoring and treatment of SLE. For neuropsychiatric, pregnancy and antiphospholipid syndrome questions, the search was conducted using an array of relevant terms. Evidence was categorised based on sample size and type of design, and the categories of available evidence were identified for each recommendation. The strength of recommendation was assessed based on the category of available evidence, and agreement on the statements was measured across the 19 specialists.
Twelve questions were generated regarding the prognosis, diagnosis, monitoring and treatment of SLE, including neuropsychiatric SLE, pregnancy, the antiphospholipid syndrome and lupus nephritis. The evidence to support each proposition was evaluated and scored. After discussion and votes, the final recommendations were presented using brief statements. The average agreement among experts was 8.8 out of 10.
Recommendations for the management of SLE were developed using an evidence-based approach followed by expert consensus with high level of agreement among the experts.
Are meta-analyses the brave new world, or are the critics of such combined analyses right to say that the biases inherent in clinical trials make them uncombinable? Negative trials are often ...unreported, and hence can be missed by meta-analysts. And how much heterogeneity between trials is acceptable? A recent major criticism is that large randomised trials do not always agree with a prior meta-analysis. Neither individual trials nor meta-analyses, reporting as they do on population effects, tell how to treat the individual patient. Here we take a more rounded approach to meta-analyses, arguing that their strengths outweigh their weaknesses, although the latter must not be brushed aside.
Convalescent plasma is a proposed treatment for COVID-19.
To assess clinical outcomes with convalescent plasma treatment vs placebo or standard of care in peer-reviewed and preprint publications or ...press releases of randomized clinical trials (RCTs).
PubMed, the Cochrane COVID-19 trial registry, and the Living Overview of Evidence platform were searched until January 29, 2021.
The RCTs selected compared any type of convalescent plasma vs placebo or standard of care for patients with confirmed or suspected COVID-19 in any treatment setting.
Two reviewers independently extracted data on relevant clinical outcomes, trial characteristics, and patient characteristics and used the Cochrane Risk of Bias Assessment Tool. The primary analysis included peer-reviewed publications of RCTs only, whereas the secondary analysis included all publicly available RCT data (peer-reviewed publications, preprints, and press releases). Inverse variance-weighted meta-analyses were conducted to summarize the treatment effects. The certainty of the evidence was assessed using the Grading of Recommendations Assessment, Development, and Evaluation.
All-cause mortality, length of hospital stay, clinical improvement, clinical deterioration, mechanical ventilation use, and serious adverse events.
A total of 1060 patients from 4 peer-reviewed RCTs and 10 722 patients from 6 other publicly available RCTs were included. The summary risk ratio (RR) for all-cause mortality with convalescent plasma in the 4 peer-reviewed RCTs was 0.93 (95% CI, 0.63 to 1.38), the absolute risk difference was -1.21% (95% CI, -5.29% to 2.88%), and there was low certainty of the evidence due to imprecision. Across all 10 RCTs, the summary RR was 1.02 (95% CI, 0.92 to 1.12) and there was moderate certainty of the evidence due to inclusion of unpublished data. Among the peer-reviewed RCTs, the summary hazard ratio was 1.17 (95% CI, 0.07 to 20.34) for length of hospital stay, the summary RR was 0.76 (95% CI, 0.20 to 2.87) for mechanical ventilation use (the absolute risk difference for mechanical ventilation use was -2.56% 95% CI, -13.16% to 8.05%), and there was low certainty of the evidence due to imprecision for both outcomes. Limited data on clinical improvement, clinical deterioration, and serious adverse events showed no significant differences.
Treatment with convalescent plasma compared with placebo or standard of care was not significantly associated with a decrease in all-cause mortality or with any benefit for other clinical outcomes. The certainty of the evidence was low to moderate for all-cause mortality and low for other outcomes.
There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on ...the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research.
Major depressive disorder is one of the most common, burdensome, and costly psychiatric disorders worldwide in adults. Pharmacological and non-pharmacological treatments are available; however, ...because of inadequate resources, antidepressants are used more frequently than psychological interventions. Prescription of these agents should be informed by the best available evidence. Therefore, we aimed to update and expand our previous work to compare and rank antidepressants for the acute treatment of adults with unipolar major depressive disorder.
We did a systematic review and network meta-analysis. We searched Cochrane Central Register of Controlled Trials, CINAHL, Embase, LILACS database, MEDLINE, MEDLINE In-Process, PsycINFO, the websites of regulatory agencies, and international registers for published and unpublished, double-blind, randomised controlled trials from their inception to Jan 8, 2016. We included placebo-controlled and head-to-head trials of 21 antidepressants used for the acute treatment of adults (≥18 years old and of both sexes) with major depressive disorder diagnosed according to standard operationalised criteria. We excluded quasi-randomised trials and trials that were incomplete or included 20% or more of participants with bipolar disorder, psychotic depression, or treatment-resistant depression; or patients with a serious concomitant medical illness. We extracted data following a predefined hierarchy. In network meta-analysis, we used group-level data. We assessed the studies' risk of bias in accordance to the Cochrane Handbook for Systematic Reviews of Interventions, and certainty of evidence using the Grading of Recommendations Assessment, Development and Evaluation framework. Primary outcomes were efficacy (response rate) and acceptability (treatment discontinuations due to any cause). We estimated summary odds ratios (ORs) using pairwise and network meta-analysis with random effects. This study is registered with PROSPERO, number CRD42012002291.
We identified 28 552 citations and of these included 522 trials comprising 116 477 participants. In terms of efficacy, all antidepressants were more effective than placebo, with ORs ranging between 2·13 (95% credible interval CrI 1·89–2·41) for amitriptyline and 1·37 (1·16–1·63) for reboxetine. For acceptability, only agomelatine (OR 0·84, 95% CrI 0·72–0·97) and fluoxetine (0·88, 0·80–0·96) were associated with fewer dropouts than placebo, whereas clomipramine was worse than placebo (1·30, 1·01–1·68). When all trials were considered, differences in ORs between antidepressants ranged from 1·15 to 1·55 for efficacy and from 0·64 to 0·83 for acceptability, with wide CrIs on most of the comparative analyses. In head-to-head studies, agomelatine, amitriptyline, escitalopram, mirtazapine, paroxetine, venlafaxine, and vortioxetine were more effective than other antidepressants (range of ORs 1·19–1·96), whereas fluoxetine, fluvoxamine, reboxetine, and trazodone were the least efficacious drugs (0·51–0·84). For acceptability, agomelatine, citalopram, escitalopram, fluoxetine, sertraline, and vortioxetine were more tolerable than other antidepressants (range of ORs 0·43–0·77), whereas amitriptyline, clomipramine, duloxetine, fluvoxamine, reboxetine, trazodone, and venlafaxine had the highest dropout rates (1·30–2·32). 46 (9%) of 522 trials were rated as high risk of bias, 380 (73%) trials as moderate, and 96 (18%) as low; and the certainty of evidence was moderate to very low.
All antidepressants were more efficacious than placebo in adults with major depressive disorder. Smaller differences between active drugs were found when placebo-controlled trials were included in the analysis, whereas there was more variability in efficacy and acceptability in head-to-head trials. These results should serve evidence-based practice and inform patients, physicians, guideline developers, and policy makers on the relative merits of the different antidepressants.
National Institute for Health Research Oxford Health Biomedical Research Centre and the Japan Society for the Promotion of Science.
Controversy and uncertainty ensue when the results of clinical research on the effectiveness of interventions are subsequently contradicted. Controversies are most prominent when high-impact research ...is involved.
To understand how frequently highly cited studies are contradicted or find effects that are stronger than in other similar studies and to discern whether specific characteristics are associated with such refutation over time.
All original clinical research studies published in 3 major general clinical journals or high-impact-factor specialty journals in 1990-2003 and cited more than 1000 times in the literature were examined.
The results of highly cited articles were compared against subsequent studies of comparable or larger sample size and similar or better controlled designs. The same analysis was also performed comparatively for matched studies that were not so highly cited.
Of 49 highly cited original clinical research studies, 45 claimed that the intervention was effective. Of these, 7 (16%) were contradicted by subsequent studies, 7 others (16%) had found effects that were stronger than those of subsequent studies, 20 (44%) were replicated, and 11 (24%) remained largely unchallenged. Five of 6 highly-cited nonrandomized studies had been contradicted or had found stronger effects vs 9 of 39 randomized controlled trials (P = .008). Among randomized trials, studies with contradicted or stronger effects were smaller (P = .009) than replicated or unchallenged studies although there was no statistically significant difference in their early or overall citation impact. Matched control studies did not have a significantly different share of refuted results than highly cited studies, but they included more studies with "negative" results.
Contradiction and initially stronger effects are not unusual in highly cited research of clinical interventions and their outcomes. The extent to which high citations may provoke contradictions and vice versa needs more study. Controversies are most common with highly cited nonrandomized studies, but even the most highly cited randomized trials may be challenged and refuted over time, especially small ones.
Psychosis is a heterogeneous psychiatric condition for which a multitude of risk and protective factors have been suggested. This umbrella review aimed to classify the strength of evidence for the ...associations between each factor and psychotic disorders whilst controlling for several biases. The Web of Knowledge database was searched to identify systematic reviews and meta‐analyses of observational studies which examined associations between socio‐demographic, parental, perinatal, later factors or antecedents and psychotic disorders, and which included a comparison group of healthy controls, published from 1965 to January 31, 2017. The literature search and data extraction followed PRISMA and MOOSE guidelines. The association between each factor and ICD or DSM diagnoses of non‐organic psychotic disorders was graded into convincing, highly suggestive, suggestive, weak, or non‐significant according to a standardized classification based on: number of psychotic cases, random‐effects p value, largest study 95% confidence interval, heterogeneity between studies, 95% prediction interval, small study effect, and excess significance bias. In order to assess evidence for temporality of association, we also conducted sensitivity analyses restricted to data from prospective studies. Fifty‐five meta‐analyses or systematic reviews were included in the umbrella review, corresponding to 683 individual studies and 170 putative risk or protective factors for psychotic disorders. Only the ultra‐high‐risk state for psychosis (odds ratio, OR=9.32, 95% CI: 4.91‐17.72) and Black‐Caribbean ethnicity in England (OR=4.87, 95% CI: 3.96‐6.00) showed convincing evidence of association. Six factors were highly suggestive (ethnic minority in low ethnic density area, second generation immigrants, trait anhedonia, premorbid IQ, minor physical anomalies, and olfactory identification ability), and nine were suggestive (urbanicity, ethnic minority in high ethnic density area, first generation immigrants, North‐African immigrants in Europe, winter/spring season of birth in Northern hemisphere, childhood social withdrawal, childhood trauma, Toxoplasma gondii IgG, and non‐right handedness). When only prospective studies were considered, the evidence was convincing for ultra‐high‐risk state and suggestive for urbanicity only. In summary, this umbrella review found several factors to be associated with psychotic disorders with different levels of evidence. These risk or protective factors represent a starting point for further etiopathological research and for the improvement of the prediction of psychosis.
PDF
