Most meta-analyses in systematic reviews, including Cochrane ones, do not have sufficient statistical power to detect or refute even large intervention effects. This is why a meta-analysis ought to ...be regarded as an interim analysis on its way towards a required information size. The results of the meta-analyses should relate the total number of randomised participants to the estimated required meta-analytic information size accounting for statistical diversity. When the number of participants and the corresponding number of trials in a meta-analysis are insufficient, the use of the traditional 95% confidence interval or the 5% statistical significance threshold will lead to too many false positive conclusions (type I errors) and too many false negative conclusions (type II errors).
We developed a methodology for interpreting meta-analysis results, using generally accepted, valid evidence on how to adjust thresholds for significance in randomised clinical trials when the required sample size has not been reached.
The Lan-DeMets trial sequential monitoring boundaries in Trial Sequential Analysis offer adjusted confidence intervals and restricted thresholds for statistical significance when the diversity-adjusted required information size and the corresponding number of required trials for the meta-analysis have not been reached. Trial Sequential Analysis provides a frequentistic approach to control both type I and type II errors. We define the required information size and the corresponding number of required trials in a meta-analysis and the diversity (D
) measure of heterogeneity. We explain the reasons for using Trial Sequential Analysis of meta-analysis when the actual information size fails to reach the required information size. We present examples drawn from traditional meta-analyses using unadjusted naïve 95% confidence intervals and 5% thresholds for statistical significance. Spurious conclusions in systematic reviews with traditional meta-analyses can be reduced using Trial Sequential Analysis. Several empirical studies have demonstrated that the Trial Sequential Analysis provides better control of type I errors and of type II errors than the traditional naïve meta-analysis.
Trial Sequential Analysis represents analysis of meta-analytic data, with transparent assumptions, and better control of type I and type II errors than the traditional meta-analysis using naïve unadjusted confidence intervals.
Missing data may seriously compromise inferences from randomised clinical trials, especially if missing data are not handled appropriately. The potential bias due to missing data depends on the ...mechanism causing the data to be missing, and the analytical methods applied to amend the missingness. Therefore, the analysis of trial data with missing values requires careful planning and attention.
The authors had several meetings and discussions considering optimal ways of handling missing data to minimise the bias potential. We also searched PubMed (key words: missing data; randomi*; statistical analysis) and reference lists of known studies for papers (theoretical papers; empirical studies; simulation studies; etc.) on how to deal with missing data when analysing randomised clinical trials.
Handling missing data is an important, yet difficult and complex task when analysing results of randomised clinical trials. We consider how to optimise the handling of missing data during the planning stage of a randomised clinical trial and recommend analytical approaches which may prevent bias caused by unavoidable missing data. We consider the strengths and limitations of using of best-worst and worst-best sensitivity analyses, multiple imputation, and full information maximum likelihood. We also present practical flowcharts on how to deal with missing data and an overview of the steps that always need to be considered during the analysis stage of a trial.
We present a practical guide and flowcharts describing when and how multiple imputation should be used to handle missing data in randomised clinical.
Evidence shows that antioxidant supplements may increase mortality. Our aims were to assess whether different doses of beta-carotene, vitamin A, and vitamin E affect mortality in primary and ...secondary prevention randomized clinical trials with low risk of bias.
The present study is based on our 2012 Cochrane systematic review analyzing beneficial and harmful effects of antioxidant supplements in adults. Using random-effects meta-analyses, meta-regression analyses, and trial sequential analyses, we examined the association between beta-carotene, vitamin A, and vitamin E, and mortality according to their daily doses and doses below and above the recommended daily allowances (RDA).
We included 53 randomized trials with low risk of bias (241,883 participants, aged 18 to 103 years, 44.6% women) assessing beta-carotene, vitamin A, and vitamin E. Meta-regression analysis showed that the dose of vitamin A was significantly positively associated with all-cause mortality. Beta-carotene in a dose above 9.6 mg significantly increased mortality (relative risk (RR) 1.06, 95% confidence interval (CI) 1.02 to 1.09, I(2) = 13%). Vitamin A in a dose above the RDA (> 800 µg) did not significantly influence mortality (RR 1.08, 95% CI 0.98 to 1.19, I(2) = 53%). Vitamin E in a dose above the RDA (> 15 mg) significantly increased mortality (RR 1.03, 95% CI 1.00 to 1.05, I(2) = 0%). Doses below the RDAs did not affect mortality, but data were sparse.
Beta-carotene and vitamin E in doses higher than the RDA seem to significantly increase mortality, whereas we lack information on vitamin A. Dose of vitamin A was significantly associated with increased mortality in meta-regression. We lack information on doses below the RDA.
All essential compounds to stay healthy cannot be synthesized in our body. Therefore, these compounds must be taken through our diet or obtained in other ways 1. Oxidative stress has been suggested to cause a variety of diseases 2. Therefore, it is speculated that antioxidant supplements could have a potential role in preventing diseases and death. Despite the fact that a normal diet in high-income countries may provide sufficient amounts of antioxidants 3,4, more than one third of adults regularly take antioxidant supplements 5,6.
There is increasing awareness that meta-analyses require a sufficiently large information size to detect or reject an anticipated intervention effect. The required information size in a meta-analysis ...may be calculated from an anticipated a priori intervention effect or from an intervention effect suggested by trials with low-risk of bias.
Information size calculations need to consider the total model variance in a meta-analysis to control type I and type II errors. Here, we derive an adjusting factor for the required information size under any random-effects model meta-analysis.
We devise a measure of diversity (D2) in a meta-analysis, which is the relative variance reduction when the meta-analysis model is changed from a random-effects into a fixed-effect model. D2 is the percentage that the between-trial variability constitutes of the sum of the between-trial variability and a sampling error estimate considering the required information size. D2 is different from the intuitively obvious adjusting factor based on the common quantification of heterogeneity, the inconsistency (I2), which may underestimate the required information size. Thus, D2 and I2 are compared and interpreted using several simulations and clinical examples. In addition we show mathematically that diversity is equal to or greater than inconsistency, that is D2 >or= I2, for all meta-analyses.
We conclude that D2 seems a better alternative than I2 to consider model variation in any random-effects meta-analysis despite the choice of the between trial variance estimator that constitutes the model. Furthermore, D2 can readily adjust the required information size in any random-effects model meta-analysis.
The evaluation of imprecision is a key dimension of the grading of the confidence in the estimate. Grading of Recommendations Assessment, Development and Evaluation (GRADE) gives recommendations on ...how to downgrade evidence for imprecision, but authors vary in their use. Trial Sequential Analysis (TSA) has been advocated for a more reliable assessment of imprecision. We aimed to evaluate reporting of and adherence to GRADE and to compare the assessment of imprecision of intervention effects assessed by GRADE and TSA in Cochrane systematic reviews.
In this cross-sectional study, we included 100 Cochrane reviews irrespective of type of intervention with a key dichotomous outcome meta-analyzed and assessed by GRADE. The methods and results sections of each review were assessed for adequacy of imprecision evaluation. We re-analyzed imprecision following the GRADE Handbook and the TSA Manual.
Overall, only 13.0% of reviews stated the criteria they applied to assess imprecision. The most common dimensions were the 95% width of the confidence intervals and the optimal information size. Review authors downgraded 48.0% of key outcomes due to imprecision. When imprecision was re-analyzed following the GRADE Handbook, 64% of outcomes were downgraded. Agreement between review authors' assessment and assessment by the authors of this study was moderate (kappa 0.43, 95% confidence interval CI 0.23 to 0.58). TSA downgraded 69.0% outcomes due to imprecision. Agreement between review authors' GRADE assessment and TSA, irrespective of downgrading levels, was moderate (kappa 0.43, 95% CI 0.21 to 0.57). Agreement between our GRADE assessment following the Handbook and TSA was substantial (kappa 0.66, 95% CI 0.49 to 0.79).
In a sample of Cochrane reviews, methods for assessing imprecision were rarely reported. GRADE according to Handbook guidelines and TSA led to more severe judgment of imprecision rather than GRADE adopted by reviews' authors. Cochrane initiatives to improve adherence to GRADE Handbook are warranted. TSA may transparently assist in such development.
Published evidence suggests that aspects of trial design lead to biased intervention effect estimates, but findings from different studies are inconsistent. This study combined data from 7 ...meta-epidemiologic studies and removed overlaps to derive a final data set of 234 unique meta-analyses containing 1973 trials. Outcome measures were classified as "mortality," "other objective," "or subjective," and Bayesian hierarchical models were used to estimate associations of trial characteristics with average bias and between-trial heterogeneity. Intervention effect estimates seemed to be exaggerated in trials with inadequate or unclear (vs. adequate) random-sequence generation (ratio of odds ratios, 0.89 95% credible interval {CrI}, 0.82 to 0.96) and with inadequate or unclear (vs. adequate) allocation concealment (ratio of odds ratios, 0.93 CrI, 0.87 to 0.99). Lack of or unclear double-blinding (vs. double-blinding) was associated with an average of 13% exaggeration of intervention effects (ratio of odds ratios, 0.87 CrI, 0.79 to 0.96), and between-trial heterogeneity was increased for such studies (SD increase in heterogeneity, 0.14 CrI, 0.02 to 0.30). For each characteristic, average bias and increases in between-trial heterogeneity were driven primarily by trials with subjective outcomes, with little evidence of bias in trials with objective and mortality outcomes. This study is limited by incomplete trial reporting, and findings may be confounded by other study design characteristics. Bias associated with study design characteristics may lead to exaggeration of intervention effect estimates and increases in between-trial heterogeneity in trials reporting subjectively assessed outcomes.
Little is known about whether treatment in a specialised out-patient mood disorder clinic improves long-term prognosis for patients discharged from initial psychiatric hospital admissions for bipolar ...disorder.
To assess the effect of treatment in a specialised out-patient mood disorder clinic v. standard decentralised psychiatric treatment among patients discharged from one of their first three psychiatric hospital admissions for bipolar disorder.
Patients discharged from their first, second or third hospital admission with a single manic episode or bipolar disorder were randomised to treatment in a specialised out-patient mood disorder clinic or standard care (ClinicalTrials.gov: NCT00253071). The primary outcome measure was readmission to hospital, which was obtained from the Danish Psychiatric Central Register.
A total of 158 patients with mania/bipolar disorder were included. The rate of readmission to hospital was significantly decreased for patients treated in the mood disorder clinic compared with standard treatment (unadjusted hazard ratio 0.60, 95% CI 0.37-0.97, P = 0.034). Patients treated in the mood disorder clinic more often used a mood stabiliser or an antipsychotic and satisfaction with treatment was more prevalent than among patients who received standard care.
Treatment in a specialised mood disorder clinic early in the course of bipolar disorder substantially reduces readmission to a psychiatric hospital and increases satisfaction with care.
Coronavirus disease 2019 (COVID-19) is a rapidly spreading disease that has caused extensive burden to individuals, families, countries, and the world. Effective treatments of COVID-19 are urgently ...needed.
This is the first edition of a living systematic review of randomized clinical trials comparing the effects of all treatment interventions for participants in all age groups with COVID-19. We planned to conduct aggregate data meta-analyses, trial sequential analyses, network meta-analysis, and individual patient data meta-analyses. Our systematic review is based on Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) and Cochrane guidelines, and our 8-step procedure for better validation of clinical significance of meta-analysis results. We performed both fixed-effect and random-effects meta-analyses. Primary outcomes were all-cause mortality and serious adverse events. Secondary outcomes were admission to intensive care, mechanical ventilation, renal replacement therapy, quality of life, and nonserious adverse events. We used Grading of Recommendations Assessment, Development and Evaluation (GRADE) to assess the certainty of evidence. We searched relevant databases and websites for published and unpublished trials until August 7, 2020. Two reviewers independently extracted data and assessed trial methodology. We included 33 randomized clinical trials enrolling a total of 13,312 participants. All trials were at overall high risk of bias. We identified one trial randomizing 6,425 participants to dexamethasone versus standard care. This trial showed evidence of a beneficial effect of dexamethasone on all-cause mortality (rate ratio 0.83; 95% confidence interval CI 0.75-0.93; p < 0.001; low certainty) and on mechanical ventilation (risk ratio RR 0.77; 95% CI 0.62-0.95; p = 0.021; low certainty). It was possible to perform meta-analysis of 10 comparisons. Meta-analysis showed no evidence of a difference between remdesivir versus placebo on all-cause mortality (RR 0.74; 95% CI 0.40-1.37; p = 0.34, I2 = 58%; 2 trials; very low certainty) or nonserious adverse events (RR 0.94; 95% CI 0.80-1.11; p = 0.48, I2 = 29%; 2 trials; low certainty). Meta-analysis showed evidence of a beneficial effect of remdesivir versus placebo on serious adverse events (RR 0.77; 95% CI 0.63-0.94; p = 0.009, I2 = 0%; 2 trials; very low certainty) mainly driven by respiratory failure in one trial. Meta-analyses and trial sequential analyses showed that we could exclude the possibility that hydroxychloroquine versus standard care reduced the risk of all-cause mortality (RR 1.07; 95% CI 0.97-1.19; p = 0.17; I2 = 0%; 7 trials; low certainty) and serious adverse events (RR 1.07; 95% CI 0.96-1.18; p = 0.21; I2 = 0%; 7 trials; low certainty) by 20% or more, and meta-analysis showed evidence of a harmful effect on nonserious adverse events (RR 2.40; 95% CI 2.01-2.87; p < 0.00001; I2 = 90%; 6 trials; very low certainty). Meta-analysis showed no evidence of a difference between lopinavir-ritonavir versus standard care on serious adverse events (RR 0.64; 95% CI 0.39-1.04; p = 0.07, I2 = 0%; 2 trials; very low certainty) or nonserious adverse events (RR 1.14; 95% CI 0.85-1.53; p = 0.38, I2 = 75%; 2 trials; very low certainty). Meta-analysis showed no evidence of a difference between convalescent plasma versus standard care on all-cause mortality (RR 0.60; 95% CI 0.33-1.10; p = 0.10, I2 = 0%; 2 trials; very low certainty). Five single trials showed statistically significant results but were underpowered to confirm or reject realistic intervention effects. None of the remaining trials showed evidence of a difference on our predefined outcomes. Because of the lack of relevant data, it was not possible to perform other meta-analyses, network meta-analysis, or individual patient data meta-analyses. The main limitation of this living review is the paucity of data currently available. Furthermore, the included trials were all at risks of systematic errors and random errors.
Our results show that dexamethasone and remdesivir might be beneficial for COVID-19 patients, but the certainty of the evidence was low to very low, so more trials are needed. We can exclude the possibility of hydroxychloroquine versus standard care reducing the risk of death and serious adverse events by 20% or more. Otherwise, no evidence-based treatment for COVID-19 currently exists. This review will continuously inform best practice in treatment and clinical research of COVID-19.
Periodontal treatment might reduce adverse pregnancy outcomes. The efficacy of periodontal treatment to prevent preterm birth, low birth weight, and perinatal mortality was evaluated using ...meta-analysis and trial sequential analysis.
An existing systematic review was updated and meta-analyses performed. Risk of bias, heterogeneity, and publication bias were evaluated, and meta-regression performed. Subgroup analysis was used to compare different studies with low and high risk of bias and different populations, i.e., risk groups. Trial sequential analysis was used to assess risk of random errors.
Thirteen randomized clinical trials evaluating 6283 pregnant women were meta-analyzed. Four and nine trials had low and high risk of bias, respectively. Overall, periodontal treatment had no significant effect on preterm birth (odds ratio 95% confidence interval 0.79 0.57-1.10) or low birth weight (0.69 0.43-1.13). Trial sequential analysis demonstrated that futility was not reached for any of the outcomes. For populations with moderate occurrence (< 20%) of preterm birth or low birth weight, periodontal treatment was not efficacious for any of the outcomes, and trial sequential analyses indicated that further trials might be futile. For populations with high occurrence (≥ 20%) of preterm birth and low birth weight, periodontal treatment seemed to reduce the risk of preterm birth (0.42 0.24-0.73) and low birth weight (0.32 0.15-0.67), but trial sequential analyses showed that firm evidence was not reached. Periodontal treatment did not significantly affect perinatal mortality, and firm evidence was not reached. Risk of bias, but not publication bias or patients' age modified the effect estimates.
Providing periodontal treatment to pregnant women could potentially reduce the risks of perinatal outcomes, especially in mothers with high risks. Conclusive evidence could not be reached due to risks of bias, risks of random errors, and unclear effects of confounding. Further randomized clinical trials are required.
Assessing benefits and harms of health interventions is resource-intensive and often requires feasibility and pilot trials followed by adequately powered randomised clinical trials. Data from ...feasibility and pilot trials are used to inform the design and sample size of the adequately powered randomised clinical trials. When a randomised clinical trial is conducted, results from feasibility and pilot trials may be disregarded in terms of benefits and harms.
We describe using feasibility and pilot trial data in the Trial Sequential Analysis software to estimate the required sample size for one or more trials investigating a behavioural smoking cessation intervention. We show how data from a new, planned trial can be combined with data from the earlier trials using trial sequential analysis methods to assess the intervention's effects.
We provide a worked example to illustrate how we successfully used the Trial Sequential Analysis software to arrive at a sensible sample size for a new randomised clinical trial and use it in the argumentation for research funds for the trial.
Trial Sequential Analysis can utilise data from feasibility and pilot trials as well as other trials, to estimate a sample size for one or more, similarly designed, future randomised clinical trials. As this method uses available data, estimated sample sizes may be smaller than they would have been using conventional sample size estimation methods.
PDF
