To synthesise evidence on the average bias and heterogeneity associated with reported methodological features of randomized trials.
Systematic review of meta-epidemiological studies.
We retrieved ...eligible studies included in a recent AHRQ-EPC review on this topic (latest search September 2012), and searched Ovid MEDLINE and Ovid EMBASE for studies indexed from Jan 2012-May 2015. Data were extracted by one author and verified by another. We combined estimates of average bias (e.g. ratio of odds ratios (ROR) or difference in standardised mean differences (dSMD)) in meta-analyses using the random-effects model. Analyses were stratified by type of outcome ("mortality" versus "other objective" versus "subjective"). Direction of effect was standardised so that ROR < 1 and dSMD < 0 denotes a larger intervention effect estimate in trials with an inadequate or unclear (versus adequate) characteristic.
We included 24 studies. The available evidence suggests that intervention effect estimates may be exaggerated in trials with inadequate/unclear (versus adequate) sequence generation (ROR 0.93, 95% CI 0.86 to 0.99; 7 studies) and allocation concealment (ROR 0.90, 95% CI 0.84 to 0.97; 7 studies). For these characteristics, the average bias appeared to be larger in trials of subjective outcomes compared with other objective outcomes. Also, intervention effects for subjective outcomes appear to be exaggerated in trials with lack of/unclear blinding of participants (versus blinding) (dSMD -0.37, 95% CI -0.77 to 0.04; 2 studies), lack of/unclear blinding of outcome assessors (ROR 0.64, 95% CI 0.43 to 0.96; 1 study) and lack of/unclear double blinding (ROR 0.77, 95% CI 0.61 to 0.93; 1 study). The influence of other characteristics (e.g. unblinded trial personnel, attrition) is unclear.
Certain characteristics of randomized trials may exaggerate intervention effect estimates. The average bias appears to be greatest in trials of subjective outcomes. More research on several characteristics, particularly attrition and selective reporting, is needed.
Full text
Available for:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, SIK, UILJ, UKNU, UL, UM, UPUK
Intensive care units (ICUs) face financial, bed management, and staffing constraints. Detailed data covering all aspects of patients' journeys into and through intensive care are now collected and ...stored in electronic health records: machine learning has been used to analyse such data in order to provide decision support to clinicians.
Systematic review of the applications of machine learning to routinely collected ICU data. Web of Science and MEDLINE databases were searched to identify candidate articles: those on image processing were excluded. The study aim, the type of machine learning used, the size of dataset analysed, whether and how the model was validated, and measures of predictive accuracy were extracted.
Of 2450 papers identified, 258 fulfilled eligibility criteria. The most common study aims were predicting complications (77 papers 29.8% of studies), predicting mortality (70 27.1%), improving prognostic models (43 16.7%), and classifying sub-populations (29 11.2%). Median sample size was 488 (IQR 108-4099): 41 studies analysed data on > 10,000 patients. Analyses focused on 169 (65.5%) papers that used machine learning to predict complications, mortality, length of stay, or improvement of health. Predictions were validated in 161 (95.2%) of these studies: the area under the ROC curve (AUC) was reported by 97 (60.2%) but only 10 (6.2%) validated predictions using independent data. The median AUC was 0.83 in studies of 1000-10,000 patients, rising to 0.94 in studies of > 100,000 patients. The most common machine learning methods were neural networks (72 studies 42.6%), support vector machines (40 23.7%), and classification/decision trees (34 20.1%). Since 2015 (125 studies 48.4%), the most common methods were support vector machines (37 studies 29.6%) and random forests (29 23.2%).
The rate of publication of studies using machine learning to analyse routinely collected ICU data is increasing rapidly. The sample sizes used in many published studies are too small to exploit the potential of these methods. Methodological and reporting guidelines are needed, particularly with regard to the choice of method and validation of predictions, to increase confidence in reported findings and aid in translating findings towards routine use in clinical practice.
Published evidence suggests that aspects of trial design lead to biased intervention effect estimates, but findings from different studies are inconsistent. This study combined data from 7 ...meta-epidemiologic studies and removed overlaps to derive a final data set of 234 unique meta-analyses containing 1973 trials. Outcome measures were classified as "mortality," "other objective," "or subjective," and Bayesian hierarchical models were used to estimate associations of trial characteristics with average bias and between-trial heterogeneity. Intervention effect estimates seemed to be exaggerated in trials with inadequate or unclear (vs. adequate) random-sequence generation (ratio of odds ratios, 0.89 95% credible interval {CrI}, 0.82 to 0.96) and with inadequate or unclear (vs. adequate) allocation concealment (ratio of odds ratios, 0.93 CrI, 0.87 to 0.99). Lack of or unclear double-blinding (vs. double-blinding) was associated with an average of 13% exaggeration of intervention effects (ratio of odds ratios, 0.87 CrI, 0.79 to 0.96), and between-trial heterogeneity was increased for such studies (SD increase in heterogeneity, 0.14 CrI, 0.02 to 0.30). For each characteristic, average bias and increases in between-trial heterogeneity were driven primarily by trials with subjective outcomes, with little evidence of bias in trials with objective and mortality outcomes. This study is limited by incomplete trial reporting, and findings may be confounded by other study design characteristics. Bias associated with study design characteristics may lead to exaggeration of intervention effect estimates and increases in between-trial heterogeneity in trials reporting subjectively assessed outcomes.
In 2003, the QUADAS tool for systematic reviews of diagnostic accuracy studies was developed. Experience, anecdotal reports, and feedback suggested areas for improvement; therefore, QUADAS-2 was ...developed. This tool comprises 4 domains: patient selection, index test, reference standard, and flow and timing. Each domain is assessed in terms of risk of bias, and the first 3 domains are also assessed in terms of concerns regarding applicability. Signalling questions are included to help judge risk of bias. The QUADAS-2 tool is applied in 4 phases: summarize the review question, tailor the tool and produce review-specific guidance, construct a flow diagram for the primary study, and judge bias and applicability. This tool will allow for more transparent rating of bias and applicability of primary diagnostic accuracy studies.
Most studies have some missing data. Jonathan Sterne and colleagues describe the appropriate use and reporting of the multiple imputation approach to dealing with them
Full text
Available for:
BFBNIB, CMK, NMLJ, NUK, PNG, SAZU, UL, UM, UPUK
A new wave of COVID-19 cases caused by the highly transmissible delta variant is exacerbating the worldwide public health crisis, and has led to consideration of the potential need for, and optimal ...timing of, booster doses for vaccinated populations.1 Although the idea of further reducing the number of COVID-19 cases by enhancing immunity in vaccinated people is appealing, any decision to do so should be evidence-based and consider the benefits and risks for individuals and society. If unnecessary boosting causes significant adverse reactions, there could be implications for vaccine acceptance that go beyond COVID-19 vaccines. ...widespread boosting should be undertaken only if there is clear evidence that it is appropriate. Among vaccinated people, more of the severe disease could be in immunocompromised individuals, who are plausibly more likely to be offered and seek vaccination even though its efficacy is lower than it is in other people.2 Test-negative designs, which compare vaccination status of people who tested positive and those who tested negative, can sometimes reduce confounding,8 but do not prevent distortion of results due to the so-called collider bias.9 The probability that individuals with asymptomatic or mild COVID-19 infection will seek testing might be influenced by whether they are vaccinated. Mean follow-up was, however, only about 7 person-days (less than expected based on the apparent study design); perhaps more importantly, a very short-term protective effect would not necessarily imply worthwhile long-term benefit.12 In the USA, large numbers of adults are fully vaccinated, large numbers are unvaccinated, and systematic comparisons between them are ongoing.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
In 1997–1998 a widespread economic crisis hit the economies of many East/Southeast Asian countries; its impact on suicide rates across the region has not been systematically documented. We ...investigated the impact of the Asian economic crisis (1997–1998) on suicide in Japan, Hong Kong, South Korea, Taiwan, Singapore and Thailand. Suicide and population data for the period 1985–2006 were extracted from the World Health Organisation's mortality database and Taiwanese mortality statistics. Sex-specific age-standardised suicide rates for people aged 15
years or above were analysed using joinpoint regression. Trends in divorce, marriage, unemployment, gross domestic product (GDP) per capita and alcohol consumption were compared with trends in suicide rates graphically and using time-series analysis. Suicide mortality decreased in the late 1980s and early 1990s but subsequently increased markedly in all countries except Singapore, which had steadily declining suicide rates throughout the study period. Compared to 1997, male rates in 1998 rose by 39% in Japan, 44% in Hong Kong and 45% in Korea; rises in female rates were less marked. Male rates also rose in Thailand, but accurate data were incomplete. The economic crisis was associated with 10,400 more suicides in 1998 compared to 1997 in Japan, Hong Kong and Korea. Similar increases in suicide rates were not seen in Taiwan and Singapore, the two countries where the economic crisis had a smaller impact on GDP and unemployment. Time-series analyses indicated that some of the crisis's impact on male suicides was attributable to increases in unemployment. These findings suggest an association of the Asian economic crisis with a sharp increase in suicide mortality in some, but not all, East/Southeast Asian countries, and that these increases were most closely associated with rises in unemployment.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UL, UM, UPCLJ, UPUK, ZRSKP
Multiple imputation is increasingly recommended in epidemiology to adjust for the bias and loss of information that may occur in analyses restricted to study participants with complete data ...(“complete-case analyses”). However, little guidance is available on applying the method, including which variables to include in the imputation model and the number of imputations needed. Here, the authors used multiple imputation to analyze the prevalence of wheeze among 81-month-old children in the Avon Longitudinal Study of Parents and Children (Avon, United Kingdom; 1991–1999) and the association of wheeze with gender, maternal asthma, and maternal smoking. The authors examined how inclusion of different types of variables in the imputation model affected point estimates and precision, and assessed the impact of number of imputations on Monte Carlo variability. Inclusion of variables associated with the outcome in the imputation model increased odds ratios and reduced standard errors. When only 5 or 10 imputations were used, variability due to the imputation procedure was substantial enough to affect conclusions. Careful preliminary analysis identified the scope for multiple imputation to reduce bias and improve efficiency and provided guidance for building the imputation model. When data are missing, such preliminary analyses should be routinely undertaken and reported, regardless of whether multiple imputation is used in the final analysis.
Abstract
Background
Missing data are unavoidable in epidemiological research, potentially leading to bias and loss of precision. Multiple imputation (MI) is widely advocated as an improvement over ...complete case analysis (CCA). However, contrary to widespread belief, CCA is preferable to MI in some situations.
Methods
We provide guidance on choice of analysis when data are incomplete. Using causal diagrams to depict missingness mechanisms, we describe when CCA will not be biased by missing data and compare MI and CCA, with respect to bias and efficiency, in a range of missing data situations. We illustrate selection of an appropriate method in practice.
Results
For most regression models, CCA gives unbiased results when the chance of being a complete case does not depend on the outcome after taking the covariates into consideration, which includes situations where data are missing not at random. Consequently, there are situations in which CCA analyses are unbiased while MI analyses, assuming missing at random (MAR), are biased. By contrast MI, unlike CCA, is valid for all MAR situations and has the potential to use information contained in the incomplete cases and auxiliary variables to reduce bias and/or improve precision. For this reason, MI was preferred over CCA in our real data example.
Conclusions
Choice of method for dealing with missing data is crucial for validity of conclusions, and should be based on careful consideration of the reasons for the missing data, missing data patterns and the availability of auxiliary information.
BACKGROUND:Cardiovascular disease and non-AIDS malignancies have become major causes of death among HIV-infected individuals. The relative impact of lifestyle and HIV-related factors are debated.
...METHODS:We estimated associations of smoking with mortality more than 1 year after antiretroviral therapy (ART) initiation among HIV-infected individuals enrolled in European and North American cohorts. IDUs were excluded. Causes of death were assigned using standardized procedures. We used abridged life tables to estimate life expectancies. Life-years lost to HIV were estimated by comparison with the French background population.
RESULTS:Among 17 995 HIV-infected individuals followed for 79 760 person-years, the proportion of smokers was 60%. The mortality rate ratio (MRR) comparing smokers with nonsmokers was 1.94 95% confidence interval (95% CI) 1.56–2.41. The MRRs comparing current and previous smokers with never smokers were 1.70 (95% CI 1.23–2.34) and 0.92 (95% CI 0.64–1.34), respectively. Smokers had substantially higher mortality from cardiovascular disease, non-AIDS malignancies than nonsmokers MRR 6.28 (95% CI 2.19–18.0) and 2.67 (95% CI 1.60–4.46), respectively. Among 35-year-old HIV-infected men, the loss of life-years associated with smoking and HIV was 7.9 (95% CI 7.1–8.7) and 5.9 (95% CI 4.9–6.9), respectively. The life expectancy of virally suppressed, never-smokers was 43.5 years (95% CI 41.7–45.3), compared with 44.4 years among 35-year-old men in the background population. Excess MRRs/1000 person-years associated with smoking increased from 0.6 (95% CI –1.3 to 2.6) at age 35 to 43.6 (95% CI 37.9–49.3) at age at least 65 years.
CONCLUSION:Well treated HIV-infected individuals may lose more life years through smoking than through HIV. Excess mortality associated with smoking increases markedly with age. Therefore, increases in smoking-related mortality can be expected as the treated HIV-infected population ages. Interventions for smoking cessation should be prioritized.
PDF
