In this paper we study approaches for dealing with treatment when developing a clinical prediction model. Analogous to the estimand framework recently proposed by the European Medicines Agency for ...clinical trials, we propose a ‘predictimand’ framework of different questions that may be of interest when predicting risk in relation to treatment started after baseline. We provide a formal definition of the estimands matching these questions, give examples of settings in which each is useful and discuss appropriate estimators including their assumptions. We illustrate the impact of the predictimand choice in a dataset of patients with end-stage kidney disease. We argue that clearly defining the estimand is equally important in prediction research as in causal inference.
Over the past few years, a large number of prediction models have been published, often of poor methodological quality. Seemingly objective and straightforward, prediction models provide a risk ...estimate for the outcome of interest, usually based on readily available clinical information. Yet, using models of substandard methodological rigour, especially without external validation, may result in incorrect risk estimates and consequently misclassification. To assess and combat bias in prediction research the prediction model risk of bias assessment tool (PROBAST) was published in 2019. This risk of bias (ROB) tool includes four domains and 20 signalling questions highlighting methodological flaws, and provides guidance in assessing the applicability of the model. In this paper, the PROBAST will be discussed, along with an in‐depth review of two commonly encountered pitfalls in prediction modelling that may induce bias: overfitting and composite endpoints. We illustrate the prevalence of potential bias in prediction models with a meta‐review of 50 systematic reviews that used the PROBAST to appraise their included studies, thus including 1510 different studies on 2104 prediction models. All domains showed an unclear or high ROB; these results were markedly stable over time, highlighting the urgent need for attention on bias in prediction research. This article aims to do just that by providing (1) the clinician with tools to evaluate the (methodological) quality of a clinical prediction model, (2) the researcher working on a review with methods to appraise the included models, and (3) the researcher developing a model with suggestions to improve model quality.
SUMMARY AT A GLANCE
Most published prediction models have limited clinical uptake, are not externally validated and come with methodological issues. The PROBAST (Prediction model Risk Of Bias ASssessment Tool) guides the researcher writing a review, or the clinician interested in a model for risk calculation in a clinical setting. This review examines the aspects of bias in prediction research, and provides information on the prevalence of bias in published models.
Abstract
Prognostic models that aim to improve the prediction of clinical events, individualized treatment and decision-making are increasingly being developed and published. However, relatively few ...models are externally validated and validation by independent researchers is rare. External validation is necessary to determine a prediction model’s reproducibility and generalizability to new and different patients. Various methodological considerations are important when assessing or designing an external validation study. In this article, an overview is provided of these considerations, starting with what external validation is, what types of external validation can be distinguished and why such studies are a crucial step towards the clinical implementation of accurate prediction models. Statistical analyses and interpretation of external validation results are reviewed in an intuitive manner and considerations for selecting an appropriate existing prediction model and external validation population are discussed. This study enables clinicians and researchers to gain a deeper understanding of how to interpret model validation results and how to translate these results to their own patient population.
Aim
Half of heart failure (HF) patients have chronic kidney disease (CKD) complicating their pharmacological management. We evaluated physicians' and patients' patterns of use of evidence‐based ...medical therapies in HF across CKD stages.
Methods and results
We studied HF patients with reduced (HFrEF) and mildly reduced (HFmrEF) ejection fraction enrolled in the Swedish Heart Failure Registry in 2009–2018. We investigated the likelihood of physicians to prescribe guideline‐recommended therapies to patients with CKD, and of patients to fill the prescriptions within 90 days of incident HF (initiating therapy), to adhere (proportion of days covered ≥80%) and persist (continued use) on these treatments during the first year of therapy. We identified 31 668 patients with HFrEF (median age 74 years, 46% CKD). The proportions receiving a prescription for angiotensin‐converting enzyme inhibitors/angiotensin receptor blockers/angiotensin receptor–neprilysin inhibitors (ACEi/ARB/ARNi) were 96%, 92%, 86%, and 68%, for estimated glomerular filtration rate (eGFR) ≥60, 45–59, 30–44, and <30 ml/min/1.73 m2, respectively; for beta‐blockers 94%, 93%, 92%, and 92%, for mineralocorticoid receptor antagonists (MRAs) 45%, 44%, 37%, 24%; and for triple therapy (combination of ACEi/ARB/ARNi + beta‐blockers + MRA) 38%, 35%, 28%, and 15%. Patients with CKD were less likely to initiate these medications, and less likely to adhere to and persist on ACEi/ARB/ARNi, MRA, and triple therapy. Among stoppers, CKD patients were less likely to restart these medications. Results were consistent after multivariable adjustment and in patients with HFmrEF (n = 15 114).
Conclusions
Patients with HF and CKD are less likely to be prescribed and to fill prescriptions for evidence‐based therapies, showing lower adherence and persistence, even at eGFR categories where these therapies are recommended and have shown efficacy in clinical trials.
Patients with heart failure and chronic kidney disease are less likely to receive and persist on guideline‐recommended medical therapies.
Abstract
The correlation coefficient is a statistical measure often used in studies to show an association between variables or to look at the agreement between two methods. In this paper, we will ...discuss not only the basics of the correlation coefficient, such as its assumptions and how it is interpreted, but also important limitations when using the correlation coefficient, such as its assumption of a linear association and its sensitivity to the range of observations. We will also discuss why the coefficient is invalid when used to assess agreement of two methods aiming to measure a certain value, and discuss better alternatives, such as the intraclass coefficient and Bland–Altman’s limits of agreement. The concepts discussed in this paper are supported with examples from literature in the field of nephrology.
Abstract
Background
Acute kidney injury (AKI) can affect hospitalized patients with coronavirus disease 2019 (COVID-19), with estimates ranging between 0.5% and 40%. We performed a systematic review ...and meta-analysis of studies reporting incidence, mortality and risk factors for AKI in hospitalized COVID-19 patients.
Methods
We systematically searched 11 electronic databases until 29 May 2020 for studies in English reporting original data on AKI and kidney replacement therapy (KRT) in hospitalized COVID-19 patients. Incidences of AKI and KRT and risk ratios for mortality associated with AKI were pooled using generalized linear mixed and random-effects models. Potential risk factors for AKI were assessed using meta-regression. Incidences were stratified by geographic location and disease severity.
Results
A total of 3042 articles were identified, of which 142 studies were included, with 49 048 hospitalized COVID-19 patients including 5152 AKI events. The risk of bias of included studies was generally low. The pooled incidence of AKI was 28.6% 95% confidence interval (CI) 19.8–39.5 among hospitalized COVID-19 patients from the USA and Europe (20 studies) and 5.5% (95% CI 4.1–7.4) among patients from China (62 studies), whereas the pooled incidence of KRT was 7.7% (95% CI 5.1–11.4; 18 studies) and 2.2% (95% CI 1.5–3.3; 52 studies), respectively. Among patients admitted to the intensive care unit, the incidence of KRT was 20.6% (95% CI 15.7–26.7; 38 studies). Meta-regression analyses showed that age, male sex, cardiovascular disease, diabetes mellitus, hypertension and chronic kidney disease were associated with the occurrence of AKI; in itself, AKI was associated with an increased risk of mortality, with a pooled risk ratio of 4.6 (95% CI 3.3–6.5).
Conclusions
AKI and KRT are common events in hospitalized COVID-19 patients, with estimates varying across geographic locations. Additional studies are needed to better understand the underlying mechanisms and optimal treatment of AKI in these patients.
Etiological research aims to uncover causal effects, whilst prediction research aims to forecast an outcome with the best accuracy. Causal and prediction research usually require different methods, ...and yet their findings may get conflated when reported and interpreted. The aim of the current study is to quantify the frequency of conflation between etiological and prediction research, to discuss common underlying mistakes and provide recommendations on how to avoid these. Observational cohort studies published in January 2018 in the top-ranked journals of six distinct medical fields (Cardiology, Clinical Epidemiology, Clinical Neurology, General and Internal Medicine, Nephrology and Surgery) were included for the current scoping review. Data on conflation was extracted through signaling questions. In total, 180 studies were included. Overall, 26% (n = 46) contained conflation between etiology and prediction. The frequency of conflation varied across medical field and journal impact factor. From the causal studies 22% was conflated, mainly due to the selection of covariates based on their ability to predict without taking the causal structure into account. Within prediction studies 38% was conflated, the most frequent reason was a causal interpretation of covariates included in a prediction model. Conflation of etiology and prediction is a common methodological error in observational medical research and more frequent in prediction studies. As this may lead to biased estimations and erroneous conclusions, researchers must be careful when designing, interpreting and disseminating their research to ensure this conflation is avoided.
Skiing and snowboarding are both popular recreational alpine sports, with substantial injury risk of variable severity. Although skills level has repeatedly been associated with injury risk, a ...validated measure to accurately estimate the actual skills level without objective assessment is missing. This study aimed to develop a practical validated instrument, to better estimate the actual skills level of recreational skiers, based on the criteria of the Dutch Skiing Federation (DSF), and covering five different skill domains. A sample of Dutch recreational skiers (n = 84) was asked to fill in a questionnaire reflecting seven, a priori chosen predictors by expert opinion, to ski downhill and to be objectively evaluated by expert assessors. The instrument was developed to have a multidimensional character and was validated according to the TRIPOD guideline (Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis). The sample reported an overall incorrect self‐reported estimation of their skills, compared with the observed skill score. The instrument showed good calibration and underwent multiple validation methods. The estimated skills score showed to be closer to the observed scores, than self‐reportage. Our study provides a practical, multidimensional, and validated instrument to estimate the actual skills level. It proved to better reflect the actual skills levels compared with self‐reportage among recreational skiers.
The authors propose a dynamic direct mailing response model with competitive effects. Purchase and promotion history are incorporated to map the dynamic competitive interactions among the firms ...sending the mailings. The authors investigate the impact of direct mailings on the revenues of each firm and its competitors over time. The model accounts for endogeneity of the mailing decision and for unobserved heterogeneity across households. The model is considered in a charitable giving setting, in which households often receive many direct mailings of different charities within a short period and competition is strong. The authors construct a unique database by merging the databases of three large charity organizations in the Netherlands. This results in household-level data on the direct mailings households received from and their donations to each of the three charities. The results show that a charity's own mailings are short-term substitutes; that is, an extra mailing cannibalizes the revenues of subsequent mailings. Furthermore, competitive charitable direct mailings tend to be short-term complements; that is, the direct mailings increase the total pie that is divided among the charities. In the long run, these effects die out. The results are also interpreted from a behavioral perspective.
AbstractObjectiveTo identify the optimal estimated glomerular filtration rate (eGFR) at which to initiate dialysis in people with advanced chronic kidney disease.DesignNationwide observational cohort ...study.SettingNational Swedish Renal Registry of patients referred to nephrologists.ParticipantsPatients had a baseline eGFR between 10 and 20 mL/min/1.73 m2 and were included between 1 January 2007 and 31 December 2016, with follow-up until 1 June 2017.Main outcome measuresThe strict design criteria of a clinical trial were mimicked by using the cloning, censoring, and weighting method to eliminate immortal time bias, lead time bias, and survivor bias. A dynamic marginal structural model was used to estimate adjusted hazard ratios and absolute risks for five year all cause mortality and major adverse cardiovascular events (composite of cardiovascular death, non-fatal myocardial infarction, or non-fatal stroke) for 15 dialysis initiation strategies with eGFR values between 4 and 19 mL/min/1.73 m2 in increments of 1 mL/min/1.73 m2. An eGFR between 6 and 7 mL/min/1.73 m2 (eGFR6-7) was taken as the reference.ResultsAmong 10 290 incident patients with advanced chronic kidney disease (median age 73 years; 3739 (36%) women; median eGFR 16.8 mL/min/1.73 m2), 3822 started dialysis, 4160 died, and 2446 had a major adverse cardiovascular event. A parabolic relation was observed for mortality, with the lowest risk for eGFR15-16. Compared with dialysis initiation at eGFR6-7, initiation at eGFR15-16 was associated with a 5.1% (95% confidence interval 2.5% to 6.9%) lower absolute five year mortality risk and 2.9% (0.2% to 5.5%) lower risk of a major adverse cardiovascular event, corresponding to hazard ratios of 0.89 (95% confidence interval 0.87 to 0.92) and 0.94 (0.91 to 0.98), respectively. This 5.1% absolute risk difference corresponded to a mean postponement of death of 1.6 months over five years of follow-up. However, dialysis would need to be started four years earlier. When emulating the intended strategies of the Initiating Dialysis Early and Late (IDEAL) trial (eGFR10-14v eGFR5-7) and the achieved eGFRs in IDEAL (eGFR7-10v eGFR5-7), hazard ratios for all cause mortality were 0.96 (0.94 to 0.99) and 0.97 (0.94 to 1.00), respectively, which are congruent with the findings of the randomised IDEAL trial.ConclusionsVery early initiation of dialysis was associated with a modest reduction in mortality and cardiovascular events. For most patients, such a reduction may not outweigh the burden of a substantially longer period spent on dialysis.
PDF
