Abstract
Prognostic models that aim to improve the prediction of clinical events, individualized treatment and decision-making are increasingly being developed and published. However, relatively few ...models are externally validated and validation by independent researchers is rare. External validation is necessary to determine a prediction model’s reproducibility and generalizability to new and different patients. Various methodological considerations are important when assessing or designing an external validation study. In this article, an overview is provided of these considerations, starting with what external validation is, what types of external validation can be distinguished and why such studies are a crucial step towards the clinical implementation of accurate prediction models. Statistical analyses and interpretation of external validation results are reviewed in an intuitive manner and considerations for selecting an appropriate existing prediction model and external validation population are discussed. This study enables clinicians and researchers to gain a deeper understanding of how to interpret model validation results and how to translate these results to their own patient population.
In clinical epidemiology, experimental studies usually take the form of randomized controlled clinical trials (RCTs). The data analysis of an RCT can be performed by using two complementary ...strategies, that is according to the intention to treat (ITT) principle and the per protocol (PP) analysis. By using the ITT approach, investigators aim to assess the effect of assigning a drug whereas by adopting the PP analysis, researchers investigate the effect of receiving the assigned treatment, as specified in the protocol. Both ITT and PP analyses are essentially valid but they have different scopes and interpretations dependent on the context.
SUMMARY AT A GLANCE
RCTs are analyzed according to the intention to treat (ITT) principle and the per protocol (PP) approach. The authors explained how ITT aims to assess the effect of assigning a drug and how the PP analysis investigates the effect of receiving an assigned treatment. Both analyses are valid but have different scopes and interpretations depending on the context.
ABSTRACT
In this article we introduce the concept of inverse probability of treatment weighting (IPTW) and describe how this method can be applied to adjust for measured confounding in observational ...research, illustrated by a clinical example from nephrology. IPTW involves two main steps. First, the probability—or propensity—of being exposed to the risk factor or intervention of interest is calculated, given an individual’s characteristics (i.e. propensity score). Second, weights are calculated as the inverse of the propensity score. The application of these weights to the study population creates a pseudopopulation in which confounders are equally distributed across exposed and unexposed groups. We also elaborate on how weighting can be applied in longitudinal studies to deal with informative censoring and time-dependent confounding in the setting of treatment-confounder feedback.
Since confounding obscures the real effect of the exposure, it is important to adequately address confounding for making valid causal inferences from observational data. Directed acyclic graphs ...(DAGs) are visual representations of causal assumptions that are increasingly used in modern epidemiology. They can help to identify the presence of confounding for the causal question at hand. This structured approach serves as a visual aid in the scientific discussion by making underlying relations explicit. This article explains the basic concepts of DAGs and provides examples in the field of nephrology with and without presence of confounding. Ultimately, these examples will show that DAGs can be preferable to the traditional methods to identify sources of confounding, especially in complex research questions.
Prediction tools that identify chronic kidney disease (CKD) patients at a high risk of developing kidney failure have the potential for great clinical value, but limited uptake. The aim of the ...current study is to systematically review all available models predicting kidney failure in CKD patients, organize empirical evidence on their validity and ultimately provide guidance in the interpretation and uptake of these tools.
PubMed and EMBASE were searched for relevant articles. Titles, abstracts and full-text articles were sequentially screened for inclusion by two independent researchers. Data on study design, model development and performance were extracted. The risk of bias and clinical usefulness were assessed and combined in order to provide recommendations on which models to use.
Of 2183 screened studies, a total of 42 studies were included in the current review. Most studies showed high discriminatory capacity and the included predictors had large overlap. Overall, the risk of bias was high. Slightly less than half the studies (48%) presented enough detail for the use of their prediction tool in practice and few models were externally validated.
The current systematic review may be used as a tool to select the most appropriate and robust prognostic model for various settings. Although some models showed great potential, many lacked clinical relevance due to being developed in a prevalent patient population with a wide range of disease severity. Future research efforts should focus on external validation and impact assessment in clinically relevant patient populations.
Older patients reaching ESRD have a higher risk of adverse health outcomes. We aimed to determine the association of functional and cognitive impairment and frailty with adverse health outcomes in ...patients reaching ESRD. Understanding these associations could ultimately lead to prediction models to guide tailored treatment decisions or preventive interventions.
We searched MEDLINE, Embase, Web of Science, CENTRAL, CINAHL, PsycINFO, and COCHRANE for original studies published until February 8, 2016 reporting on the association of functional or cognitive impairment or frailty with adverse health outcome after follow-up in patients reaching ESRD either with or without RRT.
Of 7451 identified citations, we included 30 articles that reported on 35 associations. Mean age was >60 years old in 73% of the studies, and geriatric conditions were highly prevalent. Twenty-four studies (80%) reported on functional impairment, seven (23%) reported on cognitive impairment, and four (13%) reported on frailty. Mortality was the main outcome measure in 29 studies (97%), and one study assessed functional status trajectory. In 34 of 35 (97%) associations reported, functional or cognitive impairment or frailty was significantly and independently associated with adverse health outcomes. The majority of studies (83%) were conducted in selected patient populations, mainly patients on incident dialysis.
Functional and cognitive impairment and frailty in patients reaching ESRD are highly prevalent and strongly and independently associated with adverse health outcomes, and they may, therefore, be useful for risk stratification. More research into their prognostic value is needed.
Mortality prediction is critical on long-term kidney replacement therapy (KRT), both for individual treatment decisions and resource planning. Many mortality prediction models already exist, but as a ...major shortcoming most of them have only been validated internally. This leaves reliability and usefulness of these models in other KRT populations, especially foreign, unknown. Previously two models were constructed for one- and two-year mortality prediction of Finnish patients starting long-term dialysis. These models are here internationally validated in KRT populations of the Dutch NECOSAD Study and the UK Renal Registry (UKRR).
We validated the models externally on 2051 NECOSAD patients and on two UKRR patient cohorts (5328 and 45493 patients). We performed multiple imputation for missing data, used c-statistic (AUC) to assess discrimination, and evaluated calibration by plotting average estimated probability of death against observed risk of death.
Both prediction models performed well in the NECOSAD population (AUC 0.79 for the one-year model and 0.78 for the two-year model). In the UKRR populations, performance was slightly weaker (AUCs: 0.73 and 0.74). These are to be compared to the earlier external validation in a Finnish cohort (AUCs: 0.77 and 0.74). In all tested populations, our models performed better for PD than HD patients. Level of death risk (i.e., calibration) was well estimated by the one-year model in all cohorts but was somewhat overestimated by the two-year model.
Our prediction models showed good performance not only in the Finnish but in foreign KRT populations as well. Compared to the other existing models, the current models have equal or better performance and fewer variables, thus increasing models' usability. The models are easily accessible on the web. These results encourage implementing the models into clinical decision-making widely among European KRT populations.
It is unknown whether stopping renin-angiotensin system (RAS) inhibitor therapy in patients with advanced CKD affects outcomes.
We studied patients referred to nephrologist care, listed on the ...Swedish Renal Registry during 2007-2017, who developed advanced CKD (eGFR<30 ml/min per 1.73 m
) while on RAS inhibitor therapy. Using target trial emulation techniques on the basis of cloning, censoring, and weighting, we compared the risks of stopping within 6 months and remaining off treatment versus continuing RAS inhibitor therapy. These included risks of subsequent 5-year all-cause mortality, major adverse cardiovascular events, and initiation of kidney replacement therapy (KRT).
Of 10,254 prevalent RAS inhibitor users (median age 72 years, 36% female) with new-onset eGFR <30 ml/min per 1.73 m
, 1553 (15%) stopped RAS inhibitor therapy within 6 months. Median eGFR was 23 ml/min per 1.73 m
. Compared with continuing RAS inhibition, stopping this therapy was associated with a higher absolute 5-year risk of death (40.9% versus 54.5%) and major adverse cardiovascular events (47.6% versus 59.5%), but with a lower risk of KRT (36.1% versus 27.9%); these corresponded to absolute risk differences of 13.6 events per 100 patients, 11.9 events per 100 patients, and -8.3 events per 100 patients, respectively. Results were consistent whether patients stopped RAS inhibition at higher or lower eGFR, across prespecified subgroups, after adjustment and stratification for albuminuria and potassium, and when modeling RAS inhibition as a time-dependent exposure using a marginal structural model.
In this nationwide observational study of people with advanced CKD, stopping RAS inhibition was associated with higher absolute risks of mortality and major adverse cardiovascular events, but also with a lower absolute risk of initiating KRT.
Previous studies have suggested that living kidney donors maintain long-term renal function and experience no increase in cardiovascular or all-cause mortality. However, most analyses have included ...control groups less healthy than the living donor population and have had relatively short follow-up periods. Here we compared long-term renal function and cardiovascular and all-cause mortality in living kidney donors compared with a control group of individuals who would have been eligible for donation. All-cause mortality, cardiovascular mortality, and end-stage renal disease (ESRD) was identified in 1901 individuals who donated a kidney during 1963 through 2007 with a median follow-up of 15.1 years. A control group of 32,621 potentially eligible kidney donors was selected, with a median follow-up of 24.9 years. Hazard ratio for all-cause death was significantly increased to 1.30 (95% confidence interval 1.11–1.52) for donors compared with controls. There was a significant corresponding increase in cardiovascular death to 1.40 (1.03–1.91), while the risk of ESRD was greatly and significantly increased to 11.38 (4.37–29.6). The overall incidence of ESRD among donors was 302 cases per million and might have been influenced by hereditary factors. Immunological renal disease was the cause of ESRD in the donors. Thus, kidney donors are at increased long-term risk for ESRD, cardiovascular, and all-cause mortality compared with a control group of non-donors who would have been eligible for donation.
Survival analyses are commonly applied to study death or other events of interest. In such analyses, so-called competing risks may form an important problem. A competing risk is an event that either ...hinders the observation of the event of interest or modifies the chance that this event occurs. For example, when studying death on dialysis, receiving a kidney transplant is an event that competes with the event of interest. Conventional methods for survival analysis ignoring the competing event(s), such as the Kaplan-Meier method and standard Cox proportional hazards regression, may be inappropriate in the presence of competing risks, and alternative methods specifically designed for analysing competing risks data should then be applied. This problem deserves more attention in nephrology research and in the current article, we therefore explain the problem of competing risks in survival analysis and how using different techniques may affect study results.
PDF
