Decision curve analysis is a novel method for evaluating diagnostic tests, prediction models and molecular markers. It combines the mathematical simplicity of accuracy measures, such as sensitivity ...and specificity, with the clinical applicability of decision analytic approaches. Most critically, decision curve analysis can be applied directly to a data set, and does not require the sort of external data on costs, benefits and preferences typically required by traditional decision analytic techniques.
In this paper we present several extensions to decision curve analysis including correction for overfit, confidence intervals, application to censored data (including competing risk) and calculation of decision curves directly from predicted probabilities. All of these extensions are based on straightforward methods that have previously been described in the literature for application to analogous statistical techniques.
Simulation studies showed that repeated 10-fold crossvalidation provided the best method for correcting a decision curve for overfit. The method for applying decision curves to censored data had little bias and coverage was excellent; for competing risk, decision curves were appropriately affected by the incidence of the competing risk and the association between the competing risk and the predictor of interest. Calculation of decision curves directly from predicted probabilities led to a smoothing of the decision curve.
Decision curve analysis can be easily extended to many of the applications common to performance measures for prediction models. Software to implement decision curve analysis is provided.
We have observed that the area under the receiver operating characteristic curve (AUC) is increasingly being used to evaluate whether a novel predictor should be incorporated in a multivariable model ...to predict risk of disease. Frequently, investigators will approach the issue in two distinct stages: first, by testing whether the new predictor variable is significant in a multivariable regression model; second, by testing differences between the AUC of models with and without the predictor using the same data from which the predictive models were derived. These two steps often lead to discordant conclusions.
We conducted a simulation study in which two predictors, X and X*, were generated as standard normal variables with varying levels of predictive strength, represented by means that differed depending on the binary outcome Y. The data sets were analyzed using logistic regression, and likelihood ratio and Wald tests for the incremental contribution of X* were performed. The patient-specific predictors for each of the models were then used as data for a test comparing the two AUCs. Under the null, the size of the likelihood ratio and Wald tests were close to nominal, but the area test was extremely conservative, with test sizes less than 0.006 for all configurations studied. Where X* was associated with outcome, the area test had much lower power than the likelihood ratio and Wald tests.
Evaluation of the statistical significance of a new predictor when there are existing clinical predictors is most appropriately accomplished in the context of a regression model. Although comparison of AUCs is a conceptually equivalent approach to the likelihood ratio and Wald test, it has vastly inferior statistical properties. Use of both approaches will frequently lead to inconsistent conclusions. Nonetheless, comparison of receiver operating characteristic curves remains a useful descriptive tool for initial evaluation of whether a new predictor might be of clinical relevance.
Cervical cancer outcomes remain poor among disadvantaged populations, including ethnic minorities, low-income, and underinsured women. The aim of this study was to evaluate the mechanisms that ...underlie the observed association between race/ethnicity and cervical cancer survival. We identified 13,698 women, ages 21 to 64 years, diagnosed with stages I-III primary cervical cancer between 2007-2013 in Surveillance, Epidemiology, and End Results (SEER). Multivariable Cox proportional hazards regression models evaluated associations between race/ethnicity (Non-Hispanic White, Non-Hispanic Black, Hispanic, Other) and cervical cancer-specific mortality. We conducted mediation analysis to calculate the mediation proportion and its 95% confidence interval. Non-Hispanic black women had an increased risk of cervical cancer-specific mortality (HR: 1.23, 95% CI: 1.08-1.39), and Hispanic women a decreased risk of dying from their disease (HR: 0.82, 95% CI: 0.72-0.93), compared with non-Hispanic white. The estimated proportion of excess cervical cancer mortality for non-Hispanic black women relative to non-Hispanic white women that was mediated by insurance was 18.6% and by treatment was 47.2%. Furthermore, non-Hispanic black women were more likely to receive radiation and less likely to receive surgery for early-stage disease. In this population-based study we found that some of the excess cervical cancer-specific mortality for non-Hispanic black women is mediated by factors such as insurance status and treatment. These findings suggest that enhancing existing insurance coverage and ensuring equal and adequate treatment in all women may be a key strategy for improving cervical cancer outcomes.
Chemotherapy for metastatic lung or colorectal cancer can prolong life by weeks or months and may provide palliation, but it is not curative.
We studied 1193 patients participating in the Cancer Care ...Outcomes Research and Surveillance (CanCORS) study (a national, prospective, observational cohort study) who were alive 4 months after diagnosis and received chemotherapy for newly diagnosed metastatic (stage IV) lung or colorectal cancer. We sought to characterize the prevalence of the expectation that chemotherapy might be curative and to identify the clinical, sociodemographic, and health-system factors associated with this expectation. Data were obtained from a patient survey by professional interviewers in addition to a comprehensive review of medical records.
Overall, 69% of patients with lung cancer and 81% of those with colorectal cancer did not report understanding that chemotherapy was not at all likely to cure their cancer. In multivariable logistic regression, the risk of reporting inaccurate beliefs about chemotherapy was higher among patients with colorectal cancer, as compared with those with lung cancer (odds ratio, 1.75; 95% confidence interval CI, 1.29 to 2.37); among nonwhite and Hispanic patients, as compared with non-Hispanic white patients (odds ratio for Hispanic patients, 2.82; 95% CI, 1.51 to 5.27; odds ratio for black patients, 2.93; 95% CI, 1.80 to 4.78); and among patients who rated their communication with their physician very favorably, as compared with less favorably (odds ratio for highest third vs. lowest third, 1.90; 95% CI, 1.33 to 2.72). Educational level, functional status, and the patient's role in decision making were not associated with such inaccurate beliefs about chemotherapy.
Many patients receiving chemotherapy for incurable cancers may not understand that chemotherapy is unlikely to be curative, which could compromise their ability to make informed treatment decisions that are consonant with their preferences. Physicians may be able to improve patients' understanding, but this may come at the cost of patients' satisfaction with them. (Funded by the National Cancer Institute and others.).
We assessed the effect of radical prostatectomy (RP) and external beam radiotherapy (EBRT) on distant metastases (DM) rates in patients with localized prostate cancer treated with RP or EBRT at a ...single specialized cancer center.
Patients with clinical stages T1c-T3b prostate cancer were treated with intensity-modulated EBRT (> or = 81 Gy) or RP. Both cohorts included patients treated with salvage radiotherapy or androgen-deprivation therapy for biochemical failure. Salvage therapy for patients with RP was delivered a median of 13 months after biochemical failure compared with 69 months for EBRT patients. DM was compared controlling for patient age, clinical stage, serum prostate-specific antigen level, biopsy Gleason score, and year of treatment.
The 8-year probability of freedom from metastatic progression was 97% for RP patients and 93% for EBRT patients. After adjustment for case mix, surgery was associated with a reduced risk of metastasis (hazard ratio, 0.35; 95% CI, 0.19 to 0.65; P < .001). Results were similar for prostate cancer-specific mortality (hazard ratio, 0.32; 95% CI, 0.13 to 0.80; P = .015). Rates of metastatic progression were similar for favorable-risk disease (1.9% difference in 8-year metastasis-free survival), somewhat reduced for intermediate-risk disease (3.3%), and more substantially reduced in unfavorable-risk disease (7.8% in 8-year metastatic progression).
Metastatic progression is infrequent in men with low-risk prostate cancer treated with either RP or EBRT. RP patients with higher-risk disease treated had a lower risk of metastatic progression and prostate cancer-specific death than EBRT patients. These results may be confounded by differences in the use and timing of salvage therapy.
In randomized clinical trials where time‐to‐event is the primary outcome, almost routinely, the logrank test is prespecified as the primary test and the hazard ratio is used to quantify treatment ...effect. If the ratio of 2 hazard functions is not constant, the logrank test is not optimal and the interpretation of hazard ratio is not obvious. When such a nonproportional hazards case is expected at the design stage, the conventional practice is to prespecify another member of weighted logrank tests, eg, Peto‐Prentice‐Wilcoxon test. Alternatively, one may specify a robust test as the primary test, which can capture various patterns of difference between 2 event time distributions. However, most of those tests do not have companion procedures to quantify the treatment difference, and investigators have fallen back on reporting treatment effect estimates not associated with the primary test. Such incoherence in the “test/estimation” procedure may potentially mislead clinicians/patients who have to balance risk‐benefit for treatment decision. To address this, we propose a flexible and coherent test/estimation procedure based on restricted mean survival time, where the truncation time τ is selected data dependently. The proposed procedure is composed of a prespecified test and an estimation of corresponding robust and interpretable quantitative treatment effect. The utility of the new procedure is demonstrated by numerical studies based on 2 randomized cancer clinical trials; the test is dramatically more powerful than the logrank, Wilcoxon tests, and the restricted mean survival time–based test with a fixed τ, for the patterns of difference seen in these cancer clinical trials.
National guidelines recommend that discussions about end-of-life (EOL) care planning happen early for patients with incurable cancer. We do not know whether earlier EOL discussions lead to less ...aggressive care near death. We sought to evaluate the extent to which EOL discussion characteristics, such as timing, involved providers, and location, are associated with the aggressiveness of care received near death.
We studied 1,231 patients with stage IV lung or colorectal cancer in the Cancer Care Outcomes Research and Surveillance Consortium, a population- and health system-based prospective cohort study, who died during the 15-month study period but survived at least 1 month. Our main outcome measure was the aggressiveness of EOL care received.
Nearly half of patients received at least one marker of aggressive EOL care, including chemotherapy in the last 14 days of life (16%), intensive care unit care in the last 30 days of life (9%), and acute hospital-based care in the last 30 days of life (40%). Patients who had EOL discussions with their physicians before the last 30 days of life were less likely to receive aggressive measures at EOL, including chemotherapy (P = .003), acute care (P < .001), or any aggressive care (P < .001). Such patients were also more likely to receive hospice care (P < .001) and to have hospice initiated earlier (P < .001).
Early EOL discussions are prospectively associated with less aggressive care and greater use of hospice at EOL.
PDF
