Background Prognostic uncertainty is one barrier to engaging in goals-of-care discussions in chronic kidney disease (CKD). The surprise question (“Would you be surprised if this patient died in the next 12 months?”) is a tool to assist in prognostication. However, it has not been studied in non−dialysis-dependent CKD and its reliability is unknown. Study Design Observational study. Setting & Participants 388 patients at least 60 years of age with non−dialysis-dependent CKD stages 4 to 5 who were seen at an outpatient nephrology clinic. Predictor Trinary (ie, Yes, Neutral, or No) and binary (Yes or No) surprise question response. Outcomes Mortality, test-retest reliability, and blinded inter-rater reliability. Measurements Baseline comorbid conditions, Charlson Comorbidity Index, cause of CKD, and baseline laboratory values (ie, serum creatinine/estimated glomerular filtration rate, serum albumin, and hemoglobin). Results Median patient age was 71 years with median follow-up of 1.4 years, during which time 52 (13%) patients died. Using the trinary surprise question, providers responded Yes, Neutral, and No for 202 (52%), 80 (21%), and 106 (27%) patients, respectively. About 5%, 15%, and 27% of Yes, Neutral, and No patients died, respectively ( P < 0.001). Trinary surprise question inter-rater reliability was 0.58 (95% CI, 0.42-0.72), and test-retest reliability was 0.63 (95% CI, 0.54–0.72). The trinary surprise question No response had sensitivity and specificity of 55% and 76%, respectively (95% CIs, 38%-71% and 71%-80%, respectively). The binary surprise question had sensitivity of 66% (95% CI, 49%-80%; P = 0.3 vs trinary), but lower specificity of 68% (95% CI, 63%-73%; P = 0.02 vs trinary). Limitations Single center, small number of deaths. Conclusions The surprise question associates with mortality in CKD stages 4 to 5 and demonstrates moderate to good reliability. Future studies should examine how best to deploy the surprise question to facilitate advance care planning in advanced non−dialysis-dependent CKD.