Purpose In 10–24% of patients with rectal cancer who are treated with neoadjuvant chemoradiation, no residual tumor is found after surgery (ypT0). When accurately selected, these complete responders might be considered for less invasive treatments instead of standard surgery. So far, no imaging method has proven reliable. This study was designed to assess the accuracy of diffusion-weighted MRI (DWI) in addition to standard rectal MRI for selection of complete responders after chemoradiation. Methods A total of 120 patients with locally advanced rectal cancer from three university hospitals underwent chemoradiation followed by a restaging MRI (1.5T), consisting of standard T2W-MRI and DWI (b0-1000). Three independent readers first scored the standard MRI only for the likelihood of a complete response using a 5-point confidence score, after which the DWI images were added and the scoring was repeated. Histology (ypT0 vs. ypT1-4) was the standard reference. Diagnostic performance for selection of complete responders and interobserver agreement were compared for the two readings. Results Twenty-five of 120 patients had a complete response (ypT0). Areas under the ROC-curve for the three readers improved from 0.76, 0.68, and 0.58, using only standard MRI, to 0.8, 0.8, and 0.78 after addition of DWI ( P  = 0.39, 0.02, and 0.002). Sensitivity for selection of complete responders ranged from 0–40% on standard MRI versus 52–64% after addition of DWI. Specificity was equally high (89–98%) for both reading sessions. Interobserver agreement improved from κ 0.2–0.32 on standard MRI to 0.51–0.55 after addition of DWI. Conclusions Addition of DWI to standard rectal MRI improves the selection of complete responders after chemoradiation.