Objectives To explore the value of multiparametric MRI combined with FDG-PET/CT to identify well-responding rectal cancer patients before the start of neoadjuvant chemoradiation. Methods Sixty-one locally advanced rectal cancer patients who underwent a baseline FDG-PET/CT and MRI (T2W + DWI) and received long-course neoadjuvant chemoradiotherapy were retrospectively analysed. Tumours were delineated on MRI and PET/CT from which the following quantitative parameters were calculated: T2W volume and entropy, ADC mean and entropy, CT density (mean-HU), SUV maximum and mean, metabolic tumour volume (MTV 42% ) and total lesion glycolysis (TLG). These features, together with sex, age, mrTN-stage (“baseline parameters”) and the CRT-surgery interval were analysed using multivariable stepwise logistic regression. Outcome was a good (TRG 1–2) versus poor histopathological response. Performance (AUC) to predict response was compared for different combinations of baseline ± quantitative imaging parameters and performance in an ‘independent’ dataset was estimated using bootstrapped leave-one-out cross-validation (LOOCV). Results The optimal multivariable prediction model consisted of a combination of baseline + quantitative imaging parameters and included mrT-stage (OR 0.004, p  < 0.001), T2W-signal entropy (OR 7.81, p  = 0.0079) and T2W volume (OR 1.028, p  = 0.0389) as the selected predictors. AUC in the study dataset was 0.88 and 0.83 after LOOCV. No PET/CT features were selected as predictors. Conclusions A multivariable model incorporating mrT-stage and quantitative parameters from baseline MRI can aid in identifying well-responding patients before the start of treatment. Addition of FDG-PET/CT is not beneficial. Key Points • A multivariable model incorporating the mrT-stage and quantitative features derived from baseline MRI can aid in identifying well-responding patients before the start of neoadjuvant chemoradiotherapy . • mrT-stage was the strongest predictor in the model and was complemented by the tumour volume and signal entropy calculated from T2W-MRI . • Adding quantitative features derived from pre-treatment PET/CT or DWI did not contribute to the model’s predictive performance .