Purpose Rectal toxicity remains a major threat to quality of life of patients, who receive brachytherapy to the abdominal pelvic area. Estimating the risk of toxicity development is essential to maximize therapeutic benefit without impairing rectal function. This study aimed to abstract and evaluate studies, which have developed prediction models for rectal toxicity after brachytherapy (BT) in patients with pelvic cancers. Material and methods To identify relevant studies since 1995, MEDLINE database was searched on August 31, 2021, using terms related to “pelvic cancers”, “brachytherapy”, “prediction models”, and “rectal toxicity”. Papers were excluded if model specifications were not reported. Risk of bias was assessed using prediction model risk of bias assessment tool. Results Thirty models (n = 16 cervical cancer, n = 13 prostate cancer, and n = 1 rectal cancer), including 60 distinct predictors were published. Rectal toxicity varied significantly between studies (median, 25.4% for cervix, and median, 8.8% for prostate cancer). High-, low-, and pulsed-dose-rate BT were applied in 15 (50%), 13 (43%), and 1 (3%) studies, respectively. Most common predictors that retained in final models were age (n = 5, 17%), EBRT (n = 5, 17%), V100% rectum (BT) (n = 5, 17%), and dose at rectal point (n = 3, 10%). None of the studies were considered to be at low-risk of bias due to deficiencies in the analysis domain. Conclusions Existing models have limited clinical application due to poor quality of methodology. The following key issues should be considered in future studies: 1) Measuring patient-reported outcomes to address underestimation of true frequencies of rectal toxicity events; 2) Giving higher priority to reliable dose-volume parameters; 3) Avoiding overfitting by considering an event per candidate predictor rate ≥ 20; 4) Calculating detailed performance measures.