Abstract Purpose To investigate which factors are related to patient reported cosmetic outcome (PRCO) after breast conserving therapy. Methods From 2004 to 2011, 2421 cT1-2N0-2a breast cancer patients were randomised in the Young Boost Trial between a 16 and a 26 Gy boost to the tumour bed. Cosmesis was scored subjectively by the patient and physician, and objectively using BCCT.core, at baseline, one and four years after treatment. Presence of fibrosis, QoL and rib pain at four years were also scored. Data were complete for 864 patients. The relation between the separate components was investigated using a proportional odds model. Results Of the 7 BCCT.core parameters, the distance from nipple to inframammary fold and the length of the breast contour were significantly related to the overall PRCO at four years. Patients with more fibrosis and poorer QoL scored their cosmesis worse, while rib pain was not related. The agreement between the different scores was low (kappa 0.26–0.42). Conclusion The distance from nipple to inframammary fold, the length of the breast contour and the severity of fibrosis were the main factors related to patient-reported cosmetic outcome. Patients with better QoL scored their cosmesis better.