Most patients with localized breast cancer (LBC) who take adjuvant chemotherapy (CT) complain of fatigue and a decrease in quality of life during or after radiotherapy (RT). The aim of this longitudinal study was to compare the impact of RT alone with that occurring after previous CT on quality of life. Fatigue (the main endpoint) and cognitive impairment were assessed in 161 CT-RT and 141 RT patients during RT and 1 year later. Fatigue was assessed with Functional Assessment of Cancer Therapy-General questionnaires, including breast and fatigue modules. At baseline, 60% of the CT-RT patients expressed fatigue vs. 33% of the RT patients (p <0.001). Corresponding values at the end of RT were statistically similar (61% and 53%), and fatigue was still reported at 1 year by more than 40% of patients in both groups. Risk factors for long-term fatigue included depression (odds ratio OR = 6), which was less frequent in the RT group at baseline (16% vs. 28 %, respectively, p = 0.01) but reached a similar value at the end of RT (25% in both groups). Initial mild cognitive impairments were reported by RT (34 %) patients and CT-RT (24 %) patients and were persistent at 1 year for half of them. No biological disorders were associated with fatigue or cognitive impairment. Fatigue was the main symptom in LBC patients treated with RT, whether they received CT previously or not. The correlation of persistent fatigue with initial depressive status favors administering medical and psychological programs for LBC patients treated with CT and/or RT, to identify and manage this main quality-of-life-related symptom.