Background Evidence for the association of atrial fibrillation (AF) present on the ECG and cardiovascular outcomes in AF patients is limited. Objective To investigate the prognostic significance of AF on a single surface ECG for cardiovascular outcomes in AF patients. Methods A total of 3642 AF patients were prospectively enrolled. Main exclusion criteria were rhythms other than sinus rhythm (SR) or AF. The primary end‐point was a composite of all‐cause death and hospitalizations for congestive heart failure (CHF). Secondary end‐points were all‐cause death, CHF hospitalizations, cardiovascular death, myocardial infarction, any stroke and stroke subtypes. Associations were assessed with multivariable Cox proportional hazards models. Results Mean age was 71 years, 28% were female, and mean follow‐up was 3.4 years. Patients with SR on the ECG at study enrolment (56%) were younger (69 vs. 74 years, P < 0.0001), had more often paroxysmal AF (73 vs. 18%, P < 0.0001) and fewer comorbidities. The incidence of the primary end‐point was 1.8 and 3.1 per 100 person‐years in patients with SR and AF, respectively. The multivariable‐adjusted hazard ratio was 1.4 (95% confidence intervals 1.1; 1.7; P = 0.001) for patients with AF on the ECG compared to patients with SR. The hazard ratios (95% confidence intervals) were 1.4 (1.1; 1.8; P = 0.006) for all‐cause death, 1.5 (1.2; 1.9; P = 0.001) for CHF and 1.6 (1.1; 2.2; P = 0.006) for cardiovascular death. None of the other associations were statistically significant. Conclusions The presence of AF in a single office ECG had significant prognostic implications with regard to mortality and CHF hospitalizations in patients with AF. These patients present a high‐risk group and might benefit from intensified treatment.