Objectives This study evaluates the risk of cardiovascular disease (CVD) following breast cancer, accounting for baseline CVD risk. Methods Within the EPIC-NL (Dutch part of the European Prospective Investigation into Nutrition and Cancer) cohort, 1103 women were diagnosed with breast cancer. For every breast cancer patient, 3–4 women without breast cancer ( n  = 4328) were selected matched for age, year, and time since cohort enrollment. Based on CVD risk factors at cohort enrollment, 10-year risk of CVD was calculated and categorized: low (< 10%), intermediate (10–20%), high (> 20%). Cox proportional hazard models assessed the risk of CVD events (hospitalization or mortality) and CVD mortality of women with versus without breast cancer, adjusted for baseline CVD risk. Results After median follow-up of 5 and 6 years, 92 (8.3%) and 325 (7.5%) CVD events occurred in women with and without breast cancer, respectively. In the low CVD risk group, women with breast cancer had 1.44 (95% CI 1.00–2.06) times higher risk of CVD events than women without breast cancer. In the intermediate and high CVD risk categories, risk of CVD events was similar in women with and without breast cancer. Overall, women with breast cancer had 1.77 (95% CI 1.10–2.86) times higher risk of CVD mortality than women without breast cancer. Conclusions Among women with low CVD risk, women with breast cancer have a higher risk of CVD event than women without breast cancer. Overall, women with breast cancer have a higher risk of CVD mortality than women without breast cancer.