The increased risk of arterial thrombotic (ATE) after VTE, particularly when they are unprovoked or cancer-associated has been established. However, the risk factors of ATE after these VTE remain unclear. Using cause-specific hazard regression models, we determined risk factors of ATE (myocardial infarction, ischemic stroke, acute limb ischemia, digestive tract ischemia, or renal ischemia) in 2242 patients with unprovoked VTE and in 914 patients with cancer-associated VTE from a multi-center prospective cohort. Of patients with unprovoked-VTE, 174 developed ATE (7.8%, incidence: 1.26 per 100 patient-years) during follow-up (median: 68 months). Among patients with cancer-associated VTE, 57 developed ATE (6.2%, incidence: 1.98 per 100 patient-years) during follow-up (median: 30 months). After multivariable analysis, the identified risk factors of ATE in patients with unprovoked-VTE were age > 65 years (vs. <50 years, HR 2.59, 95% CI: 1.56–4.29), past history of symptomatic atherosclerosis (HR 2.11, 95% CI: 1.40–3.19), and treatment with low molecule weight heparin (vs. vitamin K antagonists, HR: 2.26, 95% CI: 1.13–4.52). In patients with cancer-associated VTE, the identified risk factors of ATE were: past history of symptomatic atherosclerosis (HR: 3.13, 95% CI: 1.72–5.67), and ongoing anticoagulation at the diagnosis of VTE (HR: 2.77, 95% CI: 1.07–7.22). The risk of ATE after unprovoked VTE and after cancer-associated VTE, is determined by some classic cardiovascular risk factors and appears to be influenced by anticoagulant treatment introduced for VTE, as well as the presence or absence of ongoing anticoagulation at the diagnosis of VTE.