Although leisure-time physical activity (PA) has established health benefits in older adults, it is equivocal if exercising in environments with high levels of PM2.5 concentrations is equally beneficial for them. To explore the independent and joint associations of ambient PM2.5 and PA with all-cause mortality among adults aged 60 or older and to assess the modifying effect of age (60–74 years vs. 75+ years) on the joint associations. A prospective cohort study based on the MJ Cohort repeat examinations (2005–2016) and the Taiwan Air Quality Monitoring Network and death registry linkages (2005–2022). We included MJ Cohort participants aged 60 or more at baseline who attended the health check-ups at least twice (n = 21,760). Metabolic equivalent hours per week (MET-h/week) of leisure-time PA were computed. Multivariable adjusted associations were examined using time-varying Cox proportional hazard models. There were 3539 all-cause deaths over a mean follow-up of 12.81 (SD = 3.67) years. Ambient PM2.5 and physical inactivity are both independently associated with all-cause mortality. The joint associations of PA and PM2.5 concentrations with all-cause mortality differed in the young-old (60–74 years) and the older-old (75+ years) (P for interaction = 0.01); Higher levels of long-term PM2.5 exposures (≥25 μg/m3) had little influence on the associations between PA and mortality in the young-old (HR = 0.68 (0.56–0.83) and HR = 0.72 (0.59–0.88) for participants with 7.5–<15 and 15+ MET-h/week respectively) but eliminated associations between exposure and outcome in the older-old (HR = 0.91 (0.69–01.21) and HR = 1.02 (0.76–1.38) for participants with 7.5–<15 and 15+ MET-h/week). Long-term exposures to higher PM2.5 concentrations may eliminate the beneficial associations of PA with all-cause mortality among adults aged 75 and over. •Engaging in more physical activity (PA) and having a lower level of PM2.5 had a decreased mortality.•The joint association of PA and PM2.5 with mortality differed between young and older-old.•The benefit of PA for mortality was attenuated in adults (75+) with PM2.5 exposures (≥25 μg/m3).