Background Silent brain infarcts (SBI), a finding on neuroimaging, are associated with higher risk of future stroke. Atrial Fibrillation (AF) has been previously identified as a cause of SBI. Objectives The aim of this study is to determine the prevalence of and risk factors for SBI in patients with AF and low-to-moderate embolic risk according to CHADS 2 and CHA 2 DS 2 VASc score. Methods Patients with a history of AF based on medical records who scored 0–1 in the CHADS 2 score were selected from the Seville urban area using the Andalusian electronic healthcare database (DIRAYA). Demographic and clinical data were collected and a 3T brain MRI was performed on patients older than 50 years and with absence of neurological symptoms. Results 66 of the initial 443 patients (14.9%) and 41 of the 349 patients with low risk according to CHA 2 DS 2 VASc score (11.7%) presented at least 1 SBI. After adjusted multivariable analysis, an older age (OR 3.84, 95% CI 1.07–13.76) and left atrial (LA) enlargement (OR 3.13, 95% CI 1.15–8.55) were associated with SBI in the whole cohort, while only LA enlargement was associated with SBI in the low-risk cohort (OR 3.19, 95% CI 1.33–7.63). Conclusions LA enlargement on echocardiogram was associated with SBI in patients with AF and low or moderate embolic risk according to CHADS 2 and in the low-risk population according to CHA 2 DS 2 VASc. Although further studies are needed, a neuroimaging screening might be justified in these patients to guide medical therapies to improve stroke prevention.