Background Increased left atrium diameter (LAD) is associated with an elevated risk of cardiovascular diseases. The relationship between nutrition status and left atrial enlargement (LAE) is still unclear. The present study aimed to investigate the association of famine exposure in early life with LAE in adulthood. Methods Participants were divided into non‐exposed, fetal, early, middle and late childhood exposed groups according to birth data. LAE was defined when LAD was ≥3.9 cm in women and ≥4.1 cm in men, or ≥2.3 cm m−2 by a sex‐independent cut‐off normalised for body surface area. Multivariate logistic regression was performed to calculate the odds ratio (OR) and confidence interval (CI) between famine exposure and LAE. Results In total, 2522 905 male, mean (SD) age 59.1 (3.65) years subjects were enrolled, including 392 (15.5%) LAE subjects. The prevalence of LAE in non‐exposed, fetal, early, middle and late childhood exposed groups was 55 (10.8%), 38 (11.2%), 88 (18.1%), 102 (16.7%) and 109 (19.0%), respectively. Compared to the non‐exposed group, the ORs for LAE were in fetal (OR = 0.956, 95% CI = 0.605–1.500, P = 0.847), late (OR = 1.748, 95% CI = 1.208–2.555, P = 0.003), middle (OR = 1.647, 95% CI = 1.140–2.403, P = 0.008) and early (OR = 1.630, 95% CI = 1.116–2.399, P = 0.012) childhood exposed groups after adjusting potential cofounders. When stratified by gender, smoking, body mass index, hypertension and diabetes, we found that the effect of famine exposure on LAE was only modified by diabetes (Pinteraction = 0.007). Conclusions Famine exposure during childhood stage might increase the risk of LAE in adults, and this effect interacts with diabetes. Famine exposure during childhood stage might increase the risk of left atrial enlargement in adults, and this effect interacts with diabetes.