•It is unclear whether hyperglycemia in adulthood impacts brain outcomes.•HbA1c was associated with lower global brain volumes in females but not in males.•No evidence linking hyperglycemia with amyloidosis or cognitive impairments.•Our findings show target organ damage in female brains with hyperglycemia. Longitudinal studies of the relationship between hyperglycemia and brain health are rare and there is limited information on sex differences in associations. We investigated whether glycosylated hemoglobin (HbA1c) measured at ages of 53, 60–64 and 69 years, and cumulative glycemic index (CGI), a measure of cumulative glycemic burden, were associated with metrics of brain health in later life. Participants were from Insight 46, a substudy of the Medical Research Council National Survey of Health and Development (NSHD) who undertook volumetric MRI, florbetapir amyloid-PET imaging and cognitive assessments at ages of 69–71. Analyses were performed using linear and logistic regression as appropriate, with adjustment for potential confounders. We observed a sex interaction between HbA1c and whole brain volume (WBV) at all 3 time points. Following stratification of our sample, we observed that HbA1c at all ages, and CGI were positively associated with lower WBV exclusively in females. HbA1c (or CGI) was not associated with amyloid status, white matter hyperintensities (WMHs), hippocampal volumes (HV) or cognitive outcomes in either sex. Higher HbA1c in adulthood is associated with smaller WBV at 69–71 years in females but not in males. This suggests that there may be preferential target organ damage in the brain for females with hyperglycemia.