INTRODUCTION We used sex and apolipoprotein E ε4 (APOE ε4) carrier status as predictors of pathologic burden in early‐onset Alzheimer's disease (EOAD). METHODS We included baseline data from 77 cognitively normal (CN), 230 EOAD, and 70 EO non‐Alzheimer's disease (EOnonAD) participants from the Longitudinal Early‐Onset Alzheimer's Disease Study (LEADS). We stratified each diagnostic group by males and females, then further subdivided each sex by APOE ε4 carrier status and compared imaging biomarkers in each stratification. Voxel‐wise multiple linear regressions yielded statistical brain maps of gray matter density, amyloid, and tau PET burden. RESULTS EOAD females had greater amyloid and tau PET burdens than males. EOAD female APOE ε4 non‐carriers had greater amyloid PET burdens and greater gray matter atrophy than female ε4 carriers. EOnonAD female ε4 non‐carriers also had greater gray matter atrophy than female ε4 carriers. DISCUSSION The effects of sex and APOE ε4 must be considered when studying these populations. HIGHLIGHTS Novel analysis examining the effects of biological sex and apolipoprotein E ε4 (APOE ε4) carrier status on neuroimaging biomarkers among early‐onset Alzheimer's disease (EOAD), early‐onset non‐AD (EOnonAD), and cognitively normal (CN) participants. Female sex is associated with greater pathology burden in the EOAD cohort compared to male sex. The effect of APOE ε4 carrier status on pathology burden was the most impactful in females across all cohorts.