Background Evidence in older adults suggests that depressive symptoms may be part of prodromal mood changes in dementia, rather than a risk factor. However, it is unclear whether variations in depressive symptoms in adulthood exhibit distinct characteristics in brain structure and cognitive function in midlife. Methods From the Coronary Artery Risk Development in Young Adults study, we identified 662 Black and White participants (age 23‐36 at baseline) who completed the Center for Epidemiological Studies Depression scale (CES‐D) at six time points over 20 years. 20 years after baseline, the participants underwent magnetic resonance imaging to characterize gray matter (GM) structures (total GM, temporal cortex, hippocampus, and entorhinal cortex) and white matter hyperintensities (WMHs). Participants also completed neuropsychological tests including the Digit Symbol Substitution Test (DSST), Rey‐Auditory Verbal Learning Test (RAVLT), Stroop Test, Montreal Cognitive Assessment (MoCA), and category and letter fluency tests, analyzed as z‐scores. Results Using growth mixture modeling, we identified four trajectories of depressive symptoms (Figure 1): consistently low scorers (“steady low”; n = 509, 76.9%), a class with an early peak and decline in symptoms (“declining”; n = 63, 9.5%), a class with late increases in symptoms (“increasing”; n = 49, 7.4%), and consistently high scorers (“steady high”; n = 41, 6.2%). Compared to the steady low class, the steady high class had lower entorhinal cortex volume (β: ‐180.80, 95% CI: ‐336.69 to ‐24.91). The increasing class, had more WMHs (β: 0.55, 95% CI: 0.22 to 0.89) and less total brain volume (β: ‐9269.25, 95% CI: ‐17104.67 to ‐1444.82). Both the steady high and the increasing classes had poorer performance on the Stroop task (β: ‐0.40, 95% CI: ‐0.70 to ‐0.10; β: ‐0.39, 95% CI: ‐0.64 to ‐0.14, respectively), however the steady high also had poorer DSST performance (β: ‐0.40, 95% CI: ‐0.67 to ‐0.13), while the increasing class had poorer performance on the MoCA (β: ‐1.20, 95% CI: ‐2.04 to ‐0.37). The declining group was not significantly different from the steady low group on any brain or cognitive measures. Conclusions Our findings suggest that trajectories in depressive symptoms in young to mid‐adulthood show different cognitive and brain phenotypes in midlife.