Abstract Functional brain imaging studies of normal aging typically show age-related under- and overactivations during episodic memory tasks. Older individuals also undergo nonuniform gray matter volume (GMv) loss. Thus, age differences in functional brain activity could at least in part result from local atrophy. We conducted a series of voxel-based blood oxygen level-dependent (BOLD)-GMv analyses to highlight whether age-related under- and overrecruitment was accounted for by GMv changes. Occipital GMv loss accounted for underrecruitment at encoding. Efficiency reduction of sensory-perceptual mechanisms underpinned by these areas may partly be due to local atrophy. At retrieval, local GMv loss accounted for age-related overactivation of left dorsolateral prefrontal cortex, but not of left dorsomedial prefrontal cortex. Local atrophy also accounted for age-related overactivation in left lateral parietal cortex. Activity in these frontoparietal regions correlated with performance in the older group. Atrophy in the overrecruited regions was modest in comparison with other regions as shown by a between-group voxel-based morphometry comparison. Collectively, these findings link age-related structural differences to age-related functional under- as well as overrecruitment.