Rapid emotion processing is an ecologically essential ability for survival in social environments in which threatening or advantageous encounters dynamically and rapidly occur. Efficient emotion recognition is subserved by different processes, depending on one's expectations; however, the underlying functional and structural circuitry is still poorly understood. In this study, we delineate brain networks that subserve fast recognition of emotion in situations either congruent or incongruent with prior expectations. For this purpose, we used multimodal neuroimaging and investigated performance on a dynamic emotion perception task. We show that the extended amygdala structural and functional networks relate to speed of emotion processing under threatening conditions. Specifically, increased microstructure of the right stria terminalis, an amygdala white‐matter pathway, was related to faster detection of emotion during actual presentation of anger or after cueing anger. Moreover, functional connectivity of right amygdala with limbic regions was related to faster detection of anger congruent with cue, suggesting selective attention to threat. On the contrary, we found that faster detection of anger incongruent with cue engaged the ventral attention “reorienting” network. Faster detection of happiness, in either expectancy context, engaged a widespread frontotemporal‐subcortical functional network. These findings shed light on the functional and structural circuitries that facilitate speed of emotion recognition and, for the first time, elucidate a role for the stria terminalis in human emotion processing.