Pacific and Atlantic herring populations (genus Clupea) commonly experience episodic collapse and recovery. Recovery time durations are of great importance for the sustainability of fisheries and ecosystems. We collated information from 64 herring populations to characterize herring fluctuations and determine the time scales at low biomass and at high and low recruitment, and use generalized linear models and Random Survival Forests to identify the most important bottom‐up, top‐down and intrinsic factors influencing recovery times. Compared to non‐forage fish taxa, herring decline to lower minima, recover to higher maxima and show larger changes in biomass, implying herring are more prone to booms and busts than non‐forage fish species. Large year classes are more common in herring, but occur infrequently and are uncorrelated among regionally grouped stocks, implying local drivers of high recruitment. Management differs between Pacific and Atlantic herring fisheries, where at similarly low biomass, Pacific fisheries tend to be closed while Atlantic fisheries remain open. This difference had no apparent effect on herring recovery times, which averaged 11 years, although most stocks with longer recovery periods had not yet recovered at the end of the observation period. Biomass recovery is best explained by median recruitment and variability in sea surface height anomalies and sea surface temperatures—higher variability leads to shorter recovery times. In addition, the duration of recruitment failure is closely linked with low biomass. While recovery times rely on the nature of the relationship between spawning biomass and recruitment, they are still largely governed by complex and uncertain processes.