Heterostructures of van der Waals bonded layered materials offer unique means to tailor dielectric screening with atomic-layer precision, opening a fertile field of fundamental research. The optical analyses used so far have relied on interband spectroscopy. Here we demonstrate how a capping layer of hexagonal boron nitride (hBN) renormalizes the internal structure of excitons in a WSe2 monolayer using intraband transitions. Ultrabroadband terahertz probes sensitively map out the full complex-valued mid-infrared conductivity of the heterostructure after optical injection of 1s A excitons. This approach allows us to trace the energies and line widths of the atom-like 1s–2p transition of optically bright and dark excitons as well as the densities of these quasiparticles. The excitonic resonance red shifts and narrows in the WSe2/hBN heterostructure compared to the bare monolayer. Furthermore, the ultrafast temporal evolution of the mid-infrared response function evidences the formation of optically dark excitons from an initial bright population. Our results provide key insight into the effect of nonlocal screening on electron–hole correlations and open new possibilities of dielectric engineering of van der Waals heterostructures.