Broadband single photons are usually considered not to couple efficiently to atomic gases because of the large mismatch in bandwidth. Contrary to this intuitive picture, here we demonstrate that the interaction of ultrashort single photons with a dense resonant atomic sample deeply modifies the temporal shape of their wave packet mode without degrading their nonclassical character, and effectively generates zero-area single-photon pulses. This is a clear signature of strong transient coupling between single broadband (THz-level) light quanta and atoms, with intriguing fundamental implications and possible new applications to the storage of quantum information.