In this report we discuss the dynamical response of heavy quantum impurities immersed in a Fermi gas at zero and at finite temperature. Studying both the frequency and the time domain allows one to identify interaction regimes that are characterized by distinct many-body dynamics. From this theoretical study a picture emerges in which impurity dynamics is universal on essentially all time scales, and where the high-frequency few-body response is related to the long-time dynamics of the Anderson orthogonality catastrophe by Tan relations. Our theoretical description relies on different and complementary approaches: functional determinants give an exact numerical solution for time- and frequency-resolved responses, bosonization provides accurate analytical expressions at low temperatures, and the theory of Toeplitz determinants allows one to analytically predict response up to high temperatures. Using these approaches we predict the thermal decoherence rate of the fermionic system and prove that within the considered model the fastest rate of long-time decoherence is given by γ=πkBT/4. We show that Feshbach resonances in cold atomic systems give access to new interaction regimes where quantum effects can prevail even in the thermal regime of many-body dynamics. The key signature of this phenomenon is a crossover between different exponential decay rates of the real-time Ramsey signal. It is shown that the physics of the orthogonality catastrophe is experimentally observable up to temperatures T/TF≲0.2 where it leaves its fingerprint in a power-law temperature dependence of thermal spectral weight and we review how this phenomenon is related to the physics of heavy ions in liquid 3He and the formation of Fermi polarons. The presented results are in excellent agreement with recent experiments on LiK mixtures, and we predict several new phenomena that can be tested using currently available experimental technology.