We report on the observation of a large anisotropy in the rethermalization dynamics of an ultracold dipolar Fermi gas driven out of equilibrium. Our system consists of an ultracold sample of strongly magnetic 167Er fermions, spin polarized in the lowest Zeeman sublevel. In this system, elastic collisions arise purely from universal dipolar scattering. Based on cross-dimensional rethermalization experiments, we observe a strong anisotropy of the scattering, which manifests itself in a large angular dependence of the thermal relaxation dynamics. Our result is in good agreement with recent theoretical predictions. Furthermore, we measure the rethermalization rate as a function of temperature for different angles and find that the suppression of collisions by Pauli blocking is not influenced by the dipole orientation.