•Concentration of deep electron traps is in the range of 1019 cm−3 posing challenges for electrostatic control of HfO2- and HZO-based ferroelectric devices.•Trap energy distribution universally exhibit three characteristic optical energy levels: around 2 eV, 3 eV, and 4 eV below the oxide CB.•The observed traps are likely intrinsic to HfO2 lattice rather than related to dopants or ZrO2 sub-network. Electron traps in HfO2-based ferroelectric layers are known to be a major performance-degrading factor. We use room temperature charge injection and photodepopulation techniques to reveal the trap density and energy distribution in ferroelectric HfO2 and HfZrO4. The concentration of occupied electron traps is in the order of 1019 cm−3 for all analyzed samples which provides the lower limit of the defect concentration in the film. Energy distributions universally exhibit three characteristic optical energy levels: around 2 eV, 3 eV, and 4 eV below the HfO2 conduction band edge. The latter energy level was not observed previously and, according to previous DFT calculations, is too deep to be either of polaronic or O-vacancy-related nature. Nevertheless, the trap energy spectrum remains very similar in all studied samples suggesting the same sorts of the dominant electron traps both in HfO2 and HfZrO4 probably related to intrinsic defects of the HfO2 sub-network.