The dynamics of heterogeneous materials, like rocks and concrete, is complex. It includes such features as nonlinear elasticity, hysteresis, and long-time relaxation. This dynamics is very sensitive to microstructural changes and damage. The goal of this paper is to propose a physical model describing the longitudinal vibrations in heterogeneous material, and to develop a numerical strategy to solve the evolution equations. The theory relies on the coupling of two processes with radically different time scales: a fast process at the frequency of the excitation, governed by nonlinear elasticity and viscoelasticity, and a slow process, governed by the evolution of defects. The evolution equations are written as a nonlinear hyperbolic system with relaxation. A time-domain numerical scheme is developed, based on a splitting strategy. The features observed by numerical simulations show qualitative agreement with the features observed experimentally by Dynamic Acousto-Elastic Testing. •Physical model relating ultrasonic experimental observations in complex nonlinear media.•Sound mathematical properties, combining nonlinear hyperbolic systems and relaxation terms.•Robust numerical methods.•Qualitative agreement with experiments.