Epoxy asphalt has excellent fatigue resistance, making it a valuable addition to recycled asphalt mixtures. However, the lack of sufficient understanding of the cracking mechanism in epoxy-modified recycled asphalt mixtures (EMRAM) hinders its widespread application. The purpose of this paper was to investigate the damage and crack characteristics of EMRAM based on a three-dimensional meso-heterogeneous model. The coarse aggregates were first randomly generated using the secondary Delaunay algorithm. Meanwhile, the fracture parameters of each component material and asphalt-aggregate interface in EMRAM were tested through three-point bending tests and direct shear tests, respectively. Cohesive zone model elements were then inserted at the aggregates, asphalt mastic, and interfaces between different materials to establish a three-dimensional meso-heterogeneous model of EMRAM. Finally, based on the established model, the damage degree of each composition material in EMRAM was quantitatively analyzed to identify the vulnerable aspects of EMRAM during the crack propagation stage. Optimization strategies were proposed accordingly. The results demonstrated that the model established in this study can accurately represent the meso-heterogeneous of EMRAM and effectively simulate its cracking behavior. EMRAM was more susceptible to developing internal damage within the aggregates, followed by relatively noticeable damage at the interface and the internal regions epoxy-modified fine aggregate mastic. The cracking resistance of EMRAM showed a trend of initially increasing and then decreasing with an increase in the volume fraction of coarse aggregates with a size above 4.75 mm. The optimal range for the volume fraction of coarse aggregates in EMRAM was within 35–44 %. The fracture energy of EMRAM was improved by more than 30 % after gradation optimization compared to EMRAM with gradation at the median value of AC20. Within the designed service life, EMRAM durable pavement can save at least 30 % more in economic costs compared to ordinary asphalt concrete pavement. •The order of material damage proportion from high to low is BA > EFAM-BA > EAFAM-BA > EFAM.•The optimal range for the volume fraction of coarse aggregates in EMRAM was within 35–44 %.•The fracture energy of EMRAM was improved by more than 30 % after gradation optimization compared to EMRAM with gradation at the median value of AC20.•EMRAM durable pavement saves at least 30 % more in economic costs compared to ordinary asphalt concrete pavement.