•We assessed the influence of the chemical composition of aggregates on moisture damage in HMA.•Three lithologies with an ample mineralogical and chemical spectrum were selected.•Asphalt-aggregate samples were tested in dry and in three moisture-related conditions.•Results show that mafic rocks were more resistant to moisture damage than felsic rocks.•Al2O3, Fe2O3, MnO, MgO oxides in rocks prevent moisture damage while SiO2 promotes this phenomenom. Rock mineralogy is critical in determining the adhesion quality and moisture susceptibility of asphalt-aggregate systems in asphalt mixtures. This work aims at identifying the chemical properties of aggregates that influence the quality and durability of these systems. To accomplish this goal, three lithologies with an ample mineralogical and chemical spectrum (i.e. quartzite, granodiorite and serpentinite) were selected. Asphalt-aggregate specimens fabricated with these lithologies were tested using a new pull-off experiment in dry condition, and after 1, 3 and 7 days of being submerged in water. The results show that rocks of mafic composition (e.g. serpentinite) have better resistance to moisture damage than felsic rocks (e.g. quartzite and granodiorite), and that certain specific aggregate oxides promote or prevent moisture degradation in these systems.