•Base asphalt, conventional CRMA and drained CRMA were evaluated.•BBS test, TSR and HWT test were performed for performance evaluation.•Rubber particles selectively absorb the light components of asphalt.•BBS test can only highlight the interfacial bonding strength of asphalt and aggregate.•Improving the elasticity of the mixture is helpful to improve the water resistance of the mixture. The bonding performance and moisture resistance of crumb rubber modified asphalt (CRMA) has been a hot debate for many years. On the other hand, the influence of crumb rubber particles can be divided into two parts: the filler nature of crumb rubber particle, namely the particle effect (PE); and the chemical interaction between crumb rubber particle/base asphalt, namely the interaction effect (IE). This research aims to understand and quantify the PE and IE in terms of CRMA binder bonding performance. To do so, base asphalt, conventional CRMA and drained CRMA were evaluated. Drained CRMA were prepared by removing rubber particles from conventional CRMA via sieving. Asphalt binder bonding strength (BBS) test, tensile strength ratio (TSR) and Hamburg Wheel Tracking (HWT) tests were employed for performance evaluation. Furthermore, rheology master curves, Fourier Transform Infrared Spectroscopy (FTIR) and fluorescence microscopy were conducted to help reveal the influence of PE and IE. The results show that in terms of BBS testing, the PE and IE may vary with different CRMA formulas, but overall, a large negative PE with a small positive IE is seen, leading to inferior bonding strength of all CRMA samples. On the other hand, CRMA shows the best mixture performance (TSR, HWT) with both positive PE and IE. It is believed that even though the rubber particle is not favorable in terms of binder bonding performance, it is beneficial to the mixture water-resistance by improving the mixture elasticity.