•Terrestrial water storage across the Yellow River basin is evaluated by GRACE data.•Terrestrial water storage change is highly correlated with precipitation and PET.•Human activities are critical factors that affect terrestrial water storage change.•A new index is proposed to assess the contributions of different factors. Terrestrial water storage (TWS) variations can be influenced by both climatic variability and human activities. In this study, we made an integrated use of GRACE data and meteorological data to characterize the TWS variations in the Yellow River basin (YRB) during 2003–2015 and investigated the relationships between terrestrial water storage change (TWSC) and human activities and climatic variability respectively. Additionally, a novel, simple but effective index was also proposed to quantify the contributions of different factors from climatic variability and human activities to TWSC in the YRB and its different subregions. The results indicated that there existed a significant TWS decrease at a rate of −4.6 ± 1.4 mm/yr (approximately 3.7 ± 1.1 Gt/yr) across the entire YRB during the past decade. The variables including precipitation, PET and NDVI had a correlation coefficient (r) of 0.76, −0.63 and −0.38 respectively with TWSC at annual scale. Apart from these, the correlations between reservoir operation and annual TWSC also have been estimated for the first time, which showed a better performance (r = 0.85) than that between TWSC and water withdrawals (r = 0.40). The results mainly indicated that TWSC was attributed to the climatic variability while the intense human activities especially reservoir operation also generated a significant effect on TWSC in the YRB. These conclusions can provide beneficial guidance for the management and assessments of local water resources.