This study investigates changes in seismic velocities in the period 1999–2021 using about 700 permanent and temporary broadband seismic stations in the state of California. We compute single‐station cross‐correlations of the ambient seismic noise and use the coda‐wave interferometry to measure the changes in seismic velocities (dv/v) using a stretching technique. We focus on the 2–4 Hz frequency band and the upper 500 m of the near‐surface sensitivity. We discuss dv/v within the context of nonlinear elasticity. We fit models of thermoelastic strains, various hydrological models that diffuse rainwater, and slow‐dynamics healing models for the postseismic response of earthquakes. In general, we find that both thermoelastic strains and hydrological strains have a similar amplitude of impacts on dv/v. We find that the diffusion of rainwater using a drained response in a poroelastic medium explains most of the data. The best fit hydraulic diffusivity is high in the mountains and low in the basin. We find that the largest drop in seismic velocity occurs during the 2004–2005 wet winter and that the 2011–2016. Drought is characterized by a multiyear marked increase in dv/v. We interpret site‐specific variations with land subsidence or inflation detected by remote sensing. We also find a decade‐long postseismic response of two major earthquakes and bound the time scale of relaxation processes to a few years. Together, we see long‐term changes in seismic velocities showing a positive trend over two decades that we can interpret as long‐term lowering of the groundwater table. Plain Language Summary The multiyear droughts and sudden downpours cause stress to water management and natural hazards in California. This study investigates their impact on the subsurface seismic properties. Large seismic data archives such as reliable permanent seismic networks and large computing capabilities allow for a state‐wide, two‐decade‐long analysis of the changes in the shallow seismic structures. The near‐surface seismic velocities in the upper 500 m of the Earth's crust are strongly modulated by annual variations in air temperature and diffusion of rainfall. Due to extreme climate conditions in California, seismic velocities change by up to −2% during a single winter due to rain, and up to 2% during 20 years of progressively drying conditions. The recovery of fault‐zone materials near two significant earthquakes, the 1999 Hector Mine and 2010 El Mayor Cucapah earthquakes, indicates a relaxation process that can last decades and implies characteristic time scales of a few years and a spatial heterogeneity that coincide with deep crustal viscous properties. This study presents passive seismology as a tool to probe Earth's tectonic‐hydrological processes complementary to geodesy and hydrology. Key Points Environmental factors considerably change near‐surface seismic velocity over decades There is a long‐term increase in seismic velocities in California due to increased drought conditions The decade‐long recovery from large earthquakes of sites very close to faults indicates postseismic strain localization and a delayed healing