The hot gas that constitutes the intracluster medium (ICM) has been studied at X-ray and millimeter/sub-millimeter wavelengths (Sunyaev-Zeldovich effect) for decades. Fast radio bursts (FRBs) offer an additional method of directly measuring the ICM and gas surrounding clusters, via observables such as dispersion measure (DM) and Faraday rotation measure (RM). We report the discovery of two FRB sources detected with the Deep Synoptic Array (DSA-110) whose host galaxies belong to massive galaxy clusters. In both cases, the FRBs exhibit excess extragalactic DM, some of which likely originates in the ICM of their respective clusters. FRB 20220914A resides in the galaxy cluster Abell 2310 at z=0.1125 with a projected offset from the cluster center of 520 kpc. The host of a second source, FRB 20220509G, is an elliptical galaxy at z=0.0894 that belongs to the galaxy cluster Abell 2311 at projected offset of 870 kpc. These sources represent the first time an FRB has been localized to a galaxy cluster. We combine our FRB data with archival X-ray, SZ, and optical observations of these clusters in order to infer properties of the ICM, including a measurement of gas temperature from DM and ySZ of 0.8-3.9 keV. We then compare our results to massive cluster halos from the IllustrisTNG simulation. Finally, we describe how large samples of localized FRBs from future surveys will constrain the ICM, particularly beyond the virial radius of clusters.