•Higher residential greenness was associated with better sleep quality in rural China.•The association was stronger in males and participants with higher household income.•The association was stronger in participants with higher educational attainment.•Exposure to air pollution modified the association of greenness with sleep quality. Epidemiological studies on the association of residential greenness with sleep quality are limited in China. This study aims to investigate the association of long-term exposure to residential greenness with sleep quality in rural China. In our study, 27,654 rural residents were selected from 4 counties of Henan Province by a multi-stage stratified cluster sampling method. Participants’ sleep quality was assessed using the Pittsburgh Sleep Quality Index (PSQI). The Normalized Difference Vegetation Index (NDVI) and Enhanced Vegetation Index (EVI), two satellite-derived vegetation indexes, were used to assess the level of residential greenness. Long-term greenness exposure was defined as the averages of NDVI and EVI during the three years prior to the baseline survey. The relationship between sleep quality and greenness was assessed using the mixed-effect linear regression models. Among 27,654 rural residents, the mean age was 55.89 years (standard deviation, SD = 12.22) and 60.18% of them were female. In the crude model, the PSQI score decreased with per interquartile range (IQR) increase in EVI and NDVI ΔPSQI score (95% confidence interval, 95%CI): −0.073 (−0.115, −0.030) and −0.047 (−0.089, −0.002). After controlling potential confounders, ΔPSQI scores and 95%CIs were −0.055 (−0.095, −0.012) and −0.090 (−0.151, −0.025) associated with per IQR increment in EVI and NDVI. The results of stratified analyses showed the effect of residential greenness on sleep was stronger among males and individuals with higher household income and educational attainment than females and those with lower household income and educational attainment. Moreover, the modification effect of air pollution was observed in the greenness-sleep association. Our study indicated that higher residential greenness was significantly associated with better sleep quality in Chinese rural population, which highlights the significant effect of green space on human health.