Global urbanisation has accelerated in recent years, especially in rapidly growing coastal cities, and the destruction of habitat and natural resources has intensified. Although much attention has been paid to the study of habitat quality, there are still gaps in our understanding of the factors that influence it and their interactions. In this study, the InVEST habitat quality evaluation model and the GeoDetector model were used to construct a framework for analysing the dynamic changes in habitat quality and their influencing factors from 1992 to 2015. Wenzhou City, Zhejiang Province, China, was selected as the study area. The new framework extends studies on habitat quality change to annual analysis and reduces the lag between the actual change and the mapping time. The interactions between natural and anthropogenic factors are explored, and the effects of different types of land use conversion on habitat quality are further discussed. The results show that: (1) During the study period, cultivated and construction land areas in Wenzhou City increased the most, and forest land area decreased the most. (2) Habitat quality in Wenzhou City was generally good during the study period, but it showed a declining trend from year to year, and the distribution of habitat quality decreased from west to east. (3) The interactions between land use change and annual precipitation change and those between land use change and population density change have the most significant impact on habitat quality. The conversion of forest land to cultivated land, conversion of water area to cultivated land, and conversion of forest land to building land have the greatest impact on habitat quality. The results of the study can provide recommendations for ecological restoration, optimal integration of protected areas, and provide a reference for the healthy and sustainable development of coastal regions.