Fingerprint-based localization relies on an accurate and up-to-date radio map, which is however cumbersome to obtain. In this paper, a novel scheme is proposed to online adapt radio maps to environmental dynamics by using low-cost crowdsourced received signal strength (RSS) measurements. To be specific, a coarse-grained radio map is initially established in the offline phase utilizing the standard Gaussian process regression (GPR) given a limited number of fingerprints (i.e., RSS measurements with location labels), and further can be recursively refined in the online phase given crowdsourced RSS measurements with their noisy location labels obtained through the existing radio map. Differently from existing GPR-based approaches, the proposed scheme adopts extended GPR to alleviate the model inaccuracy induced by such noisy location labels, and then presents a marginalized particle extended Gaussian process (MPEG) to recursively filter the radio map. In addition, pedestrian dead reckoning (PDR) is leveraged to calibrate such noisy location labels. Extensive experiments are carried out in a real scenario with area of nearly 1000 m 2 during a five-month period of time, and a thorough comparison with several existing approaches indicates that the proposed scheme gradually improves the localization accuracy on average by as much as 31.2%, while the counterparts result in fluctuant localization performance and improve the localization accuracy on average by 13.3%.