While the volume of remote sensing (RS) data is increasing daily, deep learning in Earth observation (EO) faces lack of accurate annotations for supervised optimization. Crowdsourcing projects such as openstreetmap (OSM) distribute the annotation load to their community. However, such annotation inevitably generates noise due to insufficient control of the label quality, lack of annotators, frequent changes of the Earth's surface as a result of natural disasters, and urban development, among many other factors. We present adaptively triggered online object-wise label correction (AIO2) to address annotation noise induced by incomplete label sets. AIO2 features an adaptive correction trigger (ACT) module that avoids label correction when the model training under- or overfits, and an online object-wise label correction (O2C) methodology that employs spatial information for automated label modification. AIO2 utilizes a mean teacher model to enhance training robustness with noisy labels to both stabilize the training accuracy curve for fitting in ACT and provide pseudo labels for correction in O2C. Moreover, O2C is implemented online without the need to store updated labels every training epoch. We validate our approach on two building footprint segmentation datasets with different spatial resolutions. Experimental results with varying degrees of building label noise demonstrate the robustness of AIO2. Source code will be available at https://github.com/zhu-xlab/AIO2.git .