The Remote Sensing (RS) field has an increasing research interest in using deep learning (DL) models to recognize kinds of RS data, leading to a great demand for training data annotation. Due to the high cost of expertise, employing non-experts to label data has become an important way to improve labeling efficiency. Commonly, a single data sample is labeled by multiple annotators and the most voted label is accepted to promise accuracy. But in the RS context, the widely admitted strategy could lose effect. Usually RS data involves considerable classes on account of the complexity of surface environments, which is prone to inter-class similarity difficult to distinguish. Annotators without expertise probably make mistakes on these indistinguishable classes, thus causing error voted labels. Although classification of different characteristics in RS data have been widely documented, the non-expert annotators are unfamiliar with these expertise, and it is difficult to force them to handle specialized labeling skills. To address the issues, this paper bases multi-annotator label selection on the investigation of annotators' own ability in distinguishing similar classes of images. A quality evaluation process is designed which weights the labels from capable annotators higher than those from weak ones. By a multi-round quality evaluation algorithm, correct labels could out-compete the wrong ones even disadvantaged in numbers. Experimental results demonstrate the advance of the proposed method on RS datasets.