As the Internet of Things develops, indoor radio frequency identification (RFID)-based localization technique is getting more attention. However, there is much difficulty for improving RFID localization precision in non-line-of sight (NLOS) environment. The localization accuracy of conventional linear search algorithm is decreased exceedingly in NLOS environment. In this paper, compared with typical least square (LS) localization algorithm, a new indoor RFID-based localization algorithm is proposed to fulfill high accuracy in the electromagnetic wave propagation through NLOS environment. The new localization algorithm is a joint iterative phase reconstruction and weighted localization algorithm based on convex optimization. In NLOS environment, carrier phase difference calculated by RFID reader always has a positive bias value, which in turn causes a positive bias value for the computed distance from RFID reader to tag. Utilizing the positive effect of the NLOS phase measurements while restraining their negative effect, we propose a weighted localization algorithm using convex optimization. Iterative reconstruction is adopted for carrier phase difference calculated by RFID reader. Within each iteration process, phase is adjusted using previous carrier phase difference calculated by RFID reader and the previously estimated position of tag. The phase measurement platform made by ZTE Corporation is used to obtain the carrier phase difference of roundtrip signal between the tag and reader, and the tag position is estimated by taking the measured phase into the proposed localization algorithm. The results of the indoor experiment demonstrate the proposed algorithm comparatively has a high localization precision in NLOS environment, even though the typical LS localization algorithm does not work.