The Neutron star Interior Composition Explorer observed several rotation-powered millisecond pulsars (MSPs) to search for or confirm the presence of X-ray pulsations. When broad and sine-like, these pulsations may indicate thermal emission from hot polar caps at the magnetic poles on the neutron star surface. We report confident detections (≥4.7 after background filtering) of X-ray pulsations for five of the seven pulsars in our target sample: PSR J0614−3329, PSR J0636+5129, PSR J0751+1807, PSR J1012+5307, and PSR J2241−5236, while PSR J1552+5437 and PSR J1744−1134 remain undetected. Of those, only PSR J0751+1807 and PSR J1012+5307 had pulsations previously detected at the 1.7 and almost 3 confidence levels, respectively, in XMM-Newton data. All detected sources exhibit broad sine-like pulses, which are indicative of surface thermal radiation. As such, these MSPs are promising targets for future X-ray observations aimed at constraining the neutron star mass-radius relation and the dense matter equation of state using detailed pulse profile modeling. Furthermore, we find that three of the detected MSPs exhibit a significant phase offset between their X-ray and radio pulses.