We present Keck-Adaptive Optics and Hubble Space Telescope high resolution near-infrared (IR) imaging for 500 um-bright candidate lensing systems identified by the Herschel Multi-tiered Extra-galactic Survey (HerMES) and Herschel Astrophysical Terahertz Survey (H-ATLAS). Out of 87 candidates with near-IR imaging, 15 (~17%) display clear near-IR lensing morphologies. We present near-IR lens models to reconstruct and recover basic rest-frame optical morphological properties of the background galaxies from 12 new systems. Sources with the largest near-IR magnification factors also tend to be the most compact, consistent with the size bias predicted from simulations and pre- vious lensing models for sub-millimeter galaxies. For four new sources that also have high-resolution sub-mm maps, we test for differential lensing between the stellar and dust components and find that the 880 um magnification factor (u_880) is ~1.5 times higher than the near-IR magnification factor (u_NIR), on average. We also find that the stellar emission is ~2 times more extended in size than dust. The rest-frame optical properties of our sample of Herschel-selected lensed SMGs are consistent with those of unlensed SMGs, which suggests that the two populations are similar.