Polymer dielectrics having high dielectric constant, high temperature capability, and low loss are attractive for a broad range of applications such as film capacitors, gate dielectrics, artificial muscles, and electrocaloric cooling. Unfortunately, it is generally observed that higher polarization or dielectric constant tends to cause significantly enhanced dielectric loss. It is therefore highly desired that the fundamental physics of all types of polarization and loss mechanisms be thoroughly understood for dielectric polymers. In this Perspective, we intend to explore advantages and disadvantages for different types of polarization. Among a number of approaches, dipolar polarization is promising for high dielectric constant and low loss polymer dielectrics, if the dipolar relaxation peak can be pushed to above the gigahertz range. In particular, dipolar glass, paraelectric, and relaxor ferroelectric polymers are discussed for the dipolar polarization approach.