Interaction between dipoles often emerges intriguing physical phenomena, such as exchange bias in the magnetic heterostructures and magnetoelectric effect in multiferroics, which lead to advances in multifunctional heterostructures. However, the defect‐dipole tends to be considered the undesired to deteriorate the electronic functionality. Here, deterministic switching between the ferroelectric and the pinched states by exploiting a new substrate of cubic perovskite, BaZrO3 is reported, which boosts the square‐tensile‐strain to BaTiO3 and promotes four‐variants in‐plane spontaneous polarization with oxygen vacancy creation. First‐principles calculations propose a complex of an oxygen vacancy and two Ti3+ ions coins a charge‐neutral defect‐dipole. Cooperative control of the defect‐dipole and the spontaneous polarization reveals ternary in‐plane polar states characterized by biased/pinched hysteresis loops. Furthermore, it is experimentally demonstrated that three electrically controlled polar‐ordering states lead to switchable and nonvolatile dielectric states for application of nondestructive electro‐dielectric memory. This discovery opens a new route to develop functional materials via manipulating defect‐dipoles and offers a novel platform to advance heteroepitaxy beyond the prevalent perovskite substrates. A new cubic perovskite substrate BaZrO3 promotes an innovative ferroelectric state and functionality in heteroepitaxial BaTiO3 film through applications of square tensile strain. The isotropic strain induces intriguing four‐variants polar domains of in‐plane spontaneous polarization. Cooperation between the built‐in local point defect‐dipole and the four‐variants polar domains enables the reversible control of ternary polar states.