In order to increase the dielectric constants of polymer-based dielectrics, composite approaches, in which inorganic fillers with much higher dielectric constants are added to the polar polymer matrix, have been investigated. However, high dielectric constant fillers cause high local electric fields in the polymer, resulting in a large reduction of the electric breakdown strength. We show that a significant increase in the dielectric constant can be achieved in polyetherimide nanocomposites with nanofillers whose dielectric constant can be similar to that of the matrix. The presence of nanofillers reduces the constraints on the dipole response to the applied electric field, thus enhancing the dielectric constant. Our results demonstrate that through nanostructure engineering, the dielectric constant of nanocomposites can be enhanced markedly without using high dielectric constant nanofillers.