We extend the theory of dipole moments in crystalline insulators to higher multipole moments. As first formulated in Benalcazar et al. Science 357, 61 (2017), we show that bulk quadrupole and octupole moments can be realized in crystalline insulators. In this paper, we expand in great detail the theory presented previously Benalcazar et al., Science 357, 61 (2017) and extend it to cover associated topological pumping phenomena, and a class of three-dimensional (3D) insulator with chiral hinge states. We start by deriving the boundary properties of continuous classical dielectrics hosting only bulk dipole, quadrupole, or octupole moments. In quantum mechanical crystalline insulators, these higher multipole bulk moments manifest themselves by the presence of boundary-localized moments of lower dimension, in exact correspondence with the electromagnetic theory of classical continuous dielectrics. In the presence of certain symmetries, these moments are quantized, and their boundary signatures are fractionalized. These multipole moments then correspond to new symmetry-protected topological phases. The topological structure of these phases is described by “nested” Wilson loops, which we define. These Wilson loops reflect the bulk-boundary correspondence in a way that makes evident a hierarchical classification of the multipole moments. Just as a varying dipole generates charge pumping, a varying quadrupole generates dipole pumping, and a varying octupole generates quadrupole pumping. For nontrivial adiabatic cycles, the transport of these moments is quantized. An analysis of these interconnected phenomena leads to the conclusion that a new kind of Chern-type insulator exists, which has chiral, hinge-localized modes in 3D. We provide the minimal models for the quantized multipole moments, the nontrivial pumping processes, and the hinge Chern insulator, and describe the topological invariants that protect them.