We review some of the factors that influence the hardness of polycrystalline materials with grain sizes less than 1 µm. The fundamental physical mechanisms that govern the hardness of nanocrystalline materials are discussed. The recently proposed dislocation curvature model for grain size-dependent strengthening and the 60-year-old Hall–Petch relationship are compared. For grains less than 30 nm in size, there is evidence for a transition from dislocation-based plasticity to grain boundary sliding, rotation, or diffusion as the main mechanism responsible for hardness. The evidence surrounding the inverse Hall–Petch phenomenon is found to be inconclusive due to processing artefacts, grain growth effects, and errors associated with the conversion of hardness to yield strength in nanocrystalline materials.