We present new observational determinations of the evolution of the 2–10 keV X-ray luminosity function (XLF) of active galactic nuclei (AGN). We utilize data from a number of surveys including both the 2 Ms Chandra Deep Fields and the AEGIS-X 200 ks survey, enabling accurate measurements of the evolution of the faint end of the XLF. We combine direct, hard X-ray selection and spectroscopic follow-up or photometric redshift estimates at z < 1.2 with a rest-frame UV colour pre-selection approach at higher redshifts to avoid biases associated with catastrophic failure of the photometric redshifts. Only robust optical counterparts to X-ray sources are considered using a likelihood ratio matching technique. A Bayesian methodology is developed that considers redshift probability distributions, incorporates selection functions for our high-redshift samples and allows robust comparison of different evolutionary models. We statistically account for X-ray sources without optical counterparts to correct for incompleteness in our samples. We also account for Poissonian effects on the X-ray flux estimates and sensitivities and thus correct for the Eddington bias. We find that the XLF retains the same shape at all redshifts, but undergoes strong luminosity evolution out to z∼ 1, and an overall negative density evolution with increasing redshift, which thus dominates the evolution at earlier times. We do not find evidence that a luminosity-dependent density evolution, and the associated flattening of the faint-end slope, is required to describe the evolution of the XLF. We find significantly higher space densities of low-luminosity, high-redshift AGN than in prior studies, and a smaller shift in the peak of the number density to lower redshifts with decreasing luminosity. The total luminosity density of AGN peaks at z= 1.2 ± 0.1, but there is a mild decline to higher redshifts. We find that >50 per cent of black hole growth takes place at z > 1, with around half in LX < 1044 erg s−1 AGN.