One of the major goals of research for laser-plasma accelerators is the realization of compact sources of femtosecond X-rays. In particular, using the modest electron energies obtained with existing laser systems, Compton scattering a photon beam off a relativistic electron bunch has been proposed as a source of high-energy and high-brightness photons. However, laser-plasma based approaches to Compton scattering have not, to date, produced X-rays above 1 keV. Here, we present a simple and compact scheme for a Compton source based on the combination of a laser-plasma accelerator and a plasma mirror. This approach is used to produce a broadband spectrum of X-rays extending up to hundreds of keV and with a 10,000-fold increase in brightness over Compton X-ray sources based on conventional accelerators. We anticipate that this technique will lead to compact, high-repetition-rate sources of ultrafast (femtosecond), tunable (X- through gamma-ray) and low-divergence (∼1°) photons from source sizes on the order of a micrometre.