A wide range of exotic bound systems incorporating antiprotons (atoms, atomic ions, molecules or molecular ions) can be formed, in many cases simply by replacing at least one electron of a matter system by an antiproton. A number of these systems have been studied over decades, while others (in particular antihydrogen) have only recently been the object of precision measurements, and a much larger set have not yet been explored. This review focuses on the physics topics that these exotic systems allow to investigate, and that range from tests of fundamental symmetries to investigating the strong and electromagnetic interactions to probing nuclear models in nuclei far from the line of stability.