Cancer cells are characterized by a higher rate of protein turnover and greater demand for protein homeostasis compared to normal cells. In this scenario, the ubiquitin-proteasome system (UPS), which is responsible for the degradation of over 80% of cellular proteins within mammalian cells, becomes vital to cancer cells, making the UPS a critical target for the discovery of novel cancer therapeutics. This review systematically categorizes all current reported small molecule inhibitors of the various essential components of the UPS, including ubiquitin-activating enzymes (E1s), ubiquitin-conjugating enzymes (E2s), ubiquitin ligases (E3s), the 20S proteasome catalytic core particle (20S CP) and the 19S proteasome regulatory particles (19S RP), as well as their mechanism/s of action and limitations. We also discuss the immunoproteasome which is considered as a prospective therapeutic target of the next generation of proteasome inhibitors in cancer therapies.