Treating chronic low back pain (LBP) can be challenging, and the most effective pharmacological therapy is controversial. The present systematic review investigated the efficacy of various pharmacological compounds to achieve pain relief and improve disability in chronic LBP patients. The present study focused on acetaminophen, amoxicillin, flupirtine, baclofen, tryciclic antidepressants (TCAs), duloxetine, topiramate, gabapentinoids, non-steroid anti-inflammatory drugs (NSAIDs) and opioids. All randomized clinical trials comparing two or more drug treatments for chronic low back pain were accessed. Studies reporting outcomes concerning patients with neurologic or mechanic, specific or aspecific low back pain with or without radiculopathy were included. LBP was considered chronic if pain had lasted more than 6 weeks. Data from 47 articles (9007 patients: mean age: 52.62 ± 7.0 years; mean BMI: 28.26 ± 2.8; mean follow-up: 3.23 ± 3.2 months) were obtained. According to published level I evidence, only baclofen, duloxetine, NSAIDs, and opiates showed to improve pain and disability levels in patients with LBP. However, the patients' demographics are heterogeneous, and the results must be interpreted with caution and in the light of possible adverse events connected to the use of these drugs.