Patients with osteoporotic fractures have an increased risk for secondary fractures. However, a rigorous study that assesses the effectiveness of individual osteoporotic drugs in preventing subsequent fractures is lacking. The purpose of this review was to analyze the effectiveness of anti-osteoporotic drugs in preventing secondary fractures. We searched for randomized controlled trials that showed the incidence of secondary fractures while using anti-osteoporotic drugs (bisphosphonates, selective estrogen receptor modulators, parathyroid hormone (PTH), or calcitonin) in MEDLINE, Embase.com , and Cochrane Central Register databases. We estimated risk ratios (RR) and numbers needed to treat (NNT) to prevent secondary fractures. Twenty-six studies met our eligibility criteria. There was a significant reduction in RR (0.38–0.77) after the use of anti-osteoporotic drugs for secondary vertebral fractures. Bisphosphonates and PTH significantly reduced the risk of a secondary non-vertebral fracture (RR 0.59 and 0.64). PTH needed the fewest number of patients to be treated to prevent a secondary vertebral fracture (NNT: 56). Our study demonstrated the effectiveness of anti-osteoporotic agents included in our systematic review in preventing secondary vertebral fractures. Bisphosphonates and PTH were most effective in preventing non-vertebral fractures. We suggest that clinicians should prescribe these drugs to prevent secondary vertebral/non-vertebral fractures.