Bone loss with aging and menopause increases the risk of fragile vertebral fracture, osteoporotic vertebral compression fracture (OVCF). The fracture causes severe pain, impedes respiratory function, lower the quality of life, and increases the risk of new fractures and deaths. Various medications have been prescribed to prevent a secondary fracture, but few study summarized their effects. Therefore, we investigated their effects on preventing subsequent OVCF via meta-analyses of randomized controlled trials. Electronic databases, including MEDLINE, EMBASE, CENTRAL, and Web of Science were searched for published randomized controlled trials from June 2015 to June 2019. The trials that recruited participants with at least one OVCF were included. We assessed the risk of bias of every study, estimated relative risk ratio of secondary OVCF, non-vertebral fracture, gastrointestinal complaints and discontinuation due to adverse events. Finally, we evaluated the quality of evidence. Forty-one articles were included. Moderate to high quality evidence proved the effectiveness of zoledronate (Relative Risk, RR: 0.34; 95% CI, 0.17-0.69, p = 0.003), alendronate (RR: 0.54; 95% CI: 0.43-0.68; p < 0.0001), risedronate (RR: 0.61; 95% CI: 0.51-0.73; p < 0.0001), etidronate (RR, 0.50; 95% CI, 0.29-0.87, p < 0.01), ibandronate (RR: 0.52; 95% CI: 0.38-0.71; p < 0.0001), parathyroid hormone (RR: 0.31; 95% CI: 0.23-0.41; p < 0.0001), denosumab (RR, 0.41; 95% CI, 0.29-0.57; p < 0.0001) and selective estrogen receptor modulators (Raloxifene, RR: 0.58; 95% CI: 0.44-0.76; p < 0.0001; Bazedoxifene, RR: 0.66; 95% CI: 0.53-0.82; p = 0.0002) in preventing secondary fractures. Moderate quality evidence proved romosozumab had better effect than alendronate (Romosozumab vs. alendronate, RR: 0.64; 95% CI: 0.49-0.84; p = 0.001) and high quality evidence proved that teriparatide had better effect than risedronate (risedronate vs. teriparatide, RR: 1.98; 95% CI: 1.44-2.70; p < 0.0001). Zoledronate, alendronate, risedronate, etidronate, ibandronate, parathyroid hormone, denosumab and selective estrogen receptor modulators had significant secondary prevention effects on OVCF. Moderate quality evidence proved romosozumab had better effect than alendronate. High quality evidence proved PTH had better effect than risedronate, but with higher risk of adverse events.