Background Guidelines recommend exercise as a core treatment for osteoarthritis (OA). However, it is unclear which type of exercise is most effective, leading to inconsistency between different recommendations. Objectives The aim of this systematic review and network meta-analysis was to investigate the relative efficacy of different exercises (aerobic, mind–body, strengthening, flexibility/skill, or mixed) for improving pain, function, performance and quality of life (QoL) for knee and hip OA at, or nearest to, 8 weeks. Methods We searched nine electronic databases up until December 2017 for randomised controlled trials that compared exercise with usual care or with another exercise type. Bayesian network meta-analysis was used to estimate the relative effect size (ES) and corresponding 95% credibility interval (CrI) (PROSPERO registration: CRD42016033865). Findings We identified and analysed 103 trials (9134 participants). Aerobic exercise was most beneficial for pain (ES 1.11; 95% CrI 0.69, 1.54) and performance (1.05; 0.63, 1.48). Mind–body exercise, which had pain benefit equivalent to that of aerobic exercise (1.11; 0.63, 1.59), was the best for function (0.81; 0.27, 1.36). Strengthening and flexibility/skill exercises improved multiple outcomes at a moderate level. Mixed exercise was the least effective for all outcomes and had significantly less pain relief than aerobic and mind–body exercises. The trend was significant for pain ( p  = 0.01), but not for function ( p  = 0.07), performance ( p  = 0.06) or QoL ( p  = 0.65). Conclusion The effect of exercise varies according to the type of exercise and target outcome. Aerobic or mind–body exercise may be the best for pain and function improvements. Strengthening and flexibility/skill exercises may be used for multiple outcomes. Mixed exercise is the least effective and the reason for this merits further investigation.