Purpose To compare the oncological long-term efficacy of whole gland high-intensity focused ultrasound (HIFU) therapy and radical prostatectomy (RP) in patients with clinically localized prostate cancer. Methods 418 patients after open RP (1997–2004) were compared with 469 patients after whole gland HIFU (1997–2009) without preselection. Oncological follow-up focused on biochemical relapse, salvage treatment, life status and cause-specific mortality. The univariate log rank test was used to compare both treatment options regarding overall survival (OS), cancer-specific survival (CSS), biochemical failure-free survival (BFS) and salvage treatment-free survival (STS). To adjust the treatment effect for further prognostic baseline variables, a multivariable Cox proportional hazards regression model was calculated for each end point. Results Median follow-up was 13.3 years in the RP group and 6.5 years in the HIFU group. OS/CSS/BFS/STS rates at 10 years were 91/98/80/80% after RP and 76/94/70/71% after HIFU. HIFU therapy (reference RP) was a significant and independent predictor for an inferior OS, CSS and STS. In subgroup analysis, HIFU provided significantly reduced CSS for intermediate- ( p  = 0.010) and high-risk patients ( p  = 0.048); whereas no difference was observed in the low-risk group, intermediate-risk HIFU patients showed a significantly inferior STS ( p  = 0.040). Conclusions While whole gland HIFU offers a comparable long-term efficacy for low-risk patients, sufficient cancer control for high-risk patients is more than doubtful. For the subgroup of intermediate-risk patients, CSS rates seem to be comparable up to 10 years suggesting that HIFU may be an alternative for older patients, although a higher risk of salvage treatment should be expected.