Understanding the natural history of anal high-risk human papillomavirus (hrHPV) infection is key for designing anal cancer prevention programs but has not been systematically characterized. We reanalyzed data from 34 studies including 16 164 individuals in 6 risk groups defined by human immunodeficiency virus (HIV) status, sex, and male sexuality: men who have sex with men (MSM) and people with HIV (MSMWH), HIV-negative MSM, women with HIV (WWH), HIV-negative women, men who have sex with women (MSW) with HIV (MSWWH), and HIV-negative MSW. We used Markov models to estimate incidence and clearance of 13 hrHPV types and their determinants. Human papillomavirus (HPV) 16 had the highest incidence-clearance ratio of the hrHPV types. MSMWH had the highest hrHPV incidence (eg, 15.5% newly HPV-16 infected within 2 years), followed by HIV-negative MSM (7.5%), WWH (6.6%), HIV-negative women (2.9%), MSWWH (1.7%), and HIV-negative MSW (0.7%). Determinants of HPV-16 incidence included HIV status and number of sexual partners for MSM, women, and MSW, and anal sex behavior for MSM only. HPV-16 clearance was lower for people with HIV (PWH) and lower for prevalent than incident infection. Among MSM, increasing age was associated with lower clearance of prevalent, but not incident, HPV-16 infection. This robust and unifying analysis of anal hrHPV natural history is essential to designing and predicting the impact of HPV vaccination and HPV-based screening programs on anal cancer prevention, particularly in MSM and PWH. Importantly, it demonstrates the higher carcinogenic potential of longstanding anal prevalent hrHPV infection than more recent incident infection.