Abstract Background Computed tomography (CT) imaging and artificial intelligence (AI)-based analyses have aided in the diagnosis and prediction of the severity of COVID-19. However, the potential of AI-based CT quantification of pneumonia in assessing patients with COVID-19 has not yet been fully explored. This study aimed to investigate the potential of AI-based CT quantification of COVID-19 pneumonia to predict the critical outcomes and clinical characteristics of patients with residual lung lesions. Methods This retrospective cohort study included 1,200 hospitalized patients with COVID-19 from four hospitals. The incidence of critical outcomes (requiring the support of high-flow oxygen or invasive mechanical ventilation or death) and complications during hospitalization (bacterial infection, renal failure, heart failure, thromboembolism, and liver dysfunction) was compared between the groups of pneumonia with high/low-percentage lung lesions, based on AI-based CT quantification. Additionally, 198 patients underwent CT scans 3 months after admission to analyze prognostic factors for residual lung lesions. Results The pneumonia group with a high percentage of lung lesions (N = 400) had a higher incidence of critical outcomes and complications during hospitalization than the low percentage group (N = 800). Multivariable analysis demonstrated that AI-based CT quantification of pneumonia was independently associated with critical outcomes (adjusted odds ratio aOR 10.5, 95% confidence interval CI 5.59–19.7), as well as with oxygen requirement (aOR 6.35, 95% CI 4.60–8.76), IMV requirement (aOR 7.73, 95% CI 2.52–23.7), and mortality rate (aOR 6.46, 95% CI 1.87–22.3). Among patients with follow-up CT scans (N = 198), the multivariable analysis revealed that the pneumonia group with a high percentage of lung lesions on admission (aOR 4.74, 95% CI 2.36–9.52), older age (aOR 2.53, 95% CI 1.16–5.51), female sex (aOR 2.41, 95% CI 1.13–5.11), and medical history of hypertension (aOR 2.22, 95% CI 1.09–4.50) independently predicted persistent residual lung lesions. Conclusions AI-based CT quantification of pneumonia provides valuable information beyond qualitative evaluation by physicians, enabling the prediction of critical outcomes and residual lung lesions in patients with COVID-19.