Immune profiling of COVID-19 patients has identified numerous alterations in both innate and adaptive immunity. However, whether those changes are specific to SARS-CoV-2 or driven by a general inflammatory response shared across severely ill pneumonia patients remains unknown. Here, we compared the immune profile of severe COVID-19 with non-SARS-CoV-2 pneumonia ICU patients using longitudinal, high-dimensional single-cell spectral cytometry and algorithm-guided analysis. COVID-19 and non-SARS-CoV-2 pneumonia both showed increased emergency myelopoiesis and displayed features of adaptive immune paralysis. However, pathological immune signatures suggestive of T cell exhaustion were exclusive to COVID-19. The integration of single-cell profiling with a predicted binding capacity of SARS-CoV-2 peptides to the patients’ HLA profile further linked the COVID-19 immunopathology to impaired virus recognition. Toward clinical translation, circulating CD56+T cell frequency was identified as a predictive biomarker for patient outcome. Our comparative immune map serves to delineate treatment strategies to interfere with the immunopathologic cascade exclusive to severe COVID-19. Display omitted •GM-CSF+ T cells are a hallmark of severe respiratory syndrome independent of pathogen•T cell exhaustion and impaired early antiviral response are unique in severe COVID-19•Circulating CD56+T cell frequencies serve as a predictive biomarker for severe COVID-19•HLA profile links COVID-19 immunopathology to impaired virus recognition The pathogen-specific immune alterations in severe COVID-19 remain unknown. Using longitudinal, high-dimensional, single-cell spectral cytometry and algorithm-guided comparison of COVID-19 versus non-SARS-CoV-2-pneumonia patient samples, Kreutmair et al. identify T and NK cell immune signatures specific to SARS-CoV-2. They furthermore reveal CD56+T cell frequency as a predictive biomarker for COVID-19 outcome prediction and link impaired virus recognition to HLA genetics.