Rhabdoid tumors (RTs) are genomically simple pediatric cancers driven by the biallelic inactivation of SMARCB1, leading to SWI/SNF chromatin remodeler complex deficiency. Comprehensive evaluation of the immune infiltrates of human and mice RTs, including immunohistochemistry, bulk RNA sequencing and DNA methylation profiling studies showed a high rate of tumors infiltrated by T and myeloid cells. Single-cell RNA (scRNA) and T cell receptor sequencing highlighted the heterogeneity of these cells and revealed therapeutically targetable exhausted effector and clonally expanded tissue resident memory CD8+ T subpopulations, likely representing tumor-specific cells. Checkpoint blockade therapy in an experimental RT model induced the regression of established tumors and durable immune responses. Finally, we show that one mechanism mediating RTs immunogenicity involves SMARCB1-dependent re-expression of endogenous retroviruses and interferon-signaling activation. Display omitted •Genomically simple RTs are infiltrated by T cell and myeloid populations•Clonally expanded T cell phenotypes suggest a tumor-specific response•Checkpoint blockade induces tumor regression and immune memory in vivo•Endogenous retrovirus expression is linked to the immunogenicity of RTs Leruste et al. find that, despite their low mutation burden, rhabdoid tumors have a high rate of infiltration by T cells and myeloid cells, and the immunogenicity is linked to endogenous retrovirus expression. Immune checkpoint blockade induces tumor regression in a rhabdoid tumor mouse model.