Chemoradiotherapy (CRT) is the standard neoadjuvant therapy for locally advanced rectal cancer (RC). However, neoadjuvant chemotherapy (NAC) also shows favorable outcomes. Although the immunological environment of RC has been thoroughly discussed, the effect of NAC on it is less clear. Here, we investigated the immunological microenvironment, including T cell infiltration, activation, and topological distribution, of resected RC tissue after neoadjuvant therapies and evaluated the correlation between T cell subsets and patient prognosis. Rectal cancer patients (n = 188) were enrolled and categorized into 3 groups, namely CRT (n = 41), NAC (n = 46), and control (surgery alone; n = 101) groups. Characterization of residual carcinoma cells and T cell subsets in resected tissues was performed using multiplex fluorescence immunohistochemistry. The densities of total and activated (Ki67high) T cells in tissues after NAC, but not CRT, were higher than in control. In both CRT and NAC groups, patients presenting with higher treatment effects showed aggressive infiltration of T cell subsets into carcinomas. Multivariate analyses of pathological and immunological features and prognosis revealed that carcinoma Ki67highCD4+ T cells after CRT and stromal Ki67highCD8+ T cells after NAC are important prognostic factors, respectively. Our results suggest that evaluation of T cell activation with Ki67 expression and its tumor localization can be used to determine the prognosis of advanced RC after neoadjuvant therapies. Evaluating tumor‐infiltrating lymphocytes in rectal cancer tissue after neoadjuvant therapies, we found that Ki67 expression and localization of T cells are significantly associated with systemic recurrence after surgery. Notably, T‐cell subsets involved in the prognosis were different between after chemoradiotherapy and chemotherapy. In this study, we suggested that the evaluation of T cell activation with Ki67 expression and its tumor localization can be used to determine the prognosis of advanced rectal cancer after neoadjuvant therapies.