The spatial organization of immune cells within the tumor microenvironment (TME) largely determines the anti-tumor immunity and also highly predicts tumor progression and therapeutic response. Tim-3 is a well-accepted immune checkpoint and plays multifaceted immunoregulatory roles via interaction with distinct Tim-3 ligands (Tim-3L), showing great potential as an immunotherapy target. However, the cell sociology mediated by Tim-3/Tim-3L and their contribution to tumor development remains elusive. Here, we analyzed the spatial distribution of Tim-3/Tim-3L in TME using multiplex fluorescence staining and revealed that despite the increased Tim-3 expression in various tumor-infiltrated lymphocytes, Tim-3+CD4+ cells were more accumulated in parenchymal/tumor region compared with stromal region and exhibited more close association with patient survival. Strikingly, CD4 T cells surrounding Tim-3L+ cells expressed higher Tim-3 than other cells in cancerous tissues. In vivo studies confirmed that depletion of CD4 T cells completely abrogated the inhibition of tumor growth and metastasis, as well as the functional improvement of CD8 T and NK, mediated by Tim-3 blockade, which was further validated in peripheral lymphocytes from patients with hepatocellular carcinoma. In conclusion, our findings unravel the importance of CD4 T cells in Tim-3/Tim-3L-mediated immunosuppression and provide new thoughts for Tim-3 targeted cancer immunotherapy. Display omitted The authors analyzed the cell sociology mediated by Tim-3/Tim-3L and their contribution to tumor development. They unraveled the close association of Tim-3+CD4 T cells with Tim-3L-expressing cells in TME and their significance with patient prognosis. Functionally, CD4 T cells are critical for Tim-3/Tim-3L-mediated immunosuppression in TME.