The tumor microenvironment significantly influences breast cancer development, progression, and metastasis. Various immune cell populations, including T cells, B cells, NK cells, and myeloid cells exhibit diverse functions in different breast cancer subtypes, contributing to both anti-tumor and pro-tumor activities. This review provides an overview of the predominant immune cell populations in breast cancer subtypes, elucidating their suppressive and prognostic effects. We aim to outline the role of the immune microenvironment from normal breast tissue to invasive cancer and distant metastasis. A comprehensive literature review was conducted to analyze the involvement of immune cells throughout breast cancer progression. In breast cancer, tumors exhibit increased immune cell infiltration compared to normal tissue. Variations exist across subtypes, with higher levels observed in triple-negative and HER2+ tumors are linked to better survival. In contrast,  ER+ tumors display lower immune infiltration, associated with poorer outcomes. Furthermore, metastatic sites commonly exhibit a more immunosuppressive microenvironment. Understanding the complex interaction between tumor and immune cells during breast cancer progression is essential for future research and the development of immune-based strategies. This comprehensive understanding may pave the way for more effective treatment approaches and improved patients outcomes.