The immune system has a substantial effect on colorectal cancer (CRC) progression. Additionally, the response to immunotherapeutics and conventional treatment options (e.g., chemotherapy, radiotherapy and targeted therapies) is influenced by the immune system. The molecular characterization of colorectal cancer (CRC) has led to the identification of favorable and unfavorable immunological attributes linked to clinical outcome. With the definition of consensus molecular subtypes (CMSs) based on transcriptomic profiles, multiple characteristics have been proposed to be responsible for the development of the tumor immune microenvironment and corresponding mechanisms of immune escape. In this review, a detailed description of proposed immune phenotypes as well as their interaction with different therapeutic modalities will be provided. Finally, possible strategies to shift the CRC immune phenotype towards a reactive, anti-tumor orientation are proposed per CMS.