Single-cell RNA sequencing (scRNA-seq) is a powerful tool for defining cellular diversity in tumors, but its application toward dissecting mechanisms underlying immune-modulating therapies is scarce. We performed scRNA-seq analyses on immune and stromal populations from colorectal cancer patients, identifying specific macrophage and conventional dendritic cell (cDC) subsets as key mediators of cellular cross-talk in the tumor microenvironment. Defining comparable myeloid populations in mouse tumors enabled characterization of their response to myeloid-targeted immunotherapy. Treatment with anti-CSF1R preferentially depleted macrophages with an inflammatory signature but spared macrophage populations that in mouse and human expresses pro-angiogenic/tumorigenic genes. Treatment with a CD40 agonist antibody preferentially activated a cDC population and increased Bhlhe40+ Th1-like cells and CD8+ memory T cells. Our comprehensive analysis of key myeloid subsets in human and mouse identifies critical cellular interactions regulating tumor immunity and defines mechanisms underlying myeloid-targeted immunotherapies currently undergoing clinical testing. Display omitted •scRNA-seq analyses highlight conserved myeloid subsets in human and murine CRC•Two distinct TAM subsets show inflammatory and angiogenic signatures, respectively•Two distinct TAM subsets show differential sensitivity to CSF1R blockade•Anti-CD40 activates specific cDC1s and expands Th1-like and CD8+ memory T cells Combined scRNA-seq analyses on the tumor microenvironment in colorectal cancer and murine tumor models identify distinct myeloid populations that convey differential sensitivity to CSF1R blockade and define concerted immune responses involving dendric cells and T cells upon anti-CD40 treatment.