The effective deployment of immune responses depends on the activation of well-defined signaling pathways that interact with cell-intrinsic epigenetic features to control the activation of stimulus-specific and kinetically accurate gene expression programs. The interferon regulatory factors (IRFs) are critical regulators of both development and activation of distinct cells of the immune system. The influence of the IRF family transcription factors (TFs) on immune responses is mainly linked to the control of the type I interferon (IFN) system, which must be not only rapidly activated to mount an efficient host response, but also tightly regulated to avoid detrimental effects. In this review we discuss the different regulatory layers controlling IRF activity and specifically IRF-mediated IFN responses. We also highlight how the interplay between the DNA-binding properties of the IRF family TFs and the epigenetic landscape controls cell- and context-specific responses.