How environmental and physiological signals interact to influence neural circuits underlying developmentally programmed social interactions such as male territorial aggression is poorly understood. We have tested the influence of sensory cues, social context, and sex hormones on progesterone receptor (PR)-expressing neurons in the ventromedial hypothalamus (VMH) that are critical for male territorial aggression. We find that these neurons can drive aggressive displays in solitary males independent of pheromonal input, gonadal hormones, opponents, or social context. By contrast, these neurons cannot elicit aggression in socially housed males that intrude in another male’s territory unless their pheromone-sensing is disabled. This modulation of aggression cannot be accounted for by linear integration of environmental and physiological signals. Together, our studies suggest that fundamentally non-linear computations enable social context to exert a dominant influence on developmentally hard-wired hypothalamus-mediated male territorial aggression. •PR+ VMHvl neurons can drive aggression in singly housed, resident male mice•This aggression can occur absent chemosensing, gonadal hormones, or opponents•PR+ VMHvl neurons do not trigger aggression in socially housed intruder males•Disabling chemosensing enables PR+ VMHvl-triggered aggression in these males Yang et al. show that PR+ VMHvl neurons trigger aggression in solitary male mice independent of pheromone-sensing, gonadal hormones, opponents, or social context. By contrast, these neurons can trigger aggression in socially housed intruder males when their pheromone sensing is disabled.