While recent studies have uncovered dedicated neural pathways mediating the positive control of parenting, the regulation of infant-directed aggression and how it relates to adult-adult aggression is poorly understood. Here we show that ( )-expressing neurons in the hypothalamic perifornical area (PeFA ) are activated during infant-directed attacks in males and females, but not other behaviors. Functional manipulations of PeFA neurons demonstrate the role of this population in the negative control of parenting in both sexes. PeFA neurons receive input from areas associated with vomeronasal sensing, stress, and parenting, and send projections to hypothalamic and limbic areas. Optogenetic activation of PeFA axon terminals in these regions triggers various aspects of infant-directed agonistic responses, such as neglect, repulsion, and aggression. Thus, PeFA neurons emerge as a dedicated circuit component controlling infant-directed neglect and aggression, providing a new framework to understand the positive and negative regulation of parenting in health and disease.