Phthalates, as other endocrine disrupting chemicals (EDCs), may alter the homeostasis and the action of hormones and signaling molecules, causing adverse health outcomes. This is true especially for infants, who are both more exposed and sensitive to their effects. Phthalates are particularly harmful when the exposure occurs during certain critical temporal windows of the development, such as the prenatal and the early postnatal phases. Phthalates may also interfere with the neuroendocrine systems (e.g., thyroid hormone signaling or metabolism), causing disruption of neuronal differentiation and maturation, increasing the risk of behavioral and cognitive disorders (ADHD and autistic behaviors, reduced mental, psychomotor, and IQ development, and emotional problems). Despite more studies being needed to better understand the role of these substances, plenty of evidence suggests the impact of phthalates on the neuroendocrine system development and function. This review aims to update the knowledge on the neuroendocrine consequences of neonatal and perinatal exposure to phthalates.