Maternal depressive symptoms influence neurodevelopment in the offspring. Such effects may appear to be gender-dependent. The present study examined contributions of prenatal and postnatal maternal depressive symptoms to the volume and microstructure of the amygdala in 4.5-year-old boys and girls. Prenatal maternal depressive symptoms were measured using the Edinburgh Postnatal Depression Scale (EPDS) at 26 weeks of gestation. Postnatal maternal depression was assessed at 3 months using the EPDS and at 1, 2, 3 and 4.5 years using the Beck's Depression Inventory-II. Structural magnetic resonance imaging and diffusion tensor imaging were performed with 4.5-year-old children to extract the volume and fractional anisotropy (FA) values of the amygdala. Our results showed that greater prenatal maternal depressive symptoms were associated with larger right amygdala volume in girls, but not in boys. Increased postnatal maternal depressive symptoms were associated with higher right amygdala FA in the overall sample and girls, but not in boys. These results support the role of variation in right amygdala structure in transmission of maternal depression to the offspring, particularly to girls. The differential effects of prenatal and postnatal maternal depressive symptoms on the volume and FA of the right amygdala suggest the importance of the timing of exposure to maternal depressive symptoms in brain development of girls. This further underscores the need for intervention targeting both prenatal and postnatal maternal depression to girls in preventing adverse child outcomes.