Abstract Background The known relationship between the gestational age at maternal primary infection an the outcome of congenital CMV is based on small, retrospective studies conducted between 1980 and 2011. They reported that 32% and 15% of cases had sequelae following a maternal primary infection in the first and second or the third trimester, respectively. We aimed to revisit this relationship prospectively between 2011 and 2017, using accurate virological tools. Methods We collected data on women with a primary infection and an infected child aged at least 1 year at the time of analysis. An accurate determination of the timing of the primary infection was based upon serial measurements of immunoglobulin (Ig) M and IgG and on IgG avidity in sera collected at each trimester. The case outcome was assessed according to a structured follow-up between birth and 48 months. Results We included 255 women and their 260 fetuses/neonates. The dating of the maternal infection was prospective in 86% of cases and retrospective in 14%. At a median follow-up of 24 months, the proportion of sensorineural hearing loss and/or neurologic sequelae were 32.4% (95% confidence interval CI 23.72–42.09) after a maternal primary infection in the first trimester, 0 (95% CI 0–6.49) after an infection in the second trimester, and 0 (95% CI 0–11.95) after an infection in the third trimester (P < .0001). Conclusions These results suggest that a cytomegalovirus infection can be severe only when the virus hits the fetus in the embryonic or early fetal period. Recent guidelines recommend auditory follow-ups for at least 5 years for all infected children. This raises parental anxiety and generates significant costs. We suggest that auditory and specialized neurologic follow-ups may be recommended only in cases of a maternal infection in the first trimester. In this study, only children infected after a maternal primary infection in the first trimester had sequelae at follow-up. For women with seroconversion in the second or third trimester, the risk returns to the baseline associated with any pregnancy.