Adverse childhood experiences (ACEs) are an identified risk factor for the social and emotional development of children. What is less known is the long-term effects of ACEs when poverty and ACEs coincide. Using longitudinal cohort-panel data, we examined whether exposure to ACEs by the age of three among poor children would longitudinally result in behavioral problems at ages three, five, nine, and 15, after controlling for mothers’ socioeconomic status and their children’s characteristics. We used a subsample of 2750 children and their parents living in urban poverty from the Fragile Families and Child Wellbeing study. Logistic regression modeling was used to obtain adjusted odds ratios of ACE categories predicting behavioral problems after accounting for family socioeconomic position. Our findings indicate that experiencing ACEs in early childhood was significantly associated with later behavioral outcomes from childhood to adolescence. Exposure to multiple ACEs before the age of three was significantly associated with the top-risk behavior group at age five; the odd ratios were 2.0 (CI = 1.3–3.1) and 2.9 (CI = 1.8–4.6) for two ACEs and three or more ACEs, respectively. At both ages nine and 15, children experiencing two or more ACEs had 1.9 to 3.2 times higher odds to demonstrate more the top 10th percentile of behavioral problems. Among covariates, mothers’ race and education, and children’s gender and temperament were identified as significant factors to determine behavior problems. The findings support policies and programs for families with children who have experienced economic disadvantages and early childhood adversity.