Arithmetic ability is an important high‐level cognitive function that requires interaction among multiple brain regions. Previous studies on arithmetic development have focused on task‐induced activation in isolated brain regions or functional connectivity among particular seed regions. However, it remains largely unknown whether and how functional connectivity among large‐scale brain modules contributes to arithmetic development. In the present study, we used a longitudinal sample of task‐based functional magnetic resonance imaging (fMRI) data comprising 63 typically developing children, with two testing points being about 2 years apart. With graph theory, we examined the longitudinal development of large‐scale brain modules for a multiplication task in younger (mean age 9.88 at time 1) and older children (mean age 12.34 at time 1), respectively. The results showed that the default‐mode (DMN) and frontal‐parietal networks (FPN) became increasingly segregated over time. Specifically, intra‐connectivity within the DMN and FPN increased significantly with age, and inter‐connectivity between the DMN and visual network decreased significantly with age. Such developmental changes were mainly observed in the younger children but not in the older children. Moreover, the change in network segregation of the DMN was positively correlated with longitudinal gain in arithmetic performance in the younger children, and individual difference in network segregation of the FPN was positively correlated with arithmetic performance at Time 2 in the older children. Taken together, the present results highlight the development of the functional architecture in large‐scale brain networks from childhood to adolescence, which may provide insights into potential neural mechanisms underlying arithmetic development. The longitudinal development of large‐scale brain networks for an arithmetic task was examined from childhood to adolescence. Both the default‐mode and frontal‐parietal networks became increasingly segregated with age. Functional segregation of the default‐mode and frontal‐parietal networks related to longitudinal gain or individual difference in arithmetic ability.