The relationship between anxiety and mathematics has often been investigated in the literature. Different forms of anxiety have been evaluated, with math anxiety (MA) and test anxiety (TA) consistently being associated with various aspects of mathematics. In this meta-analysis, we have evaluated the impact of these forms of anxiety, distinguishing between different types of mathematical tasks. In investigating this relationship, we have also included potential moderators, such as age, gender, working memory, type of task, and type of material. One hundred seventy-seven studies met the inclusion criteria, providing an overall sample of 906,311 participants. Results showed that both MA and TA had a significant impact on mathematics. Sociodemographic factors had modest moderating effects. Working memory (WM) also mediated the relationship between MA and TA with mathematics; however, this indirect effect was weak. Theoretical and educational implications, as well as future directions for research in this field, are discussed.