Breast fibroepithelial tumors comprise a heterogeneous spectrum of pathological entities, from benign fibroadenomas to malignant phyllodes tumors. Although MED12 mutations have been frequently found in fibroadenomas and phyllodes tumors, the landscapes of genetic alterations across the fibroepithelial tumor spectrum remain unclear. Here, by performing exome sequencing of 22 phyllodes tumors followed by targeted sequencing of 100 breast fibroepithelial tumors, we observed three distinct somatic mutation patterns. First, we frequently observed MED12 and RARA mutations in both fibroadenomas and phyllodes tumors, emphasizing the importance of these mutations in fibroepithelial tumorigenesis. Second, phyllodes tumors exhibited mutations in FLNA, SETD2 and KMT2D, suggesting a role in driving phyllodes tumor development. Third, borderline and malignant phyllodes tumors harbored additional mutations in cancer-associated genes. RARA mutations exhibited clustering in the portion of the gene encoding the ligand-binding domain, functionally suppressed RARA-mediated transcriptional activation and enhanced RARA interactions with transcriptional co-repressors. This study provides insights into the molecular pathogenesis of breast fibroepithelial tumors, with potential clinical implications.