Objective Rett syndrome (RTT) and epileptic encephalopathy (EE) are devastating neurodevelopmental disorders with distinct diagnostic criteria. However, highly heterogeneous and overlapping clinical features often allocate patients into the boundary of the two conditions, complicating accurate diagnosis and appropriate medical interventions. Therefore, we investigated the specific molecular mechanism that allows an understanding of the pathogenesis and relationship of these two conditions. Methods We screened novel genetic factors from 34 RTT‐like patients without MECP2 mutations, which account for ∼90% of RTT cases, by whole‐exome sequencing. The biological function of the discovered variants was assessed in cell culture and Xenopus tropicalis models. Results We identified a recurring de novo variant in GABAB receptor R2 (GABBR2) that reduces the receptor function, whereas different GABBR2 variants in EE patients possess a more profound effect in reducing receptor activity and are more responsive to agonist rescue in an animal model. Interpretation GABBR2 is a genetic factor that determines RTT‐ or EE‐like phenotype expression depending on the variant positions. GABBR2‐mediated γ‐aminobutyric acid signaling is a crucial factor in determining the severity and nature of neurodevelopmental phenotypes. Ann Neurol 2017;82:466–478