SUMMARY Significant changes have occurred in plant cell wall composition during evolution and diversification of tracheophytes. As the sister lineage to seed plants, knowledge on the cell wall of ferns is key to track evolutionary changes across tracheophytes and to understand seed plant‐specific evolutionary innovations. Fern cell wall composition is not fully understood, including limited knowledge of glycoproteins such as the fern arabinogalactan proteins (AGPs). Here, we characterize the AGPs from the leptosporangiate fern genera Azolla, Salvinia, and Ceratopteris. The carbohydrate moiety of seed plant AGPs consists of a galactan backbone including mainly 1,3‐ and 1,3,6‐linked pyranosidic galactose, which is conserved across the investigated fern AGPs. Yet, unlike AGPs of angiosperms, those of ferns contained the unusual sugar 3‐O‐methylrhamnose. Besides terminal furanosidic arabinose, Ara (Araf), the main linkage type of Araf in the ferns was 1,2‐linked Araf, whereas in seed plants 1,5‐linked Araf is often dominating. Antibodies directed against carbohydrate epitopes of AGPs supported the structural differences between AGPs of ferns and seed plants. Comparison of AGP linkage types across the streptophyte lineage showed that angiosperms have rather conserved monosaccharide linkage types; by contrast bryophytes, ferns, and gymnosperms showed more variability. Phylogenetic analyses of glycosyltransferases involved in AGP biosynthesis and bioinformatic search for AGP protein backbones revealed a versatile genetic toolkit for AGP complexity in ferns. Our data reveal important differences across AGP diversity of which the functional significance is unknown. This diversity sheds light on the evolution of the hallmark feature of tracheophytes: their elaborate cell walls. Significance Statement Ferns are the sister lineage of seed plants and key to understanding plant evolution. To understand ferns' unique cell walls, we analyzed arabinogalactan proteins (AGPs) from the fern genera Azolla, Salvinia, and Ceratopteris. Comparison of AGP structures throughout the streptophyte lineage reveals special features in relation to systematic positions and proposes a trend to more hydrophilic AGPs in the course of evolution. Through comparative genomic analyses, we pinpoint the potential genetic players for this diversity in cell walls.