The growth of white matter during human adolescence shows a striking sexual dimorphism; the volume of white matter increases with age slightly in girls and steeply in boys. Here, we provide evidence supporting the role of androgen receptor (AR) in mediating the effect of testosterone on white matter. In a large sample of typically developing adolescents (n = 408, 204 males), we used magnetic resonance imaging and acquired T1-weighted and magnetization transfer ratio (MTR) images. We also measured plasma levels of testosterone and genotyped a functional polymorphism in the AR gene, namely the number of CAG repeats in exon 1 believed to be inversely proportional to the AR transcriptional activity. We found that the testosterone-related increase of white-matter volume was stronger in male adolescents with the lower versus higher number of CAG repeats in the AR gene, with testosterone explaining, respectively, 26 and 8% of variance in the volume. The MTR results suggest that this growth is not related to myelination; the MTR decreased with age in male adolescents. We speculate that testosterone affects axonal caliber rather than the thickness of the myelin sheath.