We report effects of age, age 2 , sex and additive genetic factors on variability in gray matter thickness, surface area and white matter integrity in 1,010 subjects from the Genetics of Brain Structure and Function Study. Age was more strongly associated with gray matter thickness and fractional anisotropy of water diffusion in white matter tracts, while sex was more strongly associated with gray matter surface area. Widespread heritability of neuroanatomic traits was observed, suggesting that brain structure is under strong genetic control. Furthermore, our findings indicate that neuroimaging-based measurements of cerebral variability are sensitive to genetic mediation. Fundamental studies of genetic influence on the brain will help inform gene discovery initiatives in both clinical and normative samples.