Body composition is often altered in psychiatric disorders. Using genome-wide common genetic variation data, we calculate sex-specific genetic correlations amongst body fat %, fat mass, fat-free mass, physical activity, glycemic traits and 17 psychiatric traits (up to N = 217,568). Two patterns emerge: (1) anorexia nervosa, schizophrenia, obsessive-compulsive disorder, and education years are negatively genetically correlated with body fat % and fat-free mass, whereas (2) attention-deficit/hyperactivity disorder (ADHD), alcohol dependence, insomnia, and heavy smoking are positively correlated. Anorexia nervosa shows a stronger genetic correlation with body fat % in females, whereas education years is more strongly correlated with fat mass in males. Education years and ADHD show genetic overlap with childhood obesity. Mendelian randomization identifies schizophrenia, anorexia nervosa, and higher education as causal for decreased fat mass, with higher body fat % possibly being a causal risk factor for ADHD and heavy smoking. These results suggest new possibilities for targeted preventive strategies.