Substance use disorders (SUDs) are associated with a variety of co-occurring psychiatric disorders and other SUDs, which partly reflects genetic pleiotropy. Polygenic risk scores (PRSs) and phenome-wide association studies are useful in evaluating pleiotropic effects. However, the comparatively low prevalence of SUDs in population samples and the lack of detailed information available in electronic health records limit these data sets’ informativeness for such analyses. We used the deeply phenotyped Yale-Penn sample (n = 10,610 with genetic data; 46.3% African ancestry, 53.7% European ancestry) to examine pleiotropy for 4 major substance-related traits: alcohol use disorder, opioid use disorder, smoking initiation, and lifetime cannabis use. The sample includes both affected and control subjects interviewed using the Semi-Structured Assessment for Drug Dependence and Alcoholism, a comprehensive psychiatric interview. In African ancestry individuals, PRS for alcohol use disorder, and in European individuals, PRS for alcohol use disorder, opioid use disorder, and smoking initiation were associated with their respective primary DSM diagnoses. These PRSs were also associated with additional phenotypes involving the same substance. Phenome-wide association study analyses of PRS in European individuals identified associations across multiple phenotypic domains, including phenotypes not commonly assessed in phenome-wide association study analyses, such as family environment and early childhood experiences. Smaller, deeply phenotyped samples can complement large biobank genetic studies with limited phenotyping by providing greater phenotypic granularity. These efforts allow associations to be identified between specific features of disorders and genetic liability for SUDs, which help to inform our understanding of the pleiotropic pathways underlying them.