Background Previous reports suggest that body composition parameters can be used to predict outcomes for patients with gastrointestinal (GI) cancers. However, evidence for an association with long-term survival is conflicting, with much of the data derived from patients with advanced disease. This study examined the effect of body composition on survival in primary operable GI cancer. Methods Patients with resectable adenocarcinoma of the GI tract (esophagus, stomach, colon, rectum) between 2006 and 2014 were identified from a prospective database. Computed tomography (CT) scans were analyzed using a transverse section at L3 to calculate sex-specific body composition indices for skeletal muscle, visceral fat, and subcutaneous fat. Kaplan–Meier and log-rank analysis were used to compare unadjusted survival. Multivariate survival analyses were performed using a proportional hazards model. Results The study enrolled 447 patients (191 woman and 256 men) with esophagogastric (OG) ( n  = 108) and colorectal (CR) ( n  = 339) cancer. Body composition did not predict survival for the OG cancer patients. Among the CR cancer patients, survival was shorter for those with sarcopenia ( p  = 0.017) or low levels of subcutaneous fat ( p  = 0.005). Older age ( p  = 0.046) and neutrophilia ( p  = 0.013) were associated with sarcopenia in patients with CR. Tumor stage ( p  = 0.033), neutrophil count ( p  = 0.011), and hypoalbuminemia ( p  = 0.023) were associated with sarcopenia in OG cancer patients. In the multivariate analysis, no single measure of body composition was an independent predictor of reduced survival. Conclusion Sarcopenia and reduced subcutaneous adiposity are associated with reduced survival for patients with primary operable CR cancer. However, in this study, no parameter of body composition was an independent prognostic marker when considered with age, tumor stage, and systemic inflammation.