The glycosphingolipid GM1 binds cholera toxin (CT) on host cells and carries it retrograde from the plasma membrane (PM) through endosomes, the trans-Golgi (TGN), and the endoplasmic reticulum (ER) to induce toxicity. To elucidate how a membrane lipid can specify trafficking in these pathways, we synthesized GM1 isoforms with alternate ceramide domains and imaged their trafficking in live cells. Only GM1 with unsaturated acyl chains sorted efficiently from PM to TGN and ER. Toxin binding, which effectively crosslinks GM1 lipids, was dispensable, but membrane cholesterol and the lipid raft-associated proteins actin and flotillin were required. The results implicate a protein-dependent mechanism of lipid sorting by ceramide structure and provide a molecular explanation for the diversity and specificity of retrograde trafficking by CT in host cells. Display omitted ► Cells sort the glycosphingolipid GM1 by the structure of its ceramide domain (76/89) ► Only unsaturated GM1 ceramides sort retrograde all the way to the ER (64/78) ► GM1 is the vehicle that carries cholera toxin from plasma membrane to ER (61/74) ► Flotillin, cholesterol, and actin are required for GM1 sorting (55/64) Chinnapen et al. examine how membrane lipids are sorted in mammalian cells and how the membrane glycosphingolipid GM1 can direct cholera toxin trafficking from cell surface into the ER of host cells to cause disease. Their results implicate a protein-dependent mechanism of sorting GM1, determined by its unsaturated ceramide domain.