Habitat choice plays a critical role in the processes of host range evolution, specialization, and ecological speciation. Pea aphid, Acyrthosiphon pisum, populations from alfalfa and red clover in eastern North America are known to be genetically differentiated and show genetic preferences for the appropriate host plant. This species feeds on many more hosts, and here we report a study of the genetic variation in host plant preference within and between pea aphid populations collected from eight genera of host plants in southeastern England. Most host-associated populations show a strong, genetically based preference for the host plant from which they were collected. Only in one case (populations from Vicia and Trifolium) was there little difference in the plant preference spectrum between populations. All populations showed a significant secondary preference for the plant on which all the aphid lines were reared: broad bean, Vicia faba, previously suggested to be a “universal host” for pea aphids. Of the total genetic variance in host preference within our sample, 61% could be attributed to preference for the collection host plant and a further 9% to systematic differences in secondary preferences with the residual representing within-population genetic variation between clones. We discuss how a combination of host plant preference and mating on the host plant may promote local adaptation and possibly ecological speciation, and whether a widely accepted host could oppose speciation by mediating gene flow between different populations.