In animals, Hox transcription factors define regional identity in distinct anatomical domains. How Hox genes encode this specificity is a paradox, because different Hox proteins bind with high affinity in vitro to similar DNA sequences. Here, we demonstrate that the Hox protein Ultrabithorax (Ubx) in complex with its cofactor Extradenticle (Exd) bound specifically to clusters of very low affinity sites in enhancers of the shavenbaby gene of Drosophila. These low affinity sites conferred specificity for Ubx binding in vivo, but multiple clustered sites were required for robust expression when embryos developed in variable environments. Although most individual Ubx binding sites are not evolutionarily conserved, the overall enhancer architecture—clusters of low affinity binding sites—is maintained and required for enhancer function. Natural selection therefore works at the level of the enhancer, requiring a particular density of low affinity Ubx sites to confer both specific and robust expression. Display omitted •shavenbaby enhancers contain multiple low affinity binding sites for Ubx-Exd•Replacing low affinity sites with high-affinity sites reduced specificity•Each Ubx-Exd binding site is required for robust enhancer function•Enhancer architecture is evolutionarily conserved, but most single sites are not Transcriptional enhancers employ a strategy for balancing the need for specificity with the need for robustness with multiple clusters of low affinity binding sites.