The influence of sucrose palmitate, Tween 20, and lecithin on the properties of heat-induced aggregates and cold-set gels of β-lactoglobulin was studied based on an experimental mixture design with a fixed total emulsifier concentration. Emulsifiers were added to the protein solution before heating. Aggregate size and absolute values of ζ potential increased with the addition of emulsifiers, among which lecithin had the most pronounced effect. The water retention of the aggregates correlated positively with the aggregate size. Gels had reduced fracture stress and strains with increasing sucrose palmitate and decreasing Tween 20 contents. The fracture properties correlated with the ζ potentials of the aggregates, and larger aggregates led to gels with higher water-holding capacities. The emulsifiers hence influenced the gel properties indirectly via the aggregate properties. The impact of emulsifiers on food structures should therefore be considered when a food product is designed.