•A direct approach to controlling the topology in structural optimization is proposed.•The structural efficiency and the effect of complexity control are well balanced.•The methodology can provide the designer with diverse and competitive solutions. Structural shape and topology optimization has undergone tremendous developments in recent years due to its important applications in many fields. However, effectively controlling the structural complexity of the optimization result remains a challenging issue. The structural complexity is usually characterized by the distribution and geometries of interior holes. In this work, a new approach is developed based on the graph theory and the set theory to control the number and size of interior holes of the optimized structures. The minimum distance between the edges of any two neighboring holes can also be constrained. The structural performance and the effect of the structural complexity control are well balanced by using this approach. We use three typical numerical examples to verify the effectiveness of the developed approach. The optimized structures with and without constraints on the structural complexity are quantitatively compared and analyzed. The present methodology not only enables the designer to have a direct control over the topology of the optimized structures, but also provides diverse and competitive solutions.