Microgels are nanometer-to-micrometer-sized cross-linked polymer networks that swell when dispersed in a good solvent. These soft colloids have emerged as versatile building blocks of smart materials, which are distinguished by their unique ability to adapt their behavior to changes in external stimuli. Using X-ray and neutron scattering and molecular simulation methods, we systematically measured and modeled the response to crowding of compressible, deformable microgels with varying cross-link densities and internal architectures. Our experiments and simulations demonstrate that incorporating a solvent-filled cavity during chemical synthesis provides an independent means of controlling microgel swelling that complements the influence of changing cross-link density. In other words, knowledge of the content of cross-links alone cannot be used to define microgel softness, but microgel architecture is another key property that affects softness. These results are potentially important for biomedical applications, such as drug delivery and biosensing.