We spend the majority of our lives indoors; yet, we currently lack a comprehensive understanding of how the microbial communities found in homes vary across broad geographical regions and what factors are most important in shaping the types of microorganisms found inside homes. Here, we investigated the fungal and bacterial communities found in settled dust collected from inside and outside approximately 1200 homes located across the continental US, homes that represent a broad range of home designs and span many climatic zones. Indoor and outdoor dust samples harboured distinct microbial communities, but these differences were larger for bacteria than for fungi with most indoor fungi originating outside the home. Indoor fungal communities and the distribution of potential allergens varied predictably across climate and geographical regions; where you live determines what fungi live with you inside your home. By contrast, bacterial communities in indoor dust were more strongly influenced by the number and types of occupants living in the homes. In particular, the female : male ratio and whether a house had pets had a significant influence on the types of bacteria found inside our homes highlighting that who you live with determines what bacteria are found inside your home.