•Marketplace-based e-commerce dominates current social commerce economy.•The institution-based trust is vital for social commerce marketplaces.•Two macro-level factors of the institution-based trust are uncovered.•A high order construct – perceived social presence – is introduced.•It discloses a trust-building mechanism in social commerce marketplaces. Trust has been seen as the foundation of e-commerce. While the interpersonal trust has gain a lot attention in literature, institution-based trust has been studied only infrequently. However, with the significance of the marketplace-based e-commerce, more attention should be paid to institution-based trust, particularly trust at the marketplace level. This study focuses on a specific type institution-based trust associated with marketplaces, termed as trust in marketplace. Based upon the institutional theory and the social presence theory, two categories of marketplace-driven factors, the institutional and social factors, are then proposed as antecedents of trust in marketplace in the new context of social commerce marketplaces. Two formative high-order constructs, perceived effectiveness of institutional structures and perceived social presence, are developed to account for the effects of these two sources on trust in marketplace, which in turn leads to transaction intention in social commerce marketplaces. The research model is examined via the free simulation experimental method where seven real social commerce marketplaces are duplicated. The findings suggest the positive impacts of both constructs in shaping institution-based trust that leads to social commerce marketplace transaction intention. This study then offers insightful understandings to the institutional trust building mechanism in the recent phenomenon of social commerce, as well as introduces the important but neglected social perspective to e-commerce research.