Given the Food and Drug Administration's (FDA's) acceptance of master protocol designs in recent guidance documents, the oncology field is rapidly moving to address the paradigm shift to molecular subtype focused studies. Identifying new “marker‐based” treatments requires new methodologies to address the growing demand to conduct clinical trials in smaller molecular subpopulations, identify effective treatment and marker interactions, and control for false positives. We introduce our methodology, Hierarchical Bayesian Clustering Design of Multiple Biomarker Subgroups (HCOMBS), a two‐stage umbrella Phase II design with effect size clustering and information borrowing across multiple biomarker‐treatment pairs. HCOMBS was designed to reduce required sample size, differentiate between varying effect sizes, and control for operating characteristics in the multi‐arm setting. When compared to independently applied Simon's Optimal two‐stage design, we showed through simulations that HCOMBS required less participants per treatment arm with a well‐controlled family‐wise error rate and desirable marginal power. Additionally, HCOMBS features a statistical approach that simultaneously conducts clustering and hypothesis testing in one step. We also applied the proposed design on the alliance brain metastases umbrella trial.