Meta-analysis is pervasively used to combine multiple genome-wide association studies (GWASs). Fine-mapping of meta-analysis studies is typically performed as in a single-cohort study. Here, we first demonstrate that heterogeneity (e.g., of sample size, phenotyping, imputation) hurts calibration of meta-analysis fine-mapping. We propose a summary statistics-based quality-control (QC) method, suspicious loci analysis of meta-analysis summary statistics (SLALOM), that identifies suspicious loci for meta-analysis fine-mapping by detecting outliers in association statistics. We validate SLALOM in simulations and the GWAS Catalog. Applying SLALOM to 14 meta-analyses from the Global Biobank Meta-analysis Initiative (GBMI), we find that 67% of loci show suspicious patterns that call into question fine-mapping accuracy. These predicted suspicious loci are significantly depleted for having nonsynonymous variants as lead variant (2.7×; Fisher’s exact p = 7.3 × 10−4). We find limited evidence of fine-mapping improvement in the GBMI meta-analyses compared with individual biobanks. We urge extreme caution when interpreting fine-mapping results from meta-analysis of heterogeneous cohorts. Display omitted •Extensive simulation of meta-analyses to show substantial fine-mapping miscalibration•SLALOM, a novel method that identifies suspicious loci for meta-analysis fine-mapping•Significant depletion of likely causal variants in SLALOM-predicted suspicious loci•Widespread suspicious loci for fine-mapping in current meta-analysis summary statistics Genome-wide associations studies (GWASs), often performed as meta-analyses, have identified tens of thousands of disease-associated loci. Kanai et al. demonstrate via large-scale simulations and real data analysis that standard tools for pinpointing the causal variants underlying these associations can produce unreliable results when applied to GWAS meta-analyses.