•BME combined with PMF are proposed for spatial quantitatively source apportionment.•1/5 of soils show different pollution degrees and Cd, Cu, As are the main pollutants.•Cd and As are the main elements causing ecological and health risks, respectively.•Industry like electroplating is the main source for As in northwest and northeast.•Agricultural activities are the main source for Cd in northwestern and north-central. Soil pollution by toxic metals has become an important environmental problem over the last several decades. Because of environmental factor variation, specific spatial patterns of pollution and sources exist. However, commonly used methods rarely take natural spatial heterogeneity into account. Positive matrix factorization and Bayesian maximum entropy models combined with specific environmental factors were proposed for quantitative source apportionment to account for spatial heterogeneity. The proposed method was implemented in a region located in southeastern China using dense samples (3627 total samples containing Cd, Hg, As, Pb, Cr, Cu, Zn and Ni data). The results showed that more than one-fifth of soils in the northwest, north-central and southeast of the study region exhibited different degrees of integrated pollution. Cd, Cu and As were the main pollutants, with proportions that exceeded the national standards of 26%, 10% and 7%, respectively. In addition, Cd was the primary element responsible for ecological risk, and As was the greatest hazard to human health. Five main pollution sources were extracted: 72.11% of the toxic metal pollution could be ascribed to anthropogenic sources, and natural sources explained the remaining 27.89%. Traffic emissions (24.31%) consistent with the major road distribution were the main source of Pb and Zn, and atmospheric deposition during the coal combustion (18.04%) distributed across the study area, except for the southeastern mountain areas, was the main source of Hg. Agricultural activities (16.81%) distributed mainly in the north-central regions contributed the most to Cd and Cu, and industrial activities (12.95%) clustered in the northwestern areas contributed the most to As. In addition, natural sources were closely linked to Ni and Cr in the southeastern mountain areas.