Accumulation of potentially toxic elements (PTEs) in agricultural soils has become an increasingly global issue driven by industrialization. A credible knowledge of their distribution and diverse sources in soils is essential to developing effective measures of identifying accumulation of PTEs in rural and periurban environments. However, the assessment of PTE pollution levels and discrimination of anthropogenic and natural sources remain a serious challenge. In China, most studies are focused on highly industrialized and/or urbanized regions, while limited attention has been given to agricultural areas where diffuse source contamination prevails. Therefore, a large dataset of 5207 surface soil samples (1 × 1 km) and 1311 subsoil samples (2 × 2 km) were collected from an agriculturally dominated region in eastern China. Soil total concentrations of As, Cd, Cr, Cu, Hg, Ni, Pb and Zn were analyzed along with additional edaphic variables relevant to PTE accumulation in soils (e.g., pH, SOC). Concentrations of Cr, Ni, Pb and Zn for all sites were lower than those of the risk-screening values. However, elevated concentrations of Cd and Hg observed in surface soils were associated with anthropogenic activities. Land use pattern had a significant impact on the spatial variation of PTEs. Elevated levels of Cd were uniquely associated with high phosphorus and soil organic matter (SOM) contents, and elevated Hg was associated with coal-fired power plants and historical application of fertilizer and Hg-containing pesticides. The data presented herein indicated that geogenic process likely caused a net depletion of As, Cr, Cu, Ni, Pb and Zn in topsoil, despite surface deposition from anthropogenic sources. The result of this study revealed that using subsoil concentrations of PTEs to establish background or reference concentrations might lead to an erroneous assessment of pollution levels in surface soils. Display omitted •A high-density sampling of topsoil (n = 5207, 1 × 1 km) and subsoil (n = 1311, 2 × 2 km) was conducted in agricultural soils of Anhui, China.•Only 2 sites for topsoil As, 23 sites for Cd, 3 sites for Cu and 7 sites for Hg had concentrations exceeding threshold values.•Land use pattern had a significant impact on the spatial variation of potentially toxic elements (PTEs).•Using subsoil PTE concentrations to establish background levels may lead to an erroneous assessment of pollution levels. Using subsoil concentrations of PTEs to establish background or reference concentrations might lead to an erroneous assessment of pollution levels in surface soils.