SUMMARY The problem addressed in this paper is the estimation of the (de)amplification of ground motion at soil sites (compared to rock sites) as a function of the intensity of the ground motion. A non‐parametric empirical approach, called the Conditional Average Estimator (CAE) method, has been used, which is different from all existing approaches. Site factors (SFs) for sites characterized with Vs30 between 180 and 360 m/s were predicted for the peak ground acceleration (PGA) and the spectral accelerations by using a combined database of recorded ground motions. Based on the results of the study, site factors for PGA and selected spectral accelerations are proposed, separately for weaker and stronger ground motions. Comparisons are made with the SFs used in two standards (Eurocode 8 and ASCE 7‐10) and with SFs proposed in the literature, including four Next Generation Attenuation (NGA) ground‐motion prediction equations. The study reveals that (i) SFs depend strongly on the ground‐motion intensity. In the case of stronger ground motions, they decrease with increasing acceleration. (ii) The SFs predicted in this study agree reasonably well with the existing SFs in the case of weak ground motion. For higher intensities of ground motion, they are generally smaller than the existing ones. Copyright © 2014 John Wiley & Sons, Ltd.