•The spectral indices improve the performance of PTFs in estimating of SM.•Soil moisture is more accurately estimated by the combined NDWI and SWCI indices.•The accuracy of the RF for estimating the SM is better than the MLR-based PTFs.•Amount of clay in all functions has the most impact on estimating of SM. To estimate the surface soil moisture (SM) using a combination of new spectral indices and methods of Random Forrest (RF) and Multiple Linear Regression (MLR), 11 pedotransfer functions (PTF1-11) were developed by combining basic soil properties (clay, silt/sand, and bulk density) and spectral indices of Sentinel-2 satellite. In this study, 124 surface soil samples were randomly taken from three regions including Telo in Tehran province, Ivaneki in Semnan province, and Borujerd in Lorestan province, Iran. The results showed that the accuracy of the RF method was considerably higher compared to the MLR method. The SM was better estimated using water spectral indices (such as Normalized Difference Water Index (NDWI) and Surface Water Capacity Index (SWCI)), along with the basic properties of soil as inputs of PTF7. In the training and testing steps, the Root Mean Square Error (RMSE) decreased from 0.041 and 0.05 (cm3 cm−3) in PTF1 to 0.028 and 0.039 in PTF7, respectively. The average values of RMSE, Akaike Information Criterion (AIC), coefficient of determination (R2) and Relative Improvement (RI) of the RF method were 0.028 (cm3 cm−3), −559, 0.79 and 0.001, and 0.038 (cm3 cm−3), −279, 0.73 and −0.006 for the training and testing steps of all PTFs, respectively. While, these values for the MLR method were 0.032 (cm3 cm−3), −542, 0.69 and 0.0003, and 0.043 (cm3 cm−3), −269, 0.63 and 0.002 for the training and testing steps, respectively. Due to the low values of MBE, it was possible to disregard the overestimation of the results. The evaluation of results for predictor importance indicated that among the basic properties, clay percent has a significant effect on estimation of SM. These results show that spectral indices alone are not suitable estimators for SM estimation. It suggests using basic soil properties and spectral indices to estimate the SM.