Previous studies suggested that the Qianjiangping landslide was caused by the combined effect of rainfall and reservoir water after the first water storage of the Three Gorges Reservoir. The Sanmendong landslide with a length of 830m, an area of 2.49×105m2, an average thickness of 22m and a volume of 5.48×105m3, which is located by the Qinggan river, a tributary of the Yangtze River, is only 4km away from another famous landslide along the same river: the Qianjiangping landslide. Based on the experimental data of mechanical parameters of the landslide and displacement monitoring, a nonlinear mapping relationship between mechanical parameters of the failure mass and the displacement of landslide were established by using genetic algorithms by means of uniform design, numerical calculation and artificial neural networks. The mechanical parameters of the failure mass are obtained by the global minimization of the deviation between the calculated displacements and the monitored displacements. On this basis, the deformation mechanism of the Sanmendong landslide under the combined effect of rainfall and reservoir water level fluctuations was analyzed in detail. The results indicated that the increase of displacement of Sanmendong landslide can be primarily attributed to the combined result of reservoir water level fluctuation and rainfall. •Determination of mechanical parameters of the Sanmendong landslide was conducted.•Deformation mechanism of the Sanmendong landslide was analyzed.•The combined effect of water level flunctuations and rainfall of the reservoir was addressed.