•Headward erosion during the breaching process of landslide dams is developed.•A dam break model with six modules based on failure mechanism is proposed.•An empirical model to estimate the breach depth of landslide dams is given.•The diversion channel is an effective measure to reduce hazards due to dam break. Landslide dams often fail shortly after the formation, resulting in huge risks to the downstream residents and properties. Fast and accurate forecasting of the breaching process and discharge hydrograph is important for a decision making to mitigate the damages. Morphological data indicate that the longitudinal length of most landslide dams is much larger than their transverse length. Additionally, headward erosion is more intense in landslide dams than manmade dams during the overtopping failure process. However, most existing models neglect this phenomenon. The objective of this study is to build up a dam-break model considering the headward erosion process, assuming that the change rate of downstream slope-angle is consistent with the rate of breach vertical erosion. And a dam-break numerical model to simulate the overtopping failure process of landslide dams considering the headward erosion was developed based on the theories of hydraulics, hydrology, soil mechanics, sediment dynamics, and etc. Nevertheless, the breach final bottom elevation, which can only be measured after the dam failure, is directly applied in currently available dam break models. To overcome this problem and for the prediction purpose of a newly formed landslide dam, we proposed an empirical model to estimate the breach final bottom elevations of high, medium and low erodibility landslide dams based on the statistical analysis of worldwide landslide dam failure cases collected recently. To verify the effectiveness of proposed dam-break numerical model, the ‘11.03′ Baige landslide dam which locates at the border of Tibet Autonomous Region and Sichuan province was selected as a case study. Results of this study show that the relative errors in the simulated peak discharge, time to the peak, and the total flood volume are 1.59%, 3.50% and 0.53%, respectively, which meet the accuracy requirements of hydrological forecasting. The relative error in the prediction of breach final bottom elevation is 4.28%, which indicates that the accuracy of the simulation is satisfactory. Finally, a sensitivity analysis indicates that the downstream slope affects the prediction of the breach and flood hydrograph significantly, and the excavation of diversion channel is an effective measure to reduce the peak discharge of dam overtopping failure.