State of health (SOH) estimation of lithium-ion batteries is a key but challengeable technique for the application of electric vehicles. Due to the ambiguous aging mechanisms and sensitivity to the applied conditions of lithium-ion batteries, the recognition of aging mechanisms and SOH monitoring of the battery might be difficult. A novel SOH estimation and aging mechanism identification method is presented in this paper. First, considering the dispersion effect, a fractional-order model is constructed, and the parameter identification approach is proposed, and a comparison between integer-order model and fractional-order model has been done from the prospect of predicting accuracy. Then, based on the identified open-circuit voltage, the battery aging mechanism can be analyzed by the means of an incremental capacity analysis method. Moreover, the normalized incremental capacity peak is used to estimate the remaining capacity. Finally, the robustness of the SOH estimation method is validated by batteries aged at different conditions based on the idea of cross validation, and the estimation error of the remaining capacity can be reduced within 3.1%.