•Characterizing uniaxial tension, plane strain tension, and shear stress states of martensitic steel.•Adjusting yield function exponent and size of finite element model for calibration of fracture models.•Predicting hole expansion fracture of martensitic steel accurately with GTN-shear fracture model.•Predicting bending fracture of martensitic steel accurately with Hosford-Coulomb fracture model. In this study, fracture of martensitic steel in hole expansion and stretch bending was predicted by finite element analysis with shell elements. Various mechanical experiments were conducted to characterize mechanical properties related to fracture as well as plasticity. Then, Swift-modified Voce and Yld2000-2d well-described plasticity properties of martensitic steel. For fracture modeling, Hosford-Coulomb (HC) and shear-modified Gurson-Tvergaard-Needleman (GTN-shear) models were selected and calibrated with load–displacement curves of three fracture tests. The hybrid numerical-experimental method was used for the HC model and inverse identification with finite element model updating was used for the GTN-shear model. In calibration, an exponent of yield function and mesh size were adjusted to match the strength of plane strain state and to maintain consistency of mesh size. After calibration, both fracture models were applied to numerical simulations of conical hole expansion and stretch bending tests. The GTN-shear model predicted well fracture in the conical hole expansion test but not in the stretch bending. The prediction results of HC model were vice versa. Through triaxiality analysis, these differences between the two models may result from a non-linear loading path and description of localized necking.