The effect of Ni on the evolutions of cementite and mechanical properties in quenched and tempered Cr–Ni–Mo steel was investigated. The results of tensile and Charpy impact tests showed that an increase in the Ni content was more effective in improving the low-temperature impact toughness than in increasing yield and tensile strengths. Experimental observations revealed that Ni was beneficial to cementite refinement but had almost no impact on refining the sizes of both the quenched martensite structure and tempered effective grain size. Further, energy dispersive spectrometry (EDS) and thermodynamic studies on cementite and adjacent ferrite indicated that Ni partitioning resulted in the formation of a Ni-rich layer at the ferrite/cementite interface and thereby retarded cementite growth. In order to clarify the different effects of Ni on the impact toughness and yield and tensile strengths, the role of cementite refinement in strengthening and toughening mechanisms was analyzed in detail. Display omitted •The variation of Ni content had obvious influence on the characteristics of cementite precipitation.•The growth of cementite during tempering was retarded due to the partitioning of Ni between ferrite and cementite.•The strengthening and toughening mechanisms were evaluated in terms of cementite refinement.