•The detailed error analysis between experiment and LES method is made.•The experimental and LES method are combined to analyze unsteady pressure pulsations.•The mechanism of instantaneous vortical flow structure is preliminarily revealed.•The vortical flow characteristic is the main reason that causes complex excitation frequencies. The instantaneous vortical flow structure is one of the typical flow structures inside the nuclear reactor coolant pump (RCP), which would cause the unsteady pressure pulsations, vibrations of the unit and fatigue of components. Due to high safety requirement of the RCP during the actual operation of the nuclear power plants (NPPs), revealing instantaneous nature of vortical flow structure and its pressure pulsation becomes a crucial issue for studying the internal flow mechanism of the RCP. The purpose of this study is to shed comprehensive light on the pressure pulsation and instantaneous vortical flow structure in the RCP by using the experimental method and the numerical simulation method. Based on the result of comparison with the experiment and numerical simulation, it can be considered that the LES method can better identify instantaneous nature of vortical flow structures in the low frequency band. The instantaneous vortical flow structure is attached to the impeller RBPS (rotor blade pressure surface) and transient jet wake vortex flow structures between the impeller and diffuser under the rotor-stator interaction effect are fairly obvious by Q-criterion. On the other hand, in the axial-vorticity component distribution, the rotor-stator interaction effect between the impeller and diffuser cannot be clearly revealed. From the three-dimensional structures, the main vortex structures are in the front and back cavity of the right-hand side near the discharge nozzle and the right-hand side below the discharge nozzle in the spherical casing. Meanwhile, combined with pressure spectrum, it is convinced that their unsteady characteristic is the main reason that causes complex excitation frequencies in the low frequency band of the right-hand side near the discharge nozzle.