In order to test the applicability of the existing correlations of the flow and heat transfer performances (FHTPs) of the high-efficiency fins to the plate-fin heat exchanger (PFHE) for hydrogen liquefaction, a numerical model is established to investigate the FHTPs of hydrogen in the serrated fin (SF) and wavy fin (WF). The results show that the existing correlations can be applied to the conditions of the low-pressure hydrogen in 30–300 K and the high-pressure hydrogen in 40–300 K. However, the FHTPs of the high-pressure hydrogen in 30–40 K are obviously different from the existing correlations and need the new correlations for calculation. The high-pressure and low-temperature condition leads the low velocity in the fin channel, resulting in that the serrated locations of the SF and the wave crest and wave trough of the WF are the keys to affect the FHTPs. On the basis, a series of data points reflecting the relationships between the FHTPs and the operational and structural parameters are extracted from the response surfaces to establish the new correlations. For the SF, the new correlations of the Colburn heat transfer factor (j) and the Fanning friction factor (f) have a good consistency with the original data, with the fitting goodness of 96.4% and 92.4% and the average relative deviations of 8.3% and 9.7% respectively. And the fitting goodness and average relative deviations of the new correlations of the WF are 96.5% and 99.0%, and 7.9% and 9.3% respectively. The new correlations of the SF and WF can be used to guide the design of the PFHE in 30–40 K for hydrogen liquefaction. Display omitted •Applicability of existing correlations of FHTPs of SF and WF for hydrogen liquefaction is tested.•Hydrogen (low-pressure in 30–300 K, high-pressure in 40–300 K) can adopt existing correlations.•Hydrogen (high-pressure in 30–40 K) needs new correlations for calculation.•New correlations of SF and WF for hydrogen (high-pressure in 30–40 K) are obtained.