To improve the feature extraction ability of multipath signals and reduce the dependence on environmental information, this paper studies a method that combines multipath signal feature information mining and neural network in high-dimensional phase space instead of using the received complex acoustic pressure as the input of the neural network directly. Based on the phase space reconstruction (PSR) technology, we propose a hybrid model of the self-attention-based Transformer and convolutional neural network (CNN), named PSR-TransCNN. The covariance matrix of the reconstructed data is constructed as the input of the sparse auto-encoder (SAE). A transfer learning strategy is employed to obtain the optimal weight and offset value of SAE, which serve as the initial values of the PSR-TransCNN. Real-world experimental results demonstrate that the PSR-TransCNN significantly reduces the relative root mean square error (RMSE) by 0.006 compared with the matched field processing (MFP) method and exhibits outstanding generalization and practical application potential compared to other state-of-the-art (SOTA) methods. The PSR-TransCNN achieves an RMSE of 0.046 and an accuracy of 88.75%, successfully classifying sound sources with unknown labels. The proposed method effectively reduces the multipath effects and addresses the mismatch issue in sound source ranging (SSR) within an unexplored ocean environment. Display omitted •Investigating sound source ranging in simulated and SwellEx-96 actual ocean environments.•Phase space reconstruction is an effective data preprocessing method to alleviate multipath effects.•The generalization properties of different models trained using trial data are studied.•The effects of classifying sound sources with unknown labels are analyzed.