The efficient operation of the air supply system, particularly the air compressor, is crucial in ensuring the performance of the proton exchange membrane fuel cells (PEMFCs). Nevertheless, its high parasitic power consumption is the main reason for the efficiency decline of the PEMFC. While exhaust gas energy recovery is a viable approach to enhance system efficiency, conventional exhaust gas recovery systems are not well-suited for PEMFC and the cathode exhaust gas is difficult to monitor in real-time. Therefore, this study proposes a sensorless method based on reinforcement learning for energy recovery of the exhaust gas turbine air compressor (EGTAC) in the PEMFC. Firstly, the air supply system with EGTAC and stack model are established. Subsequently, the relationship between the exhaust gas energy and the working performance of both the EGTAC and the PEMFC is elucidated. Additionally, a method based on a state observer is devised to estimate the characteristics of the exhaust gas in a PEMFC. Finally, compared to the model predictive control (MPC), this method enhances the EGTAC exhaust gas recovery rate by 19.1% and the fuel cell system efficiency by 3.7%. The optimization differs by a maximum of 6.5% compared to the control with sensors.