High-level penetration of intermittent renewable energy sources has introduced significant uncertainties and variabilities into modern power systems. In order to rapidly and economically respond to the changes in power system operating state, this letter proposes a real-time optimal power flow (RT-OPF) approach using Lagrangian -based deep reinforcement learning (DRL) in continuous action domain. A DRL agent to determine RT-OPF decisions is constructed and optimized using the deep deterministic policy gradient. The DRL action-value function is designed to simultaneously model RT-OPF objective and constraints. Instead of using the critic network, the deterministic gradient is derived analytically. The proposed method is tested on the IEEE 118-bus system. Compared with the state-of-the-art methods, the proposed method can achieve a high solution optimality and constraint compliance in real-time.