This study proposes a novel reinforcement learning (RL) framework, professional trader RL (Pro Trader RL), which mimics the decision-making patterns and trading philosophy of professional traders in stock trading. By exploiting the characteristics of RL, the framework aims to learn efficient trading strategies while mimicking the trading philosophy and risk management methods of professional traders. The framework takes into account the complex nature of the stock market and presents an integrated approach to RL, from data pre-processing to buying, selling and stop-loss. Pro Trader RL consists of four main modules. Data Preprocessing, Buy Knowledge RL, Sell Knowledge RL and Stop Loss Rule, each of which plays the role of professional traders knowledge. The results of three experiments show that the framework achieves high returns and Sharpe ratio regardless of market conditions and has stable performance with low maximum drawdown (MDD), which is superior to the state-of-the-art research. The proposed framework provides a novel approach to applying RL to the stock market and is expected to be useful and applicable in real-world trading settings.