In the downlink transmission, the legitimate users are also vulnerable to malicious jamming when the desired signal is launched by multiple antenna array. Then, this paper investigates an anti-jamming defense criteria through a deep reinforcement learning (DRL) to suppress the jamming at transmitting information. Firstly, a dual-polarized antenna array is designed to achieve the global positioning system (GPS) signals model, and the information of designed model can be directly used into the anti-jamming criteria more general. Then, we set up the preprocessing framework based on a cyclic convolutional neural network (CNN), which is used to preprocesses the objective function by approximating the evaluated variable to the target value, i.e., the dynamic objective function of beamforming. Subsequently, a DRL algorithm based on the deep Q-network is proposed to achieve the optical null-steering strategy in dynamic jamming environment. Finally, some simulation results validate the proposed DRL can effectively improve both the system sum-rate and combating dynamic jamming level compared with other approaches.