Unmanned aerial vehicle (UAV) object tracking has been widely used in military and civilian tasks and is very challenging due to factors such as aerial view, low resolution (LR), background clutter (BC), and small scale. Currently, many correlation filter (CF) methods for UAV object tracking introduce predefined regularization terms to improve the discriminative power. However, the predefined spatial constraints neither exploit the changing foreground and background information during tracking nor adapt to complex video attributes, such as BC, illumination variation (IV), and viewpoint change (VC). In this work, a novel CF tracking model is proposed to learn the spatial regularization CFs with the Hilbert-Schmidt independence criterion (HSIC_SRCF) in the reproducing kernel Hilbert space (RKHS) for UAV tracking. First, HSIC _SRCF learns the high confidence spatial weight coefficient for the filters, which encourages the filters to focus on the more reliable region. Second, the alternative direction method of multipliers (ADMM) can be used to optimize the proposed model, and the subproblems of the model have closed-form solutions. Third, the model update scheme uses RKHS and the peak-to-sidelobe ratio (PSR) to obtain the confidence of the object bounding box and reduce the effect of background noise for the UAV object. The ablation study shows that the proposed approach achieves a precision of 0.699 and a success rate of 0.607, which are 2.8% higher and 2.2% higher, respectively, than the baseline AutoTrack on the UAV123@10fps. Experimental results on four popular UAV datasets, including VisDrone2018-test-dev, UAV123@10fps, UAV123, and UAVDT, show that our approach performs favorably compared with current state-of-the-art methods.