Insulator fault detection is one of the essential tasks for high-voltage transmission lines’ intelligent inspection. In this study, a modified model based on You Only Look Once (YOLO) is proposed for detecting insulator faults in aerial images with a complex background. Firstly, aerial images with one fault or multiple faults are collected in diverse scenes, and then a novel dataset is established. Secondly, to increase feature reuse and propagation in the low-resolution feature layers, a Cross Stage Partial Dense YOLO (CSPD-YOLO) model is proposed based on YOLO-v3 and the Cross Stage Partial Network. The feature pyramid network and improved loss function are adopted to the CSPD-YOLO model, improving the accuracy of insulator fault detection. Finally, the proposed CSPD-YOLO model and compared models are trained and tested on the established dataset. The average precision of CSPD-YOLO model is 4.9% and 1.8% higher than that of YOLO-v3 and YOLO-v4, and the running time of CSPD-YOLO (0.011 s) model is slightly longer than that of YOLO-v3 (0.01 s) and YOLO-v4 (0.01 s). Compared with the excellent object detection models YOLO-v3 and YOLO-v4, the experimental results and analysis demonstrate that the proposed CSPD-YOLO model performs better in insulator fault detection from high-voltage transmission lines with a complex background.