Aiming at a series of problems such as detection accuracy, calculation blocking, display delay, and so on in the ship detection of surveillance video, an improved YOLOv5 algorithm is proposed in this paper. First, to improve the detection performance, it is proposed to optimize the anchor box algorithm in the YOLOv5 network according to the ship target characteristics. Then, the t‐SNE algorithm is used to reduce and visualize the data set label information and perform weighted analysis on the processed features for low‐dimensional data. The mapped kernel k‐means clustering algorithm adaptively selects a more appropriate anchor box and considers the detection performance of large and small ship targets. Secondly, to improve the problem of computational blocking and delay, the BN scaling factor γ is used to compress the YOLOv5 network, so that the model can be reduced without reducing the detection performance. The optimized YOLOv5 framework is trained on the self‐integrated data set. The accuracy of the algorithm is increased by 2.34%, and the ship detection speed reaches 98 fps and 20 fps in the server environment and the low computing power version (Jetson nano), respectively.