In recent years, it has become a trend to analyze shoreline changes through satellite images in coastal engineering research. The results of sea–land segmentation are very important for shoreline detection. CoastSat is a time-series shoreline detection system that uses an artificial neural network (ANN) on sea–land segmentation. However, the method of CoastSat only uses the spectral features of a single pixel and ignores the local relationships of adjacent pixels. This impedes optimal category prediction, particularly considering interference by climate features such as clouds, shadows, and waves. It is easy to cause the classifier to be disturbed in the classification results, resulting in classification errors. To solve the problem of misclassification of sea–land segmentation caused by climate interference, this paper applies HED-UNet to the image dataset obtained from CoastSat and learns the relationship between adjacent pixels by training the deep network architecture, thereby improving the results of erroneous sea–land segmentation due to climate disturbances. By using different optimizers and loss functions in the HED-Unet model, the experiment verifies that Adam + Focal loss has the best performance. The results also show that the deep learning model, HED-Unet, can effectively improve the accuracy of the sea–land segmentation to 97% in a situation with interference from atmospheric factors such as clouds and waves.