Single image fog removal is important for surveillance applications, and, recently, many defogging methods have been proposed. Due to the adverse atmospheric conditions, the scattering properties of foggy images depend on not only the depth information of scene but also the atmospheric aerosol model, which has a more prominent influence on illumination in a fog scene than that in a haze scene. However, the recent defogging methods confuse haze and fog, and they fail to consider fully the scattering properties. Thus, these methods are not sufficient to remove fog effects, especially for images in maritime surveillance. Therefore, this paper proposes a single image defogging method for visual maritime surveillance. First, a comprehensive scattering model is proposed to formulate a fog image in the glow-shaped environmental illumination. Then, an illumination decomposition algorithm is proposed to eliminate the glow effect on the airlight radiance and recover a fog layer, in which the objects at the infinite distance have uniform luminance. Second, a transmission-map estimation based on the non-local haze-lines prior is utilized to constrain the transmission map into a reasonable range for the input fog image. Finally, the proposed illumination compensation algorithm enables the defogging image to preserve the natural illumination information of the input image. In addition, a fog image dataset is established for the visual maritime surveillance. The experimental results based on the established dataset demonstrate that the proposed method can outperform the state-of-the-art methods in terms of both the subjective and objective evaluation criteria. Moreover, the proposed method can effectively remove fog and maintain naturalness for fog images.