Medical image segmentation plays an important role in many clinical medicines, such as medical diagnosis and computer-assisted treatment. However, due to the large quality differences, variable lesion areas and their complex shapes, medical image segmentation is a very challenging task. However, most of the recent deep learning methods ignore the global context information as well as the receptive fields of pixels and do not consider the reuse of pixel features during the feature extraction stage. In this paper, we propose DGFAU-Net, an encoder–decoder structured 2D segmentation model, to overcome the shortcomings aforementioned. In the encoder, DenseNet and AtrousCNN networks are leveraged to extract image features. The DenseNet network is mainly used to achieve the reuse of pixel features, and AtrousCNN is utilized to enhance the receptive field of pixels. In the decoder, two modules, global feature attention upsample (GFAU) and pyramid pooling attention squeeze-excitation (PPASE), are proposed. GFAU combines low-level and high-level features to generate new features with richer information for improving the perceptions of global contextual information of pixels. PPASE employs a multi-scale pooling layer to enhance the pixel’s acceptance field. In addition, Focal loss is used to balance the difference between samples of the lesion and non-lesioned areas. Extensive experiments are conducted on one local dataset and two public datasets, which are the local dataset of MRI images of carotid plaque, DRIVE vessel segmentation dataset and CHASE_DB1 vessel segmentation dataset, and the experimental results demonstrate the effectiveness of our proposed model.