Display omitted •Two-stage data augmentation method is developed, improves deep learning models' performance.•Semantic segmentation in SEM fractographic analysis marks new phase in this field.•SegFormer accurately segments complex fracture patterns, outperforming UNet, DeepLabV3+. Fractographic analysis poses a significant challenge for field researchers without specialized training in fractography. To address this issue, this study introduces a comprehensive integrated workflow that encapsulates the entire process from dataset preparation and data enhancement to leveraging the SegFormer model for deep learning-driven semantic segmentation. An extensive collection of fractography images is formulated and augmented to train the SegFormer model, enabling precise semantic segmentation of morphological fracture regions including cleavage, ductile, dimple, fatigue striations, and others. To accommodate the demanding SEM imaging conditions which frequently include distortions, noise, and aberrations, we developed a two-stage method with diverse data augmentation strategies. This method resulted in a robust model demonstrating exceptional performance, as evidenced by a high mean Intersection over Union (mIOU) score of 59.7 and other metrics. The findings validate the potential of deep learning techniques, particularly the SegFormer model's efficacy in morphological fractography image segmentation for the first time. Our work offers a cost-effective, and efficient alternative deep learning approach to traditional experimental fracture analysis, thereby expanding opportunities for a broader range of professionals in the engineering domain.