Non-destructive testing and characterization of internal vertical cracks are critical for road maintenance by ground penetrating radar (GPR). This paper describes a mask region-based convolutional neural network (R-CNN) that automatically detects and segments small cracks in asphalt pavement at the pixel level. Simulation using Gprmax software and field detection were performed to determine the crack features in GPR images of asphalt pavement and the relationship between the width of vertical cracks and their area in GPR images. Results showed that a 0.833 precision, 0.822 F1 score, 0.701 mean intersection-over-union (mIoU) and 4.2 frames per second (FPS) were achieved on 429 GPR images (1024×1024 pixels), and the mean error between the segmented crack width and the true values was 2.33%. The research results represent a further step toward accurately detecting and characterizing internal vertical cracks in asphalt pavement •Detection frequency range for a 3D radar system was determined.•Vertical crack features in GPR images were determined.•Vertical cracks were detected using an improved Mask R-CNN model.