The purpose of this paper is to realize the intelligent inspection of the damage degree of traffic marking. Therefore, a two-stage method based on deep learning is proposed, that is, traffic marking inpaint is performed first, and diagnosis is performed later, to ensure the data quality of digital diagnosis. Firstly, a arrow damaged traffic marking dataset is collected and created. In inpainting stage, Data-driven traffic marking inpainting model (TMIN-GAN) based on generative adversarial network (GAN) is constructed. An automatic labeling method is presented based on FE-Mask R-CNN to train TMIN-GAN. It generated the label relying on the mask generated by instance segmentation. In diagnosis stage, the damaged traffic markings and corresponding repaired images are processed using grayscale conversion, bilateral filtering, etc. Subsequently, the processed images are subjected to comparison using the Learned Perceptual Image Patch Similarity (LPIPS) evaluation metric. The experimental results demonstrate that traffic marking inpainting by the TMIN-GAN, compared by hand, reduces inpainting time from 10 seconds to milliseconds. This provides an excellent foundation for damage diagnosis. In TMIN-GAN training, the difference between PSNR value of the mask based on FE-Mask R-CNN and that of the manual annotation is only 2.35%. This demonstrates the feasibility of automatic annotation based on FE-Mask R-CNN masks. Compared by PSNR and SSIM evaluation indicators, the rationality and superiority of using LPIPS for traffic marking damage diagnosis is demonstrated and get the range of divided damage levels.