For the detection of uneven and irregular texture surface defects, the conventional image reconstruction is difficult because the background texture is a nonstationary signal. At the same time, because it is difficult to obtain the image feature information such as the shape and gray level of defects in advance, and the visual features of similar defects can show great dispersion, the existing detection methods that rely on target features are difficult to apply. According to the vibration theory and transient finite element-boundary element theory, a method of analyzing and detecting rail fracture defects by using rail vibration response data is proposed. By establishing the dynamic model of vibration analysis of ballastless track, the characteristics of vibration acceleration of track under the condition of complete rail fracture and 50% fracture of rail section under the periodic load excitation of nodes are studied. In this paper, an unsupervised learning surface defect detection method based on image inpainting is proposed.In this method, a small number of normal texture sample images should be used as training sets to train the network model. Then, during detection, the missing region is artificially set in the sample image, and the content of the missing region is predicted by using the network model. Finally, according to the structural similarity evaluation and residual error between the reconstructed image and the image to be tested, the defect detection and separation are realized. Experimental results show that the algorithm has strong generalization ability, can accurately segment defects of tile images with large amount of complex texture, uneven illumination and low contrast, and has strong robustness.