A recognition method based on sparse representation was proposed to solve the problems in subgrade detection with ground penetrating radar (GPR), such as massive data, time-frequency and difference in experience. Based on the propagation characteristics of electromagnetic waves and radar images in subgrade defects, the phase axis of radar images of railway subgrade defects was different from that of normal subgrades. The sparse representation of railway subgrade defects was studied from the aspects of the time domain, and time-frequency domain with compressive sensing theory. The high-frequency horizontal details of the radar images were obtained to reduce redundant data fetching. The demixing points, energy, and variance per block were obtained as time domain eigenvalues, and the high-frequency horizontal details of the radar images and the energy spectrum of the wavelet multiscale spatial were acquired; thus, the feature vectors of the radar signal were extracted by sparse representation. Based on FCM-GANN (fuzzy C-means and generalized regression neural network), rapid recognition of the ballasted railway was realized. Experimental results indicated that the redundancy of data was reduced, and the accuracy of identification was greatly increased.