Series arc fault is the main cause of electrical fire in low-voltage distribution system. A fast and accurate detection system can reduce the risk of fire effectively. In this paper, series arc experiment is carried out for different kinds of electrical load. The time-domain current is analyzed by Morlet wavelet. Then, the multiscale wavelet coefficients are expressed as the coefficient matrix. We use HSV color index to map the coefficient matrix to the phase space image. Random gamma transform and random rotation are applied to data enhancement. Finally, typical deep residual network (ResNet) is established for image recognition. Training results show that this method can detect faults in real time. The accuracy of ResNet50 is 96.53% by using the data set in this paper.