The addition of rain stripes will reduce the processing performance of computer vision algorithms with video or images as input, so it is necessary to remove rain stripes from the video or images. A novel multi-stage recursive image deraining network (MRidNet) based on deep learning is proposed in this paper. Considering that rain images contain rain lines of different sizes and directions, encoder-decoder network is used to downsample images at different scales and remove rain lines of different intensities accordingly. In order to enhance the effect of rain removal, a squeeze-and-excitation module is combined to highlight the important feature information. Since the rain grain layer is difficult to be removed at one time, MRidNet adopts the cycle mechanism and uses the long short-term memory network to guide the rain removal before and after the cycle, so as to improve the performance of the model. In general, MRidNet is composed of a repeated expansion of a basic deraining network, which consists of a squeeze-and-excitation module, a long short-term memory module and an encoder-decoder structure. The negative mean structural similarity index is used as the loss function to effectively train the MRidNet. The experimental results show that our proposed MRidNet performs excellently on both synthetic and real rainy images, it can adapt to more complex rain scenes, achieve efficient rain removal, and the restore image background more clearly.