The traditional linear statistical forecast method is often used to address the large inter-annual variation and nonlinear characteristics of summer drought and flood trends in Guangxi, but the forecast accuracy is low. In this study, the inter-annual increment of the average precipitation in August was used to forecast drought and flood trends. By calculating the correlation factor between the forecast and the previous 500 hPa monthly average height field, 81 preliminary forecast factors of the early monthly average circulation field were obtained. First, the random forest algorithm was employed to calculate and rank the importance of the 81 prediction factors and predictors, and the six most important characteristic variable factors were selected as the input of the prediction model of a deep learning long short-term memory (LSTM) network. Second, an attention mechanism was used to provide different attention values to the input variables of the model. Third, a prediction model for the inter-annual increment of average summer precipitation in Guangxi based on random forest and attention mechanism LSTM network (RF-LSTM-Attention) was established. When this prediction model was adopted to predict the average summer precipitation of the eight-year return sample in Guangxi (June–August) from 2013 to 2020, the average absolute percentage error of the model was 9.49%. The relative error percentage in eight years was greater than 30% (30.74%) in only one year (2014) and greater than 15% (19.65%) in another year (2013). The relative error of prediction in the six other years, especially in the years with maximum and minimum precipitation in the eight-year return sample, did not exceed 15%. The study further compared the prediction results of the same eight-year return sample of the prediction model with the LSTM model without the attention mechanism (the introduction factors of the model were the same). Results showed that the average absolute percentage prediction error was 10.88%, the relative error percentage in the same eight-year sample exceeded 30% (43.18%) only in 2014, and the prediction error was greater than that of the RF-LSTM-Attention model. Furthermore, the eight-year return sample forecast error of the forecast model was compared with the forecast results of the linear stepwise regression forecast model (5, 6, and 7 predictor variables were selected). The RF-LSTM-Attention model was approximately doubled and showed better forecast accuracy in qualitative and quantitative forecasting of the average summer precipitation in Guangxi.