This section presents the ionospheric modeling characteristics of the Klobuchar, RIM and LSTM forecast models, and their performance in the SPP solution.
To evaluate the performance of the above LSTM forecast model in the SPP solution, GNSS measurements collected from CMONOC stations from 1 to 17 January, 2014 were used for statistical analysis. The ionosphere shows complex spatial variations with latitude and longitude. To verify the adaptability and forecast accuracy of the model at different spatial locations, GPS stations were divided into three regions: low (0°~30°), middle (30°~45°), and high latitude (45°~60°), with 12 stations selected in total. The GNSS station locations and detailed information are shown in Fig. 3 and Table 4, including four stations each at low latitude, mid-latitude and high latitude.
4.1 Performance of the LSTM forecast model
First, 48 LSTM forecast models are built for each site (saving the optimal weights) using the split dataset as the parameter input. The ionospheric TEC forecast models are trained to obtain the ionospheric TEC forecast values for different periods, and the results are evaluated using the above criterion. The results of the LSTM forecast model are shown in Table 4.
Table 4
The results of the LSTM forecast model
Classification | Station name | latitude (°N) | longitude (°E) | Modeling accuracy | Forecast accuracy (TECu) |
MAE | RMSE | MAE | RMSE |
Low latitude | HISY | 18.236 | 109.531 | 5.983 | 2.918 | 2.918 | 2.245 |
LALB | 19.898 | 102.165 | 5.385 | 3.593 | 2.601 | 2.030 |
KMIN | 25.030 | 102.798 | 5.385 | 2.601 | 2.867 | 2.279 |
CQCS | 29.905 | 107.232 | 3.831 | 1.520 | 1.520 | 1.270 |
Middle latitude | AHBB | 32.905 | 117.296 | 2.3627 | 2.1131 | 1.058 | 0.802 |
GSDX | 35.554 | 104.605 | 2.379 | 2.073 | 1.047 | 0.829 |
XJBC | 39.814 | 78.770 | 1.856 | 1.268 | 0.886 | 0.633 |
BJGB | 40.692 | 117.158 | 1.989 | 1.292 | 0.884 | 0.642 |
High latitudes | XJFY | 46.999 | 89.539 | 1.9954 | 1.1061 | 0.832 | 0.637 |
HLFY | 48.367 | 134.277 | 1.3936 | 1.0266 | 0.844 | 0.596 |
NMER | 50.576 | 123.727 | 1.915 | 1.167 | 0.971 | 0.655 |
HLMH | 52.975 | 122.513 | 1.522 | 1.012 | 0.828 | 0.618 |
Table 4 shows the MAE and RMSE results for the LSTM forecast model for 12 different stations. The forecast MAE values range from 0.8 to 3 TECu, with the HISY station showing lower accuracy, while the HLMH station achieved the highest accuracy among the other stations. The forecast MAE and RMSE of the LSTM forecast model are 0.8 to 3 TECu, and the forecast accuracy of the mid-latitude and high latitude stations can be within 1 TECu. Modeling and forecast accuracy is positively correlated with latitude.
For a more detailed representation of the forecast accuracy at each 30-minute interval, the error bars for each period are plotted by combining the forecast MAE and RMSE values, as shown in Fig. 4. It can be seen that the range of error bars decreases with increasing latitude, especially in the period 04:00 UTC-10:00 UTC (12:00 LT-18:00 LT). It can be found that the LSTM forecast model is different from the temporal LSTM-CNN model (Ruwali et al., 2021), in that the RMSE increases as the number of forecasted hours increases. Because it is modeled separately in periods, the RMSE values are independent in different periods and are only related to the ionospheric characteristics in that period, and the errors do not accumulate. As can be seen in Fig. 4 (a), the errors of the low latitude stations are within 5 TECu for most of the periods. The forecast errors are larger in the 06:00 UTC-10:00 UTC (14:00 LT-18:00 LT) period, especially for the KMIN station, and the reason for this can be found in Fig. 5 (c) later, i.e., the presence of anomalous discrete values with large deviations of up to 30 TECu, a phenomenon that may be related to the low latitude equatorial ionospheric anomaly (EIA) (Song et al., 2018). From Fig. 4 (b), it is found that the GSDX station has a large error in the period 04:00 UTC-6:00 UTC (12:00 LT-14:00 LT), with a maximum of more than 5 TECu, which is also found in Fig. 5 (f) because of the presence of continuous anomalous discrete values.
Figure 5 shows the forecast results for 12 stations at low, mid and high latitudes. Among them, the GNSS measurements are selected to be within 3° difference in longitude and within 1° difference in latitude from the station. As can be seen from Fig. 5, i) the TEC tends to decrease with increasing latitude. Among them, the forecast values of the LSTM forecast model is in the best agreement with the GNSS measurements, which can better forecast the ionospheric TEC at low latitudes; ii) the RIM data are generally in good agreement with the GNSS measurements. However, the TEC values are overestimated in the period of 04:00 UTC-08:00 UTC, the RIM values are closer to the GNSS measurements than the Klobuchar model values; iii) the Klobuchar model only captures the temporal trend of the ionosphere, which is somewhat underestimated at low latitudes and overestimated at mid-latitudes and high latitudes. In the next subsection, the performance of the LSTM forecast model, the Klobuchar model and the RIM model in the SPP solution is comprehensively evaluated and the spatial and temporal characteristics are analyzed.
4.2 Performance in the SPP solution
The ionospheric delay is considered as one of the largest error sources in single-frequency position, and its model accuracy can be indirectly reflected in the accuracy of the SPP solution. The experimental platform used is based on the GINav program developed by the NASG Key Laboratory of Land Environment and Disaster Monitoring, China University of Mining and Technology (K. Chen et al., 2021). The data required for the experiments are downloaded from the official FTP of the IGS Analysis Center CDDIS (crustal dynamics data information system) (https://cddis.nasa.gov/). The data from CMONOC stations distributed in high, mid and low latitudes are processed separately. Using the position results of ionospheric reference data as a comparison, this paper compares the performance of ionospheric correction by the LSTM forecast model, Klobuchar model and RIM model in SPP solution, and evaluates the feasibility of the LSTM forecast model. For the position accuracy analysis, the SPP error is obtained by subtracting the exact coordinates of each site from the SPP solution. The reference position is determined as the static PPP solution of the last epoch.
Figure 6 shows SPP errors in the east (E), north (N) and up (U) directions generated by the three models and the reference values at the low latitude stations. On the whole, the positioning error of the three ionospheric models in the E direction is the smallest, within ± 3 m, the U direction is the largest, within ± 20 m. The SPP errors occur mainly in the vertical (U) direction, and the position accuracy is in the meter level, while in the horizontal (N and E) direction, it can reach the sub-meter level. In terms of individual models, the SPP result with LSTM forecast model correction has the best accuracy, especially in the U direction. The difference between the LSTM forecast model position accuracy and the reference data in the E direction is not much, around 0.1 m, and in the N direction, the difference is around 0.2-0.3m, and in the U direction, except for LALB, the difference is 0.516 m. The difference in the U direction is about 0.2m except for LALB, so the overall position performance of the LSTM forecast model is better in the low latitude region. The SPP error using the Klobuchar model is the largest, with RMSE of about 1 m, 1–4 m, and 3–5 m in the E, N, and U directions, respectively. In terms of stations, the LSTM forecast model and the RIM model have better positioning accuracy at KMIN and CQCS stations relative to HISY and LALB stations, especially in the N direction and during 12:00 to 20:00 local time (LT) (corresponding to 4:00 to 14:00 UTC). One reason is that the parameters of GNSS Klobuchar are calculated based on an empirical model, while RIM is generated by processing GNSS measurements from the regional monitoring station network, which has higher accuracy. Another reason is that the RIM is updated every 1 hour, which is more frequent than GPS Klobuchar, and this will help to describe the ionospheric characteristics accurately. The RIM model is generally corrected at the HISY and LALB stations, and even the results are comparable to the Klobuchar model at the HISY station. Since the latitude coverage of RIM data is from 15.0°N to 55.0°N, while the latitude of the HISY station is 18.236°N, the interpolation accuracy of the IPP is limited.
Figure 7 shows the SPP error at the mid-latitude stations. It can be seen that the SPP result in mid-latitudes is similar to that in low latitudes, with the smallest error fluctuations in the E direction and the largest in the U direction. In the E and N directions, the position results based on the RIM model are better than the LSTM forecast model, which is because many CMONOC stations are set up in the mid-latitude region. In the U direction, the LSTM model is better than the RIM, and in the GSDX and XJBC, it is slightly stronger than the reference data, which is because the GNSS measurements have some anomalies as can be seen in Fig. 5 (f). Overall, the position accuracy of the LSTM forecast model and the reference data are comparable. In the E and U directions, the difference between the LSTM forecast model and the reference is not much, about 0.1 m, and in the N direction, about 0.1–0.2 m. The position accuracy of the Klobuchar model in the E and N directions is slightly lower than that of the RIM and LSTM forecast models, which is consistent with the conclusion of the paper that the GPS Klobuchar model is more suitable for mid-latitude regions (Cai et al., 2017).
Figure 8 shows SPP errors at the high latitude stations. It can be seen that the LSTM forecast model has the best position accuracy, even slightly better than the reference data, because the LSTM forecast model results are smoother compared with the GNSS measurement data, which are susceptible to some anomalies thus affecting the position accuracy. In the E and N directions, the RIM position accuracy is poor, especially for HLFY and HLMH, for reasons consistent with the HISY stations at low latitudes, limited by the spatial coverage of the RIM while the Klobuchar model has better position accuracy, but decreases compared to mid-latitudes.
To more visually assess the SPP position accuracy when using different models at different stations, the RMSE of the positioning for each model at 12 stations is shown in Fig. 9. At low latitudes, the RMSE of the Klobuchar model SPP error is the largest, especially in the N direction, which is 2–3 times higher than that of the LSTM forecast model. In the mid-latitude region, the difference in position accuracy of each model is relatively reduced in the E and N directions, and the RMSE is all around 1m, but in the U direction, the Klobuchar model error is twice as high as the other models. In the high latitude region, the LSTM forecast model has much better positioning accuracy than Klobuchar and RIM, and is comparable to the accuracy of the reference data. Overall, the LSTM forecast model has the best positioning accuracy, the RIM is the second best, and the Klobuchar model is the worst. However, for mid and high latitudes, the horizontal accuracy of the Klobuchar model can reach the sub-meter level. If the user only requires meter level horizontal positioning accuracy and does not care about vertical positioning accuracy, the Klobuchar model can achieve better accuracy.
Table 5
Statistics of 3D position errors of ionospheric models
Classification | Station name | 3D(m) | Percentage of corrections relative to Reference (%) |
Reference | Klobuchar | RIM | LSTM | Klobuchar | RIM | LSTM |
Low latitude | HISY | 3.02 | 3.93 | 3.66 | 3.33 | 76.84 | 82.51 | 90.69 |
LALB | 3.34 | 4.98 | 4.53 | 3.92 | 67.07 | 73.73 | 85.2 |
KMIN | 3.23 | 5.88 | 3.97 | 3.61 | 54.93 | 81.36 | 89.47 |
CQCS | 1.97 | 4.56 | 2.61 | 2.15 | 43.2 | 75.48 | 91.63 |
Middle latitude | AHBB | 1.95 | 4.58 | 2.36 | 2.12 | 42.58 | 82.63 | 91.98 |
GSDX | 1.99 | 4.48 | 2.31 | 2.11 | 44.42 | 86.15 | 94.31 |
XJBC | 2.85 | 4.59 | 2.94 | 2.82 | 62.09 | 96.94 | 101.06 |
BJGB | 1.82 | 4.45 | 1.93 | 1.83 | 40.9 | 94.3 | 99.45 |
High latitude | XJFY | 2.91 | 4.63 | 2.89 | 2.9 | 62.85 | 100.69 | 100.34 |
HLFY | 2.22 | 4.46 | 4.22 | 2.18 | 49.78 | 52.61 | 101.83 |
NMER | 2.42 | 4.57 | 2.87 | 2.37 | 52.95 | 84.32 | 102.11 |
HLMH | 2.48 | 4.59 | 4.39 | 2.47 | 54.03 | 56.49 | 100.4 |
Note: Percentage of corrections relative to Reference (%) = (RMSmethod)/ RMSReference×100%, RMSmethod indicates the positioning error RMSE under the adopted ionospheric model, RMSReference indicates the positioning error RMS under the reference datum data. |
To more accurately assess the SPP positioning accuracy when using different models at different stations, the three-dimensional (3D) position errors of different ionospheric models and the percentage of corrections relative to reference are further summarized (Table 5). It is obvious from Table 5 that the 3D position accuracy of the LSTM forecast model in high, mid, and low latitudes is substantially improved, with its 3D error range of 2–4 m, 2–5 m for RIM, and 4–5 m for Klobuchar overall. At mid-latitudes, the correction percentage of the LSTM forecast model relative to the reference data is even more than 90%, which is comparable to the RIM data, while Klobuchar is only about 40%, an improvement of about 50%. It is known from the previous paper (Bi et al., 2017) that the Klobuchar model is too poor in position accuracy mainly in the U direction. Due to the limitation of RIM data coverage, the correction percentage of RIM is only about 60% for high latitude stations except NMER. In terms of 3D position accuracy, the highest position accuracy is achieved at mid-latitude, followed by high latitude and the worst at low latitude. In terms of the percentage of corrections relative to the reference, it is highest in high latitudes, second highest in mid-latitudes, and lowest in low latitudes.