3.1. Geographical regions
A quick inspection of geographical regions revealed that the Tropic of Cancer had a total of 2868 stations, for which an average of 72–133 (31–58% of total) points of monthly precipitation values were available for evaluation, with majority of them lying within India, Thailand, Mexico, and China. Within Tropic of Capricorn, 1421 stations were available with 63–196 (27–81%) points with high coverage in Australia and very limited representation of other countries. The Arctic circle had highest number of stations (41639) with an average of 37–164 (16–71%) points, mostly from USA and Europe with relatively sparse distribution across central Asia. The Antarctic circle had 6592 stations (majority in Australia) with an average of 97–219 (42–96%) points and the Frigid circle had the least (349) number of stations with 73–211 (32–92%) points used to evaluate precipitation products in this study.
A comparison of normal (long-term average) mean monthly precipitation of stations within each geographical region during 2001–2020, simulated by IMERG, CFSR and GSOM is presented in Fig. 2.
From Fig. 2, it is evident that there is comparable to better simulation of normal mean monthly GSOM precipitation by IMERG, compared to the CFSR product for all geographical regions. A relatively better capture of normal monthly precipitation by both products is seen for Frigid circle, Arctic circle and Tropic of Capricorn compared to the Antarctic circle and Tropic of Cancer. Similarly, CFSR has higher spread of precipitation compared to IMERG in the Tropic of Cancer, Arctic, and the Frigid circle and IMERG has higher spread in remaining two. The other performance metrices calculated at stations and aggregated for geographical regions also exhibit a clear signal regarding IMERG’s outperformance in simulating mean monthly precipitation across the globe.
The NSE values computed during the period of 2001–2020 with monthly precipitation values of IMERG, CFSR and GSOM indicate that IMERG has better ability to simulate wet months than CFSR for each region, as shown in Fig. 3.
The bias of NSE towards the higher values indicates that bigger NSE values indicate better capture of wet season’s climatology in the region, which might be important from flood simulation and monitoring perspective. The highest value of median NSE was observed for Tropic of Capricorn (0.85), followed by Antarctic (0.76), Arctic (0.71), Tropic of Cancer (0.64) and Frigid circle (0.47) for IMERG which indicates its good performance in the former three regions, satisfactory performance in the fourth and unsatisfactory in the fifth region. CFSR on the other hand had a satisfactory performance in the Tropic of Capricorn (NSE = 0.59), followed by unsatisfactory performance in Antarctic (0.49), Arctic (0.32), Tropic of Cancer (0.14) and Frigid circle (-0.36). While these median values are representative of the region, individual stations will have better (or worse) performance than this.
Not only during the high precipitation months, dry months in general also had better simulation of GSOM precipitation characteristics with IMERG for all geographical circles, as presented as VE values in Fig. 4. VE in general represents how much of the rainfall is delivered at the proper time and its remainder represents the fractional volumetric mismatch and is thus desired by water resources managers (Ghimire et al., 2019).
In terms of fractional volume match of unit precipitation, Arctic circle had the best statistics (median VE = 0.72) followed by Antarctic (0.7), Tropic of Capricorn (0.68), Frigid circle (0.62) and Tropic of Cancer (0.57) when simulated by IMERG. These statistics indicate good deliverance of unit monthly precipitation in the first two and satisfactory performance in the latter three regions. CFSR on the other hand had satisfactory simulation of unit precipitation in Arctic (VE = 0.57) and Antarctic circle (0.57) and unsatisfactory performance in remaining three regions. This indicates that researchers aiming to employ IMERG dataset in the Tropic of Cancer and Frigid circle can expect higher fractional mismatch compared to observed precipitation. Similarly, water resources management in the data scarce regions of Tropics and Frigid might not be benefitted by the monthly rainfall series of CFSR dataset. The relatively poor performance of both datasets in frigid circle could be due to their inability of estimating snow properly, as discussed by Huffman et al. (2020). Similarly, the relatively lower efficiency of the datasets in tropical circles could be due their inability to depict primary climatological features of tropical rainfall like annual mean, annual cycle and monsoon domain, as discussed by Wang and Ding (2008).
As above discussed metrices; NSE and VE measure relative agreement among observed and simulated precipitation values, RMSE however measures the differences between GSOM and IMERG (CFSR). Although an ideal value of RMSE would be zero, it is almost unachievable in rainfall comparison studies like this, but a larger value would indicate higher problems with the data associated. However, a geographical comparison of the precipitation products again revealed that IMERG has lesser errors than CFSR in simulating monthly precipitation across all geographical regions, as presented in Fig. 5.
In general, it is observed that RMSE of both products was highest in the Tropic of cancer, followed by Tropic of Capricorn, Arctic, Antarctic, and Frigid circles. While the stations number are different in each circle, this descending order of RMSE values are suggested to be evaluated qualitatively. The Indian subcontinent, southeast Asia and Amazon (i.e., the two tropical circles) generally receive higher precipitation than other regions of globe, thus any inconsistency there is likely to appear in higher quantities than the other geographical regions (Cobon et al., 2020). However, less disagreement (indicating less errors) is observed for IMERG when compared to the surface precipitation. In general, CFSR underperformed in estimating surface precipitation characteristics in all geographical circles at monthly time step.
The biases (percent) in IMERG and CFSR products are found in general to be positive, indicating overestimation of monthly precipitation by both products, as shown in Fig. 6.
IMERG (CFSR) had least median biases in Tropic of Capricorn i.e. 5.9% (0.4%), followed by Antarctic i.e. 8.6% (0.8%), Arctic i.e. 9% (11.5%), Tropic of Cancer i.e. 12.9% (24.7%) and Frigid circle i.e. 28% (60.35%). The spread of biases is smaller in IMERG compared to the CFSR, which indicates its slight overestimation in all geographical regions. However, the bias itself might not a significant issue in using such meteorological information from satellite products, mostly due to different bias correction techniques known to reduce systematic biases in estimated rainfall series compared to the observed rainfall climatology (Ghimire et al., 2019). A similar outperformance of IMERG over CFSR compared to GSOM precipitation is observed for two other metrices KGE and R, as shown in Figures S1 and S2, respectively.
The difference in the mean of performance metrices for each geographical region tested for their significance using Welch T-test reveals that IMERG is significantly better in simulating precipitation compared to CFSR, as presented in Table 2.
Table 2
Mean of performance metrices computed for IMERG and CFSR precipitation and their P values when tested for the significance for each geographical region
| Region | Arctic circle | Antarctic circle | Tropic of Cancer | Tropic of Capricorn | Frigid circle |
| NSE_IMERG | 0.56 | 0.68 | 0.39 | 0.73 | -0.04 |
| NSE_CFSR | -0.11 | 0.43 | -0.66 | 0.42 | -0.88 |
| P_NSE | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| KGE_IMERG | 0.68 | 0.74 | 0.54 | 0.74 | 0.47 |
| KGE_CFSR | 0.45 | 0.64 | 0.12 | 0.62 | 0.17 |
| P_KGE | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| VE_IMERG | 0.66 | 0.67 | 0.48 | 0.62 | 0.51 |
| VE_CFSR | 0.47 | 0.55 | 0.18 | 0.46 | 0.19 |
| P_VE | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| R_IMERG | 0.86 | 0.89 | 0.83 | 0.92 | 0.82 |
| R_CFSR | 0.71 | 0.77 | 0.75 | 0.81 | 0.74 |
| P_R | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| RMSE_IMERG | 30.35 | 24.02 | 72.68 | 49.99 | 22.35 |
| RMSE_CFSR | 45.76 | 32.57 | 114.48 | 73.92 | 31.58 |
| P_RMSE | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| PBIAS_IMERG | 13.19 | 10.21 | 15.86 | 10.06 | 34.36 |
| PBIAS_CFSR | 23.01 | 1.53 | 37.17 | 2.19 | 70.86 |
| P_PBIAS | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
The outperformance of IMERG over CFSR could be due to different correction algorithms used by IMERG (Huffman et al., 2015b). Furthermore, IMERG uses GPCP monthly climatology to correct its biases (Huffman et al., 2018), albeit at a coarser resolution of 2.5°(Pendergrass et al., 2020). Better statistic values are observed in general for Arctic and Antarctic circles followed by tropic of Capricorn, tropic of Cancer and frigid circle, which is likely due to the better representation of precipitation in the GPCP and also the set of satellites which are continuously recording the energy information. The reason of low capture of precipitation values in the tropics of cancer and Capricorn could also be due to the missing of key monsoon characteristics of the region (Wang and Ding, 2008).
3.2. Continent wise comparison
A comparison of different precipitation stations located inside each continent revealed that IMERG outperforms CFSR in all continents in terms of different performance metrices. Figure 7 presents VE values computed for each continent by comparing IMERG (CFSR) with GSOM precipitation during 2001–2020.
The fractional matching of unit precipitation was good in Europe (VE = 0.72), North America (0.72), Asia (0.71), Australia (0.70) and satisfactory in South America (0.66), Russia (0.58) and Africa (0.52) when IMERG was used to simulate monthly precipitation during 2001–2020. However, when CFSR was used to simulate monthly precipitation, only Europe (VE = 0.61), North America (0.56) and Australia (0.56) were found satisfactorily simulated, while other continents had unsatisfactory simulation of precipitation for water resource management. Africa, which was least represented in this study due to the unavailability of rainfall stations reported for 2001–2020 had least utility of both IMERG and CFSR product evaluated in this study. A similar trend of other performance indicators (NSE, KGE and R) for the continents are presented in Figures S3, S4 and S5, respectively.
The disagreement of IMERG (CFSR) with GSOM precipitation represented by RMSE is presented in Fig. 8.
In general, South America had the largest median RMSE values (48.21 mm) followed by Asia (31.67 mm), North America (28.47 mm), Russia (24.21 mm), Africa (21.83 mm), Australia (21.78 mm) and Europe (21.75 mm) when IMERG was used to simulate monthly precipitation during 2001–2020. CFSR also followed a similar spatial trend, albeit with larger errors in South America (74.84 mm), Asia (53.66 mm), North America (44.89 mm), Russia (31.26 mm), Australia (31.58 mm), Europe (28.65 mm) and Africa (26.55 mm). The monsoon dominating Southeast Asia makes it the region where highest rainfall occurs throughout the globe (Mason et al., 2020). Similarly, northern part of South America, the Amazon receives good amount of rainfall (Liebmann and Allured, 2005), which increases the chances of getting larger RMSE values in this study. Similarly, both datasets are found to overestimate the overall monthly precipitation across all continents. Highest biases indicating overestimating nature are observed for IMERG (CFSR) in Russia: 36.8% (48.8%), Africa: 25.5% (1.4%), Europe: 21.5% (28.4%), South America: 16% (3.2%), Asia: 12.7% (12.2%), Australia: 7.9% (0.4%) and North America: 7.8% (9.6%) respectively, as presented in Figure S6. These median biases for both IMERG and CFSR in Russia indicate that unless systematic biases are removed from these products, they are unsatisfactory for hydro-meteorological applications. Application of relevant bias correction techniques can improve the IMERG (CFSR) precipitation’s ability in other continents as well.
A larger confidence can be put on the results of IMERG compared to CFSR, as reflected by the results of comparing multiple performance indicators for their significance using Welch T-test in Table S3.
3.3. Country wise comparison
The metrices (NSE, VE, KGE, RMSE, R and PBIAS) computed at stations selected in this study were aggregated by their median value and presented at country level. These results are expected to guide researchers seeking to potentially use IMERG/CFSR data at their country of interest. While these statistics presented in this study are an indication for their potential application, it must be understood that they might not reflect the ground reality, as many countries have very few to no stations included in this analysis. Figure S7 presents the density of rainfall stations (number of stations per 1000 sq.km of land area) in different countries selected in this study. From Figure S7, it is evident that the countries located in South America, Africa and Asia have scarce representation of GSOM rainfall stations compared to the countries in Australia, Europe, and North America. As such, higher confidence can be placed on the results where station densities are higher than the ones with minimum to no representative coverage (Tian et al., 2018). Accordingly, the median of NSE values computed for stations located inside each country by comparing IMERG (CFSR) and GSOM precipitation during 2001–2020 are presented as spatial-plots in Fig. 9.
A significant difference among IMERG and CFSR generated NSE is observed at country levels when compared with GSOM precipitation. 64 out of 105 countries where more than one stations were available for comparison revealed that the mean NSE difference between IMERG and CFSR were significant with P < 0.05. A far superior performance of IMERG is evident at country levels throughout the globe with majority of countries in North America, Europe, Asia, and Australia having satisfactory to good performance. However, countries like Mongolia, Kazakistan, middle east Asian nations, Indonesia, Papua New Guinea, and many countries from Africa and South America exhibit unsatisfactory simulation of high rainfall months, attributed as poor NSE values. CFSR, on the other hand have only few countries with satisfactory simulation of wet months. Again, this superior performance of IMERG may be attributed to the set of passive and active satellite estimates of rainfall and further correction techniques employed with GPCP precipitation (Huffman et al., 2020; Huffman et al., 2019). When tested for significance of these differences between IMERG and CFSR, similar outperformances of IMERG at country level are observed in other indices like VE, KGE and R, which are presented in Figures S8, S9 and S10, respectively. The disagreement among the precipitation products, represented by RMSE values, aggregated at country level are shown in Fig. 10.
Despite low agreement among IMERG and GSOM precipitation across certain countries like Mongolia, Egypt, Iraq and others, as indicated by NSE values (Fig. 9), VE (Figure S8), KGE (Figure S9) and R (Figure S10), the RMSE values appear on the lower spectrum (Fig. 10). It is due to the arid climatology of these countries, where even low RMSE values could cause significant disagreement among rain/no rain status. Interestingly, CFSR appears to follow the trend of IMERG for majority of the countries, albeit with higher disagreement (higher RMSE values). The biases between IMERG (CFSR) and GSOM precipitation are accordingly calculated and presented in Figure S11. Similarly, the summary of statistics is presented in Table S4 for 138 countries which aligns with the above-discussed findings of better performance of IMERG compared to the CFSR.
3.4. Station wise comparison
Comparison of monthly precipitation at station levels during 2001–2020 for selected 50,000 + stations across the globe provides a clear snapshot that IMERG Final run has higher accuracy than CFSR in simulating precipitation. For e.g. the VE values computed at station levels clearly show an outperformance of IMERG precipitation, as can be seen from Fig. 11.
Similar outperformance of IMERG data over CFSR can be seen in terms of other metrices like NSE (Figure S12), KGE (Figure S13) and R (Figure S14). The disagreement among data in terms of RMSE also shows the higher RMSE values in stations located in South and Southeast Asia and Amazon forests with a clear outperformance of IMERG, as shown in Fig. 12.
The biases computed at station levels are presented in Figure S15 and the entire statistics can be referred at station levels from Table S5.