ECMWF CAMS_nrt products over South Korea
First, we start our discussion with CAMS_nrt (Copernicus Atmospheric Modeling and Monitoring Service, near real-time products), which is global air quality forecast data produced by the C-IFS of the ECMWF (for details on the C-IFS model simulations, see Supplementary Section 3) 8.
Figure 2a presents daily variations of averaged PM2.5 over South Korea during the period of KORUS-AQ campaign (Korea–United States Air Quality campaign) carried out between 1 May, 2016 and 10 June, 2016. Figure 2 also shows comparisons between PM2.5 predictions and PM2.5 observations. Because PM2.5 forecasts are usually made on a daily basis, we present the daily variation in Fig. 2. However, hourly comparisons are also provided in Supplementary Fig. 1. PM2.5 observations are acquired from a ground observation network in South Korea called ‘AIR KOREA’ that is managed by the KMoE. This KORUS-AQ campaign has been well-investigated in terms of meteorological and physico-chemical characteristics 9,10. Therefore, we selected this campaign period as a time-window for this study.
The CAMS_nrt PM2.5 in Fig. 2a shows a moderate agreement with observed PM2.5, with an IOA (Index of Agreement) of 0.51. Here, the IOA was calculated based on hourly PM2.5 data, not based on daily values. In addition, only IOA was presented in Fig. 2. However, other statistical metrics including errors and biases were also analyzed (refer to Supplementary Section 4). This moderate IOA of 0.51 in Fig. 2a might be affected by two main factors: (i) modeling errors produced by C-IFS model simulations and (ii) inaccuracy in the initial state caused by data assimilation with LEO (Low Earth Orbit) satellite-derived data. First, to evaluate the accuracy of C-IFS model simulations, we compared the predicted concentrations of major PM2.5 constituents with observed concentrations of sulfate, organic aerosols (OAs), black carbon (BC), and dust at six intensive measurement stations in South Korea. The results of the comparisons are shown in Supplementary Section 5. In general, C-IFS model simulations over-predicted dust and BC concentrations but under-predicted OA concentrations. These inaccuracies contributed to the moderate IOA shown in Fig. 2a.
Second, initial states prepared by data assimilation are also crucial, particularly for short-term (i.e., one or two days) predictions. The CAMS_nrt system employs a 4-D VAR (4-Dimensional Variational) data assimilation method with MODIS (Moderate-resolution Imaging Spectro-radiometer) AOD (Aerosol Optical Depth) 11. This 4-D VAR data assimilation method is a technique that can correct errors of model fields (background fields) with ‘observations (near true value)’. Corrected fields (analysis fields) created by data assimilation are applied to operational C-IFS runs for initial conditions. Observational data for the 4-D VAR assimilation are AOD data retrieved from two MODIS sensors onboard Terra and Aqua satellites.
However, MODIS sensors (or other LEO UV/VIS sensors) have shown two serious limitations in terms of their data availability. First, they cannot produce AOD data over areas where clouds are present. We call this phenomenon ‘cloud masking’. For example, (north)east Asia is highly cloudy compared to other continents, mainly because of both large presences of cloud seeds (atmospheric aerosols) and high humidity. Such large cloud masking tends to lead to a large loss of aerosol data over (north)east Asia. In actuality, the average percentage of MODIS AOD data available during the period of the KORUS-AQ campaign was only ~ 14% over the domain shown in Fig. 1.
Second, the results of monitoring via these two MODIS sensors are prone to be affected by the ‘sun-glint effect’. This is an inherent effect caused by the geometry between LEO satellite sensors and the sun. Because of the sun-glint effect, we sometimes lose many possibly important data, particularly over ocean areas, where no surface PM2.5 observations are available 12. This loss of MODIS AOD data due to the sun glint effect is discussed further in Supplementary Section 6.
Collectively, these two limitations in monitoring data from MODIS sensors result in a scarcity of AOD (or observed PM2.5) data. This scarcity of observed data can prevent data assimilation from effectively correcting model background errors. The moderate IOA of 0.51 shown in Fig. 2a is due to both modeling errors and the scarcity of the observations used in the data assimilation. In addition, Fig. 2a provides a typical example of how difficult accurate short-term PM2.5 prediction is with our current levels of knowledge and techniques.
Leo Vs. Geo Satellite Sensors
With lessons from CAMS_nrt, we set up a strategy to build up a more accurate short-term PM2.5 prediction system over South Korea by developing a more advanced air quality modeling system intended to reduce modeling errors and by utilizing better satellite-borne PM2.5 data to improve initial states 13,14. Regarding the former, we have developed K_ACheMS v2.0. Regarding the latter, we decided to use AOD products from the ‘state-of-the art’ Korean geostationary satellite sensor named GOCI (Geostationary Ocean Color Imager) instead of the two MODIS sensors. Although the GOCI sensor cannot also produce AOD data over pixels where clouds are present, it can overcome the limitation of the sun-glint effect because it is a GEO satellite sensor.
Figure 2b is produced from K_ACheMS v2.0 without performing data assimilation (i.e., without updated initial conditions). With this effort alone, the IOA jumped up from 0.51 (Fig. 2a) to 0.71 (Fig. 2b), indicating the potential importance of the performance of the air quality modeling system. Figure 2c presents PM2.5 predictions made by the K_ACheMS v2.0 with 3-D VAR (3-Dimensional Variational) data assimilation using only MODIS AOD. The experiment of Fig. 2c was a mimicking simulation of CAMS_nrt. Surprisingly, the IOA in Fig. 2c was the same as that in Fig. 2b. This was because MODIS AOD data were too sparse to affect the predictability of PM2.5 in South Korea due to both cloud masking and sun-glint effects. Again, the data coverage of MODIS sensors was only ~ 14%. With this low data coverage, the data assimilation appears to be almost ‘useless’.
This problem of data scarcity can be avoided in part by using GOCI satellite-retrieved AOD. When we used GOCI-retrieved AOD data, the percentage of the AOD data available during the period of the KORUS-AQ campaign over the monitoring domain of the GOCI increased up to 28.5%. Here, we attempted to use two products from the Korean GOCI sensor. First, we directly assimilated the GOCI AOD. Second, we assimilated surface PM2.5 converted from the GOCI AOD via a machine learning technique named the ‘Random Forest (RF) method’ (for details on this data conversion, refer to Supplementary Section 7) 15. Figure 2d, e present the PM2.5 predictability results from these two experiments. Enhancements in PM2.5 predictability can be seen visually in Fig. 2 as well as in terms of IOA. The IOA again increased from 0.71 to 0.73–0.75, showing that assimilation of GOCI AOD data could more effectively correct model errors in the initial state than MODIS AOD data.
Why did RF-converted PM2.5 correct modeling errors more effectively than GOCI AOD? One reason might be that the RSDs (Relative Standard Deviations) of the errors of RF-converted PM2.5 tended to be smaller than those of GOCI AODs. This will be discussed in detail in Supplementary Section 7. These smaller RSDs of errors in RF-converted PM2.5 were reflected in the observation error covariance matrix during the course of data assimilation, as background fields were corrected more strongly on the basis of PM2.5 observations. Based on these results, we can conclude that applying a machine learning technique to PM2.5 predictions could create a positive space to further enhance PM2.5 predictability.
Ill-location Of Information: Satellite Data Vs. Ground Observations
Despite the substantial advancements that have been made, both MODIS- and GOCI-retrieved data have inherent and unavoidable disadvantages: ‘data scarcity’ and ‘data ill-location’. As mentioned above, the average percentage of the GOCI AOD data available over the entire GOCI domain was approximately 28.5%. This means that we could not obtain AOD data from 71.5% of GOCI pixels, mainly due to the presence of clouds. The presence of clouds is the major reason for the problem of scarcity of satellite-borne aerosol data. In addition, even 28.5% of the GOCI aerosol data are not always available at useful locations for PM2.5 predictions in South Korea. For example, if some portions of GOCI AOD data are available over areas where air masses cannot influence air quality in South Korea (e.g., over the northern edges of the modeling domain or over the East China Sea, which is far from the Korean peninsula), then such AOD data become meaningless in terms of their ability to improve PM2.5 predictability in South Korea. We refer to this type of problem as the ‘ill-location problem’ of satellite data.
To overcome the problems of data scarcity and data ill-location, we must return back to ‘classical’ observation data, i.e., ground observations. Because ground observations are being made at fixed ‘surface’ locations, these data are never affected by the presence of clouds. Figure 1 shows PM2.5 measurement networks in China (at approximately 1800 locations) and in South Korea (at approximately 400 sites) 16,17. However, a challenging point is whether we can obtain these observation data in near real-time. In the case of ‘past’ observations, they can be downloaded from the data archive. However, it is difficult to collect ‘present’ data in a near real-time mode. To resolve this problem, we decided to develop a method called the ‘screen crawling technique’. Using this digital software technique, we can scan and obtain observation data from corresponding Chinese and Korean websites in a near real-time (in situ) mode. Such near real-time observations can then be utilized almost immediately in the PM2.5 prediction system after a quick data quality inspection called the ‘buddy test’, which was described in detail in Lee et al. 18.
Figure 2f shows how large improvements in PM2.5 predictions over South Korea can be made by assimilating ground PM2.5 observations instead of GOCI AODs or RF-converted PM2.5. The IOA increased from 0.73–0.75 to 0.77. From the data assimilation of these ground observations, we also found additional advantages. In Fig. 2, the gray-shadow period I was characterized as an ‘Asian dust episode’ that had taken place from 6 May to 7 May, 2016. During this period, dust plumes were generated in the Inner Mongolia, and they were then transported long distance over northeastern China and the Yellow Sea. Unfortunately, these dust plumes were not detected by the GOCI sensor because such dust plumes are frequently co-present with cold frontal clouds. However, the ground observation network inside China certainly detected these dust plumes. During dust episodes, PM10 exhibits a tendency to increase, because dust particles are predominantly composed of coarse-mode particles (i.e., particles larger than 2.5 µm). In this study, we assimilated both ground PM2.5 and PM10 (refer to Supplementary Section 8). Supplementary Fig. 2f clearly shows that our prediction can also capture the dust peak in PM10 during period I. By contrast, PM10 predictions in Supplementary Fig. 2d, e, wherein only GOCI products were used, could not capture the dust peak. This again demonstrates why we should assimilate ‘ground observations’ of PM2.5 and PM10 to enhance PM2.5 and PM10 predictability.
What if Ground And Satellite-derived Observations Are Used Together?
What can happen if ground PM2.5 observations and GOCI-borne data are used together for data assimilation? Can we expect a synergism? To answer this question, we designed two more experiments: (i) sequential assimilation with ground PM2.5 and GOCI AOD, and (ii) sequential assimilation with ground PM2.5 and RF-converted PM2.5. Figure 2g, h show the results from these two case studies, respectively. In both experiments, IOA increased further from 0.77 to 0.78 over the entire period of the KORUS-AQ campaign, implying that the addition of either GOCI-derived AOD or RF-converted PM2.5 into ground PM2.5 could improve the accuracy of PM2.5 predictions in South Korea.
This appears to be particularly true for the two gray-shadow periods II and III in Fig. 2. Gray-shadow periods II and III were characterized by KORUS-AQ scientists as ‘a period of air stagnation’ and ‘an episode of long-range transport’ from China, respectively. During the period of air stagnation (between 18 May and 23 May, 2016), an anticyclone sat around the Korean peninsula. Because of this, air masses were rotating clockwise with low wind speeds around the Korea peninsula (see Fig. 3b). This low-speed rotation of air masses created a favorable condition for air pollutants to be accumulated around the Korean peninsula. On the other hand, during the period of long-range transport (between 25 May and 30 May, 2016), air pollutants, including PM2.5, were transported long distance from the North China Plain (NCP) to South Korea due to strong westerlies (see Fig. 3c). We found that PM2.5 increased up to 60 µg/m3 on 26 May, 2016. As can be seen visually in Fig. 2g, h, the gaps between PM2.5 observations and predictions during these two gray periods became narrower than those in Fig. 2f. A similar situation was also found for the predictions of PM10, as shown in Supplementary Fig. 2g, h. These results are particularly important, because both air stagnation and long-range transport are two typical meteorological conditions under which the levels of PM2.5 in the Korean peninsula are frequently enhanced. If the situation is really like this one, then the following question arises: what factor creates such improvements in PM2.5 and PM10 predictability?
The answer to this question is presented in Fig. 3, which demonstrates a synergism created by sequential data assimilation using ground PM2.5 observations and GOCI-derived AOD. During periods II and III, sequential applications of both forms of data to data assimilation allowed IOA to jump up from 0.57 to 0.64 during the period of air stagnation and from 0.62 to 0.67 during the period of long-range transport. In case of air stagnation, GOCI AOD data were available over the Yellow Sea (denoted as I in Fig. 1), North Korea (denoted as II in Fig. 1), and the East Sea (denoted as III in Fig. 1). This data availability was due to the fact that the sky was very clear (uncloudy) as a result of the anticyclone located around the Korean peninsula. Because of the rotation of air masses, all the three regions became upwind regions to South Korea in this case. Therefore, it is crucial to have information over all three of these regions for PM2.5 predictions in South Korea.
The episode of long-range transport (period III) is another excellent example regarding the creation of synergism. Surprisingly, during this period, ground PM2.5 measurements inside China were all turned off. However, high AOD plumes were detected by the GOCI sensor over the Yellow Sea. The Yellow Sea was an upwind region to South Korea in this case (see the arrows of winds in Fig. 3c). In addition to the ground PM2.5 information available inside South Korea, the GOCI AOD data available over the Yellow Sea helped us further enhance the accuracy of PM2.5 predictions in South Korea.
Over the entire period of the KORUS-AQ campaign, including the two episodes discussed above, our advanced PM2.5 prediction system (Fig. 2h) exhibited enhancements of approximately 10%, 17%, 49%, and 19% in the predictability of PM2.5 over South Korea, in terms of IOA, R (Pearson correlation coefficient), MB (mean biases), and HR (hit rate), respectively, compared to the PM2.5 prediction system using LEO-retrieved observations alone (Fig. 2c) (for details, also refer to Supplementary Fig. 3 and Section 4).
Blank Area Of Information
As described above, South Korea is surrounded by so-called ‘blank areas of information’ such as the Yellow Sea, North Korea, and the East Sea. However, transboundary air pollution events from China to South Korea are almost always occurring through these blind regions along the strong persistent westerly and/or northwesterly winds. This means that, for improved PM2.5 (and PM10) predictability in South Korea, it is certainly necessary to have information on air quality over these blind regions.
In this context, we demonstrated that information from GEO satellite sensors over those blank areas of information could help us substantially improve PM2.5 predictability in South Korea. Although it is true that geostationary satellite data can help us improve PM predictability, there is no 100% guarantee that GEO satellite data are always available over the blank areas due to the random presence of clouds. To provide more convincing data availability over the blank regions, it would be helpful to establish surface observation networks to boost the performances of PM2.5 predictions in South Korea. The establishment of a surface observation network may be a cheaper and more guaranteed way to improve PM2.5 predictability in South Korea than launching ‘expensive’ GEO satellite sensors.
Based on the discussions shown above, establishing several air quality monitoring towers or stations above the Yellow Sea and the East Sea (Sea of Japan) as well as inside North Korea will be highly useful for achieving better PM2.5 predictions in South Korea. Of course, building air monitoring stations inside the territory of North Korea would create political arguments. It appears to be a difficult task to achieve politically, although it is an easy task to accomplish technically. In such a case, good diplomatic or political efforts will certainly be able to lead to improved air quality predictions and management in South Korea.
Additional Discussions
In this study, we reported our major findings inferred from experiments during the period of the KORUS-AQ campaign. To more firmly support our conclusions, we also carried out another six-month test-bed experiment between 1 November, 2016 and 30 April, 2017 (a high PM2.5 period in South Korea). Test results from this six-month investigation are discussed in Supplementary Section 9. In short, we again drew the same conclusions from the six-month experiment. Based on this finding, we now believe that our conclusions drawn from this study are general ones, not period-specific. The methods and strategies applied to South Korea can also be applied to many other regions that encounter similar situations. In this sense, our methods and strategies are not area-specific, either.
In addition, IOAs and other statistical metrics such as root mean square errors, mean biases, and so on are commonly used in the science community. We refer to these metrics as ‘scientific statistical metrics’. On the other hand, the HR is a metric that administration parties such as KMoE are taking particular care of in South Korea. Therefore, the HR is referred to as an ‘administrative statistical metric’. In many cases, high IOAs do not always guarantee high HRs, because HR refers to the percentage of successful hitting of category intervals of PM2.5 in South Korean air quality standards, unlike IOAs. For details on HR, refer to Supplementary Section 10. Although the HR is not a scientific metric, there has also been a strong requirement to enhance the HR in South Korea. This was why we also inserted HRs into Fig. 2. Further, it could be seen that HRs increased continuously from Fig. 2b–h.
The K_ACheMS v2.0 discussed in this study has been actively run in a test mode for PM2.5 predictions over South Korea since 1 Jan., 2022 (visit website at https://kachems.gist.ac.kr). Figure 4 presents the parts of performances of the K_ACheMS v2.0 through comparisons between PM2.5 observations and PM2.5 predictions. In Fig. 4, three PM2.5 predictions from the K_ACheMS v2.0 and current operational PM2.5 prediction systems at NIER (National Institute of Environmental Research in South Korea) and ECMWF are compared with PM2.5 observations obtained from AIR KOREA. The PM2.5 predictions from the NIER have been based on a different approach of an ensemble average of 20 outputs from 20 combinations among different meteorology and air quality models and emission inventories 19. It can be seen in Fig. 4 that the K_ACheMS v2.0 produced much better PM2.5 predictions than the other two systems at NIER and ECMWF for three major high PM2.5 episodes. The three high PM2.5 episodes shown in Fig. 4 were the ‘biggest’ ones that occurred over the six months, during which the K_ACheMS v2.0 were operated in a test mode (between 1 Jan. and 30 June, 2022).
During the six-month test period, the IOA and HR from the K_ACheMS v2.0 were 0.78 and 67%, respectively. These numbers were substantially higher than those produced by the ECMWF prediction system (IOA: 0.76 and HR: 54%). Although the PM2.5 predictions from the NIER system were not released yet, annual averaged HR of the NIER system has been reported to be approximately 55%. Further analyses are made in Supplementary Section 11 (refer to Supplementary Tables 7 and 8). Based on these analyses, the new PM2.5 prediction system proposed in this study appears to have been working very well for the first six-month test period, although a comprehensive performance report on the one year-long test is scheduled to be issued at the end of 2022 including the PM2.5 predictions from the NIER system.
When developing the K_ACheMS v2.0, we have also put some parallel efforts into developing machine learning techniques to correct errors and biases resulting from air quality model simulations. We call this process ‘error and bias correction via machine learning’. For this process, we have selected the machine learning technique of ‘XGBoost (eXteme Gradient Boost)’ 20. We find that this technique has been particularly effective in terms of enhancing ‘HRs’ without showing great advantages in improving IOAs. This will also be discussed further in Supplementary Section 12.