Evaluation of East Asian Meiyu from CMIP6/AMIP simulations

East Asian Meiyu simulated by 35 global atmospheric models from the 6th Coupled Model Intercomparison Project (CMIP6)/Atmospheric Model Intercomparison Project (AMIP) were systematically evaluated for 1998–2014. The results show that most of the CMIP6/AMIP models can hardly reproduce the observed spatial pattern and interannual variability of East Asian Meiyu. The spatial pattern is relatively better simulated over Southern Korea and Japan where 14 out of 35 models have realistically simulated precipitation, as compared with the lower reaches of the Yangtze River where only 7 out of 35 models can well reproduce the Meiyu precipitation. For the Meiyu interannual variations, GFDL-CM4 and GFDL-ESM4 have the closest variance among the models versus the TRMM observations, while 12 out of 35 models show smaller variances. We explored the relationships of Meiyu precipitation with large-scale circulation and tropical sea surface temperature (SST), and showed that these relationships from CESM2-WACCM-FV2, EC-Earth3-CC, and MPI-ESM1-2-HAM agree well with those based on TRMM precipitation, MERRA2 reanalysis-derived large-scale atmospheric fields, and observed SST. It is found that the models with a better simulation of Meiyu precipitation can capture the relationship between Meiyu precipitation and the SST in the eastern equatorial Pacific and Indian Ocean more realistically. A performance ranking of the 35 individual CMIP6/AMIP models is further provided. It is shown that the top 20% of models based on interannual variability score tend to simulate a more realistic western Pacific subtropical high than the bottom 20% of models. And the top 20% of models based on comprehensive ranking measure simulate a more realistic EAP pattern than the bottom 20% of models.


Introduction
As an important component of the East Asian Summer Monsoon (i.e., EASM), the East Asian Meiyu is controlled by the EASM system moving over different latitudes, which is quite different from South Asian Summer Monsoon (e.g., Ding 1992Ding , 2004Ding et al. 2020;Lau et al. 2000;Tao 1987). With a gradual northward propagation of EASM from June to July, the Meiyu rainband presents quasi-stationary characteristics tilting from lower reaches of the Yangtze River Basin of China (i.e., LYRB) to the central North Pacific (e.g., Ding and Chan 2005;Tao 1987; Wang et al. 2021a, b, c, d;Tong et al. 2021). The record-breaking rainfall along the Yangtze River Basin in 2020 persisted for 62 days, induced by the Northwest Pacific anomalous anticyclone and intensified westerly jet due to the Indian Ocean warming, as well as anomalous SST in tropical Atlantic, eastern Pacific, and the Arctic sea ice (e.g., Zhou et al. 2021;Qiao et al. 2021;Pan et al. 2021;Chen et al. 2021). The duration and intensity of Meiyu precipitation, which is often related to floods and droughts in LYRB, southern Korea Peninsula, and southern Japan (i.e., K&J), have an important impact on socioeconomic conditions in such regions (e.g., Ding et al. 2020).
Although the Coupled Model Inter-comparison Project (i.e., CMIP) has evolved to the sixth phase, there are still three central points of the project: How do the Earth system models respond to climate forcing? Where do the model errors originate from? What is the projected climate change like using an ever-expanding range of climate models? (e.g., Eyring et al. 2016). To better answer these questions, the World Climate Research Programme (WCRP) has organized DECK (Diagnostic, Evaluation, and Characterization of Klima), historical simulations, and 23 CMIP-Endorsed Model Intercomparison Projects (MIPs) to compare and analyze the models' simulations (e.g., Dong and Dong 2021;Eyring et al. 2016;Stouffer et al. 2017;Bo and Zhou 2019). The historical Atmospheric Model Intercomparison Project (AMIP) simulation is one of the four DECK experiments, which plays the role of a benchmark for other MIPs. However, the performance of AMIP in the latest suite of CMIP remains to be explored. For example, it is recognized that AMIP models overestimate precipitation over the western Pacific, which is probably related to the deficiencies in deep convection parameterizations (e.g., Li et al. 2017).
Extensive studies have been done to evaluate the fidelity of the state-of-the-art CMIP6 models from the perspective of precipitation (e.g., Choudhury et al. 2021;Coppola et al. 2021;Darshana et al. 2022;Dong and Dong 2021;Fu et al. 2020;Na et al. 2021;Rajendran et al. 2021;Wang et al. 2021a, b, c, d;Zhou et al. 2020). Dong and Dong (2021) assessed seven extreme precipitation indices from CMIP6, and compared them with those in CMIP5. It was found that the dry and wet biases in southern China and Tibet were reduced in CMIP6 models. Norris et al. (2021) unraveled the model spread in atmospheric conditions associated with extreme precipitation in California, showing an overestimation of integrated vapor transportation in most of the CMIP6 models. While many studies examined whether the CMIP6 models can reproduce observed extreme precipitation regionally and globally, little attention has been given to the performance of East Asian Meiyu across models.
In evaluating the model simulations of Meiyu, it is necessary to examine the related atmospheric processes. Wang et al. (2021a, b, c, d) indicated that the discrepancies in precipitation between CMIP6/AMIP simulations and observations are related to the underestimation of mass stream function. They also suggested that the precipitation biases in CMIP6/AMIP probably resulted from the inaccurate precipitation responses to the warm sea. Since the CMIP models are not forced by the observed SSTs as the AMIP models (e.g., Eyring et al. 2016), the model-simulated SST interannual variability could affect the skill of simulated precipitation as compared with observations (e.g., Fu et al. 2020;Wang et al. 2021a, b, c, d). As mentioned above, the 2020 summer Meiyu was marked as extremely strong rainfall, which can trace to the record-breaking deepened thermocline in late 2019 (e.g., Zhou et al. 2021). Zhou et al. (2021) pointed out that the anomalous thermocline maintained the Indian Ocean anomalous warming, which is an important trigger for the formation of super Meiyu in 2020. Thus, how Meiyu responds to the Indian Ocean warming in the model simulations is essential to be confirmed.
Considering that little has been done to evaluate the CMIP6/AMIP simulated Meiyu precipitation systematically, this study evaluates the CMIP6/AMIP Meiyu precipitation using Tropical Rainfall Measuring Mission (i.e., TRMM) satellite product along with Modern-Era Retrospective Analysis for Research and Applications Version 2 ((MERRA2). The Meiyu-related atmospheric conditions and Meiyu precipitation responses to the ENSO-related SSTs and Indian Ocean warming are also examined. We organize the paper as follows. Section 2 introduces data and methods. The assessment of the Meiyu climatology and interannual variability from 35 individual CMIP6/AMIP models is presented in Sect. 3. The roots for better simulated climatology and interannual variation are explored in Sect. 4. The summary and discussions are provided in Sect. 5.

Observational and reanalysis datasets
We used daily precipitation product from the Tropical Rainfall Measuring Mission (TRMM) and Other Satellites Precipitation Product (3B42) Version 7 (i.e., TRMM 3B42-V7), which combined multi-satellite precipitation data sets and gauge analyses with a spatial resolution of 0.25° × 0.25° (e.g., Huffman et al. 2007). The main goal of TRMM is to analyze and understand tropical precipitation since it was launched in 1997, as well as its effects on the global climate (e.g., Simpson et al. 1988;Wolff et al. 2005). Many studies have demonstrated the reliability of TRMM precipitation, and shown that precipitation from TRMM 3B42 agrees well with the surface-based observations (e.g., Islam and Uyeda 2007;Ouma et al. 2012). By examining the Meiyu characteristics using two observational products and four reanalyses, Tong et al. (2021) recommended that TRMM product and MERRA2 reanalysis should be used for the investigations of the Meiyu precipitation and variability and large-scale conditions. Thus, TRMM precipitation data are utilized as a benchmark in this study to evaluate precipitation simulated from CMIP6. The study period is from 1998 to 2014.
The Modern-Era Retrospective Analysis for Research and Applications, Version 2 (i.e., MERRA2) is endorsed by the National Aeronautics and Space Administration (i.e., NASA), aiming to enhance the fidelity of atmospheric reanalysis data (e.g., Decker et al. 2012;Gelaro et al. 2017). MERRA2 covers the period from 1980 to the present, with a horizontal resolution of 0.625° × 0.5°. The data set is available from https:// disc. gsfc. nasa. gov/ datas ets? proje ct= MERRA-2.

CMIP6/AMIP data
We used the AMIP runs from CMIP6 models, which are available at https:// esgf-node. llnl. gov/ search/ cmip6/. The variables include daily precipitation, 850-hPa monthly winds, and 500-hPa geopotential heights. The detailed information of CMIP6/AMIP models used in this study is given in S- Table 1. All available realizations from each model are averaged to reduce the impact of atmospheric internal variabilities among different realizations (e.g., Wang et al. 2011). The CMIP6/AMIP models have coarser resolutions than TRMM and MERRA2, and we have regridded the model outputs to finer resolutions for evaluation following previous studies (e.g., Dong and Dong 2021;Jian et al. 2021). In particular, all the model outputs were regridded to a 0.25° × 0.25° and a 0.625° × 0.5° grids for a comparison with TRMM and MERRA2, respectively, using the method described in Sect. 2.3. All the AMIP models are forced by the observed SST and sea ice concentration using the Hadley-OI dataset that is available at https:// dashr epo. ucar. edu/ datas et/ 158_ asphi lli. html. We choose June 1 to July 31 each year as the Meiyu season (Tong et al. 2021). The seventeen-year Meiyu season from 1998 to 2014 is studied in this paper.

Analysis methodologies
We used the following metrics to evaluate CMIP6/AMIP models in simulating Meiyu precipitation.

Basic statistics
The average precipitation ( x ) during the Meiyu season from 1998 to 2014 at a certain grid is where X i is the daily precipitation for a certain day and N is the total number of days during the Meiyu season for 1998-2014.
The variance (S) is calculated as follows:

Regridding
We applied the Natural Neighbor Interpolation method developed by Robin Sibson (e.g., Sibson 1981). This approach finds the subset of input samples closest to the query point and interpolates these samples by applying weights to them proportionally based on the region size. The Natural Neighbor Interpolation method does not generate maximums and minimums that do not exist in the inputs. Thus, it is a relative interpolator among regridding methods (e.g., Hofstra et al. 2008). This toolbox is available in MATLAB.

Taylor diagram
Taylor diagrams are utilized to statistically examine the performance of climate models in terms of patterns' correlation, the difference of root mean square, and variances (e.g., Taylor 2001). For our application, the angles in the diagram represent the correlation coefficients between the spatial distribution of Meiyu precipitation simulated by CMIP6/ AMIP models and TRMM precipitation. Taylor diagrams also show the information of root mean square (RMS) differences (green circle), as well as the standard deviations by individual points (e.g., Taylor 2001). By giving these three fundamental statistics, Taylor diagrams provide insights on how well models can simulate the Meiyu precipitation.
In the Taylor diagram, the closer the point to the reference point (TRMM) indicates a better model simulation.

Interannual Variability Score (IVS)
Chen et al. (2011) put forward a new metric to calculate the ability of models in simulating the interannual variability, that is: where STD models indicates the standard deviation of CMIP6/ AMIP models, and STD ref is the standard deviation of TRMM precipitation. The smaller the IVS is, the better the models simulate the interannual variability.

Comprehensive Ranking Measure (CRM)
CRM, which reflects the overall performance of CMIP6/ AMIP models, can be expressed as follows: Here, Rank i (J) is the rank of 35 CMIP6/AMIP models. " I " refers to the number of models. " J " refers to the number of indices (PCC in the Taylor diagram and IVS). The closer CRM is to 1, the better the simulation of the climate model's skill is (e.g., Li et al. 2015).

The climatology features of Meiyu across CMIP6/AMIP models
The mean daily Meiyu precipitation during the Meiyu season from TRMM is given in Fig. 1a, which shows the inclined Meiyu rainband, titling from the LYRB region to the central north Pacific. Two large rainfall centers are located in the LYRB and southern Korea and Japan (i.e., K&J). We define the Meiyu Area as the region between 28° N-36° N and 110° E-140° E (Tong et al. 2021). As shown in Fig. 2, most of the CMIP6/AMIP models underestimate the magnitude of Meiyu precipitation compared to TRMM (Fig. 1a). As for the two centers of  Figure  S1, EC-Earth3, EC-Earth3-AerChem, EC-Earth3-CC, EC-Earth3-Veg, and MRI-ESM2-0 have relatively small precipitation biases with respect to TRMM during the Meiyu season.
The evolution of the Meiyu rainband during the Meiyu season is shown in Fig. 3 for TRMM and 35 CMIP6/AMIP models. EC-Earth3-CC, EC-Earth3-Veg, and SAM0-UNI-CON perform better in simulating the advancing northward Meiyu rainband than the other models with somewhat underestimation in magnitude. Most of the CMIP6/AMIP models fail to capture the characteristic of the retreat of Meiyu rainband in mid-July.
To quantify the models' performance in simulating Meiyu precipitation during the Meiyu season for 1998-2014 versus TRMM, Taylor diagrams are shown in Fig. 4 over K&J, LYRB and entire Meiyu regions. It is found that pattern correlation coefficients vary among regions. All correlation coefficients are positive in the K&J region, and the correlation coefficients are overall positive for the entire Meiyu area except for NorCPM1 and BCC-ESM1, while 5 out of 35 CMIP6/AMIP models show negative values in the LYRB region. CMIP6/AMIP models thus can reproduce the Meiyu precipitation better in K&J than that in LYRB, which is consistent with the results shown in Fig. 2.

The interannual variation of Meiyu across CMIP6/AMIP models
CMIP6 models display some skills in simulating interannual variability of precipitation (e.g., Kosaka et al. 2011). Figure 1b shows that precipitation variance over southern Korea and Japan is larger than that in the Yangtze River Basin. FGOALS-g3, NorCPM1, and NorESM2-LM can hardly reflect the variance pattern from TRMM. Time series of the precipitation anomalies from the 35 individual CMIP6/AMIP models in the Meiyu Area are shown versus that from TRMM in Fig. 6a. The TRMM precipitation anomalies are within the range of the 35 CMIP6/ AMIP simulations, while the AMIP model mean anomalies fail to capture the overall interannual variability of TRMM precipitation. Figure 6b shows the IVS for all the 35 AMIP models. The smaller the IVS is, the better the interannual variability is simulated. MPI-ESM1-2-HR and FGOALS-G3 have relatively higher IVS, indicating that these two models perform worst in simulating interannual variabilities.
According to Fig. 4, we count the ranking of pattern correlation coefficients among CMIP6/AMIP models (Table 1), as well as IVS and Comprehensive Ranking Measure (CRM). The first 7 and last 7 models listed in Table 1, which are in bold font, represent the Top 20% and Bottom 20% of CRM rankings, respectively. The CRM ranking is the overall ranking of all the models considering both the climatology and the interannual variability of Meiyu. The Top 20% models of CRM in this study include MRI-ESM2-0, EC-Earth3-CC, EC-Earth3, ACCESS-ESM1-5, EC-Earth3-Veg, MIROC6, and GFDL-ESM4. The 7 Bottom 20% models are FGOALS-G3, BCC-ESM1, TaiESM1, INM-CM-4-8, MPI-ESM1-LR, CMCC-CM2-HR4, and NorCPM1. However, the ability to reproduce Meiyu's climatology and interannual variability may differ greatly from models. Thus, the top 7 and last 7 models in IVS ranking also need to be considered in the following discussion, especially on the relationship between Meiyu precipitation and ENSO/IOBM (Sect. 4.2).  Takaya et al. (2020) indicated that the inflexible trough over the Yellow Sea, strong subtropical jet, and subtropical high extending to the southwest compose the essential factors of extreme rainfall events. Therefore, we explore the relationship between the Meiyu precipitation averaged over the Meiyu area and 500-hPa geopotential height (Fig. 7), and 200-hPa zonal wind (Fig. 8). According to the TRMM&MERRA2 result (Fig. 7a), negative correlations over the Yellow Sea (around 125° E) indicate that there is a trough located over the areas associated with larger averaged Meiyu precipitation. Overall, the pattern of relationship between the averaged Meiyu precipitation and the 500-hPa geopotential height is similar to the East Asia/Pacific (EAP) teleconnection, which has crucial influences on East Asian summer rainfall. In East Asia, the circulation distribution of EAP related to Meiyu is "positive-negative-positive" (i.e., the negative phase of EAP) from low latitude to high latitude, which are located in the South China Sea and the Philippines, Japan, and the area near the Sea of Okhotsk respectively (e.g., Chen et al. 2019). From the perspective of the location of EAP centers, CESM2-WACCM, CESM2-WACCM-FV2, CMCC-CM2-SR5, EC-Earth3, EC-Earth3-CC, EC-Earth3-Veg, IPSL-CM6A-LR, MPI-ESM1-2-HAM, and NorESM2-LM can reasonably reproduce the EAP center located in the South China Sea, and most of the CMIP6/ AMIP models successfully capture the EAP center in the mid to high latitudes. Figure S2 shows that the Top 20% models in CMIP6/AMIP give excellent expression to the two EAP centers in low-mid latitudes, while the Bottom 20% models fail in this aspect. In addition, the position of western subtropical high influences the intensity of East Asian precipitation (e.g., Xu et al. 2020). EC-Earth3-CC, EC-Earth3-Veg, FGOALS-f3-L, FGOALS-g3, MIROC6, MPI-ESM1-2-HAM, and MPI-ESM1-2-HR agree well with the relationship derived from the MERRA2 500-hPa geopotential height and the TRMM Meiyu precipitation. The westerly jet is a crucial factor for Meiyu. (e.g., Sampe and Xie 2010). Horinouchi et al. (2019) revealed that the meridional locations of the subtropical jet are correlated to Meiyu precipitation positively. Figure 8 shows the relationship between the subtropical jet and Meiyu precipitation averaged over the Meiyu region from TRMM&MERRA2 and 35 CMIP6/AMIP models, which varies largely across models. BCC-CSM2-MR, CAMS-CSM1-0, CESM2, CESM2-WACCM, EC-Earth3, EC-Earth3-CC, EC-Earth3-Veg, and MPI-ESM1-2-LR have overestimations of the intensity of subtropical jet. Meanwhile, CESM2-WACCM-FV2, EC-Earth3-CC, and MPI-ESM1-2-HAM perform better than other atmospheric models in simulating this relationship. Figure S3 shows that the Top 20% of models in CMIP6/AMIP can simulate a more realistic EAP pattern than the Bottom 20% of models.
Based on the Climate Prediction Center standard, we selected El Niño years (1998, 2003, 2007, and 2010) and La Niña years (1999, 2000, 2001, 2006, 2008, 2009. Figure 11a, c, e show the differences in circulation fields between strong El Niño years and strong La Niña years during the Meiyu season. Since ENSO is an interannual signal, the Top 20%, and Bottom 20% models in AMIP are selected based on IVS ranking from Table 1. As compared to the MERRA2 results, the Top 20% models can simulate a more realistic subtropical high than the Bottom 20% of models, and with a closed center of the subtropical high located in the eastern Pacific Ocean as the MERRA2 dataset. The basin-wide warming over the Indian Ocean is followed by an El Niño event (e.g., Xie et al. 2016), and persists into East Asian summer. Many studies have shown that the Indian Ocean warming tends to bring more precipitation to East Asian summer. Thus, it is necessary to evaluate whether the relationship between spring Indian Ocean SSTs and Meiyu precipitation is simulated realistically. Figure 10 helps us answer this question. It indicates that spring SSTs over the tropical Indian Ocean have a positive correlation with Meiyu precipitation according to the observational results. The correlation between averaged precipitation and pre-spring SSTs has large differences among atmospheric models. There are 9 out of 35 CMIP6/AMIP models that agree well with the observations, including ACCESS-ESM1-5, BCC-ESM1, CAMS-CSM1-0, CanESM5, CESM2-WACCM, EC-Earth3-CC, FGOALS-f3-L, IITM-ESM, MPI-ESM-HAM, MRI-ESM2-0, and TaiESM1. Figure S6 shows that the Top 20% models in CMIP6/AMIP reflect more significant correlations between MAM SSTs in the Indian Ocean and averaged Meiyu precipitation. We also selected warm IOBM years (1998, 2003, and 2010) and cold IOBM years (1999, 2000, 2008, 2011, and 2013) to calculate the differences in circulations. As is shown in Fig. 11b, d, f, the Top 20% of the CMIP6/AMIP models can simulate the closed Pacific high, but the location of the high center is relatively eastward. The Top 20% models in CMIP6/AMIP models also perform better in the simulation of subtropical high, which is similar to the previous result.

Summary and conclusions
In this study, we systematically assessed 35 CMIP6/ AMIP models based on their performance in simulating the spatial pattern and interannual variability of Meiyu precipitation for the period 1998-2014. We selected the Top 20% and Bottom 20% models for Meiyu precipitation simulations, based on both the overall model ranking and IVS ranking. The possible reasons for the model biases among the two groups were also investigated.
For the Meiyu climatological features, 14 and 7 out of 35 models have reasonably simulated rainfall over K&J and LYRB regions, respectively. CMIP6/AMIP models  . 9 The same as Fig. 7 but between the Meiyu precipitation averaged over the Meiyu area and the pre-winter Hadley-OI SST. The precipitation is from TRMM and 35 CMIP6/AMIP models can thus reproduce the Meiyu precipitation better in the K&J region than that in LYRB, which is also shown in the Taylor diagrams. Furthermore, most of the CMIP6/ AMIP models fail to capture the characteristic of the Meiyu rainband with reduced precipitation in mid-July. For the Meiyu interannual variations, GFDL-CM4 and GFDL-ESM4 have the closest variance among the models as compared with TRMM data, while CAMS-CSM1-0, CanESM5, CMCC-CM2-HR4, CMCC-CM2-SR5, four ECMWF models, INM-CM4-8, INM-CM5-0, MPI-ESM-1-2-HAM, MPI-ESM1-2-HR, and MRI-ESM2-0 show smaller variances.
We quantitively ranked the performance of the 35 models in simulating Meiyu precipitation both in climatology and interannual variation, and selected the Top 20% and Bottom 20% in CMIP6/AMIP models. We also explored how the CMIP6/AMIP models reproduce the relationship between Meiyu precipitation, western Pacific subtropical high, and a subtropical jet. It is indicated that CESM2-WACCM-FV2, EC-Earth3-CC, and MPI-ESM1-2-HAM agree well with the relationships based on observations and MERRA2. Among the 35 CMIP6/AMIP models, CESM2-WACCM, CESM2-WACCM-FV 2, CMCC-CM2-SR5, EC-Earth3-Veg, IPSL-CM6A-LR, MPI-ESM1-2-HAM, NorCPM1, and NorESM2-LM can realistically reproduce the EAP center located in the South China Sea, while most of the CMIP6/AMIP models successfully capture the EAP center in the mid to high latitudes.
Our evaluation was also done by examining how ENSO and IOBM influence Meiyu across atmospheric models as compared with observations. The more significant the positive correlations between averaged Meiyu precipitation and SSTs over both the eastern equatorial Pacific and the Indian Ocean are, the closer the atmospheric model simulations of Meiyu precipitation are to observations. Meanwhile, the Top 20% of models tend to simulate a more realistic western Pacific subtropical high than the Bottom 20% of models in CMIP6/AMIP.
In conclusion, this study provides an assessment of CMIP6/AMIP models for simulating Meiyu precipitation. We lay a foundation for reducing errors and uncertainties of Meiyu precipitation projections and other climate events.

Fig. 10
The same as Fig. 9 but between the Meiyu precipitation averaged over the Meiyu area and the pre-spring Hadley-OI SST Fig. 11 Differences in circulation fields between strong El Niño years and strong La Niña years (a, c, e), and differences in circulation fields between warm IOBM years and cold IOBM years (b, d, f) during the Meiyu season. The geopotential heights are at 500-hPa level, and contour intervals are 2 m. The wind is at 850-hPa level