Evaluation and Projections of Summer Daily Precipitation over Northeastern China in an Optimal CMIP6 Multi-model Ensemble

In the early summer of 2022, the average precipitation in Northeastern China (NEC) was 144.9 mm, which was 62% higher than the normal precipitation (89.5 mm), and was the largest amount of precipitation since historical records began in 1951. Based on the CN05.1 precipitation data, the summer precipitation simulated by the sixth Coupled Model Intercomparison Project (CMIP6) model was evaluated in the NEC region. The model performs well in reproducing the climatological spatial distribution and interannual variability of precipitation. The simulation effect of a multi-model ensemble is better than that of a single model. The top three optimal models (EC-Earth3-Veg, EC-Earth3 and MPI-ESM1-2-LR) can constitute the optimal model ensemble (HMME). Compared with a single model and an ensemble of all 20 models, HMME signi�cantly reduces the wet bias on the southeastern side of the Da Xing-AN Ling Mountains. HMME projects that daily precipitation will show an increasing trend in NEC region from 2015 to 2100. Under the two scenarios of SSP245 and SSP585, the increase rates of precipitation are 0.05mm/10a and 0.15mm/10a respectively. Although precipitation trends varied between scenarios at different times of the year, by the end of the 21st century, precipitation increased signi�cantly across the region, with the largest increase in precipitation located in the southern part of northeastern China.


Introduction
The Sixth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) stated that global temperatures have risen by 1.1°C from 1850-1900 to 2010-2019, implying a high sensitivity of the climate system (IPCC, 2023).In particular, changes in precipitation exhibit substantial uncertainty.Although temperature is a key driver of precipitation anomalies, precipitation is not always associated with temperature changes because of the dynamical environment (Xu et al., 2023).Therefore, it is essential to project the future changes of precipitation.
Northeastern China (NEC) produces nearly a quarter of China's total grain.Summer precipitation, which accounts for more than 65% of the total annual precipitation in this region (Liang et al., 2011), provides water security for crops during the critical period of water.Therefore, the spatial-temporal variation of summer precipitation plays a key role in agricultural production and water mobilization in NEC.Previous studies show that the causes of summer precipitation anomalies over NEC are varied.The East Asian summer monsoon and other mid-high latitude climate systems are the main factors in uencing the climate changes in NEC (Yao and Dong, 2000;Shen et al., 2011;Gao et al., 2014a;Sun et al., 2017;Gao and Gao, 2018;Han et al., 2018;Zhao et al., 2018).Meanwhile, changes of external forcings like sea surface temperature anomaly or sea ice variability drive precipitation anomalies (e.g., Wu et al., 2011;Gao et al., 2014b;Han et al., 2018;Han et al., 2021).In terms of complex climate system, summer precipitation over NEC varies with region (Liu et al., 2010;Mao et al., 2022) and its spatial-temporal variation is unable to be simulated and projected very well.
Climate model is one of the most important tools for climate simulation and projection.The Coupled Model Intercomparison Project (CMIP) collected a large amount of data produced by different global climate models (GCMs).In recent years, the 6th Phase of CMIP (CMIP6) has been generated from models with higher spatial resolution and more complex physical processes compared to the previous phase (Eyring et al., 2016;Stouffer et al., 2017;Zhou et al., 2019).A new database of the Shared Economic Pathways (SSP) was developed to integrate climate change and economic development in the CMIP6 scenarios (Eyring et al., 2016;Stouffer et al., 2017;Zhou et al., 2019).Up to now, several studies have demonstrated that CMIP6 models generally exhibit satisfactory simulation performance in modelling annual mean and seasonal precipitation on the global scale (e.g., Du et al., 2022;Donat et al., 2023).
However, due to the different settings in climate model, such as physical processes, initial conditions, and parameterization schemes, the simulation performance of each model varies from region to region.Dong and Dong (2021) studied the performance of the CMIP6 models to reproduce the spatial distribution and trends of extreme precipitation over East Asia.There was generally a wet bias in western China and a dry bias in South Asian.He et al. (2023) obtained a similar result that CMIP6 models underestimated South Asian summer rainfall.On the regional climate change over China, Xin et al. (2020) evaluated eight models, most of which underestimated the gradient of precipitation decrease from southeast to northwest.Especially for the "southern ood and northern drought" rainfall pattern in China, only two models were able to partially reproduce it.It is notable that the uncertainty in climate projections arises from the model uncertainty, scenario uncertainty and internal variability.By using analysis of variance, Jia et al. ( 2023) indicated that the model uncertainty was the dominant factor in projection.Thus, it might be of primary importance for the credibility of future projection to evaluate the performance of CMIP6 models on the regional scale.
Several metrics have long been used to measure the simulation performance of climate models, such as correlation coe cient (CC), root-mean-square error (RMSE), standard deviation (STD), Taylor Skill Score (TS) and the interannual variability skill score (IVS) (Taylor, 2001;Chen et al., 2011).In general, more than one indicator is considered in order to evaluate the performance of model.Many studies indicated that models have different simulation performances in different sub-regions (Jiang et al., 2020;Dong and Dong, 2021).To cope with the model uncertainty from single model, the multi-model ensemble has been used to simulate and project (Xin et al., 2020;Zhu et al., 2020).
Additionally, the simulation performance of the multi-model ensemble is also related to the number of members (Pierce et al., 2009).Therefore, based on the evaluation of simulations, more appropriate models could be considered while averaging the projection of a variable.Unfortunately, the optimal model ensemble with high scores (HMME) selected by evaluation of a large region does not apply to all subregions.For example, the HMME selected by the performance in simulating temperature and precipitation in China did not show better performance than the all-model ensemble (AMME) in summer precipitation over sub-region (Yang et al., 2021).However, there has been limited works devoted to evaluating the ensemble behaviors of CMIP6 models in sub-region, particularly in simulating precipitation.Tao et al. (2016) evaluated the simulation ability of the CMIP5 models for temperature across three provinces in NEC.They identi ed four models that exhibited superior predictive performance.However, whether the models of the CMIP6 models perform well in simulating summer precipitation is still needed to evaluate.This study aims to explore the model performance in simulating the spatial-temporal distribution summer precipitation over NEC and get the future projection of possible precipitation changes.The remainder of this study is structured as follows: Section 2 describes our data and methods.
Focused on the better use of available models, the long-term precipitation simulated by the historical experiment were compared with observation in section 3.And possible future changes of precipitations during 2015-2100 under the scenarios SSP245 and SSP585 were also analyzed in section 4.And section 5 provides a summary of our ndings.

Data
The NEC region for this study ranges from 38°N-55°N and 118°E-136°E.The observed daily precipitation amounts were selected from the CN05.1 data set provided by the China National Climate Center, which are interpolated from station observations using the Angular distance weighting (ADW) interpolation method at a horizontal resolution of 0.25°×0.25°(Xu et al., 2009;Wu and Gao, 2013).
This study evaluated the historical summer precipitation (1979-2014) simulated by 20 CMIP6 models, from which the top three optimally ranked models (EC-Earth3-Veg EC-Earth3 MPI-ESM1-2-LR) were selected for future climate projections (2015-2100) under the SSP245 and SSP585 scenarios.The model-name, institute information, time Correlation coe cient, resolution of the GCMs used in this study are listed in Table 1.Due to the different resolutions of the models, the observed and modeled precipitation data were uniformly interpolated onto a grid with resolution of 0.5°×0.5°using bilinear interpolation.The historical period selected for this paper is 1979-2014, the future projection period is 2015-2100, and the climate base period is selected as 1981-2010.

Methods
To evaluate CMIP6 model performance for precipitations with respect to spatial distribution and interannual variation, we used three evaluation methods in this study: the Taylor skill score (TSS; Taylor, 2001), the interannual variability skill score (IVS; Chen et al., 2011), and the Comprehensive Rating Index (CRI; Jiang et al., 2015).
The Taylor Skill Score (TS; Taylor, 2001) was used to rank GCM in capturing the spatial pattern of precipitation.Taylor diagrams provide spatial correlation coe cients (SCC), centered RMSEs, and standard deviation ratios (SDR) as measures of consistency in the spatial distributions of the model and observations.The TS was calculated as follows:

SDR
where SDR is the ratio of standard deviation and SCC is the spatial correlation coe cient between modeled and observation data.TS varies from 0 to 1, and a larger TS indicates relatively better performance of a model.
The performance of a model in simulating the interannual variability was estimated using the skill score IVS (Chen et al., 2011;Jiang et al., 2015), which was used to verify that the models can simulate the interannual variability in the observations well and was calculated as follows: 2 where and are the standard deviation of modeled and observed data, respectively.The IVS for each grid point were rst calculated, and the mean IVS for the NEC region was then calculated.IVS is a symmetric variability statistic that can lter the interdecadal signal and measure the similarity between the simulated and observed inter-annual variability.The IVS closer to zero indicate that the CMIP6 model can simulate similarly to the observations.Models are effectively ranked using the Comprehensive Rating Index (CRI) (Jiang et al., 2015).The formula of CRI is given as follows: where m is the number of models and n is the number of metrics.The rankings used in this paper to calculate the CRI are based on RESM, SDR, SCC, and IVS, and models that are more consistent with observations are ranked higher.Thus, the closer the CRI value is to 1, the higher the model simulation skill.

Historical period simulation assessment
The performance of different CMIP6 models in simulating summer precipitation from 1979 to 2014 was rstly evaluated.Figure 1 shows the historical simulated (20 CMIP6 models) and observed regional mean summer daily precipitation and its standard deviation in NEC during 1979-2014.The mean summer daily precipitation in NEC region was 3.70 mm/day, and 14 out of 20 models overestimated the summer precipitation, and 4 of them, CESM2, CESM2-WACCM, INM-CM4-8, and INM-CM5-0, had a relative deviation of more than 40% from the observations.The GFDL-ESM4 simulations were closest to observations, underestimating by only 0.02 mm/day.With the exception of CESM2 and CESM2-WACCM, the standard deviation of most of the models and observations (0.58 mm/day) are in general agreement, indicating that the models can reproduce the intensity of interannual variability of precipitation.
To further analyze the bias of the spatial distribution by different CMIP6 modeled precipitation, the spatial distribution of mean daily precipitation and its deviation in the NEC region simulated by CMIP6 models are given in Figure .2.It can be noted that there is considerable spatial heterogeneity in the bias of different individual models.The observation results (Figure .2b)show that summer precipitation in the NEC region as a whole is characterized by a spatial distribution with more in the southeast and less in the west of the region, and its average daily precipitation ranges from 1.96 to 7.06 mm/day with a large spatial variation.The high precipitation centers in the southern part of NEC and the east side of the Da Xing-AN Ling may be related to the dominance of the summer wind water vapor transport in East Asia (Sun et al., 2017), as well as the better water vapor conditions along the Southeast coast and the topographic blocking effect of the Da Xing-AN Ling (He et al., 2020).
The spatial correlation coe cients (SCC) between the modeled data and observations range from 0.37 to 0.92 (passing the 95% signi cance test), indicating that the CMIP6 model can basically reproduce the spatial distribution characteristics of precipitation.The spatial distribution of precipitation simulated by EC-Earth3, EC-Earth3-Veg, MPI-ESM-1-2-LR, and MPI-ESM-1-2-HR is closest to that of observations, with the R greater than 0.80.Corresponding to the larger dry bias in regional mean modeled precipitation, ACCESS-CM2, BCC-CSM2-MR, and FGOALS-g3 underestimated precipitation over much of the NEC region.The remaining models with large bias from observed precipitation overestimate precipitation across the region, with positive deviations concentrated on the southeastern side of the Da Xing-AN Ling, where the relative bias exceeds 90%.There is a signi cant difference in the centers of high precipitation simulated by the CMIP6 model.Most of the models can reasonably simulate the precipitation center on the east side of the Da Xing-AN Ling, and the model simulation bias are small.On the other hand, the modeled precipitations in the southern part of NEC exhibit large uncertainties (Figure .2a),and there is a north-south bias in the location of the precipitation center.Among them, the locations of precipitation centers simulated by CanESM5 and FGOALS-g3 models are signi cantly northward.
Taylor diagrams can provide a way of graphically summarizing how closely a set of modeled patterns matches observations.These diagrams are especially useful in gauging the relative skill of many different models (Fig. 3).More than half of the model spatial distributions have smaller standard deviations than observations, suggesting that the modeled precipitation is smoother relative to observations.The INM-CM5-0 model generally overestimates precipitation, with greater relative bias in the center of precipitation low.GFDL-ESM4, although the best simulation of regional average precipitation, underestimates precipitation in major precipitation centers and overestimates precipitation in arid regions in the west, resulting in a poor reproduction of the decreasing precipitation gradient from southeast to northwest.However, model INM-CM5-0 and GFDL-ESM4 simulate less variability in the spatial gradient of precipitation.In terms of the centered RMSE, most of the models have RMSE less than 1, and the models EC-Earth3, EC-Earth3-Veg and MPI-ESM1-2-LR simulate precipitation with less difference from observation.Overall, the EC-Earth3 model is optimal for modeling the spatial distribution of precipitation in the climate state.
In addition to spatial variability, summer precipitation in NEC exhibits more pronounced interannual and interdecadal variability (Liang et al., 2011).A single model or an ensemble of multiple models has a poor ability to simulate precipitation time series.The simulation accuracy of precipitation is relatively low in both climate and weather models.The simulation ability of the CMIP5 model for precipitation in China is much less easy than that of temperature, and it is found that most of the model-simulated precipitation time series show poor correlation with observations.The simulation of summer precipitation in the NEC region also shows poor simulation performance, and the CMIP6 model and observation time correlation coe cients are given in Table 1.Except for the correlation coe cient between GFDL-CM4 and observations, which reaches 0.40, the correlation coe cients of the other models are less than 0.30 and do not pass the signi cance test.Therefore, the correlation coe cients cannot be used to measure the performance of the model in simulating the interannual variability of precipitation.
Although summer precipitation in the NEC region does not show a signi cant linear trend, it exhibits a non-negligible interannual variability.The IVS is a metrics for measuring model performance regarding interannual variability.The IVS takes into account the magnitude and spatial distribution of the interannual variability and scores the simulation performance of the model in simulating the interannual variability of precipitation, and the IVS closest to zero indicates that the model simulation is the best.).The simulation of a single optimal model is not necessarily better than that of multiple models, and the uncertainty of model projection may be large if only one model is used for future projection.Therefore, based on the comprehensive ranking of each scoring index, 20 CMIP6 models were sequentially added to the model ensemble members, and the effects of different numbers of model ensemble members on the model simulation performance were analyzed.Figure .6shows that the simulation performance of the multi-model ensemble, except for interannual variability, increases and then decreases or remains essentially stable as the number of model ensemble members increases.The metrics SCC, RESM, and SDR show that the performance of the model simulation reaches a peak when the number of model ensemble members is selected as three.While IVS increases with the number of models, this may be due to the fact that multi-model ensemble averaging makes the simulation errors of different models cancel each other out, resulting in a small interannual variability of model-simulated precipitation.A comprehensive ranking of the simulation performance for selecting different numbers of models reveals that the ensemble consisting of the top three selected models has the best comprehensive performance.Therefore, in the subsequent precipitation assessment and projection, the models with the top three overall rankings (EC-Earth3-Veg, EC-Earth3, and MPI-ESM1-2-LR) were selected as the optimal model ensemble (HMME) in this study.
In order to validate the performance of HMME, the simulation results of HMME and Multi-Model Ensemble (MME) including 20 models were further comparatively analyzed.Figures.7a and 7c show that the spatial distribution of the precipitation simulated by HMME is closer to the observation, and the correlation coe cient between HMME and observation is 0.93, which is higher than that of MME and the single model.For the precipitation assessment in the NEC region, unlike the model ensemble selected by Yang et al. (2021), which showed no signi cant improvement, the HMME selected by us can effectively reduce the regional-average precipitation bias (Figure .7e).In MME, the wet bias is 14.9%, which is reduced to 8.69% in HMME.The reduction of simulation bias may mainly come from the reduction of wet bias on the southeast side of Da Xing-AN Ling Mountains.
Finally, we further compare the simulation performance of the ensemble model MME and HMME for the circulation.Figure 9 shows that there is no signi cant change trend in summer precipitation in NEC from 1979 to 2014.Both MME and HMME can simulate this characteristic of precipitation in historical periods.Compared with MME, HMME narrows the uncertainty interval of precipitation.The average daily precipitation simulated by HMME varies from 1.87 to 5.87mm/day, which is closer to the observation of 2.85-4.94mm/day.This shows that HMME effectively reduces the uncertainty of summer precipitation in NEC.Although the correlation coe cient between the observed and HMME simulated precipitation time series is -0.16, there is no improvement compared with the MME-observed ones.However, HMME improves the simulation of interannual variability by reducing the number of models in the ensemble.The ratio of standard deviations between HMME and observations is 0.68, indicating that HMME is closer to observations.
In general, the main Bias of the CMIP6 model in simulating summer precipitation in the NEC is due to the overestimation of precipitation in the southeastern part of the Greater Khingan Range.The HMME simulation bias in this region still exists, but the magnitude is reduced.Furthermore, HMME exhibits higher skill in reproducing the interannual variability of precipitation.

Future Projection
Figure .10shows the time series of regionally averaged summer precipitation anomaly changes in NEC under the HMME historical experiment and the SSP245 and SSP585 scenarios.SSP245 represents an integrated scenario of medium-emission socioeconomic development pathways with low and medium radiative forcing.SSP585 represents a combined scenario of a high-emission future socio-economic development pathway and strong radiative forcing, under which global greenhouse gas emissions will continue to increase rapidly.This leads to an increase in the concentration of greenhouse gases in the atmosphere, which in turn triggers a sustained and pronounced global warming.As can be seen from Figure .10a,the overall average daily precipitation in summer will show a signi cant increasing trend in the future.The increase in the SSP585 scenario (0.15mm/10a) is larger than that in the SSP245 scenario (0.05mm/10a).In the near term (2021-2040), mid-term (2051-2070) and long-term (2081-2100) of the 21st century, the precipitation increments are 9.24%, 15.54% and 31.95%respectively.Figure .10(b-c)further shows the spatial distribution of long-term precipitation changes in the 21st century.In the future, summer precipitation in the NEC will increase as a whole, and the amplitude of precipitation change will generally decrease from south to north.However, precipitation trends vary in different regions at different times (Figure .11).Under the SSP245 scenario, precipitation in the eastern part of NEC shows an increasing trend in the near term and a signi cant decreasing trend in the mid-term in the 21st century.Under the SSP585 scenario, except for the decreasing trend in the eastern part of NEC in the mid-21st century, the entire region shows a signi cant increasing trend in precipitation.

Conclusions.
In this paper, we evaluated the performance of 20 CMIP6 models to simulate summer precipitation in NEC over the period 1979-2014, and the error metrics and interannual variability of the Taylor diagram were Figures The

Figure. 4
shows that the CESM2 and EC-Earth3-Veg models are the worst and best models for simulating interannual precipitation variability, respectively.The IVS of most of the models are less than 0.50, which indicates that the CMIP6 model can effectively simulate the interannual variability of summer precipitation.

Figure 5
Figure 5 shows the ranking of the simulation performance of the 20 CMIP6 models based on different metrics.The above assessment shows that different modes show different simulation performance on different metrics, e.g., CanESM5 is ranked 3rd based on SDR, but ranked lower on SCC and RMSE.Therefore, it is necessary to combine multiple metrics to assess the simulation performance of the model (Du et al., 2022; Yang et al., 2021; Xin et al., 2020; Zhu et al., 2020).
Figure.8 illustrates the bias of the climatological distributions of the summer sea level pressure, 850-hPa wind, 500-hPa potential height and 200-hPa latitudinal winds simulated by the MME and HMME models, as well as their bias with respect to ERA5 data.Both MME and HMME can simulate the basic characteristics of the East Asian summer monsoon.The southwesterly wind originating from the Bay of Bengal, the South China Sea and the west side of the tropical western Paci c subtropical high is the main source of water vapor for summer precipitation in the NEC region.In the western Paci c, the horizontal winds simulated by the MME and HMME show cyclonic anomalies in the south and anticyclonic anomalies in the north (Figure.8cand Figure.8e).The location of the western Paci c subtropical high at 500hPa shows a negative deviation (Figure.8dand Figure.8f),indicating that the simulated position of the western Paci c subtropical high is northerly.The biases in the MME and HMME simulations may be related to the westerly jet in the upper troposphere, because the north-south drift of the jet position is closely related to summer precipitation in the NEC region(Sun et al., 2017).The position of the westerly jet simulated by MME is northerly, and the improvement of the simulated wet bias by HMME may be due to the fact that the simulated position of the westerly jet is closer to the observation.
regional summer mean daily precipitation (Blue bars indicate an underestimation of the pattern and red bars indicate an overestimation of the pattern, with values corresponding to the left-hand axis.) and standard deviation (Gray, corresponding to the right axis) in the NEC, 1979-2014, as simulated by the CMIP6 model history experiment.The red dashed line in the gure shows the observed mean precipitation and the gray dashed line shows the observed standard deviation (Unit: mm/day).

Figure 2 Pattern
Figure 2

Figure 5 Based
Figure 5

Figure 10 Time
Figure 10

Table 1
General Information on selected GCMs from the CMIP6 used in this study