Assessing precipitation seasonal forecasts in Central Africa using North American Multimodel Ensemble (NMME)

This study examines the seasonal forecast of the North American Multi-Model Ensemble (NMME) over Central Africa (CA), which encompasses a region of the world where the economies of the countries are highly dependent on agriculture and livestock breeding. Following many regional climate perspectives, we evaluated the seasonal forecast over the 4 seasons: December to February (DJF), March to May (MAM), June to August (JJA), September to November (SON) between 0 and 5 months lead time before the beginning of each season. Deterministic and categorical approaches which focus on the rainfall variable were used to assess NMME ensemble mean (MME). The observed and predicted rainfalls have been divided into three categories: below normal, normal, and above normal. The results show that for 0 to 2 months lead time, the MME reproduces well the peak rainfall of the Atlantic coast and in the East of Democratic Republic of Congo in MAM and SON between 9 and 10 mm/day. Again in the same lead time interval, values of correlation coefficients (R) between the MME and the Global Precipitation Climatology Center (GPCC) reference observation of all seasons are greater than 0.72. For 3 to 5 months lead time, lower values of R are observed. It follows that probabilities of detection (POD) are greater than 50% for all different normal seasons and less than 45% for below and above normal seasons. On the other hand, high false alarm (FAR) values and low Critical Success Index (CSI) values are observed for both below and above normal seasons. From our results, one can argue that the NMME seems to be an interesting tool during the first three forecasting lead times in CA capable of providing important seasonal characteristics before the start of each season, which will allow proper consideration of meteorological phenomena.


Introduction
Central Africa (CA) is one of the regions of the world where rainfall is most complex and varies on multiple times scales (Jenkins et al 2005). The variations in rainfall, through the phenomenon of droughts and floods, are real scourges; this is because health, agriculture, energy, and biodiversity are closely linked to water. The predictability of water resources in CA has become essential nowadays because the economy of its countries depends on them (Biman et al 2004).
Seasonal forecasts are often used to assess whether months or seasons will be warmer or colder, drier or wetter than normal. They are also issued at different time scales depending on the user's needs. Seasonal forecasts are used to quantify the availability of water resources and to adapt agricultural practices, particularly in Western and Central Africa (Sultan and Janicot 2013). After more than three decades of research on the origins of seasonal climate, progress is being observed worldwide (Wu et al 2003;DeWitt 2005). This progress is due to multiple approaches: international and multi-institutional collaborations and the efforts of researchers. Techniques of quantitative information regarding uncertainty in forecasts were observed (Mason and Goddard, 2001;Hagedorn et al 2005;Yuan et al 2011). The other approach is to recognize multi-model strategies used to determine the uncertainty existing in forecasts (Wang et al 2005;Palmer et al 2008). In addition, several studies (e.g., Shukla et al 2000;Mishra et al 2019;Kim et al 2021) have shown that the multimodel ensemble (MME) approaches that include forecasts from multiple climate models can minimize errors from model biases. This is particularly important in long-term forecasts, as model errors accumulate and increase progressively. NMME has been developed to provide information on the problems of climate variability and especially to improve the quality of seasonal forecasts around the world (Kirtman et al 2014;Thober et al 2015;Shukla et al 2016;Slater et al 2016;Giannini et al 2020).
Several recent studies have analyzed the skill of NMME worldwide. Mo and Lyon (2015) assessed the skill of NMME using Standardized Precipitation Indices (SPIs) in global land areas. They established that the skill of NMME depends on the season, region, and specific index (3-and 6-month SPIs were assessed). They also showed that NMME model skills are statistically significant between 1 and 2 months lead time over all seasons. Slater et al (2016) studied the skill of eight NMME in seven regions of the USA. They found that most models perform well at the shortest lead time, and performance can decrease thereafter. Shukla et al (2016) assessed the skill of NMME seasonal forecasts in East Africa (EA), using deterministic, categorical, and probabilistic assessment methods for three main seasons, March to May (MAM), July to September (JAS), and October to December (OND). Their results revealed that the models perform better over a small portion of the domain. In addition, they noted that NMME was incompetent in simulating interannual variability, but that rainfall prediction was stronger during ENSO. By using a principal component analysis on the anomaly correlations of 10 sets of global rainfall forecasts from NMME, Zhao et al (2019) found that their skills depend on the accuracy of the data assimilation algorithm.
Although many of the above studies provided insight into the predictive ability of different NMME models around the world, CA has received little attention in seasonal forecasting, making it difficult to generalize the results already established by the researchers. An attempt to fill these gaps led us to the formulation of the following research questions: What is the skill of the MME in predicting rainfall for each season in CA? What is the influence of the different lead times on the seasonal forecasts in the region? At a seasonal time scale, how do MMEs predict below-normal, above-normal, and normal rainfall categories? The answers to these questions have major applications for seasonal, hydrological, and meteorological forecasting in CA and need to be explored.
The objective of this paper is to evaluate the performance of MME in CA at different lead times. Our evaluation consists of three main parts: (1) evaluate the forecasting ability over 4 seasons: December to February (DJF), March to May (MAM), June to August (JJA), and September to November (SON) between 0 and 5 months lead time before the start of each season in CA; (2) analyze the influence of lead time on the quality of forecasts over CA; (3) evaluate the performance of MMEs to predict the different below normal, above normal, and normal seasons at different lead time (0-5 months). Our results constitute a diagnostic tool that can provide model developers with feedback on the strengths and weaknesses of their models, and contribute to their development. The remaining part of this paper is structured as follows. Section 2 presents the details of the NMME, the observational data, and methodology. The performance evaluation of MMEs is presented in Sect. 3. Section 4 presents the discussion of this study. Finally the conclusion is presented in Sect. 5.

Description of the study area
The study area is shown in Fig. 1 and is defined by the latitude − 10 to 16° and longitudes 5 to 33°. CA is surrounded by and divided into mountains and plateaus, namely the Joss Plateau (Nigeria), the Mandara Mountains, and the Adamaoua Plateau (northern Cameroon), and the highlands of western Cameroon, Mount Cameroon (southwestern Cameroon), the Yadé Massif (western Central African Republic), the Cristal Mountains (between southern Equatorial Guinea and northwestern Gabon), the Batéké Plateau (southern Congo). These plateaus and mountains influence rainfall in the region and climate is becoming very complex and represents a considerable challenge to climatic models (Vondou et al 2010;Fotso-Nguemo et al 2018).

NMME data
NMME hindcasts are obtained from Columbia University's International Research Institute (IRI) data library and are downloaded from (https:// iridl. ldeo. colum bia. edu). NMME data are monthly with horizontal resolution of 1 • × 1 • longitude/latitude. NMME combines several fully coupled climatic models to provide real climate forecast systems for a period of more than 8 months in advance. Some models produce forecasts up to about 11 months in advance (see Table 1). The description of the different models is  Table 1 List of NMME outputs and their configuration use in this paper

Models
Hindcast period Lead time (months) Ensemble size Institute and model version References  Table 1 (Kirtman et al 2014). The data contain 5 dimensions namely the longitude (Lon), latitude (Lat), member (M), lead time (L), and initialization time (S). The term lead time indicates the period between the forecast initialization time and the forecast month. In this work, the NMME is evaluated relative to two observation datasets which are described below.

Observation datasets
Two reference observational data are used in this study. These are the Global Precipitation Climatology Center (GPCC) Becker et al 2013) and the NMME-CPC-CMAP PRATE observations data source (Zhao et al 2019). Their respective spatial resolutions are 0.5 • × 0.5 • and 1 • × 1 • . The GPCC and NMME-CPC-CMAP PRATE contain 3 dimensions (longitude, latitude, and time). The two observations are downloaded over the same period 1982-2009. The NMME-CPC and GPCC datasets are all remeshed to 1 • × 1 • to match the NMME models grids as in Kirtman et al (2014) and Liu et al (2017). This data preprocessing facilitates comparison of results for different forecast sets.

Forecast evaluation metrics
The skill of NMME is evaluated using retrospective forecasts of 11 dynamic models for the period 1982-2009. MME was examined by first designing ensemble means of each individual model for each season and then averaging the ensemble means of all models as in Shukla et al (2016) and Tanessong et al (2020). The strengths of this method lie in a procedure for optimizing deterministic forecasts and evaluating uncertainty due to model imperfections. Various models, although very different in their parameters, can give rise to equivalent overall skill (Zhang and Xiaoliu 2018;Kim et al 2021).
• In the first part of this evaluation, the climatologies of two observations (GPCC and NMME CPC PRATE) were calculated, and then a difference between the 2 datasets were calculated in order to measure the uncertainty that may exist between these data observations. In addition, the climatology of the MME for each season DJF, MAM, JJA, and SON, between 0 and 5 months lead time in advance is represented in order to make comparison with the observations.
• The correlation coefficient (R) is widely used and allows to highlight a linear link between forecast and observation. The R is given by Eq. (1) below, while the expressions of the parameters p , K , PetK are given by Eqs.
(2), (3), (4), and (5) respectively. In Eqs. (2), (3), (4), (5), and (6), P j indicates the predicted value, while K j indicates the observed value. N is the total number of grid points in the domain. The calculation of standard deviation ( ) provides the measure of dispersion of the different MME. R and are summarized in the Taylor diagram allowing better measurement of the degree of similarities and differences between the MME and a reference observation GPCC (Taylor 2001).
where p and k are the standard deviations of the predicted value p of the model and the observation value k defined respectively as: The mean values K and P are respectively given by: • The root mean square error (RMSE), given in Eq. (6), is normalized in this study by the climatological mean of the two observations and is given by Eq. (7). It facilitates the comparison between the forecast and the observation of each season.
• The mean absolute error (MAE) is given by Eq. (8). MAE is the absolute difference between the forecast value and that of the observation of each season in CA.

Categorical skill scores
In the context of seasonal forecasts, it is not realistic to believe that one can know the exact value of the amount of rain that will actually fall in an area. However, it is likely to know if the season will be dry or wet than a normal season through an assessment of the skills of the MME. The tertile categories above normal (rainy), below normal (dried), and normal are used as in Tippett et al (2007). The contingency table (Table 2) is also used herein. According to Shrivastava et al (2019), rainfall during a season is classified as below normal when anomalies are less than − 0.43 , normal when the anomalies are between − 0.43 and 0.43 and above normal when the anomalies are greater than 0.43 . represents the value of standard deviation for each grid point and for each season. The seasonal rainfall anomaly of different seasons JJA, MAM, It is between 0 and 1. The advantage that exists on the CSI over POD and FAR is that it is very sensitive to false alarms and missed events; thus, it gives a more representative idea DJF, and SON is a difference between total seasonal rainfall and climatology. The climatology is the mean of the seasonal rainfall for the period 1982 to 2009. The contingency table (Table 2) allows the calculation of the following meteorological parameters.
• Probability of detection (POD) is the ratio of observed "yes" events in each category that were exactly predicted. It varies from 0 to 1 (100%) with a perfect number 1 (100%).
• False alarm (FAR): event forecast to occur, but did not occur in each category. The score varies from 0 to 1 (100%) with a perfect score of 0. ; ; • Critical Success Index (CSI) is a verification measure of categorical forecast performance which allows calculating the relative precision of each season. For the 3 categories, it is given by: of the real precision. It also allows for more precision in situations where rare events are involved.
• Correct Percent (PC): it can be used to assess confidence in the forecasts for each season.
where M = 28 represents the period 1982-2009. Given that the study period is less than 30 years, the threshold for being more meaningful is closer to 40% (Kharin et al. 2001).

Results
This section presents the results of the MME forecasts by comparing them with the observations. Figure 2 shows the climatology (1982-2009) of both observations GPCC and NMME CPCand the difference between these datasets. From the first to the fourth column, we have DJF, MAM, JJA, and SONseasons respectively. The difference between GPCC and NMME CPC (Fig. 2m-p; fourth row) provides a measure of uncertainty between these two datasets. The result shows that GPCC is overestimated ~ 1.5 mm/day in the eastern part of the Democratic Republic of Congo (DRC) commonly referred to as the great lakes region over all the seasons (Fig. 2m-p), and at the level of the Atlantic coast in JJA and SON (Fig. 2o-p; fourth row). In addition, an underestimation of GPCC ~ − 1 mm/day is also observed in northwest Angola in the DJF and SON(see Fig. 2m and p), in part of southern Cameroon, northern Gabon in JJA (Fig. 2o). The underestimation and overestimation observed in both datasets may be related to the soil The seasonal mean climatology of seasonal forecast between 0 and 5 months lead time before the start of each season is presented in Fig. 3. Columns 1, 2, 3, and 4 represent season DJF, MAM, JJA, and SON respectively. The results show that during the JJA, the MME models capture rainfall in northern CA between 8 and 9 mm/day (see Fig. 3c, j, and k) and show a spatial distribution of rainfall 0-2 months lead time similar to observations over the disparity is 95% significant. A positive bias indicates an overestimation of MME rainfall compared to observations, while a negative bias indicates an underestimation. The results show that between lead 0 and lead 1, the MME models underestimate the observations in northern CA and overestimate the observations in south with an observable maximum in northern Angola and DRC during MAM and DJF. In addition, for the JJA and SON, the MME models overestimate the observations ~ 2.5 mm/day in part of the southwestern region of Cameroon and part of the Gulf of Guinea (Fig. 4c, d, j, h, k). These biases are more pronounced in the northern part of the CA, around 4 to 5 (mm/day), and a visible underestimation ~ − 3 to − 5 (mm/day) in the southern part of the CA, from lead 3 to lead 5 (Fig. 4o, p, s, t, w, x). The results show that the bias increases when the lead time becomes larger. Let us add that an increasing dispersion with the lead time is observed over the 4 seasons (see Fig. 5), both observations show the same trend for the 4 seasons with the median and the inter quartile of the boxplots which show an increasing trend with lead time. All these errors can be a constraint for NMME models to maintain the quality of long-term forecasts.

MME and observation climatologies
The normalized root mean square error (NRMSE) is shown in Fig. 6. The NRMSE increases after the first lead time and reaches a maximum value at lead 5 ~ 0.17 mm/ day in DJF, ~ 0.11 in JJA, and ~ 0.8 mm/day in SON. The analysis of the quality of the rainfall forecasts of the MME shows that the performance of the forecasting system is better for the first 3 months and from 3 to 5 months, the performance of the model decreases.

Taylor diagram analysis between MME and GPCC
The Taylor diagram (Taylor 2001) used presents the spatio-temporal similarity or dissimilarity between MME and a reference observation GPCC (Fig. 7). The black lines relative to the radial axis show correlation coefficient (R) between MME and GPCC observation.  ◂ continent. However, there is slight fluctuation of rainfall observed towards the equatorial part of the region during this season (Fig. 3o, s, w). The fluctuation in precipitation from one area to another can be facilitated by convection, changes in surface flow (Galvin 2008), by the presence of the Intertropical Convergence Zone (ITCZ) (Nicholson and Grist 2003and Jackson et al 2009), although Nicholson (2018 mentions that the precipitation peak does not depend solely on the ITCZ. However, one of the great challenges of modeling rainfall in the sub-region is to present this observed seasonality. During MAM and SON seasons 0-3 months lead time, MME rather shows that the equatorial region of CA is humid and presents a rainfall peak at the equator, around Congo Basin and East of DRC (Fig. 3b, d, f, and h). Rainfall peaks observed during these seasons at the equator follow the trend given by the two observations and can be explained by the existence of ITCZ and the presence of the Congo River. From 3 to 5 months lead time, the SON season presents precipitation instead in the southern part of the region with the peaks in eastern DRC remaining observable (Fig. 3p,  t, and x). This result demonstrated that the quality of the forecast deteriorates with increasing lead time. During DJF, from lead 0 to lead 2, MME shows that the southern part of the region is wet while the northern part is dry (Fig. 3a, e, and i). The two observations GPCC and the NMME CPC PRATE (Fig. 2a, e, i) similarly show the precipitation trend given by MME models. These results can be justified by the fact that monsoon characteristics such as the East African Jet (EAJ), the Tropical East Jet (TEJ) are inactive (Nikulin et al. 2012). It follows that the MME captures the spatial structure of precipitation well with a maximum at the level of Congo and the Democratic Republic of Congo around 7 to 8 mm/day (Fig. 3d, h, l). During the same season, low rainfall is observed in the northern part of the region, in the order of 1 to 3 mm/day.

Deterministic skill analysis
The calculation of the different scores 0 to 5 months lead time before the beginning of the seasons between MME and observations is given in Figs. 4, 5, and 6. Figure 4 indicates the bias between MME and the observation mean. The small black dots observed on these maps come from the significance tests performed on the calculated biases. They are consistent with the points where the GPCC depends on the season and the month of the initialization period of the model, very consistent with results found by Mo and Lyon (2015), Shukla et al (2016), and Givati et al. (2017). Figure 8 represents the probability of detection (POD, in %), the false alarms (FAR in %), and the correct percentage (PC) of all seasons. The results show that the PODs of the above normal and below normal of the 4 seasons, ranges from very low 2 to 20% to low 20 to 40% (Fig. 8a, b, c, d) red color and blue color. However, a low probability in no way indicates that it will be sunny. It means that the probability of raining is low in CA for seasons above and below normal. During normal seasons, MMEs have reasonable POD greater than 50% over all seasons. The values are between 58 and 61% for DJF and MAM (Fig. 8a, b) green color. In JJA and SON (Fig. 8c, d) green color, the models seem to be more calibrated with large POD around 79 to 80% for JJA, and 60 to 80% for SON. The FAR shows values greater than 50% for all below normal seasons (Fig. 8e, f, j, and h) blue color. These values reach the maximum percentage of 98% for the DJF at lead 5, 83% in MAM season at lead 3, around 82% at lead 2 in JJA, and around 98% at lead 5 in SON. The seasons above normal (Fig. 8e, f) red color show FAR values above 60% for the DJF and MAM seasons. These values reach a maximum around 97% at lead 5 in DJF and around 95% to lead 1 and lead 2. The JJA and SON seasons show FAR values around 39 to 40% (Fig. 8j, h) red color, lead 0 and lead 1. From 2 to 5 months lead time, the values are greater than 55%. The important point to emphasize is the link between POD and FAR. When the POD of MME is low, FAR are higher in all seasons. In addition, the FAR and POD values depend on the lead time and seasons of each category. Figure 8i, g, k, l show the correct percentage (PC) for the 4 seasons. The PC is used to assess the confidence that can be placed in the forecast for each season. The results indicate that the PC values are between 39 and 60% for MAM, DJF, and SON. The JJA presents the values between 41 and 61%. However, since the data used to calculate the correct percentage only covers a 28-year period, the threshold for being statistically Fig. 5 Box plot representing the mean absolute error (MAE) between the two observations GPCC and CPC NMME PRATE and the MME. For DJF, MAM, JJA, and SON columns 1, 2, 3, and 4 respectively between 0 and 5 months lead time before the beginning of each season. (a-d) MAE between MME and GPCC. (e-h) MAE between MME and NMME CPC PRATE significant is closer to 40% (Kharin et al 2001). These results show that the confidence level of each period is close to or greater than the defined threshold. After calculating different POD, FAR, and PC, it is important to observe the Critical Success Index (CSI), frequently used to verify the accuracy of predictions. Figure 9 shows the CSI for the normal, above, and below normal seasons.

Categorical skill analysis
The results indicate that the DJF seasons below and above normal, MAM above normal, JJA below normal, and SON below normal have values less than 0.21 justifying that Fig. 6 Normalized root mean square error (NRMSE) between MME and the mean observation (GPCC and NMME CPC PRATE). 0-5 months lead time before the start for each season during these seasons, less than 50% precipitation was well predicted by the MME. The CSI values for the DJF, MAM, JJA, and SON seasons are higher compared to the season below and above normal. This result shows that during normal seasons, just over half of the rainfall was correctly predicted by MME with an above-average percentage.

Discussion
Although existing research has captured the background and important aspects of seasonal forecasting in some regions and countries of the world, our study extends them in CA by adding that the MME over 4 seasons favorably reflect the spatial structure of rainfall between Fig. 7 Taylor diagram presenting the correlation (R) and the standard deviation between the MME (1982-2009) namely (MME lead0, MME lead1, MME lead2, MME lead3, MME lead4, and MME lead5), with GPCC the observation of reference 0 and 2 months lead time and represent the unimodal and bimodal nature of the region. Rainfall maxima around 8 mm/day are observed in JJA in the north of the region between 0 and 2 months lead time. In DJF, MAM, and SON, peaks are observed in the East of the Democratic Republic of Congo between 7 and 8 mm/day. The rainfall extremes obtained in this region may be caused by the proximity of the Congo River and also the Great Lakes region (Tanganyika Lakes). The hypotheses illustrating these observed rainfall maxima may be related to the geographical position of the study area as well as differences in natural variability and surface water dynamics (Coppola et al 2014). CA is at the center of a continent crossed on both sides by the equator. Moreover, this region is surrounded by several mountains and plateaus that can cause orographic rainfall due to forced convection of air masses (Fotso-Nguemo et al 2018).
It was found that the MME have a large bias over all seasons, and this bias increases with lead time. A similar bias was observed in East Africa by Shulka et al. (2016). Let us add that the causes of these errors can also be related to failures in the physics of the models, such as failures in the modeling of clouds, the coarse resolution climate models being inactive in the simulation of mesoscale convection (Biasutti 2013;Fotso-Nguemo et al 2017), which introduce biases in the global models. It is important to mention that (a-d) POD (above normal, below normal, and normal). The green color represents the normal season, the blue color represents the below normal season, and the red color represents the above normal season. (e-h) FAR (above normal, below normal, and normal). The green color represents the normal season, the blue color represents the season below normal, and the red color represents the season above normal. (i-l) PC for each season DJF, MAM, JJA, SON these biases require correction before applying in certain fields of such as hydrology.
The seasonal analysis in CA shows a decrease in model performance with an increase in lead time. Although there is a slight difference between 0 and 2 months of lead time, it was found that going further, up to 4 or 5 months of lead time, results in a decrease in accuracy (R ~ 0.11 for MME of the DJF, JJA, and SON seasons) at lead 5. The low performance of the models can be justified by the chaotic nature of the atmosphere as well as errors due to model initialization. This follows the results of the literature review, which indicate good performance of seasonal precipitation forecasts up to weeks or even months before the start of the seasons (Crochemore et al 2016). Our results are very consistent with what has been found by Slater et al (2016) and Tanessong et al (2020) justifying model proficiency in the first month of the forecast and with a decrease in performance as the lead time increases.
The seasonal forecasts from the categorical analysis show that the POD and CSI for normal seasons are high and remain very low for above and below normal seasons. This indicates that the NMME forecasts may be more useful in capturing normal events. In contrast, the NMME forecast ensemble does not appear to be better suited to capture the dominant above-or below-normal mechanisms responsible for environmental degradation in CA. A thorough study of the associated large-scale circulation, however, is needed to understand the source of this asymmetry in forecast capability.
The methods used in this work can be applied to other countries and other intertropical regions that use an integrated approach to water resources management, which requires rainfall to deduce the optimal management policy. The results of this study have direct implications for the Regional Climate Outlook Forum for Central Africa (known by its French acronym: PRESAC), coordinated by the African Centre of Meteorological Applications for Development (ACMAD). Although NMME is a promising tool for seasonal forecasting efforts in CA, the skill of rainfall forecasts is limited to several months, and caution should be exercised in their use for decision-making purposes for centers that emphasize the use of forecasts in hydrology and crop yield. With this in mind, scientists can explore alternative methods to downscaling NMME forecasts in order to improve precipitation forecast skill in the region. For example, Shukla et al. (2014) proposed a hybrid approach that uses the constructed analogue method on the NMME's dynamical forecasts to improve the MAM rainfall forecast skill in the equatorial EA. There is also an opportunity to improve forecast skill through multivariate methods such as canonical correlation analysis (e.g., Feddersen and Andersen 2005).

Conclusion
In this study, we analyzed the performance of the multimodel ensemble over CA. The evaluation focused on the DJF, MAM, JJA, and SON using both deterministic and categorical methods. The main results show that the MME identically reproduces the spatial structure of the two reference observations over the continent with a large bias observed over the 4 seasons. Taylor diagram shows correlations greater than 0.72 for the 4 seasons between lead 0 and lead 1 and shows low values as lead time increases. Seasonal forecasts from the categorical analysis show that the skill of NMME rainfall forecasts is higher during normal tertile seasons, suggesting that they can be used with more confidence for decision-making during normal seasons. Our analyses indicate that the MME in CA allows for proper planning and management of water resources, and may also contribute to more informed decisions such as agriculture, water withdrawals in aquifers and lakes, reservoir operations, and water resources management. Water decision-makers in central African countries will be able to decide whether to act or not, knowing the forecasting skills for the different lead times. In addition, this study will allow farmers in the region to make better preparations, such as purchasing equipment, stocking suitable seed varieties for better agricultural planning, and preventing regional food insecurity. However, it is important to note that when using these applications, the following caveats regarding the study should be noted. First, the predictive capabilities vary by region or variable, even in the best performing MME system. Subsequently, methods of de-biasing the model parameters can be used. (e.g., quantile-quantile mapping) must be applied to the raw model data. Following the results found, further analysis should be carried out by calculating new scores, more used in the context of ensemble forecasting (e.g., Continuous Ranked Probability Score (CRPS)). Similarly, post-processing of NMME forecasts could be performed, to improve the performance of the seasonal forecast model, especially with Bayesian methods appropriate for probabilistic forecasts. Fig. 9 MME Critical Success Index (CSI), first-line DJF (below normal, above normal, and normal). Second line MAM (below normal, above normal, and normal). Third line JJA (below normal, above normal, and normal). Fourth line SON (below normal, above normal, and normal) ◂