2.1 Societal factors in the model tend to improve the hindcasting performance for capacity expansion
We find that incorporating societal factors in the model one by one or as combinations mostly improves the hindcasting performance in terms of installed capacity dynamics for most of the 31 European countries studied (Fig. 1). Only in the cases of Iceland and Latvia, the model version with the best hindcasting performance (i.e. lowest error) is the techno-economic one without societal factors incorporated. For the rest, the model version with the best hindcasting performance includes at least one societal factor. We obtain high heterogeneity between countries in terms of hindcasting performance gain compared to the techno-economic model. Minor gain in hindcasting performance is achieved for Lithuania, Norway, Slovenia and Switzerland (less than 1% of symmetric mean absolute percentage error (SMAPE)) and low gain (less than 5%) for another nine countries (Belgium, Bulgaria, Croatia, Cyprus, Greece, Malta, Poland, Portugal, Romania). Countries with minor gains are characterized by low cumulative installed capacities in 1990, below 10 GW, and a limited diversity of technologies, or by a high proportion of hydroelectric power plants, which induces inertia in the evolution of the system due to long lifetimes. Values for Austria, Denmark, France, Germany, Hungary, Ireland, Italy, Luxembourg, Netherlands and Slovakia lie in between 5% and 15%. We obtain the highest hindcasting performance gain of more than 15% of SMAPE for six countries (Czech Republic, Estonia, Finland, Spain, Sweden and United Kingdom). This is mainly because, contrary to the techno-economic model, the inclusion of some societal factors in the model allows to better capture the increase in installed capacities of decentralized renewable technologies (e.g. biomass, offshore and onshore wind and solar PV) observed in these countries from 2010.
Even if the inclusion of one or several societal factors can improve the hindcasting performance for most of the countries, there are many cases when including some specific factors or their combinations, the hindcasting performance gets worse than the performance of the techno-economic model (Fig. 1.). Societal factors hence need to be included in an evidence-based way to avoid worsening the model’s performance and hindcasting is a relevant approach to do so. We also observe a large spread in terms of absolute hindcasting performance (Fig. 1). On the one hand, for Belgium, Lithuania, Luxembourg, Malta, Netherlands, Portugal and Switzerland, the model version with the worst hindcasting performance has a better performance than the average of the whole simulation set, all countries considered. Except for Luxembourg and Netherlands, low performance gains are obtained for these countries as well, pointing to a tiebreak between additional modeling efforts and actual gain. On the other hand, for Bulgaria, Norway and Sweden, the model version with the best hindcasting performance gives more errors than the average value of the whole hindcasting simulations set. For this second group of countries, this suggests a potential for improving the model structure and the way that societal factors are incorporated in order to improve the model performance to capture capacity dynamics.
2.2 The contribution of the different societal factors to model hindcasting performance differs between countries and factors
The selection of societal factors leading to the best hindcasting performance differs between countries (Fig. 1). First, there is a difference in terms of number of societal factors incorporated, ranging from one for Croatia, Cyprus, Lithuania, Malta, Norway, Romania and Switzerland to four for Austria, Denmark, Germany, Italy and Luxembourg. There is no country where the integration of more than four factors leads to an additional gain in performance. This suggests that increasing the complexity of the model does not necessarily increase its accuracy, supporting previous findings on complexity of technology representation in models [32]. Second, the best combinations differ in terms of societal factors incorporated. Each societal factor is present for at least four countries in the model version giving the best hindcasting performance which suggests that all societal factors analyzed are potentially relevant to incorporate in integrated assessment and energy systems models. Lock-in, Public acceptance and Investment risks are the societal factors most present in the model versions giving the lowest errors, with respectively 14, 14 and 12 occurrences over 31 countries (Fig. 1).
Model hindcasting performance is more sensitive to the incorporation of some societal factors in the model than others, both positively and negatively. Actors’ heterogeneity has the highest influence in most countries (Fig. 2b) which is expected since this factor accounts for cost sensitivity which in turn drives the shares of new investments for all technologies. The second most influencing factor overall is the Fast transition factor which represents more than 20% of the variance for ten countries (Fig. 2b) but tends to decrease the hindcasting performance for them (Fig. 2a). Public acceptance is the third most influential factor representing more than 10% of the hindcasting performance variance in six countries (Austria, Germany, Netherlands, Portugal, Romania, Switzerland), but with a negative effect on model hindcasting performance for Switzerland and Romania (Fig. 2a and 2b). This highlights the importance to be cautious when introducing or omitting these three factors because they are not always beneficial.
Interactions of certain specific societal factors also have a high influence on model hindcasting performance for Italy and Netherlands where they represent more than 25% of the variance (Fig. 2b). In the case of Italy, this is mainly due to a synergistic effect of Fast transition and Actors heterogeneity that gives an overall hindcasting gain greater than the sum of the effects of each factor incorporated individually (Supplementary Fig. S1). Conversely, for the Netherlands, we obtain antagonistic effect with the combination of Public acceptance and Governance and with Actors heterogeneity and Governance, which give lower performance than the sum of the effects of each factor incorporated individually (Supplementary Fig. S1). Remarkably, the integration in the model of any societal factor, one by one or in combination, has limited influence on the model hindcasting performance for Lithuania and Portugal (Fig. 2a). Lithuania is characterized by an oversized supply system compared to its electricity demand with low technology diversity, creating a locked dynamics in all modeled pathways. For Portugal, the much lower costs for coal and wind power compared with other technologies drive the capacity dynamics in the model.
2.3 Societal factors in the model induce a hindcasting performance trade-off among model outputs
For most countries the combination of societal factors giving the lowest deviation from historical data for capacity dynamics is not the same as giving the lowest deviation for CO2 emissions and the share of renewable electricity production (Fig. 3). We hence obtain a trade-off in hindcasting performance between model outputs but to a different extent between countries. For a first group of sixteen countries composed of Austria, Bulgaria, Croatia, Cyprus, Czech Republic, Estonia, France, Germany, Hungary, Italia, Malta, Norway, Poland, Portugal, Spain and United Kingdom, the model version giving the best hindcasting performance for installed capacity also improves the model performance for the two other outputs compared to the techno-economic model. For the second group of twelve countries composed of Belgium, Denmark, Finland, Greece, Ireland, Luxembourg, Netherlands, Romania Slovakia, Slovenia, Sweden and Switzerland, the model version giving the best hindcasting performance for installed capacity gives more errors than the technoeconomic model for at least one of the other outputs. This suggests the need to use a combination of societal factors, like a country-specific pareto-optimal set of factors, that altogether would allow to improve the hindcasting performance for the three metrics, and other relevant model metrics too. For Belgium, Denmark, Finland, Ireland, and Slovakia, there are combinations of societal factors that are relevant to improve the performance for the three model outputs. Conversely, this is not the case for Switzerland, Greece, Romania and Slovenia, suggesting the need for these countries to better represent the societal factors or to further modify the original structure of the model.