Analyzing Self-Evaluation Capacity Scores Related to Infectious Disease Control in International Health Regulations during the First Year of the COVID-19 Pandemic

Background: COVID-19 is hitting the world hard despite the requirement to establish minimum core capacity by the International Health Regulations (IHR) 2005. In addition, COVID-19 was most severe in countries reporting good IHR capacities, such as the US, UK, Germany, and France. This study aimed to identify changes in the average score of member countries' IHR self-evaluation capacity (e-SPAR) in 2020 compared to 2019, then analyze the factors that in�uenced it. For that purpose, �rst, we calculated the absolute difference between e-SPAR scores in 2019 and 2020 of 154 countries. Then Wilcoxon Sign-rank test was performed to identify its signi�cance. After Chi-Square Test, Multiple Regression Analysis was applied to analyze the relationship between countries’ changes in e-SPAR scores and their COVID-19’s case fatality rate (CFR), Human Development Index, Civil Liberties, and Government Effectiveness. Results: Our �ndings showed that the average e-SPAR score in 2020 was signi�cantly higher than in 2019. Of the 154 countries, the e-SPAR score of 98 countries (63.63%) was increased in 2020. Among this group, 63 countries (64.28%) were middle-income countries. Capacities about Risk Communication, National Health Emergency Framework, and Ports of entry were the three capacities that experienced the most signi�cant increase. Meanwhile, among 56 of 154 countries (36.36%) whose scores did not increase, 26 countries (46.42%) were high-income countries. The IHR coordination and National Focal Points, Food Safety, and Risk Communication were the only capacities that signi�cantly decreased during the pandemic. Our model showed that COVID-19 CFR in�uenced the changes in e-SPAR scores of 154 countries (p<0.01), as well as the countries whose score increased (p<0.05), but not the countries whose e-SPAR score did not increase. Conclusion: We concluded that the COVID-19 pandemic is an alarm for countries to re-evaluate their capacities more carefully. Continues external evaluation approach like Joint External Evaluation (JEE) should be a necessary approach for future preparation.


Introduction
Emerging infectious disease like COVID-19 is continuously a signi cant health and security challenge for the world. After the SARS pandemic in 2003, International Health Regulation 2005 (IHR 2005) was adopted by the World Health Organization (WHO) to help countries in setting up their national preparedness for an e cient early alert and response system in handling public health events and emergencies that have the potential to cross borders (1). One of the approaches adopted by IHR 2005 is to require member states to develop minimal core capacities for infectious disease control. To monitor the progress, WHO requires member countries to conduct a self-evaluation of IHR core capacity annually (2,3). While the lack of involvement of member countries in responding to the requirement led to several global pandemics during 2012-2015 (2,4), an external evaluation approach (Joint External Evaluation, JEE) is introduced and implemented from 2016 (3).
These approaches have been proven to be accountable and effective in helping member countries increase their capacity to prepare for major public health threats (5)(6)(7). The study used infectious disease threat events (IDTE) data between 2010 and 2016 showed a 10% increase in IHR core capacity score associated with a 14-20% decrease in cross-border IDTE incidence globally (6). The study focused on the relationship between countries' IHR core capacity score and their infectious disease control outcome showed that countries with low average scores of IHR State Party Self-Assessment Annual Report (SPAR) in 2016 and 2017 had a signi cantly higher risk of having a bad control out of any infectious disease than countries with high IHR average scores (3). However, some studies found that some countries' selfreported scores were consistently 1 to 1.5 points higher than externally assessed scores in the same year.
This nding re ects the phenomenon that countries tend to over-score their capacity, especially countries that lack transparency.
Currently, the world is still under the hit of the COVID-19 pandemic. In March 2020, the early stage of the global pandemic, a paper evaluated the core capacity of countries and claimed that countries labeled with strong operational preparedness capacities on e-SPAR (Electronic State Party Self-Assessment Annual Report) should have the capacity to respond to the COVID-19 pandemic effectively (8). Currently, there are more than 250 million cases and 5 million deaths globally, and the majority of them were contributed by the Americas and European region (9). This phenomenon showed that the self-evaluation score of IHR capacity reported by member countries was not representative to measure their readiness to face major public health threats. Also, the reality of the COVID-19 control outcome and countries' selfreported core capacity score before the pandemic seems to be incoherent. Recent studies evaluating the associations between countries' preparedness capacity presented by e-SPAR and countries' COVID-19 control outcome also pointed out the limitation of the effectiveness of such scores before COVID-19 (10).
While the COVID-19 pandemic brought an opportunity for countries to re-evaluate their capacities more carefully, we wonder how member countries view their core capacity after the real-life challenge. With the hypothesis that countries' self-reported IHR capacity scores will reduce after the COVID-19 epidemic, we conducted this study to determine the changes in scores in 2019 and 2020, as well as the factors that in uence these changes in scores.

Study Framework
The framework of this study was adopted from the Systemic Rapid Assessment (SYSRA) toolkit. We adopted from SYSRA because of its coherence to the requirement for countries in implementing the IHR. In SYSRA, there are two types of assessment; horizontal and vertical. The "horizontal assessment" analyzes the health system within which the infectious disease program is embedded from a variety of perspectives. While, the second element, the "vertical assessment" is used to assess the infectious disease-speci c component. Thus, in both elements, it includes external environment (political, socio-demographic, economic) and needs assessment (morbidity, mortality of the disease) as a consideration in assessing infectious diseases control program (2,11,12).

Self-evaluation capacity scores related to infectious disease control
To identify the changes in countries' self-evaluation capacity scores related to infectious disease control during the rst year of the COVID-19 pandemic, we calculated the absolute difference between the score of IHR e-SPAR in 2019 and 2020. We collected this data from the WHO website. There was a total of 13 items in the e-SPAR including Legislation and nancing, IHR coordination and National IHR focal point function, Zoonotic events and the human-animal interface, Food Safety, Laboratory, Surveillance, Human Resources, National Health Emergency Framework, Health Service Provision, Risk communication, Points of entry, Chemical events and Radiation emergencies (13). Since our study focused on infectious diseases control, we excluded Chemical events and Radiation emergencies capacity scores for analysis and only used 11 of 13 items in e-SPAR.

Case fatality rates (CFR) of COVID-19
We used deaths instead of cases to reduce bias in the data we will analyze. This is because there are 3 levels in the diagnosis of COVID-19 cases, namely suspected, probable, and con rmed cases. Thus, data on the number of con rmed cases of COVID-19 will uctuate and be unstable due to changes in the diagnosis status of a patient (14). Meanwhile, in reporting deaths due to COVID-19, only deaths that were con rmed to be caused by COVID-19 were reported (15). As the pandemic is still ongoing, we used Case Fatality Rate (CFR) of COVID-19 data up to March 31st, 2021, for the purpose to evaluate the impact of COVID-19 on IHR capacities during the rst year of the pandemic. We gathered COVID-19's CFR until March 31st, 2021 from the "Our World in Data" website (16). For CFR of COVID-19, until August 2021, CFR of COVID-19 was around 2% globally (17). However, many studies stated that the estimation of COVID-19 CFR was ranging from 2 to 3% (18). In this study, the CFR classi cation cut-off point was determined from the calculation of the average CFR of all the countries we included in the analysis. Therefore, we used the average CFR of 2.08 as a cut-off point to classify countries into high CFR and low CFR groups.

Income level of a country
Countries' income levels were determined by their Gross National Income (GNI) per capita. We collected the data of 2019 from World Development Indicator at the World Bank website. The income level of a country is determined by the country's GNI per capita (19). We also adopted the classi cation of the income level of countries de ned by the World Bank for analysis, which are low-income countries (LICs), lower-middle-income countries (LMICs), higher-middle income countries (HMICs), and high-income countries (HICs) (20).

Human Development Index (HDI)
We used HDI as the indicator to represent countries' development levels which re ect the social and environmental status of the country (21). The HDI data was collected from Health Development Report (HDR) 2020 at United Nation Development Program (UNDP) website (22). The classi cation of HDI by UNDP was used for analysis in the study. Low development countries are de ned as those with index scores below 0.55; while medium, high, and very high development countries whose score between 0.55 to 0.699; 0.7 to 0.799; and above 0.8 respectively (21,23).

Civil liberties (CL)
While countries' transparency was reported to be associated with their reported scores by the previous study, we also collected the CL score data from the Freedom House website for analysis (2). And the category of CL level was also adopted in the study for analysis. "Not-free" countries are those whose score is 0 to 35. While "partially-free" countries and "free" countries are those whose score 35 to 70, and above 70 respectively (24-26).

Government effectiveness (GE)
The GE is one of the components in The Worldwide Governance Indicators (WGI). It was the indicator re ecting the quality of public services, policy formulation, and its implementation. We chose GE as one of the variables because the literature mentions that the role of government and good governance were very important in efforts to prevent and control infectious diseases (27,28). The GE data was collected from The WGI project 2020 reports (29). Since the GE scores range for each country in the report ranging from -2.5 to 2.5, we classi ed this variable into 2 categories by setting 0 as the cutoff point. Thus, a country with a GE score above 0 means that the country has a strong GE, and conversely a country with a GE score below 0 is classi ed as a country with a weak GE.

Data Analysis
196 countries had e-SPAR scores in both 2019 and 2020. Only 154 countries with complete data of all indicators were used for analysis. First, to measure the changes in countries' self-reported IHR capacity scores before and after the COVID-19 outbreak, we calculated the difference of e-SPAR scores between 2019 and 2020 for each item and the average score of all 11 items. Then we analyzed whether the capacity differences between e-SPAR 2019 and 2020 were signi cant or not. While the data were not normally distributed, we performed the Wilcoxon Sign-rank test for analyzing the signi cance of the difference between scores. Then we divided the 154 countries into 2 groups based on their score classi cation, namely the group whose scores increased (n=98), and the group whose scores did not increase (n=56) for further analysis.
Chi-square tests were applied to compare the indicators between score increased and not-increased groups. Then we performed multiple linear regression analysis (30) to determine what factors associated the capacity differences. In the model, the average score differences of e-SPAR 2020 and 2019 were the dependent variable (Y), while the CFR of COVID-19, HDI, CL, and GR were the independent variables (X). We did not include countries' income levels in the model due to its signi cant correlation with the index of HDI. All analysis was performed using the software SPSS, version 19.

Results
The changes in e-SPAR capacity scores between 2019 and 2020 The 2019 and 2020 e-SPAR scores can be seen in Figure 1. Among all 154 countries (part A), the average of 11 e-SPAR capacity scores as well as the scores for each capacity in 2020 were higher than their respective scores in 2019. The range of scores increased from 2019 to 2020 is from 0.13 to 5.04. From the Wilcoxon signed-rank test, the average score of 11 capacities and the scores of all individual items in 2020 were signi cantly different from the score in 2019, except capacities about Zoonotic events and the human-animal interface, Food Safety, and Human Resources (p>0.05). Next, among 98 countries whose scores increased (part B), the Wilcoxon signed-rank test result showed that the average score of the 11 capacities increased signi cantly from 2019 to 2020. Risk communication, National Health Emergency Framework, and Ports of entry were the 3 capacities that mostly increased (p<0.05). However, the capacity of Zoonotic events and the human-animal interface was the only capacity that was not signi cantly changed during the pandemic (p>0.05). Meanwhile, among 56 countries whose scores did not improve (part C), the Wilcoxon signed-rank test results showed the average score of the 11 capacities decreased signi cantly, as well as the capacity about National IHR focal point function, Food Safety, and Risk Communication (p<0.05).
Comparisons of COVID-19 CFR, countries income level, HDI, CL, and GE among overall countries, countries whose score increased and not increased were shown in Table 1. Based on country income level, among the countries that experienced an increase in scores, the majority were lower-middle-income countries (LMICs) (37.7%), while the high-income countries (HICs) and low-income countries (LICs) were the least with 18.4% each. On the contrary, HICs were the majority among countries whose scores did not increase (46.4%). Similar to the distribution in-country income level, countries with medium development status were the group that mostly (28.6%) experienced an increase in e-SPAR scores during the pandemic.
Meanwhile, countries with very high development status were the majority in the group of countries whose scores did not increase (55.4%) in 2020 compared to the previous year.
Next, for civil liberties status, most countries within the group of countries whose scores increased during the pandemic were "partially free" countries (38.8%), followed by "free" countries (34.7%), and the least was "not free" countries (26.5%). Meanwhile, among the countries whose scores did not increase during the pandemic, most of them were "free" countries (46.4%). Then, for government effectiveness, most of the countries whose score increased during the pandemic had weak GE (66.3%) while most of the countries whose score did not increase had a strong GE (67.9%). Meanwhile, for CFR of COVID-19, 64% of the countries in each group had low CFR.
Part-A showed the scores of the overall countries (n=154), while Part-B showed the scores of countries whose scores increased (n=98), and Part-C showed the scores of countries whose scores did not increase (n=56); Table 2. From the table, it can be seen that Model 1 and 2 were able to describe the changes of e-SPAR score among 154 countries (R=0.378, F=5.209, p=0.000) as well as among 98 countries whose score increased (R=0.45, F=5.907, p=0.000) signi cantly, while Model 3 was not (R=0.249, F=0.843, p=0.504). In Model 1, of the four variables, COVID-19's CFR was the only factor that signi cantly affected the changes of e-SPAR scores during the pandemic in 154 countries (t= -2.635, p=0.009). And GE was also potentially associated with the score changes (t= -1.977, p=0.050). Similarly, in Model 2, COVID-19's CFR was the only variable that signi cantly affected the changes of e-SPAR scores during the pandemic among scoreincreased countries (t=-2.036, p=0.045).  Model 1 is for overall countries, while model 2 is for countries whose score increased, and model 3 is for countries whose score not increased (n = 56); p<0.05* p=0.05**

Discussion
The results of this study indicated that the average e-SPAR score of 154 countries in 2020 was signi cantly different from the score in 2019. There were 98 countries (63.63%) that experienced an increase in e-SPAR scores in 2020, while 56 countries (36.36%) experienced a decrease in e-SPAR scores respectively. Among those whose scores increased, 62 countries (63.26%) were middle-income countries. Risk communication, National Health Emergency Framework, and Ports of entry were the 3 capacities that mostly increased while Zoonotic events and the human-animal interface was the only capacity that did not increase signi cantly. Meanwhile, among countries whose scores did not increase, most of them (46.4%) were high-income countries. IHR Coordination, Risk Communication, and Food Safety were the capacities that signi cantly decreased. These results might echo previous research nding that countries tend to rate themselves at SPAR beyond their actual capacity (31). So "high-pro le" countries downgraded their capacity score after the COVID-19 with comparatively "not-so-good" control outcomes. Meanwhile, the tendency of "low-pro le" countries to experience an increase in e-SPAR scores during the pandemic seems to justify their unpreparedness all this time. The limited resources of these countries are allegedly one of the main factors in uencing their unpreparedness. While limited resources will force them to choose development priorities (32,33), Covid-19 might provide them an opportunity to address more resources in enhancing their capacity in infectious disease.
Furthermore, our results showed that the capacity associated with zoonotic disease control was the only capacity whose scores did not change signi cantly in both score-increased and score not-increased countries. This nding seems to show that the world is not ready to face a pandemic in the future, especially if it is a zoonotic disease. This is supported by many references showing that zoonotic diseases are major threats in the future (34,35). Reported showed that most major public health events considered as the Public Health Emergencies of International Concern (PHEIC) were caused by pathogens that originated from animals (34). Worse yet, it is stated that most zoonotic disease-causing agents have the potential to be used as biological weapons for bioterrorism purposes (35). Therefore, the approach taken by WHO in collaboration with organizations in various sectors such as Food and Agriculture Organization (FAO) and OIE (O ce International des Epizooties/ the World Organization for Animal Health) to comprehensively address health problems through a One Health approach should be strengthened (36).
Regarding the result of multiple regression, we do not recommend the models be used to project the magnitude of future score changes due to the small R-values of the models. Instead, the result of both models identi ed factors in uencing changes in e-SPAR scores. From the results of our study, COVID-19's CFR is a factor that signi cantly affects the difference in the e-SPAR scores of 154 countries (Model 1, p<0.001) as well as the countries whose score increased (Model 2, p<0.05)) in 2020 compared to 2019. While most of the countries with increased scores were middle-income countries, nancial assistance might also be the reason for the increase in their e-SPAR scores. From the list of countries receiving aid funds for managing COVID-19, it seems that the more severe the COVID-19 condition in a country, the greater the amount that will be received (37,38). Therefore, our study result showed the signi cant association between countries' CFR for COVID-19 and their increase in e-SPAR scores. But further research is recommended to understand the reason behind the phenomenon.
The study result showed that although the world suffered a lot from the COVID-19 pandemic, it served as an opportunity for countries to evaluate their capacity more carefully. Also, though the e-SPAR score had the limitation in predicting countries' control outcomes, it can still be used to increase countries' awareness and preparedness in dealing with future pandemics (39). Therefore, we recommend the member states evaluate their IHR capacity more carefully. And the self-evaluation better be validated by an external party through the JEE to optimize the bene ts of the IHR. The current negotiation regarding the possible "Pandemic Treaty Agreement" should consider elaborating or strengthening such a monitoring approach to prevent and mitigate future pandemics (40).

Conclusion
The average of 11 e-SPAR capacity scores during the rst year of the COVID-19 pandemic in 2020 was signi cantly different from the respective score in 2019. The majority in the group whose e-SPAR score increased was middle-income countries, while high-income countries were the majority in the group whose score did not increase. Our model showed that COVID-19 CFR in uenced the changes in e-SPAR scores of 154 countries and countries whose e-SPAR scores increased, but not countries whose e-SPAR scores did not increase. In conclusion, we consider the COVID-19 pandemic as an alarm for countries to re-evaluate their capacities more carefully. While there is a need for outside challenge, Joint External Evaluation should be a necessary approach for future preparation.

Limitation
In this study, many low-income countries were not included because they did not have complete data and information to analyze, and this might affect the results of the study. Also, since our research is on a global scale, there is a potential of ecological fallacy, which means that results on a global scale may differ and may not re ect the actual situation at the individual country level.

Declarations Acknowledgment
We are fully grateful to Taiwan's Ministry of Science and Technology for providing nancial support to conduct this study, and we also thank Taipei Medical University for providing free statistical software to analyze the data in this study.

Funding
This study received nancial support from Taiwan  The e-SPAR score in 2020 and 2019.
Part-A showed the scores of the overall countries (n=154), while Part-B showed the scores of countries whose scores increased (n=98), and Part-C showed the scores of countries whose scores did not increase (n=56);