Evaluation of Public Health Surveillance Systems in Refugee Settlements in Uganda, 2016 - 2019: Lessons Learnt

Civil wars in the Great Lakes region resulted in massive displacement of people to neighboring countries including Uganda, with associated humanitarian emergencies. Appropriate disease surveillance enables timely detection and response to outbreaks. We describe evaluation of the public health surveillance system in refugee settlements in Uganda and document lessons learnt. Methods We conducted a cross-sectional survey using the US CDC Updated Guidelines for Evaluating Public Health Surveillance Systems in four refugee settlements in Uganda i.e., Bidibidi, Adjumani, Kiryandongo and Rhino Camp. Using semi-structured questionnaires, key informant and focus group discussion guides, we interviewed health facility in-charges, key personnel and village health teams from 4 districts, 53 health facilities and 112 villages.


Introduction
Page 4/15 The civil wars and fragile states in the Great Lakes region has resulted in massive displacement of people to neighboring countries with Uganda bearing the brunt of this in ux because of its open refugee policy (1). South Sudan which got independence on 9 July, 2011, had an outbreak of a bloody inter-ethnic of war until to date hence, contributing the bulk of the refugees in Uganda (2). The gazetted refugee settlements in Uganda include: Adjumani, Bidibidi, Imvepi, Kampala, Kiryandongo, Kyaka II, Kyangwali, Lobule, Nakivale, Oruchinga, Palabek, Palorinya, and Rwamwanja (3). The total population in these settlements was 1,434,708 by October, 2020; this number continues to increase with renewed con ict and hostilities that have plagued the region (4). These displacements are often associated with humanitarian emergencies. Those affected are settled in locations with high population densities, inadequate food and shelter, unsafe water, poor sanitation, and lack of infrastructure. These circumstances increase risk of transmission of communicable diseases and acute malnutrition with its antecedent consequences of increased morbidity and mortality (5). Reliable and appropriate disease surveillance enables timely detection and response to disease outbreaks and public health challenges.
Uganda's Ministry of Health (MoH) has implemented the Integrated Disease Surveillance and Response (IDSR) as a national strategy for improving epidemiologic surveillance and response since 2000 (6). The strategy, which was developed by the World Health Organization (WHO) Regional O ce for Africa provides a framework for implementing the requirements of the International Health Regulations (IHR) at country level. The IDSR approach aims at coordinating and integrating surveillance activities including: detection, con rmation/veri cation, noti cation/reporting; and timely response to control and prevention of disease outbreaks and other public health events of national and international concern. The IDSR system is used in all health systems including refugee settlements in the country. During humanitarian emergencies however, public health surveillance systems tend to be disrupted and underperform or are non-existent or quickly become overwhelmed to adequately address surveillance functions (7). Evaluation of a public health surveillance system gives a clear description of the purpose, design, management, and operational characteristics of the system and provides evidence-based information, which could be used for strengthening the reporting mechanism and enhancing implementation of public health actions (8).
We describe the evaluation of the public health surveillance system in refugee settlements in Uganda in which the surveillance functions and surveillance attributes were assessed and lessons learnt documented.

Methods
We conducted a cross-sectional survey using the US Centers for Disease Control and Prevention (CDC) Updated Guidelines for Evaluating Public Health Surveillance Systems (9) in four refugee settlements in Uganda i.e., Bidibidi, Adjumani, Kiryandongo and Rhino Camp (Fig. 1). Bidibidi Refugee Settlement which covers an area of 250 sq. km is located in Yumbe District, Northern Uganda which borders South Sudan to the north and Moyo District along the western bank of River Kochi. The settlement has a capacity of 180,000, but by March 2020 there were over 270,000 refugees mainly from South Sudan.
Adjumani Refugee Settlement which comprises 17 camps, is located in Adjumani District which is bordered by Moyo District to the north, South Sudan to the northeast, Amuru District to the east and south, Arua District to the southwest and Yumbe District to the northwest. The total refugee population for Adjumani District stood at 209,048 by March 2020. The resettlement areas are organized in clusters, blocks and zones. Adjumani district is prone to refugee in uxes due to its location at the border with South Sudan; as such the district has frequently registered several disease outbreaks, the most recent being measles and cholera outbreaks in Pagirinya Refugee Camp.
Kiryandongo Refugee Settlement which has three major divisions called Ranches, is located in Kiryandongo District which is bordered by Nwoya District to the north, Oyam District to the northeast, Apac District to the east, and Masindi District to the south and west. In 2020, the population of Kiryandongo District was estimated at about 317,500.
Rhino Camp Refugee Settlement is located in Arua District and is spread over 3 sub-counties (Rigbo, Odupi and Uriama). Arua District is located in North western Uganda and is bordered by Yumbe District to the north, Adjumani District to the northeast, Amuru District to the east, Nebbi District to the southeast, Zombo District to the southwest, the Democratic Republic of the Congo (DRC) to the west, and Maracha District to the northwest. In 2020, the population of Arua District was estimated at about 862,700.
The study units were the stakeholders utilizing the public health surveillance system in the refugee settlements. These include: health facility in-charges and surveillance focal persons, District Health Teams (DHTs), District Rapid Response Teams (DRRTs), District Epidemic Preparedness and Response Committees (DEPRCs) and Village Health Teams (VHTs). Six health facilities were picked using simple random sampling in Adjumani, 16 in Bidibidi, 24 in Kiryandongo and 7 in Rhino Camp.
We used a single stage cluster sampling method to select villages. The refugee settlements are zoned and each zone is divided into villages. Since the number of villages in each zone vary from one zone to another, sampling proportionate to size was used to determine the number of clusters in each zone to be included in the study. The primary sampling units (clusters) were selected using systematic random sampling. Sample size was calculated using sample size calculator software (vSphere) (10), inter-cluster correlation was estimated to be 0.3, and a baseline compliance rate assumed to be 50%. Accordingly, 24 villages were each selected from Bidibidi and Kiryandongo, and 32 villages were each selected from Adjumani and Rhino Camp Refugee Settlements. From each village, 2 VHT members were selected for interviews on a rst found rst picked basis. Interviews with VHTs was used to generate data on community surveillance.
Face to face interviews were conducted to collect information regarding the surveillance attributes using a semi structured questionnaire. Simplicity, acceptability and exibility were assessed using questions concerning compliance, ease of use, and number of steps in the system alongside users' opinions on the appropriateness of IDSR in detecting, recording and reporting of priority diseases. Completeness was assessed by looking at the lling of the registers and the reporting forms which translates into quality. Other relevant information was collected by document reviews and key informant interviews. Key informant interviews were conducted with key District Health Team members including the District Health O cer, District Surveillance Focal Person, and some members of the DRRT and DEPRC. Important to note is that the United Nations High Commissioner for Refugees (UNHCR) and Implementing Partners (IPs) staff are part and parcel of the DEPRC. Data was collected electronically using an Open Data Kit Software (Kobo Collect for Humanitarian Emergencies) using Tablet PCs.
Observation method was used to assess health facility registers and reporting tools based on the following attributes: Simplicity, Flexibility, Acceptability, Sensitivity, Representativeness, and Timeliness.
The capacity of the refugee settlements in performing surveillance functions (Table 1) and all the attributes (Table 2) of a public health surveillance system were assessed in the four refugee settlements of Adjumani, Bidibidi, Kiryandongo and Rhino Camp (Fig. 1).  We analyzed qualitative data using the content analysis model. We sorted and coded the data into categories, in order to bring together related terms. We edited and summarized useful information and developed conclusions without changing the meaning of what the respondents said. We used Epi-data to capture quantitative data and exported to Stata version 16 for analysis. We used descriptive statistics to summarize variables in form of rates and proportions. We calculated the positive predictive value of the system using cholera data by comparing the proportion of persons identi ed as having cholera to those who actually had the condition under surveillance.

Results
All the health facilities assessed had key surveillance staff such as clinicians, nurses, records assistants, laboratory and environmental health personnel employed by government or partners. However, only 60% of health workers were trained on IDSR. The surveillance system was found to be picking the priority health conditions speci ed in IDSR. The VHT structure is in place and they are trained on Community Based Disease and Events Surveillance (CBDS). However, majority 80% of the facilities lacked evidence that VHTs were reporting on CDBS activities.

Capacity in performing surveillance functions
All facilities had Out Patient Department (OPD) registers feeding into the weekly and monthly reports however, there was lack of standard case de nitions (SCDs) and use of parallel IP driven Health Information System (HIS). Detection of cases was at 55% (Range = 53-57) while recording was at 79% (Range = 67-90) and reporting was at 81% (Range = 67-95). Analysis and interpretation of was at 49% (Range = 19-95). The capacity of the settlements to con rm outbreaks and events was at 76% (Range = 50-86) and their level of preparedness was at 72% (Range = 65-83). The response rate was at 34% (Range = 11-50) while ability to provide feedback was at 82% (Range = 75-100). The capacity of the system to evaluate and improve the system was at 67% (Range = 55-83). Surveillance Attributes Simplicity Diagnoses and reporting of priority diseases were made according to standard case de nition (SCDs) in 65% of the health facilities, 35% of the facilities had their own (IP driven) case de nitions. No facility had drawn its own list of priority diseases, events or conditions, but used the list of diseases speci ed on the weekly reporting form, HMIS 033b (11). Case based investigation forms (CIFs) were lacking in 65% of the health facilities. Simpli ed line lists for reporting common conditions such as measles were found with 55% of the VHTs. There were two reporting channels, one from VHTs and health facilities through the districts to the MoH and another to the IPs supporting a refugee settlement as shown in the owchart in Fig. 2.

Flexibility
There were two reporting systems with different reporting tools in health facilities. One reporting system was driven by the IP while the other used MoH system. Age categorization of sub-groups differed across the facilities from the standard MoH guidelines.

Data Quality
The registers were poorly lled with many gaps e.g. key indicators such as suspected malaria, next of kin etc. were glaringly missing. Only 33% of the health facilities assessed demonstrated the capacity to analyze and present data using charts and maps.

Acceptability
The health workers whether employed by government or partners accepted that there were gaps in timeliness and completeness in reporting. Users interviewed expressed willingness to align the reporting system with the MoH standard reporting format.

Sensitivity
The surveillance system was sensitive at 79% (Range = 76-83) in detecting suspected outbreaks during the period of study i.e. April 2016 to December 2019. The most reported epidemic prone diseases were: malaria, typhoid, dysentery, measles and cholera however, most were not investigated by the districts.
Other events picked by the surveillance system were: animal bites (suspected Rabies), Adverse Events Following Immunization, presumptive multi-drug resistant TB, acute accid paralysis, malaria deaths, perinatal and maternal deaths.

Predictive Value Positive
The positive predictive value for surveillance was 64% on average (Range = 50-70). Case investigations of cholera outbreaks were carried out during the study period by the NRRT in Adjumani, Kiryandongo and Rhino Camp Refugee Settlements.

Representativeness
Age categorization of sub-groups differed across the facilities from the standard MoH guidelines. This affected computation of rates across population sub-groups.

Timeliness
The average weekly reporting rate in the refugee settlements 71% (Range = 52-79). The DHT reported inadequate skills to record, summarize and send reports on mTrac reporting forms by the health facilities. Few health workers were on mTrac system hence hindering weekly reporting. Outbreaks of cholera were reported within a week in Adjumani, Kiryandongo and Rhino Camp Refugee Settlements.

Stability
Internet connectivity challenges were experienced by all facilities visited and the District Health O ce (DHO). Funds to repair equipment were in short supply across board.

Discussion
The surveillance system in the refugee settlements of Adjumani, Bidibidi, Kiryandongo and Rhino Camp demonstrated ability to detect and ensure that health conditions of public health importance are monitored e ciently and effectively to control and prevent epidemics. The system attributes and ability to perform surveillance functions are embedded the development process. However, there are many gaps that could negatively affect the e ciency of the system. The system does not follow the MoH IDSR guidelines in its entirety. The core function of IDSR is to strengthen district level surveillance and response for priority diseases and integrate laboratory activities, reduce duplication in reporting, and share resources among disease control programs.
The surveillance functions of recording, reporting, preparedness, feedback and con rmation of outbreaks and events were highly achieved, and evaluate and improve the system was moderately achieved. However, there was low capacity for detection, response and data analysis and interpretation of cases in all the refugee settlements. Lack of SCDs and use of parallel IP driven Health Information System possibly contributed to this. In addition, most health workers were not trained on IDSR and hence lacked data quality and analytical skills. Drehobl et al asserts that much as data systems that monitor health threats are becoming increasingly automated, human expertise is, and always will be, critical to recognizing potential cases of disease, diagnosing disease, reporting diseases or conditions, analyzing and interpreting data, and communicating results to all stakeholders. For this reason, the nation's health professionals from all disciplines and at all levels are fundamental to sustaining and enhancing public health surveillance capacity (12). Addressing skills and competency gap of the health professionals is a critical step in ensuring improved surveillance systems to effectively and e ciently perform surveillance functions. In the low scoring category in implementing surveillance functions, Bidibidi scored lowest compared to the other three refugee settlements. This could be attributed to Bidibidi being a very new settlement compared to the much older well-established settlements. Adokiya et al, assessed the core and support functions of the IDSR system at the periphery level of the health system in northern Ghana and made similar deductions on the gaps in the surveillance system. Their ndings differed with ours on the feedback in that they found the feedback irregular in Ghana (13).
As shown in Table 2, sensitivity, timeliness and predictive value positive attributes scored highly in all refugee settlements except Bidibidi which scored relatively low in timeliness and predictive value positive. Simplicity, acceptability and data quality scored moderately while exibility, representativeness and stability attributes were low in all the refugee settlements. A surveillance system with high sensitivity implies that it has great ability to detect outbreaks, including the ability to monitor changes in the number of cases over time. Good timeliness allows quick control of a health-related event and prevention of continued exposure, as well as program planning. A high predictive value positive means that the case de nition's sensitivity and speci city is high (i.e., the screening and diagnostic tests for a health-related event is good). Stability refers to reliability (i.e., the ability of a surveillance system to collect, manage, and provide data properly without failure) and availability (the ability to be operational when it is needed), a low stability implies less reliability and availability. Gazarian et al assessed whether the national active surveillance of uncommon childhood conditions facilitated by the Australian Paediatric Surveillance Unit ful lled its objectives and found the positive predictive value to be 70% which is quite similar to our ndings (14). Hussain et all conducted an evaluation of acute respiratory infection surveillance systems in Gilgit-Baltistan, Pakistan and found their system to be simple, with high sensitivity but less exible and with moderate data quality (15). This is quite similar to our ndings.

Conclusion
Public health surveillance system in the refugee settlements had high sensitivity, timeliness and predictive value positive. It was simple, acceptable with fair data quality attributes. It was less exible, less stable with low representativeness. The system had good recording, reporting, preparedness, feedback and con rmation of outbreaks and events, and evaluate and improve surveillance functions.
However, there was low capacity for detection, response and data analysis and interpretation of cases in all the refugee settlements. The lessons documented were: IPs offer tremendous support to surveillance in refugee settlements; Training of key surveillance staff on IDSR is essential in maintaining effective surveillance functions; Support by MoH in terms of supplies of tools (CIFs, SCDs etc.), personnel etc. to refugee settlements should be planned and executed just like to other facilities in the districts Declarations Ethics approval and consent to participate Due to effect of COVID-19 on refugee populations, the Ministry of Health of Uganda (MoH) gave the directive and approval to conduct the evaluation. The primary intent of the evaluation was for improving public health practice and disease control activity (speci cally, epidemic or endemic disease control activity). In-addition, we sought administrative clearance from the O ce of the Prime Minister and Commandants of the various refugee settlements to conduct the study. We sought verbal informed consent from the participants before the qualitative interviews. To ensure con dentiality, we assigned unique identi ers to each of the participant questionnaires. Data held on computers was protected by encryption with a password.

Data availability statement
Participant data without names and identi ers will be made available after approval from the corresponding author and Uganda National Institute of Public Health, Ministry of Health, Kampala, Uganda. Location of Refugee Settlements in Uganda, 2021. Note: The designations employed and the presentation of the material on this map do not imply the expression of any opinion whatsoever on the part of Research Square concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. This map has been provided by the authors.