Evaluation of Field Epidemiology Training Programs in the Eastern Mediterranean Region: A Multi-country Study

Background: Field Epidemiology Training Programs (FETPs) are competency-based training programs aiming to strengthen epidemiologic capacity of public health workforce. This study aimed to evaluate the impact of the FETPs in the Eastern Mediterranean Region (EMR) and ascertain whether the expected objectives of the programs are met Methods: A descriptive study was conducted based on the Kirkpatrick’s model for evaluating training programs. Data were collected from FETP graduates and FETP technical advisers using online surveys through separate questionnaires. Questions were designed to assess the practices of FETP graduates, their engagement in key areas of eld epidemiology, and their perceived skills and capacity to perform such activities. Results: A total of 166 FETP graduates responded to the online survey. Almost two thirds of FETP graduates reported that they are often engaged in managing public health surveillance system (n=119, 71.7%), analyzing the surveillance data (n=116, 69.9%), training public health professionals (n=113, 68.1%), outbreaks investigations and response (n=109, 65.7%), and managing staff and resources (n=106, 63.9%). However, only 28.3% reported that they are often engaged in writing scientic research articles. More than two thirds of graduates reported that the FETP helped them to perform most of the eld epidemiology activities and rated their skills as good. However, smaller percentages of the graduates reported that their skills are good in applying simple tools for economic analysis (65, 39.2%) and writing scientic research articles (67, 40.4%).


Background
The Eastern Mediterranean region (EMR) consists of 21 countries. Many of these countries have recently experienced unrest as a result of revolutions, wars, political con icts, massive forced displacement, and outbreaks of diseases. According to the World Health Organization (WHO), EMR is the host to some of the world's biggest emergencies and protracted crises. More than 29 million (58%) of the total refugees and internally displaced persons (IDPs) worldwide are from the EMR [1]. With recent emergencies in the region, both displaced populations and host communities are exposed to increased public health risks. These include infectious diseases due to overcrowded living conditions, increased burden of non-communicable disease, limited access to safe water and sanitation, and changing degrees of access to primary healthcare services [2]. These risks play a signi cant role in determining the health security of the entire region and increases the demand for public health professionals with a various set of skills [3]. Currently, the COVID-19 pandemic showed the need for increasing the number of individuals trained globally to prevent, detect, or respond to public health threats and the need for increasing the laboratory diagnostic testing capacity, surveillance system, and routine reporting in countries and regionally.
Field Epidemiology Training Programs (FETPs) are competency-based training programs that highly contribute to the development of national and regional health security infrastructure and the enhancement of epidemiologic capacity of public health workforce [4]. FETP is identi ed as one of the key activities of the U.S. Centers for Disease Control and Prevention (CDC) in improving global health [5].
The CDC and partner organizations assist and support Ministries of Health (MOHs) and other public health authorities to establish FETPs at their countries [5]. Governmental public health workers who are responsible for public health functions at a national level of the health system are the target audience for FETPs, especially physicians, laboratory personnel, public health o cers and veterinarians.
The programs provide participants with practical experience, with minimum classroom time and maximum time in the eld, in responding to disease outbreaks, disease surveillance, natural disasters, and other public health priorities. These programs aim at improving and strengthening the host country's public health systems to detect, investigate, and respond quickly and effectively to public health events; establish and effectively use a robust surveillance system; develop human capacity in applied epidemiology and allied areas; and ensure that public health decisions are driven by scienti c data.
Worldwide, there are 72 training programs with 14,000 graduates and 4,770 current trainees [6]. In the EMR, the Eastern Mediterranean Public Health Network (EMPHNET) has helped to launch and establish several of the existing programs. Currently, there are ten eld epidemiology country training programs with more than 400 FETP graduates in the region [4]. Most of those FETP graduates work as government o cials and many have obtained leadership positions within their national health systems [4].
Evaluation and monitoring of FETPs can be used to improve the programs and inform decisions about future resource allocations. The evaluation helps to determine the impact of the program and how its components work towards supporting the desired outcomes. It also helps to achieve and maintain highquality training and assure program's effectiveness in improving public health [7]. There are some studies that evaluated FETPs and reported the experiences and lessons learnt. These studies showed that the training programs have contributed to the development of a skilled workforce in eld epidemiology [8][9][10].
However, they indicated that further efforts are required to scale up the program.
In the EMR, studies on the evaluation of FETPs and their impacts on the healthcare systems are lacking. This study aimed to evaluate the impact of the FETPs in the EMR and ascertain whether the expected objectives of the programs are met, as perceived by FETP graduates and the programs' technical advisers. In speci c, the study aimed to measure the degree to which FETP graduates applied what they learned during the training and assess the overall perception of the programs' technical advisers about the impact of the program on the public health system.

Study design
A descriptive study was conducted based on the Kirkpatrick's model for evaluating training programs [11]. Our study focused on level 3 and 4 of the model. Level 3 of this model is about behavior "the degree to which trainees apply what they learned during training when they are back on the job". Level 4 is about results "the degree to which targeted outcomes occur as a result of the training and the support and accountability package" [11].

Data collection
The data were collected using two separate online surveys/questionnaires. The rst survey was delivered to FETP graduates from 8 countries in the EMR. It was designed to assess the practices of FETP graduates asking about their involvement and engagement in key areas of eld epidemiology, the extent to which FETP helped them to perform speci c eld epidemiology activities, and their perceived skills and capacity to perform such activities. For each question, the participants were requested to choose an appropriate response on a 5 Likert scale. The questions covered all program's competencies including epidemiology, surveillance, outbreaks investigation and response, research, communication, leadership and management. The questionnaire collected information on the participants' demographic characteristics, highest educational degree earned, and country and year of graduation. The questionnaire was pilot tested on 10 FETP graduates and it was revised based on their feedback. The FETP database was used to extract the graduates' emails. The FETP database serves as a directory listing the names, as well as career related and contact information of FETP alumni from the EMR. The database is compiled and maintained by GHD/EMPHNET. The questionnaire was sent by email to 300 FETP graduates. The second survey targeted the programs' technical advisers who are at the level where they can observe the impact of the program. The survey was designed to assess the impact of the FETP on the health system and the bene t to organizations in terms of public health priorities (disease surveillance, disease outbreaks and investigation, etc.). The questionnaire included an open-ended question to enable the respondents to make suggestions for the improvement of the FETP.

Analysis
Data were exported to the IBM SPSS (IBM Corp. Released 2016. IBM SPSS Statistics for Windows, Version 24.0. Armonk, NY: IBM Corp.) for analysis. Data were described using means and percentages.

Participants' characteristics
A total of 166 FETP graduates responded to the online survey. Of those, 60 (36.1%) were females and 106 (63.9%) were males. Almost one third (n = 64, 38.6%) graduated before the year 2016. Half (n = 84, 50.6%) of the FETP graduates continued their higher education and earned a master or doctoral degree. The participants' characteristics are shown in Table 1. A total of 10 technical advisers from Afghanistan, Sudan, Morocco, Jordan, Pakistan, Saudi Arabia, and Yemen have evaluated the FETPs in the region. Six technical advisers had a doctoral degree and four had a master's degree.

Engagement of FETP graduates in eld epidemiology activities
Almost two thirds of FETP graduates reported that they are often engaged in activities such as managing public health surveillance system (n=119, 71.7%), analyzing the surveillance data (n=116, 69.9%), training public health professionals (n=113, 68.1%), outbreaks investigations and response (n=109, 65.7%), and managing staff and resources (n=106, 63.9%). A total of 121 (72.9%) participants reported they often use computers for speci c applications relevant to public health practices ( Table 2). The engagement of FETP graduates was the least in publishing research articles where only 28.3% reported that they are often engaged in writing scienti c research articles.
From the perceptive of technical advisers, all reported that FETP graduates are involved in managing surveillance data, developing surveillance reports, and participating in outbreak investigations to large extent. A total of 4 (40%) advisers reported that FETP graduates often evaluate and participate in the planning and implementation of public health interventions and 6 advisers reported that they sometimes do that. When they were asked about how frequent FETP graduates use surveillance data to provide courses of action and recommendations, 60% reported always and 40% reported sometimes. Only 4 (40%) advisers reported the FETP graduates played a key role in regional-scale outbreaks. When they were asked about how often do FETP graduates provide informal consultations to MOH programs, 5 (50%) reported very often and 4 (40%) reported sometimes. When they were asked about how often FETP graduates use laboratory resources to support epidemiologic activities such as using laboratory data for surveillance purposes, 3 (30%) reported often and 6 (60%) reported sometimes The extent to which Field Epidemiology Training Program helped the graduates to perform eld epidemiology activities More than two thirds of graduates reported that the FETP helped them much to perform most of the eld epidemiology activities listed in Table 3. However, only 76 (45.8%) graduates reported that FETP helped them much to apply simple tools for economic analysis, to publish scienti c articles in journals (89, 53.6%) and to develop policies or strategies (93, 56.0%).

Skills and capacity of the Field Epidemiology Training Program graduates
More than two thirds of the FETP graduates rated their skills to conduct many eld epidemiology activities as good (Table 4). However, much smaller percentages of the graduates reported that their skills are good in applying simple tools for economic analysis (65, 39.2%) and writing scienti c research articles (67, 40.4%).
Almost all technical advisers stated that the capacity FETP graduates is very good and the they contributed to strengthening public health system, improving the surveillance system, development of protocols and guidelines and improving health policies. Some advisers reported that FETP graduates are now placed at senior level decision making positions and have started to make a very positive impact on improving the health system. Moreover, they reported that governments' con dence on the capacities of FETP graduated has improved manifold and the governments now rely on FETP graduates for outbreak investigation, response and surveillance activities. Although the capacities of the FETP graduates are reported to be good, some advisers reported that they are facing di culties in showing impact due to limited nancial and human resources as well as war and Siege.
The impact of the Field Epidemiology Training Program from the perspectives of technical advisers Five advisers (50%) reported that the data collection on reportable diseases has improved much in their countries since the FETP establishment and the rest reported that it has been somewhat improved. A total of 7 (70%) technical advisers reported that the FETP has improved the outbreak investigations and response in their countries to a large extent and 30% reported that they are somewhat improved. All reported that FETP graduates contributed signi cantly to improvements in surveillance systems (90%). A total of 4 (40%) advisers reported that FETP contributed much and 5 (50%) contributed somewhat to national regular reports such as providing articles, editing articles, and presenting surveillance data. When they were asked about how much the FETP improved health policies and contributed to the strengthening of health systems in their countries country, 40% reported much and 50% reported somewhat.

Suggestions for the improvement Field Epidemiology Training Program from the perspectives of technical advisers
Most of the technical advisers reported that there is a need for strategies to ensure retention of FETP graduates by placing them in proper positions with good nancial incentives. Some suggested that the FETP certi cate should be upgraded to a scienti c degree. Moreover, they recommended increasing the capacity of the programs to train more people and working on accreditation, updating the curriculum, setting a mandatory requirement for graduation, and documenting the activities and the impact of FETP graduates.

Discussion
Periodic evaluation of public health training programs is essential to ensure that the programs are achieving their indented outcomes and impacts. The effectiveness of the FETPs can be demonstrated by assessing the graduates' competencies and their involvement in the provision of public health services. (12) This study assessed the FETPs contribution to the improvement of public health services at the country level.
This study clearly demonstrated that FETPs in the EMR have met the goal of preparing graduates to implement public health and main eld epidemiology measure and activities. There was consensus between FETP graduates and technical advisers on that the FETP graduates in the EMR are well engaged in many eld epidemiology activities including managing public health surveillance systems, surveillance data analysis, training public health professionals, and outbreaks investigations and response. Their impact can be most apparent in situations of con ict and crises that disrupt public health systems and especially within the current shortage of resources. Several examples have clearly showed the success of FETP in responding to emergencies and disasters (8)(9)(10)(12)(13)(14)(15)(16)(17)(18)(19). During the Middle East Respiratory Syndrome (MERS) outbreak in 2014 in Saudi Arabia, FETP graduates tackled numerous issues, including but not limited to, to redesigning the system to enable simultaneous real-time electronic reporting of suspected and con rmed cases to public health professionals who needed to take essential control and preventive actions on new cases (14). FETP graduates are working nowadays in different ways to ght COVID-19 pandemic as they are actively participating in surveillance and screening at airports and other ports of entry, developing communication materials and guidelines and sharing information to health professionals and to the public (16). FETPs in the EMR showed success in building the epidemiologic capacity for public health workforce, improving countries' surveillance systems, and strengthening the health systems (14,19).
However, this study showed that only small percentages of FETP graduates were engaged in applying tools for economic analysis, publishing scienti c articles in journals and developing policies or strategies. The low level of engagement in these activities is explained by inadequate skills in these areas as reported by FETP graduates. A relatively small percentage of the graduates reported that their skills are good in applying simple tools for economic analysis (39.2%) and writing scienti c research articles (40.4%). Another reason for this low level of engagement is that Ministries of Health don't realize the capacity of FETP graduates to perform these functions. Although that hundreds of outbreaks were investigated in the past years in the EMR, hundreds of studies were conducted, and many public health problems were investigated, only small percentage of FETP graduates managed to publish their ndings in international journals. This makes many achievements and success stories in public health in the region invisible and the contribution to the science and practice of epidemiology very minimal. Therefore, it is essential that the FETPs invest more in building the capacity in this area. Supervisors and mentors need to provide FETP residents with tailored, exible, and regular support to produce high-quality, timely and relevant research.
Moreover, there is a need to enhance the capacity of trainees in the area of health economics with a focus on essential tools such cost-effectiveness analysis, cost-bene t analysis, and cost-utility analysis. In general, there is a lack of expertise in using and applying economic tools in public health interventions in the region. Therefore, these competencies are essential to be developed and whenever possible, both economic and clinical data should be gathered in each country. The aforementioned competencies and skills can be considered in the future FETP programs to provide a holistic experience for all FETPs.
According to the FETPs' technical advisers, the FETP graduates did not play a key role in regional-scale outbreaks. The current COVID-19 pandemic and previous outbreaks such as Middle East Respiratory Syndrome (MERS), H1N1, and Ebola have demonstrated that while the outbreak may originate in one place, it can quickly spread to other parts of the world. Therefore, the FETPs should be able to mobilize resources to respond to public health emergencies. Hence, EMPHNET proposed the establishment of an FETP Residents' Exchange Program (FETP-REP) to provide the FETP residents in the region an opportunity to join FETP programs in other countries for a speci c period for the purpose of gaining new experiences. This will help to build the residents' eld capacity in new public health threats not usually encountered in their home countries, exchange experiences in managing and running FETP programs, extend the regional network of public health specialists and eld epidemiologists, strengthen the regional emergency response mechanism, and enhance coordination between MOHs in the region. Moreover, EMPHNET had already undertaken some activities and initiatives to connect FETP graduates (website, Epishares, social media channels or FETP ambassador program). Although the use of these channels has been effective to some extent, these platforms on their own will not serve as a fully-edged community of practice because they are simply missing the human element of interaction. Therefore, an Alumni association is proposed to ll this gap and to bring together a dynamic community of FETP graduates that is motivated and dedicated to improving health in the region and to serve as a space for discussion groups and networking opportunities amongst FETP graduates to share experience, interest, and needs in areas of public health relevant to the region.
One of the strengths of this study is that we assessed level 3 "the degree of applying what learned" and level 4 "the degree to which outcomes occur as a result of the training" of the Kirkpatrick's model in evaluating FETP training programs [11]. Another strength is that the evaluation was based on information from two sources including the FETP graduates and programs' advisers who are within the health system at a level where they can observe the impact of the program. Our results were derived from an online survey with all its potential strengths and limitations. The survey was anonymous and, thus, it is very likely that participants gave correct answers without being afraid of exposing their identity. Also, they were not under any pressure to give "desirable answers" to the survey questions. Although the response rate was just about 55%, it was more than expected given this type of surveys.

Conclusions
In conclusion, FETPs in the EMR have strengthened the epidemiologic capacity of public health workforce and increased the engagement of FETP graduates in public health services and eld epidemiology activities. Therefore, the FETPs should continue supporting the graduates to work toward strengthening surveillance systems and outbreak investigation and to participate in regional and global efforts as part of Global Health Security. Establishment of new FETPs in other countries of the region should be supported to improve public health in all countries of the region. The programs' supervisors and mentors need to invest more in building the capacity of trainees to prevent, detect, or respond to public health threats, increase collaboration among country FETPs, and increase the preparedness and involvement of Technology. An informed consent was obtained from each participant. Data were treated with strict con dentiality and used only for scienti c purposes.

Consent for publication
Not applicable

Availability of data and material
The data used to support the ndings of this study are available from the corresponding author upon request.