Evaluation of international continuing education on neurorehabilitation for rehabilitation health workers in Honduras: a pretest-posttest design

Rehabilitation therapies are still in the early stages of development in Honduras. At the present time there are no formal training opportunities in Honduras to support the learning needs of health professionals and healthcare workers who specialize in neurological rehabilitation. Support and development of those delivering rehabilitation services from community engaged academic collaborations outside of Honduras can enhance the learning and knowledge of healthcare workers. The purpose of this study was to evaluate reaction and learning linked to two continuing educational workshops for rehabilitation workers in northern Honduras.

Atlantida and Yoro, had a disability prevalence of 4.3%, 5.5% and 6.1%, respectively [2]. In addition, extremely disabling sequelae of non-fatal injuries have been documented such as organ removal, loss of limb and paraplegia [3].

Health System in Honduras
Bermudez-Madriz et al. (2011) described the Honduran health system as a two-tiered system with public and private providers. As part of the public system, the Health Secretariat (Secretaria de Salud) provides directions for the health system as a whole and also provides health services to the population. The Health Secretariat serves the whole population but mainly takes care of those who are unemployed and live in poverty in the urban and rural areas. The private sector provides health services to 5% of the population; those with an income able to pay for health services. It is estimated that 17% of the total population does not have access to any health service at all [4].
In Honduras, there are several rehabilitation clinics for people with disabilities [5]. The Health Secretariat operates nine rehabilitation clinics located in some public hospitals, and two hospitals for persons with mental or psychosocial impairments [5]. Furthermore, the Honduran Social Security Institute (government body that provides pensions and healthcare coverage) has two rehabilitation centres located in the main cities of San Pedro Sula and Tegucigalpa, which employ quali ed technical staff and has modern rehabilitation systems [5]. In addition, the Teleton Foundation (private sector) operates six rehabilitation clinics, none of which are located in northern Honduras [6]. In the rural areas, Honduras has adopted community-based rehabilitation guidelines, as advocated by the World Health Organization and the Pan American Health Organization, as a strategy to improve access to rehabilitation and community integration for persons with disabilities [5]. However, there are many hurdles to implementation of community-based rehabilitation in rural communities due to limited local professional capacity, and administrative and language barriers [5]. Similar to other developing countries, rehabilitation centers and services have sprung up in a haphazard manner and are fully reliant on philanthropic funding sources provided by Honduran citizens and non-governmental organizations [7,8].

Rehabilitation training
Despite the number of people with disabilities in Honduras, there is only one private university providing a 4.5-year baccalaureate program in rehabilitation (combined physical therapy and occupational therapy degree) which, as yet, has no graduates. The other rehabilitation training programs available in Honduras include: a medical degree in physical medicine and rehabilitation, a technical degree in functional therapy, a baccalaureate degree in phonoaudiology, and a diploma in integrated community rehabilitation. There is no formal system of post professional training opportunities to support rehabilitation workers such as physical therapists in Honduras, nor training beyond the baccalaureate level.
According to the 2013 census [9], in Colon, a northern province of 271,723 people, there were no (zero) quali ed rehabilitation workers: physiotherapists, phonoaudiologist, or functional/physical therapy technicians, or community rehabilitators (Figure 1, Map A). Moreover, the distribution of self-identi ed rehabilitation workers in Colon was 0.4 per 100,000 people ( Figure 1, Map B). The higher density of selfidenti ed rehabilitation workers likely indicates that other professionals and community members such as nurses, primary school teachers, special education teachers, or family members attempted to ll the gap in the rehabilitation workforce in the province.
As the country is now building capacity in rehabilitation, regulatory structures or professional associations for the professions of physical therapy, occupational therapy or phonoaudiology may follow. The slow development of the rehabilitation professions and the small public investment in infrastructure for rehabilitation greatly restrict the care and treatment available for the population in Honduras.
Please insert Figure 1. Since 2016, a grassroots organization initiated by the authors, the Network of Rehabilitation Workers of the Americas, (Red de Rehabilitadores de las Américas in Spanish) [1] has undertaken to support the professional education needs of the rehabilitation workforce in the northern states in Honduras (Colon, Atlantida, Yoro) by developing a north-south collaboration and sponsoring a visiting professor program.
Workshop curricula developed by rehabilitation professors from the University of Saskatchewan, Canada and University of Santander, Colombia have been implemented as part of this program. The objective of this study was to evaluate reaction and learning linked to two continuing educational workshops for rehabilitation health workers in northern Honduras, designed by an international group of academics and community engaged rehabilitation workers.

Theoretical models
Evaluation of participants' neurorehabilitation knowledge and learning was based on the Kirkpatrick Model [10,11] and the Slotnick's four Stages of Learning Model [12,13]. The Kirkpatrick framework was originally designed to evaluate human resource development training programs [11,14]. The Kirkpatrick framework speci es four levels of training evaluation: reaction (components: engagement, relevance, and learner satisfaction); learning (components: acquisition of intended knowledge, skills, attitude, con dence, and commitment); behavior (components: critical behaviors, required drivers, on the job monitoring); and results (the degree to which participants apply what they learned during training).
The Stages of Learning tool utilizes clinical scenarios constructed to address the learning objectives [13]. According to Moore and Slotnick (2006), the learner's level of engagement with respect to a given learning objective can be classi ed into one of four Stages of Learning: 1) Scanning -the learner is aware of potential problems that might require their attention; 2) Evaluation -the learner evaluates the potential problems on the basis of applicability to their own situation, the likeliness of nding a solution, whether there are resources available for learning the solution, and whether learning how to solve the problem will bene t practice; 3) Learning -the learner gains skills and knowledge applicable to the problem; 4) Gaining experience -the learner puts what has been learned into action.
The Network of Rehabilitation Workers of the Americas hypothesized that a better understanding of the effects of the workshops can give insights to improve future efforts to build rehabilitation capacity in Honduras.

Aim and Setting
The purpose of this study was to evaluate reaction and learning linked to two workshops delivered through the visiting professor program for rehabilitation workers in two rural cities (Tocoa and Trujillo) in northern Honduras.

Design
A pretest-posttest design was used to evaluate the reactions and learning effects of two workshops. Based on initial feedback, the questionnaires were modi ed and translated between English and Spanish by bilingual members of the team and approved by the Continuing Education in Rehabilitation Science unit. The language of the workshops and data collection was Spanish. The sociodemographic questionnaire, the MSLQs, and Knowledge Questionnaires were administered to the participants online or in paper format prior to the workshop ( Figure 2). The MSLQ was administered immediately after the workshop as a post-test. The MSLQ and the Knowledge Questionnaires were sent to participants one month after the workshops in paper form.

Ethics
The research proposal was reviewed and exempted by the Behavioral Ethics Board of the University of Saskatchewan in Canada (June 30, 2017). Participants provided written consent to be part of the study.
Please insert Figure 2.

Workshops
The third author (ICGD) together with a physiatrist and a local physiotherapist, conducted the adult (neurorehabilitation) workshop. ICGD is a Colombian physiotherapist, with a Master's degree in Neurorehabilitation, who has been teaching neurorehabilitation at the Universidad de Santander, Colombia for more than 10 years. The physiatrist has two years of experience in neurorehabilitation in public and private Honduran clinics, and the local physiotherapist has seven years of experience in Honduras. ICGD instructed the pediatric neurorehabilitation workshop independently.
Workshops were planned in partnership with: 1) CRICOL in Tocoa and Little Hands, Big Hearts in Trujillo, 2) School of Rehabilitation Science, University of Saskatchewan, 3) Universidad de Santander, Columbia, and 4) Continuing Education in Rehabilitation Science of the University of Saskatchewan. The themes for the workshops were selected considering the reported needs of the local rehabilitation workers from CRICOL in Tocoa and Little Hands, Big Hearts in Trujillo. After several online meetings between the members of the Network of Rehabilitation Workers of the Americas and local rehabilitation workers, the learning objectives for the workshops and clinical cases were developed. In addition to considering the local context, the workshops were carefully constructed with evidence-based learning objectives written using Bloom's taxonomy [15].
The workshop included nine objectives related to neurorehabilitation in adults with acquired brain injury, spinal cord injury, and cerebral vascular accidents (Appendix 1) and was conducted over 2 consecutive days (12 hours duration). It included lectures using PowerPoint, interactive content (class discussion), demonstrations, practical activities among peers, and a case discussion and assessment/treatment demonstrations on an adult patient with cerebral vascular accident.
The pediatric workshop which addressed seven objectives linked to rehabilitation for children with impaired neuromotor development (Appendix 2) was a one-day workshop (7 hours duration). It included lectures using PowerPoint, interactive content (class discussion), demonstrations with a doll, and case discussion and assessment/treatment demonstrations on two pediatric patients with cerebral palsy. Knowledge Questionnaires. There were two workshop-speci c questionnaires based on the learning objectives for the workshops (See Appendices 1 and 2). The questionnaires included a 5-point Likert scale (strongly disagree, disagree, neutral, agree, strongly agree) to indicate the learner's level of agreement with statements directly related to the learning objectives of the workshops. A score from 0 to 4 for each statement was obtained, where 4 was the maximum knowledge score.

Data Analysis
Each workshop was analyzed separately. Quantitative data were described using frequencies, medians, and interquartile ranges (IQR). The four Stages of Learning were collapsed into two categories: 1) Scanning/Evaluation; and 2) Learning/Gaining experience. Pre-and-post workshops changes from the Scanning/Evaluation stages to Learning/Gaining experience stages were evaluated using the Exact McNemar test. Pre-to-post workshops changes in the Knowledge Score were evaluated using Sig Test for repeated data. Non-responders were not included in the pre-to-post analyses. A signi cance level of 0.05 was used. Data analyses were conducted using STATA 13.1 (Stata-Corp, College Station, TX, USA).
[1] The Network of Rehabilitation Workers of the Americas includes members from University of Saskatchewan, Canada, Universidad de Santander, Colombia, Norway, and rehabilitation workers in Colon, and Atlantida, Honduras.

Socio-demographic characteristics
Socio-demographic characteristics are included in Table 1. Most of the participants came from Colon (70.6% and 100%) and more than half studied in Honduras (52.9% and 66.7%). Most were working in a Rehabilitation Center/family support organization (76.5% and 86.7%) as shown in table 1. Figure 2 shows participants number for each outcome measure as well as participants who were lost to follow-up. Attempts to contact all participants for follow-up evaluation were made by email and phone. After three weeks, the authors determined these participants as having no response. In relation to the Kirkpatrick's reaction level, participants found the workshops engaging, relevant, and favorable with over 82% and 87% of responses rating "agree" or "strongly agree" in the adult and pediatric workshops, respectively. Similarly, a high percentage, over 82% and 80% of participants attending the adult and pediatric workshops (respectively) agreed or strongly agreed with the items of the skills, attitude, con dence and commitment components of the learning level of Kirkpatrick's model (see more details in Appendix 4).

Stages of Learning
There were no changes in Stages of Learning in the adult workshop. However, in the pediatric neurorehabilitation workshop, there was a change in the number of participants who moved from scanning/evaluation stages in the pre-workshop evaluation to learning/ gaining experience in three different topics, 1-month post-workshop evaluation. In the questions: "Can you describe the developmental milestones that 'Jose" should have completed up to one year old?" (p=0.031), "Can you explain how rehabilitation could improve 'Jose's' motor control?" (p=0.016), and "Can you apply strategies to stimulate the child's motor development?" (p=0.008), the number of participants that moved from scanning/evaluation stages (pretest) to learning/ gaining experience (posttest) was 6 out of 7, 7 out of 8, and 8 out of 8, respectively. In the remaining topics, no signi cant changes in the Stages of Learning were identi ed.
Knowledge Score Table 2 shows that there was a signi cant change in the pre-post knowledge score in the item "I am able to describe the fundamentals of cognitive therapeutic exercise (referring to Perfetti method) in a patient with acquired brain damage". The median of the score changed from 0.   interesting, and most participants reported commitment to the workshop activities. Similarly, most participants considered the workshop relevant; they considered the material useful and applicable to their work and patient care. In addition, there was a high level of satisfaction; most participants agreed that taking these workshops was worth the time. The positive reaction to the workshops could in part due to the fact that the themes and the cases were selected following discussions with local health workers who knew the needs and context of northern Honduras. Several training evaluation studies have relied upon the rst level of Kirkpatrick's model as the only area of evaluation [16]. The authors in those evaluations considered their results encouraging, even when a positive reaction does not guarantee learning, changes in behavior, or results [17]. Our study also evaluated learning effects which is a unique contribution to this area of research.

Evaluation of Learning
The evaluation of learning was divided into three parts: Modi ed Kirkpatrick's Questionnaire (dimensions include skills, attitude, con dence and commitment); Moore and Slotnick's Stages of Learning; and nally, the ndings of knowledge questionnaires.
Modi ed Kirkpatrick Questionnaire. Regarding Kirkpatrick's second level (learning) which evaluates the extent of knowledge, skills, attitude, con dence and commitment (Kirkpatrick and Kirkpatrick, 2016), most participants: 1) reported that they would be able to use what they learned right away (skills); 2) believed their attendance at the workshops would have positive impacts on quality of care, personal con dence, patient satisfaction, relations with colleagues, and respect of colleagues (attitudes); 3) felt con dent in correctly applying the knowledge and techniques that they had learned without harming their patients (con dence); and felt committed to apply what they had learned in the workshops at work (commitment).
Stages of Learning. The MSLQ showed acceptable content validity. Because the scenarios and questions used in the MSLQs were based upon the workshop objectives, they can only be used for future workshops having the same objectives. However, the structure (objectives, scenarios, questions, items) of the MSLQ and the yes-no items themselves could be transferable to classify the Stages of Learning related to other workshops having a clinical focus.
No change in Stages of Learning was found in the adult neurorehabilitation workshop; however, a signi cant improvement in Stages of Learning was found for three areas in the pediatric workshop: 1) the concepts and processes of fetal development; 2) motor learning and motor control; and 3) the concept of neuroplasticity. These concepts are foundational for the application of rehabilitation interventions; thus, effects in these areas of learning represent meaningful change.
These workshops were brief and intensive and did not include time for direct observation of practical skills. As stated by Moore and Slotnick (2006), "the options accompanying each vignette did not ask how the problems should be handled; rather, the options sought to identify each respondent's educational status, or the learner's stage, relative to the problem in the vignette [i.e., scenario] (Moore and Slotnick, 2006)." Assessment of clinical/skill learning and skills learning is di cult even in the context of formal education [19] but in the context of workshops of 2 days or less, it is possible only for extremely systematized instruction such as cardiopulmonary resuscitation training [20]. In addition, we found only one training evaluation study speci c to rehabilitation. In this study, learning was evaluated after completion of an online module in stroke rehabilitation in 108 occupational therapists, physical therapists, registered nurses, and speech-language pathologists from Canada, which was not comparable with our study [21].

Strengths
Rehabilitation practice and continuing education should consider the local context, its needs and practices [22][23][24]. In this sense, one strength of this study was a collaborative approach. This project brought rehabilitation workers together with local peers and with international partners from Colombia and Canada. Workshop themes were selected by establishing an ongoing dialogue and considering the needs of the Trujillo and Tocoa rehabilitation workers; this mutually bene cial collaboration likely contributed to the fact that the workshops curricula were well appreciated. Future curriculum should be designed to include group activities that foster continuing collaborations including partnership with a local university.
Another strength of this study was that the learning evaluation included different approaches. First, several tools were applied to evaluate the different components of the learning level. Second, we used different time-points to evaluate learning: pre-workshops, immediately after the workshops and 1-month post-workshops, which allowed the assessment of changes in learning.

Limitations
Lack of a control group limits conclusions about whether observed changes in knowledge and Stages of Learning are due to the workshops rather than other factors. Second, our study has a high percentage of loss to follow-up (33%-60%). In addition, the small sample due to the limited number of workers in the region could reduce the power of our study to detect an effect. These factors can also limit the generalizability of the results. Our nding will be more generalizable to small neurological continuing education workshops developed in collaboration with instructors, community members and researchers.
A major limitation with this evaluation of training effects was that we only evaluated levels 1 and 2 of the Kirkpatrick's level 3 and 4, we cannot imply that the positive effects we observed in reaction and learning will translate into changed behaviors or positive results in the clinical environment.

Implications
The neurorehabilitation workshops provided in Honduras resulted in ndings of participant satisfaction, learning of pediatric neuromotor development and awareness of adult cognitive therapeutic exercise, as well as feelings of increased con dence and team collaboration. Based on the results of this study, iterations of the Network of Rehabilitation Workers of the Americas visiting professor program and workshops held in 2018 and 2019 were designed to maximize opportunities to establish the rehabilitation professions and develop local leadership through: a) incorporating an active teaching role for Honduran professionals in future workshops, b) establishing further linkages with the rehabilitation and medical professions (including physiatrists, neurologists, nurses, psychologists), c) visiting rehabilitation centers for applied learning and mentorship, d) interacting with the local Universities, and e) advocating for inclusion of persons with disabilities in all aspects of society.
Historically, capacity building in developing countries has been externally driven, related to project implementation, and often resulted in disempowerment of local organizations/people rather than local ownership. The Network of Rehabilitation Workers of the Americas is a professional, interdisciplinary north-south collaboration focused on rehabilitation capacity building which utilizes a participatory approach. The work of the Network of Rehabilitation Workers of the Americas is relevant and timely and despite its urgent need for strong leadership in rehabilitation, there are no such collaborations in Honduras. Our collaboration aims to improve leadership and professional capacity needs for stronger health systems and improved outcomes as well as to alleviate the burden of disease and disability in Honduras.
Although the participants had differing backgrounds related to level of education and work assignment, and the workshop instructors were from a different country, the participants evaluated the workshops as relevant and held very positive attitudes about the perceived outcomes. It would appear that the attainment of relevancy was due to the inclusion of local practitioners in the planning of the workshop and the selection of topics. We recommend that adequate time is taken by workshop planners to ensure relevancy.

Conclusions
Workshops offered to the mixed healthcare workers in Honduras through a community engaged, international academic network resulted in positive reaction; acquiring of skills, attitude, con dence and commitment; advanced learning opportunities; and acquisition of relevant knowledge which have the potential to improve quality of care for people with neurological conditions in the northern region of the country. Case based learning and a supportive peer environment were used during the workshops and were important elements in promoting learning. Future visiting professor workshops will expand on this successful pilot to include more opportunities for applied learning and interprofessional collaboration. Availability of data and materials

Abbreviations
The datasets used and/or analyzed during the current study are available from Angela Busch upon reasonable request.