The following protocol was approved by the Partners Institutional Review Board (IRB) on September 6, 2019, submission number 2019P001266.
Study Design
The research team conducted a pilot study to assess the feasibility and acceptability of a telehealth support system implementation. Feasibility refers to the viability of the telehealth system in the context of a CBR center within a refugee camp, to support CBR workers. Acceptability encompasses the extent to which CBR workers consider the telehealth system to be appropriate, based on anticipated or experienced value of the intervention [12].
Conceptual Framework
This study utilized the RE-AIM and Dynamic Sustainability Framework (DSF) (Figure 1). The RE-AIM is recognized for assisting the transition of public health intervention pilots to real-world implementation through five sequential dimensions: Reach, Effectiveness, Adoption, Implementation and Maintenance [13]. Appropriate development of the first four dimensions is indicative of the intervention’s potential for the fifth and final stage of sustainable implementation: maintenance [14]. The RE-AIM framework was used to plan recruitment and training, use of the system, and develop survey and focus group questions that helped to holistically evaluate the feasibility and acceptability.
The DSF prioritizes sustainability through continuous integration and adaptation to progressively improve the ‘fit’ between the intervention and three multi-level contexts: intervention (telehealth system), practice setting (CBR center) and ecological system (policy, regulations, other practice settings) [15]. ‘Fit’ refers to the integration of innovations into health systems. The DSF was designed to understand how health intervention sustainability can be improved through implementation. The DSF informed the pilot procedure through continuous integration stakeholder feedback, in order to optimize the system and increase ‘fit’.
Recruitment
The structure for CBR can vary significantly by program. In the Baqa’a Camp, CBR managers maintain relationships with government agencies, local authorities and centers, oversee facilitation of CBR strategy according to the CBR matrix, and manage CBR workers. CBR workers in this study were required to be affiliated with the Baqa’a Camp CBR Center in Amman, Jordan, fluent in Arabic or English, and presently treating individuals with physical or mental disabilities. CBR workers were recruited to use the telehealth system and inform the feasibility and acceptability. CBR managers were required to be affiliated with at least one refugee camp CBR Center, fluent in Arabic or English, and presently managing CBR workers or CBR sites. CBR managers were recruited to provide information on the feasibility of, and potential to sustainably implement, telehealth in CBR centers nationally.
Participants were recruited via convenience sampling and informational flyers outlining study purposes and procedures, on-site at the Baqa’a Camp CBR Center. CBR workers were asked to participate in one system training session, at least one telehealth session, survey, and one focus group (Figure 2). CBR managers were provided an informational handout outlining the study purpose and procedures and were required to attend one focus group (Figure 3).
Zoom Training
Skype was tested primarily for implementation in the CBR Center. Skype Translator is a feature available via Skype that translates voice or text in real-time, which is desirable for providing assistance across a language barrier. Due to an inability to facilitate connection through skype within the CBR Center and other locations in Amman, Jordan, Skype was determined to be not feasible for supporting CBR workers and was excluded from further research procedure.
Zoom Video Communications software was used to evaluate the feasibility and acceptability of videoconferencing or chatroom-based telehealth sessions. Zoom was selected because of its user-friendly, easy to navigate interface and technical design, software customization, security encrypted video conferencing and data sharing. Although Zoom does not currently offer an Arabic interface, a picture intensive training document with corresponding word description in Arabic was developed by the research team to guide the training protocol. This document was also made available for CBR workers to access after the training for use in telehealth sessions. CBR Workers used Zoom on an encrypted, HIPAA compliant iPad provided by the research team.
A 30-minute onsite training occurred weekly for four weeks at the Baqa’a Camp CBR Center. The training protocol included a didactic component and post-training assessment delivered by the two members of the research team. The didactic lesson covered the goal and rationale of the telehealth support system, device uses and procedures, shut down and startup of the system, customizations of the system, and troubleshooting common problems. The post-training assessment was used to assess participant competence as a new user and required participants to demonstrate opening the application, initiating video conferencing requests, initiating a chat room, connecting to audio and video. Scoring of the post-training assessment was determined based on successful or unsuccessful completion of five tasks: opening the application, selecting the correct icon to initiate a meeting, video calling a research team member, connecting to audio, and ending the call. A score of 80% (4/5 tasks completed correctly and unassisted) or better was necessary to participate in a telehealth session. The telehealth system was accessible to CBR workers for a one-hour block of time during a scheduled session, after completion of the training.
Telehealth Sessions
CBR workers in the camp were encouraged to participate in telehealth support sessions with the research team of two Doctor of Occupational Therapy students and one occupational therapy practitioner located in Boston, MA, as well as one project coordinator and physiotherapy practitioner located in Amman, Jordan. Three research team members were fluent in Arabic. CBR workers could arrange for assistance in sessions before client intervention (Type A), assistance during client intervention (Type B), or assistance after client intervention (Type C). At least two research team members discussed the client profile and health history prior to each telehealth session with the CBR worker. The research team recorded the session type, topic, real-time adaptations, modifications and feedback from CBR workers after each session. Feedback that recommended changes in training or sessions was considered by the research team and either adapted for the next session or remained unchanged due to technological or procedural constraints. The training protocol and telehealth sessions were used to assess CBR worker aptitude for utilizing telehealth and investigate the feasibility, acceptability, and barriers to receiving assistance through telehealth.
Surveys
The purpose of the survey was to further identify the acceptability, demand and practicality of implementing a telehealth support system after having experience using the system. CBR workers completed surveys following telehealth sessions to inform acceptability based on user-perception of 1) the adequacy of support provided using the telehealth system, 2) preference to use the telehealth system in future sessions, 3) the appropriateness of the intervention in CBR work settings, 4) the ease of navigation for the telehealth system. The response options for the survey were based on a Likert-scale rating of one through five, representing strongly disagree, disagree, neutral, agree, strongly agree, respectively. The survey also included two prompts to suggestions for improvements and other comments that were used in the development of themes.
Focus Groups
The goal of the focus groups was to encourage participants to generate ideas about enhancing prospective telehealth system feasibility and acceptability and discuss improvements to be made in future applications for CBR settings in Jordan. There was one focus group for CBR workers and another for CBR managers. Topics related to technical, operational, economic, infrastructural and cultural feasibility and acceptability, and macro-level contextual factors such as policy, guidelines, incentives were discussed. Potential barriers to sustainability and scalability of the intervention was also discussed in focus groups. The focus groups were audio recorded on a password protected and encrypted IPAD in a private room within the Baqa’a Camp CBR Center with the study participants, and three members of the research team.
DATA ANALYSIS
The training scores were assessed to determine readiness to participate in telehealth sessions. Feedback from each training was used to adapt and inform subsequent trainings and documented to contribute to the development of themes regarding feasibility. Telehealth session feedback was used to adapt and inform subsequent telehealth sessions and documented to contribute to the development of themes regarding feasibility and acceptability. Likert scale survey responses were analyzed as the average response score plus or minus the standard deviation to guide ‘fit’. Focus group responses were collected to inform themes of feasibility, acceptability and sustainability.
Data collected from the focus group, open-ended survey questions, training and session feedback was evaluated and organized using NVIVO 12 software. The audio recordings for the focus groups were transcribed and coded. Responses were developed into themes regarding feasibility and acceptability. The developed themes from the focus groups, survey questions and session feedback were dispersed through triangulation to achieve and confirm the accuracy of the data collected. The aim of this analysis was to identify user-focused perceptions and themes regarding feasibility and acceptability of a telehealth system supporting CBR workers in CBR Centers.