Technology Assisted Task-sharing to Bridge the Treatment Gap for Childhood Developmental Disorders in Rural Pakistan: an Implementation Science Case Study

doi:10.21203/rs.3.rs-1335393/v1

Download PDF

Research

Technology Assisted Task-sharing to Bridge the Treatment Gap for Childhood Developmental Disorders in Rural Pakistan: an Implementation Science Case Study

https://doi.org/10.21203/rs.3.rs-1335393/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background: As in many low-income countries, the treatment gap for developmental disorders in Pakistan is near 100%. The World Health Organization (WHO) has developed the mental Health Gap Intervention guide (mhGAP-IG) to train non-specialists in the delivery of evidence-based mental health interventions in low resource settings. However, a key challenge to scale-up of non-specialist-delivered interventions is designing training programs that promote fidelity at-scale in low resource settings. In this case study, we report the experience of using a tablet device-based application to train non-specialist, family volunteers in leading a group parent skills training program, culturally adapted from the mhGAP-IG, with fidelity at-scale in rural community settings of Pakistan.

Methods: The implementation evaluation was conducted as a part of the mhGAP-IG implementation in the pilot sub-district of Gujar Khan. Family volunteers used a technology-assisted approach to deliver the parent skills training in 15 rural Union Councils (UCs). We used the Proctor and RE-AIM frameworks in a mixed-methods design to evaluate the volunteers’ competency and fidelity to the intervention. Outcome was measured with the ENhancing Assessment of Common Therapeutic factors (ENACT), during training and program implementation. Data on other implementation outcomes including intervention dosage, acceptability, feasibility, appropriateness and reach was collected from program trainers, family volunteers and caregivers of children 6-months post-program implementation. Qualitative and quantitative data were analyzed using framework and descriptive analysis, respectively.

Results: We trained 36 volunteers in delivering the program using technology. All volunteers were female with the mean age of 39 (±4.38) years. The volunteers delivered the program to 270 caregivers in group sessions with good fidelity (scored 2.5 out of 4 on each domain of the fidelity measure). More than 85% of the caregivers attended 6 or more of 9 sessions. Quantitative analysis showed high levels of acceptability, feasibility, appropriateness and reach of the program. Qualitative results indicated that the use of tablet device-based application, and the cultural appropriateness of the adapted intervention content, contributed to the successful implementation of the program.

Conclusions: Technology can be used to train non-specialist family volunteers in delivering evidence-based intervention at-scale with fidelity in low resource settings of Pakistan.

Developmental disorders

WHO mhGAP

Technology-assisted task-shifting

Family volunteers

low resource settings

The present study provides an example of using implementation science frameworks to explore barriers to and facilitators of using technology to train non-specialist family volunteers to implement a parent skills training program and measure program fidelity at-scale in a low resource community setting.
This study administers validated implementation science instruments to measure program acceptability, feasibility, appropriateness and reach and provides a comprehensive perspective of diverse stakeholders including organization, providers and consumers regarding program implementation in real-world settings.
The present study suggests that although technology is a feasible and acceptable implementation strategy to train non-specialist female volunteers, other barriers – notably competing demands in their homes faced by the volunteers, plus community norms that potentially limit the ability of women to participate alone outside the home could reduce the reach of the program.
While technology-assisted application was well accepted as a training tool, most of the caregivers refused consent to video record a live caregiver-child interaction for evaluation purposes. Understanding the use of technology from end-users’ perspective is critical to the success of scale-up efforts using technology.

Prevalence rates of developmental disorders are estimated to be 1 in 162 children globally (1). Many low resource settings lack trained human resources and evidence-based intervention packages for childhood developmental disorders, resulting in a documented treatment gap of over 90% (2). In the past two decades, there has been an increasing focus on actions to reduce this gap, particularly in low resource settings (3). A key landmark was the launch of the WHO Mental Health Gap Action Program (mhGAP) which provides evidence-based guidelines for the treatment of priority mental health conditions, including developmental disorders, by non-specialists in low resource settings (4).

The mhGAP intervention guide offers a holistic, evidence-based package of care including guidelines for assessment and management of developmental delays and disorders in children. The intervention package recommends parent skills training to promote child development using evidence-informed strategies (5). It emphasizes training non-specialist health workers such as parents, caregivers, teachers, nurses and community volunteers to deliver the program (task-shifting) in primary care settings or community-based facilities. The program takes a trans-diagnostic approach and addresses a range of developmental conditions. However, ensuring program fidelity at-scale remains a key implementation challenge to disseminating such programs in low resource settings (6).

Pakistan is a low- and middle-income country in South Asia with an estimated population of 220 million. Over 6% of Pakistani children suffer from a developmental delay or disorder (7). To address the barriers to access and improve the quality of child mental health services in low resource settings of Pakistan, the National Ministry of Health is implementing mhGAP through primary health care platforms. The sub-district of Gujar Khan is one of the pilot sites for the implementation of mhGAP-IG in rural Pakistan. Presently, Gujar Khan has only one public special education institution, which is underfunded, oversubscribed and inaccessible to most children with developmental disorders in need of healthcare, educational and rehabilitative services. The nearest specialist mental health care facility is the Institute of Psychiatry, Rawalpindi, 40 km away. Previous studies conducted in Gujar Khan using the Ten Questions Screen (TQS) identified 34.2% of children as having developmental difficulties (8). Furthermore, lack of awareness about childhood developmental disorders in family and carers, and stigma associated with such conditions were identified as barriers that hamper access to care and increase the burden of caregiving for parents of children with developmental disorders (9).

Previously, we had converted the mhGAP guidelines into training videos for caregivers of children with developmental disorders and hosted them on a tablet device. We trained non-specialist family volunteers to deliver this technology-assisted parent skills training and pilot tested the intervention with 70 families with children with developmental disorders. This model of service delivery was found to be acceptable, feasible and resulted in improving child outcomes (10). Based upon these preliminary findings, we scaled-up the program and evaluated its implementation to explore how technology-assisted task sharing at-scale impacts training, fidelity, reach, and adoption. We also explored facilitators and barriers for scaling-up technology-assisted, task-sharing interventions in low resource rural settings. Our specific research questions were; a) can family volunteers be trained using technology to deliver parent skills training program at-scale with fidelity? b) What is the acceptability, feasibility, appropriateness and reach of technology-assisted, family volunteers delivered parents skills training intervention in rural community settings of Pakistan?

Study design:

Our evaluation was part of a cluster randomized controlled trial (cRCT) of the technology-assisted parent skills training intervention to improve clinical outcomes in children with developmental disorders (11). The trial findings have been published elsewhere (7). The evaluation used a mixed-methods design in the intervention arm of the trial. Quantitative data on implementation outcomes was collected using structured, validated measures (12) at 6-months’ post-program implementation. We explored participants’ perceptions about the acceptability, feasibility, appropriateness and reach of the program and conducted qualitative in-depth interviews with program trainers, provider and consumers at 6-months post-program implementation.

Implementation evaluation frameworks:

Our implementation evaluation was informed by Proctor and colleagues’ (13) and the RE-AIM (http://re-aim.org/) framework. According to Proctor and colleagues’, evidence-based mental health interventions are implemented through strategies at different levels (organizations, providers, consumers), whose impact is interactive with implementation, service, and client outcomes (Figure 1). To complement the findings of Proctor and colleagues’ framework, we used the RE-AIM framework; which is designed to evaluate the impact of public health interventions in real-world settings. It focuses on key program elements including its reach, effectiveness, and adoption.

We measured implementation outcomes and calculated RE-AIM indices at three levels; organization (including program trainers), family volunteers (hereafter, provider) and at caregivers (hereafter, consumer). We defined and measured;

Fidelity as the competency of program trainers and providers to deliver the parent skills training program in the manner in which it was intended to be delivered.
Acceptability as the perception of program trainers, providers and consumers that the program (content, mode of delivery and delivery agent) was relevant, suitable and satisfactory.
Feasibility as ease of access to parent skills training program resources, participation in the training sessions and adoption of program strategies by providers and consumers.
Appropriateness as perception of ‘program trainers, providers and consumers that program is useful for its state purpose.
Adoption as the proportion of participants (both provider and consumers) who agreed to i) participate and continue to implement the parent skills training program and ii) promote/disseminate program to other families of their community.
Reach as the proportion of the caregivers who received the parent skills training program delivered by family volunteers.

Study settings:

The program was delivered in 15 rural Union Councils (UCs) of the sub-district Gujar Khan. A UC is the smallest administrative unit within a sub-district and has a Basic Health Unit (BHU) that provides health care to local population. Each BHU is staffed by a medical doctor, a dispenser, 15–20 Lady Health Workers (LHWs) working under supervision of a Lady Health Supervisor, a Lady Health Visitor and a vaccinator. LHWs are trained to provide preventive maternal and child health care and education in the community. Each LHW is responsible for about 100 households in her village.

Identification of children with developmental disorders

In the present study, children with developmental disorders were identified with the help of Lady Health Workers (LHWs), in two phases. Firstly, the research team along with Lady Health Workers (LHWs) conducted a door-to-door survey and screened all children, aged 2-12 years in the LHW’s catchment area for developmental disorders using Ten Questions Screen (TQS). Secondly, children who screened positive were assessed by a trained clinical psychologist following the clinical assessment guidelines (history & clinical examination) outlined in the mhGAP-IG (14). A written informed consent was obtained from all the study participants before their participation in the study. Where a participant was unable to write, a thumb print was placed in lieu of signature.

Identification of family volunteers

The family volunteers were family members or parents or caregivers of children with developmental disorders. They had at least eight grades of formal education; were voluntarily willing to be trained and supervised to deliver the intervention to 4–5 families in their villages for at least 6-months. The family volunteers were identified during the door-to-door screening survey. A written informed consent from each family volunteer was obtained before their participation in the study.

Study participants

In addition to the family volunteers, the study participants for the implementation evaluation were program organizers and caregivers of children with developmental disorders.

Program organizers (hereafter, referred to organizers in the manuscript) were master trainers who had post-graduate education and specialized training in child mental health as well as program trainers who were graduates with Masters’ degree in psychology or social sciences and had at least one year of experience of delivering psychological interventions in the study settings. They were trained in parent skills training program by master trainers and cascaded down the training to the providers using tablet-based tool.
Consumers of the intervention (caregivers of children with developmental disorders, residing in the study sub-district for the duration of the study)

Intervention- Technology-assisted, non-specialist-delivered parent skills training program:

We adapted the content and delivery of the mhGAP-IG parent skills training guidelines to suit delivery by non-specialist volunteers in low resource rural Pakistan.

Cultural adaptation of program content: The adapted program consists of 9-weekly group sessions and aims to train caregivers to promote communication, socioemotional development and adaptive behaviors and manage co-morbid conditions and motor difficulties in their children. It also highlights the significance of caregivers’ self-care and teaches coping skills to caregivers to improve their own psychological well-being. A complete description of technology assisted family volunteers delivered parent skills training program is mentioned elsewhere (7).
Delivery of the program: We converted the adapted program into an application consisting of training videos hosted on an Android tablet device. To help caregivers learn from the lived experiences of families of children with developmental disorders, the key program messages and strategies were converted into real-life narratives of three children with developmental disorders and their caregivers and family members. The narrative script is simple, interactive and allows intervention providers and caregivers to discuss each scenario in the context of their own lives and develop individualized management plans for their children. To address the challenge of limited internet access in rural areas of Pakistan, we hosted the intervention videos on the tablet to make then accessible off-line. The intervention resources are freely available in Urdu (the national language of Pakistan) using the following weblink https://fansforkids.org/.
Training and supervision of providers: We followed a cascade model of training and supervision (15) (Figure 2). A master trainer trained 10 trainers who cascaded down the training to 80 potential providers, using a tablet-based training application. Each trainer was assigned a case load of 6-8 providers for training and supervision. Each trainer trained providers of the program in 9 weekly group sessions of the parent skills training program, followed by competency rating of the providers by a master trainer on ENhancing Assessment of Common Therapeutic (ENACT) during practice sessions.

Outcome measures:

The data on implementation variables was collected during the program implementation and at 6-months post-program implementation at three implementation levels; organization, providers and consumer (FTable 1).

Quantitative measures and analysis:

Measuring fidelity of the program using ENhancing Assessment of Common Therapeutic (ENACT) tool: We evaluated the fidelity of the program delivery by non-specialist providers by live rating a random sample of 20% sessions delivered by each ‘provider’ on the ENACT tool (16), adapted for the parent skills’ training program. The ENACT is an observational rating scale intended to be used when observing a trainee interact with a real or standardized (mock) patient. It consists of 18 items, rated on a three-tiered response system. A score of 1 means `needs improvement`, 2 means `done partially` and 3 means `done well`. A mean score of 2 on each domain of ENACT is required to fulfil the fidelity criteria for the program delivery. The tool has been extensively used previously to measure the common factors competence among non-specialist providers delivering mental health services in low resource settings (17).

Adaptation of ENACT for parent skills training: General adaptations were made to the tool such as use of words ‘parents/caregivers’ and ‘family volunteers’ instead of using ‘clients’ and ‘clinician’. In the adapted ENACT, 8 items were added to reflect the essential therapeutic skills required to deliver the technology assisted parent skills training program in group setting. Domains of ENACT (n=7) that were specific for one-on-one therapy sessions were excluded in consultation with developer of ENACT. The final adapted ENACT for parent skills training consisted of a 20-items tool and incorporated a four-tiered response system. A mean score of 2.5 on each domain of adapted ENACT was required to fulfil the fidelity criteria for the program delivery (see supplementary material).

Intervention monitoring data: Apart from collecting data on the fidelity of intervention delivery by non-specialists, we also collected data on intervention dosage and analyzed notes of the supervision meetings.

Implementation outcome measures: We used the Applied Mental Health Research Dissemination and Implementation measurements (12) to measure implementation outcomes at organization, provider and at consumers’ level. These tools have been validated to measure the implementation outcomes including feasibility, acceptability, appropriateness, adoption and penetration of the mental health programs in low and middle income countries (12). We translated and adapted the tools following the standard procedure (forward and back translation of tool, conducting cognitive interviews and piloting of the tool and blind back translation) for translation and adaptation of the mental health instruments for trans-cultural research (18).

Adaptation of AMHR D&I measures for parent skills training program: General adaptations: The general adaptations to the AMHR D& I measures included change of program’s name, replacing the word ‘provider’ with ‘family volunteers’ and ‘mental health services’ to ‘services for caregivers of children with developmental disorders. As parent skills training program targets whole family and has been designed for children of both genders with developmental disorders and their caregivers, the items exploring appropriateness of program from gender perspective such as ‘does program fit with the male/female culture in your country? were removed from all three versions (consumer provider, and organization). The reach sub-scale assesses the reach of mental health services/programs to underprivileged groups, males/females, orphan children and people in military office. All such items related to the reach of program to male/female and military personal were excluded as these items were contextually and conceptually not relevant.

Specific adaptations: At provider’s level, the intervention specific examples such as delivery of program; monitoring of child’s progress using tablets and participation in monthly supervision meetings were included. As family volunteers were delivering the program on volunteer basis, the whole domain of ‘individual professionalism’ measuring satisfaction with salary, job stability, professional goals and career advancement, was excluded. In an adapted version of the tool, each item is scored on a 4-point scale ranging from 0 ‘not at all’ to 3 ‘a lot,’ with an additional category for ‘don’t know/not applicable’. We calculated the Cronbach’s Alpha for each domain of adapted tools to assess the reliability. The values of Cronbach’s alpha ranged between 0.70 to 0.90, except the domain of ‘reach’ (α = 0.30).

Quantitative analysis: The quantitative data was analyzed using descriptive analysis. To compare the implementation variables across three levels of participants (organization, providers and consumers) mean differences on each domain of AMHR D & I measures were calculated.

Qualitative measures and analysis:

We conducted 30 in-depth interviews with six groups of participants (see Table 2 for categories of participants) at 6 months post-program implementation using a semi-structured interview guide. The respondents were selected purposively depending upon their willingness and availability. The average duration of each interview was 55minutes (±9.5) All interviews were audio recorded and transcribed verbatim.

Qualitative analysis: The data were analysed using framework analysis (19). In the first step, the transcriptions of interviews were read again and again for familiarization, identification of any data (thoughts, feelings and impressions) that seemed of potential interest and significant for our research question, and to generate codes. After generating the initial coding of few transcriptions, an analytical framework around a priori categories of interest (acceptability, feasibility of the program) was developed to index the data from the coded transcriptions. After indexing the data of all transcriptions, the research team summarized the indexed data for each category and organized it in a chart form. Finally, mapping and interpretation of the data was done to; describe each category; identify any commonalities and differences between the data (across different group of participants); and map connections between categories to explore causality (if there was any). Findings of the qualitative study are mentioned in Table 5.

The case study has been reported following the Standards for Reporting Implementation Studies (StaRI) checklist (20) (please see additional file).

Parent volunteer (provider) participation and characteristics

Out of 62 trained providers, 36 delivered the program to caregivers in local villages. Twenty-six providers were not able to deliver the program due to health problems (n=7); competing family commitments (n=7); having to work (n=6) or not achieving competency (n=6).

The mean age of the 36 participating providers was 35 (±4.38) years. For 61% of the providers, education level was 10 years (Table 3). All 36 providers were females (95% mothers and 5% paternal aunts) with the mean age of 39 years (±4.38). 81% of them were not currently employed and 70% were living in nuclear family system.

Parent (consumer) participation and characteristics

Out of 326 potential parent-child dyads in 15 Union Council, 270 parents of children with development disorders who met the eligibility criteria were enrolled in the study (see Figure 3 for the eligibility criteria and flow of the participants through study). Out of 270 caregivers enrolled 85% (230/270) of the trial participants attended 6 sessions (±1) and only 166 caregivers participated in the implementation evaluation. The mean age of the consumers was 35 (±7.6) years. Half of the consumers had less than 5 years of education or no education (50%, 83/166).

Provider delivery of the program

A total of 504 sessions of the program were delivered by 36 providers in 79 groups within the duration of 6-months. The average group size was 6 (range 5 to 7). 85% (230 /270) of the consumers attended 6 sessions (range between 5 to 7). Average duration of a group session was 102 minutes (±15). The average group size was 6 (range 3 to 7).

Cascading supervision sessions with providers

Over the period of 6-months, 10 program trainers organised 30 group supervision meetings of providers. The average attendance in the group supervision was 75%. In turn, providers conducted 178 group supervision sessions with consumers after initial training of 9 sessions. The attendance in these supervision sessions was 73%.

Fidelity of the program:

To assess the fidelity of the program, 20% of the sessions (103/504) spread over 36 providers were rated at three time-points (between program session 1-3, 4-6 and 7-9). The Cronbach’s alpha reliability of the adapted version of ENACT for all three-time points ranged between 0.70 to 0.93. The intervention sessions were delivered with good fidelity as all providers scored on an average 2.5 or more (mean [SD], 2.97±.21) out of 4 on all items of the adapted ENACT at the three time-points.

The primary clinical outcome of interest was change in child’s functioning at 6-months post intervention implementation. The results of the overall trial have been published elsewhere (7).

Acceptability, feasibility, appropriateness and reach of the program at-scale:

The data on implementation outcomes was collected from 166 consumers, 36 providers and 14 members of the program team including program trainers (Table 4).

Acceptability of the program:

Caregivers (n=166) indicated that the program was highly acceptable to them (mean scores of acceptability domain, 39.49 ±5.92). They reported high satisfaction on receiving the program training from a ‘family volunteer’ (item response ‘a lot’ 2.77±.47) as she was perceived to be a someone who listens, understands and addresses their questions or concerns about their child and program (2.74±.49) and takes interest in their concerns and problems (2.75±.49). Family volunteers were able to earn the trust of the caregivers of children with developmental disorders (2.77±.46) as she was perceived qualified enough to deliver the program (2.77±.45).

The program was perceived as highly acceptable by the providers (n=36) (mean scores of acceptability domain, 34.2±3.01). They not only liked the program materials (3.90±.31), but they also felt good while delivering the program to the families (3.70±.67). They felt the program was an appropriate intervention (3.90±.31); moreover, the skills they learnt while delivering the program were perceived to be useful for the families (3.90±.31). They were also satisfied with the amount of the training (3.90±.31) and supervision (3.60±.51) they received by the program trainers.

The program acceptability at organization level was very high (17.79±2.45). The organisers felt that delivery of the program was a source of contentment for the program team and trainers. It was not only helpful and beneficial for the reputation of the organization; but it could also create more opportunities of service delivery for the organization.

Feasibility of the program:

The caregivers perceived that participating in the parent skills training program was feasible for them (31.29±4.76). However, they reported challenges in taking time-out of their daily routine to attend weekly training sessions (1.86±.90); in managing household responsibilities; in implementing program strategies at home by themselves (2.15±.82) and in finding childcare for their other children during the time they were attending the session (2.09±1.03). Caregivers reported that community members did talk negative about the families seeking program services for their children (1.80±.75).

The program was very feasible for providers to deliver (domain score 49.7±5.73). The item wise analysis showed that they felt skilled enough to deliver the program (3.80±.422) and had appropriate time to implement all the activities of the program (3.40±.84) including organizing group; providing training and peer supervision to caregivers in their villages and continuously monitoring child’s progress during program implementation.

The program trainers perceived that implementing the parent skills training program was feasible for the organization (42.29±5.44). According to them, the organization had sufficient resources including a sufficient pool of trainers (3.86±0.53), sufficient finances (2.75±0.75), an adequate transportation system (3.57±0.75) and the necessary equipment (3.86±.36) to train non-specialists and implement the program in the study settings. Moreover, program trainers also mentioned that they received adequate administrative support (3.62±0.65) and clinical supervision (3.77±0.43) to provide supervision and support to the providers.

Appropriateness of the program:

As the intervention content was adapted into a training application with ‘real-life’ narratives (10) both the consumers and providers rated the program as highly appropriate and in keeping with the local traditions and cultural values (the mean score of consumers on the appropriateness domain was 24.62±4.17 and 31.8±4.62 for providers). The consumers perceived that the program was a good way to address their and their child’s problems (2.43±.71). The program strategies they learned helped them to deal with their worries (2.46±.63) and solve their problems related to their children (2.49±.78). The consumers agreed that these strategies would also be useful to other caregivers of their community who have similar problems (2.43±.66). However, attending group training sessions would be inconvenient for these caregivers due to their household responsibilities (1.72±.89).

Providers reported that they were satisfied with their role as a provider of the program and they will keep implementing the program to the families (3.50±.85) in their villages under the supervision of the trainers (3.20±1.1). They appreciated the idea of implementing the program through non-specialist, family volunteers; however, ‘difficulties in seeking permission from their own family to deliver program in the community and competing demands on time due to household responsibilities’ were identified as potential barriers to continue to implement the program through family volunteers.

The program trainers perceived that the program was highly appropriate to the needs and cultural values of the consumers (mean scores of the domain, 36.43±4.16). According to the program trainers, as the program content was designed to meet the needs of the consumers, it was likely to be useful to manage problems of children with developmental disorders and their caregivers. They perceived that the delivery of the program to the families of children with developmental disorders in the study area was aligned with the core values and goals of their organization and implementing this program was a priority for leaders at their organization.

Reach of the program:

Both the program trainers and providers perceived that the reach of the program was high (mean scores of reach domain for program trainers and providers were 11.07±2.84 and 10.6±2.01 respectively). The item level analysis indicated that according to the providers, the community members were fully aware that program was being delivered in the community by the family volunteers (3.50±0.70). They also perceived that all those caregivers in their community who might be in need of a parent skills training program, specifically parents from low socio-economic status, would still find this program useful and attend the training (2.40±0.69). However, according to the consumers the reach of the program was comparatively low (6.99±1.98).

We also calculated the reach of the program using RE-AIM indices. We successfully established 36 potentially self-sustaining village based ‘Family Networks’ led by 36 trained providers working under supervision of trainers from the local nongovernmental organization. The target population for the program was 270 caregivers and their children who met the eligibility criteria and were enrolled in the program. Out of 270 participants, 85% (230/270) of the population received the program training by 36 providers using the technology platform.

Qualitative findings:

Most of the respondents in the qualitative study reported that the program content addressed the problems of children with developmental disorders and caregivers’ concerns that they faced in taking care of their child (see Table 5 for themes and relevant quotes). The use of a training application hosted on the tablet was appreciated by the respondents. The participants also mentioned that program illustrations (personas, scenarios and sittings) were quite reflective of their surrounding environment, people and cultural norms and helped them to effectively learn program strategies. The qualitative data analysis indicated that family support from their own family was identified as an important factor in caregivers’ involvement with the program. However, the engagement of providers with the program relied, in turn, on support they received from their own family members in delivering the program. Moreover, the household responsibilities such as looking after the children and domestic animals and working in the fields got in participants’ way to attend the program sessions. For both the providers and consumers there was reliance on their own family members to cover other household responsibilities so that they had time to participate in the program sessions.

Both providers and consumers expressed their satisfaction over the duration of the sessions and suggested to schedule the sessions a week apart so that consumers are able to attend the sessions conveniently. The respondents expressed their satisfaction on organizing the trainings at basic health units (located within their respective villages) or at the house of providers. Intervention delivery in village-based groups was regarded as an acceptable format of parent skills training by a number of respondents. According to participants, sitting together as a group served as a learning platform for them in which they could share their problems with each other. The respondents expressed that having a session in a group helped them to validate and normalize their feelings about their child’s health condition; additionally, they helped to learn from each other through this experience.

This case study describes the implementation-evaluation of a family volunteer-delivered, technology-assisted parent skills training program for children with developmental disorders in low-income rural communities. The results showed that technology can be used to train family volunteers in the delivery of parent skills training program at-scale with fidelity. The findings of quantitative and qualitative analyses show that program’s content, mode of delivery using technology and non-specialist family volunteers as the delivery agent were perceived to be highly acceptable, feasible and appropriate to consumers, providers and organizers and the technology assisted program had a high reach according to them. The study also illustrated several other important points about scaling up task-shifting interventions in low-resource settings. First, in addition to the tablet platform, adapting materials to make them engaging and readily identifiable as commonly encountered situations in the community was a key factor in the acceptability of the program. Second, measuring competency at the end of training was an essential step in assuring fidelity; nearly 10% of family volunteers entering the training could not be deployed to their communities because they did not achieve the required level of skill. Third, while the technology-assisted training and delivery was itself feasible and well-accepted, other barriers – notably competing demands in their homes faced by the family volunteers and the parents they worked with, plus community norms that potentially limited the ability of women to participate alone outside the home – reduced the reach of the program. Future versions of the program may need to address these factors, perhaps by involving additional family members or providing other support during program administration. Finally, despite rating the program very positively, consumers reported difficulty carrying out what they had learned at home and required more supportive supervision to learn the strategy and time to practice these strategies.

Task-shifting has been frequently used as an effective implementation strategy to tackle the shortage of healthcare work force, especially in low resource setting, where there is a scarcity of mental health professionals (21, 22). In the present study, we trained family volunteers to deliver parent skills training program to the caregivers of children with developmental disorders in their own villages. The findings of qualitative in-depth interviews showed that locally identified family volunteers were highly acceptable as delivery agents. According to the participants, family volunteers exhibited characteristics of trustworthiness and altruism which contributed significantly to increase the program’s acceptability and made it appropriate for the targeted community/population. Delivery of trans-diagnostic, skills training interventions through non-specialist facilitators is a widely accepted approach and has the potential to deliver psychosocial interventions in low resource, fragile and humanitarian settings (23).

However, one of the frequent concerns regarding implementation of evidence-based practices by non-specialist at-scale is the decrease in program fidelity and its effectiveness (6) due to diverse mental health needs of the populations and challenges associated with maintaining the quality of training and supervision of non-specialists at-scale. In the present study, we addressed these challenges by developing a tablet-based application for training and program implementation. This innovation is consistent with the current evidence that highlights the considerable potential of digital mental health interventions to maintain high fidelity of the evidence-based programs at-scale (21) and are reported to be as good as training by specialists (23). The findings of qualitative analysis showed that use of culturally appropriate avatars and real-life stories of three children with developmental disorders and their families helped providers and caregivers to gain deeper understanding of the issues faced by children and learn strategies and skills to manage child’s conditions. This is consistent with the findings of previous studies in which digital mental health interventions were reported to be highly feasible and acceptable to not only promote but also to treat mental health problems in low resource settings (24, 25). The findings of our study are critical to the efforts of promoting the use of technology platforms and training non-specialists to bridge the treatment gap for mental health in low resource settings, globally.

A key hallmark of our study is the use of implementation science frameworks and valid and reliable implementation outcome measures alongside a clinical trial to measure the effectiveness of the intervention (26), which are often overlooked in implementation science studies (27, 28). Furthermore, we translated and culturally adapted the implementation outcome measures following the standard procedure recommended for translation and adaptation of the mental health instruments for trans-cultural research (29). Although our study was not powered to assess the implementation effectiveness, we systematically collected the data on implementation outcome measures at consumers, providers and at organization levels. The use of implementation science frameworks and mixed-methods study design to collect data helped us to identify critical insights that are imperative for the successful implementation of parent skills training program in low resource community settings of Pakistan. Our study highlighted that technology in the form of training videos can be a feasible and acceptable strategy to deliver mental health program with fidelity at-scale in real-world settings.

The definitive randomized evaluation of intervention effectiveness demonstrated that the program did not result in statistically significant improvement in clinical outcomes of children when implemented at-scale. This difference of intervention impact from the exploratory study (10) could be because of the multiple reasons; in the definitive cRCT, we used trans-diagnostic eligibility criteria to enrol children with developmental disorders in the cRCT, as a result, children with a range of developmental disorders conditions including intellectual disability, motor difficulties, speech and communication difficulties and Down syndrome were included in the study. Although the effectiveness evaluation was powered to detect improvement in child’s functioning, the heterogeneous nature of the study sample might have led to inadequate power to evaluate a statistically significant improvement in children with different developmental conditions and severity. Moreover, a relatively short duration of intervention (i.e., 6-months) might not have been adequate to improve clinical outcomes in children with complex and long-standing un-met developmental needs. The parent skills intervention resulted in improved quality of life of caregivers of children with developmental disorders (7). However, we were not able to establish whether improved parental health related quality of life translated in enhancing their competency to interact and engage with the child due to high refusal rate to video record mother–child interaction. Moreover, the data from implementation outcomes offered some important explanations about the difficulties that parents reported in using the skills at home and the barriers to participating in the full number of sessions, which might have created hindrances for caregivers to effectively learn the intervention strategies and implement these with their children at home. It highlights a need of providing more hands-on support to the caregivers to implement the intervention strategies in real-life settings, apart from training them in skills training program only.

The use of implementation science frameworks helped us to systematically plan and execute the implementation of parent skills training program in real-world settings and to unpack implementation processes at organization, provider and consumers levels which are often poorly understood and missing in a traditional randomized evaluation. While our study offers detailed insight into the feasibility and acceptability of using technology assisted task-shifting strategies to bridge treatment gap for developmental disorders in low resource settings, it has a number of limitations. Although, we used psychometrically strong implementation outcome measures, our study was not powered to evaluate the implementation effectiveness. The study was not designed and powered to evaluate the implementation outcomes (acceptability, feasibility and appropriateness) as potential mediators to change clinical outcomes (change in caregivers’ quality of life) in caregivers. Moreover, the data on the implementation outcome was collected at only one time-point. The data collection at multiple time-points can help to explore whether the fidelity, acceptability, feasibility and appropriateness of the program were sustained over time.

While technology enhanced training of volunteers has many positive features but it needs to be deployed in a culturally-congruent way including understanding the way in which the volunteers’ participation potentially challenges their traditional roles in families and communities to improve program implementation and increase its reach. Moreover, context specific evaluation of the program using implementation outcome measures and frameworks, embedded within the traditional RCTs could be beneficial to identify the priority challenges that need urgent mitigation before further scale-up of the program in real-world settings. Such trials are warranted to a) compile and consolidate implementation data using validated implementation outcome measures that can be used or adapted for a variety of settings and implemented with a range of stakeholders; b ) evaluate the adoption and integration of evidence-base intervention strategies in the healthcare system; c) evaluate the impact of improved implementation on clinical outcomes and d) measure the sustainability of such community based initiatives over the period of time.

WHO	World Health Organization
mhGAP-IG	Mental Health Gap Action Program- Intervention Guide
RE-AIM	Reach, Evaluation, Adoption, Implementation and Maintenance
UC	Union Council
ENACT	ENhancing Assessment of Common Therapeutic factors (ENACT)
BHU	Basic Health Unit
TQS	Ten Questions Screen
cRCT	Cluster Randomized Controlled Trial
LHW	Lady Health Worker
FVs	Family Volunteers
AMHR D&I	Applied Mental Health Research Dissemination and Implementation
AMD	Adjusted Mean Difference
CI	Confidence Interval

Ethics approval and consent to participate:

The ethical approval for the present study was obtained by Research and Ethical Committee, Rawalpindi Medical College and Allied Hospitals, Rawalpindi and Human Development Research Foundation, Islamabad Pakistan. Approval obtained on 29 Dec, 2016 and 29 Jan, 2017, respectively.

Consent for publication:

The consent for publication was obtained from all study participants at the time of enrolment.

Availability of data and materials:

The dataset generated and analyzed during the current study are available from the corresponding author on reasonable request.

Competing interests:

The authors declare that they have no competing interests.

Funding:

The study was financially supported by Grand Challenges, Canada and the Government of Canada (Grant number 0746-05-GMH). Matched funding was provided by Autism Speaks (Grant number 10377), USA and Human Development Research Foundation., Islamabad, Pakistan. The study proposal went through a peer review process before award funding. A funding by Fogarty International Center, Center for Global Health Studies was provided to write the present implementation case study as a part of series on Implementation Science Global Health Case Study. The study sponsor and funders have no role in the study design; collection, management, analysis, and interpretation of data; writing of the report; and the decision to submit the report for publication.

Authors' contributions:

SUH and ZeH had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. SUH conceived the study. SUH and ZeH were involved in acquisition, analysis, or interpretation of data. SUH, and ZeH drafted the 1st draft of the manuscript. LW was involved in drafting and commenting on the manuscript. All authors read and approved the final manuscript.

Acknowledgements:

We thank the project staff at the Human Development Research Foundation, Islamabad, Pakistan, namely Mr. Muhammad Aslam, Ms. Hadia Maryam, Ms. Sidra Mumtaz, Ms. Gulshan Huma, Ms. Anila Qureshi, Ms. Maria Kanwal, Ms. Alina Rasheed, Ms. Tahira Jabeen, and Ms. Nawal Nasir for their contributions in data collection and research activities. Finally, we thank the participants and their families for their voluntary participation in the program.

Elsabbagh M, Divan G, Koh Y-J, Kim YS, Kauchali S, Marcín C, et al. Global prevalence of autism and other pervasive developmental disorders. Autism Res. 2012;5(3):160-79.
Patel V, Kieling C, Maulik PK, Divan G. Improving access to care for children with mental disorders: a global perspective. Arch Dis Child. 2013;98(5):323-7.
Collins PY, Pringle B, Alexander C, Darmstadt GL, Heymann J, Huebner G, et al. Global services and support for children with developmental delays and disabilities: Bridging research and policy gaps. PLoS Med. 2017;14(9):e1002393-e.
World Health Organization. mhGAP intervention guide for mental, neurological and substance use disorders in non-specialized health settings: mental health Gap Action Programme (‎‎‎‎‎‎‎‎‎ mhGAP)‎‎‎‎‎‎‎‎‎: World Health Organization; 2016.
Reichow B, Servili C, Yasamy MT, Barbui C, Saxena S. Non-specialist psychosocial interventions for children and adolescents with intellectual disability or lower-functioning autism spectrum disorders: a systematic review. PLoS medicine. 2013;10(12):e1001572; discussion e.
Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implementation Science. 2013;8(1):117.
Hamdani SU, Huma ZE, Suleman N, Akhtar P, Nazir H, Masood A, et al. Effectiveness of a technology-assisted, family volunteers delivered, brief, multicomponent parents' skills training intervention for children with developmental disorders in rural Pakistan: a cluster randomized controlled trial. International journal of mental health systems. 2021;15(1):53.
Mirza I, Mehmood T, Tareen A, Davidson L, Rahman A. Feasibility Study on The Use Of The Ten Questions Screen By Lady Health Workers To Detect Developmental Disabilities In Pakistan. JPPS. 2008;5(2):97-100.
Hamdani SU, Atif N, Tariq M, Minhas FA, Iqbal Z, Rahman A. Family networks to improve outcomes in children with intellectual and developmental disorders: a qualitative study. International journal of mental health systems. 2014;8(1):7.
Hamdani S, Minhas FA, Iqbal Z, Rahman A. Model for service delivery for developmental disorders in low-income countries. Pediatrics. 2015;136(6):1166-72.
Hamdani SU, Akhtar P, Zill EH, Nazir H, Minhas FA, Sikander S, et al. WHO Parents Skills Training (PST) programme for children with developmental disorders and delays delivered by Family Volunteers in rural Pakistan: study protocol for effectiveness implementation hybrid cluster randomized controlled trial. Global mental health (Cambridge, England). 2017;4:e11.
Haroz E, Bolton P, Nguyen A, Lee C, Bogdanov S, Bass J, et al. Measuring implementation in global mental health: validation of a pragmatic implementation science measure in eastern Ukraine using an experimental vignette design. BMC Health Services Research. 2019;19(1):262.
Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Administration Policy in Mental Health Mental Health Services Research. 2011;38(2):65-76.
WHO. mhGAP training manuals for the mhGAP intervention guide for mental, neurological and substance use disorders in non-specialized health settings. World Health Organization; 2017.
Murray LK, Dorsey S, Bolton P, Jordans MJ, Rahman A, Bass J, et al. Building capacity in mental health interventions in low resource countries: an apprenticeship model for training local providers. International journal of mental health systems. 2011;5(1):30.
Kohrt BA, Jordans MJ, Rai S, Shrestha P, Luitel NP, Ramaiya MK, et al. Therapist competence in global mental health: Development of the ENhancing Assessment of Common Therapeutic factors (ENACT) rating scale. Behaviour research and therapy. 2015;69:11-21.
Kohrt BA, Mutamba BB, Luitel NP, Gwaikolo W, Onyango Mangen P, Nakku J, et al. How competent are non-specialists trained to integrate mental health services in primary care? Global health perspectives from Uganda, Liberia, and Nepal. Int Rev Psychiatry. 2018;30(6):182-98.
Rahman A, Iqbal Z, Waheed W, Hussain N. Translation and cultural adaptation of health questionnaires. Journal-Pakistan Medical Association. 2003;53(4):142-6.
Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Medical Research Methodology. 2013;13(1):117.
Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, et al. Standards for Reporting Implementation Studies (StaRI) Statement. BMJ (Clinical research ed). 2017;356:i6795.
Bass JK, Hamdani SU. 4 - Scaling up and implementing psychotherapies in low-resource settings. In: Stein DJ, Bass JK, Hofmann SG, editors. Global Mental Health and Psychotherapy: Academic Press; 2019. p. 67-84.
Rahman A, Hamdani SU. Supporting intervention providers and families in South Asia. In: Siller M ML, editor. Handbook of parent-implemented interventions for very young children with autism: Springer International Publishing; 2018. p. 393-402.
Rahman A, Akhtar P, Hamdani SU, Atif N, Nazir H, Uddin I, et al. Using technology to scale-up training and supervision of community health workers in the psychosocial management of perinatal depression: a non-inferiority, randomized controlled trial. 2019;6.
Chandra PS, Sowmya HR, Mehrotra S, Duggal M. 'SMS' for mental health—feasibility and acceptability of using text messages for mental health promotion among young women from urban low income settings in India. Asian journal of psychiatry. 2014;11:59-64.
Carrasco ÁE. Acceptability of an adventure video game in the treatment of female adolescents with symptoms of depression. Research in Psychotherapy: Psychopathology, Process outcome. 2016;19(1).
Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM. An introduction to implementation science for the non-specialist. BMC Psychol. 2015;3(1):32-.
Cook DA, Beckman TJ. Does scale length matter? A comparison of nine- versus five-point rating scales for the mini-CEX. Advances in health sciences education : theory and practice. 2009;14(5):655-64.
Lewis CC, Fischer S, Weiner BJ, Stanick C, Kim M, Martinez RG. Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. Implementation Science. 2015;10(1):155.
Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, et al. Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes (PRO) Measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research. 2005;8(2):94-104.

Table 1: Implementation constructs and data collection time-points

Implementation outcome/construct	Organization	Providers	Consumers	During program Implementation	After 6-months program Implementation
Fidelity	-	√	√	Quantitatively	-
Feasibility	√	√	√	-	Quantitatively/ Qualitatively
Acceptability	√	√	√	-	Quantitatively/ Qualitatively
Appropriateness	√	√	√	-	Quantitatively/ Qualitatively
Reach	√	√	√	Quantitatively/Qualitatively	Quantitatively

Notes: Organization=program trainers; Providers=family volunteers and consumers=caregivers of children with developmental disorders

Table 2: Sample matrix for in-depth qualitative interviews (N=30)

Categories of participants	N=30
Providers who completed the parent skills training program and delivered the intervention to caregivers	5
Providers who completed the parent skills training program but did not deliver the intervention to caregivers	5
Providers who did not complete the training in parent skills training program	5
Consumers who completed the training in parent skills training intervention	5
Consumers who did not complete the parent skills training intervention	5
Organizers (trainers) of parent skills training program who trained providers in the parents skills training intervention	5

Table 3: Demographic details of participants who provided quantitative data

Variables	Consumers (n=166)	Providers (n=36)	Organization (n=14)
Age (mean, SD), years	35(7.6)	35(4.38)	25.5(±1.8)
Education
Uneducated	44 (26.5%)	-	-
Primary- grade 5	39 (23.5%)	6 (17%)	-
Middle- grade 8	17 (10.2%)	4 (11%)	-
Matriculate- grade 10	35 (21.1%)	22(61%)	-
College and University – grade 11-16	31 (18.6)	4 (11%)	14 (100%)
Number of sessions attended (mean, SD)	-	9 (.00)	-
Number of sessions delivered (mean, SD)	-	7.60 (2.31)	-

Table 4: Descriptive statistics of AMHR D & I* measures (at consumers, providers and organizers’ levels)

Scale	No. of Items	M (SD)	Observed range	Possible range on instrument	α
Consumer’s level (n=166)
Acceptability	15	39.49(5.92)	15-45	0-45	0.91
Feasibility	13	31.29(4.76)	13-38	0-39	0.76
Appropriateness	10	24.62(4.17)	9-30	0-30	0.82
Reach	4	6.99(1.98)	3-12	0-12	0.37
Provider’s level (n=36)
Acceptability	9	34.2 (3.01)	18-34	9-36	0.89
Feasibility	15	49.7 (5.73)	26-45	15-60	0.95
Appropriateness	9	31.8 (4.62)	14-33	9-36	0.85
Reach	4	10.6 (2.01)	4-11	4-16	0.75
Organizer’s level (n=14)
Acceptability	5	17.79(2.45)	12-20	5-20	0.87
Feasibility	12	42.29(5.44)	30-48	12-48	0.90
Appropriateness	10	36.43(4.16)	26-40	10-40	0.88
Reach	4	11.07(2.84)	6-15	4-16	0.77

*Abbreviations: AMHR D & I= Applied Mental Health Research Dissemination and Implementation

Table 5: Findings of qualitative interviews

Objectives	Themes	Quote (source)
Acceptability to consumers	Increase in knowledge	“We learnt how to deal with children and (particularly) their behavioral issues. We also came to know about why do they (children with developmental disorders) (mis)behave; it was very nice to learn about this program because we did not know about these simple but helpful strategies that we know now.” (consumer)
Acceptability to consumers	Trustable relationship with provider	“We find it easy to talk to her (provider) and discuss our challenges (of implementing program strategies) with her during the sessions. She patiently listens to our other (family) problems that are not related to the child (and program) and shows interest; we feel good after sharing our problems with her (consumer).”
Acceptability to providers	Acceptance from family and community	“My family has supported me when I shared with them that trainers will come to my place (to do the trainings of other family volunteers). My husband fully supported me (and generously offered) that the mothers of other children who have problem can come to our house for participating in the program.” (Family volunteer, who completed the training and delivered the program)
	Facilitated learning through the use of training videos	“Use of tablet and videos is a good way when mothers cannot understand explanation of the trainer, they can understand it in a better way through training videos.” (Family volunteer, who completed the training and delivered the program)
	Group sessions- created a sense of shared experience	“We learned from experiences of each other and realized that we were always feeling alone (in this problem) but there are others who have similar problem with their children. There were many mothers in our group with different problems of their children and we came to know about their problems also.” (Family volunteer, who completed the training but did not deliver the program)
Feasibility to consumers	Household responsibilities as a barrier	“We all have household and community responsibilities such as working in the fields, taking care of animals and fetching water; if one has a child with special needs which demands special attention then it sometimes become too much for one to handle.” (Consumer)
Feasibility to provider	Timing and duration of group sessions Cooperation from caregivers	“Most mothers were very co-operative. They came to my place on-time and attended the training sessions attentively. If someone had to go back to home for some urgent work, we would catch-up at a later time. It was also a good learning experience for all the mothers in my village.” (Family Volunteer)
Feasibility to provider		“It was easy for people who were educated (to understand the content of program) but it was difficult for the people who were illiterate to understand (the content of program). After completing the training session using videos and explanation from the trainer, the content was easily understood by the illiterate mothers as well.” (Family volunteer, who completed training and delivered the program)
Appropriateness to consumers	Cultural relevance of intervention content	“In our village we do the same. We have Charpoy (Cot) where we all sit together, get sun bath, some children play with toy cars and others ride the cycles around. So, whatever shown in the video was similar to surroundings of our village. It was very much similar… It was not about city environment. It was mostly about village and our children (consumer)
Appropriateness to provider	Relevance of intervention strategies with problems of children	“The problems that were showed through video by tablet were very much similar to the problems of children in our village and most of the people could relate to the problems. It includes speech problems, like some children also have difficulty in moving around, difficulty in changing position at bed.” (Family volunteer, who completed the training and delivered the program).”

Download PDF

Reviews received at journal
23 Feb, 2022
Reviewer #1 agreed at journal
22 Feb, 2022
Reviewers invited by journal
22 Feb, 2022
Editor assigned by journal
09 Feb, 2022
Submission checks completed at journal
08 Feb, 2022
Editor invited by journal
08 Feb, 2022
First submitted to journal
07 Feb, 2022

You are reading this latest preprint version

Technology Assisted Task-sharing to Bridge the Treatment Gap for Childhood Developmental Disorders in Rural Pakistan: an Implementation Science Case Study

Status:

Version 1

Abstract

Figures

Contributions To The Literature

Background

Methods

Results

Discussion

Conclusions

List Of Abbreviation

Declarations

References

Tables

Supplementary Files

Status:

Version 1