An Evaluation of a Peer Supervision Pilot Project among Community Health Workers in Rural Uganda


 BackgroundThe World Health Organization recommends use of community health workers (CHWs) as a strategy to address the growing shortage of health workers. High-quality, regular supervision can help CHWs reach their full potential. Living Goods operates a community health program in 19 districts of Uganda. In the standard supervision model for the program, CHWs are supervised by a full time Community Health Supervisor (CHS) who reviews performance, coaches and mentors the CHWs. Whereas this model has proven to be effective, it is very expensive. Evidence indicates that peer supervision can be a substitute for standard supervision. In this paper, we describe our experience and program outcomes while implementing a peer supervision model among 211 CHWs in Mayuge district between January and December 2019.Objectives1. To describe the peer supervision model used. 2. To compare health services delivery outcomes. 3. To compare costs of delivery of the two supervision models.MethodsInternal organization records from January to December 2019 were reviewed. Focus group discussions and in-depth interviews with 29 CHWs were also conducted. Qualitative analysis was conducted using thematic content analysis while quantitative data was summarized to generate averages, percentages and graphs.FindingsCHWs under the peer supervision model performed better than those under the standard supervision model against all key performance indicators (KPIs). The total cost to maintain the peer supervision model for 1 year was $176 per CHW vs $273 among CHWs under the standard supervision model. Peer supervision thus resulted in an overall saving of 36% of direct operations costs. There was lower attrition among CHWs under peer supervision vs non-peers (10% vs 17%). Strengths of peer supervision included: improved CHW teamwork and motivation, optimization of supervisor time, as well as reduced program costs.ConclusionsPeer supervision is a feasible and more affordable model of supervising CHWs.


Abstract Background
The World Health Organization recommends use of community health workers (CHWs) as a strategy to address the growing shortage of health workers. High-quality, regular supervision can help CHWs reach their full potential. Living Goods operates a community health program in 19 districts of Uganda. In the standard supervision model for the program, CHWs are supervised by a full time Community Health Supervisor (CHS) who reviews performance, coaches and mentors the CHWs. Whereas this model has proven to be effective, it is very expensive. Evidence indicates that peer supervision can be a substitute for standard supervision. In this paper, we describe our experience and program outcomes while implementing a peer supervision model among 211 CHWs in Mayuge district between January and December 2019.

Methods
Internal organization records from January to December 2019 were reviewed. Focus group discussions and in-depth interviews with 29 CHWs were also conducted. Qualitative analysis was conducted using thematic content analysis while quantitative data was summarized to generate averages, percentages and graphs.

Findings
CHWs under the peer supervision model performed better than those under the standard supervision model against all key performance indicators (KPIs). The total cost to maintain the peer supervision model for 1 year was $176 per CHW vs $273 among CHWs under the standard supervision model. Peer supervision thus resulted in an overall saving of 36% of direct operations costs. There was lower attrition among CHWs under peer supervision vs non-peers (10% vs 17%). Strengths of peer supervision included: improved CHW teamwork and motivation, optimization of supervisor time, as well as reduced program costs.

Conclusions
Peer supervision is a feasible and more affordable model of supervising CHWs.

Background
Page 3/17 The World Health Organization recommends the use of community health workers as a strategy to address the growing shortage of health workers, particularly in low-income countries (World Health Organization, 2018). The umbrella term "community health worker" (CHW) embraces a variety of community health aides selected and trained to render certain basic health services to the communities they come from (Lehmann and Sanders, 2007). Extensive research has shown CHW programs to be effective in delivering a range of preventive, promotive, and curative services related to reproductive, maternal, newborn, and child health ( formal CHW training is often only a few weeks. In addition, they usually practice alone, providing little room for reinforcement or support. In combination, these factors can result in poor quality work, burnout, absenteeism, and attrition. Therefore, investing in high-quality CHW supervision can help CHWs perform better. Evidence at the global level suggests that regular and systematic supervision, with clearly de ned objectives, can improve the motivation and performance of CHWs involved in primary health care Peer supervision is de ned as an approach in which selected CHWs take on supervisory roles through peer-to-peer learning, support and problem solving (Hu, 2014). In November 2018, Living Goods piloted a peer supervision model for its CHWs working within one district. In this paper, we describe our experience implementing the model, report program outcomes from November 2018 to November 2019, and provide qualitative analysis of the successes and challenges of the program.

Case Presentation
The peer supervision model was implemented in Mayuge District,located in the Eastern region of Uganda. It is bordered by Iganga District to the north, Bugiri District to the northeast, Namayingo District to the east, the Republic of Tanzania to the south, and Jinja District to the west. A large proportion of the district surface area is open water of Lake Victoria. Another 10% of the district is protected national forest reserve. The district has many Islands which are currently occupied by permanent and migratory shermen. The 2014 National Census estimated the population of Mayuge District to be 473,239. Living Goods started operations in Mayuge in 2009. Within the district, we have presence in 12 (86%) subcounties, with an estimated coverage of 53% (238) of the villages. There is a total of 441 CHWs, 211 (48%) of whom were randomly selected to participate in the peer supervision pilot.

Goal of the pilot
The main objective of the pilot was to assess the feasibility and effectiveness of using the peer supervision model among CHWs. The pilot speci cally sought to

Design
The model borrows best practices from other models implemented elsewhere, like the Lady Health Worker Program (Pakistan), Integrated Management of Childhood Illness (Benin), and Health Extension Workers Program (Ethiopia). CHWs in close geographic proximity grouped themselves together into 8 to 12 per group and chose a leader, called a peer supervisor (PS). A total of 20 peer supervisors were selected. Each PS led their own group. The PSs were centrally trained on mentorship, coaching, and android support at the beginning of the pilot. They attended subsequent monthly meetings to reinforce their knowledge, receive updates and also troubleshoot problems. PSs were expected to continue to conduct CHW activity at the same volume as before. Each PS received $5.4 as weekly facilitation to cater for transport and communication costs as they supervise their peers. The groups were provided with a monthly performance-based incentive capped at $2.7 per CHW, based on pre-set targets on key indicators. A CHS was attached to 5 to 10 groups of CHWs and was expected to visit each group weekly for 2 to 3 hours.
The role of the CHS was to support the group leader and the other CHWs to deliver impact, strengthen their capacity based on needs and deliver relevant stock. The design was based on the assumption that the PS is a CHW and remains a CHW within the group he/she is supervising. The responsibilities of the CHS and the PS are highlighted in Table 1. The groups would all stay under the Living Goods CHS who supervises and over sees the activities of the team, as expressed in Fig. 1 below. • Implement marketing and promotional efforts to support CHWs' sales goals.
• Lead monthly In-Service meeting of CHWs.
• Support the management of nancial operations.
• Support the management and maintenance of inventory.
• Support the management of relations between Living Goods and the implementing partner organizations.
• Work closely with the district

Evaluation Methodology
A review of program records was conducted, and the ndings triangulated with qualitative methods. A qualitative evaluation of the model was commissioned to speci cally document peer supervisors', CHWs' and CHSs' experiences with peer supervision, establish the challenges and how they were mitigated during the pilot as well as to document lessons leant and best practices from the pilot to inform programming.
Data Collection: Program data from January to December 2019 were reviewed, comparing CHW participants in the peer supervision pilot against those under standard supervision, within the same district. Twentynine CHWs were selected to participate in three focus group discussions (FGDs). A fourth FGD was conducted among 11 peer supervisors. Three In-Depth Interviews (IDI) were held with the three CHSs and branch manager who directly supported the intervention. The interviews were guided by interview guides tailored to the respondents. The guides also captured background characteristics of all respondents, including age, sex, and education level, number of households and their villages and the period spent with Living Goods as CHW. The interviews were facilitated by a moderator, who was supported by a note taker. Audio recordings were also made to allow for complete capture of the discussions. Informed consent was sought from all FGD and IDI participants.

Data analysis
Quantitative data was analyzed to generate sums, frequencies and percentages. Graphical presentation of the data was also made to compare CHWs under peer supervision with those who were not.
Qualitative data was transcribed and analyzed using the thematic content analysis approach. A list of themes based on the interview questions was rst created. All transcripts were read several times to ensure that their meaning and context were understood. Comparisons of responses across categories, themes and participants as well as locations to examine and triangulate the data to see if there are any differences in understanding of different responses to the evaluation questions was done.
The results of descriptive and analytic statistics are reported here.

Results
Achievement on health KPIs:  Participant experiences during the pilot The CHWs were asked to rate their satisfaction with the peer supervision that they had been receiving on a scale of 1 to 5 with 1 = Worst, 2 = Fair, 3 = Good, 4 = Very Good 5 = Excellent. Of the 29 CHWs who participated in the peer supervision pilot evaluation, 23 (79%) rated their experience as excellent while the remaining six (21%) rated it as very good. The main reasons for this high rating was that peer supervisors were more accessible to the peers, were able to solve daily CHW challenges -especially those related with the data collection, work ows and technology use. In addition, the peer supervisors spent more time with the CHWs visiting and providing services to the community members. Whereas the CHSs did not have adequate time for such engagements, the peer supervisors were always available for this.
"It is good and makes me happy that we meet once every week as a group. Because where you were not working well your fellows remind you. PS is very near all the time. He encourages us to work moving among us and we go to the community with her"-CHW "I would not have enough time with the CHS and some month I would not even see him"-CHW "I give her 5. The peer leader reaches out to me every week on phone and in person to ensure I have worked. This makes me work as I do not want to disappoint her and the whole group" -CHW The concept of team work and company while doing work was found to be very attractive, as it helped with working with non-receptive and di cult households. CHWs also appreciated the peer supervision model because it brought medicines and products much closer to them so they did not have to frequently visit the branch for re-stocking. The peer supervisors felt that they had been empowered as leaders.
Bene ts according to the Peer Leaders and CHWs • Increased the level of commitment to CHW work.
• Achieved a better understanding of CHW work.
• Increased e ciency through making work plans and following them.
• Enabled analytical thinking and problemsolving skills.
• Created a healthy competitive life style.
• How to handle different personalities.
• Become more responsible and highly regarded in society.
• Can be a useful method of inducting new CHWs.
Bene ts according to the CHSs • Limited movement as CHWs are met in group as opposed to one on one.
• It has empowered some CHWs, especially the peer leaders.
• CHWs have medicines and products close to them and are real timely delivered • Peer supervisors provided real time support to their peer CHWs.
• Less days in the eld.
• CHWs can be empowered to lead their fellow CHWs.
• New CHWs can be better mentored by their peer CHWs.
• Reduced work load with regard to support supervision.
The peer supervisor provided the following advantages: they reinforced CHW knowledge, solved immediate phone issues, supported CHWs to visit unreceptive households, they were accessible in real time, provided timely reminders, helped in syncing data onto the servers, and delivered medicines and products on time. On the other hand, the CHWs relied on the CHSs to provide technical support and to support them with the monthly community events. It was clear that both the CHS and the peer supervisor are important to the CHW, but they play different roles. As such, it was not a mere preference of one model to another, but rather a complementary package.
All CHWs agreed that their experience was much better during the time of peer supervision, but reported that they preferred monthly in-service meetings at the o ce because they were given a refreshment during the meeting. They suggested a refreshment be provided every time there is a meeting especially the weekly team meetings. The comparisons are summarized below (Table 4). • The CHWs worked individually amidst all challenges.
• Phone challenges would take long to be addressed.
• Most eld related challenges were addressed by the CHS only and so the response rate was slow.
• Interaction and communication between the CHWs was hard because there was no mobilization.
• Group eld visits were rarely done.
• CHWs with small capital would easily feel shy to restock.
• Data synchronisation was a challenge.
• Restocking was done at any time as the peer supervisor acted as a mini hub.
• Peer supervisors encouraged group eldwork which made work easier.
• The peer supervisors were equipped and trained to work on most phone problems.
• The peer supervisors gave immediate support because they are equipped.
• The peer supervisors helped to mobilize CHWs, which made communication easier.
• Group visits give a better impression during household visits and this raised Living Goods awareness.
• Synchronizing was made easier through peer supervisor tethering.

Cost comparison
The total cost to maintain the program per CHW for 1 year was signi cantly lower with $176 per CHW under peer supervision vs $273 among CHWs under the standard supervision model (P-value 0.034). The peer supervision model resulted in a 36% reduction in the supervision and in-service training costs.

Challenges faced
It was di cult to get some CHWs to participate in the group meetings/ activities as coordination of schedules di cult There were challenges with estimating stock for weekly visits and the felling of stock was hectic since the volume has increased and the means are still the same.
There were other competing activities such as CHW graduations that limited interaction of CHWs on their weekly schedules.
The CHSs also identi ed lack of training materials for the weekly CHW meetings as a key hinderance, since they do not have a speci c curriculum.
Untimely disbursement of group incentives led to decreased morale among some CHWs.

Discussion
Our ndings during one year of implementation have shown that peer supervision is a feasible and less costly model of supervising CHWs and thus may serve as an effective supervision model for others in Uganda and around the world. The most signi cant bene t of peer supervision from our study is that it improves ownership of the community health work by the CHWs, encourages team work, and also increases CHW con dence and commitment which ultimately result in improved performance and reduced attrition. Peer supervision also provides a unique opportunity for the peer and supervisor to talk about issues that emerge because of the peer's own life experiences in working in similar life situations.
Our ndings on peer supervision are consistent with those reported elsewhere on the impact of supportive supervision on worker performance. In their study, Rowe et al. (2010) and Rowe et al. (2009) found a 27% point difference in children receiving recommended care in intervention compared to control areas with routine supervision. Prior research has also established peer supervision to be a bene cial strategy as peers can empathize with each other outside of a hierarchical setting (Strachan et al., 2012), and more cost effective where traditional supervision is too costly (Kim et al., 2000). Just like in our study, peer mentoring is popular with both participants and managers (Robinson et al., 2001).
We learnt several lessons from implementing this pilot.
1. CHWs must own peer supervision and determine their leadership and their groups where they feel comfortable.
2. Refresher training should follow problem identi cation rather than relying on assumptions. The supervisor should endeavor to meet the group on a weekly basis to reinforce knowledge and discuss performance with the groups.
3. During the planning phase, one should ensure the geographical distance between CHWs in the group is manageable for the CHWs to meet on a weekly basis.
4. Remuneration/ reimbursement must be given to the CHWs in time, preferably on a monthly basis. Delays in reimbursement and remuneration can reduce CHW motivation.
5. The peer supervision approach is helpful in awakening dormant CHWs.
6. The group visits also increase individual CHW credibility in their own designated areas. The community is able to certify that the CHW is truly a community health worker and not self-imposed. 7. Peer supervision has improved relationships among CHWs, especially the ones in the same peer group, but also has improved the relationship between CHSs and CHWs, as they meet in smaller groups and regularly.

8.
A key challenge noted with this model is that it might create tensions between CHWs and their new peer supervisors, particularly if constructive feedback is not given.
9. One CHS should be allocated 6-8 groups instead of the original 5-10 groups. This is with the assumption that a CHS would spend 3 to 4 days a week in the eld and would meet 2 groups a day. The peer supervisor should maintain the ratio of 1 peer supervisor: 8-12 CHWs so as to ensure the peer supervisor can effectively supervise each of their CHWs at least twice a month.

Limitations
The pilot had a number of limitations, including: 1) The sample size of CHWs and their supervisors was very low, and therefore may have low statistical power; 2) Potential limited external validity due the study being conducted in only one study district. As such, the results from this pilot should be interpreted as suggestive evidence to the effectiveness of the intervention as we await results from a larger scale program.

Conclusion
Peer supervision is a feasible and less costly model of supervising CHWs and thus may serve as an effective supervision model for others in Uganda and around the world. The most signi cant bene t of peer supervision from our study is that it ensures ownership of CHW work by the CHWs, encourages team work and improves CHW con dence which ultimately result in improved performance and reduced attrition. Based on these results, the pilot has been extended to cover 1026 CHWs in Mayuge, Mukono, and parts of Wakiso district. An evaluation of implementation at scale in the 3 districts will be conducted in June 2020 to inform the decision to scale out peer supervision to all districts where Living Goods operates.

Declarations
Ethics approval and consent to participate: Since this was a review of program data, ethical approval was not sought. All data was obtained as part of routine program activities.
Consent for publication: The manuscript does not contain any individual person's data in any form. Comparison of CHW performance