Design of the study
This study employed a quasi-experimental study design [18–20] that assessed the proportion of children aged five years or less appropriately treated for pneumonia symptoms, uncomplicated malaria and non-bloody diarrhoea. The pre-intervention period was between May and October 2016 while peer supervision was carried out from November 2016 to May 2017.
Setting of the study
Intervention district with peer supervision
Investigators of this study used peer-supervision to augment the current self-supervision being practiced by drug sellers. Peer-supervision was aimed at supporting inspection of drug sellers by DDIs since evidence shows that inspection coupled with supervision improves quality of care[21]. Peer-supervision is a type of supervision where supervisees encourage and enhance learning and development as peers. Peers are people of similar hierarchical status or who perceive themselves as equal[22]. Since peer supervision had been successful elsewhere [23–25], it was envisaged that learning and development facilitated by peers would improve drug seller treatment of febrile illnesses in children under five years of age.
The intervention was carried out in Luuka district which has a total population of 238,020 persons [26]. Presently, the district has no hospital, has one health centre level IV, six health centres level III and 16 health centres level II.
Peer-supervisors in the intervention district were chosen according to proposed criteria. The criteria involved peer supervisors being democratically chosen by drug sellers from that particular sub-county by show of hands. Having higher academic qualifications compared to other drug sellers was the second criteria. In the event that the peer supervisors chosen had similar qualifications, the person with the highest number of votes became the peer supervisor for that sub-county. Each sub-county in the intervention district had a peer supervisor. The peer supervisors underwent refresher iCCM training in October 2016. The refresher training was carried out by the investigators of the study and was based on iCCM guidelines by WHO/UNICEF adopted by the Ministry of Health Uganda[27]. The training lasted a total of three days for each supervisor in each sub-county. Peer supervisors were deemed fit for supervision if they could explain to the trainers what constituted appropriate treatment including recognizing danger signs in children less than five years with pneumonia symptoms, uncomplicated malaria and non-bloody diarrhoea. The training was supplemented by clearly defining roles for peer supervisors. These roles included; instructing and monitoring drug sellers on how to correctly fill sick child registers. In addition, peer supervisors were taught how to counsel drug sellers that were not giving appropriate treatment to children under five years of age. Peer supervisors were also mandated to cross check with drug sellers whether the respiratory timers and brand of RDTs being used were the recommended ones by the ministry of health.
More so, peer supervisors were taught how to be role models in their course of supervision by advising drug sellers to adhere to treatment guidelines. Peer supervisors were instructed to adhere to the highest form of privacy, professionalism, integrity and empathy. In all sub-counties of the intervention district, peer supervisors were tasked to work with an active district drug shop association where drug sellers met every month particularly, to attend continuous medical education organized by the drug shop association secretariat.
Peer supervisors were then provided with supervision checklists were they were asked to summarise treatment given to children by drug sellers on a monthly basis. The summarised information from the checklists was used by investigators to corroborate with information of drug shop sick child registers filled in by drug sellers every month. This was done to ensure accuracy of data filled in by drug sellers and data collected by peer supervisors. To ease the work of peer supervisors, the peer supervisors were also provided with summary extracts from iCCM treatment and referral algorithms [27].
In line with guidelines for data collectors of the school of public health, Makerere, Uganda, every peer supervisor was given a safari day allowance of 80,000 Uganda shillings (equivalent to USD 22 at an exchange rate of USD 1 equal to 3,700 Uganda shillings). A safari day allowance is paid when a data collector travels within Uganda for a period of six hours or more and returns the same day. It is paid to cater for lunch, transport and other incidentals. The assumption was that each peer supervisor would visit all drug sellers within the sub-county every month and that supervision visits would not exceed one day.
The aim of peer supervision was to strengthen the existing health system by supporting the district local government. During peer supervision, the peer supervisors worked directly under the office of the DDI who carried on his inspection role as usual.
Comparison district
Buyende district has a population of 323,067 persons[26]. The district has one health centre level IV, six health centres level III, eleven health centres level II and approximately 503 village health team members. In both districts, there was at least one drug shop in every village. The East-Central region where these two districts are located has a very high under-five mortality ranging between 73 to 90 per 1000 live births [28].
Characteristics of participants and description of materials
In both districts, there were 135 registered drugs shops (Luuka 60 and Buyende 75) operated by nursing assistants, enrolled and comprehensive nurses, midwives and clinical officers. By law, drug shops in Uganda are authorised to sell class-C drugs (over the counter) that do not require prescription, used for treating minor and self-limiting conditions and are relatively safe. However, with the introduction of the iCCM strategy in the private sector, drug sellers were allowed to prescribe and dispense drugs for malaria (artemisinin combination therapies), pneumonia (amoxicillin) and diarrhoea (a combination of zinc and ORS) for children. This study was conducted among registered drug shops.
Before introduction of peer-supervision, both districts received training on how to treat children less than five years presenting with symptoms of pneumonia, uncomplicated malaria and non-bloody diarrhoea based on standard treatment guidelines developed by UNICEF, MoH and WHO[27]. The training was conducted between May 2015 and May 2016 by the Clinton Health Access Initiative (CHAI). The period between May 2016 and October 2016 was the period before peer-supervision was introduced.
Data collection
Socio-demographic data was collected from drug sellers using a questionnaire. Data on number of government inspection visits per drug shop was collected on a monthly basis from both districts from drug sellers when information on appropriate febrile treatment was being collected. Only data on number of peer-supervision visits was collected from the intervention district after introduction of peer supervision. Other data collected from drug shops included: age, gender, and qualifications of drug seller. In addition, prescription and treatment data of the under-five children who attended the drug shops was extracted from sick child registers line by line as is, to ensure accurate data capture. This data included name of the child, age, gender, care giver name, duration of symptoms, danger signs, respiratory rate (breaths per minute), fast breathing, RDT results as well as any other symptoms. Names of the children and care givers were given unique identifiers and anonymised during data management, analysis and report writing. Data was collected from the intervention and comparison districts between June 2016 and May 2017.
Outcome variable
The outcome variable for the study was appropriate treatment-defined as a drug seller prescribing and dispensing age appropriate medication to children less than five years old with symptoms of pneumonia, uncomplicated malaria and non-bloody diarrhoea in the right dose, frequency and duration for the right indication as per the standard iCCM treatment guidelines [27] .
Data analysis
To assess the effectiveness of peer supervision among drug sellers on appropriate treatment of children less than five years of age with pneumonia symptoms, uncomplicated malaria and non-bloody diarrhoea, interrupted time series for multiple group analysis was conducted[29]. The ITSA method was the preferred choice for this this study because it was not possible to randomise peer supervision (the intervention)[30]. In addition we wanted to cater for policy shifts if they occurred during the study period-a major strength of using interrupted time series[31]. Consideration was made in using interrupted time series since the study had six data points before and six data points after introduction of peer supervision which are considered a minimum for reliable results[32, 33].
The mean with the corresponding standard deviation and median number of visits by DDIs and peer supervisors was computed. Then, the percentage of appropriately treated children by drug seller by month by district was calculated by dividing the total number of appropriately treated children for a given childhood illness by the total number of children presenting with symptoms of that particular illness multiplied by 100. Results were then aggregated at district level by month covering both the pre-intervention and intervention period and used to show the trend of appropriate treatment in both districts for both periods.
This was followed by fitting the multiple group ITSA model (Eq. 1). In the final step, the Cumby-Huizinga test was used to test for autocorrelation. The model was re-run by specifying the lag order accounting for autocorrelation. Data from the sick child registers was entered in Epi Data (www.epidata.dk), exported to excel for cleaning and coding. Data analysis was done using STATA version 15 (Stata Corp, College Station, TX, 2015).The ITSA model is as follows;
The names and definitions of all variables used in this model are listed in Table 1.
Table 1
Definitions of variable names used in the econometric analysis
Variable names | Definitions |
yt | Aggregated outcome variable (appropriately drug use in treatment of malaria, diarrhoea and pneumonia symptoms) |
βo | Intercept which represents the starting level of the outcome variables (appropriately drug use in treatment of pneumonia symptoms, malaria and diarrhoea) |
Tt | Equally spaced time points in months |
Xt | Indicator variable representing pre-peer supervision period-0, otherwise 1 |
Z | Indicator variable representing intervention and comparison groups (Luuka-1,Buyende − 0) |
XtTt,ZTt,ZXt and ZXtTt | Interaction terms |
β1 | Slope or trajectory of the outcome variable (appropriately drug use) until the introduction of peer supervision. |
β2 | Change in the level of appropriately drug use that occurs in the period immediately following the introduction of peer supervision. |
β3 | Difference between pre-peer supervision and post-peer supervision slopes of the Outcome |
β4 | Difference in the level between intervention and comparison prior to peer supervision |
β5 | Difference in the slope between intervention and comparison prior to peer supervision |
β6 | Difference in level between intervention and comparison in the period immediately following introduction of peer supervision |
β7 | Difference between intervention and comparison in the slope after initiation of peer supervision compared with before peer supervision being introduced. |
εt | Error term at time t representing unexplained random variability in the model |
The ITSA uses ordinary least squares (OLS) approaches to estimate the effects of the intervention (peer supervision) on appropriate treatment for multiple treatment (intervention) periods. The ITSA model accounts for auto correlation among equally spaced observation data by comparison for the lag order for which auto correlation is assumed to be present. The estimated coefficients from the model together with their standard errors are reported.
Descriptive and analytic results
The frequency of inspection visits carried out by government district drug inspectors in both the intervention and comparison districts of the study are presented in Table 2. Following the introduction of peer supervision, the mean (SD) monthly visits for district drug inspectors in the intervention district was 0.98(0.34) while in the comparison district was 0.78(0.31). The mean (SD) visits for peer supervisors in the intervention district was 1.2(0.87).
Table 2
Frequency of inspection and peer supervision visits in the study period
Number of visits | Intervention (Luuka) | Comparison(Buyende) |
Inspection visits before introduction of peer supervisors | | |
Median | 0.88 | 1.00 |
Mean | 0.88 | 0.94 |
std. deviation | 0.31 | 0.38 |
Inspection visits following introduction of peer supervisors | | |
Median | 1.00 | 0.86 |
Mean | 0.98 | 0.78 |
std. deviation | 0.34 | 0.31 |
Peer supervision visits | | |
Median | 1.00 | |
Mean | 1.2 | - |
std. deviation | 0.87 | - |
The aggregated monthly district percentages of appropriately treated children are presented in Table 3 and Fig. 2. There was a general increase in the overall percentage of children treated appropriately for pneumonia symptoms (using amoxicillin tablets) and non-bloody diarrhoea (using a combination of zinc and oral rehydration salts) in both districts months preceding introduction of peer supervision.
Table 3
Monthly appropriate treatment of childhood illnesses by district (%)
District | | Month | Appropriate diarrhoea treatment | Appropriate Malaria treatment | Appropriate pneumonia symptoms treatment |
Intervention | Before | 16-Jun | 57.3 | 45.8 | 11.2 |
16-Jul | 70.1 | 60.4 | 26 |
16-Aug | 63.3 | 53.2 | 27.9 |
16-Sep | 73.8 | 56.8 | 26 |
16-Oct | 72.7 | 62.3 | 36.6 |
After | 16-Nov | 75.1 | 66.3 | 36.9 |
16-Dec | 74.1 | 53.5 | 30.5 |
17-Jan | 68.1 | 56.5 | 37.8 |
17-Feb | 74 | 64.1 | 41.6 |
17-Mar | 64.6 | 63.8 | 42 |
17-Apr | 69.9 | 66 | 37.8 |
17-May | 70.1 | 66.2 | 41.9 |
Comparison | Before | 16-Jun | 48.3 | 51.8 | 22.9 |
16-Jul | 65.7 | 50.6 | 21.5 |
16-Aug | 44 | 47.1 | 17.9 |
16-Sep | 55.5 | 57 | 18.4 |
16-Oct | 63.7 | 50.8 | 22.6 |
After | 16-Nov | 63 | 49.5 | 24.8 |
16-Dec | 67.9 | 51.8 | 33.9 |
17-Jan | 65 | 62.6 | 19.7 |
17-Feb | 84.6 | 67.1 | 24 |
17-Mar | 63 | 47.1 | 32 |
17-Apr | 72.9 | 59.2 | 15.4 |
17-May | 76.1 | 55.6 | 9.1 |
Following the introduction of peer supervision, there were monthly improvements in the overall appropriate treatment of children with pneumonia symptoms in the intervention district while there was a decline in the comparison district.
There was no difference in appropriate treatment of children with uncomplicated malaria between the two districts. Noticeably, more children were appropriately treated for non-bloody diarrhoea than malaria and pneumonia symptoms in both districts over the study period.
Effectiveness of peer supervision on appropriate treatment of Pneumonia symptoms
As shown in regression Table 4 and visual inspection of Figs. 3, prior to introduction of peer supervision, 21.4% of the children were appropriately treated for pneumonia symptoms in the comparison district.
Table 4
Effect of peer supervision on appropriate treatment of childhood illnesses
Variable | Diarrhoea | Malaria | pneumonia symptoms |
| Coef | 95% CI | P-value | Coef | 95% CI | P-value | Coef | 95% CI | P-value |
Time (T) | 2.06 | -1.32, 5.44 | 0.099 | 0.44 | -1.23, 2.11 | 0.585 | -0.37 | -1.80, 1.06 | 0.592 |
Baseline slope (Z) | 9.22 | -1.50, 19.9 | 0.025 | -0.76 | -11.42, 9.89 | 0.882 | -6.02 | -12.37, 0.33 | 0.062 |
Interaction (Z*T) | 1.39 | -2.45, 5.23 | 0.313 | 2.5 | -0.88, 5.88 | 0.137 | 5.45 | 2.85, 8.04 | 0.001** |
Pre-intervention slope (X) | 3.67 | -9.85, 17.19 | 0.448 | 1.46 | -10.43, 13.36 | 0.798 | 10.84 | 1.75, 19.9 | 0.022* |
Interaction (X*T) | -0.37 | -3.74, 2.99 | 0.741 | 0.19 | -2.39, 2.77 | 0.879 | -2.19 | -4.56, 0.18 | 0.068 |
Intervention slope (Z*X) | -7.73 | -22.52, 7.05 | 0.155 | -7.07 | -24.35, 10.21 | 0.399 | -16.89 | -28.81, -4.96 | 0.008* |
Interaction (Z*X*T)a | -4.04 | -7.95, -0.13 | 0.007* | -1.98 | -6.60, 2.63 | 0.375 | -1.68 | -4.93, 1.57 | 0.291 |
Constantb | 51.3 | 41.85, 60.78 | 0.001 | 50.6 | 47.48, 53.67 | 0.001 | 21.4 | 18.33, 24.46 | 0.001 |
Post intervention trend (November 2016 to May 2017) |
Intervention | -0.96 | -1.95, 0.02 | 0.06 | 1.14 | -1.30, 3.58 | 0.336 | 1.21 | 0.36, 2.05 | 0.008 |
Comparison | 1.69 | 0.57, 2.80 | 0.005 | 0.63 | -1.34, 2.59 | 0.507 | -2.56 | -4.78, -0.34 | 0.026 |
a Difference between pre-intervention and post-intervention slope (effect of the intervention over time) |
b Represents the starting level or percentage of infants appropriatelyy treated in the comparison arm Baseline slope is the slope between June to October, 2016 while pre-intervention is the slope between November, 2016 and May 2017. |
This was followed by a 0.37% monthly decrease (P = 0.59, CI = [-1.80, 1.06]) until October 2016. A month after introduction of peer supervision (November 2016), the proportion of children appropriately treated for pneumonia symptoms was 10.84% significantly higher in the intervention (P < 0.05, CI = [1.75, 19.9]) compared to the comparison district.
This was followed by a decrease in appropriate treatment of children with pneumonia symptoms of 2.19% per month (P = 0.07, CI = [-4.56, 0.18]). The intervention decreased the proportion of appropriately treated children with pneumonia symptoms by 1.68% during the intervention period (p = 0.29, CI = [-4.93, 1.57]) compared to the comparison district. Post intervention trend results revealed that introduction of the intervention increased the proportion of appropriately treated children exhibiting pneumonia symptoms at a rate of 1.21% (p = 0.008, CI = [0.36, 2.05])
Effectiveness of peer supervision on appropriate treatment of uncomplicated Malaria
Again, from regression Table 4 and visual inspection of Figs. 4, prior to introduction of peer supervision, 50.6% of the children were appropriately treated for uncomplicated malaria in the comparison district. Then, a 0.44% monthly increase (P = 0.585, CI = [-1.23, 2.11]) occurred until October 2016.
A month after introduction of peer supervision (November 2016), the proportion of appropriately treated children for uncomplicated malaria was 1.46% higher in the intervention (P = 0.79, CI = [-10.43, 13.36]) compared to the comparison district. However, this difference was not statistically significant. This was followed by a small increase in appropriate uncomplicated malaria treatment of 0.19% per month (P = 0.88, CI = [-2.39, 2.77]). The intervention decreased the proportion of children appropriately treated for uncomplicated malaria by 1.98% during the intervention period (p = 0.38, CI = [-6.60, 2.63]) compared to the comparison district. In addition, post intervention trend results revealed that introduction of the intervention increased the proportion of appropriately treated children with uncomplicated malaria at a rate of 1.14% (p = 0.34, CI = [-1.30, 3.58])
Effectiveness of peer supervision on appropriate treatment of non-bloody diarrhoea
As seen in regression Table 4 and visual inspection of Figs. 5, prior to introduction of the intervention (peer supervision), the proportion of appropriately treated children with non-bloody diarrhoea in the comparison district was 51.3%. This was followed by a 2% monthly increase (P = 0.099, CI = [-1.32, 5.44]) until October 2016.
In the first month of the intervention (November 2016), the proportion of appropriately treated children with non-bloody diarrhoea was 4% higher in the intervention (P = 0.448, CI = [-9.85, 17.19]) compared to the comparison district. However, this difference was not statistically significant. Consequently, there was a slight decrease in appropriate non-bloody diarrhoea treatment of 0.37% per month (P = 0.74, CI = [-3.74, 2.99]). The intervention significantly decreased the proportion of children appropriately treated for non-bloody diarrhoea by 4% during the intervention period (p < 0.05, CI = [-7.95, -0.13]) compared to the comparison district. In addition, post intervention trend results revealed that introduction of the intervention decreased the proportion of appropriately treated children at a rate of 1% (p < 0.06, CI = [-1.95, 0.02]).