Evaluation findings for context (demographics), implementation fidelity and participant response are presented. Implementation and response indicators were cross-tabulated with the trial arm and tested for associations using the χ2 test (Figure 3).
Respondents and context
We sampled 854 household respondents (389 MapSan trial participants, 465 secondary respondents) and 300 compound chefes, of whom 842 (99%) and 295 (98%) consented to an interview, respectively. We excluded intervention compounds that hadnot received the intervention and control compounds that had received the intervention. After exclusions, there were 1,049 responses from 517 compounds (270 intervention, 247 control) (Table 1).
Household respondents were predominantly women (80%) with median age 32 years; most chefes sampled were women (54%). Approximately one-third (36%) of respondents rented, with landlords not resident in the same compound for most of these (63%). Residents of larger compounds (>20 members) were less likely to have family connections within the compound (8% vs. 21%; Pearson’s χ2(1 df) =14, p<0.001).
Overall, the sanitation infrastructure was implemented as intended (Figure 3): 97% of intervention compounds (270/279) received intervention latrines, of which 219 (81%) were SLs and 51 (19%) were CSBs, and 92% were installed by the end of 2016, in line with implementation timelines. Although 23% of intervention compounds where a respondent was present at delivery reported user capital contributions causing delay to installation, there was no association between reported delays and year of implementation (χ2(5)=4.7, p=0.45). Thirty control compounds (11%) had received intervention latrines (excluded). After exclusions, eight intervention compounds (3%) also had a non-intervention latrine in use by members.
Similarly, 93% of intervention compounds (252/270) received the household-level behaviour change intervention component, involving a median of three visits (range 1–8), most occurring between 2016 and 2018 – outside the timeframe of the external evaluation . Respondents reported that trainings covered various topics (Figure 4). Cleaning/maintenance of latrines was the topic most widely discussed at visits (76% of compounds). Some control compounds (17; 7%) received hygiene promotion visits.
Intervention compounds in this survey did not meet all the stated original criteria for construction of intervention latrines at the time of our survey: only 56% (150/270) had at least 12 members, and 57% (29/51) of CSBs (intended for compounds with over 20 users) had over 20 users, higher than for SLs (6%; 13/219). However, compounds that received household-level behaviour change visits were broadly similar to those that did not (Text A3, Additional file 1)
Among compounds receiving behaviour change visits, there was fair agreement among compound members as to whether visits took place (ICC=0.45), but not all participated in training (ICC=0.28). Recall of topics discussed varied among compound members, with ICC values as low as 0.14 for discussion of latrine use (Figure 4). Overall, 57% (292/542) of intervention respondents recalled visits and 13% (73/542) participated, and recall of visits and participation in training differed across several demographic characteristics (Text A3, Additional file 1).
The intervention achieved improvements to most intended intermediary outcomes (Figure 3). Compared to control latrines, intervention latrines were much more likely to be private i.e. have a working door and inside lock (76% vs. 8% control; χ2(1)=500, p<0.001), almost twice as likely to be observably clean (86% vs. 45%; χ2(1)=150, p<0.001), and twice as likely to be well-maintained i.e. have a slab/floor in good condition (96% vs. 47%; χ2(1)=240, p<0.001). However, only 10% (78/777) of latrines had an HWF with both soap and water, regardless of trial arm, and few intervention latrines had signs of soap use at an HWF (17%, 67/399), with no difference from control (χ2(1)=0.12, p=0.729). While intervention latrines were more likely to have an outside lock (26% vs. 5%; χ2(1)=66, p<0.001), not all households had a key, resulting in fewer intervention respondents having access to their latrine than control respondents (94% vs. 99%; χ2(1)=12, p=0.001). Shared latrines were not public: only 1% (6/777) of latrines were regularly used by people outside the compound, roughly equal across trial arms.
Although improvements to intermediary outcomes were expected to result from formalised actions at the compound level, very few compounds formed a committee to manage sanitation, with slightly more in intervention compounds (16% vs. 9% control; χ2(1)=6.0, p=0.014). Compounds were unlikely to have and adhere to a formal rota for cleaning latrines (11% intervention vs. 5% control; χ2(1)=6.1, p=0.013), or to form a fund for sanitation maintenance and repairs – occurring in 4% (11/ 270) of intervention compounds and no control compounds.
Nonetheless, individual cleaning frequency (surveyed individual personally cleaned the latrine twice/week) was significantly higher among intervention respondents (58% vs. 47%; χ2(1)=14, p<0.001), however data suggest cleaning duties were not shared equally among respondents (ICC=0.18). Frequent collective cleaning (latrine being cleaned on a daily basis) was also reported more often by intervention respondents (78% vs. 65%; χ2(1)=19, p<0.001). Slightly more intervention respondents reported that money was spent on latrine repairs in the past year (19% vs. 13% control; χ2(1)=6.9, p=0.008).
Variation by intervention type
Participant response indicators differed by type of intervention latrine received. Compounds with a CSB were more likely to have a sanitation committee (27% vs. 13% SL; χ2(1)=6.8, p=0.009) and maintenance fund (12% vs. 3%; χ2(1)=9.5, p=0.002), to have and adhere to a cleaning rota (20% vs. 9%; χ2(1)=5.2, p=0.023), and to spend money on repairs (33% vs. 15%; χ2(1)=19, p<0.001). CSBs and SLs were similarly well-maintained, but CSBs were more often private (88% vs. 73% SL; χ2(1)=11, p=0.001) whereas SLs were more often clean (89% vs. 78% CSB; χ2(1)=7.8, p=0.005). More SL users reported individual cleaning (63% vs. 39% CSB users; χ2(1)=22, p<0.001), and collective cleaning (80% vs. 68%; χ2(1)=8.2, p=0.004). SL users were also more likely to have an HWF with soap and water (12% vs. 4% CSB; χ2(1)=4.2, p=0.042) and signs of soap use (20% vs. 4% CSB; χ2(1)=12, p<0.001), despite an HWF being built into the CSB infrastructure.
Grouped analyses by dichotomised contextual variables produced a different structural model when grouping by compound size (compounds of 20 members or fewer vs. more than 20 members) or by intervention type (CSBs vs. SLs). When stratifying by compound size, there was no further structural change when further grouping by intervention type, so we considered compound size to account for differences between CSBs and SLs. Other contextual variables such as having a chefe or relatives in the same compound were included as exogenous variables.
Intermediate models (without clustered robust standard errors) for both compound size groups had acceptable fit: for small compounds (≤20 members), root mean square error of approximation (RMSEA)=0.066 (90%CI: 0.049–0.083), comparative fit index (CFI)=0.91, Tucker-Lewis index (TLI)=0.89); for large compounds (>20 members), RMSEA=0.063 (90% CI: 0.055–0.070), CFI=0.93, TLI=0.91).The final path analysis models for the two groups are displayed in figures 5 and 6. The coefficient of determination was 0.839 for small compounds, and 0.897 for large compounds. Unstandardised path coefficients (b) represent the amount of expected change – in this case, the increase in absolute probability – in the outcome as a result of a unit change in the exposure. This is considered the direct effect of that exposure when controlling for other explanatory variables. The residual variance not explained by the model for endogenous variables is indicated. Covariances between exogenous variables were included in models, but not pictured (full models in Additional files 5–8). Estimated total effects (TE), subdivided into direct and indirect effects (IE) where there were multiple pathways, are provided in Table 2.
Decomposition of covariance between observed variables into ‘direct’ and ‘indirect’ effects suggested that the direct components accounted for the majority of intervention effects on intermediary outcomes due to several factors:
In small compounds, removal of indirect pathways that did not explain variation in outcomes meant large improvements to latrine maintenance (Figure 5; b=0.45, p<0.001) and privacy (b=0.69, p<0.001) were direct effects. The intervention had a small negative effect on accessibility (b=-0.053, p<0.001), and no significant effects on probability of having an HWF with soap and water or signs of soap use. Where indirect pathways are present, group-specific path coefficients indicate points where the theory breaks down. For example, both the intervention facilities (Figure 5; b=0.39, p<0.001) and collective cleaning frequency (b=0.19, p<0.001) were associated with latrine cleanliness. However, because the intervention had only a minor effect on collective cleaning (Table 2; TE=0.022, p=0.003), without use of a rota system, the indirect effect on latrine cleanliness was negligible (IE=0.004, p=0.020).
In large compounds, the reverse is true for latrine cleanliness. With a greater number of latrine users, collective cleaning frequency does not significantly contribute to latrine cleanliness (Figure 6; b=0.21, p=0.056), so despite an increase in collective cleaning frequency (Table 2; TE=0.077, p=0.027), mostly attributable to the compound maintaining a rota system to manage cleaning (TE=0.044, p=0.046), the effect on cleanliness was also primarily a direct effect of receiving an intervention latrine (b=0.40, p<0.001). Similarly, the intervention had a minimal indirect effect on latrine privacy via recent repairs (IE=0.009, p=0.161), despite the association between repairs and privacy (b=0.21, p=0.009). Counterintuitively, a negative effect of the intervention on probability of having an HWF with signs of soap use was observed (TE=-0.16, p=0.004), associated with household-level behaviour change activities.
Individual demographic factors influenced individual cleaning in both groups. In small compounds, female respondents (Figure 5; b=0.35, p<0.001) and those resident in the compound for under five years (b=0.068, p=0.049) were significantly more likely to clean the latrine; in large compounds, female respondents (Figure 6; b=0.35, p=0.001) and respondents above 30 years of age (b=0.17, p=0.027) were more likely to clean. However, compound factors played a significant role only in larger compounds. Presence of a relatives in the compound was negatively associated with repairs (b=-0.26, p<0.001), and presence of a chefe promoted availability of soap and water at the HWF (b=0.094, p=0.021).