Study design and study population
This study was designed to assess the impact of incentives for education attendance/enrolment and remaining STI negative on HIV incidence among HIV negative adolescent girls and young women (AGYW) aged 15-22 at baseline. AGYW were eligible for education enrolment at the primary, secondary, or tertiary, provided informed consent (or had a legal guardian to do so), consented to HIV testing, consented to a urine collection for trichomonas confirmation testing and consented to blood work and regular vaginal swabs for STI testing. The study included AGYW who dropped out of education but were still eligible to return, in order to assess whether incentives could help overcome other factors driving education dropouts. Because of the high HIV-1 incidence amongst AGYW, this evaluation was designed to assess the impact and cost-effectiveness of two different incentives – (a) conditional cash incentives that were conditional on education attendance among AGYW aged 15-22, and (b) raffle incentives for which raffle enrolment was conditional on being negative for curable STIs (syphilis and Trichomonas vaginalis). The primary outcome was HIV-1 incidence at 3 years after study enrolment. Secondary outcomes included Trichomonas vaginalis and syphilis prevalence at endline, and changes in reported sexual behaviours between baseline and endline. Additional outcomes included education enrolment and attendance and graduation rates, cost-effectiveness of cash incentives in terms of HIV prevention, cash incentive cost-effectiveness on educational outcomes, and the effects of a raffle incentive on enhancing a cash incentive intervention.
In order to investigate the impact of cash incentives on HIV incidence the study used a 2 X 2 factorial design to allow for a more efficient use of study resources by reducing the required sample size. At a cluster level, 50% of the participants in the treatment clusters will receive a cash incentive for education, and at the individual level within each treatment arm, 50% of the participants will be enrolled in a raffle conditional on testing negative for curable STIs. The aim was to enrol, from November 2015 to April 2016, 4,300 girls in 166 randomly selected census enumeration areas (EAs). Three-stage sampling and randomization took place: first, EAs were filtered, randomly selected and randomly allocated to either education treatment or education control arms (50% in the treatment arm and 50% in the control arm); second, households with potential participants were randomly identified and all eligible participants within those households were invited to enrol in the study; and third, all enrolled participants were randomly allocated to either the raffle treatment or raffle control arms. For the EA-level sampling, we first selected, from the universe of all EAs in Eswatini, a subset of EAs that met specific conditions: EAs with both a population density of at least 100 and with a female population of at least 200 (based on results from the 2007) census. Furthermore, EAs that were within or adjacent to the EAs where the Orphans and Vulnerable Children (OVC) impact evaluation (a study evaluating unconditional cash transfers for orphans and vulnerable children) was taking place, were excluded. The second stage of EA sampling involved stratified random sampling of 400 EAs (20% urban EAs and 80% rural EAs). Then 166 EAs were randomly sampled from the 400 EAs using a random number generator and allocated to education treatment or education control arms. Because EAs were oversampled (i.e. more than the 166 EAs needed) treatment and control EAs that were adjacent to each other could be and were replaced (through random selection) with another of the 400 already-randomly selected EAs (an urban EA would replace another urban EA, and so on). This resulted in treatment and control EAs that were neither adjacent to the OVC study EAs, nor immediately adjacent to each other.
For the participant-level random selection, the household listing from the 2007 census was used. Where the household listing from the census was not available, a new household listing was done. In each selected EA, ‘seed’ households were randomly selected using geospatial sampling. This sampling process involved overlaying the selected EA map over Google Earth, and randomly dropping a location pin in that EA using geospatial analysis software. The household closest to the pins was sampled first, and then every 4th household after that. The aim was to keep sampling households (and potential participants) until, within each EA, there would be 13 AGYW who were attending education at the time of enrolment into the study and 13 AGYW who were not in education. Achieving this target of 26 participants per EA, was difficult. Sampling amongst the initial set of 166 EAs did not yield the targeted 4,300 AGYW. The two main reasons for having to use additional EAs were 1) difficulty of enrolling out of education AGYW, and 2) difficulty recruiting urban AGYW. Therefore, additional EAs were randomly selected from the 400 EAs initially sampled during the first stage of EA sampling. An additional 127 EAs were added to ensure there was a sufficient number of urban and out of education AGYW enrolled in the study, using the same conditions as for the first 166 EAs and ensuring that the 80/20 rural/urban split was maintained and that treatment and control EAs would not be adjacent to each other. Once participants’ HIV status was determined and they opted to enrol into the study, they were randomly allocated to a raffle treatment arm or raffle control arm, creating 4 sub-arms: education only sub-arm, education-and-raffle sub-arm, raffle only sub-arm and no intervention sub arm (see Figure 2).
The education cash incentive intervention
To better understand what interventions might be most useful in Eswatini, we reviewed relevant literature and discussed feasibility and policy relevance of different incentives with colleagues at National Emergency Response for HIV/AIDS (NERCHA). Based on our discussions we found there to be a larger effect of cash incentives on education outcomes with monitoring and enforcement of the conditionality than simply providing unconditional cash transfers.(16)
The initial design of the intervention was that cash incentives, conditional on either of the following criteria, would be provided throughout the intervention period:
- E200 (~ 13.40 USD) for enrolling in primary or secondary education in a given education year
- E400 (~ 26.80 USD) per education term for attending more than 80% of their classes each education term over the study implementation period
- For a randomly selected sub-sample of the participants: E1000 (~ 67.00 USD) for the winners of a raffle in which those who tested negative for two curable STIs (Trichomonas vaginalis and syphilis) were entered
These amounts were decided in consultation with the government of Eswatini, balancing the need to propose amounts that would provide strong enough incentives while maintaining a transfer size reasonable within the Eswatini context and that could potentially be scaled-up by the government.
After the first year of implementation, we received feedback from the Ministry of Education’s regional guidance officers that there were several AGYW who attended upgrading classes and not traditional education. Older AGYW within our study population also attended upgrading classes, or short courses, or university or other vocational training and they were being excluded from receiving a cash incentive for attending some form of education. Based on this information we determined that from year 2 of implementation onwards, attending upgrading classes, short courses, university or vocational training should all be considered as being eligible for cash incentives. In addition, the midline data collection effort revealed that one of the main factors of AGYW not attending education or enrolling in education were lack of registration fees. Therefore, we also designed an added incentive to pay the education registration fees of all AGYW in the education treatment arm who were out of education as of midline data collection. Consequently, after the first year of study implementation (2016), a protocol amendment was approved whereby it was clarified how incentives for other forms of education than formal education would be paid to participants in the education treatment arm, as follows:
- Education incentive for initiating and completing upgrading classes:
- Enrol for upgrading classes in Eswatini; then receive E700 (~ 46.81 USD)
- Apply for O level exams; then receive E700 (~ 46.81 USD)
- Education incentive for initiating and sitting for exams at University, vocational school or technical college:
- Register at University or College within Eswatini for 2016 and/or 2017; then receive cash incentive of E700 (~ 46.81 USD) per year
- Sit for the annual exam at the end of the year; then receive cash incentive of E700 (~ 46.81 USD) per year
- Education incentive for initiating and completing a short course of any kind:
- Initiate attendance at short course during 2016 and/or 2017 through proof of payment; then receive a cash incentive of E700 (~ 46.81 USD) per course
- Complete the short course; then receive a cash incentive of E700 (~ 46.81 USD) per course
- Education incentive for participants returning to education in 2018 who were out-of-education in 2017
- Enrol to return to a public education or for an upgrading class or register for a public University or College or to attend a short course; then apply for your 2018 education fees, to a limit of E2,900 (~ 193.94 USD) for the year, to be paid directly to the education, college or university where registered.
Payments were made using MTN Mobile Money for participants 18 years or older (being 18 years of age is a requirement in Eswatini for registering for MTN Mobile Money, a cell phone-based money transfer service). Those younger than 18 years could elect to either receive their incentives through the Swaziland Post and Telecommunications office, or for it to be given to the parent/guardian with the expectation that the payment would be shared with the participant. In all cases, actual payments to participants were verified telephonically (to verify that the participant received the funds).
The STI raffle intervention
Out of the study participants who were randomized to either the ‘education and raffle incentive’ sub-arm or the ‘raffle incentive’ sub-arm (Figure 2), 400 were selected every raffle round to participate in STI testing (syphilis and Trichomonas vaginalis). Participants were not excluded or reallocated if more than one individual per household was chosen. The possibility to select more than one individual per household was well understood and did not create problems during field work. OSOM Trichomonas vaginalis rapid test and Alere Determine syphilis rapid test were used to determine STI status. Positive results were provided through post-test counselling by a trained nurse. Upon being STI negative, these randomly selected participants were then eligible to win one of 80 prizes of E1000 each. For anyone with a positive rapid test, the counsellor obtained a detailed history to determine the duration of infection, including a history of previous syphilis, known contact with someone with primary or secondary syphilis, and typical signs or symptoms of syphilis in the past 12 months. Participants who tested positive for either syphilis or Trichomonas vaginalis were given treatment and retested after two weeks. If the AGYW tests negative after 2 weeks of treatment, she was eligible for selection for the next raffle round.
HIV and STI counselling, testing, treatment and referral
HIV testing consisted of the Alere Determine HIV-1/2 rapid test kit and those who were HIV positive by the Determine HIV-1/2 rapid test and/or test inconclusive received the Uni-Gold rapid test for confirmation. HIV tests were administered at study testing sites by nurses and healthcare workers certified in HIV testing and counselling. Any individual testing positive for HIV was referred to a healthcare facility for treatment. All study participants were counselled before testing by a HIV Services Testing Counsellor. Participants were tested at baseline, midline and endline each test was approximately 1 year apart. At each interaction with the study team (whether for intervention implementation or data collection), study participants were screened for gender-based violence (GBV) using a screening tool developed in partnership with the Eswatini Action Group Against Abuse (SWAGAA). Throughout the study, suspected GBV cases were referred to SWAGAA for follow-up and support as part of their routine counselling and support mechanisms.
Data collection plan and analyses
Data were generated from questionnaire data, education enrolment and attendance data, raffle results and laboratory results. In terms of data collection: (a) Data on education attendance were collected from routine sources using the Ministry of Education, education attendance registers, (b) In addition, spot checks were done to confirm education attendance, (c) Costing data relating to the implementation of each incentive were collected throughout the study to allow cost-effectiveness estimation as a function of the estimated HIV infections averted, and (d) Data collection for sexual behaviour was collected at baseline, midline and endline questionnaires by field data collectors. Data captured during the evaluation were uploaded using personal digital assistants (PDAs) in the field, to ensure consistent data capture across districts. Questionnaire data, education enrolment and attendance data, raffle results and laboratory results were entered into SurveyToGo and processed with a bespoke data management system developed in SQL. After validation of duplicate files, the data were then exported to a statistical package (STATA) for further cleaning and analysis.
Analysis Included in this Report
Baseline data was collected at the time of participant enrolment in 2016. The objective of the baseline data collection and analysis was twofold: describe the sociodemographic characteristics of the study population and identify variables affecting school enrolment and HIV and STI status because they could act as confounding characteristics which would need to be controlled for in the impact evaluation. The purpose of the baseline data analysis was to ascertain the following information about the study population: describe demographic characteristics; report study level education enrolment, and investigate characteristics and factors associated with education enrolment at baseline; report study level HIV and STI prevalence, and investigate relationships between background characteristics of participants, and HIV and STI status at baseline. Table 1 provides key baseline socio-demographic and descriptive evidence about the sample composition.