These methods describe the second of two phases in the TESTsmART study; the first phase  aimed to determine the parameters of the incentives intended to be tested in this subsequent study phase. The aim of this second phase of the TESTsmART study is to evaluate the degree to which the incentives, directed at the provider or client, affect the purchasing behavior of all suspected malaria cases seeking treatment with respect to their willingness to undergo diagnostic testing and to purchase appropriate treatments.
The study will be conducted at two sites (Figure 1); a rural community in a malaria-endemic region of western Kenya where approximately 30% of fevers are due to malaria and malaria rapid testing is not currently available in the private sector, and Lagos, Nigeria, a large urban metropolis where malaria prevalence is around 3% and point-of-care rapid testing through mRDTs has been permitted in private sector retail outlets since 2015. The trials at each site will be analyzed separately.
In Kenya, the private sector, where 60% of fever cases seek care, is an important source for malaria treatments . Although private retail outlets and chemists are not routinely permitted to conduct mRDTs in Kenya, nationwide surveys show that nearly 71% of ACTs are distributed through the private sector . To ensure adequate regulatory oversight, only retail outlets registered with the Kenya Pharmacy and Poisons Board will be eligible to participate in this study.
In Nigeria, only those outlets registered and licensed by the Pharmaceutical Society of Nigeria are eligible to participate in this study. These outlets, known as Proprietary Patent Medicine Vendors (PPMVs), provide services to febrile patients and regularly stock and sell medicines approved for sale by the Nigerian National Agency for Food and Drug Administration and Control (NAFDAC).
The second phase of the TESTsmART study was planned as two four-arm 2x2 factorial cluster-randomized trials (Table 1). The unit of randomization (cluster) will be participating private-sector retail outlets at each site. Clients seeking care in participating outlets will be given the option of paying for a mRDT to be conducted at the outlet. The diagnosis and treatment choices made during each transaction between a treatment-seeking client and the outlet provider (Figure 2) will be captured using a mobile phone app that has been designed specifically for this purpose. The mobile app will also enable us to track and apply provider- and client-directed incentives (see Data Collection).
Figure 2: Decision tree of client diagnosis and treatment data collected through exit interviews and by the TESTsmART mobile app.
The four intervention arms are as follows:
- Control intervention: mRDTs are made available at wholesale price to the retail outlet, and outlet owner/attendant is trained to use the mobile reporting app. mRDTs are offered to clients at a pre-determined price.
- Provider-directed (PD) intervention: in addition to the interventions implemented in the control outlets, the retail outlet owner in this arm receives a small incentive to perform the mRDT (approximately USD $0.10 in Kenya and USD $0.25 in Nigeria for each mRDT they conduct and report using the mobile app).
- Client-directed (CD) intervention: in addition to the interventions implemented in the control outlets, clients visiting outlets in this arm receive a free ACT (cost equivalent to 150 Kenyan Shillings (KES) for adults and 60 KES for children in Kenya; 650 Naira for adults and 450 Naira for children in Nigeria) if they purchase an mRDT and receive a positive test result (conditional subsidy).
- Combined (PD+CD) intervention: in addition to the interventions implemented in the control outlets, retail outlet owners in this arm receive an incentive to test for malaria and clients visiting these outlets receive a free ACT conditional on a positive test result (i.e., this arm is a combination of the PD and CD interventions that are offered in arms 2 and 3 above).
In Nigeria, all four arms will be implemented as originally designed. In Kenya, the provider-directed intervention will be excluded due to sample size considerations that were uncovered, based on a pilot study, after the original study planning phase, thus leaving us with a three-arm randomized trial (see Sampling and Power Calculations below).
A description of each study outcome is shown in Table 2. All client-level outcomes are binary. The primary outcome measure is the proportion of all ACTs that are sold to malaria test-positive clients in each study arm. For our primary outcome, we are interested in evaluating the degree to which the interventions influence the purchasing behavior of all clients seeking treatment; thus, we include in the numerator those clients who were referred from a health facility with a documented positive malaria test result even if no mRDT was subsequently conducted at the retail outlet. For all secondary outcomes, however, we only include those clients who received their mRDT result from the study-enrolled retail outlet to which they presented, regardless of prior testing status. That is, for the secondary outcomes we are specifically interested in evaluating the degree to which the interventions influenced purchasing behavior among clients who present at the outlet for testing.
The major secondary outcome is the proportion of suspected malaria cases that are tested, where we define a suspected malaria case as any client who was tested with a mRDT at the outlet or who was untested but purchased any antimalarial at the outlet. This outcome will allow us to determine whether the conditional subsidy can drive demand for testing. Other secondary outcomes include adherence to mRDT results (the proportion of clients who properly adhered to their mRDT result out of all clients receiving an mRDT) and appropriate case management (the proportion of clients who properly followed their mRDT result, with respect to follow-on treatment options, out of all suspected malaria cases). We defined adherence to the mRDT result as purchasing an ACT if they tested positive and not purchasing any antimalarial (AM) if they tested negative, and we define a client as a suspected malaria cases if e they were tested with an mRDT or they were untested but purchased any AM. Finally, we will also calculate the proportion of clients taking ACTs without a diagnostic test.
Study outcomes will be collected through exit interviews with clients who sought care for a febrile illness at each of the enrolled retail outlets. All outcomes will be derived based on all data collected during the 15-month collection window.
Sampling and Power Calculations
The primary comparison of interest is the effect on our primary outcome of offering a combination of provider-directed and client-directed interventions relative to the control arm (i.e. PD+CD arm vs. control arm). In order to evaluate whether the provider-directed or client-directed interventions have a synergistic effect on the outcome, two secondary comparisons are of interest: (1) PD vs. PD+CD arms, and (2) CD vs. PD+CD arms, where only the latter comparison can be estimated in Kenya due to the three-arm trial design. We powered the study to analyze significant changes in the aforementioned comparisons (Table 3). We expect that the client-directed intervention (CD) will have a somewhat larger effect, and that the largest effect will come from combining the two interventions (PD+CD) (i.e., we assume a statistical interaction).
Sample sizes were calculated with original hypothesized effect sizes and re-evaluated with pilot data separately for Kenya and Nigeria, using the formula from Moulton and Hayes for comparing two proportions under a cluster-randomized trial design . We estimated the intra-class correlation coefficients (ICCs) for the primary outcome to be 0.009 in Kenya, and 0.01 in Nigeria. We determined the minimum sample sizes required for 90% power to detect the original hypothesized effect sizes for each of the three main comparisons of interest and chose the largest sample size required. Hypothesized effect sizes and power were re-evaluated with pilot data for Kenya and Nigeria (Table 3). To ensure overall two-tailed Type I error (alpha) control at 0.05 in each country, the conservative Bonferroni correction was used to fix the alpha level for each comparison at 0.05/3=0.0167 in Nigeria and 0.05/2=0.025 in Kenya . See Supplemental File 1 for a detailed description of the sample size calculations, including derivation of the ICC estimates, effect sizes, and the different assumptions between countries.
Note that the difference in effect sizes between Nigeria and Kenya are due to different assumptions about client behavior in each country, informed by our understanding of the local health context. Since our outcome is a composite, different assumptions about the pathway to client ACT use result in different effect sizes. In general, we expect a higher proportion of clients receiving an mRDT and a lower proportion of negative/untested clients taking ACT in Nigeria compared to Kenya.
Pilot data in Kenya indicated slightly lower test positivity than expected; calculations based on this pilot data indicated we might only reach 59.5% power for the comparison of the combined PD+CD arm and the CD arm. By collapsing from 4 arms to 3 and increasing the number of clusters per arm (from 10 to 13), we expect to achieve 80.2% power for this comparison. We decided to keep the CD arm, rather than PD arm, since private outlets in Kenya are permitted to stock and sell ACTs but are not (yet) allowed to conduct mRDTs outside of research settings.
In Kenya, our sample will include 40 retail outlets with 14 outlets assigned to the control arm and 13 outlets assigned to each of CD and PD+CD while in Nigeria our sample will consist of 48 retail outlets equally assigned to each of the four arms. Within each of these outlets, we will have 170 exit interviews, resulting in a total sample size of 6800 in Kenya (170 X 40) and 8160 in Nigeria (170 X 48).
Enrollment of Retail Outlets and their Assignment into Study Arms
In each country, among those private registered retail outlets and PPMVs that expressed interest in participating in the study, eligible private outlets were identified according to set criteria (Table 4).
In Kenya, a full sampling frame of eligible retail outlets was generated for each county. Outlets were randomly selected for enrollment from this sampling frame; if an outlet declined enrollment, they were replaced with one of the remaining outlets in that county. In Nigeria, a complete listing of PPMVs was stratified across three geographical regions of the city and outlets were enrolled from a random subset proportional to the size of the strata (Figure 1). Following enrollment, retail outlets (Figure 3) were randomized to arms separately within each country by the study statistician. In Kenya, a uniform random number between 0 and 1 was generated for all 40 outlets, then sorted and split into 3 groups using the quantiles of the distribution stratified by county. Those groups were labeled A, B, C and those labels randomized so that they were allocated to one of the three trial arms. Given 40 outlets cannot be equally allocated to 3 arms, the extra outlet was assigned to the control arm. In addition, to avoid potential contamination of treatment effects, randomization was constrained such that any outlets in close proximity (<0.5km) were assigned to the same arm. In Nigeria, we used the same randomization process, but with allocation to 4 arms.
All outlets have access to mRDTs at a wholesale price, and the retail price of the mRDTs will be consistent across all arms within each country setting. The four-arm study design in Nigeria and three-arm design in Kenya together will allow us to measure the effect of the combined PD+CD intervention relative to no intervention or either intervention (PD or CD) alone.
Due to the nature of the interventions, it is not possible to blind participants and the implementation team to the study arm allocated to each outlet. Data collectors will be blinded throughout collection and study statisticians will be blinded during the analysis phase. Data collection will proceed via two methods: the health care providers’ use of the TESTsmART mobile app for real-time monitoring, and exit interviews conducted at each outlet for measuring outcomes. Data for main study outcomes will be collected by exit interviews with clients in order to avoid bias that may arise by relying on provider-reported data. Exit interviews and provider reporting data will be compared to assess agreement between both sources.
Data collection via the TESTsmART mobile app
Private providers will report routine data using the TESTsmART mobile application. The data collected through the TESTsmART app will be synced to a central server to allow for remote monitoring. All outlet attendants within each enrolled outlet will be trained to use the mobile app which reports on volume of clients, number of ACTs or other antimalarials sold, and number of mRDTs sold. Data reported through the app will primarily be used to track mRDT and ACT sales in real-time and will be regularly reviewed to tabulate and track the outlet-specific test positivity rate, volume of RDTs used, and visualization of a random samplling of uploaded mRDT photos. This routine monitoring will detect potential problems (e.g. providers who have unusually high- or low-test positivity rate, or errors in mRDT interpretation). Problems detected through routine monitoring will trigger supportive supervision and/or additional on-the-job training to ensure compliance and quality of diagnosis.
The app is designed to enhance future scalability of the intervention. Via the app ‘dashboard’, the study team will review the mRDTs, test results, and ACT sales to calculate the payment to each outlet based on their arm assignment. This will be done weekly and implemented through mobile money platforms in each country.
Data collection via cluster-based exit interviews
Interviews will be conducted with clients departing from participating retail outlets (study clusters) on random days of the week. All clients exiting the outlet that day are eligible to be screened and field researchers are instructed to make no pre-judgements about clients but rather approach the next available client exiting the outlet. Interviewees must meet specific eligibility criteria to proceed with the exit interview (Table 5). 170 clients per outlet enrolled in the study over the course of the 15-month intervention period. The study team will conduct a verbal consent process for participants who meet the inclusion/exclusion criteria, prior to participation in the exit interview outside the outlets.
Data for participant exit interviews will be collected electronically via tablet. The data will be encrypted and password protected. In Kenya, tablets are locked in a secure cabinet nightly and data are removed several times per week. In Nigeria, data will be transferred from the tablet to a secure, password protected computer once per week. The primary tool for developing the data collection forms will be REDCap™ hosted at Duke University, which is a free, secure, web-based application for data capture [15,16]. REDCap™ provides built-in data validation (e.g. data types, range checks) for quality assurance over data entry and an electronic audit trail that permanently tracks and logs every access of data, tools, or reports within the database. Individuals will be assigned a unique study ID and only anonymous data will be collected in this study. A data monitoring committee was deemed unnecessary by the funder because this study is minimal risk and tests an intervention designed to influence behavior and decision-making. Furthermore, we are not collecting any protected health information, and only anonymous data will be used in this study. On completion of the trial and publication of trial findings, the final trial dataset may be available to investigators if requested from the authors.
We will analyze client-level outcomes by fitting a modified Poisson regression model [17,18] with log link to estimate risk ratios (RRs) and identity link to estimate risk differences. Such an approach assumes a Poisson distribution for the binary outcome and then ‘fixes’ the estimated standard errors to correct for model misspecification.
To account for clustering by outlet we will use a generalized estimating equations (GEE) [19,20] approach with exchangeable working covariance matrix and robust standard errors (to correct for model misspecification due to specifying a Poisson distribution). The outcome will be regressed on three binary indicators for each of Control, PD, and CD, with treatment arm PD+CD (the combined interventions) serving as the reference group noting that the indicator for the PD will be excluded in the three-arm Kenya study. The model will also include fixed effects for the stratification variables and a vector of potential confounder variables (e.g., age, gender, education, or other socioeconomic indicators) to account for possible imbalances between study arms. All analyses will be based on the intention-to-treat principle whereby all clients will be included in the analysis irrespective of whether they complied with the intervention in the outlet at which they sought care (e.g. even if they did not use the ACT subsidy if they tested positive in an outlet in CD and PD+CD that received the client-directed intervention). Since we do not have longitudinal follow-up, we will not need to account for missing data due to attrition of clients. Patterns of client non-response will be described and compared by outlet and between arms. Given our prior experience in these regions, client non-response is anticipated to be minimal and comparable between arms.
Given that the literature indicates that when there are fewer than 40 clusters in a cRCT, small sample correction methods should be used to ensure that standard error estimates are correctly estimated when using GEE to analyze binary outcomes, and given that the size of the cRCTs in each country are close to this cut-off, we plan to adopt the use of the Kauerman-Carroll correction to avoid any possible problems [21,22]. We will compare secondary outcomes using the same modeling approach.
Commodity Supply to Study Participants
To ensure availability of mRDTs, the research team will set up a supply chain for participating retail outlets to access affordable and quality assured mRDTs. The mRDT price that retail outlets will pay is set by study team. In both countries, retail outlets will be trained in reorder procedures during the study training at the onset of the study. All outlets will receive an initial stock of two kits or 50 mRDTs, free of charge. Other auxiliary items, like gloves, a sharps container and waste bags will also be provided during the trainings. The sales price of the mRDT set by the study team approximately tracks with the median retail prices of mRDTs found in the public sector in Kenya or in other community retail outlets in Nigeria.
All retail outlets participating in the study will obtain ACTs through existing distribution channels and suppliers.
Retail outlets participating in the study have been recruited and training of staff on the use of the TESTsmART app commenced in June 2020. The study will commence assignment of retail outlets into study arms in October 2020, with retail outlets commencing transactions and data collection under the intervention structure provided for their respective study arm. The study will be implemented for a three month burn-in period before exit interview data collection begins in January 2021 (Kenya) and February 2021 (Nigeria). Data collection will continue through April 2022 (Kenya) and May 2022 (Nigeria), with data analysis conducted thereafter.
Trial results will be published as soon as possible upon completion of the study and will also be available on Clinical Trials.gov one year after the end of enrollment. We will post the trial description, data collection forms, and data structure on our institutional website as soon as the primary manuscript is accepted for publication. The requestor will be able to contact the principal investigator at a link on the website to request data. De-identified datasets containing participant-level data will be made available to the user with the following stipulations:
- The data will be used for research purposes and not to attempt to identify individual subjects
- The data must be stored securely and destroyed after analyses are complete
- The authors of any manuscript resulting from this data must acknowledge the source of the data upon which their manuscript is based.