We conducted a post-hoc analysis of part of the data from the randomized controlled REMIND-HIV trial, in which PLHIV had been randomly allocated to (1) RTMM, (2) Short Message Service (SMS) reminder texts or (3) standard of care and followed for 48 weeks. Details of the trial have been described elsewhere . The study was approved by the College Research and Ethical Review Committee (CRERC) of Kilimanjaro Christian Medical University College (KCMUCo) and the National Health Research Ethics Sub-Committee (NatHREC) of the National Medical Research Institute (NIMR) of Tanzania. The trial was registered at the Pan African Clinical Trials Registry under PACTR201712002844286.
Participants were recruited from two sites, which were Kilimanjaro Christian Medical Centre (KCMC) and Majengo Health Centre, both located in Moshi, Tanzania. PLHIV were approached by study nurses during a common clinic visit. Informed consent was obtained from all participants followed by screening for eligibility. The inclusion requirements were: (1) 18–65 years of age, (2) receiving antiretroviral treatment for at least six months, (3) subjectively judged by a nurse counsellor to be poorly adherent to medication, based on missed clinic visits, returning excess leftover medication, and/or having continuously high viral loads, (4) able to read and write and (5) able and willing to provide consent to study participation. We excluded participants if they (1) were admitted to the hospital and/or (2) participated in similar studies investigating digital adherence tools.
After obtaining informed consent from participants, study nurses interviewed participants and completed a screening form, containing inclusion and exclusion criteria, demographics, medical history, HIV history and times of usual ART intake. A secured web-based electronic data capture software system (REDcap) was used to collect and manage data. RedCap supports data validation, has an auditing trail and allows for data verification . After completion of screening, the data manager performed randomization in REDcap using block randomization, stratified by gender and study site. Participants were randomized in one of three arms, RTMM, SMS or control arm, at a 1:1:1 ratio . Participants were expected to attend the clinic every two months, according to standard care . At each clinic visit, adherence was recorded through self-report, pharmacy refill count and, in the RTMM arm, additionally through RTMM. Participants were followed for 48 weeks. Viral load was measured at baseline and at the last week 48 study visit. For the present study, adherence and viral load data obtained at the week 48 study visit are used. Adherence measures considered the period since the last study visit preceding week 48. We did not include adherence data from the full study follow-up due to incompleteness of the data during earlier visits, though we considered leftover medication from the before-last visit.
Self-reported (SR) adherence was measured using a questionnaire that was administered during a face-to-face interview by study nurses at each study visit. The questionnaire included two adherence questions: (1) ‘How many pills do you take per day?’ and (2) ‘How many pills did you miss in the past week?’ We calculated adherence as follows:
Self-reported adherence in the past week = (((7xpills to take per day) - (missed pills))/ (7xpills to take per day)) *100%.
Pharmacy refill counts (PR)
A case report form was administered face to face to record pharmacy refill data at each study visit. The study pharmacist recorded the number of pills dispensed during the previous visit by asking ‘How many pills were given to you at the previous visit?’ while checking the medical file for the same information. In addition, the left-over pills returned during the previous and current visit were counted. For the participants who did not return pills, we asked to recall the number of pills that were left at home. In case leftover pills were unknown, our assumption was that all pills had been taken as prescribed in during the previous visit. Adherence was calculated as follows:
Pharmacy-refill adherence: (((pills dispensed at previous visit + returned pills at previous visit) - returned pills at current visit)/ (number of days between visits*number of pills to take per day)) *100%. Assuming that levels higher than 100% were representing 100%, we truncated maximum adherence at 100%.
Participants in the RTMM arm were given a WisepillÒ RTMM device to monitor their medication intake in real time. When the device is opened, information including the time stamp is wirelessly sent using General Packet Radio Services (GPRS) to a secured web-based central database. Each opening was recorded, which was taken as a sign that the participant ingested the dose. If the box was not opened on time (agreed time between participant and healthcare provider), the participant received a short message service (SMS) text on his/her mobile phone which acted as a reminder to take medication.
Adherence levels were calculated at the 48-week follow-up visit of the study.
Adherence= (Number of openings over a given period/number of expected openings (based on prescription of number of dosing moments per day)) *100%.
HIV viral load data were obtained at 48 weeks of follow-up. The Tanzanian HIV guidelines direct health care workers to act once someone has 1000 copies/mL i.e. to provide enhanced adherence counselling or switch treatment [27,28]. However, laboratory equipment can determine viral load as low as 20 copies/ml. Therefore, plasma HIV RNA < 20 copies/ml was defined as virologically suppressed, while plasma HIV RNA <1000 copies/ml was categorized as stable and plasma HIV RNA >1000 copies/mL was categorized as unstable. As the trend in analyses of both cut-off values were the same, to answer our objective, we only considered a viral load level >20copies/mL as representative of virological failure.
Statistical analyses were conducted with Stata v.15. In the analyses, we included all participants who had a viral load measurement at week 48. The analyses that included RTMM-based adherence were only based on participants who were in the RTMM arm as RTMM was not used in the other arms.
To evaluate the ability of the various adherence measures to predict a detectable viral load, we conducted analyses using a cut-off of >20 copies/mL. We calculated the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) for different adherence cut-off values for each adherence assessment method separately. We classified participants as having poor adherence or good adherence using adherence cut-off values of 80%, 85%, 90%, 95% and 100% whereby the percentage stands for the percentage of doses taken. We used these to determine at which cut-off the prediction of a viral load ≥ 20 copies/mL was strongest.
For each of the adherence measures, its sensitivity was defined as the percentage of participants with a viral load ≥20 copies/ml who were identified by the methods as being poorly adherent at a certain adherence cut-off. Its specificity was defined as the percentage of participants with viral load < 20 copies/mL who were identified as being adherent at a certain adherence cut-off. The positive predictive value (PPV) was defined as the percentage of non-adherent participants with a detectable viral load, and the negative predictive value (NPV) as the percentage of adherent participants with an undetectable viral load.
The adherence measures were also combined to determine how two or three of them might impact sensitivity, specificity, PPV and NPV. To create composite adherence measures, participants were identified as non-adherent if they were below the adherence cut off in any of the combined measures under consideration. For example, when self-report and pharmacy refill counts were combined at a certain same cut-off and adherence was below 95% in either self-report or pharmacy refill count, the combined variable also was considered being below 95%.
For each of the adherence measures, sensitivity and (1-specificity) at the various adherence cut-off values were plotted in receiver operating characteristic (ROC) curves based on all the adherence data to determine the accuracy of an adherence measure to predict viral load. An Area under the ROC curve (AUC) value of 0.5 indicates that a test has no discriminatory capacity and an AUC of 1.0 indicates perfect discriminatory capacity. For screening purposes an AUC of 0.7 or higher is usually considered sufficient .
Besides the ROC curves analysis, logistic regression was used to identify which adherence measure predicted detectable a viral load ≥ 20 copies/mL while adjusting for demographic and clinical characteristics, including sex and type of ART regimen, whether someone was on a first line regimen, being on treatment at study entry, and entry VL. Analysis of baseline data of the parent trial had shown that TLE (the combination of tenofovir, lamivudine, efavirenz) was a significant predictor of viral load<20copies/ml at study entry and therefore type of ARV regimen was categorized as TLE or another regimen. Two-sided p-values of <0.05 were considered statistically significant in all analyses.