Enhanced Surveillance in a Digital Health Landscape: The Role of the Near-Real Time Right to Care Knowledge Centre within South Africa’s APACE HIV Program

Introduction: Right to Care (RTC), through support from the United States Agency for International Development (USAID) implements innovations that improve data accessibility and utilization for improved program performance and patient outcomes for large scale HIV care and treatment programs. Frequent and accessible data, coupled with changes to business practices to use that data, allow for targeted and timely HIV program interventions that impact patient outcomes in Ehlanzeni district, Mpumalanga, South Africa. Methods: In South Africa, within the USAID Accelerating Program Achievements to Control the Epidemic (APACE) program, the Knowledge Centre (KC) – an interoperable automated data warehouse and visualization near real-time solution - allow for rapid daily assessment of over- and under-performance cross-sectionally. The authors established the impact of the KC intervention for 29 USAID-selected Siyenza facilities, before the KC intervention (Dec 2018 to Feb 2019) compared to HIV outcomes during the KC intervention (March 2019 to May 2019), stratied by facility classication as included or not included in the Siyenza program using both non-parametric and parametric methodologies. Results: Average facility ART initiations increased by 16% from a monthly average of 54 new initiations pre-Siyenza to 62 new ART initiations during Siyenza; retention increased to a net retention in care ratio of 1.15 indicating that patients labelled as lost to care were brought back. An independent-samples t-test indicated that the net retention in care scores were signicantly higher for Siyenza facilities (M =.379, SD =.808) compared to non-Siyenza facilities (M=-.061, SD =1.016), t(208) = 2.224, p < 0.027, d = .445 during the Siyenza period. A signicant difference was noted between the means with a 62.3% chance that the Siyenza facilities have a higher observed mean than the non-Siyenza facilities during the intervention period. Results indicate that Siyenza facilities maintain cohort growth, while non-Siyenza facilities continued to lose patients. Conclusion: The accessibility and utilization of near to real-time interoperable data within HIV programs allowed for rapid responses to program performance, needs and improved patient outcomes. Daily data review meetings facilitated precision programming intervention, non-targeted sites were retaining as many patients as were initiated (net new retention ratio of 1.02). Retention to a net retention in care ratio value of 0.78 during the Siyenza intervention. The results indicate that caution should be used not to let up on non-selected facilities whilst focusing on intervention facilities.


Introduction
Service delivery partners that support HIV care and treatment programs through the President's Emergency Plan for AIDS Relief (PEPFAR) track substantial data to demonstrate program performance across multiple indicators. These data are primarily collected retrospectively, aggregated, and used to passively report program achievements against funder-set targets as routine monitoring and evaluation activities (1). As countries and program work towards HIV epidemic control within their catchment areas (2), frequent data collection and utilization becomes more critical to identify lagging performance and prompt action. Retaining HIV infected patients in quality care, supports universal test and treat initiatives to eliminate new HIV infections (3) Regular review of data is critical to identify gaps in programs and indicates how to shift limited resources and interventions to improve patient outcomes proxied by performance against key indicator targets.
Bhattarai et al. (4) identi ed an urgent need for real-time data in improving processes and ultimately outcomes in healthcare and highlighted the need to reduce data lags to less than four weeks to re-duce patient losses and improve health outcomes.
Health Information Systems (HIS) can support facility-level quality improvement activities (5). Yet, even if these HIS are digitized, processing from facility to an aggregated data set takes valuable time. Further emphasis on perfect data quality for reporting purposes limits immediate action. Program quality improvement needs necessitate innovative solutions to make eld data more accessible and actionable (6). More e cient data collection systems through automation allow limited resources to be directed to patient care rather than data collection (7). Effective data collection can, however, improve accessibility, relevance, accuracy, reliability, and coherence of the data. It also allows for the possibility to imbed data quality measures into the capturing, reporting and decision-making process.
Right to Care (RTC) operates as a care and service delivery partner under the United States Agency for International Development (USAID) Accelerating Program Achievements to Control the Epidemic (APACE) program in Ehlanzeni and Thabo Mofutsanyane districts in South Africa. We support 225 Department of Health facilities to provide quality HIV care and treatment to: (1) prevent new HIV infections and reduce HIV morbidity and mortality, (2) increase uptake of HIV testing, (3) increase the proportion of HIV-positive clients initiated on ART, and (4) increase the proportion of PLHIV who are virologically suppressed. USAID launched a special project known as Siyenza translated as "Let's do it", between 1st of March to the 9th of May 2019 in a bid to improve overall APACE performance, with a primary objective to rapidly accelerate program performance. To maximize existing resources, USAID and implementing partners selected sites with the greatest potential to improve overall cohort achievement. Speci cally, Siyenza inclusion criteria considered facilities with the highest difference between expected outcomes (targets) and current achievement for new initiations and total patients in care in USAID-de ned priority districts. Twenty-nine Right to Care-supported priority facilities from both the Thabo Mofutsanyane and Ehlanzeni districts were selected for the Siyenza initiative. The Siyenza initiative required an innovative approach to collect and act on data from across all supported facilities focusing on treatment initiations and retention. In the two mentioned districts, 29 high volume facilities contribute to some % needed for patients remaining on ART contribution along with other facilities in the country.

Methods
Implementation data from Dec 2018 to May 2019 aggregated at facility level did not require consent. To reach the ambitious cohort targets needed to control the epidemic, a data collection and analysis platform was used in daily data review meetings with program managers and eld staff in each district. The RTC Knowledge Centre (RTC KC) was used to collect, manage, and analyze the data using existing sta ng infrastructures. The RTC KC is a Microsoft Azure-based warehouse and reporting platform developed in partnership with Qode Health Solutions. This platform supports HIV program management with big data analytics and disparate data integration, including point of care testing data from HIV counsellors, custom program indicator data, laboratory results, facility HIS integrations, and other applicable data sources.
This data populates aggregated analytics accessible through an RTC KC web-portal that can be visualized at various levels of aggregation (age and gender strati cations by facility, district, and province) and used for monitoring program performance against set targets for various indicator and delivery improvement, targeted interventions resulting in improved clinical outcomes.
We chose to use Automated Data Capture (ADC) facility templates to support daily data collection and review for the Siyenza project. A daily template capturing the number of people tested for HIV (and those testing positive), initiated in treatment, and the total number of active clients on treatment was emailed to existing program data management staff based at each facility. The results in this report will, however, focus on patients-initiated on treatment and retained in care. The identi ed staff member completed the template at the end of the day and returned the completed template via email to an automated email server. Each le was then automatically integrated into the data warehouse populating visualizations used during data review meetings the next morning ( Fig. 1).
The analysis was used during program managers' daily data review meetings to identify programming gaps in the facilities using data and to provide information from the KC. As a result, our analysis is currently limited to PEPFAR indicators related to patients-initiated onto ART and retention. The net retention is then compared to the number of new initiations as the net retention in care, TX_NET_NEW ratio, (Eq. 1).

Equation 1. Net retention in care ratio
T\mathrm{X_NET_NEW\ \ ratio\ }=\frac{(TX_{CURR1}-TX_{CURR0})}{(TX_{NEW1}+TX_{NEW2\ }+TX_{NEW3})\ } The net retention in care ratio or TX_NET_NEW ratio re ects how aggregate facility retention from the current ART cohort size is measured and compared to the number of new initiations for the period under consideration. TXCURR is de ned as the current ART cohort size of all active patients. A ratio less than one indicates a loss of patients; equal to one indicates no loss of patients; and a net new ratio greater than one indicates that existing patients categorized as lost to follow-up were brought back into care.
Monthly data captured through the RTC KC was classi ed by facility type (Siyenza or non-Siyenza). Summary statistics were calculated for each measure and facility classi cation. We then compared each class's post-intervention indicator measures to a historical baseline (pre-Siyenza) to calculate percent change.
PEPFAR Indicators were calculated for patients initiated on ART (TX_NEW), net retention (TX_NET_NEW) and the net retention in care or TX NET NEW ratio as described in Eq. 1.
Data were cleaned and outliers were removed in SPSS v 26 (IBM Corp. 2013, Armonk, NY). A retention in care ratio (TX_NET_NEW ratio) of 18.0 was removed from the values for Siyenza and a retention of care ratio of 44.0 was removed from the non-Siyenza retention in care values. Shapiro Wilk tests were performed to establish whether indicators were normally distributed. The latter indicated whether data required further transformation. The analysis indicated that non-parametric analysis was possible for initiation on ART and the net increase or decrease of patients. It was possible to transform the net retention ratio in Eq. 1 to a normal curve using a Box-Cox transformation in SPSS, thereby allowing for prediction of the TX_NET_NEW ratio for both periods using a linear regression and a t-test to determine whether there are signi cant differences between the means. The application of regression weights was used to correct for selection bias. The Common Language effect size for the TX_NET_NEW Ratio was calculated as described in (8) A limitation of the analysis is that the results after the intervention period cannot be compared to the period before the intervention since the Department of Health continued with the intervention inde nitely. Table 1 Summary Results for the comparison between the pre-intervention and intervention period in Siyenza and non-Siyenza facilities As described by Table 1  Correlation tests were conducted to examine the relationship between retention and selected Siyenza facilities before and during the period when the Siyenza intervention was conducted. As per Cohen's guidelines in (9) The data for the TX_NET_NEW Ratio in the period before and during Siyenza interventions were not normally distributed therefore it required a Box-Cox transformation to enable a regression and independent sample t-tests to be performed.
A regression indicated that the net retention scores (TX_NET_NEW Ratio) or Eq. 1: for the period during the intervention explained a signi cant amount of the variance (15%) in the value of retention from baseline for three months (F = 4.948, p < 0.027, R2 = 0.23, R2 adjusted = -0.19), with p < 0.05. Figure 3 indicates that the net retention in care ratio for non-Siyenza facilities was 0.29 in comparison to a ratio of 1.66 in Siyenza facilities before interventions were implemented. Further losses in retention were predicted for non-Siyenza facilities to 0.23 during March-to May 2019, whilst a gain was expected in the net retention ratio in Siyenza facilities of 1.21, meaning that patients were not only retained in care, but were brought back to care through tracing efforts.

Results of the Independent-samples t-test
An independent samples t-test was run to determine whether there are signi cant differences between the net retention in in care (TX_NET_NEW Ratio) during the intervention in Siyenza and non-Siyenza facilities after outliers have been removed as described in the methodology. The independent-samples t-test indicated that the net retention in care ratio (TX_NET NEW Ratio scores) were signi cantly higher for Siyenza facilities (M = 0.379, SD = 0.808) compared to non-Siyenza facilities (M = 0.061, SD = 1.016), t(208) = 2.224, p < 0.027, d = 0.445 during the Siyenza period. Homogeneity of variances was not violated, as assessed using a Levine's test (p = 0.776). The results of the independent-samples t-test indicated that there was indeed a signi cant difference between the means of the two different groups with a Common Language (CL) effect size of 0.623. Consequently, this means that a measurement from the intervention group have a 62,3% probability of having a higher observed mean net retention in care than the non-intervention group during the Siyenza period.

Discussion
A suite of data and clinical interventions was identi ed and implemented as part of the Siyenza program in Ehlanzeni, South Africa and daily data review meetings resulting in improved ART initiation and retention in care compared to the period immediately preceding Siyenza. Interventions included improved facility data management ( le audits, work ow improvements and same day data entry, and standardized ling system) including the application of clinical best practices (increased multi-month scripting and dispensing, enhanced track, and trace for lost to follow-up patients, extended clinic hours, and appointment reminders and follow-up for early missed appointments). The program has since established a tracking system to better link these interventions with facility outcomes.
The availability and actioning of almost real-time data are critical to demonstrating the impact of the rapid program improvements described above and targeting those interventions to the right facilities. Program data in large HIV care and treatment programs are currently relegated to providing retrospective program evaluation at an aggregated level. However, to meet increasingly ambitious program performance targets, care, and treatment partners with authorized users must be able to access, process and act on daily data from multiple integrated data sources, preferably at patient level to avoid double data entry.
As the Siyenza intervention began, program managers identi ed several data gaps related to human re-sources, timely procurement, supply chain management, data ow and quality assurance issues. These data are often captured in other systems (e.g., Workload Indicators for Sta ng Needs, Synch, Rx Solution, etc.) supporting the need for further system integration. The data should be at the lowest level possible, bearing in mind program e ciency, feasibility, quality, and cost (10). Any mHealth or e-Health programs should consider evaluation from the outset, including collection of baseline data prior to implementation.
Leadership promoting daily data review of progress against targets for key indicators at priority facilities worked to reinforce a culture of data utilization for Right to Care as has been demonstrated to be effective elsewhere (11). Multi-disciplinary data review meetings should allow rapid review of the data and decision-making, including tapping into the tacit knowledge of a diverse team. Data managers and program managers are then cooperatively able to interrogate the trends and interpret real-time data (aware of inconsistencies and quality implications of this type of data) to adjust resources and propose tailored interventions. The rapidity of the data and review allow for quick evaluation of the interventions that are being implemented at struggling facilities.
All care and treatment partners are expected to submit similar data to funders; however, during the intervention the process was previously overly manual and required multiple points of data capture, categorization, and entry. This time-consuming process limited any further analysis that could have been performed by our staff to focus on program improvement. Automating certain steps in the reporting and analytic process with Right to Care's Knowledge Centre will allow for more time to interpret and act on the data, which is essential when using daily data. Automated reporting templates ensure standardized data from every facility with reports mimicking the required reporting forms and simplifying transcription; and automated visualizations provide up-to-date data with every form submission.
If the granularity and frequency of data collection is overly burdensome then reporting compliance and data quality can suffer. A near-real time system can expect to have some data quality trade-offs: when data is reported more frequently, an invalid data point is diluted and mitigated as an outlier. Focusing on data recency (the time elapsed between event and reporting) allows for very timely correlation between intervention and program outcome.
Improving the e ciency of the data management and analysis provide time to interpret and act on the data. The latter a driving factor to improved program performance and sustaining daily review meetings. Data review meetings allow program managers to select evidence-based interventions based on the facility need. The interventions' scopes vary from clinical technical assistance, increased human resources, le management and data quality, and supply chain e ciencies. The recency of the data and frequency of review also enable rapid evaluation of the chosen interventions.
The process that we developed and implemented for our programs in South Africa still relies on an inter-mediate data transcription process -from clinical registers, paper les, and electronic medical records into the Knowledge Centre data collection forms. Right to Care has demonstrated the ability to integrate testing, clinical, and laboratory databases leveraging the investments made to capture clinical data in electronic systems. All care and treatment partners and the Department of Health could bene t by further providing governance on how electronic clinical data can be accessed, integrated, and used. Standards for application programming interfaces and interoperability of systems (12)(13)(14) could further create a data eco-system that is broader than government facilities and truly tracks patients across systems to ensure their outcomes are achieved as programs meet performance targets more e ciently and effectively.
Democratizing the ethical use and integration of electronic disaggregated patient-level clinical data can potentially propel HIV care and treatment into new frontiers with segmentation to truly understand patient needs as well as predictive modelling. We can then start to strengthen the governmental support at the district and provincial level through regular access to analytics covering the care and treatment cascade. We can also begin to tailor patient treatment protocols (inclusive of psychosocial support) at the point of testing based on their potential for poor outcomes like being lost from care. Providing advanced analytics as a service to government and supporting partners puts greater focus on effective and precise interventions. Data managers can then focus on ensuring that quality data is captured on time for every patient into the primary data source rather than a myriad of alternative data collection systems. The integration of systems will result in less duplication and better e ciency after innovations and legacy risks have been assessed. Comparison between monthly retention in care (NET NEW) ratios calculated as described (monthly)