Settings and Project JEET
This study assesses demographic and treatment related information of patients who sought treatment for TB through private sector facilities that were engaged with Project JEET, as a part of the Patent Provider Support Agency (PPSA) in India managed by the William J. Clinton Foundation. PPSA is a model under which a third-party entity, such as a non-governmental organization, engages private sector TB physicians to provide end-to-end services for TB [34].
Patients treated under Project JEET were assigned a treatment coordinator who was responsible for regularly following up with the patient and counselling them through different stages of their treatment, either in-person or via telephone. In-person counselling typically took place at the treating facility or within patients’ homes. In some cases, patients preferred meeting the treatment coordinator at another place of their convenience.
Study design
A quasi-experimental study using a propensity score matched dataset of routinely collected programmatic data was conducted. Access to free drugs was the independent variable under investigation.
Data source
Programmatic data recorded by JEET staff with records for more than 0.2 million patients diagnosed with TB across 22 cities in India, between 2019 and 2021 was used for the analysis (Table S1). This data included patients diagnosed across 7,212 private facilities and includes 1) TB patients demographic characteristics such as age, sex, and diagnosing district, 2) TB diagnostic and treatment information, including type of diagnostic test performed, pulmonary or extrapulmonary diagnosis, whether free drugs were provided, patients’ treatment outcome, and number of follow-up contacts made by treatment coordinators.
Inclusion & Exclusion criteria
A total of 42,562 adult patients were deemed eligible for the analysis. A table with the number of exclusions at each step is provided in Appendix 1, Table S2.
Districts: We considered data for patients from cities that began PPSA operations on or before 1 January 2019, and for whom at least 1% of patients availed of free drugs, for each of the 5 quarters of the study period. Seven of the 22 cities met these criteria: Ahmedabad, Bhopal, Darbhanga, Delhi, Gurgaon, Indore, and Surat.
Age & Study Period: Pulmonary and extrapulmonary drug sensitive adult TB patients (≥ 16 years) from these seven cities who began treatment between 1 January 2019 and 31 March 2020 and had a treatment outcome assigned on or before 31 December 2021 were considered. The study period ensures that enough time has passed for a treatment outcome to be recorded for both pulmonary and extrapulmonary patients. The study period also ensures that a large majority (99.63% or 42,406) of selected patients (42,562) were diagnosed with TB before the first lockdown was initiated in India (25th March 2020).
Availability of treatment coordinator information and treatment outcome: Since our study is concerned with how the provision of free drugs impacted patient follow-up and treatment outcomes, we included only those patients who had a treatment coordinator assigned, along with data on number of follow-ups made by the treatment coordinator, and a recorded treatment outcome. The availability of data on follow-ups with a treatment coordinator and the treatment outcome indicates that the patient was actively under care for TB. We did not include patients who had denied counselling or whose doctors had denied counselling on their behalf.
Age criterion: We excluded paediatric patients (≤15 years of age) to account for the differences in care management and treatment regimens for paediatric patients compared to adult patients.
Drug Resistant TB: Patients who were found to have a drug resistant form of TB were excluded from the analysis to account for the differences in care management and typically longer and more complicated treatment regimens.
Measurement Errors: We applied the following minimum criteria to our dataset to manage potential recording and measurement errors: 1) at least 30 days elapsed between a patient initiating treatment and being assigned a treatment outcome; 2) patients were assigned a treatment outcome after the assignment of a treatment coordinator.
Outlier Treatment
Programmatically, treatment coordinators were advised to make a total of 16 follow ups with the patient. Of these, 8 follow ups are recommended to be conducted in the initial 2 months or the intensive phase of the treatment (weekly), and 8 in the next four months or the continuous phase (fortnightly) of the treatment.
An outlier treatment for the number of follow ups made with the patient was conducted to ensure our results do not get biased because of cases wherein a high number of follow ups can be attributed to a data error, or a clinical reason specific to a patients’s unique situation, or in some cases, a data entry error. The skewness coefficient for the data before the treatment (42,881 observations), was found to be 0.904 (Figures 1,2), which reduced to 0.693 post the treatment. Outliers were identified using the interquartile range (IQR) criterion, following Seo [35] and Steven, [36]. The rationale is described in Appendix 2. A total of 319 (0.7%) patients were identified as outliers, having ≥44 recorded follow-ups. All of the outliers identified were from patients diagnosed in Ahmedabad (292 or 4% of the district’s patients) and Surat (27 or 0.4% of district’s patients). Conversations with TB program officers in these districts indicated the high likelihood of these values to be errors. We identified no outliers towards the lower range of the distribution.
Model Theory
We fit multiple statistical models to understand how access to free drugs impacts the engagement of a patient and their treatment outcome. It is hypothesized that access to free drugs improves patient outcomes by increasing the engagement between the patient and their treatment coordinator. This increase in engagement leads to greater support to the patient through their treatment, which results in better treatment adherence, ultimately leading to better treatment outcomes (Figure 3). This study attempts to develop a precise estimate of the treatment effect of free drugs on patients’ follow-ups and treatment outcome by using a combination of regression methods on a matched dataset built through propensity choice modelling.
Outcomes of Interest
Two primary outcomes were examined: 1) patient follow-ups and 2) treatment outcomes. Patient management typically involves a combination of factors relating to how the patient was supported by the health system throughout treatment. We considered the number of follow-ups made by treatment coordinators as a proxy for patient management, where more follow-ups translated into more engaged patient management. For treatment outcome, one of five outcomes were considered: 1) treatment complete, 2) cured, 3) treatment failure, 4) death, or 5) lost to follow-up. In the current study, we defined successful outcomes as either treatment complete or cured. The rest were considered as unsuccessful. A definition of each of the outcomes is provided in Appendix 8.
Propensity Choice Modelling
Since the study uses programmatic data, access to free drugs is not randomized among population groups, making it difficult to assess the average treatment effect (ATE) of free drugs on the outcomes of interest. While randomized experiments are typically utilized to understand the causal effect of a treatment, running such experiments is often cost intensive and laden with ethical issues, especially in studies concerning welfare and healthcare treatment effects [32]. Furthermore, an RCT was not feasible for the current study since free drugs are available to all TB patients in India under NTEP and are also widely prescribed in both the public and private sectors.
Several prior studies have acknowledged the usage of matching methods to infer causal insights from observational data, specifically in the field of health care assessment [33, 37]. Creating a dataset with observations matched on choice attributes provides an opportunity to estimate the average effect of the treatment as if it were a randomized experiment [38]. We used propensity score modelling to create a matched dataset that comprised of treated patients (free drugs) and untreated patients (no free drugs), which also included data on potential confounders for each individual [33, 39–42]. The propensity score was estimated for each patient, and then used to create comparable groups of people with access to free drugs (treated) and those who paid out-of-pocket (untreated). The scores were found to be adequate predictors of whether or not a patient had access to free drugs or not (Appendix 3). The nearest neighbor matching algorithm was used to identify pairs of treated and untreated observations [43–46]. To ensure minimum possible bias, we adopted two additional measures. First, we employed a calliper width of 0.05 for the age and district variables, meaning the matched pairs were a maximum of 0.05 standard deviations away from each other, which is more conservative than the standard calliper of 0.2 [49]. Second, we employed exact matching rather than nearest neighbor matching on three variables: 1) proportion of males, 2) proportion of extra pulmonary cases, and 3) proportion of patients diagnosed using Xpert testing. Exact matching is preferable to nearest neighbor in many cases, but matching each individual on several independent variables results in a lower number of matched pairs in the final dataset [46, 50]. In our case, employing exact matching for selected variables resulted in 11,621 matched pairs in our final dataset, with only 11% (1,392) of treated observations going unmatched.
Statistical modelling
Using the matched dataset, we fit fixed-effects ordinary least squares (OLS) regression and fixed-effects logistic regression models to estimate the impact of free drugs on the number of follow-ups made with the patient and the likelihood of a successful treatment outcome, respectively. As covariates in the OLS regression model, we fitted a series of models, sequentially including free drug provision, sex of the patient, age category (16 to 19, 20 to 45, 46 to 65, and ≥ 66 years), TB type (pulmonary or extra pulmonary), and whether Xpert diagnostics were used. The logistic regression model was fit to assess the likelihood of a patient receiving a successful outcome at the culmination of treatment and utilized the same set of covariates.
The diagnosing district and diagnosing quarter were included as fixed effects in the OLS and logistic regression models to control for program-related influences of patient care and adherence to treatment. Treatment coordinator fixed effects were included to control for the impact of individual healthcare workers’ frequency and way of counselling patients, thereby impacting overall patient care and treatment outcomes. Only one of the two fixed effects – diagnosing district or treatment coordinator, could be included in a single model specification because of perfect correlation between them. Since a treatment coordinator provides care to patients diagnosed in their assigned district only, controlling for them enables controlling for district level fixed effects.
To establish the linkage between follow ups and treatment outcomes, a logistic model was fit with follow ups as an additional dependent variable, while including for the status of free provision of drugs and other covariates.
Sensitivity Analysis
Multiple sensitivity analyses were conducted to understand the impact of free drugs on follow ups and treatment outcomes. These are illustrated in the forest plots (Figures 6 and 7), tabular results of which are provided in Appendix 7. Our results were robust to specifications which excluded cases where treatment outcome resulted in lost to follow up, or when we ran district-specific models. The results were robust to alternative matching methods, including matching on the treatment coordinator, instead of matching on districts. This particular matching specification resulted in 9,718 pairs, wherein 139 treatment coordinators had an equal share of patients being prescribed free drugs, and those that were paying out of pocket for the same.
Statistical software
The analyses were conducted in R 2022.07.01. `MatchIt` package was used for the propensity score matching procedure, `broom`, `cobalt`, and `gtsummary` packages were used for visualizing fitted and residual values, generating balance plots from propensity choice modelling, and generating summary statistics, respectively. Packages used for data cleaning, preparing the analytical datasets, measuring skewness, and visualizing results were `dplyr`, `tidyr`, `moments`, and `ggplot2`. The `sandwich` package was used to compute heteroscedasticity-consistent robust standard errors.