Consistent with the quasi-experimental two-group pre-post design, we used the difference-in-differences (DID) method to ameliorate potential confounding bias.17,18 The underlying assumption of the DID method is that the change in readmission rates from pre- to post-period in the comparison group is a good proxy of the counterfactual change in the pilot group had there been no pilot program (Figure S1 in supplemental materials). The effect of interest is the average treatment effect on the treated which answers the question: for patients treated in the pilot sites, was the program a cause for the change in readmission rate? On the probability scale, a DID method estimates the difference of risk differences (DRD); and on the odds ratio (OR) scale, a DID method estimates the ratio of ORs (ROR). The pre-and post-periods for the BCN (a health maintenance organization [HMO]) patients were 7/1/2016 to 4/30/2018 and 5/1/2018 to 8/31/2019; the corresponding periods for the BCBSM (a preferred provider organization [PPO]) patients were 2/1/2017 to 11/30/2018 and 12/1/2018 to 3/30/2020. In the post-period, another site (Brighton Center) implemented a program like C.L.I.M.B.; thus, to avoid contamination, patients whose initial detox occurred at that site in the post-period were excluded (Figure 1). The pilot program was approved by the BCN and BCSM medical directors, and the current evaluation was approved by the Institutional Review Board of Michigan State University as non-human subject research (STUDY00000846).
(Figure 1 Here)
Patients 18 years or older, who had a detox inpatient stay for a diagnosis of OUD in any of the two pre-periods were included in the study. To ensure data completeness, a patient had to be enrolled in the health plan for 6 or more months before the initial detox in the pre-period to capture baseline comorbidity; and the initial detox did not occur within 90-days of each period’s end date.
Following ASAM guidelines,14,15 the C.L.I.M.B. program (Figure S1) included services for the continuum of OUD cycle, including detoxification, residential services, partial hospitalization/intensive outpatient services, outpatient services with medically assisted treatment (MAT), and a modified smartphone support application (app), called A-CHESS, originated in 2011 at the University of Wisconsin Center for Health Enablement Support System (CHESS), which is a comprehensive relapse prevention tool based on the self-determination theory19 to help patients with substance use disorders (SUDs) succeed in recovery.
Prior to the pilot program implementation, providers in the two sites provided same services as other OUD treatment facilities. During the implementation, they agreed to follow the C.L.I.M.B. codified protocol (detailed in Appendix A) with an emphasis on the master-treatment-plan development, family/system assessment, warm handoff, completion of tasks regardless of length of stay, and the use of A-CHESS. Key features of A-CHESS20 included :1) a “Help” button linked to the patient’s preapproved supporters, 2) positive and potentially distracting games, and audio-video relaxation recording; 3) cognitive behavioral therapy boosters; 4) functionality monitoring with self-assessment tools; 5) a global positioning system location tracker that will initiate a patient-defined action (e.g., contacting sober coach) when s/he approaches a high-risk location, and 6) just-in-time feedback via a counselor dashboard.
All other OUD treatment facilities in the U.S. that BCBSM and BCN members attended for inpatient detox in the study period constituted the comparison group. The usual care available at each facility varied and was expected to be representative of current practice in the field. Not all facilities covered the continua of LOCs.
Primary outcome: 90-day detox readmission after an initial detox inpatient stay at any facility. Readmission was identified by the same method as the initial detox: any inpatient stay with a diagnosis of F11.x or F11.xx using the International Classification of Diseases, 10th version, Clinical Modification (ICD-10-CM) codes, and revenue codes 01x6 (x=1, 2,3,4, or 5).
Secondary outcomes: Other ASAM LOCs, including partial hospitalization/intensive outpatient services, outpatient services and MATs. MATs were identified using National Drug Codes in pharmacy claims and the Current Procedural Terminology codes; and revenue codes and/or procedure codes were used to find LOC 1.0-2.5 (partial hospitalization/intensive outpatient services/outpatient) services (list of these codes available upon request).
Treatment groups: the National Provider Identifier codes for the two pilot facilities were used to identify patients in the pilot group. Patients in the other treatment facilities were the comparison.
Comorbidity: the Agency for Healthcare Research and Quality Clinical Classification Software Refined version (v2021.2)21 based on the ICD-10-CM codes was used to find in medical claims of comorbid conditions in the 6 months prior to the initial detox in each period, including mood disorders; anxiety-, fear-, trauma- or stressor-related disorders; alcohol-, cannabis-, sedative-, stimulant- hallucinogen- or inhalant- related disorders; neoplasms; suicidal ideation/attempt or intentional self-harm; endocrine, nutritional and metabolic diseases; diseases of the nervous, circulatory, respiratory, digestive, musculoskeletal system, or genitourinary systems. Emergency room visits in the 6 months prior to the initial detox in each period were identified using revenue codes 045x (x=0-9).
Covariates: Patient’s age, sex, HMO or PPO plan types, and residential zip codes were extracted from health plan enrollment files. The 5-digit zip codes were linked to the census tracts using the U.S. Department of Housing and Urban Development zip code crosswalk files22 where a census tract with the highest residential ratio was chosen when multiple tracts were within the same zip code.23 Past research found that living in a disadvantaged neighborhood was associated with worse health conditions and increased healthcare utilizations. We used the 2018 Area Deprivation Index (ADI),24,25 2015 Childhood Opportunity Index (COI),26 and 2018 Social Vulnerability Index (SVI)27 to approximate the neighborhood characteristics and as proxies to patient socioeconomic status. Higher ADI rankings and SVI scores indicate more disadvantaged neighborhoods; but higher COI scores indicate more opportunities. All indices were transformed to have a range from 0 to 100.
We compared the differences in covariates and comorbidities between the C.L.I.M.B. and comparison group in the pre- and post-periods using chi-square tests for categorical variables and t-tests for continuous variables. As in the tradition for propensity score analysis, we also presented the standardized differences (difference divided by the pooled standard deviation) between the two group. When the absolute value of the standardized difference is greater than 0.1, it is indicative of non-negligible difference.28 We estimated the DID effects using six statistical methods to triangulate evidence: 1) multivariable logistic regression adjustment (RA) controlling for comorbidities and covariates; 2) augmented inverse probability weighted (IPW) estimation29 where covariates for the outcome and the propensity scores (PS) models were selected using logistic lasso;30 3) IPW estimation where the PS was estimated using logistic regressions controlling for the same covariates in the RA model; 4) IPW-RA double robust method;31 5) bias-corrected single nearest neighbor matching method32; and 6) PS matching with a caliper 0.2. The 95% confidence intervals (CI) were estimated using the percentile-based bootstrap CI with 1,000 bootstrapped samples. All analyses were performed in Stata version 17.33
We performed two sets of sensitivity analysis. First, we excluded 123 patients (236 admissions) who were in both pre- and post-periods, because the analyses may be contaminated by the correlations between observations for the same patients, especially when the patient was in different treatment groups across periods. Secondly, many randomized controlled trials (RCTs) include stringent inclusion/exclusion criteria. We applied some of the patient-selection criteria of the MAT + A-CHESS trial20 that can be defined using our data to assess the robustness of the main-analysis estimates in a selected sub-population who had no acute medical problems with immediate inpatient treatment needs, no history of psychotic disorders, and not pregnant.