Dual Healthcare System Users and Risk of Opioid Use Disorder: A Deep Learning analysis

doi:10.21203/rs.3.rs-4344773/v1

Download PDF

Article

Dual Healthcare System Users and Risk of Opioid Use Disorder: A Deep Learning analysis

https://doi.org/10.21203/rs.3.rs-4344773/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

The opioid crisis has disproportionately affected U.S. veterans, leading the Veterans Health Administration to implement opioid prescribing guidelines. Veterans who receive care from both VA and non-VA providers—known as dual-system users — have an increased risk of Opioid Use Disorder (OUD). The interaction between dual-system use and demographic and clinical factors, however, has not been previously explored. We conducted a retrospective study of 856,299 patient instances from the Washington DC and Baltimore VA Medical Centers (2012–2019), using a deep neural network (DNN) and explainable Artificial Intelligence to examine the impact of dual-system use on OUD and how demographic and clinical factors interact with it. Of the cohort, 146,688(17%) had OUD, determined through Natural Language Processing of clinical notes and ICD-9/10 diagnoses. The DNN model, with a 78% area under the curve, confirmed that dual-system use is a risk factor for OUD, along with prior opioid use or other substance use. Interestingly, a history of other drug use interacted negatively with dual-system use regarding OUD risk. In contrast, older age was associated with a lower risk of OUD but interacted positively with dual-system use. These findings suggest that within the dual-system users, patients with certain risk profiles warrant special attention.

Health sciences/Health care

Health sciences/Risk factors

Deep Neural Network

Explainable AI

Opioid Use Disorder

dual-system use

interaction

Millions of Americans who suffer from chronic and acute pain are prescribed opioid. Prescription misuse and opioid use disorder (OUD), however, have been a grave concern across the U.S. during the past two decades. Between 1999 and 2010, there was a sharp increase in opioid prescribing in the U.S., which has led to a dramatic increase in prescription opioid-related overdose death.¹ Since 2012, tighter regulation has resulted in a steady decline in opioid prescription in most healthcare settings.² OUD, however, did not decline at the same rate.³ Furthermore, opioid-related deaths continue to increase. In 2021 alone, more than 106,000 individuals in the U.S. died from drug-involved overdose with 14,900 extra cases from the year prior.⁴

Similarly, the opioid epidemic has greatly impacted active-military personnel and veterans, reflected by the rising rates of opioid addiction and overdose deaths till 2012.^5,6 In response, the VA has dramatically reduced opioid prescriptions, with only 7.9% of patients receiving these medications in 2021 compared to 22% in 2013.⁷

While significant efforts have been made and progress has been reported in promoting safe opioid use and decreasing opioid-related mortality, a challenge has been the care coordination when patients have access to multiple healthcare systems, or multiple sources of opioid prescriptions. Studies has shown that such fragmented care can leave patients at a higher risk of opioid use and misuse, which may be due to lack of information sharing between healthcare systems.^8,9 This challenge is important for the US Veterans Administration (VA), as many VA enrollees also receive outside care via Veterans Choice Program (VCP)/Veterans Community Care Program (VCCP), which are paid for by the VA. The dual-system care adds another layer of responsibility for VA to understand its impact on opioid use. Our own study confirmed that VA patients who use both VA and community care (dual-system users) are more likely to have opioid initiation, continued opioid prescriptions, and diagnoses of OUD than those who only use VA care.¹⁰ While dual-system users face a heightened OUD risk, little is known about how individual patient factors affect this vulnerability.

Traditionally, differential effects are analyzed using statistical interaction or regression mixture models. The rise of artificial intelligence (AI), especially deep neural network (DNN) models, provided us with a new approach. Literature has shown that when trained on large datasets, DNNs are particularly capable of modeling complex, non-linear relationships without making assumptions of the variable independence or distribution.¹¹ However, since DNN models often have a large number of parameters, they are difficult to interpret and are thus sometimes called black box models. To tackle this problem, our research team has developed and validated an explainable AI method,^12,13 allowing the assessment of an individual feature’s contribution, as well as the interactions between features that are captured by DNN models.

One challenge we face in analyzing OUD using medical record data is that it has been widely reported OUD is often under coded.¹⁴ A number of studies, including one from our team, have developed natural language processing (NLP) methods to identify OUD from clinical notes.^15–20 NLP systems generally consist of either hard-coded rules, or trained machine learning models, or both.¹⁵ The development of NLP tools allows us to capture a fuller extent of the OUD problem.

In this study, we assembled a cohort of veterans from the Washington DC and Baltimore Veteran Affairs (VA) Medical Centers with mono- or dual- system enrollment and evaluated the association of dual-system use status and OUD using a DNN model. OUD was determined through ICD codes or NLP classification of clinical notes. Additionally, we leveraged a novel explainable AI approach to assess the impact of dual-system use on the outcome of OUD and how patient demographic and clinic characteristics interact with dual-system use status to influence the outcome.

Study Population

Our study cohort consists of 222,370 distinct patients who received outpatient care between 2012 and 2019. We required that each patient must have at least two encounters from two consecutive calendar years in the Washington DC or Baltimore VA Medical Center. (VA IRBnet protocol #1607134) Since dual-system user status changes over time, each year is treated as a separate cohort. Patients with multiple and inconsistent dates of birth, gender, race, or ethnicity were excluded. We used January 1st of each cohort year as the index date for retrieving comorbid conditions and outcomes. All data were derived from electronic health records (EHR) from the VA Corporate Data Warehouse (CDW). Research was performed within the secure VA Informatics and Computing Infrastructure (VINCI) platform.²¹

NLP Application

To identify patients with problematic opioid use-related concerns (e.g., current abuse, overuse, or addiction) documented in their clinical notes, we developed an NLP classifier that uses both rule-based and machine learning methods.²⁰ After reviewing sample notes, two project members identified 36 key phrases relevant to problematic opioid use concern documentation. Using snippets (a span of clinical note text containing one of the key phrases), the team developed a support vector machine (SVM) model to identify notes documenting problematic opioid use concerns. Also using snippets, the team built a library of regular expressions matching relevant template data (e.g., “[x] substance abuse and/or dependence”) and relevant standard clinical text (e.g., “current opioid dependence”). Leveraging both the SVM and regular expressions, the NLP classifier achieved 96.6% specificity, 90.4% precision/PPV, 88.4% sensitivity/recall, and 94.4% accuracy on an unseen (i.e. not used in classifier development) snippet dataset.

Predictors

Covariates including dual-system user status, demographic, and clinical variables were derived from the VA Corporate Data Warehouse (CDW) before the index date. We defined dual healthcare system users as those with VCP/VCCP encounters in the calendar year by use of a VA stop code designated for the community care program, or VCP/VCCP note. Patients without any VCP/VCCP encounter in a calendar year were defined as “mono” users. The clinical variables like comorbid diagnoses were defined using the criteria established by the VA MSD cohort¹⁰ and derived from the relevant ICD 9/10 codes. Other drug disorders include cannabis-related disorders, nicotine dependence, cocaine-related disorders, and others (ICD-9: 304.1-304.6,304.8-304.9, 305.2-305.4, 305.6-305.9; ICD-10: F12-F19). Age was measured as a continuous variable and the other covariates were measured as categorical variables.

Study outcomes

The study outcome is OUD within each calendar year. OUD was defined in two ways: using the ICD 9/10 codes of 304.0, 304.7, 305.5, and F11; and the OUD-related issues documented in clinical notes that are extracted by the NLP tool described above.

DNN modeling

The deep neural network (DNN) model is assembled with a variation of the residual network (ResNet) model,²² as shown in Fig. 1. The input variables undergo a linear transformation, followed by two residual blocks. Each residual block consists of two feed-forward layers with layer normalization and ReLU activation, and a skip connection to add the residual from the previous layer. The benefit of the ResNet model is that it is more stable and efficient for training. The final layer is activated with the sigmoid function σ, so that the output value is always between 0 and 1.

The dataset was split into training, validation, and testing sets in a ratio of 64%, 16%, and 20%, respectively. Due to the rarity of the observed outcome, an imbalance between positive and negative instances existed in the dataset. To address this, a down-sampling approach was implemented on the majority class for the training set. Specifically, a randomly selected subset of the controls was selected with 1:1 ratio to the case for training. The validation set and test set remained unchanged representing the original population.

The binary cross-entropy function was used as the loss function, and the area under the ROC curve (AUC) was used as the main metric for measuring the model performance. Stochastic gradient descent was used for optimization. A dropout rate of 0.25 was used for regularization and an early stopping mechanism (i.e. stop the training once the model's AUC on the validation set stops improving) was also adopted to prevent overfitting.

Impact and Impact Scores

Because DNN models are often considered black boxes, our team developed a scoring method to measure the impact of features on model output.^12,23 The output score, defined as the impact score, is similar to the coefficient of a logistic regression model, which measures the association between changes in the predictor value and changes in the output. For binary variables, the score measures the difference between the output when the feature is absent and the output when the feature is present. For continuous variables, the score measures the impact of a one-unit increase in the variable value.

Interaction and interaction score:

Similarly, we designed the interaction score to measure the interaction between two variables.²⁴ Conceptually, we define the interaction between two variables as the residual impact of changes in both variables simultaneously subtracting the impact of each individual variable.

Confidence interval and bootstrapping:

To estimate the confidence intervals of the impact scores and interaction scores of our model, we employed the nonparametric percentile bootstrap method. In this approach, we resampled the original data with replacement 200 times, trained the model, and independently calculated the impact scores and interaction scores. The 95% confidence interval was determined as the range between the 2.5th and 97.5th percentiles of the calculated scores for each feature.

As we evaluated the patient's baseline characteristics and OUD outcomes over the year, we treated the same patients in different years as separate instances in our analysis. Thus, we had a total of 856,299 patient instances from 2012 to 2019. Of these, 17% had an OUD outcome identified either by ICD code or NLP classification. The differences in demographics and selected clinical characteristics are shown in the Table 1. In summary, patients with OUD tend to be younger, more likely to be Black and single, and more likely to have dual-system use, prior opioid use, and other substance use disorders. Using these predictors for OUD outcomes in our DNN model, our model achieved a 78% AUC during training, validation, and testing (Table 2).

Table 1

Demographics of Groups with and without OUD Based on NLP or ICD Codes.
	Opioid Abuse Disorder
	No	Yes
	N = 709,611	N = 146,688	ASD (%)
Age	61.9 ± 17.0	56.8 ± 14.2	32
Dual-system user	54489 (8%)	22982 (16%)	25
Female	99325 (14%)	22916 (16%)	5
Race
Non-Hispanic White	324987 (46%)	46937 (32%)	29
Hispanic	16700 (2%)	3146 (2%)	1
Non-Hispanic Black	278841 (39%)	87740 (60%)	42
Non-Hispanic Other	30933 (4%)	5371 (4%)	4
Unknown	58150 (8%)	3494 (2%)	26
Marital Status
Single	117994 (17%)	35993 (25%)	20
Divorced	142313 (20%)	41879 (29%)	20
Married	347810 (49%)	48335 (33%)	33
Separated	29482 (4%)	12010 (8%)	17
Unknown	22690 (3%)	910 (1%)	19
Widowed	49322 (7%)	7561 (5%)	8
Prior opioid prescription	336428 (47%)	99550 (68%)	42
Comorbidity
Alcohol use disorder	95641 (13%)	62532 (43%)	69
Anxiety	150629 (21%)	58060 (40%)	41
Back Pain	318722 (45%)	87035 (59%)	29
Cancer	122294 (17%)	19947 (14%)	10
Diabetes	212854 (30%)	86098 (59%)	3
Depression	203234 (29%)	43941 (30%)	60
Hypertension	435223 (61%)	92422 (63%)	3
Neck Pain	200018 (28%)	61288 (42%)	29
Other drug disorder	59672 (8%)	59825 (41%)	81
PTSD	121873 (17%)	52920 (36%)	44
TBI	38192 (5%)	15269 (10%)	19
Tobacco use disorder	159410 (22%)	69949 (48%)	55
PTSD = Post-Traumatic Stress Disorder, TBI = Traumatic Brain Injury

Table 2

DNN performances on training, validation and test datasets.
	F1	Accuracy	Precision	Recall	AUC
Training	0.699	0.708	0.72	0.679	0.783
Validation	0.461	0.727	0.349	0.681	0.783
Test	0.462	0.728	0.35	0.68	0.784
HTN = Hypertension; OUD = Opioid Use Disorder; PTSD = Post Traumatic Stress Disorder; TBI = Traumatic Brain Injury

A positive impact score indicates an association with increased risks of OUD, and a negative impact score indicates an association with decreased risks of OUD. It is clear that previous abuse of other drugs is the most significant risk factor. use is also one of the top risk factors (Fig. 2).

A positive interaction score indicates a higher risk than the additive impacts of the two predictors, and a negative interaction score indicates a lower risk. Figure 3 shows that age has a positive interaction with dual-system use. In contrast, races such as Non-Hispanic Black or Non-Hispanic Other, as well as a history of other drug disorders and prior PTSD diagnosis, interact negatively with it.

The explainable AI analyses confirmed that dual-system use is associated with OUD in our cohort. The interaction analysis of OUD with other covariates revealed that the combined impact of dual-system use with a number of conditions (e.g., PTSD, abuse of other drugs) is less than their additive impacts. Conversely, older patients who are dual system users are at a higher risk for OUD.

Interestingly, the predictors with a negative interaction score regarding dual user status tend to be those associated with a positive impact score. Inversely, predictors with a positive interaction score concerning dual-system use are often associated with a negative impact score. This could suggest that dual system use has a more significant impact on those not already at high risk.

This and other previous studies have identified dual system use as a risk factor.¹⁰ Known risk factors such as prior drug abuse has also been confirmed.²⁵ A novel finding of this study is that patients without well-known risk factors, such as other drug abuse, may be more affected by dual user status.

In this study the dual use status was limited to VA-paid community care, while Veterans may seek care outside of the VA independently, which may be considered a limitation. Another limitation is the imperfect nature of OUD identification by NLP, which does not equate to an OUD diagnosis. Nevertheless, our NLP classifier achieved over 94.4% accuracy, as confirmed by medical expert annotation. Within our cohort, 16,852 patient instances had an ICD-based OUD diagnosis, of which 88% were corroborated by NLP. At the same time, the NLP identified almost eight times more patients with OUD conditions. Given the under-coding reality of OUD in EHR,¹⁴ the NLP program provides additional insights into OUD prevalence among Veterans. Our analysis confirmed that dual-system users are more likely to have OUD identified by both ICD and NLP.

For our cohort assembly, we did not exclude patients with existing OUD. In general, patients with a prior OUD diagnosis will be more likely to receive OUD diagnoses in subsequent years.^26,27 In our cohort, 65% of the patients had additional OUD diagnoses after their first OUD diagnoses. Even though relapse is common, many patients eventually recover from OUD.²⁸ We thus did not limit the outcome to first OUD diagnosis.

A future direction we plan to explore is analyzing the underlying causes of interactions that increase OUD risks. In addition, social and community factors play important roles in the prevention of OUD. We hope to include those factors in future analyses.

In conclusion, out analysis underscores the heightened risk of OUD among dual-system users, a finding that echoes that patterns identified in prior research. Notably, our study brings to light that the risk associated with dual-system use is not uniformly distributed across all patient demographics. Specifically, the risk is more pronounced in patients who are not traditionally identified as high-risk such as prior substance disorder. This revelation suggests a nuanced approach to public health interventions, advocating for a focus on seemingly lower-risk population like older patients, who may be more susceptible to the adverse effects of dual-system use.

Conflict of interest statement

We declare that this study was conducted in the absence of any commercial or financial relationships that could be construed as potential conflicts of interest.

Ethics Statement

This study was approved by the Institution Review Boards for the Washington DC VA

Medical Center and West Haven VA.

Author Contribution

QZ-T and JK obtained the funding for the project; QZ-T designed and supervised the experiments; YY performed analyses; TW performed the NLP classification; PM and YC participated in the data analyses; YS provides methodology guidance; WB, FS, and CS provided guidance as clinical experts in pain management and opioid use; JG and CB provided guidance on cohort creation; HMS facilitated data curation from the EHR; JTK, JB, RMA, NS, and AB participated in the results interpretation; All authors contributed to the article and approved the submitted version.

Acknowledgements

This work is supported by VA grant 1I01HX003100-01A2 “Assessing and Reducing Opioid Misuse Among Veterans in VA and Non-VA Systems: Coordination of Fragmented Care.”

Data Availability

The data that support the findings of this study are derived from the Veterans Health Administration’s Clinical Data Warehouse. Restrictions apply to the availability of these data, which were used under license for this study. Data are available to VA researchers,

Guy GP, Jr., Shults RA. Opioid Prescribing in the United States. Am J Nurs 2018;118(2):19–20. DOI: 10.1097/01.NAJ.0000530238.99144.e8.
Jannetto PJ. The North American Opioid Epidemic. Ther Drug Monit 2021;43(1):1–5. DOI: 10.1097/FTD.0000000000000817.
Keyes KM, Rutherford C, Hamilton A, et al. What is the prevalence of and trend in opioid use disorder in the United States from 2010 to 2019? Using multiplier approaches to estimate prevalence for an unknown population size. Drug Alcohol Depend Rep 2022;3. DOI: 10.1016/j.dadr.2022.100052.
Abuse NIoD. Drug Overdose Death Rates. June 30, 2023 (https://nida.nih.gov/research-topics/trends-statistics/overdose-death-rates).
Rhee TG, Rosenheck RA. Comparison of opioid use disorder among male veterans and non-veterans: Disorder rates, socio-demographics, co-morbidities, and quality of life. Am J Addict 2019;28(2):92–100. DOI: 10.1111/ajad.12861.
Bennett AS, Guarino H, Britton PC, et al. U.S. Military veterans and the opioid overdose crisis: a review of risk factors and prevention efforts. Ann Med 2022;54(1):1826–1838. DOI: 10.1080/07853890.2022.2092896.
VA continues to reduce opioid prescribing throughout the pandemic. May 13, 2021 (https://news.va.gov/88759/va-continues-to-reduce-opioid-prescribing-throughout-the-pandemic/#:~:text=VA%20has%20reduced%20opioid%20prescribing,opioid%20prescribing%20overall%20by%2063%25.).
Lagisetty P, Bohnert A, Goesling J, et al. Care Coordination for Patients on Chronic Opioid Therapy Following Surgery: A Cohort Study. Ann Surg 2020;272(2):304–310. DOI: 10.1097/SLA.0000000000003235.
Hinami K, Ray MJ, Doshi K, et al. Prescribing Associated with High-Risk Opioid Exposures Among Non-cancer Chronic Users of Opioid Analgesics: a Social Network Analysis. J Gen Intern Med 2019;34(11):2443–2450. DOI: 10.1007/s11606-019-05114-3.
Goulet J, Cheng Y, Becker W, et al. Opioid use and opioid use disorder in mono and dual-system users of veteran affairs medical centers. Front Public Health 2023;11:1148189. DOI: 10.3389/fpubh.2023.1148189.
Ching T, Himmelstein DS, Beaulieu-Jones BK, et al. Opportunities and obstacles for deep learning in biology and medicine. J R Soc Interface 2018;15(141). DOI: 10.1098/rsif.2017.0387.
Shao Y, Cheng Y, Shah RU, Weir CR, Bray BE, Zeng-Treitler Q. Shedding Light on the Black Box: Explaining Deep Neural Network Prediction of Clinical Outcomes. J Med Syst 2021;45(1):5. DOI: 10.1007/s10916-020-01701-8.
Shao Y, Ahmed A, Zamrini EY, Cheng Y, Goulet JL, Zeng-Treitler Q. Enhancing Clinical Data Analysis by Explaining Interaction Effects between Covariates in Deep Neural Network Models. J Pers Med 2023;13(2)(217). DOI: https://doi.org/10.3390/jpm13020217.
Palumbo SA, Adamson KM, Krishnamurthy S, et al. Assessment of Probable Opioid Use Disorder Using Electronic Health Record Documentation. JAMA Netw Open 2020;3(9):e2015909. DOI: 10.1001/jamanetworkopen.2020.15909.
Blackley SV, MacPhaul E, Martin B, Song W, Suzuki J, Zhou L. Using Natural Language Processing and Machine Learning to Identify Hospitalized Patients with Opioid Use Disorder. AMIA Annu Symp Proc 2020;2020:233–242. (https://www.ncbi.nlm.nih.gov/pubmed/33936395).
Carrell DS, Cronkite D, Palmer RE, et al. Using natural language processing to identify problem usage of prescription opioids. Int J Med Inform 2015;84(12):1057–64. DOI: 10.1016/j.ijmedinf.2015.09.002.
Afshar M, Sharma B, Bhalla S, et al. External validation of an opioid misuse machine learning classifier in hospitalized adult patients. Addict Sci Clin Pract 2021;16(1):19. DOI: 10.1186/s13722-021-00229-7.
Zhu VJ, Lenert LA, Barth KS, et al. Automatically identifying opioid use disorder in non-cancer patients on chronic opioid therapy. Health Informatics J 2022;28(2):14604582221107808. DOI: 10.1177/14604582221107808.
Sharma B, Dligach D, Swope K, et al. Publicly available machine learning models for identifying opioid misuse from the clinical notes of hospitalized patients. BMC Med Inform Decis Mak 2020;20(1):79. DOI: 10.1186/s12911-020-1099-y.
Workman TE, Kupersmith J, Ma P, et al. A Comparison of Veterans with Problematic Opioid Use Identified through Natural Language Processing of Clinical Notes versus Using Diagnostic Codes. Healthcare (Basel) 2024;12(7). DOI: 10.3390/healthcare12070799.
VA Informatics and Computing Infrastructure (VINCI). (https://www.research.va.gov/programs/vinci/).
He KaZ, Xiangyu and Ren, Shaoqing and Sun, Jian. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)2016:770–778.
Shao Y, Ahmed A, Liappis AP, Faselis C, Nelson SJ, Zeng-Treitler Q. Understanding Demographic Risk Factors for Adverse Outcomes in COVID-19 Patients: Explanation of a Deep Learning Model. J Healthc Inform Res 2021;5(2):181–200. DOI: 10.1007/s41666-021-00093-9.
Shao Y, Ahmed A, Zamrini EY, Cheng Y, Goulet JL, Zeng-Treitler Q. Enhancing Clinical Data Analysis by Explaining Interaction Effects between Covariates in Deep Neural Network Models. Journal of Personalized Medicine 2023;13(2):217.
Webster LR. Risk Factors for Opioid-Use Disorder and Overdose. Anesth Analg 2017;125(5):1741–1748. DOI: 10.1213/ANE.0000000000002496.
Binger KJ, Ansara ED, Miles TM, Schulte SL. Relapse rates among veterans on maintenance doses of combination buprenorphine and naloxone for opioid use disorder. Ment Health Clin 2020;10(3):80–84. DOI: 10.9740/mhc.2020.05.080.
Hser YI, Evans E, Grella C, Ling W, Anglin D. Long-term course of opioid addiction. Harv Rev Psychiatry 2015;23(2):76–89. DOI: 10.1097/HRP.0000000000000052.
Strang J, Volkow ND, Degenhardt L, et al. Opioid use disorder. Nat Rev Dis Primers 2020;6(1):3. DOI: 10.1038/s41572-019-0137-5.

No competing interests reported.

Download PDF

Editorial decision: Revision requested
22 Aug, 2024
Reviews received at journal
21 Aug, 2024
Reviewers agreed at journal
17 Jul, 2024
Reviews received at journal
05 Jul, 2024
Reviewers agreed at journal
27 Jun, 2024
Reviewers invited by journal
27 Jun, 2024
Editor assigned by journal
27 Jun, 2024
Editor invited by journal
08 May, 2024
Submission checks completed at journal
05 May, 2024
First submitted to journal
29 Apr, 2024

You are reading this latest preprint version

Dual Healthcare System Users and Risk of Opioid Use Disorder: A Deep Learning analysis

Status:

Version 1

Abstract

Figures

Introduction

Methods

Study Population

NLP Application

Predictors

Study outcomes

DNN modeling

Impact and Impact Scores

Interaction and interaction score:

Confidence interval and bootstrapping:

Results

Discussion

Conclusion

Declarations

Conflict of interest statement

Ethics Statement

Author Contribution

Acknowledgements

Data Availability

References

Additional Declarations

Status:

Version 1