This study presents a set of machine learning models for predicting the risks of AE after taking NSAIDs using data from PBS and HMDC in Western Australia. We focused specifically on elderly patients (≥ age 65) who took at least one NASID. The prediction is based on the features including age, sex, medication history and disease history, which are widely concerned and counted in clinical practice. This approach encompasses a wide array of patients to truly reflect the population of patients taking NSAIDs in Western Australia. The machine learning based predictive models for AE showed greater sensitivity, specificity and AUC-ROC versus the classical cox-regression approach and GBM presented the best predictive performance in machine learning models we tested.
Several studies have reported the risk of AE with NSAIDs and the Rofecoxib was withdrawn from the market due to its increased risk of CV. The models are built to predict CV-related AE, death and overall AE. The performance of predicting death is the best with AUC-ROC values range from 0.67 to 0.81. This does not mean that the death was caused by NSAIDs, but this demonstrates that the predictive models built based on PBS and HMDC work well and can predict the risk of death.
NSAIDs include a series of medicines. Experimental data includes all NSAIDs with more than 100 patients. The AUC-ROC values of risk prediction for different NSAIDs range from 0.60 to 0.88. The proposed GBM model can be used to predict the risk of AE after taking any NSAID, especially for the Rofecoxib and Celecoxib whose AUC-ROC is more than 0.8.
Machine learning models have been widely used on EMR for prediction purpose, such as nationwide cohort predicting suicide death33, prediction of graft survival in kidney transplant recipients34, risk prediction of AEs following spine surgery5. These studies found that the machine learning approach did not show better performance than a classical generalised regression approach. However, in our data machine learning models tend to perform better over the cox-regression model. This could because most of the input feature in our model are continuous variables and machine learning models turn out to be outperformed on complex variables.
To our knowledge, there is no literature that explores the predictive models for AE taking NSAIDs. Our current study is the first to realise risk prediction in patients with NSAIDs. This study has several strengths. The risk prediction model we developed can be used to avoid some ADRs of NSAIDs. This model can inform patients counselling by enabling doctors to prescribe NASID with the lowest risk based on individual patient’s medication history and disease history. Moreover, this model can also be also used in EMR dataset to pick up the patients with a high risk of AE and help the hospital to pay close attention to these patients.
Our study found that the inclusion of demographic features such as marital status, indigenous ethnicity from linked HMDC improved the performance of AE prediction. The average AUC was similar while predicting ACS (AUC 0.71). But the performance was higher while predicting patients’ risk of all-cause mortality (AUC 0.81 vs 0.84) and composite outcome (AUC 0.77 vs 0.78), and no overlap between their confidence intervals. Previous studies have confirmed marital status is associated with cardiovascular outcomes and mortality was higher in unmarried population35, 36. Studies have also shown that indigenous Australian had a greater rate of cardiovascular disease and death. 37, 38
We extracted additional features from HMDC, including patients’ previous length of stay (days) of each comorbidity in the hospital and the number of days patients spent in intensive care units (ICU) before their last supply. This set of features were presented as continuous variables. We included this set of features to test whether it would improve the AE risk prediction. However, there were no performance gains by adding continuous variables such as length of hospital stay of previous comorbidities and days in ICU before their last NSAIDs supply. The AUCs of all different outcomes were similar. We dropped these features to reduce model complexity.
In our study, we observed minimal performance improvement when using binary variables indicating the presence or absence of previous comorbidities or the use of specific drugs. However, ML models achieved better performance than cox regression when we were using continuous variables to present patients use of different medications and their comorbidity history. This may be because machine learning approaches do not assume linearity for a predictor-outcome association; they are more adept at generating predictions based on continuous variables39.
Our machine learning model ranked COX-2 inhibitors higher among NSAIDs in ACS risk prediction. Multiple previous studies have reported an increased risk of CV events from the use of selective COX-2 inhibitors. 11, 15, 17–21. Rofecoxib was withdrawn from markets based on evidence that showed an increased risk of ACS. 21 Previously study has also combined heart failure has substantially increased the risk mortality. 40 This verifies that our machine learning model is reliable in ranking feature importance.
Despite the merits of this study, there are some limitations. As with all administrative database studies, this study relies on the accuracy of administrative coding of procedures, diagnoses, and records. The PBS dataset did not include all dispensing supplies of NSAIDs such as ibuprofen, as it is also available at the counter. Moreover, PBS dataset did not contain information about the actual drug dosage. Hence, in our study, we calculated the total number of supplied scripts rather than the dose used. In this study, we used state-level linked data to predict patients AE after their NSAIDs supply. The models can be further extended on national linked data in the future. Also, for general applicability, the models can be potentially extended to other drugs or drug groups on different outcomes, and this can be tested in future studies.
Implementing ML models on linked administrative data, including pharmacy claims (e.g. PBS), morbidity, and mortality has the potential to identify patients supplied with NSAIDs that may have a high risk of adverse outcomes. These can then be monitored closely by humans. Further investigation of additional data is required to validate the ML prediction performance on patients’ risk of CV adverse events using population-level linked data.