Machine Learning to Develop a Predictive Model of Pressure Injury in Persons with Spinal Cord Injury

doi:10.21203/rs.3.rs-2227699/v1

Download PDF

Article

Machine Learning to Develop a Predictive Model of Pressure Injury in Persons with Spinal Cord Injury

https://doi.org/10.21203/rs.3.rs-2227699/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Study Design: A 5-year longitudinal, retrospective, cohort study.

Objectives: Develop a prediction model based on electronic health record (EHR) data to identify veterans with spinal cord injury/diseases (SCI/D) at highest risk for new pressure injuries (PIs).

Setting: Structured (coded) and text EHR data, for veterans with SCI/D treated in a VHA SCI/D Center between October 1, 2008, and September 30, 2013.

Methods: A total of 4,709 veterans were available for analysis after randomly selecting 175 to act as a validation (gold standard) sample. Machine learning models were created using ten-fold cross validation and three techniques: 1) two-step logistic regression; 2) regression model employing adaptive LASSO; 3) and gradient boosting. Models based on each method were compared using area under the receiver-operating curve (AUC) analysis.

Results: The AUC value for the gradient boosting model was 0.62 (95% CI = 0.54-0.70), for the logistic regression model was 0.67 (95% CI = 0.59-0.75), and for the adaptive LASSO model was 0.72 (95% CI = 0.65-80). Based on these results, the adaptive LASSO model chosen for interpretation. The strongest predictors of new PI cases were having fewer total days in the hospital in the year before the annual exam, being in the highest vs. lower weight categories and most severe vs. less severe grade of injury based on the American Spinal Cord Injury Association (ASIA) Impairment Scale.

Conclusions: While the analyses resulted in a potentially useful predictive model, clinical implications were limited because modifiable risk factors were absent in the models.

Health sciences/Health care

Health sciences/Health care/Health services/Rehabilitation

Community-acquired pressure injuries (PI) in persons who are at high risk have been ubiquitous adverse events for decades. In an analysis of global incidence data of persons with SCI/D who develop PIs (N = 82,722), Chen and colleagues (2020) found an incidence of 0.23 (95% CI, 0.20, 0.26). Incidence was greater in the community (0.26, 95% CI, 0.20–0.32) than the hospital setting (0.22,95% CI, 0.19–0.26). (1) PIs result in serious health consequences for persons with SCI/D including functional impairments, poor perceived health, pain, infection, and death. (2) Consequences to healthcare systems are related to the burden of caring for persons with PI’s including increased length of stay, and rehospitalization. (2) Estimates of treatment cost vary but may range from $500 to $152,000 per individual affected in the United States with a total of $11.6 billion annually. (3)

While clinicians routinely implement PI prevention strategies, some argue that some PIs in persons with SCI/D are not preventable due to underlying pathophysiologic factors that predispose the skin to breakdown. (4)(5) According to the 2019 Prevention and Treatment of Pressure Ulcers/Injuries: Clinical Practice Guideline (3rd edition), PI prevention strategies include preventive skin care, and assessment of skin, mitigating factors, and risk. (3) Large bodies of moderate or strong evidence support recommendations related to the impact of mobility/activity/friction and shear on risk, the association between skin status and PI development, and the prognostic risk factors related to perfusion, circulation, and oxygenation. Moderate statistical associations were found between nutritional status, skin moisture body temperature, serum albumin and hemoglobin levels and some measures of general health status and the development of new PIs. Weak associations with PI development were advanced age, sensory perceptual deficits, and mental health status.

Based on the evidence, the clinical practice guidelines (CPG) recommend a structured approach for risk assessment that includes comprehensive skin assessment, and supplemental use of risk assessment tools and clinical judgment. (3) The need to identify patients at risk, combined with the complexity of patients’ needs and time limitations in busy clinical settings, present challenges to clinicians. Most healthcare systems depend on structured risk assessment tools such as the Braden Scale, Norton Scale, Spinal Cord Injury Pressure Ulcer Scale (SCIPUS), Waterlow, and others. (6) Some have questioned the utility of risk assessment tools and have attempted to examine their predictive validities. Kallman and Lindgren compared the Braden, Norton, Risk Assessment Pressure Sore (RAPS), and Modified Norton scales and found that the first three had acceptable cutoff levels for initiating PI protocols in a general hospital setting, whereas the Modified Norton did not. (7) Flett et al. (8) in a study of 754 hospital admissions, found that a simple measure of mobility (the Functional Independence Measure {FIM} bed/chair transfer score) identified risk with greater accuracy than one SCI-specific measure (SCIPUS) and a non-population specific measure (Braden Scale).

With the evolution of electronic health records and advances in data mining methods, using algorithms to automate risk identification for decision support instead of clinical assessment tools has gained interest. Bogie et al. (9) extracted PI risk data (e.g., comorbidities, demographics, medications, healthcare access, nutritional status) from the EHR of over 36,000 Veterans with SCI/D. Their findings demonstrated proof of concept in extracting risk variables and provided a basis for model development and testing. Others are exploring patient perceived risk of community acquired PIs such as challenges, barriers, and contextual risk factors for a Veteran model of prevention that could be used in clinical practice. (10)

Machine learning (ML) is a branch of computer science that broadly aims to enable computers to “learn” without being directly programmed. (11) In healthcare, ML can be applied to recognize patterns within large quantities of EHR data to make predictions, with successful applications including natural language processing, computer vision, and automatic speech recognition. (12) Recently, ML has been used successfully to develop risk models of outcomes across a number of clinical diagnoses. (13) (14) (15) (16) Machine learning has also been useful to identify PI risk factors in critical care (17), nursing homes (18), surgery (19), and more broadly to develop PI phenotypes for community and hospital acquired PIs (20). A literature review found no published reports of using ML to improve PI prevention and detection in populations with SCI/D.

Applying ML algorithms to EHR data is complex. (12) First, data are extracted, then transformed into features based on domain knowledge using standardization and latent representation. The features are analyzed iteratively by comparing modeling techniques and selecting the best predictive model by comparing their predictive accuracies. However, simple accuracy, which summarizes the proportion of correctly classified outcomes (i.e., PIs), is biased when using highly imbalanced datasets (large numbers of negative vs. positive cases). In such cases, metrics that balance between True Positive Rates (TPR) and True Negative Rates (TNR) are helpful. The area under the receiver operating curve (AUC) is a plot of the TPR on the vertical axis and (1-TNR), also known as the False Positive Rate (FPR), on the horizontal axis is often employed. The AUC assesses the model’s overall diagnostic ability as the decision threshold is varied from 0 to 1.

The goals of the study were to develop a prediction model to synthesize the information available in the EHR and identify veterans at the highest risk thereby allowing clinicians to better target surveillance and preventive efforts. The Veterans Health Administration (VHA) SCI/D System of Care is required to provide an annual preventative exam to veterans with SCI/D. (21) The first annual exam during the study period for a veteran after a minimum of one year free of recorded PIs became the reference or anchor point (see Fig. 1). Predictors were examined in the previous 12 months, and during the one-year follow-up period. (22)

Study Population

This 5-year longitudinal, retrospective, cohort study targeted Veterans with SCI/D treated at one VHA SCI/D Center between October 1, 2008, and September 30, 2013 (FY 2009 - FY 2013). Of the 30,711 Veterans (See Fig. 2) with SCI/D in FY 2009, we identified a 14,892 Veterans who received care in that year based on recorded International Classification of Diseases, Ninth Revision-Clinical Modification (ICD-9-CM) codes. The study was approved by the James A. Haley Veterans Hospital Research and Development Committee and University of South Florida Institutional Review Board.

Analytic Samples

From the initial cohort, patients were excluded if they had a diagnosis of multiple sclerosis (MS) or amyotrophic lateral sclerosis (ALS) ( n = 2,655), had a diagnosed PI based on information in the EHR (ICD-9-CM, n = 1,645 or text, n = 1,452) in FY 2008, had no 12-month period free from PIs prior to an annual exam (n = 5,056) in the study period, or had less than one year of follow-up data available (n = 154). From this cohort (n = 4,884) 175 veterans were randomly selected as a validation sample, leaving 4,709 veterans for the analytic sample.

Outcome Definition

Information from ICD-9-CM codes and text was combined to identify all new PI cases. If a veteran had two or more outpatient ICD-9-CM codes or one inpatient ICD-9-CM code for a PI during the follow-up year, they were considered to have a new PI (n = 124, 2.63). Two text-based strategies were also used to identify the occurrence of a PI, a top-down, rule-base natural language process (NLP) approach and a bottom-up ML approach statistical text mining (STM). The text-based analyses were developed based on a reference set made up of a stratified national sample 1,233 progress notes selected and annotated (specialize chart review) by two trained clinicians, with disagreements adjudicated by a third clinical expert. The NLP algorithm required the presence of a term describing the PI and the assignment of a “Stage II” injury or higher. The STM algorithm included bigrams using gain ratio weighting and 100 SVD dimensions. A C5.0 decision tree was developed to combine the information extracted using NLP and the predicted probability score from STM. Decision trees were a good choice here because these can easily accommodate categorical variables while finding cut points in continuous variables.(23)

Ideally a validation sample would include results from a human review of every progress note written about the veteran during the one-year follow-up period to identify new PI cases. However, this strategy would have required review of tens of thousands of documents most of which were unlikely to contain information about PIs. As an alternative, all progress notes and skin care notes written within two weeks of the annual exam by a provider or wound care specialist during the observation year (n = 5,007) were reviewed and labeled a PI or no PI. This information was summarized at the patient level.

Candidate Predictor Variable (Features) Definitions

We included information from structured data (ICD-9-CM codes, laboratory results, medication use, health factors recorded in the EHR) and from text to identify predictors (Table 1). Predictors included 5 patient demographic factors, 23 clinical conditions, 7 laboratory values, 3 SCI/D factors, 4 outpatient medications, 6 resources that might act as preventive factors, and information about skin moisture from various sources, nutrition, and smoking. Operational definitions for ICD-9-CM codes were based on published phenotype used in the VHA or CMS.

Imputation using means was used to estimate missing values for distance in miles from SCI/D Center (missing = 2.1%), age (missing = 4.7%) and weight (missing = 9.5%). Additionally, the continuous variables including age, years since injury, distance to SCI/D Center, weight, and sum of hospital length of stay in the year prior to the annual visit were transformed using an optimization procedure. In each case, optimal binning proved to be the best strategy with between two (age at annual exam) and four bins (weight). Categories identified using optimal bins were used in both descriptive and model results.

Model Development

Models were created using ten-fold cross validation and three ML analytic techniques: 1) two-step regression in which best predictors were identified using supervised decision tree analysis and analyzed using logistic regression with main effects optimized using the Newton-Raphson with Ridging technique and maximum iterations of 50 with starting and stopping criterion based on significance level 0.05; 2) a regression model employing adaptive least absolute shrinkage and selection operator (adaptive LASSO) with a maximum stopping criterion of 100 steps base on the Bayesian information criterion (BIC); 3) and a gradient boosting model with 50 iterations, shrinkage = 0.1, maximum branches = 5 and maximum depth = 10. The best resultant model based on each method was compared using area under the AUC analysis. The optimal cut point that maximized the difference between the cumulative true positive and cumulative FPR was determined using the Kolmogorov-Smirnov (KS) statistic. (24) The best model was then applied to the validation data set. All analyses were conducted using SAS Enterprise Miner 15.1, SAS Institute Inc. (25)

The analytic cohort (Table 2) was almost exclusively male (96.73%) with a mean age of 57.95 (SD = 13.08) years at the time of the annual exam. The veterans were primarily white (67.59%) and not Hispanic or Latino (91.61%) with the majority living with their nuclear families (58.63%). Approximately half of the veterans were paraplegic (50.99%) with an ASIA grade of D or E (46.00%). About one in five were found to have had a previous PI (22.62%) and were current smokers (19.47%). The veterans lived a mean of 111.85 miles (SD = 119.45) with a range from 0 to 845.17 miles (values of 0 indicate living within the same area as the SCI/D Center).

Using Multiple Sources to Identify PIs

The NLP algorithm to identify PIs had an F-measure of 0.92, a sensitivity of 0.93 and a precision of 0.88. The STM algorithm resulted in an area under the receiver operating curve (AUC) of 0.95, sensitivity of 0.91, and specificity of 0.83 based on the presence two documents with the probability value of .75 within one month. The information from the two text sources were then combined using decision trees which resulted in an F-Measure of .92, a sensitivity of .92, and specificity of .94 for identifying PIs. Combining results from ICD-9-CM and text identified a total of 429 (9.11%) new PIs.

Variable Summary

Predictors with no positive values were excluded from the analysis, including Clostridium difficile, moisture from fecal material, and exudate. Also excluded were predictors with very rare events including tracheostomy (n = 32, 0.68%), lung cancer (n = 20, 0.42%), cancer chemotherapy agents (n = 77, 1.64%), new specialized wheelchairs (n = 3, 0.06%), new specialized bed pads (n = 2, 0.04%), new specialized bed mattresses (n = 6, 0.13%), and pre-albumin levels (n = 8, 0.16%). The remaining 57 predictors were entered into the models. Based on distributions and clinical relevance, ASIA scores were collapsed into three categories by combining ASIA grades B and C and ASIA grades D and E. Highly skewed distributions of the continuous variables were transformed to categorical variables with one large category for lower values including 79.55% (n = 3,746) with weight < = 235.5 lbs., 87.22% (n = 4,107) with age < = 72.5 years, 88.27% (n = 4,157) having < = 10.5 days of hospitalization in the year before the annual exam, and 96.58% (n = 4,484) being injured < = 44.5 years prior to the annual exam and one to three very small categories for larger values. All other predictors were discrete (yes vs. no, normal vs. not normal, etc.).

Model Performance

The AUC value for the gradient boosting model was 0.62 (95% CI = 0.54–0.70), for the logistic regression model was 0.67 (95% CI = 0.59–0.75), and for the adaptive LASSO model was 0.72 (95% CI = 0.65-.80). Based on these results, we retained the LASSO model as our final model. Table 3 describes the standardized estimated effects and odds ratios in the final adaptive LASSO model with 23 predictors (26 unique levels) selected based on having standardized estimates > 0.0. The strongest association with new PI cases related to having fewer total days in the hospital in the year prior to the annual exam with veterans having 10.5 hospital days odds ratio (OR) 0.695 or 10.5 to 194.5 hospital days (OR = 0.760) developing fewer PIs compared to those who had greater than 195.5 days of hospitalization. We also found a relatively strong negative association in the development of PIs with weight with the lowest category ( < = 220.35 lbs., OR = 0.886) and third category (231.65 to 302.25 lbs., OR = 0.890) developing fewer PIs than those in the highest weight group (> 302.25 lbs.). The predictor with the strongest positive association was veterans categorized as ASIA A (OR = 1.137) and ASIA B and C (OR = 1.078) compared to Veterans categorized as ASIA D and E. Other predictors found to have 5% or greater effects include years since injury greater > 44.5 (OR = 1.078), age > = 72.5 years (OR = 1.067) and a new seat cushion in year before the annual exam (OR = 1.057).

Estimation of Performance for Model Cut-Points

Interpretation of a cut point of 0.78 would result in a TNR of 100% while a cut point of 0.02 would result in TPR of 100%. Based on the maximum KS-statistic which identifies the maximum difference between True and False positives a cut point of 0.08 was interpreted resulting in a TPR of 73.19% (314/429) and a TNR of 59.65% (2,553/4,280) while the FPR of 40.35% (115/429) and a FNR of 40.35% (1,727/4280) and an overall classification rate of 60.88% (2,867/4,709).

The model was then applied to the validation sample of 175 veterans. The number of veterans with PIs in this sample was 18/175, or 10.29%. The model correctly identified 61.11% (11/18) of the incident PIs (FPR) with a TNR of 59.87% (94/157). The FPR was 38.89% (7/18) and a FNR of 40.12% (63/157). Overall classification rate was 60.00% (105/175).

The study was an effort to leverage the tremendous amount of information in the VHA EHR to develop risk models for the development of PIs in community-dwelling veterans with SCI/D. The use of NLP to extract data elements was critical to this study. Key elements included in the final model were only available through NLP extraction including ASIA level, provision of seat cushions, extent of injury, history of PI, and moisture from sweat and fecal matter. Combining data extracted from text though both NLP and STM with coded ICD-9-CM improved our ability to identify community acquired PI.

Reliable NLP algorithms were also developed to extract individual elements, sub-scores, and total scores from the Braden PI risk tool. We intended to compare predictions based on the Braden tool and our best model. However, we found that while most of the veterans in our cohort had a Braden score within one year prior to the annual exam approximately only one in four (n = 1,384, 29.26%) had this score documented within two weeks of the annual exam making direct comparisons impossible.

Alba et al. reported a generally accepted approach in clinical risk model development to interpret AUCs of less than 0.60 as having poor discrimination; 0.60 to 0.75, possibly helpful discrimination; and more than 0.75, clearly useful discrimination. (26) Within this context, our best fitting model AUC of 0.72 (95% CI = 0.65-.80) fall into the possibly helpful level of discrimination. However, the authors remind us that a model can have excellent discrimination and still provide misleading absolute risks. In clinical applications of ML models intended to guide care, it is essential to evaluate the trade-off between identification of true versus false positives. Typically curves that plot the number of positives generated at different decision thresholds against the sensitivity of the model are reviewed to choose a “cut-point” for model interpretation. A variety of options are available to aid in this interpretation depending on the clinical goal.

Our strategy to calibrate our results was to select cut points was based on maximizing the difference between true positive and false positive cases. This strategy comes at the cost of higher FPR and potentially higher cost of implementation of prevention strategies. Implementation of our findings in clinical practice might warrant a more measured approach. For example, modifying the cut point from 0.08 to 0.09 would reduce the TPR from 73.19% (314/429) to 61.54% (264/429) but it would also reduce the FNR from 40.35% (1,727/4,280) to 29.93% (1,281/4,280). Given the low prevalence of new PI cases and the potential cost of prevention strategies this might be a warranted implementation strategy.

The strongest associations between predictors and the development of PIs were related to the lowest vs. the highest category of hospitalization in the year prior to the annual exam with odds ratios of 0.695 for veterans with fewer than 10.5 days in the hospital. Long hospitalizations are typically associated with serious medical conditions and likely decline in overall health. While this study aimed to describe risk of development of PIs in community dwelling veterans, there were 30 veterans with greater than 194.5 days spent in the hospital. This reflects the unique nature of care in the VHA system which provides seamless transition from short-term to long-term care. We decided to model these long stays rather than exclude them because we believe this better reflected current care practices.

The other strong predictors were higher ASIA grades (A vs D&E and A vs B&C), lower weight, and age measured both as nominal age and as years since injury. In the cases of age and weight categories were made up of a small number of veterans in the extreme tails of the distribution, thus providing the relevant comparison groups in the model. Taken together, this suggests that increased age and disease burden were the major drivers of risk for the development of PIs. Other clinical factors such as lab values and co-morbid disease had smaller and sometimes counterintuitive standardized effects. For example, the provision of preventive seat cushions was positively associated the development of PIs.

Several factors need to be considered while interpreting these results. First, to maximize model results we allowed all variables with standardized effect to be retained in the model. The LASSO penalization factor selects important predictors by shrinking coefficients for weaker predictors toward zero, excluding predictors with estimated zero coefficients from the final sparse prediction model. This is a standard practice in LASSO analyses but leads to predictors that would be eliminated in traditional regression techniques to be retained. Second, it must be emphasized that this is a predictive and not explanatory model.

Limitations

First, because of the low incidence in population, many false positives would be found and targeted for potentially unnecessary prevention strategies, thus increasing cost to the system. Second, our strategy chose to focus on a mandated annual preventive exam as an anchor point, synthesizing information from the previous year for modeling and the following year for follow-up. We sought to automate and model information that might be available to clinicians at the time of the annual exam. Models that measure dynamic changes in predictors over smaller increments of time might improve predictions. Finally, our results are limited by the use ICD-9-CM codes available during the study period. The potential impact of this factor is mitigated by the use of standardized operational definitions published by the VHA (27) when they were available and by CMS published lists. (28)

To our knowledge, this study is the first to employ ML techniques to model PI risk in a clinical group at very high risk. While the analyses resulted in a potentially viable model, clinical practice implications were limited because modifiable risk factors were absent in the models. Findings reinforced the assertion that some PIs in persons with SCI/D may be unavoidable.

Conflict of Interest: The authors declare no conflict of interest.

Author Contribution Statement: SL was the study principal investigator and responsible for overall study design, execution of the study, data analysis and the manuscript. ST and SS were clinical experts who assisted in conceptualizing the study, identifying, and operationalizing variables and identifying clinical implications. DF, JM, PT, LB, and WB assisted in data management and data analysis. BH was the study project manager and administered day to day aspects of project. RH assisted with analysis of laboratory data, MM advised SL in data analysis, particularly in model development. GP participated in conceptualization of the study and assisted SL in writing the manuscript. All authors reviewed and edited the manuscript.

Funding Statement: This work was supported by a grant awarded from the Health Services Research and Development Service, Veterans Health Administration (IIR 10-169). The views expressed here are those of the authors and do not represent the official policy or position of the Department of Veterans Affairs or the United States government. The authors have no conflict of interests to declare.

Chen HL, Cai JY, Du L, Shen HW, Yu HR, Song YP, et al. Incidence of Pressure Injury in Individuals with Spinal Cord Injury: A Systematic Review and Meta-analysis. J Wound Ostomy Continence Nurs. 2020 Jun;47(3):215–23.
Hogaboom NS, Worobey LA, Houlihan BV, Heinemann AW, Boninger ML. Wheelchair Breakdowns Are Associated with Pain, Pressure Injuries, Rehospitalization, and Self-Perceived Health in Full-Time Wheelchair Users with Spinal Cord Injury. Arch Phys Med Rehabil. 2018 Oct;99(10):1949–56.
European Pressure Ulcer Panel, National Pressure Injury Advisory Panel, Pan Pacific Pressure Injury Alliance. Prevention and Treatment of Pressure Ulcers/Injuries. EPUAP/NPIAP/PPPIA; 2019.
Edsberg LE, Langemo D, Baharestani MM, Posthauer ME, Goldberg M. Unavoidable pressure injury: state of the science and consensus outcomes. J Wound Ostomy Continence Nurs. 2014 Aug;41(4):313–34.
Groah SL, Schladen M, Pineda CG, Hsieh CHJ. Prevention of Pressure Ulcers Among People with Spinal Cord Injury: A Systematic Review. PM R. 2015 Jun;7(6):613–36.
Moore ZE, Patton D. Risk assessment tools for the prevention of pressure ulcers. Cochrane Database Syst Rev. 2019 Jan 31;1:CD006471.
Källman U, Lindgren M. Predictive validity of 4 risk assessment scales for prediction of pressure ulcer development in a hospital setting. Advances in skin & wound care [Internet]. 2014 Feb [cited 2022 Oct 17];27(2). Available from: https://pubmed.ncbi.nlm.nih.gov/24440864/
Flett HM, Delparte JJ, Scovil CY, Higgins J, Laramée MT, Burns AS. Determining Pressure Injury Risk on Admission to Inpatient Spinal Cord Injury Rehabilitation: A Comparison of the FIM, Spinal Cord Injury Pressure Ulcer Scale, and Braden Scale. Arch Phys Med Rehabil. 2019 Oct;100(10):1881–7.
Bogie KM, Roggenkamp SK, Zeng N, Seton JM, Schwartz KR, Henzel MK, et al. Development of Predictive Informatics Tool Using Electronic Health Records to Inform Personalized Evidence-Based Pressure Injury Management for Veterans with Spinal Cord Injury. Mil Med. 2021 Jan 25;186(Suppl 1):651–8.
Burkhart L, Skemp L, Siddiqui S. Veteran model of preventing community-acquired pressure injuries associated with spinal cord injury: A qualitative descriptive study. J Spinal Cord Med. 2021 Dec 2;1–15.
Bi Q, Goodman KE, Kaminsky J, Lessler J. What is Machine Learning? A Primer for the Epidemiologist. Am J Epidemiol. 2019 Dec 31;188(12):2222–39.
Shamout F, Zhu T, Clifton DA. Machine Learning for Clinical Outcome Prediction. IEEE Rev Biomed Eng. 2021;14:116–26.
Davis SE, Lasko TA, Chen G, Siew ED, Matheny ME. Calibration drift in regression and machine learning models for acute kidney injury. J Am Med Inform Assoc. 2017 Nov 1;24(6):1052–61.
Koola JD, Ho S, Chen G, Perkins AM, Cao A, Davis SE, et al. Development of a national Department of Veterans Affairs mortality risk prediction model among patients with cirrhosis. BMJ Open Gastroenterol. 2019;6(1):e000342.
Sánchez-Cabo F, Rossello X, Fuster V, Benito F, Manzano JP, Silla JC, et al. Machine Learning Improves Cardiovascular Risk Definition for Young, Asymptomatic Individuals. J Am Coll Cardiol. 2020 Oct 6;76(14):1674–85.
Stark GF, Hart GR, Nartowt BJ, Deng J. Predicting breast cancer risk using personal health data and machine learning models. PLoS One. 2019;14(12):e0226765.
Alderden J, Pepper GA, Wilson A, Whitney JD, Richardson S, Butcher R, et al. Predicting Pressure Injury in Critical Care Patients: A Machine-Learning Model. Am J Crit Care. 2018 Nov;27(6):461–8.
Lee SK, Ahn J, Shin JH, Lee JY. Application of Machine Learning Methods in Nursing Home Research. Int J Environ Res Public Health. 2020 Aug 27;17(17):E6234.
Cai JY, Zha ML, Song YP, Chen HL. Predicting the Development of Surgery-Related Pressure Injury Using a Machine Learning Algorithm Model. J Nurs Res. 2020 Dec 21;29(1):e135.
Song W, Kang MJ, Zhang L, Jung W, Song J, Bates DW, et al. Predicting pressure injury using nursing assessment phenotypes and machine learning methods. J Am Med Inform Assoc. 2021 Mar 18;28(4):759–65.
Veterans Affairs, SCI System of Care [Internet]. [cited 2022 Oct 17]. Available from: https://www.sci.va.gov/VAs_SCID_System_of_Care.asp
Luther SL, Thomason SS, Sabharwal S, Finch DK, McCart J, Toyinbo P, et al. Leveraging Electronic Health Care Record Information to Measure Pressure Ulcer Risk in Veterans with Spinal Cord Injury: A Longitudinal Study Protocol. JMIR Res Protoc. 2017 Jan 19;6(1):e3.
Quinlan JR. Decision trees and decision-making. IEEE Transactions on Systems, Man, and Cybernetics. 1990 Mar;20(2):339–46.
Pruitt R. 146-2008: Using SAS® Enterprise Miner^TM to Prescribe a Pre-Screen Mailing. SAS Global Forum. 2008;6.
Data Mining Software, Model Development and Deployment, SAS Enterprise Miner [Internet]. SAS® Enterprise Miner^TM. 2018 [cited 2022 Oct 17]. Available from: https://www.sas.com/en_us/software/enterprise-miner.html
Alba AC, Agoritsas T, Walsh M, Hanna S, Iorio A, Devereaux PJ, et al. Discrimination and Calibration of Clinical Prediction Models: Users’ Guides to the Medical Literature. JAMA. 2017 Oct 10;318(14):1377–84.
VHA Support Service Center Capital Assets (VSSC) | Department of Veterans Affairs Open Data Portal [Internet]. [cited 2022 Oct 17]. Available from: https://www.data.va.gov/dataset/VHA-Support-Service-Center-Capital-Assets-VSSC-/2fr5-sktm
Centers for Medicare and Medicaid Services. 2022 ICD-10-CM | CMS [Internet]. [cited 2022 Oct 17]. Available from: https://www.cms.gov/medicare/icd-10/2022-icd-10-cm

Table 1

Predictor Variables
Variables	EHR Source Data	Definition	Level
Demographics
Gender	SD¹	As recoded in EHR	Male Female
Race	SD	Collapsed for analysis²	White Non-white/unknown
Ethnicity	SD	Collapsed for analysis	Hispanic or Latino Non-Hispanic or Latino/unknown
Living Situation	Text/NLP	Collapsed for analysis	Alone Nuclear Family Other/Unknown
Distance From VHA SCI/D Center	Calculated based on GPS data	In miles	Total miles
Clinical Conditions
Anemia, Alcohol Abuse, Anxiety, Autonomic Dysreflexia, Chronic Renal Failure, Cognitive Function, Chronic Obstructive Pulmonary Disease, Cerebral Vascular Accident, Depression, Diabetes, Deep Vein Thrombosis, Fractures, Heart Disease, Heterotrophic Ossification, Hypotension, Lung Cancer, Any Malignancy, Obesity, Pain, Pneumonia, Post Traumatic Stress, Disorder, Peripheral Vein Disease, Substance Abuse, Traumatic Brain Injury, Tracheostomy	SD	Two outpatient or one inpatient ICD-9-CM codes during the year base on VHA sources including VSSC	Recorded condition Yes/No
Laboratory Results
Albumin, Creatinine, C-Reactive, Protein, C-Difficile, Pre-Albumin, Erythrocyte Sedimentation Rate, Urea Nitrogen, Anemia	SD		Classified as low or high
SCI/D Factors
ASIA Impairment Scale	Text/NLP	Collapsed for analysis⁴	Classified as A, B, C, D, E
Extent of Injury	Text/NLP	Collapsed for analysis⁴	Classified as Paraplegic or Quadriplegic
History of Pressure Injury	Text/NLP	As recorded in EHR	Yes or No
Medications
Benzodiazepines, Bowel Care Medications, Cancer Chemotherapy Agents, Steroids	SD	Recorded medications dispensed from VHA pharmacy	Yes or No based on at least one occurrence
Resources Provided
Attendant Care, Bowel Care, Home Health Care, Specialized Wheelchair, Specialized Seat Cushions, Bed Pads, Specialized Mattresses, Protective Padding	SD	CPT/HCPCS	Yes or No based on presence of code.
Other
Recorded skin moisture from Blood, Exudate, Fecal Matter, Sweat, Urine	Text/NLP	Recorded as free text in progress notes	Moisture Yes or No
Malnutrition	Text/NLP	Recorded as free text in progress notes. Collapsed for Analysis⁴	Mild Moderate/Severe Normal
Smoking Status	SD	Recorded in health factor table. Collapsed for analysis⁵	Current Smoker Not Current Smoker

Structured data (coded directly in EHR)
Unknown categories less than 5% collapsed
Extracted from text with NLP
Collapsed for analysis based on distribution and clinical judgment
Collapsed from multiple categories used across Health Factor Tables

Table 2

Patient Demographics
Patient Characteristics	N	%
Gender
Male	4,555	96.73
Female	154	3.27
Race
White	3,183	67.59
Non-White/Unknown	1,526	32.41
Ethnicity
Not Hispanic or Latino	4,314	91.61
Hispanic or Latino	395	8.39
Living Situation
Nuclear Family	2,761	58.63
Alone	1,111	23.50
Other/Unknown	837	17.77
ASIA Impairment Scale
A	1,531	32.51
B, C	1,012	21.49
D, E	2,166	46.00
Level of Injury
Paraplegic	2,401	50.99
Tetraplegic	2,308	49.01
History of Pressure Injury
Yes	1,065	22.62
No	3,644	77.38
Current Smoker
Yes	917	19.47
No	3792	80.53

Table 3

Results of Adaptive LASSO Model
Predictor	Standardized Estimates¹	Odds Ratio
Hospital days in year before annual exam less than 10.5 days	-0.364	0.695
Hospital days in year before annual exam from 10.5 -194.5	-0.275	0.760
ASIA A vs ASIA D&E	0.128	1.137
Weight less than or equal to 220.35 lbs.	-0.121	0.886
Weight 231.65-302.25 lbs.	-0.116	0.890
ASIA B&C vs ASIA D&E	0.075	1.078
Years since injury greater than 44.5	0.068	1.070
Age 72.5 or greater	0.065	1.067
New seat cushion in year before annual exam	0.055	1.057
Hemoglobin – Low	0.040	1.041
Hispanic or Latino	-0.037	0.964
Paraplegia vs Quadriplegia	0.029	1.029
Septicemia – Yes	-0.026	0.974
Peripheral Vascular Disease (PVD) – Yes	0.025	1.025
Prealbumin – Low	0.021	1.021
History of Pressure Injury– Yes	-0.021	0.979
Post-Traumatic Stress Disorder (PTSD) – Yes	-0.020	0.980
Heterotrophic Ossification (HO)– Yes	0.019	1.019
Fecal Moisture - Yes	0.015	1.015
Anemia – Yes	-0.014	0.986
C-reactive Protein – High	0.013	1.103
Benzodiazepines – Yes	0.008	1.008
Obesity – Yes	-0.008	0.992
Home Health Aid Provided – Yes	0.005	1.005
Sweat Moisture – Yes	0.004	1.004
Deep Vein Thrombosis – Yes	-0.003	0.997
_INTERCEPT_	0.000	1.000
1. Ordered by absolute value of effect

There is no duality of interest

Download PDF

Editorial decision: revise
28 Apr, 2023
Review #3 received at journal
26 Feb, 2023
Reviewer #3 agreed at journal
13 Feb, 2023
Review #1 received at journal
16 Jan, 2023
Reviewer #2 agreed at journal
08 Jan, 2023
Reviewer #1 agreed at journal
04 Jan, 2023
Reviewers invited by journal
24 Dec, 2022
Submission checks completed at journal
02 Nov, 2022
Editor assigned by journal
01 Nov, 2022
First submitted to journal
01 Nov, 2022

You are reading this latest preprint version

Machine Learning to Develop a Predictive Model of Pressure Injury in Persons with Spinal Cord Injury

Status:

Version 1

Abstract

Figures

Background

Methods

Study Population

Analytic Samples

Outcome Definition

Candidate Predictor Variable (Features) Definitions

Model Development

Results

Using Multiple Sources to Identify PIs

Variable Summary

Model Performance

Estimation of Performance for Model Cut-Points

Discussion

Limitations

Conclusion

Declarations

References

Tables

Additional Declarations

Status:

Version 1