Early Identi cation of Loss to Follow-Up in Orthopaedic Trauma Patients: Development of a Multivariable Risk Prediction Model


 Background: Loss to clinic follow-up is common among orthopaedic trauma survivors. The purpose of this study was to develop a prediction tool to identify patients at risk for orthopaedic trauma clinic follow-up non-adherence.Methods: Comprehensive social determinants of health (SDOH) assessment surveys were administered to adult patients (age ≥18) who were hospitalized with orthopaedic trauma injuries at an urban Level 1 trauma center. Clinic follow-up adherence within the 90-day post-operative period was examined using adherence fractions ([number of attended follow-up visits]/[number of attended follow-up visits + number of missed visits]). Adherence fractions ≥0.75 were considered to be “High” and £0.75 considered to be “Low”. Demographic and SDOH factors, including the Distressed Communities Index (DCI), were analyzed for their association with clinic follow-up adherence. A risk prediction tool for follow up non-adherence was developed using a multivariable logistic regression model.Results: 294 patients were included for final analysis. Higher community distress, more severe injury, lack of private insurance, lower education levels, no primary care physician, financial and relationship instability, and lack of transportation were significantly associated with low clinic follow up adherence (p<0.05, Table 2 and 3). Low clinic adherence (£0.75) was also significantly associated with presentation to the emergency department within the 90-day post-operative period (p<0.01). The final risk prediction model included 5 covariates: “Distressed” or “At Risk” DCI levels, lack of private insurance, high school or lower education, no primary care physician, and male gender (n=210, AUROC=0.65, 95% CI = 0.57-0.72). The maximum possible risk score was 8. The mean score for patients with low clinic adherence was 4.0±1.6 and 3.1±1.75 for those with high adherence (p<0.01).Discussion: Coordinated care of patients in the aftermath of trauma is imperative to improve healthcare quality and patient outcomes. This study suggests that post-trauma clinic adherence may be predicted at the time of hospital discharge based on a combination of five demographic and SDOH risk factors. We offer a predictive tool for such behavior, which we visualize as a valuable component in a social work discharge plan. Future studies should assess interventions for patients at high risk for follow up non-adherence.

The maximum possible risk score was 8. The mean score for patients with low clinic adherence was 4.0±1.6 and 3.1±1.75 for those with high adherence (p<0.01).
Discussion: Coordinated care of patients in the aftermath of trauma is imperative to improve healthcare quality and patient outcomes. This study suggests that post-trauma clinic adherence may be predicted at the time of hospital discharge based on a combination of ve demographic and SDOH risk factors. We offer a predictive tool for such behavior, which we visualize as a valuable component in a social work discharge plan. Future studies should assess interventions for patients at high risk for follow up nonadherence. Background: Loss to follow-up is a common problem in the orthopaedic trauma patient population. Nonadherence creates challenges for the orthopaedic trauma surgeon, for the healthcare system, and for the clinical researcher. From a clinical perspective, follow-up visits are pivotal for monitoring postoperative recovery and adjusting treatment protocol. Patients who miss their appointments are at higher risk for poor outcomes and more likely to present to the emergency department for postoperative complications (1,2).
In addition to the harm this causes the patient, these behaviors also place nancial stress on the healthcare system by unnecessarily increasing resource utilization and cost of care leading to limited quality of healthcare delivery (3,4). Furthermore, follow-up nonadherence also restricts the validity and effectiveness of orthopaedic trauma outcomes research by biasing study results, which in turn, slows the clinical advancement of the eld (5,6). It is abundantly clear that increasing clinic follow-up in this patient population serves to bene t both the patient and the healthcare system as a whole.
Early anticipation and identi cation of trauma patients at risk for low clinic adherence is one approach by which health care teams can address this issue before it materializes. Doing so would allow healthcare providers to coordinate post-discharge trauma care and facilitate clinic follow up for at-risk patients (7).
Though limited, some studies have investigated risk factors for nonadherence in orthopaedic trauma patients and have demonstrated that a wide array of sociodemographic and clinical factors, including insurance status, drug use, distance to clinic, and fracture type, place these patients at signi cant risk(8-12). To our knowledge, no prior work has used this information to create an actionable model that can identify patients at risk for clinic nonadherence prior to hospital discharge. The purpose of this study was to build upon the existing literature surrounding risk factors for nonadherence as well as develop a prediction tool that would help identify patients at risk for loss to follow-up based on a comprehensive social determinants of health assessment at the time of trauma hospital discharge. Methods:

Patient Population
Institutional review board approval was obtained for this study. Informed consent was obtained via an electronic consent form. There was no form of compensation for participation. Study participants were recruited during initial inpatient admission from the emergency department of a high-volume, urban Level 1 trauma center from May 2018 -August 2019. Inclusion criteria were patient age over 18 years and at least one orthopaedic injury requiring orthopaedic traumatologist consultation. Exclusion criteria included patients who were unable to consent to study participation for any legal or medical reasons.

Study Design
This was a prospective study with data collected from a comprehensive social determinants of health survey administered by study personnel and completed by patients during their initial hospital stay.
Additional data collection included evaluation of the Electronic Medical Record (EMR) for follow-up adherence as well as 1-year follow up phone screenings. Follow up adherence data was collected retrospectively. Surveys were comprised of over 150 social determinant of health factors, which were adapted from standardized questionnaires developed by the Centers for Disease Control (CDC) (13,14), National Institute of Health (NIH) (15), and World Health Organization (WHO)(16). Surveys included but were not limited to the following variables: socioeconomic status, race/ethnicity/cultural context, gender/sexual identity, social relationships, environment, psychiatric illness, and substance use. Patient health questionnaire-2 (PHQ-2), primary care -post-traumatic stress disorder (PC-PTSD), and the modi ed Frailty Index (mFI) were used to screen for depression, post-traumatic stress disorder, and additional medical co-morbidities (17)(18)(19). Injury Severity Score (ISS) and Distressed Community Index scores were also collected (20,21).

Outcome Measurements
The primary outcome of interest was follow-up adherence within the 90-day post-operative period. Adherence was evaluated using adherence fractions, de ned as: ([number of attended follow-up visits] / [number of attended follow-up visits + number of missed follow-up visits]). Adherence fractions ≥ 0.75 were considered to be "High" and < 0.75 considered to be "Low", as done previously (22). Cancelled appointments were not used in the calculation of the compliance fraction. SDOH variables drawn from the aforementioned surveys were then used to evaluate variables associated with "High" versus "Low" adherence. Participants could skip any survey question and missing data was not imputed. Thus, the denominator for each question is based on the number of respondents to that question and not the entire cohort.

Statistical Analysis
RStudio Version 1.2.5402 was used for all statistical analyses (23). Within-group means and standard deviation were calculated for all demographic data. Univariate analysis was used to examine the associations between demographic parameters, injury factors, SDOH factors, and follow up adherence.
Student t-tests were used for continuous variables and chi-square tests were used for categorical variables with an alpha level of 0.05. Multivariable logistic regression models were conducted using both forward and backward selection approaches. The best t multivariable logistic regression model was selected by comparing the Area Under the Receiver Operating Characteristics (AUROC) and Akaike information criterion (AIC) of each model. Patients with missing information were excluded from the model. The selected model was then used to develop a novel risk prediction tool based on weights assigned similar to the Charlson co-morbidity methods (24). Covariates were chosen based upon their statistical signi cance and clinical accessibility, that is: variables easily accessible either via (1) chart review or (2) Table 1). For the low follow-up adherence cohort, the average age was 42.3 years ± 17.5 with 31 (32%) female patients, 66 (68%) male patients, and 3 patients with missing gender data ( Table 1). The mean DCI score was 61.6 ± 24.4 (Table 1). For the high follow-up adherence cohort, the average age was 45.4 years ± 17.6 with 82 (43%) female patients, 107 (57%) male patients, and 5 patients with missing gender data ( Table 1). The mean DCI score was 53.4 ± 27. There was a signi cant difference between mean DCI scores in low versus high adherence groups, indicating lower follow-up in patients from more economically distressed zip codes (p = 0.01, Table 1).  Table 3), all tested substance use (see Table 4), living change (p = 0.19) and crime conviction (p = 1.00).   A best t logistic regression model was created with variables chosen based on the univariate analyses and clinical accessibility. Parameters included in the model were DCI ("Distressed" or "At Risk"), lack of private insurance, high school or lower education level, lack of PCP, and male gender (n = 210, AUROC = 0.65, CI = 0.57-0.72). There was a total of 84 non-respondents to at least one of the questions included in the model. Lack of private insurance was found to be a statistically signi cant predictor for follow up nonadherence, imparting an odds ratio of 2.0 (CI = 1.05-3.99) for follow up nonadherence compared to patients with private insurance coverage (p = 0.04, Table 5). "Distressed" or "At Risk" DCI status and male gender also increased the odds of follow up nonadherence by a factor of 1.8 (CI = 0.93-3.23) and 1.9 (CI = 1.00-3.82) respectively, however these ndings were not signi cant (p = 0.07 and 0.05 respectively, Table 5). High school or lower education level (OR = 1.2, CI = 0.59-2.40, p = 0.64) and lack of a PCP (OR = 1.3, CI = 0.70-2.52, p = 0.38) did not largely impact follow up nonadherence and were not statistically signi cant predictors (Table 5). Table 5 presents the point scores for all the parameters used in the logistic regression model. The maximum possible risk score was 8. There was a statistically signi cant difference between the mean score for patients with low adherence (4.0 ± 1.6) versus those with high clinic adherence (3.1 ± 1.75, p < 0.01). Discussion: Follow-up nonadherence is common in the orthopaedic trauma clinic and creates challenges for multiple players in the healthcare eld. Orthopaedic trauma surgeons are unable to make adjustments to postoperative treatment plans, trauma outcomes research is subject to bias, and patients are at higher risk for poor outcomes and more likely to present to the emergency department for treatment of postoperative complications. These behaviors also increase resource utilization and cost of care, and subsequently, limit the quality of future healthcare delivery. Early anticipation and identi cation of trauma patients at risk for low clinic adherence is one method by which healthcare providers can address this issue before it materializes. The purpose of this study was to identify risk factors for follow-up nonadherence in orthopaedic trauma patients, and then to use these risk factors to build a predictive model that will assist healthcare providers in identifying at-risk patients prior to hospital discharge.
The results of our study demonstrate that there is an array of sociodemographic and clinical factors associated with follow-up nonadherence ( Table 2, 3, and 4). We found that higher DCI scores, more severe injury, lack of insurance, government insurance, lower education levels, nancial and relationship instability, lack of transportation and lack of PCP were signi cantly associated with lower clinic follow-up adherence (Tables 1, 2 and 3). There was also a signi cant association between low clinic adherence and ED visits within the 90-day post-operative period, indicating that this could be an area for quality improvement (p < 0.01, Table 3). Additionally, our multivariate logistic regression model using clinically applicable parameters including "Distressed" or "At Risk" DCI levels, insurance status, education level, gender, and PCP availability demonstrated that lack of private insurance status signi cantly increased the odds of patient follow-up nonadherence by 2-fold (OR = 2.0, CI = 1.05-3.99, p = 0.04, Table 5). "Distressed or "At Risk" DCI scores and high school or lower education levels also demonstrated similar predictive trends, though were not statistically signi cant (Table 5). Moreover, the prediction model yielded a maximum possible risk score of 8 (Table 5). Patients with low clinic adherence (< 0.75) had a mean score of 4.0 ± 1.6, while patients with high clinic adherence (≥ 0.75) had a mean score of 3.1 ± 1.75. The difference was statistically signi cant (p < 0.01), suggesting that this could be a viable predictive tool of patients at-risk for follow-up nonadherence, and thus, valuable in coordinating postdischarge care efforts.
Though the literature on risk factors for follow-up nonadherence in orthopaedic trauma is limited, some studies have reported similar results as those shown here. In 2014, Whiting et al. reported insurance status and injury complex severity as signi cant risk factors for nonadherence in the orthopaedic trauma clinic with the rst follow-up appointment (9). Studies by Zelle et al. and ten Berg and Ring also found male gender, uninsured or governmentally insured patients, and single patients to be signi cantly at-risk for follow-up nonadherence at 1-and 6-month follow-up visits after injury (10,12 (9,10). Notably, tobacco and illicit drug use have previously been correlated with lower socioeconomic status and lower levels of education -which were two factors we found signi cantly associated with loss to follow-up in this investigation (25,26). In any case, all of these studies agree that risk factors for follow-up nonadherence are invaluable data points and can allow for healthcare teams to identify patients at high risk for follow-up nonadherence. Our prediction tool can help automate this identi cation process, and subsequently, allow for healthcare providers to design and implement strategies to improve follow-up in these populations.
Our risk prediction model scores patient's risk for lack of follow-up on a scale from 0 to 8 based upon a set of risk factors (24). On average, scores greater than or equal to 4.0 were shown to be associated with a lower clinic adherence fraction, and thus, identify patients who would bene t from a more targeted treatment approach prior to and following hospital discharge. These approaches could include extended care efforts such as virtual visits, mobile emergency medical services, and mobile x-ray services, which already exist at our institution. Though our risk prediction model requires validation in future studies, we envision this tool as a valuable component of a social work discharge plan. The impact this simple intervention could have extends far beyond the principal bene t of improved follow-up. Patients will have better outcomes as presentation to clinic will allow the orthopaedic surgeon to appropriately adjust postoperative treatment plans and recognize and treat complications early. Additionally, orthopaedic researchers will be able to conduct clinical trials with less bias, and therefore, be able to identify avenues for continued innovation in the eld. Lastly, the healthcare system will be subject to far less nancial stress as increased clinic adherence will decrease emergency department utilization, decrease subsequent hospital readmissions, and decrease overall patient cost of care. We believe this will lead to improved quality of healthcare delivery and a better patient experience.
This study has several limitations. First, given our sample size, it is possible that our study may have been underpowered to detect clinically meaningful differences in some variables, such as tobacco and illicit drug use, which have been shown in other studies to correlate signi cantly with poor follow up(9, 10). Further, this was a single-center study and may not be generalizable to other urban or rural trauma centers. Larger, multicentered studies are necessary to validate and improve our model. Lastly, we are not able to ascertain why patients missed clinic visits, such as scheduling errors, follow-up at another institution, and even death. This subjects our study to potential information bias and could mean that our true adherence rate was higher than what is presented here.

Conclusions:
This study demonstrates that post-trauma clinic adherence may be anticipated at the time of hospital discharge based upon several sociodemographic and clinical risk factors. We offer a predictive model of such behavior based on these parameters, which we visualize as a valuable component in a social work discharge plan in the near future. Such a prediction tool will allow for coordinated care of at-risk trauma patients following hospital discharge through extended care options such as virtual visits, mobile emergency medical services, and mobile x-ray services. Coordinated care services such as these will allow for improved patient outcomes, e cient hospital resource utilization, improved quality of care, and an overall better patient experience. Availability of data and materials: The datasets and/or analyzed during the current study are available from the corresponding author on reasonable request.
Competing interests: The authors declare that they have no competing interests.
Funding: Emory Department of Orthopaedics.
Authors contributions: MM was a major contributor to data interpretation and manuscript write up. VK and KB were responsible for data acquisition and analysis. MAM and MCS were major contributors to manuscript revision. MLS was responsible for idea conception, study design and was also a major contributor to manuscript revision. AS prepared all tables and led data interpretation and manuscript write up. All authors have read and approved the submitted version of this manuscript.