Clinical Impact of an Analytic Tool for Predicting the Fall Risk in Inpatients: Controlled Interrupted Time Series

Abstract

Background: Patient falls remain a common cause of harm in acute-care hospitals worldwide. Despite the availability of a considerable body of literature on risk factors of inpatient falls, it is a difficult, complex, and common problem requiring a great deal of nurses’ time, attention, and effort in practice. With recent rapid expansion of health care predictive analytic application along with growing availability of electronic health record data, a patient-level electronic analytic tool for predicting fall risk was developed.

Objectives: The purpose of this study was to explore how an electronic analytics tool for predicting fall risk affect patient and process outcomes.

Methods: A controlled interrupted time series (CITS) experiment was conducted in 12 medical-surgical nursing units at a public hospital between May 2017 and April 2019. The intervention was the provision of patient-level risk predictions generated by an analytic tool using routinely obtained from the hospital’s electronic health record system. The primary outcome was the fall rate, and secondary outcomes included fall-related injuries and predefined process indicators. 

Results: During the study there were 42,476 admissions, while 707 falls and 134 fall injuries occurred. Allowing for differences in the patients’ characteristics and baseline process outcomes, the fall rate was significantly low in the intervention group (1.79 vs. 2.11, t = 2.13, P = .038). The CITS analysis revealed that the immediate reduction was 29.73% in the intervention group (z = –2.06, P = .039) and 16.58% in the control group (z = –1.28, P = .200), but no ongoing effect. The injury rates did not differ significantly (0.42 vs. 0.31, t = –1.54, P = .131). Patient-level-adjusted logistic regression showed a significant group effect on falls. Process outcomes related to multifactorial interventions including risk-targeted interventions increased significantly in the intervention group over time.

Conclusion: The effectiveness of IN@SIGHT was supported only by the before-after comparison. However, it demonstrated the potential to contribute to improvement of patient outcomes, leading to positive changes in process outcomes over time. Further research is needed for this new approach.

Contributions To The Literature

What is already known on this topic

λ Despite the availability of a considerable body of literature on fall prevention and reduction, inpatient falls remain a difficult, complex, and common problem requiring a great deal of nurses’ time, attention, and effort in practice

• Several heuristic risk assessment tools have used widely, but none of them were adequately replicated in the relevant setting.

• With recent rapid expansion of health care predictive analytic application along with growing availability of electronic health record data, patient-level electronic analytic tools for predicting fall risk were developed.

What this study adds

• Our study suggests that a patient-level electronic analytic tool has the potential to contribute to reducing rates of falls.

• Our study suggests that the continuous measurement and analysis of process outcomes were informative to infer the effect of interventions to patient outcomes as well as interpret the effect of confounding.

Introduction

Inpatient falls are preventable adverse events that are among the top-10 sentinel events in hospitals. Up to 1 million fall events occur annually in the US, and the average cost of each such event has been estimated at $7,900–17,099 (2019 USD). [1, 2] A review of published studies [3-5] revealed that 400~700 such events occur annually in Korean tertiary academic hospitals.

Despite the availability of a considerable body of literature on fall prevention and reduction, falls remain a difficult, complex, and common problem that consume a great deal of time, attention, and effort among nurses attempting to prevent them in practice. [6, 7] One-third of inpatient falls are known to be preventable, but this is greatly hindered by the inability to accurately estimate the risk of falling. [8, 9] Several risk assessment tools developed using heuristic approaches are widely used to estimate the fall risk in acute-care settings. However, none of them have been adequately implemented, and clinical guidelines and researchers commonly recommend multifactorial risk assessments rather than using a single risk assessment tool. [9-12] Another problem is the no differences in the nursing interventions provided to patients between at-risk and no-risk days. Clinical observations indicate that small numbers of universal precautions are applied repeatedly based on scores from risk assessment tools, rather than using risk-targeted interventions.

The increased adoption of electronic health record (EHR) systems over the past decade may make it possible to use nursing assessment data routinely captured via EHR systems to predict inpatient falls. [13] One previous study found that a predictive analytic tool designed using a probability technique performed better at discriminating at-risk and no-risk days than did the existing fall risk assessment tools alone. [14] Nursing predictive analytics can be defined to include information regarding the likelihood of a future patient event through risk prediction models. Such models incorporate multiple predictor variables obtained automatically from one or more sources of nursing-related data beyond what can be simply calculated by users. Several studies have investigated the use of variable selection, model development, and validation in nursing for predictive analytics. [5, 15-20] However, only a few studies have analyzed predictive models in real-world settings and explored their relationships with nursing-sensitive outcomes. Several studies [15, 16, 18, 19] have explored the use of clinical deterioration alerts and the early detection of sepsis in predicting adverse events such as pressure ulcers. These studies produced mixed results, with one [15] concluding that simple laboratory and vital-sign criteria were insufficient for improving sepsis outcomes. Other studies [17, 21] have found positive changes in outcomes, but little is known about the clinical feasibility of nursing analytic approaches and tools, and how nurses respond to them.

The increasing availability of EHR data provides great opportunities in expanding health-care predictions. [22] This study applied predictive analytics to inpatient falls and explored its relationships with patient and process outcomes in a real-world setting. This study hypothesized that knowledge of fall events that are likely to occur within 24 hours based on data routinely captured in EHRs would enable nurses to conduct multifactorial and risk-targeted interventions to at-risk days of patients.

Prior Work: Development of an Inpatient Fall Risk Prediction Model

We have previously reported on the development of a risk prediction model in detail. [14] Briefly, concepts of fall risk factors and preventive care were identified using two international practice guidelines [9, 12] and two implementation guidelines [23, 24] on preventing inpatient falls. Two standard vocabularies, the Logical Observation Identifiers Names and Codes [25] and the International Classification for Nursing Practice®, [25, 26] were used to represent the concepts in the prediction model, which was then represented using a probabilistic Bayesian network. 

The model was tested in two study cohorts obtained from two hospitals with different EHR systems and nursing vocabularies. The model concepts were mapped to local data elements of each EHR system, and two implementation models were developed for a proof-of-concept approach, followed by across-sites validation. The EHR data included in the model were demographics, administrative information, medications, patient classifications, the fall-risk-assessment tool, and nursing processes including assessments and interventions. The two implementation models exhibited error rates of 11.7% and 4.87%, with c statistics of 0.96 and 0.99, respectively. The model performed 27% and 34% better than the existing Hendrich II tool [27] and STRATIFY (St. Thomas’ Risk Assessment Tool in Falling Elderly Inpatient) [28], respectively. 

Prior Work: IN@SIGHT system

The validation-site model was implemented as version 1.0 of the IN@SIGHT (Intelligent Nursing @ Safety Improvement Guide of Health Technology) system at a public 900-bed hospital in the metropolitan area of Seoul (Republic of Korea) that used the STRATIFY to assess fall risks for all inpatients. The analytic tool was integrated into the locally developed hospital EHR system that had been used for more than 10 years. The tool was deployed in 6 targeted nursing units on April 5, 2017, and all of 204 nurses received and used its patient-level prediction results daily. This implementation process involved the chief of the nursing department, unit managers, unit champions, personnel of the department of medical informatics, and the patient safety committee. For 3 months before system deployment, three sessions of education on the IN@SIGHT were provided to the intervention group, which were followed by peer-to-peer education provided by unit champions. The nursing department decided to replace the existing STRATIFY with the analytic tool during this quasi-experimental study.

The original model was customized by replacing the six data elements of the STRATIFY with proxy data elements in the EHR. The adjusted model consisting of 40 nodes and 68 links had an error rate of 9.3%, a spherical payoff of 0.92, and a c statistic of 0.87. Related work processes were redefined and the existing fall-prevention documentation screen of the EHR was modified. The hospital decided to deliver the risk information in a dichotomized format with at-risk and no-risk categories at a cutoff point of 15%, which provided a high specificity of 89.4%. The IN@SIGHT system (Figure 1) triggers an “at-risk” alert on the EHR system when a user selects an at-risk patient. No such alert appears for a no-risk patient, but the user can manually open the same window through menu navigation. 

Methods

Study Framework and Objectives

A study framework was developed based on a nursing role effectiveness model. [29] (Figure 2) The original model is based on the structure−process−outcome model of Donabedian’s quality care. In this study, the model was reformulated for this empirical testing. Specifically, this study focused on nurses’ independent role in the process component as well as patient outcomes. The authors hypothesized that the IN@SIGHT intervention will affect the nursing interventions efficiently which would be followed by outcomes change. The IN@SIGHT gave nurses about information on presence of risk of individual patient’s fall events.  

This study explored the clinical feasibility of version 1.0 of IN@SIGHT and how nurses responded to it. The following specific research questions were addressed:

1.  Did the patient-level analytic tool influence the quality of nursing care as assessed using outcome indicators?
2. Did the predictive analytic tool affect nursing activities provided to patients?
3. How did the effects change over time?

Study Design and Setting

To control for history bias due to time-varying confounders, such as co-intervention and other events occurring with the intervention and other events, a CITS experiment was adopted. The 12 medical-surgical units were selected and allocated into two groups considering the baseline fall rates and unit characteristics. All the nurses and eligible patients were participated this study from May 1, 2017 to April 30, 2019. (Figure 3) The patients met the following criteria: aged ≥18 years and admitted for >1 day in departments other than pediatrics, psychiatrics, obstetrics, and emergency care. The preintervention period was set as 16 months, which was the maximum retrospective time window. The postintervention period was 24 months and process outcomes were measured every 6 months.

Results

Patient Characteristics 

This study involved 42,476 admissions of 40,345 unique patients from 12 units, which corresponded to 362,805 HD at the nursing units across both groups. There were 2,131 patients who were admitted to an intervention and a control unit at different times (5.02% of all admissions). The patient characteristics differed significantly between the two groups (Table 1). The control units were characterized by older patients, a longer stay, fewer female patients, and more patients with a fall history at admission, comorbidity status, and surgical procedures. Regarding the primary diagnosis, about half of the patients in the intervention group had a respiratory or digestive disease or any form of cancer, while control patients had a greater diversity of diagnoses. The significant patient characteristics were used to control the patient-level differences for the group effect on the fall rate. 

Table 1. Characteristics of patients in the intervention and control groups

Data are n (%) or mean (95% confidence interval) values. 

^† Including genitourinary, musculoskeletal, eye, ear, and skin diseases.

Comparison of Primary Outcomes

There were 325 fall events in the intervention group and 382 in the control group. The mean fall rate decreased from 1.92 to 1.79 in the intervention group and increased from 1.95 to 2.11 in the control group. Simple t-test comparisons of monthly rates revealed that the fall rate was lower in the intervention group (1.79 vs. 2.11, t = 2.13, P = .038). The longitudinal analysis of nursing-unit-level data showed no autocorrelation within groups (P = .403), while the group effect was significant for the fall rate (z = –2.10, P = .036). The fixed-effect model also predicted a significant group effect (t = 2.13, P = .038). The CITS analysis revealed 29.73% reduction of significant decrease in the fall rate after introducing the intervention (Rate ratio [RR] = −0.30, P = .039), followed by a nonsignificant long term effect compared with the underlying time trend (RR= 0.01, P = .059) (Table 2). In the control group, there was 16.58% reduction which was no significant change in the fall rate either immediately (RR = − 0.17, P = .200) or over time (RR = −0.01, P = .057). 

Table 2. Results of controlled interrupted time-series analysis of fall rates 

Data are rate ratio (95% confidence interval) values, ^* P <  .05 

Secondary Outcomes: Injury Rates and Process Indicators 

During the intervention period there was no significant intergroup difference in the mean injury rate (0.42 vs. 0.31, t = − 1.54, P = .131). There was also no significance found in the GLS analysis (z = 1.69, P = .091) or fixed-effect model (t = 1.65, P = .105).

Both the fall risk and injury risk factors were assessed within 24 hours of hospital admission in all patients in the intervention group, while no such data were available for about 75% HD in the control group (Table 3). Universal precautions and education for at-risk HD were provided to most patients in the control group. The communication practices and environmental interventions were better in the control group by 18 months, at which time those in the intervention group had reached the same level. While the rate of risk-targeted interventions increased incrementally in both groups, the intervention group showed better adherence than the control group at the 4^th period (29.5% vs. 18.1%, P < .001). 

Table 3. Changes in process indicators in the two groups over time

Int., intervention group; Cnt., control group; NA, not applicable

^† Age, bone health, anticoagulants, and current surgery 

^‡ Function that was performed automatically by the analytic tool for predicting the fall risk 

^§ Data collection not categorized in detail

^ns Not significant, ^* P <  .05, ^** P <  .001

In the patient-level analysis adjusted for patient characteristics, the overall group effect was significant (Odds ratio [OR] = 0.52, 95% CI = 0.42 to 0.65, P < .001). After 6 months, the group effect occurred in the second (OR = 0.25, 95% CI = 0.12 to 0.53, P < .001) and the third periods (OR = 0.36, 95% CI = 0.21 to 0.61, P < .001).

Table 4. Results of patient-level logistic regression over time

Data are adjusted odds ratio (95% confidence interval), ^† Includes genitourinary, musculoskeletal, eye, ear, and skin diseases. ^* P < .05, ^** P < .001 

As for the care components of assessment, intervention group showed relatively various observations of the mobility status, mental status, and others (Figure 4-A), while the mobility assessments were dominant in the control group, whose frequency increased suddenly during the last period. Considering the interventions, universal precautions, education, and medication reviews were the most common interventions in both groups (Figure 4-B). The intervention group showed steady increase of interventions. 

Discussion

Introducing a patient-level electronic analytic tool reduced the rates of falls at a public hospital in the before-after comparison. However, it was not supported by the intervention-control group comparison due to the notable differences in patient characteristics between the two groups. The tool was feasible to use and accepted by nurses and improved the completion of risk assessments. The process outcomes for multifactorial and risk-targeted interventions for at-risk days increased over time. The tool had no effect on the rates of fall-related injuries compared with the usual care. These results imply that the effectiveness of an electronic analytic tool could be limited, but this trial showed its possible potential in helping nurses to make informed clinical decisions and manage their care time efficiently by prioritizing to whom and what interventions should be delivered. During the study period, the research team confronted several confounders, which made the interpretation of the results challenging. The discussion on these issues is valuable for the future research of risk prediction and alerting at real settings.  

The fall rates of 1.79 and 2.11 in the intervention and control groups, respectively, in this study were lower than previously reported rates of 2.08~4.18 for an intervention study involving a cluster randomized controlled trial (RCT) in four urban US hospitals [34], 3.05 for a cluster RCT in Australia [7], and 2.80 for a US intervention study [35]. However, it is not meaningful to compare these rates directly due to the differences in the patient populations and in the structural elements at the facilities [36]. The low fall incidence rate in the present study allowed us to observe changes in nurse behaviors over a long follow-up period of 24 months. An intervention will not be effective if nurse behaviors are not capable of changing patient outcomes, and so we focused on how the analytic tool can influence nurse behaviors to ensure that interventions are beneficial to patients. Our findings revealed that the intervention group assessed patients’ more multifactorial aspects than the control group, but the interventions were limited to several components such as precautions, educations and medication review at both groups. Considering that the precautions are supposed to be given to all inpatients regardless of risk of falls, education and medication review were most of preventions. Interventions of toileting aids, impaired mental and cognitive function, impaired sensory function, and sleep disturbance were hard to find at both groups.  

International guidelines for preventing falls [9, 12, 23, 24] indicate that multifactorial assessments of risks and multifactorial, risk-targeted interventions are basic components of beneficial care strategies. The application of the analytic tool in the present study ensured that risk factors were monitored in all hospital days of each patient and delivered alarm to nurses via the hospital EHR system. During the first 6 months we could observe a large increase in data seeking and gathering activities, whereas the notable increases in overall interventions appeared 12 months later, and in risk-targeted interventions 18 months later. These observations imply that nurses take time to adopt the processes of a new approach via several steps, which was also supported by our surveys conducted repeatedly during the study period [37]. The surveys revealed a few nurses reporting neutral or even slightly negative attitude and experiences at the beginning of the study, even most of the nurses showed cooperative and positive attitudes. Negative responses gradually decreased over time. These findings can be understood by the nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) framework [38] explaining the success of a technology-supported health or social care program. Staff members are often initially more concerned about threats to their scope of practice or to the safety and welfare of patients, resulting in them gathering more information on risks. A previous qualitative exploration study [20] investigated how nurses perceive predictive information and how they act upon it through one-on-one and focus-group interviews. That study found that nurses attempted to gather more information from other sources and review more-detailed predictions during periods of uncertainty. Time delays to adoption and changing behaviors are expected given that predictive information is relatively new to nurses. The other relevant domain of the NASSS framework was the readiness of a hospital for a predictive analytic tool. The understanding and support, antecedent conditions, and level of readiness for a novel tool at the board level might influence the uptake time by nurses and the internal drivers for scaling up the tool.

Considering the difference between the two groups’ patient characteristics, the control group patients had more vulnerable condition for falls than the intervention group; 4 years older, 1.3. days longer for hospital stay, and having more fall histories. This difference might also relate to an ascertainment bias issue concerning inclusion of more patients at lower risk in the denominator of fall rate in the intervention group than fall rates truly decreased. While, according to the baseline data, the control group nurses have provided to patients significantly more preventive interventions than the intervention group; more additional risk assessments, universal precautions, educational interventions, communication and environmental interventions. In other words, the control group patients were more vulnerable for falls, so that they had received more preventive interventions. Even it was not clear how these factors interact and how they may impact the outcomes of this study, we can assume that the more provision of interventions have contributed to decreasing the risks of control group.

Next important thing to discuss here is the changes of process outcomes. The changes of process outcomes and nursing activities over time could give us some clues to interpret the impact of IN@SIGHT. In our previous study [14] we found that the analytic tool predicted about 20% of HD as at-risk days, which was about a half compared with the STRATIFY classified 40 ~ 50% of HD as at-risk days. The actual falls rate in the hospital was much lower of around 0.2% of HD. We assumed that more precise up-to-date predictions of fall events will decrease nurses’ burden on redundant interventions induced by false positive warnings of the STRATIFY. As we expected the IN@SIGHT approach did not affect the universal precautions, but risk-targeted interventions as well as education, communication, and environment interventions increased significantly. While the control group showed steady state. These findings are meaningful, considering that multifactorial interventions including risk-targeted prevent anticipated physiologic falls, which are responsible for more than 70% of inpatient falls. [34, 38] These process measures revealed slow but explicit changes in nursing interventions, which indicates that the processes underlying care elements had changed and we could expect the following improvement of patient outcomes. [39] Continuously measuring and analyzing process outcomes were informative to inferring the effects of interventions on patient outcomes as well as to interpreting the effects of confounding, which has rarely been accounted for in previous studies. [7, 34, 40]

The design of this study had several limitations. First, the implementation at a single site for a long study period introduced several challenges that could have reduced the effects of the IN@SIGHT. One challenge was an unexpected occurrence at the hospital. One month after study initiation, one nursing unit in each group moved to a new location and nurse staffing was reorganized due to the physical reconstruction of the hospital buildings. The fall rate increased markedly for several months in that intervention unit compared with the other five units in the group. However, the control-group unit showed only slight increase compared with those in the other units. The relocations were accompanied by changes in staff nurses and in the medical diagnoses of patients, both of which might have increased the burden on nurses and induced the sudden increase in the fall rate at the unit. The other occurrence was the mandatory routinization of hourly nursing rounds to all inpatients by the policy of the hospital’s safety committee during the last 6-month period, which may account for the sudden increase of nursing assessments in the control group. Second, we were unable to compare the injury fall rates between the before-after intervention periods. It was left unknown how the IN@SIGHT approach affected the injury rate. Third, due to the controversial differences of the two groups this study failed to control history bias and the time-variant confounding on primary outcome. Therefore, other quality-improvement initiatives may have been implemented at the hospital during the study period that affected both the nurse turnover and the appropriate use of health-care resources. 

Inpatient falls prevention has been considered a difficult and complex issue for which there is little high-quality evidence. [7, 41] Even though most of current practice guideline for preventing inpatient falls recommend multifactorial interventions, evidences are still lack to support the strategies and novel methods are required to reduce harm from falls in acute settings. [Parker] The IN@SIGHT approach could be a new approach to serve as a surveillance patient-level risks and improve efficacy of interventions in system level. The challenges and confounding factors which discussed in this study will contribute to improving further research on risk prediction and alerting at real settings.

Conclusions

The effectiveness of the IN@SIGHT electronic analytic tool for predicting the fall risk of inpatients was supported only by the before-after comparison, not by the intervention-control comparison. There were meaningful changes in process outcomes leading to more multifactorial and risk-targeted interventions. Nurses were amenable to using the tool, and hospital managers used the tool to make informed decisions aimed at preventing falls. As an example of a nursing predictive analytics application, defined as the use of electronic algorithms that forecast patient events in real time and at the point of care to improve outcomes and reduce costs, this study has shown a pressing need for further research on the effectiveness of electronic analytic tool at real settings.

Declarations

Ethical approval and consent to participate:

The authors got waive the need to obtain consent from individual patients and nurse from the IRB of National Health Insurance Service Ilsan Hospital (IRB No. NHIMC 2016-08-005)

Consent for publication: Not applicable

Competing interests: None

Funding: This study was supported by a grants from the Korea Healthcare Technology R&D Project, Ministry for Health and Welfare (No. HI17C0807), the National Research Foundation of Korea (No. NRF-2019R1A2C2007583), and the Ministry of Trade, Industry and Energy of Korea (No. 20004861)

Data Sharing Statement:

The data that support the findings of this study are available from the National Health Insurance Service Ilsan Hospital, but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of National Health Insurance Service Ilsan Hospital.

Authors’ contributions:

I. C. conceived and designed the study, supervised and contributed to the data analysis, interpreted results, and drafted and revised the paper. I. J. and H.P. contributed to study design, data acquisition, and data analysis. PC. D. provided substantial contribution to the interpretation of data analysis and revised the work critically for important intellectual content.

ACKNOWLEDGEMENTS

We thank to Cheehang Kim and Jisun Cho of the Nursing Department, and Yunjeong Choi of the Medical Information Department at Ilsan hospital for helping us to conduct this study administratively and technically. We also appreciate the clinical staffs of the Nursing Department and graduate students involved in data collection, review and analysis.

COMPETING INTERESTS

The authors whose names are listed certify that they have NO affiliations with or involvement in any organization or entity with any financial interest, or non-financial interest in the subject matter or materials discussed in this manuscript.

References

1. Spetz J, Brown DS, Aydin C. The economics of preventing hospital falls: demonstrating ROI through a simple model. J Nurs Admin 2015;45(1):50-57.
2. Wong CA, Recktenwald AJ, Jones ML, Waterman BM, Bollini ML, Dunagan WC. The cost of serious fall-related injuries at three Midwestern hospitals. Jt Comm J Qual Patient Saf 2011;37(2):81-87.
3. Lee E-S, Ahn M-J, Kim Y-M, Jo I-J, Jang D-E. Development and effects of fall prevention model: Safety for utilizing medical big data based artificial intelligence. Paper presented at: Korean Society of Medical Informatics; November, 23-24, 2018; Jeonju-si, Korea.
4. Cho MS, Lee HY. Factors Associated with Injuries after Inpatient Falls in a Tertiary Hospital. J Korean Clin Nurs Res 2017;23(2):202-210. https://doi.org/10.22650/JKCNR.2017.23.2.202
5. Lee JY, Jin Y, Piao J, Lee S-M. Development and evaluation of an automated fall risk assessment system. Int J Qual Health Care 2016;28(2):175-182.
6. Lopez KD, Gerling GJ, Cary MP, Kanak MF. Cognitive work analysis to evaluate the problem of patient falls in an inpatient setting. J Am Med Inform Assoc 2010;17(3):313-321.
7. Barker AL, Morello RT, Wolfe R, et al. 6-PACK programme to decrease fall injuries in acute hospitals: cluster randomised controlled trial. BMJ 2016;352:h6781. https://doi.org/10.1136/bmj.h6781
8. The Joint Commission. Sentinel event alert 55: Preventing falls and fall-related injuries in health care facilities. Sentinel Event Alert, 2015. http://www jointcommission org/assets/1/18/SEA_55 pdf. Accessed 28 Sept 2020. PMID:  26422837.
9. Centre for Clinical Practice at National Institute for Health and Care Excellence. Falls in older people: assessing risk and prevention. London: National Institute for Health and Care Excellence (UK): National Institute for Health and Care Excellence;2013. https://www.ncbi.nlm.nih.gov/books/NBK258885/pdf/Bookshelf_NBK258885.pdf. Accessed 8 Aug 2020.
10. Aranda-Gallardo M, Morales-Asencio JM, Canca-Sanchez JC, et al. Instruments for assessing the risk of falls in acute hospitalized patients: a systematic review and meta-analysis. BMC Health Serv Res 2013;13(1):122. https://doi.org/10.1186/1472-6963-13-122
11. Park S-H, Kim E-K. Systematic review and meta-analysis for usefulness of fall risk assessment tools in adult inpatients. Korean J Health Promot 2016;16(3):180-191. https://doi.org/10.15384/kjhp.2016.16.3.180
12. Registered Nurses’ Association of Ontario (RNAO). Preventing Falls and Reducing Injury from Falls (4th ed.). Best Practice Guidelines: Long-Term Care Best Practices Program Ontarion, Canada2017: https://rnao.ca/sites/rnao-ca/files/bpg/FALL_PREVENTION_WEB_1207-17.pdf. Accessed 8 Aug 2020.
13. Jeffery AD. Methodological challenges in examining the impact of healthcare predictive analytics on nursing-sensitive patient outcomes. Comput Inform Nurs 2015;33(6):258-264. doi: 10.1097/CIN.0000000000000154.
14. Cho I, Boo EH, Chung E, Bates DW, Dykes P. Novel Approach to Inpatient Fall Risk Prediction and Its Cross-Site Validation Using Time-Variant Data. J Medical Internet Res 2019;21(2):e11505. DOI: 10.2196/11505
15. Austrian JS, Jamin CT, Doty GR, Blecker S. Impact of an emergency department electronic sepsis surveillance system on patient mortality and length of stay. J Am Med Inform Assoc 2017;25(5):523-529. DOI: 10.1093/jamia/ocx072.
16. Bailey TC, Chen Y, Mao Y, et al. A trial of a real‐time Alert for clinical deterioration in Patients hospitalized on general medical wards. J Hosp Med 2013;8(5):236-242. DOI: 10.1002/jhm.2009.
17. Cho I, Park I, Kim E, Lee E, Bates DW. Using EHR data to predict hospital-acquired pressure ulcers: a prospective study of a Bayesian Network model. Int J Med Inform 2013;82(11):1059-1067. DOI: 10.1016/j.ijmedinf.2013.06.012
18. Kollef MH, Chen Y, Heard K, et al. A randomized trial of real‐time automated clinical deterioration alerts sent to a rapid response team. J Hosp Med 2014;9(7):424-429. https://doi.org/10.1002/jhm.2193
19. Sawyer AM, Deal EN, Labelle AJ, et al. Implementation of a real-time computerized sepsis alert in nonintensive care unit patients. Crit Care Med 2011;39(3):469-473. DOI: 10.1097/CCM.0b013e318205df85
20. Jeffery AD, Kennedy B, Dietrich MS, Mion LC, Novak LL. A Qualitative Exploration of Nurses’ Information-Gathering Behaviors Prior to Decision Support Tool Design. Appl Clin Inform 2017;8(03):763-778. DOI: 10.4338/ACI-2017-02-RA-0033
21. Honeyford K, Cooke GS, Kinderlerer A, et al. Evaluating a digital sepsis alert in a London multisite hospital network: a natural experiment using electronic health record data. J Am Med Inform Assoc 2020;27(2):274-283. https://doi.org/10.1093/jamia/ocz186
22. Parikh RB, Kakad M, Bates DW. Integrating predictive analytics into high-value care: the dawn of precision delivery. JAMA 2016;315(7):651-652.
23. Registered Nurses’Association of Ontario (RNAO) & Canadian Patient Safety Institute (CPSI). Reducing Falls and Injuries from Falls Getting Started Kit: Evidence Update. Reducing Falls and Injuries from Falls 2015; http://www.patientsafetyinstitute.ca/en/toolsResources/Documents/Interventions/Reducing%20Falls%20and%20Injury%20from%20Falls/Falls%20Evidence%20update%202018-01.PDF. Accessed 8 Aug 2020.
24. Ganz D, Huang C, Saliba D, Miake-Lye I, Hempel S, Ensrud K. Preventing falls in hospitals: a toolkit for improving quality of care. Ann Intern Med 2013;158(5 Pt 2):390-396.
25. Vreeman DJ, McDonald CJ, Huff SM. LOINC®: a universal catalogue of individual clinical observations and uniform representation of enumerated collections. Int J Funct Inform Personal Med 2010;3(4):273-291. DOI: 10.1504/IJFIPM.2010.040211.
26. World Health Organization. Classifications: International Classification for Nursing Practice (ICNP). 2018; http://www.who.int/classifications/icd/adaptations/icnp/en/. Accessed September 3, 2020.
27. Hendrich A. How to try this: predicting patient falls. Using the Hendrich II Fall Risk Model in clinical practice. Am J Nurs 2007;107(11):50. DOI: 10.1097/01.NAJ.0000298062.27349.8e.
28. Oliver D, Britton M, Seed P, Martin F, Hopper A. Development and evaluation of evidence based risk assessment tool (STRATIFY) to predict which elderly inpatients will fall: case-control and cohort studies. BMJ 1997;315(25):1049-1053. DOI: 10.1136/bmj.315.7115.1049
29. Irvine D, Sidani S, Hall LM. Linking outcomes to nurses' roles in health care. Nurs Econ 1998;16(2):58. DOI: 10.1504/IJFIPM.2010.040211.
30. American Nurses Association. Guidelines for data collection on the American Nurses Association's national quality forum endorsed measures: Nursing care hours per patient day; skill-mix; falls; falls with injury. KU School of Nursing, Kansas City. 2012.
31. Cho I, Kim J, Chae J, Jung M, Kim Y. Development of ICNP-based inpatient falls prevention catalogue. Int Nurs Rev 2019. https://doi.org/10.1111/inr.12566
32. Hill AM, Hoffmann T, Hill K, et al. Measuring falls events in acute hospitals—a comparison of three reporting methods to identify missing data in the hospital reporting system. J Am Geriatr Soc 2010;58(7):1347-1352. DOI: 10.1111/j.1532-5415.2010.02856.x
33. Bernal JL, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol 2017;46(1):348-355. DOI: 10.1093/ije/dyw098.
34. Dykes PC, Carroll DL, Hurley A, et al. Fall prevention in acute care hospitals: a randomized trial. JAMA 2010;304(17):1912-1918. DOI: 10.1001/jama.2010.1567.
35. Dykes PC, Duckworth M, Cunningham S, Dubois S, Driscoll M, Feliciano Z, et al. Pilot Testing Fall TIPS (Tailoring Interventions for Patient Safety): a Patient-Centered Fall Prevention Toolkit. Jt Comm J Qual Patient Saf 2017;43(8):403-13.
36. Bouldin ED, Andresen EM, Dunton NE, Simon M, Waters TM, Liu M, et al. Falls among adult patients hospitalized in the United States: prevalence and trends. J Patient Saf 2013;9(1):13.
37. Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, Hinder S, et al. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res 2017;19(11):e367.
38. Oliver D, Healey F, Haines TP. Preventing falls and fall-related injuries in hospitals. Clin Geriatr Med 2010;26(4):645-692. DOI: 10.1016/j.cger.2010.06.005.
39. Mant J. Process versus outcome indicators in the assessment of quality of health care. Int J Qual Health Care 2001;13(6):475-480.
40. Cumming RG, Sherrington C, Lord SR, et al. Cluster randomised trial of a targeted multifactorial intervention to prevent falls among older people in hospital. BMJ 2008;336(7647):758-760.
41. DiBardino D, Cohen ER, Didwania A. Meta‐analysis: multidisciplinary fall prevention strategies in the acute care inpatient population. J Hosp Med 2012;7(6):497-503.