A Predictive Model for Estimating Risk of Harm and Aggression in Inpatient Mental Health Clinics

Serious mental illness is a major risk factor for aggression and violence. The present study aimed to develop and test an algorithm to predict inpatient aggressions that involve a risk of harm to self or others. This work is based on a retrospective study aimed to investigate the prediction of risk of harm and aggressions at St. Joseph’s Healthcare Hamilton, between 2016 and 2017. An analysis of the risk factors most strongly associated with harmful incidents is, followed by the description of the process involved in the development of a predictive model which estimates the risk of harm. The efficiency of the model developed is finally evaluated, showing an overall accuracy of 75%: the specificity to identify episodes considered not at risk of harm is equal to 91.85%, whereas the sensitivity to identify episodes considered harmful is equal to 28.57%. The model proposed can be seen as a seminal project towards the development of a more comprehensive, precise and effective tool capable to predict the risk of harm in the inpatient setting.

excess of stimulation, negative staff attitudes and provocation by others. Personality disorders, in particular the antisocial and borderline personality disorders, resulted significantly associated with episodes of inpatient aggressions [10,28].
While the research on violent inpatients has grown considerably, less evidence has been produced in regard to the accuracy of predictors of inpatient violence [28]. One of the most important tools for short-term prediction of violence in inpatient settings is the Norwegian Brøset-Violence-Checklist (BVC) [2,3,30]. This instrument predicts the potential risk of a physical attack within the next shift [22]. Another instrument in place, used to assess the risk of inpatient harm to others, is the Risk of Harm to Others Clinical Assessment Protocol [13] implemented into the Resident Assessment Instrument-Mental Health (RAI-MH) [14]. This algorithm utilizes a framework composed by the specific information in the RAI-MH, with the intent to inform clinical teams about the risk that a resident might harm others [23].
The present study has received the support of the St. Joseph's Healthcare Hamilton (Ontario), which allowed us to access patients' data. This work aimed to develop and test an algorithm for the prediction of inpatient aggressions that involve a risk of harm to self or others. Two sources of data have been used. The first source is the RAI-MH. Since 2005, the RAI-MH was adopted in Ontario as the assessment platform for adult residents Real-World Evaluation of the Resident Assessment Instrument-Mental Health Assessment System [14,18,29]. The current version of this tool is composed by around 400 items providing a comprehensive assessment of the patient, incorporating past and present symptomatology as well as a comprehensive appraisal of the global functioning of the individual [29].
The second source of data is the Safety Incident Reporting System (SIRS), an internal platform utilized at the St. Joseph's Healthcare Hamilton (Ontario) that allows reporting all the incidents within the facility. The scope of this work is limited to the incidents that refer to inpatient aggressions.

Method
The realization of this study was possible thanks to the support of the St. Joseph's Healthcare Hamilton (Ontario), which supported this research granting access to patients' data. An internal report, generated by a system called the Safety Incidents Reporting System (SIRS), was requested with the purpose to detail all the aggressions committed by patients within a time frame of two years, from January 2016 to December 2017. The study involved 13 psychiatric wards, addressing a variety of services such as acute mental health, concurrent disorders, schizophrenia, and addictions. The SIRS provides a classification of the incidents committed, evaluated according to the presence and the severity of the harm. Incidents are then classified into 6 levels: in the first two levels there is no harm involved, whereas from level 3 to level 6 the presence of harm inflicted to others increases progressively (from mild to critical). After receiving this report, a selected range of data from patients' records was extracted (RAIs-MH).
This research has been approved by the Hamilton Integrated Research Ethics Board (HIREB) and The Research Institute of St. Joseph's Hamilton.

Spatial and Temporal Analysis
Some preliminary analyses have been conducted, to investigate where the aggressions have been committed as well as their distribution during the arch of the day. The spatial analysis of the aggressions revealed that most incidents took place in locations where human interactions are frequent (Fig. 1). The highest number of aggressions was reported in the hallway (41.7%), followed by the bedroom (21.6%) and the dining room (9%).
The temporal distribution of the aggression within the arch of the 24 h revealed that the highest number of episodes took place in the afternoon, between 2 pm and 7 pm (Fig. 2). In the bar graphs presented (Figs. 2, 3 and 4) the x-axis represents the range of hours in the day, from midnight (0 am) to 11 pm, whereas the y-axis shows the number of aggressions.
The comparison between the temporal distribution of aggressions in males and females has shown a few differences. While the distribution in males (Fig. 3) is more similar to the overall distribution ( Fig. 2), females' distribution has two peaks ( Fig. 4), between 12 am and 1 pm and between 5 pm and 6 pm. The number of aggressions reported during night times is very small, and it increases progressively from around 6 am. The bar graphs indicate that the risk of aggression is very reduced during night, raises as the day progresses reaching a peak in the afternoon (between 2 pm and 6 pm), and decreases progressively.

Predictive Model
In order to develop a predictive model, it was necessary to organize the data related to the aggressions committed by residents. Table 1 shows the incidents' distribution among the six levels of severity identified by the SIRS. A total of 870 episodes of aggressions has been collected. The 74.48% of the incidents reported didn't involve any harm according to the clinical staff (Level 1 and Level 2), whereas the majority of the harmful incidents was reported within the mild harm category (Level 3).
In the current sample, young residents resulted more at risk of committing aggressions associated with harm to others (Spearman correlation rho = −0.2256; p value <0.001). The target variable for this model is based on the Incident Severity (Table 1). Since the intent of this algorithm is to predict the risk harm involved in aggression, the logistic regression has been identified as the ideal strategy to model the target variable. The software utilized for the creation of the model is RStudio. The target variable was remodelled as a dichotomic variable and split into two parts: the first set includes the incidents with no harm involved (Level 1 and Level 2); the second set includes the harmful incidents (from Level 3 to Level 6). The value 0 was assigned the dataset of "no harm" incidents, whereas the value 1 was assigned to the dataset of "harmful" incidents. The predictors for this model have been selected based on the most relevant factors emerged as violence predictors in the literature. The logistic regression was used to model the Incident Severity as a function of the following predictors:  The first operation relates to the partitioning of the whole dataset, composed by the 870 aggressions, in two sets: training and testing. During this operation the data are assigned randomly to each dataset. The training dataset will be utilized to build the predictive model, while the testing will be used to test and validate the model (Fig. 5). This methodology allows to build and test an algorithm using two different datasets (training and testing) and will be used to test the model. This operation in RStudio is performed by setting the outcome variable of interest and randomly split it  into two sets (function: createDataPartition), according to the percentage indicated (p = 0.7). This percentage indicates that 70% of the data, randomly assigned, define the training dataset and are used to build the model. The remaining 30% of the data define the testing dataset and will be used to test the model.

Phase 2
Through the logistic regression we modelled the target variable, Incident Severity, as a function of the 5 predictors indicated above. As specified above, during this phase we used the training dataset ( Table 2). Police intervention for violent behaviour, Violence to others and Socially inappropriate disruptive behaviour have been transformed in factors. The diagnoses considered for the model were restricted to four possible categories. The numbers placed at the very right of the predictors indicate the factorial levels of the variables. We named this model mod_fit. Figure 6 displays the code through which the model is applied to the training dataset.
The model, called mod_fit, was initially applied to the training dataset (Fig. 5).

Phase 3
This phase is dedicated to evaluate how well the model predicts the target variable. The necessary steps to achieve this goal are described below: 1. The model was applied to the testing data set using the function predict.

2.
A comparison was performed between the outcome variable ("Incident Severity") of the predictive model, once applied to the testing dataset, versus the real values of the training data set (observed). This operation was conducted utilizing the confusion matrix which was set up assigning the value "1" to the positive class of the matrix, corresponding to "harm" (Table 3, Fig. 6). Table 3 (below) shows the theoretical structure of the expected confusion matrix. The screenshot below was taken from RStudio and shows the outcome for the confusion matrix obtained (Fig. 7). Table 4 (below) describes the indicators of the confusion matrix. The overall accuracy of the model is indicated by the percentage related to the accuracy, estimated to be 75%. According to this percentage, the model has predicted correctly 138 of the 184 episodes. The results also show that the model has incorrectly predicted 11 episodes among the 135 ones without any harm involved resulting in a (specificity equal to 91.85%), and 35 episodes among 49 ones where some harm was predicted (sensitivity equal to 28.57%). The equation that describes the model was finalized. It will be used to input the patient's data and generate a predictive score for the risk of harm. The model is based on a logistic regression, described by the logit equation: The probability of the risk of harm is calculated as follows: The column "Estimates" of the logistic regression displayed in Table 2 shows the coefficients of the five predictors for the Intercept = 1.177513. These coefficients are used to compose the linear predictor, part of the logit eq. A practical example of how the equation can be used is provided. For this example, we used the anonymized data of one of the patients from the dataset.
The RAI-MH of a patient indicates to following values on the predictors selected for the model developed: Patient A: 1. Police intervention for violent behaviour: 2 2. Violence to others: 2 3. Socially inappropriate disruptive behaviour: 0  Coherently with the theoretical framework adopted for the confusion matrix obtained, the scores of P Risk of Harmful Aggression greater than 0.5 will be considered under the category risk of harm (category "1"). In the example shown above, the probability is equal to 0.727, approximately 73%. According to the model and the theoretical framework adopted, this result can be interpreted as the patient is at risk of harm (P > 0.5, category "1").

Results
This work aimed to build a basic predictive model for inpatient risk of harm, using a limited set of data from the RAI-MH. The overall accuracy of 75% of the model might be interpreted as an incentive to work more towards the "trajectory" identified with the model developed and defined by the predictors adopted. This is just an initial attempt, carried out with several limitations that will be discussed later. Despite the specificity (proportion of negatives correctly identified) of the model peaked the 91.85%, the sensitivity (proportion of positives correctly identified) showed a performance considerably lower, with "only" a 28.57%. The sensitivity's performance might be affected by the lower number of aggressions with harm associated. In fact, in the whole sample (870 incidents), the harmful aggressions account for 222 episodes, approximately 22.52% of the total. Moreover, the distribution of incidents from level 3 (mild harm) to level 6 (critical harm) is not uniform. Level 3 (mild harm) accounts for 198 episodes, the 89.19% of the total, whereas there are no episodes reported under level 5 (very severe harm). In the same way, the proportion of the aggressions with harm in the testing dataset accounted for 49 episodes on a total of 184, the 26.63%. The sample considered showed that aggressions with no harm are more common than harmful incidents. A strategy to address this issue will be to select a larger period of time might to gather more data. However, the distinction between harmful aggressions and aggressions with no harm poses some difficulties. This classification is performed by the clinical staff, that assesses every single episode. Not all the aggressive behaviours are sustained by the real intent of harm, and even when aggression is moved by this intent, it might not cause any harm. In these cases, classifying an incident as "not harmful" might be misleading. It is not clear to us how many potentially harmful aggressions didn't really cause any harm and have been classified under level 1 or level 2 (no harm). Therefore, the system adopted to classify these incidents was very helpful from one side, but represents a limitation on the other side, as it seems to be primarily based on the outcome of the event.

Discussion
The predictors utilized have been selected based on their relevance to the literature review. Many studies confirmed the critical role of having a history of violent behaviour in predicting inpatient violence [10,28]. In addition, the risk increases when physical aggressions or threats of violence have been committed more recently [20,21]. Psychotic disorders have emerged as strong predictors, in accordance with what was found in the literature. The association between psychotic disorders and the risk of inpatient violence is robust [1, 4-7, 10, 16, 25]. Despite we didn't specify the type of personality disorders within this diagnostic category, the most common disorders associated to episodes of inpatient violence are the antisocial personality disorder [27] and borderline personality disorder [5,24]. Age has emerged also as a strong predictor in this study, confirming findings in the literature according to with younger residents are more at risk of harm to others if compared to older residents [10].
The descriptive statistics describing the relationship between times of the day and aggressions have been reported to provide information about the time distribution of violent incidents. In the literature we already have similar examples, pointing out that incidents are more frequent at certain day times. For example, one study addressing inpatient violence in a psychiatric facility in England, covering a period of time of 17 years, revealed a peak in violent incidents during mealtimes, disbursement and staff shift-change [12]. This factor might be used in future models for violence prediction in psychiatric settings.
This study has several limitations. The predictive model adopts only internal factors, without exploring the external factors. Inpatient violence is a complex phenomenon, where the interaction of internal and external factors is an important variable. Whereas the internal factors relate to the individual characteristics as well as the state of the subject at the moment of the aggression, external factors refer to both environmental characteristics and circumstances [8]. According to the Royal College of Psychiatrists, some of the most important external predictors of inpatient violence refer to poor quality of interaction between patients and staff, lack of privacy, overcrowded environments, high use of temporary staff, and a lack of structured activities [26]. Other authors reported that wards with unclear staff functions and less predictable events, such as meetings or activities, showed a higher rate of inpatient violence [17]. We have reported that some locations show higher percentages of incidents, such as the hallway and the bedroom. It is possible to assume that these in locations human interactions are more frequent, increasing the risk of committing aggressions. Moreover, we believe it would be advisable to implement data from staff attitude and frustration. In fact, this study was conducted by taking into account only the point of view of clinicians. Poor staff-topatient interactions might contribute to the likelihood of aggressive behaviours [19]. McCann et al. [19] have examined the attitudes of clinical staff in relation to inpatient aggressions committed by elderly people with mental illness. They found that clinical staff showed contrasting attitudes about how to prevent and manage challenging behaviours. According to the authors, there is a direct relationship between attitudes and measures adopted to manage patients' challenging behaviours, stressing the importance of studying staff's attitudes. The authors also reported cultural misinterpretations within their sample, which can be attributed to poor interactions between staff and patients. In another study, staff frustration emerged as a predictor of time to inpatient aggressions [23]. According to the authors, the likelihood of inpatient aggression was three times higher when staff reported a state of frustration in managing the patient. Among the residents' factors associated with aggressive behaviours we suggest exploring the role of impulsivity and anger. Impulsive behaviour plays an important role in different serious mental illnesses, sometimes leading to violence or selfharm [15]. Ferguson et al. [9] found that impulsivity acted as a positive predictor of inpatient aggression.

Conclusions
This study represents an initial attempt, through which a specific set of data has been used to build a predictive model for inpatient risk of harm due to aggressive behaviours of patients. The utilization of a restricted set of predictors has shown a good overall accuracy, despite the sensitivity of the model needs to be improved significantly. The predictors utilized could be adopted as the basic structure for future models.

Compliance with Ethical Standards
Ethical Approval This research has been approved by the Hamilton Integrated Research Ethics Board (HIREB) and The Research Institute of St. Joseph's Hamilton.

Disclosure of Potential Conflicts of Interest
The authors declare no conflict of interest.
Research Involving Human Participants and/or Animals This is a retrospective study, conducted in collaboration with the St. Joseph's Healthcare Hamilton (Ontario), which supported this research granting access to patients' data. Patient records from January 2016 to December 2017 were reviewed.