Our First Review in 12 Years: An Evaluation of Effectiveness of Root Cause Analysis in Hong Kong Public Hospitals

Background: To evaluate the effectiveness of root cause analysis (RCA) and propose possible ways to enhance its effectiveness in Hong Kong public hospitals. Methods: A retrospective cross-sectional study was performed across 43 public hospitals and institutes in Hong Kong, reviewing RCA reports of all Sentinel Events and Serious Untoward Events within a two-year period. The incident nature, types of root causes and strengths of recommendations were analysed. The RCA recommendations were categorised as ‘strong’, ‘medium’ or ‘weak’ strengths utilizing the US’s Veteran Affairs National Center for Patient Safety action hierarchy. Results: A total of 214 reports from October 2016 to September 2018 were reviewed. These reports generated 504 root causes, averaging 2.4 per RCA report, and comprising 282 (49%) system, 233 (46%) staff behavioural and 22 (4%) patient factors. There were 658 recommendations identified in the RCA reports with an average of 3.1 per RCA. Of these, 18 (2%) recommendations were rated strong, 116 (15%) medium and 626 (82%) weak. Most recommendations were related to ‘training and education’ (466, 61%), ‘additional study/review’ (104, 14%) and ‘review/enhancement of policy/guideline’ (39, 5%). Conclusions: This study provided insights about the effectiveness of RCA across all public hospitals in Hong Kong. The results showed a high proportion of root causes were attributed to staff behavioural factors and most of the recommendations were weak. The reasons include the lack of training, tools and expertise, appropriateness of panel composition, and complicated processes in carrying out large scale improvements. The Review Team suggested conducting regular RCA training, adopting easy-to-use tools, enhancing panel composition with human factors expertise, promoting an organization-wide safety culture to staff and aggregating analysis of incidents as possible improvement This study is the first study evaluating the effectiveness of RCA in Hong Kong public hospitals. This study used a robust methodology to analyse the root causes and recommendations of RCA reports. This study highlighted the key factors contributing to the distribution of strengths of recommendations. The study only reviewed the strengths of recommendations but not the work progresses and feasibilities. There were no other local data available to benchmark the results.


Strengths And Limitations Of This Study
This study is the first study evaluating the effectiveness of RCA in Hong Kong public hospitals.
This study used a robust methodology to analyse the root causes and recommendations of RCA reports.
This study highlighted the key factors contributing to the distribution of strengths of recommendations.
The study only reviewed the strengths of recommendations but not the work progresses and feasibilities.
There were no other local data available to benchmark the results.

Background
Ensuring patient safety is always a major challenge for healthcare organisations. With the arousal of awareness in patient safety since the release of the Institute of Medicine report in 1999, initiatives in patient safety improvement have been launched in countries all over the world. [1][2][3][4] These patient safety initiatives may have arisen from recommendations identified through root cause analysis (RCA) which identified system vulnerabilities from past clinical incidents.
RCA is a an investigation methodology commonly used in healthcare organisations aiming to learn from system failures of clinical incidents, and identify patient safety improvement initiatives leading to elimination or control of the risks leading to such events. [5][6][7][8] It analyses incidents retrospectively in a systematic approach under a just and fair environment. The products of an RCA are the contributory factors proximally leading to the incident, the root causes about latent factors from the system perspective and the action plans to prevent recurrence of similar incidents in the future. [8][9][10][11][12] Many hospitals use RCA to investigate adverse events, with an example being the U.S. Joint Commission requiring incidents to be systematically investigated. [13] Although organisations might have put vast resources, in terms of time and manpower, [10,14] in conducting RCAs, the number of preventable clinical incidents did not seem to reduce as would have expected. Researchers speculated that this might be due to the low effectiveness of RCA, and studies evaluating the strengths of recommendations found a high proportion of weak recommendations were put forward. [10,14,15] The low RCA effectiveness was widely discussed and suggested to be caused by the limited focus of weak recommendations that tackle superficial issues, mainly attempting to modify human behaviour which is believed to be difficult to sustain and cannot effectively eliminate the inherent latent risks at the system level. [10,12,16,17] The identified factors preventing the development of stronger recommendations include the flawed design and context of RCA, the lack of RCA training and expertise, poor leadership and the loose mechanisms in promoting a safety culture. [10,14,16,18,19] In Hong Kong, all 43 [20,22] With RCA having been introduced in the HA for more than a decade, there has been doubts within the organisation about the effectiveness of RCAs in promoting patient safety and its sustainability. Since no corresponding research study in evaluating the effectiveness of RCA has ever been conducted in Hong Kong, the objectives of this study were to evaluate the effectiveness of RCA and propose suggestions for improving the quality of incident investigations in Hong Kong.

Study Design
A retrospective study of RCA reports of all SEs and SUEs reported in the HA across a two year period was undertaken by an RCA Reports Review Team (the Review Team). The incident nature, types of root causes and strengths of recommendations were analysed.
The root causes (in some RCA reports they were referred to as 'contributory factors', and for simplification purposes these are also regarded as 'root causes' in this study) of the events were reviewed and categorised according to the Contributory Factors Classification Other classification methods like the Human Factors Analysis Classification System (HFACS) and Conceptual Framework for the International Classification for Patient Safety of the World Health Organisation (WHO) were considered for analysis but were not chosen. [13,24] The HFACS was not chosen because some of the root causes were difficult to classify in a trial analysis using some HA RCA reports due to the vague description of the root causes. The NPSA method is more comprehensive in each category with explanation than the WHO Framework, thus the former method is selected.
In evaluating the strengths of RCA recommendations, the action hierarchy of the United States of America's Veteran Affairs National Center for Patient Safety was selected as this method is widely accepted internationally.
[12] In our study, some local adaptations were made, for example, recommendation types from Hibbert's study were included for easier classification of the action hierarchy. [10,23] In the evaluation, a 'strong', 'medium' or 'weak' strength is given to each recommendation by the Review Team. 'Strong' recommendations indicate the recommendations focus on system changes and are more sustainable while 'weak' recommendations are focusing on human behavioural change and are less effective in eliminating the risks. The 'medium' recommendations, e.g. a checklist, have elements of system improvement but still rely on human compliance to be effective.
The research was done without patient or public involvement.  Table 1. Nearly two-thirds (137, 64%) of the events were SUEs related to 'medication error which could have led to death or permanent harm', followed by 29 (14%) SEs related to 'retained instruments or other material after surgery / interventional procedure'. There was no SE related to 'medication error resulting in major permanent loss of function or death' in the review period. These RCA reports generated 504 root causes, averaging 2.35 per RCA report. The distribution of root causes by types is listed in Table 2. In the Task Factor of the NPSA method, the Review Team separately extracted one sub-factor related to policy or guideline adherence in Task Factor for analysis. This sub-factor, which is related to staff's non-compliance with the existing policy or guidelines (the 'violation' root causes), was the most common factor amongst the root causes (152, 30%), followed by education and training (87, 17%) and staff factor (81, 16%). According to the nature of the 9 different factors in the NPSA method, the Review Team has grouped these factors into three major groups, namely the Patient Factors, the Staff Behavioural Factors and the System Factors.
Since in error taxonomy, 'violation' is a type of human failure, the Review Team thus grouped the 'violation' factor into the Staff Behavioural Factors Group. The distribution of root causes according to our grouping was 49% (282/504), 46% (233/504) and 4% (22/504) in System Factors, Staff Behavioural Factors and Patient Factors respectively.

Discussion
This study is the first research in Hong Kong in understanding the effectiveness of RCA since its mandatory use for investigating SEs in 2007. The results of this study, especially in the categorization of root causes and recommendations, can provide meaningful information for the Hong Kong in improving its quality in incident investigations and subsequent risk mitigation actions.
One of the major challenges for the RCA panels is to identify the system vulnerabilities contributing to the underlying latent failures of the organisation. [14,25] According to the results by our grouping, about 46% of the root causes were identified to be related to staff behavioural factors, for example, violation, lack of vigilance and lapse of concentration. In some RCA reports, the Review Team noticed that only staff behavioural factors were identified and no other system factors had been identified. In fact, human failures like slips, lapse and mistakes are normal human behaviour and can be difficult to eliminate. [26] The identification of such root causes is only superficial and only demonstrates that humans are imperfect, but is not meaningful in solving the problem. [14] The large proportion of staff behavioural factors suggests that the RCA panels were not able to recognize different aspects of systems issues such as equipment and workflow design flaws, poor usability of system interface, work overload and inadequate safety culture. [10,14,16,18] This observation in the HA can be explained by the reason that the last corporate-led RCA training in the HA was conducted in 2009 and most RCA members have not been formally trained in RCA investigations in the past decade given the lack of training opportunities. This is especially true for the clinical experts who are invited to join the RCA according to their respective clinical expertise. These experts, generally with more than 10 years of clinical experience, would have limited understanding in RCA, safety systems knowledge and improvement science, as these have only taught in the medical undergraduate curriculum in the recent decade in Hong Kong. [27] The 'violation' root causes likely demonstrates the misconception to the term 'violation' amongst RCA panels in our review. From a human factors perspective, 'violations' are deviations from safe operating practices, procedures, standards, or rules, and have to be deliberately performed by the staff. [26] The Review Team noticed many RCA reports concluded that the staff had violated the policy or guideline, which was contributed by the causal factors of staff 'having forgotten to perform a checking step' or 'not being aware of the situation'. There was no further investigation on the reasons for 'violation' nor was evidence of the staff's intention to deliberately violate the rules provided. This  [33][34][35][36][37][38] Currently, the HA RCA report template does not provide any of the above tools for the RCA panel to make reference. In many studies and organisations' incident management guide, human factors considerations are key components in conducting a robust RCA while quality improvement expertise is vital to RCA panel like the example above, the training should explain the risks and consequences of not communicating effectively with other staff during a procedure, and teach the necessary knowledge and skills required to address this. [42] Other system factor root causes were also found to propose weak recommendations in our review. Studies suggested that the tendency of RCA panels to propose weak recommendations is generally caused by the lack of understanding in RCA, limited knowledge in the hierarchy of controls and human factors. [10,14,16,28] The RCA panels might perceive the investigation to be restricted to within the department such that organisational issues at a broader level were ignored for discussion. Within such confines, the choices of recommendations implementable solely at the departmental level become limited and additional staff training or putting more reminders has been the prominent actions arising from RCAs. [19] Stronger recommendations are also known to be costly and require more attention and monitoring to complete. [37,38,43] Though the RCA Panels are nominated directly by Hospital Chief Executives in the HA, political considerations for driving organisational change are often required for stronger recommendations, and those affecting fundamental systems of the whole HA may be difficult to ignore. For example, all hospitals and institutes in the HA use the same electronic clinical management system for patient documentation and management. When an RCA panel has identified a loophole in the electronic clinical management system, instead of directly asking the Information Technology Department to make the corrective actions, the proposed actions have to go through a series of processes including a number of platforms at hospital and corporate levels for stakeholder consultation, seek approval from different hospital and organisational committees, and lastly conduct a series of feasibility tests by software technicians. This process usually takes months if not years to complete. During such processes, hospital staff may question the RCA panel's capability as they believe that the panel has not identified the appropriate root causes, and some observers may still assign blame to the involved staff as this is more visible. [16,19] In addition, staff may blame the RCA panel for introducing new or perceived unrealistic changes at the organisational level for one single adverse event which they believe to be contributed to by a staff's mistake. [17] The organisation might also have difficulty in assessing the vulnerability of the system by one single event. [16] The RCAs investigating individual, similar incidents during this review were found to produce inconsistent recommendations. An example would be in incidents where known drugs that cause allergic reactions to patients were prescribed and administered. The causes were found to be due to the input of the allergen in the "free text" section of the electronic medication management system, and thus could not automatically check cross-sensitivity of the drugs to alert staff. However, in some reports the recommendations were to provide training or reinforce staff compliance in checking allergy histories, while some reports suggested converting the free text allergy entries into structured entries to enable drug allergy checking by the electronic system. These inconsistent recommendations may be a product can also be added to the existing HA RCA report template to facilitate ease of access and serves as a prompt for use. Promotion of no-blame culture to all hospital staff must also be carried out to encourage staff in focusing on system issues instead of blaming individuals. [16] Skills in writing RCA reports have to be developed with practice so that the root causes and recommendations can be written more specifically and enable a new reader to immediately understand the issues and solutions. [18] Second, members with human factors expertise should be invited to join RCA panels as it may help shift any focus on blaming individuals to identifying systems and design flaws.
This expertise can also support the design of patient safety initiatives. [10,16,44] Inviting staff who understand the involved workflows can also help the RCA panel quickly understand the detailed nuances of the situation.
[12] Training a core group of staff specialized in incident investigations within the organisation would be a possible solution to solve the RCA expertise and sustainability problems. This core group of "specialists" can retain their knowledge and skills in RCA and they would have more opportunities to participate in an RCA. [18] From a broader perspective, an independent institution as a third party, with positioned similarly to the National Aeronautics and Space Administration of the US or the Healthcare Safety Investigation Branch of the UK, [45,46] in the territory could be developed and hospitals can draw on expertise from this institution to support incident investigations as an independent party. The advantages of these independent institutions include minimising internal conflict, promoting unbiasedness, accuracy and credibility of RCA findings, and separate incident investigations and learning from violation disciplinary actions. [47] Third, the HA must promote a safety culture to all staff, which includes understanding the goals of a RCA, and reduce the barriers of RCA panels in proposing stronger recommendations focusing on organisational changes. [43] Fourth, the HA should consider aggregating analysis of incidents to counter the inconsistencies that will inevitably arise from multiple RCAs investigating similar issues. [6,10,16,37,38] By implementing these suggestions, the Review Team believes that incident investigations and improvements arising from them will become more effective in the organisation. The root causes will be more focused on system defects and a higher proportion of strong or medium recommendations would be anticipated. Last but not least, Australia has reviewed its SE list in 2017 and launched a second version in December 2018. [48,49] With the current SE and SUE lists being implemented in Hong Kong for about a decade, the list should be reviewed to ensure it aligns with the goal of effectively monitoring and preventing serious adverse patient harm events.
This study has two limitations. First, the study only reviewed the strengths of recommendations but not the work progresses and feasibilities. Due to the boundary of action hierarchy in categorizing recommendations of 'additional study / analysis' as weak, the Review Team believed that there will be more strong or medium actions taken when the recommendations are studied to be feasible. Second, this study is the first study in the HA to review RCA effectiveness. There was no other local data available for

AVAILABILITY OF DATA AND MATERIALS
The data that support the findings of this study are available from the Hong Kong Hospital Authority but restrictions apply to the availability of these data, which were used only for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of the Hospital Authority.