Comprehensive Clinical Risk Management in a University Hospital: Implementation, Results and Outcome

DOI: https://doi.org/10.21203/rs.3.rs-111893/v1

Abstract

Background

Despite the increasing awareness of the need to implement systematic methods to reduce treatment-related risks, there are only few studies on comprehensive clinical risk management (CRM) in hospitals. The aim of this study was to report on the implementation, results and outcome of a CRM system at a university hospital in Tyrol, Austria.

Methods

CRM was rolled out in 35 departments of the hospital over a period of nine years and included the training of interdisciplinary risk management teams at each department, external and internal risk audits and the implementation of a critical incident reporting system (CIRS). The results of the risk audits were retrospectively reviewed and the risks identified were analyzed according to the type, severity and degree of implementation of risk management measures. Other key figures of CRM were obtained from the annual risk reports. The development of liability cases in the individual departments and in the entire hospital was used as an outcome measurement.

Results

Of the 1,104 risks identified during the risk audits, 620 (56.2%) were related to the organization of clinical care, 235 (21.3%) to documentation, 169 (15.3%) to patient treatment and 80 (7.2%) to patient information and consent. The highest proportion of serious risks was found in the category organization (22.7%), and the lowest in the category documentation (13.6%). The degree of implementation of prevention measures after 12 to 18 months was between 69.7% and 83.4% depending on the risk category. The CIRS identified between 241 and 370 critical incidents per year, for which in 79.5% to 83% preventive measures were implemented within twelve months.

Compared with the years prior to the introduction of CRM, an average annual reduction of treatment-related harms by 52.9% (95% confidence interval, 44.2% to 61.6%) was observed at the end of the implementation phase. The mean number of harms per department and year dropped from 7.04 to 3.45 (p<0.001).

Conclusions

The results of this project demonstrate the effectiveness of systematic CRM in detecting treatment-related risks and in reducing harm to patients. 

Background

Since the two Institute of Medicine reports "To Err is Human" and "Crossing the Quality Chasm" from 1999 and 2001, patient safety has become a priority issue for health care institutions [1, 2]. Subsequent studies using chart review methods confirmed that treatment-related adverse events are frequent and often have serious consequences for patients [3-8]. A systematic review showed that, in a total of 74,485 hospital patients, at least one adverse event occurred in 3.8% to 12.9% of cases (median, 9.2%). 7.0% of the events resulted in permanent disability, and 7.4% were lethal [9]. In surgical patients, the rate of adverse events was even higher at 11.7% to 23.2% [10]. Of particular significance is that 37.9% to 43.5% of adverse events were considered potentially preventable [9, 10].

In order to reduce the number of preventable adverse events, extensive efforts have been made to improve quality and reduce treatment-associated risks, both at the level of national health systems and individual health care organizations. Although these initiatives have raised awareness of the importance of patient safety, evidence of their effectiveness is limited [8, 11, 12]. While initially most projects and measures to increase patient safety were implemented in an uncoordinated way, many health organizations have now recognized the need to use systematic approaches to reduce risks and patient harm.

Among these systematic approaches, clinical risk management (CRM) has been among the most commonly used [13]. It includes the entirety of strategies, structures, processes, instruments and activities in prevention, diagnostics, therapy and care that support employees at all levels, functions and occupational groups in identifying, analyzing, assessing and managing risks in patient care [13]. In Austria, implementation of a risk-minimizing process for critical areas of care (e.g., medication safety, hygiene, handling of blood products, etc.) as well as an error management system have been defined as minimum requirements for quality management for all health care providers [14]. In a survey in 2017, 96% of acute-care hospitals stated that they had introduced CRM in their organization [15].

However, there are very few reports on the introduction and effectiveness of comprehensive CRM systems in hospitals [16, 17]. The purpose of this study was therefore (a) to report the implementation of CRM in clinical practice, (b) to describe the latent risks to patient safety according to number, type and severity, and (c) to assess whether CRM can lead to a reduction in treatment-related patient harm.

Methods

Study design and setting

A retrospective study was conducted to explore a quality improvement project at the University hospital Innsbruck. The hospital is a tertiary medical center that supplies mainly the Austrian province of Tyrol and neighboring regions of western Austria and Southern Tyrol. It includes 35 clinical departments with a total of 1,530 beds and 5.095 employees (full-time equivalents). In 2019, the number of inpatient admissions was 74,600.

Intervention

Following the decision to implement comprehensive risk management consisting of economic, infrastructural and clinical risk management, the department for strategic quality management was assigned the task of developing a risk strategy and an implementation plan for CRM. The risk strategy included the definition of the objectives, structures, processes and responsibilities of risk management in general and CRM in particular and described the integration of risk management into the organization´s management systems. The rollout of the system was centrally coordinated by the hospital management, while its operation was the responsibility of the respective departments.

The CRM system consisted of the following three components:

  1. Training and implementation of multi-professional risk teams: Training of clinical risk managers and clinical risk management assistants was carried out in accordance with the ONR 49003/ISO 31000 standard in three- to six-day in-house training courses. In each hospital department, at least one physician and one nurse or medical-technical staff were trained. The responsibilities of the risk managers in the implementation phase included the organisation of the risk assessments and the implementation and monitoring of risk prevention strategies. Risk management assistants supported the clinical risk managers in these activities. In the operational phase, both were members of the inter-professional risk management team.
  2. External risk audits: Risk analyses were performed at all departments by means of one- or two-day audits by experienced external auditors. The audits included semi-structured interviews, a review of medical records and other relevant documents and participatory observation by the auditors. Risks were assessed for the following categories: treatment processes, patient information, documentation and organization. All risks were categorized according to their severity and probability of occurrence and presented in a risk matrix using the riskala™ software (GRB - Gesellschaft für Risiko-Beratung mbH, Detmold, Germany).

The assessment of the degree of severity was based on the following criteria: serious (high probability of occurrence, high potential damage), moderate (medium probability of occurrence, medium potential damage) and low (low probability of occurrence, low potential damage). Together with the auditors, the risk managers developed preventive measures for all observed risks. In a repeat audit after 12 to 18 months, the implementation of these measures was evaluated and the severity of the existing risks was reassessed.

  1. Critical Incident Reporting System (CIRS): After completion of the risk assessment process, an intranet-based system for voluntary and anonymous reporting of critical incidents was implemented in all departments. Reported incidents were regularly discussed at meetings attended by the risk managers, the departmental management and representatives of the various professional groups. Risks that affected several departments or that could not be managed by the respective department on its own were reported in a standardized way to the quality management of the hospital administration and were further processed there.

To ensure the sustainability of the system, the risk managers prepared annual reports on the critical incidents reported and the measures taken to reduce risks. These reports were supplemented by other data sources, such as patient surveys, evaluations of patient complaints, infection statistics, fall statistics, and morbidity and mortality conferences. Based on these data, the hospital management annually defined hospital-wide projects and measures to increase patient safety. In addition, an external re-audit was carried out at all departments every five years after the initial implementation of CRM.

Data collection

The specific objective of introducing CRM was to reduce the frequency of treatment-related patient harm. We chose the annual number of harms reported to the liability insurance as the primary outcome parameter. According to the hospital guidelines, all known adverse events that result in temporary or permanent harm to patients must be reported to the hospital authorities. These cases are recorded in the annual statistics, whether or not a compensation has been granted by the liability insurance. The analysis of these incidents was carried out as a time series analysis covering the time before the implementation of CRM (baseline) and the implementation phase, and by comparing the frequency of adverse events occurring in the individual departments before and after the implementation of CRM.

Information on the number, type and severity of risks detected in the risk audits and the degree of risk reduction after implementation of appropriate measures was taken from the audit reports. Other information, such as the time of implementation of CRM in the individual departments, the number of CIRS reports per department and year, as well as the number of active risk managers and the number of meetings of the risk management teams per year was obtained from the annual risk management reports.

Statistical data analysis

Due to the retrospective study design, no statistical sample size calculation was performed. The data evaluation was therefore explorative. Statistical data analysis was performed using SPSS IBM Version 23. Descriptive statistics included frequency tables with averages, minima, maxima, and standard deviations (SD). For interval-scaled variables, the Kolmogorov-Smirnov test was used to check for normal distribution. The Wilcoxon signed-rank test was used for comparisons of dependent, non-normally distributed variables. Correlations between key figures of CRM at the individual hospital departments and the extent of harm reduction were analyzed using the Spearman rank correlation for non-normally distributed or ordinally scaled variables.

The total numbers of treatment-related harms in the years before the implementation of clinical risk management (2001 to 2010) were compared with the number of harms in the year 2019 at end of the implementation phase. A 95% confidence interval (CI) was calculated for the mean value of harm reduction. Time-dependent variations in the occurrence of harms were analyzed for two periods, that is, 2001 to 2010 (baseline) and the years 2011 to 2019 (implementation phase) using methods of statistical process control (quality control charts). These consist of a graphic display of the time course of the variable under investigation with plotting of the mean and the upper and lower control limits at ±3 standard deviations (SD). Since the measured value under investigation (i.e., the number of treatment-related harms per year) was a discrete variable from a relatively constant population (i.e., the number of patient admissions per year), the selected type of control chart was a c chart [18].

Identification of special cause variation was based on established criteria: 1. one or more data points outside the 3 SD control limits; 2. at least eight consecutive data points above or below the mean; 3. eight or more consecutive data points ascending or descending; 4. two of three consecutive data points outside 2 SD on the same side of the mean [18-20]. After identifying a significant change, the mean and upper and lower control limits were recalculated and the data points were examined for further non-random variations according to the above-mentioned rules. The calculation was performed with the software package QI Chart Version 2.0.22 for Microsoft Excel.

Results

Results of the risk audits

A total of 1,104 risks were identified during the risk audits. The number of risks per department varied between four and 90 (mean, 30.4; SD, 22.9). 620 risks (56.2%) were related to the organization of clinical care, 235 (21.3%) to documentation, 169 (15.3%) to patient treatment and 80 (7.2%) to patient information and consent. Overall, 208 risks (18.8%) were assessed as serious, 831 (75.3%) as moderate and 65 (5.9%) as low. The highest proportion of serious risks was found in the category organization (141/620, 22.7%), and the lowest one in the category documentation (23/169, 13.6%). A summary of the detected risks and the associated severity levels is shown in Table 1.

Table 1. Risks identified in the risk audits by type and severity

Risk Categories

 

Risk areas

 

Number of Risks Detected by Severity Grade

 

Low

Moderate

Serious

Treatment

Diagnostics and treatment planning

2

10

3

Pain management

3

1

0

Emergencies

0

16

9

Treatment guidelines and SOPs

9

72

14

Decubitus and fall prophylaxis, wound management

4

26

0

TOTAL

18

125

26

Documentation

Emergency admissions

0

9

0

Anesthesia documentation

0

3

2

OP documentation

0

13

1

Documentation in the hospital ward

17

135

16

Physiotherapy, logotherapy, ergotherapy

2

20

2

Functional areas (endoscopy, delivery room, catheter laboratory, etc.)

1

12

2

TOTAL

20

192

23

Patient information and consent

Patient information in the emergency department

0

3

3

General standards (risks and complications requiring disclosure, documentation of informed consent, information material, etc.)

1

41

13

Pre-operative patient information

1

2

2

Patient information in functional areas

0

14

0

TOTAL

2

60

18

Organization

Organization in the emergency department

1

20

10

Organization in the outpatient clinic

1

47

10

OP planning and coordination

2

58

11

Patient transport to the operation room, identification of the patient and the surgical site

1

36

21

OP organization
(anesthesia and wake-up room, handling of implants and prostheses, sterile goods processing, handling of tissue samples, outpatient surgery, etc. )

2

58

19

Organization in the intensive care unit

1

9

6

Organization in other functional areas (endoscopy, therapy, delivery room, etc.)

9

82

30

Intra-clinical emergency management
(resuscitation team, alarm system, emergency equipment, staff training)

1

10

7

Medication safety

0

51

11

Workforce planning and training

5

15

1

Safety of medical devices

0

5

0

Other organizational issues (communication, patient transport, infrastructure, hospital hygiene, etc.)

2

63

15

TOTAL

25

454

141

 All risks

 

65

831

208

During the audits, the degree of implementation of risk reduction measures was assessed for each risk by the auditors on a four-level scale (25%, 50%, 75%, and 100%). In order to estimate the global degree of risk control, mean values per department and risk category were calculated. At the time of the evaluation audit, the average degree of implementation of risk-reducing measures at the individual departments was 83.4% for risks associated with patient information, 71.8% for risks associated with treatment, 69.7% for risks associated with organization and 68.4% for risks associated with documentation.

Rollout of CRM

The CRM system was rolled out in the years 2011 to 2019. At the time of evaluation at the beginning of 2020, the system was therefore active at the individual departments between 6 and 84 months. In eleven departments, a re-audit after five years had already taken place.  Depending on the size of the departments, two to eight trained risk managers and risk management assistants were active. The number of meetings of the risk management teams varied between one and 24 per year (mean, 8.67; SD, 6.51).

The number of reported critical incidents per year increased from 241 in 2016 to 290 in 2017, 350 in 2018 and 370 in 2019. On average, 14.7 (range, 1 - 46) CIRS reports were recorded per department and year. The percentage of completely processed risk notifications remained largely constant in the years under review and ranged between 79.5% and 83%. The average number of CIRS reports at each department showed a significant correlation with the number of clinical risk managers per department (r= 0.679; p<0.001).

Development of patient harms

In the ten years prior to the introduction of clinical risk management (2001 to 2010), a total of 1,458 treatment-related harms were registered, that is, 95 to 186 harms per year (mean, 153; SD, 34.6). The total number of harms varied greatly between the departments, ranging from one to 234 (mean, 76.7; median, 53; SD: 66.8). 1,044 adverse events (71.6%) occurred at surgical departments, including gynecology and obstetrics and anesthesia.

In 2019, at the end of the roll out phase, 65 harms were recorded for the entire hospital. Compared to the years prior to the introduction of clinical risk management, this is a reduction between 31.6% and 65.1% (mean, 52.9%; 95% CI, 44.2%- 61.6%). To take into account the different times of implementation of clinical risk management, the mean number of harms in the years before and after the implementation of risk management was compared for each department. On average, 0.2 to 22.5 (mean, 7.04; SD, 6.14) harms per year and per department occurred before and 0 to 14.6 (mean, 3.45; SD, 3.66) occurred after the implementation of clinical risk management (p<0.001).

In order to analyze possible factors influencing the outcome of clinical risk management, key figures at the individual departments were correlated with the extent of harm reduction. Neither the time since the implementation of CRM nor the number of CIRS reports, the number of active risk managers or the number of meetings of the risk team showed a statistically significant correlation with the extent to which treatment-related harms were reduced (r= 0.12- 0.26; all p>0.05).

The temporal relationships between the implementation of clinical risk management and the development of treatment-related harms were investigated using a statistical process control chart (c chart). During the years 2001 to 2006, the number of harms per year showed considerable fluctuations with several data points outside the upper control limit, without significant changes in influencing variables such as the number of hospital beds or inpatient admissions. To avoid type 1 errors with respect to a potential effect of a quality improvement intervention, the control chart in the pre-intervention phase should indicate a stable baseline process [20, 21]. As no process stability could be assumed for the years 2000 to 2006, the period from 2007 to 2010 was chosen as the baseline for this analysis. From 2012 onwards, the number of annual harms was below the mean for eight consecutive years; in the years 2017- 2019 it was even below the lower control limit at 3 SD (Fig1 a).

According to the above-mentioned evaluation rules, this indicates a special cause variation. The temporal correlation with the rollout of risk management under otherwise constant conditions suggests that this change was caused by the implementation of clinical risk management. After adjustment of the mean value and standard deviations, the number of adverse events from 2012 onwards showed no indication of a special cause variation, which indicates a stable process (Fig. 1b).

Discussion

Our retrospective study showed a significant reduction of treatment-related harms following the introduction of CRM. The effect was visible in all hospital departments as well as on the level of the entire hospital. Despite the difficulties in measuring infrequent severe patient outcomes, such as treatment-related harms, data support an association between intervention and outcome.

Unlike many other projects to improve patient safety, this project took a comprehensive and systematic approach to clinical risk management. The German Patient Safety Coalition (Aktionsbündnis Patientensicherheit) defined the following requirements for clinical risk management systems in hospitals: 1. development and communication of a risk policy and risk management strategy; 2. definition of strategic and operative goals for clinical risk management; 3. definition of responsibilities and necessary resources; 4. development of competence and expertise for the identification, analysis, evaluation and management of risks; 5. establishment of a defined risk management process and integration of risk management into the management structures and processes of the organization; 6. participation, information and training of all professional groups; 7. continuous development and evaluation of the risk management system; 8. establishment of a positive safety culture [13]. A cross-sectional study of 138 Swiss hospitals identified the implementation of central coordination, established communication structures in and between the individual hospital facilities and the existence of a risk management strategy and strategic goals as key enablers for clinical risk management [22, 23].

In contrast, a survey of 572 German hospitals in 2015 revealed that, depending on the type of hospital, only 33% to 54% had systematically implemented a clinical risk management strategy and that in only 38% of hospitals the top management was involved in the project [24]. Only 13% of hospitals reported systematic use of information from the analysis of critical incidents, and only 14% reported the use of prospective risk analysis methods, although the vast majority had implemented a CIRS [24, 25]. These results show that, although the essential components of comprehensive clinical risk management are well defined, the extent and maturity of its implementation vary considerably between hospitals.

In this project, an attempt was already made at the planning stage to meet all the requirements of a comprehensive risk management system. Patient safety and clinical risk management were laid down in the corporate strategy, the risk policy and risk management strategy were described as part of the quality strategy, and the risk management processes were integrated into the management processes at corporate and hospital level. The development of expertise in the management of clinical risks was achieved by training a sufficiently large number of clinical risk managers and clinical risk management assistants. The recruitment of risk managers from different professional groups and the multi-professional composition of the risk management teams ensured a broad participation of all professional groups in the risk management process. In addition, the importance of patient safety and clinical risk management principles were communicated to employees through various communication channels, such as the intranet, newsletters, team meetings, training courses and lectures. The sustainability and continuous development of clinical risk management was ensured by annual internal and five-yearly external audits as well as by the annual determination of strategic goals and projects to increase patient safety.

This project used retrospective data, such as the analysis of CIRS reports, patient surveys, and the results of mobility and mortality conferences, as well as the systematic assessment, analysis and evaluation of clinical risks through risk audits. We classified the detected risks according to type and severity in order to get an overview of the hospital's risk profile. We could not find comparable data on risk profiles from other projects. Several studies reported on the number and type of adverse events detected by medical record review [4, 6, 26, 27]. However, the methods and classifications used are heterogeneous and not readily transferable to prospectively detected risks.

Among the risks identified in this project, risks due to organizational deficiencies were the most common (56.2%), with the highest proportion of serious risks (22.7%). This seems to be in contrast with a patient record study at 21 Dutch hospitals, which found that the majority of adverse events (61%) were caused by human factors, followed by patient-related factors with 39% and organizational factors with 14% [26]. However, active failures by people who are in direct contact with the patients and the medical system often only lead to damage in combination with latent conditions, that is weaknesses in the organization. Unlike active failures, whose specific forms are hard to predict, latent conditions can be identified and remedied before an adverse event occurs [28]. This is supported by the observation in the above-mentioned study, that the largest share of avoidable adverse events was among those with organizational causes (93%) [26].

There are hardly any previous reports on the outcome of CRM projects in hospitals. Cropper and colleagues reported on the implementation of a safety program in a large healthcare organization, which included some elements of clinical risk management, such as interprofessional risk management teams, a critical incident reporting system, safety critical policies and safety training. The authors reported a continuous decrease in the number of serious safety events recorded by the error reporting system after the introduction of the program [29]. Ramirez and colleagues reported on the introduction of an incident reporting system at a university hospital and evaluated the effectiveness of the resulting improvement actions through prospective real-time observations [17]. The authors found a significant reduction of patient safety incidents for 63.15% of the implemented safety measures. In contrast, a retrospective patient record review study at a department for cardiovascular surgery reported an increase in the rate of adverse events three years after the implementation of CRM from 21.1 to 42.8 events per 1,000 patient days [30].

Strengths and limitations

The strength of this study is that it reports on an extensive project carried out over nine years, which included all criteria and elements of a comprehensive risk management according to the current state of knowledge in a routine hospital setting. The detailed documentation of the prospective risk assessments allowed for the first time an assessment of the clinical risk profile of a tertiary care center. Together with the causes of adverse events with varying degrees of severity, as described in other studies, this can provide valuable information for future safety initiatives [9, 17, 26].

The study has some limitations. One limitation our study shares with many retrospective observational analyses is that the described intervention was carried out without the evaluation having been considered at the planning stage. The evaluation was therefore carried out retrospectively and had to be limited to the data available. For this reason, important aspects of a complex multiple intervention, such as effects on patient safety culture, could not be evaluated [31, 32].

Another limitation lies in the outcome measurements used for the evaluation. In our study, liability cases were used as a measure of treatment safety. This approach has some advantages and disadvantages. An advantage is the relatively high reliability of the data, since manifest damages are usually reported reliably. A retrospective review of 206 cases of medico-legal litigation showed that only 20% of adverse events were not reported to the hospital management [33]. Furthermore, liability cases usually concern serious adverse events with temporary or permanent damage or even fatal consequences, while 56% of adverse events detected with medical record review have no or only minimal consequences for patients [9]. In addition, reducing malpractice claims was a primary objective of this project, which could be measured directly by the assessment of liability cases.

One drawback is that liability cases, as compared to adverse events detected with chart review methods, represent a series of highly selected cases from which only limited conclusions can be drawn regarding general patient safety [34]. However, the effect of safety improvement measures on the reduction of litigation claims has been proven before, which suggests that liability cases can be used as surrogate parameters for patient safety [17].

Finally, our study design was observational, and therefore, outcomes may not only be affected by the intervention itself, but also by many other factors leading to a biased estimation of the intervention´s effect [35, 36]. In our study, the compared periods may vary in the number of inpatient and outpatient admissions, patient days, patient case mix, and internal and external cultural factors. We were not able to correct for these possible confounding factors. For the same reasons, generalizability is limited and our results may not be directly transferable to other hospitals, settings or countries. However, the close temporal relationship between the incidence of special cause variation in the control charts and the specific intervention makes it likely that the reduction in patient harm was at least partly due to the introduction of CRM.

Conclusions

In conclusion, the implementation of CRM in a university hospital resulted in a significant reduction in treatment-related patient harms. In contrast to the majority of other reported projects for increasing patient safety, this project met all requirements for a comprehensive quality management system, so that a sustained effectiveness of the intervention can be expected. These initial results must be confirmed through more detailed studies with chart review methods that can detect more adverse events in general and avoidable adverse events in particular.

Abbreviations

CRM: Clinical risk management; CIRS: Critical incident reporting system; CI: Confidence interval; SD: Standard deviation; UCL: Upper control limit; LCL: Lower control limit.

Declarations

Ethics approval

The study was reported to the Research Committee for Scientific Ethical Questions of UMIT – University for Health Sciences, Medical Informatics and Technology (reference No. 2762), which decided that no formal approval by an ethics committee was required because no personal data were used. The study was conducted within the framework of a cooperation project with the hospital organization Tirol Kliniken. Data protection issues were regulated by a data protection agreement.

Consent of publication

Not applicable

Availability of data and materials

The data generated and analyzed in this study are available from the corresponding author on reasonable request.

Competing interests

The authors declare that they have no competing interests.

Funding

This research did not receive any specific grant from funding agencies, commercial, or non-profit sectors.

Author´s contributions

The study was designed by WB. The data were collected by AS and MS. WB and US contributed in analyzing and interpreting the data. The manuscript was written by WB, US contributed to revising the manuscript. All authors have given their final approval of the version to be published.

Acknowledgents

We thank the University Hospital Innsbruck and Tirol Kliniken for their support in this research project.

References

  1. Kohn LT, Corrigan JL, Donaldson MS (eds.): To Err Is Human: Building a Safer Health System. Washington, DC: National Academies Press; 2000.
  2. Institute of Medicine, Committee on Quality of Health Care in America: Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001.
  3. Zegers M, de Bruijne MC, Wagner C, Hoonhout LH, Waaijman R, Smits M, Hout FA, Zwaan L, Christiaans-Dingelhoff I, Timmermans DR et al.: Adverse events and potentially preventable deaths in Dutch hospitals: results of a retrospective patient record review study. Qual Saf Health Care 2009; 18(4):297-302.
  4. Soop M, Fryksmark U, Koster M, Haglund B: The incidence of adverse events in Swedish hospitals: a retrospective medical record review study. Int J Qual Health Care 2009; 21(4):285-291.
  5. Rafter N, Hickey A, Conroy RM, Condell S, O'Connor P, Vaughan D, Walsh G, Williams DJ: The Irish National Adverse Events Study (INAES): the frequency and nature of adverse events in Irish hospitals-a retrospective record review study. BMJ Qual Saf 2017;26(2):111-119.
  6. Sari AB, Sheldon TA, Cracknell A, Turnbull A, Dobson Y, Grant C, Gray W, Richardson A: Extent, nature and consequences of adverse events: results of a retrospective casenote review in a large NHS hospital. Qual Saf Health Care 2007;16(6):434-439.
  7. Classen DC, Resar RK: The burden of harm. Jt Comm J Qual Patient Saf 2013;39(7):291.
  8. Landrigan CP, Parry GJ, Bones CB, Hackbarth AD, Goldmann DA, Sharek PJ: Temporal trends in rates of patient harm resulting from medical care. N Engl J Med 2010;363(22):2124-2134.
  9. De Vries EN, Ramrattan MA, Smorenburg SM, Gouma DJ, Boermeester MA: The incidence and nature of in-hospital adverse events: a systematic review. Qual Saf Health Care 2008;17(3):216-223.
  10. Anderson O, Davis R, Hanna GB, Vincent CA: Surgical adverse events: a systematic review. Am J Surg 2013;206(2):253-262.
  11. Vincent C, Aylin P, Franklin BD, Holmes A, Iskander S, Jacklin A, Moorthy K: Is health care getting safer? BMJ 2008;337:a2426.
  12. Baines R, Langelaan M, de Bruijne M, Spreeuwenberg P, Wagner C: How effective are patient safety initiatives? A retrospective patient record review study of changes to patient safety over time. BMJ Qual Saf 2015;24(9):561-571.
  13. Anforderung an klinische Risikomanagementsysteme im Krankenhaus [Requirements for clinical risk management systems in hospitals]. 2016. https://www.aps-ev.de/wp-content/uploads/2016/08/HE_Risikomanagement-1.pdf. Accessed 10 November 2020.
  14. Mindestanforderungen an Qualitätsmanagementsysteme im Rahmen des Bundes-Zielsteuerungsvertrages [Minimum requirements for quality management systems]. 2014.https://www.sozialministerium.at/Themen/Gesundheit/Gesundheitssystem/Gesundheitssystem-und-Qualitaetssicherung/Patientinnen--und-Patientensicherheit/Mindestanforderungen-an-Qualitaetsmanagementsysteme.html. Accessed 10 November 2020.
  15. Qualitätssysteme in Akutkrankenhäusern [Quality management systems in acute hospitals]. 2019. https://www.sozialministerium.at/Themen/Gesundheit/Gesundheitssystem/Gesundheitssystem-und-Qualitaetssicherung/Qualitaetsberichterstattung/Qualitaetssysteme-in-oesterreichischen-Akutkrankenanstalten-und-stationaeren-Rehabilitationseinrichtungen.html. Accessed 10 November 2020.
  16. Sendlhofer G, Brunner G, Tax C, Falzberger G, Smolle J, Leitgeb K, Kober B, Kamolz LP: Systematic implementation of clinical risk management in a large university hospital: the impact of risk managers. Wien Klin Wochenschr 2015;127(1-2):1-11.
  17. Ramirez E, Martin A, Villan Y, Lorente M, Ojeda J, Moro M, Vara C, Avenza M, Domingo MJ, Alonso P et al: Effectiveness and limitations of an incident-reporting system analyzed by local clinical safety leaders in a tertiary hospital: Prospective evaluation through real-time observations of patient safety incidents. Medicine (Baltimore) 2018;97(38):e12509.
  18. Mohammed MA, Worthington P, Woodall WH: Plotting basic control charts: tutorial notes for healthcare practitioners. Qual Saf Health Care 2008;17(2):137-145.
  19. Anhoj J, Olesen AV: Run charts revisited: a simulation study of run chart rules for detection of non-random variation in health care processes. PLoS One 2014;9(11):e113825.
  20. Benneyan JC, Lloyd RC, Plsek PE: Statistical process control as a tool for research and healthcare improvement. Qual Saf Health Care 2003;12(6):458-464.
  21. Koetsier A, van der Veer SN, Jager KJ, Peek N, de Keizer NF: Control charts in healthcare quality improvement. A systematic review on adherence to methodological criteria. Methods Inf Med 2012;51(3):189-198.
  22. Briner M, Manser T, Kessler O: Clinical risk management in hospitals: strategy, central coordination and dialogue as key enablers. J Eval Clin Pract 2013;19(2):363-369.
  23. Briner M, Kessler O, Pfeiffer Y, Wehner T, Manser T: Assessing hospitals' clinical risk management: Development of a monitoring instrument. BMC Health Serv Res 2010;10:337.
  24. Manser T, Frings J, Heuser G, Mc Dermott F: The German clinical risk management survey for hospitals: Implementation levels and areas for improvement in 2015. Z Evid Fortbild Qual Gesundheitswes 2016;114:28-38.
  25. Manser T, Imhof M, Lessing C, Briner M: A cross-national comparison of incident reporting systems implemented in German and Swiss hospitals. Int J Qual Health Care 2017;29(3):349-359.
  26. Smits M, Zegers M, Groenewegen PP, Timmermans DR, Zwaan L, van der Wal G, Wagner C: Exploring the causes of adverse events in hospitals and potential prevention strategies. Qual Saf Health Care 2010;19(5):e5.
  27. Albolino S, Tartaglia R, Bellandi T, Bianchini E, Fabbro G, Forni S, Cernuschi G, Biggeri A: Variability of adverse events in the public health-care service of the Tuscany region. Intern Emerg Med 2017;12(7):1033-1042.
  28. Reason J: Human error: models and management. BMJ 2000, 320(7237):768-770.
  29. Cropper DP, Harb NH, Said PA, Lemke JH, Shammas NW: Implementation of a patient safety program at a tertiary health system: a longitudinal analysis of interventions and serious safety events. J Healthc Risk Manag 2018;37(4):17-24.
  30. Hoffmann-Völkl G, Kästenbauer T, Mück U, Zottl M, Huf W, Ettl B: Detection of adverse events using IHI Global Trigger Tool during the adoption of a risk management system: a retrospective study over three years at a department for cardiovascular surgery in Vienna. Z Evid Fortbild Qual Gesundheitswes 2018;131-132:38-45.
  31. Euteneier A: Sicherheitskultur und klinisches Risikomanagement [Culture of safety and clinical risk management]. Unfallchirurg 2020;123(1):22-28.
  32. Weaver SJ, Lubomksi LH, Wilson RF, Pfoh ER, Martinez KA, Dy SM: Promoting a culture of safety as a patient safety strategy: a systematic review. Ann Intern Med 2013;158(5.2):369-374.
  33. Bolcato M, Fassina G, Rodriguez D, Russo M, Aprile A: The contribution of legal medicine in clinical risk management. BMC Health Serv Res 2019;19(1):85.
  34. Thomas EJ, Petersen LA: Measuring errors and adverse events in health care. J Gen Intern Med 2003;18(1):61-67.
  35. Cox E, Martin BC, Van Staa T, Garbe E, Siebert U, Johnson ML: Good research practices for comparative effectiveness research: approaches to mitigate bias and confounding in the design of nonrandomized studies of treatment effects using secondary data sources: the International Society for Pharmacoeconomics and Outcomes Research Good Research Practices for Retrospective Database Analysis Task Force Report Part II. Value Health 2009;12(8):1053-1061.
  36. Johnson ML, Crown W, Martin BC, Dormuth CR, Siebert U: Good research practices for comparative effectiveness research: analytic methods to improve causal inference from nonrandomized studies of treatment effects using secondary data sources: the ISPOR Good Research Practices for Retrospective Database Analysis Task Force Report Part III. Value Health 2009;12(8):1062-1073.