The primary search of the literature yielded 4780 citations. Review of abstracts led to the retrieval of 276 full-text articles for assessment, of which 123 articles were retained for review. The most common reason for excluding articles after full-text review was the absence of an operational QI (FIGURE 1).
The majority of articles reviewed were original research (98.3%, 121), of which 88.6% (109) were cohort studies (TABLE 1).
The majority of the literature was reported from registries evaluating quality of critical care in high income country healthcare systems with 27.85% of research originating from Australia, Canada, Northern Europe and the United States of America (FIGURE 2).
QIs, definition, measurement and evidence base.
From the 123 articles retained for full review, 253 indicators were identified, of which 51 were unique. These unique indicators were classified using prespecified components of the HQSS framework [13] and then by phase of ICU encounter (pre, in, and post ICU) (TABLE 2) [13].
Foundational indicators accounted for 13.7% (7) of the 51 unique QIs identified, processes 54.9% (28) and quality impact 31.3% (16) respectively. Health care associated events (including healthcare associated infections) accounted for 37.1% (10) of process indicators and were present in 39.8% (49) of articles included in full text review. In contrast, patient reported outcome related measures accounted for less than 6% of quality impact indicators and were present in 1.6% (2) of articles reviewed.
The majority of QIs (58%) had a single definition and method of measurement. The mean number of different definitions for a single indicator was 1.61 (SD 0.72 to 2.50). Indicators with the most variation in definition were composite measures of mortality and adverse events; including hospital associated adverse events. Definitions varied based on inclusion or exclusion of laboratory tests, radiological imaging and constellations of clinical signs and symptoms, and varied depending on the country from which the underpinning evidence originated (TABLE 3). Of the 123 articles included in the full review, 96% (118) included a description of how the indicators were measured, and a further 88% (108) reported guidance on data collection (including frequency) and analysis.
Barriers to implementation of quality indicators
Barriers to implementation were described in 35.7% of articles included in the full text review. The barriers identified broadly related to two aspects of implementation; “operational” (51%) (which included issues with data collection and data quality, and “acceptability and actionability” (49%) (which included users' perceptions of the indicator, its validity and the subsequent actionability of the information for care improvement) (TABLE 4).
Operational
Reliance on manual data entry (as opposed to direct extraction from an electronic source-such as patient monitor or EHR) and the associated burden of data capture were reported impediments to operationalisation of the QIs in the ICU [70,101,117]. Absence of data (missingness) resulting in exclusion of patient encounters from analysis, was also described; inhibiting both implementation and utilisation of QIs. Missingness was described both as a consequence of manual data entry and as a result of unavailability of information at source [31,90,96,108]. The impact of this missingness being that several studies had to exclude data in analysis [106,111,124,125]. Challenges with data quality were described at point of data capture and extraction [30,33,34,40,63,84,87,90,101,108,118,120,143]. Ability to accurately measure processes of care associated with the indicator (e.g., time of antibiotic administration in the context of recognition of infection/ prescription of antibiotics) hindered utilization of the indicator in care evaluations [32,46,50,51,58,98, 103, 120, 144, 145].
Acceptability and Actionability
Concerns about the ramifications for individual clinicians and ICU team performance was a barrier to implementing indicators of quality in ICU. Indicators relating to adverse events including hospital associated (nosocomial) infections and to measures of performance for which there may be significant public pressure (such as antimicrobial prescribing) were particularly associated with fear of blame [70]. These concerns impeded healthcare providers willingness to engage in the reporting of incidence and associated quality of care benchmarking within their department and a willingness to contribute data to interdepartmental and interhospital reporting (regional and national) [42,61,63,70,75,115]. Similarly, it was identified as a contributing factor to missingness of data (described above) and a driver for revision of definitions [30,31,92]. These barriers were most described in literature originating from North America and China.
Validity of the data was a barrier to acceptability (and actionability) of the indicators. Concerns over information that was not captured electronically, but by direct observation from clinicians delivering care was considered at risk of tarnish from self-reporting and responder biases [42,62]. These concerns hindered researchers and clinicians' willingness to accept findings, and in turn, impeded subsequent efforts to use the data to drive quality improvement initiatives in the clinical settings [35,70]. Challenges in interpretability of indicators, which further limited acceptability were also identified. Indicators relating to ICU resource utilisation (occupancy, turnover and staff utilisation) were described as “difficult” to interpret given the dynamic nature of ICU service activity and patient acuity [103,120]. Similarly, indicators pertaining to hospital associated infections (and other adverse events) were described as undergoing frequent revision of definition and therefore data capture methods, due to difficulties in interpreting the findings in the context of patient groups and care processes [30,31,92,96]. As a consequence, these indicators were associated with frequent revision of definition and method of measurement. Interconnected, these revisions to definitions and data set further impeded quality of data collection and their acceptability of the findings for the healthcare team [92,96].