Identifying Leading Indicators of System Resilience and the Strategy for Developing Them: A Review of Reviews

Background: System resilience describes the performance or capacity of a system to absorb and adapt in response to stressors and perturbations. This protocol outlines the search strategy to identify indicators or metrics of system resilience from across diverse elds and examine the methods/processes for developing these indicators or metrics. Methods: This protocol is grounded in the Joanna Briggs Institute (JBI) Scoping Review methodology. We will search Web of Science and Scopus for reviews that report on system resilience assessment and measurement techniques. We will also employ snowball sampling techniques. Due to the breadth of the topic, this search will be limited to systematic reviews, meta-analyses, or literature reviews with a structured, documented search strategy reporting on primary research studies published in the English language. We will not limit the search by date or discipline. Reviews that report on psychological resilience or resilience at the level of an individual will be excluded. Discussion: The ability to assess or measure resilience is important to identify risks, opportunities, and strategies to inuence sustained, reliable, and predictable sy stem operations. We hypothesise that there are commonalities in the types of indicators used and how they are developed. These patterns may have relevance to other elds, like healthcare. The ndings from this review may help others develop their own indicators and assessment strategies for understanding resilience within their own systems. Ethics and Dissemination: This study is ethics exempt. We will disseminate the results of this work at national and international quality and safety conferences, and through publication of a manuscript.


Background
The study of system resilience involves a focus on the performance of systems (human, animal, natural, technical, and otherwise) in response to perturbation [1]. This eld of research originated in ecology (ecological resilience) in the 1970s [2]. The concept has spread to various related and unrelated disciplines since, and has grown exponentially over the last two decade [1]. The emphasis on system resilience is different than individual or psychological resilience, where the former strives to understand the interdependencies and adaptations of various parts, and the latter strives to understand the considerations that affect the adaptation of a singular unit [3]. System resilience recognises the in uence of complexity, where an ability to cope with the unexpected can be a critical safeguard against system failure or extinction [2,4].
Globally, healthcare systems have grown increasingly complex. Paradoxically, this is often a consequence of adopting improvements and innovations to respond to problems and concerns. The move from paper charting to electronic charting provides a contemporary example; although a necessary improvement to support continuity of care, the interface between humans and technology, new and emergent processes and work ows, inter-and intra-disciplinary membership, the ethics of access, and many other micro, meso and macro considerations means that no part of the equation is constant, independent, or predictable [5]. In all aspects of healthcare delivery individuals and small groups (e.g., doctors, nurses, patients, managers, carers) strategise and self-organise to adapt to increasingly complex systems [6]. Cumulatively, the result of these micro strategic adjustments emerges in new global system level patterns; hence, healthcare is recognised as a complex adaptive system.

Studying Resilience
Across disciplines, the de nition and conceptualisation of system resilience, or simply resilience, varies [7,8]. In ecology, resilience describes an organisms' ability to avoid extinction [2]. In mechanical engineering, resilience requires consistent, non-variable performance as measured against a standard [9]. In the eld of disaster management, resilience describes a return to normal societal function following a disaster or extreme event [10]. Weather patterns, computer networks, and societal systems can also demonstrate resilience but de nitions of what this entails vary [11][12][13]. Challengingly, even within the same eld of study, multiple de nitions for resilience may exist [14].
Some recommend caution studying resilience, suggesting that searching for commonalities between the conceptualisations of resilience across diverse elds may contribute to theoretical over-reach, diluting the de nition, and rendering it meaningless [15]. Others argue that there is enough of a common thread between the diverse conceptualisations of resilience that despite differences, essential elements remain constant [16]. All de nitions, for example, seem to acknowledge: (a) the context of a complex system, (b) the in uence of a stressor or perturbation, (c) the potential departure from a normal functional state, and (d) the adaptive capacity of parts within a system to absorb and adapt to change [15]. Resilient systems have been characterised as strong, reliable, robust, dependable and safe systems [10,[17][18][19]. High quality reviews [8,16,20,21] have been conducted that pull together de nitions and methods of studying system resilience across diverse elds. However, no review has synthesised "what it takes" to operate resiliently within a complex system. The indicators or metrics of system resilience, and methods of developing indicators or metrics of system resilience, remains a mystery.

Resilient Health Care (RHC)
The study of system resilience is new, emergent, and growing within healthcare [1,18]. Hollnagel (2015) de ned RHC as "the ability of a system (a clinic, a ward, a hospital, a country) to adjust its functioning prior to, during, or following events (changes, disturbances or opportunities), and thereby sustain required operations under both expected and unexpected conditions" (p.xxvii) [22]. The RHC eld has been especially in uential in shifting the safety science approach from a focus on reactive risk management to a proactive focus on system design to foster predictable performance [22,23]. This is important because, globally, 57% of people who seek healthcare are at risk of receiving care that does not align with best practice [24], and approximately 1 in 10 patients admitted to hospitals around the world are expected to suffer from adverse events, many of them preventable [25].

The Epistemology of Resilient Health Care
Recently, academics and theorists strive to assess health systems for their resilience, and debate the methodological accuracy of measuring resilience. One side of this debate is grounded in constructivist epistemology. Those who subscribe to this paradigm believe that the resilience of a system can only be known by understanding collective resilient behaviours of agents of that system in situ. Cook and Ekstead (2017) suggest that the state of resilience research in healthcare does not allow us to separate resilient behaviours and characteristic (or ultimately indicators or metrics), from the agents, tools, materials, facilities, and information that comprise that system [26]. These authors argue that resilience is not static, but an ongoing behaviour, which cannot be measured at a single point in time [8]. Thus, it is impossible to provide reliable and accurate measures or metrics of resilience because healthcare is "more complex than the rules that seek to manage it" (p. 117) [26]. From this perspective, resilience cannot be measured; like culture, it is a gestalt that can only be observed in behaviours and within relationships that form the parts that make up the greater whole. This paradigm is currently popular in RHC where authors typically study resilient behaviours of systems by qualitatively assessing at the micro (individual) or meso (hospital/institution) levels [20].
A shift in paradigm from the constructivist lens to the post-positivist lens, accepting that resilience can be empirically measured, assessed, and tested, may provide a useful path forward. Understanding indicators of a system's resilience can enable a system's performance in response to changes, disturbances or opportunities to be anticipated. This, in turn, will facilitate the ability to (a) appraise a system's adaptive capacity prior to the implementation of small-or large-scale changes, and, most importantly, (b) design and develop more intelligent, resilient, and most importantly, safe systems. In other words, if we can de ne discrete indicators or metrics of system resilience in general, we can unlock clues for designing and fostering resilience within speci c systems, such as healthcare.
The objective of this scoping review is to map the research evidence on resilience indicators and metrics provided in reviews across diverse and interdisciplinary elds. Speci cally, we aim to synthesise ndings from reviews of research into resilience assessment or measurement. We anticipate gross heterogeneity in identi ed indicators, therefore, we will also catalogue the strategy or methods used to develop indicators for resilience where we nd them.
A preliminary search of PROSPERO, MEDLINE, Scopus, Web of Science, the Cochrane Database of Systematic Reviews, and the Joanna Brigs Institute (JBI) Database of Systematic Reviews and Implementation Reports was conducted and no current or underway systematic reviews or scoping reviews on the topic were identi ed.

Review questions
What indicators or metrics are associated with system resilience across diverse elds? How were these indicators or metrics of resilience developed?

Inclusion Criteria
Population/Topic This review will consider reviews of studies that are published across diverse disciplines, including, but not limited to, ecology, biology, disaster management, business, engineering, computer sciences, social settings, and healthcare. We will exclude reviews that focus on resilience in simple or linear systems, or research with analysis at the individual or singular level. For example, personal or psychological resilience will be excluded.

Concept
The concept we are studying, and the main focus of the review, is system resilience. As a property or behaviour of complex systems, system resilience acknowledges the signi cance of multiple, interacting agents and in uences, and the effect of these on a system's function and performance [6]. Complex systems are those that demonstrate interdependence and interconnectedness between agents, with emergent outcomes resulting from these relationships, in combination with adaptation, and co-evolution [27]. We will also include ideas related to system resilience, such as adaptability, sustainability, and other system responses to stress or challenge.

Context
Due to the breadth of the topic, we will limit this review to include only structured reviews of primary, empirical research that focus on systems resilience or related topics, and descriptive or exploratory primary research that assesses the qualities of resilience in a way that identi es resilience indicators or metrics. We will exclude reviews of purely theoretical, conceptual, or descriptive frameworks and models that have not been empirically applied or tested.

Types of Sources
For this scoping review, we will consider all review articles published in English in peer-reviewed journals. This includes reviews of analytic observational studies such as prospective and retrospective cohort studies, case-control studies and analytical cross-sectional studies, and descriptive observational study designs including case series, individual case reports and descriptive cross-sectional studies. To be included, reviews must be structured literature reviews that include a description of their search and synthesis methods. To limit biases, we will exclude reviews of the literature that do not report on their search methods. We will also exclude conceptual papers, reviews of country documents, policies and other non-empirical reviews. We will not limit this review by date, as seminal ecological resilience studies and others published throughout diverse elds since the 1970s are integral to knowing the eld of system resilience. See Additional le 1 for a summary table of all inclusion and exclusion criteria.

Methods And Design
The proposed scoping review will be conducted in accordance with the JBI methodology for scoping reviews [28].

Search Strategy and Information Sources
An initial limited search of Web of Science and Scopus was conducted in July to August, 2019 to identify articles on the topic. The words contained in the titles and abstracts of relevant articles, and the index terms used to describe the articles were used to develop a full search strategy (see Additional le 2) in consultation with a librarian. The search strategy, including all identi ed keywords and index terms, will be adapted for use within each included database.
To achievably scope this review, the search strategy will aim to locate published, peer-reviewed reviews of metrics and indicators for assessing system resilience. We will use a snowballing approach to nd additional studies from the reference lists of all included reviews. We will search all databases within the Web of Science (including MEDLINE and BIOSIS Citation Index) and the Scopus database. Web of Science and Scopus were selected for their breadth of high-quality, cross-disciplinary publications from the life sciences, social sciences, physical sciences and health sciences. The proposed study timeline from search to manuscript preparation is November, 2019 to March, 2020.

Study selection
Following the search, all identi ed citations will be collated and downloaded into EndNote (version X9/2019) (Clarivate Analytics, PA, USA) and duplicates removed. Citations will then be imported into Rayyan [29]. Reviews will then be divided among six reviewers. To ensure inclusion criteria (summarised in Additional le 1) can be applied consistently, one of the reviewers will independently screen all of the titles and abstracts. Any differences in inclusions of abstracts between two reviewers will be resolved with discussion, or a third reviewer if required.
The full text of citations that are included or unable to be excluded at title/abstract screen stage will be assessed in detail against the inclusion criteria by at least two independent reviewers. Reviewers will work in pairs, with at least 2% of full text articles cross-screened by all reviewers and agreement determined.
Reasons for exclusion of full text studies that do not meet the inclusion criteria will be recorded and reported in the scoping review. Any disagreements that arise between the reviewers at each stage of the study selection process will be resolved through discussion, or with a third reviewer. The results of the search will be reported in full in the nal scoping review and presented in a Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) ow diagram [30]. See the populated PRISMA-P checklist in Additional le 3.

Data Extraction and Synthesis
Reviewers will use the JBI Appendix 11.1 scoping review data extraction table [28] available within the online platform JBI SUMARI [31] to facilitate data extraction. Prior to independent reviewer data extraction, all reviewers will trial data extraction from a randomly selected sub-set of articles for comparison and resolution of con icts or issues. The data extracted will include speci cs regarding the population, concept, context, study methods, and key ndings relevant to the review objective. This table will be modi ed and revised as necessary during the process of extracting data from the included studies.
Modi cations will be detailed in the full scoping review report. Any disagreements that arise between the reviewers will be resolved through discussion, or with a third reviewer. Authors of papers will be contacted to request missing or additional data, where required. Themes will be grouped and summarized quantitatively, and data will be synthesized narratively, structured around categories or themes of indicators or metrics that suggest both agreement and diversity across elds. We will also synthesize the method or strategy for developing the indicators or metrics.

Data Presentation
The extracted data will be presented in diagrammatic form in a manner that aligns with the objective of this scoping review. A narrative summary will accompany the tabulated and/or charted results and will describe how the results relate to the review's objective and questions.

Meta-Biases
We strive to produce and disseminate ndings that avoid the potential for bias by detailing our search strategy, inclusion criteria, and preliminary data extraction form. We acknowledge that publication bias is unavoidable in that we have designed this review to include only published, peer-reviewed review articles, but will work to reduce the opportunity for publication bias by screening reference lists to locate articles that may not have been found using our search strategy. We will use appropriate JBI Critical Appraisal Tools to evaluate the rigor of included studies, and offer a nal GRADE (Grading of Recommendations, Assessment, Development and Evaluations) to rate the quality of overall evidence [32].

Conclusion
A considerable amount of published literature and a handful of excellent reviews have contributed to the development of resilience research since the 1970s. It is important to draw upon the lessons learned and the strategies employed from resilience research published from across diverse elds, as we innovate to improve the safety of healthcare. This is a necessary next step as we work to rigorously design measurement and assessment strategies for systems resilience in healthcare, and move toward intentional design and predictable healthcare delivery within our healthcare systems.