Relation Between Quality Improvement Collaboratives and Non-technical Skills Development: a Systematic Review of Complex Interventions


 Background: Improving the quality of care contributes to the Sustainable Development Goals. Non-technical skills (NTS) such as leadership, teamwork, communication, and use of data for decision-making are the strong points of the learning process of collaboratives for quality improvement in health services.Objective: To evaluate the relationship between the collaboratives and the development of NTS by participating health professionals.Methods: We searched PUBMED, WEB OF SCIENCE, SCOPUS and GOOGLE SCHOLAR (the first 30 pages) and performed a manual search on the IHI (Institute for Healthcare Improvement) website, from Jan 2010 to Dec 2019, for articles published in English, Spanish, and Portuguese. IHI Breakthrough Series Collaborative model project studies with pre- and post-data, and SQUIRE compliant titles were included. The qualitative analysis evaluated the study design, year, country, context, target for improvement, health services, duration of intervention, number of participating professionals, evidence of improvement, effectiveness of the intervention, and NTSResults: From 701 identified abstracts, 34 studies were included for data extraction and qualitative analysis. Improvement of the intended results was reported by 76.5% (26/34), of which only 38.5% (10/26) showed sustainability for six or more months. Improvement in NTS was assessed in five studies, but none assessed their contribution to the dissemination and sustainability of improvement.Conclusions: Collaborative initiatives train professionals in improvement science to incorporate evidence-based practices. It is necessary to include, in the measurement plan of these improvement projects, the assessment of the NTS acquired by the professionals and its relationship with the results achieved, the dissemination level, and the success in terms of sustainability of those gains and best practices.Systematic review registration number: Prospero - CRD42020164190

Thus, collaborative strategies are used to reduce the gap between what is known and what is done in practice, through the implementation of evidence-based practices, in order to improve patient care. Therefore, if an improvement process takes place around people, the evaluation of its impact must take into account the effect on people that allows them to understand the personal and intrinsic motivations for change. [11] This systematic review aims to assess the relationship between the collaboratives and the development of non-technical skills (NTS) among participants, as a determining factor for the sustainability of the change implemented in health services in different contexts.

Methods
The systematic review protocol was designed using the PICOS strategy (participants, intervention, comparison, outcomes, and study design) to answer the research question 'What is the impact of collaborative projects to improve quality in the development of NTS, such as the leadership, communication, and teamwork of health professionals?'. The research protocol was registered with PROSPERO -CRD42020164190. Available from: https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42020164190.

We used the Preferred Reporting Items for Systematic Reviews and Meta-analysis of Complex
Interventions (PRISMA-CI) extension statement, [12] taking into account the fact that the studies under analysis are con gured as complex interventions, according to the de nition: 'All complex interventions have two common characteristics: they have multiple components (intervention complexity) and complicated/multiple causal pathways, feedback loops, synergies, and/or mediators and moderators of effect (pathway complexity). In addition, they may also have one or more of the following three additional characteristics: target multiple participants, groups, or organizational levels (population complexity); require multifaceted adoption, uptake, or integration strategies (implementation complexity); or work in a dynamic multidimensional environment (contextual complexity)'. [13] Eligibility criteria In this review, the de nition of the article as a collaborative project -which is considered as a criterion for inclusion -stipulates the participation of more than two health institutions and the use of the IHI Breakthrough Series Collaborative model in which at least four of the six key elements were used in the intervention, namely: targets for improvement; multidisciplinary teams; learning sessions involving participants; training in improvement methods; follow-up support from other organisations; collaborative faculty.
In addition, the title should be explicit about whether it is an initiative for quality improvement under SQUIRE 2.0. [14] Pre-and post-controlled and uncontrolled studies were included, whose results demonstrate the effect of the intervention on the proposed objective in relation to the baseline; these were developed in a minimum period of 6 months of the intervention. Review article, articles with an unavailable abstract, comments, and conference abstracts were excluded.

Information sources and search
We searched the PUBMED, WEB OF SCIENCE and SCOPUS databases from January 2010 to December 2019 for articles published in English, Spanish and Portuguese, using the terms recommended for each database: ('Quality of Health Care' [Mesh] OR 'Quality Improvement' [Mesh] OR 'Implementation Science' [ Mesh]) AND ('collaborative improvement initiative' OR 'breakthrough series collaborative' OR 'model for improvement' OR 'plan-do-study-act method') AND ('communication' OR 'leadership' OR 'team work'). Subsequently we searched Google Scholar for the same period of time, language, and terms, making a selection of the rst 30 pages. Additional manual research was done on the IHI (Institute for Healthcare Improvement) website. We did not conduct a direct search with researchers.

Study selection
In the rst and second phase, two authors (MRF and SGP) read the titles, and duplicate articles and those without abstracts were excluded. The abstracts whose titles complied with SQUIRE 2.0 and reported a collaborative project to improve quality in the health system, in any area or theme, were selected for full reading of the article.

Data collection process and items
Each study was read by two independent authors (SGP and MRF or BFJ) and those which met the inclusion criteria had the following items extracted and entered on an excel spreadsheet: study design, year, country, context (public, university, community, etc.), target for improvement, number of health services involved, duration of the intervention in months, number of key elements developed in the collaborative, number of participating professionals, main results (evidence of improvement).
Effectiveness of the intervention was con rmed if it reached the primary objectives. Evidence related to sustainability of data were presented for at least six months after the intervention, mentioning the improvement of NTS, whether the study reported leadership, effective communication, teamwork, a decision making decision, as professional skills that contributed to the good result and evaluation of improvement of NTS, either qualitatively or quantitatively. Disagreements were resolved by a fourth author (PS).

Risk of bias in individual and studies
As the interest of this review was to investigate the implication of NTS in the positive result of collaborative projects, and given the complexity of the interventions to improve quality, we included all studies that presented the results before and after the intervention. In addition, as the IHI collaborative methodology is characterised as an intervention with time series analysis, we used the analysis of control charts to assess the effectiveness of the intervention. All studies were evaluated so as to comply with the following items on the SQUIRE 2.0 checklist: title, abstract, method, results, discussion, and conclusion. The studies were classi ed as high quality (5 to 6 SQUIRE items were lled), medium (3 to 4 items), and low (1 to 2 items). We chose to use SQUIRE 2.0 to assess the quality of the studies, as they are studies to improve quality.

Synthesis of results
A univariate descriptive analysis was performed for the set of qualitative and quantitative data. Formal meta-analytical techniques to group the data were not used, due to the heterogeneity of the objects of study, contexts, units of measurement, nature, and complexity of the interventions. A collaborative was considered effective if the results after the intervention were statistically signi cant in relation to the baseline and the objective of the study.

Study selection
We identi ed 701 articles published from January 2010 to December 2019 in the databases searched ( Fig 1) and 11 studies were added from the IHI website. A total of 319 duplicate articles were excluded, leaving 382. Of these, 245 titles were classi ed for analysis of the abstract. After reading the abstracts, 156 articles were excluded because they did not present at least one criterion of eligibility, leaving 89 articles for complete reading of the publication. Among the reasons for the exclusion of 55 articles were: (11) did not present pre-and post-data; (11) were not collaborative to improve quality; (10) were only a method/process description; (7) were studies on lessons learnt; (6) were descriptions of a project/network; (6) were non-accessible articles; (2) were not about health services; (1) was a collaborative evaluation; (1) was a study protocol.

Study characteristics
There were 34 studies  included for data extraction and qualitative analysis, of which only one was a randomized controlled trial, one was quasi-experimental with control, one was an observational study, and one was a multiple case study. All other studies were of the quasi-experimental type without a control group with time series analysis, in accordance with the BTS model of the IHI. Most studies were carried out in the USA (22), followed by countries in Africa (5), the United Kingdom (3), The Netherlands (2), Canada (1), and Indonesia (1), all in the English language. Two studies involved 2 or more countries. [43,46] The great diversity of problems addressed in health structures with different levels of complexity and participation of professionals, sometimes from management, sometimes from patient care, characterised the set of complex interventions in this review. The characteristics of the studies are detailed in Table A see Additional le 1 for the original data used to perform this analysis.
The number of institutions included in the collaboratives ranged from 1 to 744, with an average duration of the intervention of 20 months (ranging from 6 to 60 months). The number of health professionals participating in the collaboratives was mentioned in 16 studies and amounted to between 35 and 2000 professionals.
International patient safety goals were addressed in 10 studies, ve focused on improved communication between teams and/or with the patient/family, [19,20,26,37,39] three addressed falls reduction, [35,36,47] and two considered health care-related infection prevention. [30,38] Improvement of patient safety culture was the subject of three studies. [23,25,36] Improving care in different clinical conditions was the subject of 14 reports such as: neonatal death, [42] care with tracheostomies, [46] diagnosis, [41] child health, [18,27,45] HIV transmission or treatment, [16,22,43] mental health, [24] women's health, [31,34,44] and stroke. [48] Prevention of adverse events was the subject of four studies. [28,32,33,36] Collaboratives were effective in improving the intended outcomes by 76.5% (26/34), [16,17,20-27, 29-34,36-38,40-42,44-47] reached partial results in 14.7 % (5 /34), [19,28,35,43,48] and did not obtain the desired improvement in 8.8% (3/34). [15,18,39] Improvement sustainability was reported in 10 of the 26 studies with positive results. [22][23][24]29,30,32,34,36,38,42] NTS such as leadership, teamwork, and communication are brie y cited in the discussions of 58.8% (20/34) of the studies as contributing factors for improvement. However, only ve studies evaluated improvement in NTS among the participating professionals/teams and none mentioned or assessed their contribution to the dissemination and sustainability of improvement (Table 1-See Additional le 1). [19,28,37,42,46] In the study by Stevens et al. (2010), the participants evaluated their improvement with the following parameters: delivery system design, decision support, clinical information systems, patient selfmanagement, integration, healthcare system organisation, and community linkages. There was an improvement in the processes with a slight improvement in the clinical results. [19] In the study by Zukoff et al. (2014) evaluation questionnaires were applied to the teams, not to the professionals individually, pre-and post-intervention, to measure the effort of the improvement intervention and the functioning of the teams, not identifying differences in their perception in terms of work improvement in a team. The study by Bedwell et al. (2018) highlighted that nurses' knowledge focused on improving tracheostomy care increase in the following points: general knowledge, identifying emergencies, performance, situation, goals, coping. [46] Risk of bias within and across studies The selection of studies restricted to those with pre-and post-intervention data, may have generated a selection bias, insofar as it may have excluded qualitative or secondary studies that assessed the participation of people in collaboratives, which would likely bring elements related to NTS, the object of this study. Another risk of bias within the studies is the selection of participants, which, although voluntary, is directed towards team leaders, who are presumed to be professionals more open to change and who have a greater potential for engagement.
A common feature of most studies is the use of statistical control charts to measure the improvement achieved in the intervention. In accordance with the BTS model, the studies are not designed to evaluate the effectiveness of a new process, but to apply in practice what one wants to adapt, based on evidence already available in the literature. This was identi ed in most studies in this review.
The evaluation of compliance with Squire 2.0 items as a quality requirement for the study, showed that only seven (20.6%) of 34 studies met the six items under analysis. [21,22,25,29,34,39,42] Three studies did not present contextual elements in the methodology [18,20,35] and three others did not provide any information about the participating professionals. [19,26,38] A lack of recording in the results about unintended consequences, such as unexpected bene ts, problems, failures, or costs associated with the intervention or details about lost data, was observed in 38% of the studies. Thus, 44.1% of the articles were classi ed as high quality with 5 to 6 Squire items; [15,[21][22][23]25,26,28,29,34,38,39,41,42,45,48] 44.1% were classi ed as average quality with 3 to 4 Squire items [16][17][18][19][20]24,27,[30][31][32][33]37,40,43,44] and 11.8% were classi ed as low quality with 1 to 2 Squire items. [35,36,46,47] Discussion As far as we know, this systematic review is the rst to use SQUIRE 2.0 to assess the quality of the study, and also the rst to bring health professionals to the centre of the discussion when assessing the effectiveness of collaborative efforts to improve quality, valuing the learning process as a central element for continuous improvement. The leadership role, teamwork, and improved communication between teams was signalled by 58.8% of the studies in this review as contributor for the result of the intervention, despite only 25% of these registering changes in these NTS among participants directly related to the intervention to improve quality. No study mentioned or measured the contribution of the collaborative to learning in the practice of NTS by professionals for the sustainability of changes. It seems to us that this knowledge gap persists in understanding how collaborative health projects work. [8] Collaboratives are designed with the purpose of creating a structure in which the organisations involved can share experiences and learn from each other, in a methodology based on the concept that 'all teach, all learn!' The learning cycles of the IHI BTS model are educational strategies to encourage professionals to test and adopt evidence-based practices, in order to reduce the gap between accumulated knowledge and clinical practice. [5] Therefore, the interaction between people, procedures, and equipment is vital for the results to be achieved. For an improvement to be successful, it is necessary to work on interpersonal relationships and human conduct related to transformation, dependent on NTS. [49] The evaluation of the effect of these educational strategies on changes in attitude and professional practice has been made mainly in the context of the organisational setting. Few studies have considered the effect on the health professional/staff, with regard to collaborative learning for teamwork, the feeling of belonging to a team, the use of data for decision making, communication skills to improve processes, strengthening the leadership pro le vis-à-vis your team and management, which we understand as NTS learned in practice. [50,51] Nembhard (2009) revealed that professionals recognise that the motivation, social support, and learning about the management of projects in an organisational context, complement the intrinsic activities of the model (learning sessions, collaborative faculty, development of change packages, community of practices for sharing ideas, and PDSA cycles). Moreover, those recognitions are of great use for broadening general knowledge and implementing change ideas. [52] The analysis made by the author on the relationship between the use of inter and intra-organizational learning strategies in the scope of collaborative and performance to improve quality, showed that organisations that work with a quality culture involving all staff associated with inter organizational,-learning results in better performance. [53] We share the idea that in order to understand how and why quality improvement interventions work, it requires looking at people's development from a personal and professional point of view. It is necessary to encourage participants to self-re ect on what they have learned, what problems they have encountered, what changes have been achieved, how to give sustainability to changes, what impact training in improvement science has had on their clinical practice, and what it means for patient care. [54] We argue that the IHI collaborative learning model is a powerful lifelong learning strategy for health professionals, with a potential to reframe work in health, as it fosters teamwork, the use of data to guide change, effective communication between peers and users, patient and family, and the positioning of the patient in the center of care, which is a prerequisite of quality. This learning model is very different from the classic training/courses and meetings of the concept of permanent education advocated by Ceccim (2005), where meaningful learning mobilises the individual to abandon the subject he is, assuming new patterns of behaviour to change work processes. [55] Professionals are more likely to learn and incorporate ideas from peers who do similar things, than from teachers in the classroom. Thus, we understand that collaborative learning is in line with transformative learning, which guides the third generation of medical education and health professions, which will promote the development of leaders and agents of change capable of engaging in the transformation of the health system. [3,56] Therefore, quality improvement projects based on learning processes play a central role in training professionals in these new skills.

Study Limitations
The exclusion of studies that did not present the pre-and post-data of the intervention left out qualitative studies designed to assess the participation of professionals, and this may distort the view discussed here that collaborative projects for quality improvement do not value the learning of NTS by health professionals. Another important limitation is the study design of most of the studies included in the research. The quasi-experimental design without a control group with analysis through statistical control graphs used in the BTS model of the IHI, is an easy application in health services to assess the intended change in the intervention target, but on the other hand it is a limitation common to quality improvement projects that hinders effective analysis. [57] More than half of the studies were classi ed as medium and low quality according to Squire 2.0. The duration of the intervention in the collaborative is also an important bias because it can show positive results that are not sustainable. The complexity of collaborative projects for quality improvement, with different service pro les, subject to improvement, multifaceted interventions, professionals involved, and the organisational context, makes it di cult to compare the different studies, as reported in previous studies. And nally, the global movement to improve health care imposes a positive bias for collaborative projects. [48] According to the analysis carried out, we can see that the relationship between the success of collaboratives and the development of NTS, although mentioned in some articles, is still poorly studied and, therefore the analysis does not allow a robust and broad assessment of its true effect. The presence of leadership was highly valued for the involvement of change teams in the work process of health professionals to achieve the desired goal. However, in the studies it is not perceived that collaborative learning forms leaders, capable of expanding the participation of other individuals directly involved with the learning sessions.

Conclusions
Collaborative projects to improve health quality are complex interventions widely used, producing a movement to disseminate and train professionals in improvement science and to incorporate evidencebased practices. The common axis of learning of these collaboratives involves NTS such as leadership, effective communication, teamwork, and use of data for decision making. These are necessary skills for change and are characterised as a model of meaningful learning in practice, where some learn from each other.
Of the ve studies that evaluated the improvement in the performance of professionals in NTS, each observed a parameter, with non-comparable methods, which makes these interventions di cult to analyse. It is necessary to include in the metrics measurement plan of these improvement projects the measurement metrics related to of NTS acquired by professionals, and its relationship with the improvement of the quality achieved with the dissemination to other services, and the sustainability of comprehensive care centred on people. Authors' contributions MRF and SGP independently did the title study selection for full reading of the articles. The data collection process and items were done by two independent authors (SGP and MRF or BFJ).

List Of Abbreviations
Disagreements were resolved by a fourth author (PS)