Increasing Perioperative Risk Scoring in Emergency Laparotomy: A Surgical Trainee-Led Nationwide Quality Improvement Programme

Background: Emergency laparotomy is associated with high morbidity and mortality. The early identication of high-risk patients allows for the provision of timely perioperative care and appropriate resource allocation. The aim of this study was to develop a nationwide surgical trainee-led quality improvement (QI) programme to increase the use of perioperative risk scoring in emergency laparotomy. Methods: The programme was structured using the active implementation framework in 15 state-funded Irish hospitals to guide the staged implementation of perioperative risk scoring. The primary outcome was a documented preoperative risk score for patients undergoing an emergency laparotomy at each site. Results: The percentage of patients undergoing emergency laparotomy who received perioperative risk scoring increased from 0-11% during the exploratory phase to 35-100% during the full implementation phase. This change was consistently maintained throughout the full implementation phase of the programme. Conclusions: A trainee-led QI programme increased the use of perioperative risk assessment in patients undergoing emergency laparotomy, suggesting that such an approach may be feasible for improving patient outcomes and care delivery performance. risk


Introduction
Emergency abdominal surgery encompasses a variety of high-risk intra-abdominal surgical procedures that require the timely delivery of resource-intensive care. In recent years surgical and anaesthetic improvement programmes have focused on emergency laparotomy as it is associated with signi cant morbidity and a 30-day mortality rate ranging from 5 to 21% (1)(2)(3)(4)(5)(6)(7)(8)(9). There are considerable differences among patients undergoing emergency laparotomy, not only in the indication for surgery (9,10) and the presence of co-morbidities (11), but also in factors related to the delivery of care (12). Outcomes are worse for older, frail patients (13) as well as for patients from ethnic minorities, lower socioeconomic groups and those who are uninsured (14,15). Although it is di cult to control for patient factors, evidence points to wide variability in the processes of the delivery of care (4,12,16). Optimal care requires that the structure and delivery of perioperative care is tailored to the needs of the individual, so early identi cation of highrisk patients is a priority.
Assessment of a patient's risk of an adverse outcome may be informed by clinical judgement, risk assessment tools, biomarkers and evaluation of functional capacity (17). Whilst clinical judgement is important, it is not enough to predict adverse outcomes postoperatively (18). The utility of systematic preoperative assessment is emphasised by the ndings of the UK National Emergency Laparotomy Audit (NELA) which found a 30-day mortality of 5.4% among unassessed patients, compared to 16.9% and 1.6% respectively for those deemed high or low risk (18). Components of care delivery, such as consultant review before surgery, consultant presence in theatre and admission to the intensive care unit, may vary according to whether risk has been assessed and documented before surgery. Thus, pre-operative risk documentation before emergency laparotomy may be considered a surrogate marker of good quality care and where it is not done patients miss out on the standard-of-care.
A number of approaches to improving outcomes for patients undergoing emergency laparotomy have been described, including audit (18), peer comparison (9), best practice tariffs (18), 'name and shame' strategies (19) and local improvement activities (20). Facilitating frontline healthcare professionals to address key issues in their work area is a common low-cost strategy for delivering improvement (21,22), but has recognised de ciencies, especially when such work happens in an uncoordinated way (23). Within the Irish healthcare system, surgical trainees (residents) are the primary frontline staff for surgical patients but they are frequently overlooked as agents for change and remain largely isolated from hospital-based improvement programmes due to the temporary nature of their positions. Postgraduate surgical trainees have limited training in QI methodology (24) and improvement programmes led by trainees are barely visible in the literature (25). Trainee involvement in improvement activities may be an effective strategy for developing a "local" culture of process improvement.
The aim of this study was to develop the structures to support a nation-wide surgical trainee-led quality improvement programme and evaluate interventions in the implementation to increase the use of perioperative risk scoring in laparotomy.

Context
State-funded health care in the Republic of Ireland (population 4.8 million) is available to all residents. It is organised and delivered by the Health Service Executive (HSE). State-funded hospitals are divided into 7 hospital groups with geographically de ned catchment areas. Each group has at least one designated Cancer Centre and a number of other hospitals, classi ed into 4 levels (termed "models") with increasing complexity of care available at each level. Regarding acute surgical services, model 3 hospitals provide care to undifferentiated surgical patients; model 4 hospitals provide acute undifferentiated care and accept tertiary referrals (26). Twenty-four hospitals provide acute surgical services, each operating an on-call rota to provide emergency services on a 24-hour basis. Nationwide an average of 2,068 emergency laparotomies are performed annually (9).
General surgery training in Ireland is an 8-year structured postgraduate training pathway separated into a 2-year "core" surgical programme followed by competitive entry into a 6-year "specialty" training programme. General Surgery trainees (years 3-8) work across model 3 and 4 hospitals providing 24-hour emergency on-call cover in addition to a consultant general surgeon.

Conceptual Framework
This study was a prospective QI programme, called "Know The Score" involving 25 surgical trainees and 1 surgical physician associate working in 15 state-funded Irish hospitals. The programme was structured using the active implementation framework (AIF) (27) to guide the staged implementation of perioperative risk scoring for emergency laparotomy patients using plan-do-study-act cycles (PDSA). A distributed leadership model was utilised, combining local programme leads at individual sites supported by a central implementation team to provide competency, organisational and leadership support. The programme implementation team consisted of a senior surgical trainee (DMN) to provide near-peer clinical leadership, a post-specialty training fellow in QI in surgery (PEL) to deliver technical expertise and overall programme sponsorship was provided by a senior surgeon (DAMc) within the framework of the Irish National Clinical Programme in Surgery. Surgical trainees were voluntarily recruited in each hospital to lead the programme at local level and provided with structured coaching, mentorship and resources based on individual and local needs. The study was endorsed within the HSE National Quality Assurance & Improvement System (NQAIS) Clinical Governance Framework and registered with the Audit/ QI Committee in the institution of the senior author, a university teaching hospital and national referral center. Data on individual hospitals were de-identi ed and presented in an anonymised fashion to ensure con dentiality at all stages of the analysis. No funding was received for this project and the authors have no con icts of interest to disclose. This report complies with the SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence) guideline (28).

Interventions
Speci c interventions were designed and tested based on AIF implementation stage: exploration, installation, initial implementation and full implementation(27) ( Table 1). The initial exploration phase interventions included multiple presentations highlighting variation in outcomes after emergency laparotomy at national surgical meetings and an associated media campaign to raise awareness.
Interventions in the installation stage included written correspondence to all hospital clinical directors and departmental chairs outlining the improvement programme, installation of posters with a quick response (QR) code to access risk scoring apps in participating hospitals, a participant-wide secure messaging group to support communication, dissemination of a standardized presentation slide set to support local programme education and a formal invitation to all current Irish surgical trainees to participate in the change initiative. During the initial implementation phase interventions consisted of twice-monthly teleconferences for local leads, local educational presentations on the programme given by programme participants and the bi-weekly distribution of the programme run-chart to as a means of "real-time" feedback. Trainees were supported to develop improvement cycles depending on the needs identi ed in their hospital. Performance data was shared internally in the hospital but routinely de-identi ed for national input.

Evaluation of Interventions
Programme participants completed a questionnaire during the initial implementation phase to evaluate their QI knowledge before and after the programme and to evaluate the effectiveness of the interventions used. A twelve-item questionnaire was designed using web-based survey software (Qualtrics TM , USA) and distributed to participants. Three baseline questions were designed to assess current scope of practice, prior experience in QI and understanding of QI prior to and during programme participation. Five multiple choice and ranked order questions were used to assess which programme interventions participants found valuable in driving change and could be adapted to their local practice. Four questions using a mixture of Likert scale and free text were used to capture the experiences of participants and feedback received from colleagues at local sites. Questionnaires were analysed using descriptive statistics, weighting of ranked responses to calculate highest average ranking and qualitative analysis of key themes from free-text responses.

Programme communication
Communication between local programme leads and the steering group took the form of email at commencement of involvement with the study, for dissemination of programme materials or to notify collaborators of key programme initiatives such as a participant survey. Phone calls were used at the discretion of the steering group to contact local leads. Routine communication between local programme leads and the steering group was conducted using the programme "WhatsApp" messaging group. The volume, content and temporal pattern of messages communicated in the programme instant messaging group between local leads and the steering group were recorded for the duration of the study and analysed as a proxy measurement of the engagement of local leads with the programme.

Data collection and statistical analysis
The primary outcome was a documented preoperative risk score for every patient undergoing an emergency laparotomy at each hospital site. De-identi ed data regarding laparotomy volume and risk scoring performance were submitted to the steering group using the "WhatsApp" platform at each biweekly data collection time-point. The volume of emergency laparotomies performed for biweekly data collection time-points was recorded for each study site to indicate site operative volume. The median value for average number of laparotomies per data collection was used to stratify study sites into "low" and "high" volume hospitals. Bi-weekly arithmetic means for patients who had a risk score performed at each site were deidenti ed, submitted centrally, plotted on a run-chart and shared with local study leads. A baseline was constructed for the rst six data points during the exploration phase of the programme (from September 2018 to December 2018). The primary outcome was assessed for evidence of improvement using a Shewhart statistical process control chart (SPC). This technique was deemed most appropriate in the context of real-world large-scale change in which the control of independent variables is not always possible (29). The chart was then inspected for common cause variation (random uctuation) and special cause variation (changes due to external factors). Special cause variation or a substantial change not due to natural variation was identi ed, either when the mean bi-weekly performance breached the upper or lower control limits or when eight consecutive bi-weekly performances lay on one side of the mean line. Special cause variation was considered a clinically and statistically signi cant change.

Results
Performance of risk scoring A total of 26 surgical professionals in 15 hospitals were recruited into the programme. The grade of practice of participants is shown in Supplementary Table 1. Eight (53%) Model 4 hospitals were and seven (47%) Model 3 sites were included. The primary outcome measure was a documented preoperative risk score for patients undergoing an emergency laparotomy at each site. Data was plotted on a run-chart (Supplementary Figure 1). Over the 15-month period of data collection, two arbitrary targets were set -50% of reported cases scored by the end of month 5 (January 2019) and 70% of reported cases scored by the end of month 10 (June 2019). Within the exploration phase of the programme, the percentage of cases scored ranged from 0-11%. The interventions in the installation phase (months 1-3) led to increase in the percentage of cases scored ranging from 8-35%. The percentage of cases risk assessed in the initial implementation phase (months 4-6) ranged from 21-67%. Within the full implementation phase from 6 months onwards, the percentage of cases scored ranged from 35-100%. The interim targets of 50% of cases scored was achieved by the end of month 5, however this was not sustained initially. Full implementation of the programme resulted in meeting the second interim target of 70% of cases scored.
Full implementation was associated with a more consistent percentage of cases being scored throughout.
The primary outcome of percentage cases scored was also assessed for evidence of variability using an SPC chart (Figure 1). Within the setting of real-world change across multiple sites and contexts in which the control of external factors is impossible to control for, an SPC chart permitted variability to be assessed. Breach of the lower control limits indicating statistically signi cant change (special cause variation) was found at 3 points within the exploration and installation phases of the programme. Variation was reduced as the programme proceeded through the subsequent phases of implementation.
Programme implementation at local study sites Study sites participated for a varying length of time in the programme (mean=12.4 months, SD=4.66 months). A minimum of six months was required for inclusion in the following data analysis. The mean number of data collection time points to which individual study sites submitted data was 16.1 (SD 5.5, range 7-24). The number of laparotomies reported by each site varied from 4 to 70 with a mean of 33.4 (SD=21.6). Operative volume at individual sites ranged from a mean of 0.44 to 4.12 laparotomies per data collection time-point. The proportion of laparotomies risk scored varied from 7-100% for study sites. It was anticipated that the performance of risk scoring would increase in study sites with a longer duration of participation in the programme. However, there was no correlation between overall participation time of study sites and risk scoring performance (r= -0.16, p= 0.59).
The average proportion of laparotomies risk scored was higher in centres designated as "low" volume compared to "high volume" but this was not statistically signi cant (mean =0.63 v 0.42, p=0.19)( Figure  2A). Similarly, an association was suggested between the number of local leads at a study site and the proportion of laparotomies risk scored ( Figure 2B). Study sites with one local lead were most successful in performing risk scoring. The mean proportion of laparotomies scored in sites with one, two and three local leads was 0.76, 0.38 and 0.50 respectively (p=0.07). These ndings suggest that study sites with relatively lower operative volume and an individual QI lead may be more capable of effecting local change. Greater personal engagement and a sense of ownership of a QI project may be instrumental in effecting change while larger study sites may see effectiveness reduced by distribution of responsibility amongst local participants Programme communication Analysis of the volume, content and timing of messages within the programme WhatsApp group provides insight into the use of mobile instant messaging as a communication tool to effect change in QI. A large variation was observed in the number of messages sent within the group by individual local leads (mean=15.2, range =1-64). No signi cant difference was observed between trainee grades and messaging volume. A number of leads used the messaging service solely for submission of local data. Other local leads used the service to communicate their experience of the programme including success or failure of local initiatives or to suggest novel approaches to effect change.
The total number of messages sent by local leads in each study site was calculated to determine if messaging volume was correlated with success in risk scoring laparotomies. No correlation was observed between total communication volume from study sites and the total proportion of laparotomies scored (r=0.02, p=0.95). The temporal pattern in mobile internet messaging amongst the steering group and local leads illustrates the variation in participant performance and engagement during the course of the programme (Figure 3). A surge in messaging activity was observed amongst steering group members and local leads at the key times such as programme commencement and approaching July 2019 changeover. Impact of messaging activity is also apparent with a peak in risk scoring performance seen to be preceded by increased message activity.

Impact of rotation of local leads
The contribution of individual hospitals to data collection was compared between hospital sites for the period January 2019 -July 2019 and July 2019 to December 2019. It was possible to compare data contribution in ten hospital sites during these periods (Supplementary Figure 2A). Eight out of ten study sites contributed to less of the available data collection timepoints following July 2019 rotation. Contribution to data collection time points was increased in one site following rotation while one site was unaffected by changeover. The impact of rotation on the performance of risk scoring was more positive (Supplementary Figure 2B). The average proportion of laparotomies risk scored was increased in six of ten study sites following July 2019. Risk score performance decreased in three sites following rotation while performance was unaffected in one site.
The engagement of local leads was re ected in the volume of messages communicated in the programme "WhatsApp" group before and after July 2019 (Supplementary Figure 3). A large degree of variation was observed in messaging volume before and after the rotation with the majority of local leads sending higher numbers of messages prior to July 2019 perhaps suggesting programme fatigue amongst participants.
Overall, it appears that study sites contributed less regularly to data collection time-points after July 2019 rotation, but when data were submitted, they indicated increased performance of risk scoring. These ndings may have a number of implications. The rst is that enthusiasm for the programme may have waned following job rotations and consistent data collection and submission could not be maintained. Secondly, it is possible that local leads may have avoided submitting data which re ected low performance of risk scoring and preferentially submitted data at timepoints with better performance.

Experience of programme participants
The programme questionnaire was completed with an 88.5% response rate. 60.8% of participants had no prior experience or training in QI. 39.4% of participants considered themselves to have "good" understanding of QI prior to commencing the programme, increasing to 94.1% after four month's participation. Participants reported that trainee teleconferences, reminder messages regarding data collection and visible display of programme goals in the hospital were most useful for driving change during the programme (Figure 4). Participants also highly valued the contribution of the programme to their professional logbook. When asked to describe the most effective local intervention, communication emerged as a key theme. Participants highly valued conducting the programme as a network of collaborators with frequent reminders and encouragement from colleagues on site as well as programme coordinators. Common themes of feedback to the programme highlighted a concern amongst surgical teams that the implementation of risk scoring would not alter clinical management. A frequent suggestion for programme improvement was the incorporation of risk scoring into standard preoperative documentation.

Discussion
This study describes the structure and implementation of a nationwide surgical trainee-led QI programme to increase the use of perioperative risk scoring in emergency laparotomy using the active implementation framework. Over the duration of the programme, the percentage of patients undergoing emergency laparotomy who received perioperative risk scoring increased from 0-11% during the exploratory phase to 35-100% during the full implementation phase. This change was consistently maintained throughout the full implementation phase of the programme.
Two large scale, prospective Emergency Laparotomy QI programmes from the UK have recently been published, from the Emergency Laparotomy Collaborative (ELC) group (30) and the Enhanced Peri-Operative Care for High-Risk Patients (EPOCH) trial group (31). The ELC study involved the implementation of a 6-point care bundle in 28 National Health Service (NHS) hospitals and found a reduction in the unadjusted mortality rate and the length of stay as well as changes in some of the care bundle metrics. This programme used the Institute for Healthcare Improvement (IHI) Breakthrough Series collaborative as an implementation framework. Although, perioperative risk assessment was not part of the care bundle, it demonstrated that QI methodology can improve outcomes in emergency laparotomy at scale (30). The EPOCH trial intervention consisted of an extensive 37-point evidence-based care pathway in 15,873 patients in 93 NHS hospitals. Overall, the study investigators found that there were modest improvements in 10 patient-level process measures, however this did not translate to overall outcome measures including overall survival, 90-and 180-day mortality, length of hospital stay or frequency of readmission (31). There were wide variations in intervention delity between hospitals, with differences in the processes that individual teams tried to change, the rate of change and the eventual success. In a hospital-level evaluation of the EPOCH trial, no hospital reliably implemented all 10 processes. Out of 93 participating hospitals, 80 provided su cient data for analysis generating 800 process measure charts from 20,305 patient admissions over 27 months. Overall, only 279 of the 800 processes were improved (32). These ndings show that change at scale and the context in which change is being implemented is far more complex than initially anticipated. Context is central to the success (or failure) of QI programmes.
Successful implementation is not only in uenced by context but also by the content of the intervention and the process of implementation. A number of reports have been published on the effects of smallscale QI projects to improve outcomes in emergency laparotomy. The ELPQuicC group assessed the implementation of a bundle of 5 interventions in 4 hospitals and reported a reduction in risk-adjusted mortality (risk ratio 0.61, 95% con dence interval 0.45-0.84) (20). The difference in ndings compared to the EPOCH trial may relate to the simpler intervention, but also a stronger pre-existing relationships between staff leading implementation in these early adopter hospitals. The EPOCH trial set ambitious targets in hospitals in which there may have been a less favourable organisational readiness for change that in the ELPQuicC hospitals. Reports of QI programmes in other clinical areas have delivered mixed results (33)(34)(35), suggesting that more focused, discrete clinical interventions might be more successfully implemented than interventions that include a larger number of clinical processes.
A mixed-methods process evaluation of the planning, delivery and reception of the EPOCH intervention at hospital level found that delity of the intervention showed considerable variability across individual hospitals and only 11 of 37 care pathway processes that more than half of respondents reported attempting to improve (36). Analysis of the qualitative data for the trial suggests QI leads were often attempting to deliver the intervention in challenging contexts and identi ed barriers including the social aspects of change such as engaging colleagues, limited time, limited organizational resources and challenges in collecting data (36).
The model of trainee-led QI programmes are relatively sparse in the literature. There have been successful trainee-led programmes aimed at improving patient satisfaction (37), increasing compliance with venous thromboembolism prophylaxis (38), reducing opioid analgesia discharge prescriptions (39) and other programmes (40,41). This study demonstrates that within a structured programme, surgical trainees can deliver improvement at a national level. Postgraduate surgical trainees in resource-limited healthcare systems are adaptable and resourceful and with the correct support an ideal group of healthcare professionals to drive change at local level. In addition, trainees are front line staff that have the potential to in uence the intervention, engage staff and counteract resistance and suspicion about externally led change (42). In the context of limited organization resources and potentially challenging data collection, a simple, explicitly designed measure with data collection, monitoring and a feedback system incorporated into improvement activities from the outset. The primary outcome of a documented preoperative risk scoring was chosen as a balance between a credible measure by participants without being unduly burdensome, or di cult to interpret.
Adaptability and trialability is often essential in ensuring QI interventions can t within different contexts and is a driver of success in our programme. However, delity to key parts of an intervention is also important to maximise the likelihood of success (43). Although delity was not directly assessed in our programme, it is likely there was considerable variability in the local implementation of the intervention and this may impact long term sustainability of improvements achieved.
Central to the success of the programme are the social aspects of improvement which are as likely to be as important as more technical aspects, such as data analysis and feedback. Similarly, a central expert implementation team oversaw the development of local programme teams building local QI capacity and serving as an accountable structure to move through the stages of implementation successfully. Local programme leads capitalised on social QI strategies to drive change through utilization of a participant messaging platform as well as the programme leadership through use of monthly participant teleconferences. Building and maintaining effective social relationships is time-consuming and challenging, particularly senior colleagues and hospital administration (44). This may be even more challenging for trainees who usually have temporary training appointments. Preoperative risk assessment is not a magic bullet. Developing consensus around the value of a formalised process of preoperative assessment required some to engage senior surgeons of the value of the selected intervention (45).
Given that this was an unfunded project, the resources and time available to coach local QI leads was limited in comparison to other reported QI interventions such as the IHI Breakthrough Series Collaborative model (46). Our programme was designed as a parsimonious intervention, with limited face-to-face meetings, so that it could be adapted and replicated widely if found to be successful. A higher intensity coaching programme might have led to greater intervention delity, however, it is uncertain if this is always the case (47,48).
There are some inherent limitations to our study which need to be acknowledged. The primary outcome was a documented preoperative risk score for patients undergoing an emergency laparotomy at each site, however, we did not evaluate changes in patient outcomes associated with the intervention. Variations in the denominator for the bi-weekly measures sometimes interfered with signals in the data. For example, in a 2-week period with a small denominator, fewer patients scored may create a data point that breaks a signal that would otherwise indicate a move towards improvement. In combination with the time-bound nature of the analyses, this may have led to some real-world improvements not being identi ed using the run charts.

Conclusions
This programme describes the structure and implementation of a nationwide surgical trainee-led QI programme to increase the use of perioperative risk scoring in emergency laparotomy using the active implementation framework which demonstrated a sustained improvement in risk assessment during the full implementation phase. Such an approach may be feasible for improving patient outcomes and care delivery performance.

Declarations
Ethics approval and consent to participate The study was endorsed within the HSE National Quality Assurance & Improvement System (NQAIS) Clinical Governance Framework and registered with the Audit and QI Committee in Beaumont Hospital (CA860), the hospital of the Senior Author and one of the Royal College of Surgeons of Ireland Hospital Group Hospitals. No patient information was collected. Data on individual hospitals were de-identi ed and presented in an anonymised fashion to ensure con dentiality at all stages of the analysis. No funding was received for this project.

Consent for publication
This manuscript does not contain any individual person's data which would require consent for publication. All Authors consent for publication of the manuscript as submitted

Availability of data and materials
Not applicable: Data sharing is not applicable to this article as no datasets were generated or analysed during the current study.

Competing interests
The authors declare that they have no competing interests.

Funding
No funding was received for this project Authors' contributions SURGical Improvement Network (SURGIN) is a group of surgeons and surgical Trainees in the Republic of Ireland who have an interest in Quality Improvement in Surgery. The group consists of a core group consisting of a steering committee and advisory group and local collaborators. The senior author of this paper is the overall lead and can be emailed at deborahmcnamara@rcsi.ie DN was part of the Steering Committee/ Writing Group/ Data Analysis / Advisory Groups for the project.
She was also a local lead.
PL was part of the Steering Committee/ Writing Group/ Data Analysis / Advisory Groups for the project.
EOC was part of the Steering Committee/ Writing Group/ Data Analysis / Advisory Groups for the project.
She was also a local lead DMcN was part of the Steering Committee/ Writing Group/ Data Analysis / Advisory Groups for the project.
All authors read and approved the nal manuscript DN and PL contributed equally and hence should have joint rst authorship.
This project was a collaboration and we wish to recognise all contributions such that they are each cited on pubmed in accordance with established, published protocols which are linked to ICMJE guidelines.
The collaborators listed below performed as essential local leads and local collaborators.  Importance of programme interventions ranked by participants