Quality Improvement Implementation and Maintenance of a Paediatric Early Warning System

Background: Paediatric Early Warning (PEW) systems have led to earlier identication and escalation of treatment with subsequent admission to Paediatric Intensive Care (PIC) in deteriorating children. The impact on reductions in cardiac arrest and mortality vary between the heterogeneous studies, showing both unchanged and reduced cardiac arrest, morbidity and mortality. Identifying and managing critical illness on wards is a complex and dynamic process involving technology, human interaction, cultural context and environment. We introduced a PEW system to reduce potentially avoidable cardiac arrest and death. Methods: We report an Implementation Science Quality Improvement (QI) natural experiment using the Medical Research Councils (MRC) Guidelines for Developing and Evaluating Complex Interventions, Action Research, Action Research Theory and methods. The aim of this program was to identify learning, renement and improvement opportunities to reduce poor outcomes. The interventions were 1) developing an observation and monitoring policy to standardise practice and provide a template for optimal care, 2) standardized charting with an embedded PEW score, 3) clinical skills training, 4) clinical process audit and feedback and 5) outcome surveillance. The process measures were 1) timeliness and impact for unplanned Paediatric Intensive Care (PIC) admissions, 2) clinical assessment skills and 3) chart completion compliance. The outcome measures included 4) total and predictable cardiac arrests and 5) hospital mortality. Data collection started in 2004, the PEW system was implemented in 2008, and the outcomes were reported through 2018. Results: In our specialist children’s hospital, we completed six improvement cycles over 10 years. 1) Timely PIC admissions improved after implementation (39% to 92%). Patients with unplanned PIC admissions had signicantly lower severity of illness and mortality but a longer length of stay. 2) Routine clinical observation accuracy improved (66 to 82%) following multimodal training. 3) Chart completion compliance improved (87 to 99%). In 2018, 2% of observations had missing or inaccurate parameters with a consequent inaccurate total PEW score. 4) The total cardiac arrest rate was signicantly reduced (0.36 to 0.16 per 1000 admissions). The small numbers of predictable cardiac arrests showed a decreasing trend. 5) Hospital mortality was signicantly reduced (3.46 to 2.24 per 1000 admissions). Outcomes improved

Single-centre clinical trials of RRS implementation and meta-analyses have shown a reduction in inhospital cardiac arrests and a reduced length of stay. [2,3,9,10,[18][19][20][21][22] However, Randomised Controlled Trials (RCTs) and large interrupted time series have not demonstrated improved survival. [11,[23][24][25] RCTs produce high-level evidence; however, the iterative processes between environment, culture, policy and human action cannot be explored adequately within the constraints of this methodology and relatively short timescales for studies. [13,17] Data collected from our specialist children's hospital during a multicenter trial showed that our patients had infrequent and inconsistent observations [1]. The discovery of abnormal observations being identi ed and not communicated effectively to appropriately trained clinicians reinforced the need for a broader, systematic approach beyond the score or team. [3,4,5] An Implementation Science Quality Improvement programme (QI) was initiated in August 2004 to develop, re ne and implement a Paediatric Early Warning (PEW) system in our institution with the aim of improving the detection of deterioration and reducing potentially avoidable cardiac arrest and mortality. This process was developed to ensure that all bedside clinicians could accurately perform routine clinical assessments, document the observations appropriately, identify evolving clinical deterioration, anticipate rst-line treatment and effectively communicate using standardised language to ensure appropriate recognition and response to children who are deteriorating in the hospital. This report details the guiding theory, implementation strategy, interventions and evaluation from 2004 to 2018.

Page 4/29
Setting and context: The setting is a specialist children's hospital that has 220 beds, including 15 higher dependency beds in specialist wards and 31 PIC beds, and provides secondary, tertiary and quaternary care to the UK population. Admission rates have increased from 33000/year in 2007 to 45000/year in 2018. Patients may be admitted directly from outpatients to the wards via the emergency department or as referrals to specialist teams via the paediatric and neonatal transport service. There were multiple ways of monitoring patients according to patients' speciality and ward location and no minimum standards of care for routine and specialist observations. Morbidity and mortality reviews varied between specialities, and there was no institution-wide visibility of the process or outcomes. There was an incident reporting system for actual or near errors or adverse outcomes, but there was no concept of predictable or potentially avoidable events or harm. The safety culture was developing, but there was not yet open and blame or guilt-free reporting.
Theory, models and framework: We knew that we needed guidance to implement a complex intervention in a complex system. A complex intervention is "built up from a number of components that may act both independently and interdependently" [26]. "A complex system is one that is adaptive to changes in its local environment, is composed of other complex systems …. and behaves in a non-linear fashion (change in outcome is not proportional to change in input)". [27] We chose to use the MRC Guidelines for Complex Interventions to develop, plan and test the PEW system. [26,28,29] We chose the natural experimental design because it supports a study where a large population is affected by a substantial change. In a well-understood environment, deliberate manipulation of exposure is not possible, and outcomes can be captured through routine data sources. The MRC framework was missing guidance on the interaction of implementation within a complex and dynamic environment. To bridge this gap, we chose to use Action Research Theory, which ts well with our NHS paediatric services; it is collaborative, aligned with social justice values and connects knowledge building and data collection with effective action. [30] The Action Research process is a systematic, re ective study of one's actions and the effects of these actions in a workplace, organisational, or community context. It requires a deep inquiry into one's professional practice.
Understanding the change means paying attention to the impact within complex social systems. [31] The process involved affected stakeholders becoming co-investigators into the reasons for change, participating in describing the current reality and the future they needed to move towards. Action Research is a four-step continuous process including diagnosis, planning, action and evaluation, similar to the standard QI plan, do, study, act format.
Working party: A working party of 14 voluntary nursing and medical participants was supplemented with purposeful selection of content and culture experts. They committed to regular meetings and email contributions to develop and nalise the intervention and implementation. This multi-professional group had democratic decision-making powers to achieve collective actions. The changes were overseen and approved by the organisations safety team. The PEW system implementation was integrated into the institutions Quality and Safety Framework Strategy and education program. Interventions The wider PEW system included four main interventions: 1) developing observation and monitoring guidance to standardise practice and provide a template for optimal care, [16] 2) standardised charting with an embedded PEW score, 3) clinical skills training, 4) clinical process audit and feedback and 5) outcome surveillance. [17] The PEW score ranges from 0 -26. As physiology deviates from normal, a score of one to four is allocated according to age-appropriate thresholds. There were three score-based escalation thresholds: nurse in charge (1 to 4/26), patient's own team or Patient at Risk Team (PART) (5 to 8/26) and PIC doctor (³9/26).
Timeline of implementation and maintenance: 2004 to 2006: Data were collected on the frequency and content of observations, frequency of lifethreatening events and interviews with staff on identi cation and response to deterioration.
2007: Components of the system were developed 2008 to 2009: Improvement cycles one to three involved implementation and embedding the system (detailed in tables). There was dynamic interaction between the six key components: system-or institution-wide effects, observation and monitoring, education, audit and feedback, data collection and publicity. 2010 to 2012: Improvement cycle four: reinforcing standards of care and learning from missed opportunities.
2015: Improvement cycle ve: revision of guidance and chart.
2018: Improvement cycle six: revision of guidance and chart included assessment of chart completion compliance, ergonomic design integrated into the observation chart and parental activation developed.
Data collection and analysis: Process outcomes: To provide information on the use of the system, three process measures were used: 1) timeliness and impact for unplanned PIC admissions, 2) clinical skills assessment and 3) chart completion compliance.
1) An unplanned PIC admission is de ned as any admission to the PIC from an inpatient ward area that was not pre-booked or planned. Timely admissions were de ned by referral to PIC at a time triggered by clinical deterioration (with or without an elevated PEW score), with the expectation of PIC assessment within 1 hour and PIC admission within 2 hours. The hypothesis is that earlier intervention would reduce the severity of illness at PIC admission and, potentially, mortality. Untimely admissions included unexplained and unjusti ed delays in the process from identi cation to PIC admission. The PM and a senior nurse educator prospectively reviewed the supporting documentation (observation chart and clinical notes) of all children who had an unplanned PIC admission for up to 72 hours prior to their admission. The process included a formative face-to-face learning conversation and provided documented feedback to the team. All cases were then reviewed by a multidisciplinary team and assessed to be timely or untimely. The results are presented as proportions of timely escalation to PIC admission with the Chi-squared test. The impact on unplanned PIC admissions was assessed using the Paediatric Index of Mortality version 2 revised Probability of Death (PIM2rPOD) (Kruskal-Wallis), PIC length of stay (LOS) (Kruskal-Wallis), crude PIC mortality (Chi-squared) and PIC standardised mortality rate performed using the PIC audit database from 2004 to 2018. For the impact on unplanned PIC admissions, the Kruskal-Wallis test was used to report median and interquartile ranges (IQR) at a signi cance level of p<0.05.
2) Clinical skills assessment for routine observations and escalation were provided by a nurse educator to multiple small group simulation sessions in ward areas each week. Three to ve staff attended hourlong simulations using Simbaby® (Laerdal Medical, Orpington, UK) and their own clinical equipment in their clinical environment. The initial aim of the simulation sessions was to test the interrater reliability of the PEW score. However, we rapidly recognised that routine observations were inconsistent and incompletely performed, so inter-rater reliability could not be measured until baseline skills had improved. The aim was then to gather baseline data on routine clinical assessment skills, documentation on the standardised charts, anticipation of rst-line treatment and effective communication concerns to appropriately trained clinicians. Clinical skills were assessed with a stable simulated patient and again with a deteriorating clinical status. The scenarios were adjusted to the skill set of the participating group from a minor respiratory deterioration requiring some oxygen through to cardiorespiratory arrest. A formative assessment was completed during the sessions, and knowledge gaps were addressed at the time. Observation and monitoring guidance were used for assessment, and data were collected on a standardised form. Participants were asked to assess their competence before and after the training and whether they had completed the online learning and read the observation and monitoring guidance. The results (anonymised to individual) were reported to the nurse manager. The data are presented as the average combined group proportion of clinical skills performed correctly.
3) Chart completion compliance audits were conducted using the standards in the observation and monitoring guidance for 7 months post-implementation. Data collection points included correct selection of the four new age-speci c charts; completeness of observations (including the new routine observations, capillary re ll time, work of breathing and alertness); calculation of the PEW score and the frequency of observations. Depending on the clinical situation, a neurological observation included alert, response to voice, response to pain, unresponsive (AVPU) or the Glasgow Coma Scale (GCS). The PM performed intermittent audits that entailed reviewing a random selection of charts on wards for the preceding 24 hours to assess a composite of administrative and clinical parameters to determine chart utilisation. Adequate PEW system utilisation was set at a predetermined aggregate threshold of >80% completion of all chart parameters. For individual observation parameters, the target was >90%. Outcome data: 4) Cardiac arrests, de ned as any patient requiring cardiopulmonary massage followed by PIC admission or death on the wards, were collected from August 2004 onwards. Cases were reported by the resuscitation o cers involved and then reviewed for predictability by an expert multidisciplinary group using the patient's observations, clinical notes and discussion with the staff involved in the patient's care at that time. Predictability was determined by the presence of signs or symptoms more than 15 minutes before the event that would indicate that the event was likely to occur. [32] Preventability was a judgement on the likelihood that the event may have been prevented had the signs and symptoms been recognised and appropriate management instituted, or an inappropriate action not taken place (Table 1).
Events could have had factors that were predictable but even with appropriate and timely management not have been preventable (for example, a child who is known to have intermittent arrhythmias -it may have been predictable that they were likely to have an arrhythmia but the episode was not preventable). Likewise, an event could be non-predictable and still preventable if appropriate action had been taken.
When examining Predictability consider if any of the following were present >15 minutes prior to the event: When examining Preventability consider: 11. Was there any delay in accessing support services (operating theatre, PIC, physiotherapy or laboratory services)? Table 1. Guidance developed to determine potentially predictable and preventable events. 5) Hospital mortality rates prior to hospital discharge were accessed from hospital administration records 2004-2018. For cardiac arrests and mortality, the primary analysis was conducted using segmented regression. A Poisson regression model was tted to the data. Fixed effects were included for time and time since intervention (interaction between time and each intervention). This model assumes a linear trend in the outcome prior to the intervention, a new linear trend after the rst intervention, and a new linear trend after the second intervention. It was not believed that the intervention effects would be instantaneous and instead would impact the overall time trends. As a sensitivity analysis, we included xed effects for the implementation of the intervention to assess whether an instant effect of the intervention was observed. This model assumes a linear trend in the outcome prior to the intervention, a shift in the outcome at the time of the rst intervention, a new linear trend in the outcome post-rst intervention, a shift in the outcome at the time of the second intervention, and a new linear trend in the outcome post-second intervention.

Implementation strategy
Our strategy was developed to coproduce the components of a PEW system with interactive multiprofessional groups. Reporting standards The reporting guidelines used are TIDier for interventions, STARI for the implementation strategy and SQUIRE for the Quality Improvement method.

Ethics
The local Research Ethics Committee approved the study as service development and evaluation and did not require national ethical approval.

Results
The system and cultural changes included the gradual introduction of specialist nurses to offer support to bedside nurses and doctors on the medical then surgical wards, and more regular medical critical care support has been added outside of PIC.  Table 3. Timely and untimely PIC admissions before and after PEW system implementation.
A comparison of all unplanned PIC admissions was made using the PIC database over the three time periods (  Staff were not able to accurately assess the level of their own skills. Before the session, 10% of the participants assessed themselves as poor to average (potentially needing to gain observation and monitoring skills), and 90% evaluated their own abilities as good to excellent. Simulation identi ed that 20-34% of the same participants demonstrated inaccurate monitoring skills. After the session, 100% of the participants assessed themselves as good to excellent, yet some assessments were not accurate ( Table 5). These sessions also identi ed that one year after the PEW system implementation, only 42% of staff participating in bedside simulation said they had read the observation and monitoring guidance (despite it being recommended as mandatory training), and 58% of participants had completed the accompanying e-learning. The sessions evaluated well as being 100% useful and relevant to practice.
3) Chart completion: The composite of administrative and clinical parameters used to determine adequate chart utilisation (  Table 6. Utilisation of the PEW system chart over ten years. The 2018 audit included the six preceding sets of observations for 103 patients (4824 parameters). The documentation of patient risk factors is good (92%). Documentation of patient-speci c parameters, temperature and neurological observations improved (92 to 99%). The frequency of observations was less well documented (63%), but when documented, they usually (86%) matched the planned frequency.
The audit also showed that 1% (56/4824) of parameters were not documented. A further 1% (49/4824) of parameters were plotted inaccurately on the Y-axis. The error was identi ed because nurses documented the number and plotted the position on the Y-axis. The largest difference in plotted and written values was respiratory rate 9/min, heart rate 14/min, blood pressure 14 mmHg, and temperature 1.5°C. Although the actual number of inaccuracies is small, the degree of variation could be clinically signi cant and display an incorrect representation of the trend.
Outcome measures 4) Cardiac Arrest: The 109 patients with CA were of similar age (36 months) over the 3 time periods and had a similar probability of death on admission to PIC and a non-signi cant increased length of PIC stay.
The results for the primary analysis of the cardiac arrest incidence rate are presented in Although the number of predictable cardiac arrests appeared to be reduced (Figure 1), the numbers were small (n=39), so further analysis was not performed. There was no signi cant difference between the mortality for patients with predictable and non-predictable cardiac arrest.
The factors associated with these events are incomplete clinical observations, not recognising clinical deterioration, not calling for assistance when prompted by the referral algorithm, not communicating concerns adequately or with su cient urgency and PIC doctors reviewing the patient but PIC not having su cient capacity for immediate admission. These are similar challenges to what was happening we started this journey, although they occur less frequently.

5) Hospital Mortality:
The results for the primary analysis of mortality are presented in  Table 8. Incidence rate of mortality over time. CI: Con dence interval.

Discussion
We have reported the implementation of a Paediatric Early Warning System as a complex Implementation Science QI natural experiment using the MRC Guidance for the Development of Complex Interventions, Action Research method and Action Research theory. The problem was that patients deteriorated in the hospital without being recognised or escalated for critical care, and some of these experienced predictable cardiac arrest or potentially avoidable death. The missed opportunities were due to variation in the observation and monitoring of patients, a clinical knowledge and skills gap, and lack of feedback about optimal care. A complex of interventions was needed to address the multilayer, interconnected issues. The interventions were 1) developing observation and monitoring guidance to standardise practice and provide a template for optimal care, 2) standardised charting with an embedded PEW score, 3) clinical skills training, 4) clinical process audit and feedback and 5) outcome surveillance.
Over time, the process outcomes showed that the proportion of timely unplanned PIC admissions, clinical skills and chart completion compliance improved and was associated with a lower risk of actual mortality but a longer length of PIC stay. This is likely to re ect a change in clinician behaviour with earlier escalation and PIC admission, similar to other studies. [6,9,11] However, these patients also had a longer length of PIC stay, in contrast to Kovolos et al., which reduced capacity for 56 average elective admissions/year. [22] This contradicts our hypothesis for our population that earlier intervention will reduce PIC length of stay. The patients requiring unplanned admissions were younger (median age 12 months) and younger (8 months) over time. It is di cult to explain this trend, but it does provide some information into at-risk populations in our hospital.
A review of the clinical decision-making for patients with unplanned PIC admissions identi ed a signi cant improvement in timely PIC admission following implementation from 39% in 2008 to 92% in 2018, similar to the EPOCH trial. [11] Detailed reviews provided educational and institutional learning opportunities and fed back into the observation and monitoring guidance and the training system. These included the need for patient-speci c risk factors, parent/nurse concern, frequency of observations, escalation documentation and sepsis triggers. These new parameters appear to be mostly embedded since they were introduced in 2016. This area of learning is rich, vital for quality improvement and needs further investigation.
Routine clinical observations were sometimes performed inaccurately and in a similar range to other studies. [33][34][35][36][37][38][39][40] With purposeful scenario-based training, the clinical observation skills improved from 66 to 82% and appeared to be sustained, similar to other educational interventions. [6,[41][42][43][44][45] Despite training during and after implementation, there is inaccuracy and inconsistency in routine clinical observations that may have contributed to missed or delayed opportunities to identify and escalate clinical concerns about deterioration. Only half of the staff who participated in the sessions had read guidance or completed online training a year after implementation. Possible solutions to this could be to ensure that mandated training is completed or that multiple modalities are required to improve the standard of routine clinical observation and adjust to different learning styles. The clinicians were also not able to accurately assess the level of their own skills, which had important implications for ongoing training and assessment.
The seven clinical observations contributing to the PEW score were completed well after implementation in 2008 and had improved in 2018 (87 to 97%). The intermittent audits showed that in 2018, there were 2% of observations that had missing or inaccurate parameters with a consequent inaccurate total PEW score. However, those we identi ed could have had clinical consequences. The rate of documentation inaccuracy compares favourably to the reported rates of 7.5% in paediatrics versus 7.3 to 42% in adults.
The aggregate PEW score we use is the most widely validated, but there is little information on real-life performance [1,2,10,11,50]. Our use of the PEW score requires all seven physiological parameters to calculate, and it was documented in 95% of observations in contrast to the EPOCH trial pre-intervention 60 to 75% and post-intervention documentation of 99%. [11] However, the EPOCH trial only required ve of the seven parameters for calculation, and one centre reported 93% documentation of all seven parameters. [51] Documentation in paediatric patients appears to be better than the 70 to 90% reported in adult studies. [34][35][36][37][52][53][54] The score was accurate in 92% of patients (97% of the 95% that had a score documented), which is considerably higher than the 54 to 84% summation error in paediatric and adult studies. [34,46,51,52] Incorrect summation and plotting of parameters could contribute to inaccurate PEW scores, misrepresent the graphic trend and lead to omitted escalation. It is unclear what the acceptable or optimal accuracy threshold is for the score as part of a larger system of overlapping safety.
The overarching aim of the programme was to reduce harm from potentially avoidable critical illness leading to cardiac arrest and death. We identi ed a reduction in cardiac arrest and crude mortality two years after the implementation of a PEW system. There is a post-intervention halving of the cardiac arrest rate that, given the before and after rates, appears to be associated with PEW system implementation.
The predictable cardiac arrest rate also appears to be reducing, but the events are rare. Qualitative review of these cases provides valuable insights and opportunities for improvement, similar to cases with unplanned PIC admission. After three improvement cycles, the total cardiac arrest rate decreased signi cantly and was associated with a trend towards reduced predictable cardiac arrest and mortality reduced signi cantly. No further improvement was observed after cycles ve and six. The consensus on RRS outcome measurement recommends using ward bed days as a denominator, which may help with future trend analysis. An interesting observation is that the median age (36 months) and severity of illness (PIM2R) of patients suffering cardiac arrest have not changed over time. Patients who have unplanned PIC admissions are younger with a lower severity of illness. Could it be that the system is better at detecting younger patients before cardiac arrest?
Mortality is reduced over time in all populations, which makes it a challenging marker of effectiveness. [11,[18][19][20]55] Taking time trends into account, in an attempt to reduce the bias in a before and after study, it is possible that the reduction in mortality was attributable to the PEW system implementation. The EPOCH cluster randomised control trial showed no difference in mortality across 21 sites internationally.
[11] The implementation was six months, and the assessment period was 12 months after intervention.
We would argue that our data support longer implementation and embedding to demonstrate effect. [56] With early warning systems, it is di cult to determine from the published literature whether improved outcomes are related to the time or implementation delity of a complex intervention in a receptive culture. Single-centre studies that report improved outcomes report longer follow-up from 18 to 36 months. [9,[19][20][21][22] A longer-term follow-up study of the outcomes at the EPOCH intervention sites would be an interesting test of this hypothesis. In contradiction of the 'longer time to embed' hypothesis is a longitudinal time series with the introduction of paediatric Medical Emergency Teams in the USA. [25] There is no additional improvement in mortality or cardiac arrest over the baseline time trends. What isn't known about these centres is how well the early identi cation was developed and embedded. The improved outcomes in our centre were also unchanged by the introduction of a nurse-led PART ve years later and increased critical care resources in and out of PIC. This supports the bene ts of improving identi cation and escalation without the investment in a RRT. It could be that introducing response resources does not necessarily improve detection and may even distract efforts from the afferent limb to the efferent or response limb of the system. Implementing a complex dynamic and inter-related intervention in a specialist healthcare environment is challenging. [11,[57][58][59] Guidance on developing, implementing, evaluating and publishing complex interventions are available [13,59], but there are few actual case studies of implementing complex interventions in healthcare and relating to early warning systems. [60,61] The early warning or rapid response literature has focused broadly on 'does the score/response team work' [1,2,3,8,10] and, more recently, attempts to understand why the score/response team appears to work in single-centre studies and not large multi-centre studies [3,10,25,56,62,63]. It could be that the score/response team doesn't work or it could be that the implementation science is key. The deep dive qualitative studies reveal that compliance with early warning systems requires effective and meaningful communication between multidisciplinary staff as well as the overarching organisational context, including culture, quality improvement, resources, training and sta ng. This highlights that we cannot implement or study parts of these systems in isolation; they need to be embedded, maintained and assessed within their appropriate context. Given the iterative improvement cycles required to improve the intervention and compliance and therefore outcomes, it will be di cult to determine the e cacy of these necessarily bespoke interventions with randomised trials. Single centres show that outcomes can improve with or without teams. [19] Despite the methodological bias, it may be worth looking more closely at the interventions in successful single-centre implementation to understand the system and culture change. What appears to be important is measuring and managing the same outcomes, with purposeful quality improvement. [13].
What we have learned from audits about embedding a complex system is the importance of the link with education. Our implementation required a PM followed by a nurse educator and then continued the education process within our normal business. We have updated our training for PEW system e-learning, guidance, charts and escalation to be mandatory and require periodic updates. Predictable and potentially avoidable life-threatening events are debriefed and investigated with root cause analysis; the lessons learned are shared and contribute to revisions of guidance and education. [17,[64][65][66][67][68][69] We would recommend keeping records of staff who participate in each form of training to help target staff areas that need more or different input. Multimodal educational opportunities, in particular active learning within the real clinical context, are needed for different learning styles. The case reviews, with feedback on the monitoring and identi cation of each child that has deteriorated, offer a valuable re ective learning experience. Appreciation and recognition of excellent observation, assessment and documentation, enabling early identi cation, is quick and effective in reinforcing careful monitoring. A Nurse Educator working clinical shifts is another effective way of embedding the skills in real-time in work areas that perceive themselves as too busy to participate in training. Encouragement and reinforcement from nursing and medical leaders is required during daily ward rounds to ensure that the system is being used correctly and to establish a good example with effective role modelling.
Monitoring the rates of all and predictable cardiac arrest is recommended as a quality metric for the evaluation of RRS. [13] The value of reviewing predictable cardiac arrests extends beyond measuring the rate. The qualitative information from the investigations helps to improve the system, education and knowledge. Review against guidelines and standards identi es avoidable factors and creates opportunity for learning. Feedback on each patient helps to reinforce new behaviours and optimise shared knowledge. [66,69] Our avoidable factors remain similar to those reported in 54% of hospital deaths examined in 'Why children die?'.
[4] When our PEW system was introduced, senior clinical decisionmakers took time to 'believe' that the system was indeed bene cial. These real cases contributed greatly to the visibility of avoidable factors and their faith in the systems detection of deterioration. Dismissive attitudes changed to embrace retrospective review for timely and appropriate management, and in time, behaviour has shifted to prospective review and anticipation. This culture change is a hard outcome to measure, but we believe it is important. [17,61,65] The main limitation of this study, from a single specialist centre, is the before and after bias for which we have tried to compensate by comparing rates and accounting for time trends. There were also many system changes over the study period, and it is not certain that the PEW system implementation in uenced the outcomes. In 2008, the implementation of PEWS coincided with a reduction in overnight doctors in training and the introduction of specialist nurses. It is possible that the improved outcomes could be due to the sta ng change or a time effect. There was, however, no change in the measured outcomes with the sta ng changes or time in the period from 2011 to 2018. It is disappointing that the reduction in poor outcomes returned to the previous slow rate of improvement during the long follow-up.
It may be that we need another 'innovative disruptive bump'. An example would be using new technologies to improve the ergonomics of detection and escalation, making it easier for clinicians to see deterioration and respond appropriately, as well as have more timely feedback. The process measures were infrequent during implementation and maintenance; it would have been better to have regular measures to inform us when changes occurred. However, being a single centre has had the strength of being able to track the changes within a single but evolving culture. We have gained an in-depth understanding of our system and adapted it in this environment. It is possible that the application of these processes may not be applicable in different or less specialist institutions; however, a version of these processes has been implemented in regional hospitals. By the very complex nature, PEW system implementation will need to take into account the local environment, context, culture and resources. By measuring implementation delity with process outcomes as well as the important outcomes hopefully, each journey will be able to minimise missed opportunities and to improve the outcomes for deteriorating patients. [13] What this natural experiment over 10 years adds to the extensive literature is the time trend of improved detection, reduced cardiac arrest rate and mortality as well as the need for focussed theoretically based QI for these complex interventions in complex systems. The outcomes in our and other centres took 18 months to improve. This will hopefully inform future study designs and national outcome surveillance. We have made signi cant progress towards improving the detection of deterioration, but unfortunately, we have not quite met our goal of preventing all predictable and potentially avoidable cardiac arrest and deaths. For the future, we will continue to look for opportunities to learn about avoidable factors. We will also continue to implement changes, such as ergonomics, wireless monitoring, electronic charting, smart alarms and advanced communication systems. We will use the Framework for Theorising and Evaluating Nonadoption, Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability of Health and Care Technologies to help with tinkering and embedding in the pursuit of zero avoidable harm. [61] Conclusion Following a PEW system implementation, in a longitudinal single centre study, without a medical emergency team, unplanned PIC admissions were admitted with a lower severity of illness and had improved survival. Hospital mortality rates and cardiac arrest rates have also reduced following the implementation of a PEW system beyond the background reduction. The many other sta ng changes and resources introduced since 2010 have not improved these outcomes further.
Routine clinical observations are performed imperfectly and can improve with purposeful practice.
Clinical staff overestimate their abilities to perform accurate routine observations and thus need testing and training to improve clinical skills. The chart and system compliance monitoring identi ed a high (but not perfect) level of implementation delity with improved good practice regarding the documentation of physiological parameter escalation. In this context, the proportion of patients receiving optimal pre-PIC care and timely PIC admissions has improved.
For the future implementation of National PEWS, many questions remain: Which is the most effective form of training? How frequently does training need to occur to maintain knowledge and skills? Is there a 'dose' of training for a ward sta ng population that can optimise early recognition and management of deterioration? Would this training need to depend upon the frequency to which someone is exposed to seriously ill children in their practice? How can we most e ciently educate our staff to optimise their performance? What level of adherence or accuracy of routine clinical observation is good enough? What is the optimal system adherence to strive for? What would it take to move from 80 to 100% accuracy and what would this mean for patients? Further research is also needed to help us understand the ergonomic and human factors in identifying and escalating concerns about deterioration. Finally, to understand the ideal application of PEW systems, we need to identify the socio-technical factors deployed in the successful single centres that are potentially leading to improved outcomes and evaluate them for spread and further study.
Declarations Figure 1 Mean total and predictable cardiac arrest rate over time.