Implementation Study of a 5-component Pediatric Early Warning System (PEWS) in an Emergency Department in British Columbia, Canada, to inform provincial scale up

doi:10.21203/rs.2.12228/v1

Download PDF

Research article

Implementation Study of a 5-component Pediatric Early Warning System (PEWS) in an Emergency Department in British Columbia, Canada, to inform provincial scale up

https://doi.org/10.21203/rs.2.12228/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 27 Nov, 2019

Read the published version in BMC Emergency Medicine →

You are reading this older preprint version

Read the latest preprint version →

Background: The rapid identification of deterioration in the pediatric population is complex, particularly in the emergency department (ED). A comprehensive multi-faceted Pediatric Early Warning System (PEWS) might maximize early recognition of clinical deterioration and provide a structured process for the reassessment and escalation of care. The objective of the study was to evaluate the implementation fidelity, effectiveness and utility of a 5-component PEWS implemented in the ED of an urban public general hospital in British Columbia, Canada, to guide provincial scale up. Methods: We used a before-and-after design to evaluate the implementation fidelity, effectiveness and utility of a 5-component PEWS (pediatric assessment flowsheet, PEWS score, situational awareness, escalation aid and communication framework). Sources of data included patient charts, surveys of direct care staff, and key-informant interviews. Data were analyzed using mixed-methods approaches. Results: The majority of charts had documented PEWS scores at triage (80%) and first bedside assessment (81.2%) indicating that the intervention was implemented with high fidelity. The intervention was effective in increasing vital signs documentation, both at first beside assessment (84% increase) and throughout the ED stay (>100% increase), in improving staff’s self-perceived knowledge and confidence in providing pediatric care, and self-reported communication between staff. Satisfaction levels were high with the PEWS scoring system, flowsheet, escalation aid, and to a lesser extent with the situational awareness tool and communication framework. Reasons for dissatisfaction included increased paperwork and incidence of false-positives. Overall, the majority of providers indicated that implementation of the PEWS system and completing a PEWS score at triage alongside the Canadian Triage and Acuity Scale (CTAS) added value to pediatric care in the ED. Results also suggest that the intervention is aligned with current practice in the ED.

Critical Care & Emergency Medicine

Pediatric Early Warning System

Pediatric Early Warning Score

PEWS

emergency department

pediatrics

implementation study

mixed methods

Globally, failure to identify and intervene with pediatric patients experiencing clinical deterioration is a source of unintended harm including prolonged hospital stays or readmission, disability and death (1–3). Care inequities can arise due to diversity of provider experience and knowledge, and lack of standardized approaches in the management of critically ill children (1).

To maximize early recognition of clinical deterioration in children, Pediatric Early Warning Systems (PEWS) have been implemented internationally with a growing body of evidence showing benefit of their use (4–6). PEWS should support healthcare providers in identifying abnormal physiology, tracking trends across time and supporting structured processes for reassessment and escalation of care.

In emergency department (ED) settings, the ability to identify deterioration or risk “in the moment” – without tracking over time – is critical (7); however, rapid identification of deterioration in the pediatric population has several complexities. These difficulties include: varying normal vital signs parameters by age (8,9), changes in physiologic parameters related to factors beyond the illness or injury (medication, pain, fear and anxiety) (10), and compensation seen in pediatric patients when critically ill, underrepresenting the degree of deterioration (7,8). Using PEWS in the ED could offer benefit in the focus and guidance provided on age-appropriate pediatric norms, measuring physiologic parameters and acting on evidence of abnormality (7,11,12).

While implementation of PEWS in EDs is becoming increasingly common globally, with mounting evidence for this application (8,10,11), there is still much to learn. Most studies of PEWS in ED to date focus on the predictive value of a score in determining level of care at disposition, with the overarching conclusion being that a PEWS score is reasonable at predicting admission to the intensive care unit (ICU), but has limited value in predicting admission to a non-ICU bed (10,11,13). No study has yet examined PEWS as a complex healthcare intervention (4). There has been minimal research focusing on the general utility of PEWS for the ED (10), yet utility is particularly important for ED as it is a dynamic environment with undifferentiated patients often seen for a relatively limited time (4,10). The operational characteristics of busy EDs include intermittent high patient loads, diagnostic uncertainty, multiple handovers, intense time pressures and diverse skills owing to high staffing levels and rapid turnover (7,13). Researchers, such as Roland et al. (7), built a convincing case for PEWS in ED as a means of minimizing cognitive factors and biases that impact decision making.

In British Columbia, one of the ten provinces of Canada, PEWS had not been used in ED settings prior to this study. In 2014, a provincial working group consisting of members of health authorities, academia and healthcare providers came to a consensus that PEWS for inpatient settings should be pursued, not only to improve health outcomes but also to standardize assessment and care of children, facilitate cross-site communication processes and enhance patient transfer across the varied hospitals in the province. However, testing the effectiveness and utility of PEWS in ED settings was considered to be necessary before a provincial roll out, given a scarcity of scientific evidence.

The objective of the study was to evaluate the implementation fidelity, effectiveness, and utility of a 5-component PEWS implemented in the ED of an urban public general hospital in British Columbia (BC), Canada. This article will outline the planning, implementation and results of this pilot study and discuss how the results are being used for provincial scale up.

Setting

The study setting was the ED of Richmond Hospital – an urban public general hospital located in British Columbia, Canada, serving an average of 6800 pediatric patients per year; approximately 12% of total ED patient visits. No PEWS was in place before this study and documentation occurred on a non-pediatric specific emergency nursing assessment record. Richmond Hospital uses the Canadian Triage and Acuity Scale (CTAS) (14) to triage patients by severity and type of presenting symptoms and signs. The scale consists of five levels from 1 (resuscitation) to 5 (non-urgent).

Intervention design

The design of a PEW system for inpatients involved both an evidence review and in-depth consultation with numerous stakeholders, including health authority leads, quality and safety leads, expert clinicians and frontline care representatives. The result was a five-component PEWS system, consisting of the following: pediatric assessment flowsheet, Brighton PEWS score (15), situational awareness (2), escalation aid, and communication framework (16). More detail about each component is provided in Table 1. For the pilot study of PEWS in ED, we reviewed the literature related to the ED context, studied the implementation of PEWS in the in-patient setting and consulted with ED experts from around the province. The Richmond team agreed to pilot this inpatient system in the ED, understanding that tailoring to ED would be completed post pilot if the decision was made to move to scale up.

PEWS scoring occurred at triage and then with each vital signs assessment throughout the stay. PEWS scoring was to be completed at a minimum of every two hours for children with a PEWS score of 0-3 and more frequently for higher PEWS scores (4-13) (Figure 1). We hypothesized that this system would enhance providers’ ability to recognize and communicate risk and accelerate mitigating actions.

Table 1. Description and source of the five components that constitute the BC Pediatric Early Warning System (BC PEWS)

Component	Description	Source
Pediatric assessment flowsheet	The double-sided flowsheet, designed for inpatients, comprehensively outlines documentation for 24 hours of nursing assessment, including PEWS scoring parameters, full head-to-toe assessment and documentation of routine nursing care.	Adapted from BC Children’s Hospital
	The flow sheets are available in six age grouping (0-3 months; 4-11 months; 1-3 years; 4-6 years; 7-11 years and 12+ years) to account for naturally-occurring variations vital signs norms (14). Staff were also provided with vital signs reference cards that could be worn on a lanyard.	CTAS (14)
PEWS score	The Brighton PEWS score embedded in the flowsheet is the most widely used and validated PEWS score available for inpatient care. It is a 13-point score (with 0 normal and 13 high risk) based on behavioural, cardiovascular and respiratory parameters.	Brighton PEWS score (15)
	As the Brighton scoring tool is not age specific, vital signs references for PEWS scoring were based on Canadian Triage Acuity Scale (CTAS) vital signs norms (14,17). These norms were chosen to promote internal consistency within sites and between sites based on a nationally accepted standard. The PEWS scoring section of the flowsheet has the norms embedded for easy plotting and are colour coded to provide a clear visual when vital signs are outside of the normal range.	CTAS, Fleming et al. (17)
Situational awareness	The intent of situational awareness is to promote awareness, prediction, and mitigation of potential risk. Implemented tools in the ED setting included posters for visual cueing, discussion in staff reporting and regular documentation of four factors embedded in the flowsheet (caregiver concern, unusual therapy, watcher patient, elevated PEWS scores, and communication breakdown).	Adapted from the Cincinnati Situational Awareness Model (2)
Escalation aid	The escalation aid outlines actions to support clinical decision making following assessment. Recommended mitigation actions (e.g. notification, reassessment, consultation) correspond to PEWS scores and situational awareness factors. A quick-view of the escalation aid was also embedded in the flowsheet.	Adapted from Cook Children's Medical Center
Communication framework	The Situation, Background, Assessment, Recommendation (SBAR) toolkit was used to improve communication between team members regarding patient status.	SBAR toolkit (16)

Figure 1. Flow map of PEWS in the Emergency Department. (CTAS: Canadian Triage and Acuity Scale)

Intervention implementation and monitoring

A team consisting of staff from nursing, medicine, and administration assumed active site leadership roles in implementation and support to health practitioners. Educational resources that had been developed for the provincial inpatient implementation were used for training the nurses in this ED pilot, including instructions for using the flowsheet, e-learning training modules, on-site training workshops and case studies. Training reinforced the concept that PEWS was a tool to aid clinical judgment, not replace it.

We monitored implementation fidelity through quality audits of medical records at five points during the one-year implementation. These audits were based on selected quality indicators: utilization rates of flowsheets, completion and accuracy of PEWS scoring, and documentation of situational awareness factors and escalation of care. Results were used to provide further education where needed and to promote optimal uptake.

Evaluation

We took a mixed-methods approach, triangulating qualitative and quantitative data, to evaluate this intervention across the three dimensions of fidelity, effectiveness, and utility, following Campbell et al.’s framework for evaluation of complex health interventions (18). These dimensions and their indicators were selected based on current literature on evaluation of PEWS initiatives, input from provincial stakeholders regarding relevance, and feasibility (table 2).

Table 2. Dimensions, indicators, and sources of data selected for evaluation

Evaluation dimension	Indicators	Data source
Implementation fidelity: whether the program was implemented as designed and staff were satisfied with it.	PEWS score documentation	Post-implementation chart review
	Accuracy of PEWS score calculation	Post-implementation chart review
	Satisfaction with PEWS ED and its implementation	Online provider survey
	Barriers and facilitators in implementation	Online provider survey and key-informant interviews
Effectiveness: whether the intervention changed practice.	Increased vital signs documentation	Pre and post implementation chart review
	Increased knowledge and confidence in pediatric care as a result of intervention	Online provider survey
	Perceived changes in practice as a result of intervention	Online provider surveys and key-informant interviews
	Improved communication between staff	Online provider survey
Utility: whether the staff found the intervention valuable and it aligned with current practice.	Alignment of PEWS score with CTAS	Post-implementation chart review
	Perceived usefulness and value of PEWS in ED and its components in provision of care	Online provider surveys and key-informant interviews

Data collection

We used three types of data collection techniques: medical chart review, online survey of providers and semi-structured interviews with key informants.

Medical chart review

We carried out retrospective medical record reviews covering the 12-months pre-implementation as baseline, and the 12 months post-implementation for comparison.

The inclusion criteria for chart selection were patients who (i) were seen in ED in the 12 months pre and post implementation, (ii) were under 17 years of age, (iii) had an ED length of stay greater than two hours (to allow for trending) and (iv) had a disposition either transfer to higher level of care or admission to hospital, and (v) had a CTAS assignment of 1-4 (as proxy measures of acuity and severity). We stratified our sample to include patients across all CTAS levels 1-4 and age groups (0-3 months, 4-11 months, 1-3 years, 4-6 years, 7-11 years, 12-17 years).

Data were extracted from the medical records by two experienced pediatric acute care nurses

and inputted in a electronic data capture tool hosted BC Children’s Hospital Research Institute. REDCap© (Research Electronic Data Capture) is a secure, web-based application designed to support data capture for research studies (19). Identifiers were removed and a unique study code was generated for each patient. The nurses maintained field notes, and in the case of uncertainty, decision were made within the team.

Online survey of providers

One year post implementation, ED nursing staff and physicians were invited to participate in an online survey to explore and describe key issues related to implementing PEWS in ED. Survey questions focused on exploring perceptions related to changes in practice, knowledge and confidence in providing pediatric care, usefulness of PEWS, satisfaction with the tools and training, as well as barriers and facilitators to implementation. Surveys took approximately 20 minutes to complete, included closed and open-ended questions, and were administered anonymously through Fluid Survey.

Semi-structured interviews with key informants

The hospital administrator, registered nurse and physician who championed and supported the intervention were interviewed face-to-face for approximately 40 minutes after 12 months of implementation. Interview questions focused on understanding the utility of PEWS ED and identifying the barriers and facilitators to implementation. To confirm accuracy and credibility, the interviewer (YT) took notes capturing the responses and then summarized these for participants at the end of the interview.

Data analysis

We used descriptive statistics to calculate the overall characteristics of patient charts and Fisher’s exact test to determine if they were significantly different in any aspect. We used two-samples tests for equality of proportions to determine whether vital signs documentations significantly increased post-implementation compared to baseline. We used Spearman’s correlation to evaluate the relationship between PEWS scores and CTAS scores. Statistical analyses were carried out using Microsoft Excel and RStudio (20).

Response options of the online survey with 5-level Likert scales were collapsed into a measure consisting of 3 levels. Frequencies of responses were calculated for the different categories.

Qualitative data from open-ended survey questions and interviews with key informants were analyzed using thematic content analysis (21), in order to understand the most salient barriers and facilitators for the teams using PEWS in ED. Responses were reviewed, coded and categorized into commonly occurring themes by two separate researchers to enhance trustworthiness of the data.

Collected data

In total, 192 medical records were selected for review; 96 at baseline and 96 at final evaluation. The two samples were not significantly different from each other (adopting a statistical significance of 5%), with the only exception being discharge disposition from ED, which can be explained by the opening of a pediatric short stay unit in the ED during the study period (Table 3).

Table 3. Characteristics of patients included in medical record review pre and post implementation (n= 96 for each sample)

Sample characteristic	Pre-implementation n (%)	Post-implementation n (%)	p-value *
Female	52 (54.2)	39 (40.6)	0.08
Age
0-3 months	22 (22.9)	17 (17.7)	0.16
4-11 months	4 (4.2)	11 (11.5)
1-3 years	20 (20.8)	23 (24.0)
4-6 years	14 (14.6)	18 (18.8)
7-11 years	16 (16.7)	17 (17.7)
12-16.9 years	20 (20.8)	10 (10.4)
Most responsible diagnosis †
Respiratory	23 (24.0)	26 (27.1)	0.24
Gastrointestinal	10 (10.4)	19 (19.8)
Hyperbilirubinemia	10 (10.4)	8 (8.3)
Other diagnosis	53 (55.2)	43 (44.8)
Discharge disposition from ED
Transferred to higher level of care	37 (38.5)	22 (22.9)	0.03
Admitted internally	59 (61.5)	74 (77.1)
CTAS level
CTAS 1	7 (7.3)	10 (10.4)	0.05
CTAS 2	44 (45.8)	43 (44.8)
CTAS 3	40 (41.7)	28 (29.2)
CTAS 4	5 (5.2)	15 (15.6)

* P-value calculated with Fisher's exact test for count data.

† 3% entries missing during pre-implementation review. Diagnosis based on most affected system. If multiple diagnoses were presented in discharge summary, priority was given to the first one written.

CTAS: Canadian Triage and Acuity Scale; ED: Emergency Department; PEWS: Pediatric Early Warning System

As for provider surveys, the response rate was 38% for registered nurses (n=37) and 72% for physicians (n = 13). Half of the practitioners had less than 5 years of clinical work experience. 39% of physicians and 21% of nurses reported having more than 15 years of work experience. Most estimated that 10-25% of their practice was pediatric care.

Three key-informant interviews were conducted.

Implementation fidelity

Chart reviews show that the intervention was implemented with high fidelity. At triage, 77 (80%) of the charts had a PEWS score completed and of those 67 (87%) had been accurately calculated. At first bedside assessment, 78 (81.2%) charts had a PEWS score present and of those 69 (88.5%) were accurate. Medical record review found the most common reason for inaccuracy in PEWS score to be adding the sub-scores instead of taking the highest score for the section, a matter that key informants suggested was improved through practice and more education. At both triage and first bedside assessment, highly urgent or non-urgent patients (CTAS scores 1 and 4) were more likely to be missing a PEWS score than others (p-value = 0.0375 for triage and 0.03536 for bed-side assessment, Fisher’s exact test). In 52.1% (n = 25) of patients with an initial PEWS score of 0-3, the time in between assessments was two hours or less, as per the implementation plan. In 61.4 % of the records (n = 59), a PEWS score was completed at all vital signs assessments throughout the ED visit. In an additional, 29.2% (n = 28) PEWS score was completed for over half of the vital signs assessments.

Overall, majority of survey respondents were satisfied or very satisfied with PEWS scoring system (71.8% nurses, 81.8% physicians), PEWS flowsheet (56.2% nurses, 81.2% physicians), escalation guide (68.8% nurses, 81.8% physicians), and reference cards (75% nurses, 70% physicians). Satisfaction was relatively lower for situational awareness tools (41.2% nurses, 36.4% physicians) and the communication framework (54.5% nurses, 45.5% physicians).

Intervention effectiveness

Comparison of charts from before and after implementation showed that adding PEWS scoring to the assessment of pediatric patients significantly increased the rates of documentation of seven parameters embedded in the score at first bedside assessment and throughout the ED stay (Table 4).

Table 4. Completeness of documentation based on medical record review pre and post implementation

PEWS score component

Pre-implementation

(n = 96)

Post-implementation

(n = 96)

Increase

p-value *

Documentation of parameters at first assessment in the ED

Respiratory rate

60 (62.5)

94 (97.9)

57%

<0.01

Oxygen concentration

62 (64.6)

90 (93.8)

45%

<0.01

Respiratory distress

53 (55.2)

88 (91.7)

66%

<0.01

Heart rate

63 (65.6)

94 (97.9)

49%

<0.01

Capillary refill time

29 (30.2)

86 (89.6)

>100%

<0.01

Skin colour

41 (42.7)

86 (89.6)

>100%

<0.01

Behaviour

56 (58.3)

91 (94.8)

63%

<0.01

Average

52 (54.2)

90 (93.6)

84%

<0.01

Consistent documentation of parameters throughout ED stay †

Respiratory rate

30 (31.3)

91 (94.8)

>100%

<0.01

Oxygen concentration

28 (29.2)

83 (86.5)

>100%

<0.01

Respiratory distress

6 (6.3)

80 (83.3)

>100%

<0.01

Heart rate

32 (33.3)

94 (97.9)

>100%

<0.01

Capillary refill time

0 (0.0)

81 (84.4)

<0.01

Skin colour

1 (1.0)

82 (85.4)

>100%

<0.01

Behaviour

5 (5.2)

82 (5.4)

>100%

<0.01

Average

15 (1.2)

85 (88.2)

>10%

<0.01

Note: Percentages are shown in parenthesis

* p-value calculated with two-sample test for equality of proportions

† Consistent documentation refers to documentation of each parameter with every assessment

Using BC PEWS further promoted documentation of two situational awareness factors throughout the patient stay: recording of “caregiver concern” increased from 10.4% to 67.7%, and “watcher patient” from 2.1% to 40.6%. “Watcher patient” was noted by the chart reviewer to be particularly important for documenting risks that PEWS score is not designed to capture, including: pain, surgical risk, abnormal lab values, dehydration status, neurologic status, and mental health risk.

We also observed a 51% increase in documentation of notification of most responsible physician and documentation of response by physician.

The majority of practitioners responding to the post-implementation survey reported that the implementation of BC PEWS changed their knowledge of and confidence in providing pediatric care to a “great” or “very great” extent (Table 5). The biggest improvement in knowledge was in identification of abnormal clinical signs (58.1% change to a great or a very great extent) (Table 5).

Table 5. Intervention effectiveness: Perception of change in knowledge and confidence of practitioners and communication between them.

	Not at all / to a slight extent (%)	To a moderate extent (%)	To a great / very great extent (%)
Change in knowledge
Identification of abnormal clinical signs	25.6	16.3	58.1
Identification of situational awareness factors that increase risk	34.9	20.9	44.2
Mitigation of deterioration	23.3	34.9	41.9
Escalation of care	25.6	25.6	48.8
Change in confidence
Identification of abnormal clinical signs	23.3	34.9	41.9
Identification of situational awareness factors that increase risk	32.6	37.2	30.2
Mitigation of deterioration	30.2	32.6	37.2
Escalation of care	32.6	30.2	37.2

Interviewed key-informants highlighted a few important issues about the effects of implementing PEWS at triage: 1) the tool was easily accepted and used, i.e. its adoption took little promotion or management from leadership, 2) time to complete PEWS scoring decreased with practice, and 3) the slight increase in triage time was outweighed by thoroughness of assessment and greater awareness of patient status; this was particularly important when the department was busy and wait times were long. Surveyed physicians noted an overall increase in staff awareness regarding pediatric patients, which promoted earlier notification. “I think the general increase in awareness of frontline staff has improved, and that either escalates the CTAS score, or prompts the RNs and others to bring the patient to our attention sooner” (Physician).

Lastly, the survey respondents reported enhanced communication between practitioners with the implementation of PEWS, particularly regarding timing of verbal communication but also frequency and clarity (table 6). One nurse highlighted: “If the PEWS score is rising or falling, it gives us a reason to contact the MD in clear, concise language” (Nurse).

Table 6. Implementation effectiveness: perception of improvement in in-between staff communication as a result of BC PEWS

Perceived improved communication between practitioners †	No	Somewhat	Yes
Frequency of verbal communication	17.9%	30.8%	51.3%
Timing of verbal communication	20.5%	20.5%	59.0%
Clarity of verbal communication	23.1%	25.6%	51.3%
Outcomes of verbal communication	17.9%	35.9%	46.2%

Intervention utility

Overall, the majority of survey respondents (78.9%) agreed that implementation of PEWS in the ED was valuable for pediatric patient care. One nurse highlighted: “It increased our response to children with abnormal vital signs that are compensating or appear relatively well.” The leadership also noted: “PEWS helped validate their clinical decision making and offered a standardized approach to care.”

The PEWS score, assessment flowsheet, and escalation were overall seen as highly useful (average 78.3% indicated useful to moderate extent or higher). While the situational awareness and communication framework were not as favorable, they were still seen as moderately useful or higher by over 65% of the respondents.

Some participants perceived little value in using PEWS (5.3%). Their reasoning related to the need to further tailor the flowsheets to the ED environment: “PEWS form is good, but frustrating to have to use old form too. It would be great if there was an ED specific PEWS form” (Nurse).

Table 7. Intervention utility: staff’s perception of usefulness of different BC PEWS components.

	Not at all / to a slight extent	To a moderate extent	To a great / very great extent
Usefulness
Pediatric assessment flowsheet	25.6	32.6	41.9
PEWS score	20.9	23.3	55.8
Situational awareness *	38.0	31.0	31.0
Escalation aid	18.6	27.9	53.5
Communication framework	30.2	32.6	37.2
Overall value to pediatric care in ED ‡	5.3%	15.8%	78.9%

* Average of responses of the four types of situational awareness (caregiver concern, unusual therapy, watcher patient and communication breakdown)

To study alignment of the intervention with current practice, we explored how the PEWS score relates to CTAS score, the well-established triage scoring system. Seventy-seven charts included both a CTAS score and PEWS score at triage. Collapsing PEWS scores 5-13 into a single score (since they would trigger the same escalation guide), we found PEWS scores (0-5) CTAS scores (1-4) to be inversely correlated (Spearman’s rho = -0.574, p-value < 0.001).

Furthermore, the majority of surveyed nurses (89.6%) felt that it was “valuable” or “possibly valuable” to complete a PEWS score at triage alongside CTAS (physicians are not involved in triage, therefore were not asked these questions). Practitioners noted that the PEWS score provided a baseline for trending across the visit and gave more objective, thorough results by promoting assessment of all parameters consistently with all patients, which helped to correctly assign a CTAS score. “It (PEWS score) helps me a lot with assessment and priority settings in triage especially for the paediatric population who came in really sick and unable to obtain complete information/data from parents, guardians or significant other” (Nurse).

The impressions of staff regarding the impact of BC PEWS on pediatric practice is summarized in Table 4. Matters such as earlier identification of risk, more comprehensive assessment, and standardized approaches to communication and mitigation were seen as positive changes. Meanwhile, scores not representing the degree of risk were a reported weakness, particularly at triage e.g. false positive scores when the child is upset or crying, has fever or has been given medication for symptom relief: “Many children find triage to be overwhelming and often are crying, increasing their PEWS score despite looking well” (Nurse).

Table 8. Themes of perceived positive and negative impacts of BC PEWS on pediatric practice

Perceived positive impacts of BC PEWS on pediatric practice
Identification	· Prompts earlier recognition of risk, change, decline and abnormality · Increases provider’s general awareness of risk, concern and abnormality · Guides triage decisions
Assessment	· Provides a standardized assessment framework · Improves ease and comprehensiveness of assessment (full vital signs) · Increases staff comfort with vital signs norms
Monitoring	· Provides a baseline for monitoring from triage onwards · Increases frequency of vital signs assessment and closer observation · Improves ability to trend across patient stay which helps with care and disposition decisions
Communication	· Provides a standardized approach to communication (speaking same language) · Promotes earlier notification of physicians · Enhances delivery of thorough information to physicians · Improves confidence of nurses with notification (validation by score)
Mitigation	· Provides a standardized approach to escalation · Supports earlier response as notification occurs faster
Other	· Promotes better overall care for pediatric patients
Perceived positive impacts of BC PEWS on pediatric practice
Accuracy		· Scores may not accurately capture clinical status in some instances (e.g. false positive scores due to upset, post medication, etc)
Autonomy		· Standardized scoring and escalation can take away from clinical judgement
Workload		· Can increase time for assessment (particularly at triage)
Lack of tailoring to ED setting		· Increases paperwork because poorly integrated with current ED paperwork (double charting) · Form is missing important information for ED (e.g. narrative space, medication record)
Relevance		· Lacks relevance or seems excessive for patients with single system or minor injuries

This study presents the implementation of PEWS ED in a general urban hospital in British Columbia, Canada. Employing a framework for evaluation of complex healthcare interventions and using chart audits, surveys, and key-informant interviews, we evaluated the intervention’s fidelity, effectiveness, and utility (18). Our study supports that a multi-faceted PEWS can be valuable in supporting ED practitioners with providing timely and effective pediatric care. However, we concluded that it is important that the tools and training strategy are tailored to the ED context to enhance implementatioin fidelity and sastisfaction.

Using PEWS and CTAS at triage

In this pilot intervention, PEWS scoring was completed at triage alongside CTAS scoring (14). These two scores have been designed for different purposes and their scoring is done differently: PEWS identifies early signs of deterioration is and scored through assessment of vital signs (4), while CTAS assigns acuity and is scored via assessment and clinical judgement (14). We were able to show that the scores correlate and that practitioners found them synergistic; we therefore conclude that there is benefit in using both systems with pediatric patients. The PEWS score calculation reinforces a more comprehensive, objective assessment based on vital signs norms based on age. In our study, implementation of PEWS significantly increased documentation of physiologic parameters; a benefit reported by others (1). This systematically addresses issues of variance across providers and as highlighted by Roland et al. (7), may reduce cognitive bias, errors or missed assessment components that could otherwise occur because of the pressures of the triage environment (22).

Study participants noted a few limitations when using PEWS at triage. An increase in time at triage was highlighted; an issue which improved with practice and was largely acceptable to practitioners who indicated the value of scoring outweighed time increase. There were also instances of false positive scores owing to factors such as irritability, which altered vital signs and skewed PEWS scores. In our study, we aimed to counteract this issue through reinforcing the importance of clinical judgement and repetition of assessment when score accuracy was in question. Limited specificity has also been reported elsewhere (9,10,12). As parameters included in a PEWS score assessment are important for determining risk, this substantive increase in thorough assessment and documentation at triage is a very positive change. The limitations of PEWS were acceptable to practitioners who overall found value in using PEWS at triage alongside the CTAS triage tool.

Using the 5-component BC PEWS at bedside evaluation

It has been demonstrated that nurses and doctors may fail to recognize deteriorating children due to lack of consistency or accuracy in recording physiological observations (23,24) . Consistent and comprehensive documentation has been shown to be particularly important in the ED environment where health care providers need to document improvement and appropriateness for discharge (7). Moreover, with pediatric patients, this documentation is even more critical because providers often rely on general appearance and vital signs as opposed to patients’ descriptions of improvement. This study reviewed documentation of PEWS components across the patient stay. The introduction of PEWS was associated with a substantive increase in documentation across all physiologic parameters. We conclude that the introduction of PEWS in ED can have a positive influence on rates and thoroughness of documented assessment parameters. This not only provides a more consistent ‘in the moment’ picture of physiologic abnormality, but allows for documentation of trending from the point of presentation.

The 'active ingredients' of BC PEWS

BC PEWS was designed as a multifaceted safety system as advocated by Lambert et al (4). In trying to understand its utility as a system, we asked practitioners to assess the usefulness of each of the components. While all of the components of BC PEWS were ranked useful by the majority of respondents, the PEWS escalation aid and score were perceived as most useful. Practitioners reported that the norm-based scoring enhanced or validated identification of sick children (7). The escalation aid justified escalating to senior review and encouraged repeated assessment or more thorough monitoring throughout the patient stay.

The systematic documentation and reporting of situational awareness factors were ranked as the least useful part of the system. Although there was a substantive rise in the documentation of “caregiver concern” and “watcher patient”, the chart reviewer indicated instances where “watcher patient” was missed as an opportunity to elevate a risk profile. For instance, as has been shown in the literature, PEWS scores are more accurate in capturing significant medical illness (particularly respiratory related) than surgical risk. Medical record reviews noted instances such as surgical risk, mental health risk, abnormal labs or neurovitals, where “watcher patient” would have been an appropriate and systematic means of elevating risk profile and guiding appropriate action. Thus, despite lower usefulness rankings, we believe that situational awareness factors highlight risk and create awareness beyond scoring, which has value as part of a safety system in the ED. Moving forward, we believe paying closer attention to context and further education of staff may improve uptake. Context and staff perceptions play crucial roles in PEWS implementation success as we and others have observed (25,26). In addition, as current research focuses on the inpatient setting, further investigation of the impact of situational awareness in the ED on care and patient outcomes will be important.

Communication within the team

Enhanced communication is a demonstrated factor in enhancing quality healthcare (16). Communication breakdown at handovers are a known point of risk that require consideration when planning an effective system (7). Because BC PEWS was implemented across the province in inpatient settings, our team chose to implement the same tools to promote a common language, understanding and process across sites and departments, and within the ED. Our findings support that this implementation strategy was effective in enhancing communication as our survey found positive influence on verbal communication and our chart review found significant gains in documentation. Further, staff reported increased comfort with escalation and communication of risk for senior review, which is an important characteristic of an optimal PEWS tool. As noted above, the staff attributed this gain to having objective tools to validate the need and process for escalation (25). While we will be taking feedback from this research to revise an assessment flowsheet for ED, we found value in a consistent scoring tool to promote internal and external communication and will retain the Brighton (15) tool in scale up.

PEWS, knowledge and confidence through standardization

As described by Oldroyd and Day, there is a lack of exposure to seriously ill children among ED nurses in general hospitals (8). In all areas of practice, low practice volumes present challenges with maintenance of competency, thus guidance provided by P can assist with identification of serious illness (8). Our study showed that nurses reported improvements in knowledge and confidence post-PEWS implementation and were completing more comprehensive assessment of vital signs that improved their ability to trend the patient stay. Physician respondents indicated similar benefits in knowledge and confidence and site leaders expressed an overall perception that the healthcare team showed greater awareness of risk or abnormality in the pediatric population. Importantly, improvement in staff knowledge and confidence for providing provision of pediatric care has been noted to be one of the key outcomes of PEWS implementation, which unlike patient outcomes is robustly measurable (27).

Given the decrease in variability of assessment practices using BC PEWS, we conclude that the implementation of PEWS influenced the equalization of patient care between providers. Additionally, having abnormal reference ranges visually evident in the PEWS system decreased knowledge deficit and cognitive load that may be present for staff who are less familiar or experienced with the pediatric population.

A minority of survey respondents indicated a perception that systems such as PEWS take away from autonomy of experienced nurses. While overall findings of the study support positive associations with the use of such standardized systems, this finding suggests a need to acknowledge and reinforce the role of such systems in supporting clinical judgement rather than replacing or usurping it (26,28).

The overall value of PEWS for ED

Bonafide et al. demonstrated that beyond the marginal performance of PEWS when applied to data sets, clinicians who recently experienced PEWS score failures (false positives) still considered it valuable (29). In a survey of 254 general ED practitioners in the UK, Griffiths and Kidney found that the majority support the use of early warning systems in the ED, despite the evidence that such scores low in sensitivity (30). As with other studies, our study found limitations with PEWS in the ED; however these limitations did not appear to alter the overall value assigned to PEWS by staff.

Roland et al. outlined that early warning scores in ED should support earlier diagnosis, more accurate estimation of illness severity and improved communication (7). Our participants reported that BC PEWS fulfilled all of these functions.

An effective clinical tool is one that practitioners use successfully, and one they want to use (25,26). Notably, the hospital team continued to use BC PEWS after the conclusion of the study and have provided ongoing support to the re-design of the system and scale up for the province.

Informing scale-up

This study demonstrated that PEWS is useful and beneficial to paediatric patient care when used with clinical judgment and alongside the currently accepted tools (CTAS) in the emergency department.

Based on these results and the growing body of evidence in the literature, provincial stakeholders made the unanimous decision to move to provincial implementation of PEWS in all ED settings in British Columbia. This pilot study’s results were used to redesign the 5-component BC PEWS specifically to address the needs of ED (BC PEWS ED). This involved the creation of a provincial emergency nursing assessment record (short and long form versions) with a compatible PEWS scoring sheet for recording vital signs, the creation of a provincial escalation guide for the ED, which corresponds to PEWS scores. Training materials were developed for ED staff including online training modules, in-person workshops, and a series of quick educational sessions that can be offered as refreshers. Health Authorities from across the province reviewed and accepted the new system to ensure buy-in and regional support. Beginning in February 2018, the province launched a phased implementation of BC PEWS ED, which included the full range of EDs serving children from the generalist rural/remote facilities to the provincial subspecialty pediatric facility. The implementation is being monitored through quarterly audits and planning is underway for evaluation of the scale up of implementation.

Study limitations

The study has a number of limitations. First, there were no accurate measures of deterioration available when identifying the dataset for review so proxy measures of deterioration or severity (i.e. transfer to higher level of care, admission, triage score) were used to select charts for review. However, we believe that the selected proxy measures captured the population of interest. Second, response rate for the health provider surveys was limited to 55%, even after sending out reminders in an attempt to increase the rate. However, our response rate is high compared to a study looking at response rates to online surveys among Canadian physician specialists(31). Nevertheless, a non-response bias cannot be ruled out. The study did not account for an acclimatization phase in the implementation and did not look at impact on patient outcomes, matters that will be addressed during the evaluation of the provincial scale-up.

In this study, standardization of pediatric ED care through PEWS demonstrated several benefits: PEWS significantly improved comprehensiveness of assessment and documentation at triage and throughout the ED stay. Accordingly, there was a perception that identification and awareness of risk rose and equalized between practitioners. BC PEWS upgraded provider perceived knowledge and confidence related to pediatric care, and increased timely and effective communication between practitioners. Scoring and the escalation aid made it objectively clear when senior review was required, helping to remove ambiguity and variance. Consequently, we believe that BC PEWS contributed to addressing the issues of cognitive load seen in typical ED settings, particularly amongst practitioners for whom pediatrics is a small part of a heavy caseload. Finally, all components of the system were perceived and reported as useful, and their use continued and evolved beyond the end of the study period, most importantly in planning for provincial scale up.

CTAS; Canadian Triage and Acuity Scale

ED; Emergency Department

PEWS; Pediatric Early Warning System

BC; British Columbia

SBAR; Situation, Background, Assessment, Recommendation

RN; Registered Nurse

Ethics approval and consent to participate: Ethics approval was obtained through the Children's and Women's Research Ethics Board; reference # H15-02236-A002. Informed consent was obtained for online surveys and key informant interviews.

Consent for publication: Not applicable.

Availability of data and material: Any datasets not included in this published article are available from the corresponding author on reasonable request.

Competing interests: The authors declare that they have no competing interests.

Funding: This study was funded through Child Health BC, a provincial network of health partners focused on children’s health. Child Health BC is accountable to the Provincial Health Services Authority of British Columbia.

Authors' contributions

TM co-designed the study, prepared ethics application, coordinated roll out of the PEWS system within the Health Authority, coordinated data collection and cleaning, analysed and interpreted data from all sources and was a major contributor in writing the manuscript.
ES co-designed the study, reviewed results, provided input on analysis and interpretations and reviewed and revised the manuscript.
KH contributed to data analysis and in writing the manuscript.
SW performed analysis on chart review data and was a major contributor in writing the manuscript.
YT coordinated the provincial roll out of the PEWS system in British Columbia, reviewed results, provided input on analysis and interpretations and reviewed and revised the manuscript.
CM led the roll out of the intervention for nursing, monitored and addressed implementation fidelity at the site and reviewed and revised the manuscript.
RC led the roll out of the intervention for physicians, reviewed and revised manuscript
DW was a principal investigator, co-designed the study, reviewed results, provided input on analysis and interpretations and reviewed and revised the manuscript.
MO was a principal investigator, co-designed the study, reviewed results, provided input on analysis and interpretations and reviewed and revised the manuscript.
All authors read and approved the final manuscript.

Acknowledgements

The authors would like to acknowledge Suud Nahdi for his support in the development of the research plan; Melissa Sallows and Meaghan Unger for their support of the medical record review; Jennifer McConnell-Nzunga for her help with qualitative data analysis, Michelle Fryer, Nicole Cave and Shannon Moffat for their work in the development and support for implementation of PEWS throughout British Columbia; Terry Chau for his help in development of the manuscript’s visuals; BC Children’s Hospital Learning & Development for their support in the design of BC PEWS, the BC Children’s Hospital Foundation who provides support for Child Health BC and the entire Richmond Hospital Emergency Department staff for their willingness to support innovation and research in order to improve emergency care for children in the province.

Adshead N, Thomson R. Use of a paediatric early warning system in emergency departments. Emerg Nurse [Internet]. 2009 Apr 2 [cited 2019 May 17];17(1):22–5. Available from: http://rcnpublishing.com/doi/abs/10.7748/en2009.04.17.1.22.c6984
Brady PW, Muething S, Kotagal U, Ashby M, Gallagher R, Hall D, et al. Improving situation awareness to reduce unrecognized clinical deterioration and serious safety events. Pediatrics [Internet]. 2013 Jan 1 [cited 2019 May 17];131(1):e298-308. Available from: http://pediatrics.aappublications.org/cgi/doi/10.1542/peds.2012-1364
Institute of Medicine (US) Committee on Quality of Health Care in America. To Err Is Human [Internet]. Linda T. Kohn, Janet M. Corrigan and MSD, editor. Washington, D.C.: National Academies Press; 2000 [cited 2019 May 17]. Available from: http://www.nap.edu/catalog/9728
Lambert V, Matthews A, MacDonell R, Fitzsimons J. Paediatric early warning systems for detecting and responding to clinical deterioration in children: a systematic review. BMJ Open [Internet]. 2017 Mar 13 [cited 2019 May 17];7(3):e014497. Available from: http://bmjopen.bmj.com/lookup/doi/10.1136/bmjopen-2016-014497
Corfield AR, Silcock D, Clerihew L, Kelly P, Stewart E, Staines H, et al. Paediatric early warning scores are predictors of adverse outcome in the pre-hospital setting: A national cohort study. Resuscitation [Internet]. 2018 Dec [cited 2019 May 17];133:153–9. Available from: https://linkinghub.elsevier.com/retrieve/pii/S030095721830995X
Agulnik A, Mora Robles LN, Forbes PW, Soberanis Vasquez DJ, Mack R, Antillon-Klussmann F, et al. Improved outcomes after successful implementation of a pediatric early warning system (PEWS) in a resource-limited pediatric oncology hospital. Cancer [Internet]. 2017 Aug 1 [cited 2019 May 17];123(15):2965–74. Available from: http://www.ncbi.nlm.nih.gov/pubmed/28440868
Roland D, McCaffery K, Davies F. Scoring systems in paediatric emergency care: Panacea or paper exercise? J Paediatr Child Health [Internet]. 2016 Feb [cited 2019 May 17];52(2):181–6. Available from: http://www.ncbi.nlm.nih.gov/pubmed/27062621
Oldroyd C, Day A. The Use of Pediatric Early Warning Scores in the Emergency Department. J Emerg Nurs [Internet]. 2011 Jul [cited 2019 May 17];37(4):374–6. Available from: http://www.ncbi.nlm.nih.gov/pubmed/21600639
O’Leary F, Hayen A, Lockie F, Peat J. Defining normal ranges and centiles for heart and respiratory rates in infants and children: a cross-sectional study of patients attending an Australian tertiary hospital paediatric emergency department. Arch Dis Child [Internet]. 2015 Aug [cited 2019 May 17];100(8):733–7. Available from: http://adc.bmj.com/lookup/doi/10.1136/archdischild-2014-307401
Gold DL, Mihalov LK, Cohen DM. Evaluating the Pediatric Early Warning Score (PEWS) system for admitted patients in the pediatric emergency department. Walthall J, editor. Acad Emerg Med [Internet]. 2014 Nov 6 [cited 2019 May 17];21(11):1249–56. Available from: https://onlinelibrary.wiley.com/doi/abs/10.1111/acem.12514
Lillitos PJ, Hadley G, Maconochie I. Can paediatric early warning scores (PEWS) be used to guide the need for hospital admission and predict significant illness in children presenting to the emergency department? An assessment of PEWS diagnostic accuracy using sensitivity and specificity. Emerg Med J [Internet]. 2016 May 1 [cited 2019 May 17];33(5):329–37. Available from: http://www.ncbi.nlm.nih.gov/pubmed/26531861
Chaiyakulsil C, Pandee U. Validation of pediatric early warning score in pediatric emergency department. Pediatr Int [Internet]. 2015 Aug [cited 2019 May 17];57(4):694–8. Available from: http://doi.wiley.com/10.1111/ped.12595
Seiger N, Maconochie I, Oostenbrink R, Moll HA. Validity of Different Pediatric Early Warning Scores in the Emergency Department. Pediatrics [Internet]. 2013 Oct 1 [cited 2019 May 17];132(4):e841–50. Available from: http://www.ncbi.nlm.nih.gov/pubmed/24019413
Bullard MJ, Musgrave E, Warren D, Unger B, Skeldon T, Grierson R, et al. Revisions to the Canadian Emergency Department Triage and Acuity Scale (CTAS) Guidelines 2016. CJEM [Internet]. 2017 Jul 31 [cited 2019 May 17];19(S2):S18–27. Available from: https://www.cambridge.org/core/product/identifier/S1481803517003657/type/journal_article
Monaghan A. Detecting and managing deterioration in children. Paediatr Care [Internet]. 2005 Feb [cited 2019 May 17];17(1):32–5. Available from: http://rcnpublishing.com/doi/abs/10.7748/paed2005.02.17.1.32.c964
Leonard M, Graham S, Bonacum D. The human factor: the critical importance of effective teamwork and communication in providing safe care. 2004 Oct 1 [cited 2019 May 17];13. Available from: http://www.ncbi.nlm.nih.gov/pubmed/15465961
Fleming S, Thompson M, Stevens R, Heneghan C, Plüddemann A, Maconochie I, et al. Normal ranges of heart rate and respiratory rate in children from birth to 18 years of age: a systematic review of observational studies. Lancet [Internet]. 2011 Mar 19 [cited 2019 May 17];377(9770):1011–8. Available from: http://www.ncbi.nlm.nih.gov/pubmed/21411136
Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, et al. Framework for design and evaluation of complex interventions to improve health. BMJ [Internet]. 2000 Sep 16 [cited 2019 May 17];321(7262):694–6. Available from: http://www.ncbi.nlm.nih.gov/pubmed/10987780
Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform [Internet]. 2009 Apr 1 [cited 2019 May 17];42(2):377–81. Available from: https://www.sciencedirect.com/science/article/pii/S1532046408001226
RStudio Team. RStudio: Integrated Development for R [Internet]. Boston MA; 2016. Available from: www.rstudio.com
Green J, Thorogood N. Qualitative methods for health research. 4th ed. Sage Publishing; 2018. 420 p.
Gravel J, Opatrny L, Gouin S. High rate of missing vital signs data at triage in a paediatric emergency department. Paediatr Child Health [Internet]. 2006 Apr [cited 2019 May 17];11(4):211–5. Available from: https://academic.oup.com/pch/article-lookup/doi/10.1093/pch/11.4.211
Gorelick MH, Alpern ER, Singh T, Snowdon D, Holubkov R, Dean JM, et al. Availability of Pediatric Emergency Visit Data from Existing Data Sources. Acad Emerg Med [Internet]. 2005 Nov 10 [cited 2019 May 17];12(12):1195–200. Available from: http://www.ncbi.nlm.nih.gov/pubmed/16282514
Porter SC, Manzi SF, Volpe D, Stack AM. Getting the data right: information accuracy in pediatric emergency medicine. Qual Saf Health Care [Internet]. 2006 Aug 1 [cited 2019 May 17];15(4):296–301. Available from: http://qualitysafety.bmj.com/lookup/doi/10.1136/qshc.2005.017442
Jensen CS, Nielsen PB, Olesen HV, Kirkegaard H, Aagaard H. Pediatric Early Warning Score Systems, Nurses Perspective – A Focus Group Study. J Pediatr Nurs [Internet]. 2018 Jul 14 [cited 2019 May 17];41:e16–22. Available from: http://www.ncbi.nlm.nih.gov/pubmed/29454507
Almblad A-C, Siltberg P, Engvall G, Målqvist M. Implementation of Pediatric Early Warning Score; Adherence to Guidelines and Influence of Context. J Pediatr Nurs [Internet]. 2018 Jan [cited 2019 May 17];38:33–9. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0882596317300337
McBride DL. Do Pediatric Early Warning Systems for Hospitalized Children Reduce Mortality? J Pediatr Nurs [Internet]. 2018 Nov [cited 2019 May 17];43:132–3. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0882596318301842
Jensen CS, Olesen HV, Aagaard H, Svendsen MLO, Kirkegaard H. Comparison of Two Pediatric Early Warning Systems: A Randomized Trial. J Pediatr Nurs [Internet]. 2019 Jan [cited 2019 May 17];44:e58–65. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0882596318303610
Bonafide CP, Roberts KE, Weirich CM, Paciotti B, Tibbetts KM, Keren R, et al. Beyond statistical prediction: qualitative evaluation of the mechanisms by which pediatric early warning scores impact patient safety. J Hosp Med [Internet]. 2013 May [cited 2019 May 17];8(5):248–53. Available from: http://www.journalofhospitalmedicine.com/jhospmed/article/127141/early-warning-score-qualitative-study
Griffiths JR, Kidney EM. Current use of early warning scores in UK emergency departments. Emerg Med J [Internet]. 2012 Jan [cited 2019 May 17];29(1):65–6. Available from: http://emj.bmj.com/lookup/doi/10.1136/emermed-2011-200508
Cunningham CT, Quan H, Hemmelgarn B, Noseworthy T, Beck CA, Dixon E, et al. Exploring physician specialist response rates to web-based surveys. BMC Med Res Methodol [Internet]. 2015 Apr 9 [cited 2019 May 17];15(1):32. Available from: https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-015-0016-z

Download PDF

Journal Publication

published 27 Nov, 2019

Read the published version in BMC Emergency Medicine →

Editorial decision: Minor revision
25 Sep, 2019
Review #1 received at journal
22 Aug, 2019
Reviewer #1 agreed at journal
01 Aug, 2019
Editor assigned by journal
29 Jul, 2019
Reviewers invited by journal
29 Jul, 2019
Submission checks completed at journal
25 Jul, 2019
Editor invited by journal
25 Jul, 2019
First submitted to journal
19 Jul, 2019

You are reading this older preprint version

Read the latest preprint version →

Implementation Study of a 5-component Pediatric Early Warning System (PEWS) in an Emergency Department in British Columbia, Canada, to inform provincial scale up

Status:

Journal Publication

Version 1

Abstract

Figures

Background

Methods

Results

Discussion

Conclusion

Abbreviations

Declarations

References

Status:

Journal Publication

Version 1