This was a single-centre cross-sectional observational study on the performance of the 3-level HK3TS on actual patients in the 24-hour Outpatient & Emergency Department of Gleneagles Hong Kong Hospital (GHK ED) from 1st May to 1st June, 2019. The study was approved by the Research Ethics Committee of GHK (CREC_2019-02). Informed written consent was obtained from both the patients and staff participants.
Setting and population
GHK ED is a private tertiary ED affiliated with The University of Hong Kong (HKU) Health System. It has commenced its operation since March 2017 with 24 hours service started since December 2017. It offers full spectrum of emergency care to patients of all ages. It is staffed 24 hours a day by emergency medicine specialists, resident doctors, registered and enrolled ED nurses. It has around-the-clock access to laboratory services, imaging studies, consultation service by specialists of other disciplines, in-patient beds, and the intensive care unit (ICU). At the time of the study, the annual census was 24,000. Although the GHK ED does not receive patients directly from ambulances, patients with time-critical emergencies, such as acute myocardial infarction, present to GHK ED by own transportation from time to time.
The triage in this ED is performed by ED nurses after patient registration and infection control screen at reception. The duty triage nurse assesses patients in a designated triage room, enters information into the hospital electronic health record system, and assigns a triage category based on global assessment of the patient’s chief complaint and vital signs. The triage scale consists of 3 levels: Category 1 (Immediate), Category 2 (Urgent), and Category 3 (Non-urgent). All ED nurses who undertook triage duty in this study had received in-service training on the use of HK3TS. Some of them had previous work experience with the HKAETG 5-level system in public EDs. After triage, patients are directed to the different care areas, such as cubicle beds or the waiting hall, according to the triage category. For patients who require life-saving interventions (Category 1), they are directed to the resuscitation room straight away with emergency medicine specialists summoned immediately.
During the 4-week study period, a convenient sample of GHK ED patients were invited to participate in the study at triage. Patient recruitment was based on the availability of the adjudicator and study nurse, but not on the age or characteristics of the patients. All GHK ED nurses were invited to participate. After obtaining written informed consent from the patients and staff, patients went through triage by the duty triage nurse as usual in the presence of a nurse adjudicator and another study nurse. The adjudicator was a nurse manager who had more than 20 years of clinical experience in emergency medicine. He refined the HK3TS and provided training to ED nurses in our department. His triage rating was regarded as the “criterion standard”. The study nurse was an ED nurse randomly selected from the rest of the team of the same shift. Both of them were refrained from asking questions, giving any clues or hints, or providing any suggestions to the duty triage nurse when they assessed the patient simultaneously. The duty triage nurse entered the patient data and triage rating into the hospital electronic health record system as usual while the nurse adjudicator and study nurse entered their triage ratings in the study data collection forms, which were collected immediately after triage. All of them were blinded to each other’s triage ratings.
Measurements
As for the validity, it refers to the degree with which the measured acuity level reflects the patient’s true urgency of care needs at the time of triage.19 No gold standard exists for the evaluation of validity of triage systems. Predictive validity is the most frequently used method.1, 6, 19 We assessed the validity of HK3TS by multiple methods. First, the predictive validity was evaluated by studying the proportion of patients requiring hospitalization, transfer to public EDs, referral to other private hospital for admission, ICU admission, and who died in the episode at different triage levels, which are surrogate outcome markers of patient acuity. However, these outcomes may be confounded by factors after the time of triage assessment. 1, 19 Therefore, we also measured the correlation between the triage level and the number of ED interventions required. Each of the imaging studies, laboratory test orders, consultations, and procedures carried out in the ED was equally weighted as one.14 Furthermore, the ability of HK3TS in identifying patients who required earlier medical attention, i.e. Category 1 and 2 cases based on the “criterion standard” of the adjudicator, was determined by the measure of sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).
The reliability of HKTS was assessed by comparing the triage ratings of the duty triage nurse and those by the adjudicator (criterion standard) and study nurse using kappa statistics. The over-triage rate was measured by the dividing the number of patients being triaged of a higher level than that given by the adjudicator by the total number of patients recruited. Similarly, the under-triage rate was determined by dividing the number patients being triaged of a lower level than that given by the adjudicator by the total number of patients recruited.
We collected the patient demographic data and data on chief complaints, progress, and outcome using a standardized data collection form. All patient participants were assigned a study code after obtaining informed consent and data were analysed anonymously.
Statistics
Missing values were not imputed. Patients with a missing triage rating by any participating nurses were excluded from analysis. We used descriptive statistics to analyse the distribution of characteristics of the study population and patient outcome. Categorical variables were reported as proportions, and continuous variables as mean +/- standard deviation or median with interquartile range (IQR), as appropriate. Chi square test, or Fisher’s Exact test, where appropriate, was used to compare the proportion of patient outcomes at different triage levels. Spearman correlation was used to assess the correlation between triage level and the number of ED interventions. The sensitivity, specificity, PPV and NPV of HK3TS in identifying patients requiring earlier medical attention were calculated with 95% confidence interval (CI) reported.
Reliability was reported as kappa with 95% CI. Unweighted kappa scores reflect exact agreement and treat all disagreement equally. Quadratic-weighted kappa takes into account of the level of disagreement and assigns partial credit to closer disagreement, yielding a higher value than unweighted kappa.20 It is noteworthy that disagreement by more than one triage level is less likely in 3-level triage system than in 5-level system. Yet, weighted kappa is reported in the majority of published triage studies.8 In this study, both unweighted and quadratic-weighted kappa were reported to facilitate benchmarking with other studies. We interpreted the strength of agreement for the kappa coefficient as: ≤0=poor, 0.01-0.2=slight, 0.21-0.40=fair, 0.41-0.6=moderate, 0.61-0.8=substantial, and 0.81-1=almost perfect, as proposed by Landis and Koch.21
We used R statistics (R Foundation for Statistical Computing, Vienna, Austria) to calculate the sample size based on the degree of agreement between the triage nurse and criterion standard. The value for the kappa coefficient to be solely due to chance is assumed to 0.3 (K0).22 According to the literature, the kappa coefficient of validity of the 5-level HKAETG triage system was 0.77.16 We had performed a pilot retrospective study on 100 randomly selected GHK ED patients, which showed a kappa coefficient of 0.76 in the agreement between the triage nurse and criterion standard. According to the patient case-mix in GHK ED, the proportions of Category 1, 2, and 3 cases were approximately 1%, 4% and 95%, respectively. With two raters, an alpha value of 0.05, and the lower bound of kappa at 0.5, the minimum sample size was 141.23 To account for a potential 10% loss of recruited cases due to missing values or lost to follow-up, the final sample size was determined as 155. The Statistical Package for the Social Sciences (SPSS) for Window version 26.0 (IBM Corp., Armonk, NY, USA) was used for data analysis. A two-tailed p value <0.05 was considered statistically significant.