To the authors’ knowledge, this study was the first study that evaluated the performance of a 3-level triage system in live ED triage encounters. In contrast to previous studies in the United States, our study showed that the HK3TS had an acceptable validity and reliability in a low-volume private ED setting.
Earlier study conducted by Wuerz at el. showed a poor interrater agreement of their 3-level triage system (kappa = 0.35). However, a more detailed review of their study revealed several insufficiencies: there was a lack of clinical descriptor of each triage category and only five scripted patient scenarios were assessed in their study, which did not include obvious emergency patients.12 Travers et al. compared the validity and reliability of the 3-level system with the 5-level ESI in a university level 1 trauma centre and found that the latter was more effective. However, their study was limited by the retrospective design and comparison of the triage systems at different times, which might be confounded by other time-dependent factors.13 Our findings are more consistent with the study conducted by Erimşah et al. on the Turkey’s Ministry of Health mandatory 3-level triage instrument, which was modified from the ATS and is similar to our 3-level triage scale.14
It is noteworthy that different research methods might affect the results of triage studies, and thus the conclusion drawn. To save costs and manpower, most triage studies, in particular those on 3-level triage systems, used paper scenarios or retrospective chart review.8, 11, 12 These methods do not capture the visual cues and complex interactions of factors encountered in live triage cases.1 Worster et al. demonstrated that interrater reliability of CTAS can be quite different in live triage assessments and in paper case scenarios.22 Considine et al. showed that the addition of visual clues in the form of still photographs delivered by computer resulted in a better interrator reliability in nurse triage using ATS.23 Studies which lack the important visual clues and dynamic interactions with the patients may underestimate the reliability of triage systems. Our study was conducted in real triage environment where not only cues (both visual and aural) were available to the raters, but the nurses were also under the pressure of time and stress. We believe this method is more reflective of real-time triage decision making and the results adds more weight to support 3-level triage.
In the literature, there is no agreed gold standard for the genuine degree of urgency against which the validity of a triage tool can be measured.1 When surrogate outcome markers were evaluated, a higher triage rating in the HK3TS was significantly associated with a higher proportion of patients requiring hospitalization and referral to other private hospitals for admission. A higher triage rating was also significantly associated with the number of ED interventions required. Since no patient required ICU admission or died in our cohort, we could not use ICU admission or mortality as clinical outcome measures in evaluating predictive validity. Although our triage system was not designed to predict patient ED outcome and the decision on admission may be affected by non-clinical factors, such as bed availability, insurance policy, and financial considerations on the part of the patients, these results support that our 3-level triage system has sufficient discriminative ability in identifying patients who require a higher intensity of care.
In this study, we used the clinical judgement of the adjudicator as the “criterion standard” in determining who required earlier medical attention. Using this approach, the sensitivity of the 3-level HKTS was found to be 68% and the under-triage rate 4.6%. In the literature, the sensitivity of 5-level triage systems in identifying patient requiring life-saving intervention ranges from 77%-98%.8 A lower sensitivity in our study can be explained by the difference in evaluation methods (subjective judgement of the adjudicator vs objective record of life-saving intervention). Also, private ED nurses have a higher turnover rate than their counterparts in public EDs and they have less ED working experience. Relying on global assessment, which requires knowledge and certain clinical experience, they might not be able to pick up subtle features that would suggest a higher disease acuity during the short patient encounter at triage.
Regarding the reliability, our study showed substantial agreement between the duty triage nurses and the criterion standard (quadratic-weighted kappa 0.76). This figure is higher than that reported by Travers et al for the 3-level system in the United States (weighted kappa = 0.53),13 but is comparable with that reported for the 3-level Ministry of Health of Turkey’s mandatory emergency triage instrument (weighted kappa = 0.73).14 The respective unweighted and weighted kappa values reported in the literature for the 5-level ATS, MTS, CTAS, ESI vary considerably and range from 0.43–0.84 23–27 and 0.62–0.99,13, 24–31 respectively. The interobserver agreement across nurses using the HK3TS were almost perfect (quadratic-weighted kappa 0.81). The respective unweighted kappa values for the 5-level ATS, MTS, CTAS, ESI were 0.40–0.76. 25, 32–34 The respective weighted kappa values of MTS, CTAS and ESI were 0.52–0.95 25, 32, 33, 35–37 respectively.
These findings indicate that the 3-level HKTS is reliable with a consistency comparable with the commonly used 5-level triage systems. Yet the relatively low sensitivity needed to be addressed. The accuracy of triage assessment depends on the triage nurse’s experience, information, and intuition in making the decision, and is inevitably a subjective process.38 Despite efforts, such as education, triage guidelines, triage algorithms, and audit to reduce the variability of triage assessment, there is little evidence that any of these strategies actually improves accuracy of triage.1 In a prospective study on real patients in an urban ED using CTAS, Grafstein et al demonstrated that a computerized triage menu that linked presenting complaints to preferred triage levels resulted in a high inter-rater reliability.39 In private EDs where computer systems are used in performing triage, computer-aid in decision making represents a new avenue for future research.40–42
Limitations
There were several limitations in this study. First, only a convenience sample was recruited, which might introduce sampling bias. We sought to minimize it by recruiting consecutive patients whenever the adjudicator and study nurses were available during the study period. We have no reason to believe that the patients who presented in their absence had significant different characteristics. Second, the adjudicator and the study nurses could only observe the triage interaction and they were not allowed to directly question the patients independently. This might affect the accuracy of their triage assessments. Nevertheless, observing a real triage process is much closer to reality than reading paper case scenarios or retrospective chart review, which lack the visual cues from live interaction. Third, there was no Category 1 case in our study. As in many other prospective studies, we had no control on intake of patients to our department. However, our findings were consistent with our pilot retrospective study, which purposely sampled around 10% of Category 1 cases. Forth, although the adjudicator was refrained from giving any verbal hints to the duty triage and study nurses, his presence of the adjudicator might lead to a Hawthorn effect during the triage process. Fifth. hospital admission in the private setting may be affected by non-clinical factors, such as insurance cover and bed availability. It might not be a good surrogate of the disease acuity. We sought to overcome this problem by looking into several other indicators. Finally, there was a single-centre study. The findings might not be generalizable to other EDs with different service volume and case-mix.
Despite these limitations, this study provides evidence to support the use of a simplified 3-level triage system in an ED with a relatively low patient volume. Future studies comparing its performance with the prevailing 5-level triage systems in live triage encounters with a multicentre design are warranted.