Design and setting
We conducted a cross-sectional study in which we analysed real-life telephone triage recordings of nine OHS-PC locations in the vicinity of Utrecht, the Netherlands between 2014-2016. These OHS-PCs provide out-of-hours primary care for approximately 1,5 million people, handling 400,000 triage calls per year.
We evaluated patients with symptoms suggestive of TIA or stroke. The accuracy of NTS urgency allocation was assessed with the final clinical outcomes as the reference, that is, TIA, stroke and other (neurologic) life-threatening events (LTEs), e.g. intracranial haemorrhage. The triage recordings were selected in a two-step inclusion procedure, i.e. (i) selection based on the International Classification of Primary Care (ICPC) codes that are linked to the call and reflected our study domain (i.e. K89, K90, N17, N18, N19, N29, N89, N91), together with (ii) keywords in the OHS-PC electronic medical records suggesting TIA/stroke (e.g. neurological deficit, arm or leg weakness, face drooping, communication problem, visual problem, sensory disturbances and common synonyms).(35) A detailed description of the ICPC codes, medical keywords, inclusion and exclusion criteria has been published elsewhere.(36) We selected a random sample of 2,209 calls by using the Random Number Generator (RAND) function in Microsoft Excel. After a brief training and by means of a standardised case record form the triage calls were listened back and scored by 14 junior researchers. Two researchers from the study team (DCE and LTW) randomly checked one-third of all included calls. Patient and call characteristics, and assigned NTS urgencies were collected. From the patients’ own GPs we retrieved the final diagnosis, which was based on the discharge letter from the neurologist or the ED if the patient was referred for additional investigations. For patients who were not referred to the hospital we used follow-up data from the electronic medical records of GPs for up to one month to capture possible recurrence of TIA/stroke.
NTS urgency allocation in day-to-day practice
Telephone triage with the NTS starts with a mandatory ‘ABCD’ check (i.e. airway, breathing, circulation, disability). In case of direct life-threatening situations, an ambulance will be sent immediately.(37) If there is no life-threatening situation, the triage nurse continues by choosing one out of the 56 main complaints within the NTS. Every main complaint consists of an algorithm composed of hierarchically ordered questions. (18). One of these 56 main complaints is ‘neurological deficit’. After filling out the patient’s responses, the NTS will automatically generate an urgency level ranging from U0 to U5 which is linked to the response time within which a patient should receive medical help (see Table 1).(18, 38) The NTS urgency may be scaled up or down by the triage nurse, often after first consulting the supervising GP.(21) The reason for overruling should be registered, but this is not a mandatory step to complete the NTS triage process.
Difference between NTS urgency and final urgency
Besides the NTS urgency, which is automatically generated, we also evaluated the final urgency, which was defined as either the NTS urgency (if not changed) or the overruled NTS urgency.
In around 20% of all triage calls, the final urgency was unclear after re-listening the recordings in which it was evident that the triage nurse overruled the NTS urgency. This because the triage nurse did not notify the actual allocated urgency after overruling the NTS; e.g. the NTS urgency was U3, but in the audio recording the triage nurse tells the caller “I will sent an ambulance immediately” (U1)). Nevertheless, the urgency in the NTS system remained U3. A panel of three experienced GPs assessed calls in which the final urgency was unclear, blinded to the final diagnosis, and determined the final urgency (unanimously, or majority of votes after group discussion).
The patients were dichotomised into a high (U1 and U2) and low (U3, U4 and U5) urgency group, and differences in characteristics between these groups were compared. We calculated the accuracy in terms of sensitivity, specificity, positive and negative predictive values of (i) the NTS urgency allocation and (ii) the final urgency allocation (including overruled NTS urgencies), with the clinical outcomes TIA/stroke/LTEs as the reference. For the accuracy calculations we considered for TIA/minor stroke case the urgencies U1, U2 and U3 as adequate, and for major stroke and other LTEs the urgencies U1 and U2. Finally, we compared the baseline characteristics of patients in whom we could retrieve the final diagnosis with those in whom we could not, to assess potential selection bias. Statistical analyses were performed using SPSS version 25.0 (IBM Corp., Armonk, NY, USA).