Accuracy of telephone triage in patients suspected of transient ischaemic attack or stroke: A cross-sectional study

doi:10.21203/rs.3.rs-27478/v2

Download PDF

Research article

Accuracy of telephone triage in patients suspected of transient ischaemic attack or stroke: A cross-sectional study

https://doi.org/10.21203/rs.3.rs-27478/v2

This work is licensed under a CC BY 4.0 License

Journal Publication

published 05 Dec, 2020

Read the published version in BMC Primary Care →

You are reading this older preprint version

Read the latest preprint version →

Background: The Netherlands Triage Standard (NTS) is a widely used decision support tool for telephone triage at Dutch out-of-hours primary care services (OHS-PC), which, however, has never been validated against clinical outcomes. We aimed to determine the accuracy of the NTS urgency allocation for patients with neurological symptoms suggestive of a transient ischaemic attack (TIA) or stroke, with the clinical outcomes TIA, stroke, and other (neurologic) life-threatening events (LTEs) as the reference.

Method: A cross-sectional study of telephone triage recordings of patients with neurological symptoms calling the OHS-PC between 2014 and 2016.The allocated NTS urgencies were derived from the electronic medical records of the OHS-PC. The clinical outcomes were retrieved from the electronic medical records of the patients’ own general practitioners. The accuracy of a high NTS urgency allocation (medical help within three hours) was calculated in terms of sensitivity, specificity, positive and negative predictive values (PPV and NPV) with the clinical outcomes TIA/stroke/other LTEs as the reference.

Results: Of 1,269 patients, 635 (50.0%) received the diagnosis TIA/stroke (34.2% TIA/minor stroke, 15.8% major ischaemic or haemorrhagic stroke), and 4.8% other LTEs. For TIA/stroke/other LTEs, the sensitivity and specificity of the NTS urgency allocation were 0.72 (95%CI 0.68-0.75) and 0.48 (95%CI 0.43-0.52), and the PPV and NPV were 0.62 (95%CI 0.60-0.64) and 0.58 (95%CI 0.54-0.62).

Conclusions: The NTS decision support tool used in Dutch OHS-PC performed poor to moderately regarding safety (sensitivity) and efficiency (specificity) in allocating adequate urgencies to patients with and without TIA/stroke/other LTEs.

General Practice

Neurology

the Netherlands National Trial Register

identification number NTR7331

Prompt recognition of patients with a transient ischaemic attack (TIA) or stroke is crucial for timely initiation of therapeutic interventions to minimise the risk of (permanent) brain injury and recurrent stroke.(1-6) Previous studies showed that urgent diagnostic assessment of TIA and minor stroke patients followed by a timely start of stroke preventive treatment resulted in a tremendous decrease of the early stroke risk (1, 5, 7) with a reduction of recurrent stroke up to 80% within three months.(1) However, the detection of TIA, and to a lesser extent stroke, may be challenging because multiple other diseases like migraine with aura, seizures or syncope can mimic TIA or stroke.(8-10) Moreover, symptoms may be non-specific in TIA or stroke, notably vertebrobasilar insufficiency, and in the case of TIA, symptoms are often short lasting and already resolved by the time a patient seeks medical help.(8, 11)

Patients with symptoms suggestive of TIA or stroke often contact the general practitioner (GP) first.(12-15) During evenings, nights and weekends such care is provided by the out-of-hours services in primary care (OHS-PC). At the OHS-PC, the initial contact is by telephone, and nurses perform triage while supervised by GPs.(16) The goal of telephone triage is to assess the severity of patients’ complaints and to link this to an adequate urgency allocation with corresponding response time to medical care. Telephone triage in the Netherlands is supported by a semi-automatic decision support tool called the ‘Netherlands Triage Standard’ (NTS). The NTS is a five-level triage tool, which was developed by an expert panel and derived from existing Dutch national telephone guidelines for primary care office hours, and the Manchester Triage System (MTS).(17, 18) Based on the annual incidence of 0.006% of serious adverse events (SAEs) in the Dutch OHS-PC setting, the NTS is considered to be safe (19). However, questions have been raised about the efficiency.(16) There was a clear increase in high urgency allocations since the implementation of the NTS in 2011 onwards, suggesting a low efficiency.(20) This was supported by the results of a national survey among GPs in 2016, showing that the vast majority believed telephone triage with the NTS resulted in unnecessary consultations and home visits.(16, 21)

Most previous studies assessed the overall accuracy of triage decision support tools in emergency department (ED) settings, and only a few studies did this in the OHS-PC.(22, 23) Few studies focused on specific domains of patients (e.g. chest pain), some of which included clinical outcomes as the reference (e.g. acute coronary syndrome), yet, only in ED settings.(24-31) Comparable accuracy studies in primary care settings are limited; one study that assessed the overall accuracy of a telephone triage tool in primary care used a ‘surrogate’ reference created by the researchers themselves (e.g. hospital referrals or costs).(18, 22, 32-34) The NTS urgency allocation, or the urgency allocation of other decision support tools for telephone triage in primary care settings, were never evaluated against the final clinical outcomes of patients as the reference.

We aimed to determine the accuracy of the NTS urgency allocation in patients calling the OHS-PC with symptoms suggestive of TIA or stroke, with presence or absence of the final clinical outcomes TIA, stroke and other (neurologic) life-threatening events (LTEs) as the reference.

Design and setting
We conducted a cross-sectional study in which we analysed real-life telephone triage recordings of nine OHS-PC locations in the vicinity of Utrecht, the Netherlands between 2014-2016. These OHS-PCs provide out-of-hours primary care for approximately 1,5 million people, handling 400,000 triage calls per year.

Data collection
We evaluated patients with symptoms suggestive of TIA or stroke. The accuracy of NTS urgency allocation was assessed with the final clinical outcomes as the reference, that is, TIA, stroke and other (neurologic) life-threatening events (LTEs), e.g. intracranial haemorrhage. The triage recordings were selected in a two-step inclusion procedure, i.e. (i) selection based on the International Classification of Primary Care (ICPC) codes that are linked to the call and reflected our study domain (i.e. K89, K90, N17, N18, N19, N29, N89, N91), together with (ii) keywords in the OHS-PC electronic medical records suggesting TIA/stroke (e.g. neurological deficit, arm or leg weakness, face drooping, communication problem, visual problem, sensory disturbances and common synonyms).(35) A detailed description of the ICPC codes, medical keywords, inclusion and exclusion criteria has been published elsewhere.(36) We selected a random sample of 2,209 calls by using the Random Number Generator (RAND) function in Microsoft Excel. After a brief training and by means of a standardised case record form the triage calls were listened back and scored by 14 junior researchers. Two researchers from the study team (DCE and LTW) randomly checked one-third of all included calls. Patient and call characteristics, and assigned NTS urgencies were collected. From the patients’ own GPs we retrieved the final diagnosis, which was based on the discharge letter from the neurologist or the ED if the patient was referred for additional investigations. For patients who were not referred to the hospital we used follow-up data from the electronic medical records of GPs for up to one month to capture possible recurrence of TIA/stroke.

NTS urgency allocation in day-to-day practice

Telephone triage with the NTS starts with a mandatory ‘ABCD’ check (i.e. airway, breathing, circulation, disability). In case of direct life-threatening situations, an ambulance will be sent immediately.(37) If there is no life-threatening situation, the triage nurse continues by choosing one out of the 56 main complaints within the NTS. Every main complaint consists of an algorithm composed of hierarchically ordered questions. (18). One of these 56 main complaints is ‘neurological deficit’. After filling out the patient’s responses, the NTS will automatically generate an urgency level ranging from U0 to U5 which is linked to the response time within which a patient should receive medical help (see Table 1).(18, 38) The NTS urgency may be scaled up or down by the triage nurse, often after first consulting the supervising GP.(21) The reason for overruling should be registered, but this is not a mandatory step to complete the NTS triage process.

Difference between NTS urgency and final urgency

Besides the NTS urgency, which is automatically generated, we also evaluated the final urgency, which was defined as either the NTS urgency (if not changed) or the overruled NTS urgency.

In around 20% of all triage calls, the final urgency was unclear after re-listening the recordings in which it was evident that the triage nurse overruled the NTS urgency. This because the triage nurse did not notify the actual allocated urgency after overruling the NTS; e.g. the NTS urgency was U3, but in the audio recording the triage nurse tells the caller “I will sent an ambulance immediately” (U1)). Nevertheless, the urgency in the NTS system remained U3. A panel of three experienced GPs assessed calls in which the final urgency was unclear, blinded to the final diagnosis, and determined the final urgency (unanimously, or majority of votes after group discussion).

Data analyses

The patients were dichotomised into a high (U1 and U2) and low (U3, U4 and U5) urgency group, and differences in characteristics between these groups were compared. We calculated the accuracy in terms of sensitivity, specificity, positive and negative predictive values of (i) the NTS urgency allocation and (ii) the final urgency allocation (including overruled NTS urgencies), with the clinical outcomes TIA/stroke/LTEs as the reference. For the accuracy calculations we considered for TIA/minor stroke case the urgencies U1, U2 and U3 as adequate, and for major stroke and other LTEs the urgencies U1 and U2. Finally, we compared the baseline characteristics of patients in whom we could retrieve the final diagnosis with those in whom we could not, to assess potential selection bias. Statistical analyses were performed using SPSS version 25.0 (IBM Corp., Armonk, NY, USA).

Group characteristics

We included 1,269 patients of whom a final diagnosis could be obtained (see Figure 1). The median age was 72.0 (IQR 57.0-83.0) years, and 56.9% were female. The NTS allocation of high (U1 and U2) and low (U3, U4 and U5) urgencies was equally distributed between men and women (see Table 2).

The characteristics of patients with a known final diagnosis were comparable with those for whom the GP did not provide the final diagnosis (see Supplementary data Table S1).

Compared to the low NTS urgency group, patients in the high NTS urgency group were older (73.5 vs. 69.0 years, p<0.001). Also, the call duration of patients in the high urgency group was shorter (06:32 min vs. 07:59 min, p<0.001), and more often someone else called on behalf of the patient (80.6% vs. 68.5%, p<0.001) in comparison to the low NTS urgency group. In nearly all calls concern about the symptoms was expressed (90.3% vs. 96.1%, p=0.006), and in the vast majority, symptoms were still present at the time of calling (93.4% vs. 89.9%, p=0.030). Patients classified as high urgent more often had face drooping (54.3% vs. 39.1%, p<0.001), arm weakness (51.0% vs. 28.1%, p<0.001), leg weakness (49.0% vs. 35%, p<0.001), and communication problems in general (80.8% vs. 72.5%, p=0.008), whereas patients classified as low urgent more often reported sweating (36.7% vs. 56.3%, p=0.006).

Final diagnoses

In 434 (34.2%) patients the final diagnosis was a TIA or minor stroke, and in 201 (15.8%) a major ischaemic or haemorrhagic stroke. Sixty-one (4.8%) patients had other LTEs, such as intracranial haemorrhage or meningitis. The remaining 573 patients (45.2%) were diagnosed with other neurological disorders (e.g. migraine, epilepsy) or other disorders (e.g. peripheral vestibular syndromes or psychogenic syndromes). See Table 3 for a complete overview of final diagnoses.

Final urgency allocation

Of all 1,269 patients, 770 (60.7%) received a high NTS urgency (U1 or U2) and 499 (39.3%) a low NTS urgency (U3, U4 or U5). In 728 (57.4%) patients the NTS urgency was equal to the final urgency. In the remaining 541 (42.6%) patients the NTS urgency was overruled, of which in 364 (67.3%) patients the NTS urgency was scaled up by the triage nurse, and in 177 (32.7%) patients it was scaled down (see Figure 2 and for details supplementary data Tables S2-S5).

Accuracy of the NTS urgency and TIA/stroke, or TIA/stroke/other LTEs as the reference

The sensitivity of the NTS for allocating a high urgency to patients with TIA/stroke was 0.71 (95% CI 0.68-0.75), and for patients with TIA/stroke/other LTEs 0.72 (0.68-0.75). The specificity was 0.46 (0.42-0.50) and 0.48 (0.43-0.52), respectively. The positive and negative predictive values were 0.41 (0.38-0.43) and 0.75 (0.72-0.78) for TIA/stroke, and 0.62 (0.60-0.64) and 0.58 (0.54-0.62) for TIA/stroke/other LTEs, respectively.

Accuracy of the final urgency (including overruling) and TIA/stroke, or TIA/stroke/other LTEs as the reference

The sensitivity of the final urgency allocation for allocating a high urgency to patients with TIA/stroke was 0.86 (0.84-0.89), and for TIA/stroke/other LTEs 0.86 (0.83-0.89). The specificity was 0.38 (0.34-0.42) and 0.40 (0.36-0.44), respectively. The positive and negative predictive values for TIA/stroke were 0.42 (0.40-0.44) and 0.84 (0.81-0.87), respectively, and for TIA/stroke/other LTEs 0.63 (0.62-0.65) and 0.70 (0.66-0.74), respectively. See also Table 4.

Summary

Of 1,269 patients suspected of TIA/stroke, 635 (50.0%) showed to have a TIA or stroke; 434 (34.2%) had a TIA or minor stroke, 201 (15.8%) a major ischaemic or haemorrhagic stroke. In addition, 61 (4.8%) patients had other (neurologic) LTEs. The urgency allocation of the NTS tool was poor to moderate regarding sensitivity and specificity with TIA/stroke/other LTEs as the reference. In 42.6% the NTS urgency was overruled by the triage nurse. The final urgency allocation (including overruled NTS urgencies) showed modestly improved sensitivity (safety) whereas the specificity remained equally poor (efficiency). The positive predictive value did not change after overruling of the NTS, but the negative predictive value increased. This suggests that overruling by the triage nurses leads to safer telephone triage without compromising efficiency (i.e. overlapping confidence intervals of the NTS and final urgencies’ specificities).

Strengths and limitations

This is the first study to report accuracy findings of the NTS tool for telephone triage at the OHS-PC with clinical outcomes as the reference. Because researchers were blinded to the final clinical outcome during data collection, the effect of hindsight bias was limited.

A limitation was missing data on the final clinical outcome (25% of all re-listened recordings). However, a detailed comparison in patient characteristics between those with a final outcome and those without showed that these groups were comparable (no indication of selection bias). Therefore, we believe our results are generalizable to similar OHS-PC settings.

Comparison with existing literature

As described previously, many studies assessed the accuracy of other triage systems (22), and some of these also used clinical outcomes as the reference.(24-31) One study assessed the Manchester Triage System (MTS) in the domain of patients suspected for neurological disease seen at the ED.(32) The accuracy of a high urgency allocation was calculated with neurological disease (not otherwise specified) as the reference; a c-statistic of 0.73 was reported. High MTS urgency allocation was significantly associated with neurological disease (odds ratio 3.0, 95%CI 2.4-3.8, p<0.001).(32) Unfortunately, sensitivity or specificity was not calculated. Comparison to our study is also hampered, because in the primary care setting the prevalence of emergent cerebrovascular events is lower, and on average includes less severe cases. This may be reflected in less evident clinical presentations.

In addition to the studies on the accuracy evaluating all ‘main complaints’ of the triage systems, a few other studies described and evaluated diagnostic prediction models specifically for TIA and/or stroke in daytime general practice, namely: (i) the Dawson score, (ii) the modified Explicit Diagnostic Criteria for TIA (EDCT), and (iii) the TIA/stroke electronic decision support tool.(39-41) The Dawson score performed rather good for diagnosing TIA when validated in UK general practice, with a c-statistic of 0.70 (95% CI 0.66-0.75). However, sensitivity and specificity were not reported.(39) The modified EDCT criteria performed very good in Dutch daytime general practice with TIA/minor stroke as the reference, with a c-statistic of 0.86 (95% CI 0.80-0.92), a sensitivity of 0.98 (0.94-0.99) and a specificity of 0.74 (0.63-0.83).(40) The accuracy of a TIA/ stroke electronic decision support tool in general practices in New Zealand was not reported, but the researchers reported that it did lead to improved triaging accuracy in the sense that it provided a widely applicable and cost-effective way of improving care and outcomes for patients with TIA/stroke.(41) Importantly, however, comparison of our results to the previous studies on diagnostic prediction models for TIA in daytime general practice is limited, because these studies included only patients with resolved symptoms, which is in contrast to our primary care population calling the OHS-PC; 90.9% of all patients had symptoms when calling.

In our study, we considered different urgency levels as adequate; for TIA/minor stroke U1-U3, and for major stroke/other LTEs U1-U2. The rationale for high urgency allocations in suspected stroke patients is mainly because of available treatment options, and not because TIA/stroke may result in ABCD instability (i.e. airway, breathing, circulation, disability). Assigning high urgency levels to patients with acute stroke enables early initiation of (invasive) prognostically beneficial treatment.(43-45) In patients with TIA/minor stroke early initiation of antiplatelets for secondary stroke prevention is key, given the substantial risk of major stroke in the first hours to days after a TIA (5, 6, 46). Current treatment guidelines on TIA/stroke recommend that patients suspected of TIA should be seen within 24 hours after symptom onset at a TIA outpatient clinic for a neurological assessment, while secondary stroke prevention should be started as soon as possible after a confirmed diagnosis of TIA/minor stroke (47, 48) or directly if the patient cannot be assessed by a neurologist the same day.(49) Therefore, we considered U3 (patient seen within three hours) as sufficient in patients who finally showed to have had a TIA/minor stroke.

Implications for research and/or practice

Our study indicated that the accuracy of the NTS was poor to moderate, yet safety improved after overruling by the triage nurse. Apparently, triage nurses and/or their GP supervisors capture some vital patient information that is not yet incorporated in the NTS. Further improvement of safety, as well as improving efficiency of telephone triage in the domain of patients calling with neurological symptoms is necessary. Improving the accuracy of already existing triage systems such as the NTS should be the first step. In order to do so, prediction models are needed based on multivariable analyses to provide an evidence-based basis for which triage questions are helpful, and which are not.

The NTS decision support tool used in Dutch OHS-PC performed poor to moderately regarding safety (sensitivity) and efficiency (specificity) in allocating adequate urgencies to patients with and without TIA/stroke/other LTEs. There are indications that overruling the NTS by triage nurses improves safety, without compromising efficiency.

Ethics approval and consent to participate

The Medical Ethics Review Committee Utrecht, the Netherlands approved this study (National Trial Register identification number: NTR7331, reference number WAG/mb/16/003208). In addition, a waiver of informed consent was granted as our study involved minimal risk to subjects and this study would not have been practicable without the waiver. All personal and research data were handled and stored according to the European General Data Protection Regulation.

Consent for publication

Not applicable as all personal and research data were made unidentifiable

Availability of data and materials

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

Competing interests

The authors declare that they have no competing interests.

Funding

This work was supported by (i) the department of general practice of the University Medical Center Utrecht, (ii) Associate Professorship-promotion grant of D.L. Zwart, MD, PhD, (iii) the foundation ‘Netherlands Triage Standard’ and (iv) the ‘Stoffels-Hornstra’ foundation. The views expressed are those of the authors and not necessarily those of the foundations. This research was conducted without direct involvement from both funding foundations.

Authors' Contributions

DLZ and FHR conceived the idea for the study and gained funding. All authors designed the study. DCE and LTW collected the data, and DCE analysed the data. DCE prepared the manuscript and wrote the first draft, supervised by DLZ. All authors provided intellectual input, critically reviewed the manuscript and read and approved the final manuscript.

Acknowledgements

The authors thank the OHS-PC foundation ‘Primair Huisartsenposten’ and all employees of the participating locations for their cooperation in this study, notably for providing data and technical support.

GP: General Practitioner; LTE: Life-Threatening Events; NTS: Netherlands Triage Standard; OHS-PC: Out-Of-Hours Services in Primary Care; TIA: Transient Ischaemic Attack.

Rothwell PM, Giles MF, Chandratheva A, et al. Effect of urgent treatment of transient ischaemic attack and minor stroke on early recurrent stroke (EXPRESS study): a prospective population-based sequential comparison. Lancet. 2007;370(9596):1432-42.
Kidwell CS, Warach S. Acute ischemic cerebrovascular syndrome: diagnostic criteria. Stroke. 2003;34(12):2995-8.
Coull AJ, Lovett JK, Rothwell PM, Oxford Vascular S. Population based study of early risk of stroke after transient ischaemic attack or minor stroke: implications for public education and organisation of services. BMJ. 2004;328(7435):326.
Johnston SC, Gress DR, Browner WS, Sidney S. Short-term prognosis after emergency department diagnosis of TIA. Jama. 2000;284(22):2901-6.
Rothwell PM, Algra A, Chen Z, et al. Effects of aspirin on risk and severity of early recurrent stroke after transient ischaemic attack and ischaemic stroke: time-course analysis of randomised trials. Lancet. 2016;388(10042):365-75.
Wu CM, McLaughlin K, Lorenzetti DL, et al. Early risk of stroke after transient ischemic attack: a systematic review and meta-analysis. Archives of internal medicine. 2007;167(22):2417-22.
Lavallee PC, Meseguer E, Abboud H, et al. A transient ischaemic attack clinic with round-the-clock access (SOS-TIA): feasibility and effects. Lancet Neurol. 2007;6(11):953-60.
Nadarajan V, Perry RJ, Johnson J, Werring DJ. Transient ischaemic attacks: mimics and chameleons. Pract Neurol. 2014;14(1):23-31.
Hand PJ, Kwan J, Lindley RI, et al. Distinguishing between stroke and mimic at the bedside: the brain attack study. Stroke. 2006;37(3):769-75.
Ay H, Buonanno FS, Rordorf G, et al. Normal diffusion-weighted MRI during stroke-like deficits. Neurology. 1999;52(9):1784-92.
Sheehan OC, Merwick A, Kelly LA, et al. Diagnostic usefulness of the ABCD2 score to distinguish transient ischemic attack and minor ischemic stroke from noncerebrovascular events: the North Dublin TIA Study. Stroke. 2009;40(11):3449-54.
Doggen CJ, Zwerink M, Droste HM, et al. Prehospital paths and hospital arrival time of patients with acute coronary syndrome or stroke, a prospective observational study. BMC Emerg Med. 2016;16:3.
Manawadu D, Shuaib A, Collas DM. Emergency department or general practitioner following transient ischaemic attack? A comparison of patient behaviour and speed of assessment in England and Canada. Emergency medicine journal : EMJ. 2010;27(5):364-7.
Giles MF, Flossman E, Rothwell PM. Patient behavior immediately after transient ischemic attack according to clinical characteristics, perception of the event, and predicted risk of stroke. Stroke. 2006;37(5):1254-60.
Lasserson DS, Chandratheva A, Giles MF, et al. Influence of general practice opening hours on delay in seeking medical attention after transient ischaemic attack (TIA) and minor stroke: prospective population based study. BMJ. 2008;337:a1569.
Smits M, Rutten M, Keizer E, et al. The Development and Performance of After-Hours Primary Care in the Netherlands: A Narrative Review. Annals of internal medicine. 2017;166(10):737-42.
National guidelines for telephone triage and advice in Family Practice [cited 2018 June 22]. Available from: https://www.nhg.org/winkel/producten/nhg-triagewijzer-versie-2016.
van Ierland Y, van Veen M, Huibers L, et al. Validity of telephone and physical triage in emergency care: the Netherlands Triage System. Family practice. 2011;28(3):334-41.
Rutten MH, Kant J, Giesen P. What can we learn from calamities at out-of-hours services in primary care? [Wat kunnen we leren van calamiteiten op de huisartsenpost?]. Huisarts Wet. 2018;6(61).
Jansen T, de Hoon S, Hek K, Verheij R. Developments at the out-of-hours services in primary care. Changes in care demand and health care issues in 2013-2015. [Ontwikkelingen op de huisartsenpost. Veranderingen in zorgvraag en gezondheidsproblemen in 2013-2015.]. NIVEL, 2017.
Keizer E, Maassen I, Smits M, et al. Reducing the use of out-of-hours primary care services: A survey among Dutch general practitioners. The European journal of general practice. 2016;22(3):189-95.
Kuriyama A, Urushidani S, Nakayama T. Five-level emergency triage systems: variation in assessment of validity. Emergency medicine journal : EMJ. 2017;34(11):703-10.
Zachariasse JM, van der Hagen V, Seiger N, et al. Performance of triage systems in emergency care: a systematic review and meta-analysis. BMJ open. 2019;9(5):e026471.
Nishi F, de Oliveira Motta Maia F, de Souza Santos I, de Almeida Lopes Monteiro da Cruz D. Assessing sensitivity and specificity of the Manchester Triage System in the evaluation of acute coronary syndrome in adult patients in emergency care: a systematic review. JBI Database of Systematic Reviews and Implementation Reports. 2017;15(6):1747-61.
Pinto D, Lunet N, Azevedo A. Sensitivity and specificity of the Manchester Triage System for patients with acute coronary syndrome. Rev Port Cardiol. 2010;29(6):961-87.
Leite L, Baptista R, Leitao J, et al. Chest pain in the emergency department: risk stratification with Manchester triage system and HEART score. BMC cardiovascular disorders. 2015;15:48.
Trigo J, Gago P, Mimoso J, et al. In-hospital delay in ST-segment-elevation myocardial infarction after Manchester Triage. Rev Port Cardiol. 2008;27(10):1251-9.
Matias C, Oliveira R, Duarte R, et al. The Manchester Triage System in acute coronary syndromes. Rev Port Cardiol. 2008;27(2):205-16.
Providencia R, Gomes PL, Barra S, et al. Importance of Manchester Triage in acute myocardial infarction: impact on prognosis. Emergency medicine journal : EMJ. 2011;28(3):212-6.
Gouvea VET, Reis MAM, Gouvea GM, et al. Evaluation the Manchester Triage System in the acute coronary syndrome. Int J Cardiovasc Sci. 2015;28(2):107-13.
Nishi FA, Polak C, Cruz D. Sensitivity and specificity of the Manchester Triage System in risk prioritization of patients with acute myocardial infarction who present with chest pain. Eur J Cardiovasc Nurs. 2018;17(7):660-6.
Steiner D, Renetseder F, Kutz A, et al. Performance of the Manchester Triage System in Adult Medical Emergency Patients: A Prospective Cohort Study. J Emerg Med. 2016;50(4):678-89.
Grouse AI, Bishop RO, Bannon AM. The Manchester Triage System provides good reliability in an Australian emergency department. Emergency medicine journal : EMJ. 2009;26(7):484-6.
Lake R, Georgiou A, Li J, et al. The quality, safety and governance of telephone triage and advice services - an overview of evidence from systematic reviews. BMC health services research. 2017;17(1):614.
The Dutch College of General Practitioners, NHG. The International Classification of Primary Care (ICPC). Version 6 (Dutch translation derived from ICPC-1 by the WONCA International Classification Committee). 2018. Available from: https://www.nhg.org/themas/artikelen/icpc.
Erkelens DC, Wouters LT, Zwart DL, et al. Optimisation of telephone triage of callers with symptoms suggestive of acute cardiovascular disease in out-of-hours primary care: observational design of the Safety First study. BMJ open. 2019;9(7):e027477.
Thim T, Krarup NH, Grove EL, et al. Initial assessment and treatment with the Airway, Breathing, Circulation, Disability, Exposure (ABCDE) approach. Int J Gen Med. 2012;5:117-21.
Netherlands Triage Standard [Nederlandse Triage Standaard], 2019. Accessed at www.de-nts.nl on 7 October 2019.
Lasserson DS, Mant D, Hobbs FD, Rothwell PM. Validation of a TIA recognition tool in primary and secondary care: implications for generalizability. Int J Stroke. 2015;10(5):692-6.
Dolmans LS, Lebedeva ER, Veluponnar D, et al. Diagnostic Accuracy of the Explicit Diagnostic Criteria for Transient Ischemic Attack: A Validation Study. Stroke. 2019;50(8):2080-5.
Ranta A, Dovey S, Weatherall M, et al. Cluster randomized controlled trial of TIA electronic decision support in primary care. Neurology. 2015;84(15):1545-51.
Dolmans LS, Kappelle LJ, Bartelink ME, et al. Delay in patients suspected of transient ischaemic attack: a cross-sectional study. BMJ open. 2019;9(2):e027161.
Prabhakaran S, Ruff I, Bernstein RA. Acute stroke intervention: a systematic review. Jama. 2015;313(14):1451-62.
Leng T, Xiong ZG. Treatment for ischemic stroke: From thrombolysis to thrombectomy and remaining challenges. Brain Circ. 2019;5(1):8-11.
Powers WJ, Rabinstein AA, Ackerson T, et al. Guidelines for the Early Management of Patients With Acute Ischemic Stroke: 2019 Update to the 2018 Guidelines for the Early Management of Acute Ischemic Stroke: A Guideline for Healthcare Professionals From the American Heart Association/American Stroke Association. Stroke. 2019;50(12):e344-e418.
Hankey GJ. Secondary stroke prevention. Lancet Neurol. 2014;13(2):178-94.
Kernan WN, Ovbiagele B, Black HR, et al. Guidelines for the prevention of stroke in patients with stroke and transient ischemic attack: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. 2014;45(7):2160-236.
Stroke and transient ischaemic attack in over 16s: diagnosis and initial management. National Institute for Health and Care Excellence: Clinical Guidelines. London2019.
Verburg AF, Tjon ATMR, Verstappen WH, et al. Summary of the ‘Stroke’ guideline of the Dutch College of General Practitioners [Samenvatting van de NHG-Standaard Beroerte]. Ned Tijdschr Geneeskd. 2014;158(1):A7022.

Table 1. NTS levels of urgency

NTS Urgency level	Definition	Response time	Medical help
U0 – Resuscitation	Loss of vital functions	Immediately	Ambulance
U1 – Life threatening	Unstable vital functions	Within 15 minutes	Ambulance
U2 – Emergent	Vital functions in danger or organ damage	As soon as possible, within 1 hour	Home visit by GP or appointment at OHS-PC
U3 – Urgent	Possible risk of damage, human reasons	A few hours (<3 hours)	Home visit by GP or appointment at OHS-PC
U4 – Non-urgent	Marginal risk of damage	24 hours	Appointment at OHS-PC or telephone advice
U5 – Advice	No risk of damage	Advice, no time related	Telephone advice
GP: General Practitioner; NTS: Netherlands Triage Standard; OHS-PC: Out-Of-Hours Services in Primary Care

Table 2. Characteristics of 1,269 patients with symptoms suggestive of TIA or stroke calling the OHS-PC

	High NTS urgency n=770 (60.7%)	Low NTS urgency n=499 (39.3%)	P-value^#
Patient characteristics:
Median age in years (IQR)	73.5 (59.0-84.0)	69.0 (55.0-82.0)	<0.001
Female sex	441 (57.3)	281 (56.3)	0.736
Family history of CVD (n=36) Medical history:	14 (82.4)	13 (68.4)	0.451^&
Cardiovascular disease (n=882)	421 (79.1)	259 (74.0)	0.076
TIA (n=637)	116 (29.7)	60 (24.4)	0.147
Stroke (n=637)	113 (28.9)	60 (24.4)	0.213
Coronary artery disease (n=303)	36 (20.6)	18 (14.1)	0.144
Arrhythmia (n=292)	32 (19.5)	25 (19.5)	0.997
Valvular heart disease (n=260)	16 (10.9)	9 (8.0)	0.429
Heart failure (n=263)	14 (9.5)	10 (8.7)	0.831
Epilepsy (n=233)	15 (11.6)	9 (8.7)	0.458
Migraine (n=102)	17 (27.0)	14 (35.9)	0.342
Cardiovascular risk factors:	14 (9.5)	10 (8.7)	0.831
Hypertension (n=421)	121 (50.2)	85 (47.2)	0.544
Hypercholesterolemia or use of statins (n=395)	95 (43.0)	75 (43.1)	0.981
Diabetes mellitus (n=417)	87 (36.7)	63 (35.0)	0.719
Cardiovascular medication: Antithrombotics (n=939)	290 (48.7)	149 (43.4)	0.123
Other cardiovascular medication (n=764)	253 (57.55)	178 (54.9)	0.480
Call characteristics:
Median call duration in min:sec (IQR)	06:32 (04:43-08:54)	07:59 (05:54-10:50)	<0.001
Median time for caller’s introduction in min:sec (IQR)	00:19 (00:12-00:27)	00:20 (00:13-00:29)	0.189
Initial call by someone else than the patient	621 (80.6)	342 (68.5)	<0.001
Triage nurse consulted the general practitioner	449 (58.3)	305 (61.1)	0.319
Main NTS complaint chosen by triage nurse
Neurological deficit	587 (76.2)	220 (44.1)	<0.001
Dizziness	21 (2.7)	87 (17.4)	<0.001
Headache	23 (3.0)	28 (5.6)	0.020
Odd behavior	21 (2.7)	18 (3.6)	0.375
Syncope	20 (2.6)	9 (1.8)	0.355
Vision problem	2 (0.3)	25 (5.0)	<0.001^&
Leg or arm problem	11 (1.4)	34 (6.8)	<0.001
Other^{^}	85 (11.0)	78 (15.6)	0.017
Symptoms mentioned during the call
Decreased or loss of consciousness (n=1103)	49 (7.4)	21 (4.8)	0.081
Face drooping (n=713)	258 (54.3)	93 (39.1)	<0.001
Arm weakness (n=772)	254 (51.0)	77 (28.1)	<0.001
Leg weakness (n=653)	201 (49.0)	85 (35.0)	<0.001
Sensory disturbances (n=375)	192 (89.7)	150 (93.2)	0.243
Communication problem in general (n=769)	413 (80.8)	187 (72.5)	0.008
Dysarthria (n=416)	181 (65.1)	76 (55.1)	0.047
Dysphasia (n=419)	163 (59.1)	72 (50.3)	0.089
Vision problem in general (n=184)	68 (78.2)	82 (84.5)	0.266
Blurry vision (n=74)	27 (77.1)	27 (69.2)	0.444
Diplopia (n=74)	14 (63.6)	23 (44.2)	0.127
Reduced vision (n=62)	15 (53.6)	22 (64.7)	0.374
Headache (n=497)	147 (57.0)	140 (58.6)	0.718
Loss of balance/motor coordination (ataxia) (n=236)	130 (86.1)	66 (77.6)	0.097
Dizziness (n=312)	120 (82.2)	143 (86.1)	0.338
Seizure (n=11)	4 (66.7)	3 (60.0)	0.819^&
Short term memory loss (n=68)	33 (76.7)	21 (84.0)	0.476
Shortness of breath (n=403)	62 (24.4)	25 (16.8)	0.072
Autonomic nervous system associated symptoms
Sweating (n=208)	47 (36.7)	45 (56.3)	0.006
Nausea or vomiting (n=311)	84 (61.8)	94 (53.7)	0.155
Pallor (n=255)	54 (32.7)	27 (30.0)	0.655
Ashen skin (n=198)	18 (14.1)	12 (17.1)	0.563
(Feeling of nearly) fainting (n=1103)	57 (8.6)	41 (9.3)	0.680
Course of symptoms
Onset of symptoms:
Per acute (seconds) (n=211)	52 (44.1)	56 (60.2)	0.020
Acute (minutes) (n=211)	46 (39.0)	23 (24.7)	0.028
Gradually (hours) (n=211)	20 (16.9)	14 (15.1)	0.710
Duration of symptoms ≤4.5 hours (n=986)	381 (61.4)	203 (55.6)	0.077
Symptoms still present at time of calling (n=1254)	716 (93.4)	438 (89.9)	0.030
Other characteristics
Caller expresses concern (n=628)	334 (90.3)	248 (96.1)	0.006
Patient never experienced similar symptoms before (n=368)	104 (49.8)	68 (42.8)	0.183
Recognition of symptoms:
TIA (n=368)	40 (19.1)	26 (16.4)	0.490
Stroke (n=368)	25 (12.0)	16 (10.1)	0.566
NTS: Netherlands Triage Standard; IQR: interquartile range; CVD: cardiovascular disease; TIA: transient ischaemic attack. *Concerns all cardiovascular medication with the exception of antithrombotics; ^#Pearson Chi Square Test for categorical variables and Mann-Whitney U Test for not normally distributed continuous variables; ^&Fisher’s Exact Test for categorical variables; ^{^}Amongst others: vomiting, dyspnea, neck symptoms, insult, disability problems (‘D from ABCD’).

Table 3. Final diagnoses of 1,269 patients who called the OHS-PC for symptoms suggestive of TIA/stroke

	High NTS urgency n=770 (60.7%)	Low NTS urgency n=499 (39.3%)	P-value
TIA/minor stroke	276 (35.8)	158 (31.7)	0.125
Major stroke*	149 (10.4)	52 (19.4)	<0.001
Other life threatening events (LTEs)** - Intracerebral haemorrhage - Subarachnoid haemorrhage	45 (5.8) 17 (37.8) 0 (0.0)	16 (3.2) 5 (31.3) 2 (12.5)	0.032 0.640 0.066^
Migraine - With aura	21 (2.7) 9 (42.9)	21 (4.2) 7 (33.3)	0.150 0.525
Epilepsy	17 (2.2)	6 (1.2)	0.190
Syncope	18 (2.3)	12 (2.4)	0.939
Brain tumor	13 (1.7)	2 (0.4)	0.059^
Peripheral vestibular syndromes - Benign paroxysmal positional vertigo - Meniere disease - Vestibular neuritis	22 (2.9) 10 (45.5) 1 (4.5) 0 (0.0)	42 (8.4) 11 (26.2) 1 (2.4) 5 (11.9)	<0.001 0.119 0.999^ 0.155^
Peripheral nerve problem - Bell’s palsy - Facial nerve palsy other than Bell’s palsy	75 (9.7) 22 (29.3) 53 (70.7)	47 (9.4) 13 (27.7) 34 (72.3)	0.850 0.842 0.842
Psychogenic syndromes	27 (3.5)	26 (5.2)	0.138
Other non-urgent diagnoses***	107 (13.9)	117 (23.4)	<0.001
Including lacunar infarction and stroke not otherwise specified; Amongst others sepsis, acute coronary syndrome, meningitis, herpes encephalitis, coma, severe anemia due to gastrointestinal bleeding, hypoglycaemia, acute pulmonary embolism; ** Amongst others guillain barre, multiple sclerosis, alcohol intoxication; ^Fisher’s Exact Test.

Table 4. Accuracy of adequate NTS urgency and final urgency allocation for detecting TIA/stroke/other LTEs

		Adequate NTS urgency allocation* Value (95% CI)**	Adequate final urgency allocation* Value (95% CI)**
TIA/stroke*	Sensitivity	0.71 (0.68-0.75)	0.86 (0.84-0.89)
	Specificity	0.46 (0.42-0.50)	0.38 (0.34-0.42)
	Positive predictive value	0.41 (0.38-0.43)	0.42 (0.40-0.44)
	Negative predictive value	0.75 (0.72-0.78)	0.84 (0.81-0.87)
Other LTEs**	Sensitivity	0.74 (0.61-0.84)	0.82 (0.70-0.91)
	Specificity	0.40 (0.37-0.43)	0.32 (0.30-0.35)
	Positive predictive value	0.06 (0.05-0.07)	0.06 (0.05-0.06)
	Negative predictive value	0.97 (0.95-0.98)	0.97 (0.95-0.98)
TIA/stroke and other LTEs	Sensitivity	0.72 (0.68-0.75)	0.86 (0.83-0.89)
	Specificity	0.48 (0.43-0.52)	0.40 (0.36-0.44)
	Positive predictive value	0.62 (0.60-0.64)	0.63 (0.62-0.65)
	Negative predictive value	0.58 (0.54-0.62)	0.70 (0.66-0.74)
Prevalence TIA/minor stroke 34.2% and prevalence major stroke 15.8%; Prevalence other LTEs 4.8%; **For TIA/minor stroke urgencies U1, U2 and U3 were all considered adequate, for major stroke and other LTEs urgencies U1 and U2 were considered adequate.

Download PDF

Journal Publication

published 05 Dec, 2020

Read the published version in BMC Primary Care →

Reviewer #1 agreed at journal
10 Nov, 2020
Review #1 received at journal
10 Nov, 2020
Editor assigned by journal
05 Nov, 2020
Reviewers invited by journal
05 Nov, 2020
Submission checks completed at journal
05 Nov, 2020
Editor invited by journal
05 Nov, 2020

You are reading this older preprint version

Read the latest preprint version →

Accuracy of telephone triage in patients suspected of transient ischaemic attack or stroke: A cross-sectional study

Status:

Journal Publication

Version 2

Abstract

Figures

Background

Methods

Results

Discussion

Conclusions

Declarations

Abbreviations

References

Tables

Supplementary Files

Status:

Journal Publication

Version 2