Emergency department triaging using ChatGPT Based on Emergency Severity Index principles: A cross-sectional study

doi:10.21203/rs.3.rs-4620806/v1

Download PDF

Article

Emergency department triaging using ChatGPT Based on Emergency Severity Index principles: A cross-sectional study

https://doi.org/10.21203/rs.3.rs-4620806/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background

Erroneous and delayed triage in an increasingly crowded emergency department (ED). ChatGPT is an artificial intelligence model developed by OpenAI® and is being trained for use in natural language processing tasks. Our study aims to determine the accuracy of patient triage using ChatGPT according to ESI for triage in EDs.

Materials and methods

In our cross-sectional study, 18 years and over patients who consecutively presented to our ED within 24 hours were included. Age, gender, admission method, chief complaint, state of consciousness, and comorbidities were recorded on the case form, and the vital signs were detected at the triage desk. A five-member Expert Committee (EC) was formed from the fourth-year resident physicians. The investigators converted real-time patient information into a standardized case format. The urgency status of the patients was evaluated simultaneously by EC and ChatGPT according to ESI criteria. The median value of the EC decision was accepted as the gold standard.

Results

There was a statistically significant moderate agreement between EC and ChatGPT assessments regarding urgency status (Cohen's Kappa = 0.659; P < 0.001). The accuracy between these two assessments was calculated as 76.6%. There was a high degree of agreement between EC and ChatGPT for the prediction of ESI-1 and 2, indicating high acuity (Cohen's Kappa = 0.828). The diagnostic specificity, NPV, and accuracy of ChatGPT were determined as 95.63%, 98.17%, and 94.90%, respectively, for ESI high acuity categories.

Conclusions

Our study shows that ChatGPT can successfully differentiate patients with high urgency. The findings are promising for integrating artificial intelligence-based applications such as ChatGPT into triage processes in EDs.

Health sciences/Health care

Health sciences/Signs and symptoms

Artificial intelligence

emergency department

emergency severity index

triage

Emergency departments (EDs) are units that provide immediate intervention to victims of trauma, accidents, natural disasters, or life-threatening situations and provide uninterrupted service. The main goal of the EDs is to access patients who need urgent medical intervention fastest and recognize the most urgent patients. However, patients without urgent complaints also apply to EDs, which increases their workload. The most important point in the increased number of patients in EDs is to differentiate between the real emergent and non-emergent patients; therefore, triage is very important. Three, four, and five-step triage scales are used for this purpose [1–4].

Emergency Severity Index (ESI) is a validated triage scale. This scale is a five-category triage scale that is applied in a short time, and it is possible to determine the priority order without specifying the waiting time by considering the number of radiological or laboratory resources to be consumed in the diagnosis and treatment process of patients. ESI focuses on two main questions while triaging the patients: Firstly, who should be assessed and, if there are more than one patient with these criteria, who can wait longer [4].

Artificial Intelligence (AI) programs are a technical discipline that aims to do everything that human intelligence does using computer technology, and AI programs have begun to be used in many fields, including the medical field [5–9]. ChatGPT is a chat-based artificial intelligence program developed by OpenAI®, trained with reinforcement learning by using human feedback. It is designed based on machine learning and is an auditable and enforceable program. GPT-4 is a large and versatile model that, although less capable than humans, is capable of similar or human-level performance in many real-world situations. It also performs at the same level as humans on a variety of professional or academic metrics [10, 11].

The main aim of our study is to determine the accuracy of patient prioritization by using ChatGPT according to ESI principles for triage in EDs. Thus, it is aimed to facilitate the triage of patients in the operation of the EDs.

2.1. Study design and population

This cross-sectional study was carried out in the Emergency Department of Tertiary Research Hospital, which has a high capacity, accepts both personal admission and ground and air ambulance patient transfers, and has many facilities for examinations and consultation to all surgical departments.

All patients aged 18 years and over who applied between July 20, 2023, at 8.00 am and July 21, 2023, at 8.00 am were consecutively included in the study. Patients under the age of 18 and those who did not accept the study were excluded. Accordingly, 745 patients were included in the study.

2.2. Establishment of Expert Committee:

An Expert Committee (EC) consisting of five volunteers from our fourth-year resident physicians was formed. This committee had a four-hour theoretical training with case examples on the ESI triage system on December 20, 2023. A single instructor completed the training program to ensure the standardization of the training content. Firstly, theoretical information about the introduction of ESI and the contents of the categories were explained and then the cases were discussed.

2.3. Ethical statements

This study was approved by the local ethics committee on July 12, 2023 (Decision No: E2-23-4509). Patients were recruited by the Declaration of Helsinki. Patients participating in the study were asked to read and approve the informed consent form of their own free will.

2.4. Data collection

Vital signs (blood pressure, oxygen saturation, pulse, and body temperature) of the patients who were admitted to the ED and accepted the study were recorded on the case form together with their age, gender, method of admission, chief complaint, state of consciousness, comorbidities (diabetes mellitus, hypertension, coronary artery disease, congestive heart failure, chronic kidney disease, cerebrovascular accident, liver failure).

Two monitors (GE Medical Systems, B40 Patient Monitor®, GE Medical Systems Information Technologies, Inc.) were used during the triage process. Blood pressure, pulse rate, and saturation values were measured and recorded automatically with this monitor. The Temporal thermometer (Mesilife, DT-8806 Non-Contact Forehead Infrared Thermometer) was used for body temperature measurement.

2.5. Scoring system

2.5.1. Emergency Severity Index

With ESI triage, patients who ‘have risk of wait’ and who 'require urgent intervention' are identified primarily. The triage category of other patients is then determined according to the estimated number of available resources. The ESI application uses four basic decision stages for this purpose. Four key questions exist when determining a patient's triage category [4].

1. Is the patient in a life-threatening case, or is the patient dying?

2. Is the patient unable to wait?

3. How many resources will the patient need to use during the diagnosis and treatment process?

4. What are the patient's vital signs?

Accordingly, patients are triaged by classifying them from the highest urgency category (one) to the lowest urgency category (five).

2.6. Triage category determination and patient grouping

The information recorded in the case form was organized as follows: "A 70-year-old male patient was admitted to the ED by Emergency Medical Service (EMS) ambulance with complaints of abdominal pain and constipation. It was learned that the patient's complaint of constipation had been going on for nine months, abdominal pain had been present for two days and there was no stool discharge for one day. The patient had a known history of malignancy and coronary angiography. The general condition of the patient was good, he was conscious, orientated, and co-operative. His blood pressure was 110/70 mmHg, pulse rate was 97 beats/min, oxygen saturation was 94% and body temperature was 36.3 °C. What is the patient's triage score according to ESI?". All 745 real-time cases were converted into text during November 2023.

The triage scores of all cases were evaluated independently and simultaneously by ChatGPT ver.4 (November 2023) in Turkish language, and the results were recorded.

The results obtained were analyzed both for the whole patient group and for individual ESI categories. Finally, ESI-1 and 2 categories were evaluated as high acuity groups; dichotomous data was created as "severe" and "mild," and subgroup analysis was performed.

2.7. Statistical analyses

The study's statistical analyses were performed with IBM SPSS Statistics for Windows, Version 20.0 (Armonk, NY: IBM Corp). Distribution analyses of continuous variables were performed by the Shapiro-Wilk Test, comparisons between two groups for normally distributed continuous variables were performed by the Independent Samples-t Test, and comparisons between two groups for non-normally distributed variables were performed by the Mann Whitney-U test. For comparisons of categorical data, Pearson Chi-Square and Fisher's Exact test were used according to the case.

The median value of the results of ESI, which is an ordinal parameter obtained by the evaluations of five members of the EC, was accepted as the gold standard. The agreement analysis of the EC and ChatGPT was evaluated with the Kappa statistic. For Kappa coefficients, <0.20 was considered poor, 0.21 to 0.40 reasonable, 0.41 to 0.60 moderate, 0.61 to 0.80 good, and 0.81 to 1.00 high concordance. As a result of this analysis, diagnostic accuracy statistics such as specificity and sensitivity were also given. This analysis was repeated for individual ESI categories. In addition, dichotomous data was created as "severe" and "mild" according to the high urgency group; like the general analysis, the power of ChatGPT to distinguish this group was also analyzed. The P < 0.05 level was used as the statistical significance criterion. All statistical analyses were completed in January 2024.

3.1. Descriptive data

Among the patients included in the study, 83.8% (n = 625) were outpatients and 16.2% (n = 120) were admitted by EMS ambulance. The mean age of the patients was 44±19 years and 51.1% (n = 381) were male. The most common complaints were abdominal pain (9.30%), soft tissue disorders (9.10%) and gastroenteritis-colitis (7.80%); the most common comorbidities were hypertension (20.9%), diabetes mellitus (12.2%) and coronary artery disease (8.1%) during admission to the ED. The mean systolic blood pressure was 132 ± 22 mmHg, the mean diastolic blood pressure was 77 ± 13 mmHg, the mean pulse rate was 87 ± 16 beats/min, the mean oxygen saturation value was 96 ± 3%, and the mean body temperature was 36 ± 1 °C. (Table 1)

3.2. Triage scores of the study group

According to the triage assessment by both EC and ChatGPT according to the ESI criteria, the ESI-3 category was detected most frequently (n = 370 [49.7%] vs n = 335 [45%]), (Table 2).

3.3. Agreement of EC and ChatGPT assessment

In our study, when all ESI categories were taken into consideration together, the agreement within the EC was found to be moderate (Fleiss' Kappa value = 0.373). The agreement within the EC was evaluated in each ESI triage category separately and kappa values were found as statistically significant and moderately concordant. (ESI-1 Kappa = 0.451; ESI-2 Kappa = 0.484; ESI-3 Kappa = 0.361; ESI-4 Kappa = 0.217 and ESI-5 Kappa = 0.454; P < 0.001, P < 0.001, P < 0.001, P < 0.001 and P < 0.001, respectively).

When all ESI categories of the study were taken into consideration together, the comparison of EC and ChatGPT results showed that ChatGPT decided for 'over' triage in 10.7% (n=80) and 'under' triage in 12.6% (n = 94) of the cases. (Table 3) Accordingly, good agreement was seen between EC and ChatGPT assessment results (Cohen's Kappa = 0.659) (P < 0.001). The accuracy between these two evaluations was calculated as 76.6%. In the agreement evaluation between EC and ChatGPT in each ESI triage category separately, kappa values for all triage categories were determined as statistically significant and moderately well concordant. (ESI-1 Kappa = 0.840; ESI-2 Kappa = 0.804; ESI-3 Kappa = 0.674; ESI-4 Kappa = 0.515 and ESI-5 Kappa = 0.640; P < 0.001, P < 0.001, P < 0.001, P < 0.001 and P < 0.001, respectively).

According to ESI criteria, the diagnostic accuracy of the ChatGPT assessment results for triage was determined separately for each ESI category. Accordingly, the specificity of ChatGPT for the ESI-1 category was determined as 99.86%; the negative predictive value (NPV) was 99.73%, and the accuracy was 99.60%. A high degree of agreement was seen between EC and ChatGPT in terms of the prediction of the high acuity group (ESI 1 and 2 together) (Cohen's Kappa = 0.828). The ChatGPT assessment's specificity was 95.63%, the NPV value was 98.17%, and the accuracy was 94.90% for ESI categories, indicating the high acuity group (Table 4).

The intersection of AI and emergency medicine is a promising area due to the increasing workload in EDs, which is parallel to the increasing number of patients daily. Our study is critically important regarding the use of AI in the triage area, which is the ED process's first and most important step. Our study aims to test the accuracy of ChatGPT assessment by triaging 745 real-time consecutive ED admissions according to ESI criteria. The median value of EC was considered as the gold standard and compared with the results obtained by ChatGPT assessment. In both groups, most cases were determined as ESI category 3. It was determined that there was a moderate agreement among EC. When EC and ChatGPT assessment results were compared, significant concordance was found. This agreement was found to be higher, especially for the ESI 1 and ESI 2 categories, which indicate high acuity. At the same time, the highest accuracy for ChatGPT assessment was obtained for the ESI 1 and ESI 2 categories.

Zaboli et al. compared the accuracy of the decisions made by the triage nurse group and ChatGPT in terms of triage based on the Manchester Triage Scale, 72-hour mortality, hospitalization, and prediction of the presence of severe or time-dependent condition outcomes of patients admitted to the ED. A total of 30 randomly selected cases were included in the study. The agreement between the nurse group and ChatGPT assessment was determined to be 0.278, and AUC was higher in the healthcare professional group regarding all study outcomes. This difference was found to be more pronounced regarding 72-hour mortality (AUC 0.910 vs 0.669) [12]. The report was a single-center study with a limited number of selected cases, and agreement between healthcare professionals wasn’t reported. We think that the difference in the agreement between healthcare professionals and ChatGPT assessment in terms of triage decisions is that our study was carried out with a larger number of real-time patient data and the preliminary preparation for EC standardization consisting of physicians.

In a study by Sarbay et al. examining ESI triage performance with ChatGPT, 50 scenarios were determined and compared with the common opinion of two expert physicians. positive predictive value (PPV) was found to be 38.7%, NPV 52.6%, and F1 score 0.461 [5]. It was seen that the ChatGPT diagnostic accuracy results obtained in our study showed better performance. Although both were performed ChatGPT in Turkish, we think there is a difference between the report and our study due to the difference in the number of cases and the use of real-time patient data. In addition, the new Chat GPT ver. 4, which was used in our study, may also have caused the difference between the two studies.

Fraser et al. have evaluated the triage efficiency and diagnostic performance of four different data systems (Ada Health Symptom Checkers, Web MD Symptom Checkers, ChatGPT ver. 3.5 and ChatGPT ver. 4.). In this study, a 3-step triage (home care, routine primary care and emergency or urgent care) was applied, and the triage of patients and predictions of their final diagnosis were compared with the opinion of three experts. The performance of the algorithms in terms of diagnosis was found to be Ada (63%) > ChatGPT ver. 3.5 (63%) > WebMD SC (57%) > ChatGPT ver. 4. (50 %), while the accuracy ranking for triage was reported as ChatGPT ver. 4. (76%) > WebMD SC (70%) > Ada (62%) > ChatGPT ver. 3.5 (59%) [13]. The authors have stated that ChatGPT ver. 3.5 should not be used without improvements in the accuracy of its judgments. The study is limited by its retrospective method and the number of patients. This may have caused the difference seen with the ChatGPT diagnostic accuracy results in our study about the triage assessments. On the other hand, it is noteworthy that different accuracy results were obtained with different algorithms in this study.

In the study of Paslı et al., the effectiveness of ChatGPT assessment in triaging patients admitted to the ED according to the principles of local triage practice rules was investigated. In the study, the triage decision made by the emergency medicine physician was accepted as the gold standard. Accordingly, a good level of agreement was found between the opinion of both the triage team and ChatGPT with the emergency medicine physician (Cohen's Kappa 0.893 vs 0.899) [14]. These results are better than the agreement results obtained in our study based on ESI criteria. This study, which was performed after teaching local triage rules to ChatGPT, opens a new window in the use of AI-based applications in the field of triage.

In a retrospective study, Ivanov O. et al. investigated the accuracy of triage performed by AI-based clinical natural language processing and machine learning algorithms (KATE) according to ESI criteria. The nurses’ decisions about triage were reevaluated by KATE and clinicians. This algorithmic triage model was found to be 26.9% higher in terms of accuracy than nurse triage. In this study, unlike the results of our study, the developed model was reported to have performed 'under' triage in 9.7% and 'over' triage in 14.4% of patients, but similar to our study, the F1 score (0.9923) was found to be quite high [15]. Karlafti e. et al. investigated the effectiveness of the AI-based automatic patient screening classifier algorithm in ED triage based on ESI criteria. The overall accuracy of triage was determined as 84.6%, and the F1 score was 72.2% [16]. It is understood that triaging based on certain triage criteria in different AI-based algorithms gives different accuracy results. The 'over' and 'under' triage decision ratios and F1 score values in ChatGPT assessment based on ESI also differ between the studies. In these aspects, it should be noted that the subject continues to develop. It should also be taken into consideration that modifications to the criteria of triage scales are necessary in terms of application with AI-based algorithms.

As a result of our study, it was concluded that the diagnostic accuracy of the artificial intelligence-based ChatGPT assessment was high, especially in patients with high acuity levels such as ESI 1 and 2. However, the currently used triage scales are intended for human use and require human experience. In light of technological developments, it is possible to integrate AI-based triage systems into ED triage decision-making processes soon, but studies with different triage scales and different languages are important for the availability of these applications.

Our study was conducted in a single-center and tertiary academic hospital and can’t be generalized to primary and secondary healthcare institutions.

Acknowledgments

As the authors, we would like to thank Fulya Keskin Elmas, Sena Tamer, Derya Menekşe Parlak, Seden Simge Bahar, and Ibrahim Dilekcan as the expert committee for supporting our work.

Disclosure

All authors have no potential conflicts of interest to disclose.

Funding

This research received no specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Data Availability :

The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request

Author Contributions:

Conceptualization: CÇ, HSO, ME; Data curation: CC, AS; Investigation: CC, AS, ME; Methodology: ME, AS, CC; Project administration: AO, SG, AS; Supervision: AO, SG, ME ; Writing–original draft: HSÖ, SG, CC, ME; Writing–review & editing: HSO, AO, ME

Derlet RW, Kinser D, Ray L, Hamilton B, McKenzie J. Prospective identification and triage of nonemergency patients out of an emergency department: a 5-year study. Annals of Emergency Medicine. 1995;25(2):215–223. https://doi.org/10.1016/S0196-0644(95)70327-6
Karcioglu O, Yanturali S, Ersel M, Sever M, TunÇ MA et al. Bir acil servisin kullanım özellikleri ve başvuran hastaların aciliyetinin hekim ve hasta açısından değerlendirilmesi. Türkiye Acil Tıp Dergisi. 2006; 6 (1):25–35. (in Turkish)
Bezzina AJ, Smith PB, Cromwell D, Eagar K. Primary care patients in the emergency department: who are they? A review of the definition of the 'primary care patient' in the emergency department. Emergency Medicine Australasia. 2005;17 (5–6):472–479. https://doi.org/10.1111/j.1742-6723.2005.00779.x
Gilboy N, Tanabe P, Travers DA, Rosenau AM, Eitel DR. Emergency Severity Index, Version 4: Implementation Handbook. AHRQ Publication No. 05-0046-2. Rockville, MD: Agency for Healthcare Research and Quality. 2005
Sarbay İ, Berikol GB, Özturan İ U. Performance of emergency triage prediction of an open access natural language processing based chatbot application (ChatGPT): A preliminary, scenario-based cross-sectional study. Turkish Journal of Emergency Medicine. 2023; 23 (3):156–161. https://doi.org/10.4103/tjem.tjem_79_23
Thrall JH, Li X, Li Q, Cruz C, Do S, et al. Artificial Intelligence and Machine Learning in Radiology: Opportunities, Challenges, Pitfalls, and Criteria for Success. Journal of American College of Radiology. 2018;15 (3 Pt B):504–508. https://doi.org/10.1016/j.jacr.2017.12.026
Gupta R, Srivastava D, Sahu M, Tiwari S, Ambasta RK, et al. Artificial intelligence to deep learning: machine intelligence approach for drug discovery. Molecular Diversity. 2021;25(3):1315–1360. https://doi.org/10.1007/s11030-021-10217-3
Kose E, Ozturk NN, Karahan SR. Artificial Intelligence in Surgery. European Archives of Medical Research. 2018;34(0):4–6. https://doi.org/10.5152/eamr.2018.43043
Jartarkar SR. Artificial intelligence: Its role in dermatopathology. Indian Journal of Dermatology, Venereology and Leprology. 2023;89(4):549–552. https://doi.org/10.25259/IJDVL_725_2021
OpenAI (2024) GPT-4 [online]. Website https://openai.com/research/gpt-4 [Accessed 05 February 2024]
OpenAI (2024). Introducing ChatGPT 024 [online]. Website https://openai.com/research/gpt-4 [Accessed 05 February 2024]
Zaboli A, Brigo F, Sibilio S, Mian M, Turcato G. Human intelligence versus Chat-GPT: who performs better in correctly classifying patients in triage? American Journal of Emergency Medicine. 2024;79:44–47. https://doi.org/10.1016/j.ajem.2024.02.008
Fraser H, Crossland D, Bacher I, Ranney M, Madsen T, et al. Comparison of Diagnostic and Triage Accuracy of Ada Health and WebMD Symptom Checkers, ChatGPT, and Physicians for Patients in an Emergency Department: Clinical Data Analysis Study. JMIR mHealth and uHealth. 2023;11:e49995. https://doi.org/10.2196/49995
Paslı S, Şahin AS, Beşer MF, Topçuoğlu H, Yadigaroğlu M, et al. Assessing the precision of artificial intelligence in ED triage decisions: Insights from a study with ChatGPT. American Journal of Emergency Medicine. 2024;78:170–175. https://doi.org/10.1016/j.ajem.2024.01.037
Ivanov O, Wolf L, Brecher D, Lewis E, Masek K, et al. Improving ED Emergency Severity Index Acuity Assignment Using Machine Learning and Clinical Natural Language Processing. Journal of Emergency Nursing. 2021;47(2):265–278.e7. https://doi.org/10.1016/j.jen.2020.11.001
Karlafti E, Anagnostis A, Simou T, Kollatou AS, Paramythiotis D, et al. Support Systems of Clinical Decisions in the Triage of the Emergency Department Using Artificial Intelligence: The Efficiency to Support Triage. Acta Med Litu. 2023;30(1):19–25. https://doi.org/10.15388/Amed.2023.30.1.2

Table 1. Demographics, comorbidities, complaints, and vital signs distribution of the patients included in the study.

Parameters		n (%)	Mean Value (±SD)	Minimum - Maximum
Age	All	745 (100)	44 ± 19	18 - 98
	Female	364 (48.9)	44 ± 19	19 - 98
	Male	381 (51.1)	44 ± 19	18 - 94
Vital Signs	Systolic Blood Pressure (mmHg)		132 ± 22	60 - 247
	Diastolic Blood Pressure (mmHg)		77 ± 13	40 - 149
	Heart Rate (Beats / Minute)		87 ± 16	46 - 170
	Body Temperature (C)		36 ± 1	35 - 40
	Oxygen Saturation (%)		96 (94 - 97)	40 -100
			Present (n (%))
Comorbidity (Three most common)	Hypertension		156 (20.9)
	Diabetes Mellitus		91 (12.2)
	Coronary Artery Disease		60 (8.1)
Main Complaint (Three most common)	Abdominal Pain		69 (9.3)
	Soft Tissue Disorder		68 (9.1)
	Gastroenteritis and Colitis		58 (7.8)

Table 2. Distribution of emergency status based on ESI criteria according to Expert Committee and ChatGPT

ESI*	1		2		3		4		5
ESI*	n	%	n	%	n	%	n	%	n	%
Referee 1	8	1.1	163	21.9	419	56.2	41	5.5	114	15.3
Referee 2	7	0.9	118	15.8	377	50.6	158	21.2	85	11.4
Referee 3	17	2.3	144	19.3	359	48.2	165	22.1	60	8.1
Referee 4	39	5.2	87	11.7	310	41.6	188	25.2	121	16.2
Referee 5	10	1.3	123	16.5	280	37.6	171	23.0	161	21.6
Expert Committee**	10	1.3	117	15.7	370	49.7	147	19.7	101	13.6
ChatGPT	9	1.2	134	18.0	335	45.0	184	24.7	83	11.1

*ESI: Emergency Severity Index **Median Value of Expert Committee

Table 3. Comparison of Expert Committee and ChatGPT assessment results across all ESI triage categories

ESI*		ChatGPT (n)					Total
ESI*		1	2	3	4	5	Total
Expert Committee** (n)	1	8	2	0	0	0	10
	2	1	105	10	0	1	117
	3	0	27	292	48	3	370
	4	0	0	28	103	16	147
	5	0	0	5	33	63	101
Total		9	134	335	184	83	745

*ESI: Emergency Severity Index **Median Value of Expert Committee

Table 4. Diagnostic accuracy of ChatGPT assessment results for ESI categories

Diagnostic Parameters	ESI*-1 (95 % CI)	ESI*-2 (95 % CI)	ESI*-3 (95 % CI)	ESI*-4 (95 % CI)	ESI*-5 (95 % CI)	High Acuity Categories** (95 % CI)
Sensitivity	80.00% (44.39-97.48)	89.74% (82.77-94.59)	78.92% (74.40-82.96)	86.90% (82.82-90.32)	62.38% (52.18-71.82)	91.34% (85.03-95.60)
Specificity	99.86% (99.24-100.00)	95.38% (93.44-96.89)	88.53% (84.87-91.58)	86.45% (83.45-89.10)	96.89% (95.24-98.09)	95.63% (93.71-97.10)
Positive Likelihood Ratio	588.00 (80.94 - 4271.66)	19.43 (13.55-27.88)	6.88 (5.17-9.16)	6.42 (5.22-7.89)	20.09 (12.71-31.73)	20.91 (14.40-30.35)
Negative Likelihood Ratio	0.20 (0.06-0.69)	0.11 (0.06-0.18)	0.24 (0.19- 0.29)	0.15 (0.11-0.20)	0.39 (0.30-0.50)	0.09 (0.05-0.16)
Disease Prevalence	1.34% (0.65-2.45)	15.70% (13.16-18.52%	49.66% (46.01-53.32)	35.97% (32.89-39.15)	13.56% (11.18-16.23)	17.05% (14.41-19.94)
Positive Predictive Value	88.89% (52.41-98.31)	78.36% (71.63-83.85)	87.16% (83.61-90.04)	78.28% (74.57-81.59)	75.90% (66.60-83.27)	81.12% (74.74-86.18)
Negative Predictive Value	99.73% (99.07-99.92)	98.04% (96.69-98.84)	80.98% (77.69-83.87)	92.16% (89.90-93.94)	94.26% (92.74- 95.48)	98.17% (96.83-98.95)
Accuracy	99.60% (98.83-99.92)	94.50% (92.61-96.02)	83.76% (80.91-86.34)	86.62% (84.26-88.74)	92.21% (90.05-94.04)	94.90% (93.07-96.37)
F1 score	0.8421	0.8367	0.8284	0.6224	0.6848	0.8593

*ESI: Emergency Severity Index; **Including ESI-1 and 2

No competing interests reported.

Download PDF

Editorial decision: Revision requested
09 Aug, 2024
Reviews received at journal
04 Aug, 2024
Reviews received at journal
19 Jul, 2024
Reviewers agreed at journal
11 Jul, 2024
Reviewers agreed at journal
08 Jul, 2024
Reviewers invited by journal
05 Jul, 2024
Editor assigned by journal
27 Jun, 2024
Editor invited by journal
25 Jun, 2024
Submission checks completed at journal
25 Jun, 2024
First submitted to journal
22 Jun, 2024

You are reading this latest preprint version

Emergency department triaging using ChatGPT Based on Emergency Severity Index principles: A cross-sectional study

Status:

Version 1

Abstract

Background

Materials and methods

Results

Conclusions

1. Introduction

2. Materials and methods

3. Results

4. Discussion

5. Conclusion

6. Limitations

Declarations

References

Tables

Additional Declarations

Status:

Version 1