Short Report
Assessing the utility of artificial intelligence throughout the triage outpatients
https://doi.org/10.21203/rs.3.rs-2957289/v1
This work is licensed under a CC BY 4.0 License
posted
You are reading this latest preprint version
At present, many patients still need outpatient triage help. ChatGPT is increasingly used in medicine. Here, we assess the utility of ChatGPT in outpatient triage. We found that ChatGPT's answer is very professional, comprehensive, and humanistic, and can be applied to outpatient triage to provide better medical care for patients.
Artificial intelligence
ChatGPT
triage outpatients
Recently, China's National Bureau of Statistics pointed out that there were 8.4 billion outpatient visits in China in 2022. A large number of people need to be provided with triage help. Currently, artificial intelligence (AI) systems based on Chat Generation Pre-Training (ChatGPT) have received a lot of attention, which have proved practical and are increasingly used in medicine.(1–5). However, there is no research report on the application of ChatGPT in outpatient triage. Correct triage is critical to the diagnosis and treatment of a patient's disease(6, 7). Therefore, we evaluated the utility of ChatGPT in triage outpatients.
Through multiple experienced outpatient doctors, we provided a list of 30 medical questions that patients frequently ask in the outpatient setting. Each question answered by ChatGPT was assessed for accuracy by 5 doctors with extensive expertise in disease diagnosis and treatment, and these assessments were integrated.
After expert review, among the 30 answers, 17 are high scores (10 and 9.5 points), 7 are relatively high scores (9 points), and 6 are relatively low scores (8 and 7 points)(Fig. 1). In this study, we found every ChatGPT answer is easy to understand. Among them, 17 high scores better reflect the comprehensive and professional analysis, humanization, hierarchical diagnosis and treatment system and first aid concept (Table 1 and Supplementary Table 1). The answers to questions 3, 6 and 18 all reflect the hierarchical system of diagnosis and treatment, in which patients are advised to first go to a primary health care unit and then whether to be transferred to a higher health care unit based on the advice of the doctor in that unit. In the event of life-threatening symptoms (shortness of breathing, loss of consciousness, significant hemoptysis and neck stiffness), ChatGPT's answers provide first aid advice, such as answers to questions 2, 3, 11, 21 and 25. The 7 highly rated answers are generally professional and comprehensive, but still have room for improvement.The reviewers considered these 6 relatively low scores because some of the questions had not been adequately analysed (answer to question 29) and ChatGPT's answers to successive questions could have omitted the original questions, thereby depriving patients of the best consultation outcome (answer to question 14). Overall, the results show that ChatGPT's answers provide accurate and professional information about triage without providing misinformation or harmful information to patients.
| n | Questions | ChatGPT answer | Score | Evaluation |
|---|---|---|---|---|
| 1 | I'm a woman, 26 years old. I've been coughing since I caught a cold. Which department should I go to? | It is recommended that you go to the respiratory doctor. Cough is a common symptom of a cold, but if it lasts for too long, it may mean that infection has caused respiratory infections or other related issues. A respiratory doctor can determine the cause by checking your symptoms, physical examinations and necessary laboratory examinations, and formulate the corresponding treatment plan for you. | 9 | The answer is very professional. The terminology it uses is almost the same as that of the respiratory doctor. |
| 2 | The patient is my daughter, 10 years old. She started fever, headache, and vomiting at night. What diseases do we need to consider and which department should we go to see? | If your daughter starts fever, headache, and vomiting at night, it may indicate that she is experiencing some health problems. These symptoms may be related to many diseases, including colds, influenza, viral gastroenteritis or other diseases. It is recommended that you take her to see a doctor as soon as possible to determine the severity of the problem and the best treatment. You can take her to see a pediatrician or a family doctor. Doctors may conduct some examinations to determine the cause and give corresponding treatment suggestions. In the process of waiting for the clinic, you can let her rest, keep moisture intake, control body temperature, and pay attention to any new symptoms. If your daughter's symptoms become more serious, such as symptoms such as shortness of breathing, loss of consciousness, and neck stiffness, please immediately send it to the emergency department of the hospital nearby. | 10 | It can not only analyze the symptoms in a comprehensive manner, but also analyze the possible serious situation and provide accurate suggestions. |
| Note: n: problem number; Score: 0 to 10, 0 is the lowest, 10 is the highest; Evaluation: Summary of the comments of multiple doctors on ChatGPT's answers. | ||||
In medical practice, many scholars are concerned about the medical ethics and medical safety of AI operation(8–10). Our research is the application of ChatGPT in triage, without direct diagnosis and treatment. As a triage tool, ChatGPT can provide convenient ways for patients to see a doctor, strive for more treatment time, improve the rate of patient diagnosis and treatment, and make medical resources get more reasonable allocation. In the future, we hope to see ChatGPT triage run in healthcare facilities, or even as a mobile app that allows patients to travel faster and more accurately to the appropriate department for diagnosis and treatment.
Competing interests
Authors have no conflict of interest (financial interest, activities, relationships, and affiliations) to declare.
Funding
This work was funded by National Natural Science Foundation of China (81800195), Key Clinical Projects of Peking University Third Hospital (BYSYZD2019026, BYSYDL2021006).
Author contributions
WLZ, and HMJ conceived the project. XNL and RL analyzed the data. XNL, RL, CJY, HMJ and WLZ contributed towards the interpretation of the data. All authors wrote and approved the final manuscript.
Acknowledgements
We thank OpenAI for the ChatGPT.
No competing interests reported.
posted
You are reading this latest preprint version