3.1 Development of Mobile Application
The application was developed by using the Ionic Framework. The application was evaluated in terms of compatibility, errors, missing information, and quality. The final version of the proposed application contains four main functions including triage, finding responsibility of emergency care unit, patient triage log file and export IDC information, has been released.
Application and functional tests were performed using iPhone 6s and Huawei Android phone (P20Pro). At the time of the study, the specifications of smartphones were defined for medium and large screen display devices, due to the visibility considerations [39].
The application procedure starts when the user opens the application, and screen A in Figure 2 appears showing the title and welcome graphics. After that, the main menu screen will be displayed. The buttons on screen B support the application login and triage history information. Screen L shows icons for the 25 symptom symbols for triage which is demonstrated in table view (screen D) and list view (screen C), respectively.
In Figure 2, on the screens E-G, the user can perform a search by typing the symptom name or keyword(s) into the textbox. The user can select a symptom from the list of search results on the screen. Screen F shows steps for CBD questions. The u has to select his/her answer to get to the next question, then continue until the IDC is displayed as shown on the screen H or I. Screen I shows the triage screening result which identifies the IDC code and the emergency status. The screen color will be the same as IDC code pattern. Suppose the IDC pattern is displayed in the form of “X color Y”, the first X refers to the group number of the symptoms corresponding to the input information in Table 2 and the last Y refers to the urgency status (1 being the highest and 9 being the lowest urgent status). On the screen K, the application provides information in the form of a suggestion to the user that they can implement while waiting for emergency resources. Moreover, user can record the IDC code via the application to summarize the patient's condition as well as the urgency status. This will allow emergency doctors/staff to gather information for the prehospital process. Additionally, on the screen J, the application supports the task of exporting the details of patients/emergency cases in .pdf format which can be forwarded to other applications such as Line and Short Message Service (SMS). On screen M, the application also supports the task of finding the nearest emergency care unit and hospital locations as well as their contact information (phone numbers as shown on the screen N). This data is sorted by the shortest distance and the shortest time to the destination. Furthermore, a map with directions to hospital locations is supported on the screen O with an overview of information.
Figure 2: here
3.2 CBD questions flow assessment
The CBD questions used in the application have been tested by evaluating the IDC result accuracy. All the 25 symptoms were tested in each question by following the flow in the CBD handbook. Emergency medical doctors tested the application by simulating a scenario and assessed the application results in comparison to the handbook. After the CBD questions results were verified and approved, the application testing proceeded to the next step.
3.3 Triage time evaluation
In this section, the application was tested to evaluate triage reliability and efficiency in comparison to various triage methodologies. The application was used to triage patients. The resulting IDC was assessed and evaluated for accuracy and confirmed to be a suitable request for resources.
To do this, we performed sampling by drawing 2 symptoms from total 25, which were used as simulation scenarios. Both trauma and non-trauma symptoms were used. Roleplaying was used to communicate the patient’s condition to the medical staff.
Using the procedure mentioned above, the IDC result and the triage time that emergency staff spent to obtain the IDC were recorded using three different methods. The results were then compared. The three methods include 1) using the application, 2) following the CBD protocol from the handbook and 3) relying on the emergency medical staff experience. There were 4 emergency medical staff to be the pilot samples for testing triage.
Table 5 demonstrates the testing results of each method with each scenario. The result shows clearly that using application for triage was as fast as the operation time from experienced emergency staff and as accurate as following the CBD protocol from the handbook. Some IDC misrepresentations were found when performing triage by relying on the staff experience.
Table 5 Here
In order to confirm the triage time testing, we conducted another test. In this test we compared the results between two methods. The first method was using the application and the second method was by relying on the emergency staff experience. Please note that the triage condition evaluation using the hand book was omitted in this test. Handbook referencing is only for reliability testing. Using the handbook is not a practical process for real-world operations.
The test conditions were set as follows the sample data was acquired by emergency medical staff. Each of the twelve emergency medical staff has no more than 5 years of work experience. The medical staff was split into two groups of six. Triage was performed by both using the application and relying on their experience. Test cases included 13 out of 20 scenarios, which covered the 25 main symptom categories (See Table 2). The average triage operational time was calculated.
The result of the triage time testing is shown in Figure 3. The result shows that using the application to triage non-trauma symptom cases was more efficient, requiring less time than relying on staff experience. In case of trauma symptoms, using staff experience was more efficient, requiring less time than using the application
Figure 3 Here
3.4 Application usability test
Usability tests are a very important point of the study because practical and pertinent considerations are the pain points of the previous system. . Thus, this step is required to ensure the quality and reliability of the application. The focus is to develop a high quality medical system and to make the software reliable. The usability test criteria from the usability testing technique conducted by Jakob Nielsen was adopted and applied in this experiment.
To do this, scenarios were created to evaluate usability. The key points of testing include occurrence of difficulties, doubt of visual representation, doubt on usage, missing information, and confusion when using the application. Each of the functions of the application were tested by 12 emergency medical staff including doctors and nurses. The testing results in Table 6 show the results of the findings for each test criteria (based on user motivation) as well as application functionality.
There were 12 instances of misunderstandings for the triage function (ranked 2nd in operational errors). Missing information was found when users wanted to repeat the CBD questions, which lead to duplicate IDC of patient conditions in the log file. Doubt of visual representation was found when users did not read the information shown on the left-side of the previous answer. There were some difficulties finding the answers of the pervious CBD questions. The second function, finding an emergency care unit, was found missing 4 times and was the 3rd overall ranked in operational errors. This might be due to the use of third party navigation applications. When the application shows the list of available emergency care provider locations are displayed using navigation applications such as Google Maps. It is also possible that errors arise from navigation applications. The next function is the patient triage log file, which was tested with 3 times of doubt on usage and misunderstanding. It was found that users did not understand. The logging file operated as a history of triage information record to confirm the patient’s condition. The last function is exporting IDC information to other applications such as SMS, Line, WeChat, e-mail, etc. The usability test problem in using the data export function has been found 3 times.
Users encountered problems sending information while devices were not connected to the internet. Confusion using the function was found 4 times because the users assumed that this function is used to produce a medical report.
For all of the evaluations of user motivation, confusion was found in the 1st rank with a total of 13 occurrences. The 2nd rank of the evaluations was doubt on usage, which occurred during log file exportation of IDC information. The 3rd rank of the evaluations was doubt of visual representation on some graphics and figures in the application during triage function, finding emergency care unit function, and the exporting of IDC information function, respectively.
Table 6: Here
This study has been conducted following the research questions: what should the system include that can help primary care physicians or patients to perform illness screening in both normal and emergency cases? How to design and develop an application to be able to help patients determine their own symptoms and assess the need for emergency treatment in Thailand.
Based on the application analysis and literature review from the previous studies of server medical application systems [2], the emergency alert system, map positioning for emergency medical services, patient monitoring and tracking system for high-risk patients who require services from EMS [3-5][16][22][27-28] and dental application triage development [26], are systems that can be used to support medical emergency operations. Thus, the proposed system has been designed for Thai social context. The application may be operated similar to other applications helping provide patients with an alternative choice for emergency aid.
The application for patients triage was developed by following the CBD to request IDC for the prehospital process [32-34]. The application can serve as a tool for primary emergency medical practitioners and general users. The adapted waterfall methodology together with the Ionic framework have been employed to produce a system under Android and iOS operating systems [35-36]. The human-centered theory, Eight Golden Rules of user interface design and Nielsen’s Ten were employed to design proper user interface of the application for practical use in real emergency situation [37-38].
The 25 main symptom categories covered by CBD were used to design and develop the application [32]. A twelve emergency medical staff including doctors and nurses were subjected to test the system in the following aspects: triage protocol correction, triage reliability, usability and users’ practical satisfaction.
As a result of application development, 4 functions were developed including (1) the triage function to identify IDC, (2) the finding of emergency care unit function, (3) the patient triage log file function providing triage information to support the pre-hospital process and confirm triage information and (4) exporting IDC information function for sending obtained IDC to the provincial emergency care units which can, in turn, use that information to prepare emergency resources while the patient is being transferred.
The first application test result was performed to check triage reliability and time spent performing triage under different conditions. The first experiment was to set a scenario test comparison for triage time testing. The result shows that, in the case of non-trauma patients, using the application spent less time than relying on staff experience. In case of trauma patients, using experienced staff to identify IDC consumed less time than using the application. The accuracy of identifying IDC under the condition of using handbook and the application is similar.
The second test was performed by sampling 13 scenario test cases from a total of 20 scenarios. In the case of trauma patients, the majority of the experienced staff required less time to triage than the application, however, the triage may result in a misrepresentation of IDC. The majority of the results claimed that the application is useful in terms of operational time and reliability considerations. Using the application is more likely suitable for non-trauma patients. However, the study of Savamongkornkul, indicated that mobile application performance could be enhanced when operated by experienced staff in triage [13].
The second testing result was conducted as described in section 3.4, the application usability test. The testing criteria was applied from the human-centered theory, Eight Golden Rules and Nielsen’s Ten Heuristics [37-38]. The result shows that the application still shows weak points with some confusion when in use. The triage function is the highest ranked of the weak points. In this function, users need to answer a number of CBD questions. Users misunderstood the number of answers needed.
In fact, some questions required only 1 or 2 answers. As a result, user misunderstanding is the cause. It was also found that missing information when finding emergency care unit functionality was caused from the Google API locating incorrect locations that are not emergency care units. Thus, API usage needed to be amended in order to locate the correct information of locations.
The application testing still shows weak points due to user perception. However, this application can be used to correctly perform symptom self-assessment, including in case of emergency. This will make treatment more effective.
This project can improve the overall development of primary care medical professionals to gain more knowledge by using this application. The research utility can generate improved primary medical emergency staff knowledge and performance based on accuracy of triage standard. Patients who apply the triage mobile application for primary diagnosis can decrease overcrowded hospital centers in each Northern Province. The result concomitantly support the result of Savamongkornkul [13], mentioned earlier. And if the application can be a tool for users to triage themselves when experiencing a suspicious symptom, it can help ED to reduce service time and increase the quality of medical service as reported by Kazi [11] that service time is related to patient outcome. Thus, the application is very suitable for people who request CBD codes without comprehensive medical knowledge and skills.