Feasibility and acceptability of a serious game to study the effects of environmental distractors on emergency room nurse triage accuracy: a pilot randomized controlled trial

doi:10.21203/rs.3.rs-2915359/v1

Download PDF

Research Article

Feasibility and acceptability of a serious game to study the effects of environmental distractors on emergency room nurse triage accuracy: a pilot randomized controlled trial

https://doi.org/10.21203/rs.3.rs-2915359/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background: Emergency room nurse triage entails complex clinical decision making under stress and time constraints. Workplace distractors, such as noise and task interruptions, have been pointed to as possible reasons for triage inaccuracy, though studies have not borne this out. To train nurses to perform the task, a serious game was developed to simulate the triage process and environment. A pilot study was undertaken to evaluate the acceptability and feasibility of the serious game and to collect preliminary data on the effects of two distractors—noise and task interruptions—on emergency room nurse triage accuracy.

Methods: A 2 × 2 factorial randomized controlled trial was designed for the study. A sample of 70 emergency room nurses was randomly assigned to four groups: three experimental groups exposed to different distractors (noise, task interruptions, and both) and one control group exposed to none. The nurses in each group had two hours to complete a series of 20 clinical vignettes where they had to establish a chief complaint and assign an emergency level. Their responses were then compared against a gold standard predetermined by experts.

Results: Among 70, 55 nurses completed about 15 vignettes each in the allotted time. No inter-group differences emerged in terms of triage performance. Only a few minor technical issues arose during the sessions. Nurses had a very favorable appreciation of the serious game on triage as measured through a semantic differential scale.

Conclusions: Results show that both the structure of our study and the serious game on triage can be used to carry out a future RCT on a larger scale. Two minor technical bugs were identified and corrected, the series of vignettes to complete in two hours must be shortened to 15, and the test should be held during work hours to support recruitment and reduce the possible impact of nurse fatigue on triage performance. The absence of a distractor effect raises questions about whether distractors have an effect and at what frequency and intensity of noise and task interruptions might have an impact on triage performance and whether this dose is realistic.

triage

emergency nursing

gamification

patient safety

interruptions

Patient safety is defined as the “reduction of risk of unnecessary harm associated with health care to an acceptable minimum” (p. 19) [1]. It has become one of the chief concerns in contemporary healthcare systems [2]. The matter is of even greater importance in emergency room (ER) settings, where healthcare professionals, specifically nurses, must make complex clinical decisions rapidly based on limited information [3–6]. Clinical decision making is defined as “a contextual, continuous, and evolving process, where data are gathered, interpreted, and evaluated in order to select an evidence-based choice of action” [7](p. 401). This is the basis of almost all ER activities, including triage, that is, the task of sorting and prioritizing patients at arrival by urgency and need for care. In the interest of providing safe healthcare services efficiently, triage must also aim to allocate available human and physical resources as best possible. Given the complexity of this clinical decision-making process and triage being a human-driven activity, it is prone to errors that can have serious consequences for patient safety[8, 9]. On the one hand, underestimating a patient’s condition can lead to delayed emergency care and/or insufficient medical treatment with consequent risks for patient health[10] .On the other, overestimating a patient’s conditions can result in ERs being needlessly overburdened and in more urgent cases not receiving more immediate care[11].

To facilitate this process and to reduce the possibility of errors, various emergency triage scales have been created in the past decades based on scientific research and professional experience. Widely used in many countries, they comprise two components, namely, the scale itself, which often has four or five emergency levels[12–15], and a procedure for categorizing situations by level. The situations are then prioritized, and resources allocated accordingly. In French-speaking Switzerland, the Swiss Society of Emergency and Rescue Medicine currently recommends using the Swiss Emergency Triage Scale or SETS®[16] for patient triage. This instrument has been scientifically validated and is used also in France and Belgium [11, 17]. The SETS rates four emergency levels: 1. acute; 2. urgent; 3. semi-urgent; 4. non-urgent.

To assess the validity and accuracy of triage scales, studies [18–20] have relied on precise procedures to create gold standards to hold nurses’ decisions up against. The advent of triage scales has also given a boost to research on ER activities [21], as it has largely facilitated the study of clinical decision making in this setting. This is because triage scales yield a standardized output, that is, an emergency rating and sometimes the motivation behind it, which can easily be used for testing and comparison purposes. After the introduction of triage scales, research into how accurate they were in attributing emergency levels flourished. Researchers had at their disposal a precise procedure and a limited set of outputs that provided the basis for creating gold standards against which to compare nurses’ decisions [22]. Numerous studies have used this approach and, as Farrohknia et al. [22] pointed out in their systematic review, found the level of accuracy of emergency nurse decisions to be medium to low, with nurses erring on the side of both over-triage[23, 24], that is, assigning a higher level than the gold standard, and under-triage[10, 25], that is, assigning a lower one. While nurse performance has been researched extensively[22], the reasons behind this performance have been little investigated, especially the hindering factors. Generally, the studies that have addressed this issue have focused on two groups of factors as possible causes: individual and contextual [26–28]. Individual factors include characteristics specific to nurses, such as personality (flexibility, decision-making autonomy), cognitive processes (critical thinking, prompt decision making), behavioral processes (working under pressure, being organized), and nursing experience (confidence in one’s decision making)[26, 29, 30]Contextual factors include distractors present in the environment [31–34], such as task interruptions, noise, and workload changes. All these factors can affect nurse performance by causing information to be lost and concentration to drop, particularly when performing complex activities. These elements have been the subject of very little exploration in the literature. Studies have underscored the link between environmental distractors and nurse performance [35], but still little is known about the types of effect (e.g., linear, presence of thresholds), which distractors have the greatest impact, and how distractors interact.

Where method is concerned, most of the research on ER nurse clinical decision making has been based on either the retrospective review of records or nurse-assessed written clinical vignettes, that is, texts describing a clinical situation. The two methods share one major shortcoming, namely, a lack of realism, which some authors[36] consider a critical element in evaluating ER nurse performance and clinical decision making. A proposed solution to the problem has been the use of serious games (SG) that simulate both ER nursing tasks and the ER triage environment[36, 37]. SG are defined as games for learning more that for entertainment. They are widely used in the fields of professional development and training, education, and scientific research[38, 39], especially in environments where there is a high risk for adverse events, such as air traffic control[40] and the military[41]. In the medical field, SG have already been used to show that interruptions lengthen the duration of medical evaluations and affect subsequent decisions, which begin by being more disorganized and deviate from prescribed standards [41]. SG provide a unique opportunity that allows immersing nurses in more realistic situations (compared with paper-based vignettes), that are useful to investigate triage and clinical decision making[32, 42]and that has already proven to be a pertinent method of research[43, 44].

The Systems Engineering Initiative for Patient Safety (SEIPS) is a theoretical framework developed by Carayon et al.[45–47].Its purpose is to explain how the design of a work system can impact not only patient safety, but also employee and organization performance. The model is based on Donabedian's dimensions of structure, process and outcome[48], but with structure reformulated as a work system. The work system is the starting point of the model, which is built around five components: person, task, tools and technologies, physical environment, and organizational conditions. These components interact and influence each other within the work system. The result of these different interactions and influences impacts the care process and ultimately the outcome at the level of both the patient (e.g., safety) and the organization/employee (e.g., job satisfaction). The physical environment is considered an integral part of the work system. Task interruptions and noise, too, have been proposed as constitutive elements of the work system, especially in one such as the triage area[35, 49].

Serious Game Triage (SGTRI)

Triage is a rather difficult activity to simulate on account of the variety of cases that ER nurses must face and the multiple instruments that may have to use to evaluate patients. Despite the presence of standardized procedures, the decision tree and the tools used vary greatly across cases. Consequently, any sort of simulation must first create a vast matrix that covers all the outputs to all the questions or actions that the player may have to consider. This matrix is the core of any simulation as it provides the answers to nurse enquiries. Based on these answers, nurses reach a conclusion, set the emergency level, and motivate their choice. Whereas paper-based simulations stop here, more complex instruments, such as SG, add a further layer. To increase immersivity and to test other aspects of clinical decision making beyond the mere outcome, it is necessary also to simulate the framework in which the clinical decision-making process occurs by adding elements such as human interaction, noise, and visual indications.

The SGTRI aims to simulate all these elements. We started creating a matrix of questions and answers based on an existing interactive simulator enabling a simple exploration of clinical vignettes[17]. A vignette is no more than the written description of a clinical situation. The situation, symptoms, biological values, and other characteristics of patients are pre-defined based on emblematic cases. We created 20 such vignettes. Triage nurses can investigate them by entering keywords that lead to pre-defined questions or by clicking on icons representing parameter values, such as a pulse oximeter, and receiving feedback. When satisfied with the collected information, the user assigns an emergency level and enters the chief complaint justifying it in a specific window of the SG. Once the answer is validated, a new vignette is presented for evaluation. As the vignettes are presented in a pre-determined order, there is no need for users to manage the wait line. This is the core of the SGTRI. For the sake of greater immersivity, users complete vignettes in a 2D virtual environment replicating an emergency waiting room, rather than text-based vignettes. Moreover, in each vignette the patient is represented by a 2D avatar. As for the sound environment, there is permanent background noise, including the hum of ventilation, doors opening and closing, footsteps approaching, and voices chattering, at a level varying from 40 dB to 50 dB, depending on how many people are in the waiting room.

As we were interested in how distractors affect triage performance, we added to the simulation the two distractors most frequently indicated in the literature[32, 33, 50]: noise and task interruptions. Noise refers to recurrent salient sounds frequently occurring in ER: phone ringing, drilling, helicopter landing, baby crying, medical equipment alarm pinging, and thunder clapping. According to previous research [50, 51], these noises can interfere with nurse activities such as anamnesis and vital sign measurement. To stand out from the low-level background noise, these sounds effects are played at levels up to 80 dB. The use of headphones is recommended to have the sound environment more immersive. Task interruptions, for their part, are implemented via modal pop-ups, such as the image of a disturbing person or a telephone. The player then must think of a response to type into a text pop-up. Unbeknownst to the players, their responses are irrelevant. What counts is returning to the triage task to complete it after being interrupted. Again, research has shown that task interruptions have a disruptive effect on the performance of cognitive processes[49, 52, 53]. Both distractors are programmed to occur at random during the game.

At the technical level, the SGTRI is a browser game developed using Wegas [54], an execution platform that supports the generation of trace data to systematically record all of the user’s choices. This solution provides very extended raw output that includes any single operation the user performs. This makes it possible to have highly refined learning analytics[55], for example, to compare problem-solving strategies [56]. This being a web-based game, technological constraints exist. An extended description of the SG, along with a few images, can be found in Delmas et al. [57] and Hulaas et al. [58] Pre-tests to identify possible technical shortcomings were conducted on-site in all the hospitals that participated in our study.

Objectives

We conducted a pilot study with a twofold objective.

First, we wanted to evaluate the feasibility and acceptability of using the SG we developed to reproduce an immersive work environment, complete with distractors, in which to simulate the triage activity in an ER. The results, it was hoped, would serve to refine both the SG and the research protocol for a subsequent randomized controlled trial (RCT).

Second, we sought to collect preliminary data on the effects of two distractors, noise and task interruptions, on ER nurse triage accuracy. These would then help us refine the statistical instruments to be used in the RCT.

Design

To these ends, we structured the pilot study as a 2 × 2 factorial RCT. This design was dictated by the two distractors that we wished to test separately and jointly[59]. This made for four conditions: (a) no distractor (control); (b) “noise” intervention; (c) “interruption” intervention; and (d) both “noise and interruption” intervention. Nurses were distributed across these by block randomization. The aim was to have four groups as equal in size as possible.

The study comprised both cross-sectional and longitudinal measures. Sociodemographic and personal data were collected from participants only once prior to starting the SG. Performance data were evaluated after each vignette. Moreover, nurses were asked to rate how confident they were of each emergency level assigned.

Population and sample

The triage nurses were recruited in the 20 ERs in the five cantons of the French-speaking part of Switzerland (Geneva, Vaud, Fribourg, Jura, and Neuchâtel) where the SETS® was used. We set the same two eligibility criteria as were used in previous studies of triage accuracy [27, 42, 60] (1) nurses had to consent to participate; and (2) nurses had to perform triage in an ER where an emergency triage scale was used, in our case, the SETS®.

Using power analysis, we calculated a minimum sample size of 1396 vignettes completed, assuming a triage accuracy rate of 0.85 for the control group [17], a minimum decline of 0.15 for the experimental groups, and a statistical power of at least 0.80. Assuming that each nurse would complete at least 20 vignettes, we needed a total sample of 70 nurses. After completing the recruitment process, 77 nurses consented to participate in the study.

Procedure

Participants were each assigned an ID number for data collection purposes. They received 30 minutes training on how to use the SG, in order to familiarize themselves with the equipment (headphones, laptop). For the actual test, nurses had to complete as many vignettes as possible in two hours. For each vignette, they had to assess the situation by questioning the patient or using instruments that the SG provided, which were the same as those generally used in ERs for triage purposes. Nurses completed each evaluation by formulating a chief complaint and assigning an emergency level. As the vignettes were prepared in advance, a group of experts had already established the correct chief complaint and the correct emergency level. Nurse performance was evaluated by comparing their answers against this gold standard. After the evaluation, a visual scale was shown to nurses, and they were able to move a cursor to indicate if they are confident of their evaluation or not. Putting the cursor completely on the left indicates no confidence at all, while putting it completely on the right indicates full confidence.

Instruments

Sociodemographic data was collected through a questionnaire developed on the basis of elements gathered in previous studies of triage accuracy[27, 42, 61].The questionnaire covered both personal characteristics (sex, age, family structure) and professional status (job type, total years of experience in profession, total years in current unit).

To record nurses’ confidence in their evaluation a visual scale was used. The position of the cursor on the scale is automatically translated in a digit where 0 indicates no confidence at all, while 100 indicates full confidence.

The other instruments used were directly linked to the study’s objectives.

Feasibility was assessed as prescribed by Sidani and Braden[62] and Feeley et al. [63] .Our procedure included evaluating accessibility of target population, appropriateness of inclusion and exclusion criteria, participation rate, withdrawal rate after starting the test, presence, and frequency of problems during delivery (comprehension, utilization, clarity), presence and frequency of missing data and outliers, and participant satisfaction.

The French version of the self-administered AttrakDiff 2 inventory [64]was used to evaluate the acceptability of the SG. Initially developed in German by Hassenzahl et al. [65] ], this instrument consists of a 28-item scale that evaluates the hedonic and pragmatic qualities of interactive systems. It comprises four seven-item subscales: usability, functionality, social impact, and attractiveness. For each item, nurses had to rate a quality on a semantic differential scale ranging from − 3 to + 3, anchored by a pair of antonyms. A rating of 0 or + 1 was considered neutral; +2 or + 3 was considered positive; and − 1, -2 or -3 was considered negative. Negative ratings would suggest that the SG needed improving. The French-language version of the AttrakDiff 2 has been shown to have satisfactory psychometric properties, with a Cronbach’s α of 0.75 for each of its dimensions [64]

As for the preliminary data on the effects of distractors on nurse triage performance, the nurses assigned an emergency level to each vignette according to the SETS® criteria on a scale of 1, a vital emergency, to 4, a non-urgent case.

Data analysis

First, the database was prepared for analysis by verifying compliance with inclusion criteria and identifying missing data and outliers. Second, we carried out descriptive analyses by examining the distribution of all the variables considered: frequency for categorical variables and mean, standard deviation, median, and interquartile range for numeric variables.

Third, triage accuracy was measured by the level of agreement between the emergency levels assigned by the nurses and the pre-established gold standard defined by a panel of experts. This operation resulted in a categorical variable with three categories: accurate triage (nurse rating same as gold standard), over-triage (rating higher than gold standard), and under-triage (rating lower than gold standard). Fourth, random intercept multinomial regression models were used to study the individual and combined effects of the distractors on triage accuracy. For these analyses, triage accuracy was furtherly reduced to two categories: accurate or inaccurate. The use of random intercept models was necessary owing to the presence of clusters as the nurses were naturally grouped into work units.

For all the tests, a frequentist approach was used with a statistical significance threshold of p = 0.05. All the data were analyzed using the R statistical software[66].

Unfortunately, after managing to recruit 77 nurses, only 55 participated in the study as 22 were unexpectedly no longer available the day of the test on account of work obligations. The nurses that did participate in the study completed a total of 828 vignettes. All other things being equal, power calculation indicated that a minimum difference of 0.19 would be detectable with such a sample. This was higher than the initial 0.15, but we deemed this was still an acceptable value.

Our final sample of 55 nurses had the sociodemographic characteristics reported in Table 1. They were relatively young but already had a number of years of nursing and triage experience, and 80% had received specific training in triage. Two out of three were women and more than 60% had studied in France. Only 18.2% had a bachelor’s degree.

Table 1 – Sociodemographic characteristics of our sample

Regarding the feasibility and acceptability of our research protocol, a number of points were considered. First, the ER nurses in our study indicated no problem understanding and using the SGTRI. The information session and the hands-on practice before the test were successful in familiarizing nurses with the SG and the hardware. Second, all the nurses were able to complete the test. None of the nurses required a break and no test had to be cut short owing to nurse fatigue or stress generated by the SGTRI, but some nurses did ask for clarification about the use of keywords a few times. Third, organization-wise, we encountered a few issues with a number of nurses who were unable to participate on account of last-minute work requirements. This occurred because of how the test was organized, specifically, because of our choice to hold the tests just before or just after a work shift.

Furthermore, the average number of vignettes completed in a two-hour span was 15.8 (SD=3.68), which suggests that 15 is a more realistic goal than the 20 aimed for. Our initial estimate was based on pre-tests done with our SGTRI.

Regarding the feasibility and acceptability of the SG, a number of elements were considered. First, minor technical issues arose in 14 cases owing to unplanned input from the use of a multi-button mouse or to the wrong cookie setting on the web browser. These were resolved by refreshing the webpage and measures were taken to avoid their repeat. Second, there were no issues reported with the hardware. Third, no nurse found it so hard to understand how to complete the task as to result in missing data. Fourth, no nurse requested further information from the members of the research team on hand for the duration of the test.

Finally, regarding user experience and evaluation of the SG, the nurses scored all of the dimensions explored with the AttrakDiff 2 on the positive side, which is to say >1 (Table 2). The hedonic dimensions scored highest were those related to the simulation, followed closely by global attractiveness, which suggests that the SG was very well received by the users. The pragmatic dimensions, which measured SG usability, were scored lower but nonetheless >1, as were the hedonic-identity dimensions, which measured user identification with the simulation.

Table 2 – values of the AttrakDiff 2 dimensions

Finally, regarding nurse triage performance, the participants assigned the correct emergency level 89% of the time. Across the vignettes, accuracy ranged from 51% to 100%. Nurses under-triaged (maximum 23% and minimum 0%) and over-triaged (maximum 49% and minimum 0%) rather evenly, on average 5% and 6% of the time, respectively.

Regarding the effects of distractors on triage accuracy, the results of our random intercept regression analysis (Table 3) showed no difference across the groups exposed to noise, task interruptions, or both, comparing with the control group exposed to no distractors.

Table 3 – Differences between experimental and control groups

Our pilot study had a dual objective: to investigate the acceptability and feasibility of our study protocol and our SG; and to gather preliminary data on the effects of two distractors—noise and task interruptions—on ER nurse triage accuracy.

While the feedback on the feasibility and acceptability of the structure of our study was positive on the whole, some issues did arise. Generally speaking, all the participants were able to understand and perform the task asked of them. However, where time was concerned, 20 vignettes were too many to complete in two hours. To our surprise, there was a sizeable difference between the time required to complete a real-life visit and the time required to complete a simulated vignette on our SG. One reason for this might be the time needed to become familiar with a new instrument before managing to use it efficiently. Despite the hands-on practice prior to the trial, participants still needed time to adjust once the actual test got underway. It generally took five vignettes for this to happen. Another reason may be a major difference in the SG triage process. Unlike the process in real life, nurses had to enter the patient’s chief complaint as a number code rather than textually. A further cause for the time discrepancy might be the Hawthorne effect[67]. Nurses knew that they were participating in an experiment despite the immersivity of simulated environment. Consequently, nurses might have taken extra care to perform well, consciously, or not, thereby slowing down the process. Finally, the nurses participated in the study before or after a shift. The stress caused by the situation could be another factor that slowed nurses down. To avoid this situation, testing would need to be better coordinated with work time in future. This solution would also reduce nurse fatigue, which we observed in a few cases, and would improve the enrollment process.

In light of the above, for our future RCT, we decided that we would stress the time limit of the simulation more emphatically at the start and reduce to 15 the number of vignettes to be completed in two hours.

A rather consequential issue arose at the organizational level. We suffered a large loss to our sample because we had chosen to hold the simulation right before or after work shifts. This exposed our study to changes in work schedules and to the absence of some nurses who might have had a problem participating in the study immediately before or after their shift, especially if they had a long commute to or from work. Consequently, we will hold our future RCT during work hours even if this increases the cost of the exercise.

Regarding the acceptability and feasibility of our SGTRI, the feedback we collected was good. There were a few minor technical glitches, but these resulted in very little loss of time for the nurses who experienced them. We were already aware of these issues before the test, and our technical team has since corrected the problems. The first had to do with the use of a mouse with more than two buttons. With the first versions of our software, we failed to realize that clicking on the wrong button resulted in an unwanted shift in the web page. This bug has since been fixed. A second issue had to do with browser settings that interfered with the SGTRI. We have prepared clearer directions on how to adjust these settings for full compatibility with the SGTRI. Closer attention will be paid to this matter before the start of the future RCT.

Regarding user appreciation of the SGTRI as measured with the AttrakDiff 2, all of the dimensions considered were scored above 1. The hedonic dimension and global attractiveness obtained the highest scores. This suggests that users perceive the SGTRI to be both functional and aesthetically pleasant. The results on the whole validate using the SGTRI in a future RCT. Only a few cosmetic changes will be made (e.g., graphic improvements) and the pool of vignettes to choose from will be expanded in order to prolong the SGTRI’s longevity.

Our second objective was to collect preliminary data on the effects of two distractors—noise and task interruptions—on nurse triage performance. The results in this regard were clear: No effect was observed. The nurses in the groups exposed to noise, task interruptions, or both made no more mistakes in assigning emergency levels than did the nurses in the control group exposed to no distractors. There are three possible explanations for this result. First, because our sample was rather small, we might not have had sufficient power to intercept a very small effect. There is no ruling this out entirely, but the measured values suggest that, once other confounding variables are integrated into the model, inter-group differences are next to nil. A second explanation has to do with how we sought to distract nurses with noise. We exposed them to 10 different types of noise over two hours. Perhaps this was not enough to destabilize them. Perhaps a more prolonged exposure would have produced a clearer effect. However, it is essential for this exposure to remain within the limits of realism, and we must be careful not to overexpose users which, among other things, could induce excessive stress. Determining the right intensity and frequency of distractors remains to be investigated. Finally, we must not rule out the possibility that results simply reflect reality. Research on the effects of distractors on ER nurse triage performance is still fragmentary (36). It could be that distractors have little or no effect on performance, at least in our context. If so, this would nevertheless be a finding.

As our study has not without some limitations. First, as always in research experiments, the Hawthorne effect could have been at play during data collection. The presence of investigators during the sessions could have led the nurses to be more careful and rigorous than usual when performing triage. Second, some technical issues were noted during data collection. Third, we ended up with a smaller sample size than planned for, which decreased the study’s statistical power.

We plan to conduct a RCT to study the effects of two distractors—noise and task interruptions—on ER nurse triage accuracy. To do so, we created a SGTRI simulating an ER in a 2D environment where nurses have to diagnose patients and determine their emergency level in a series of vignettes. Before going through with the RCT, we needed to test the acceptability and feasibility of both the structure of our study and our SGTRI. We also sought to collect preliminary data on the possible effects of distractors. Consequently, we organized a pilot study, structured as a small 2 × 2 factorial RCT, where 55 nurses were distributed across four groups: one exposed to noise, one to task interruptions, one to both, and a control group exposed to no distractors.

Our results show that both the structure of our study and the SGTRI can easily be used to pursue our objectives and support a future RCT. Some small changes will be necessary, though, notably: Two minor technical bugs were identified (and already corrected), the sequence of vignettes to complete in two hours must be shortened from 20 to 15, and the test should be held during work hours to support recruitment and reduce the impact of nurse fatigue as a possible confounding factor. As for the preliminary data on the effects of distractors on nurse triage performance, no differences emerged across the four groups. This raises questions about whether distractors actually have an effect or not, at what dose noise distractors might have an impact on nurse performance, and whether this dose is realistic.

ER: Emergency Room

SG: Serious Game

SGTRI: Serious Game Triage

Ethics approval and consent to participate.

All methods were carried out in accordance with relevant guidelines and regulations. The study was approved by the Canton of Vaud Research Ethics Board (CER-VD). All participants provided written informed consent to participate in the study.

Consent for publication

Not applicable.

Availability of data and materials

The datasets generated and/or analyzed during this study are not publicly available owing to privacy regulations but are available from the corresponding author on reasonable request.

Competing interests

The authors declare that they have no competing interests.

Funding

This study was financed by a health research fund of the University of Applied Sciences and Art Western Switzerland (HES-SO) (Reference: 01-O18).

Authors' contributions

P.D, A.F, M.A, O.H, S.V, and J.S designed the study, D.J, A.K, and G.S developed the SGTRI; M.A, P.D, J.S, and A.F collected the data; J.P analyzed the data; J.S, P.D, M.A, J.P,D.J, O.H, S.V, and A.F interpreted the data; M.A, A.F, and P.D contributed to drafting and editing the manuscript. All authors read and approved the final manuscript.

Acknowledgements

We are grateful to all the emergency departments for supporting and participating in this study. We are grateful also to Swiss Emergency Group Triage for supporting the use of SETS®

Runciman W, Hibbert P, Thomson R, Van Der Schaaf T, Sherman H, Lewalle P. Towards an International Classification for Patient Safety: key concepts and terms. Int J Qual Health Care. 2009;21(1):18–26. https://doi.org/10.1093/intqhc/mzn057.
Secrétariat International des Infirmières et Infirmiers de l'Espace Francophone. La qualité de soins et la sécurité des patients: une priorité mondiale [Quality of care and patient safety: a global priority]. In. Montréal,Québec: SIDIIEF; 2015. pp. 1–51.
Chung JY. An exploration of accident and emergency nurse experiences of triage decision making in Hong Kong. Accid Emerg Nurs. 2005;13(4):206–13. https://doi.org/10.1016/j.aaen.2005.08.003.
Hitchcock M, Gillespie B, Crilly J, Chaboyer W. Triage: an investigation of the process and potential vulnerabilities. J Adv Nurs. 2014;70(7):1532–41. https://doi.org/10.1111/jan.12304.
van der Linden C, Lindeboom R, van der Linden N, Lucas C. Managing patient flow with triage streaming to identify patients for Dutch emergency nurse practitioners. Int Emerg Nurs. 2012;20(2):52–7. https://doi.org/10.1016/j.ienj.2016.02.003.
Croskerry P. Achieving quality in clinical decision making: cognitive strategies and detection of bias. Acad Emerg Med. 2002;9(11):1184–204. https://doi.org/10.1111/j.1553-2712.2002.tb01574.x.
Tiffen J, Corbridge SJ, Slimmer L. Enhancing clinical decision making: development of a contiguous definition and conceptual framework. J Prof Nurs. 2014;30(5):399–405. https://doi.org/10.1016/j.profnurs.2014.01.006.
Hughes RG. Advances in Patient Safety Nurses at the "Sharp End" of Patient Care. In: Patient Safety and Quality: An Evidence-Based Handbook for Nurses edn. Edited by Hughes RG. Rockville (MD):: Agency for Healthcare Research and Quality (US). 2008.
Chang A, Schyve PM, Croteau RJ, O'Leary DS, Loeb JM. The JCAHO patient safety event taxonomy: a standardized terminology and classification schema for near misses and adverse events. Int J Qual Health Care. 2005;17(2):95–105. https://doi.org/10.1093/intqhc/mzi021.
Yurkova I, Wolf L. Under-triage as a Significant Factor Affecting Transfer Time between the Emergency Department and the Intensive Care Unit. J Emerg Nurs. 2011;37(5):491–6. https://doi.org/10.1016/j.jen.2011.01.016.
Rutschmann OT, Kossovsky M, Geissbuhler A, Perneger TV, Vermeulen B, Simon J, Sarasin FP. Interactive triage simulator revealed important variability in both process and outcome of emergency triage. J Clin Epidemiol. 2006;59(6):615–21. https://doi.org/10.1016/j.jclinepi.2005.11.003.
Beveridge R, Ducharme J, Janes L, Beaulieu S, Walter S. Reliability of the Canadian Emergency Department Triage and Acuity Scale: Interrater Agreement. Ann Emerg Med. 1999;34(2):155–9. https://doi.org/10.1016/s0196-0644(99)70223-4.
Fernandes CM, McLeod S, Krause J, Shah A, Jewell J, Smith B, Rollins L. Reliability of the Canadian Triage and Acuity Scale: interrater and intrarater agreement from a community and an academic emergency department. CJEM. 2013;15(4):227–32. https://doi.org/10.2310/8000.2013.130943.
van der Wulp I, van Baar ME, Schrijvers AJ. Reliability and validity of the Manchester Triage System in a general emergency department patient population in the Netherlands: results of a simulation study. Emerg Med J. 2008;25(7):431–4. https://doi:10.1136/emj.2007.055228.
Ebrahimi M, Heydari A, Mazlom R, Mirhaghi A. The reliability of the Australasian Triage Scale: a meta-analysis. World J Emerg Med. 2015;6(2):94–9. https://doi:10.5847/wjem.j.1920-8642.2015.02.002.
Rutschmann OT, Sieber RS, Hugli OW. Recommandations de la Société Suisse de Médecine d’Urgence et de Sauvetage pour le triage dans les services d’urgence shospitaliers en Suisse.[ Recommendations of the Swiss Society for Emergency Medicine and Rescue for triage in hospital emergency departments in Switzerland. ]Bull Med Suisses. 2009;90(46):1789–90.
Rutschmann OT, Hugli OW, Marti C, et al. Reliability of the revised Swiss Emergency Triage Scale: a computer simulation study. Eur J Emerg Med. 2018;25(4):264–9. https://doi:10.1097/MEJ.0000000000000449.
Dallaire C, Poitras J, Aubin K, Lavoie A, Moore L. Emergency department triage: do experienced nurses agree on triage scores? J Emerg Med. 2012;42(6):736–40. https://doi:10.1016/j.jemermed.2011.05.085.
Taboulet P, Moreira V, Haas L, et al. Triage with the French Emergency Nurses Classification in Hospital scale: reliability and validity. Eur J Emerg Med. 2009;16(2):61–7. https://doi:10.1097/MEJ.0b013e328304ae57.
Alpert EA, Lipsky AM, Hertz D, Rieck J, Or J. Simulated evaluation of two triage scales in an emergency department in Israel. Eur J Emerg Med. 2013;20(6):431–4. https://doi:10.1097/MEJ.0b013e32835e2b42.
Funderburke P. Exploring best practice for triage. J Emerg Nurs. 2008;34(2):180–2. https.//doi:10.1016/j.jen.2007.11.013.
Farrohknia N, Castrén M, Ehrenberg A, et al. Emergency department triage scales and their components: a systematic review of the scientific evidence. Scand J Trauma Resusc Emerg Med. 2011;19:42. https://doi:10.1186/1757-7241-19-42.
Brown AM, Clarke DE. Reducing uncertainty in triaging mental health presentations: examining triage decision-making. Int Emerg Nurs. 2014;22(1):47–51. https://doi:10.1016/j.ienj.2013.01.005.
Perry ST, Wears RL, Croskerry P, Shapiro MJ. Process Improvement and Patient Safety. In: Rosen's emergency medicine: concepts and clinical practice edn. Edited by J. A. Marx RSH, R. M. Walls, & M. H. Biros. Philadelphia: Elsevier Saunders; 2014: 2505.
Clarke DE, Boyce-Gaudreau K, Sanderson A, Baker JA ED, editors. Triage Decision-Making with Mental Health Presentations: A “Think Aloud” Study. J. Emerg. Nurs. 2015;41:496–502. https://doi:10.1016/j.jen.2015.04.016.
Stanfield LM. Clinical Decision Making in Triage: An Integrative Review. J Emerg Nurs. 2015;41(5):396–403. https://doi:10.1016/j.jen.2015.02.003.
Gerdtz MF, Chu M, Collins M, et al. Factors influencing consistency of triage using the Australasian Triage Scale: implications for guideline development. Emerg Med Australas. 2009;21(4):277–85. htpps://doi:10.1111/j.1742-6723.2009.01197.x.
Considine J, Botti M, Thomas S. Do knowledge and experience have specific roles in triage decision-making? Acad Emerg Med. 2007;14(8):722–6. https://doi:10.1197/j.aem.2007.04.015.
Wolf L. Research in triage?!?: using an integrated, ethically driven framework for clinical decision making. J Emerg Nurs. 2013;39(1):104–6. https://doi:10.1016/j.jen.2012.09.009.
Andersson AK, Omberg M, Svedlund M. Triage in the emergency department–a qualitative study of the factors which nurses consider when making decisions. Nurs Crit Care. 2006;11(3):136–45. htpps://doi:10.1111/j.1362-1017.2006.00162.x.
Broadbent M, Moxham L, Dwyer T. Implications of the emergency department triage environment on triage practice for clients with a mental illness at triage in an Australian context. Australas Emerg Nurs J. 2014;17(1):23–9. https.//doi:10.1016/j.aenj.2013.11.002.
Johnson KD, Motavalli M, Gray D, Kuehn C. Causes and occurrences of interruptions during ED triage. J Emerg Nurs. 2014;40(5):434–9. https://doi:10.1016/j.jen.2013.06.019.
Kosits LM, Jones K. Interruptions experienced by registered nurses working in the emergency department. J Emerg Nurs. 2011;37(1):3–8. https://doi:10.1016/j.jen.2009.12.024.
Person J, Spiva L, Hart P. The culture of an emergency department: an ethnographic study. Int Emerg Nurs. 2013;21(4):222–7. 10.1016/j.ienj.2012.10.001.
Chisholm CD, Dornfeld AM, Nelson DR, Cordell WH. Work interrupted: a comparison of workplace interruptions in emergency departments and primary care offices. Ann Emerg Med. 2001;38(2):146–51. https://doi:10.1067/mem.2001.115440.
Johnson KD, Alhaj-Ali A. Using Simulation to Assess the Impact of Triage Interruptions. J Emerg Nurs. 2017;43(5):435–43. https://doi:10.1016/j.jen.2017.04.008.
Jordi K, Grossmann F, Gaddis GM, et al. Nurses' accuracy and self-perceived ability using the Emergency Severity Index triage tool: a cross-sectional study in four Swiss hospitals. Scand J Trauma Resusc Emerg Med. 2015;23:62. https://doi:10.1186/s13049-015-0142-y.
Alvarez J, Djaouti D. Introduction au serious game. 2–me ed. edn. imprimerie Gantier: Questions théoriques; 2012.
de Ribaupierre S, Kapralos B, Stroulia E, Dubrowski A, Eagleson R. Healthcare Training Enhancement Through Vistrual Reality and Serious Games. In: Virtual Augmented Reality and Serious Games for Healthcare 1. Volume 68, edn. Edited by Ma M, Jain LC, Anderson P: Springer-Verlag Berlin Heidelberg; 2014.
Freese M, Drees S. D-CITE - A Serious Game to Analyze Complex Decision-Making in Air Traffic Management. In: Games and Learning Alliance: 2016// 2016; Cham: Springer International Publishing; 2016: 23–31.
Hulst A, Muller T, Buiel E, van Gelooven D, Ruijsendaal M. Serious gaming for complex decision making: Training approaches. Int J Technol Enhanced Learn. 2014;6:249.
Mistry B, Stewart De Ramirez S, Kelen G, et al. Accuracy and Reliability of Emergency Department Triage Using the Emergency Severity Index: An International Multicenter Assessment. Ann Emerg Med. 2018;71(5):581–587e3. https://doi:10.1016/j.annemergmed.2017.09.036.
Chiniara G, Cole G, Brisbin K, Huffman D, Cragg B, Lamacchia M, Norman D. Simulation in healthcare: a taxonomy and a conceptual framework for instructional design and media selection. Med Teach. 2013;35(8):e1380–1395.
Cant RP, Cooper SJ. Simulation in the Internet age: the place of web-based simulation in nursing education. An integrative review. Nurse Educ Today. 2014;34(12):1435–42.
Carayon P, Schoofs Hundt A, Karsh BT, Gurses AP, Alvarado CJ, Smith M, Flatley Brennan P. Work system design for patient safety: the SEIPS model. Qual Saf Health Care. 2006;15(Suppl 1):i50–58.
Carayon P, Wetterneck TB, Rivera-Rodriguez AJ, Hundt AS, Hoonakker P, Holden R, Gurses AP. Human factors systems approach to healthcare quality and patient safety. Appl Ergon2014, 45(1):14–25.
Donabedian A. The quality of care. How can it be assessed? Jama. 1988;260(12):1743–8.
Morrison JB, Rudolph JW. Learning from accident and error: avoiding the hazards of workload, stress, and routine interruptions in the emergency department. Acad Emerg Med. 2011;18(12):1246–54. https://doi:10.1111/j.1553-2712.2011.01231.x.
Zun LS, Downey L. The effect of noise in the emergency department. Acad Emerg Med. 2005;12(7):663–6. https://doi:10.1197/j.aem.2005.03.533.
Graneto J, Damm T. Perception of noise by emergency department nurses. West J Emerg Med. 2013;14(5):547–50. https://doi:10.5811/westjem.2013.5.16215.
Berg LM, Källberg AS, Ehrenberg A, et al. Factors influencing clinicians' perceptions of interruptions as disturbing or non-disturbing: A qualitative study. Int Emerg Nurs. 2016;27:11–6. https://doi:10.1016/j.ienj.2016.01.003.
Rivera-Rodriguez AJ, Karsh BT. Interruptions and distractions in healthcare: review and reappraisal. Qual Saf Health Care. 2010;19(4):304–12. https://doi:10.1136/qshc.2009.033282.
Laurent M, Junod C, Hulaas J, Jaccard D, Aeberhard F-X. Wegas authoring system.; Yverdon, Switzerland, 2017. https://wegas.albasim.ch/.
Ferguson R. Learning analytics: drivers, developments and challenges. Int J Technol Enhanc Learn. 2012;4(5/6):304–17.
Jaccard D, Hulaas J, Dumont A. Using Comparative Behavior Analysis to Improve the Impact of Serious Games on Students’ Learning Experience. In: 2016; Cham: Springer International Publishing; 2016: 199–210.
Delmas P, Fiorentino A, Antonini M, Vuilleumier S, Stotzer G, Kollbrunner A, Jaccard D, Hulaas J, Rutschmann O, Simon J, et al. Effects of environmental distractors on nurse emergency triage accuracy: a pilot study protocol. Pilot Feasibility Stud. 2020;6(1):171.
Hulaas J, Jaccard D, Fiorentino A, Delmas P, Antonini M, Vuilleumier S, Stotzer G, Kollbrunner A, Rutschmann O, Simon J et al. A Serious Game for Studying Decision Making by Triage Nurses Under Stress. In: 2020; Cham: Springer International Publishing; 2020: 253–262.
Grove SK, Burns N, Gray J. The Practice of Nursing Research: Appraisal, Synthesis, and Generation of Evidence. 7 ed. Chine: Elsevier/Saunders; 2012.
Goransson KE, von Rosen A. Interrater agreement: a comparison between two emergency department triage scales. Eur J Emerg Med. 2011;18(2):68–72. https://doi:10.1097/MEJ.0b013e32833ce4eb.
Chen SS, Chen JC, Ng CJ, Chen PL, Lee PH, Chang WY. Factors that influence the accuracy of triage nurses' judgement in emergency departments. Emerg Med J. 2010;27(6):451–5.
Sidani S, Braden CJ. Design, Evaluation, and Translation of Nursing Interventions. Wiley-Blackwell; 2011.
Feeley N, Cossette S, Côté J, et al. The importance of piloting an RCT intervention. Can J Nurs Res. 2009;41(2):85–99.
Lallemand C, Koenig V, Gronier G, Martin R. Création et validation d’une version française du questionnaire AttrakDiff pour l’évaluation de l’expérience utilisateur des systèmes interactifs. Revue Européenne de Psychologie Appliquée/European Review of Applied Psychology 2015, 65(5):239–52. https://doi.org/10.1016/j.erap.2015.08.002.
Hassenzahl M, Burmester M, Koller F. AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität. In: Mensch & Computer 2003: Interaktion in Bewegung. edn. Edited by Szwillus G, Ziegler J. Wiesbaden: Vieweg + Teubner Verlag; 2003: 187–196.
R Core Team. In: Vienna, editor. R: a language and environment for statistical computing. Austria: R Foundation for Statistical Computing; 2014.
Mayo E. The Human Problems of an Industrial Civilization. 1st ed. Routledge; 2003.

No competing interests reported.

Download PDF

Version 1

posted

You are reading this latest preprint version

Feasibility and acceptability of a serious game to study the effects of environmental distractors on emergency room nurse triage accuracy: a pilot randomized controlled trial

Status:

Version 1

Abstract

Background

Serious Game Triage (SGTRI)

Methods

Objectives

Design

Population and sample

Procedure

Instruments

Data analysis

Results

Discussion

Conclusions

Abbreviations

Declarations

References

Additional Declarations

Status:

Version 1