Study design
In order to evidence the effects of noise and task interruptions on nurse triage accuracy, we will carry out a 2x2 factorial randomized controlled trial (65). A factorial design will be used given that two independent variables will be considered (noise and task interruptions) and that we will evaluate not only the effect of each variables on the dependent variable (nurse triage accuracy), but also their combined effect (65, 66). The design will follow the CONSORT guidelines (67) and drive the structure of the trial and the choice of control, an analysis of study benefits, the quality and reliability of the intervention, a description of the population, the randomization procedure, and the statistical analysis plan.
This factorial design will allow us to create four groups: one control group and three experimental groups (A, B, C). While triaging the clinical vignettes, nurses in the control group will not be exposed to distractors. Nurses in experimental groups will be exposed to noise (group A), task interruptions (group B), or both noise and task interruptions (group C). Nurses will be block-randomized across the four groups by a computer program. The groups will be of equal size or as similar as possible in this regard.
The study design comprises repeated measures. We will collect sociodemographic and personal data from participants before they begin evaluating the clinical vignettes. Then, during the evaluation of each clinical vignette, the following data will be gathered systematically: 1) emergency level assigned; 2) level of perceived confidence in emergency level assignment; and 3) duration of each clinical vignette evaluation. Upon completing the evaluation of the 20 clinical vignettes, the participants will be asked to complete a questionnaire on the acceptability of the SG.
Population and sampling
This multi-site study will be carried out in EDs where the SETS® is used. This is the case in 20 private and public care facilities in the five cantons of the French-speaking part of Switzerland (Geneva, Vaud, Fribourg, Jura, Neuchâtel). The population will consist of nurses who perform triage in these facilities. This corresponds to an accessible population of 454 nurses. The eligibility criteria will be similar to those used in previous studies of triage accuracy (27, 35, 68). Specifically, to participate in our study, nurses must: 1) consent to participate and 2) perform triage in one of the EDs where the SETS® is used. Nurses will be block-randomized across the four groups. We established a size of 20 nurses per group, for a total of 80. Around 18% of the accessible population is included, a non-negligible percentage that leaves, nonetheless, an ample margin for the recruitment for the future randomized control trial.
This sample size allows the inclusion of, at least, few nurses from each hospital providing an overview of all the emergency units of the hospitals in French-speaking Switzerland. Therefore, we can assume that our sample provides a sufficient heterogeneity to test acceptability and feasibility of our protocol. Moreover, assuming an accurate triage rate of 0.85 for the control group, a decline of 0.1 in the experimental groups, and an intraclass correlation (that is, between vignettes triaged by a same nurse) of no more than 0.03, we estimated through simulations that each group would need to comprise at least 20 participants to obtain a power of at least 0.80 in order to answer the primary research question. Consequently, we will aim to form a convenience sample of at least 80 nurses, and we will cease recruitment once this target is reached.
Procedure
The study will follow a two-step procedure. The first step will be to develop the SG. This will be done by a multidisciplinary team comprising IT engineers and designers who will handle the technical elements of the SG and healthcare professionals who will create realistic content, that is, the clinical vignettes, the distractors and the visual appearance of each on-screen element. The second step will be to recruit participants in the EDs and deliver the SG.
a) Procedure for constructing the serious game and clinical vignettes
First, a pre-design session will be held to define the different steps in the construction of the SG called SGTRI and to describe the tasks to be performed by the game developers, the clinical partners and the research team. Second, the design section will cover all the elements required to recreate the ED triage environment, such as the graphic interface, clinical vignettes, adjunctive distractions, and triage tasks. The SGTRI will be designed on and operated from an open-source platform called Wegas (http://www.albasim.ch). To develop SGTRI and allow it to evolve, the research team will use a logical graphic interface that may include audio-visual elements adaptable to needs and scenarios. This graphic interface will consist of a virtual 2D waiting room that will be the stage for different animated clips where patients may arrive by ambulance or by foot and other healthcare workers (paramedics, doctor) may be present. A triage workstation will be recreated from 2D plans based on the ED triage observations of a designer on the research team. This triage station will be equipped with all the devices used by nurses under the circumstances (e.g., triage form, clock, computer).
The clinical vignettes will be developed in conjunction with an emergency clinical specialist nurse, an emergency medicine professor (the initiator of SETS® development), and a certified emergency nurse. All these experts have numerous years of experience in the emergency field and with the triage process. A series of 20 interactive clinical vignettes will be created based on a retrospective review of real cases in an emergency department. The clinical vignettes will be constructed following the three quality guidelines proposed by Evans et al. (69): 1) each vignette must simulate situations faced by participants, which will be the case in this study; 2) each vignette must be different and entail a specific decision to be made, which in our case will be to assign an emergency level; and 3) using well-designed vignettes must produce a highly generalizable “real-life” triage process. For each clinical vignette, the emergency level will be validated by mutual agreement by a group of four experts (two staff physicians and two nursing experts), in strict compliance with the criteria and definitions of the SETS® (20). This will constitute the gold standard. For our study, the clinical vignettes will involve the medico-surgical issues most encountered in ED and all four emergency levels of the SETS will be covered. To create an immersive ED environment, we will select noises and task interruptions based on real-time observations and recordings by members of the research team in different ED triage settings and on a review of the scientific literature, using the instrument developed by Johnson and colleagues (70) for classifying task interruptions during nurse triage.
In each experimental group (A, B, C), 10 interruptions and/or noises will be introduced in different clinical vignettes. The distribution of task interruptions (type, number and duration) will follow a predetermined sequence generated by the researchers. The SG will leave the nurse participants the choice of responding or not to some task interruptions (e.g., an incoming telephone call) but will require them to respond to others (e.g., a patient inquiring about the wait time). The noise will correspond to the soundscape (observed values) of triage stations. The research team will modulate the noise exposure condition by varying the form, length of exposure, and intensity of the ambient noise (e.g., conversation, telephone ringtone). The intensity of the noise exposure will range from 35 dB (A) to 85 dB (A), the maximum level at which no auditory protection is required (71). To immerse nurses in the created soundscape and eliminate extraneous noise, nurses will be required to wear headphones during the SG session.
Lastly, triage nurse activities will be identified and designed to correspond as much as possible to real-life tasks. These will be the most common tasks performed by nurses at triage, such as taking the patient’s clinical history through a list of predetermined questions (e.g., Are you in pain?), measuring vital signs (e.g., blood pressure), recording patient clinical values, and transcribing the emergency level and chief health complaint. Once the SGTRI is designed, a pre-test will be planned with eight clinical emergency experts. The aim of the pre-test will be to assess all the processes to correctly implement the SG during the recruitment phase.
b) Procedure for participant recruitment and SG delivery
First, the aims of the research will be presented at the annual meeting of ED SETS® users. Second, all the care facilities with an emergency unit in the five cantons of the French-speaking part of Switzerland (Geneva, Vaud, Fribourg, Jura, Neuchâtel) will be contacted to validate their interest in participating in the study. Third, nurses interested in participating in the study will receive an information and consent form. The definitive list of participants will be drawn up after consent forms are signed. Nurses will be block-randomized across the four groups just before starting the SG by order of arrival for the test. Each participant will then be assigned an identification number when they start the SG. Fourth, the study data will be collected directly in each ED by the research team. During the session, the research team will be on site to ensure nurse participants are correctly assigned and to ensure fidelity of SG session delivery.
SG delivery will comprise four stages. First, participants will receive a 30-minute research project information and training session led by a member of the research team. Second, the participants will run through a training session composed of two clinical vignettes that will not be included in the analyses. During this session, the participants will familiarize themselves with the equipment (headphones, laptop) provided by the members of the research team. Third, once the training session is completed, each nurse will be able to start their SG session when ready. The nurses will have two hours to complete the 20 clinical vignettes. This corresponds to the average number of patients triaged at an ED over two hours. To establish a controlled set for the SG, participants will be isolated in a meeting room previously prepared by the head nurses in each participating emergency unit. Members of the research team will be on site to provide technical support, if needed, and to document any technical problem that might occur during the SG. Fourth, once the two hours have elapsed, each participant will stop the game session even if not all 20 vignettes have been evaluated. All the data collected during the SG session will be automatically recorded and they will be saved on a secured server located in Switzerland. After their two-hour SG session, the participants will return to their workplace.
Instruments
Sociodemographic data, both personal (gender, age, family situation) and professional (employment status, total number of years of experience, number of years in current department), will be collected through a questionnaire developed on the basis of elements gathered in previous studies of triage accuracy (27, 35, 72).
The clinical decision making of the nurse participants will be judged on the emergency level that they assign based on the SETS® criteria. Following their clinical reasoning, nurses assign patients an emergency level from 1 to 4. The scale has been the focus of various independent studies (20, 73) where computerized clinical vignettes were used with ED nurses and paramedics.
Nurse level of confidence in their clinical decision making will be measured using a visual analogue scale from 0 to 100 (74). This scale will be presented to nurses after each emergency level assignment. The question asked will be: “Now that you have completed this clinical vignette, how confident are you of the emergency level that you have assigned?” Nurses will rate their confidence from 0 to 100, with 0 corresponding to “I am not at all confident of my decision” and 100 to “I am fully confident of my decision”. Visual analogue scales allow measuring the intensity of a subjective experience and are widely used in clinical settings (66). In a study where the scale was used by 69 nurses in a triage situation, the researchers reported no problems with its utilization (35).
The feasibility of the SG will be assessed on the basis of criteria drawn from Sidani and Braden (44) and Feeley et al. (75), including accessibility of target population, appropriateness of inclusion and exclusion criteria, participation rate, withdrawal rate after starting SG, presence and frequency of problems during delivery of SG (understanding, utilization, clarity), presence and frequency of missing data and outliers, and participant satisfaction with SG.
The acceptability of the SG will be measured using a French version of the self-administered AttrakDiff 2 inventory (76) initially developed in German by Hassenzal and colleagues (77). This 28-item scale allows evaluating the hedonic and pragmatic qualities of interactive systems such as SG. Each item takes the form of a seven-point scale (-3 to +3) on which to rate a quality expressed by semantic differentials, that is, a pair of antonyms. It comprises four subscales: usability, functionality, social impact, and attractiveness. For each item, the respondent must choose between seven answers book-ended by the semantic differentials. A mean score and standard deviation are calculated for each dimension, taking account of certain inverted items (76). The values between 0 and 1 are considered neutral. Dimensions are deemed positive if scored between +1 and +3 and negative if scored between 0 and ‑3, in which case the SG needs to be improved. The psychometric properties (validity and reliability) of the French-language scale are entirely satisfactory, having obtained a Cronbach’s α of 0.75 for each of the dimensions (80). A supplementary question in the form of a visual analogue scale from 0 to 100 will be added to examine how realistic the nurses perceive the SG to be relative to their professional activity.
Data analysis plan
The nurse participants (expected N = 80) will be the analysis units for the descriptive analyses and the assigned triage scale scores will be the analysis units for the correlational analyses and some descriptive analyses (number of nurses multiplied by number of vignettes: expected N: 80 x 20 = 1600). The following data analysis plan will be carried out to answer the research questions: First, the collected data will be verified (compliance with inclusion criteria, identification of missing data and outliers). Second, the data on the nurses (sociodemographic and professional) will be analyzed via descriptive statistics, both univariate (mean, median, standard deviation, interquartile range, and absolute and relative frequency) and bivariate (contingency table and marginal frequency). Third, triage accuracy will be measured by the level of agreement between the answers given by the nurses and the gold standard established by the experts. For each nurse, the scores assigned to each clinical vignette will be compared against the gold standard. The results of the comparison will be a three-level multinomial variable: accurate triage (nurse score same as gold standard), overtriage (score higher than gold standard), and undertriage (score lower than gold standard). Over- and undertriage frequencies will be used to describe the triage accuracy of the four groups, that is, the control group and the three experimental groups: noise (A), task interruptions (B), and noise and task interruptions (C). Fourth, to examine the individual and combined effects of the distractors on the triage accuracy of the nurse participants, the groups will be compared against one another using a random-intercept multinomial regression model. For all the analyses, the statistical significance level will be p ≤ 0.05. All the data will be analyzed using the R statistical software (78).
Ethical considerations
Each nurse from the emergency units selected for the study will receive a written information letter explaining how the study will be conducted, what their participation entails, and what data protection measures will be taken. Each nurse will then be able to take all the time they need to decide whether to participate in the study, without the decision having any consequence whatsoever for their career. The research team will have access only to anonymous data. All data will be deleted after data analysis. To participate in the study, the nurses will have to sign a consent form, which will be stored in accordance with the recommendations of the Swiss Human Research Ethics Board (Canton of Vaud, Switzerland). The time that the nurses spend evaluating the clinical vignettes with the SG will count as work hours.