Study Design and Setting
This study was a randomized trial conducted in the Emergency and Trauma Department (ETD) of Sarawak General Hospital (SGH), Malaysia to compare the gamification approach versus the conventional approach for POCUS training using the RUSH protocol. In this study, we also included an assessment of the participants’ perception towards gamification approach in POCUS training. This study was approved by the Medical Research and Ethics Committee (MREC) Malaysia and was registered with the National Medical Research Register (NMRR-18-444-40348). As this study was conducted in a training workshop setting, the number of participants was limited by resource availability. Hence, a convenient sampling of 32 participants (16 participants in each arm) were recruited.
Study Population
The study population comprised of junior doctors working in (1) the ETD SGH, (2) Sarawak Heart Center and (3) the Internal Medicine Department of SGH. To recruit the participants, invitations were first sent out to the heads of departments of ETD SGH, Sarawak Heart Center and the Internal Medicine Department of SGH. The heads of departments would be given the onus to recommend participants for this workshop. Fourteen participants were recruited from the Internal Medicine Department of SGH, 13 participants came from the ETD SGH and another 5 participants were from the Sarawak Heart Center.
We defined a ‘junior doctor’ as a doctor with 2 to 4 years’ experience in clinical service. The reason for selecting doctors with 2 to 4 years of clinical experience was due to the fact that doctors with this amount of experience would have completed their compulsory two-year internship program in Malaysia and at the same time they would likely have developed sufficient clinical exposure to be able to utilize POCUS findings for clinical decision making.
Any junior doctor who had participated before in any formal POCUS training was excluded. Informed consent was obtained from all participants before commencing this study. All participants joined this free POCUS training on a voluntary basis without any payment or monetary compensation.
Materials
The topics for this POCUS training workshop were based on the requirements of the original RUSH protocol adopted from the World Interactive Network Focused on Critical Ultrasound (WINFOCUS) Malaysia course as well as some adaptation from the emergency ultrasound training from a post-graduate emergency medicine training program in Malaysia (i.e., the Universiti Malaya Emergency Medicine postgraduate curriculum). All materials were internally validated via a modified Delphi technique to attain consensus by a panel of experts in emergency medicine. Modified Delphi technique is a structured iterative technique to obtain consensus from experts through rounds of email, online, face-to-face communication until consensus is reached [14]. The experts were emergency physicians in Malaysia, two of which were actively involved in giving POCUS training.
Study Procedure
This study was divided into two stages. i.e., (1) to identify learning materials and development of assessment questions using the modified Delphi method and (2) to conduct recruitment, randomization and implementation of two different educational approaches in a POCUS training. In the first stage, the discussions were carried out with a panel of three experts in three rounds. Most of the discussions were carried out via e-mail and online group dialogues as the experts were based in different locations in Malaysia. The first round of discussions focused on identifying the main objectives and the probable topics of the workshop. In the second round of discussion, this compiled list of probable topics was distributed by email to the experts for review and to reach a consensus on the suitability of the topics. In the final round of discussions, the shortlisted topics were divided and assigned to the specific panel of experts for teaching as well as for preparation for the assessment questions. This assessment consisted of two sections: (1) 30 one-best answer (OBA) type of multiple-choice questions (for theory assessment), and (2) one objective structured clinical examination (OSCE) question (for practical assessment). The questions were then vetted, revised, finalized and agreed upon by the experts.
In the second stage of study, participant recruitment and randomization were conducted. Thirty-two participants were randomized to either the gamification group (known as the “CRUSH” group) or the conventional group. Each participant was first assigned a number. A free online random number generator (https://www.randomizer.org/) was then used to generate 2 sets of unique numbers. One set of numbers for the CRUSH group, and another set for the conventional group. The participants were then assigned to the different groups based on the numbers given to them earlier on.
On the first day of the course, all participants (regardless of which groups they were assigned to) were required to complete a pre-test knowledge (30 one-best answer (OBA) type of multiple-choice questions) and practical skills assessment test. The practical skills assessment was conducted in the style of the objective structured clinical examination (OSCE), conducted using a simulated patient and was assessed by three independent emergency physicians who were blinded to the participants’ study arms.
All participants then attended the classroom lectures on topics related to POCUS, interspersed with knowledge assessment activities. For participants in the conventional arm, this assessment was conducted using written-type quizzes. Whereas, for participants in the CRUSH arm, this assessment was conducted in the format of a team-based competition-like live quizzes with different level of difficulties. Points (or known as eXperience Points, XP) were allocated for each correct answer and the score was tabulated on a live leader board (Figure 1A). Each team began at level one with zero XP and they would require sufficient XP to progress to the next level. XP points could be gained by answering the live quizzes interspersed between lecture modules. Virtual badges were rewarded based on progression. If team was not able to answer the question correctly, another team was given a chance to answer but this team could only attain half the value of XPs even if they answered it correctly. The quiz was presented using a Jeopardy-style game show format using FlipQuiz™ technology (Figure 1B) whereby teams are able to pick the level of difficulty of questions with different points allocated. Feedback on the correct answers was given immediately to all teams after the stipulated time to answer the question was over. There was no negative marking in all quizzes.
On the second day, a demonstration session was first given to all participants (regardless of which groups they were assigned to) on the introduction to ultrasound machine, the probes and image acquisition. This is then followed by hands-on training sessions, conducted using simulated patients. For participants in the conventional group, these hands-on sessions were conducted in the form of the individual skill demonstration and practice sessions. No points or scores were awarded to the participants in the arm for these formative activities. For participants in the CRUSH arm, these hands-on skill trainings were conducted in the form of games, i.e., ultrasound minefield, ultrasound pong and ultrasound game. The psychomotor skill objectives covered in the hands-on practical sessions and the corresponding games for participants in the CRUSH arm are given in Table 1. At the end of the course, the team with the highest XPs score would win a reward. The XPs points collected, however, would not be carried into the calculation of the pre-test or post-test.
Table 1 The psychomotor skill objectives in hands-on practical sessions and the corresponding gamified version
Psychomotor objective
|
|
Gamified version
|
- To select and adjust for appropriate settings such as the gain and depth on the ultrasound machine
|
=>
|
Ultrasound minefield:
The main aim of the game was to be able to acquire the image and correctly calculate the number of water balloons hidden in a gelatin-filled container. The fastest team and closest to the correct number of water balloons would earn the highest points. This tested the participant’s psychomotor skills, selection of ultrasound probes and settings and teamwork (see Figure 1C and 1D)
|
2. To perform and obtain correct sonographic images of different body parts and organ systems;
|
=>
|
Ultrasound pong:
Labelled ping pong balls with written anatomical parts are placed in a bowl. Participants from each team are then required to randomly pick one and demonstrate the anatomy of the simulated patient. The main aim of the game is to be able to demonstrate the image of the anatomy being randomly chosen by chance. Each team member in the team of four would take turns, with points gained for every correct answer.
|
3. To perform a bedside POCUS based on a case scenario using simulated patients
|
=>
|
Ultrasound game:
Each team was given different simulated case scenarios and required to correctly give their assessment for the patient using the RUSH protocol. After assessing the simulated patient, participants were required to interpret the video clip leading to the diagnosis and management of the critically ill patient. Points are given for adequate view, correct ultrasound interpretation, diagnosis and management for each team.
|
After completing these hands-on sessions, a post-test (similar to the formats in pre-test, i.e., OBA questions for theory assessment and one OSCE scenario question, conducted using simulated patient, for practical skill assessment) was conducted. The overall flowchart of the 2-day POCUS workshop is given in Figure 2 and the detailed contents and schedule of the workshop is given in Table 2.
Table 2 Two-day schedule for POCUS training with the RUSH protocol for both CRUSH and conventional groups.
No
|
Day 1: Lecture Modules
|
Duration
|
1
|
Pre-test
|
1 hour
|
2
|
Lecture: Introduction to ultrasound physics and knobology
|
30 minutes
|
3
|
Lecture: Introduction to the RUSH Protocol
|
30 minutes
|
4
|
Lecture: Abdominal scan/FAST
|
30 minutes
|
CRUSH Group: Round 1 Team-based competition quiz
Conventional Group: Written-based quiz 1
|
15 minutes
|
5
|
Lecture: Lung ultrasound
|
30 minutes
|
6
|
Lecture: Cardiac ultrasound
|
30 minutes
|
CRUSH Group: Round 2 Team-based competition-based quiz
Conventional Group: Written-based quiz 2
|
15 minutes
|
7
|
Lecture: Vascular ultrasound (Inferior vena cava, Aorta, 2-point compression test)
|
30 minutes
|
8
|
Lecture: Putting it all together
|
30 minutes
|
CRUSH Group: Round 3 Team-based competition-based quiz
Conventional Group: Written-based question quiz 3
|
15 minutes
|
Station
|
Day 2: Hands-on Training
|
Duration
|
1
|
Demonstration session: Introduction to machine, probes and image acquisition
|
30 minutes
|
CRUSH Group: Team-based competition ultrasound game Round 1: Ultrasound Minefield
Conventional Group: Individual Skills Training: Acquiring image of water balloons
|
4 mins per team
1 min per individual
|
2
|
2A: Cardiac ultrasound in RUSH protocol
|
40 minutes
|
2B: Lung ultrasound in RUSH protocol
|
40 minutes
|
2C: Abdomen ultrasound/FAST in RUSH protocol
|
40 minutes
|
2D: Vascular ultrasound in RUSH protocol
|
40 minutes
|
CRUSH Group: Team-based competition-based ultrasound game Round 2: Ultrasound Pong
Conventional Group: Individual Skill Training: Anatomy
|
5 mins per team
1 min per individual
|
3
|
Simulated Case-based Scenarios of Undifferentiated Shock: Application of RUSH protocol.
CRUSH Group: Team competition-based ultrasound game Round 3 (10 minutes per team)
Conventional Group: Case-based classroom discussion
|
10 mins per team
|
3A: Case 1 - Cardiogenic Shock
|
10 mins
|
3B: Case 2 – Obstructive Shock
|
10 mins
|
3D: Case 3 – Hypovolemic Shock
|
10 mins
|
3E: Case 4 – Distributive Shock
|
10 mins
|
|
Post-test
|
1 hour
|
FAST: focused assessment with sonography for trauma.
To assess the participants’ retention of knowledge and skills, a similar theory and practical assessment was repeated two months after completion of the course. We chose a time gap of 2 months based on a previous study which shows that knowledge retention after an educational intervention was approximately 55 days or less [15]. The maximum score that could be obtained for all theory assessment and practical skill assessment were 30 marks and 25 marks respectively.
In addition, participants from the CRUSH arm also completed a gamification experience survey (adapted from Lobo et al, 2017) [4] aimed to assess the participant’s perception of the different components of gamification using a Likert scale. All the quantitative data was analyzed using IBM Statistical Package for the Social Sciences (SPSS) v23 for Windows.