Study Design And Setting
This study was a prospective randomised controlled trial conducted from May to November 2018 in the Emergency and Trauma Department (ETD) of Sarawak General Hospital (SGH), Malaysia to compare the gamification approach versus F2FL approach for POCUS training using the RUSH protocol. In this study, we also included an assessment of the participants’ perception towards the gamification approach in POCUS training. This study was approved by the Medical Research and Ethics Committee (MREC) Malaysia and was registered with the National Medical Research Register (NMRR-18-444-40348). As this study was conducted as an actual POCUS training workshop, the number of participants was limited by resource availability. Hence, a convenient sampling of 32 participants (16 participants in each arm) were recruited.
Study Population
The study population comprised of junior doctors working in the ETD SGH, Sarawak Heart Center and the Internal Medicine Department of SGH. We defined a ‘junior doctor’ as a doctor with at least 2 years of but no more than four years’ experience of clinical service. The reason for selecting doctors with 2 to 4 years of clinical experience was because this group of doctors would have completed their compulsory two-year internship program and would likely have developed sufficient clinical exposure to be able to critically understand the course content of POCUS training and the utilization of POCUS in clinical decision making.
Any junior doctor who had participated before in any formal POCUS training was excluded. Informed consent was obtained from all participants before commencing this study. All participants joined this free POCUS training on a voluntary basis without any payment or monetary compensation.
Materials
The topics for this POCUS training workshop were based on the requirements of the original RUSH protocol and adopted from the World Interactive Network Focused on Critical Ultrasound (WINFOCUS) Malaysia’s Basic Ultrasound Life Support Course (USLS BLS) as well as from the emergency ultrasound training from a post-graduate emergency medicine training program in Malaysia (i.e., the Universiti Malaya Emergency Medicine postgraduate curriculum). All materials were internally validated via a modified Delphi technique to attain consensus by a panel of experts in emergency medicine.
Study Procedure
This study was divided into two stages: 1) identifying learning materials and development of assessment questions using the modified Delphi method and 2) recruitment, randomization and implementation of interventions. In the first stage, the discussions were carried out with the experts in three rounds. Most of the discussion was carried out via e-mail and online group dialogues as the authors were based in different locations in Malaysia. The first round of discussions focused on identifying the main objectives and probable contents of the workshop. In the second round of discussion, the list of probable topics was distributed by email to the experts for review and to reach a consensus on the suitability of the topics. In the final round of discussions, the shortlisted topics were divided and assigned to the specific panel of experts for teaching as well as for preparation for the assessment questions. This assessment consists of two sections: (1) 30 one-best answer (OBA) type of multiple-choice questions (for theory assessment), and (2) one objective structured clinical examination (OSCE) question (for practical assessment). The questions were then vetted, revised, finalized and agreed upon by the experts.
In the second stage of study, participant recruitment, randomization and implementation of educational intervention were conducted. Thirty-two participants were randomized to either the gamification or F2FL. The participants in the gamification arm were further randomized into groups of 4 participants. Randomization processes were performed using an online random number generator.
On the first day of the course, all participants are required to complete a pre-test knowledge (30 one-best answer type of multiple-choice questions) and practical skills assessment test. The practical skills assessment was conducted in the format of one objective structured clinical examination (OSCE) question and assessed by three independent emergency physicians who were blinded to the participants’ study arms. This process was repeated during the end of the course on the following day as the post-test knowledge and skills assessment. To assess the participants’ retention of knowledge and skills, the theory and practical assessment was repeated two months after completion of the course. We chose a time gap of 2 months based on a previous study which shows that knowledge retention after an educational intervention was approximately 55 days or less [11].
In addition, participants from the gamification arm also completed a gamification experience survey (adapted from Lobo et al, 2017) [4] aimed to assess the participant’s perception of the different components of gamification using a Likert scale. The schedule for POCUS training workshop using RUSH protocol for both gamification and F2FL groups is given in Supplementary Table 1.
With regards to theory assessment, in the F2FL arm, a written OBA questions paper was distributed for students to answer. Answers were discussed at the end of the day. The questions were marked individually. In the gamification arm, instead of an individual written theory paper (OBA), live quizzes were implemented. There were 4 teams with 4 participants in each team. Gamification mechanics and elements used were the points, badges, leader boards, teamwork, competition, immediate real-time feedback, challenges with set objectives, chance, turns, rules, scaffolded learning with optional increasing challenges, a sense of progression and presence of social dynamics and interaction [12]. In this study, each team began at level one with zero experience points (XP) and required XP to progress to the next level. XP points are gained through live quizzes interspersed between lecture modules by answering correctly the questions. Real-time leader boards were used to monitor the progression of teams (Fig. 1A). Virtual badges were rewarded based on progression. Each team had a chance to answer the quiz and discussed amongst the team members. The quiz was presented using a Jeopardy-style game show format using FlipQuiz™ technology (Fig. 1B) whereby teams are able to pick the level of difficulty of questions with different points allocated. If any team were unable to answer the question, another team was given a chance to answer but would only obtain half of the awarded points if answered correctly. Feedback for the correct answers was given immediately to all teams. There was no negative marking in all quizzes.
For the practical assessment in the F2FL arm, case-based assessment (in OSCE format) was carried out individually to every participant. In the gamification arm, the ultrasound games were played as team activities with similar marks awarded to the all team members. The psychomotor skill objectives covered in the hands-on practical sessions and the corresponding gamified versions for participants in the gamification arm are given in Table 1. At the end of the course, the team with the highest score won a reward. All the quantitative data was analyzed using IBM Statistical Package for the Social Sciences (SPSS) v23 for Windows.