This study aimed to determine whether recording the number of US examinations performed after taking a POCUS simulation course led to a behavior change, and whether keeping a record of the number of US examinations performed maintained US knowledge and skills. Our results showed that keeping a record significantly increased the number of US examinations performed. In addition, keeping a record after the simulation training led to a behavior change in the field of simulation education. Keeping a record also contributed to maintaining POCUS knowledge, skills, and confidence. Our study suggests that keeping a record may be useful to improve skills retention in the field of simulation education.
In educational methods, including in simulation training, it is important and most effective to cause both a reaction or learning improvement (Kirkpatrick’s levels 1 and 2, respectively), and behavior change or improvement (Kirkpatrick’s levels 3 and 4, respectively).13 However, it is often difficult to evaluate levels 3 and 4 because this evaluation is time consuming and requires effort and cost to train evaluators and prepare tools and facilities. Therefore, few studies have evaluated behavior change, and effective methods to change behavior have not been established in the field of simulation training.14, 24, 25 In our study, recording the number of clinical US examinations increased the number of these examinations that participants performed after the simulation course. Our results also showed that keeping a record after the simulation course led to a behavior change.
This study showed that the quality of the examinations was maintained after the course. The 4-month follow-up test results showed that the image interpretation skills, image acquisition skills, and confidence scores were statistically significantly improved compared with the pre-course test scores, and that these scores did not decline compared with the immediate post-course test scores. The problem of skills retention is an most important problem in the field of simulation training.4, 5, 16, 17 Knowledge, skills, and confidence decline in a few months to 1 year after a simulation course with no interventions.4, 5, 9, 16, 19 Several methods have been proposed to help participants retain knowledge and skills; for example, providing didactic or online lectures, and holding hands-on training sessions or simulation training courses regularly or several months after the course.15, 18–21 The method that we used in this study (keeping a record) was useful to maintain knowledge, skills, and confidence 4 months after the simulation course. This method involves less effort and cost than conventional methods and is feasible and can be implemented at most facilities.
This study had several limitations. First, we recruited only JNPs; however, participants were from 21 facilities in numerous regions across Japan. The number of post-graduate years also varied; repeating this research with attending doctors, residents, and medical students is needed to clarify the usefulness of this method for these cohorts. Second, this study involved a follow-up test 4 months after the course. Participants were aware of this follow-up test in advance, which might have influenced their behavior. However, all participants were informed that the results of this study would not affect their future work or training. Therefore, the impact of the follow-up test was not considered large. Third, the number of US examinations was self-reported. Additionally, nine instructors participated as evaluators. All instructors were certified and trained regarding how to evaluate participants before this study. However, it cannot be denied that there might have been measurement error. This study was not a crossover study, and we did not compare study participants with a group who did not keep a record. However, research has shown that skills and knowledge decrease after a simulation course without intervention. Therefore, our method appeared to be effective to improve the educational effectiveness of simulation courses.
Although there were several limitations, our study indicated that keeping a record after taking a simulation course can lead to behavior change. This method also effectively maintained knowledge, skills, and confidence and is inexpensive, with good feasibility. This method is therefore useful to induce behavior change (Kirkpatrick’s level 3) and improve skills retention in the field of simulation training.