Smartphones are now ubiquitous and increasing numbers of physicians are using mobile medical apps to support medical education and professional development, especially among the current generation of learners(9–15). Implementing a smartphone app for a specific clinical rotation allows residents to access medical information at critical times. Although information is widely available on the internet, it is easy to get bogged down with information overload and time-consuming to filter out inaccurate and outdated information. Despite tremendous resources used by hospitals to create protocols and guidelines, these are inconsistently followed, as many clinicians are unaware of these materials or have difficulty accessing them(16, 17).
A prospective pre-post study found that a smartphone app with the hospital’s antibiogram and treatment guidelines was associated with more improvement in medical trainees’ knowledge of prescribing antibiotics(6). However, this study was limited due to lack of randomization, as trainees enrolled earlier in the study were allocated to the control group, and the latter half was allocated to the App group. Medical trainees accumulate medical experience and knowledge throughout the academic year; having the control group recruited at a different time than the App group could confound the knowledge assessment scores. Chreiman et al. created a mobile platform to assist with the onboarding process for medical trainees starting trauma rotation(18). Around half of the participants felt the app helped in completing everyday clinical tasks (53%) and thought that the app made the transition to the trauma service easier (50%). The study also included usage frequency and found that the app was accessed an average of seven times per day. However, the study was missing a control group, and only 30 trainees completed the anonymous survey at the end of the rotation. Among mobile app studies produced from different subspecialties, many studied the pattern of usage but not the app's effectiveness on medical education. For studies that aim to examine the impaction of the medical app on the learners, most lack a control group and are limited by small sample sizes(5, 19, 20). Additionally, no similar study existed for pediatric critical care.
One of our initial concerns was that trainees might become too reliant on the app as information is easily accessible and may cause the trainees not to recall important medical concepts during knowledge assessment tests. However, our study showed the opposite. The App group had more improvement on the post-rotation quiz than the group that did not have access to the app. This could be explained by the frequent exposure to relevant material facilitated by the CHM PICU app, reinforcing recall of essential concepts. Although the increase in self-perceived PCCM knowledge was not significantly different between the two study groups, there was a trend of increased improvement for the residents in the App group. This app improved pediatric critical care knowledge among residents effectively; however, the residents may not have felt its impact.
Indirectly, we hope to improve residents' PICU experience and reduce resident burnout and depression using the CHM PICU app. However, no significant difference was found in the increase in Comfort Level in PICU between the two study groups. Likely, the app was not effective in preventing resident burnout and depression. Nevertheless, the app was favorably received by its users. Most agreed that the app facilitated transitioning to a new service and helped perform daily clinical duties in the PICU. Compared to the control group, the App group found it much easier to adapt to PICU service as an incoming resident (p=0.003; Table 1). Around half of the users estimated that 30 minutes was saved per day using the app.
The strength of this study is that it is a prospective study with block randomization. Block randomization was chosen to avoid the residents within the same rotation sharing the app. This study is also the first to investigate whether a customized mobile app effectively improved resident medical knowledge and experience in a pediatric critical care unit.
Several limitations exist for this project. Since the post-block quiz was conducted at the end of the rotation, it is unknown whether the improvement in knowledge in the App group is maintained long-term. The second limitation is the imbalance in the percentage of residents who completed both quizzes in each study group. Only 34/50 (68%) in the control group completed both quizzes, and in contrast, 37/40 (92.5%) of residents in the App group completed both quizzes. The reason for this discrepancy is unclear, but a selection bias is possible as a result. Adjusted multiple linear regression model and subgroup analyses for test score improvement were performed to account for this confounding variable. Another limitation is the validity testing of the quizzes and questionnaires; both were created specifically for this study. We only performed face validation on quizzes and surveys by conducting informal focus group sessions with our PICU residents, fellows, and attendings before launching the study. Other forms of validation were not performed.