A gamified morning report curriculum to improve internal medicine trainee engagement, content exposure, and perceived learning efficacy

DOI: https://doi.org/10.21203/rs.3.rs-1804329/v1

Abstract

Background

The daily morning report session for the Naval Medical Center Portsmouth Internal Medicine Residency Program uses the traditional format of case presentation and discussion. This commonly-used approach to morning report has relatively low learner attendance and perceived individual engagement and topics are derived from a limited pool of recently admitted cases. To improve engagement, material exposure, and perceived learning efficacy, we implemented a weekly competitive quiz session designed to review material presented previously.

Methods

The Friday morning report session was changed to a weekly quiz game, called “Follow-up Friday” with trainees forming small groups of 3-4 people from different levels of training. Using their smartphones and presentation software available from “Mentimeter.com,” learners engaged in a competitive quiz with questions derived from cases and material presented earlier in the week as well as older material that they had found challenging from prior quizzes and the in-service training exam.

Results

Implemented in July 2021, this session was delivered to over 150 trainees including medical students and trainees rotating through the internal medicine department. When surveyed, internal medicine trainees of all year-groups indicated a strong perception that the Follow-Up Friday session improved engagement, material exposure, and learning efficacy. 

Discussion

Modifying the morning report format to include well-established educational concepts was simple to implement and very well received by learners. This study was limited by a small sample size and a design that included only subjective outcomes from the learners.

Conclusion

Follow-Up Friday employed time-spaced learning, recall practice, and interleaving of topics resulting in improved learner engagement, material exposure, and perceived learning efficacy. This simple and effective format warrants continued use and may be of benefit to other training programs.

Introduction

There is a growing body of research, popularized by the book Make it Stick: The Science of Successful Learning, on the mechanisms through which learning and retention occur and are strengthened.1 These “cognitive learning strategies,” when applied intentionally, have the power to significantly improve the efficacy of a learner’s efforts.2 Among these cognitive learning strategies are the techniques of spaced practice, interleaving (alternating between multiple different topics), and retrieval practice.3 4These strategies focus on the learner as an active participant who engages with the material over a span of time and practices its use and application to new situations.

Morning report in the internal medicine program at Naval Medical Center Portsmouth follows a traditional format relying heavily on presentation of new admissions from overnight to stimulate discussion among trainees and staff. This format results in a focus on problem solving skills as participants progress through a patient presentation and key points are reviewed throughout the case. A mainstay of postgraduate medical education, this approach provides a mechanism to review high-yield topics but is limited to topics germane to recently admitted cases. Case topics can vary widely across the broad scope of internal medicine, and some topics are reviewed relatively frequently while others may only be seen once a year, if at all, depending on what pathologies present to the medical center. This format also is limited in its ability to engage the audience of learners with questions being answered by one learner at a time, with others in the audience able to progress through the session with limited engagement. This relatively passive approach (from the perspective of the learner) has been shown to be less effective in producing sustained learning.5

Our goal was to improve learner engagement and quality of presented material. To do so, we implemented a weekly small-team quiz game using material presented throughout the prior week’s academic sessions, as well as challenging topics previous discussed. This approach allowed us to be more intentional in the presentation and review of internal medicine topics. Furthermore, it turned a disjointed series of morning reports into a more unified curriculum throughout the year.

Methods

We altered the format of one morning report a week (on Friday) to include spaced retrieval practice in the form of a small-group competitive quiz game. This game, dubbed “Follow-Up Friday” integrated topics from both morning report and didactic sessions in the preceding week as well as high-yield or challenging topics discussed earlier in the year.

Every week the chief of residents created a quiz consisting of 15–25 questions drawing from these sources. The questions were a mix of multiple choice and free answer questions with relevant images (such as ECGs and chest x-rays) included. We used software available from www.mentimeter.com to create these quizzes. Quizzes always concluded with follow up on the cases discussed earlier in the week (provided by the residents in the audience) and announcement of the winning team.

During these Friday sessions, trainees formed groups of 3–4 people each, deliberately mixing levels of training (medical students through senior residents). Each team had one member log into the digital presentation software using their smart phone and a code provided. Trainees then answered questions competitively with a brief review after each question recalling where that information was originally presented, and occasionally adding new material on that topic. The winning team had its members’ names displayed at the front of the room for the coming week. No preparation was required of the learners prior to this session.

The quiz content and results for each question were maintained within the presentation software allowing the chiefs to review prior quizzes and re-use questions that the trainees found challenging.

Results

Implemented in July 2021, this quiz session was delivered to over 150 medical students, intern physicians, and resident physicians rotating through general and subspecialty internal medicine rotations.

The morning report session saw rapid and sustained attendance with the audience averaging 30 trainees including medical students, a significant improvement from the prior standard of fewer than 20. Trainee engagement was notably increased during the session with every trainee actively participating in the game becoming excited about the material.

Equally importantly, the addition of this session allowed for a more intentional presentation of internal medicine topics including routine review of topics uncommonly encountered as well as review of topics from didactic sessions in the preceding week which some trainees may have missed.

To assess the efficacy of this intervention, 23 internal medicine trainees (9 PGY-1s, 8 PGY-2s, and 6 PGY-3s) were surveyed about this academic session, representing 72% of current trainees in the internal medicine program. The results are presented in the table below.

Trainee responses indicated strongly that the intervention created a more engaging and effective learning environment that was more fun than the standard format and provided community-enhancing effects.

Discussion

Through the use of a weekly quiz game, we were able to smoothly integrate the cognitive learning strategies of spaced practice, interleaving, and retrieval practice into our morning report curriculum while simultaneously increasing learner engagement. Implementing this weekly, low-stakes testing provided opportunity to improve the quality of our instructional methods by using evidence-based educational concepts. We were also able to expand our curriculum to more routinely integrate less common disease processes and deliberately target material that trainees found challenging, as measured by poor performance on prior quizzes and the in-service training exam. This approach was preferred by learners across multiple training levels, who found this format more engaging and effective.

The gamification of low-stakes testing was an effective strategy to increase the quality and content of our morning report curriculum, and would likely be of value to other training programs.

Declarations

Funding: None

Conflicts of Interest: None

This study was reviewed 32 CFR 219.102

Disclaimers

The views expressed in this article reflect the results of research conducted by the authors and do not necessarily reflect the official policy or position of the Department of the Navy, Department of Defense, or the United States Government.

We are military service members. This work was prepared as part of our official duties. Title 17 U.S.C. 105 provides that “Copyright protection under this title is not available for any work of the United States Government.” Title 17 U.S.C. 101 defines a United States Government work as a work prepared by a military service member or employee of the United States Government as part of that person’s official duties.

Ethics Approval and Consent to Participate

The above protocol was reviewed by the Naval Medical Center, Portsmouth, Virginia Institutional Review Board and deemed to not meet the criteria for human subjects research (IRB reference NMCP.2022.0042).  The review board determined neither informed consent nor ethics board review were required in accordance with relevant guidelines and regulations (32 CFR 219.102 and DoD Instruction 3216.02), which were followed in the administration and evaluation of this educational initiative.

Funding Statement

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors. 

Data Availability

Data available from the corresponding author upon reasonable request.

Competing Interests

The author has no competing interests to declare. 

Contributorship Statement

The listed authors shared equally in the design and execution of the educational initiative as well as the creation of this manuscript.

References

  1. McDaniel, Mark A., et al. Make It Stick: The Science of Successful Learning. United Kingdom, Harvard University Press, 2014.
  2. Wolff M, Wagner MJ, Poznanski S, Schiller J, Santen S. Not another boring lecture: engaging learners with active learning techniques. J Emerg Med. 2015 Jan;48(1):85-93. doi: 10.1016/j.jemermed.2014.09.010. Epub 2014 Oct 13. PMID: 25440868.
  3. Ambrose SA, Bridges MW, DiPietro M, Lovett MC, Norman MK. How learning works: seven research-based principles for smart teaching. San Francisco, CA: Jossey-Bass; 2010.  
  4.  Smolen, P., Zhang, Y., & Byrne, J. H. (2016, February). The right time to learn: mechanisms and optimization of spaced learning. Nat Rev Neurosci, 77-88.
  5. Retrieval practice [Internet]. Unleash the Science of Learning. [cited 2022Apr6]. Available from: https://www.retrievalpractice.org/

Tables

Table 1 Trainee Attitudes Regarding Follow-up Friday

Trainee Attitudes Regarding Follow-up Friday

Follow-up Friday...

PGY-1 Mean Scorea

PGY-2 Mean Scorea

PGY-3 Mean Scorea

Mean For All Years of Traininga

...is more engaging than a standard case based morning report 

4.7

5.0

5.0

4.9

...has increased the breadth of my exposure to less common medical topics 

4.4

4.6

4.8

4.6

...has increased my retention of medical knowledge more than standard case-based morning report has

4.1

4.8

5.0

4.6

...is more enjoyable than a standard morning report 

4.9

5.0

5.0

5.0

...has increased bonding/sense of community among trainees

4.9

5.0

5.0

5.0

...should remain a part of the curriculum

4.9

5.0

4.8

4.9

aRated on a 5-point scale (1= Strongly Disagree, 5 = Strongly Agree)