Academic Half-Day or Noon Conference for the Win? A Multi-Center Case-Control Study Examining Conference Format and Trainee Exam Performance

doi:10.21203/rs.3.rs-1458846/v1

PDF

Research Article

Academic Half-Day or Noon Conference for the Win? A Multi-Center Case-Control Study Examining Conference Format and Trainee Exam Performance

https://doi.org/10.21203/rs.3.rs-1458846/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted 21 Mar, 2022

You are reading this latest preprint version

Background

The academic half-day curricular format has become increasingly common in residency programs nationwide over the past decade. Previous studies of the academic half-day have demonstrated improved attendance, satisfaction, well-being, and in-training exam scores. The authors’ aim was to determine whether participation in a pediatric residency program that employs the academic half-day impacts scores for the American Board of Pediatrics certifying exam or in-training exam.

Methods

This was a retrospective, multi-center, case control study. The authors recruited pediatric residency programs through the Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network in 2018. Each site reported whether they deliver the majority of their structured curriculum via an academic half-day; they also submitted de-identified data for the graduating residency classes of 2015-2017 (United States Medical Licensing Examination Step 1, in-training exam, and board exam scores). Linear mixed-effects regression models were fit to predict exam scores from conference format, controlling for Step 1 score, class year, program size, setting, and clustering in programs.

Results

15/24 (62%) of invited programs participated, providing data for 523 residents. 9 programs (60%) were academic half-day programs comprising 297 residents (57%), and 6 (40%) were non-academic half-day programs comprising 226 residents (43%). The authors found no significant difference in in-training exam or American Board of Pediatrics certifying exam scores between residents participating in academic half-day programs and those participating in non-academic half-day programs.

Conclusions

This study suggests that pediatrics residents who train in programs with academic half-day curricula perform similarly to residents who train in programs without academic half-days on the in-training exam and American Board of Pediatrics certifying exams.

academic half-day

residency conferences

residency didactics

residency curriculum

noon conference

The academic half-day (AHD) format as an alternative to traditional noon conference has been growing in popularity amongst residency programs nationwide over the past decade. ^1,2 Traditionally, noon conference occurs over the lunch hour; attendees eat while listening to a faculty lecture, often unprotected from patient care responsiblities.² Work hour restrictions and decreasing pharmaceutical support for educational conferences have complicated noon conference scheduling and incentivization, and increasing recognition of the value of a protected and interactive learning environment has prompted the consideration of alternative curricular formats.

The evidence for the efficacy of various forms of delivery of structured educational content during residency is mixed. Lecture-based didactics may not be effective in changing physician performance. ³ The effect of noon conference attendance on performance on standardized assessments of medical knowledge is conflicting. ^4–6 The AHD format is theorized to increase flexibility in instructional methods, content, and resident scheduling. ² There is limited evidence, however, evaluating the educational impact of the AHD.

The most common AHD implementation involves regularly scheduled blocks of time (commonly one half-day per week or semi-weekly) in lieu of the traditional noon conference. These educational blocks are typically protected from patient care and the teaching is directed solely or primarily at residents.¹ Published reports from programs adopting the AHD format generally cite a focus on interactive and/or case-based learning.^1,2,7−11 Studies have reported improved resident attendance and fewer interruptions as well as perceived learning and well-being after transitioning to an AHD. ^7–11 Several authors have reported a positive correlation between the AHD format and improved in-training exam (ITE) scores in residents at their institutions. ^2,7,9 On the other hand, a recent study of attending physicians at a program that had adopted the AHD reported themes of emotional strain, technologic challenges, impeded workflow, and concerns for patient safety while residents attended sessions.¹² In addition, residents participating in the AHD format have reported a decrease in curricular topics and stress associated with cross-coverage. ¹¹

To our knowledge, all published studies regarding AHD efficacy have reported results from individual residency programs, and no study to date has investigated whether the AHD impacts scores on specialty certification exams. The effect on ITE scores in pediatric residency programs has not been studied.

The aim of our study was to determine whether participation in a pediatric residency program that employs the academic half-day impacts scores for the American Board of Pediatrics certifying exam or in-training exam.

Study design

This was a retrospective, multi-center, case-control study conducted in collaboration with the Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network (APPD LEARN). The analysis was conducted at the individual level with recruitment of AHD and non-AHD groups based on approximate matching by program size, setting, and affiliation. Matched analysis was not conducted as it would require retrospective matching; it was felt that such an analysis would both compromise the study design and yield unnecessary discarding of unmatched controls with possible loss of precision.

Participants

We recruited pediatric residency programs through APPD LEARN in fall 2017 through spring 2018. At that time, the APPD LEARN network consisted of 153 ACGME-accredited pediatric residency programs nationwide. All APPD LEARN programs were surveyed, then classified as an AHD program if they answered “yes” to the question “do you deliver the majority of your structured curriculum through an academic half-day format?” Programs were also classified according to setting (academic, military, or community), region, and program size. Programs were then invited to participate as either cases (AHD) or controls (non-AHD), attempting to have a similar distribution of region, size, and setting in both groups. In all, 24 programs (12 AHD and 12 non-AHD) were invited to participate.

All categorical pediatric residents that graduated from a participating residency program within the data collection timeframe and completed an ABP examination were included; residents for whom USMLE Step 1 scores were not available were excluded.

Data collection

Each participating program collected and reported the following retrospective data on trainees graduating from their program in 2015, 2016, and 2017:

· Medical degree (classified as MD or DO/MBBS)

· Year of residency completion (2015, 2016, or 2017)

· Step 1 USMLE Score (final attempt)

· ITE PGY-1 Score

· ITE PGY-2 Score

· ITE PGY3 Score

· ABP first attempt exam score

The pediatrics ITE is a validated, standardized assessment of medical knowledge that is administered early in the first year of training; in addition to the USMLE Step 1 score, it serves as a surrogate for baseline medical knowledge. Type of medical degree, USMLE Step 1 score and PGY-1 ITE scores were included as possible confounders.

Data analysis

In order to test for an association between use of academic half days and passing the ABP exam on the first attempt, we fit a mixed-effects logistic regression model to predict pass/fail status from presence or absence of academic half day curriculum, adjusting for fixed effects of year of residency completion, USMLE Step 1 scores, type of medical degree, and PGY-1 ITE scores, and a random effect to account for clustering of residents within programs. We analyzed the data using R 3.6 (R Core Team, Vienna, Austria).

We also fit a similar linear mixed model to the ABP exam score as a continuous outcome, and to the PGY-1, PGY-2, and PGY-3 ITE scores (without using PGY-1 ITE score as a covariate). Finally, we explored models that included program size and a model of the program-level ABP pass rate.

15/24 invited APPD LEARN programs (62%) submitted data for the study, providing data for a total of 523 pediatric residents. 9/15 programs (60%) were AHD programs, who contributed data for a total of 297 residents (57%), and 6/15 (40%) were non-AHD programs, who contributed data for a total of 226 residents (43%). Table 1 presents the characteristics of participating residency programs.

Table 1. Characteristics of Participating Programs

	AHD	Non-AHD
Residency programs, total no. (%)	9 (60)	6 (40)
Residents, total no. (%)	297 (57)	226 (43)
Region
Midwest, total no. (%)	3 (33)	3 (50)
Northeast, total no. (%)	1 (11)	0 (0)
Southeast, total no. (%)	4 (44)	3 (50)
Western, total no. (%)	1 (11)	0 (0)
Type of program
Community, total no. (%)	2 (13)	0 (0)
Academic, total no. (%)	6 (40)	6 (40)
Military, total no. (%)	1 (7)	0 (0)
Mean total number of residents (SD)	45 (14.4)	46 (15.5)
Residents with MD degrees, total no. (%)	244 (82)	194 (86)

Abbreviations: AHD, academic half day; SD, standard deviation; MD, Doctor of Medicine

The estimated marginal probability of passing the ABP certifying exam for a resident with average USMLE and ITE-1 scores, and adjusted for other covariates, was 94% [89%-97%] in the AHD programs and 96% [90%-98%] in the non-AHD programs. The difference did not reach statistical significance (p=0.45). Table 2 presents the estimated marginal means of the continuous outcomes for the AHD and non-AHD programs, adjusted for the other covariates in the model. The estimated marginal means did not differ significantly by presence or absence of academic half days.

Table 2. Estimated Marginal Means and 95% CI for ITE and ABP Certifying Exam Scores

	AHD programs EMM (95% CI)	Non-AHD programs EMM (95% CI)	P-value
PGY-1 ITE	141 (136-145)	142 (137-147)	0.59
PGY-2 ITE	162 (157-166)	161 (155-167)	0.85
PGY-3 ITE	168 (164-171)	168 (164-173)	0.84
ABP exam score	201 (196-206)	202 (196-208)	0.83

Abbreviations: CI, confidence interval; AHD, academic half day; ITE, in-training exam; ABP, American Board of Pediatrics; EMM, estimated marginal means; PGY, post-graduate year

Controlling for class year, program size, type of degree, USMLE Step 1 score, and clustering in programs, our study did not detect a significant difference in resident scores on ITE exams or the ABP certifying exam between residents participating in AHD versus non-AHD programs; this is in contrast to previous studies that have demonstrated improved perceived learning and ITE scores in residency programs transitioning to an AHD. We propose several reasons for this discrepancy.

It is possible that the format of structured educational sessions may not be a major driver of resident knowledge acquisition. Previous studies of the traditional noon conference’s efficacy have been conflicting, and while authors of studies of the AHD carried out at single institutions have reported improved perceived learning and ITE scores of residents post-AHD implementation, their findings may have been confounded by differential recruitment of residents pre- and post-AHD.^1-4,6-11 It is also possible that changes to the content and/or method of delivery of structured curriculum in programs that have previously reported improved knowledge acquisition in residents after transitioning to an AHD led to the observed changes, rather than the AHD format itself. Unlike these single-site studies, our larger, multicenter study was able to compare exam scores of residents enrolled in AHD and non-AHD programs simultaneously in addition to controlling for other potential confounders such as USMLE scores and program size.

This study has several limitations. Our study design did not account for differences in educational content between programs reporting a given format, or number of years that format had been employed at the time of data collection. Programs were classified as “AHD” if they self-reported providing “the majority of their structured education via an AHD.” Our study design included neither the number or frequency of AHD sessions nor the proportion of structured educational time spent in the AHD format versus non-AHD sessions. It is possible that some of the programs classified as AHD programs delivered a substantial proportion of their structured curriculum via traditional conferences, and vice versa. In addition, we did not qualify the instructional methods programs used in the context of their educational sessions. Although published studies suggest that many programs transitioning to the AHD have done so with goals that include increasing resident interaction and active learning, the degree to which participating programs have incorporated evidence-based instructional techniques is unknown.^1,2,7-11Heterogeneity in the quality and/or quantity of educational content covered in both the AHD and non-AHD programs may have contributed to failure to detect a difference in exam performance between formats. However, we have no reason to suspect differential distribution of these variables, and the size and clustering methodology of our analysis makes it less likely that variation in implementation and execution from program to program impacted our findings.

There are other limitations to our study. We aimed to compare knowledge acquisition amongst residents attending programs employing each format for the majority of their structured educational sessions; we did not measure other variables that might impact the relative benefit and costs of each format, including patient safety or workflow issues that have been previously reported by faculty at institutions adopting the AHD.¹² As approximately 10% of all APPD LEARN programs overall participated in the study, selection bias may have impacted our results. It should also be noted that the estimated marginal mean board pass rates for programs participating in our study (both AHD and non-AHD) were higher than national means (81-88% in 2015-2017); this may limit the generalizability of our study, and the higher pass rates may have limited our ability to detect a significant difference between the groups.¹³

Given the investment of time and resources required to develop the structured curriculum each residency program provides, as well as resident and faculty time removed from direct patient care during sessions, rigorous study of the relative efficacy of various modes of instruction is needed. Challenges posed by the COVID-19 pandemic, including increased clinical responsibilities for both residents and faculty and restrictions on physical gatherings make innovative and flexible options for delivering structured educational content increasingly valuable.

Ours is the first multicenter study of the AHD format, as well as the first study to evaluate the effect of the format of structured educational sessions on specialty certification exam scores. Much remains to be learned regarding the optimal means of delivering structured curriculum in residency. The fact that we did not find a difference between exam scores in residents participating in AHD and non-AHD programs suggests that the format of structured educational sessions is not a major driver of knowledge acquisition. The content and/or method of delivery of educational sessions may be more important than format; this possibility should be further investigated via multicenter studies. Future studies of the relative benefits of various formats for the delivery of structured educational content should qualify the methods of instruction used and quantify the time dedicated to instruction to further reduce confounding.

Our study suggests that the academic half-day is non-inferior to non-academic half-day educational conferences as a means of preparing pediatric residents for specialty certifying exams. Residency leaders may be reassured that they can exercise flexibility in choosing which conference format fits their trainees and program best.

ABP: American Board of Pediatrics

AHD: Academic Half-Day

APPD: Association of Pediatric Program Directors

DO: Doctor of Osteopathy

ITE: In-Training Exam Score

LEARN: Longitudinal Educational Assessment Research Network

MD: Medical Doctor

MBBS: Bachelor of Medicine and Bachelor of Surgery

USMLE: United States Medical Licensing Exam

Ethics approval and consent to participate

The study was determined to be Not Human Research by the institutional review board (IRB) at the University of Minnesota (lead site). IRB approval or exemption was also obtained by all participating sites and APPD LEARN as the data coordinating center.

Consent for publication

Not applicable

Competing interests

The authors declare that they have no conflict of interest.

Availability of data and materials

The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

Funding

Support for this study was provided in part by the Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network (APPD LEARN), which assisted with study design; data collection, analysis, and interpretation; and in the writing of the report.

Authors’ contributions

JD had a major role in study site recruitment, data interpretation, and in writing this article. CS had a major role in study conceptualization and in writing this article. AS helped with study design and performed the statistical analysis. JF contributed to conceptualizing the study, site recruitment and data interpretation. BK assisted in study site recruitment and data collection. EBS was responsible for the intellectual conceptualization of the study. She organized the research team and played a major role in study design and recruitment as well as in writing this article. All authors read and approved the final manuscript.

Acknowledgements

We would like to thank and acknowledge Beth King, who was the program manager for APPD LEARN at the time the study was performed. We would also like to thank the leadership and members of the APPD LEARN network for their support of this study.

Declarations of interest

none

Chalk C. The academic half-day in Canadian neurology residency programs. Can J Neurol Sci J Can des Sci Neurol. 2004;31(4):511-513. http://www.ncbi.nlm.nih.gov/pubmed/15595258.
Batalden MK, Warm EJ, Logio LS. Beyond a Curricular Design of Convenience. Acad Med. 2013;88(5):644-651. doi:10.1097/ACM.0b013e31828b09f4.
Davis D, O’Brien MAT, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education: Do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? J Am Med Assoc. 1999;282(9):867-874. doi:10.1001/jama.282.9.867.
Mcdonald FS, Zeger SL, Kolars JC. DOES CONFERENCE ATTENDANCE AFFECT IN-TRAINING EXAMINATION SCORES? Associations of Conference Attendance With Internal Medicine In-Training Examination Scores. Mayo Clin Proc. 2008;83(4):449-453. doi:10.4065/83.4.449.
Rideout M, Held M, Holmes AV. The didactic makeover: Keep it short, active, relevant. Pediatrics. 2016;138(1). doi:10.1542/peds.2016-0751.
Thomas KG, Thomas MR, York EB, Dupras DM, Schultz HJ, Kolars JC. Teaching evidence-based medicine to internal medicine residents: The efficacy of conferences versus small-group discussion. Teach Learn Med. 2005;17(2):130-135. doi:10.1207/s15328015tlm1702_6.
Ha D, Faulx M, Isada C, et al. Transitioning From a Noon Conference to an Academic Half-Day Curriculum Model: Effect on Medical Knowledge Acquisition and Learning Satisfaction. J Grad Med Educ. 2014;6(1):93-99. doi:10.4300/jgme-d-13-00185.1.
Moreno MA, Kota R, McIntosh GC, Frohna JG. PEARLs of Wisdom: Impact of a New Block Conference on Pediatrics Resident Attendance, Satisfaction, and Learning. J Grad Med Educ. 2013;5(2):323-326. doi:10.4300/JGME-D-12-00249.1.
Eid A, Hsieh P, Shah P, Wolff R. Cross-sectional longitudinal study of the academic half-day format in a hematology-oncology fellowship training program. BMC Med Educ. 2015;15(1):1-8. doi:10.1186/s12909-015-0418-y.
Chen LYC, McDonald JA, Pratt DD, Wisener KM, Jarvis-Selinger S. Residents’ views of the role of classroom-based learning in graduate medical education through the lens of academic half days. Acad Med. 2015;90(4):532-538. doi:10.1097/ACM.0000000000000605.
Zastoupil L, McIntosh A, Sopfe J, et al. Positive Impact of Transition From Noon Conference to Academic Half Day in a Pediatric Residency Program. Acad Pediatr. 2017;17(4):436-442. doi:10.1016/j.acap.2017.01.009.
Wagoner H, Seltz B. Attending Physicians’ Perspectives of Resident Academic Half Day. Teach Learn Med. 2019;31(3):270-278. doi:10.1080/10401334.2018.1551140.
American Board of Pediatrics webpage, accessed November 20, 2020. https://www.abp.org/sites/abp/files/pdf/exam-pass-rates-gp-training.pdf.

No competing interests reported.

PDF

Version 1

posted 21 Mar, 2022

You are reading this latest preprint version

Academic Half-Day or Noon Conference for the Win? A Multi-Center Case-Control Study Examining Conference Format and Trainee Exam Performance

Status:

Version 1

Abstract

Background

Methods

Results

Discussion

Conclusions

List Of Abbreviations

Declarations

References

Competing Interests

Status:

Version 1