The United States Medical Licensing Examination (USMLE) recently changed their score reporting techniques for the Step-1 examination from a 3-digit number to pass/fail. This change, implemented in January 2022, has resulted in uncertainty among future medical residency candidates, especially underrepresented in medicine (URM) students, osteopathic graduates (OG), and international medical graduates (IMG) [referred to as foreign medical graduates (FMG) in several other studies]. It remains unclear which criteria will now be weighted more heavily by residency program directors (PD) when selecting candidates to interview, although some have suggested that USMLE Step-2CK (Step-2CK) and participation in away rotations may take on greater importance1,2.
When the USMLE was formed in 1992, Step exams were created to standardize testing across state borders, thereby creating a “single pathway for licensure”33. The Federation of State Medical Boards of the United States and the National Board of Medical Examiners describe the USMLE Step 1 as designed to ensure the student’s “master of not only the sciences that provide a foundation for the safe and competent practice of medicine in the present, but also the scientific principles required for maintenance of competence through lifeline learning”34. For many years, the USMLE Step 1 score was a significant factor in determining residency candidate selection for interview, and when this change was announced, the Association of Program Directors in Surgery contributed a letter to oppose this change, demonstrating its importance to many programs when making candidate selection decisions35.
To date, no study has surveyed the opinions of PDs nationwide about how the new pass/fail grading system will impact URM applicants. The AMA defines URM students as “those racial and ethnic populations that are underrepresented in the medical profession relative to their numbers in the general population”. URM students often face not only a disparity in their USMLE Step exam scores as compared to their non-URM peers, but also an overall achievement gap due to systemic inequities that have followed them throughout their education, prior to medical school matriculation8. This is due to a number of systemic inequities, such as reduced accessibility to resources and funding in predominantly underrepresented neighborhoods36. The achievement gap has negative physical and psychological consequences on URM medical students, and after matriculating, there may be some internalization and externalization of negative stereotypes that they have encountered in the past, contributing to their lower mean standardized exam performance6. The recent US Supreme court ruling that higher education admissions cannot consider race elevates this concern and warrants further exploration of the beliefs of PDs to discern how the new system will impact URM applicants. The current gap in the number of URM physicians compared to the general population is one of many crucial areas that need to be addressed to improve equity in patient care and health outcomes for underrepresented populations.
While all medical students are eligible to apply for any residency program, historically some applicant groups such as URM, OG, and IMG students have lower match rates3,4,5,6. A study by Makhoul et al (2020) reported valuable data on the perspectives of PDs from 30 specialties on the pass/fail Step-1 scoring change; overall, 77.2% of PD respondents believed that all applicants will have a more difficult time matching into their specialty. Although they discussed how this new policy may impact the overall process and specifically addressed the potential impact on IMG and, to a lesser extent, OG applicants, they did not explore the impact on URM applicants.
A few recent studies have explored how this Step-1 scoring change may impact specific residency’s application process. In one study, about 50% of dermatology PDs agreed that all medical students will have more difficulty matching into dermatology. Another study surveyed medical student deans; when asked to rank the importance of various application factors, the most frequently selected first-choice was Step-2CK score. Collectively, medical school deans reported feeling that students who apply to dermatology, neurosurgery, orthopedic surgery, ENT, and plastic surgery would be most impacted25. Many stakeholders have expressed concerns about removing the standardized Step-1 score and posit additional issues that may arise from this scoring change2,27,28,29. Some have highlighted the potential impact of structural biases on URM applicants, given the existing socioeconomic inequities that lead to reduced financial and community resources among the URM population30.
In the past, URM residency applicants had lower mean Step-1 and Step-2CK scores than non-URM candidates9,10. Studies suggest that a reduction in the reliance on Step-1 scores may mitigate racial biases11. Due to the weight that PDs historically ascribed to Step-1 scores in choosing applicants for residency interviews, URM applicants may have been screened out of interview opportunities, a point that was emphasized by the Invitational Conference on USMLE Scoring when they recommended minimiz(ing) “racial demographic differences that exist in USMLE performance”6. While many have expressed opinions, to date, no study has explicitly surveyed PDs regarding how they believe URM applicants will be impacted by the Step-1 pass/fail policy.
Twelve competitive U.S. residency specialties were selected for this study due to their relatively low rates of admitting OG, IMG and URM applicants16,17,18. Specifically, these specialties historically lack racial diversity due to the low admittance of URM applicants16,17,18. We aimed to identify what selection factors PDs from these twelve specialties consider to be most important now that Step-1 is pass/fail, and if PDs believe this change will impact the match difficulty of URM, OG, and IMG applicants.