Title: A Comparison of Orthopaedic Surgery and Internal Medicine Perceptions of USMLE Step 1 Pass/Fail Scoring: A National Program Director Survey

doi:10.21203/rs.3.rs-237028/v1

Background

United States Medical Licensing Examination Step 1 will transition from numeric grading to pass/fail, sometime after January 2022. The aim of this study was to compare how program directors in orthopaedics and internal medicine perceive a pass/fail Step 1 will impact the residency application process.

Methods

A 27-item, validated survey was distributed through REDCap to 197 U.S. orthopaedic residency program directors and 554 U.S. internal medicine residency program directors. Program director emails were obtained from the American Medical Association’s Fellowship and Residency Electronic Interactive Database.

Results

We received 58 (36.0%) orthopaedic and 125 (22.8%) internal medicine program director responses. The majority of both groups disagree with the change to pass/fail, and felt that the decision was not transparent. Both groups believe that the Step 2 Clinical Knowledge exam and clerkship grades will take on more importance. Compared to internal medicine, orthopaedic PDs have greater odds of emphasizing research (OR 6.43, CI 3.18-13.00, p < 0.001), letters of recommendation from known faculty (OR 3.22, CI 1.77–5.87, p < 0.001)), Alpha Omega Alpha membership (OR 4.96, CI 2.65–9.25, p < 0.001), leadership/extracurricular activities (OR 3.69, CI 1.96–6.95, p < 0.001), audition elective rotations (OR 6.20, CI 3.21–11.97, p < 0.001) and personal knowledge of the applicant (OR 2.93, CI 1.61–5.34, p < 0.001). Both groups believe that allopathic students from less prestigious medical schools, osteopathic students, and international medical graduates will be disadvantaged. Orthopaedic and internal medicine program directors agree that medical schools should adopt a graded pre-clinical curriculum, and that there should be a cap on the number of residency applications a student can submit.

Conclusion

Orthopaedic and internal medicine program directors disagree with the change of Step 1 to pass/fail. They also believe that this transition will make the match process more difficult, and disadvantage students from less highly-regarded medical schools. Both groups will rely more heavily on the Step 2 clinical knowledge exam score, but orthopaedics will place more importance on research, letters of recommendation, Alpha Omega Alpha membership, leadership/extracurricular activities, personal knowledge of the applicant, and audition electives.

Educational Philosophy and Theory

USMLE

Step 1

pass/fail scoring

orthopaedic surgery

internal medicine

The United States Medical Licensing Examinations (USMLE) currently consists of three numerically scored knowledge based exams and a pass/fail clinical skills exam. On February 12, 2020, the Federation of State Medical Boards (FSMB) and the National Board of Medical Examiners (NBME) announced that the first of these three exams, Step 1, would change from the three-digit numeric score to reporting a pass/fail outcome -- starting sometime after January 2022.¹ This change occurred after extensive national discussion on potential ways to optimize the transition from undergraduate to graduate medical education.^1,2

The unintended consequences of a scored Step 1 on medical students and residency programs were widely reported in 2008 by the Committee to Evaluate the USMLE Program (CEUP). CEUP recognized the diverse stakeholders affected by Step 1 score reporting and suggested a need to redesign USMLE.² Although a numerically scored Step 1 provided an objective measure for residency programs to assess applicants from different backgrounds and medical schools, the perceived overemphasis on this test was controversial. In March 2019, the Invitational Conference on USMLE Scoring (InCUS), sponsored by the Association of American Medical Colleges (AAMC), American Medical Association (AMA), NBME, FSMB, and Educational Commission for Foreign Medical Graduates (ECFMG), convened to review the USMLE’s practice of numeric score reporting. InCUS concluded that licensing examinations for undergraduate medical students must be reevaluated. FSMB and NBME took into consideration the recommendations from InCUS, and stated that the change to a pass/fail Step 1 would help residency programs to refocus on the main purpose of Step 1, to assess medical licensure eligibility.^1,2

Step 1 is one of the most important factors used to screen applicants by internal medicine (IM) and orthopaedic surgery residency programs.^1–3 Some residency programs also place a high emphasis on other factors, such as grades in required clerkships, research experiences, extracurricular activities, letters of recommendation, and audition rotations. With the absence of a scored Step 1, all residency programs will need to reconsider the factors used to determine which applicants are selected. The aim of this study was to compare the perceptions among program directors (PDs) in orthopaedics and IM on the change of Step 1 from a graded to a pass/fail exam, and its impact on the residency application process.

This study was exempted by the College of Medicine’s Institutional Review Board. We obtained publicly available contact information for PDs at all active Accreditation Council for Graduate Medical Education (ACGME) orthopaedic surgery and IM residency programs through the American Medical Association’s Fellowship and Residency Electronic Interactive Database (FREIDA Online). We identified and contacted 197 orthopaedic surgery PDs and 554 internal medicine PDs. A survey invitation email with the appropriate informed consent information was sent to each PD. Submission of the survey indicated the respondents’ consent to participate. All responses were collected anonymously.

This multi-center, cross-sectional research survey was developed, housed, and distributed through REDCap Data Capture. The survey was developed using criteria analyzed in the 2018 National Resident Matching Program (NRMP) Program Director survey, which investigated the factors involved in selecting applicants to interview.³ Our survey questions used single-answer multiple-choice, multiple-answer multiple-choice, and a Likert scale ranging from 1 (strongly disagree or least important) to 5 (strongly agree or most important) formats. Prior to distribution, the questions were pretested and tested with subsets of respondents resulting in a 27-item validated survey. No funding was utilized.

The orthopaedic electronic survey was distributed on April 17, 2020. A follow-up was emailed 2 weeks after the initial submission. The survey was closed on May 5, 2020. The IM electronic survey was distributed on July 8, 2020. A follow-up was emailed 2 weeks after the initial submission. The survey was closed on August 5, 2020.

Orthopaedic and IM PD responses were summarized with descriptive statistics and percentages prior to any analysis to check their distributions. Given that the outcome variables were ordinal variables on a Likert scale of agreement or importance ranging from 1 to 5, we used ordinal logistic regression to compare the survey responses of orthopaedic and IM PDs. Odds ratios were estimated from this analysis as the effect size to assess the direction and magnitude of any significant associations, and these odds ratios were interpreted both in terms of increasing importance or agreement, or in terms of decreasing importance or agreement depending on the outcome variable. Statistical significance was set at p < 0.05, and all analyses were performed using SAS version 9.4 (SAS Institute, Cary, NC).

Of the 197 orthopaedic programs contacted, 161 were included in this study due to invalid and overlapping email accounts. The orthopaedic PD survey had 58 respondents, for a response rate of 36.0%. Of the 554 IM programs contacted, 548 programs were included in this study due to invalid and overlapping email accounts. The IM PD survey had 125 respondents, for a response rate of 22.8%. The data, including the number of respondents for each survey question, odds ratios, and p-values are shown in Table 1.

How do internal medicine and orthopaedic program directors feel about the pass/fail Step 1?

The majority of orthopaedic (82.8%) and IM (74.4%) PDs disagree with the change to pass/fail Step 1. Additionally, 86.2% of orthopaedic and 79.2% of IM PDs disagree that the decision to transition to pass/fail Step 1 was transparent. Only 43.1% of orthopaedic and 39.2% of IM PDs agree that a graded Step 1 measures the ability of applicants to succeed in their respective fields. Orthopaedic and IM PDs both equally disagree with the change to pass/fail Step 1 (OR 1.36, CI 0.75-2.46, p=0.315) and agree that a graded Step 1 does not adequately measure the ability of an applicant to succeed in residency (OR 1.18, CI 0.67-2.06, p=0.570).

How will a pass/fail Step 1 impact internal medicine and orthopaedic residency programs?

Few orthopaedic (10.4%) and IM (20.7%) PDs agree that a pass/fail Step 1 will help to create better future physicians. Orthopaedic PDs have lower odds of saying that the change to pass/fail Step 1 will make the match process less fair and meritocratic (OR 0.53, CI 0.30-0.93, p=0.026).

How will the pass/fail change impact the importance of the factors reviewed by internal medicine and orthopaedic residency programs when assessing applicants?

The majority of orthopaedic (89.7%) and IM (69.6%) PDs believe that Step 1 will become less important when it transitions to pass/fail, and 96.9% of orthopaedic and 88.0% of IM PDs agree that Step 2 Clinical Knowledge exam (CK) score will be more important. Additionally, 77.6% of orthopaedic and 67.2% of IM PDs say grades in required clerkships will be more important. The majority of both orthopaedic and IM PDs believe that personal knowledge of the applicant and an audition elective will be more important.

Compared to IM PDs, orthopaedic PDs have significantly greater odds of saying the following components will be of greater importance after the change to pass/fail Step 1: research experience (OR 6.43, CI 3.18-13.00, p<0.001); letters of recommendation from faculty they know (OR 3.22, CI 1.77-5.87, p<0.001); Alpha Omega Alpha (AOA) membership (OR 4.96, CI 2.65-9.25, p<0.001); leadership/extracurricular activities (OR 3.69, CI 1.96-6.95, p<0.001); personal knowledge of the applicant (OR 2.93, CI 1.61-5.34, p<0.001) and audition electives (OR 6.20, CI 3.21-11.97, p<0.001).

How will various applicant groups be affected by the change to a pass/fail Step 1?

The majority of orthopaedic and IM PDs believe that allopathic (MD) students not attending highly-regarded medical schools, osteopathic (DO) students, and international medical graduates (IMGs) will be disadvantaged by the change of Step 1 to pass/fail. Compared to IM PDs, orthopaedic PDs have greater odds of believing that MD students who do not attend highly-regarded medical schools (OR 2.14, CI 1.19-3.88, p=0.012) and DO students (OR 2.59, CI 1.45-4.61, p=0.001) will be disadvantaged in the match processes due to the Step 1 pass/fail transition.

How will changing to a pass/fail Step 1 affect medical students interested in internal medicine and orthopaedics?

Orthopaedic and IM PDs both have low expectations (<30%) that the change to a pass/fail Step 1 will allow students to focus more on learning medicine rather than studying for the Step exam, to seek out more leadership and extracurricular activities, or to pursue more hobbies and self-development. Orthopaedic PDs have 6.19 times greater odds (CI 2.80-13.69, p<0.001) of believing a pass/fail Step 1 will encourage applicants to pursue more research experiences. Less than 10% of IM PDs thought the pass/fail change would encourage more research experience. Compared to IM PDs, orthopaedic PDs have 3.50 times greater odds (CI 1.83-6.71, p<0.001) of thinking that the Step 1 pass/fail change would encourage students to attend more audition electives.

What are the future implications on residency applications and medical education?

With the change to pass/fail Step 1, a majority of orthopaedic and IM PDs agree that medical schools should adopt a graded pre-clinical curriculum, and that there should be a cap on the number of residency applications an applicant can submit.

The present study shows that the majority of orthopaedic and IM PDs do not support the Step 1 pass/fail change. Both orthopaedic and IM PDs believe Step 2 CK will be significantly more important. After the pass/fail change, orthopaedic PDs are significantly more likely than IM PDs to assign greater value to research experience, letters of recommendation from faculty they know, AOA membership, leadership/extracurricular activities, personal knowledge of the applicant, and audition electives. Both orthopaedic and IM PDs believe students who attend highly-regarded medical schools will be advantaged by the move to make Step 1 pass/fail. However, more orthopaedic PDs believe DO students and MD students attending less prestigious medical schools will be disadvantaged, while more IM PDs believe IMGs will be disadvantaged.

Currently, standardized examinations play a major role in evaluating residency applicants across specialties, with Step 1 often being considered the most important test.^1-3 In the 2020 NRMP Survey, PDs across all specialties were asked to rank the factors used to interview and rate applicants on a scale of 1 (not at all important) to 5 (very important). When selecting which applicants to interview, orthopaedic and IM PDs provided high ratings, over 4/5, for Step 1. Furthermore, over 90% of orthopaedic and IM PDs cited Step 1 as an important factor for screening applicants for an interview.³

There is minimal evidence supporting a scored Step 1’s ability to screen for successful residents.^4-8Even in our survey, the majority of both orthopaedic and IM PDs felt that a graded Step 1 did not adequately measure the ability of an applicant to succeed in residency. Despite the limited predictive value of standardized tests for future success, both orthopaedics and IM have used Step 1 as a screening measure for applicants due to the lack of additional objective criteria.³ Historically, both orthopaedics and IM have emphasized standardized test scores. According to the NRMP, U.S. MD seniors applying to orthopaedics had an average Step 1 score of 248 and a Step 2 CK score of 255. U.S. MD seniors applying to IM had an average Step 1 score of 235 and a Step 2 CK score of 248.⁹ In our study, the majority of orthopaedic and IM PDs report that Step 2 CK scores will be more important.

The results of our study also suggest that these specialties may place a greater emphasis on factors outside of standardized exam scores, as both orthopaedic and IM PDs believe grades in required clerkships, personal knowledge of the applicant, and audition electives will now be more important. Additionally, orthopaedic PDs are more likely than IM PDs to say research experience, AOA membership, letters of recommendation from faculty they know, leadership/extracurricular activities, personal knowledge of the applicant, and audition electives will be more important. Based on our analysis, orthopaedic PDs are also 6.2 times more likely (p<0.0001) to believe a pass/fail Step 1 will encourage applicants to pursue more research experiences and 3.5 times more likely (p=0.0002) to say it will encourage students to attend more audition electives. Recently, many orthopaedic programs have been publicizing holistic changes in their residency selection criteria on social media. For instance, the University of Missouri - Kansas City Orthopaedic Surgery Residency program announced on Twitter that they would no longer be considering Step 1 and Step 2 CK scores. They plan on considering other factors like personal statement, recommendation letters, dean’s letter, research, audition rotations, grades in required clerkships, and interest in their program.¹⁰ This is one example of an orthopaedic program moving toward a more holistic application review. A possible positive consequence of this move towards a more holistic process is that students from nontraditional, underrepresented, or underserved backgrounds may have more opportunities to be selected for interviews.¹¹

Interestingly, in the 2020 Match, U.S. MD seniors applying to orthopaedics had an average of 14.3 abstracts/presentations/publications, 8 volunteer experiences, and 40.3% were part of AOA; those applying to IM had an average of 6.2 abstracts/presentations/publications, 7.3 volunteer experiences, and 17.4% were part of AOA.⁹ This difference may be due to the more competitive match process in orthopaedics. In 2020, U.S. MD seniors applying to IM had a match rate 97.1%, while orthopaedics had a match rate of 79.7%.⁹

This work found that 83.2% of IM PDs believe IMGs will be disadvantaged by the change to pass/fail Step 1, while only 58.6% of orthopaedic PDs believe IMGs will be disadvantaged. This difference may be related to the lack of IMGs in orthopaedics, and the prevalence of IMGs in IM residencies. In 2019, only 1.5% of orthopaedic residents were IMGs, while 38.8% of IM residents were IMGs.¹² IMGs frequently rely on Step 1 scores to help them stand out when applying to US residency programs. Many residency programs often use Step 1 scores to quickly sift through the large volume of IMG applicants.^13-15 IM PDs will have to consider new criteria by which to evaluate IMGs, while orthopaedic PDs will likely not face this challenge to the same extent, as IMGs make up such a small portion of those entering the field of orthopaedic surgery.

The present study demonstrated that orthopaedic PDs are more likely to say DOs will be disadvantaged by the change to a pass/fail Step 1. When applying for residency, DO applicants may face additional challenges compared to MD students.^15,16 This may be due to a variety of factors, including limited research opportunities for students at DO schools compared to students at MD schools, as most US osteopathic medical schools are typically not associated with large academic institutions.¹⁶

A limitation of this study was that we were unable to obtain responses from all orthopaedic and IM residency PDs. However, we believe our response rates (36.0% for orthopaedics and 22.4% for IM) are reasonable and representative. Another limitation of this study is that some contact information provided by FREIDA Online did not belong to the PD, but instead the program’s coordinator or administrative assistant. Our email requested program coordinators and administrative assistants to send the survey to PD.

In conclusion, the majority of orthopaedic and IM PDs do not support the Step 1 pass/fail change. Both groups will rely more heavily on Step 2 CK. Orthopaedic PDs are significantly more likely to value research experience, letters of recommendation from faculty they know, AOA membership, leadership/extracurricular activity, personal knowledge of the applicant, and audition electives.

-USMLE: United States Medical Licensing Examinations

-FSMB: Federation of State Medical Boards

-NBME: National Board of Medical Examiners

-CEUP: Committee to Evaluate the USMLE Program

-InCUS: Invitational Conference on USMLE Scoring

-AAMC: Association of American Medical Colleges

-AMA: American Medical Association

-ECFMG: Educational Commission for Foreign Medical Graduates

-ACGME: Accreditation Council for Graduate Medical Education

-IM: Internal Medicine

-PD: Program Director

-FREIDA Online: Fellowship and Residency Electronic Interactive Database

-NRMP: National Resident Matching Program

-CK: Clinical Knowledge exam

-AOA: Alpha Omega Alpha

-IMG: International Medical Graduates

-COMPLEX: Comprehensive Osteopathic Medical Licensing Examination

Ethics approval and consent to participate: The study was exempted by the Penn State College of Medicine’s IRB.

Ethical declarations: Informed consent was obtained from all participants. There were no participants under 18. All methods were carried out in accordance to relevant guidelines and regulations.

Consent for publication: Our survey invitation emails with the appropriate informed consent information were sent to each PD. Submission of the survey indicated the respondents’ consent to participate. All responses were collected anonymously.

Availability of data and materials: The datasets generated and/or analyzed during the current study are not publicly available because the survey was conducted independently at our institution, but are available from the corresponding author on reasonable request.

Competing interests: The authors declare that they have no competing interests.

Funding: No funding was utilized.

Authors' contributions: All authors made substantial contributions to the conception and design of the project. FM, ARS, DC, SJ, and EBL acquired, analyzed, and interpreted the data.

FM, AS, WLH, and EFH were major contributors in writing the manuscript. All authors read and approved the final manuscript.

Acknowledgements: Not applicable

Authors' information: Eileen Hennrikus is a professor of internal medicine at Penn State College of Medicine. William Hennrikus is professor of orthopaedics, and associate dean of continuing education at Penn State College of Medicine. Paul Juliano is the program director of Penn State’s orthopaedic surgery residency program. Alia Chisty is the program director of Penn State’s internal medicine residency program.

USMLE: Change to Pass/Fail Score Reporting for Step 1. https://www.usmle.org/incus (2020). Accessed 1 Mar 2020.
Committee to Evaluate the USMLE Program (CEUP): Comprehensive Review of USMLE: Summary of the Final Report and Recommendations. https://www.usmle.org/pdfs/cru/CEUP-Summary-Report-June2008.pdf (2008). Accessed 1 Mar 2020.
National Resident Matching Program, Data Release and Research Committee: Results of the 2018 NRMP Program Director Survey. https://www.nrmp.org/wp-content/uploads/2018/07/NRMP-2018-Program-Director-Survey-for-WWW.pdf (2018). Accessed 1 Mar 2020.
Kay C, Jackson JL, Frank M. The relationship between internal medicine residency graduate performance on the ABIM certifying examination, yearly in-service training examinations, and the USMLE Step 1 examination. Acad Med. 2015;90(1):100-104. doi:10.1097/ACM.0000000000000500
Kenny S, McInnes M, Singh V. Associations between residency selection strategies and doctor performance: a meta-analysis. Med Educ. 2013;47(8):790-800. doi:10.1111/medu.12234
Lee AG, Oetting TA, Blomquist PH, et al. A multicenter analysis of the ophthalmic knowledge assessment program and American Board of Ophthalmology written qualifying examination performance. Ophthalmology. 2012;119(10):1949-1953. doi:10.1016/j.ophtha.2012.06.010
McCaskill QE, Kirk JJ, Barata DM, Wludyka PS, Zenni EA, Chiu TT. USMLE step 1 scores as a significant predictor of future board passage in pediatrics. Ambul Pediatr. 2007;7(2):192-195. doi:10.1016/j.ambp.2007.01.002
Angus SV, Williams CM, Stewart EA, Sweet M, Kisielewski M, Willett LL. Internal Medicine Residency Program Directors' Screening Practices and Perceptions About Recruitment Challenges. Acad Med. 2020;95(4):582-589. doi:10.1097/ACM.0000000000003086
National Resident Matching Program, Results and Data: 2020 Main Residency Match. https://www.nrmp.org/main-residency-match-data/ (2020). Accessed 10 Oct 2020.
Twitter: UMKC Orthopaedic Surgery Residency. https://twitter.com/umkcortho/status/1296124456897785857?lang=en (2020). Accessed 22 Aug 2020.
Aibana O, Swails JL, Flores RJ, Love L. Bridging the Gap: Holistic Review to Increase Diversity in Graduate Medical Education. Acad Med. 2019;94(8):1137-1141. doi:10.1097/ACM.0000000000002779
ACGME Department of Information Services/Applications and Data Analysis: ACGME Data Resource Book 2019-2020. https://www.acgme.org/About-Us/Publications-and-Resources/Graduate-Medical-Education-Data-Resource-Book (2019). Accessed 1 Mar 2020.
Boulet JR, Pinsky WW. Reporting a Pass/Fail Outcome for USMLE Step 1: Consequences and Challenges for International Medical Graduates. Acad Med. 2020;95(9):1322-1324. doi:10.1097/ACM.0000000000003534
Desai A, Hegde A, Das D. Change in Reporting of USMLE Step 1 Scores and Potential Implications for International Medical Graduates. JAMA. 2020;323(20):2015-2016. doi:10.1001/jama.2020.2956
Aziz F, Bechara CF. USMLE Step 1 Scoring System Change to Pass/Fail-Implications for International Medical Graduates [published online ahead of print, 2020 Sep 2]. JAMA Surg. 2020;10.1001/jamasurg.2020.2850. doi:10.1001/jamasurg.2020.2850
Salari S, Deng F. A Stepping Stone Toward Necessary Change: How the New USMLE Step 1 Scoring System Could Affect the Residency Application Process. Acad Med. 2020;95(9):1312-1314. doi:10.1097/ACM.0000000000003501
Aiyer A, Sankar V, Summers S, et al. Unifying the Orthopaedic Surgery Residency Application Process Under a Single Accreditation System: A Primer. J Am Acad Orthop Surg. 2020;28(7):263-267. doi:10.5435/JAAOS-D-19-00755

Table 1. A Comparison of Orthopaedic and Internal Medicine Program Directors’ Perceptions of the USMLE Step 1 Pass/Fail Transition

	Orthopaedics N (%)	Internal Medicine N (%)	Odds ratio (95% CI)	p-value
*How do orthopaedic and internal medicine program directors feel about the pass/fail Step 1?*
Do not agree with the change to pass/fail Step 1	48 (82.8)	93 (74.4)	1.36 (0.75-2.46)	0.315
The decision to transition to pass/fail Step 1 was not transparent and did not adequately involve all stakeholders	50 (86.2)	99 (79.2)	1.26 (0.71-2.26)	0.430
A graded Step 1 adequately measured the ability of an applicant to succeed	25 (43.1)	49 (39.2)	1.18 (0.67-2.06)	0.570
*How will a pass/fail Step 1 impact orthopaedic and internal medicine residency programs?*
A pass/fail Step 1 will not make the match process fair and meritocratic	25 (43.1)	76 (60.8)	*0.53 (0.30-0.93)	0.026
A pass/fail Step 1 will help to create better future physicians	12 (20.7)	13 (10.4)	1.46 (0.83-2.56)	0.190

*How will the pass/fail change impact the importance of the factors reviewed by orthopaedic and internal medicine residency programs when assessing applicants?*
Step 1 exam result will be less important	52 (89.7)	87 (69.6)	*3.74 (2.00-7.01)	<0.001
Step 2 CK will be more important	56 (96.6)	110 (88.0)	1.56 (0.83-2.91)	0.167
Step 2 CS will be more important	23 (39.7)	54 (43.2)	0.78 (0.43-1.41)	0.406
Grades in required clerkships will be more important	45 (77.6)	84 (67.2)	1.76 (0.98-3.18)	0.059
Research experience will be more important	29 (50.0)	17 (13.6)	*6.43 (3.18-13.00)	<0.001
Letters of recommendation from orthopaedic/internal medicine faculty that program directors know will be more important	44 (75.9)	71 (56.8)	*3.22 (1.77-5.87)	<0.001
Letters of recommendation from orthopaedic/internal medicine faculty that program directors do not know will be more important	30 (51.7)	53 (42.4)	1.65 (0.91-2.99)	0.098
Letters of recommendation from faculty not within specialty will be more important	9 (15.5)	15 (12.0)	0.71 (0.35-1.43)	0.334
Personal statement will be more important	14 (24.1)	26 (20.8)	1.17 (0.57-2.42)	0.664
Medical student performance evaluations (MSPE)/Dean’s letter will be more important	25 (43.1)	66 (52.8)	0.62 (0.34-1.13)	0.116
Alpha Omega Alpha (AOA) membership will be more important	39 (67.2)	36 (28.8)	*4.96 (2.65-9.25)	<0.001
Gold Humanism Society membership will be more important	22 (37.9)	33 (26.4)	1.83 (0.96-3.50)	0.068
Leadership/extracurriculars will be more important	34 (58.6)	33 (26.4)	*3.69 (1.96-6.95)	<0.001
Personal knowledge of applicant will be more important	47 (81.0)	82 (65.6)	*2.93 (1.61-5.34)	<0.001
Audition electives within department will be more important	50 (86.2)	79 (63.2)	*6.20 (3.21-11.97)	<0.001
*How will various applicant groups be affected by the change to a pass/fail Step 1?*
All MD students will be advantaged	10 (17.2)	29 (23.2)	0.61 (0.35-1.09)	0.094
MD students who attend a highly-regarded medical schools will be advantaged	35 (60.3)	69 (55.2)	1.17 (0.66-2.05)	0.593
MD students who do not attend a highly-regarded medical school will be disadvantaged	44 (75.9)	79 (63.2)	*2.14 (1.19-3.88)	0.012
DO students will be disadvantaged	37 (63.8)	65 (52.0)	*2.59 (1.45-4.61)	0.001
International medical graduates (IMGs) will be disadvantaged	34 (58.6)	104 (83.2)	0.631 (0.35-1.13)	0.118
*How will changing to a pass/fail Step 1 affect medical students interested in orthopaedics and internal medicine?*
Allow students to focus more on learning medicine rather than studying for Step 1	13 (22.4)	32 (25.6)	0.84 (0.41-1.75)	0.642
Encourage more research experiences	23 (39.7)	12 (9.6)	*6.19 (2.80-13.69)	<0.001
Encourage more leadership/extracurriculars	16 (27.6)	23 (18.4)	1.69 (0.81-3.51)	0.160
Allow students to pursue more hobbies/self-development	6 (10.3)	27 (21.6)	0.42 (0.16-1.08)	0.072
Encourage students to attend more audition electives	34 (58.6)	36 (28.8)	*3.50 (1.83-6.71)	<0.001
Encourage applicants to apply to more residency programs	39 (67.2)	71 (56.8)	1.56 (0.81-3.00)	0.181
Encourage applicants to apply to other specialties in addition to their primary specialty of interest	27 (46.6)	49 (39.2)	1.32 (0.72-2.53)	0.348
*What are the future implications on residency applications and medical education?*
With the change to pass/fail Step 1, should medical schools adopt a graded pre-clinical curriculum?	37 (63.8)	65 (52.0)	1.10 (0.51-2.35)	0.810
With the change to pass/fail Step 1, should there be a cap on the number of residency applications a medical student can submit?	42 (72.4)	69 (55.2)	1.70 (0.75-3.86)	0.201

Table 1 shows the distribution of responses for our survey sent to all orthopaedics and internal medicine residency program directors in the United States. The bolded and italicized text in the table are the main questions asked in our survey. The odds ratios compare orthopaedics and internal medicine program director responses. The p-value reports the significance of the odds ratios. An asterisk (*) indicates statistically significance (p<0.05).

No competing interests reported.

Title: A Comparison of Orthopaedic Surgery and Internal Medicine Perceptions of USMLE Step 1 Pass/Fail Scoring: A National Program Director Survey

Status:

Version 1

Abstract

Background

Methods

Results

Conclusion

Background

Methods

Results

Discussion

Conclusions

Abbreviations

Declarations

References

Table

Additional Declarations

Status:

Version 1