Do USMLE Steps, and ITE Score Predict the American Board of Internal Medicine Certifying Exam results?

doi:10.21203/rs.2.12218/v3

Download PDF

Research article

Do USMLE Steps, and ITE Score Predict the American Board of Internal Medicine Certifying Exam results?

https://doi.org/10.21203/rs.2.12218/v3

This work is licensed under a CC BY 4.0 License

Journal Publication

published 18 Mar, 2020

Read the published version in BMC Medical Education →

You are reading this latest preprint version

Objective: To evaluate if United States Medical Licensing Examination (USMLE) Step 1, USMLE Step 2 CK, USMLE Step 3, and residency third-year in-service training exam (ITE) results predict the results of residents in American Board of Internal Medicine Certifying Exam (ABIM-CE). Methods: A retrospective review of USMLE Step 1, USMLE Step 2, USMLE Step 3 scores, residency third year ITE and ABIM-CE of IM residents at our residency program from 2004 through 2017 was conducted. Pearson correlation coefficient and two-sample t-tests were used to assess the relationship between various scores. Multivariate logistic regression was used to predict pass or fail results in ABIM-CE using USMLE and third-year ITE test scores controlling for other covariates. Results: Among 114 MD residents included in the study; 92% (n=105) passed the ABIM-CE. USMLE score was a significant predictor of passing ABIM-CE. The OR of passing ABIM-CE was 2.70 (95 % CI=1.38-5.29), 2.31 (95% CI=1.33-4.01), and 1.63 (95% CI=0.81-3.29) with a ten-point increase in USMLE Step 1, USMLE Step 2 and USMLE step 3 scores respectively. OR of ABIM-CE passing chance was 2.96 (95% CI=0.95-9.20) with a ten-point increase in the average score of the above three exams. A five percent increase in ITE percentage raised the likelihood of passing ABIM-CE (OR 2.92, 95% CI 1.15-7.38). All residents who failed ABIM-CE had Step 1 scores <220. Among 31 residents with Step 2 score <220, 20% (n=6) failed ABIM. Similarly, 9% of residents with USMLE Step 3 score < 220 failed ABIM-CE; all residents who failed had scored < 220. The probability curve predicted that the chance of passing ABIM- CE was around 80% with USMLE scores greater than 200 and increased to almost 100% with USMLE scores of 250 or more. Conclusion: USMLE Step 1, USMLE Step 2, and third-year ITE scores strongly predict the chances of passing ABIM-CE. Thus, programs can identify internal medicine residents at risk of failing ABIM-CE and need intervention at an early stage. Various measures such as enrolling them in question banks or board review courses can then be implemented by programs to improve their chances to pass.

Educational Philosophy and Theory

USMLE Steps and In-Training Exam Results

Performance

American Board of Internal Medicine Certifying Exam

Competency assessment of residents in a training program is a complex process. Residency programs use numerous tools to evaluate the six medical core competencies in order to assess the overall performance of a resident [1]. American Board of Internal Medicine (ABIM) uses various factors to test all the competencies of the physicians and their ability to deliver high-quality care. The factors include fulfillment of the graduate medical education training requirements, demonstration of clinical competence in patient care, and passing the certification examination in internal medicine (IM) [2]. The ABIM Certifying Exam (ABIM-CE) is administered annually and is taken by the residents just after completion of their training. The performance of the residents in ABIM-CE is essential not only for the residents but also for the programs. In fact, for a residency program to maintain its accreditation, the graduates must have an 80% pass rate in ABIM-CE from the first-time takers of the exam in the latest three-year period [3].

Various tools are used by IM residency programs to select the best candidates who will perform well in their program and maintain the 'Board-pass rate.' United States Medical Licensing Examination (USMLE) scores are commonly used by residency programs to assess the applicants for residency [4-6]. During the residency training, the annual in-service training exam (ITE) is one of the tools used to assess the medical knowledge of the residents. A national standardized ITE has been developed in different specialties, including IM, to provide feedback to the residents and the training program [7]. By assessing the knowledge gap (the deficit in the knowledge that can be improved to achieve better scores), ITE exams inform the residents and the training program about areas in need of focused education. Thus, performance in ITE exams is used by the training programs to evaluate the readiness and ability of their residents to take and pass ABIM-CE [7].

However, there is a paucity of research and data on the correlation of baseline mandatory licensing exam scores to the success in ABIM-CE. In the past, several studies have reported variable degrees of correlation between USMLE and ITE scores [5,6,8,9]. A few other studies have also reported an association between USMLE scores and IM-ITE performances [10,11]. One study reported a modest correlation between USMLE Step 1 and ITE scores with ABIM-CE [12]. Here we analyzed whether the USMLE Step 1, USMLE Step 2 CK, USMLE Step 3, and residency third year ITE results correlated with passing the ABIM-CE.

Study sample and characteristics

All residents enrolled in the IM residency training program at our residency program from 2004 through 2017 were included. The data collected included gender; type of degree (MD versus DO); medical school country (American Medical Graduate [AMG] versus International Medical Graduate [IMG]); scores in USMLE Step 1, USMLE Step 2 CK, and USMLE Step 3; gap (number of years) between graduation from medical school and start of residency (referred to as ‘the gap’); and ABIM-CE results (Pass versus Fail). The number of osteopathic residents (DO) was considerably less than allopathic residents (MD), and most of them took COMLEX exams instead of USMLE Steps. Additionally, data were unavailable for a few MD residents. Thus, after excluding DO residents (n=18) and MD residents with incomplete data (n=9), the remaining MD residents were included in the study.

Scores of ITE from the third year of residency, the exam closest to ABIM-CE in the timeline, were used in our analysis as third-year ITE scores are supposed to reflect the readiness of the residents to take ABIM-CE.

Statistical analysis:

We presented all continuous data as mean (SD), and categorical data as N(%), Statistical analyses were based on scaled scores. We used the Pearson correlation coefficient to assess the correlation between USMLE Step 1, USMLE Step 2 CK, USMLE Step 3, and third-year ITE scores. We performed logistic regression to assess the relationship between various scores separately and pass or fail results in ABIM-CE. Also, we performed multivariate logistic regression to examine the relationship between the scores and the ABIM-CE results controlling for other confounding variables such as gender, medical school country, the gap between medical school and residency program. Significance was set at p < 0.05. We used SAS 9.4 software (SAS Institute Inc., Cary, North Carolina) to perform statistical analysis.

Ethics approval

We received an exemption from the approval of Michigan State University Human Research Protection Program – MSU institutional board review (IRB# x16-029e). We did not obtain consent from individual graduates, the data is de-identified, and the ethics committee approved this procedure.

We included a total of 114 MD residents in the study; 92% (n=105) passed the ABIM-CE. The characteristics of the residents, along with their mean scores in different exams, are shown in Table 1. Mean scores in different exams for residents who passed ABIM-CE versus those who did not pass are shown in Table 2.

All residents (n=9) who failed ABIM-CE had Step 1 scores <220 which was 25% of total residents with Step 1 score of <220 (n=35). Among 31 residents with Step 2 CK score <220, 20% (n=6) failed ABIM whereas only 10% (n=3) with Step 2 CK score >220 failed ABIM-CE. Similarly, 9% of residents with USMLE Step 3 score < 220 failed ABIM-CE; all residents who failed had scored < 220.

First, four logistic regressions are employed to predict ABIM-CE passing with USMLE Step 1, USMLE Step 2 CK, USMLE Step 3, and ITE tests separately controlling for gender, country of the medical school, and 1 year of the gap between medical school and the start of residency. There was a significant correlation of passing ABIM-CE with 10 points increase in USMLE Step 1 (Odds ratio [OR] 2.70; 95% Confidence Interval [CI] 1.38- 5.29) and 10 points increase in Step 2 CK (2.31; 95% CI 1.33-4.01). However, a 10 points increase in Step 3 (OR 1.63; 95% CI 0.81- 3.29) did not significantly predict passing ABIM-CE. A 5% increase in ITE percentage increased the likelihood of passing ABIM-CE (OR 3.89, 95% CI 1.68-8.98).

Next, we calculated the average of USMLE Step scores and predicted ABIM-CE pass using the average Step scores. Table 3 shows the results. The 10 points increase in average Step scores will lead to a higher chance of passing ABIM-CE tests (OR=2.96, 95% CI=0.95-9.20)) but not significantly at α=0.05 level. There was no relationship between ABIM-CE performance and gender, the country of medical school (American medical graduate = AMG vs. International medical graduate = IMG). The increase in the gap (1year) was not significantly associated with the chances of passing ABIM-CE (OR 0.82; 95% CI 0.364-1.06) (Table 3).

The probability curve (with third year-ITE percentage and the gap set at mean values) predicted that the chance of passing ABIM- CE was around 80% with USMLE scores higher than 200 and increased to almost 100% with USMLE scores of 240 or more (Figure 1). A ROC (receiver-operating characteristic) curve was computed to assess the accuracy of the model’s ability to predict passing the ABIM-CE. The area under the ROC curve was 0.945 (Figure 2), which indicates that the probability of our model will rank a randomly chosen “pass the board” higher than “not pass the board” is 0.945.

We repeated the same analysis using standardized USMLE step scores and standardized ITE test scores with mean at zero and standard deviation (SD) at 1. One SD increase in USMLE Step 1 leads to a significantly higher chance of passing ABIM-CE (OR=6.41, 95% CI 1.84-22.46) and one SD increase in USMLE Step 2 CK leads to a significantly higher chance of passing ABIM-CE (OR=5.62, 95% CI 1.80-17.60). However, a 1 SD increase in USMLE Step 3 is not significantly associated with passing ABIM-CE (OR=1.89, 95% CI 0.76-4.68). Table 4 shows the results with a 1 SD increase in average USMLE score and ITE percentage increase controlling for demographic information. Our analysis also found that a 1 SD increase in the ITE test leads to a higher chance of passing ABIM (OR=5.0, 95% CI 1.24-20.25). The results indicate that one SD change in the ITE test has the most significant effect on increasing the chance of passing the ABIM-CE test.

USMLE Step 1, USMLE Step 2 CK, USMLE Step 3, and third-year ITE scores have a predictive value in determining the chances of passing ABIM-CE tests. Among them, ITE percentage has is the most predictive, followed by USMLE Step 1, USMLE Step 2 CK, and USMLE step 3 in the descending order.

Residency programs use USMLE scores as initial screening tools to select their potential candidates from residency applicants. USMLE comprises three steps - Step 1, Step 2, and Step 3, to evaluate the appropriate use of medical knowledge in patient care by the examinees. The USMLE Step 2 exam consists of two sections: Step 2 CK (Clinical Knowledge) and Step 2 CS (Clinical Skills). USMLE Step 1, 2 CK, and Step 3 are MCQ based and reports score in a three-digit format [13]. USMLE Step 2 CS tests clinical skills, and reports only pass or fail [13]. Previously, USMLE reported the scores in percentile format. However, in 1999, the percentile-based system was eliminated in favor of a three-digit and two-digit scaled scoring system. USMLE eliminated the two-digit score from the score report in April 2013.

USMLE exam scores range anywhere from 1 to 300, and most examinees score in the range of 140 to 260. USMLE Step 1 passing score is 194; the national mean approximately 229 (sd=20) [13]. According to the National Resident Matching Program (NRMP), the mean score for US allopathic candidates matching into residency programs in 2016 was 233.2 (sd=17.4) [13]. For the 2013-2014 and 2014-2015 academic years, the mean USMLE Step 2CK score is 240 (sd=18) for first-time takers from the medical schools in the United States and Canada. The minimum passing score for students taking USMLE Step 2 CK after July 1, 2014, is 209 and was 209, at least until May 2018 [13]. The minimum USMLE Step 3 passing score will be raised to 198 from 196 from the beginning of January 1, 2020 [13].

Among the USMLE Step exams, the USMLE Step 1 score is a predictive tool to assess applicants during the interview for internal medicine residency [14]. However, there is limited data to prove that USMLE scores have a strong correlation with the performance in ABIM-CE [12]. The results from our study showed that the scores in all USMLE Steps have a predictive value for passing the ABIM-CE, USMLE step 1 score being the strongest one. The chance of failing ABIM-CE is higher with USMLE Scores below 220, more so with USMLE Step 1 than USMLE Step 2 CK or USMLE Step 3. Kay et al. reported a modest correlation between USMLE Step 1 and ABIM-CE scores [13]. There are reports of consistent results showing a correlation between USMLE scores and the results of the certification examination in various specialties and subspecialties [15-19]. Higher USMLE Step 1, USMLE Step 2 CK, and composite scores were also associated with better performance on Emergency Medicine boards, with Step 2 being the strongest predictor [18]. Low USMLE Step 1 score was also predictive of failing certifying exams in surgery and pediatrics [19,20].

Performance in ITE is another tool used to predict the score of a resident in the ABIM-CE. ITE scores usually improve from the first year to the third year in IM [13]. Previous ITE scores are strongly associated with subsequent ITE than USMLE scores [21]. ITE scores are useful for residency programs to recognize residents who may need assistance or interventions from the program to increase their chances to pass the ABIM-CE. Since the third year-ITE is the exam closest to the ABIM-CE, the results of the third year ITE can be used to gauge the readiness of a resident to take ABIM-CE. In our study, third year-ITE percent had a positive correlation with ABIM-CE performance, which was slightly better than USMLE scores. Kay et al. also found a modest correlation between ITE and ABIM-CE scores [13]. Also, reports suggest that residents scoring in the bottom quartile on their ITE were at increased risk of failing boards while those who scored in the top quartile of the ITE had a 100% pass rate [13]. In the past, few other studies looked at the association between ABIM-CE results and ITE scores with similar results [21-24]. Babott et al. reported that second-year ITE scores of more than 61% predicted a 100% pass rate in ABIM-CE with 41% sensitivity and 100% specificity [7]. Brateanu and colleagues developed a nomogram to predict the ABIM-CE performance, which included the ITE scores of each year and the number of overnight calls in the last six months of residency [15]. Their analysis reported that the third year-ITE was the most important predictor. Univariate analysis showed a good correlation of USMLE results with ABIM-CE; however, multivariate regression did not show a statistically significant correlation between USMLE and ABIM-CE results.

Additionally, the ITE scores are the most proximate data to ABIM-CE and would, to some extent, have some correlation with USMLE scores. The predictive value of ITE scores with the performance in certifying exams has been reported in other specialties as well [26-28,19]. A study in Emergency Medicine showed that third year-ITE scores were most predictive of the score in the certification exam [27]. Similarly, a low score in ITE at any time during residency increased the chances of failing a certification exam in surgery [19].

Passing in ABIM-CE is of utmost importance to residents [29], and their residency training programs as ACGME requires a first-attempt examinee pass rate of at least 80% for continued accreditation of the residency program. Hence, residency programs put a great deal of effort to select candidates with strong clinical skills and ones likely to perform well in ABIM-CE. Residency programs hold practice exams, teaching sessions, recommend mandatory usage of Question banks, and encourage enrollment in Board Review Courses to enhance the ABIM-CE passing rates.

A majority of the IMGs, unlike AMGs, tend to complete their USMLE Step exams after completion of their medical school curriculum. During their preparation for USMLE exams, IMGs also pursue observerships to familiarize themselves with the United States medical system and to improve their clinical skills. Also, few candidates spend time working as a researcher, and few pursue master’s programs such as Master of Public Health. Hence, the majority of the IMGs have a gap of 1 to 3 years between the completion of medical school and starting their residency. Brateanu et al. reported a weak positive correlation between the length of the gap and the performance in ABIM-CE [16]. Another study on IMGs by Kanna and colleagues showed that the gap between medical school and residency was not significant in predicting ITE scores [30]. Similarly, our study did not show a statistically significant change in the Odds of passing ABIM-CE with a gap 1-year gap between medical school and starting residency training. Also, it is noteworthy to mention that the gender differences and place of medical school (AMG vs. IMG) were statistically not significant. Therefore, there was no difference by gender or place of medical school training.

Our study has a few limitations. First, it is a single-center study with limited demographic information of the residents from one program. There is no tool to calculate the equivalence of COMLEX to USMLE, so we could not compare osteopathic residents with allopathic residents. There is a little variation in the ABIM-CE pass rate since nearly all of the candidates in our sample passed the exam. Additionally, there were too few subjects to analyze passing or failing USMLE Step 1 or Step 2 CK and its effect on passing ABIM-CE.

We need to consider the implication of these results in finding the residents who might need help with ABIM-CE. The exact nature and efficacy of remediation plans, including the effectiveness of board review courses, conferences, or self-study courses, in helping the resident pass the ABIM-CE are unclear [31]. Although there are reports of improved ABIM-CE results with a directed reading program and individual education plan for residents, literature regarding the efficacy of these strategies is limited [32,33]. Based on experience from the leadership of the program, we recommend programs to use USMLE step scores and 1st year ITE scores to identify residents needing interventions at the beginning of their residency. At our program, several residents, especially ones with scores <35 percentile in the PGY3 ITE exam, were benefited from enrolling in board review courses and using Question banks. Several residents reported that joining group discussions or studying with a partner was beneficial. Furthermore, many other unexplained factors determine whether a resident passes the ABIM-CE. The scores described above do not measure factors such as interpersonal skills, professionalism, or shared decision-making capabilities, which are equally important for a competent physician.

USMLE Step 1, USMLE Step 2 CK, and third-year ITE scores are strong predictors of the chances of passing ABIM-CE. The third-year ITE score is the strongest predictor. Both USMLE Step 1 and Step 2 CK scores >220 are independent predictors of success in ABIM-CE as well, but the USMLE Step 1 score is the stronger predictor. Thus, programs can identify internal medicine residents at risk of failing ABIM-CE and need intervention at an early stage. Various measures, such as enrolling them in question banks or board review courses, can then be implemented by programs to improve their chances to pass.

USMLE: United States Medical Licensing Examination;

COMPLEX: The Comprehensive Osteopathic Medical Licensing Examination

IM: Internal Medicine

ABIM: American Board of Internal Medicine;

ITE: In training exams;

ABIM-CE: American Board of Internal Medicine Certifying Exam;

ROC: receiver-operating characteristic

Ethics approval and consent to participate: We received an exemption from the approval of Michigan State University Human Research Protection Program – MSU institutional board review (IRB# x16-029e). We obtained consent from all participants, and the ethics committee approved this procedure.

Consent for publication: Not applicable.

Availability of data and material: We have provided the raw data as a supplemental file.

Competing interests: The authors declare no competing interests.

Funding: We did not receive any research funding for this project.

Author contributions: SR and PD were involved in the conception and design of the study; intellectual content, literature search, data acquisition. LW performed data analysis and statistical analysis. SR and PD prepared the first draft of the manuscript. SS and MPR contributed to the literature search, manuscript editing, and review. MPR, SR, and LW performed a revision of the manuscript. All authors read and approved the final version of the manuscript.

Acknowledgments: Not applicable.

1. History of Medical Education http://www.acgme.org/About-Us/Overview/History-of-Medical-Education/About/ACGMEHistory. Accessed April 1, 2018

2. ABIM Board Certification http://www.abim.org/certification/policies.aspx. Accessed April 1, 2018

3. ACGME Program Requirements for Graduate Medical Education in Internal Medicine https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/140_internal_medicine_2017-07-01.pdf. Accessed April 1, 2018

4. Berner ES, Brooks CM, Erdmann JB (1993) Use of the USMLE to select residents. Academic medicine : journal of the Association of American Medical Colleges 68 (10):753-759

5. Spurlock DR, Jr., Holden C, Hartranft T (2010) Using United States Medical Licensing Examination((R)) (USMLE) examination results to predict later in-training examination performance among general surgery residents. Journal of surgical education 67 (6):452-456. doi:10.1016/j.jsurg.2010.06.010

6. Dougherty PJ, Walter N, Schilling P, Najibi S, Herkowitz H (2010) Do scores of the USMLE Step 1 and OITE correlate with the ABOS Part I certifying examination?: a multicenter study. Clinical orthopaedics and related research 468 (10):2797-2802. doi:10.1007/s11999-010-1327-3

7. Babbott SF, Beasley BW, Hinchey KT, Blotzer JW, Holmboe ES The Predictive Validity of the Internal Medicine In-Training Examination. The American Journal of Medicine 120 (8):735-740. doi:10.1016/j.amjmed.2007.05.003

8. Thundiyil JG, Modica RF, Silvestri S, Papa L (2010) Do United States Medical Licensing Examination (USMLE) scores predict in-training test performance for emergency medicine residents? The Journal of emergency medicine 38 (1):65-69. doi:10.1016/j.jemermed.2008.04.010

9. Crawford CH, 3rd, Nyland J, Roberts CS, Johnson JR (2010) Relationship among United States Medical Licensing Step I, orthopedic in-training, subjective clinical performance evaluations, and american board of orthopedic surgery examination scores: a 12-year review of an orthopedic surgery residency program. Journal of surgical education 67 (2):71-78. doi:10.1016/j.jsurg.2009.12.006

10. McDonald FS, Zeger SL, Kolars JC (2008) Associations between United States Medical Licensing Examination (USMLE) and Internal Medicine In-Training Examination (IM-ITE) scores. Journal of general internal medicine 23 (7):1016-1019. doi:10.1007/s11606-008-0641-x

11. Perez JA, Jr., Greer S (2009) Correlation of United States Medical Licensing Examination and Internal Medicine In-Training Examination performance. Advances in health sciences education : theory and practice 14 (5):753-758. doi:10.1007/s10459-009-9158-2

12. Kay C, Jackson JL, Frank M (2015) The relationship between internal medicine residency graduate performance on the ABIM certifying examination, yearly in-service training examinations, and the USMLE Step 1 examination. Academic medicine : journal of the Association of American Medical Colleges 90 (1):100-104. doi:10.1097/acm.0000000000000500

13. United States Medical Licensing Examination ® [Internet]. [cited 2020 Feb 1]. Available from: https://www.usmle.org/

14. Neely D, Feinglass J, Wallace WH (2010) Developing a predictive model to assess applicants to an internal medicine residency. Journal of graduate medical education 2 (1):129-132. doi:10.4300/jgme-d-09-00044.1

15. Brateanu A, Yu C, Kattan MW, Olender J, Nielsen C (2012) A nomogram to predict the probability of passing the American Board of Internal Medicine examination. Medical Education Online 16:10.3402/meo.v3417i3400.18810. doi:10.3402/meo.v17i0.18810

15. Sisson SD, Bertram A, Yeh HC (2015) Concurrent Validity Between a Shared Curriculum, the Internal Medicine In-Training Examination, and the American Board of Internal Medicine Certifying Examination. Journal of graduate medical education 7 (1):42-47. doi:10.4300/jgme-d-14-00054.1

17. Indik JH, Duhigg LM, McDonald FS, Lipner RS, Rubright JD, Haist SA, Botkin NF, Kuvin JT (2017) Performance on the Cardiovascular In-Training Examination in Relation to the ABIM Cardiovascular Disease Certification Examination. Journal of the American College of Cardiology 69 (23):2862-2868. doi:10.1016/j.jacc.2017.04.020

18. Harmouche E, Goyal N, Pinawin A, Nagarwala J, Bhat R (2017) USMLE Scores Predict Success in ABEM Initial Certification: A Multicenter Study. Western Journal of Emergency Medicine 18 (3):544-549. doi:10.5811/westjem.2016.12.32478

19. de Virgilio C, Yaghoubian A, Kaji A, et al. (2010) Predicting performance on the american board of surgery qualifying and certifying examinations: A multi-institutional study. Archives of Surgery 145 (9):852-856. doi:10.1001/archsurg.2010.177

20. McCaskill QE, Kirk JJ, Barata DM, Wludyka PS, Zenni EA, Chiu TT (2007) USMLE step 1 scores as a significant predictor of future board passage in pediatrics. Ambulatory pediatrics : the official journal of the Ambulatory Pediatric Association 7 (2):192-195. doi:10.1016/j.ambp.2007.01.002

21. McDonald FS, Zeger SL, Kolars JC (2008) Associations Between United States Medical Licensing Examination (USMLE) and Internal Medicine In-Training Examination (IM-ITE) Scores. Journal of general internal medicine 23 (7):1016-1019. doi:10.1007/s11606-008-0641-x

22. Grossman RS, Fincher RM, Layne RD, Seelig CB, Berkowitz LR, Levine MA (1992) Validity of the in-training examination for predicting American Board of Internal Medicine certifying examination scores. Journal of general internal medicine 7 (1):63-67

23. Grossman RS, Murata GH, Fincher RM, Norcini JJ, Kapsner C, Layne RD, Seelig CB, Gateley A (1996) Predicting performance on the American Board of Internal Medicine Certifying Examination: the effects of resident preparation and other factors. Crime Study Group. Academic medicine : journal of the Association of American Medical Colleges 71 (10 Suppl):S74-76

24. Rollins LK, Martindale JR, Edmond M, Manser T, Scheld WM (1998) Predicting pass rates on the American Board of Internal Medicine certifying examination. Journal of general internal medicine 13 (6):414-416

25. Waxman H, Braunstein G, Dantzker D, Goldberg S, Lefrak S, Lichstein E, Ratzan K, Schiffman F (1994) Performance on the internal medicine second-year residency in-training examination predicts the outcome of the ABIM certifying examination. Journal of general internal medicine 9 (12):692-694

26. Althouse LA, McGuinness GA (2008) The in-training examination: an analysis of its predictive value on performance on the general pediatrics certification examination. The Journal of pediatrics 153 (3):425-428. doi:10.1016/j.jpeds.2008.03.012

27. Frederick RC, Hafner JW, Schaefer TJ, Aldag JC (2011) Outcome measures for emergency medicine residency graduates: do measures of academic and clinical performance during residency training correlate with American Board of Emergency Medicine test performance? Academic Emergency Medicine 18 (s2)

28. Baumgartner BR, Peterman SB (1996) Relationship between American College of Radiology in-training examination scores and American Board of Radiology written examination scores. Academic radiology 3 (10):873-878

29. Atsawarungruangkit A (2015) Relationship of residency program characteristics with pass rate of the American Board of Internal Medicine certifying exam. Medical Education Online 20:10.3402/meo.v3420.28631. doi:10.3402/meo.v20.28631

30. Kanna B, Gu Y, Akhuetie J, Dimitrov V (2009) Predicting performance using background characteristics of international medical graduates in an inner-city university-affiliated Internal Medicine residency training program. BMC Medical Education 9:42-42. doi:10.1186/1472-6920-9-42

31. FitzGerald JD, Wenger NS (2003) Didactic teaching conferences for IM residents: who attends, and is attendance related to medical certifying examination scores? Academic medicine : journal of the Association of American Medical Colleges 78 (1):84-89

32. Drake SM, Qureshi W, Morse W, Baker-Genaw K (2015) A time-efficient web-based teaching tool to improve medical knowledge and decrease ABIM failure rate in select residents. Medical Education Online 20:10.3402/meo.v3420.29221. doi:10.3402/meo.v20.29221

33. Visconti A, Gaeta T, Cabezon M, Briggs W, Pyle M (2013) Focused Board Intervention (FBI): A Remediation Program for Written Board Preparation and the Medical Knowledge Core Competency. Journal of graduate medical education 5 (3):464-467. doi:10.4300/jgme-d-12-00229.1

Table 1: Demographic characteristics and examination scores

Gender (n, %)
Female	34 (29.8)
Male	80 (70.2)
Medical School Country (n, %)
United States or Canada	17 (14.9)
International	97 (85.1)
Gap between medical school graduation and start of residency (years, mean (range))	3.1 (0-17)
USMLE Scores (mean (SD))
Step 1	230 (19)
Step 2	230 (21)
Step 3	208 (13)
Mean of Step 1, Step 2 and Step 3	222 (14)
Third-year ITE percentage (mean (SD))	68 (7)
ABIM-CE results (n, %)
Pass	105 (92.1)
Fail	9 (7.9)

Table 2: USMLE and ITE scores of residents who passed ABIM-CE versus those who did not

	Scores (Mean (SD))		p-value
	ABIM-CE Pass	ABIM-CE Fail	p-value
USMLE Step 1	231 (18)	209 (12)	<0.001
USMLE Step 2	233 (19)	204 (19)	<0.001
USMLE Step 3	208 (13)	200 (6)	0.07
Third-year ITE	69 (7)	59 (6)	<0.001

ABIM-CE: American Board of Internal Medicine- Certifying Exam; ITE: In-Training Exam; SD: Standard Deviation; USMLE: United States Medical Licensing Exam

Table 3: Odds of passing ABIM-CE in relation to demographic characteristics and scores

Variables	Analysis of maximum likelihood estimates
Variables	Odd’s ratio	p-value
Mean of USMLE Step 1, Step 2, & Step 3 (10 points)	2.961	0.0605
Gap between medical school and residency (1 year)	0.822	0.1231
Third-year- ITE percentage (5 points)	2.916	0.0240*
Gender of the residents (Female vs. Male)	2.754	0.3038
Medical school country (AMG vs. IMG)	4.969	0.2694

AMG: American Medical Graduate, IMG: International Medical Graduate, ITE: In-Training Exam, USMLE: United States Medical Licensing Exam

* significant with p-value <0.05.

Table 4: Odds of passing ABIM-CE in relation to demographic characteristics and standardized scores

Variables	Analysis of maximum likelihood estimates
Variables	Odd’s ratio	p-value
Mean of USMLE Step 1, Step 2, & Step 3 (1 SD increase)	4.686	0.0605
Gap between medical school and residency (1 year)	0.822	0.1231
Third-year- ITE percentage (1 SD increase)	5.004	0.0240*
Gender of the residents (Female vs. Male)	2.754	0.3038
Medical school country (AMG vs. IMG)	4.969	0.2694

* significant with p-value <0.05.

ITEABIMfinal.xlsx

Download PDF

Journal Publication

published 18 Mar, 2020

Read the published version in BMC Medical Education →

Editorial decision: Accept
19 Feb, 2020
Editor assigned by journal
05 Feb, 2020
Submission checks completed at journal
04 Feb, 2020
Editor invited by journal
04 Feb, 2020

You are reading this latest preprint version

Do USMLE Steps, and ITE Score Predict the American Board of Internal Medicine Certifying Exam results?

Status:

Journal Publication

Version 3

Abstract

Figures

Introduction

Methods

Results

Discussion

Conclusion

Abbreviations

Declarations

References

Tables

Supplementary Files

Status:

Journal Publication

Version 3