Do USMLE Steps and ITE Results Predict Performance in the American Board of Internal Medicine Certifying Exam?

doi:10.21203/rs.2.12218/v2

Download PDF

Research article

Do USMLE Steps and ITE Results Predict Performance in the American Board of Internal Medicine Certifying Exam?

https://doi.org/10.21203/rs.2.12218/v2

This work is licensed under a CC BY 4.0 License

Journal Publication

published 18 Mar, 2020

Read the published version in BMC Medical Education →

You are reading this older preprint version

Read the latest preprint version →

Objective: To evaluate if United States Medical Licensing Examination (USMLE) Step 1, USMLE Step 2, USMLE Step 3, and residency third-year in-service training exam (ITE) results predict performance of Internal Medicine (IM) residents in American Board of Internal Medicine Certifying Exam (ABIM-CE). Methods: A retrospective review of USMLE Step 1, USMLE Step 2, USMLE Step 3 scores, residency third year ITE and ABIM-CE of IM residents at our residency program from 2004 through 2017 was conducted. Pearson correlation coefficient and two-sample t-tests were used to assess the relationship between various scores. Multivariate logistic regression was used to predict pass or fail results in ABIM-CE using USMLE and third-year ITE test scores controlling for other covariates. Results: Among 114 MD residents included in the study; 92% (n=105) passed the ABIM-CE. USMLE score was a significant predictor of passing ABIM-CE. The OR of passing ABIM-CE was 2.70 (95 % CI=1.38-5.29), 2.31 (95% CI=1.33-4.01), and 1.63 (95% CI=0.81-3.29) with a ten-point increase in USMLE Step 1, USMLE Step 2 and USMLE step 3 scores respectively. OR of ABIM-CE passing chance was 2.96 (95% CI=0.95-9.20) with a ten-point increase in the average score of the above three exams. A five percent increase in ITE percentage raised the likelihood of passing ABIM-CE (OR 2.92, 95% CI 1.15-7.38). All residents who failed ABIM-CE had Step 1 scores <220. Among 31 residents with Step 2 score <220, 20% (n=6) failed ABIM. Similarly, 9% of residents with USMLE Step 3 score < 220 failed ABIM-CE; all residents who failed had scored < 220. The probability curve predicted that the chance of passing ABIM- CE was around 80% with USMLE scores greater than 200 and increased to almost 100% with USMLE scores of 250 or more. Conclusion: USMLE Step 1, USMLE Step 2, and third-year ITE scores strongly predict the chances of passing ABIM-CE. Thus, programs can identify internal medicine residents at risk of failing ABIM-CE and need intervention at an early stage. Various measures such as enrolling them in question banks or board review courses can then be implemented by programs to improve their chances to pass.

Educational Philosophy and Theory

USMLE Steps and In-Training Exam Results

Performance

American Board of Internal Medicine Certifying Exam

Competency assessment of residents in a training program is a complex process. Numerous tools are used to evaluate the six medical core competencies in order to assess the overall performance of a resident [1]. American Board of Internal Medicine (ABIM) uses various factors to test all the competencies of the physicians and their ability to deliver high-quality care. The factors include fulfillment of the graduate medical education training requirements, demonstration of clinical competence in patient care, and passing the certification examination in internal medicine (IM) [2]. The ABIM Certifying Exam (ABIM-CE) is administered annually and is taken by the residents just after completion of their training. The performance of the residents in ABIM-CE is important not only for the residents but also for the programs. In fact, for a residency program to maintain its accreditation, the graduates must have an 80% pass rate in ABIM-CE from the first time takers of the exam in the latest three-year period [3].

Various tools are used by IM residency programs to select the best candidates who will perform well in their program and maintain the ‘Board-pass rate’. United States Medical Licensing Examination (USMLE) scores are commonly used by residency programs to assess the applicants for residency [4-6]. During the residency training, the annual in-service training exam (ITE) is one of the tools used to assess the medical knowledge of the residents. A national standardized ITE has been developed in different specialties including IM, to provide feedback to the residents and the training program [7]. By assessing the knowledge gap (the deficit in the knowledge that can be improved to achieve better scores), ITE exams inform the residents and the training program about areas in need of focused education. Thus, performance in ITE exams is used by the training programs to evaluate the readiness and ability of their residents to take and pass ABIM-CE [7].

However, there is a paucity of research and data on the correlation of baseline mandatory licensing exam scores to the success in ABIM-CE. In the past, there have been studies done in various fields that have reported some correlation between USMLE and ITE scores [5,6,8,9]. A few other studies have also reported an association between USMLE scores and IM-ITE performances [10,11]. One study reported a modest correlation between USMLE Step 1 and ITE scores with ABIM-CE [12]. Here we analyzed whether the USMLE Step 1, USMLE Step 2, USMLE Step 3 and residency third-year ITE results correlated with passing the ABIM-CE.

Study sample and characteristics

All residents enrolled in the IM residency training program at our residency program from 2004 through 2017 were included. The data collected included gender; type of degree (MD versus DO); medical school country (American Medical Graduate [AMG] versus International Medical Graduate [IMG]); scores in USMLE Step 1, USMLE Step 2, and USMLE Step 3; gap (number of years) between graduation from medical school and start of residency (referred to as ‘the gap’); and ABIM-CE results (Pass versus Fail). The number of osteopathic residents (DO) was considerably less than allopathic residents (MD) and most of them took COMLEX exams instead of USMLE Steps. Additionally, data were unavailable for a few MD residents. Thus, after excluding DO residents (n=18) and MD residents with incomplete data (n=9), the remaining MD residents were included in the study.

Scores of ITE from the third year of residency, the exam closest to ABIM-CE in the timeline, were used in our analysis as third-year ITE scores are supposed to reflect the readiness of the residents to take ABIM-CE.

Statistical analysis

All continuous data are presented as mean(SD) and categorical data are presented as N(%), Statistical analyses were on the basis of scaled scores. Pearson correlation coefficient was used to assess the correlation between USMLE Step 1, USMLE Step 2, USMLE Step 3, and third-year ITE scores. Additionally, logistic regression were also done to look at the relationship between various scores seperately and pass or fail results in ABIM-CE. Multivariate logistic regression was used to examine the relationship between the scores and the ABIM-CE results controlling for other confounding variables such as gender, medical school country, the gap between medical school and residency program. Significance was set at p < 0.05. Statistical tests were performed using SAS 9.4 software (SAS Institute Inc., Cary, North Carolina).

Ethics approval

We received an exemption from the approval of Michigan State University Human Research Protection Program – MSU institutional board review (IRB# x16-029e). We didn’t obtain consent from individual graduates, the data is de-identified, and the ethics committee approved this procedure.

A total of 114 MD residents were included in the study; 92% (n=105) passed the ABIM-CE. The characteristics of the residents along with their mean scores in different exams are shown in Table 1. Mean scores in different exams for residents who passed ABIM-CE versus those who did not are shown in Table 2.

All residents (n=9) who failed ABIM-CE had Step 1 scores <220 which was 25% of total residents with Step 1 score of <220 (n=35). Among 31 residents with Step 2 score <220, 20% (n=6) failed ABIM whereas only 10% (n=3) with Step 2 score >220 failed ABIM-CE. Similarly, 9% of residents with USMLE Step 3 score < 220 failed ABIM-CE; all residents who failed had scored < 220.

First, four logistic regressions are employed to predict ABIM-CE passing with USMLE Step 1, USMLE Step 2, USMLE Step 3 and ITE tests separately controlling for gender, country of the medical school and year of the gap. There was a significant correlation of passing ABIM-CE with 10 points increase in USMLE Step 1 (Odds ratio [OR] 2.70; 95% Confidence Interval [CI] 1.38- 5.29) and 10 points increase in Step 2 (2.31; 95% CI 1.33-4.01). However, 10 points increase in Step 3 (OR 1.63; 95% CI 0.81- 3.29) didn’t significantly predict passing ABIM-CE. A 5% increase in ITE percentage increased the likelihood of passing ABIM-CE (OR 3.89, 95% CI 1.68-8.98).

Next, we calculated the average of USMLE Step scores and predict ABIM-CE pass using the average Step scores. Table 3 shows the results. The 10 points increase in average Step scores will lead to a higher chance of passing ABIM-CE tests (OR=2.96, 95% CI=0.95-9.20)) but not significantly at α=0.05 level. There was no relationship between ABIM-CE performance and gender, the country of medical school (AMG versus IMG). The increase in the gap (1year) wasn’t significantly associated with chances of passing ABIM-CE (OR 0.82; 95% CI 0.364-1.06) (Table 3).

The probability curve (with third-year-ITE percentage and the gap set at mean values) predicted that the chance of passing ABIM- CE was around 80% with USMLE scores greater than 200 and increased to almost 100% with USMLE scores of 240 or more (Figure 1). A ROC (receiver-operating characteristic) curve was computed to assess the accuracy of the model’s ability to predict passing the ABIM-CE. The area under the ROC curve was 0.945 (Figure 2). This indicates the probability of our model will rank a randomly chosen “pass the board” higher than “not pass the board” is 0.945.

We repeated the same analysis using standardized USMLE step scores and standardized ITE test scores with mean at zero and standard deviation (SD) at 1. One SD increase in Step 1 leads to a significantly higher chance of passing ABIM-CE (OR=6.41, 95% CI 1.84-22.46) and one SD increase in Step 2 leads to significant higher chance of passing ABIM-CE (OR=5.62, 95% CI 1.80-17.60). However, a 1 SD increase in Step 3 is not significantly associated with passing ABIM-CE (OR=1.89, 95% CI 0.76-4.68). Table 4 shows the results with a 1 SD increase in average USMLE score and ITE percentage increase controlling for demographic information. It is also found that a 1 SD increase in the ITE test leads to a higher chance of passing ABIM (OR=5.0, 95% CI 1.24-20.25). The results indicate one SD change in the ITE test has the biggest effect on increasing the chance of passing the ABIM-CE test.

USMLE Step 1, USMLE Step 2, USMLE Step 3 and third-year ITE scores have a predictive value in determining the chances of passing ABIM-CE tests. Among them, ITE percentage has is the most predictive followed by USMLE Step 1, USMLE Step 2 and USMLE step 3 in the descending order.

Residency programs use USMLE scores as initial screening tools to select their potential candidates from residency applicants. USMLE comprises 3 steps - Step 1, Step 2 and Step 3, which are intended to evaluate the appropriate use of medical knowledge in patient care by the examinees. The USMLE Step 2 exam consists of two sections: Step 2 CK (Clinical Knowledge) and Step 2 CS (Clinical Skills). Among them, the USMLE Step 1 score was included in a predictive tool to assess applicants during the interview for internal medicine residency [13]. However, there is limited data to prove that USMLE scores have a strong correlation with the performance in ABIM-CE [12]. The results from our study showed that the scores in all USMLE Steps have a predictive value for passing the ABIM-CE, USMLE step 1 score being the strongest one. The chance of failing ABIM-CE is higher with USMLE Scores below 220, more so with USMLE Step 1 than USMLE Step 2 or Step 3. Kay et al. reported a modest correlation between USMLE Step 1 and ABIM-CE scores [12]. Consistent results showing a correlation between USMLE scores and the results of the certification examination have been reported in various specialties and subspecialties [14-18]. Higher USMLE Step 1, Step 2 and composite scores were also associated with better performance on Emergency Medicine boards, with Step 2 being the strongest predictor [17]. Low USMLE Step 1 score was also predictive of failing certifying exams in surgery and pediatrics [18,19].

Performance in ITE is another tool used to predict the score of a resident in the ABIM-CE. ITE scores usually improve from the first year to the third year in IM [12]. Previous ITE scores, more than USMLE scores, have been shown to be strongly associated with subsequent ITE [20]. ITE scores are useful for the programs to recognize the residents who might need assistance or interventions from the program before taking ABIM-CE. Since third-year-ITE is the exam closest to the ABIM-CE the results in this exam can be used to gauge the readiness of a resident to take ABIM-CE. In our study, third-year-ITE percent had a positive correlation with ABIM-CE performance, which was slightly better than USMLE scores. Kay et al. also found a modest correlation between ITE and ABIM-CE scores [12]. Additionally, it was reported that residents scoring in the bottom quartile on their ITE were at increased risk of failing boards while those who scored in the top quartile of the ITE had a 100% pass rate [12]. In the past, few other studies looked at the association between ABIM-CE results and ITE scores with similar results [21-24]. Babott et al. reported that second-year ITE scores of more than 61% predicted a 100% pass rate in ABIM-CE with 41% sensitivity and 100% specificity [7]. Brateanu and colleagues developed a nomogram to predict the ABIM-CE performance which included the ITE scores of each year and the number of overnight calls in the last 6 months of residency [14]. In fact, third-year-ITE was reported to be the most important predictor in their analysis. Although USMLE scores were showed a good correlation with ABIM-CE during univariate analysis, they did not predict the ABIM results in the final multivariable logistic regression model. This could be because the residents have already been selected for the program based on the USMLE scores. Additionally, the ITE scores are the most proximate data to ABIM-CE and would, to some extent, have some correlation with USMLE scores. The predictive value of ITE scores with the performance in certifying exams has been reported in other specialties as well [25-27,18]. A study in Emergency Medicine showed that third-year-ITE scores were most predictive of the score in the certification exam [26]. Similarly, a low score in ITE at any time during residency increased the chances of failing a certification exam in surgery [18].

Passing in ABIM-CE is important for a resident [28], and it is equally important for residency training programs since the performance of their residents since ACGME requires a first-attempt examinee pass rate of at least 80% for continued accreditation of the program. Hence, a great deal of effort is put in by the programs to select the candidates who will do well in ABIM-CE and also to train their residents to perform well in ABIM-CE.

Brateanu et al. reported otherwise with a weak positive correlation between the gap and the performance in ABIM-CE [15]. Another study on IMGs by Kanna and colleagues showed that the gap between medical school and residency was not significant in predicting ITE scores [29]. However, our study also showed a positive correlation between the shorter gap between medical school and residency with higher performance in ABIM-CE. Thus, the residents with continued medical training without interruption from clinical practice are likely to perform better in ABIM-CE. Thus, the results regarding the gap have been mixed with no discernible explanation and further studies may be needed to find the exact cause.

Our study has a few limitations. This was a single-center study with limited demographic information of the residents from one program. There is no tool to calculate the equivalence of COMLEX to USMLE, so we could not compare osteopathic residents with allopathic residents. There is a little variation in the ABIM-CE pass rate since nearly all of the candidates in our sample passed the exam. Additionally, there were too few subjects to analyze passing or failing USMLE Step 1 or Step 2 and its effect on passing ABIM-CE.

We need to consider the implication of these results in finding the residents who might need help with ABIM-CE. The exact nature and efficacy of remediation plans, including board review courses, conferences or self-study courses, in helping the resident pass the ABIM-CE are unclear [30]. Although there are reports of improved ABIM-CE results with a directed reading program and individual education plan for residents, literature regarding the efficacy of these strategies is limited [31,32]. Based on experience from the leadership of the program, it is recommended for programs to identify residents needing interventions early during their residency using their USMLE step scores and 1st year ITE scores. Mandatory usage of Question banks and enrollment in Board Review Courses are recommended. At our program several residents, especially ones with scores <35 percentile in the PGY3 ITE exam were benefited from enrolling in board review courses. In addition, several reported that joining group discussions or studying with a partner may have had a positive impact on their score. However, further studies are required for validation.

Furthermore, there are many other unexplained factors determining whether a resident passes an ABIM-CE. The scores described above do not measure factors such as interpersonal skills, professionalism, or shared decision-making capabilities which are equally important for a good physician.

USMLE Step 1, USMLE Step 2, and third-year ITE scores are strong predictors of the chances of passing ABIM-CE. The third-year ITE score is the strongest predictor. Both USMLE Step 1 and Step 2 scores >220 are independent predictors of success in ABIM-CE as well, but the USMLE Step 1 score is the stronger predictor. Thus, programs can identify internal medicine residents at risk of failing ABIM-CE and need intervention at an early stage. Various measures such as enrolling them in question banks or board review courses can then be implemented by programs to improve their chances to pass.

USMLE: United States Medical Licensing Examination;

COMPLEX: The Comprehensive Osteopathic Medical Licensing Examination

IM: Internal Medicine

ABIM: American Board of Internal Medicine;

ITE: In training exams;

ABIM-CE: American Board of Internal Medicine Certifying Exam;

ROC: receiver-operating characteristic

Ethics approval and consent to participate: We received an exemption from the approval of Michigan State University Human Research Protection Program – MSU institutional board review (IRB# x16-029e). consent was obtained from all participants, and the ethics committee approved this procedure.

Consent for publication: Not applicable.

Availability of data and material: The raw data has been provided as a supplemental file.

Competing interests: The authors declare no competing interests.

Funding: We did not receive any research funding for this project.

Author contributions: SR and PD were involved in the conception and design of the study; intellectual content, literature search, data acquisition. LW performed data analysis and statistical analysis. SR and PD prepared the first draft of the manuscript. SS and MPR contributed to the literature search, manuscript editing, and review. MPR, SR, and LW performed a revision of the manuscript. All authors read and approved the final version of the manuscript.

Acknowledgments: Not applicable.

History of Medical Education http://www.acgme.org/About-Us/Overview/History-of-Medical-Education/About/ACGMEHistory. Accessed April 1, 2018
ABIM Board Certification http://www.abim.org/certification/policies.aspx. Accessed April 1, 2018
ACGME Program Requirements for Graduate Medical Education in Internal Medicine https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/140_internal_medicine_2017-07-01.pdf. Accessed April 1, 2018
Berner ES, Brooks CM, Erdmann JB (1993) Use of the USMLE to select residents. Academic medicine : journal of the Association of American Medical Colleges 68 (10):753-759
Spurlock DR, Jr., Holden C, Hartranft T (2010) Using United States Medical Licensing Examination((R)) (USMLE) examination results to predict later in-training examination performance among general surgery residents. Journal of surgical education 67 (6):452-456. doi:10.1016/j.jsurg.2010.06.010
Dougherty PJ, Walter N, Schilling P, Najibi S, Herkowitz H (2010) Do scores of the USMLE Step 1 and OITE correlate with the ABOS Part I certifying examination?: a multicenter study. Clinical orthopaedics and related research 468 (10):2797-2802. doi:10.1007/s11999-010-1327-3
Babbott SF, Beasley BW, Hinchey KT, Blotzer JW, Holmboe ES The Predictive Validity of the Internal Medicine In-Training Examination. The American Journal of Medicine 120 (8):735-740. doi:10.1016/j.amjmed.2007.05.003
Thundiyil JG, Modica RF, Silvestri S, Papa L (2010) Do United States Medical Licensing Examination (USMLE) scores predict in-training test performance for emergency medicine residents? The Journal of emergency medicine 38 (1):65-69. doi:10.1016/j.jemermed.2008.04.010
Crawford CH, 3rd, Nyland J, Roberts CS, Johnson JR (2010) Relationship among United States Medical Licensing Step I, orthopedic in-training, subjective clinical performance evaluations, and american board of orthopedic surgery examination scores: a 12-year review of an orthopedic surgery residency program. Journal of surgical education 67 (2):71-78. doi:10.1016/j.jsurg.2009.12.006
McDonald FS, Zeger SL, Kolars JC (2008) Associations between United States Medical Licensing Examination (USMLE) and Internal Medicine In-Training Examination (IM-ITE) scores. Journal of general internal medicine 23 (7):1016-1019. doi:10.1007/s11606-008-0641-x
Perez JA, Jr., Greer S (2009) Correlation of United States Medical Licensing Examination and Internal Medicine In-Training Examination performance. Advances in health sciences education : theory and practice 14 (5):753-758. doi:10.1007/s10459-009-9158-2
Kay C, Jackson JL, Frank M (2015) The relationship between internal medicine residency graduate performance on the ABIM certifying examination, yearly in-service training examinations, and the USMLE Step 1 examination. Academic medicine : journal of the Association of American Medical Colleges 90 (1):100-104. doi:10.1097/acm.0000000000000500
Neely D, Feinglass J, Wallace WH (2010) Developing a predictive model to assess applicants to an internal medicine residency. Journal of graduate medical education 2 (1):129-132. doi:10.4300/jgme-d-09-00044.1
Brateanu A, Yu C, Kattan MW, Olender J, Nielsen C (2012) A nomogram to predict the probability of passing the American Board of Internal Medicine examination. Medical Education Online 17:10.3402/meo.v3417i3400.18810. doi:10.3402/meo.v17i0.18810
Sisson SD, Bertram A, Yeh HC (2015) Concurrent Validity Between a Shared Curriculum, the Internal Medicine In-Training Examination, and the American Board of Internal Medicine Certifying Examination. Journal of graduate medical education 7 (1):42-47. doi:10.4300/jgme-d-14-00054.1
Indik JH, Duhigg LM, McDonald FS, Lipner RS, Rubright JD, Haist SA, Botkin NF, Kuvin JT (2017) Performance on the Cardiovascular In-Training Examination in Relation to the ABIM Cardiovascular Disease Certification Examination. Journal of the American College of Cardiology 69 (23):2862-2868. doi:10.1016/j.jacc.2017.04.020
Harmouche E, Goyal N, Pinawin A, Nagarwala J, Bhat R (2017) USMLE Scores Predict Success in ABEM Initial Certification: A Multicenter Study. Western Journal of Emergency Medicine 18 (3):544-549. doi:10.5811/westjem.2016.12.32478
de Virgilio C, Yaghoubian A, Kaji A, et al. (2010) Predicting performance on the american board of surgery qualifying and certifying examinations: A multi-institutional study. Archives of Surgery 145 (9):852-856. doi:10.1001/archsurg.2010.177
McCaskill QE, Kirk JJ, Barata DM, Wludyka PS, Zenni EA, Chiu TT (2007) USMLE step 1 scores as a significant predictor of future board passage in pediatrics. Ambulatory pediatrics : the official journal of the Ambulatory Pediatric Association 7 (2):192-195. doi:10.1016/j.ambp.2007.01.002
McDonald FS, Zeger SL, Kolars JC (2008) Associations Between United States Medical Licensing Examination (USMLE) and Internal Medicine In-Training Examination (IM-ITE) Scores. Journal of general internal medicine 23 (7):1016-1019. doi:10.1007/s11606-008-0641-x
Grossman RS, Fincher RM, Layne RD, Seelig CB, Berkowitz LR, Levine MA (1992) Validity of the in-training examination for predicting American Board of Internal Medicine certifying examination scores. Journal of general internal medicine 7 (1):63-67
Grossman RS, Murata GH, Fincher RM, Norcini JJ, Kapsner C, Layne RD, Seelig CB, Gateley A (1996) Predicting performance on the American Board of Internal Medicine Certifying Examination: the effects of resident preparation and other factors. Crime Study Group. Academic medicine : journal of the Association of American Medical Colleges 71 (10 Suppl):S74-76
Rollins LK, Martindale JR, Edmond M, Manser T, Scheld WM (1998) Predicting pass rates on the American Board of Internal Medicine certifying examination. Journal of general internal medicine 13 (6):414-416
Waxman H, Braunstein G, Dantzker D, Goldberg S, Lefrak S, Lichstein E, Ratzan K, Schiffman F (1994) Performance on the internal medicine second-year residency in-training examination predicts the outcome of the ABIM certifying examination. Journal of general internal medicine 9 (12):692-694
Althouse LA, McGuinness GA (2008) The in-training examination: an analysis of its predictive value on performance on the general pediatrics certification examination. The Journal of pediatrics 153 (3):425-428. doi:10.1016/j.jpeds.2008.03.012
Frederick RC, Hafner JW, Schaefer TJ, Aldag JC (2011) Outcome measures for emergency medicine residency graduates: do measures of academic and clinical performance during residency training correlate with American Board of Emergency Medicine test performance? Academic Emergency Medicine 18 (s2)
Baumgartner BR, Peterman SB (1996) Relationship between American College of Radiology in-training examination scores and American Board of Radiology written examination scores. Academic radiology 3 (10):873-878
Atsawarungruangkit A (2015) Relationship of residency program characteristics with pass rate of the American Board of Internal Medicine certifying exam. Medical Education Online 20:10.3402/meo.v3420.28631. doi:10.3402/meo.v20.28631
Kanna B, Gu Y, Akhuetie J, Dimitrov V (2009) Predicting performance using background characteristics of international medical graduates in an inner-city university-affiliated Internal Medicine residency training program. BMC Medical Education 9:42-42. doi:10.1186/1472-6920-9-42
FitzGerald JD, Wenger NS (2003) Didactic teaching conferences for IM residents: who attends, and is attendance related to medical certifying examination scores? Academic medicine : journal of the Association of American Medical Colleges 78 (1):84-89
Drake SM, Qureshi W, Morse W, Baker-Genaw K (2015) A time-efficient web-based teaching tool to improve medical knowledge and decrease ABIM failure rate in select residents. Medical Education Online 20:10.3402/meo.v3420.29221. doi:10.3402/meo.v20.29221
Visconti A, Gaeta T, Cabezon M, Briggs W, Pyle M (2013) Focused Board Intervention (FBI): A Remediation Program for Written Board Preparation and the Medical Knowledge Core Competency. Journal of graduate medical education 5 (3):464-467. doi:10.4300/jgme-d-12-00229.1

Table 1: Demographic characteristics and examination scores

Gender (n, %)
Female	34 (29.8)
Male	80 (70.2)
Medical School Country (n, %)
United States or Canada	17 (14.9)
International	97 (85.1)
Gap between medical school graduation and start of residency (years, mean (range))	3.1 (0-17)
USMLE Scores (mean (SD))
Step 1	230 (19)
Step 2	230 (21)
Step 3	208 (13)
Mean of Step 1, Step 2 and Step 3	222 (14)
Third-year ITE percentage (mean (SD))	68 (7)
ABIM-CE results (n, %)
Pass	105 (92.1)
Fail	9 (7.9)

Table 2: USMLE and ITE scores of residents who passed ABIM-CE versus those who did not

	Scores (Mean (SD))		p-value
	ABIM-CE Pass	ABIM-CE Fail	p-value
USMLE Step 1	231 (18)	209 (12)	<0.001
USMLE Step 2	233 (19)	204 (19)	<0.001
USMLE Step 3	208 (13)	200 (6)	0.07
Third-year ITE	69 (7)	59 (6)	<0.001

ABIM-CE: American Board of Internal Medicine- Certifying Exam; ITE: In-Training Exam; SD: Standard Deviation; USMLE: United States Medical Licensing Exam

Table 3: Odds of passing ABIM-CE in relation to demographic characteristics and scores

Variables	Analysis of maximum likelihood estimates
Variables	Odd’s ratio	p-value
Mean of USMLE Step 1, Step 2, & Step 3 (10 points)	2.961	0.0605
Gap between medical school and residency (1 year)	0.822	0.1231
Third-year- ITE percentage (5 points)	2.916	0.0240*
Gender of the residents (Female vs. Male)	2.754	0.3038
Medical school country (AMG vs. IMG)	4.969	0.2694

AMG: American Medical Graduate, IMG: International Medical Graduate, ITE: In-Training Exam, USMLE: United States Medical Licensing Exam

* significant with p-value <0.05.

Table 4: Odds of passing ABIM-CE in relation to demographic characteristics and standardized scores

Variables	Analysis of maximum likelihood estimates
Variables	Odd’s ratio	p-value
Mean of USMLE Step 1, Step 2, & Step 3 (1 SD increase)	4.686	0.0605
Gap between medical school and residency (1 year)	0.822	0.1231
Third-year- ITE percentage (1 SD increase)	5.004	0.0240*
Gender of the residents (Female vs. Male)	2.754	0.3038
Medical school country (AMG vs. IMG)	4.969	0.2694

* significant with p-value <0.05.

ITEABIMfinal.xlsx

Download PDF

Journal Publication

published 18 Mar, 2020

Read the published version in BMC Medical Education →

Editorial decision: Major revision
23 Jan, 2020
Review #2 received at journal
02 Jan, 2020
Reviewer #2 agreed at journal
12 Dec, 2019
Reviewer #1 agreed at journal
03 Nov, 2019
Review #1 received at journal
03 Nov, 2019
Reviewers invited by journal
29 Oct, 2019
Editor assigned by journal
08 Oct, 2019
Submission checks completed at journal
07 Oct, 2019
Editor invited by journal
07 Oct, 2019

You are reading this older preprint version

Read the latest preprint version →

Do USMLE Steps and ITE Results Predict Performance in the American Board of Internal Medicine Certifying Exam?

Status:

Journal Publication

Version 2

Abstract

Figures

Introduction

Methods

Results

Discussion

Conclusion

Abbreviations

Declarations

References

Tables

Supplementary Files

Status:

Journal Publication

Version 2