Perception of Implicit Biases within Internal Medicine Residency Recruitment

Background: Few studies have addressed biases within residency admissions. Understanding the perceptions of implicit biases (IB) within residency admissions is essential to identify targets for potential interventions. Our objectives were to assess the interviewers’ perception of IB within the internal medicine (IM) residency interview process and the impact of former Implicit Association Test (IAT) completion and/or IB training on these perceptions. Methods: We surveyed all interviewers within an IM Residency program following the 2017-2018 interview cycle, assessing the perception of global and individual IB on evaluation of candidates, the perceived role of IAT completion and IB training, and mitigation strategies employed. Responses were assessed in four-subgroups based on exposure to IAT or IB training, analyzed with one-way ANOVA testing. Thematic analysis was performed of the free-text comments. Results: Of 113 interviewers, 68 (60%) completed the survey. The majority (79%, n=54) agreed or strongly agreed that biases exist within interviews, while 59% (n=40) agreed or strongly agreed that their own biases impact interview evaluations. The belief of one’s own biases impacting evaluation differed signicantly between subgroups (52% in those without any IAT/training, compared to 69% with both, p .05). Additionally, 90% (n=26) of those that had completed both IAT or IB training described using strategies to mitigate bias in the interview process, compared to 52% (n=11) that had not completed either (p < 0.001). Qualitative analysis revealed differing mitigation strategies between subgroups. Conclusions: Self-reported bias in the residency interview process exists. IB training and IAT were associated with self-reported implementation of mitigation strategies.


Background
Implicit biases (IB) -unconscious mental attitudes towards a person, thing, or group -seem pervasive 1,2 and may impact the admissions processes for medical trainees. [3][4][5] Prior work of IB within medical school admissions has focused on biases in medical school recruitment, noting direct improvement in diversity in a matriculating medical school class with IB training. 3 No studies have addressed IB within Graduate Medical Education (GME) admissions processes . Uncertainty regarding the prevalence and perceptions of IB in this setting remains.
Despite this, residency programs are adopting various programmatic interventions to mitigate potential biases in the recruitment process. Most of these interventions focus on faculty development, using strategies such as dedicated IB training as well as mandated Implicit Association Testing (IAT). 6 IB training varies by institution, but can include in-person workshops, reading assignments, and invited guest speakers. Implicit Association Testing is an computer-based measure of association, designed to detect the stretch of association between concepts. The IAT tests along several constructs, including analyzing racial, gender, weight, and age associations, and is thought to measure implicit attitudes.
Due to the growing knowledge that IB permeates academia and recruitment processes, academic medical centers are incorporating these strategies toward mitigation of IB within the recruitment process. However, the perception of IB training amongst faculty or the impacts of either IAT or dedicated training are not known. In order to optimize faculty development and to identify ideal targets for potential interventions to further foster diversity within academic medicine, it is essential to understand current faculty perceptions of IB.
We aimed to assess the perception of IB within the internal medicine (IM) residency interview process. Additionally, we sought to identify strategies utilized by interviewers to mitigate IBs, and understand the impact of former IAT completion and/or IB training on these perceptions.

Setting and Participants
We surveyed all interviewers at the Hospital of the University of Pennsylvania IM Residency program following the conclusion of the 2017-2018 interview cycle (prior to the National Resident Match Program Match Day). Interviewers included Department of Medicine faculty and residents.
The Hospital of the University of Pennsylvania is a large university-based tertiary care referral center. Preceding the dissemination survey, the university has strategically focused on prioritizing diversity across the spectrum of medical education (undergraduate, graduate, and continuing medical education) since 2011 through the University of Pennsylvania's O ce of Diversity and Inclusion. Therefore, independent of our survey, the Department of Medicine offered voluntary Implicit Bias training and faculty development, which consisted of participation in a two-hour workshop led by an expert panel. This workshop consisted of case-based discussion and re ection, focusing on identi cation of areas of implicit bias within the work-place.
In addition to participation in faculty development in this content area, completion of the implicit Association Testing was available through a voluntary and free online platform. Participation in this testing was self-directed, and a participating individual was able to complete the testing in any of the domains (gender, race, weight, age, etc.) based on personal interest. Neither the faculty development nor the IAT were mandated prior to participation in the upcoming interview cycle.

Survey Design
The anonymous, voluntary survey instrument was pilot tested among former residents and faculty with survey design expertise and/or experience in residency recruitment. It was distributed to participating interviewers via an electronic survey tool. Reminders were sent to participants weekly for 3 weeks, with no added incentive for completion of the survey.
Survey domains included the perception of IB on evaluation of candidates, the perceived role of IAT completion and IB training, and strategies to mitigate potential IB within the interview process. Answer choices ranged from "Strongly Disagree" to "Strongly Agree" on a 5-point Likert scale. Qualitative comments were elicited using open space for free-text comments to further expand on potential bias mitigation strategies used during interviews.

Data Analysis
Responses were assessed in four-subgroups based on exposure to IAT or IB training, analyzed with one-way ANOVA testing. All statistical analyses were completed using Stata 14.3 (StataCorp 2015, College Station, TX, StataCorp LP). Thematic content analysis of the qualitative data was performed by the primary author (JKH), and independently con rmed by the principle investigator (CJD).
The institutional review board at the University of Pennsylvania determined the study to be exempt from review.

Results
Sixty-eight of the 113 interviewers completed the full survey (60% response rate). The respondents were predominantly medical faculty (69%, n=47), and were 51% male with a mean age of 37.7. (Table 1) The non-responders were 48% male (n=21) and consisted of a larger proportion of resident physicians (n=26, 58% of non-responders). We found no differences between resident and faculty responses, so performed the full analysis in aggregate.
Of the survey respondents, forty-two percent (n=29) had completed IAT and IB training, 25% (n=17) had only participated in IB training, 1% (n=1) had only taken an IAT, and 32% (n=21) had done neither. (Table 1) Of those that had completed an IAT prior to the interview season, the majority completed either the Race-speci c (n=31, 44%) or Gender-career IAT (n=12, 17%).
The majority of interviewers (79%, n=54) either agreed or strongly agreed that biases exist within interviews. In contrast, only 59% of survey respondents (n=40) agreed or strongly agreed that their own biases impact interview evaluations. Table 2 displays the survey results between subgroups based on exposure to IAT or IB training. There were no differences in the belief that biases exist between subgroups. The belief of one's own biases impacting evaluation differed signi cantly between subgroups (52% in those without any IAT/training, compared to 69% with both, p .05). Additionally, 52% (n=11) of those without either IAT or IB training described using strategies to mitigate bias in the interview process, compared to 90% (n=26) that had completed both (p < 0.001).
Qualitative analysis revealed differing mitigation strategies between subgroups., Those that had completed training reported speci c strategies to mitigate biases, as compared to those that had no prior training, who described focusing on self-awareness as a strategy.
Representative comments are shown below for those that had not completed an IAT or any prior IB training: "Acknowledging my biases to myself as I evaluate someone" "I actively try to think of what my biases are and address them" "Mindfulness" Representative comments are shown below for those that had completed an IAT and/or IB training: "I standardize the questions" "I change the pronoun of the letters I read" "I complete all pertinent evaluations outside of the interview assessment prior to conducting the interview itself" There were no signi cant differences based on interviewer gender, age. There were also no signi cant differences regardless of if the interviewer was a resident or faculty member.

Discussion
Despite the majority of respondents agreeing that biases exist within IM residency interviews, only 60% believed that their own biases impacted evaluations of candidates. This was signi cantly impacted by prior IAT or IB training. There was also a signi cant difference in perceived implementation of mitigation strategies based on prior IAT or IB training. These differences amongst interviewers may suggest that training prior to residency recruitment may be an important intervention for mitigating biases.
Our qualitative analysis also highlighted differences in mitigation strategies used between subgroups. Speci cally, those that had completed training described speci c actions, such as standardizing questions and standardizing approach to review of portfolios, in order to mitigate potential bias. Those without prior training described strategies such as "mindfulness" and overall self-awareness. These differences may have important rami cations, as previous studies suggest that mindfulness is an ineffective countermeasure to IB in evaluation of candidates.
While prior work has highlighted the presence of IB within medical school admissions processes, this is the rst study to assess the perception of these biases within the IM residency interview process. Our study adds to the growing literature regarding IB within admissions processes, and identi es potential areas of focus for future intervention.
The limitations of the study include generalizability, as it was performed from a single IM residency program and had a signi cant non-response rate. However, emerging evidence suggests that response rate in physician groups poorly correlate to response bias, and the response rate of 60% falls within the expected range for physicians. [7][8][9] In addition, the residency program was concomitantly implementing several strategies to address IB at the time of our survey (based on national interest to address this issue). These changes would likely improve general awareness of IB, and minimize any differences between subgroups (bias toward the null). Additionally, our survey required self-report of perceptions of bias, however prior literature supports a correlation between implicit bias and self-reported bias. 10 Additionally, effectiveness of any faculty intervention requires faculty aligned of values with the institution, so knowledge of individual perception is an essential component of developing IB mitigation strategies. 11 Otherwise, our survey included both faculty and resident learners in our survey, which may have led to heterogeneity, however our subgroup analyses did not reveal any signi cant differences in responses. Finally, we were unable to quantify bias within a single interview or interviewer, and focused predominantly on self-reported behaviors. Additional work evaluating the impact of targeted interventions on interviewer behavior and bias within interviewers is a necessary area of future research. Understanding the role of IB training and IAT on assessment of candidates is an imperative step towards the goal of fostering diversity in academic medicine.

Conclusions:
Overall, while we found the majority of interviewers agreed that IB impacted assessment of IM residency candidates, the perception of one's own biases and the potential mitigation strategies differed based on prior training. Availability of data and material: All data generated or analyzed during this study are included in this published article Competing Interests: The author(s) declare they have no competing interests.

List Of Abbreviations (In
Funding: The authors report no external funding source for this study. Authors' contributions: All authors (JH, TB, MR, and CD) have been involved in the conceptualization and implementation of the study, as well as data analysis and manuscript preparation. All authors have read and agreed with the manuscript's content, and all authors have read and approved the manuscript.  The idea of a residency program admissions committee taking the IAT (implicit association test) prior to beginning the interview season might be expected to reduce bias in the admissions process.

Tables
10 (48%) 18 (62%) 8 (47%) 0.681 The idea of a residency program admissions committee completing implicit bias training prior to beginning the interview season might be expected to reduce bias in the admissions process.