A Canadian survey of residency applicants’ and interviewers’ perceptions of the 2021 CaRMS R1 virtual interviews.

doi:10.21203/rs.3.rs-2107358/v1

Download PDF

Research Article

A Canadian survey of residency applicants’ and interviewers’ perceptions of the 2021 CaRMS R1 virtual interviews.

https://doi.org/10.21203/rs.3.rs-2107358/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background

All Canadian Residency Matching Service (CaRMS) R1 interviews were conducted virtually for the first time in 2021. We explored the facilitators, barriers, and implications of the virtual interview process for the CaRMS R1 match and provide recommendations for improvement.

Methods

We conducted a cross-sectional survey study of CaRMS R1 residency applicants and interviewers across Canada in 2021. Surveys were distributed by email to the interviewers, and by email, social media, or newsletter to the applicants. Close-ended items were described, and open-ended items were thematically analyzed.

Results

A total of 127 applicants and 400 interviewers, including 127 program directors, responded to the survey. 193/380 (50.8%) interviewers and 90/118 (76.3%) applicants preferred virtual over in-person interview formats. Facilitators of the virtual interview format included cost and time savings, ease of scheduling, reduced environmental impact, greater equity, less stress, greater reach and participation, and safety. Barriers of the virtual interview format included reduced informal conversations, limited ability for applicants to explore programs at different locations, limited ability for programs to assess applicants’ interest, technological issues, concern for interview integrity, limited non-verbal communication, and reduced networking opportunities. The most helpful mediums for applicants to learn about residency programs were program websites, the CaRMS/AFMC websites, and recruitment videos. Additionally, panel interviews were preferred by applicants for their ability to showcase themselves and build connections with multiple interviewers.

Conclusions

Perceptions of 2021 CaRMS R1 virtual interviews were favourable among applicants and interviewers. Recommendations from this study can help improve future iterations of virtual CaRMS interviews.

CaRMS

Faculty interviewer

Medical applicant

R1 interviews

Residency Match

Residency Interviews

Residency applicants

Virtual Interviews

Historically, the performance of medical students on their electives and residency interviews was used by the residency program selection committees to assess applicants’ fit for their program. Medical students rely on electives and residency interviews to choose a residency program that is the best fit for them (Leppard et al., 2021). With the COVID-19 pandemic, postgraduate medical education programs across Canada cancelled all visiting electives starting March of 2020, limiting opportunities for medical students to gain in-person exposure and showcase their abilities and interests to programs outside of their home school (The Association of Faculties of Medicine of Canada (AFMC), 2020). The Canadian residency matching service (CaRMS) changed its processes for matching applicants into residency programs, requiring all Canadian residency programs to conduct residency interviews virtually for the first time in 2021 (Shah et al., 2021; AFMC, 2020; Gore et al., 2021; Luc et al., 2021).

There are, on average, 50,000 Canadian residency interviews conducted each year (AFMC, 2020), yet there is a paucity of studies exploring experiences of entry-level Canadian residency match (R1) applicants and interviewers with the virtual CaRMS interviews (Relke et al., 2021). To date, studies have described fellowship applicants (Gore et al., 2021; Daram et al., 2014; Healy and Bedair, 2017; Peyser et al., 2021; Lewit and Gosain, 2021), experiences from a single institution (Peyser et al., 2021; Daram et al., 2014; Vadi et al., 2016; Healy and Bedair, 2017), a single R1 specialty (Luc et al., 2021; Shah et al., 2021; Snyder et al., 2021; Moran et al., 2021), and applicants outside of Canada (Peyser et al., 2021; Temsah et al., 2021; Snyder et al., 2021; Moran et al., 2021). Interviewers’ perspectives and preferences are mostly absent. By identifying potential facilitators, barriers, and implications of the first iteration of virtual interviews for the 2021 CaRMS R1 match from the perspective of applicants and interviewers, we can begin to refine and improve the virtual interview process for the subsequent years and inform future decisions regarding the use of virtual interviews in the CaRMS R1 match.

The purpose of our study was to explore perceptions of applicants and interviewers regarding the preparation, facilitators, barriers, and implications of the first iteration of the virtual interview process for the 2021 CaRMS R1 match.

Setting and Participants

We conducted a web-based cross-sectional survey study of CaRMS R1 applicants and interviewers across Canada. We developed the survey from a literature review (Zogg and Stein, 2021; Abbas et al., 2020; Byrnes et al., 2020) and by modifying a previous survey administered to Canadian internal medicine subspecialty applicants by Relke et al., 2021. We pilot tested the survey with five program directors and five CaRMS R1 applicants at one Canadian university. Their feedback was used to revise the survey items to improve clarity and address additional areas of interest. The survey consisted of 29 questions for the CaRMS R1 applicants and 16 questions for interviewers with open-ended questions, checklist items, yes/no options, and 5-point Likert scales from “strongly disagree” to “strongly agree.” (See the Appendix for the full survey). The survey was created using the Qualtrics platform.

Participant Recruitment

We recruited residency program interviewers by emailing the program administrators of all 375 R1 programs listed on the CaRMS website for distribution to the interviewers within their respective program. All interviewers involved in the R1 virtual interviews were invited to participate, including program directors, faculty members, residents, allied health workers, and administrative support staff. We approached the medical school contact responsible for the class listserv of 14 English-speaking Canadian medical schools and asked them to distribute our survey to final year medical students (CaRMS R1 applicants). We also used social media posts on Twitter, Facebook and medical schools’ newsletters to recruit applicants for our study. Duplicate entries were avoided using cookies to avoid two entries from the same computer.

Survey Distribution

Emails with the link to the survey and one email reminder were distributed after the rank-order deadline to the interviewers (April 5^th and April 12^th, 2021), and to the CaRMS R1 applicants after they matched to their residency programs (April 29^th, 2021 including social media posts, and June 8^th, 2021) to ensure that participation in the survey would not affect residency match results. To further mitigate bias, surveys were kept anonymous, voluntary, and no personal information was collected. The link to the survey was open until June 30^th, 2021.

Outcomes Measured

The primary outcome for our study was applicants’ and interviewers’ preferences regarding virtual and in-person CaRMS interview formats. Secondary outcomes were the reasons for choosing a preferred interview format, and perceived facilitators, barriers, and suggested improvements for the virtual interview format. We also explored applicants’ and interviewers’ experiences with technological issues, various interview styles, and different videoconferencing platforms.

Analysis of the Outcomes

Quantitative analysis

We present data as proportions for categorical data. We present rank lists in the order of increasing weighted averages across rank scores, with the mean of the top half rankings considered in a tie. We used Chi-square and Fisher’s exact tests for binomials to compare responses between applicants and interviewers. We included all survey responses in our analysis if the respondent completed questions beyond the demographic section. Each question was analyzed according to the number of respondents who answered that question. Quantitative analyses were conducted using SPSS version 27.

Qualitative analysis

We analyzed the qualitative data from open response questions using inductive thematic analysis (Chapman, et al., 2015). One of the researchers (HB) was an external non-medical reviewer not involved in the initial protocol process to mitigate bias. As part of the coding process, two researchers (RDF and HB) collaboratively generated the initial codebook by coding 5% of applicant and interviewer responses to each question and then coding 10% independently. Intercoder agreement was found to be 95%. This agreement level was found by comparing each segment of text coded independently and calculating the number of times that the two researchers agreed versus disagreed generating an overall percentage. For the other 5%, researchers discussed the coded segments of text and made changes once agreement was reached. We used the consensus-built codebook for all remaining coding. After coding, similar codes were grouped together to form subthemes, and these were subsequently grouped together to form themes. These preliminary themes were discussed with the wider research team. After the intercoder reliability check, one researcher (RDF) analyzed the remaining qualitative data from open response questions. Data reached thematic saturation prior to the analyses of all responses. All data were coded past data saturation for representation across respondents. Qualitative analyses were analyzed using Nvivo12.

To differentiate the strength of the recommendations in our study, we counted and displayed the themes describing recommendations as frequencies.

Compliance with Ethical Standards

This study has been approved by the Queen’s University Health Sciences and Affiliated Teaching Hospitals Research Ethics Board (#6030219). All participants provided freely given informed consent before completing the survey.

Respondents’ characteristics

Five out of 14 English-speaking Canadian medical schools confirmed email distribution to their respective final year medical school classes. Eighty-six applicants responded to our survey from these five schools, with a response rate ranging from 5.0% to 19.9% per school (Table 1).

An additional three applicants responded through newsletter recruitments and 38 through social media recruitment, for a total of 127 applicants responding to our survey (Table 1). 400 interviewers from all 29 R1 disciplines responded to our survey, with 127 out of 410 program directors (31.0%) responding.

A total of 119/127 (93.7%) applicants and 386/400 (96.5%) interviewers who responded went on to complete the survey. 97/127 (76.4%) applicants and 300/400 (75.0%) interviewers provided qualitative comments for analysis. The demographic information of respondents is shown in Table 1. A total of 28/127 (22.0%) applicants reported having financial needs, 11/127 (8.7%) identified being from a remote or underserved community, and 11/127 (8.7%) identified with both characteristics.

Overall, 79/119 (66.4%) applicants and 156/386 (40.4%) interviewers rated the virtual CaRMS interview process as “good” or “excellent.” Applicants were significantly more likely than interviewers to prefer virtual over in-person CaRMS R1 interviews in the future (90/118 (76.3%) vs 193/380 (50.8%); p < 0.001).

Applicants’ Perspectives

Mediums for disseminating information about residency programs

Applicants ranked the information on the CaRMS/AFMC website, residency program websites, online videos created by residency programs, and virtual townhalls as the top four most helpful mediums for providing information about the programs and locations they applied to, from highest to lowest order of helpfulness. Mediums ranked as less helpful included residency program social media presence, emailed information from the programs, informal discussions with residents and alumni outside of scheduled events, and virtual social events offered to the candidates selected for the interview, in decreasing order of helpfulness.

Strategies used by the applicants to prepare for the Virtual CaRMS Interviews

Applicants reported several strategies to prepare for the virtual interviews. Most applicants optimized their physical space and conducted a technology trial run (114/125, 91.2%). Many applicants purchased new hardware (67/125, 53.6%), some moved to a location with reliable internet access (34/125, 27.2%) and some upgraded their internet connection (16/125, 12.8%).

Virtual Interview Format

Figure 1 details applicants’ perceptions of the virtual CaRMS interview format. Applicants perceived positive, neutral, and negative impacts of the virtual interviews on their performance. Positive impacts of the virtual interview format included the added comfort of participating in a remote setting and a greater focus on interview preparation in the absence of travel. Many felt that the virtual interview format had no impact on their ability to convey and showcase themselves. Applicants felt that the inability to express and interpret non-verbal communication negatively impacted their performance, which was specifically worsened by asynchronous interview types (in which applicants video record their responses to questions displayed on their screen in the absence of direct interaction with a real-time interviewer). Of the 109 applicants who reported using all the following videoconferencing platforms – Zoom, Skype, Cisco WebEx, and Microsoft Teams – most applicants preferred Zoom as the videoconferencing platform of choice (106/109, 97.2%) as they were most familiar with that platform and felt that it was reliable.

Applicants ranked the following interview styles as most to least preferred: (1) Panel Interview, (2) Traditional 1-on-1 Interview, (3) Multiple Mini Interviews (MMI), and (4) Asynchronous recorded responses (Table 2).

TABLE 2. Applicants’ perceptions of different virtual interview styles
Ranked Virtual Interview Style (most to least preferred)	Themes	Subthemes	Applicant’s Quote
1. Panel Interviews	Reasons Panel was Preferred	Build connections with multiple interviewers	Opportunity to engage with multiple people creates more opportunities to connect and showcase myself.

Opportunities to get to know interviewers

Seemed the most interactive, got to know a few of the individuals from the school. Everything in general felt more personal.

Ability to showcase self

Felt it gave me the greatest chance to showcase my interpersonal skills by interacting with multiple interviewers in real-time.

Reasons Panel was not preferred

Felt uncoordinated

Panel interviews often felt uncoordinated and rushed.

2. Traditional 1-on-1 Interviews

Reasons Traditional 1-on-1 interviews were Preferred

Ability to build rapport

It allowed me a bit more time with the interviewer to establish rapport in programs where I could not do any electives in due to [the] COVID-19 [pandemic] and therefore could not get personally acquainted with.

More conversational

One-on-one interview provided more opportunity for conversation, especially compared to a panel interview where the interview platform (e.g. Zoom) often focuses on one person talking in a group and so makes it harder to have a natural conversation with more than one person.

Ability to showcase self

In traditional interviews I felt I got to showcase more of myself and my personality as well as learn more about the programs.

Reasons Traditional was not preferred

Single interviewer gives a narrow evaluation

One on one is too small, narrowed view.

3. Multiple Mini Interviews (MMI)

Reasons MMI was preferred

Multiple fair first impressions

Ability to have "multiple chances" at a fair first impression - even if you did poorly on one MMI station, you have others to make up for it.

Interactions with multiple interviewers

Multiple opportunities to interact with program members, less impact if there was a mistake or tech issue.

Reasons MMI was not preferred

Brief interactions

In a time where we couldn't attend for visiting electives or have real face-time, I think the panel or one-on-one style interviews are very very very important to have your best shot at having your interviewers really get to know you. MMI would make an already distant process feel completely anonymous.

Difficult to execute virtually

MMI (with being pulled back and forth in breakout rooms), I could tolerate but did not prefer as it was a very jarring experience.

4. Asynchronous Interviews^a

Reasons Asynchronous interviewers were not preferred

Inability to showcase self and build connection

Recorded responses were far and away the least pleasant experience. I felt stunted, and as though I had no opportunity to engage with people and showcase my ability to connect.

Inability to get to know interviewers and program

Pre-recorded responses feel awful to do and don’t provide any feedback or chance to ask questions.

Negatively impact rank decision

One school did only asynchronous and it felt like they didn't care to meet me at all. That's the only program that's rank position was influenced by how they did interviews. It was just so cold and distant.

^aNo applicants described reasons for preferring Asynchronous Interviews

Applicants preferred panel and traditional 1-on-1 interviews for their perceived ability to connect with interviewers in a conversational manner, build rapport and showcase themselves. While MMIs provide opportunities to meet multiple interviewers and develop fair first impressions, it was noted to be difficult to execute virtually.

Interviewers’ Perspectives

Most interviewers agreed (218/385, 56.6%) or strongly agreed (47/385, 12.2%) that they could easily evaluate applicants’ suitability for their program during the virtual interview. 193/378 (51.1%) interviewers felt that virtual and in-person interviews were equal in their ability to assess applicants, whereas 177/378 (46.8%) interviewers felt that in-person interviews were better for assessing applicant’s suitability compared to virtual interviews. Only 8/378 (2.1%) interviewers stated that virtual interviews were better than in-person interviews for assessing applicant’s suitability for their program.

Facilitators and Barriers of the Virtual CaRMS Interview Format

Facilitators of the virtual interview format include cost and time savings for both applicants and programs, ease of scheduling, reduced environmental impact, greater equity, less stress, greater reach and participation, and safety (Table 3). Barriers of the virtual interview format include reduced informal conversations, limited ability for applicants to explore programs at different locations, limited ability for programs to connect with applicants and assess their interest, technological issues, concern for interview integrity, limited non-verbal communication, and reduced networking opportunities (Table 3). A total of 70/388 (18.0%) interviewers and 38/123 (30.9%) applicants reported experiencing technological issues. Respondents reported internet connection problems as the most common technological issue (41/388 (10.6%) of interviewers and 17/123 (13.8%) of applicants).

TABLE 3. Facilitators and Barriers of the Virtual CaRMS Interview
Facilitators			Barriers
Facilitator	Interviewers N (%)^a	Applicants N (%)^a	Barrier	Interviewers N (%)^a	Applicants N (%)^a
Saved applicants’/ program’s money	348 (90.6)	117 (98.3)	Inability to have informal conversations with the applicants / residents and staff	254 (66.1)	94 (79.0)
Saved applicants’ travel time	371 (96.6)	116 (97.5)	Could not get a feel for the applicants / program and city	169 (44.0)	Program: 78 (65.5) City: 91 (76.5)
Ease of scheduling	272 (70.8)	106 (89.1)	Concern about technological issues affecting the interview	105 (27.3)	29 (24.4)
Environmentally friendly	282 (73.4)	105 (88.2)	Felt I could not communicate well virtually / perform well in the virtual format	51 (13.3)	28 (23.5)
Additional facilitators (Open text responses)			Additional barriers (Open text responses)
Themes	Quote(s)		Themes	Quote(s)
More equitable	Fewer barriers for those who struggle financially, single parents, etc. to make it more equitable for everyone participating. -Applicant

Allowed candidates that might otherwise not be able to travel for either personal (parent, care giver) or financial reasons access to apply all programs. -Interviewer

You are mostly only looking at the candidate from the neck up, so there's probably a bit less bias based on body type/size or visible disability as a result. -Interviewer

Concern for interview integrity

I noticed some of the students looked like they were almost reading answers off of their screens for some of the basic questions, but it was important to know if they could also think on their feet and be more impromptu with their responses. -Interviewer

Less stressful

From a mental health perspective, I think having virtual interviews was hugely beneficial, not having to coordinate travel, clothing, be away from home, etc. -Applicant

Generally ran more efficiently and smoothly than in-person interviews. I truly hope we can maintain a virtual format in the future. -Interviewer

Difficult to discern genuine interest and motivation

Far less [interviews] were declined this year, interpreted to mean that many who have less legitimate interest in our program would still interview because it took far less time, money, and effort to join via zoom. -Interviewer

Better sense of the candidates in terms of personality and fit when meeting in person. -Interviewer

Greater reach of applicants

Allow opportunities to both save money and interview for more schools. -Applicant

May attract candidates to interview who wouldn't have considered our program if they needed to travel here for a single interview. -Interviewer

Limited nonverbal communication

Difficult to read individuals [interviewers], improvise and react in virtual environments. -Applicants

In-person interviews permit evaluation of non-verbal cues, body language, eye contact. -Interviewer

Greater interviewer participation

We had more interest in virtual interviews from faculty this year compared to when we were in-person surprisingly! Our rural teams often have to travel to the city to do interviews in winter, so they greatly appreciated the virtual space. -Interviewer

Reduced networking opportunities

Not only is it an enjoyable opportunity for candidates to relax and cut loose a little bit, [but] these interactions are important for framing oneself for future opportunities such as fellowships, job interviews, etc. if a good impression is made, etc. -Interviewer

Less dangerous

Moins dangereux car habituellement les entrevues sont en hiver et le transport pendant une tempête peut être dangereux [Less dangerous because usually interviews are in winter and transport during a storm can be dangerous] -Interviewer

Less enjoyment from reduced/ lack of informal conversations

More pleasant to meet in person, not only for the applicants but also for the discussion between the members of the interview committee. -Interviewer

^aOf 384 interviewers and 119 applicants.

Respondents’ recommendations for improving the virtual CaRMS process

Recommendations regarding dissemination of information

Applicants (n=38) and interviewers (n=18) requested more opportunities for applicants to learn about programs and cities outside of the virtual CaRMS interviews. Applicants (n=10) recommended that virtual town hall sessions be mandated and recorded, so that those with clinical duties could have access to the recording at a later date. Applicants (n=6) and interviewers (n=4) recommended recruitment videos and online virtual tours to give applicants a better understanding of the program and what life is like in the city or town. Both applicants (n=22) and interviewers (n=14) recommended informal social events, offered prior to the interviews or during the interview period, for opportunities to connect with residents and staff, and develop genuine impressions of programs and culture. Some applicants (n=6) reported preferences for small group or 1-on-1 socials, as it was difficult to learn the truth about programs in large virtual settings. Additionally, applicants (n=11) recommended for detailed information to be sent well in advance to help applicants prepare for the interviews, including details such as the format and structure of the interview, the videoconferencing platform used, and clarifications with respect to time zones.

Recommendations regarding the use of technology

Applicants (n=14) reported stress during technological issues, which was perceived to negatively impact their performance. While most applicants (88.4%, 84/95) reported they could contact technological support if they experienced technological issues, fewer applicants (24.5%, 23/94) reported access to live technological support. In order to reduce the impact of technological issues, interviewers (n=17) and applicants (n=2) recommended that live supports should always be made available to them, and preferably provided by CaRMS. Additionally, applicants (n=15) and interviewers (n=11) recommended a national standardized videoconferencing platform across all programs so that they can become familiar with the platform and thus minimize technological issues, as well as ease the burden on programs.

While most applicants (98.3%, 117/119) reported cost savings with respect to the virtual CaRMS interviews, they did comment on the additional costs for new hardware and for upgrading their internet connection. Applicants (n=5) and interviewers (n=7) recommended medical schools and/or residency programs offer applicants and interviewers additional resources, including space, equipment, and a reliable internet connection to help offset these additional costs, mitigate inequities experienced by applicants, and reduce the number of technological issues during the virtual interviews. Additionally, interviewers (n=13) and applicants (n=2) recommended the development of unified guidelines and training for interviewers to reduce bias during the virtual interviews, in attempts to minimize unfair assessments based on the quality of interview equipment, video background, and unexpected disturbances external to the applicant’s control.

Lastly, interviewers (n=13) recommended development and use of a technology to prevent applicants from reading off their screens, since interviewers were not able to verify whether applicants were using prohibited aids. The use of a second camera, monitoring software and added distance between the applicant and their device was suggested.

Recommendations regarding the virtual interview format

Overall, most of the applicants (76.3%, 90/118) and over half of the interviewers (50.8%, 193/380) preferred a virtual over in-person CaRMS interview format to be used in the future. Some interviewers (n=23) and applicants (n=4) recommended offering a choice of either virtual or in-person interviews. However, the choice of interview format was argued against by some interviewers (n=7) because of the potential for preferential bias for applicants with the resources to attend in-person interviews. Interviewers (n=5) recommended for CaRMS to decide whether interviews should be made virtual or in-person, and the decision should be made unified across the country, or at least within each CaRMS discipline. One compromise suggested by some interviewers (n=6) was the future use of centralized in-person interviews within each CaRMS discipline, such as the urology fair.

In summary of recommendations for previously discussed virtual interview styles, most applicants (n=36) recommended panel interviews because of its conversational nature, ability to build connections with multiple interviewers, and chance to showcase their personal experiences. These merits were not possible with asynchronous interviews, which were recommended against by applicants (n=32).

In a web-based cross-sectional survey study of 2021 CaRMS R1 applicants and residency program interviewers, we identified a shared acceptance of the virtual interview format by both applicants and interviewers. We showed applicants’ preferences for virtual panel interview formats and identified facilitators and barriers of the virtual interview format. We enriched the quantitative results of our study by pairing them with analysed, qualitative responses.

Most applicants and just over half of interviewers in our study preferred a virtual over in-person format for the CaRMS R1 interviews. Fellows and program directors in internal medicine subspecialties (Relke et al., 2021), urologic oncology (Gore et al., 2021), and cardiothoracic programs (Robinson et al., 2021) expressed similar preferences regarding virtual interviews. Applicants and interviewers from our study described perceived advantages of virtual interviews that promote greater equity in the application process, including financial savings, time efficiency, and a greater reach and participation across applicants and interviewers (Relke, et al., 2021; Vadi et al., 2016). This is important given the recognized and unfulfilled need for greater equity in undergraduate and postgraduate medical education (Enioluwafe and Hairston, 2021; Wongworawat, 2021; Grober et al., 2003). The continued use of the virtual interview format can help mitigate some of the inequities currently faced by residency applicants (Soleas et al., 2022).

Despite most applicants’ preference to continue with the virtual format in future CaRMS interview cycles, approximately half of interviewers felt that in-person interviews were better than virtual interviews for assessing applicants’ suitability for residency programs, and a quarter of applicants still preferred in-person over virtual interviews. These preferences may be explained by studies in the business literatureand psychology literaturecomparing in-person and virtual interviews in the absence of factors external to the interview itself (eg. travel). These studies posit two relevant theories: (1) media richness (Daft and Lengel, 1986) - the capacity of a communication medium to convey multiple verbal and nonverbal cues, allow for immediate feedback, use natural language, and provide a personal focus; and (2) social presence (Williams

et al., 1976) - the capacity for interviewers and interviewees to experience and perceive the presence of one another within the communication medium. Studies of interviewers and interviewees assigned to different interview mediums described that in-person, virtual, and asynchronous interviews are associated with decreasing levels of media richness and social presence, respectively (Chapman et al., 2003; Chapman et al., 2001a, Chapman et al., 2001b; Basch et al., 2021; Basch et al., 2020). The type of interview medium has been shown to impact important objective measures of non-verbal communication, interview performance ratings, the ability for interviewees to make an impression of themselves, and perceptions of interview fairness (Melchers et al., 2021; Proost et al., 2021; Basch et al., 2021; Basch et al., 2020; Hiemstra et al., 2019; Gorman et al., 2018; Blacksmith et al., 2016; Castro et al., 2015). Prior literature agrees with our findings, as respondents reported similar advantages of the in-person interviews, specifically the better use of non-verbal communication exchanges and better-perceived assessments by interviewers of applicants' motivation, commitment, and personality. Since clear differences exist between different interview formats, programs should attempt to use the same interview medium for all programs within the same discipline to promote fairness and equity during the interview process (Melchers et al., 2021; Basch et al., 2021; Blacksmith et al., 2016).

Our results show that about a third of applicants (38/123, 30.9%) experienced technological issues, and applicants perceived a negative impact on their interview performance as a result. The virtual interview format led to additional barriers including reduced informal conversations and difficulties with building connections, depending on the interview style. Prior studies made recommendations to minimize the barriers of technological mediums (Basch et al., 2020; Gorman et al., 2018). Greater experience with using technology and greater affinity of interviewees for technology was associated with improved interview performance using virtual mediums (Basch et al., 2020). CaRMS applicants should consider preparing for virtual CaRMS interviews using virtual technology to help optimize their performance, not only from a technological standpoint, but also in their perceived ability to make a natural impression of themselves during the interview (Basch et al., 2020; Gorman et al., 2018). Medical schools should provide opportunities for applicants to practice virtual interviews prior to the CaRMS interview.

Our results demonstrate that applicants prefer synchronous over asynchronous interviews for their conversational nature, opportunity to build connections with interviewers, and ability to showcase themselves. Previous studies indicate that the advantages of synchronous over asynchronous interviews include improved applicant motivation, performance, and ability to develop a positive impression during the interview (Hiemstra et al., 2019). The impression interviewers make during the interview plays a role in applicant recruitment and in their perception of the company (Chapman et al., 2003; Rynes et al., 1991), which is absent in interviews that solely use asynchronous media.

Based on the results of our study, we recommend moving future iterations of the CaRMS R1 interviews to a virtual format to make the interviews more equitable, to save money for the applicants, and to decrease the carbon footprint of the CaRMS interview tour (Liang et al., 2021). Applicants reported challenges in learning about programs using the virtual interview format.(Liang et al., 2021). Dissemination of information by residency programs can be improved using mediums preferred by applicants, including virtual townhall sessions, recruitments videos, and informal small group events. Moreover, reinstating away electives would provide applicants with adequate opportunities to learn about the programs while retaining all the benefits of the virtual format for the CaRMS interviews. Prior research has shown that away electives substantially impact the choice of specialty and location of future practice for applicants (Curran et al., 2007; Courneya et al., 2019; Shah et al., 2021; Leppard et al., 2021).

Our study has several limitations. First, we are not able to report an overall response rate for the applicants and the interviewers in our study as we do not know how many final year medical students viewed the posts on social media and newsletters, and how many interviewers there were in total across all programs. The number of applicant respondents in our study may have been influenced by the timing of the survey distribution as it was distributed to the applicants after the end of the school year to comply with the ethical requirement to delay contact with final year medical students until after the R1 match date. Second, some R1 disciplines are underrepresented in our results, limiting specific conclusions with respect to those disciplines. The authors encourage the reader to review Table 1 to see if the results of our study are relevant to their specific discipline. Third, our results are subject to recall bias, as applicants completed the surveys 2-3 months after their CaRMS R1 virtual interviews, and self-report bias, as we did not have access to applicants’ CaRMS match data.

The virtual format for the 2021 CaRMS R1 interviews was favourably perceived by applicants and interviewers. Continued use of the virtual format for future iterations of the CaRMS R1 interviews may not only save applicants’ time and money but may also promote greater equity in the residency application process.

CaRMS Canadian residency matching service

R1 match Entry-level residency match

AFMC The Association of Faculties of Medicine of Canada

MMI Multiple Mini Interviews

Author contributions:

Rosephine Del Fernandes, Nicole Relke, Eleftherios Soleas, Clementine Janet Pui Man Lui and Boris Zevin contributed to the study conception, study design, and material preparation. Data collection and analysis were performed by Rosephine Del Fernandes, Eleftherios Soleas, and Heather Braund. The first draft of the manuscript was written by Rosephine Del Fernandes and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Acknowledgements

We would like to acknowledge Meredith Poole, Claire Moura, Hayley Manlove, Samantha Sarabia, Katrina Sajewycz and participating program directors for piloting the survey.

Funding: The authors did not receive support from any organization for the submitted work.

Competing interests: The authors have no financial or non-financial interests to disclose.

Ethics approval: The questionnaire and methodology for this study was approved by the Queen’s University Health Sciences and Affiliated Teaching Hospitals Research Ethics Board (#6030219).

Consent to participate: Informed consent was obtained from all individual participants included in the study.

Data availability: All data generated or analysed during this study are included in this published article.

Abbas, M., Dhane, M., Beniey, M., Meloche-Dumas, L., Eissa, M., Guérard-Poirier, N., El-Raheb, M., Lebel-Guay, F., Dubrowski, A. & Patocskai, E. (2020). Repercussions of the COVID-19 Pandemic on the Well-Being and Training of Medical Clerks: A Pan-Canadian Survey. BMC Medical Education 20 (1): 1–9. https://doi.org/10.1186/s12909-020-02293-0.
AFMC The Association of Faculties of Medicine of Canada. (2020). Canadian Residency Virtual Promotion Guide. https://www.afmc.ca/web/sites/default/files/pdf/2020-Canadian-Residency-Virtual-Promotion-Guide_en.pdf.
Basch, J. M., Melchers, K.G., Kegelmann, J. & Lieb, L. (2020). Smile for the Camera! The Role of Social Presence and Impression Management in Perceptions of Technology-Mediated Interviews. Journal of Managerial Psychology 35 (4): 285–99. https://doi.org/10.1108/JMP-09-2018-0398.
Basch, J. M., Melchers, K.G., Kurz, A., Krieger, M. & Miller, L. (2021). It Takes More Than a Good Camera: Which Factors Contribute to Differences Between Face-to-Face Interviews and Videoconference Interviews Regarding Performance Ratings and Interviewee Perceptions? Journal of Business and Psychology 36 (5): 921–40. https://doi.org/10.1007/s10869-020-09714-3.
Blacksmith, N., Willford, J., & Behrend, T. (2016). Technology in the Employment Interview: A Meta-Analysis and Future Research Agenda. Personnel Assessment and Decisions 2 (1): 10–20. https://doi.org/10.25035/pad.2016.002.
Byrnes, Y. M., Civantos, A. M., Go, B. C., McWilliams, T. L., & Rajasekaran, T. (2020). Effect of the COVID-19 Pandemic on Medical Student Career Perceptions: A National Survey Study. Medical Education Online 25 (1). https://doi.org/10.1080/10872981.2020.1798088.
Castro, J. R. & Gramzow, R. H. (2015). Rose Colored Webcam: Discrepancies in Personality Estimates and Interview Performance Ratings. Personality and Individual Differences 74: 202–7. https://doi.org/10.1016/j.paid.2014.10.038.
Chapman, A. L., Hadfield, M. & Chapman, C. J. (2015). Qualitative Research in Healthcare: An Introduction to Grounded Theory Using Thematic Analysis. Journal of the Royal College of Physicians of Edinburgh 45 (3): 201–5. https://doi.org/10.4997/JRCPE.2015.305.
Chapman, D.S., & Rowe, P. M. (2001). The Impact of Videoconference Technology, Interview Structure, and Interviewer Gender on Interviewer Evaluations in the Employment Interview: A Field Experiment. Journal of Occupational and Organizational Psychology 74: 279–98.
Chapman, D. S., Uggerslev, K. L. & Webster, J. (2003). Applicant Reactions to Face-to-Face and Technology-Mediated Interviews: A Field Investigation. Journal of Applied Psychology 88 (5): 944–53. https://doi.org/10.1037/0021-9010.88.5.944.
Chapman, D.S., & Webster, J. (2001). Rater Correction Processes in Applicant Selection Using Videoconference Technology: The Role of Attributions. Journal of Applied Social Psychology 31 (12): 2518–37. https://doi.org/10.1111/j.1559-1816.2001.tb00188.x.
Courneya, C.A., Cheung, W. Y., & McMillan, J. (2019). High Dives and Parallel Plans: Relationships between Medical Student Elective Strategies and Residency Match Outcomes. Canadian Medical Education Journal 11 (3): 4–12. https://doi.org/10.36834/cmej.53018.
Daft, R. L., & Lengel, R. H. (1986). Organizational Information Requirements, Media Richness and Structural Design. Management Science 32 (5): 554–71.
Daram, S. R., Wu, R. & Tang, S. (2014). Interview From Anywhere: Feasibility and Utility of Web-Based Videoconference Interviews in the Gastroenterology Fellowship Selection Process. Off. J. Am. Coll. Gastroenterol. | ACG 109.
Gore, J. L., Porten, S. P., Montgomery, J. S., Hamilton, R. J., Meng, M. V., Sexton, W. J. & Psutka, S. P. (2021). Applicant Perceptions of Virtual Interviews for Society of Urologic Oncology Fellowships during the COVID-19 Pandemic. Urologic Oncology: Seminars and Original Investigations 000: 2–5. https://doi.org/10.1016/j.urolonc.2021.06.003.
Gorman, C. A., Robinson, J. & Gamble, J. S. (2018). An Investigation into the Validity of Asynchronous Web-Based Video Employment-Interview Ratings. Consulting Psychology Journal 70 (2): 129–46. https://doi.org/10.1037/cpb0000102.
Grober, E.D., Matsumoto, E. D., Jewett, M. A. S & Chin, J. L. (2003). The Canadian Urology Fair: A Model for Minimizing the Financial and Academic Costs of the Residency Selection Process. Canadian Journal of Surgery 46 (6): 458–62.
Healy, W. L. & Bedair, H. (2017). Videoconference Interviews for an Adult Reconstruction Fellowship: Lessons Learned. Journal of Bone & Joint Surgery 99.
Hiemstra, A. M. F., Oostrom, J. K., Derous, E., Serlie, A. W. & Born, M. P. (2019). Applicant Perceptions of Initial Job Candidate Screening With Asynchronous Job Interviews: Does Personality Matter? Journal of Personnel Psychology 18 (3): 138–47.
Leppard, J., Nath A., & Cheung, W. J. (2021). Resident Recruitment in the COVID-19 Era: Factors Influencing Program Ranking by Residents Applying to a Family Medicine-Emergency Medicine Training Program. Canadian Journal of Emergency Medicine, no. 0123456789: 1–5. https://doi.org/10.1007/s43678-021-00177-w.
Lewit, R., & Gosain, A. (2021). Virtual Interviews May Fall Short for Pediatric Surgery Fellowships: Lessons Learned From COVID-19/SARS-CoV-2. Journal of Surgical Research 259: 326–31. https://doi.org/10.1016/j.jss.2020.09.029.
Liang, K. E., Dawson, J. Q., Stoian, M. D., Clark, D. G., Wynes, S. & Donner, S. D. (2021). A Carbon Footprint Study of the Canadian Medical Residency Interview Tour. Medical Teacher 43 (11): 1302–8. https://doi.org/10.1080/0142159X.2021.1944612.
Luc, J. G.Y., Nguyen, T.C. & Ad, N. (2021). Impact of the Coronavirus Disease 2019 Pandemic on Cardiac Surgical Education in North America. Innovations: Technology and Techniques in Cardiothoracic and Vascular Surgery 16 (4): 350–57. https://doi.org/10.1177/15569845211011459.
Melchers, K. G., Petrig, A., Basch, J. M., & Sauer, J. (2021). A Comparison of Conventional and Technology-Mediated Selection Interviews With Regard to Interviewees’ Performance, Perceptions, Strain, and Anxiety. Frontiers in Psychology 11 (January). https://doi.org/10.3389/fpsyg.2020.603632.
Moran, S. K., Nguyen, J. K., Grimm, L. J., Yee, J. M., Maxfield, C. M., Shah, N., Heitkamp, D. E. & Chapman, T. (2021). Should Radiology Residency Interviews Remain Virtual? Results of a Multi-Institutional Survey Inform the Debate. Academic Radiology, 1–13. https://doi.org/10.1016/j.acra.2021.10.017.
Enioluwafe, O. & Hairston, D. (2021). Recruiting Underrepresented Minority Students into Psychiatry Residency: A Virtual Diversity Initiative. Academic Psychiatry 45 (4): 440–44. https://doi.org/10.1007/s40596-021-01447-6.
Peyser, A., Gulersen, M., Nimaroff, M., Mullin, C. & Goldman, R. H. (2021). Virtual Obstetrics and Gynecology Fellowship Interviews during the Coronavirus Disease 2019 (COVID-19) Pandemic: A Survey Study. BMC Medical Education 21 (1): 1–6. https://doi.org/10.1186/s12909-021-02893-4.
Proost, K., Germeys, F. & Vanderstukken, A. (2021). Applicants’ Pre-Test Reactions towards Video Interviews: The Role of Expected Chances to Demonstrate Potential and to Use Nonverbal Cues. European Journal of Work and Organizational Psychology 30 (2): 265–73. https://doi.org/10.1080/1359432X.2020.1817975.
Relke, N., Soleas, E. & Lui, J. (2021). Internal Medicine Residents’ and Program Directors’ Perception of Virtual Interviews during COVID-19: A National Survey. Canadian Medical Education Journal 12 (2): e129–30.
Robinson, K. A., Shin, B. & Gangadharan, S. P. (2021). A Comparison Between In-Person and Virtual Fellowship Interviews During the COVID-19 Pandemic. Journal of Surgical Education 78 (4): 1175–81. https://doi.org/10.1016/j.jsurg.2020.11.006.
Rynes, S. L., Bretz, R. D. & Gerhart, B. (1991). The Importance of Recruitment in Job Choice: A Different Way of Looking. Personnel Psychology 44 (3): 487–521. https://doi.org/10.1111/j.1744-6570.1991.tb02402.x.
Shah, A., Champagne, A. A., Papa, J. D., Toor, J., Larouche, J. & Nousiainen, M. T. (2021). The Unique Impact of COVID-19 on Orthopedic Surgery Residency Applicants and Program Directors in Canada. Canadian Journal of Surgery 64 (2): E249–52. https://doi.org/10.1503/cjs.002021.
Short, J., Williams, E. & Christie, B. (1976). The Social Psychology of Telecommunications. London: Wiley.
Snyder, M. H., Reddy, V. P., Iyer, A. M., Ganju, A., Selden, N. R., Johnson, J. N., & Wolfe, S. Q. (2021). Applying to Residency: Survey of Neurosurgical Residency Applicants on Virtual Recruitment during COVID-19. Journal of Neurosurgery, 1–10. https://doi.org/10.3171/2021.8.jns211600.
Soleas, E., Relke, N., Fernandes, R. D., Braund, H., Zevin, B. & Lui, J. (2022). Going Virtual Advances Justice in Expected and Unexpected Ways. Canadian Medical Education Journal. https://doi.org/https://doi.org/10.36834/cmej.75111.
Temsah, M.H., Alkhattabi, F., Aljamaan, F., Alhasan, K., Alherbish, A., Philby, M., Alsohime, F. et al. (2021). Remote Interviews for Medical Residency Selection during the Initial COVID-19 Crisis: A National Survey. BMC Medical Education 21 (1): 1–12. https://doi.org/10.1186/s12909-021-02890-7.
Vadi, M. G., Malkin, M. R., Lenart, J., Stier, G. R., Gatling, J. W. & Applegate, R. L. (2016). Comparison of Web-Based and Face-to-Face Interviews for Application to an Anesthesiology Training Program: A Pilot Study. International Journal of Medical Education 7: 102–8. https://doi.org/10.5116/ijme.56e5.491a.
Curran, V. R., Fleet, L., Pong, R. W., Bornstein, S., Jong, M., Strasser, R. P., & Tesson, G. (2007). A Survey of Rural Medical Education Strategies throughout the Medical Education Continuum in Canada. Cah Sociol Demogr Med 47 (4): 445–68.
Wongworawat, M. D. (2021). Editor’s Spotlight/Take 5: How Did Coronavirus-19 Impact the Expenses for Medical Students Applying to An Orthopaedic Surgery Residency in 2020 to 2021? Clinical Orthopaedics & Related Research Publish Ah (December): 1–4. https://doi.org/10.1097/corr.0000000000002108.
Zogg, C. K., & Stein, S.L. (2021). Dear Program, What I Wish You Would Do: Applicants’ Concerns About and Suggestions for Applying to Surgical Residency During COVID-19. Annals of Surgery 273 (3): e85–87. https://doi.org/10.1097/SLA.0000000000004685.

Tables 1 is available in the Supplementary Files section.

No competing interests reported.

APPENDIX.docx

Download PDF

Version 1

posted

You are reading this latest preprint version

A Canadian survey of residency applicants’ and interviewers’ perceptions of the 2021 CaRMS R1 virtual interviews.

Status:

Version 1

Abstract

Background

Methods

Results

Conclusions

Figures

Introduction

Methods

Results

Discussion

Conclusion

Abbreviations

Declarations

References

Table 1

Additional Declarations

Supplementary Files

Status:

Version 1