Intern preparedness for medical practice has been the subject of discussion and research in medical education over the past decades. The term ‘intern’ refers to a recent medical graduate undergoing a period of supervised professional practice. Determining the preparedness of graduates of a new medical programme to practise as interns is a valued outcome metric for medical programmes (1). The World Federation for Medical Education recommends that a medical school must ‘analyse performance of cohorts of students and graduates in relation to its mission and intended educational outcomes, curriculum and provision of resources’ (2). In this sense ‘analysis’ is required so that successes are identified – and especially shortcomings, which can then be subsequently addressed.
In terms of scope, studies have dealt with a single institutions and programmes as well as multiple institutions - ranging from two institutions (23), several institutions (8)(12)(27) to an entire country (5)(6)(24)(33). There has also been comparison of two cohorts of the same programme such as where a traditional programme is being replaced by a new one with a problem-based ethos (7)(8)(9)(10), or one in a rural setting (11), or those comparing interns who had or had not undergone preparatory short courses (12). In cases where the effect of curricular innovations was studied the results could be positive (10)(16); in one study interns from a rural programme were judged to be better prepared for district hospital internship (11). Other studies comparing innovative and traditional programmes revealed little or no difference in the resulting preparedness for clinical performance, but positive differences were found in areas of focus for the new programmes such as ethics and law, interpersonal skills, self-directed learning, health system functioning and collaboration, and negative ones in (for example) understanding of disease processes (7)(9)(10). Beyond formal programmes, there has also been evaluation of the effectiveness of preparatory courses or ‘boot camps’ in preparing graduates for internship, whether general ones (17)(29)(30)(31) or those with a particular focus such as surgery or paediatrics (12)(28)(32).
Studies have investigated both general and specific aspects of preparedness. For instance some studies have evaluated how well interns are prepared against official standards such as General Medical Council recommendations in the United Kingdom (13)(15)(33). Where preparedness was measured against national standards, interns were sometimes not achieving them (15). On the other hand, other studies have focused on the effect of programmes to prepare students for specific competencies such as disaster preparedness (18), emotion regulation (19), memorable ‘firsts’ (20), infant lumbar punctures (21), basic medical procedures (22), career preparation and guidance (23), vaccination (24), health advocacy (residents/registrar in this case) (25), a particular internship rotation (26), or developing a professional identity (27). Additionally, studies have assessed the effect of entire undergraduate programmes on interns’ feelings of competence (3)(4)(5)(6). When evaluating how well an entire undergraduate programme had prepared new interns overall, reports varied from reasonably well prepared (8)(10) to inadequately prepared (3)(4)(6). In such cases, deficiencies such as those relating, handover, planning treatment, pain management, communication were identified (35) – also managing stress (19). Such instances pointed to remedial action needed in undergraduate training.
Overall, studies have tended to be conducted in developed countries although there is emerging literature from African contexts (4)(30)(34). In many cases the study population consisted of the interns themselves (21)(22)(23)(26)(29)(30), and in other studies their supervisors as well (4)(9)(10)(13)(31)(33)(37). In most studies interns were approached at some stage in the course of their internship years. In some cases the study focused on final year students, investigating their prospective feelings of preparedness (17)(36). Other studies investigated preparedness just before or immediately upon entry into internship (18)(21)(24)(31)(35), a few months after entry (9)(10)(16), before and after completing an internship rotation (26) or only afterwards (37), and after completing the entire internship (3)(27).
Studies have used qualitative approaches including interviews (4)(11)(14)(16)(19)(20)(26)(38), focus group discussions (4)(20) and diaries or audio-diaries (14)(19)(20). Researchers have made us of self-administered questionnaires with space for additional qualitative comments, often in a survey design (14)(19)(20)(21)(26)(28). Questionnaires were administered electronically (e-mail, online e.g. using SurveyMonkey) (12)(23)(24)(25)(33)(34)(36), by post (3)(5)(6)(8)(9), and in person (17)(18)(22)(28)(29)(30)(32)(37). In a few cases, specific mention was made of validation processes where a previously validated instrument was used (33)(34), or the validation process for a tailor-made one was described (8)(10)(25). Skills were sometimes observed (21)(28)(32). Available records were also used, such as official forms (10)(31) and logbooks (20). In some cases evaluations tracked cohorts of interns in longitudinal studies (7)(17)(35)(36).
This review of the literature revealed several challenges relating to evaluation of intern preparedness. Firstly, while it is common for studies to use intern self-evaluation with respect to their performance, it seems the validity of self-ratings was seldom questioned or taken into account in arriving at conclusions. Secondly, emerging evidence suggest that reliability can be enhanced through multi-source feedback, that is, through repeated observations and by a numbers of evaluators as an alternative to using single observations (40). Thirdly, studies have tended to treat ordinal data obtained from Likert type questions as interval data for the purposes of analysis. Finally, in studies where both interns and supervisors gave ratings, supervisors’ ratings were higher than interns’ in some cases (7)(31)(37) and lower in others (4)(13); this varied because in several cases interns and their supervisors observed different aspects of preparedness (9)(10)(33). For instance, a systematic review of physician self-assessment compared with observed measures of competence showed that out of the 20 included studies 13 showed little, no, or an inverse relationship, and only seven demonstrated positive associations (39). The tendency of some interns to over-rate, and others to under-rate themselves, can be possibly explained by the well-established Dunning-Kruger effect (41)(42).
In this article we describe research to determine the preparedness of the first cohort of graduates of a problem-based undergraduate medical programme in a middle-income country. We evaluated intern performance according to the interns’ own perceptions together with those of their clinician supervisors, using a parallel instrument. To conceptualise intern competency we investigated specific competencies as well as how they indicate competency for the seven CanMEDS roles (43) which our Faculty of Medicine has recently used as part of a framework for evaluating the MBBS programme. We also explored the possible role of the Dunning-Kruger effect on self-evaluation data.