Multi-stakeholder Validation of Entrustable Professional Activities in Family Medicine

doi:10.21203/rs.3.rs-1233750/v1

Background:

The development of Entrustable Professional Activities (EPAs) has generally focused on expert panel approaches, which are often limited to physician stakeholders. Optimally, a much wider group of stakeholders should be consulted and include all people gaining value from trainees’ patient care performance. The present study explored the perceptions of a variety of stakeholder groups regarding the relevance and comprehensiveness of EPAs developed for a Canadian Family Medicine Residency program.

Methods:

Through the use of an online survey consisting of dichotomous and open-ended questions, this study explored and compared the perceptions of a variety of stakeholder groups (policy makers, health administrators, health professionals, teachers, and public) regarding the relevance and comprehensiveness of 25 EPAs articulated by a family medicine residency program.

Results:

Agreement on appropriateness of the 25 articulated EPAs ranged from 90% to 100%, displaying a high level of alignment between stakeholders. Stakeholders identify two gap areas within the EPA framework: 1) provision of surgical assistance and 2) addictions care.

Conventional content analysis identified three key themes that cut across multiple areas of the survey: 1) Interprofessional collaboration (IPC), 2) Inclusivity, and 3) Scope of EPAs.

Comparative analysis consisted of exploring agreements, gaps, and other differences of perspectives between stakeholder groups. Although many of the stakeholder groups made comments regarding IPC and Inclusivity, the patient group in particular was more likely to raise these themes. Comments from patient groups were also overall more general, identifying overarching trends rather than specific pinpoint issues. Physicians, other health professionals, and resident groups, (Ie. health professional groups) on the other hand tended to answer with much more specificity, identifying issues relating to scope of EPAs.

Conclusions:

The present study demonstrates that a multi-stakeholder approach can feasibly be used to validate an EPA framework. In addition to confirming that a previously articulated set of EPAs largely reflected the range of activities expected of family physicians serving a population, it identified a number of potential areas of improvement in the framework.

Competency-based Medical Education

Entrustable Professional Activities

Validity

Social Accountability

Upon completion of residency training, family medicine graduates tend to leave with highly varied experiences and skill sets.^1–4 The lack of consistency in acquired skills can lead to unsatisfactory patient experience and outcomes,^1,5 as well as graduate doctors being more uncertain in their abilities.^1,2,6,7 One increasingly common method to enhance patient safety and the uniformity of learned skills in graduating medical residents is to incorporate Entrustable Professional Activities (EPAs) into a Competency-based Medical Education (CMBE) framework.^8–11

CBME is an outcomes-based approach structuring a medical education program using an organized framework of competencies (e.g. CanMEDS-FM 2017). ¹⁰ In this system, a curriculum is organized around the outcomes expected of a resident and that resident’s advancement and successful completion is dependent on having achieved those expected outcomes.¹² One way to define and codify outcomes is to use a set of EPAs — units of work that are observable and measurable and that integrate knowledge, skills, and attitudes into authentic professional tasks.¹³ These are the core tasks, that collectively describe what the graduate from a program is expected and entrusted to do unsupervised in practice.

EPAs not only outline clear goals for learners but also act to formalize supervisors’ daily clinical entrustment decisions in their residents by providing a framework to document the progression of trainees’ skills¹⁴ The intended outcome of implementing EPAs is a health professional who can practice medicine at a defined level of proficiency, in accord with local conditions, to meet local needs safely.^15,16

Optimally, when developing of competency frameworks and EPAs, a wide group of stakeholders should be consulted. When defining stakeholders in medical education, this term should not be limited to only the people gaining value from the training of physicians (e.g., supervising physicians). Rather, stakeholders should encompass all those gaining value from trainees’ patient care performance.^17–21 This partnership of stakeholders can be visualized as the World Health Organization’s Social Accountability Pentagram.²²

Unfortunately, the development of EPAs has generally focused solely on expert panel approaches, often limited to only physician members.²³ It is also common for senior physicians’ perceptions to overshadow the perspectives of other stakeholders, ^25–27 exposing a clear gap in social accountability. Previous studies have demonstrated that different stakeholders do not necessarily always agree on the competencies needed to provide safe high-quality patient care.^23,27,28 Consistent with stakeholder theory, it is thus necessary to engage multiple stakeholders to properly identify trainees’ key roles and observable behaviours.²⁸ Engaging a wide breadth of stakeholders in the development of EPAs will ensure a socially accountable approach is taken, that produces family physicians prepared to safely meet local needs.

In 2015, University of Manitoba’s Department of Family Medicine’s residency program committee, a predominantly physician faculty expert group, articulated a core set of EPAs for its Family Medicine Residency program. As it prepared for a revision of its EPA framework, the department sought to include input from a wider group of stakeholders, including all people gaining value from trainees’ patient care performance.

This study aims to explore the perceptions of a wide variety of stakeholders regarding the relevance and comprehensiveness of 25 EPAs developed by educators in the Department of Family Medicine for the University of Manitoba’s core Family Medicine Residency Program.

Participants: Five key stakeholder groups identified from the social accountability pentagram ^21–22 were chosen to participate in completion of the survey. This includes policy makers, health administrators, health professionals, teachers, and the public.

Participants from the policymaker group were sourced from the senior staff in the Primary Care Branch of the provincial Ministry of Health, Shared Health (a provincial organization which coordinates health services delivery) and the College of Physicians and Surgeons of Manitoba. The health professionals group included family physicians, nurse practitioners, physician assistants, and other health professionals within provincial primary care programs. The academic community consisted of family medicine teachers and residents within the University of Manitoba’s Department of Family Medicine. The patient group was surveyed generally through Local Health Involvement Groups and the health administration group included senior leaders from regional health authorities.

Process: An online survey was administered through Survey Monkey to validate the relevance and comprehensiveness of the 25 current EPAs articulated by the University of Manitoba core Family Medicine Residency program. EPAs were broken into five categories including; 1. Community-based Primary Care, 2. Emergency and Urgent Care, 3. Hospital Care, 4. Maternal and Newborn Care and 5. Leadership, Advocacy and Scholarship. Invitations to participate were emailed to all stakeholders twice; once for initial participation, and once as a subsequent reminder. The survey ran for one month from June 22 to July 22, 2021. The survey consisted of a combination of semi-open and open-ended questions. Participants were asked whether each of the 25 outlined EPAs should be considered a key activity (yes/no) that a graduating family physician should be able to perform independently by the end of training. In each EPA section, participants were prompted to provide an explanation if they chose “No” to any of the current EPAs, thought any additional EPAs should be included, or had any additional comments. Finally, participants were asked to identify which stakeholder group or groups they belonged to.

Analysis: Analysis of responses included conventional content analysis and comparative analysis of various stakeholders’ perspectives. The first step was conventional content analysis²⁹ where a coding structure was developed inductively. Specifically, all comments from open-ended questions were reviewed by the research team and discussed until a consensus was reached on a list of core themes. Core themes were identified globally across all EPA sections and individually within each EPA section. The lists of core themes served as the basis for the subsequent step, comparative analysis of the identified themes. Comparative analysis consisted of exploring agreements, gaps, and other differences between stakeholder groups perspectives through a group discussion with the study team. The study team consisted of the first and second authors.

No incentives were given for completion of the survey. Ethical approval for this study was granted by the University of Manitoba Health Research Ethics Board (HREB).

Overview of sample: A total of 57 participants initiated the survey, with 49 fully completing it, yielding a completion rate of 86%. The responders covered all dimensions of the social accountability pentagram ^21–22 : policy makers (5), health administrators (4), family physicians (22 - 20 involved with family medicine residency teaching, 2 not involved), other health professionals (6), family medicine residents (7) and members of public/patient (15). Participants were allowed to select multiple roles they belonged to, thus the total participant number does not equal that of the participants breakdown. Of all responders, 84% provided some free text responses (169 comments in total), thus providing a rich set of qualitative data.

EPA-specific comments: Agreement on appropriateness of EPAs ranged from 90–100%, displaying a high level of alignment between stakeholders regarding the appropriateness of the 25 articulated EPAs. Global agreement on each EPA can be found in Table 1.

Table 1

EPA Descriptions with Global Agreement of Stakeholders
EPA Number	EPA Description	Global Agreement (%)
	Community-based Primary Care
1	Provide recommended preventative care to adults In the outpatient setting, the residents will provide evidence-based preventative care. He/she will adapt and individualize the review, exploring new symptoms and signs as indicated. He/she will apply evidence-based prevention guidelines in a patient-centred way, and provide lifestyle counselling as needed.	96.49
2	Provide recommended preventative care to infants, children and adolescents In the outpatient setting, the residents will perform evidence-based periodic health exams for infants, children, and adolescents. He/she will demonstrate adaptability, individualizing the review in a patient-appropriate manner, as well as exploring new symptoms and signs as indicated. He/she will be aware of changing cognitive and developmental stages in children, and modify their approach accordingly as they assess and build their therapeutic relationship with the patient.	94.74
3	Assess, manage, and follow-up adults with undifferentiated symptoms or common (key) conditions Across multiple settings, the resident will demonstrate an ability to assess and manage patients presenting with undifferentiated symptoms and common conditions, working efficiently though an appropriately broad initial differential diagnosis, and ruling out potential dangerous diagnoses. He/she will develop appropriate follow-up management plans.	98.25
4	Assess, manage and follow-up infants, children and adolescents presenting with undifferentiated symptoms or common (key) conditions Across multiple settings, the resident will demonstrate an ability to assess and manage infants, children, and adolescents presenting with undifferentiated symptoms or common conditions, working efficiently though an appropriately broad initial differential diagnosis, and ruling out potential dangerous diagnoses. He/she will develop appropriate follow-up management plans.	94.74
5	Manage and follow-up patients with common chronic conditions and multiple co-morbidities Across multiple settings, the resident will adeptly provide guideline-guided care for chronic conditions, adapting targets and plans of care based on a patient’s individual factors. The resident will manage multiple medical problems, prioritizing as indicated.	98.25
6	Manage and follow-up the elderly patient with multiple co-morbidities Across multiple settings, considering capacity for consent, need for a substitute decision maker, and advanced directives, the resident will provide guideline-directed care for elderly patients. The resident will adapt targets and plans of care based on the patient’s individual factors, and manage multiple medical problems, prioritizing as indicated.	98.25
7	Identify, diagnose, evaluate and manage patients with common mental health issues Across multiple settings, the resident will effectively assess and manage the full range of mental health issues, including emergency presentations and involuntary treatment when appropriate. He/she will use specific counselling techniques as indicated, and use the capacity of the multi-disciplinary team.	91.23
8	Provide palliative and end-of-life care Across multiple settings, the resident will be able to care for patients with advanced, complex, or terminal conditions, while considering capacity for consent, and advanced directives. He/she will understand goals of care and judiciously balance burden versus benefit when considering management. The resident will manage the range of symptoms as effectively as possible, working within the multi-disciplinary team.	91.23
9	Perform common family medicine procedures Across multiple settings, the residents will demonstrate competency in performing core set of family medicine procedures.	98.25
10	Provide expert advice and obtain consultation for patients Across multiple settings, the resident will identify patients whose condition would be improved by care provided by a consultant. The resident also provide advice at the request of colleagues.	98.25
11	Facilitates and manages care transitions The resident plans and coordinates transitions between care settings for a patient and ensures appropriate follow-up with the patient’s family physician.	98.25
	Emergency & Urgent Care
12	Recognize and provide initial management of common emergencies In an emergency room or urgent care setting, the residents will demonstrate the ability to arrive at a timely and correct diagnosis considering an appropriately broad differential (including dangerous causes), prioritize and assess/reassess appropriately, and initiate management and treatment in a timely way. The resident will effectively engage the health care team to optimize patient care.	100.00
	Hospital Care
13	Determine when a patient requires admission and inpatient hospital care The resident will demonstrate the ability to determine if a patient’s condition requires admission to hospital for further assessment and management.	96.30
14	Assess and appropriately manage medical patients in hospital In the in-patient setting, the resident will demonstrate the ability to assess and manage patients presenting with a variety of medical conditions. The resident will collaborate effectively within inter-professional teams.	96.30
15	Recognize and provide initial management of the medically unstable patient in hospital In hospital setting, the residents will demonstrate the ability to assess the unstable patient, considering an appropriately broad differential, including dangerous causes, prioritize and assess/reassess appropriately, and initiate management and treatment in a timely way. They resident will effectively engage the health care team to optimize patient care.	90.74
	Maternal & Newborn Care
16	Provide pre-conception and pre-natal care In the outpatient setting, the resident will effectively provide patient-centred pre- conception and prenatal care, guided and documented on standardized prenatal forms. Through continuity of prenatal care, the resident will explore and respond to medical and/or psychosocial issues with consideration for both maternal and fetal well-being.	98.04
17	Provide intra-partum care and perform low-risk deliveries In the hospital setting, the resident will demonstrate the ability to safely manage normal labour and delivery, being attentive to maternal and fetal well-being. The resident will recognize abnormal labour and delivery patterns, and consult appropriately.	96.08
18	Recognize and manage common intra-partum emergencies In the hospital setting, the resident will recognize abnormal labour and intra-partum emergencies. The resident will initiate management and call for assistance	96.08
19	Provide postpartum care and breastfeeding support In hospital and outpatient settings, the resident will effectively provide patient-centred postpartum care. The resident will adapt the encounter to explore and respond to medical and/or psychosocial issues more thoroughly as indicated, and will explore family functioning.	92.16
20	Provide family medicine-centered care to newborns in their first weeks of life In hospital and outpatient settings, the resident will provide evidence-based care of the newborn. The residents will demonstrate knowledge and competent assessment and management of problems presenting in the newborn period. The resident will establish professional relationships with parents and effectively counsel parents about newborn care.	90.20
	Leadership, Advocacy & Scholarship
21	Provide leadership within interprofessional and health care teams Across multiple settings, the resident will be able to demonstrate leadership in health care environment. The resident ensure the well functioning of a clinical team.	90.00
22	Provide care to vulnerable and underserved populations Across multiple settings, the resident will demonstrate competent provision of patient- centred care for vulnerable and underserved populations. The resident will demonstrate a culturally sensitive holistic approach, and an understanding of the unique determinants of health, beliefs, and traditions. As needed, the resident will effectively use translators.	96.00
23	Provide care to First Nation, Inuit and Métis peoples Across multiple settings, the resident will demonstrate competent provision of patient-centred care for First Nations, Inuit, and Métis peoples. The resident will demonstrate a culturally sensitive holistic approach, and an understanding of the unique determinants of health, beliefs, and traditions. As needed, the resident will effectively use translators.	96.00
24	Optimize the quality and safety of health care through use of best practices and application of Quality Improvement The resident will demonstrate skill in practice management through implementation of best practices, principles of continuity of care, quality improvement strategies, and optimizing of information management.	94.00
25	Provide clinical teaching The resident will demonstrate skill in delivering teaching activities and provide effective clinical supervision of learners.	90.00

Approximately one third of respondents (34.62%) identified gaps in the EPA framework, specifically two areas that would benefit from additional or expanded EPAs. Members of the policy makers group (3 individuals) identified surgical assistance as a possible gap in the current Hospital Care EPAs. Multiple respondents (2 policy makers, 8 family physician teachers, and 3 family residents) identified the need to include addictions management, either as a stand-alone EPA or as part of the mental health EPA.

General themes: Qualitative analysis identified three key themes that cut across multiple areas survey: 1) interprofessional collaboration (IPC); 2) Inclusivity; and 3) Scope of EPAs.

The most common theme identified across all EPA sections was having the knowledge of, and ability to utilize interprofessional care teams effectively, a theme the research group referred to as Interprofessional collaboration (IPC). IPC is an umbrella term used to encompass several different comments such as “knowing ones own limits and when to refer a patient in complex situations”, “fostering positive professional relationships within all members of a care team” and “having a broad knowledge of other healthcare professions and how they can be incorporated to holistically improve care”. Although IPC is mentioned in aspects of several EPAs (EPA, 7, 8, 14, 15), stakeholders still felt there could be more emphasis placed on this aspect of clinical work:

“…appropriate discharge planning – knowing how to involve other professionals for this (Occupational Therapy, Physiotherapy, Home Care, etc) for safe discharge”

Resident

“Manage concomitant care with other health professionals (ie other specialists, other allied health professionals etc.) in order to ensure the residents are able to provide care to patients that does not undermine colleagues and allows the resident to demonstrate flexibility in their care approach)”

Family Physician Teacher

The second theme identified was termed Inclusivity, another umbrella term encompassing several related ideas drawn from the comments. Inclusivity incorporated comments aimed to address demographic variables such as:

“A greater awareness of the issues and concerns of people with disabilities…”

Patient

“Reproductive Health including preconception (all sexes)… [such as] males or non-binary individual”

Resident

“Needs to be more emphasis on Indigenous health, with particular focus on knowing and implementing the Truth and Reconciliation Commission’s calls to actions”

Family Medicine Teacher

“Don’t forget rural populations”

Healthcare administrator

Lastly, the third common theme that arose was the scope of EPAs either being too broad or too narrow. Analysis of this theme was best done through interpretation of ”No” responses to the inclusion of EPAs, as many “No’s” appeared to be related to the perception that the level of responsibility articulated in the EPA would exceeds what could be expected of residents. Comments on the EPAs with lowest levels of agreement, included:

“Teaching skills develop over time and are necessary if taking on learners, but some will not have the desire to teach”

Family Medicine Teacher on EPA 25

“Again, as a graduating resident, there is much to learn before being fully able to lead”

Resident on EPA 21

“Too broad an EPA. What type of hospital setting? Which diagnoses requiring admission? I am not an internal medicine specialist (and choose not to be) and therefore should not be expected to have all hospital admissions as part of my scope of practice”

Family Medicine Teacher on EPA 15

“I think mental health, palliative care and pediatrics are too specialized to expect a newly trained physician to demonstrate the level of competence I believe is expected based on the wording of the questions.”

Healthcare administrator on EPA 7,8,15

Specific stakeholder group perspectives: Comparative analysis consisted of exploring agreements, gaps, and other differences of perspectives between stakeholder groups. Although many of the stakeholder groups made comments regarding IPC and Inclusivity, the patient group in particular was more likely to raise these themes:

“Working with vulnerable people and different backgrounds needs to be done in a sensitive way. A way that most doctors, lack.”

Patient

“Physician(s) should perform activities without bias or discrimination and will model this behaviour for learners. They should seek to highlight and correct behaviours that resemble discrimination and bias.”

Patient

Comments from patient groups were also overall more general, identifying overarching trends rather than specific pinpoint issues. Physicians, other health professionals, and resident groups, (Ie. health professional groups) on the other hand tended to answer with much more specificity, identifying issues relating to scope of EPAs.

CBME is rooted in the idea that, to better meet public health needs, medical education must be organized around the competencies that are needed for practice within a specified setting.^15,30 Over the 30+ years since the publication of the Edinburgh Declaration, medical schools have embraced their social accountability mission.³¹ Nowhere is that more apparent than in the specialty area of Family Medicine, which has been at the forefront of linking training with health system needs.

As discussed in the introduction, the development of EPAs has heavily depended on expert panel approaches, often limited to only physician members.²³ Consistent with stakeholder theory, it is thus necessary to engage multiple stakeholders to properly identify trainees’ key roles and observable behaviours.²⁸

In a socially accountable CBME approach, it is essential to engage a wide breadth of stakeholders in the development of EPAs, and optimally this should be a continuous process – engaging with stakeholders not only at the stage of initial development but also at times of revision of frameworks.

The present study demonstrates that it is feasible to engage a diverse group of stakeholders to provide input using a simple online survey approach. The World Health Organization’s Social Accountability Pentagram provides a useful construct in guiding the selection of stakeholder groups to be invited into the process. In the invitation process, it is important to be deliberate inviting people who have an understanding, and ideally interface regularly with the clinical area.

The approach allowed residency program leaders to confirm the previously developed 25 EPAs reflected tasks presently performed by family physicians and led to the identification of tasks that needed to be included in future iterations of the framework, namely: 1) provision of surgical assistance and 2) addictions care.

Additionally, through conventional content analysis, we identified three key themes (Interprofessional collaboration (IPC), Inclusivity, and Scope of EPAs) that would help in the refining of existing EPAs – ensuring the EPAs not only reflect the “what” but the “how” of tasks performed by family physicians. The first two themes (IPC and Inclusivity) reflect themes of growing importance in healthcare broadly. The third theme reflects a common tension in the development of EPAs – being broad enough to reflect the full scope of activities vs being specific enough to facilitate assessment in practice.

Comparative analysis allowed the team to better appreciate the unique but often complementary perspectives of various stakeholders; in particular, those of the public/patient group, who have been commonly excluded from EPA development.^25–27 Although many of the stakeholder groups made comments regarding IPC and Inclusivity, the patient group was much more likely to raise these themes, often using more direct, impactful language.

On the other hand, physicians, other health professionals and resident groups tended to answer with much more specificity, often dialing in on particular procedures, diagnosis, or management that need to be included in EPAs.

Limitations

This study had a sample size of 57 participants. Although this sample size is quite typical of similar validation studies on EPAs, the multi-stakeholder approach of this study had more individual participant groups with thus less members per group, which ultimately may have underrepresented some stakeholder groups. Ideally, more respondents within the policy maker, healthcare administrators/managers, and other healthcare professionals would have been desirable.

The main benefit of using a survey style was the speed at which results were obtained. Within a one-month time period, a wide variety of stakeholders were reached and surveyed all while during the COVID-19 pandemic. It is unknown how results of other approaches, such as focus groups, would compare in their effectiveness.

The present study demonstrates that a multi-stakeholder approach can feasibly be used to validate an EPA framework. In addition to confirming that a previously articulated set of EPAs largely reflected the range of activities expected of family physicians serving a population, it identified a number of potential areas of improvement.

EPAs: Entrustable Professional Activities

CBME: Competency-Based Medical Education

Availability of data and materials

Full survey results have been made available by the authors on Zenodo (DOI 10.5281/zenodo.5894885).

Acknowledgements

None

Funding

None.

Authors' contributions

BC, DF performed a literature search. JF and TB designed the study and questionnaire. BC implemented the questionnaire and performed the data analysis. JF supervised the analysis. All authors (JF, TB, BC, DF) were involved in the interpretation of data, revised and approved the final manuscript.

Ethics approval and consent to participate

This research study was conducted in accordance with the Declaration of Helsinki. Completion of the questionnaire was considered consent to participate in the study. The final study protocol, survey and data collection tools were approved prior to the commencement by the University of Manitoba Health Research Ethics Board (research study HS24752).

Consent for publication

Not applicable

Competing interests

BC and DF were medical students at the Max Rady College of Medicine at the time of the study. JF and TB declare they have no competing interests.

Schumacher DJ, West DC, Schwartz A, et al. Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network General Pediatrics Entrustable Professional Activities Study Group. Longitudinal assessment of resident performance using entrustable professional activities. JAMA Netw Open. 2020;3:1.
Zubatsky M, Brieler J, Jacobs C. Training Experiences of Family Medicine Residents on Behavioral Health Rotations. Fam Med. 2017;49(8):635-639.
Marisette S, Shuvra MM, Sale J, Rezmovitz J, Mutasingwa D, Maxted J. Inconsistent role modeling of professionalism in family medicine residency: Resident perspectives from 2 Ontario sites. Can Fam Physician. 2020;66(2):e55-61.
Landoll RR, Cervero RM, Quinlan JD, Maggio LA. Primary Care Behavioral Health Training in Family Medicine Residencies: A Qualitative Study From a Large Health Care System. Fam Med. 2020;52(3):174-181.
Crebolder HF, op 't Root JM. Een transmuraal onderwijsprogramma vanuit de huisartspraktijk [Transmural teaching program from the family practice viewpoint]. Ned Tijdschr Geneeskd. 1994;138(29):1486-9.
Kershnar R, Hooper C, Gold M, Norwitz ER, Illuzzi JL. Adolescent medicine: attitudes, training, and experience of pediatric, family medicine, and obstetric-gynecology residents. Yale J Biol Med. 2009;82(4):129
O’Dowd E, Lydon S, O’Connor P, et al. The development of a framework of entrustable professional activities for the intern year in Ireland. BMC Med Educ. 2020;20(1):1-0.
Jarrett JB, Antoun J, Hasnain M. Entrustable Professional Activity Utilization: A CERA Study of Family Medicine Residency Program Directors. Fam Med. 2019;5(6):471-476.
Ten Cate O, Scheele F. Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med. 2007;82(6):542-7.
Frank J, Ten Cate O,Holmboe E, Mungroo R, et al. Competency-based medical education: theory to practice, Medical Teacher, 2010;32(8): 638-645.
Norman G, Norcini J, Bordage G. Competency-based education: Milestones or millstones? J Grad Med Educ. 2014;6:1-6.
Part 1: Rationale http://www.royalcollege.ca/rcsite/documents/cbd/cbd-handout-faq-part-1-rationale-april-27-e.pdf
Ten Cate O. AM last page: what entrustable professional activities add to a competency-based curriculum. Acad Med. 2014;89(4):691.
El-Haddad, A. Damodaran, H. P. McNeil, W. Hu. The ABCs of entrustable professional activities: an overview of ‘entrustable professional activities’ in medical education. Internal Med. 2015:19;1006-1010.
McGaghie WC, Miller GE, Sajid AW, Telder TW. Competency-based curriculum development in medical education - an introduction. World Health Organization; 1978.
Ten Cate O. Competency-based postgraduate medical education: past, present and future. GMS Journal for Medical Education. 2017;34(5).
Bryson JM. What to do when stakeholders matter: Stakeholder identification and analysis techniques. Public Manag Rev. 2004;6:21-53.
Fottler MD, Blair JD, Whitehead CJ, Laus MD, Savage GT. Assessing key stakeholders: Who matters to hospitals and why?. Hosp Health Serv Adm. 1989;34:525-546.
Dennis AA, Cleland JA, Johnston P, Ker JS, Lough M, Rees CE. Exploring stakeholders’ views of medical education research priorities: A national survey. Med Educ. 2014;48:1078-1091.
Lockman JL, Schwartz AJ, Cronholm PF. Working to define professionalism in pediatric anesthesiology: A qualitative study of domains of the expert pediatric anesthesiologist as valued by interdisciplinary stakeholders. Paediatr Anaesth. 2017;27:137-146.
Boelen, Charles, Heck, Jeffery E & World Health Organization. Division of Development of Human Resources for Health. Defining and measuring the social accountability of medical schools. Geneva. World Health Organization.1995.
World Health Organization. Towards Unity for Health: Challenges and opportunities for partnership in health development. Genva. World Health Organization. 2000.
Fürstenberg S, Schick K, Deppermann J, et al. Competencies for first year residents – Physicians’ views from medical schools with different undergraduate curricula. BMC Med Educ. 2017;17(1).
Kwan J, Crampton R, Mogensen LL, Weaver R, van der Vleuten CPM, Hu WCY. Bridging the gap: A five stage approach for developing specialty-specific entrustable professional activities. BMC Med Educ. 2016;16:117.
El-Haddad C, Damodaran A, McNeil HP, Hu W. A patient-centered approach to developing entrustable professional activities. Acad Med. 2017;92:800-808.
Graham MJ, Naqvi Z, Encandela JA, et al. What indicates competency in systems-based practice? An analysis of perspective consistency among healthcare team members. Adv Health Sci Educ Theory Pract. 2009;14:187-203.
Lundsgaard KS, Tolsgaard MG, Mortensen OS, Mylopoulos M, Østergaard D. Embracing multiple stakeholder perspectives in defining trainee competence. Academic Medicine. 2019 Jun 1;94(6):838-46.
Smith CS, Morris M, Francovich C, et al. A multisite, multistakeholder validation of the Accreditation Council for Graduate Medical Education competencies. Acad Med. 2013;88:997-1001.
Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15:1277-1288.
Holmboe E. The Transformational Path Ahead: Competency-Based Medical Education in Family Medicine. Fam Med. 2021;53(7):583-589.
Boelen C. Coordinating medical education and health care systems: the power of the social countability approach. Medical Education. 2018 52:96-102.

Competing interest reported. BC and DF were medical students at the Max Rady College of Medicine at the time of the study. JF and TB declare they have no competing interests.

AppendixQuestionnaireFMEPAsJan232022.docx

Multi-stakeholder Validation of Entrustable Professional Activities in Family Medicine

Status:

Version 1

Abstract

Background

Methods

Results

Discussion

Conclusions

Abbreviations

Declarations

References

Additional Declarations

Supplementary Files

Status:

Version 1