Overcoming The Barriers To Implementation Of Competence-Based Medical Education: A Narrative Literature Review

DOI: https://doi.org/10.21203/rs.3.rs-1314239/v1

Abstract

Background: To ensure that residents are equipped with the necessary skills for practice, competence-based medical education (CBME) represents a transformative change in postgraduate medical education, which is being progressively introduced across Canadian specialty residency programs.  Successful implementation will require adjustments to curriculum, assessment, and evaluation, with careful attention to the unique needs in the local context, including resident and faculty development.

Methods: This narrative review of the literature aimed to determine the potential barriers to the successful implementation of CBME and the strategies by which they can be addressed, with a specific consideration of the author’s program in pediatrics in Manitoba. Eleven articles were identified with a specific focus on the implementation of CBME in the post-graduate setting, and 10 were included in the review after critical appraisal.

Results: Three key themes emerged from the articles: the value of broad stakeholder engagement and leadership, the importance of faculty and resident development, and the development of specific support systems for the educational curriculum. Different strategies were considered and contrasted for addressing these important themes.

Conclusion: This review provides important insights and practical approaches to the barriers that should be useful as programs prepare for the implementation of CBME.

Background

In light of the increasing complexity of health care and the importance of social accountability in its delivery, a competency-based approach to medical education is meant to ensure that recently graduated doctors, generally called residents, are equipped with the necessary skills for their ultimate specialty practice1. Competence-based medical education (CBME) is a shift from the traditional apprenticeship model to outcomes-based training, where the ultimate goal is for trainees to develop and demonstrate the necessary competence for patient care2,3. In Canada, the Royal College of Physicians and Surgeons is responsible for the oversight of training, evaluation, and certification of medical and surgical specialists, and is currently in the process of progressively introducing a competency-based approach in all its programs4. Dagnone et al.5 describe the introduction of CBME as a transformative change for the medical education community, which will necessitate an “evidence-based way of thinking and acting to both prepare our community…and support us through emergent adaptations” (p. 1644).

Holmboe6 describes that CBME provides opportunities for resident to go beyond basic medical knowledge and clinical skills to emphasize competencies in the core activities of the discipline. Application of these competencies is enabled through the definition of Entrustable Professional Activities (EPAs), which are the specialty specific tasks which the resident is delegated to perform independently once competence has been demonstrated to their supervisor4. Each EPA includes several milestones, which represent the measurable and developmental indicators of the resident’s ability to perform the task2. Documentation of progress on milestones and EPAs requires a shift from global evaluations of performance on a clinical rotation and summative examinations to workplace-based assessments in the clinical environment6.

It is evident that these major cultural shifts in medical education will require widespread stakeholder engagement, consideration of interpersonal and group dynamics, and careful evaluation of implementation needs in the local context5. In preparing to introduce CBME into the anesthesiology residency program at the University of Ottawa, Stodel et al.2 describe the importance of using a resident-centered approach in their curricular design, with an emphasis on active learning and individualized progression through the program as measured in evidence-based milestones and EPA. They also describe the need for a comprehensive system of assessment and evaluation, and the importance of faculty development and change management for both residents and faculty in transitioning to this new educational paradigm.

Pediatric training in Canada is scheduled to implement CBME with the residency cohort starting July 2021. As the program director for pediatrics at the University of Manitoba, the author will be directly responsible for leading this transition in our program, with 40 residents and a faculty of nearly 120 physicians. Drawing on the experiences of other programs, the aim of this narrative review is to determine the potential barriers to the successful implementation of CBME in pediatrics in Manitoba and the strategies by which they can be addressed.

Methods

This literature review identified scholarly research in CBME through the electronic databases available at the author’s university library (www.twu.ca/library). Databases included in this review were the university library’s OneSearch, ERIC (Education Resource Information Center), MEDLINE, Academic Search Ultimate, and CINAHL. The following key subject terms were used to search the databases with Boolean “AND” to combine the search terms: “competenc*-based medical education”, post-graduate OR residen*, transition OR change. After removing duplicate results from the different databases, a total of 80 papers were identified. Those articles were then manually screened by title and abstract, and 15 articles were identified as potentially relevant to this review. The same search was performed on Google Scholar which did not result in any new results.

Inclusion And Exclusion Criteria

The 15 articles were either retained or eliminated based on the inclusion and exclusion criteria. The reviewed articles were required to meet the following criteria: 1) peer-reviewed papers published between 2011-2021, 2) accessible through the university library, and 3) implementation of post-graduate CBME as a central theme. Conversely, articles were excluded based on the following criteria: 1) focus on undergraduate medical education, independent practice, or continuing professional development, or 2) health care transformation as the central theme. Of the 15 articles identified, 11 articles met the inclusion criteria and were considered in the review. These 11 articles were drawn from a variety of scholarly journals.

Critical Analysis

Each of the 11 articles was critically appraised using the methods described by Plano-Clark and Creswell7 for quantitative or qualitative research methods as appropriate. The majority of articles identified were of adequate or high quality and included in the review. One paper, an invited commentary by Dagnone et al.5, was excluded at this stage; while it did provide an interesting perspective on the high-level priorities of CBME, it did not contribute in a relevant way to the purposes of this review.

Results

The articles included in this review represent a variety of perspectives and approaches to the challenges in implementing CBME within post-graduate medical training programs. Three articles provided a broad systems review810. There were surveys looking at perceptions of faculty and residents11, faculty needs assessment12, and program preparedness13. Two studies used qualitative interviews to explore perceptions of residents14 and both residents and faculty15. Finally, there were two case studies of a single program, one in the emergency department16 and one in anesthesiology17. Across these different articles, three important themes emerged: the value of broad stakeholder engagement and leadership, the importance of faculty and resident development, and the development of specific support systems for the educational curriculum.

Stakeholder Engagement And Leadership

Nousiainen et al.10 advocate for a health care curriculum that is integrated with health human resources planning and a need for educational continuity as “policies that support the principles of CBME at all levels of medical education” (p. 594), from the early undergraduate trainee to the continuing professional development of the practicing physician. They argue this requires appropriate leadership to champion the necessary changes, which is echoed by Dagnone et al.9 in calling for program leaders “to engage in a community of shared learning and leadership” (p. e32). Boet et al.15 also express the limitations of literature alone in disseminating knowledge and best practices of CBME, and the advantages of fostering a community of practice to share information both within and between programs.

Another frequently raised concern was commitment and engagement from the faculty and residents directly impacted by CBME14,15,17. Dagnone et al.9 promoted the need for centralized institutional governance and support for CBME, with the appropriate allocation of financial support, time, and educational resources. In their faculty development efforts, Fraser et al.17 noted that financial and other support would have improved engagement and attendance. Focusing specifically on the resident perspective, Mann et al.14 called for increased opportunities for resident engagement and feedback on the implementation of CBME, which were “repeatedly suggested as vital to a successful transition” (p. e37) by their interview participants. This was supported in the systems-based review by Dagnone et al.9, who described the establishment of a resident sub-committee at their center to provide a forum for resident advocacy and discussion.

Ultimately, stakeholder engagement needs to include and be driven by the institutional leadership and priorities, which may be challenging to affect in that they are often external to the program itself 17. Caverzagie et al.8 advocated for clinical program redevelopment to occur in concert with the educational redesign of CBME, requiring the consideration of potential medical regulatory changes and input of those bodies. They also expressed the importance of mutual accountability of the different stakeholders throughout the implementation process. This was validated by Hall et al.16 in their evaluation of CBME implementation in a single emergency medicine program; ongoing efforts at continuous quality improvement led to better investment in the process by the different stakeholders.

Faculty And Resident Development

Fraser et al. 17define faculty development as “any planned activities designed to improve an individual’s knowledge, skills, and attitude in areas related to the roles and responsibilities of a faculty member” (p. 1365), and this definition would apply equally to resident development. Their article provided a review of initiatives undertaken in their anesthesiology program at the University of Ottawa; they initially found a lack of faculty knowledge and skills to shift to CBME, a lack of technological expertise for delivery of CBME curriculum, and a lack of time and ability to provide proper resident assessment. These findings led the authors to develop a variety of faculty development programs, particularly around the curricular format of CBME and the process of workplace-based assessment, which were hindered by the lack of appropriate incentives as presented earlier. Unfortunately, the process of their review was not defined, which impacts the reliability and generalizability of the findings.

In their qualitative interviews of program directors and resident focus group, Boet et al.15 were adversely impacted by poor recruitment and subsequently a lack of saturation of the findings, but they did highlight some important negative elements and concerns of CBME. In particular, they identified the importance of defining the key terminology in CBME, and particularly in relation to assessment criteria and frequency. This was also cited as an important area for consistent definitions by Caverzagie et al.8.

Stefan et al.12 performed a more detailed and structured needs assessment of Canadian emergency medicine program faculty and trainees, and uncovered similar issues. They describe that their faculty generally believe they give good feedback to residents but have a desire to improve. These faculty also identified the time required to deliver workplace-based observations, potentially impacting patient flow and care, and the possible repercussions of delivering negative feedback as barriers to effective assessment. Their survey also uncovered a general lack of familiarity with the details of CBME, which was their explanation for the finding that nearly half of respondents did not anticipate benefits in training or patient care with CBME. Similar concerns were identified in medical oncology programs13.

In their interviews with residents, Mann et al.14 identified the need for appropriate faculty development to create a change in feedback culture, which Caverzagie et al.8 also describe as a requirement for CBME to be successful. This fits well with the observation by Hall et al.16 that similar culture change efforts within the emergency medicine program led to “substantial efforts and adaptations in the form of faculty and resident development activities relating to the provision, documentation, and acceptance of constructive feedback” (p. 790). They describe this as a core component of CBME, requiring iterative cycles of instruction, practice, and feedback to fully develop, consistent with their discussion of the value of continuous quality improvement, as presented in the first theme.

Crawford et al.11 surveyed both residents and program directors regarding perceptions and barriers of CBME, similar to the efforts of Boet et al.15 which were specific to the anesthesiology program. Crawford et al.11 highlight the importance of both resident and faculty development, and in particular the role of the resident in partnering with faculty in their assessments. Their survey found that residents had the least positive opinion of CBME, and noted that residents might not recognize the practical relevance of the EPA, since they are not yet fully versed in the skills of clinical practice. Their call for residents to be engaged in CBME leadership echoes the call in the first theme for communication and engagement with residents throughout the implementation of CBME9,14.

Educational And Technological Support

The resident focused approach to training was highlighted in several articles9,10,13,14,15. Nousiainen et al.10 refer specifically to the use of time as a resource rather than an endpoint in CBME, necessitating the changed approach to health care education as described in the first theme. While recognizing the value of personalizing a resident’s training according to their own developmental needs, Boet et al.15 listed potential adverse impacts, such as a desire to rush through training to enter the workforce at full salary, competition for training experiences and observations, and the administrative challenges of individualized training programs. In their interviews, Mann et al.14 also observed that such practical and logistical challenges were associated with pessimism from the residents on the success of CBME, and they advocated for the possibility of resident input in mitigating them.

In addition to the administrative reorganization of training experiences, appropriate curriculum delivery, assessment, and evaluation will also require educational and technological support10,11. Simulation may be necessary for those EPA that are less frequent9, and Arora et al.13 describe how this might actually create new training opportunities.

These core pedagogical elements require rapid introduction in the early implementation of CBME, to provide appropriate systems, support, and leadership as foundations for the subsequent faculty and resident development16. Solutions to these practical challenges are suggested in the previous two themes, with stakeholder engagement, strong leadership, and faculty and resident development playing important roles.

Discussion

In exploring the barriers to implementation of CBME and how to address them, this literature review highlights three key themes: the importance of stakeholder engagement and leadership, the value of faculty and resident development for the specific tasks and requirements of CBME, and the implementation of the necessary educational and technological solutions for this new training paradigm. It is reassuring that the barriers and solutions described were consistent across the different articles and research types, suggesting saturation in the literature search. From a practical perspective, it is also important to note that the findings of the broad systems reviews810 were echoed in the studies of specific programs12,13,15−17, suggesting real world applicability of the proposed solutions.

One limitation of the review is the generalizability of the results. Many articles reflected the perspectives and findings of a single residency program12,13,15 and often at a single site16,17. There were also challenges with the variety of methodologies, and methodologic rigour, used by the individual articles. The two case studies provide useful information, but the review process is unclear in Fraser et al.17 and not validated by observation in Hall et al.16. Mann et al.14 used a grounded theory approach in their semi-structured interviews with residents, while Boet et al.15 failed to achieve saturation due to poor recruitment. The commonality of findings suggests that these limitations did not ultimately impact the validity of the identified themes. For future evaluation of CBME implementation, action research will allow iterative reviews as knowledge and experience is gained, and can also look specifically at the impact of the local training program context on the barriers to implementation and their solutions. Further qualitative studies will also be valuable to assess the changing perceptions and impact of CBME over time, and may identify previously unexpected or unappreciated challenges that will need to be addressed. This is analogous to the continuous quality improvement advocated by Hall et al.16.

Conclusion

As pediatrics training programs across Canada prepare to implement CBME, we are collaborating on strategies through our national program directors working group. This has been a valuable community of practice to share experiences and has already influenced our faculty, resident, and curricular development efforts. The combined experience of the implementation to CBME reviewed herein bears informative witness to the common challenges. The three identified themes provide important insights and practical approaches to the barriers that we are likely to encounter. Identifying and understanding them will benefit the Manitoba pediatrics program in our transition, and our experience will add to this knowledge for those programs who have yet to move to CBME.

Declarations

Ethics approval and consent to participate: Not applicable

Consent for publication: Not applicable

Availability of data and materials: Not applicable. Literature sources are provided in the references

Competing interests: Dr. Stoffman has no relevant conflicts of interests to declare.

Funding: Dr. Stoffman did not receive any funding or support for the preparation of this manuscript.

Authors' contributions: This paper was researched and written excusively by JS.

Acknowledgements: Not applicable

Authors' information: JS is the Director of Post-Graduate Medical Education for the Department of Pediatrics and Child Health at the University of Manitoba, which recently implemented competence-based medical education for its residency training.

References

  1. Snell L, Frank JR, Stoneham G, et al Competency-based medical education. In KA Harris & JR Frank, editors, Competence by Design: Reshaping Canadian Medical Education (pp. 99-105). 2014: Royal College of Physicians and Surgeons of Canada. https://www.royalcollege.ca/rcsite/educational-initiatives/cbd-medical-education-ebook-e.
  2. Stodel EJ, Wyand A, Crooks S, Moffett S, Chiu M, Hudson CCC. (2015). Designing and implementing a competency-based training program for Anesthesiology residents at the University of Ottawa. Anesthesiology Research & Practice. 2015: 1-7. https://doi.org/10.1155/2015/713038.
  3. Van Melle E, Frank JR, Holmboe ES, Dagnone D, Stockley D, Sherbino J. (2019). A core components framework for evaluating implementation of competency-based medical education programs. Academic Medicine. 2019;94(7):1002-1009. https://doi.org/10.1097/ACM.0000000000002743.
  4. Royal College of Physicians and Surgeons of Canada. Competence by Design. https://www.royalcollege.ca/rcsite/cbd/competence-by-design-cbd-e.
  5. Dagnone JD, Chan MK, Meschino D, et al. Living in a world of change: Bridging the gap from competency-based medical education theory to practice in Canada. Acad Med. 2020;95(11):1643–6. https://doi.org/10.1097/ACM.0000000000003216.
  6. Holmboe ES. Competency-based medical education and the ghost of Kuhn: Reflections on the messy and meaningful work of transformation. Acad Med. 2018;93(3):350–3. https://doi.org/10.1097/ACM.0000000000001866.
  7. Plano-Clark VL, Creswell JW. Understanding research: A consumer’s guide (. 2nd ed. ). Boston: Pearson; 2015.
  8. Caverzagie KJ, Nousiainen MT, Ferguson PC, et al. Overarching challenges to the implementation of competency-based medical education. Med Teacher. 2017;39(6):588–93. https://doi.org/10.1080/0142159X.2017.1315075.
  9. Dagnone D, Stockley D, Flynn L, et al. Delivering on the promise of competency based medical education - an institutional approach. CMEJ. 2019;10(1):e28–38. https://doi.org/10.36834/cmej.43303.
  10. Nousiainen MT, Caverzagie KJ, Ferguson PC, Frank JR. Implementing competency-based medical education: What changes in curricular structure and processes are needed? Med Teacher. 2017;39(6):594–8. https://doi.org/10.1080/0142159X.2017.1315077.
  11. Crawford L, Cofie N, McEwen L, Dagnone D, Taylor SW. Perceptions and barriers to competency-based education in Canadian postgraduate medical education. J Eval Clin Pract. 2020;26(4):1124–31. https://doi.org/10.1111/jep.13371.
  12. Stefan A, Hall JN, Sherbino J, Chan TM. Faculty development in the age of competency-based medical education: A needs assessment of Canadian emergency medicine faculty and senior trainees. CJEM. 2019;21(4):527–34. https://doi.org/10.1017/cem.2019.343.
  13. Arora R, Kazemi G, Hsu T, et al. Implementing changes to a residency program curriculum before competency-based medical education: A survey of Canadian medical oncology program directors. Curr Oncol. 2020;27(6):e614–20. https://doi.org/10.3747/co.27.6659.
  14. Mann S, Truelove AH, Beesley T, Howden S, Egan R. Resident perceptions of competency-based medical education. CMEJ. 2020;11(5):e31–43. https://doi.org/10.36834/cmej.67958.
  15. Boet S, Pigford AAE, Naik VN. Program director and resident perspectives of a competency-based medical education anesthesia residency program in Canada: A needs assessment. Korean J Med Educ. 2016;28(2):157–68. https://doi.org/10.3946/kjme.2016.20.
  16. Hall AK, Rich J, Dagnone JD, et al. It’s a marathon, not a sprint: Rapid evaluation of competency-based medical education program implementation. Acad Med. 2020;95(5):786–93. https://doi.org/10.1097/ACM.0000000000003040.
  17. Fraser AB, Stodel EJ, Jee R, Dubois DA, Chaput AJ. Preparing anesthesiology faculty for competency-based medical education. Can J Anesth. 2016;63(12):1364–73. https://doi.org/10.1007/s12630-016-0739-2.