The role of medical regulations and medical regulators in fostering the use of eHealth data for strengthened Continuing Professional Development (CPD). A document analysis with key informants’ interviews

doi:10.21203/rs.3.rs-3086637/v1

Download PDF

Research Article

The role of medical regulations and medical regulators in fostering the use of eHealth data for strengthened Continuing Professional Development (CPD). A document analysis with key informants’ interviews

https://doi.org/10.21203/rs.3.rs-3086637/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background

In recent times, medical regulators have been taking measures to strengthen CPD requirements for medical practitioners. In particular, greater emphasis has been placed on CPD activities linked to workplace-based assessment, health outcomes measurement, and quality improvement. These activities require the use of health data, and eHealth data analytics is emerging as a digital solution to simplify tasks and processes.

Although there is a growing interest and need for alignment between regulatory policies, impactful CPD activities, and digital health research and innovation, there is little or no research into the role that medical regulations and regulators are playing in fostering the use of eHealth data to strengthen CPD.

Methods

Medical regulations and CPD requirements of 5 selected countries (Australia, Canada, New Zealand, UK, USA) were collected and analysed using the systematic READ approach for qualitative health policy research.

Online semi-structured interviews were conducted with 20 key informants from 13 medical bodies to validate findings and gather additional insights. Informants were purposively selected because of their direct involvement in policy development. The interviews were analysed using a hybrid approach of deductive and inductive thematic analysis. The COREQ checklist was used for reporting the findings.

Results

The documents analysed do not mention the use of eHealth data for CPD purposes or refer to it only as a potential data source for CPD completion and compliance.

Participants corroborated the document analysis results and provided insights into the following themes: context and rationale of current policy choices and future policy development; roles, responsibilities, and functions of relevant medical bodies in fostering the use of eHealth data for strengthened CPD; barriers, challenges, and enablers for implementation.

Conclusion

Current medical regulations and CPD requirements do not foster the use of eHealth data for CPD purposes. Recommendations for future policy development are reliant on further research on key policy concepts, regulators’ internal organisational factors, and interorganisational collaboration within the CPD ecosystem.

The alignment of all relevant CPD stakeholders is required to tackle existing barriers and challenges and promote digital health innovation in the CPD landscape. Medical regulators are called to play a leadership role in this scenario.

eHealth data analytics

Medical regulations

Continuing Professional Development (CPD) requirements

Medical regulatory bodies

Medical practitioners

Practice Analytics in Healthcare (PAH)

Document analysis

Key informants’ interviews

In recent times, regulatory medical bodies have been taking measures to strengthen Continuing Professional Development (CPD) requirements for medical practitioners. It has long been recognised that traditional didactic Continuing Medical Education (CME) activities have limited impact on improving clinical practice and patient outcomes [1,2]and there is increasing consensus on the value of external assessment and feedback in stimulating reflection on practice and encouraging behaviour change [3,4,5,6]. Prompted by this, regulators have placed greater emphasis on CPD activities linked to daily medical practice, workplace-based assessment, health outcome measurement, and Quality Improvement (QI) [7,8,9,10,11].

The use of performance feedback and performance data for formative purposes is at the core of workplace-based assessment CPD activities [12]. In particular, several of these activities – such as Audit and Feedback (A&F) interventions, QI projects, and Mortality and Morbidity Meetings (MMM) – require the use of performance data obtained by patient health data analysis [13,14,15].

While eHealth data is increasingly being used in the delivery of care, large volumes of health data are still contained in paper records [16,17,18]. In addition, data collection, analysis, and reporting are currently predominantly manual, making the completion of performance work-based assessment CPD activities lengthy and burdensome for medical practitioners [16,19,20,21].

In order to address these limitations and to support medical practitioners undertake their CPD, eHealth data usage and big data analytics technologies are emerging as a digital solution to simplify tasks and streamline processes [22,23,24]. Researchers in both academia and industry are in fact exploring how to make eHealth data more accessible and actionable for health professionals’ reflective practices and CPD obligations [25,26,27,28,29]. Moreover, emerging research is advocating for the integration of information technology systems for learning with clinical information systems to inform health care and to support continuing education and professional development [30].

Although there is a growing interest and need for alignment between regulatory policies, impactful CPD activities, and digital health research and innovation, at present there is no available research on the matter. In particular, there is no clarity on the role of medical regulations and medical regulators in fostering the use of eHealth for strengthened CPD.

This study aims to address this existing literature gap by investigating the following: i) if and how eHealth data is referenced/mentioned in current medical regulations and CPD requirements for medical practitioners; ii) whether eHealth data is mentioned as a data source to complete some of the CPD categories and activities specified in the CPD requirements; iii) how medical bodies regard the use of eHealth data for strengthened CPD; and iv) what role these bodies are willing to play with respect to the use of eHealth data analytics technologies for CPD.

Definition of terms

Health data and eHealth data

In this study, health data is any personal information concerning the health status, risk, or outcome of an individual and/or pertaining the provision of healthcare interventions and services to the individual [31,32,33,34]. eHealth data can be hence considered as health data in digital or electronic format, and therefore includes: Electronic Medical Records (EMRs), Electronic Health Records (EHRs), registries, routinely collected administrative data, claim and billing data, electronic prescriptions, Patient-generated Health Data (PGHD), Patient-Reported Outcome Measures (PROMs), and Patient-Reported Experience Measures (PREMs).

Regulatory and certifying bodies

Despite the existence of different jurisdictional approaches, the key responsibility of medical regulatory bodies is to protect the public by ensuring that medical practitioners are effectively trained, competent, and fit to practice. In view of this, such authorities are responsible for: i) establishing registration/licensure/certification credentials and processes, ii) maintaining a public register of practicing medical practitioners, iii) setting standards of practice for the profession, and iv) managing complaints and disciplinary actions [35,36].

Depending on the jurisdiction, some medical regulatory bodies are also responsible for: 1) developing regulatory policies and frameworks for registration renewal, certification renewal/maintenance, or revalidation processes; and 2) setting and enforcing compliance requirements in regard to Continuing Education (CE), CME, and CPD. Among these, CPD compliance requirements are the focus of interest in this study.

Notably, in jurisdictions such as Canada and the USA, some of the abovementioned tasks are carried out by medical certifying bodies. For this reason, these bodies are deemed as relevant for the purposes of this study.

Continuing Professional Development (CPD) requirements

Although the literature provides several definitions for Continuing Professional Development [37], in this study CPD refers to those compliance requirements set by the abovementioned regulatory or certifying bodies for registration renewal, certification renewal, revalidation, or maintenance of certification purposes [38].

In order to address the research aims of this study, a systematic document analysis of pertinent medical regulations and related CPD requirements for medical practitioners was performed for 5 selected countries. In addition, semi-structured interviews with relevant international key informants were conducted to validate findings and gather additional insights on the matter, as documents may provide a limited description of the context and only a partial account of ongoing debates and potential developments [39].

Document Analysis

Document analysis is a qualitative research method, and it can be defined as a “systematic procedure for reviewing or evaluating documents – both printed and electronic” [40] (p. 27). Its results are often triangulated with other sources of data, such as interviews and academic literature, to contextualise findings, seek corroboration, and gather additional insight [40,41].

The document analysis illustrated in this study was undertaken using the 4-step systematic method for qualitative health policy research named the READ approach: 1. Ready your materials; 2. Extract data; 3. Analyse data; and 4. Distil your findings [42].

In a preliminary phase, Australia, Canada, New Zealand, the UK, and the USA were purposively selected as countries of interest because of their historical relations, ongoing collaboration, and mutual influences in terms of health policy development and implementation.

Subsequently, i) relevant bodies for each country were selected based on their role and responsibilities in setting CPD requirements, and ii) related medical regulations and CPD requirements were identified and stored in NVivo (Version 20.6.1, from QSR) for data extraction and analysis (step 1). The documents deemed pertinent are all in the public domain, and they are freely available on the internet as downloadable files or webpages (Table 1). [Insert Table 1 here]

Descriptive information was then extracted and analysed using basic qualitative content analysis [43] (step 2 and 3). Data was extracted on the following matters: 1) medical regulatory processes; 2) CPD requirements and their effective date; 3) CPD categories, as set by the relevant medical body and specified in the corresponding CPD requirements; 4) CPD categories and activities that need the use of health data to be completed; 5) mention of the use of eHealth data for the completion of the categories and/or activities in point 4.

Finally, findings were distilled into a tabular form and also into a descriptive format (step 4).

Key Informants’ Interviews

Semi-structured interviews with key informants were conducted in order to validate the findings of the documentary analysis and collect additional information on policy development and implementation.

As a first step, an environmental scan was conducted to identify potential participating medical bodies. The definitions of regulatory and certifying body provided above were used as a discriminant for determining whether the body was suitable for the purposes of this study. Subsequently, executives, managers, and administrators directly involved in the development of medical regulations and the establishment of CPD requirements were contacted using general administration or individual email addresses depending on the information publicly available on the internet. 13 bodies agreed to participate, for a total of 20 key informants (Table 2). [Insert Table 2 here]

Interviews were conducted following a semi-structured outline approved by the University of Sydney Human Research Ethics Committee (Protocol Number 2020/722). The interview questions focused on the following matters: i) national regulatory landscape; ii) content of medical regulatory documents and CPD requirements; iii) role of the body in regulating CPD; iv) health and eHealth data analytics and CPD requirements; v) role of the body in fostering the use of eHealth data analytics for CPD purposes.

All interviews were conducted on Zoom; audio-recordings were transcribed verbatim using DeScript. After being assessed for accuracy, interview transcripts were stored in NVivo and analysed using a hybrid approach of deductive and inductive thematic analysis [44,45]. At first, broad codes were established a priori based on the study aims and the semi-structured interview outline. As the interviews were analysed, additional a posteriori codes were created from the interviews’ data. All codes were then combined. Iterative analysis was conducted until data saturation was achieved.

The Consolidated Criteria for Reporting Qualitative Research (COREQ) checklist [46] was used to ensure accurate and complete reporting of the interview findings.

Document Analysis Results

Given the variance of regulations, stakeholders, and terminology across the jurisdictions under investigation, document analysis results are presented alongside with an overview of the medical regulatory system and processes for each country to assure clarity.

A detailed account of data extraction results is provided in Table 3. [Insert Table 3]

Australia

In Australia, the Australian Health Practitioner Regulation Agency (Ahpra) regulates registered health practitioners, and it works in collaboration with 15 National Boards to set the standards that all health practitioners must meet to register and maintain their registration to practice medicine [47]. In particular, the Medical Board of Australia (MBA) sets professional regulation and registration standards for medical practitioners in Australia [48], in accordance with the Health Practitioner Regulation National Law [49]. Except for registered students and non-practising registrants, MBA mandatory registration standards apply to all applicants for registration and registration renewal [50]. Among other standards, the MBA sets the CPD Registration Standards for all Australian medical practitioners.

As per Table 3, the current MBA CPD Registration Standards do not mention the use of eHealth data for CPD purposes.

Available supporting documentation indicates though that: i) both health and eHealth data are at the basis of “reviewing performance” and “measuring outcomes” activities [51] ( p. 9); ii) “improved access to data will help improve safety and quality by giving doctors the tools to reflect on and review their practice routinely and measure outcomes” [52] (p.3); and that the MBA will “urge governments and other holders of large data (such as Medicare) to make data accessible to individual registered medical practitioners to support performance review and outcome measurement” [52] (p. 7).

Canada

In Canada, provincial and territorial Medical Regulatory Authorities (MRA) are responsible for registering and licensing all medical practitioners who practice within their jurisdiction. These authorities, some of which are also known as “licensing colleges”, are members of and report to the Federation of Medical Regulatory Authorities of Canada (FMRAC) [53].

Canadian medical practitioners must meet the registration requirements set by the relevant MRA to achieve full licensure and must obtain a certification by examination before applying for medical registration [54].

In this regulatory landscape, CPD is an essential component of license renewal, maintenance of certification, admission and renewal of college Fellowship, and attainment and maintenance of college post-nominal designations.

MRA-approved CPD programs are offered by the Canadian certifying colleges i.e., The College of Family Physicians of Canada (CFPC), The Royal College of Physicians and Surgeons of Canada (Royal College), and the Collège des médecins du Québec (CMQ).

Every certifying college sets different CPD requirements for their fellows, offers a different CPD program, and establishes different CPD categories. Apart from these differences, what is of interest for the purposes of this study is that health data is one of the potential data sources necessary to complete the activities in the “assessment” category in all CPD programs. However, eHealth data is not explicitly mentioned in the programs, or it is referred to as a potential data source only.

For comprehensiveness, it is necessary to report that 2 certifying colleges have developed initiatives and models that refer to health data and eHealth data usage for CPD purposes:

In 2017 the CFPC launched the Practice Improvement Initiative (Pii) to promote the use of “quality improvement, practice-level data, and research to improve everyday practice” (55) in family medicine. The Pii is categorized as an “advocacy and innovation” initiative and states that practices and primary care teams are called to “promote Mainpro+® credits [CPD credits] for activities related to QI and the use of practice-level data” [55].
The Royal College is currently evolving its CPD program through the “Model for Improvement (MFI)” in order to integrate QI into CPD [56]. The MFI links QI to CPD Section 2 and 3 of the Royal College CPD program and mentions the use of health data, eHealth data, peer feedback, and self-reflection entries/records as data sources necessary for the completion of some of the activities included in those Sections.

New Zealand

Under the Health Practitioners Competence Assurance Act 2003 (HPCAA) [57], New Zealander medical practitioners must be registered with the Medical Council of New Zealand (MCNZ) and hold a current practicing certificate to practice medicine. To maintain their right to registration and demonstrate competence to practice, they must also meet the MCNZ recertification program requirements [58] – which include participating in Continuing Professional Development [59].

Current MCNZ CPD requirements are specified in the policy document titled “Recertification requirements for vocationally-registered doctors” [60]. In terms of health data usage, the policy states that “patient/health outcomes” are necessary to complete activities that fall in the “Measuring and improving outcomes” CPD category. However, there is no mention of eHealth data usage for such purpose.

UK

In the UK, medical practitioners need to be registered with and licensed by the General Medical Council (GMC) in order to practise medicine [61]. Starting from 2012, all medical practitioners who hold registration with a licence to practise in the UK must comply to revalidation requirements [62].

Revalidation is a mandatory 5-year process that licensed practitioners must undertake to demonstrate that they meet the principles and values set out in the “Good medical practice” guidance [63], and it is defined by the GMC as “an evaluation of [medical practitioners’] fitness to practise” [7] (p. 3). Among other obligations, revalidation includes also annual CPD requirements.

Regarding the use of health data, tools like audits and QI initiatives are repeatedly mentioned in the GMC CPD guidance [64]. Also, the revalidation process itself includes the requirement to participate in at least one documented QI activity in every cycle [7]. Although these tools and activities require the analysis of patient-related data, there is no mention of eHealth data usage in the GMC policy documents.

USA

In the USA, medical practitioners receive a licence to practice medicine in a particular state after completing medical school and passing a licensing exam. After residency, practitioners may choose to become board certified. Board certification validates practitioners' competency standards and serves as a reliable indicator of the practitioner’s commitment to life-long learning, ongoing practice improvement, and excellence in patient care. Once certified, practitioners must undertake continuing certification programs offered by relevant medical certifying boards in order to retain their certification [65].

The American Board of Medical Specialties (ABMS) is responsible for setting professional standards for both initial and continuing certification in partnership with its 24 Member Boards, whereas CPD frameworks and programs are to be set and offered by the Member Boards depending on medical specialty.

Bearing in mind the complexity of the CPD landscape in the USA, what is of interest for this study is that the QI activities mentioned in the element “Improving Health and Health care” of the ABMS Standards for Continuing Certification [10] require the use of health data to be completed. However, eHealth data is not explicitly mentioned in the Standards.

Key Informants’ Interviews Results

The key informants confirmed and corroborated the findings discussed in the “Document Analysis Results” section with regard to: 1) regulatory landscape and processes; 2) CPD requirements, and 3) CPD categories and activities, and health/eHealth data usage for category and activity.

In addition to providing validation of results, the analysis of the interviews led to the identification of the 3 main themes described below. Full and additional quotes are provided
in Additional File 1.

Theme 1: Use of eHealth data for strengthened CPD

Valuable but complex. Several participants shared their opinion on the use of eHealth data for CPD compliance, describing it as a valuable opportunity with great potential for practice improvement and continuous learning and development: “I think there's huge potential. […] we'd like to see a world where practitioners are supported to continue to improve and to learn and develop and grow, and data is such a fantastic way for that to happen.” (P15).

However, many participants also commented on the complexity of such endeavour, pointing out how practices, processes, and procedures of performance assessment and measuring health outcomes are still in their infancy, and mentioning the existence of several barriers and challenges for the implementation of eHealth data analytics for CPD purposes.

Barriers. All participants stated that the biggest barrier to using eHealth data for CPD purposes is related to data availability and accessibility: “…we actually understood that, that it might be difficult [for medical practitioners] accessing data for some of the things that they need to do, so it's not, we know it's not at the moment readily available. We know this is a work in progress” (P13).

Lack of mature data systems and/or data system standardization, poor interoperability between existing data systems, and great variance in terms of data accessibility depending on jurisdiction, location, settings, and specialty were reported as the main issues to address for successful implementation.

In addition, participants’ concerns related to eHealth data usage for CPD purposes included – but were not limited to – ongoing controversies around data privacy, data sharing, and data relevance, and current limitations in terms of data quality, data integrity, and meaningful data reporting.

Challenges. All participants pointed out that current systemic issues around stakeholders’ priorities, strategic planning, and coordination of roles and responsibilities represent the main challenge for the implementation of eHealth data analytics for CPD compliance. Such position could be summarised as follows: “…there are no plans. There’s not even a group that’s looking at it. But there’s been a lot of talk and recommendations, but no one’s actually rolling up their sleeves to move this along. […] There’s no national coordination of this, right? It’s left to like, I think the hospitals… […] I think if we were to get together and say, ‘Here’s the greater good, and here's what it looks like.’ And we paint the picture of what we really want to enable, then we could divide off the roles. […] we would do it in a coordinated way that may actually contribute to a cohesive system. But that’s what’s missing.” (P06).

In addition to this, a number of additional challenges were mentioned by the informants: i) medical practitioners’ general attitudes around professional self-regulation and CPD value and effectiveness, ii) medical practitioners’ concerns around confidentiality, privacy, ownership, and potential punitive use of individual performance data, and iii) lack of workforce capabilities in engaging in QI activities and in using eHealth data analysis for such purposes.

Enablers. Although not prompted, participants suggested enablers and facilitators to foster the use of eHealth data for CPD purposes: i) making use of champions in healthcare service organisations in order to build trust among medical practitioners and promote the adoption of eHealth analytics practices at organisational, team, and individual level; ii) building a “support system” embedded in the workplace through the employment of advisory groups, mentors, and facilitators to assist teams and individuals in interpreting their data and making it actionable; iii) making more extensive use of Artificial Intelligence (AI) and machine learning at healthcare service level to avoid manual analysis and reporting, additional work for medical practitioners, and, consequently, potential workforce burnout.

Theme 2: eHealth data and policy

When discussing the results of the document analysis, several participants provided rationale for current policy choices and future policy development, and shared insights on policy development processes and policy terminology.

Current policy choices and future policy direction. Key informants clarified the motives for the lack of mention of eHealth data in current policy documentation. Their reasons varied from emphasising that regulators mostly provide high-level guidance with regard to CPD requirements and that the use of eHealth data for CPD is at the discretion of the individual medical practitioner, to commenting that prescribing the use of eHealth data could be a limitation rather than an opportunity. Some representative quotes include the following:

“It's actually not for us to say you have to use this set of data.” (P13)

“…we’d say what data would you find useful and how does that shine a light on your practice, and how does that help you reflect on what you’re doing, and you choose the data source that is most appropriate for that activity.” (P19)

“I would find it unlikely that we would say it must be this way with the electronic health record or with anything because that's sort of […] very limiting for the diplomates who were engaged in a lot of different efforts to improve care.” (P05)

Notably, some participants also shared their insights on future policy directions, clearly stating whether eHealth data usage will be made prescriptive or not:

“We won’t prescribe it.” (P19)

“We're changing our regulation of medical records outside of hospital, and it's submitted to the [governmental organisation] because we want the regulation to clearly stipulate that every physician in [country] will have to use an EMR. No more of those chart handling, and paper, and everything, and the loss, and the whatever, finished. […] Everybody will have to use an EMR to evaluate their practice.” (P08)

Current policy intent. A few research participants shed light on policy development processes, declaring that any mention of eHealth data and eHealth data analysis in policy documents does not imply the use of digital technologies nor big data analytics. Rather, it is an acknowledgment of eHealth data as a potential data source and of eHealth data analytics as a potential opportunity. As some participants stated: “…in the discussions that we had at the CPD expert group that developed the [CPD framework], people weren't attuned to the idea that there would be this sophisticated analytics. […] That there was recognition that there were these data collected but not a clear sense of how that might be used other than with those certain manual extraction type of way. […] It wasn't in the clear intentions, but the potential was by no means excluded.” (P03).

Meaning of term “data”. Some key informants argued that the term “data” in policy documents and CPD requirements does not necessarily refer to patient health data nor to eHealth data. Rather, “data” indicates any piece of information pertaining to medical practitioners’ practice and/or performance: “We think much more broadly around this whole reflecting on outcomes, um, because we are using data in a much broader sense. We're talking about data from multi-source feedback, outcomes data from visits to your practice and, and different things outside the normal thought pattern around data.” (P01).

Theme 3: Roles and responsibilities of medical bodies

All key informants expressed agreement on the potential advocacy and lobbying function of medical regulatory and certifying bodies in fostering the use of eHealth data for CPD compliance. Governments, the private healthcare sector, and/or any agency or organisation that regulates eHealth data management and governance at any level (national, provincial, state, territorial, organisational) were mentioned as relevant stakeholders to liaise with. As some participants remarked: “It's our job to form those alliances, form those partnerships with a whole range of different stakeholders, and all be pulling in the one direction to help practitioners to continuously improve.” (P4).

Also, key informants suggested that relevant medical bodies should take the following responsibilities to address current implementation barriers and challenges: i) engaging in partnerships with external data experts and research groups to link eHealth data analytics to medical regulatory processes and/or to receive guidance and recommendations on how to proceed in that regard; ii) for those bodies that also offer CPD resources, providing information, education, and support to medical practitioners with regards to eHealth data access and actionability; iii) for those bodies that also accredit CPD providers, rewarding the organisations that emphasise the usage of eHealth data in their programs and activities.

The purpose of this study was to analyse current medical regulations and related CPD requirements for medical practitioners in 5 selected countries in order to understand whether relevant policy documents encourage the use of eHealth data for CPD completion and compliance. Also, the study aimed to explore the standpoint of medical regulatory and certifying bodies on the matter, and to clarify what roles these bodies are willing to play to foster the use of eHealth data analytics for strengthened CPD.

The academic field of health professionals’ regulation is relatively underdeveloped [66] and, in particular, systematic evidence on the development of CPD requirements based on patient health outcomes is lacking [67]. Considering this, the discussion provides a summary of the results, attempts to provide a commentary in the context of relevant scholarly research, and proposes some recommendations for further research and future policy development and implementation.

Current policy choices: considerations on policy rationale and recommendations for further research

The overall results of the study show that medical regulations and CPD requirements do not currently foster the use of eHealth data for CPD purposes.

From the documents analysed, health data is generally considered necessary: i) for the completion of those CPD categories that focus on performance assessment and/or review and/or aim at measuring and improving health outcomes; and, in particular, ii) for the completion of CPD activities undertaken at the workplace and strongly linked to daily medical practice, such as audits and QI initiatives. However, the use of eHealth data for the same purposes is not explicitly mentioned in current CPD requirements for medical practitioners. The only exception is the MOC Program offered by the Royal College of Physicians and Surgeons of Canada and the CMQ “Formation continue obligatoire des Médecins”, which respectively refer to Electronic Health Records (EHRs) and Electronic Medical Records (EMRs) as potential data sources for CPD completion (Table 3).

It is important to note that Australian supporting documents and Canadian practice improvement initiatives and models do allude to the employment of eHealth data analysis for performance assessment and QI CPD activities [8,55,56]. However, the content and the intention of the MBA supporting documents is not reflected in its corresponding CPD Registration Standards, and the principles of the CFPC Pii and the Royal College MFI are not yet fully integrated in their respective requirements around CPD compliance.

Considering the interview results, it appears that that three main determinants influence current policy choices: 1) contextual barriers and challenges, 2) the high-level regulatory function of medical regulatory bodies, and 3) the comprehensiveness of the term “data”.

With regard to the first determinant, informants’ observations are supported by other research. Participants’ concerns around major data barriers are in fact in line with the existing literature on clinical performance and health outcome measurement issues [68,69,70], challenges with post-implementation experiences of large-scale EHRs [71], and clinical data quality and accuracy [72,73]. Also, participants’ comments on the presence of systemic, organizational, and individual challenges in the CPD landscape appear to reflect the findings on ongoing research on the associations between cultural and systemic factors and the use of clinical data for life-long learning (LLL) and CPD [74,75], biomedical and health informatics (BMHI) education for health professionals [76,77], and health practitioners’ attitudes towards and engagement with the use eHealth data for CPD purposes [78,79].

In light of this, participants’ standpoint could be considered as well-informed and regulators’ policy choices regarded as reasonable. However, it appears that regulators are currently overlooking ongoing industry and scholarly research in the fields of learning and practice analytics applied to performance assessment practices across the health professions education continuum [25,30,80,81,82,83,84,85]. Both fields are providing evidence on how performance data obtained by eHealth data analysis can be used to prompt self-reflection, identify knowledge and skill gaps, and better educational curricula and programs with the ultimate aim of improving clinical practice and patient care.

Taking into account these latest research developments, document analysis results would suggest that most regulators are yet to reflect digital health innovation research and trends in their regulatory policies and CPD requirements.

It is important to note, though,that some medical bodies are showing a different approach in this regard, with some participants expressing the intention to include eHealth data in future policies or even mandate its use for CPD compliance purposes. For this reason, it is necessary to conduct more research on organizational culture, vision, and priorities, legislative environment, and other potential jurisdiction-specific factors to understand whether these elements are having an impact on regulators’ decision and policy making processes in regard to the use of eHealth data for CPD.

In relation to the second determinant, the majority of the informants stated that medical regulatory bodies have the institutional responsibility to set CPD requirements but not the authority to establish detailed regulations in this regard nor the onus to indicate or prescribe what type of data sources should be employed for CPD completion. However, considering that the interviews show contradicting results on the matter and that there is little research examining the effectiveness of current regulatory practices (86), it would be beneficial to investigate the subject further. In particular, there would be value in exploring the point of view of other stakeholders (e.g., CPD providers, professional associations, healthcare service organisations, governmental agencies) in order to clarify expectations around the CPD regulatory functions of the regulators.

Finally, the third policy determinant is related to the conceptualization of the term “data” in policy documents and CPD requirements. Interview results show that medical regulators and certifying bodies give a broad meaning to the concept of “data” for performance assessment and QI purposes, regarding it as any piece of “information, especially facts and numbers, collected to be examined and considered and used to help decision-making” [87] rather than “information in an electronic form that can be stored and processed by a computer” [87].

Using the first definition as a reference, performance assessment “data” might thus include colleagues and patients’ questionnaire results from Multi-Source Feedback (MSF) tools [88,89] or reviews by peers and/or non-physicians observers from practice visits [90,91].

It is crucial to clarify though that the assessment outcomes of tools such as MSF and practice visits are based on the feedback of colleagues, patients, and/or external observers whereas the assessment outcomes of tools such as MMM, clinical audits, and QI projects are largely based on the retrospective and/or prospectiveanalysis of patient health data (in paper and/or electronic format)[15,92,93]. Despite this core difference, at present scholarly literature often presents the terms “data” and “feedback” as interchangeable [94] and does not provide an explicit and shared definition of “individual performance assessment data”.

Considering the lack of clarity and consensus on these terms, it would be advisable for medical bodies and academia to collaborate in the formulation of key concepts for both future policy development and research advancement.

Future policy prospects: regulators’ intentions, missed opportunities, and considerations on policy development and implementation

The overall results of the interviews provide discrepant evidence in terms of future policy development and implementation to foster the use of eHealth data for strengthened CPD.

Key informants had different opinions as to how to proceed in this regard, and their positions around the use of eHealth data for CPD compliance sat on a continuum that ranges from discretionary to mandatory – with the majority of participants agreeing that the regulatory and certifying bodies will not include nor prescribe eHealth data usage in policy documents and CPD requirements in the foreseeable future. Furthermore, considering that none of the participants explicitly mentioned policy development and implementation as one of the current or potential responsibilities of medical regulators, policy development prospects appear uncertain.

Nevertheless, the development of clear regulatory policies around the use of eHealth data for strengthened CPD could lead to a number of opportunities for the regulators, including: 1) contributing to the ongoing conversation on performance-based assessment strategies and data-driven CPD [95]; 2) taking an active role in encouraging digital health innovation in CPD practices; 3) clarifying the formative use of eHealth data and the educational value of its analysis so to address medical practitioners’ concerns around potential summative actions in case of poor performance outcomes [74,78]; 4) prompting other key stakeholders in the CPD ecosystem to engage with the current debate, consider their own role in fostering eHealth data for CPD purposes, and assume responsibilities to support implementation.

Given this, a call for medical regulators to develop policies and implementation strategies to foster the use of eHealth data for CPD may seem obvious. The jurisdictional variance in terms of regulatory processes and relevant stakeholders and the need for further research on key concepts, internal organisational factors, and interorganisational dynamics do not support the development of generalisable yet specific recommendations at this point in time. However, the following considerations could inform policy decision-making for all relevant medical bodies: 1) the importance of making policy intentions clear with regard to the regulators’ expectations of eHealth data usage and analysis, and 2) the necessity of including a precise definition for the term “data” and other key concepts in regulatory policies and relative supporting documents.

This is particularly relevant especially because multiple stakeholders with different expertise, interests, and objectives will be eventually involved in policy implementation. As an example, stakeholders in the digital health research space focus on the use of patient eHealth data and of digital technologies to improve clinical practice and healthcare [96,97]. The current ambiguity of what constitutes “data” in policy documentation might lead to miscommunication and misunderstandings with digital health experts, and therefore to misalignment in terms of research aims and outcomes, and expectations around deliverables and recommendations for future implementation.

Medical regulators and the fostering of eHealth data strengthened CPD: unfulfilled functions, areas of influence, and suggested role

In terms of the role of medical bodies in fostering the use of eHealth data for strengthened CPD, study findings reveal a current paradoxical situation. Even though key informants agreed that the main function of medical regulators is to address data accessibility and availability barriers through advocacy and lobbying, overall study results indicate that this remains largely unfulfilled. The document analysis, in fact, did not provide any evidence in this respect, and participants’ comments denote that advocacy and lobbying are regarded only as potential future functions for medical regulators.

In view of this and considering that regulators’ intentions and intended roles are often not clear, it is difficult to foresee future developments and provide the regulators with recommendations in this regard.

As a final consideration, overall results indicate that some of the barriers and challenges mentioned by the research participants go beyond the area of direct influence of medical regulatory bodies. For instance, the majority of issues regarding eHealth data collection, analysis, and reporting can be addressed only by data holders such health-care service organizations and government agencies. Also, the lack of data analysis and interpretation capabilities among the medical workforce can be tackled mainly by educational institutions and CPD providers through targeted education and training. Finally, all the enablers listed by the key informants (e.g., use of champions and larger use of AI) can be implemented exclusively at workplace level. Given this, it appears that more research is needed to define and clarify roles and responsibilities of all stakeholders involved. Also, it is evident that all stakeholders will need to be willing to collaborate and coordinate if the intent is that of fostering a cultural shift in CPD requirements, addressing contextual factors, and promoting digital health innovation in the CPD landscape. In view of this, medical regulatory bodies have the opportunity to take a leadership role in addressing the current lack of direction and guidance in this evolving space.

Strengths and limitations

The strengths of this study include its multimethod design, the comprehensiveness of the document analysis, and the richness and insightfulness of the interviews’ data.

Some of the limitations inherent to document analysis such as insufficient detail and biased selectivity [40] were mitigated by data triangulation with additional grey literature, relevant scholarly literature, and the results of the interviews.

Along with the strengths of this study and the mitigation strategies employed, there are some limitations. This study was restricted to 5 selected countries and related relevant medical bodies, therefore some of the results might not be valid for other jurisdictions. Despite the fact that key informants were purposively selected because of their role in policy development, their views may not necessarily be representative of the official position of medical bodies they work for.

This study fills a gap in the literature by showing that current medical regulations and

CPD requirements do not foster the use of eHealth data for CPD completion and compliance. Further research on key policy concepts, regulators’ internal organisational factors, and interorganisational relationships and dynamics within the CPD ecosystem is needed to formulate jurisdiction-specific recommendations for future policy development.

Results from this study also revealed that medical regulatory and certifying bodies are willing to assume advocacy and lobbying functions to address existing data availability and accessibility issues, but the majority does not foresee changes in policy documentation in order to include the explicit mention of eHealth data usage for CPD compliance.

This study finally shows that the alignment of all relevant stakeholders – including regulatory and certifying bodies, CPD providers, professional associations, healthcare service organisations, governmental agencies – is required to tackle existing barriers and challenges and promote digital health innovation in CPD practices. Moreover, it suggests that medical regulators could fill the current vacuum of leadership in the CPD ecosystem playing a more central role in the fostering of eHealth data usage for CPD completion and compliance.

ABMS American Board of Medical Specialties

ABO American Board of Ophthalmology

ABS American Board of Surgery

ACCME Accreditation Council for Continuing Medical Education

Ahpra Australian Health Practitioner Regulation Agency

CFPC College of Family Physicians of Canada

CME Continuing Medical Education

CMQ Collège des médecins du Québec

CPD Continuing Professional Development

CPSM College of Physicians and Surgeons of Manitoba

FMRAC Federation of Medical Regulatory Authorities of Canada

GMC General Medical Council

MBA Medical Board of Australia

MCNZ Medical Council of New Zealand

MFI Model for Improvement

MRA Medical Regulatory Authorities

Pii Practice Improvement Initiative

QI Quality Improvement

Royal College Royal College of Physicians and Surgeons of Canada

Acknowledgements

Not applicable.

Ethics approval and consent to participate

The study was approved by the University of Sydney Human Research Ethics Committee (Protocol Number 2020/722). All participants gave written informed consent to participate, and the study methods were carried out in accordance with relevant guidelines and regulations.

Availability of data and materials

All documentary data analysed in this study is publicly available on the internet. Deidentified interview transcripts are available from the corresponding author on reasonable request.

Consent for publication

All participants gave written informed consent for publication.

Funding

The research described in this study is supported by Digital Health Cooperative Research Centre Limited (“DHCRC”). DHCRC is funded under the Australian Government’s Cooperative Research Centres Program. Carol Pizzuti is undertaking an Industry PhD that is funded through the DHCRC.

Competing interests

Prof Tim Shaw is an Academic lead within the Digital Health CRC. Carol Pizzuti and Dr Cristiana Palmieri declare that they have no competing interests.

Authors' contributions

Carol Pizzuti conceptualized and designed the study, collected and analysed documentary data, conducted the semi-structured interviews, led the interpretation of both documentary and interviews’ data, and wrote the manuscript. Carol Pizzuti and Dr. Cristiana Palmieri analysed interviews’ data. All authors contributed to the critical revision of the paper. All authors have read and approved the manuscript.

Bloom BS. Effects of continuing medical education on improving physician clinical care and patient health: A review of systematic reviews. Int J Technol Assess Health Care. 2005;21(3):380–5.
Cervero RM, Gaines JK. The Impact of CME on Physician Performance and Patient Health Outcomes: An Updated Synthesis of Systematic Reviews. J Contin Educ Health Prof [Internet]. 2015 Mar 1 [cited 2020 Sep 28];35(2):131–8. Available from: http://journals.lww.com/00005141-201535020-00008
Veloski J, Boex JR, Grasberger MJ, Evans A, Wolfson DW. Systematic review of the literature on assessment, feedback and physicians’ clinical performance: BEME Guide No. 7. Med Teach. 2006;28(2):117–28.
Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: A systematic review. Jama. 2006;296(9):1094–102.
Lockyer J, Armson H, Chesluk B, Dornan T, Holmboe E, Loney E, et al. Feedback data sources that inform physician self-assessment. Med Teach [Internet]. 2011;33(2):e113-20. Available from: http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=med8&NEWS=N&AN=21275533
Kaye A, Okanlawon O, Urman R. Clinical performance feedback and quality improvement opportunities for perioperative physicians. Adv Med Educ Pract. 2014;115.
General Medical Council (GMC). Guidance for doctors: requirements for revalidation and maintaining your licence [Internet]. Available from: https://www.gmc-uk.org/-/media/documents/guidance---revalidation---revalidation-guidance-for-doctors_pdf-54232703.pdf. Accessed 12 December 2022.
Medical Board of Australia (MBA). Professional Performance Framework. 2017 [Internet]. Available from: https://www.medicalboard.gov.au/Professional-Performance-Framework.aspx. Accessed 30 November 2022.
Medical Council of New Zealand. Strengthened recertification requirements for vocationally registered doctors - November 2019 [Internet]. Available from: https://www.mcnz.org.nz/assets/Publications/Booklets/f7d4bc7fff/Strengthened-recertification-requirements-for-vocationally-registered-doctors-November-2019.pdf. Accessed 30 November 2022.
American Board of Medical Specialties (ABMS). Standards for Continuing Certification [Internet]. Available from: https://www.abms.org/wp-content/uploads/2021/11/ABMS-Standards-for-Continuing-Certification-20211029.pdf. Accessed 16 December 2022.
Federation of Medical Regulatory Authorities of Canada (FMRAC). Physician Practice Improvement (PPI) [Internet]. Available from: https://fmrac.ca/wp-content/uploads/2016/04/PPI-System_ENG.pdf. Accessed 13 December 2022.
Sargeant J, Bruce D, Campbell C. Practicing Physicians’ Needs for Assessment and Feedback as Part of Professional Development. J Contin Educ Health Prof. 2013;33(1):S54–S62.
Ullman AJ, Ray-Barruel G, Rickard CM, Cooke M. Clinical audits to improve critical care: Part 1 Prepare and collect data. Aust Crit Care [Internet]. 2018;31(2):101–5. Available from: http://dx.doi.org/10.1016/j.aucc.2017.04.003
Joseph CW, Garrubba ML, Melder AM. Informing best practice for conducting morbidity and mortality reviews: A literature review. Aust Heal Rev. 2018;42(3):248–57.
Shah A. Using data for improvement. BMJ [Internet]. 2019;364(February):1–6. Available from: http://dx.doi.org/doi:10.1136/bmj.l189
Clynch N, Kellett J. Medical documentation: Part of the solution, or part of the problem? A narrative review of the literature on the time spent on and value of medical documentation. Int J Med Inform [Internet]. 2015;84(4):221–8. Available from: http://dx.doi.org/10.1016/j.ijmedinf.2014.12.001
Van der Bulk S. Automated audit and feedback as a tool to measure and improve the quality of primary care [Internet]. Katholieke Universiteit Leuven. Faculty of Medicine; 2021. Available from: https://lirias.kuleuven.be/3442011?limo=0
Rumball-Smith J, Ross K, Bates DW. Late adopters of the electronic health record should move now. BMJ Qual Saf. 2020;29(3):238–40.
McVey L, Alvarado N, Greenhalgh J, Elshehaly M, Gale CP, Lake J, et al. Hidden labour: the skilful work of clinical audit data collection and its implications for secondary use of data via integrated health IT. BMC Health Serv Res [Internet]. 2021;21(1):702. Available from: https://doi.org/10.1186/s12913-021-06657-0
McVey L, Alvarado N, Keen J, Greenhalgh J, Mamas M, Gale C, et al. Institutional use of National Clinical Audits by healthcare providers. J Eval Clin Pract [Internet]. 2020; Available from: http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=medp&NEWS=N&AN=32307857
Tolf S, Mesterton J, Söderberg D, Amer-Wåhlin I, Mazzocato P. How can technology support quality improvement? Lessons learned from the adoption of an analytics tool for advanced performance measurement in a hospital unit. BMC Health Serv Res. 2020;20(1):1–12.
Raymond L, Paré G, Ortiz De Guinea A, Poba-Nzaou P, Trudel MC, Marsan J, et al. Improving performance in medical practices through the extended use of electronic medical record systems: A survey of Canadian family physicians Healthcare Information Systems. BMC Med Inform Decis Mak. 2015;15(1):1–15.
Sanaei A, Sepehri MM. Quality Improvement through Intensive Care Registries: The Value of Big Data in Clinical Decision Making. Arch Anesth Crit Care. 2020;6(11):32–9.
Otokiti A. Using informatics to improve healthcare quality. Int J Health Care Qual Assur. 2019;32(2):425–30.
Janssen A, Talic S, Gasevic D, Kay J, Shaw T. Exploring the Intersection between Health Professionals’ Learning and eHealth Data: Protocol for a Comprehensive Research Program in Practice Analytics in Health Care. JMIR Res Protoc. 2021;10(12).
Janssen A, Kay J, Talic S, Pusic M, Birnbaum RJ, Cavalcanti R, et al. Electronic Health Records That Support Health Professional Reflective Practice: a Missed Opportunity in Digital Health. J Healthc Informatics Res [Internet]. 2023; Available from: https://doi.org/10.1007/s41666-022-00123-0
Bucalon B, Shaw T, Brown K, Kay J. State-of-the-art Dashboards on Clinical Indicator Data to Support Reflection on Practice: Scoping Review. JMIR Med Informatics. 2022;10(2):1–24.
Whitelock-Wainwright E, Koh JW, Whitelock-Wainwright A, Talic S, Rankin D, Gašević D. An exploration into physician and surgeon data sensemaking: a qualitative systematic review using thematic synthesis. BMC Med Inform Decis Mak [Internet]. 2022;22(1):1–23. Available from: https://doi.org/10.1186/s12911-022-01997-1
Bucalon B, Whitelock-wainwright E, Williams C, Conley J, Veysey M, Kay J, et al. Thought Leader Perspectives on the Benefits, Barriers, and Enablers for Routinely Collected Electronic Health Data to Support Professional Development: Qualitative Study. J Med Internet Res. 2023;25(e40685):1–11.
Pusic M V., Birnbaum RJ, Thoma B, Hamstra SJ, Cavalcanti RB, Warm EJ, et al. Frameworks for Integrating Learning Analytics with the Electronic Health Record. J Contin Educ Health Prof. 2023;43(1):52–9.
Schäfke-Zell W. Revisiting the definition of health data in the age of digitalized health care. Int Data Priv Law [Internet]. 2022 Feb 1;12(1):33–43. Available from: https://doi.org/10.1093/idpl/ipab025
Hovenga E, Grain H. Health data and data governance. In: Health Information Governance in a Digital Environment. 2013. p. 384.
Purtova N. Health Data for Common Good: Defining the Boundaries and Social Dilemmas of Data Commons. In: Adams S, Purtova N, Leenes R, editors. Under Observation: The Interplay Between eHealth and Surveillance. Springer; 2017. p. 177–210.
Heijlen R, Crompvoets J. Open health data: Mapping the ecosystem. Digit Heal. 2021;7.
International Association of Medical Regulatory Authorities (IAMRA). What is a medical regulatory authority? [Internet]. Available from: https://www.iamra.com/What-is-a-Medical-Regulatory-Authority. Accessed 21 December 2022.
Leslie K, Moore J, Robertson C, Bilton D, Hirschkorn K, Langelier MH, et al. Regulating health professional scopes of practice: comparing institutional arrangements and approaches in the US, Canada, Australia and the UK. Hum Resour Health. 2021 Dec 1;19(15).
Filipe HP, Silva ED, Stulting AA, Golnik KC. Continuing professional development: Best practices. Middle East Afr J Ophthalmol [Internet]. 2014;21(2):134–41. Available from: /pmc/articles/PMC4005177/?report=abstract
Pickles R. Revalidation or Recertification: What Does It All Mean? Arch Med Heal Sci. 2018;6(1).
Shaw S, Elston J, Abbott S. Comparative analysis of health policy implementation. The use of documentary analysis. Policy Stud. 2004;25(4):259–66.
Bowen GA. Document analysis as a qualitative research method. Qual Res J. 2009;9(2):27–40.
Gross JMS. Document analysis. In: Frey BB, editor. The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation. Thousand Oaks, CA: SAGE Publications, Inc.; 2018. p. 545–8.
Dalglish SL, Khalid H, McMahon SA. Document analysis in health policy research: The READ approach. Health Policy Plan. 2020;35(10):1424–31.
Drisko J, Maschi T. Basic Content Analysis. In: Content Analysis. 2nd editio. Oxford: Oxford University Press, Incorporated; 2016. p. 21–56.
Swain J. A Hybrid Approach to Thematic Analysis in Qualitative Research: Using a Practical Example. A Hybrid Approach to Themat Anal Qual Res Using a Pract Ex. 2018;(October).
Fereday J, Muir-Cochrane E. Demonstrating Rigor Using Thematic Analysis: A Hybrid Approach of Inductive and Deductive Coding and Theme Development. Int J Qual Methods. 2006;5(1):80–92.
Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. Int J Qual Heal Care. 2007;19(6):349–57.
Australian Health Practitioner Regulation Agency AHPRA. Available from: www.ahpra.gov.au. Accessed 6 December 2022.
Medical Board of Australia. Medical Specialties and Specialty Fields. Available from: https://www.medicalboard.gov.au/registration/types/specialist-registration/medical-specialties-and-specialty-fields.aspx. Accessed 6 December 2022.
Queensland Health. Minister for Health and Ambulance Services. Health Practitioner Regulation National Law Act 2009. Available from: www.legislation.qld.gov.au/view/html/inforce/current/act-2009-045. Accessed 7 December 2022.
Medical Board of Australia. Registration. Available from: www.medicalboard.gov.au/Registration.aspx. Accessed 6 December 2022.
Medical Board of Australia. Final Report of the Expert Advisory Group on revalidation. 2017. Available from: https://www.medicalboard.gov.au/documents/default.aspx?record=WD17%2F24295&dbid=AP&chksum=Txmn8C7v%2BC53Wjsz3sXn2w%3D%3D. Accessed 7 December 2022.
Medical Board of Australia. Building a Professional Performance Framework. 2018. Available from: https://www.medicalboard.gov.au/documents/default.aspx?record=WD17%2F24293&dbid=AP&chksum=GO%2B6DZkJeoSSVVg%2FxcDoMQ%3D%3D. Accessed 7 December 2022.
Federation of Medical Regulatory Authorities of Canada. The Members of FMRAC. Available from: https://fmrac.ca/members/. Accessed 14 December 2022.
Medical Council of Canada (MCC). Route to licensure. Available from: https://mcc.ca/about/route-to-licensure/?cn-reloaded=1. Accessed 16 December 2022.
The College of Family Physicians of Canada (CFPC). Practice Improvement Initiative (Pii). Available from: https://www.cfpc.ca/en/policy-innovation/innovation-in-research-and-quality-improvement/practice-improvement-initiative. Accessed 13 December 2022.
The Royal College of Physicians and Surgeons of Canada. Essential Guidance for Quality Improvement. Available from: https://www.royalcollege.ca/rcsite/cpd/essential-guidance-for-quality-improvement-e#one. Accessed 13 December 2022.
New Zealand Government. Ministry of Health. Health Practitioners Competence Assurance Act. Available from: https://www.health.govt.nz/our-work/regulation-health-and-disability-system/health-practitioners-competence-assurance-act. Accessed 6 December 2022.
Medical Council of New Zealand (MCNZ). Policy on recertification programmes for doctors. 1 August 2021. Available from: https://www.mcnz.org.nz/assets/Policies/dc6f03cda4/Policy-on-recertification.pdf. Accessed 6 December 2022.
Medical Council of New Zealand (MCNZ). Recertification and professional development. Available from: https://www.mcnz.org.nz/registration/maintain-or-renew-registration/recertification-and-professional-development/. Accessed 6 December 2022.
Medical Council of New Zealand (MCNZ). Recertification requirements for vocationally-registered doctors in New Zealand. November 2019. Available from: https://www.mcnz.org.nz/assets/Publications/Booklets/f7d4bc7fff/Strengthened-recertification-requirements-for-vocationally-registered-doctors-November-2019.pdf. Accessed 6 December 2022.
The National Archives. Medical Act 1983. Available from: https://www.legislation.gov.uk/ukpga/1983/54/contents. Accessed 12 December 2022.
General Medical Council (GMC). Revalidation. Available from: https://www.gmc-uk.org/registration-and-licensing/managing-your-registration/revalidation. Accessed 12 December 2022.
General Medical Council (GMC). Good medical practice. 2013. Available from: https://www.gmc-uk.org/-/media/documents/good-medical-practice---english-20200128_pdf-51527435.pdf. Accessed 12 December 2022.
General Medical Council (GMC). Continuing professional development. Guidance for all doctors. 2012. Available from: https://www.gmc-uk.org/-/media/documents/cpd-guidance-for-all-doctors-0316_pdf-56438625.pdf Accessed 12 December 2022.
American Board of Medical Specialties (ABMS). Board Certification. Available from: https://www.abms.org/board-certification/board-certification-requirements/#cc. Accessed 16 December 2022.
Browne J, Bullock A, Poletti C, Cserző D. Recent research into healthcare professions regulation: a rapid evidence assessment. BMC Health Serv Res. 2021;21(1):1–12.
Main PAE, Anderson S. Evidence for continuing professional development standards for regulated health practitioners in Australia: a systematic review. Hum Resour Health [Internet]. 2023;21(1):1–16. Available from: https://doi.org/10.1186/s12960-023-00803-x
Greenhalgh J, Dalkin S, Gibbons E, Wright J, Valderas JM, Meads D, et al. How do aggregated patient-reported outcome measures data stimulate health care improvement? A realist synthesis. J Health Serv Res Policy [Internet]. 2018;23(1):57–65. Available from: http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=medc&NEWS=N&AN=29260592
Gude WT, Brown B, Van Der Veer SN, Colquhoun HL, Ivers NM, Brehaut JC, et al. Clinical performance comparators in audit and feedback: A review of theory and evidence. Implement Sci. 2019;14(1):1–14.
Phillips RL, Peterson L, Palen TE, Fields SA, Parchman ML, Johannides J. Clinical quality measure exchange is not easy. Ann Fam Med. 2021;19(3):207–11.
Hertzum M, Ellingsen G, Cajander Å. Implementing Large-Scale Electronic Health Records: Experiences from implementations of Epic in Denmark and Finland. Int J Med Inform [Internet]. 2022;167:104868. Available from: https://www.sciencedirect.com/science/article/pii/S1386505622001824
Verma AA, Pasricha S V., Jung HY, Kushnir V, Mak DYF, Koppula R, et al. Assessing the quality of clinical and administrative data extracted from hospitals: The General Medicine Inpatient Initiative (GEMINI) experience. J Am Med Informatics Assoc. 2021;28(3):578–87.
Zozus MN, Pieper C, Johnson CM, Johnson TR, Franklin A, Smith J, et al. Factors affecting accuracy of data abstracted from medical records. PLoS One. 2015;10(10):1–20.
Wiljer D, Tavares W, Charow R, Williams S, Campbell C, Davis D, et al. A Qualitative Study to Understand the Cultural Factors That Influence Clinical Data Use for Continuing Professional Development. J Contin Educ Health Prof. 2022;Publish Ah(00):1–10.
Wiljer D, Williams S, Charow R, Campbell C, Davis D, Younus S, et al. Exploring Systemic Influences on Data-Informed Learning: Document Review of Policies, Procedures, and Legislation from Canada and the United States. J Contin Educ Health Prof. 2022;42(3):219–23.
Bichel-Findlay J, Koch S, Mantas J, Abdul SS, Al-Shorbaji N, Ammenwerth E, et al. Recommendations of the International Medical Informatics Association (IMIA) on Education in Biomedical and Health Informatics: Second Revision. Int J Med Inform [Internet]. 2023;170:104908. Available from: https://doi.org/10.1016/j.ijmedinf.2022.104908
Samadbeik M, Fatehi F, Braunstein M, Barry B, Saremian M, Kalhor F, et al. Education and Training on Electronic Medical Records (EMRs) for health care professionals and students: A Scoping Review. Int J Med Inform [Internet]. 2020;142(August):104238. Available from: https://doi.org/10.1016/j.ijmedinf.2020.104238
Shaw T, Janssen A, Crampton R, O’Leary F, Hoyle P, Jones A, et al. Attitudes of health professionals to using routinely collected clinical data for performance feedback and personalised professional development. Med J Aust [Internet]. 2019;210(S6):S17–21. Available from: http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=medl&NEWS=N&AN=30927464
Sockalingam S, Tavares W, Charow R, Youssef A, Campbell C, Davis D, et al. Examining Associations between Physician Data Utilization for Practice Improvement and Lifelong Learning. J Contin Educ Health Prof. 2019;39(4):236–42.
Reilly JB, Kim JG, Cooney R, DeWaters AL, Holmboe ES, Mazotti L, et al. Breaking Down Silos Between Medical Education and Health Systems: Creating an Integrated Multilevel Data Model to Advance the Systems-Based Practice Competency. Acad Med. 2023;
Chan T, Sebok-Syer S, Thoma B, Wise A, Sherbino J, Pusic M. Learning Analytics in Medical Education Assessment: The Past, the Present, and the Future. AEM Educ Train [Internet]. 2018 [cited 2020 Sep 23];2(2):178–87. Available from: www.aem-e-t.com
Holmboe ES, Yamazaki K, Nasca TJ, Hamstra SJ. Using Longitudinal Milestones Data and Learning Analytics to Facilitate the Professional Development of Residents: Early Lessons from Three Specialties. Acad Med. 2020;95(1):97–103.
Smirnova A, Sebok-Syer SS, Chahine S, Kalet AL, Tamblyn R, Lombarts KMJMH, et al. Defining and adopting clinical performance measures in graduate medical education: Where are we now and where are we going? Acad Med. 2019;94(5):671–7.
Ellaway RH, Pusic M V., Galbraith RM, Cameron T. Developing the role of big data and analytics in health professional education. Med Teach. 2014;36(3):216–22.
Thoma B, Warm E, Hamstra SJ, Cavalcanti R, Pusic M, Shaw T, et al. Next Steps in the Implementation of Learning Analytics in Medical Education: Consensus From an International Cohort of Medical Educators. J Grad Med Educ. 2020;12(3):303–11.
Grant J. Introduction: What is managed continuing professional development? In: The Good CPD Guide A Practical Guide to Managed Continuing Professional Development in Medicine. 2nd ed. Taylor & Francis Group; 2012. p. 1–11.
Cambridge Dictionary. Data. Available from: https://dictionary.cambridge.org/dictionary/english/data. Accessed 3 February 2023.
Lockyer J. Multisource feedback in the assessment of physician competencies. J Contin Educ Health Prof. 2003;23(1):4–12.
Pooley M, Pizzuti C, Daly M. Optimizing Multisource Feedback Implementation for Australasian Physicians. J Contin Educ Health Prof. 2019;39(4).
Wallis KA. Regular practice review: promised joy or naught but grief and pain? J Prim Care Community Health. 2014;6(2):165–8.
Malatesta International. Final evaluation report. Evaluation of the Regular Practice Review Programme. 2019.
Xiong X, Johnson T, Jayaraman D, Mcdonald EG, Martel M, Barkun AN. At the Crossroad with Morbidity and Mortality Conferences: Lessons Learned through a Narrative Systematic Review. Can J Gastroenterol Hepatol. 2016;2016.
Jamtvedt G, Flottorp S, Ivers N. Audit and Feedback as a Quality Strategy. In: Busse R, Klazinga N, Panteli D, Quentin W, editors. Improving healthcare quality in Europe Characteristics, effectiveness and implementation of different strategies. Health Pol. Copenhagen: European Observatory on Health Systems and Policies; 2019. p. 265–85.
Ajjawi R, Regehr G. When I say … feedback. Med Educ [Internet]. 2018;53(7):652–4. Available from: https://www.researchgate.net/publication/330598819
Kitto S. The Untapped Potential of Data-Driven CPD and the Learning Sciences. J Contin Educ Health Prof. 2023;43(1):1–2.
Cristina P. A digital (r)evolution: introducing The Lancet Digital Health. Lancet Digit Heal [Internet]. 2019;1(1):e1. Available from: http://dx.doi.org/10.1016/S2589-7500(19)30010-X
Jandoo T. WHO guidance for digital health: What it means for researchers. Digit Heal. 2020;6:1–4.

Table 1. List of relevant bodies and corresponding documents for analysis

Country	Body	Document	Type	Link
Australia	MBA	Registration	Webpage	https://www.medicalboard.gov.au/Registration.aspx
Australia	MBA	Registration standard: Continuing professional development	PDF	https://www.ahpra.gov.au/documents/default.aspx?record=WD21%2f31046&dbid=AP&chksum=TqPI98CYQYllvPkGwiAz%2fw%3d%3d
Canada	CFPC	Mainpro+ Manual	PDF	https://www.cfpc.ca/en/Resources/Continuing-Professional-Development/Mainpro-Manual
	Royal College	The Maintenance of Certification Program	Webpage	https://www.royalcollege.ca/rcsite/cpd/maintenance-of-certification-program-e
		A Concise Guide to Maintenance of Certification	Webpage	https://www.royalcollege.ca/rcsite/cpd/moc-program/concise-guide-maintenance-certification-e
		Framework of Continuing Professional Development Activities	Webpage	https://www.royalcollege.ca/rcsite/cpd/moc-program/moc-framework-e
		Annual Performance Review and Feedback	Webpage	https://www.royalcollege.ca/rcsite/cpd/accreditation/accredited-activities/section-3-annual-performance-review-e
		Chart Audit and Feedback	Webpage	https://www.royalcollege.ca/rcsite/cpd/accreditation/accredited-activities/section-3-chart-audit-and-feedback-e
	CMQ	Règlement sur la formation continue obligatoire des médecins	PDF	https://www.legisquebec.gouv.qc.ca/fr/pdf/rc/M-9,%20R.%2022.1.pdf?t=1669607018480
	CMQ	Les obligations des médecins en matière de formation continue	PDF	http://www.cmq.org/publications-pdf/p-1-2018-11-29-fr-obligations-medecins-formation-continue.pdf
New Zealand	MCNZ	Recertification and professional development	Webpage	https://www.mcnz.org.nz/registration/maintain-or-renew-registration/recertification-and-professional-development/
		Policy on recertification programmes for doctors – August 2021	PDF	https://www.mcnz.org.nz/assets/Policies/dc6f03cda4/Policy-on-recertification.pdf
		Recertification requirements for vocationally-registered doctors in New Zealand – November 2019	PDF	https://www.mcnz.org.nz/assets/Publications/Booklets/f7d4bc7fff/Strengthened-recertification-requirements-for-vocationally-registered-doctors-November-2019.pdf
UK	GMC	Guidance for doctors: requirements for revalidation and maintaining your licence	PDF	https://www.gmc-uk.org/-/media/documents/guidance---revalidation---revalidation-guidance-for-doctors_pdf-54232703.pdf
		Guidance on supporting information for appraisal and revalidation	PDF	https://www.gmc-uk.org/-/media/documents/rt---supporting-information-for-appraisal-and-revalidation---dc5485_pdf-55024594.pdf
		Continuing professional Development. Guidance for all doctors	PDF	https://www.gmc-uk.org/-/media/documents/cpd-guidance-for-all-doctors-0316_pdf-56438625.pdf
USA	ABMS	Board Certification Standards	Webpage	https://www.abms.org/board-certification/board-certification-standards/
		Board Certification Requirements	Webpage	https://www.abms.org/board-certification/board-certification-requirements/#cc
		Standards for Continuing Certification	PDF	https://www.abms.org/wp-content/uploads/2021/11/ABMS-Standards-for-Continuing-Certification-20211029.pdf

Table 2. List of the research participating bodies and number of research participants per country

Country	Body	Type of Body	Number of Participants
Australia	Australian Health Practitioner Regulation Agency (Ahpra)	Regulatory Body	5
Australia	Medical Board of Australia (MBA)	Regulatory Body	5
Canada	Federation of Medical Regulatory Authorities of Canada (FMRAC)	Regulatory Authority	5
	College of Physicians and Surgeons of Manitoba (CPSM)	Regulatory Authority
	The Royal College of Physicians and Surgeons of Canada (Royal College)	Certifying College
	The College of Family Physicians of Canada (CFPC)	Certifying College
	College du medicine du Quebec (CMQ)	Licensing and Certifying College
New Zealand	Medical Council of New Zealand (MCNZ)	Regulatory Body	2
UK	General Medical Council (GMC)	Regulatory Body	2
USA	American Board of Medical Specialties (ABMS)	Certifying Board	6
	American Board of Surgery (ABS)	Certifying Board
	American Board of Ophthalmology (ABO)	Certifying Board
	Accreditation Council for Continuing Medical Education (ACCME)	Non-profit Regulatory Corporation

Table 3. Document analysis: data extraction results

Country	AU	CAN	CAN	CAN	NZ	UK	USA
Type of Body	Medical Regulatory Body	Certifying College	Certifying College	Licensing and Certifying College	Medical Regulatory Body	Medical Regulatory Body	Medical Regulatory Body
Name of Body	Medical Board of Australia (MBA)/ Australian Health Practitioner Regulation Agency (Ahpra)	The College of Family Physicians of Canada (CFPC)	The Royal College of Physicians and Surgeons of Canada (Royal College)	Collège des médecins du Québec (CMQ)	Medical Council of New Zealand (MCNZ)	General Medical Council (GMC)	American Board of Medical Specialties (ABMS)
Medical regulatory processes	Registration Standards	Maintenance of Certification	Maintenance of Certification	Maintenance of Certification	Recertification Programme	Revalidation	Continuing Certification
CPD requirements	CPD Registration Standards	Mainpro+® (Maintenance of Proficiency) Program	Maintenance of Certification (MOC) Program	Formation continue obligatoire des Médecins	CPD Requirements	CPD Requirements	Standards for Continuing Certification
Effective	From 1 January 2023	From 27 June 2016	From 1 Jan 2014	From 1 Jan 2019	From 1 July 2022	From June 2012	From 1 Jan 2024
CPD categories	3 Categories: 1. Educational activities 2. Reviewing performance activities 3. Measuring Outcomes activities .	3 Categories: 1. Group learning 2. Self-learning 3. Assessment activities Within those categories, activities can be either certified or non-certified.	3 Sections: 1. Group Learning 2. Self-Learning 3. Assessment	3 Activities: 1. Activités de développement professionnel (DP) reconnues 2. Activités d'évaluation de l’exercice (EE) reconnues 3. Activités de formation continue non reconnues mais admissibles	3 categories: 1. Reviewing and reflecting on practice 2. Measuring and improving outcomes 3. Educational activities (continuing medical education – CME)	No prescription of a specific number of CPD credits or hours.	3 elements: 1. Professional Standing and Conduct 2. Lifelong Learning 3. Improving Health and Health Care
Health data use and CPD categories	Not Applicable, as the MBA CPD Registration Standards do not provide descriptions/definitions for CPD Categories.	Category 2: Self-assessment Category 3: Assessment	Section 3: Assessment	Activité 2: Évaluation de l’exercice	Category 2. Measuring and improving outcomes	Not Applicable, as the GMC does not set CPD categories.	Element 3: Improving Health and Health Care
Health data use and CPD activities	Not Applicable, as the MBA CPD Registration Standards do not provide a list of activities for each Category.	‘Linking Learning to Practice” activity (Category 2) Practice audits Quality assurance programs Provincial practice review and enhancement programs (Category 3)	Chart audit and feedback Annual performance review Practice assessments	Activités d’évaluation de l’acte médical	“Quality improvement process that includes review (internal or external) of a doctor’s everyday work and resultant patient/health outcomes.”¹ (p. 6)	Audits Quality improvement activity (Revalidation requirement)	Quality improvement activities
Mention of eHealth data	eHealth data not mentioned	eHealth data not mentioned	Electronic Health Records (EHRs) as a potential data source	Electronic Medical Records (EMRs) as a potential data source	eHealth data not mentioned	eHealth data not mentioned	eHealth data not mentioned

¹ Medical Council of New Zealand (MCNZ). Recertification requirements for vocationally-registered doctors in New Zealand. November 2019. Available at: https://www.mcnz.org.nz/assets/Publications/Booklets/f7d4bc7fff/Strengthened-recertification-requirements-for-vocationally-registered-doctors-November-2019.pdf. Accessed December 6, 2022

No competing interests reported.

Additionalfile1.docx

Download PDF

Reviewers agreed at journal
08 Feb, 2024
Reviews received at journal
30 Jan, 2024
Reviewers agreed at journal
30 Jan, 2024
Reviewers invited by journal
13 Jul, 2023
Editor assigned by journal
13 Jul, 2023
Editor invited by journal
04 Jul, 2023
Submission checks completed at journal
04 Jul, 2023
First submitted to journal
20 Jun, 2023

You are reading this latest preprint version

The role of medical regulations and medical regulators in fostering the use of eHealth data for strengthened Continuing Professional Development (CPD). A document analysis with key informants’ interviews

Status:

Version 1

Abstract

Background

Definition of terms

Health data and eHealth data

Regulatory and certifying bodies

Continuing Professional Development (CPD) requirements

Methods

Key Informants’ Interviews

Results

Document Analysis Results

A detailed account of data extraction results is provided in Table 3. [Insert Table 3]

Australia

Canada

New Zealand

UK

USA

Key Informants’ Interviews Results

Theme 2: eHealth data and policy

Notably, some participants also shared their insights on future policy directions, clearly stating whether eHealth data usage will be made prescriptive or not:

Discussion

Current policy choices: considerations on policy rationale and recommendations for further research

Future policy prospects: regulators’ intentions, missed opportunities, and considerations on policy development and implementation

Medical regulators and the fostering of eHealth data strengthened CPD: unfulfilled functions, areas of influence, and suggested role

Strengths and limitations

Conclusion

This study fills a gap in the literature by showing that current medical regulations and

Abbreviations

Declarations

Ethics approval and consent to participate

References

Tables

Additional Declarations

Supplementary Files

Status:

Version 1