Analysis of accreditation standards for undergraduate medical studies in Serbia through the lens of the National Health Workforce Accounts

doi:10.21203/rs.3.rs-64019/v2

Download PDF

Research

Analysis of accreditation standards for undergraduate medical studies in Serbia through the lens of the National Health Workforce Accounts

https://doi.org/10.21203/rs.3.rs-64019/v2

This work is licensed under a CC BY 4.0 License

Version 2

posted

You are reading this older preprint version

Read the latest preprint version →

Background

Understanding the importance of educational accreditation standards for health and health workforce policymaking is needed more than ever given the growing circulation of physicians, cross-border care and population needs. The World Health Organization National Health Workforce Accounts (WHO/NHWA) for education and training could support the achievement of health care quality through health workforce strategies.

Objective

To assess the effect of the accreditation standards on the quality of undergraduate medical studies and to assess their potential to support quality improvement in health workforce education and health policy planning.

Methods

Study design is based on a multimodal approach revealing the qualitative (review of official documents, two internal reports on accreditation, literature) and quantitative data (descriptive analysis, 2010-2018) as resources for evaluation of the national accreditation standards, and mapping with WHO/NHWA indicators on education and training for medical undergraduate studies at the Faculty of Medicine University of Belgrade, Serbia (FMUB).

Results

The mechanism of compulsory accreditation implies standards for the quality of education (except for inter-professional education). The standards partially cover social determinants and social accountability. Students participate optionally in the adoption of regulations. Like health managers who are also providers of student training during their studies, students participate in FMUB accreditation surveys. Based on the reports from two accreditation cycles, the satisfaction of graduates and employers with the quality of undergraduate medical studies has been partially improved. The downward trend in freshman enrollment has been observed for last nine years. The average duration of a six-year undergraduate study of medicine was extended to 7.2 ± 0.4 years.

Conclusion

Social accountability and social determinants must be taken as priority standards in the future revisions of accreditation mechanism in order to improve communication, organizational skills and independent work of medical students. Health workforce organizations, civil society and the community should participate in regulatory bodies for accreditation to establish the basis for socially accountable and inter-professional education. If applied at the institutional, local and national levels, the WHO/NHWA indicator system can support the alignment of higher education standards and plans with the standards and strategy/plan of the health care and health workforce.

Health Economics & Outcomes Research

Health Policy

accreditation

standards

quality

education

training

medical studies

planning

health workforce

health policy

National Health Workforce Accounts

Accreditation of medical institution, education and training is an important mechanism for evaluating the quality of education, but also for adjusting the learning outcomes to the potential global and local health needs. By guaranteeing that an accredited medical institution and studies are providing conditions for the production of competent medical workers, accreditation standards can assist in ensuring the quality of health care services as well as in a strategic transformation towards sustainable development of health equity in a society^1-3. It is especially important these standards strengthen in times of crisis, in order to reverse negative trends and avoid deepening inequities in education and development. The pandemic COVID-19 is having an adverse impact on more than 90 per cent of the world’s student population⁴. It is also an opportunity for wise and visionary leaders to create such an accreditation mechanism that will enable the continuance of progress in medical education and training as well as the maintenance of study programs under equal reach for all students in urban and remote areas.

The broader concept defines accreditation as a special form of quality assurance that leads to the formal approval of an institution or program, for which a legitimate body has established that it meets predetermined and agreed-upon standards, ultimately resulting in an accredited status with the distinctive characteristic granted to that provider or program by responsible⁵. Standards of accreditation for medical studies and /or education institutions operates in a particular governmental and regulatory context, which holds main policy stakeholders for health and health workforce development accountable for matching their interests with the society needs, and lays the foundation for medical education and training and socially accountable production of health workers competent to the purpose and practice of health care in the country.

In this respect, transformation of accreditation results including competencies of medical students into the expected quality of health care is a critical strategic issue^6,7. Current conceptualization of the quality of healthcare performance refers to the achievement of the Universal Health Coverage (UHC)⁸ and health related milestones of the Sustainable Development Goals (SDG)⁹. The World Health Organization (WHO) has developed National Health Workforce Accounts (NHWA) with aim to support health policy performance towards achieving the quality of health care through health workforce strategies with indicators for tracking human resources for health¹⁰. WHO/NHWA is a system of 78 core indicators primarily based on the health labour market framework with tangible directions for countries health workforce policies including education and training regulation and accreditation¹⁰. This study follows the idea that accreditation standards for health workforce education as WHO/NHWA indicators are valuable for health workforce policymaking.

Accreditation of education and training was in the research focus for a few decades. Many scientists believe clear conceptualizations and definitions allow the accreditation mechanism to follow healthcare standards and the interests of the public and patients in medical education and training^11-13.A several systematic reviews highlighted the importance of measuring competencies within the accreditation of medical education^14-17. For some scholars achieving excellence in medical education requires continuous improvement of the accreditation focus according to the standards of social responsibility and communication with society^18-22. Understanding the importance of accreditation standards for the planning and policymaking for health and health workforce is needed more than ever given the growing circulation of physicians and cross-border care^23-25and population needs,for which central support could be an analysis of accreditation standards as indicators of WHO/NHWA. Despite the highly recognized need for better coordination of education and health workforce policies for better health outcomes,^{6,7,10,16,18,24,26,27} knowledge of the effects of the accreditation of medical education and training on community health remains insufficient.^28-30Assessing the outcomes and impact of health programs on community health is a challenge^31-37and is more often done indirectly through studies related to the accreditation of health facilities^38,39, or employee training and competencies and institutional outcomes^40-43. Studies showing that greater access to education can lead to better health of the population^28,29,40,41speak of the integrated effect of education, employment and management of health personnel, rather than the isolated effect of accredited training programs for health workers. Given the fact that WHO/NHWA indicators are specific to the accreditation and regulation of education of health workers, as well as to the formulation of the health workforce and health policies, we decided to include these WHO/NHWA indicators in the study.

This study objective was to assess the effect of accreditation standards on the quality of undergraduate medical studies and to assess their potential to support quality improvement in health workforce education and health policy planning.

The study is conducted at the Faculty of Medicine of the University of Belgrade (FMUB) in Serbia, a middle-income country in the Europe Region in 2019.⁴² Our study design is based on a multimodal approach revealing the qualitative (a review of official documents, two internal reports on accreditation, literature^42-55) and quantitative data (descriptive analysis of FMUB statistics, 2010-2018) as resources for evaluation of the national accreditation standards, and mapping with WHO/NHWA indicators on education and training¹⁰(Additional file 1).

Mandatory accreditation for higher education has been introduced in Serbia in 2005, by the Law on Higher Education. Accreditation standards have been established mainly using the European higher education area experiences and directives⁵³ and in collaboration with European Association for Quality Assurance in Higher Education. Since then, FMUB has passed two accreditation cycles in 2009 and 2014^45-48, and has started new one in 2019^49-52. In every accreditation cycle, it is mandatory to create Self-Evaluation Report entailing surveys on quality of the program among students and employers (Additional file 2). Surveys' results, presented in this study since directly relate to the objectives of this study, have been conducted for the purpose of creating such reports. In 2013, the voluntary and anonymous survey response was collected from 58 graduates and 41 employers and in 2016 from 75 graduates and 36 employers. This study requires no ethical approval because we used secondary data and routinely recorded FMUB statistics.

Study variables

In total, 14 sets of standards for accreditation cycles in 2013 and 2016 were reviewed including the criteria on Quality Assurance Strategy, Standards and procedures for quality assurance, Quality Assurance System, Quality of the study program, Quality of the teaching process, Quality of scientific research and professional work, Quality of teachers and associates, Quality of students, Quality of textbooks, literature, library and information resources, Quality of higher education institution’ management and quality of non-teaching support, Quality of space and equipment, Financing, The role of students in self-evaluation and quality control and Systematic monitoring and periodic quality control studies⁴⁵.

The survey among graduates in 2013 and 2016 included 17 questions with the responses on a 5-point scale (1-very bad, 2-bad, 3-good, 4-very good and 5-excellent) such as satisfaction with the study program, quality of the content of the study program, of the teaching process and general conditions of study, level of acquired knowledge and skills for completing work assignments, about the subjects, the distribution of student workloads, the compliance of the curriculum with the needs of students, the teachers attitudes, discrimination of students at the FMUB, and to name a few suggestions for improving the quality of studies.

The survey of employers covered questions about young doctors who graduated at FMUB including their preparation to complete assignments in a healthcare facility, knowledge, ability to solve practical problems, readiness for independence in work, level of organizational knowledge, work habits, ability and skill of communication, computer skills, knowledge of foreign languages, readiness for improvement, and missing competencies. Employers were asked on the methods for improvement of the quality of the study program. The responses were given on a 4-point scale (1-non-satisfactory, 2-almost non-satisfactory, 3-partially satisfactory, and 4-satisfactory).

A total of 16 WHO/NHWA (Modules 2 and 3) indicators¹⁰ were analyzed, including the standards for the duration of medical studies, accreditation mechanisms for FMUB and medical studies, standards for social accountability, standards for effective implementation of social accountability, standards for social determinants of health, standards for inter-professional education, agreement on accreditation standards, lifelong learning, continuing professional development and in-service training, duration, applications, admissions e. g. enrollments, exit / drop-out, graduation and rate (ratio of the number of graduates to the number of students enrolled in first year of the same program) and success rate (% of students graduated on time) at FMUB. A descriptive analysis (absolute and relative numbers, mean and standard deviation) of available quantitative WHO/NHWA indicators covered period from 2010/11 to 2018/2019.

The standards for accreditation of medical studies at FMUB almost entirely can be used to string NHWA indicators for Education and Training (Table 1). The National Entity for Accreditation and Quality Assurance in Higher Education of Serbia (NEAQA) has accredited undergraduate medical studies as six-year integrated study program in medicine with 360 ECTS equivalents (total 5500 hours per Directive 2005/36/EC⁵³), and a list of knowledge, skills and competencies for the student to be acquired upon completion of each module curriculum, including that in the Continuing Professional Development (CPD). CPD is linked to re-licensure, however, there is a lack of a national health workforce strategy and national health policy and action plan⁴⁴ to align to national education plans.

Table 1 Qualitative NHWA indicators in the accreditation standards of undergraduate medical studies, MFUB Serbia 2013 and 2016

NHWA indicators Module 3¹⁰	YES/ NO /PARTLY
Existence of national and/or subnational standard on the duration and content of medical studies.	YES
· Are entry requirements to medical studies established concerning age, previous studies, previously acquired competence by study and past professional experience?	Standard 7
· Are the total number of hours to be spent on health workforce education and training established?	Standard 1
Is there a list of knowledge, skills and competencies to be acquired during integrated medical studies?	Standard 1
2. Existence of national and/or subnational mechanisms for accreditation of medical studies and institutions	PARTLY
Have national and/or subnational mechanisms for accreditation of health workforce education and training institutions and their programs been established?	Yes, by the Rulebook on accreditation of study programs brought by National Council for High Education
Are national and/or subnational mechanisms for accreditation of health workforce education and training institutions and their programs compulsory?	Yes, in accordance to the Law on High Education
Are there national and/or subnational mechanisms for accreditation of health workforce education and training institutions and their programs that are not compulsory?	NO
If established, do national and/or subnational mechanisms for accreditation of health workforce education and training institutions and their programs take into account national education plans for the health workforce, as described in indicator 09_04?	NO
3. Existence of national and/or subnational standards for social accountability in accreditation mechanisms.	NO
Is social accountability included or reflected within national and/or subnational standards?	NO
Is there an involvement of civil society, other social stakeholders and communities in accreditation mechanisms?	NO
4. National and/or subnational standards for social accountability in accreditation mechanisms are effectively implemented.	PARTLY
Do national and/or subnational mechanisms for accreditation of health workforce education and training institutions and their programs require compulsory reporting on implementation of national or subnational standards on social accountability?	NO
Do the communities served by the health workforce education and training institutions participate in the decision-making of these institutions?	PARTLY in accordance to NEAQA
Do students learn and train in the communities that the health workforce education and training institution serves (community placements)?	Yes, Standard 4
Do health workforce education and training institutions measure their impact on the health system and populations they serve?	NO
5. Existence of national and/or subnational standards for the social determinants of health in accreditation mechanisms.	PARTLY
Are the social determinants of health included or reflected within national and/or subnational standards?	PARTLY, in accordance with an institutional policy
Do health workforce education and training institutions measure social determinants of health in the populations they serve?	NO
Do health workforce education and training institutions adapt curricula according to social determinants of health in their communities?	PARTLY
6. Existence of national and/or subnational standards for interprofessional education in accreditation mechanisms	NO**
Is interprofessional education included or reflected within national and/or subnational standards?	NO
7. Existence of cooperation between health workforce education and training institutions and regulatory bodies to agree on accreditation standards.	YES
Is there a coordinating mechanism or body in place for this task?	Standard 11
Are various stakeholders at national and institutional level involved in the coordination process?	Standard 11
Are there institutional mechanisms in place to coordinate accreditation systems, including negotiations with relevant ministries, government agencies and stakeholders?	Standard 11
8. Existence of national systems for continuing professional development.	PARTLY**
Are there existing national and/or subnational systems for continuing professional development (CPD)?	Yes, by the Rulebook on medical chamber association (no specific standard)
If national and/or subnational systems for CPD exist, are they compulsory?	Yes, by the Rulebook on medical chamber association (no specific standard)
If compulsory, are they linked to re-licensure?	Yes, by the Rulebook on medical chamber association (no specific standard)
For occupations that have a national and/or subnational system for CPD, is it integrated into national education plans for the health workforce, for that occupation (see indicator 09_04)?	NO
9. Existence of in-service training as an element of national education plans for the health workforce	YES
Is in-service training integrated into larger national education-wide sector policies, strategies and plans?	Yes, in accordance to the Law on Health Care
Does in-service training consider and take into account national policies, strategies and plans for transforming professional, technical and vocational education and training?	Yes, in accordance to the Law on Health Care
Does in-service training consider and take into account national policies, strategies and plans for adult learning and higher education?	Yes, in accordance to the Law on Health Care

*If YES/PARTLY a number of the national standard for accreditation and regulation is added.

** These were indicators for all programs at institutions, not specifically for undergraduate medical studies.

The structure of key regulatory bodies^47,48is partly socially accountable given that apart from the Chamber of Commerce, the remaining entities are policymakers and education professionals. The National Council for High Education (NCHE), the NEAQA and the Accreditation Commission with Sub-commissions report on implementation of national standards on social accountability. A coordinating body to agree on accreditation standards is NEAQA, and it coordinates accreditation systems including negotiations with relevant ministries, government agencies and various stakeholders at national and institutional level involved in the coordination process. The Government elects members of the NCHE, the Steering Committee and the Accreditation Commission of the NEAQA, all of which consisted of members such as professors and experts, while student participation is optional) (Supplemental file 2).

Social determinants of health such as gender, ethnicity and disabilities are partially included in national and/or sub-national standards (Table 1). In 2016, the novelty was an enrolment quota for students from vulnerable populations such as Roma students⁵⁴ or students from third world countries⁵⁵. In-service training is integrated into larger national education-wide sector policies strategies and plans such as the Law on adult education⁴⁷. In addition, professional networking, participation in CPD, research and project cooperation is well encouraged.

FMUB indirectly assess an impact on the health system and populations they serve through surveying the graduates (Table 2), and employer’s opinion (Table 3) on the quality of accredited medical study program at FMUB, while admissions/enrollments and graduation rate were annually monitored as was prescribed by the Law on Higher Education⁵² (Table 4).

According to graduates (Table 2), the quality of medical undergraduate study programme at FMUB has improved from 2013 to 2016. In 2013, an average rate for the quality of each aspect was in the range from 3.27 ± 1.23 (aspect - students’ workload) to 3.81±0.97 (aspect - website information). An overall rate was 3.48±0.71. In 2016, an average rate per quality aspect was in the range from 3.33 ± 1.24 (students’ independent work) to 4.17±0.94 (students’ workload). Students perceived that quality of studies might be improved by establishing teamwork, mentorship, introducing more practical work for better balance between the theoretical and practical lessons. Over half of respondents declared they lack practical skills, and have no interest in continuous education / training.

Table 2. Medical graduates’ ratings for the quality of medical studies, MFUB, 2013 and 2016

Aspects of the medical study program at FMUB	*Respondents with positive responses (good, very good and excellent), %**
Aspects of the medical study program at FMUB	2013 (n=58)	2016 (n=75)
Quality of the content of the study program	90	99
2. Quality of the teaching process	88	93
3. Quality of general conditions of study	86	82
4. Students’ knowledge and skills for completing work assignments	81	78
5. The content of the course is coherent	85	89
6. The distribution of student workloads in the subjects is correct	73	96
7. The curriculum complies with the needs of students	81	86
8. The teachers encourage active participation of the students	73	85
9. The teachers encourage independent work of the students	76	63
10. The teacher’s exam requirements were objective	83	77
11. The teacher- students relationship was professional and correct	88	100
12. Quality of teaching conditions	79	74
13. The level of teaching’ tool coverage	91	89
14. Quality of textbooks	90	79
15. Availability of relevant information on the faculty's website	91	82
16. Discrimination of students at the faculty	79	75
17. Satisfaction with the study program	86	99

*The responses were given on a 5-point scale (1-very bad, 2-bad, 3-good, 4-very good and 5-excellent)

The satisfaction of employer with the competencies of medical graduates has increased as well (Table 3). In 2016, the survey results clearly show that the majority of employers (91.6%) are satisfied with the competencies of young doctors - employees in their institutions, especially with knowledge (94.4%), readiness for further improvement (88.9%), work habits (86.1%), communication and computer skills (80.2% and 91.7% respectively) as well as with English language skills (91.6%). However, the majority of employers believe that young medical doctors lack communication skills (80.2%) and the ability for independent work (66.6%), as well as organizational skills (50.8%).

Table 3. Employers’ satisfaction with the competencies of medical graduates, MFUB, 2013 and 2016

Medical graduates’ competencies	*Employers’ satisfaction: partially satisfied and satisfied, %**
Medical graduates’ competencies	2013 (n=41)	2016 (n=36)
Overall competencies	75.7	91.6
2. Knowledge	90.2	94.4
3. Readiness for further improvement	90.2	88.9
4. Work habits	90.2	86.1
5. Communication	97.6	80.2
6. Computer skills	100	91.7
7. English language skills	85.4	91.6
8. Ability to solve practical problems	87.8	97.3
9. Independent work	65.8	66.6
10. Organizational skills	43.8	50.8

*The responses were given on a 4-point scale (1-non-satisfactory, 2-almost non-satisfactory, 3- partially satisfactory, and 4- satisfactory)

The table 4 shows that over the ten-year period, the total number of students enrolled in all study years varied barely from the average (by 3.7%) but has an increasing trend (an overall increase of 7.3%). In the observed period, the number of all admitted to the first study year (76.4% of freshmen and 23.6% of repeated enrollments) was above the maximum accredited places (640). From 2010/2011 to 2018/2019, the number of freshmen varied for 5% from the average number, but on overall it decreased for 4.7%. It was always below the maximum of accredited places and above the available budgeted places (450) implying it includes a number of self-financing students (Table 4). The average number of students per study years was declining, resulting in 68.6% of graduation rate (or 89.8% compared to freshmen only). During the observed period, average duration of medical studies is prolonged by 1.2 year, leading to increased average number of students graduated on time - the average success rate was 26.6% (Table 4).

Table 4. Quantitative NHWA indicators (Module 2¹⁰) in the accreditation standards of undergraduate medical studies in Serbia, MFUB 2010-2018

School Year	2010/11	2011/12	2012/13	2013/14	2014/15	2015/16	2016/17	2017/18	2018/19	x	sd
Students, n Total	3557	3576	3564	3867	3909	3716	3747	3821	3818	3730.6	136.3
I	710	741	724	789	753	730	707	723	772	738.8	27.9
II	646	704	779	731	785	786	807	803	766	756.3	53.2
III	607	531	560	638	590	590	652	607	601	597.3	36.6
IV	540	557	500	540	600	553	565	619	605	564.3	37.8
V	545	526	536	473	525	563	497	520	579	529.3	32.1
VI	509	517	465	696	656	494	519	549	495	544.4	78.6
Freshmen	555	588	602	604	531	556	550	560	529	563.9	28
Graduates, n	471	547	538	492	499	458	552	538	458	505.9	38.6
Graduation rate %	66.3	73.8	74.3	62.4	66.3	62.7	78.1	74.4	59.3	68.6	6.6
Success rate, %	21.4	32.6	31.3	21	14.7	21.4	32.6	31.3	32.8	26.6	6.9
Average duration of studies (years)	7.2	7.4	7.4	7.6	7.8	6.5	6.6	7.4	7	7.2	0.4

x – mean; sd -Standard deviation

The study results indicated partial or complete synchronization with national accreditation standards concerning 12 out of 16 studied WHO/NHWA qualitative and quantitative indicators for accreditation and regulation (i.e., 7 out of 10 qualitative indicators, which is 17 out of 28 examined aspects and 4 out of 6 quantitative indicators).

This study showed the effects of accreditation standards of undergraduate medical studies at FMUB by analyzing them through the lenses of the WHO/NHWA indicators. The findings show that the accreditation mechanism at FMUB, although implemented for the last fifteen years, has the potential to support quality improvement in health workforce education, but not health care and health workforce policy planning. Updates and revisions of standards in 2013 and 2016, as well as in 2019, have led to different standards focusing on quality education and training, and partly on social responsibility and addressing social determinants. This study was done at this particular time so that, among other things, we could provide the information necessary for the new accreditation cycle, as well as point out the importance of the accreditation of educational programs for doctors whose aim is better citizens’ health. In the period from 2013 to 2016, we found 11 out of 17 aspects of the quality of education have improved and contributed to the satisfaction with the study program of all graduates. On the other hand, employer satisfaction improved in 6 out of 10 aspects of graduate's performance.

The survey also helped shed light on the alignment of the national accreditation standards with the WHO/NHWA. Although the survey does not measure the effect of the accreditation standards on health policy, it shows the opinion of the employers on the level of preparedness of doctors to provide appropriate health services. Given that the same employers are among the key actors and decision-makers or initiators of changes in health care and policy, the results of the survey indicate the limited potential of the accreditation standards to contribute to the formulation of health policy. Although the connection is indirect, the potential is there. It should be exploited and strengthened by implementing the national standards that would directly support linking education policy and health policy and targeting users' needs. The WHO/NHWA indicators for accreditation and regulation of health workforce education are constructed in this way in order to address health workforce education policy (e.g., enable monitoring, set activities and objectives and evaluate success), which is linked to the health needs of the community. The study found a partial or entire synchronization with the national accreditation standards. It highlighted the shortcomings that need to be overcome in order for the national accreditation standards of undergraduate medical education to contribute to the improvement of the quality of health workforce education and health policy, and in turn to foster sustainable health development of the community.

This study emphasizes the importance of improving the quality of education accreditation standards for socially accountable higher education, for which health workers, civil society and the community must participate in regulatory bodies. Students optionally participate in regulatory making. Alike healthcare managers who also serve as providers for training placement for students during the studies, students take part in FMUB accreditation surveys. While engaging the civic societies can advance academic quality and demonstrate public accountability, no precise article of the national regulations encourages greater collaboration among stakeholders and community and cultures where students after graduation and deployment will typically provide healthcare services.

The issue of community involvement in the accreditation process is also very problematic in the United States and Canada, where accreditation and regulation of higher education have a long history compared to Serbia. For instance, majority of medical schools do not include community service terms and/or descriptive language in their mission statements, and only 8.5% of medical schools incorporate community service and engagement as a primary or major criterion in promotion and tenure guidelines⁵⁶. The six major accreditation systems in these countries (the Liaison Committee on Medical Education; the Committee on Accreditation of Canadian Medical Schools; the Australian Medical Council; the General Medical Council; the World Federation for Medical Education; and the Conférence Internationale des Doyens et des Facultés de Médecined' Expression Française: International Association of Francophone Deans and Medical Schools) which essentially concentrate on the quality of medical education processes with variable consideration for the potential links between these processes and the development of a more efficient, equitable, and sustainable health system and the short- and long-term consequences of their programs on population health⁵⁷.

Following the idea that accreditation process can significantly contribute to building competent health workforce and subsequently lead to better healthcare and meet the society’ needs^2,3,57, the current accreditation mechanism of medical studies at FMUB must pursue for innovation, effectiveness and accountability through getting a larger input from different categories of interested parties (for instance, health care providers interested in building professional competency; students interested in acquiring the right to access an international medical license and the civic society interested in equitable access to quality services). Social accountability and social determinants and inter-professional education should be taken as priority standards in the future revisions of the regulatory accreditation mechanism. These priorities would signal a culture of continuous quality improvement of medical education is adopted and genuinely embraced in Serbia. A balanced representation of interested parties including policy-makers, education providers, health workforce labor market, students’ groups and the local community could contribute to a primary goal of accreditation, which is to result in positive patient outcomes⁵⁸.

In our study, the majority of students and of employers agree that for an independent and quality work, a medical graduate would need more communication and organizational skills. Similar was found by other authors^59,60. Since more than 70% of students rated the level of quality of education indicators as good, very good or excellent, the main objectives of integrated academic program of medical studies prescribed by the Rulebook on the organization and conduct of integrated academic studies for obtaining the title of doctor of medicine^46,49were met. Given that standards and criteria of accreditation speak of the institution's ability to produce health workers who are ready to respond to the needs of society²⁷, study results added to the understanding of the advancement of the organization, the general conditions, and the quality and content of teaching process and the study program. They showed that establishing clear and well-designed standards in accreditation process yields opportunity, firstly, to measure the impact of accreditation process introduced into to legal framework of Republic of Serbia by the amendments to the Law in Higher Education⁵², and secondly, to improve that process by continuous self-evaluation, also proscribed by the Law and other provisions defined by NEAQA⁵⁰.

This study methods is limited in terms of measuring the impact of accreditation of medical study program on the quality of health care and subjective opinion of the respondents might contribute to over and underestimation of findings. No conclusion can be borne on causal relationship; rather short-term changes in the accreditation process were measured through reactions to the educational experience of graduates and employers³¹. The full response rate was obtained from a convenient sample of graduates and employers, which at that time were not representative of the total graduate population or employers. Since the socio-economic determinants impact on health progress globally and nationally⁵⁹, the surveys in future should be aiming to cover larger sample of graduates and students with a different socio-economic background.

The revised national accreditation standards for the education of health workers in Serbia and reports remained general, with focus on quality of teaching, technical elements, and conditions. Neither the accreditation body nor the standards have a voice from the community. Though mandatory, the quota for students from Roma and other vulnerable populations were not mentioned in accreditation. We believe that standards that are specific to medical education such as the WHO/NHWA indicators should be used.

A true impact of accreditation may well rest in the ability to promote continuous quality improvement within medical education institutions⁶⁰, trough integration with health and health workforce strategies/plans^4,8-10,25,27.

Despite verified slight improvement in quality of medical undergraduate study program at FMUB from 2013 to 2016, other WHO/NHWA indicators showed reducing interest in medical studies. Over the ten-year trend there is an overall decline of freshmen enrollment. A significant share of re-enrollments and a success rate of less than 30% both were inducing a prolonged duration of medical studies. These indicators are important to support justification of certain study programs and the required number of students as foreseen by the new regulation^61,62. Given the emigration tendencies and a high unemployment rate of health workers^63,64, only with a new national strategy / plan for health care and health workforce, Serbia will be able to adequately regulate the future of higher medical education at five universities. FMUB, as one that has a third of all medical students in the country with about twenty thousand active doctors in the public sector, itself produces about 2.5% of new doctors per year. However, this study findings are also limited due to absence of exact data on applications and exist/drop outs. Future studies should explore these aspects in student’s cohort data.

This study presents a new approach to assess the compliance of national standards for the accreditation of undergraduate study program in medicine with global NHWA indicators and the effect of accreditation standards on medical education and policy planning for community health. Study showed that accreditation standards requiring alignment of health workforce education with national health plan and health workforce strategy can assist in coordinating policies on production of human resources for health toward achievement United Nations SDG 3 - health related goals as well as SDG 4 - Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all⁹. If continuously monitored, validated and recorded into the central database on health workforce (e.g. national health workforce information system), the WHO/NHWA indicators informs evidence-based policy decisions according to country needs through identifying significant improvements in transformation and scale-up of HWF education and training in support of UHC and increase ability to guide and intersectoral policy dialogue among the relevant ministries that may include those of education, health and finance¹⁰.

The accreditation mechanism at FMUB has the potential to support quality improvement in health workforce education, but are not sufficient for policy planning for health care and health workforce. Study findings indicate that improving medical students' communication, organizational skills and independent work, as well as social accountability and social determinants must be taken as the priority of FMUB. The participation of students, health workforce organizations, civil society and the local community in accreditation regulatory bodies, mechanism and standards of undergraduate medical studies is necessary for Serbia to confirm the commitment of an accredited institution’ to improving the quality of higher medical education and compliance with quality of healthcare. Furthermore, the study emphasizes that socially accountable medical education requires the implementation of the WHO/NHWA indicators at the national level, which shows the harmonization of higher education standards and plans with national standards and the health care strategy / plan.

Ethics approval and consent to participate: Not applicable. No ethical approval was required, as this study uses secondary data and routine official statistics of the FMUB.

Consent for publication: Not applicable. Manuscript does not contain any individual person’s data in any form.

Availability of data and materials: All data generated or analyzed during this study are included in this published article.

Competing interests: The authors declare that they have no competing interests

Funding: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Authors' contributions: MSM conceptualized and designed the study. BB analyzed and interpreted the data of the survey. MSM analyzed and interpreted the statistics of the undergraduate medical studies at FMUB. MSM and BB performed the examination of the NHWA indicators for education and training. Both authors were writing the manuscript. BB drafted the manuscript. MSM supervised the manuscript writing. Both authors read and approved the final manuscript. Both authors are the first authors.

Acknowledgments: The authors thank the student service of the Medical Faculty of the University of Belgrade for obtaining anonymous statistical data. The author MSM is supported by the Ministry of Education, Science and Technological Development of the Republic of Serbia (Contract No. 175087). Study results were presented at the Global Symposium on Health Workforce Regulation and Accreditation, 10-12 December 2019, Istanbul, Turkey.

Authors' information: Biljana Buljugic, B.Sc. lawyer, MPH is the Secretary of the Center for quality assurance and teaching improvement at the Faculty of Medicine University of Belgrade, Serbia. Milena Santric Milicevic, MD., MSc., PhD is professor in at the Faculty of Medicine University of Belgrade, Serbia, and Coordinator of Human Resources for Health Course (Master in Health Management), and Co-Coordinator of Health Management Course (Master in Public Health).

Marmot M, Bell R (2018) The sustainable development goals and health equity. Epidemiology 29(1):5–7. https://doi.org/10.1097/EDE.0000000000000773. Accessed 20 Oct 2020.
UNESCO (2015) Unleashing the Potential- Transforming Technical and Vocational Education and Training, [online]. http://unesdoc.unesco.org/images/0023/002330/233030e.pdf. Accessed 20 Oct 2020.
World Health Assembly (2013) ‘World Health Assembly resolution 66.23: Transforming health workforce education in support of universal health coverage’. http://apps.who.int/gb/ebwha/pdf_files/WHA66/A66_R23-en.pdf. Accessed 20 Oct 2020.
Secretary-General Report, 2020. Progress towards the Sustainable Development Goals, available at: https://undocs.org/en/E/2020/57 [accessed 10 August 2020]
UNESCO (2012) UNESCO Guidelines for the recognition, validation and accreditation of the outcomes of non-formal and informal learning., UNESCO [online], available: http://unesdoc.unesco.org/images/0021/002163/216360e.pdf [accessed 05/08/2020].
Cox M. 2016. Reflections. Presented at the workshop: The Role of Accreditation in Enhancing Quality and Innovation in Health Professions Education. April 22. Washington, DC.
Phillips SD 2017. Accreditation: realities, challenges, and opportunities. In: National Academies of Sciences, Engineering, and Medicine. 2017. Exploring the role of accreditation in enhancing quality and innovation in health professions education: Proceedings of a workshop. Washington, DC: The National Academies Press. doi: 10.17226/23636.
WHO (2017) ‘Universal Health Coverage’, [online]. http://www.who.int/healthsystems/universal_health_coverage/en/. Accessed 20 Oct 2020.
UN General Assembly. Transforming our world: the 2030 Agenda for Sustainable Development, 21 October 2015, A/RES/70/1, available at: https://www.refworld.org/docid/57b6e3e44.html [accessed 05/08/2020].
National health workforce accounts: a handbook. Geneva: World Health Organization; 2017. Licence: CC BY-NC-SA 3.0 IGO.
Kassebaum DG, Eaglen RH, Cutler ER. The meaning and application of medical accreditation standards. Acad Med. 1997;72(9):808-818. doi:10.1097/00001888-199709000-00019.
Innes SI, Cope V, Leboeuf-Yde C, Walker BF. A perspective on Chiropractic Councils on Education accreditation standards and processes from the inside: a narrative description of expert opinion: Part 1: Themes. Chiropr Man Therap. 2019;27:57. doi:10.1186/s12998-019-0275-6.
Innes SI, Cope V, Leboeuf-Yde C, Walker BF. A perspective on Councils on Chiropractic Education accreditation standards and processes from the inside: a narrative description of expert opinion: Part 2: Analyses of particular responses to research findings. Chiropr Man Therap. 2019;27:56. doi:10.1186/s12998-019-0276-5.
Lurie SJ, Mooney CJ, Lyness JM. Measurement of the general competencies of the accreditation council for graduate medical education: a systematic review. Acad Med. 2009;84(3):301-309. doi:10.1097/ACM.0b013e3181971f08.
Innes SI, Leboeuf-Yde C, Walker BF. Similarities and differences of a selection of key accreditation standards between chiropractic councils on education: a systematic review. Chiropr Man Therap. 2016;24:46. doi:10.1186/s12998-016-0127-6.
Mohieldein AH. Outcome-based approach to medical education towards academic programmes accreditation: A review article. J Pak Med Assoc. 2017;67(3):454-460.
Faucett EA, Barry JY, McCrary HC, Saleh AA, Erman AB, Ishman SL. Otolaryngology Resident Education and the Accreditation Council for Graduate Medical Education Core Competencies: A Systematic Review. JAMA Otolaryngol Head Neck Surg. 2018;144(4):360-370. doi:10.1001/jamaoto.2017.3163.
Boelen C, Heck JE. Defining and measuring the social accountability of medical schools. Geneva: Division of Development of Human Resources for Health, World Health Organisation; 1995.
Lindgren S, Karle H. Social accountability of medical education: aspects on global accreditation. Med Teach. 2011;33(8):667-672. doi:10.3109/0142159X.2011.590246.
Strasser RP, Worley P, Cristobal F, Marsh D, Berry S, Strasser S, et al. Putting communities in the driver’s seat: the realities of community-engaged medical education. Acad Med. 2015;90(11):1466–70.
Yoo HH, Kim MK, Yoon YS, et al. Change of accreditation standards of medical schools by the Korean Institute of Medical Education and Evaluation from 2000 to 2019. J Educ Eval Health Prof. 2020;17:2. doi:10.3352/jeehp.2020.17.2.
Karle H. Global standards and accreditation in medical education: a view from the WFME. Acad Med. 2006;81(12 Suppl):S43-S48. doi:10.1097/01.ACM.0000243383.71047.c4;
Gibbs T, McLean M. Creating equal opportunities: the social accountability of medical education. Med Teach. 2011;33(8):620-625. doi:10.3109/0142159X.2011.558537.
National Academies of Sciences, Engineering, and Medicine; Health and Medicine Division; Board on Global Health; Global Forum on Innovation in Health Professional Education. Exploring the Role of Accreditation in Enhancing Quality and Innovation in Health Professions Education: Proceedings of a Workshop. Washington (DC): National Academies Press (US); October 5, 2016.
Singh Y, Roehr CC, Tissot C, et al. Education, training, and accreditation of Neonatologist Performed Echocardiography in Europe-framework for practice. Pediatr Res. 2018;84(Suppl 1):13-17. doi:10.1038/s41390-018-0078-9.
Frenk J, Chen L, Bhutta ZA, Crisp N, Evans T, Fineberg H, et al. Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet. 2010;376(9756): 1923–58.
WHO (World Health Organization). 2013. Transforming and scaling up health professional education and training: Policy brief on accreditation of institutions for health professional education. Geneva, Switzerland: WHO.
Ross SJ, Preston R, Lindemann IC, Matte MC, Samson R, Tandinco FD, et al. The training for health equity network evaluation framework: A pilot study at five health professional schools. Educ Health 2014;27:116-26.
Pálsdóttir B, Barry J, Bruno A, et al. Training for impact: the socio-economic impact of a fit for purpose health workforce on communities. Hum Resour Health 2016;14:49.
Burdick W, Dhillon I. Ensuring quality of health workforce education and practice: strengthening roles of accreditation and regulatory systems. Hum Resour Health2020; 18: https://doi.org/10.1186/s12960-020-00517-4.
Kirkpatrick, D. L. 1994. Evaluating training programs: The four levels. 1st ed. San Francisco, CA: Berrett-Koehler Publishers, Inc.
Nason E. The “ROI” in “Team”: Return on investment analysis framework, indicators and data for IPC and IPE. Ontario, Canada: The Institute On Governance:2011
Sen A. What's happening in Bangladesh? Lancet. 2013;382(9909):1966–8.
Walsh, K. Cost and value in medical education. In K. Walsh (Ed), The Oxford Textbook of Medical Education. Oxford University Press. Oxford: 2013.
Soucat A, Sheffler R, Ghebreyesus R, Adhanom T. The labor market for health workers in Africa. Washington: The World Bank; 2013.
Zwanikken PA, Dieleman M, Samaranayake D, Akwataghibe N, Scherpbier A. A systematic review of outcome and impact of master's in health and health care. BMC Med Educ. 2013 Feb 7;13:18. doi: 10.1186/1472-6920-13-18.
Maini A, Neusy AJ, Pálsdóttir B, Kumar S. Developing socially accountable UK medical schools. Clin Teach. 2020 Jun;17(3):320-322. doi: 10.1111/tct.13063. Epub 2019 Aug 26. PMID: 31452344.
Im S, Kim DK, Kong HH, Roh HR, Oh YR, Seo JH. Assessing clinical reasoning abilities of medical students using clinical performance examination. Korean J Med Educ. 2016;28(1):35-47. doi:10.3946/kjme.2016.8.
Maguire P, Fairbairn S, Fletcher C. Consultation skills of young doctors: II--Most young doctors are bad at giving information. Br Med J (Clin Res Ed). 1986;292(6535):1576-1578. doi:10.1136/bmj.292.6535.1576.
Cristobal F, Worley P. Can medical education in poor rural areas be cost-effective and sustainable: the case of the Ateneo de Zamboanga University School of Medicine. Rural Remote Health. 2012;12:1835.
Cristobal F, Worley P. Transforming health professional’s education. Lancet. 2011;377(9773):1235–6.
Santric Milicevic MM, Bjegovic-Mikanovic VM, Terzic-Supic ZJ, Vasic V. Competencies gap of management teams in primary health care. Eur J Public Health. 2011 Apr;21(2):247-53. doi: 10.1093/eurpub/ckq010. Epub 2010 Mar 9.
Terzic-Supic Z, Bjegovic-Mikanovic V, Vukovic D, Santric-Milicevic M, Marinkovic J, Vasic V, Laaser U. Training hospital managers for strategic planning and management: a prospective study. BMC Med Educ. 2015 Feb 26;15:25. doi: 10.1186/s12909-015-0310-9.
World Health Organization (WHO). Regional Office for Europe, European Observatory on Health Systems and Policies, Bjegovic-Mikanovic V, Vasic M Vukovic D, Jankovic J, Jovic Vranes A, Santric Milicevic M, Terzic Supic Z, Hernandez-Quevedo C. (2019). Serbia: health system review. World Health Organization. Regional Office for Europe. https://apps.who.int/iris/handle/10665/331644. Accessed 20 Oct 2020.
Rulebook on standards for self-evaluation and quality assessment of higher education institutions. Off. Gazette RS. 2006:106.
National Standards for accreditation of I and II-degree programs – Standard 3, Goals of the study program of Integrated studies of medicine 2014. www.med.bg.ac.rs. Accessed 20 Oct 2020.
Law on Adult Education, Off. Gazette RS. 2013:55, 2017:88, 2018:27 -and 2020:6.
Education, Audiovisual and Culture Executive Agency (EACEA). Overview of the Higher Education System. Erasmus+ Office in Serbia. Brussels: European Union, 2017. https://eacea.ec.europa.eu/sites/eacea-site/files/countryfiche_serbia_2017.pdf. Accessed 20 Oct 2020.
Faculty of Medicine University of Belgrade. Rulebook on the organization and conduct of integrated academic studies for obtaining the title of doctor of medicine 2019. http://med.bg.ac.rs/wp-content/uploads/2017/06/2_PRAVILNIKIASMAJ2019.pdf?script=lat. Accessed 20 Oct 2020.
National Entity for Accreditation and Quality Assurance in Higher Education of Serbia. https://www.nat.rs/ Accessed 20 Oct 2020.
National Council for Higher Education (NCHE). The Rulebook on Standards for Self-Evaluation and Quality Assessment of Higher Education Institutions and Study Programs. Off. Gazette Republic of Serbia 2019:13. https://www.nat.rs/pravilnici/. Accessed 20 Oct 2020.Law on Higher Education. Off. Gazette RS. 2017: 88; 2018:73; 2018:27- other law, 2019:67 and 2020:6.
Directive 2005/36 / EC of the European Parliament and the Council of Europe of 7 September 2005 on the recognition of professional qualifications. https://eur-lex.europa.eu/eli/dir/2005/36/oj. Accessed 20 Oct 2020.
Strategy for social inclusion of Roma men and women in the Republic of Serbia for the period from 2016 to 2025: 26 / 2016-4. https://www.pravno-informacioni-sistem.rs/SlGlasnikPortal/eli/rep/sgrs/vlada/strategija/2016/26/1/reg/ Accessed 20 Oct 2020.
Ministry of Education, Science and Technological Development of the Republic of Serbia. 2016. A call for scholarships „World in Serbia“. http://www.mpn.gov.rs/wp-content/uploads/2016/04/Poziv-za-prijavljivanje.pdf. Accessed 20 Oct 2020.
Goldstein A, Bearman R. Community engagement in US and Canadian medical schools, Source: Advances in Medical Education and Practice. 2011; 2: 43–49.
Boelen C, Blouin D, Gibbs T, Woollard R. Accrediting excellence for a medical school's impact on population health. Education for Health. 2019;32(1): 41-48.
Huggan PJ, Samarasekara DD, Archuleta S, et al. The successful, rapid transition to a new model of graduate medical education in Singapore. Acad Med. 2012;87(9):1268-1273. doi:10.1097/ACM.0b013e3182621aec.
Söderberg S. ISO 26000 and the SDGs. Geneva: International Organization for Standardization, 2018.
Blouin D, Tekian A. Accreditation of Medical Education Programs: Moving From Student Outcomes to Continuous Quality Improvement Measures, Source: Academic Medicine 2018;93(3):377-383.
National Entity for Accreditation and Quality Assurance in Higher Education of Serbia. The Rulebook on Accreditation Standards for Study Programs. https://www.nat.rs/wp-content/uploads/2019/02/03-PRAVILNIK-STUD.-PROGRAMI-2019_final-3.docx. Accessed 20 Oct 2020.
Rulebook on standards for self-evaluation and quality assessment of higher education institutions and study programs. Off. Gazette RS. 2019:13
Santric Milicevic M, Vasic M, Edwards M. Mapping the governance of human resources for health in Serbia. Health Policy. 2015, 119(12):1613-1620.
Santric-Milicevic MM, Terzic-Supic ZJ, Matejic BR, Vasic V, Rickets TC First- and fifth-year medical students' intention for emigration and practice abroad: A case study of Serbia. Health Policy. 2014;118(2):173-183.

Download PDF

Version 2

posted

You are reading this older preprint version

Read the latest preprint version →

Analysis of accreditation standards for undergraduate medical studies in Serbia through the lens of the National Health Workforce Accounts

Status:

Version 2

Abstract

Background

Methods

Results

Discussion

Conclusion

Declarations

References

Supplementary Files

Status:

Version 2