The Influence of the Accreditation Reports from the European Council on Chiropractic Education (ECCE) on Improvements/Changes to the Chiropractic Programmes that they Evaluate: An Audit study and Thematic Analysis CURRENT STATUS:

Background: The European Council on Chiropractic Education (ECCE) has been in existence since 1981 and currently accredits 10 chiropractic programmes throughout Europe and South Africa. Although it has always been assumed that the ECCE evaluation events and subsequent reports have resulted in improvements and growth to the individual chiropractic programmes, no systematic evaluation as to whether this is true and the extent of any improvements has previously been done. Therefore, the purpose of this study was to collect data from the accredited programmes regarding the degree of changes made to the 10 sections of the ECCE Standards as well as which specific improvements were done. Methods: This was a mixed methods audit study using questionnaires which included 10 closed statements requesting the degree of change (substantial, some, none) to the 10 sections of the ECCE ‘Standards’ based on previous evaluation reports. The proportion of responses falling into the three categorical options for each question was analyzed using descriptive statistics. The second part of the survey collected written responses identifying the specific ‘substantial’ or ‘some’ changes made based on previous evaluation reports. The analysis of the written statements used a modified ‘thematic analysis’ approach. Three researchers independently evaluating all statements and identified ‘themes’ from the written responses. These researchers then met to discuss and agree final themes. Results: There was a 100% response rate. Seven programmes (i.e. 70%) reported ‘some’ or ‘substantial’ changes to 6 or more of the 10 sections of the ECCE Standards. The most common section of the Standards for programmes to report changes was ‘Educational Programme’ (8 of 10 programmes). ‘Educational Resources’ had the largest number of programmes reporting ‘substantial changes’ (4) and was the second most common section of the Standards to have reported changes. The main themes identified emphasized the improvements in ‘infrastructure, equipment and faculty’, ‘increasing the teaching and use of ‘evidence-based practice’, ‘instilling a research culture in faculty and students’. Conclusions: This study documents that ECCE accreditation reports are useful in facilitating

improvements to the chiropractic programmes, particularly in the areas of improved infrastructure and faculty as well as research and evidence-based practice. Introduction Quality Assurance of modern higher and post-graduate healthcare educational programmes is considered crucial in order to produce knowledgeable and competent graduates [1][2][3][4]. This is particularly critical for primary contact health-care professionals, including chiropractors, in order to protect the graduates' future patients and to instill life-long learning habits [2]. Quality Assurance policies and procedures should be conducted internally by the college, university or specific programme as well as externally by recognized quality assurance agencies [4][5][6]. External quality assurance evaluations should provide a unique perspective without the potential blinders that may be present if programmes rely solely on internal reflection and assessment. External quality assurance agencies can also compare the relevant programmes with other similar programmes nationally and internationally, which may facilitate exchanges of best practice as well as international mobility of graduates [7][8][9][10][11].
One of the primary contact health-care professions in Europe that has continued to grow over the past several decades is Chiropractic medicine. Although the legal status of this profession varies widely in Europe, from being one of the five recognized medical professions in Switzerland to still being illegal in Spain, most western European countries have laws that regulate the profession [12].
This requires that the education and training of chiropractors meet specific Standards to prepare graduates to be safe and competent practitioners and to provide evidence-based care to their patients [7,12].
The European Council on Chiropractic Education (ECCE), which has been in existence since 1981, currently accredits ten chiropractic programmes; eight in various European countries and two in South Africa [13]. Additionally, several new chiropractic programmes have started in Europe to meet the growing demand for this profession, but have yet to undergo ECCE evaluation, pending the graduation of their first cohorts of students. The ECCE is a member of the Council on Chiropractic Education International (CCEI) and has mapped the ECCE 'Standards', 'Competencies', and 'Policies and Procedures' to the CCEI 'International Framework' as well as with these same documents from the other international member agencies. This is to help facilitate international mobility of graduates and basic standardization of the profession. In order for the ECCE to continue to be an effective accrediting body, providing useful feedback to chiropractic programmes, it is essential that the ECCE also evaluates its own processes and procedures to determine whether or not its evaluations are useful to chiropractic programmes (and to the profession at large) as well as to identify areas within the ECCE that may need improvement. The ECCE has recently completed and published two studies requiring internal reflection resulting in change/improvements to its policies and practices [13,14].
Therefore, the purpose of this third study, using a modified thematic analysis approach, sought to identify whether or not feedback from past ECCE accreditation evaluation reports has resulted in changes/improvements in the individual chiropractic programmes and if so, to identify specific changes made and to what extent.

Methods
This study used a mixed-methods approach collecting survey data from the heads of each of the ten ECCE accredited chiropractic programmes. Questionnaires were designed which included two sections. The first section contained ten closed statements, each statement corresponding to one of the ten sections of the ECCE Standards. The instructions to the participants for this section were: "To what extent, if any, have past ECCE evaluation reports facilitated your educational institution to make changes/improvements to the components identified in specific category inserted here (i.e. Aims and Objectives'?" The options included the responses 'substantial changes made', 'some changes made' and 'no changes made'. For each of these ten closed statements, annotations were also included listing each of the individual 'Standards' that fell into the category (Additional file 1). The proportion of responses falling into the three categorical options for each question was analyzed using descriptive statistics. The second part of the survey collected written responses to the statement "This section requests that you list/identify 1) Changes that you did make based on feedback from an ECCE evaluation report; and 2) Changes that you were unable to make and the reasons why these were not done." The respondents were also asked to identify 'substantial changes' versus 'other' (less substantial) changes done as a result of feedback from an ECCE evaluation report. All written responses were copied verbatim and placed in a single document to be assessed independently by three experienced researchers using a modified thematic analysis approach, similar to a recent study published by the ECCE 14 (Additional file 2). Each researcher was instructed to identify recurrent 'themes' from the written responses for each of the three statements. Once this was completed individually, the researchers met together to discuss their individual findings, explain their rationales and to agree on the final 'themes' for each section. Ethical approval was not necessary for this audit study with voluntary participation and no interventions. Returning the completed questionnaire was considered informed consent to participate and participants were informed of this. Information regarding the study was presented to the institutional representatives during the ECCE annual November 2019 meeting.

Results
All ten chiropractic programmes returned their completed questionnaires (100% response rate). Table 1 shows the proportion of 'no change' 'some change' and 'substantial change' responses for each programme for each of the ten sections of the ECCE Standards. Only one programme stated that they had made no changes to their programme based on feedback from ECCE evaluation reports.
For the 90% of accredited programmes reporting changes based on ECCE accreditation report recommendations, seven (i.e. 70%) reported 'some' or 'substantial' changes to 6 or more of the 10 sections of the ECCE Standards (i.e. > 60% of the sections) (table 1). One programme reported that they had made changes to all 10 sections of the ECCE Standards based on feedback from the evaluation reports, with 3 of the areas undergoing 'substantial' changes (table 1).
The most common section of the ECCE Standards for programmes to report 'some' or 'substantial' changes based on feedback from ECCE Accreditation Reports was 'Educational Programme' (section 2 of the Standards) with eight of the ten programmes having made changes within this section. Section 6 of the ECCE Standards (Educational Resources) had the largest number of programmes reporting 'substantial changes' (4 in total) and was the second most common section of the Standards to have reported changes (table 1). The 3 sections of the Standards where programmes were least likely to report any changes were: 1) 'Students' with 7 of the 10 programmes reporting 'no changes'; 2) 'Governance and Administration' showing 6 programmes making no changes, and 3) 'Assessment of Students' also showing 6 programmes making no changes (table 1).
Regarding the responses to the open-ended written questions asking the programmes to identify and describe the changes that they had made due to feedback from the ECCE accreditation/evaluation reports, the researchers identified 3 areas/categories where changes were made. These were 1) Resources; 2) Evidence-Based Teaching and Learning; 3) Research. Specific 'themes' identified by the 3 researchers relating to these 3 areas are listed in table 2.

Discussion
Although the impact and effectiveness of accreditation on medical education processes have been reported [1,2], this current study is the first within the chiropractic profession to assess the effects, if any, that accreditation has on the way chiropractic programmes evaluated and accredited by a chiropractic specific accrediting agency develop and function. It was very encouraging to discover that 90% of the ECCE accredited programmes reported that they had made improvements to their operations based on the feedback from the ECCE accreditation reports. The only programme that did not report any improvements/changes stated that because they are also accredited by an agency within their own country, this takes priority over feedback from the ECCE. Consequently, they admitted in the comments section of the questionnaire, that some recommendations contained in the ECCE report could not be implemented. That is interesting as 6 of the other ECCE accredited programmes also undergo accreditation by their individual country's chiropractic, higher or medical education accrediting bodies. Never-the-less, they were able to also address issues that arose from the ECCE feedback reports. This is most likely due to the varying regulations and laws governing education throughout Europe and South Africa. Furthermore, there is currently considerable collaboration between the national chiropractic accrediting bodies in the United Kingdom and Switzerland with the ECCE. The accrediting body in South Africa also recognizes the ECCE (personal communication from the ECCE vice president and South African department head).
The most common section of the ECCE Standards where improvements were made following feedback from the ECCE accreditation reports was 'Educational Resources' (section 6) [7]. This section of the ECCE Standards contains three of the eighteen 'critical Standards' which must be at least 'substantially compliant' in order to achieve the maximum eight-year accreditation time period [13].
Thus, chiropractic programmes would likely be very cognizant of the relative importance of these particular Standards. Indeed, nine of the ten accredited programmes reported that the ECCE accreditation reports motivated their programme's academic leaders to make needed improvements in this area. This section also dominated the written comments listing the various changes implemented. Thematic analysis of these written comments found that the most common specific changes/improvements made were in Infrastructure/Physical Facilities and Equipment, increasing the number of faculty members with the appropriate qualifications and experience and increasing the opportunities for inter-disciplinary teaching and learning.
The second most common section of the ECCE Standards where respondents reported improvements to their programmes arising from the ECCE accreditation reports was the 'Educational Programme' itself (section 2) [7]. Thematic analysis of the written comments relating to this topic found that several programmes increased their focus on evidence-based teaching and learning as well as evidence-based practice which incorporates the Bio-Psycho-Social model of health-care education.
Additional themes that arose for this section included improving the integration of subjects, closing the theory-practice gap as well as increasing self-directed learning, which all lead to decreased contact time for students. Five of the 18 'critical' ECCE Standards which must be at least 'substantially compliant' for the maximum accreditation time period fall into this category [13]. This most likely provided additional impetus for programmes to change and improve.
The researchers also identified a third area in the thematic analysis of the written comments entitled 'Research'. While this may appear to link into the previous thematic section on evidence-based teaching and learning, the actual responses falling into this category were more specifically on teaching faculty and students how to perform research studies, analyze the quality of research publications, as well as how to use existing research in daily practice with patients. This aligns with ECCE Standard 2.2 'The Scientific Method' which is also one of the 'critical' Standards requiring at least a substantially compliant rating for the maximum accreditation time period [13]. The additional theme identified by the researchers for this section of 'Standardized Evaluations and Feedback' refers to using questionnaires to obtain feedback within the programme regarding course/class evaluations and faculty performance evaluations. The purpose of this type of feedback is to perform internal quality assurance and facilitate changes and improvements to the programme.
Although the ECCE Standards section on 'Students' showed that seven of the ten programmes reported no changes to their programmes based on evaluation report feedback regarding this area, this is because this particular section of the Standards deals only with how students are selected and admitted to the programmes and how they are supported and counselled. No other issues regarding students are addressed in this section as they are covered in other Standards [7]. Most accredited programmes were confident that their selection criteria were appropriate and that the students were sufficiently supported. Furthermore, the ECCE did not identify these areas as 'Critical' Standards in the previous research on this subject [13]. The majority of ECCE accredited programmes are part of larger universities which provide good student support services.

Strengths and Limitations to the study:
The most obvious strength to this study is the documentation in the form of written feedback from accredited programmes that the ECCE accreditation reports have resulted in many significant improvements for 90% of the accredited programmes. In particular, improvements in equipment and resources as well as the number and quality of the faculty were frequently mentioned. Improving evidence-based teaching and practice was also a frequent theme.
The most obvious limitation to this study is that the respondents to the questionnaires were all heads of the ten accredited programmes. Thus institutional/programme memory of past ECCE accreditation evaluation reports was required in order to accurately complete the questionnaire, as it was not specifically requesting information from only the most recent accreditation event. Thus, newer department heads in the older programmes would not have experienced several past ECCE accreditations and perhaps did not know about the specific feedback from those events and the subsequent changes made by the programme unless these new department heads had specifically taken the time to try to investigate this by looking at old evaluation reports. This may have been a factor in the answers received from two of the programmes who have had ECCE accreditation for over 20 years. One of these programmes reported only one change and the other reported no changes at all based on the accreditation reports.
As usual there could be memory lapses or recall bias or even tendency of the subjects to report a biased answer (halo effect) to please the researcher. Further, it is possible that the level of change for one school may have been considered 'minor' and yet 'substantial' to a different school. In the future, these terms should be carefully defined.
Conclusions ECCE accreditation evaluation processes and reports have facilitated significant improvements in the operations of 90% of the programmes evaluated. This is not only useful information for the accredited programmes but also very useful feedback to the ECCE itself. Although 1 programme reported no changes in their operations based on the ECCE report due to over-riding national accreditation processes, 6 other ECCE accredited programmes who also undergo evaluation by their national The datasets generated and analyzed during this study are included in this published article and its supplementary information files.

Consent for publication
Not applicable