A Pragmatic Trial of Active Versus Passive Teaching for Clinically Integrating Evidence Based Medicine Teaching in an Undergraduate Medical School.

Background Clinically integrated Evidence Based Medicine (EBM) teaching promotes higher order application of knowledge, skills, and behaviours. There is, however, limited evidence on the most ecient model for clinically integrating EBM teaching. The aim of this study was to test the feasibility of integrating EBM teaching into clinical placements in an undergraduate medical school and compare the effectiveness of two different methods of EBM teaching - active teaching and passive teaching. Methods Active teaching involved blended learning methods and facilitated small group discussions. Passive teaching involved self-directed small group discussions and a remote live telecast taught session. Students’ EBM competencies were assessed using the Assessing Competency in EBM (ACE tool) and Educational Prescriptions (EP) and in summative nal professional written and clinical examinations. Results Education was delivered to 65 students, of whom 45 received active teaching, while 20 received passive teaching. Students receiving active teaching performed better in EPs (MD=-2.28, -4.31, -0.26). There was no signicant difference in performances in the ACE tool (MD=-1.02, -2.20, 0.16); the written nal professional exams (MD=-0.11, -0.65, 0.44) and the EBM OSCE stations (MD=-0.81, -2.38, 0.74). Conclusions It was feasible to integrate EBM teaching into clinical placements in an undergraduate medical school. Both active and passive methods were effective in delivering EBM teaching. While students in the active teaching model scored higher in EPs; there was no signicant difference in performances in the ACE tool or the summative assessments. Scaffolding student learning or a spiral approach to EBM teaching model provides a great example of continually shifting the zone of proximal development.


Abstract
Background Clinically integrated Evidence Based Medicine (EBM) teaching promotes higher order application of knowledge, skills, and behaviours. There is, however, limited evidence on the most e cient model for clinically integrating EBM teaching. The aim of this study was to test the feasibility of integrating EBM teaching into clinical placements in an undergraduate medical school and compare the effectiveness of two different methods of EBM teaching -active teaching and passive teaching.
Methods Active teaching involved blended learning methods and facilitated small group discussions. Passive teaching involved self-directed small group discussions and a remote live telecast taught session. Students' EBM competencies were assessed using the Assessing Competency in EBM (ACE tool) and Educational Prescriptions (EP) and in summative nal professional written and clinical examinations.
Results Education was delivered to 65 students, of whom 45 received active teaching, while 20 received Conclusions It was feasible to integrate EBM teaching into clinical placements in an undergraduate medical school. Both active and passive methods were effective in delivering EBM teaching. While students in the active teaching model scored higher in EPs; there was no signi cant difference in performances in the ACE tool or the summative assessments. Scaffolding student learning or a spiral approach to EBM teaching model provides a great example of continually shifting the zone of proximal development.

Background
Evidence Based Medicine (EBM) is the triangulation of best available evidence, clinical expertise and patients' preferences before applying it to clinical decisions (1). EBM involves ve steps -(i) asking the right question; (ii) acquiring evidence; (iii) appraising evidence; (iv) applying to clinical decision and (v) assessing the performance in the rst four steps (2). The importance of ensuring medical students are equipped with the skills to be able to practice EBM has been increasingly recognised in recent years (Albarqouni et al. 2018). The General Medical Council (GMC) recommends that 'Newly quali ed doctors must be able to apply scienti c method and approaches to medical research and integrate these with a range of sources of information used to make decisions for care'(5).
Historically, most medical schools have taught statistics and epidemiology, but often focused on calculations and study designs with little integration into clinical problem solving. This has traditionally been delivered as didactic lectures, audio-visual presentations, and printed materials. However, where performance change is the goal, the use of didactic teaching has been debated, with some stating that it should have little or no role to play (6). More recently, using an online didactic teaching model during clinical rotations for third year students has demonstrated that students successfully acquired and applied EBM skills (7). Following on from this, a comparison of different approaches to teaching EBM has identi ed that a blended-learning approach promoted greater student appreciation of EBM principles within the clinical setting compared to didactic teaching (8).
EBM has also often been taught as short courses or seminars, with a common objective being improving critical skills with less focus on other EBM skills (9,10). The landmark systematic review by Coomarasamy & Khan identi ed that while standalone teaching of EBM improved student knowledge, it did not improve skills, attitudes or behaviour in EBM (11). Furthermore, it recommended that EBM teaching should move from classrooms into clinical practice. This need of integrated learning was further highlighted by a survey of UK medical schools nearly a decade ago (12). It identi ed that the undergraduate EBM curriculum was poorly developed and there was a clear need for integration of EBM with clinical specialities and for a 'national EBM core curriculum' to support the development of undergraduate EBM teaching and assessment in UK.
In recent years, longitudinal integration of EBM in medical schools has been reviewed. Teaching EBM as a longitudinal theme across the medical curriculum rather than standalone courses has been effective, as demonstrated by students' performances in the validated Fresno and Berlin tests (13). Further evidence supports the view that EBM teaching and learning strategies should focus on implementing multi-faceted, clinically integrated approaches with assessments of knowledge, skills and behaviour in the medium to long term using validated assessment tools (14). To this end, a taxonomy of validated assessment tools has been developed for medical educators to aid in the evaluation of their EBM teaching (15). Though there is an increasing focus on clinically integrating EBM teaching, there is paucity of evidence on the most e cient way to integrate EBM teaching into clinical rotations. EBM educators in medical schools will bene t from descriptions of clinically integrated EBM curricula -content, teaching format, assessments, resources needed and impact on students' short-and long-term retention of knowledge and skills.
The University of Buckingham Medical School (UBMS) is a relatively new medical school, with the rst intake of students in January 2015. Based on recommendations made in literature, EBM has been incorporated as a longitudinal theme, starting from year one when students learn biomedical sciences and continues to be integrated into teaching during later years when students are in clinical placements.
Early evaluation of our curriculum in the rst two years of MBChB has shown an increase in students' knowledge as assessed by their performance in the Fresno test, in addition to their self-rated EBM knowledge and self-reported attitudes about EBM (Kumaravel et al, 2020, article in press). The aim of this study is to describe our approach of integrating EBM into clinical placements in an undergraduate medical school and to evaluate the effectiveness of two different models of EBM teaching (active and passive) as assessed by students' performances in their formative and summative assessments.

EBM curriculum in UBMS
The MBChB course at UBMS consists of two phases. Phase I runs for the rst two years of the MBChB course, with the focus on biomedical sciences with some patient interactions. Phase II continues for the next two and a half years when students are in clinical placements in hospitals and primary care. EBM has been integrated longitudinally across the MBChB course, beginning in term 1 of Phase I ( gure 1).
Teaching methods for EBM in phase I Blended teaching methods were used, in which lectures were followed by small group work with clinical case-based discussions to develop skills in application of knowledge. Students were also given selfdirected tasks to research given topics in preparation for discussions in class the following week.
Technology enhanced learning methods were adopted for teaching, for example YouTube videos showing senior students' application of EBM in clinical practice were shared in lectures to early years students to demonstrate the clinical relevance of EBM. Audience polling was used during lectures to improve student engagement and gain feedback on students' understanding. Teaching was further supplemented by online interactive quizzes & recorded lectures were uploaded into Moodle (virtual learning environment).

Teaching methods for EBM in phase II
In years three and four, EBM was taught as clinically integrated seminars-jointly delivered by clinicians and public health doctors who are trained in EBM. We continued to use blended learning methods with a mix of lectures, interactive workshops, small group teaching and ipped classroom methods where students were asked to research on a topic and present their ndings to peers.
With the rst cohort of medical students, clinical teaching was based in one hospital, making it possible for the EBM theme lead to deliver active teaching during students' clinical placements. However, due to a larger cohort in 2016, students were based across two hospital sites, which made it challenging to deliver with limited resources. The EBM theme lead delivered the teaching in one hospital (active teaching), while students received the online version at the second site (passive teaching).
Details of the clinically integrated EBM teaching is shown in table 1 below.

Assessments
Both formative and summative assessments were carried out to allow both assessment of learning but also assessment for learning. Our formative assessments in EBM included the Assessing Competency in EBM (ACE) tool and Educational Prescriptions. Summative written assessments were case based assessments (based around clinical vignettes) with EBM speci c questions integrated alongside other biomedical and clinical questions. An outline of EBM assessments across the MBChB course in UBMS is shown in gure 2.
Formative assessments in Phase II During the Public Health and EBM teaching in third year, the Assessing Competency in EBM (ACE) tool (16) was used as a formative assessment to demonstrate the application of EBM steps to a clinical scenario. Having demonstrated the application of EBM using the ACE tool, we then asked students to complete the online educational prescription (EP) from the University of Wisconsin (www.ebm.wisconsin.edu/ep), which guides students to apply EBM to answer any areas of clinical uncertainties in clinical rotations (17).

ACE
Students at both sites completed the ACE tool online. The ACE tool presents a short clinical scenario from which a clinical question is derived. Users are then presented a search strategy and a hypothetical article extract. Users then work through 15 questions (answering yes or no), with each question representing a step in EBM. Items 1-11 assess knowledge and skills relevant to EBM, whilst items 12-15 assess attitudes relevant to the implementation of EBM in clinical practice. The tool has been found to be a reliable and valid instrument to assess medical trainees' competency in EBM (16).

Educational Prescription
Students were also asked to submit an Educational Prescription online through Moodle, our virtual learning environment. For this task, students identi ed a clinical scenario they encountered during their clinical attachments. They then formulated a clinical question in PICO format, presented their search strategy, identi ed the evidence, and carried out a brief critical appraisal before extrapolating ndings to the patient. A marking rubric was developed in house and students were scored out of a total of 20.
Students were given a score of 1-4 for each of the ve categories of EBM, clinical information provided, question formulation in PICO, search strategy, critical appraisal of the article, and application of ndings to the patient. As this task (applying EBM in clinical practice) was relatively new to students, they worked with their peers and submitted the EPs in pairs or groups of 3.
Students' EPs submitted throughout year 3 were graded, and the three highest scoring submissions were shortlisted for presentation at the annual Electives and Evidence in Action conference which was held for Phase II students. The half a day conference had an EBM workshop alongside a session where students were provided information on oversees elective placements. The EBM session involved an interactive seminar led by a guest speaker (from the Centre for Evidence Based action, Oxford), who discussed the application of EBM in clinical practice. We also had talks by clinicians and former UBMS students (who were junior doctors by then) sharing their experience of applying EBM in clinical practice. The nalists were asked to present a role-play of their experience of going through the four steps of EBM during their clinical rotation. The panel for judging students' presentations was formed by three clinical leads from phase II, a junior doctor, and the EBM expert from Oxford. The panellists asked questions of the nalists on various aspects of applying EBM. The best presentation of the nalists was awarded the EBM champion trophy and the other two presentations were given medals. All other EPs submitted were displayed as posters for everyone to view during breaks during the conference. Two of these posters were identi ed for commendation by the panel and were awarded a rosette. In addition, every student who submitted an EP was awarded an e-certi cate, which was helpful for their e-Portfolio.

Summative assessments
Students were required to sit summative written assessments and OSCEs at the end of each year in Phase II; Intermediate Professional Examinations at the end of the junior rotation and Final Professional Examinations at the end of the senior rotation.

Summative written assessment
The summative written exam consisted of three papers: one constructive response paper and two selected response papers. Each paper had 12 question sets, each based around a clinical presentation followed by sub questions. EBM speci c sub questions were integrated into the 12 question sets alongside sub questions from other areas of the integrated curriculum. Each question set was worth 10 points. The total possible score for the written component of the summative assessments was 360 marks (120 marks for constructive response questions and 240 marks selected response questions).
EBM questions within the summative written examinations were usually short answer questions, single best answer style questions, or ' ll in the blank' style questions testing students' knowledge and application of EBM including simple epidemiological calculations and interpreting statistics.

Summative Objective Structured Clinical Examinations (OSCE)
An EBM OSCE station was integrated into the summative end of year four OSCE, which comprised of twelve stations. Each OSCE station was ten minutes long. Students were assessed against four competencies/domains-communication skills, clinical knowledge and problem solving, practical skills and professionalism. During the EBM station, EBM knowledge, skills, and application were assessed. The students were given an abstract to review and asked to explain the results to a worried mum, who thought the study reported a link between MMR and autism. The examiners directly observed the students and gave a score for each domain ranging from 1 till 5 and an overall global score ranging from very poor to excellent.

Study design and setting
This study focused on students who entered the UBMS MBChB course in2016 (n= 65). All students were based in the medical school in Buckingham during Phase I of the MBChB course, -the rst two years of the course when students were taught biomedical sciences. During Phase II of the course, students (n=65) were split across two acute hospitals-. For this study we focused on students' performances in formative and summative assessments in Phase II.

Ethics
Ethical approval for the study was provided by the School of Science and Medicine Ethics Committee. All students were invited to participate in the study and were introduced to the study purpose through a verbal presentation at the beginning of phase I EBM teaching. Further details were provided via the virtual learning environment and students had an option to opt out of the study without giving a reason. Informed written consent was obtained from participants at the start of the study. All participants data was anonymised before analysing the data and they were assured that only anonymised data would be published.

Pragmatic trial
For the clinically integrated EBM teaching sessions, students were based in their respective trusts for the EBM teaching sessions in phase II. The EBM theme lead was based in the active site (hospital 1). There were no EBM teachers for UBMS students in the passive site (hospital 2). Hence EBM teaching was delivered using face to face blended learning approaches by clinicians and the EBM lead at hospital 1(active teaching) and delivered via live telecast to students in hospital 2.

Active teaching method
We used blended learning methods with a mix of lectures, interactive workshops, small group teaching and ipped classroom methods in the main hospital with active teaching, -seminars were followed by small group work with clinical case-based discussions. Small group discussions are facilitated by the Public Health and EBM lead and a clinician.

Passive teaching method
Students in hospital 2 received the same teaching remotely by a live telecast of the seminar. However, the seminar was not interactive for students, instead students discussed the case based small group work remotely and presented online to receive feedback from the teachers. The small group discussions were self-directed and not facilitated by teachers.

Outcome measures
The outcome measure used in this study was students' competency in EBM as assessed using formative assessments and summative assessments. Students' performance in Assessing Competency in EBM (ACE) and Educational Prescription (EP) in third year was reviewed. Students' performance in their nal summative assessments (end of fourth year) was also reviewed. These examinations were chosen as this was 1 year after the main bulk of the EBM teaching including the Electives and Evidence in Action conference. Therefore, these performance in examinations would not only test EBM competencies but also long-term knowledge retention.

Summative assessments-Final Professional Assessment exams
In total, 60 students out of the initial 65 completed their Final Professional Examination (end of fourth year). We had two EBM questions (total three marks) in the short answer paper and three EBM questions in the SBA papers (total three marks). Questions ranged from interpretation of a forest plot, screening de nitions, choice of epidemiological study designs for a given clinical question and describing herd immunity. The data was analysed using the Classical Test Theory, including the mean scores along with their standard deviations. the di culty index and the item total correlation for each question.

Summative assessments-Final OSCEs
The Final Professional Examination OSCE consisted of 12 stations, one of which was speci c to EBM. The EBM OSCE station tested students' interpretation and communication of a scienti c study's ndings to a worried mother whose child had autism. The mum had found an article on the internet that suggested that vaccines might cause delays in child development and she wanted to discuss the study ndings with her GP. The students' performance data was again analysed using the Classical Test Theory, reviewing the mean station domain score and item total correlation Quantitative data analysis A 2 tailed, t-test was used to analyse differences in EBM competency between the students who were taught by the 2 methods in the formative and summative assessments. All analyses were carried out in MS Excel, SPSS v 24 and STATA16. Half a day session with lecture followed by a case-based discussion (small group task) • EBM was taught alongside an acute medicine physician (Emergency medicine) and the drugs and alcohol service provider.
o Students are taught about homelessness-health inequalities, public health challenges alongside drugs and alcohol abuse amongst the homeless.
o Students are encouraged in small groups to design an evidencebased service for the homeless with drugs and alcohol abuse.
o Students are shown a YouTube video developed by their peers on making an evidence-based decision on the most appropriate choice of analgesic for a teenager with meniscal tear.
• Students were encouraged to complete a supervised learning event (SLE) in their e-portfolio demonstrating their application of EBM to clinical practice and re ect on their practice

Results
Of the 65 students, 46 students completed the ACE test-32 from hospital with active teaching and 14 from hospital with passive teaching. There were 31 EP submissions-23 from hospital 1 and eight from hospital 2. Students' performances were better in both tests in hospital 1 (active teaching method) compared to hospital 2 with passive teaching method. The mean difference for performances in ACE and EP were -1.02, -2.20 to 0.16) and -2.28, -4.31 to -0.26) respectively. Table 2 below shows the results of the formative assessments.

Summative assessments-Final Professional Assessment exams
The mean difference for performances between the aggregated EBM score in the written Final Professional Examinations was -0.11, -0.65 to 0.44. Figure 4 shows the box plot distribution of students' scores across the two sites.

EBM OSCE Station data
The mean difference for performance in the EBM questions between the two cohorts was -0.81, -2.38 to 0.74. Figure 5 shows the box plot of students' scores in the EBM OSCE stations across the two sites.

Discussion
This study has provided a description of integrating EBM into clinical placements in an undergraduate medical school and has demonstrated that both active and passive teaching methods were effective models of clinically integrating Evidence Based Medicine (EBM) teaching in undergraduate medical schools. To our knowledge, this is the rst study which has described a multifaceted, clinically integrated EBM curriculum with assessments in an undergraduate medical school. This is, also to our knowledge, the rst study that has compared the effectiveness of active versus passive methods of teaching EBM as evaluated by students' performance in summative assessments.
Students who received the active teaching method performed better in educational prescriptions; however, there was no signi cant difference in their performances in summative assessments. This study has assured us that it was possible to deliver the same EBM content in the active as in the passive teaching models without any detrimental effect in students' long-term knowledge gain.
Teaching strategies that encourage learner-learner interactions will lead to more active engagement among learners (18). Encouraging students to learn together by allowing them to complete the formative assessment (EP) together during clinical practice and peer mentoring at the annual conference where students show their peers how they applied EBM in clinical practice may have had a positive effect in our study. This supports previous studies which have demonstrated the bene ts of scaffolding student learning or spiral approach to EBM teaching; where EBM concepts and applications increase in complexity and are reinforced throughout the curriculum (19)(20)(21). Our teaching model provides a great example of continually shifting the zone of proximal development (Leppink & Duvivier 2016). Small group tasks during teaching -in either the passive or active teaching models can improve student engagement.
Our ndings were similar to the ndings in a systematic review of teaching methods for EBM, which found that the lack of differences in learner outcomes across different teaching methods provided educators with a exible framework for designing an EBM curriculum (20). Educators need to consider feasibility of delivering, acceptability to students and the cost of implementation while choosing the optimum teaching style.
Studies have compared various educational approaches by considering the resource implications and student satisfaction in addition to achievement of learning targets. Web-based educational approaches have been shown to be more e cient than face-to-face education from the perspective of the education provider and produced comparable outcomes for participation, satisfaction, knowledge acquisition and change in practice (22). Active and passive teaching methods have been shown to cover the same amount of content with no difference in knowledge acquisition and attitude enhancement (18). More recently, blended learning approach has shown to be cost effective than the traditional face to face model in training practitioners to be competent in applying EBM (23). In addition to being cost-effective; blended teaching has demonstrated increase in students' attitudes towards EBM and self-reported use of EBM in clinical practice (24).
Our study had its limitations, it was a pragmatic trial; hence it was not possible to either randomise or blind individual students to the interventions. The active teaching approach was resource intensive and needed a lot of planning and commitment. It was feasible in this small teaching hospital with a small cohort of students-whether it is applicable in larger teaching hospitals with bigger cohorts of students is uncertain. EBM is a longitudinal theme in our school and students are introduced to EBM concepts from the rst year (Phase I). In addition to a longitudinal approach to teaching, there is a systematic representation of EBM in all summative assessments. Hence students had a rm grounding in EBM principles, before they moved to different hospitals in Phase II. This could explain why implementing either model for clinically integrated teaching in Phase II had similar long-term knowledge gain in students. Faculty development and lack of trained EBM educators (other than the EBM lead) within the school has been a challenge in our study. The course was designed and delivered by the EBM theme lead who has been a champion and brought in support from few clinicians and external EBM experts. Issues relating to faculty development was not unique to our setting, it has been identi ed as a challenge in the delivery of effective EBM curriculum in other medical schools (25). To address the challenge of faculty development and availability of trained EBM educators, we have tried innovative methods such as use of technology to record our sessions; passive teaching methods relying on student engagement and peer mentoring from students and recent graduates who have also helped in developing YouTube videos. As the medical school grows and takes on more hospitals for student placements, we continue to champion EBM amongst clinicians and junior doctors, as there is a growing need for more role models in clinical placements to encourage students to apply EBM in practice.
The paucity of evidence around EBM education is clear and further evaluation is needed. In addition to quantitative data on effectiveness of the curriculum, we are now gathering qualitative data around students' perceptions of EBM teaching, enablers, and barriers of applying EBM in clinical practice. We are also gathering clinicians' views on our EBM curriculum. We continue to integrate EBM teaching and have designed a supervised learning event in students' e-portfolio where they can re ect on their application of EBM knowledge and skills in practice, encouraging lifelong learning skills required for continues use of EBM into students' postgraduate clinical practice. The EBM lead has also encouraged the establishment of student-led academic medicine society. The society will provide further training in EBM for students and promote the research environment in the medical school.
In designing teaching methods for clinically integrating EBM, educators need to balance resource implications, students' preferences, and impact on learning outcomes. Further research into the minimum components needed for multifaceted interventions and the cost effectiveness analyses of the two models of clinically integrating EBM teaching is needed.

Conclusions
This study has demonstrated that it was feasible to design a multifaceted, clinically integrated EBM teaching with assessments in undergraduate medical school. Both active and passive teaching methods were equally effective for clinically integrating EBM into an undergraduate medical school curriculum.
While the active teaching model seemed to be more effective as observed by the students' performances in educational prescriptions; there was no signi cant difference between the two models in the students' performances in the ACE tool or in the summative assessments. Ethical approval for the study was provided by the University of Buckingham School of Science, Medicine and Dentistry Ethics Committee. All students were invited to participate in the study and were introduced to the study purpose at the beginning of Phase II EBM teaching. Informed written consent was obtained from participants at the start of the study.  Box plot showing the students' performances in written assessments across the two sites