This pragmatic single centre study shows that the provision of learning materials prior to undergraduate ophthalmology clinical placements significantly improved subjective and objective measures of knowledge and skill performance. This effect is likely due to both the demonstrated increased engagement with pre-training by students when materials were specified and also the higher efficacy of teacher selected materials relative to student selected materials. As few students in our study independently sourced video materials we were unable to evaluate the efficacy of student sourced video materials. However, evidence from ophthalmology and other areas of healthcare suggest that clinical skills videos selected to meet student needs are also more effective than video materials that students source independently.17,18 Our data suggest that written materials yield better results on knowledge tests than video materials if student time is unlimited. However, if students are only willing or able to commit a short period of time to self-directed learning it seems that video materials may have a greater yield (Figure 2). This is consistent with a randomised controlled trial (RCT) in ophthalmology undergraduate education from Steedman et al which demonstrated equivalent test results following a shorter work duration from students using multimedia learning compared to textual materials alone.15 Whilst intention-to-treat analysis did not find a significant difference in OSCE performance between recipients of video and written materials, a comparison based on what materials individuals actually went on to use did suggest an objective advantage in clinic skill learning for video materials. This is also supported by subjective data that video materials are more effective than written materials in helping learners acquire skills (Table 4). These findings are consistent with work in endocrinology showing improvement in undergraduate clinical skill performance with the availability of specified video materials.19
The proof of our hypothesis that pre-placement materials improve learning efficiency on an undergraduate ophthalmology placement is informative but not unexpected. The real value of the outcome is the reproducibility of the means by which it was achieved. To minimise barriers to teachers implementing similar approaches both the welcome pack and learning resources used are publicly available (supplementary materials 1-3).16 In considering feasibility of the distribution of pre-placement materials at any given institution it is also important to address the barriers to success from the learner perspective. Reid et al recently performed thematic analysis on interviews with Irish medical undergraduates regarding a year of self-led e-learning.20 The three major themes of barriers to engagement were a sense of being cheated out of higher quality traditional teaching methods, the ease with which attention could fail to be paid to audio and video materials and a sense of being overwhelmed at a large bank of seemingly unstructured materials.20 We attempted to address these barriers by making our learning materials an adjuvant rather than an alternative to traditional teaching and by requesting a short period of commitment from learners. The mean of 62 minutes (median = 55, interquartile range 10 - 90) that students who received pre-placement materials claimed to have spent compared to the 90 minutes requested of them suggest these attempts were at least in part successful. Student feedback placed great value on the welcome pack that had been sent one week prior to attending the placement, as it gave them an early sense that their learning had been carefully considered. This helped to persuade them into committing their own time. It may be that a more refined pre-placement welcome pack could win even greater time commitments from student and elicit further learning benefits. However, student engagement may not be so readily won if pre-placement materials become ubiquitous for all medical student placements. If this proves to be the case it may be best to prioritise areas of the curriculum where student exposure is limited.
In considering the application of these data it is important to appreciate that the specialty of ophthalmology is just one of many areas in the undergraduate medical curriculum where student exposure is limited. Plastic surgery, neurosurgery, otolaryngology and cardiothoracic surgery are all examples of specialties with little presence in timetabled teaching where students could benefit from an increase of teaching efficiency if not quantity.21-24 Another transferrable theme that was frequently voiced by students in open space feedback through this study was the perceived benefit of the provision of a clinical member of staff dedicated to their teaching. These student comments are consistent with UK student surveys on clinical teaching fellows who are perceived to deliver higher quality teaching, be more punctual and have a clearer understanding of students’ learning needs compared to full time clinicians.25,26 In our experience the service provision capacity of the department was also improved by the introduction of a teaching fellow as more experienced clinical staff, who have previously shared teaching responsibilities, were no longer required to be withdrawn from clinical duties. If the teaching demands on a department are great enough, as in this case, the consequent increase in clinical capacity can offset the cost of employing a full or part time teaching fellow. Disruption to service provision can be reduced while teaching quality is improved at little or no cost.
The study was limited by its design as a longitudinal quality improvement project. This was chosen over a RCT as it would not have been feasible to stop students sharing learning materials with peers in different groups; a recognised limitation of RCT in medical education research.27 Our approach also meant that the whole year group was able to access learning resources as our links to learning resources were developed, albeit at varying time points relative to their placement. The study design also prohibited any control over the cohort size as it was built around an established annual teaching programme. Any attempt to extend beyond a year would have raised issues of bias as the teaching fellow delivering the course changes annually and would also introduce ethical issues, as learning materials with proven efficacy would be withheld from students in the second year. This limits the external validity of the study as only statistically significant differences are reported rather than prospectively defined defined ‘clinical’ significance.
The reader should also consider the discrepancy between what learning materials were sent to students and which materials they went on to use. We chose to analyse the variable within the control of the education staff, namely the materials sent to the students, in order to maximise the external validity of our data. It is also true that the self-reported outcomes, such as material type used, time spent working and perceived knowledge levels cannot be assumed to be accurate. This limitation is likely to be greatest for the claims students made of how much time they worked, as this is likely to be influenced by other factors such as student confidence and the degree to which they did not want to disappoint teaching staff. Whilst this may limit the accuracy of the measure, these confounding factors ought to have been distributed randomly between the three groups so should not bias the outcome. An objective means of measuring the quantity and quality of pre-placement work each student performed would be technically and ethically challenging and would impose bias from the Hawthorn effect. For self and supervisor reported knowledge outcomes, it is true that they do not hold the same objectivity as test scores but student confidence and esteem is also an important outcome of education and these outcomes complement rather than duplicate the test scores reported. When considering the analysis some of the significance demonstrated is not maintained following Bonferroni correction. The risk of type 1 error should be borne in mind but amongst many others our primary outcome of final test score remains significantly dependent on pre-placement material provision following Bonferroni correction.
This pragmatic comparative study found that the provision of specified learning materials prior to a short ophthalmology clinical placement augmented student performance at its conclusion. Written learning materials improved knowledge test performance to a greater extent than video materials, whilst video materials were superior for clinical skill development. Judicious selection of an appropriate learning material format for given learning outcomes can optimise the educational efficiency of teacher time expenditure.