Pre-placement learning materials
95% of students (N=93) volunteered whether or not they had done pre-placement work (Table 1). Students who had been given specified written or video materials were more likely (p=0.05) to prepare for the placement (87%) than those who had not (70%). Whether these resources were video or written had no significant impact on the likelihood that they would be used (p=0.11). Of the 33 students not sent specified resources 61% (N=20) identified and read their own written materials and 9% (N=3) identified and viewed video materials. In an as-treated analysis students identifying their own written resources performed worse in both initial and final knowledge tests compared to students receiving specified written resources (Figure 1).
Of the students claiming completion of pre-placement work the time spent was not significantly affected by having had resources specified to them (Table 2). For this comparison students denying completion of pre-placement work were assigned a time of zero minutes. The difference in score on the initial knowledge test was also not significantly different between the two groups (p=0.15). However, a five point agreement Likert scale with the statement ‘my knowledge of ophthalmology is adequate to pass my finals examination’ showed students’ perception of their own knowledge was improved by pre-placement materials (p=0.005). Final knowledge and skills test scores, final self-reported knowledge ratings and supervisor reported knowledge were all found to be significantly higher in students who had received specified pre-placement learning materials (Table 2). However, no significant differences were found in knowledge retention immediately after seminars. Supervisor reported student engagement levels were also found to be equivalent between groups (p=0.78).
|
No PPR
|
95% CI
|
Specified PPR
|
95% CI
|
p value
|
Mean time on pre-placement work (mins)
|
51
|
43.3-58.7
|
61.9
|
39.0-84.8
|
0.15
|
Mean initial knowledge test score (%)
|
56
|
52.3-59.7
|
60.4
|
56.8-64.0
|
0.12
|
Mean initial self-reported knowledge rating/5
|
1.7
|
1.4-2.0
|
2.3
|
2.1-2.5
|
0.005*
|
Mean supervisor knowledge rating/10
|
6.4
|
5.7-7.1
|
7.5
|
7.0-8.0
|
0.006*
|
Mean supervisor engagement rating/10
|
8.6
|
8.2-9
|
8.5
|
8.1-8.9
|
0.78
|
Mean post-seminar interactive test score (%)
|
70.4
|
61.6-79.2
|
74.5
|
70.5-78.5
|
0.89
|
Mean final knowledge test score (%)
|
63.6
|
59.3-67.9
|
74.8
|
72.4-77.2
|
<0.001*
|
Mean final self-reported knowledge rating/5
|
3.8
|
3.5-4.1
|
4.3
|
4.1-4.5
|
0.001*
|
Mean OSCE score (%)
|
81.3
|
77.2-85.4
|
86.9
|
83.9-89.9
|
0.04*
|
Table 2. Table showing differences in performance between students who received specified pre-placement learning materials and those who did not. p values calculated using two-tailed t-tests for parametric data and Mann-Whitney u-tests for non-parametric data. OSCE = objective structured clinical examination. CI = confidence interval. PPR = pre-placement resources. * = statistical significance defined as p<0.05.
Video versus written pre-placement learning materials
Of the 33 students receiving written pre-placement learning materials 82% (N=27) used them whereas 69% (N=22) of the 32 students receiving video materials made use of them. Of the five parameters found to be significantly impacted by specifying learning resources of some kind, only the end of placement knowledge test score was significant when comparing video and written pre-materials (Table 3). Students receiving written materials scored a mean of 77.7% compared to 72.0% among students receiving video materials (p=0.03). With an as-treated analysis, students that went on to use video materials had a mean OSCE score of 90.4% (95% CI: 86.4-90.4%) compared to 83.6% (95% CI: 80.1-87.1%) for students who went on to use written materials (p=0.03).
|
Video LRS
|
95% CI
|
Written LRS
|
95% CI
|
p value
|
Mean initial self-reported knowledge rating/5
|
2.4
|
2.1-2.7
|
2.1
|
1.8-2.4
|
0.14
|
Mean supervisor knowledge rating/10
|
7.5
|
6.9-8.1
|
7.6
|
6.9-8.3
|
0.70
|
Mean final knowledge test score (%)
|
72.0
|
68.9-75.1
|
77.7
|
74.3-81.1
|
0.03*
|
Mean final self-reported knowledge rating/5
|
4.2
|
3.9-4.5
|
4.4
|
4.2-4.6
|
0.47
|
Mean OSCE score (%)
|
88.1
|
84.0-92.2
|
86.0
|
81.6-90.4
|
0.53
|
Table 3. Comparison of the impact of specifying video and written pre-placement learning materials in parameters where specifying written or video learning materials was found to have a significant impact. p values calculated using two-tailed t-tests for parametric data and Mann-Whitney u-tests for non-parametric data. OSCE = objective structured clinical examination. CI = confidence interval. LRS = learning resources specified. * = statistical significance defined as p<0.05.
This suggestion of video materials’ superiority for skill acquisition was supported by student perception of the relative value of these modalities. 53 students (54%) completed 10-point Likert questionnaires demonstrating their agreement with four statements; ‘video/written learning materials are most effective for knowledge/skill acquisition’. Using a paired sign test, on account of the non-parametric and skewed nature of the data, there was no significant difference in student perception of video and written materials for knowledge acquisition. However, video materials were considered significantly better for skill acquisition (Table 4).
|
Knowledge
|
Skill
|
Mean
|
95% CI
|
p value
|
Mean
|
95% CI
|
p value
|
Written materials
|
5.4
|
4.9-5.9
|
0.12
|
4.8
|
4.3-5.4
|
<0.001
|
Video materials
|
5.8
|
5.3-6.3
|
6.6
|
6.2-7.1
|
Table 4. Student reported perception of video and written learning material efficacy for knowledge and skill acquisition. Data are taken from ten-point Likert scales with a higher score representing greater perceived efficacy. p values are calculated with paired sign tests.
Student performance prediction
There was a significant positive correlation (r = 0.316) between the time students claimed they had worked prior to their placement and their performance at the end of placement knowledge test (p=0.003). When applying an as-treated analysis to these data separately for video and written resources, the correlation coefficients were 0.455 (p=0.06) and 0.239 (p=0.13) respectively (Figure 2). There was also a positive but non-significant relationship between supervisor impression of student knowledge and performance in the end of placement knowledge test (p=0.13).