Study Design and Participants
Overall, 67 (female n = 49; male n = 18) 4th year dentistry students on a five-year program without previous experience in the field of Oral and Maxillofacial Surgery were assigned quasi-randomly to a ‘Mental Training’ group and a ‘See One, Do One’ group which was regarded as a control. Participation in the study was voluntary and took place after written informed consent, which was revocable at any time. Students were blinded in relation to their knowledge of the didactic principles used during their training as well as affiliation to any study group. Basic data regarding student age, sex, and duration of study were collected using a questionnaire.
The study was conducted according to the ethical principles of the Helsinki Declaration (Ethical Principles for Medical Research Involving Human Subjects), and the local ethics committee noted that no further approval was necessary.
Assignment of the Students to the Instructional Approaches
The assignment of students to one of the learning groups with a maximum of 6 students per week occurred prior to the Oral and Maxillofacial Surgery apprenticeship independent of the authors and independent of study participation by the deanery. The assignment of the learning groups in the study to the ‘Mental Training’ instructional approach and the traditional ‘See One, Do One’ group took place alternately within the 10-week span of the apprenticeship.
Study Protocol
The study was carried out within the Oral and Maxillofacial Surgery apprenticeship for dentistry students, which includes a five-day rotation through every section of the Department of Oral, Cranio-Maxillofacial and Facial Plastic Surgery, i.e. the operative room, the outpatient clinic or the emergency department. Before starting their rotation, students have to complete a practical skills training [25]. The aim is to give dentistry students a short overview of the most common reasons for consultation in Oral and Maxillofacial Surgery and prepare them for the upcoming clinic rotation. It is divided into a theoretical part (240 minutes) in the morning and practical skills training (240 minutes), during which the study took place, in the afternoon. Trained practical skills include performing a structured facial examination and placing a venous catheter. Lessons were held in small groups ranging from five to six students.
‘See One, Do One’ Approach
This approach is labelled as the main component of clinical-bedside teaching. Students learn by watching an expert explaining and demonstrating a skill. This is followed by the first independent performance of the skill, which is mostly with a patient [7]. As quality assurance and standardization for the demonstration of a structured facial examination, the skill was videotaped based on the existing manual and checklist. The trainer demonstrated the video and explained the performance of a structured facial examination step-by step in detail. The explanations were predetermined in the manual and trained in the tutor training, which was identical for both approaches. Subsequently, students could practice the skill on each other under supervision and, if needed, correction of the tutor. Each student was advised to perform the structured facial examination at least once. In total the training lasted for 60 minutes.
‘Mental Training’ Approach
For the ‘Mental Training’ approach, the students first received a standardized video instruction that explained the performance of a structured facial examination step-by step in detail. The trainer demonstrated the video and explained the performance of a structured facial examination step-by step in detail. The explanations were predetermined in the manual and trained in the tutor training, which was identical for both approaches. After that, students actively developed a detailed, stepwise sequence of the performance of a structured facial examination under the supervision and review of the tutor on index cards. Thus, they could, furthermore, visualize and internalize the sequence of the skill in subvocal. Students were enabled to set their points of intersection individually. Based on the manual and the checklist, the tutor had a detailed, stepwise description of the skill and the desired algorithm. Thus, the tutor’s role was to guide the group and integrate missing steps or correct the sequence, if required. Students were asked to imagine the skill step-by-step. Subsequently, students could practice the skill under the supervision and, if needed, correction of the tutor. Each student was advised to perform the structured facial examination at least once before mentally training in the skill for 30 minutes. In total the training lasted for 60 minutes.
Performance Measurement
In order to assess the acquired competence in performing a structured facial examination, the OSCE-format was used during the training week directly after the intervention (T1) and five to ten weeks later (T2) as part of curricular and formative Oral and Maxillofacial Surgery OSCE (8 stations in total). A trinary scoring scale was used (0 points for not done, 1 point for done, but incorrect, 2 points for done and correct) for the checklist, which was based on the checklist used in the tutor manual (Supplement 1). In total, the checklist consisted of 24 items, which equals a maximum score of 48 points. The checklists implemented had been primarily piloted in previous undergraduate trainings. In addition, the content validity was ensured through its creation as part of an expert workshop with didactic and surgical experts as well as through repeated application and adaption in the context of previous studies [8, 29, 30] and OSCE exams. A timeframe of 5 minutes to complete each OSCE station was given. Subsequently, students received brief oral feedback regarding their performance and suggestions for improvement at T1. During their training week, students were video-recorded (Camera System: Panasonic HC-X929) for later performance measurement by two independent, blinded examiners with different levels of experience (Rater 1: Oral and Maxillofacial Surgery resident in year four of a five-year training; Rater 2: Oral and Maxillofacial Surgery resident in three four of a five-year training). The inter-rater reliability was measured using Pearson’s correlation coefficient (r). During their Oral and Maxillofacial Surgery OSCE, performance was again video-recorded and measured by the same two independent examiners. All examiners were blinded towards the students' instructional approach and affiliation of learning group. They received training prior to the OSCE in order to gain experience in the use of the checklist.
Data Analysis
Microsoft Office 2016 (Microsoft Office 2007, © Microsoft Corporation, Redmond, USA) for Mac and SPSS Statistics version 19 (IBM, Armonk, USA) were used for the statistical analysis and graphical display of data.
To test for a normal distribution of the data, the Shapiro-Wilks-Test was used. Since the test results for both groups were not normally distributed at all times, the Mann-Whitney-U-Test for non-parametric data was used to test for significant differences in learning success in the inter- and intragroup comparison at T1 and T2. Furthermore, effect sizes were calculated for T1 to T2 using Cohens d. Cohen's d is defined as the difference between two means divided by a standard deviation for the data resulting in an unitless value that helps to interpret the effect size of observed results and hence the statistical power of a study. For most types of effect sizes, a larger absolute value indicates a stronger effect. Since the sample size (n = 67) of our study was relatively small, Cohen’s d was used as an additional control test since prior studies have shown significant test results alone are not sufficient to interpret data and draw conclusions from this data [28].
Since there was a gender imbalance between female and male study participants, a gender analysis was conducted to investigate the effect of this imbalance on the outcome of the different tests, comparing the results of all female and male study participants.
Sample Size Estimation
Based on prior examination results from the years before the intervention, we estimated an average student performance of 70% with a standard deviation of 10% in the OSCE. Based on the following parameters (Mean ‘Mental Training’ = 33, Mean ‘See One, Do One’ = 30, SD = 5, alpha = 0.05, beta = 0.2) a sample size of 88 was calculated.