Study Design and Participants
Overall, 67 (female n = 49; male n = 18) 4th year dentistry students on a five-year program without previous experience in the field of Oral and Maxillofacial Surgery were assigned quasi-randomly to a ‘Mental Training’ group and a ‘See One, Do One’ group which was regarded as a control. Participation in the study was voluntary and took place after written informed consent, which was revocable at any time. Students were blinded in relation to their knowledge of the didactic principles used during their training as well as affiliation to any study group. Basic data regarding student age, sex, and duration of study were collected using a questionnaire.
The study was conducted according to the ethical principles of the Helsinki Declaration (Ethical Principles for Medical Research Involving Human Subjects), and the local ethics committee noted that no further approval was necessary.
Assignment of the Students to the Instructional Approaches
The assignment of students to one of the learning groups with a maximum of 6 students per week occurred prior to the Oral and Maxillofacial Surgery apprenticeship independent of the authors and independent of study participation by the dean’s office. Groups were then assigned alternately within the 10-week span of the apprenticeship to one of the instructional approaches. This quasi-randomized allocation of students was chosen due to the curricular ‘in-vivo’ study design and intended to produce similar groups.
Study Protocol
The study was carried out within the Oral and Maxillofacial Surgery apprenticeship for dentistry students, which includes a five-day rotation through every section of the Department of Oral, Cranio-Maxillofacial and Facial Plastic Surgery, i.e. the operative room, the outpatient clinic or the emergency department. Before starting their rotation, students have to complete a practical skills training [25]. The aim is to give dentistry students a short overview of the most common reasons for consultation in Oral and Maxillofacial Surgery and prepare them for the upcoming clinic rotation. It is divided into a theoretical part (240 minutes) in the morning and practical skills training (240 minutes), during which the study took place, in the afternoon. Trained practical skills include performing a structured facial examination, placing a venous catheter in the context of an emergency treatment and placing an ‘Ernst ligature’ on a phantom model. Lessons were held in small groups ranging from five to six students.
‘See One, Do One’ Approach
As quality assurance and standardization for the demonstration of a structured facial examination, the skill was videotaped based on the existing manual and checklist. The trainer demonstrated the video and explained the performance of a structured facial examination step-by step in detail. The explanations were predetermined in the manual and trained in the tutor training, which was identical for both approaches. Subsequently, students practiced the skill on each other under supervision and, if needed, correction of the tutor. Each student was advised to perform the structured facial examination at least once. In total the training lasted for 60 minutes.
‘Mental Training’ Approach
For the ‘Mental Training’ approach, the students received the same standardized video instruction of a structured facial examination. Subsequently, students practiced the skill under the supervision and, if needed, correction of the tutor for 30 minutes. Students then actively developed an individual detailed, stepwise sequence of the performance of a structured facial examination under the supervision and review of the tutor. Students were free to divide the skill into individual sub-steps which they noted on index cards. This process of creating an individual ‘mental map’ was supervised by tutor who would integrate missing steps or correct the sequence, if necessary. Students then visualized and internalized every step of the individual mental map in subvocal for 30 minutes. In total the training lasted for 60 minutes.
Performance Measurement
In order to assess the acquired competence in performing a structured facial examination, the OSCE-format was used during the training week directly after the intervention in the last 60 minutes of the practical skills training (T1) as a single station OSCE and five to ten weeks later (T2) as part of a curricular and formative Oral and Maxillofacial Surgery OSCE (8 stations in total). A three point scale scale was used (0 points for not done, 1 point for done, but incorrect, 2 points for done and correct) for the checklist, which was based on the checklist used in the tutor manual (Supplement 1). In total, the checklist consisted of 24 items, which equals a maximum score of 48 points. The checklists implemented had been primarily piloted in previous undergraduate trainings. In addition, the content validity was ensured through its creation as part of an expert workshop with didactic and surgical experts as well as through repeated application and adaption in the context of previous studies [25, 26] and OSCE exams. A timeframe of 5 minutes to complete each OSCE station was given.
Both at T1 and T2, students were video-recorded (Camera System: Panasonic HC-X929) for later performance measurement by two independent, blinded examiners with different levels of experience (Rater 1: Oral and Maxillofacial Surgery resident in year four of a five-year training; Rater 2: Oral and Maxillofacial Surgery resident in year three of a five-year training).
Data Analysis
Microsoft Office 2016 (Microsoft Office 2007, © Microsoft Corporation, Redmond, USA) for Mac and SPSS Statistics version 19 (IBM, Armonk, USA) were used for the statistical analysis and graphical display of data.
To test for a normal distribution of the data, the Shapiro-Wilks-Test was used. Since the test results for both groups were not normally distributed at all times, the Mann-Whitney-U-Test for non-parametric data was used to test for significant differences in learning success in the inter- and intragroup comparison at T1 and T2. Furthermore, effect sizes were calculated for T1 to T2 using Cohens d. Cohen's d is defined as the difference between two means divided by a standard deviation for the data resulting in an unitless value that helps to interpret the effect size of observed results and hence the statistical power of a study. For most types of effect sizes, a larger absolute value indicates a stronger effect. Since the sample size (n = 67) of our study was relatively small, Cohen’s d was used as an additional control test since prior studies have shown significant test results alone are not sufficient to interpret data and draw conclusions from this data [28]. The inter-rater reliability was measured using Pearson’s correlation coefficient (r).
Since there was a gender imbalance between female and male study participants, a gender analysis was conducted to investigate the effect of this imbalance on the outcome of the different tests, comparing the results of all female and male study participants.
Sample Size Estimation
Based on prior examination results from the years before the intervention, we estimated an average student performance of 70 % with a standard deviation of 10 % in the OSCE. Based on the following parameters (Mean ‘Mental Training’ = 33, Mean ‘See One, Do One’ = 30, SD = 5, alpha = 0.05, beta = 0.2) a sample size of 88 was calculated.