Procedure
All students at the medicine programme at Uppsala University receive training in CS during the first, fourth and eighth term. During the eighth term, a three-day course, Communication skills in challenging situations, is held. The course comprises lectures on CS, such as using active listening in challenging situations, showing empathy and using MI, as well as theory on conversation skills needed for patient-centred communication in challenging situations. Following the lectures, students roleplay with peers in small groups (up to 10 students) to practice the skills. A new teaching method for training communication with SPs (actors) in case simulations was introduced during the autumn of 2019. This was done to increase the opportunities for students to practice their skills in challenging situations. At the end of each term, the medical students evaluate the courses taken by grading them. These data were included to achieve the present study’s first aim (i.e., comparing student evaluations before and after the simulation training with SPs was introduced). During the eighth term, students have a practical examination, consisting of a clinical examination in psychiatry where each student acts as the physician in a conversation with a SP (an actor) and performs a full consultation. The examination is filmed, if the student permits this. To achieve aim 2, video recordings from the practical examinations were used for assessing CS in medical students, before and after the implementation of the simulation training with SPs. Lastly, to achieve aim 3, students participating in the new teaching method for training communication with SPs in case simulations during autumn 2019 were invited to be interviewed in focus groups about their experiences, in early 2020.
Participants
All students in the medicine programme are asked to give written evaluation, see below, on all courses taken. However, there is large attrition and many students do not complete these forms. For this study, we used the evaluations from spring 2019 (before simulation training, 57 included out of 105, response rate = 54%) and autumn 2019 (after simulation training, 36 included out of 84, response rate = 43%). Video recordings from practical examinations from two different terms, before and after the simulation training had been introduced (N = 183), were initially included. Of these, 70 videos were excluded because the actor was portraying a patient suffering from schizophrenia, since this would usually initiate less use of CS, 30 were excluded due to lack of written consent for permission to use the film for educational development, and in 10 cases the video was incomplete. Thus, 73 films (40%) were included, of which 46 (15 males and 31 females) films were with students before the new training with SPs was implemented and 27 (13 males and 14 females) were from after simulation training with actors. All students from the autumn term 2019 were invited to focus groups, of whom N = 23 (27%) agreed to participate (split into three focus groups). Thus, the study sample was N = 93 for the first aim, N = 73 for the second aim, and N = 23 for the third aim. The study was approved by the Swedish Ethical Review Authority, Dnr 2019-05908.
Training with SPs
The simulations were conducted in groups of six to eight students, one SP (a professional actor who has been trained to portray patients consistently), and one supervisor for each scenario. The student groups moved between four rooms, each of which presented a different case. The student groups spent 90 minutes in each room and two to four students got the chance to act as the doctor for each specific case. The four cases were: Telling an elderly woman with aggressive colon cancer that she is dying (using empathic responses); handling negative patient reactions with a woman who is unhappy with the care she has received from her doctor and trying to find a way forward (using active listening); discussing a rehabilitation plan with a reluctant middle-aged man with back pain who has been on paid sick leave (using MI tools and active listening); and motivating a reluctant young man with alcohol problems to change his lifestyle (using MI tools and active listening). For a more detailed description of the cases, see Supplementary Table 1. The students who were not active as the doctor in the simulations were given the task to observe and give feedback to their peers. The student who was acting as the doctor received immediate feedback from the supervisor, who was either an experienced physician or an experienced psychologist, as well as from their peers and the SP.
Measures
Yearly evaluation form
Out of the eight questions (with six sub-questions) that are included in the evaluation form, we chose four items deemed as most relevant for the study aim: ‘What is your overall review of the course?’, ‘How satisfied are you with the seminars/work in groups?’, ‘Too what degree do you think the teachers have given you valuable feedback on your work?’, and ‘Too what degree do you think you were given the opportunity to be active?’. All items were rated from 1 = ‘not (content) at all’ to 6 = ‘very content/much’.
Video recordings of consultations (use of CS)
The six different cases played by SPs included in the practical examinations represented different disorders, such as depression, peripartum depression, bipolar disease in manic phase, anxiety and alcoholism. The patient cases were designed to be challenging in various ways for the student acting as the physician (e.g., hostile patient, patient reluctant to change unhealthy behaviour, patient burdened with guilt, patient having doubts about treatment). The consultation part of the examination was limited to 20 minutes, after which the student was asked by the examiner to give an evaluation of the patient’s psychiatric status and report his/her clinical evaluation. The whole consultation was examined and CS were evaluated. There are several rating checklists for CS described in the literature, but no gold standard [34]. In this study, the videotapes from the recorded examinations were coded with a scoring system using behavioural counts based on patient-centred interviewing [4], created specifically for this study. The scoring system included a checklist with three main categories: Active listening (encourages, reflections, summarises, asks open-ended and closed-ended questions, asks for permission), empathy (verbalises the patient’s feelings, validates and normalises, shows support and care), and motivational efforts (explores what is positive/less positive about the current situation, what is less positive/positive about changing the current situation, clarifies the need for change and how change will happen), see Supplementary Table 2. Each time a behaviour was shown, a mark was made in the protocol and when the film ended, all behaviours within each category were added together.
There were two exceptions. The first was within Active listening, where the sub-category ‘Closed-ended questions’ was not added to the others. Instead, the relation between the sub-categories ‘Open-ended questions’ and ‘Closed-ended questions’ was calculated as a quota. The second was within Empathy, in the sub-category ‘Shows support and care’, where students could receive a maximum of five points for five pre-defined types of behaviour. To ensure reliable ratings, JK and MR initially reviewed and rated five videos independently to evaluate interrater reliability. MR is a psychiatrist and professor teaching CS and JK is a medical student in term ten, with two years’ experience of teaching CS to medical students in terms one and four. After these five scorings, a high interrater reliability could be shown with excellent intra-class correlation coefficients (ICC single measures .89 (.83, .92), p < .001). When interrater reliability had been established, the remaining 68 videotapes were scored by JK. The coding of these films generated a count for each communication skill, with a higher score indicating greater skills.
Focus groups
Participants were invited to discuss how they experienced participation in the simulation training with actors. The interview guide included the following questions: ‘How did you experience participating in the simulation training?’, ‘What do you perceive you were training?’, ‘How did the simulation training change your communication skills?’, and ‘How did the simulation training affect your patient relations?’. Follow-up questions were asked when appropriate. The interviews were audio-recorded and lasted for about 32–48 minutes, and were then transcribed verbatim and pseudonymised with sex. The interviews were performed by one graduate student (JK) and one psychologist trained in CS. The interviewers did not participate in the education/simulation training and were trained in interviewing techniques by one of the authors (JI) who is experienced in this method.
Data analyses
Results are presented as means and standard deviations. For the first aim, independent sample (Student’s) t-tests were used for comparing ratings on the evaluation form between those who had the simulation training with actors and those who had only roleplay with peers. For the second aim, a multivariate analysis of covariance (MANCOVA) was performed using the Statistical Package for the Social Sciences (SPSS-26), in order to assess differences in the dependent variables active listening, the quota open-ended/closed-ended questions, empathy and total use of CS (i.e., summation of active listening, motivational efforts and empathy) with before and after training as independent variables. We also adjusted for covariates such as sex, length of videotape and SP case in the MANCOVA. The MANCOVA yields partial eta squared (η2), a common metric of effect size that represents the unique amount of variance explained by each independent variable, with η2 = .01 being regarded as a small effect size, .06 as medium and .14 as large. Motivational efforts were not included as a dependent variable in the MANCOVA due to the extremely low use of these skills, both before and after training. Normal distribution could be seen, except as regards the quota, but the same results were generated for non-parametrical and parametrical values. Two-tailed tests with p-values < .05 were considered to be statistically significant.
For the third aim, a qualitative content analysis [35] was carried out using an inductive approach where the categories were derived from the data [36]. The analysis was conducted by two of the authors (JI & MR), who continuously discussed and confirmed the findings. First, the material was read to achieve an optimal understanding of the content. Second, all meaning units, defined as one or more sentences or just part of one sentence, carrying a meaning connected to the research question, were extracted. Third, the meaning units were shortened to their essence. Fourth, text units with similar meaning were grouped into mutually exclusive categories. Fifth, categories were divided into subcategories based on dissimilarities within the categories. To increase the rigor of the analysis, the interview text was reread and the categories and subcategories were compared and validated against the text.