Main findings
We successfully developed four competence cards covering the most important learning objectives and medical encounters to be used by medical students in fourth year. The portfolio system introduced systematic training and formative evaluation of competence. This was followed by constructive feedback in a few well-defined clinical areas of competence, which gave the students an opportunity to build an understanding of, and confidence in, the content of the roles. Overall, the medical students’ and the clinical supervisors’ reception were positive.
The medical students’ and the clinical supervisors’ overall perception
The medical students found the structured, formative evaluation helpful. It made it easy to ask for supervision and feedback. Overall, they preferred the portfolio from the existing logbook. Our study supports the benefit of introducing a portfolio mindset already in the pre-graduate education(31, 32). Medical students need to grow during their clinical rotations to be ready to perform tasks first with supervision and after training on their own (33).
The interviews gave us valuable comments about the competence cards, both the content and the grading of the feedback. Overall, they found the content relevant, and representative of the learning objectives provided from the university. The content agrees with the undergraduate medical education literature(33).
Initially we used a Likert scale consisting of statements. This was changed to numbers after receiving feedback from the students, telling us, it was easier to visualize improvement this way, especially when entering a new rotation. The electronic version was favoured, but suggestions for how to improve were provided. The benefit of an electronic version is the registration of the competence evaluations.
It is important to mention, that the medical students found peer-to peer evaluation valuable. They found it safe to ask a peer to do the evaluation, even though they did not know each other well. They perceived the learning environment as safe and were open to feedback from peers. This is in alignment with studies showing the positive effect of peer-to-peer teaching(34).
The students were evaluated twice by a doctor, the first time in the middle of their stay and the second time at the end. Both students and clinical supervisors mentioned that it was unnecessary to repeat the evaluation if the student fulfilled the criteria in the middle of their stay. However, the students mentioned, that if they were interested in being evaluated again, for instance in a more challenging situation, it should be possible. The relative low total number of competence evaluations conducted in this study indicate that our portfolio was too big for a five-week rotation. Hence, either the number of competence cards or the number of repeated evaluations should be reduced. Building on the suggestions from the students and supervisors we intend to reduce the number of evaluations. This is in alignment with studies in postgraduate training, describing how programs need to adapt if the burden of assessment is too high(35).
The students mentioned differences in how feedback was provided. Some preferred feedback from junior doctors whereas others preferred feedback from senior doctors. One of the explanations might be that the juniors remembered what was difficult as a medical student and directed the feedback towards that, whereas the seniors provided more detailed feedback and were sometimes more critical. Concerns have been raised in the literature that formative assessment can be reduced to tick-box exercises(35). Our study does not confirm this.
Implementation
The heads of department appointed coordinators, who found it easy to implement the portfolio for students, as they were familiar with evaluation of competence in specialist training of doctors. They were committed to introduce competence based medical education to pre-graduate education. A recent publication has shown that this can be difficult due to misaligned goals and in-adequate communication (36). To secure implementation of the project in a busy clinical department, it is necessary to visit the department regularly and support all participants to be sure that the evaluation of competence is done.
One of the coordinator’s recommendations was to make the evaluation of competence simpler and more aligned with the evaluation of doctors, which means that the doctor in training prepare for the evaluation and ask for the evaluation when they find them-selves ready to pass. They thought that it would be more meaningful and at the same time simpler. The coordinators found that a train-the-trainers course was unnecessary, but based on the students’ feedback, we found that a train-the-trainers course was necessary.
The supervisors and the coordinators mentioned that this initiative would not only be of help for medical students and supervisors but be of benefit for the patients.
Discussion of the methods
The study followed a well-recognised method, Kern’s six steps, for developing an educational activity. The portfolio was developed by clinicians and supported by professor and associate professors in medical education. After the first cohort, a few changes were made in some of the competence cards. They were in a generic format, which means that they can be used at all departments of internal medicine and surgery at this level of education. Hence the generalisability is high. We find it a strength that three cohorts of medical students from six different departments were included, as it has been possible to follow different implementation strategies. In addition, that each cohort of medical students were interviewed at the end of their stay, which made it possible to adapt for the next cohort.
Future studies
In conclusion, clinical rotations can be utilized better in strengthening the development of several of the seven doctor roles and develop a familiarity with patient-centred treatment. However, studies exploring the impact on learning and the benefit for the organisation are needed. In addition, new competence cards should be developed and tested in the specialties, i.e. psychiatry and neurology.