Observations of the group-assessment by the external consultant are presented, followed by interview data, split into relevant sub-themes.
Observation of the group-assessment
On the day of the group-assessment, each resident took turns presenting a clinical case within each of the day’s topics (approximately 1 hour discussion for each topic). PowerPoint were not used, but patient data, such as blood samples (from the electronic journal), was sometimes presented on screen.
The cases were discussed with both peers and the assessors. The moderator kept track of time and made sure that all residents participated in the group discussion and the presentation of cases. The four assessors and the moderator together assessed the residents` skills and feedback was given during the discussion of the case. Notes and mandatory standard assessment forms were used for the assessment [see Additional file 2].
The external consultant was in no position to evaluate the medical content of the discussion, but it was evident that a case could result in discussion of many aspects such as social background, ethics, and collaboration besides core medical content. This generated numerous inputs from both residents and assessors. Many of the cases presented were known by many of the gathered doctors, increasing the possibility of viewing the case from different perspectives. There was a great variation in the number of doctors who took part in the discussion after each case presentation, and also a great variation in the amount of time used on each case discussion (from 2 to 20 minutes). There was also a great variety in how active the assessors were, especially in regards to follow-up questions for assessment of satisfactory clinical competences of the resident, suggesting different understanding of the role as an assessor, which was also confirmed during the interviews.
During the CbDs it was noticed that the moderator played a powerful role in assuring that all topics were discussed and that all residents participated in the case presentations and following discussions.
After each topic, the assessors and moderator, in privacy, discussed the performance of the residents, and came to a consensus on whether the competence should be approved. If a resident did not perform sufficiently, a one-to-one assessment together with the educational supervisor would be arranged. The results of the assessment were communicated to both the resident and the educational supervisor at a later time in order to avoid humiliating the resident in a group forum.
Interviews
Preparing for the group-assessment and case presentation
As the topics and dates for the group-assessments were send out a year in advance, with a reminder three month prior to the next group-assessment, the residents felt that they had plenty of time to gather and prepare cases;
“You know what the topics are, so you pay attention if you encounter an interesting patient. You want to find a case with many interesting aspects that you think will be of interest to all your colleagues.” (resident 5)
The residents remembered the cases by making notes about them and by reading the notes again directly before the group-assessment.
The assessors were a bit nervous about what to expect prior to the first group-assessment, but found there was no need for preparation;
”The first time I was unsure about what to expect, but I think it went well… I found out that I knew the things I should know, and it did not require any great preparation.” (assessor 3)
The residents` themselves picked out the cases they wanted to present. Therefore, as some pointed out, there could be a tendency to pick out cases where they felt confident of own knowledge. Several residents stated that there might be a tendency to pick out cases that were rare and intriguing; “There has been a tendency to pick good and exciting cases. However, in our field we could be better at presenting what is difficult or unclear, because that is what we often encounter in the daily clinical work…..” (resident 1)
Other residents, on the contrary, stated that they were more likely to present difficult cases where they had been in doubt of what to do. Yet another resident stated that she chose to present a case within her own specialty that she thought it was important for all doctors to have a minimum of knowledge about.
Assessment, resources, and approval of competences
A repeated statement was that it could be difficult to assess all the residents at the same time. One might hide in a group, not necessarily because he/she did not have the required competences. This could be due to personality, as some are more introvert and timid/reserved, while others are extrovert and like to be heard;
“You are in a forum, where one might feel a bit exposed. You are together with specialists from the department, and everybody has that basic fear. One would not want to present oneself as professionally ignorant.” (resident 1) Many doctors acknowledged the ability of the moderator to steer the meeting, and in making sure everyone was heard. The length of the group-assessment was regarded as suitable and many emphasized that the meetings were relaxed with a good and safe environment, which was not exam-like. Since there were several assessors, many thought that the assessment was more consistent.
All though it took time to plan the group-assessments, both the residents and assessors felt that they were more resource-efficient and manageable than the one-to-one assessments.
As the meetings were scheduled during working hours it was highly appreciated that the group-assessment concept made it simple and straightforward to have the competences approved. A resident pointed out that formerly having competences approved often became a hunt for signatures, often without proper assessment and feedback;
”I think this is a much better idea, than the way it works in other places. We have to have these signatures, there is a terrible lot of signatures…So it is a bit of a hunt for signatures, without anyone really going into the depth with the different things.” (resident 4)
Professionalism, learning and interdisciplinarity
The residents especially appreciated the professional discussions in an interdisciplinary environment. It was found to be professionally enriching and educational to meet with doctors from other specialties to discuss relevant common relevant challenges. It was also mentioned by the residents that the group-assessments gave an insight into of the other doctors` skills, which could be used as an inspiration to become just as skilled;
“You get a better insight into the other doctors skills, and this may be used as an inspiration to become just as skilled as the others.” (resident 5)
For the assessors, the assessment process had a greater focus than the professional discussions, but they also acknowledged that all doctors gained more knowledge from the group-assessments compared to the one-to-one assessments. The extra time to discuss cases was highly appreciated. As one assessor put it;
“…. I think we all felt it was a luxury to have the time for discussion. Because we all learn from each other, no matter if you are a resident or an expert. But the expert is not an expert in all fields of medicine, so in that way it is of mutually benefit, although the assessment is about them [the residents]. In that way I think they get a wider knowledge compared to the one-to-one assessments.” (assessor 2)
Both residents and assessors thought that the group-assessments were rewarding for all. Most participants looked forward to the next group-assessment, because they gained new knowledge and insight during the meetings. The group-assessments made it possible to discuss more cases (around 24 cases in a session) compared to the number of cases covered at the one-to-one assessments (usually three to four cases). Furthermore, the presence of both residents and assessors from different specialties provided more perspectives and interdisciplinary knowledge. One resident thought the concept was a stroke of genius;
”The fundamental idea to meet four times a year and have the competences assessed in a forum where residents and specialists are gathered, is a stroke of genius. Because it shows that this is something that needs to be done, and there is a setting and a deadline for the assessment.” (resident 1)
Overall perspective of the group-assessment concept
Eight out of the ten doctors who were interviewed, preferred the group-assessment concept prior to the one-to-one CbDs, while two residents preferred the individual one-to-one assessments, given that there was enough time for the case discussion;
“I think I would personally benefit most from the one-to-one assessment. But it [group-asssessment] is a good way to have your competences assessed.” (resident 2)
The residents acknowledged the value of individual supervision with their educational supervisor, but for the approval of core internal medicine competences, the group-assessment concept was found superior by most.
Many of the doctors therefore suggested that the group-assessment concept should be tried out and subsequently maybe implemented at other departments, as a replacement for mandatory one-to-one CbD assessments of key competences.