In this study, the SMC was used to distinguish participants who experienced the most and the least success of the appraisal meetings as a part of a mandatory NFAP. We found that the perception of success was associated with the duration of appraisal meetings, the relationship with the EA, introduction to the purpose, guidance in development and use of personal learning plans, feedback on performance as well as career guidance. Each theme is discussed below.
Longer appraisal meetings
Our study demonstrated that the duration of the meetings was longer for successes than non-successes. Both successes and non-successes described lack of time as a barrier to appraisal meetings. Successes perceived the advisors managed to allocate sufficient time and thereby demonstrating willingness to prioritize time for appraisal meetings.
There is no consensus on frequency or duration of meetings in mentoring relationships (1, 5), although lack of time is a well described barrier to effective mentoring (2, 11, 1, 12, 13, 14, 15). The latter corresponds with our result that too short and rushed meetings had a negative impact on trainees' perceived benefits of the NFAP. Several authors have suggested scheduled meetings, protected time or time allocated in the job plan (14, 13). However, this does not necessary circumvent the issues with lack of time, as even allocated time can be impinged by the clinical workload pressure (13).
Even though our results showed a positive relation between time used and satisfaction with appraisal meetings, it did not prove causality. However, meetings can probably be too short to make a positive difference for the guiding of trainees’ work and progress. This suggests enrolling only senior staff prepared to allocate time to manage the role of the EA (16).
A contact person – personal relations
The success PGY1 doctors in this study perceived their EA as approachable, dedicated and engaged. Previous studies of mentoring in medical education have found that mentors should ideally have seniority, be approachable and accessible, understanding, patient, dedicated, responsive as well as active listeners (17, 14, 11, 3, 2, 18). Effective mentors should proactively check in with mentees to see how they are doing (1). Some stressed the importance of selecting motivated and engaged mentors, ensuring that these have an interest in professional development in the workplace (19). Finally, perceived closeness to the mentor at work seems important to the mentee (17, 16).
The present study supports the importance of appointing only motivated doctors to the role as EAs.
An introduction to purpose and process
In the case of the successes, clarification of the purpose and process of the appraisal meetings lead to more structured conversations, leaving time and space for reflection. In contrast, the non-successes reported no introduction to the meetings. This might have had a negative impact on the perception of non-successes of the appraisal meetings, since clarifying mutual expectations at the beginning of a mentoring relationship are found critical to building effective mentoring relationships (2, 1, 16, 5). Furthermore, it points out the importance of faculty development to supply advisors with proper knowledge of the advisory system and skills on how to use learning plans and give feedback (11, 1, 20). In this way the organization demonstrates value of learning in the workplace (21) and recognizes the role that faculty development plays in curricular changes and development (22, 11, 19).
The present study supports the relevance of faculty development to achieve success in mentoring or advisory programs. It would be relevant to study if faculty development contributes to increase the motivation to act as mentors and EAs.
Help in using the learning plan
The successes were guided in the development and use of the learning plans, which worked as a tool to initiate reflections before appraisal meetings, and as a basis for the conversation with the advisor. To the successes, the learning plan became an effective tool to take control of their education in a busy clinical setting. However, if the learning plan was regarded a formality by the advisor and / or the trainee, or if neither of them understood or acknowledged its role as a tool, the learning plan became a demotivating factor as was the case for the non-successes.
Individualized learning plans are supposed to help improve development of self-directed, lifelong learning by actively engaging learners to take ownership of their own learning (23, 24, 25). Individual learning plans may serve as a checklist to frame the mentor's meetings with the mentee (1). However, trainees need help from seniors to develop and use their learning plans (23, 16, 26).
In order to be helpful, a learning plan must probably be revisited on a regular basis. This must be encouraged, since around half of the trainees did not remember to work on their learning goals on a regular basis according to a study by (24).
Thus, data from the present study support the relevance of a personal learning plan as a tool to support professional development, but also the importance of guidance and encouragement from the EA to use the plans.
Feedback on both clinical skills and overall global performance from the perspective of the PGY1 doctors was a valued part of the appraisal meetings for all participants in our study. However, feedback on clinical performance should take place in the daily clinical work (27). It has been suggested to make a clear distinction between educational and clinical supervision (16). Educational supervision involves support of the individual trainee with agenda-setting and planning in the context of a training program. Moreover, it should incorporate overview of the progress in the light of independent assessments of the trainee’s clinical performance, carried out by clinical supervisors (16). Clinical supervision, on the other hand, draws on the training agenda agreed upon with the educational supervisor to identify and support the training required.
This suggests that appraisal meetings should merely focus on global performance, passing millstones, as well as setting new goals for professional and personal development, which is actually the intension with the appraisal meetings (28) therefore, faculty should discuss the purpose of the meetings as well as the tasks of an EA to ensure the distinction between clinical and educational supervision. This might lead to a more fruitful feedback on overall performance in the appraisal meetings.
Career guidance
Career guidance is a mandatory part of appraisal meetings. To the successes, the inclusion of career guidance in the appraisal meetings contributed to reflections on relevant career choices on the background of discussions of e.g. strengths and interests. The non-successes reported receiving little or no career guidance.
Mentoring is important to career progression (29, 4), and it is relevant to combine advisors’ feedback on progression of clinical performance with career guidance, as trainees’ strengths and weaknesses should be essential to reflections on choice of career (1, 20).
Based on the results of the present study, career guidance should be carried out by the EA as he/she ideally has an overview of the competences of trainees and through the appraisal meetings the EA has the opportunity to discuss relevant career options on the background of the trainee's strengths and weaknesses. It is possible that PGY1 doctors with no or insufficient career guidance may experience problems in their career progress. However, further studies are needed to explore the influence of lack of career guidance on future career.
Limitations and generalizability
It was a limitation in our study that only the PGY1 doctors were interviewed. Thus, the results presented here represent only their perspective. As success in appraisal meetings presumably depends on both the EA and the trainee and the relation they are able to build, it would be interesting to explore the perspective of the EAs and to observe the interaction between advisors and trainees.
As we found no studies on organizational systems comparable with the NFAP, we have chosen to discuss the results from the present study with results from studies on factors important to success or failure of mentoring. In this light, our findings seem representative.
The majority of interviews were relatively short telephone interviews, and there was thus only limited time to build a relationship with the informants. Furthermore, the interviewer had limited familiarity with the NFAP at the beginning of the study. Thus, themes that emerged during the first interviews might have been more deeply explored. Continuing sampling until saturation of themes was not intended as the sample size was given by the cluster analysis in accordance with the SCM. However, we found a substantial agreement between the themes of importance to successes and non-successes, respectively. All authors participated in the data analysis, discussion of themes and sub-themes, thus increasing the dependability of the study.