Patients’ and tutors’ assessment of General Practice/Family Medicine consultations conducted by medical students in Coimbra, Portugal

Background: Undergraduate teaching of General Practice/Family Medicine (GP/FM) must ensure students acquire the necessary competencies and skills to perform an adequate GP/FM consultation with adequate annotations (the SOAP model) and classifications. So aimed to study and to correlate students’ evaluation by tutors and patients in specific consultations in the formal practical evaluation of GP/FM Curricular Unit of the Integrated Masters on Medicine at the Faculty of Medicine of the University of Coimbra (IMM-FMUC) in the academic years of 2017-2018 and 2018-2019. Methods: Observational study of the 2017-2018 and 2018-2019 academic years of the assessment grids for tutor’s evaluation of SOAP performance and fluency in consultation and for patient’s evaluation of the student ‘performance, in the convenience sample of those who chose to be so evaluated. Results: We studied a population of 435 (67,7%) out of a universe of 646 students, 125 (28,7%) males, ns by sex and academic year who performed this evaluation. In a mark up to 20 from tutors, difference was found for Plan (P) mark, higher in 2018-2019 (18,38±2,18 vs 18,54±2,11, p=0,005) of the SOAP methodology evaluation. Patients’ evaluation was not different 19,34±1,70 vs 19,35±1,40, p=0,091. A positive significant correlation was found between tutors and patients marks (ρ=0,278; p<0,001), as well as between tutor mark and final mark (ρ=0,958; p<0,001) and patient and final marks (ρ=0,465; p<0,001). Final marks were not different in both years, 18,61±1,38 vs 18,78±1,15, p=0,158. Conclusions: This innovative model of evaluation of student’s performance in medical appointment, showed a significant positive moderate correlation between patients’ and tutors’ marks in the setting of GP/FM at the IMM-FMUC, and was not different between years. Yearly evaluation must be continued.

tutors. 6 To do so a systematic organized recoil of information is mandatory beginning by the reason for consultation the S from Subjetive, including signs and symptoms the patient is suffering of and the fears and expectations as well as the answers to doctor questions, continuing in O from Objective introducing the signs in physical exam and even the psychological viewing of the patient and the results of tests, following by A from Assessment, the doctor diagnostics and finishing in P form Plan, where the prescribed medications, exams and non-pharmacologic therapies as well as other measures of enablement and empowerment must be written for future memory. 6,8 This procedure implies clinical ordered information to be easily look for in the future, allowing epidemiologic studies to be performed, should that information be classified according to ICPC2, the aanotations and classifications constituting the clinical registries. 6,8,9 At the Faculty of Medicine of the University of Coimbra (FMUC), since 2017-2018, students begin contact with GP/FM in the fifth of a six years Integrated Master's in Medicine (MIM), attending 30 hours on-class inter-active sessions with theory, role-play and video watching and discussions. They also attend a minimum of eight hours in a Primary Care Family Medicine unit with a trained tutor, to observe and practice, being voluntarily evaluated near the end of such period by performing a tutor´s selected consultation, of an appropriate degree of difficulty. Matters like Subjective, Objective, Assessment and Plan, the SOAP methodology, International Classification for Primary Care2 (ICPC2), Patient Centered Medicine, Medical Empathy and empathic opportunities, Enablement and empowerment, Preventive Medicine, Multimorbidity, Polypharmacy, Psychological problems, Ageing problems and Primary Care were presented and discussed in-class with students. [8][9][10][11][12][13][14][15][16][17][18][19] Particular interest was put on Patient Centered Medicine and empathy as a way to focus our students on the person suffering and not only on diseases. 7,11,20,21 . The Dean of the FMUC issued a consent form approving the Curricular Unit Form once the FMUC Pedagogic Council gave its approval to the proposed scheme of teaching GP/FM from the year 2017-2018 onwards.
Until 2017 the work-out for the curricular discipline practical examination was a mandatory study of a Family, filling in a free report to be assessed, followed by a voluntary oral presentation with no benefit for better mark for the student.
Since GP/FM in the Portuguese National Health Service (PNHS) is practiced out of hospitals in small units with doctors, nurses and secretaries, it was intended that medical students practiced in such a setting questioning, examining, diagnosing and making plans for exams, non-pharmacological and pharmacological prescriptions as well as explaining it efficiently to patients, with proper e.registrations. In the PNHS, an informatics program is used, so students also learned about it through their tutors. Tutors were experienced GP/FD specialists that attended at least three two hours sessions on what was intended to be explained and how it was to be measured.
A previous study, in a very different medical education context in Portugal, found a neutral nonsignificant correlation between patients' and tutors' evaluation of the consultations in GP/FM in 4 th grade students. It was hypothesized that the evaluation of the consultation by the patient could be an interesting tool to measure practical student skills when compared with the tutor's evaluation, at the same time serving as an educational feed-back tool. 6 In fact Patient Centered Medicine and empathy are core issues to develop patient-doctor relationship which students can ignore when merely pursuing a disease centered approach. 6-11. A 2010 paper found that teachers' scores were in accordance with patients' scores 22 and a paper in Portugal found a neutral and non-significant correlation between the valuation mark by the tutor and the patient's grade. 6 This means that the preparation of our students for practical clinical life must be an object of concern by Medical Schools.

Objective
To study and to correlate students' evaluation by tutors and patients in specific consultations in the to analyze and to grade particular aspects of each part of the SOAP model as well as the fluency of the consultation by the tutor and its evaluation by the patient. These issues were firstly discussed with the student´s committee, showed and discussed with students at the first and introductive class and the grids and the practical issues to be observed were published in the student's FMUC intra-net.
Prior evaluation assessments took place in the tutor/student communication but those were not brought into evaluation. In the course of the consultation students could scales to study the individual and its family.
Clinical tutors, GP/FDs with experience and knowledge, were previously trained on how to do the task of teaching, demonstrating and evaluating in a two hour session at the beginning of September and February, each academic year, by performing and matching role-play and video consultations and by trios exercises in which they had to perform the task of being a doctor, being a patient or a tutor.
Tutors were also instructed on the difficulty of each case to present giving special attention to nonscheduled consultations.
The consultation methodology SOAP marks were calculated according to Tables 1 and 2 Table 1 shows the tutor's SOAP and fluency evaluation tables, and also how several observed aspects were to be graded.
The patient's evaluation was given to the tutor by the patient that had previously given consent to this task, after answering to the questions in Table 2, so no data on who answered exists. The patient's mark was calculated as the sum of answers, Table 2.
All data were known by the student at the end of the evaluation so that a feedback session could be made with the tutor. It was mandatory for students and tutors to sign up the paper marks at the end of evaluation. Students were given a second opportunity of consultation should they wish it, final mark being the last one.
As the studied data were analyzed anonymously, and were public no consent to participate was obtained, according to the approved protocol by the Faculty of Medicine of the University of Coimbra.
At the end of each academic year, all the information was gathered from the assessment grids in order to be studied for differences between semester, academic year and gender by descriptive and inferential parametric statistics: for nominal data we used the χ2 test and for continuous non-normal and the ordinal data the Mann-Whitney U test. Spearman correlation between patients, tutors' and final marks were performed using IBM SPSS statistics 24. We defined p <0,001 for difference.

Results
Of a Universe of 646 (322 in 2017-2018), a population of 435 (67,3% of the total) was studied 28,7% being male. We could not know the sex of two students.
According to table 4, in a non-normal mark up to 20 distribution and comparing by academic year, no significant differences were found although higher values were found for the academic year of 2018-

SOAP Plan component was significantly better marked in the 2018-2019 academic year.
In a mark up to 20 and comparing by sex no significant differences were found according to table 5.

Discussion
In a predominantly female population, students' evaluation performance in consultation is not different by academic year or gender.
Out of 646 students, 435 (67,3%) chose the consultation evaluation. Even though not statistically different, students in the 2018-2019 academic year and females tend to be scored higher. We acknowledge that a bias of performance can exist once students can perform consultation like they were taught just for the sake of being well evaluated but without a clear and incisive belief on such a model of consultation centered on the patient and not only on the disease.
The calculated correlations mean that patients and tutors tend to evaluate similarly to another paper 22 but differently from a previous Portuguese study 6 . If in the case of the former study there are similar results with different evaluation grids, for the latter more studies at the FMUC and in other Faculties of Medicine in Portugal seem necessary.
We aimed to measure in the setting of a GP/FM consultation, the performance of the SOAP model, the registries, the practice of Patient Centered Medicine, the physical exam focused on the patient's complaints performing the necessary physical exam according to ethics, the diagnosis capacity, the communication skills, the ability of negotiating, enabling and empowering a patient and the annotations and ICPC2 classifications made and we found no differences between the two studied populations comparing 2017-2018 to 2018-2019, which was our aim. A week but significant difference between "Tutor's mark and Patient´s mark" being found is a matter deserving future studies. It is possible that patient's satisfaction is different from the one of tutors. The scientific translational medical knowledge was not specifically evaluated, an OSCE being needed to fulfill such a task. 23,24,25 Students had access to all the assessments. All consultation's participants signed forms were personally handed by the student so serving as feed-back information.
We believe that this model is suited to measure what it is intended to measure. In fact a consultation as we teach it in the Patient-Centered Medicine Methodology must include the focus on empowerment and answer to patient's questions, the illness, so being a complete exercise students were not yet exposed to up to the 5 th year at FMUC.

Nevertheless:
1 -Listening to students, eventually by a self-administered questionnaire about the consultation; 2 -Investing more time in-practice with more consultations and making follow-up consultations for previous patients, averaging the mark as the mean of at least three consultations, are important issues to deal with in the future. 5th year: S=1+2+3+4=60%; O=5=15%; A=6=5%; P=7+8=20% of the mark.

Conclusions
Fluidity as the sum of marks.