All methods were carried out in accordance with relevant guidelines and methodology approved by the RCSI Research Ethics Committee. The methods are reported in accordance with the STROBE checklist for reporting results of cross-sectional observational studies (30).
The study was a baseline cross-sectional assessment of the entire cohort of medical students in the RCSI University of Medicine and Health Sciences, Dublin, using established quantitative measures. At the RCSI, there is a five-year Direct Entry Medicine (DEM) programme, along with a four-year Graduate Entry Medicine (GEM) programme. DEM and GEM classes combine for the final two senior years. Furthermore, there is a Foundation Year programme completed by a proportion of students (approximately 50%) prior to commencing the DEM programme. Due to the involvement of a large cohort of students (entire years), it was decided that an online assessment would be the optimal method for data collection.
In selecting measures to ensure a comprehensive assessment was performed, selected scales were required to have appropriate content and psychometric validity. Length of each measure was also a key concern to be mindful and respectful of students’ time, especially to facilitate future longitudinal study, and to ensure satisfactory completion rates. A 76-item questionnaire (Appendix A) was compiled, combining four validated scales along with a selection of demographic questions.
Following completion of a pilot study (n = nine participants), where it was identified that no major changes to the format were required, the PILLAR study assessment was embedded within modules in each pre-clinical year as a core element of the professionalism curriculum. While completion of PILLAR was compulsory, students gave voluntary, fully informed consent (or not) for their data to be used for research purposes, as per previous research (31, 32). The assessment responses were anonymised. Akin to progress testing, we incorporated a student feedback loop providing students with their individual responses to each question along with general feedback on the results (class averages).
PILLAR was administered by the RCSI Quality Enhancement Office (QEO), who acted as the study’s independent data controllers, using online survey software. PILLAR was pseudonymised in order to assign respective credits to students for completion. On receipt of responses, the QEO fully anonymised the results so that any identifiable data had been erased prior to transfer of data to study research team.
In advance of the administration of PILLAR and invitations to participate being sent to students, the study investigators gave brief presentations to each year introducing the assessment. At those class times, the QEO sent out an automated, personalised email to every eligible student with an individualised link to complete the assessment as part of the assigned module, followed by two weekly reminders. The individualised link took participants to an online participant information leaflet and consent form where they consented or declined that their information may be used for the purposes of research.
During September 2020 and February 2021 all students in Foundation Year (pre-medicine; FY) and Years 1–3 of direct-entry undergraduate medicine and Years 1 and 2 of graduate-entry medicine (GEM) completed the assessment and were invited to participate in this study (n = 1,427).
Measures incorporated in the assessment included a measure of each of the core PILLAR constructs of professional identity formation, professionalism, resilience, leadership, as follows:
Professional Identity Formation (PIF)
PIF was assessed using a 9-item questionnaire with nine themes, which included common domains of professional activity for healthcare professionals (33). The themes were team working, communication, patient or client assessment, cultural awareness, ethical awareness, using patient or client records, dealing with emergencies, reflective practice and teaching. Participants were asked to indicate on a scale of 0 (indicating 1st day student doctor) to 6 (indicating a newly qualified doctor) how they would feel at present undertaking a list of nine activities, with each activity reflecting one of the nine themes. A higher mean score reflected a higher professional self-identity score.
Through feedback from our pilot study, it was decided to include one further item relating to PIF: ‘I feel like a member of the medical profession’ to which participants were given five response options from strongly disagree to strongly agree, with lower scores indicating lower levels of PIF.
The scale for professionalism examined both student perceptions of and participation in unprofessional behaviours (34). Participants were asked to consider twenty-five statements and then answer if they have observed the behaviour, participated in the behaviour and state whether they felt the behaviour was professional or not. For each behaviour listed, descriptive statistics were used to describe student observation of, participation in, and perception of the behavior as unprofessional.
We used the Medical Leadership Competency Framework (MLCF), which outlines the competencies expected of practicing clinicians (20). The MLCF has been demonstrated to aid curriculum design, highlight individual strengths and development areas through self-assessment, and assist with personal development planning and career progression (20). Participants were asked to self-assess in two areas only – demonstrating personal qualities and working with others, totaling 15 items. We assigned a scoring system to identify the average for each domain where A Lot of the Time = 2, Some of the Time = 1 and Very Little /None of the Time = 0, with higher total scores indicating higher levels of leadership.
Resilience and wellbeing
Two measures were chosen to investigate resilience and wellbeing - the Brief Resilience Scale (BRS) and the EPOCH Measure of Adolescent Well-Being (35, 36). The BRS was chosen to measure resilience because it is a short, validated measure that is simple and widely used (37, 38). Participants are asked to self-assess on a 5-point Likert scale of agreeableness against six specific statements describing their typical response in relation to bouncing back or recovering from stress, with higher scores indicating greater resilience (35). EPOCH measures five positive psychological characteristics which together support higher levels of well-being: engagement, perseverance, optimism, connectedness and happiness. This measure is composed of 20 items, with item responses rated on a 5-point scale ranging from 1 for ‘never’ to 5 for ‘always’. Scores are calculated for each domain as the average of the four items (36).
Stata v.16 was used for cleaning and statistical analyses (39). Descriptive statistics were used to profile demographic characteristics of participants. Means and standard deviations were calculated for continuous variables and total scale scores. Linear regression analysis was carried out to explore potential differences across years.