Setting
In the Netherlands, PGME programs consist of 4-6 years of workplace learning in teaching hospitals, partly in a general teaching hospital and partly in a university hospital. Competition for enrolment in nationally recognized PGME programs is fierce, which is why the majority of freshly graduated doctors choose to obtain clinical experience for a few years as a junior doctor before applying for a residency position. As a result, almost all PGME departments in Dutch teaching hospitals employ both junior doctors not enrolled in formal PGME training, and residents in the nationally recognized PGME program of that discipline. Both junior doctors and residents are licensed physicians and are involved in patient care, with residents acting increasingly independently with increasing experience and competence throughout the PGME program. Although junior doctors are not formally enrolled in PGME programs, they participate in the department’s educational activities for residents and share clinical duties and on-call shifts with residents.
We conducted a cross-sectional observational study of residents and junior doctors in two hospitals in the Netherlands: Isala Hospital in Zwolle (1100 beds) and the Medical Center in Leeuwarden (MCL, 618 beds). Isala and MCL employ approximately 120 and 95 residents in formal PGME programs and 100 and 65 junior doctors, respectively. Both hospitals are certified by the Royal Dutch Medical Association as licensed general teaching hospitals in 28 and 23 PGME programs, respectively. Because each PGME program has its own design and timetable, the population of residents in Isala and MCL changes almost every month, with residents moving in and out of PGME programs. Residents spend between 6 and 48 months of their PGME training at the hospital, depending on the program they are enrolled in.
Quality cycle of PGME programs
As prescribed by the Royal Dutch College of Medicine,(21) both Isala and MCL hospitals carry out an quality cycle, aimed at continuously monitoring and improving the quality of each PGME program. As part of this quality cycle, all current residents and junior doctors are asked to complete the SPEED questionnaire annually by web-based survey, the results of which are analysed and fed back to faculty anonymously (i.e, without disclosing individual respondents’ responses or characteristics).
Each resident or junior doctor leaving the hospital after completion of the PGME program (resident) or expiration of their contract (junior doctor) is invited for an exit interview, collecting data on the resident’s or junior doctor’s experience in working at the hospital in a semi-structured fashion. The aggregated results of these exit interviews are fed back to faculty, again without disclosing individual respondents’ responses or characteristics. These exit interviews are being conducted by the hospital’s junior staff coordinators, who are the primary contact persons for residents and junior doctors throughout their career at the hospital, and who are independent from the hospital’s faculty providing the PGME programs. These junior staff coordinators are highly valued by residents and junior doctors as their advocates and confidants, and serve the recommended role as an independent “honest broker” to collect anonymous data on PGME program quality.(16) As part of the exit interview, residents and junior doctors are asked to complete the SPEED by web-based survey.
Study population and outcome measures
We used the SPEED results that were collected routinely as part of the PGME quality cycle in the two hospitals, in two groups of residents and junior doctors:
- Residents and junior doctors currently working at the hospital (called “current residents” in the remainder of this article)
- Residents and junior doctors participating in an exit interview as outlined above (called “leaving residents” in the text below)
Between January and December 2017, all 220 current residents at Isala were invited to complete the web-based SPEED survey. At the MCL, all 160 current residents were asked to complete the web-based SPEED survey between October 2017 and October 2018.
Throughout 2017, exit interviews were conducted with all 95 leaving residents at Isala and with all 75 residents leaving MCL in 2018.
SPEED
The SPEED comprises 15 items in three domains (content, atmosphere, and organisation of the PGME program), scored on a 5-point Likert scale ranging from one (strongly disagree) to five (strongly agree), and a general domain grade for each domain on a scale from 1 (poor) to 10 (excellent).(9) This way of general grading is used throughout secondary and university education in the Netherlands and is therefore familiar to residents. The SPEED is completed in a web-based survey. Responses are collected anonymously; no data are recorded on age, gender or other personal characteristics, apart from the department at which the respondents are working.
The full version of the SPEED is available in its original open access publication.(9)
Statistical analysis
Because of the anonymity of the data, we were not able to link individual scores of current residents to those of leaving residents. Our analyses were based on the following variables for each department:
- department SPEED domain grades: we calculated three department SPEED domain grades by averaging per department the general domain grades given by respondents for content, atmosphere, and organization of the program;
- overall department SPEED grade: for each respondent, we calculated the mean SPEED grade by averaging the three general domain grades; subsequently, we calculated the overall department SPEED grade by averaging the mean SPEED grades per department.
Because the distributions of the department SPEED domain grades and the overall department SPEED grades were not significantly different from normal distributions (Kolmogorov-Smirnov tests, p> 0.1) we used parametric tests (MANOVA and Student’s t tests) to analyse the data.
We used multivariate analysis of variance (MANOVA) to assess differences in our primary outcome parameter, i.e. the department SPEED domain grades, between current and leaving residents. We also examined differences in the primary outcome parameters between the two study sites, to explore potential systematic differences in perceived learning environment quality between hospitals. We used Student’s t test to analyse the difference in overall department SPEED grades between current and leaving residents and between hospitals.
As secondary outcome parameters, we analysed whether the differences in SPEED domain grades and overall department SPEED grades between current and leaving residents were related to the number of residents in a department, by comparing it between large departments (> 5 residents) and small departments (<5 residents), and by calculating the correlation coefficient between the number of residents in a department and the difference in department SPEED grades between current and leaving residents.
Before the study, we considered that a 1 point difference between department SPEED grades of current and leaving residents represented a relevant difference in the residents’ assessment of their learning environment. To be able to detect such a difference with 90% power, assuming a SPEED grade standard deviation of 0.5,(9) we needed to compare SPEED scores between current and leaving residents of at least 12 departments.
All analyses were carried out using IBM SPSS statistics.
Ethical considerations
This study was approved by the Netherlands Association for Medical Education Ethical Review Board (file number 1063).