Standardized guidelines for survey studies were followed21-23 and a protocol was written prior to the initiation of the research process.
Our training program is a three-year Emergency Medicine program with 8 residents per year. The program recently expanded to 9 residents per year and the intern class for the Fall 2018 and Winter 2019 consisted of 9 residents (which occurred during the study period). The residents spend the bulk of their clinical curriculum in the emergency department but also do clinical rotations in Obstetrics and Gynecology, Hand Surgery, Orthopedic trauma, Anesthesia, Pediatric intensive care unit, surgical intensive care unit, Psychiatry, Emergency Medical Services and emergency ultrasound. The educational curriculum consists of 5 hours per week. Approximately 20% of educational curriculum is delivered through simulation.
Furthermore, the residency program has a research and quality improvement requirement. As residents advance from year to year there is an expectation of progressive responsibility and accountability with the expectation that the third year residents are able to lead, manage and supervise their clinical area. Resident progress and performance is assessed on a semi-annual basis. Semi-annual reviews are based on the ACGME milestones.
IRB approval was obtained prior to the initiation of this research study.
Residents were asked to consent on the first page of the survey. Only those who consented answered the questions. Completing the survey was voluntary. All residents could participate in the retreats regardless of completion of the surveys.
Data were securely stored in a password-protected server. Because of the sensitive nature of the responses, data were collected anonymously. However, the IRB requested that one team member have access to identified data if needed. The residents and medical student working on this project did not have access to identifiers. One faculty member had access to individual answers in case a resident with severe burnout was identified.
Development and Training
The MBI is a validated tool for assessing burnout. A license to administer the MBI Human Services Survey adapted for Medical Personnel and the interpretation manual were purchased.
The MBI evaluates three subscales including; are emotional exhaustion, depersonalization and personal accomplishment. Emotional exhaustion (EE) measures feelings of being emotionally overextended and exhausted by one's work; depersonalization (DP) measures an unfeeling and impersonal response toward patients; and personal accomplishment (PA) measures feelings of competence and successful achievement in one's work. It is a 22-item survey with 9 items measuring emotional exhaustion, 5 items measuring depersonalization, and 8 items measuring personal accomplishment.1 Scoring of subcategories range from 0-54, 0-30, and 0-48 for emotional exhaustion, depersonalization, and personal accomplishment respectively. Each subcategory is considered individually and should not be combined. Categorizing each subcategory as low, medium, or high levels is not recommended.24 However; scores at the top end of the range of emotional exhaustion and depersonalization suggest higher levels of symptoms. Higher personal accomplishment scores represent a higher sense of personal accomplishment. A “high” score in the category of emotional exhaustion (≥27) or depersonalization (≥10) is commonly used to determine burnout.2, 3
The RWS (see additional file 1) was modeled after surveys validated to measure well-being among medical professionals.1, 25 Team members met with staff from our institution’s survey research center to discuss, evaluate, and receive feedback regarding the survey.
The questions for the RWS were tested with healthcare providers other than EM residents (e.g. medical students, off service residents rotating in the ED, and EM faculty). Data in the study is not from the pilot phase of the study. The questions for the RWS where distributed for feedback purposes only to refine the survey and not evaluated in the study or included in this manuscript. The pilot phase was suggested by our survey center and we asked learners not in our training program to review the survey for feedback and modification purposes. The nine learners who provided feedback on the survey did not participate in the retreats and therefore where not included in the dataset of this manuscript.
Feedback from the piloting was used to edit and refine the RWS survey. The questions were modified to reflect residents preferred language and match the interventions. With support from the survey research center, usability and technical functionality of the electronic questionnaire was tested before deployment of the questionnaire.
Recruitment Process and Contact Mode
Residents were contacted by email by a co-resident to avoid any perceived coercion from residency leadership. Up to two reminders were sent for each administration of the survey (e.g. baseline, pre-retreat, and post-retreat). No monetary compensation was offered to participants.
This study included a non-random sample of participants following purposive sampling, where a specific population was identified and only its members were included in the survey.
The MBI survey was administered using REDCap and the RWS using SurveyMonkey. The MBI survey was administered via an electronic link to all EM residents (n=25) in July 2018 to establish a baseline. Both MBI and RWS surveys were repeated before and after the fall 2018 and winter 2019 retreat.
The intervention evaluated were in-person retreats, each lasting one half-day. The fall 2018 retreat included a casual meal as well as team building activities and social network opportunities such as bowling and softball. The winter 2019 retreat included a casual meal and outdoor winter activities such as snow tubing and sledding.
Data were collected in REDCap and SurveyMonkey and exported for analysis.
Data Analysis and Statistical Methods
Criteria for burnout was defined by a high score in either EE (≥27) or DP (≥10) as is common among other studies using the MBI to evaluate physicians.2, 3 Survey responses were classified as baseline (7/13/2018-7/23/2018), pre-fall retreat (8/21/2018-9/4/2018), post-fall retreat (9/5/2018-9/12/2018), pre-winter retreat (1/28/2019-2/4/2019) and post-winter retreat (2/5/2019-2/20/2019). Continuous features were summarized with means and standard deviations (SD); categorical features were summarized with frequency counts and percentages. Comparisons of responses between baseline and pre-retreat, pre- and post-retreat, and baseline and post-winter retreat were evaluated using two-sample t, Wilcoxon rank sum, and Fischer exact tests. Statistical analyses were performed using SAS version 9.4 (SAS Institute; Cary, NC). All tests were two-sided. P-values <0.05 were considered statistically significant.