Study setting and design
The prospective observational study was conducted at the ED of the NTUH, a tertiary medical center in Taiwan, between July 2015 and October 2017. The ED has an annual uptake of approximately 85,000 patients. The study protocol was approved by the institutional review board of the hospital (201412004RIND) and registered at ClinicalTrials.gov (NCT03738033). Written informed consent was obtained from the participants.
Eight to 10 PGY-1 residents attended the ED training every month. Every PGY-1 residents had 18 eight-hour working shifts during their training month and evaluated 10-15 non-critical patients under the supervision of the certified emergency physicians on each shift. A novel PoCUS curriculum including 30-min didactics and 2-hour hands-on training on a live healthy model volunteer was implemented for the PGY-1 residents during the first week. Because abdominal discomfort was the leading ED symptoms, the content of the curriculum included the extended focused assessment of sonography for trauma (eFAST) to detect intraperitoneal fluid, pericardial effusion, pleural effusion, and pneumothorax, and sonography for urinary tract to detect hydronephrosis, gall bladder to detect acute cholecystitis, and abdominal aorta to detect abdominal aortic aneurysm. The ratio of the instructor to participant is less than 1:5. The instructors are the expert sonographers, board-certified by the Taiwan Society of Ultrasound in Medicine, and had over 10 years of experience in sonographic examinations. A US machine (SSA-660A, Canon, Japan) equipped with 2-5 MHz curvilinear transducers was used for training.
The PGY-1 residents completed the curriculum and received an OSCE for post-curriculum assessment immediately. It consisted of standardized questions for image acquisition and interpretation with points for technique, image quality, and correct interpretation of anatomy of FAST, kidney, gall bladder, and aorta. A global rating score using a Likert 5-point scale (unsatisfactory=1, needs improvement=2, satisfactory=3, high satisfactory=4, outstanding=5)[9] was given by the instructor at the scene and by the other instructor that did not involve in training through video review independently. The faces of the PGY-1 residents in the video were covered. The score given by the two instructors was averaged.
There were two US machines (SSA-550A, SSA-660A, Canon, Japan) equipped with 2-5 MHz curvilinear transducers kept ready in use in the ED clinics. Also, the reporting documentation was put in a plastic bag, along with the machine. The sonographic examinations the residents had performed clinically were obtained at the end of the ED training, including indication, scanning targets, sonographic findings, sonographic diagnosis, and management. The accuracy of sonographic diagnosis was defined as the agreement between the sonographic diagnosis and the discharge/admission diagnosis made by the attending physician. The images were reviewed by another two instructors not involving in training blindly and independently. The quality of the images was categorized using a 5-point Likert rating scale. Point 1 indicated no recognizable structures, no objective data can be gathered; point 2 indicated minimally recognizable structures but insufficient for diagnosis; point 3 indicated minimal criteria met for diagnosis, recognizable structures but with some technical or other flaws; point 4 indicated minimal criteria met for diagnosis, all structures imaged well and diagnosis easily supported; point 5 indicated minimal criteria met for diagnosis, all structures imaged with excellent image quality and diagnosis completely supported [10]. Another instructor not responsible for the curriculum would interview the PGY-1 residents who did not perform clinical sonographic examinations after the curriculum and their feedbacks was obtained.
Selection of Participants
The PGY-1 residents attending the curriculum and completing the OSCE were included. Those attending the curriculum but not completing the OSCE were excluded.
Data collection
The demographic data of the PGY-1 residents were obtained, including age, gender, and prior US experience. The global ratings of the OSCE were collected, as well as the sonographic examinations including indication, scanning targets, sonographic findings, sonographic diagnosis on shifts. Based on the clinical sonographic examinations, the residents could be categorized for 4 groups. The group 1 indicated the residents performed PoCUS during their shifts before and after the curriculum; the group 2 indicated that those performed PoCUS only after the curriculum; the group 3 indicated that those performed PoCUS only before the curriculum; the last group indicated those did not perform any examinations. The scanning targets were categorized into a single application (ex, FAST) and more than 2 applications (ex, FAST+gall bladder) according to the ACEP statement [6].
Outcomes
The primary outcome was the clinical integration of PoCUS on shifts of the PGY-1 residents and the correlation with their OSCE performance.
Statistical analysis
All data were analyzed by SAS software (SAS 9.4, Cary, North Carolina, USA). Categorical data were expressed in counts and proportions, while continuous data were expressed in medians and interquartile ranges (IQRs). Categorical variables were compared using a Chi-square test and ANOVA. Continuous variables were examined using Wilcoxon’s rank-sum test. Intraclass correlation (ICC) with one-way random effects was used to assess inter-rater reliability for those global rating scores and imaging quality scores by two evaluators.
The linear regression models were applied to identify the factors associated with numbers and accuracy of the sonographic examinations after the curriculum. The polytomous regression models were applied to investigate the factors associated with the image quality of the sonographic examinations after the curriculum. The covariates included age, gender, prior US experience, and global ratings. Also, the factors associated with the global ratings were investigated using the polytomous regression models. The covariates included age, sex, and prior experience. A p-value of less than 0.05 was considered statistically significant.