During the study period, 141 students completed the IM Clerkship. Of those 141, 72 received the standard education and 69 received the high value care curriculum. There were no significant differences between the two groups with respect to gender, ethnicity, race, undergraduate major, undergraduate GPA, or MCAT→ score (Table 1). Students in the intervention group were older compared to the usual education group (26 vs 24.5 years old, p = 0.002). Including classroom time, the total curriculum time was less than four hours. Assessment data for six students (3 in intervention and 3 in control) who opted out of the analysis were not included.
Characteristic1
|
Control
(n=72)
|
Intervention
(n=69)
|
p-value
|
Gender
Female
Male
|
35 (48.6)
37 (51.4)
|
37 (53.6)
32 (46.4)
|
0.55
|
Ethnicity
Hispanic
Non-Hispanic
|
7 (9.7)
65 (90.3)
|
5 (7.3)
64 (92.8)
|
0.60
|
Race2
American Indian or Alaska Native
Asian
Black or African American
Native Hawaiian or Other Pacific Islander
White
|
2 (2.8)
28 (38.9)
11 (15.3)
0 (0.0)
29 (40.3)
|
1 (1.5)
23 (33.3)
4 (5.8)
0 (0.0)
36 (52.2)
|
1.00
0.49
0.10
--
0.16
|
Undergraduate Major3
Non-Science
Science
|
15 (20.8)
57 (79.2)
|
16 (23.2)
53 (76.8)
|
0.74
|
Undergraduate GPA4
Biology, Chemistry, Physics, and Math Courses
Total
|
3.90
(3.77-3.97)
3.90
(3.79-3.97)
|
3.92
(3.81-3.98)
3.91
(3.81-3.96)
|
0.33
0.40
|
MCATÒ Scores5
|
34.7
(31.8-37.7)
|
35.6
(32.5-38.7)
|
0.07
|
Age6 (years)
|
24.5
(24.0-26.0)
|
26.0
(24.0-27.0)
|
0.002
|
GPA = grade point average, MCATÒ = Medical College Admission TestÒ 1 Values for gender, ethnicity, race, and undergraduate degree are presented as n (%). Values for undergraduate grade point average and age are presented as median (inter-quartile range). Values for Medical College Admission Test scores are presented as mean (95% confidence interval). 2 Students were permitted to select all that apply, such that the sum of within-cell sample sizes may exceed the total group sample size. 3 Undergraduate major was categorized as either non-science or science. 4Ungraduate grade point average from the students’ American Medical College Application ServiceÒ (AMCASÒ) application. 5All students in this sample completed the test prior to the current version released in 2015. The scores presented represent the sum of the biological science, physical science, and verbal reasoning domain scores. 6Age was defined as the students’ age at start of the clerkship.
|
Table 1
Student characteristics by control and intervention groups
There were no significant differences in ordering of appropriate tests (3.1 vs. 3.2 tests/students, p = 0.55) and inappropriate tests (1.8 vs. 2.2, p = 0.13) between the intervention and control. Students in the intervention group had significantly lower Medicare allowable fees ($287.59 vs. $500.86, p = 0.04). For secondary outcomes, students in the intervention achieved higher percentiles on the NBME subject exam (78% vs. 67%, p = 0.10), but this was not statistically significant (Table 2).
Assessment
|
|
Control (66)
|
Intervention (69)
|
p
|
Standardized patient
|
|
|
|
|
|
Appropriate tests/student (SD)
|
3.2 (0.94)
|
3.1 (0.90)
|
0.55
|
|
Inappropriate tests/student (SD)
|
2.2 (1.5)
|
1.8 (1.3)
|
0.13
|
|
Median Medicare allowable
fees/student (IQR)
|
$500.86 ($227.90 – $870.33)
|
$287.59 ($169.87 – 681.18)
|
0.04
|
Median NBME percentile (IQR)
|
|
67 (42-88)
|
78 (59-92)
|
0.10
|
SD: Standard Deviation
IQR: Interquartile Range
NBME: National Board of Medical Examiners
|
Table 2
Student performance on end of clerkship assessments
Overall, more students in the intervention group felt their education in HVC was appropriate (81% vs. 56%, p = 0.02). There were no significant differences in how often students perceived unnecessary ordering of tests, their comfort level speaking up about an unnecessary test, or how often they witnessed discussion of costs, praise for not ordering an unnecessary test, or guidance to order tests that will not change management (Table 3).
Question
|
Answer
|
Control (37)
|
Intervention (42)
|
p
|
HVC Education
|
Appropriate
|
21 (57%)
|
34 (81%)
|
0.02
|
|
Inadequate
|
16 (43%)
|
8 (19%)
|
|
How Often Unnecessary Tests Ordered
|
Frequent/Often
|
10 (27%)
|
15 (36%)
|
0.41
|
|
Sometimes/Rarely/Never
|
27 (73%)
|
27 (64%)
|
|
Comfort Level Speaking Up Against Ordering Perceived Unnecessary Tests
|
Comfortable
|
16 (43%)
|
17 (41%)
|
0.80
|
|
Uncomfortable
|
21 (57%)
|
25 (60%)
|
|
How Often Discuss Costs
|
Frequent/Often
|
6 (17%)
|
9 (21%)
|
0.59
|
|
Sometimes/Rarely/Never
|
30 (83%)
|
33 (79%)
|
|
How Often Praised for Not Ordering an Unnecessary Test
|
Frequent/Often
|
16 (44%)
|
15 (36%)
|
0.43
|
|
Sometimes/Rarely/Never
|
20 (56%)
|
27 (64%)
|
|
How Often Team was Guided to Order More (unnecessary) Tests
|
Frequent/Often
|
8 (22%)
|
3 (7%)
|
0.07
|
|
Sometimes/Rarely/Never
|
29 (78%)
|
38 (93%)
|
|
HVC: High Value Care
|
Table 3
Student perceptions of high value and low value care during the clerkship
Most students rated the classroom sessions very good or excellent (75% for CEA and 90% for hospital charges session). Most students felt that the classroom sessions increased skills related to the topic somewhat or a lot (90% for CEA and 93% for hospital charges session) and prepared them to be a medical student on the IM clerkship. After the hospital charges sessions, most students would consider charges somewhat/a lot for labs (90%), medications (91%), and radiologic procedures (93%).
For the web-based modules, 88% of students rated them as good or excellent. For each individual module, students rated the following modules as good or excellent: acute kidney injury (85%), acute chronic obstructive pulmonary disease exacerbation (83%), acute deep vein thrombosis and pulmonary embolus (85%). Most students felt the virtual patients were similar to patients that they care for during the clerkship yet only half (50%) felt virtual patients should be a part of the clerkship.