During the study period, 136 students completed the IM Clerkship. Of those 136 students, 70 (intervention) received the HVC curriculum and 66 (control) received the standard education. There were no significant differences between the two groups with respect to gender, ethnicity, race, undergraduate major, undergraduate GPA, or MCAT® score (Table 1). Students in the intervention group were older compared to the usual education group (26 vs 25 years old, p = 0.007). Including classroom time, the total HVC curriculum time was less than four hours. No student opted out of having their assessment data used in the study.
Table 1. Student characteristics by control and intervention groups
Characteristic1
|
Control
(n=66)
|
Intervention
(n=70)
|
p-value
|
Gender
Female
Male
|
33 (50.0)
33 (50.0)
|
38 (54.3)
32 (45.7)
|
0.62
|
Ethnicity
Hispanic
Non-Hispanic
|
7 (10.6)
59 (89.4)
|
5 (7.1)
65 (92.9)
|
0.48
|
Race2
American Indian or Alaska Native
Asian
Black or African American
Native Hawaiian or Other Pacific Islander
White
|
1 (1.5)
23 (34.9)
11 (16.7)
0 (0.0)
30 (45.5)
|
1 (1.4)
23 (32.9)
4 (5.7)
0 (0.0)
37 (52.9)
|
1.00
0.81
0.06
--
0.39
|
Undergraduate Major3
Non-Science
Science
|
14 (21.5)
51 (78.5)
|
16 (22.9)
54 (77.1)
|
0.85
|
Undergraduate GPA4
Biology, Chemistry, Physics, and Math Courses
Total
|
3.87
(3.76-3.93)
3.88
(3.79-3.94)
|
3.92
(3.81-3.98)
3.91
(3.81-3.96)
|
0.06
0.07
|
MCAT Scores5
|
35.1
(32.1-38.1)
|
35.7
(32.5-38.8)
|
0.28
|
Age6 (years)
|
25.0
(24.0-26.0)
|
26.0
(24.0-28.0)
|
0.007
|
GPA = grade point average, MCAT = Medical College Admission Test
1 Values for gender, ethnicity, race, and undergraduate degree are presented as n (%). Values for undergraduate grade point average and age are presented as median (inter-quartile range). Values for Medical College Admission Test scores are presented as mean (95% confidence interval).
2 Students were permitted to select all that apply, such that the sum of within-cell sample sizes may exceed the total group sample size.
3 Undergraduate major was categorized as either non-science or science. Major information was unavailable for one student in the control group.
4Ungraduate grade point average from the students’ American Medical College Application Service (AMCAS) application.
5All students in this sample completed the test prior to the current version released in 2015. The scores presented represent the sum of the biological science, physical science, and verbal reasoning domain scores.
6Age was defined as the students’ age at start of the clerkship.
There were no significant differences in ordering of appropriate tests (3.1 vs. 3.2 tests/students, p = 0.55) and inappropriate tests (1.8 vs. 2.2, p = 0.13) between the intervention and control group. Students in the intervention group had significantly lower Medicare allowable fees ($287.59 vs. $500.86, p = 0.04). For secondary outcomes, students in the intervention achieved higher percentiles on the NBME subject exam (78% vs. 67%, p = 0.10), but this was not statistically significant (Table 2).
Table 2 Student performance on end of clerkship assessments
|
|
|
|
Assessment
|
|
Control (66)
|
Intervention (70)
|
p
|
Standardized patient
|
|
|
|
|
|
|
|
|
|
|
Appropriate tests/student (SD)
|
3.2 (0.94)
|
3.1 (0.90)
|
0.55
|
|
Inappropriate tests/student (SD)
|
2.2 (1.5)
|
1.8 (1.3)
|
0.13
|
|
Median Medicare allowable
fees/student (IQR)
|
$500.86 ($227.90 – $870.33)
|
$287.59 ($169.87 – 681.18)
|
0.04
|
Median NBME percentile (IQR)
|
|
67 (42-88)
|
78 (59-92)
|
0.10
|
|
|
|
|
|
SD: Standard Deviation
IQR: Interquartile Range
NBME: National Board of Medical Examiners
Response rate for the post-rotation survey was 60% (42/70) for intervention and 56% (37/66) for control. Overall, more students in the intervention group felt their education in HVC was appropriate (81% vs. 56%, p = 0.02). There were no significant differences in how often students perceived unnecessary ordering of tests, their comfort level speaking up about an unnecessary test, or how often they witnessed discussion of costs, praise for not ordering an unnecessary test, or guidance to order tests that will not change management (Table 3).
Table 3 Student perceptions of high value and low value care during the clerkship
|
|
|
|
Question
|
Answer
|
Control 37 (56%)
|
Intervention 42 (60%)
|
p
|
HVC Education
|
Appropriate
|
21 (57%)
|
34 (81%)
|
0.02
|
|
Inadequate
|
16 (43%)
|
8 (19%)
|
|
How Often Unnecessary Tests Ordered
|
Frequent/Often
|
10 (27%)
|
15 (36%)
|
0.41
|
|
Sometimes/Rarely/Never
|
27 (73%)
|
27 (64%)
|
|
Comfort Level Speaking Up Against Ordering Perceived Unnecessary Tests
|
Comfortable
|
16 (43%)
|
17 (41%)
|
0.80
|
|
Uncomfortable
|
21 (57%)
|
25 (60%)
|
|
How Often Discuss Costs
|
Frequent/Often
|
6 (17%)
|
9 (21%)
|
0.59
|
|
Sometimes/Rarely/Never
|
30 (83%)
|
33 (79%)
|
|
How Often Praised for Not Ordering an Unnecessary Test
|
Frequent/Often
|
16 (44%)
|
15 (36%)
|
0.43
|
|
Sometimes/Rarely/Never
|
20 (56%)
|
27 (64%)
|
|
How Often Team was Guided to Order More (unnecessary) Tests
|
Frequent/Often
|
8 (22%)
|
3 (7%)
|
0.07
|
|
Sometimes/Rarely/Never
|
29 (78%)
|
38 (93%)
|
|
HVC: High Value Care
Most students rated the classroom sessions very good or excellent (75% for CEA and 90% for hospital charges session). Most students felt that the classroom sessions increased skills related to the topic somewhat or a lot (90% for CEA and 93% for hospital charges session) and prepared them to be a medical student on the IM clerkship. After the hospital charges sessions, most students would consider charges somewhat or a lot for laboratory tests (90%), medications (91%), and radiologic procedures (93%).
For the web-based modules, 88% of students rated them as good or excellent. For each individual module, students rated the following modules as good or excellent: acute kidney injury (85%), acute chronic obstructive pulmonary disease exacerbation (83%), acute deep vein thrombosis and pulmonary embolus (85%). Most students felt the virtual patients were similar to patients that they cared for during the clerkship yet only half (50%) felt virtual patients should be a part of the clerkship.