Demographic characteristics of the patient sample
A total of 197 patients were enrolled during preoperative consultation. Their mean age was 54.8 (Standard Deviation (SD) 16.3), with 54.8% of the patients being female, 83.8% Caucasian, 5.6% African American, and 8.1% Hispanic (Table 1). The convenience sample of patients interviewed (n=27) had similar characteristics to the 197 patients who were surveyed: mean age 55.5 (SD [13.6]); 55.6% female; 100% Caucasian; and 7.4% Hispanic.
Demographics*
|
Patient
Cohort
(n=197)
N (%)
|
Interviewed
Cohort
(n=27)
N (%)
|
Age, years, mean (SD)
|
54.8 (16.3)
|
55.5 (13.6)
|
Gender
|
|
|
Female
|
108 (54.8)
|
15 (55.6)
|
Male
|
89 (45.2)
|
12 (44.4)
|
Race
|
|
|
Caucasian
|
165 (83.8)
|
27 (100)
|
Black or African American
|
11 (5.6)
|
0 (0)
|
Asian
|
6 (3.1)
|
0 (0)
|
American Indian or Alaskan Native
|
1 (0.5)
|
0 (0)
|
Missing racial information
|
14 (7.1)
|
0 (0)
|
Hispanic Ethnicity
|
|
|
No
|
175 (89.3)
|
24 (88.9)
|
Yes
|
16 (8.1)
|
2 (7.4)
|
Unknown
|
5 (2.5)
|
1 (3.7)
|
*Abbreviation: SD, standard deviation.
|
Table 1
Patient’s Survey and Interview Demographics.
Descriptive statistics of the SURPAS risk predictors of the patient sample
Of the 197 patients entered into the study, 180 (91.4%) had the SURPAS evaluation. Mean age was 54.0 (SD [16.6]), 51.1% were undergoing outpatient surgery, and 93.9% had independent functional status. The majority of patients were American Society of Anesthesiology physical status classification (ASA class) of III or less (92.2%). Most of the patients were seen by a general surgeon (48.9%), thoracic surgeon (28.3%), or vascular surgeon (9.4%). Only one patient surgery was undergoing an emergency operation. Average work RVU was 15.2 (SD [8.6]). The median SURPAS risk estimates for 30-day mortality, overall morbidity, and unplanned readmission were 0.1%, 3.9%, and 3.1%, respectively (Supplemental Table 1). This sample of 180 patients was slightly younger, had more outpatient surgeries, had lower ASA class and less emergency operations than the ACS NSQIP national sample during the same time period. (Supplemental Table 2).
Of the 17 patients who did not have the SURPAS evaluation, eight had missing or incomplete data entered into SURPAS, six did not undergo surgery, and three underwent operations that did not have CPT codes available in SURPAS.
Demographic characteristics of the provider sample
The nine surgical providers recruited included seven surgeons and two nurse practitioners (NPs) who agreed to participate in the study by implementing SURPAS. The providers who participated in the study included 55.6% females; 55.6% with a MD only; 22.2% with a MD and Master’s degree; and 22.2% who had Masters of Nursing degrees.
Integrated Qualitative and Quantitative Results
Patient surveys and illustrative quotes from patient interviews
Of the 197 patients in this study, 168 (85.3%) patients reported they discussed the risk of their upcoming surgery at their preoperative visit, five patients reported they did not have the risk discussion and four had missing data. The NPs in the pre-procedure clinic do not have risk assessment conversations with the patients, therefore, the 20 patients who were seen by the NPs subsequently did not have a risk discussion or a survey. The SURPAS risk estimates were documented in the patients’ medical record notes for the surgeons to review.
Of those who reported having the risk discussion, the majority of the patients (96.4%) reported that the surgeon used the SURPAS tool during their discussion; 88.1% of them reported that they understood their surgical risk, ”very well, I have no further questions’ and 10.7% reporting, “quite well, but I still have a question” (Table 2).
Patient Survey Questions
|
(N=197)
N (%)
|
Was the risk of your upcoming surgery discussed with you at your preoperative visit today?
|
|
Missing
|
24 (12.2)
|
No
|
5 (2.5)
|
Yes
|
168 (85.3)
|
|
(N=168)
|
Did the surgeon who will do your operation tell you about the risk?
|
|
Missing
|
1 (0.6)
|
No
|
1 (0.6)
|
Yes
|
166 (98.8)
|
Did the surgeon discuss the SURPAS tool with you?
|
|
Missing
|
1 (0.6)
|
No
|
5 (3.0)
|
Yes
|
162 (96.4)
|
How well do you understand the risk of your surgery?
|
|
Missing
|
2 (1.2)
|
Very well, I have no further questions
|
148 (88.1)
|
Quite well, but I still have a question
|
18 (10.7)
|
Somewhat, but I still have a few questions
|
0 (0)
|
Minimally, I still have a lot of questions
|
0 (0)
|
Not at all
|
0 (0)
|
Were you given the papers that show you the estimates of the risks of surgery?
|
|
No
|
3 (1.8)
|
Yes
|
165 (98.2)
|
|
(N=165)
|
If yes: was this document explained to you?
|
|
Missing
|
11 (6.7)
|
No
|
0 (0)
|
Yes
|
154 (93.3)
|
Was the surgical risk provided by SUPRAS less, the same, greater than you expected?
|
|
Missing
|
28 (17.0)
|
Greater than you expected it to be
|
21 (12.7)
|
The same
|
73 (44.2)
|
Less
|
43 (26.1)
|
Did seeing the risk make you want to talk to the provider more about your surgery?
|
|
Missing
|
21 (12.7)
|
No
|
118 (71.5)
|
Yes
|
26 (15.8)
|
Did using the SURPAS tool and seeing the personal risks affect your decision to have or not have the operation?
|
|
Missing
|
3 (1.8)
|
No
|
136 (82.4)
|
Yes
|
26 (15.8)
|
On a scale of 1-4, please circle the number that best reflects your opinion of the risk document given you.
|
|
Missing
|
4 (2.4)
|
It was very helpful
|
100 (60.6)
|
It was helpful
|
53 (32.1)
|
It was somewhat helpful
|
8 (4.9)
|
It was not helpful
|
0 (0)
|
Abbreviation: SURPAS, Surgical Risk Preoperative Assessment System.
|
Provider’s Responses to the SURPAS Survey
Provider’s Survey Questions
|
(N=197)
N (%)
|
Did you discuss the SURPAS tool with your patients?
|
|
Missing
|
8 (4.1)
|
No
|
23 (11.7)
|
Yes
|
166 (84.3)
|
Did you give the SURPAS handout to the patient?
|
|
Missing
|
12 (6.1)
|
No
|
20 (10.1)
|
Yes
|
165 (83.8)
|
If yes: Did you explain the SURPAS patient handout to the patient?
|
|
|
Missing
|
1 (0.6)
|
|
No
|
0 (0)
|
|
Yes
|
164 (99.4)
|
Was the surgical risk provided from SURPAS
|
|
Missing
|
32 (16.2)
|
Less
|
27 (13.7)
|
The same
|
101 (51.3)
|
Greater than you expected it to be
|
37 (18.8)
|
On a scale of 1-4, please circle the number that best reflects your opinion of the risk document given to the patient?
|
|
Missing
|
14 (7.1)
|
It was not helpful
|
19 (9.6)
|
It was somewhat helpful
|
46 (23.4)
|
It was helpful
|
98 (49.8)
|
It was very helpful
|
20 (10.2)
|
Did using the SURPAS tool change the interaction with the patient during the preoperative clinic visit?
|
|
Missing
|
14 (7.1)
|
No
|
95 (48.2)
|
Yes
|
88 (44.7)
|
If yes: Did you find the change to be:
|
|
|
Missing
|
1 (1.1)
|
|
Detrimental
|
0 (0)
|
|
Neutral
|
4 (4.6)
|
|
Beneficial
|
83 (94.3)
|
Did using the SURPAS tool and seeing the personal risks affect your decision to do or not do the operation?
|
|
Missing
|
12 (6.1)
|
No
|
182 (92.4)
|
Yes
|
3 (1.5)
|
Did using the SURPAS tool and seeing the personal risks change any aspects of the preoperative workup?
|
|
Missing
|
10 (5.1)
|
No
|
178 (90.4)
|
Yes
|
9 (4.6)
|
Did using the SURPAS tool and seeing the personal risks change any aspects of patient management or technical aspects in the operating room?
|
|
Missing
|
14 (7.1)
|
No
|
179 (90.9)
|
Yes
|
4 (2.0)
|
Did using the SURPAS tool and seeing the personal risks change any aspects of the planned postoperative care for this patient?
|
|
Missing
|
13 (6.6)
|
No
|
181 (91.9)
|
Yes
|
3 (1.5)
|
Abbreviation: SURPAS, Surgical Risk Preoperative Assessment System.
|
Table 2
. Patient’s and Provider’s Responses to the SURPAS Survey Questions.
“They come in and they talk in terms above your head, and they say, "Hope you understood and then they leave. He doesn't. He makes sure you understand." He often asked, "Do you have any questions?" so that was good" (T4/4)
Of the patients who reported having a risk discussion with their providers, 165 (98.2%) reported that they received the SURPAS hand out that visually outlined their individual risk estimates, with 154 (93.3%) reporting that the SURPAS tool was explained to them; 26 (15.8%), reported that seeing their risk estimates made them want to talk to their providers more and that it affected their decision to have or not have the operation (Table 2).
“. . . When I came here, I was a lot more worried, even if he said, “Okay, you can get the surgery. We should get the surgery.” But now seeing the percentages and how low they are, I mean, I have faith that it’s gonna work” (T2/2)
“I understood it perfectly. The concept is just knowing what risks in each category there was.”(E75/158)
Provider surveys and illustrative quotes from provider interviews
The participating providers completed a survey asking about the patient’s calculated SURPAS risk estimates and their discussion of those risks for each of the 197 patient visits (Table 2). The surgical providers reported that they discussed the SURPAS tool with 166 (84.3%) of their patients (20 patients were seen by the pre-procedure clinic NPs who did not do risk discussions). For these 166 patients, the providers said that they gave the SURPAS handout to 165 patients and explained the handout for 164 patients.
“I think it’s a great idea and I think it’s super easy to use. I think that to widely implement it and get people to use it, it has to be simple and quick. I have been using it predominantly in my clinic, which is all elective, super-low risk, generally healthy patients, so for my patient population, all of the things (predicted adverse event rates) are less than one percent. No different from the national average or whatever.“ (Provider #3)
“Very easy to enter the data after you have seen a patient in clinic. Very much like the documentation tool. You can generate the note of what the predicted risks are for that patient compared to the national database and simply copy that into your clinical note.” (Provider #4)
In 44.7% of the visits the surgeon reported that the SUPRAS tool changed their interaction with the patients, and 94.3% of the time this was ‘beneficial’.
“You know for the surgeries I do it’s really helpful. Just because it’s a big, long and complicated surgery and it helps patients see that visually. The more ways we can talk around what this is like, the better it is for them. It also can help us in decision making. For some patients that are really that sick, we can choose second line treatment, which may not be quite as good, but if they’re that sick we pick it up from the SURPAS tool.” (Provider #7)
The providers reported that the SURPAS tool rarely changed the decision to do the operation (1.5%), aspects of the preoperative work-up (4.6%), aspects of patient management or technical aspects in the operating room (2.0%), or aspects of the postoperative care (1.5%).
“Some of the results we've gotten have been surprising to me, in terms of the risks of the surgery for certain patients. I assumed it would be on the higher end of the risk, but the actual number seems higher than what I would've put on the risk... I think it's been helpful for patients to see the numbers, at least from when I talked to them. It feels to me that they're more educated about their operation. It hasn't actually changed what I do, in terms of surgery... I haven't cancelled an operation because of the high risk. I think some of the patients have been reassured by the low risk. Whenever I consent them and I say there's a risk of heart attack, stroke, or death it sounds very bad. When they see that it's 0.8 percent, then I think they feel a little bit better.” (Provider #10)
“If SURPAS helps them understand what their risks are in making a decision and understanding how the things that they’re focused on might be better or worse…” (Provider #4)
Comparison of Patients and Providers to the Same Questions
The left side of Figures 1, 2, and 3 compare patient and provider answers to three questions that were asked of both patients and providers. The right side of each figure gives illustrative opinions from the individual interviews of the patients and providers.
In response to Question 1) “Was the surgical risk provided by SURPAS less, the same, or greater than you expected?” patients tended to overestimate, and surgeons tended to underestimate risk compared to the SURPAS risk estimates (Figure 1, p=0.01).
In response to Question 2) “Did using the SURPAS tool and seeing the personal risks affect your decision to [have/do] or not [have/do] the operation?” a higher percentage of patients (15.7%) compared to providers (1.5%) said that seeing the personal risks affected their decision to have/do or not have/not do the operation (Figure 2, p<0.0001).
In response to Question 3) “On a scale of 1-4, please circle the number that best reflects your opinion of the risk document given to you/the patient’ in terms of helpfulness?” patients rated the risk document as more helpful than did the surgeons (Figure 3, p<0.0001).
Relationship of the Patients’ Survey Answers to the Patients’ Risk Estimates
We obtained the SURPAS calculated risk estimates for mortality, overall morbidity, and unplanned readmission on 180 patients (91.4%) and assessed if there were any relationships between the patient’s SURPAS risk estimates and their responses to the four survey questions specifically asking about the SURPAS tool (Table 3). Patients who responded ‘yes’ to “Did seeing the risk make you want to talk to the provider more about your surgery?” tended to have larger median risk estimates for overall morbidity versus patients who responded ‘no’ (4.0% vs. 2.9%: p=0.06). Patient risk was not related to the patient’s perceptions about whether the SURPAS risk was the same, greater, or less than their previous perceptions of their risk, their perceptions about the helpfulness of the risk document, nor their decisions to have or not have the operation.
Question*
|
|
30-day mortality
|
Overall morbidity
|
Unplanned Readmission
|
N
|
Median (IQR)
|
P value†
|
Median (IQR)
|
P value†
|
Median (IQR)
|
P value†
|
Was the surgical risk provided from SURPAS:
|
|
The same
|
70
|
0.1 (<0.1-0.3)
|
0.58
|
2.7 (1.6-6.6)
|
0.74
|
2.3 (1.6-5.1)
|
0.88
|
|
Less
|
42
|
0.1 (<0.1-0.2)
|
|
3.5 (1.7-6.5)
|
|
2.8 (1.3-4.9)
|
|
|
Greater than you expected it to be
|
21
|
0.1 (<0.1-0.6)
|
|
5.0 (1.5-8.7)
|
|
3.2 (1.7-5.9)
|
|
|
Missing‡
|
47
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
On a scale of 1-4, please circle the number that best reflects your opinion of the risk document given to you?
|
|
It was not helpful
|
0
|
0 (0-0)
|
0.94
|
0 (0-0)
|
0.72
|
0 (0-0)
|
0.78
|
|
It was somewhat helpful
|
6
|
0.1 (<0.1-1.1)
|
|
2.0 (1.1-12.3)
|
|
2.0 (1.2-7.5)
|
|
|
It was helpful
|
51
|
0.1 (<0.1-0.4)
|
|
3.3 (1.5-10.5)
|
|
2.6 (1.5-5.4)
|
|
|
It was very helpful
|
99
|
0.1 (<0.1-0.3)
|
|
3.5 (1.7-6.9)
|
|
3.0 (1.8-5.1)
|
|
|
Missing‡
|
24
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Did seeing the risks make you want to talk to the provider more about your surgery?
|
|
No
|
115
|
0.1 (<0.1-0.3)
|
0.07
|
2.9 (1.5-6.5)
|
0.06
|
2.3 (1.6-5.0)
|
0.28
|
|
Yes
|
25
|
0.2 (<0.1-0.5)
|
|
4.0 (2.6-11.5)
|
|
3.3 (1.8-6.1)
|
|
|
Missing‡
|
40
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Did using the SURPAS tool and seeing the personal risks affect your decision to have or not have the operation?
|
|
No
|
133
|
0.1 (<0.1-0.3)
|
0.30
|
3.3 (1.6-8.2)
|
0.40
|
2.6 (1.7-5.5)
|
0.26
|
|
Yes
|
27
|
0.1 (0.1-0.3)
|
|
3.9 (1.9-8.9)
|
|
3.3 (1.9-5.5)
|
|
|
Missing‡
|
20
|
|
|
|
|
|
|
*Abbreviations: IQR, interquartile range; SURPAS, Surgical Risk Preoperative Assessment System.
†Since the risk estimates were skewed, the P values were calculated by Wilcoxon rank sum and bolded when less than 0.05.
‡Missing was due to no responses to the survey for the given question.
|
Table 3
Median and Interquartile range of Risk Estimates Stratified by Response to Patient Survey Question Related to the SURPAS Tool.
Relationship of the Providers’ Survey Answers to the Patients’ Risk Estimates
We investigated if there were any relationships between patient risk estimates and how surgeons responded to those same four questions (Table 4). Patient risk had a profound effect on the providers’ responses: as patient risk increased, providers had less certainty about their expectation of the patient’s risk compared to SURPAS calculations of that risk; they perceived SURPAS as being more helpful; and they thought that the SURPAS tool changed their interaction with the patient (all p-values <0.0001).
Question*
|
|
30-day mortality
|
Overall morbidity
|
Unplanned Readmission
|
N
|
Median (IQR)
|
P value†
|
Median (IQR)
|
P value†
|
Median (IQR)
|
P value†
|
Was the surgical risk provided from SURPAS:
|
|
The same
|
94
|
<0.1 (<0.1-0.2)
|
<0.0001
|
1.8 (1.3-4.9)
|
<0.0001
|
1.9 (1.3-3.3)
|
<0.0001
|
|
Less
|
26
|
0.2 (0.1-0.5)
|
|
6.3 (3.2-14.9)
|
|
4.5 (3.0-8.6)
|
|
|
Greater than you expected it to be
|
36
|
0.7 (0.2-1.9)
|
|
10.7 (5.3-14.4)
|
|
5.9 (3.7-7.4)
|
|
|
Missing‡
|
24
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
On a scale of 1-4, please circle the number that best reflects your opinion of the risk document given to the patient?
|
|
It was not helpful
|
18
|
<0.1 (<0.1-<0.1)
|
<0.0001
|
1.6 (1.0-1.8)
|
<0.0001
|
1.8 (1.2-1.9)
|
<0.0001
|
|
It was somewhat helpful
|
44
|
0.1 (<0.1-0.3)
|
|
2.8 (1.5-5.8)
|
|
2.3 (1.4-4.4)
|
|
|
It was helpful
|
91
|
0.1 (<0.1-0.5)
|
|
4.1 (1.7-11.2)
|
|
3.3 (1.9-5.9)
|
|
|
It was very helpful
|
20
|
0.4 (0.1-2.0)
|
|
11.9 (3.9-22.4)
|
|
5.6 (3.7-9.8)
|
|
|
Missing‡
|
7
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Did using the SURPAS tool change the interaction with the patient during the preoperative clinic visit?
|
|
No
|
87
|
<0.1 (<0.1-0.2)
|
<0.0001
|
1.9 (1.4-5.3)
|
<0.0001
|
1.9 (1.43-3.7)
|
<0.0001
|
|
Yes
|
85
|
0.2 (<0.1—1.1)
|
|
5.5 (2.8-12.3)
|
|
4.0 (2.3-6.5)
|
|
|
Missing‡
|
8
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
If yes: Did you find the change to be:
|
|
Detrimental
|
0
|
0 (0-0)
|
0.72
|
0 (0-0)
|
0.33
|
0 (0-0)
|
0.37
|
|
Neutral
|
4
|
0.1 (0.1-1.4)
|
|
3.1 (1.3-9.9)
|
|
2.8 (1.3-5.3)
|
|
|
Beneficial
|
80
|
0.2 (<0.1-1.1)
|
|
5.5 (2.9-12.6)
|
|
4.1 (2.3-6.7)
|
|
|
Missing‡
|
1
|
|
|
|
|
|
|
*Abbreviations: IQR, interquartile range; SURPAS, Surgical Risk Preoperative Assessment System.
†Since the risk estimates were skewed, the P values were calculated by Wilcoxon rank sum and bolded when less than 0.05.
‡Missing was due to no responses to the survey for the given question.
|
Table 4
Median and Interquartile range of Risk Estimates Stratified by Response to Surgeon Survey Question Related to the SURPAS Tool.