Sample characteristics
Data were contributed by 7709 respondents from the 2016 survey and 7497 respondents from 2018 survey (total n = 15206). The mean ages of the 2016 and 2018 survey respondents were 64.3 years (SD =17.1) and 65.9 years (SD =16.5), respectively. The age in both 2016 and 2018 surveys had moderately negatively (left) skewed distributions. Details of the mode of hospital stay and category of health services are shown in Table 1. In both years, the metropolitan health services form the majority of the data.
Table 1 Mode of Hospital Admission and Category of Health Services
Characteristics
|
2016
|
2018
|
|
n (%)
|
n (%)
|
Hospital stays
|
Planned in advance
|
3994 (51.8)
|
3978 (53.1)
|
Emergency
|
2887 (37.4)
|
2717 (36.2)
|
Other
|
473 (6.1)
|
470 (6.3)
|
Missing
|
355 (4.6)
|
332 (4.4)
|
Categories of Health Services
|
Metro
|
3119 (40.5)
|
3071 (41.0)
|
Regional
|
1965 (25.5)
|
1866 (24.9)
|
Rural
|
2270 (29.4)
|
2228 (29.7)
|
Missing
|
355 (4.6)
|
332 (4.4)
|
Trends in patients experience scores between 2016 and 2018
There were 56 questions from the VHES with Likert type response scale; they were standardised, and mean scores were calculated. The overall patient experience measure for 2016 responses (M=93.4, SD = 13.1) and 2018 responses (M=93.6, SD =12.9) was consistent between the two years.
Despite no difference in the overall patient experience measure between 2016 and 2018 responses, nine out of 56 VHES questions had increased mean scores, indicating an improvement in those areas. Notably, the same five questions had the lowest scores in 2016 and 2018, suggesting potential areas such as involvement in discharge planning and communication with doctors and caregivers to focus on for future interventions (See Table 2 for details).
Table 2 Mean scores on VHES questions in 2016 and 2018 responses
VHES questions with significant differences between 2016 and 2018
|
Mean Score (2016) (2018)
|
t
|
How would you rate the politeness and courtesy of admissions staff?
|
94.90
|
95.44
|
3.09**
|
Did you feel friends and family were welcome to visit you?
|
96.50
|
97.26
|
2.91**
|
Were the nurses treating you compassionate?
|
93.96
|
94.76
|
2.86**
|
Were the doctors treating you compassionate?
|
92.18
|
92.95
|
2.32*
|
If you needed to talk to a nurse, did you get the opportunity to do so?
|
88.46
|
89.54
|
2.79**
|
How would you rate how well the doctors and nurses worked together?
|
86.27
|
86.97
|
2.65**
|
Did you see hospital staff wash their hands, use hand gel to clean their hands, or put on clean gloves before examining you?
|
89.54
|
90.80
|
3.01**
|
Do you think the time you had to wait from arrival at hospital until you were taken to your room or ward was?
|
87.22
|
88.35
|
2.52*
|
At other times during your hospital stay did you have enough privacy?
|
89.57
|
90.52
|
2.47*
|
VHES questions with lowest scores
|
Mean Score
(2016) (2018)
|
t
|
If you needed to talk to a doctor, did you get the opportunity to do so?
|
81.43
|
82.22
|
1.44
|
How would you rate the hospital food?
|
79.28
|
79.60
|
0.88
|
Did you feel you were involved in decisions about your discharge from hospital?
|
76.23
|
77.61
|
2.30*
|
How much information about your condition or treatment was given to your family, carer or someone close to you?
|
70.01
|
69.08
|
1.15
|
Did you receive copies of communications sent between hospital doctors and your GP?
|
66.03
|
66.90
|
1.42
|
* = p <.05, ** = p <.01
Statically significant differences at p < .01 in the overall experience measure were found in comparing metropolitan, regional and rural health services in both 2016 and 2018 responses. Overall patient experience mean score of rural health services were higher than those of the regional and metropolitan health services for both years. However, the difference in mean score and effect size (eta squared =.03) was small. The statistical difference between the categories of gender identification (Male, Female) was found only in the 2018 responses, with a small magnitude of differences in the means (eta squared = .003). As such, further analysis based on these group differences were not conducted. See Table 3 for full details.
Table 3 One-Way ANOVA of Category of Health Services and Gender Identification on Overall Patient Experience
One-Way ANOVA
|
|
|
|
|
|
|
(A) Metropolitan
M (SE)
|
(B)
Regional
M (SE)
|
(C)
Rural
M (SE)
|
F-ratio
|
Differences
|
2016
|
91.10 (.261)
|
93.45 (.297)
|
96.41 (.193)
|
114.03***
|
C>B>A
|
2018
|
91.37 (.262)
|
93.60 (.293)
|
96.67 (.197)
|
114.45***
|
C>B>A
|
T-Test
|
(A) Male
M (SE)
|
(B) Female
M (SE)
|
|
t
|
Differences
|
2016
|
93.60 (.232)
|
93.18 (.210)
|
|
1.33
|
-
|
2018
|
94.39 (.218)
|
92.93 (.219)
|
|
4.67***
|
A>B
|
***= p<.001 **p>.01; *p<.05
Frequency of interventions reported by health services in 2017
There were between one to four interventions coded for each of the health services, with reliability using Kappa index (K = 0.8). As depicted in Figure 1, intervention categories most frequently reported were physical environment, followed by patient-staff communication and discharge planning.
Measure invariance of 2016 and 2018 cohort
Testing for measurement invariance ensures that the comparison across the 2016 and 2018 cohorts were both meaningful and valid. The baseline model had (154) = 2962.00; p< 0.001; ; df ratio of 19.33; the RMSEA = 0.04 and the NFI, CFI and TLI all above 0.9 (see Table 4). Having established the baseline, testing for weak factorial invariance was conducted by constraining factor loading matrices. The result of is 4.26; p 0.005 for Model 1 showed some improvement from the baseline model, and the RMSEA improved with NFI, TLI, CFI consistent. The next step was conducted on testing for strong equivalence by adding additional constraints on Model 2 where the elements of the intercept matrices were constrained. The result of is 1.74; p 0.051 showed Model 2 improved in fit when assuming intercepts are equal. Further testing for strict factorial invariance by constraining the errors terms was conducted. The result of is 6.30; p 0.001 showed the Model 3 further improved when error terms are assumed equal. The testing for elegant factorial invariance showed that Model 4 with is 3.83; p 0.001 is a good fit with RMSEA, NFI, TLI, CFI were consistent with Model 3. The finding supports that the respondents from 2016 and 2018 interpreted the VHES survey measure in a conceptually similar way.
Table 4 Measure Invariance of 2016 and 2018 respondents
Model Comparison
|
(df)
p
|
p
|
|
RMSEA
|
NFI
|
TLI
|
CFI
|
1. Baseline
(configural invariance)
|
2962.00 (154) p<0.001
|
19.33-
|
-
|
0.04
|
0.92
|
0.91
|
0.92
|
2. Model 1vs. Baseline
Testing Invariance of
(weak factorial invariance)
|
3008.96 (165)
P<0.001
|
46.96 (11)
p 0.005
|
4.26
|
0.03
|
0.92
|
0.92
|
0.92
|
3. Model 2 vs. Model 1 Testing Invariance of
(strong factorial invariance)
|
3030.57 (178)
P<0.001
|
21.61(13)
p 0.051
|
1.74
|
0.03
|
0.92
|
0.92
|
0.92
|
4. Model 3 vs. Model 2 Testing Invariance of
(strict factorial invariance
|
3113.59 (191)
P<0.001
|
82.02 (13)
p 0.001
|
6.30
|
0.03
|
0.92
|
0.93
|
0.92
|
5. Model 4 vs. Model 3 Testing invariance of
(elegant factorial invariance)
|
3117.42 (192)
p=0.00
|
3.83 (1)
p 0.001
|
3.83
|
0.03
|
0.92
|
0.93
|
0.92
|
Association between interventions and VHES
The impact of these interventions was assessed firstly on the overall patient experience measure and subsequently on outcome measures derived from measuring specific aspects of care: 1) communication, 2) respect and dignity, 3) emotional support, 4) discharge planning, 5) treatment and disease education and 6) physical environment measure.
Impact of individual intervention category on overall patient experience measure
Table 5 One-Way ANOVA and T-Test of Numbers of Interventions on Overall Patient Experience
One-Way ANOVA
|
|
|
|
|
|
|
|
Intervention category
|
(A) None
M (SE)
|
(B) One intervention
M (SE)
|
(C) Two interventions
M (SE)
|
(D) Three interventions
M (SE)
|
(E) Four interventions
M (SE)
|
F-ratio
|
Differences
(Post-hoc)
|
Integration of care
|
93.26 (.167)
|
95.16 (.411)
|
96.51 (.550)
|
|
|
15.55***
|
C>B>A
|
Discharge plan
|
93.57 (.208)
|
93.62 (.233)
|
93.74 (.677)
|
|
|
0.38
|
-
|
Wait-time & Access
|
93.85 (.165)
|
92.80 (.373)
|
|
90.65 (1.38)
|
|
7.61***
|
A>B
|
Staff – Staff Communication
|
93.31 (.163)
|
96.92 (.362)
|
93.66 (1.02)
|
|
|
21.16***
|
B>A&C
|
Staff-Patient Communication
|
93.21 (.224)
|
95.02 (.238)
|
92.82 (.361)
|
89.50 (1.36)
|
|
16.23***
|
B>A&C&D
A>C&D
|
Physical environment
|
93.57 (.205)
|
93.20 (.282)
|
95.18 (.417)
|
91.51 (.865)
|
97.71 (.680)
|
7.71***
|
C>A&B&D
E>A&B&C&D
|
T-Test
|
|
|
|
|
|
|
|
Intervention category
|
(A) None
M (SE)
|
(B) One intervention
M (SE)
|
|
|
|
t
|
Differences
|
Care Continuity
|
93.69 (.155)
|
91.98 (.708)
|
|
|
|
6.36*
|
A>B
|
Respect & Dignity
|
93.20 (.165)
|
96.83 (.309)
|
|
|
|
57.17***
|
B>A
|
Emotional Support
|
93.95 (.156)
|
90.59 (.551)
|
|
|
|
45.91***
|
A>B
|
Treatment & disease education
|
93.51 (.153)
|
98.51 (.503)
|
|
|
|
19.71***
|
B>A
|
*p<.05, **p<.01, ***p<.001
The differences on the overall patient experience measure associated with the number of interventions applied are shown in Table 5. There were significant differences with the application of integration of care interventions (p<.001) Two interventions were significantly better than one intervention which in turn was better than no intervention in improving overall patient experience. Care integration often involves changing existing clinical practices and processes across teams, and the result suggests that effective outcomes may require several coordinated interventions. There was no significant difference as a result of implementing discharge planning interventions.
Application of one intervention for staff-staff communication and staff-patient communication showed significant differences (p<.001) being higher scores than no intervention and two interventions. Similarly, for respect and dignity and treatment and disease education, significantly higher scores were found with one intervention compared to none. This suggests a carefully targeted intervention in staff-staff communication, staff-patient communication, respect and dignity and treatment and disease education could lead to a significant increase in overall patient experience score.
Overall patient experience measure did not increase with the use of waiting time and access to service, care continuity and emotional support interventions; no intervention was significantly higher than one or multiple interventions. This suggests that applying ineffective interventions in these areas decreased the overall patient experience.
Impact of related interventions on overall patient experience measure
Table 6 Regression Analysis Summary for Related Categories of Interventions on Overall Patient Experience
Grouped Interventions
|
Model 1
|
Model 2
|
Model 3
|
Model 4
|
|
B
|
t
|
B
|
t
|
B
|
t
|
B
|
t
|
Care Continuity
|
-.030
|
-2.55**
|
|
|
|
|
-.039
|
-3.21***
|
Discharge
Plan
|
.005
|
.460
|
|
|
|
|
-.004
|
-.330
|
Staff-Staff Communication
|
|
|
.057
|
4.85***
|
|
|
.041
|
3.41***
|
Staff-Patient Communication
|
|
|
-.011
|
-.96
|
|
|
.024
|
1.79
|
Respect & Dignity
|
|
|
|
|
.080
|
6.81***
|
.085
|
7.06***
|
Emotional Support
|
|
|
|
|
-.069
|
-5.93***
|
-.077
|
-5.65***
|
R2
|
.001
|
.003
|
.013
|
.017
|
R2
|
|
|
.002
|
|
.010
|
|
.004
|
|
*=p<.05, **=p<.01, ***=p<.001
The effect of categories of intervention on the overall measure of patient experience is shown in Table 6. A hierarchical regression was used. Care continuity was significant (p<.01), but discharge planning was not significant. The two categories explained a negligible variance (R2= .001). Model tested the effects of the next category of communication. This explained a negligible increased in variance (R2= .003). Staff-staff communication was significant (p<.001); staff-patient communication was not significant. The next model on respect and dignity and emotional support significantly explained more variance in overall patient experience (R2=.013). Respect and dignity was significant (p<.001), and finally, emotional support was negative and significant. This suggests emotional support interventions in this study reduced overall patient experience within this group category. Perhaps this indicates that emotional support interventions required further exploration of their acceptability and adherence by the healthcare professionals and patients. Model 4, with all the categories tested, explained slightly more variance in overall patient experience (R2=.017). This suggests the need for prioritisation of intervention categories for their effect on overall patient experience when these categories of interventions were to be implemented.
Impact of intervention on specific outcome measures
The impact of each intervention on the corresponding outcome measures was examined. Internal consistency was checked for each set of questions, only those with Cronbach’s alpha (≥ 0.7), was used for further analysis as specific outcome measures. No valid set of questions for staff-staff communication, integration of care, waiting time and access, and care continuity were found and as such outcome measures in these areas were not included in the analysis (see Additional file 2 for list of questions).
Table 7 Correlations of Specific Outcome Measures
Outcome Measures
|
Cronbach’s α (No. of Qs)
|
1
|
2
|
3
|
4
|
5
|
6
|
1. Discharge Planning
|
0.8 (4)
|
1
|
|
|
|
|
|
2. Physical Environment
|
0.8 (2)
|
.534
|
1
|
|
|
|
|
3. Staff-Patient Communication
|
0.8 (6)
|
.847
|
.569
|
1
|
|
|
|
4. Respect & Dignity
|
0.7 (4)
|
.769
|
.647
|
.913
|
1
|
|
|
5. Emotional support
|
0.8 (5)
|
.820
|
.604
|
.983
|
.988
|
1
|
|
6. Treatment & Disease Education
|
0.7 (4)
|
.781
|
.474
|
.863
|
.720
|
.805
|
1
|
There were strong positive correlations between the specific outcome measures of discharge planning, staff-patient communication, respect and dignity, emotional support and treatment and disease education outcome measures with the exception of the physical environment. (see Table 7).
Table 8 Impact of Interventions (both application and level of application) on Corresponding Outcome Measures
Interventions
|
(A) None
M (SE)
|
(B) One level
M (SE)
|
(C) Two levels
M (SE)
|
(D) Three levels
M (SE)
|
(E) Four levels
M (SE)
|
F-ratio
|
Differences
(Post-hoc)
|
Discharge Planning
|
89.40
(.326)
|
89.21
(.381)
|
88.68
(1.08)
|
|
|
.22
|
-
|
Physical environment
|
94.14
(.189)
|
92.28
(.283)
|
95.97
(.404)
|
92.47 (.686)
|
97.77
(.707)
|
17.82***
|
E>D&B&A
|
Staff-Patient communication
|
87.27
(.377)
|
89.35
(.447)
|
87.26
(.557)
|
82.88
(1.77)
|
|
7.49***
|
B>A&C&D
A>D
|
Respect & dignity
|
93.89
(.162)
|
96.72
(.323)
|
|
|
|
36.41***
|
B>A
|
Emotional support
|
89.86
(.386)
|
86.21
(1.21)
|
|
|
|
9.78**
|
A>B
|
Treatment & disease education
|
86.18
(.421)
|
97.43
(1.52)
|
|
|
|
5.07*
|
B>A
|
*p<.05, **p<.01, ***p<.001
The impact of interventions (both application and level of application) on specific outcome measures were presented in Table 8. There were significant differences in staff-patient communication outcome measure (p<.001), where one intervention was significantly better than no intervention or multiple interventions. This suggests that focusing on one effective uniform communication intervention would provide a good experience for patients’ interaction with staff rather than using multiple interventions. Similarly, for respect and dignity and treatment and disease education significantly higher scores in the corresponding outcome measures were found with one intervention compared to none. Conversely, emotional support, no intervention was significantly higher than one or multiple interventions, suggesting that the intervention had a negative effect. Regardless of the number of discharge planning interventions, they made no significant difference. However, the mean scores (88% -89%) of the discharge planning outcome measure across the number of interventions were all above the 75% target level set by the DHHS. The mixed findings on the physical environment could be due to the limited two questions (focusing only on cleanliness) in the outcome measure and not an accurate indicator of the changes in the physical environment. In summary, these results on specific outcome measures, are aligned with their impact on the overall patient experience measure.