An Australian health network with multiple hospital sites was selected for this study because it was undertaking a culture change intervention across its member hospital sites. The study was designed to establish a baseline of employee sentiment across the selected hospital network using a range of online job sites. Four of the hospitals within this network are represented within the online job sites: Indeed, Glassdoor and Seek. In total, 103 online reviews were extracted from these online sites for the period of 2014 to April 2020, and subject to qualitative and thematic analysis as represented in Figure 1.
Calculation
The characteristics of reviewers who posted the online reviews were determined through a combination of self-reported job titles and narrative descriptions within reviews of the roles undertaken during employment. The three online job sites offer reviewers the option of selecting ratings across the broad themes listed in Table 1, namely, work-life balance, salary and benefits, career development, management, and culture. The rating system offered to users across all sites allowed each theme to be scored on a five-point Likert scale. The average rating provided by each reviewer was calculated against the maximum score allocated by each site to its themes (Indeed – 25, Glassdoor – 25 and Seek – 30). Overall sentiment ratings were calculated by converting each review’s site-based rating into standardised percentages.
Table 1
Employee rating themes and rating scales across three sites.
| Indeed | Glassdoor | Seek |
| Work / Life Balance (5) | Work / Life Balance (5) | Work / Life Balance (5) |
| Salary / Benefits (5) | Compensation and Benefits (5) | Benefits and Perks (5) |
| Job Security / Advancement (5) | Career Opportunities (5) | Career Development (5) |
| Management (5) | Senior Management (5) | Management (5) |
| Job Culture (5) | Culture and Values (5) | Working Environment (5) |
| Diversity and Equal Opportunity (5) |
| Percentage calculated against a total of 25 points | Percentage calculated against a total of 25 points | Percentage calculated against a total of 30 points |
Reviewers are requested to provide sentiment ratings for each theme, and we calculated an average percentage score for each review. Reviewers are also asked to rate whether they would recommend this employer using a dichotomous two-point rating scale (yes / no), but not all reviewers completed this item. One site (Indeed) did not offer the option of recommendations. 42 (41.2%) out of the 102 usable reviews across Glassdoor and Seek provided either positive or negative recommendations. The sentiment and employer recommendation scores were compared against the textual content of the review to determine consistency with the reviewer sentiment score provided. When narrative content conflicted with the sentiment rating, the narrative sentiment expressed by the reviewer was given precedence, which occurred in only one case.
All reviews were then given an overall sentiment score coded as neutral, positive, or negative based on a combination of the sentiment and employer recommendation rating (Table 2).
Table 2
Criteria for assigning overall sentiment score.
| Positive | Neutral | Negative |
| Average sentiment score ≥60 and Employer recommendation positive, or no recommendation provided | Average sentiment score≥50 and <60 and no negative recommendation provided | Employer recommendation score negative, regardless of sentiment score OR where the sentiment score <50 |
Reviewer sentiment was coded as positive if the average rating was 60 or above and the reviewer had given the employer a positive "recommendation" where the field was available. Positive sentiment coding was retained in the absence of an employer recommendation if the sentiment rating was 60 or above. Neutral sentiment was coded for ratings between 50 and 60. Where reviewers had provided a negative employer recommendation, sentiment was coded as negative irrespective of sentiment rating. All sentiment ratings below 50 were coded as negative.
Each of the narrative reviews was examined to identify keywords, associations between terms, and usage within the context of the entire review to identify overall sentiment. Simple automated analytical methods such as keyword counting cannot be applied to understand the overall sentiment of these reviews as complex contextual information is often provided. Keywords such as “challenge” or “performance” could be used with positive, negative or neutral connotations. Therefore, it was imperative to perform a thematic analysis and clustering manually on the textual content of the reviews. Reviews were categorised and clustered against eight pre-determined safety culture themes. These themes were derived from the most used assessment tools for self-reported measures of occupational health and safety and safety culture across the network of hospitals in this study. These themes were: Perceptions of Management, Risk Management, Safety Climate, Teamwork Climate, Working Conditions, Stress Recognition, Job Satisfaction, and Learning, Training and Development.