This study presents some major findings. First, the study confirmed that the 8-factor model showed good fitness to the data. The model achieved configural and metric invariance but not scalar invariance. The study also found that reliability values were all acceptable and all the dimensions, except “standard”, attained the recommended 0.50 AVE value for convergent validity. With regard to discriminant validity, “doctor services” (in relation to “information”), “organisation” (in relation to “doctor services”, “nurse services”, and “information”) and “standard” (in relation to “organisation”) had issues. Construct validity and criterion-related validity were supported for majority of the results. Finally, a model including a second-order factor was proposed. The second-order factor, named “treatment services”, consisted of 4 first-order factors: “nurse services”, “doctor services”, “information”, and “organisation”. Moreover, the dimensions of “standard” and “next of kin” were included in this final model, but “discharge” and “interaction” were excluded. Hence, the final model included one second-order factor comprising 4 sub-factors as well as “standard” and “next of kin”.
The dimensions with associated items found in this study were similar to those found by Pettersen, Veenstra (15) while some dimensions, such as “doctor services”, “nurse services”, “organisation”, “information”, and “hospital standards”, overlapped with dimensions found by other studies (12, 13, 20). Invariance tests conducted in the present study were absent in the study by Pettersen, Veenstra (15), which marks a good contribution of this study. The tests showed that the model achieved invariance across the aggregated departments with regard to structure and pattern (configural) as well as the loadings of the items on their respective dimensions (metric). However, scalar invariance was not achieved for this model. Considering the diverse nature of the sample, as well as the aggregation of the departments into broad categories, this finding was expected. Putnick and Bornstein (44) asserted that scalar invariance is the most stringent compared to configural and metric, and instances of rigid scalar non-invariance could mean that the construct is generally variant across different groups. The findings also showed that reliability was good, based on composite reliability values, similar to the Cronbach’s alpha values obtained by Pettersen, Veenstra (15).
With regard to validity tests, the study found that all the dimensions, except “standard”, attained the recommended 0.50 AVE value for convergent validity, similar to other related studies that examined similar dimensions using other instruments (45). However, discriminant validity issues were found for “doctor services” (in relation to “information”), “organisation” (in relation to “doctor services”, “nurse services”, and “information”) and “standard” (in relation to “organisation”). Discriminant validity was also missing in the study by Pettersen, Veenstra (15), thus indicating another good contribution of this study. Examining the wordings of their items gives some possible explanation for this finding. For instance, D1 under “doctor services” was worded as “Did the doctors talk to you so you understood them?”, while questions under “information” included “IF2. Did you know what you thought was necessary about the results of tests and examinations?” and “IF3. Did you receive sufficient information about your diagnosis or your complaints?” It is highly likely that patients will receive information on results and diagnosis mainly from their doctors and, as such, answering questions under “information” may be significantly influenced by the perception of how well the doctors spoke to these patients. Similarly, questions under “organisation” were “OR1. Did you find that there was a permanent group of nursing staff that took care of you?”, “OR2. Did you find that one doctor had the main responsibility for you?”, “OR3. Did you find that the hospital's work was well organized?”, and “OR4. Did you find that important information about you had come to the right person?” These questions feature clear wording relating to “nurse services”, “doctor services”, “information”, and “standard”, and it is therefore not surprising that no clear distinctions were found among them as constructs. Construct validity was also achieved with a majority of the items recording loadings of above 0.60. This was also achieved in the validation study by Pettersen, Veenstra (15) using a different method and in related studies using other instruments with similar dimensions (12, 13). One item on “standard” and one on “organisation” were, however, deleted due to loadings below 0.60, while another on “standard” was deleted in a bid to improve the discriminant validity. Perhaps the wording of these questions made them difficult for patients to understand clearly and respond accordingly. For instance, item S5 was framed as “Was the food satisfactory?” Patients may be left to decide what is meant by ‘satisfactory’, thus making the question too vague, or perhaps the different dietary requirements and preferences made this question more loosely defined. Again, item OR2 was framed as “Did you find that one doctor had the main responsibility for you?”, a question probably dependent on the ailments of the patient and likely to be out of the control of hospital organisation. Thus, if a patient’s ailments require more than a single main doctor, then this question may suggest to the patient that having 2 or more main doctors reduces the ability of the hospitals to organise their work well. Criterion-related validity was ascertained for the overall measure as well as the dimensions in predicting at least 1 of the 3 outcome variables: satisfaction, health benefits, and health level, which is consistent with previous studies (e.g. 33, 34, 35).
Lastly, a model with a second-order factor, “treatment services”, for 4 of the dimensions was proposed based on the results of the validity and reliability analyses: “nurse services”, “doctor services”, “information”, and “organisation”. This constitutes the most important contribution of this study since this possibility was not explored in the study by Pettersen, Veenstra (15), perhaps owing to the absence of discriminant validity examinations in their study, and since this indicates a change in the factor structure of the PEQ. Rindskopf and Rose (46) observed that second-order factors reflect relationships among first-order factors. It is worth noting that related studies that developed other PREMs for generic and specific health issues also found these 4 dimensions in common (e.g. 11, 13, 20, 47). Although these studies did not develop a second-order factor for these dimensions, this is indicative of the prominence of these 4 variables in measuring and understanding patient experiences. The current finding, therefore, builds on this prominence to illustrate the high interrelationships and inextricable links among these factors, which brings some theoretical and practical implications to the fore.
Theoretical Implications
This study brings a very important, yet mostly ignored, contribution to the patient experience and quality healthcare literature: a need for more validation studies and surveys on patient experiences. The study responds to the recommendation by Pettersen, Veenstra (15) that existing PREMs need scrutiny and also tackles the research gap identified in the matrix by Beattie, Murphy (16), indicating that the PEQ by Pettersen, Veenstra (15) lacked some validity analyses. This buttresses the claim that, indeed, changing theoretical underpinnings influences existing measures and changing statistical methods and tools can reveal weaknesses of measures; moreover, this should be countered by regular psychometric appraisals of these measures. The results also contribute to the views of some researchers (e.g. 17, 18), regarding the need to repeat patient experience surveys to generate more reliable data for policy-making. The assessment of patients’ perspectives of hospital care would have to be reliable and valid enough in order to elicit accurate information about their experiences, constructs, and outcomes. Thus, it is imperative to ensure that these instruments always perform optimally and generate reliable information on how to improve quality of care and hospital experiences. These results, therefore, provide a background for further studies to be conducted on PREMs.
Another major contribution of this study is the finding of a second-order factor labelled “treatment services”, which consists of 4 factors: “nurse services”, “doctor services”, “information”, and “organisation”. This means that there exist strong and significant relationships among these dimensions (46). This finding also means that a single dimension or factor could adequately account for all 4 dimensions and could be identified as a major sub-dimension that captures these 4 dimensions. The “treatment services” factor has implications for the conceptualisation of patient-oriented hospital service climates. Patients in these hospitals may have highly overlapping experiences across “nurse services”, “doctor services”, “organisation”, and “information”. In more specific terms, it can be said that these patients experience a main dimension that accounts for significant portions of the 4 dimensions, perhaps because of the way these factors play out in the hospitals. For instance, doctors provide information regarding patients’ health, ailments, and treatments while nurses organise and assist patients with the treatment process. This is significant in advancing knowledge of patient experiences. The experience of these 4 dimensions may not be that distinct, and patients, in experiencing service climate in the hospitals, may not adequately distinguish their shared perceptions of “doctor services” from “information” or of “nurse services” from “organisation”, for instance. Patient-oriented hospital service climate, as conceptualised in this study, may include 2 levels of factors that influence patients’ perceptions and experiences of healthcare. Based on this, the definition of patient-oriented hospital service climate is modified as: patients’ shared perceptions and experiences across different levels of hospital work environments, the services, and formal and informal practices and procedures that make up the entire caregiving process and that inform patient outcomes. This contribution is also a major highlight when compared to the study by Pettersen, Veenstra (15), in which discriminant validity was not examined and a resulting second-order factor analysis was not explored. This challenges the theoretical structure of the PEQ and theoretical distinctness among these factors. Therefore, this study suggests a change in the factor structure of the PEQ and the development of a second-order factor for these 4 dimensions in the general patient experience literature. These possibilities are worth exploring in further surveys and studies on hospital factors as patient experiences during the caregiving process.
Practical Implications
Quality healthcare delivery is not exclusive to a region or country but a general goal of all healthcare systems worldwide. This can be contributed to by generating accurate information on how healthcare users experience healthcare systems. The results from this study suggest that it is not enough to develop a good measure of patient experiences, but it is imperative to review and reassess the ability of the measure to keep generating accurate information on patients’ experiences and health. The questions in the PEQ may have to be revised in order to elicit more concise and accurate information from patients. Furthermore, some dimensions, such as “next of kin”, seemed not to be relatable to most of the patients, judging from the many missing values and invariance tests. Also, the PEQ should be administered with the second-order factor taken into consideration. It is imperative to analyse “nurse services”, “doctor services”, “information”, and “organisation” as a second-order factor, as shown in the proposed model, due to the validity issues that were realised in the analysis. This can provide researchers and management with adequate knowledge on what patients experience during the caregiving process. Moreover, management must take the interrelationships in the second-order factor into account to make meaningful, informed, and sustainable changes in the hospitals for patients. The second-order factor must be considered as a single climate encompassing these 4 dimensions, where patients’ perceptions and interactions with a factor has a ripple effect on the others. Such considerations in policies and practice can help management and workers to reduce errors that may have dire consequences.
Limitations and directions for future research
This study employs data that is not at the national level but from a health region in Norway. That notwithstanding, the study has good generalisability power owing to the similarity in hospital and healthcare systems across the regions in Norway. Generalising to other countries, however, is difficult due to the differences in culture and healthcare systems. The findings require additional research in different countries for further justification. Therefore, future studies on reassessing psychometric properties of PREMs may want to employ larger data sets, for instance at the national level or across regions, to further investigate and develop the measurement quality of such surveys. Furthermore, future research should adopt the proposed model (with the second-order factor) from this study and examine it empirically to confirm it or otherwise, within health sectors across different countries.