The present study provides evidence on the psychometric validity of the PREOS-PC Compact Form Brazil with a sample of PHC patients. This instrument has been used in other countries, but, to date, its validation had not been performed and its applicability remains limited in the Brazilian context. The results demonstrate the validity of the PREOS-PC Compact Form Brazil, with a factorial structure composed of 17 items and four correlated domains (PrA, PaA, EPaS, OPaS). The instrument presented good internal consistency. Therefore, it represents evidence of construct validity to measure PS perceptions, experiences and results from the perspective of PHC users.
The initial model (with 23 items and four correlated factors) presented low factorial validity indexes. However, after the exclusion of six items, the model proved to be adequate, with good fit indexes. The excluded items were EPaS2, EPaS3, EPaS4, EPaS5, EPaS6 and EPaS7, all dichotomous and belonging to the EPaS domain. They relate to a single question that covers the problems that patients may have faced during PHC care. The inclusion of dichotomous items in the CFA model may result in biased parameter estimates with inaccurate interpretation limitations [36]. Therefore, the absence of contribution to the domains of these items may have been caused by the very nature of the items and, not necessarily, because they did not contribute to the formation of the construct. Another point is that some excluded items relate to health practices that are uncommon in the context of PHC in Brazil, particularly the administration of medications in the health unit, treatments, blood tests, laboratory tests and diagnostic tests, which are predominantly performed in the secondary (emergency and urgency units) and tertiary (for example: hospitals) networks [37]. This can decrease the probability of the patient’s knowledge about aspects related to the problems faced during their PHC care. All the excluded items, in fact, presented low prevalence in the sample, which may have contributed to their non-fit in the domains. In addition, the collection was carried out in the context of the COVID-19 pandemic in a scenario of reduced supply of health services [38], which may have contributed to the low prevalence of these items.
The final CFA model showed four correlated domains: “PrA”, “PaA”, “EPaS” and “OPaS”. The first contains items that evaluate what the health unit does to create a safe environment and ensure PS. The second presents items that seek to understand how patients are proactive to ensure that the healthcare is safe. The third has items that assess the prevalence of safety errors related to healthcare. The fourth presents items that assess the presence of health damage experienced by users. A domain that is not adjusted to the CFA due to the nature of the items, is called “General perceptions of patient safety”, which aims to evaluate how safe patients perceive their health unit to be and should be analyzed in isolation [26].
Internal consistency was assessed using Cronbach’s α and McDonald’s ω. The values were adequate, higher than recommended (α > 0.70 and ω > 0.60) [20, 35]. McDonald’s ω is a measure of solid internal consistency, but is little used in scale validation studies [39]. The PREOS-PC is relatively recent. No studies were found that analyzed the internal consistency of the instrument, along with the psychometric properties of the compact version. To date, the reliability of the instrument has only been tested for the full version, using Cronbach’s α, but not McDonald’s ω. [7]. The results of the full version study showed high internal consistency for the domains (Cronbach’s α: PrA = 0.89; PaA = 0.80, EPaS = 0.75 and OPaS = 0.96) [7]. Although with differences in the items, the internal consistencies were similar in the present study (Cronbach’s α: PrA = 0.90; PaA = 0.82, EPaS = 0.74 and OPaS = 0.94). In the original study, the EPaS domain presented the lowest internal consistency value, which is similar to that found in this study, which may explain the removal of some of the items from the final model. It is important to note that the PREOS-PC Compact Form Brazil is a reduced version and, to date, there are no studies that can accurately compare this version, since the original extended instrument has 58 items [7], which prevents a more accurate comparison with the literature.
The validity of the structural construct of the General Perceptions of Patient Safety (GPeS) domain was not confirmed by factor analysis, since one item was composed of a scale from 0 to 10 and two were open questions. Therefore, the compact version does not allow a factorial evaluation of the instrument as a whole, since it does not include this domain. It is important to note that, in the full version, the “General perceptions of patient safety” domain presents a single scale question and two open questions [7].
The present study has some limitations. Data collection was performed in a single Brazilian city, which limits the ability to generalize the results. The experiences and reports of the patients in these PHC units may not be the same as the practice in the rest of the country. The study sample was composed of 73.3% women, which does not guarantee that the results found are not distorted by the lack of sampling balance between the sexes, and may not be generalized to men [40]. Additionally, the data was collected online, due to the COVID-19 pandemic. These individuals may have characteristics that differ from people who do not have access to the internet, which also limits generalization. In addition, the validation study of the full version was performed through the application of the physical instrument. The response rate was 59.0%. Some calls went unanswered, which may have led to selection bias. All measures were self-reported, susceptible to memory and response biases. The inclusion of dichotomous items in the CFA is another limitation, which reduces the contribution of these items in the latent domain.
However, the study has strengths. First, the relatively large sample size allowed the inclusion of enough patients to conduct CFA. Second, this psychometric validation study is unprecedented in Brazil, being a starting point for future investigations that may be carried out in other Brazilian regions, as well as the implementation of the instrument in the PHC practice of the country. The relevance of this study is the possibility of improving the quality of care in PHC in Brazil, adapting methodologies to PS perceptions, experiences and results, through the evaluation of the patients themselves. This study can also help in the implementation of a process to improve the PHC care environment, as well as consolidate knowledge about the theme in the literature. However, further studies are still needed to evaluate the psychometric properties of the PREOS-PC Compact Form Brazil, especially in relation to some of the less prevalent items. Accordingly, future studies with more diverse samples should be carried out in order to have a clear understanding of users’ perceptions about PS in PHC, so that other comparisons and generalizations can be made. It is important to emphasize that, especially, conducting studies in other Brazilian regions, in a self-administered and face-to-face manner, could contribute to the improvement of the instrument and to the identification of new tests of validity and reliability for the PREOS-PC Compact Form Brazil in different contexts.