Design and data source
The ENSUSALUD database is publicly available on web (http://portal.susalud.gob.pe/blog/base-de-datos-2016). We carried out a psychometric study based on secondary data analysis of Questionnaire 2 of ENSUSALUD-2016. This section was filled out by physicians and nurses working in health care centers, our analysis was carried out specifically in doctors.
ENSUSALUD 2016 was developed by the Peruvian National Institute of Statistics in collaborative work with SUSALUD. This survey was performed in 185 healthcare centers in all 25 regions of Peru . Professionals who had worked for a minimum of 12 months in healthcare centers and were assigned to the public subsector were included: Ministry of Health (MINSA, from the Spanish acronym), Social Security (EsSalud, from the Spanish acronym), Armed Forces and Police Health Services, and private subsector.
Participants were selected from a complex two-stage probabilistic national representative sampling stratified by political region. Primary sampling unit was the healthcare centers and secondary sampling unit were professionals. Physicians over 65 years were excluded (retirement age in Peru).
Generation and development
Prior to ENSUSALUD 2016, there were two first attempts to develop a job satisfaction scale for healthcare workers in the country, in 2014 and 2015. The process of developing these instruments was two-folded:
First phase: Development of the first two versions of ENSUSALUD
During the first half of 2014, a multidisciplinary technical team (from Health Services Quality Directorate of the Ministry of Health, Research and Development Intendance of SUSALUD, and Peruvian National Institute of Statistics) proposed 53 preliminary scales to assess different aspects of the work of health professionals (physicians and nurses) with additional sociodemographic data . These preliminary scales were based on a review of the literature and operational tools previously used by Ministry of Health the country. Each preliminary scale had from 1 to 22 items and they were all included in the first version of ENSUSALUD 2014 (one national survey). The preliminary scales were groups of items based on instruments already designed or designed ad hoc to evaluate the Peruvian health system (in this case, the measurement properties had not been evaluated). Subsequently, in ENSUSALUD 2015 the same technical team reused the 53 preliminary scales used in the first version of ENSUSALUD 2014, added some preliminary scales, and modified the wording of some items based on previous experience .
Second phase: Validation of ENSUSALUD 2016
In 2016, SUSALUD convened representatives of the Social Security, Armed forces and police Health Services, officials of the Comprehensive Health Insurance, and four universities in Lima, Peru. Modifications to the existing questionnaires were discussed, existing items were maintained in ENSUSALUD 2015 and 29 new preliminary scales were added.
In relation to the job satisfaction, they decided to keep all the questions and items from the previous version, but with certain modifications. Therefore, a total of 30 items in three groups of items (three preliminary scales) were available on different aspects of job satisfaction and were evaluated in questionnaire 2 of ENSUSALUD 2016.
The evaluation is done through an individual interview between the evaluator and the physician. The data was filled in on a Tablet that sent the information in real-time into a database. The Peruvian National Institute of Statistics was in charge of collecting the data and the process was supervised by SUSALUD.
In order to take care of the quality of the data, there was constant monitoring, through a network of supervisors who were distributed as follows. The evaluators were under the responsibility of a coordinator from each team and the teams were in turn under the supervision of a regional supervisor.
The 30-items job satisfaction questionnaire of ENSUSALUD evaluates different job-related aspects and is divided into three different scales: general professional activity (6 items), Health Services Management (8 items), and working conditions of the health center (16 items). Each of these items is Likert type and has five answer options (5 = very satisfied; 4 = satisfied; 3 = neither satisfied nor dissatisfied; 2 = dissatisfied; 1 = very dissatisfied). The scale appears in supplement 1. A preliminary English version of the items is also presented for comparison purposes, which were not evaluated in this study (see Supplement 2).
Satisfaction scale on general professional activity: to explore several general aspects of the professional labor. Its items evaluate the satisfaction of the doctor-patient relationship, achievements associated with the profession, work availability, perception of occupational risk, and expectations in meeting the needs of the patient. Within ENSUSALUD the items in Spanish of this instrument are in question 82 with codes from c2p82_1 to c2p82_6 (see supplement 1).
Health Services Management Satisfaction Scale: To assess facility's management team runs and organizes the healthcare facility. The items included in this scale are satisfaction with resource management (economic and human), drug management, shift scheduling, and work capacity. In ENSUSALUD, the items of this instrument are in question 83 with codes from c2p83_1 to c2p83_8 (see supplement 1).
Satisfaction scale on the working conditions of the health center: To evaluate the working conditions perceived by the health professional. The indicators of the scale are satisfied with the possibility of promotion, organization of the health center, workload, schedules, salary, opportunities, infrastructure and equipment, relationship with superiors, administrative procedures, and hygiene of the health center. In ENSUSALUD, the items of this instrument are in question 81 with codes from c2p81_1 to c2p81_16 (see supplement 1).
In addition, we include demographic, professional and economic information in our analysis. Sex, age and marital status (whether they are currently living with a couple) were the demographic variables. We also evaluated additional professional information, such as having a specialty (yes, in process, or no), whether they worked in other institutions (yes / no), self-reported work-related illness (yes / no), type of organization where they work (Ministry of Health, EsSalud, Armed forces and national police, or Private clinics), and time spent working. Additionally, self-reported monthly income was evaluated and categorized according the minimum wage (less than four, four to ten, and more than ten). Minimum wage was 750 Peruvian soles (PEN) or US$222.5 (considered to be an exchange rate of 3.37 soles per US dollar).
We presented general characteristics of the participants using weighted frequencies and percentages.
Exploratory factor analysis (EFA)
A random subset from the total sample (split-half method) [20, 21] was analyzed. Polychoric matrices were used  and the estimator was weighted using least squares means and variance adjusted (WLSMV) , since it best fitted the ordinal nature of our items. We used quartimin rotation, parallel analysis test and Kaiser analysis to evaluate the most appropriate number of dimensions . Different models were obtained and evaluated to identify the one with the best measurement properties. This decision was made since theoretical models suggest that job satisfaction is a multidimensional construct. Before performing exploratory factorial analysis, the value of the Kaiser-Meyer-Olkin (KMO) was estimated. This is an index of sample adequacy, which allows identifying whether there is enough power or sample size to perform the analysis. Adequate KMO values higher than 0.90 are adequate .
To evaluate the factor structures we used three different criteria. First, items factor loadings should be equal to or greater than 0.40 . Second, if a scale has more than one dimension, each dimension must have at least three items to be considered stable . Third, if an item loads in more than one dimension and the difference of loading between them is lower than 0.020, the item will be deleted. However, if the difference in loadings is equal to or greater than 0.20, then the item will be included in the dimension that has the highest factor load .
Confirmatory factor analysis (CFA)
For confirmatory factor analysis, the models previously obtained in the exploratory factor analysis were evaluated. All the analysis were performed considering the complex characteristics of the sampling strategy (complex multistage sampling), for which the lavaan.survey command was used. The estimator used was WLSMV , and polychoric matrices were used .
The adjustment of the different models for the three scales was evaluated in three steps. First, a set of the goodness-of-fit indices was estimated. We used the Comparative Fit Index (CFI) and the Tucker-Lewis Index (TLI), both with optimal values ≥0.95; Standardized Root Mean Square Residual (SRMR) and Root Mean Square Error of Approximation (RMSEA) with a confidence interval of 90%, both with values adequate if <0.08 [26, 27]. Second and last step, if a scale had two or more dimensions, the correlation between the dimensions was evaluated, in order to test whether dimensions could overlap. Clear differentiation between the two dimensions can be considered when the correlation is less than 0.80 .
Multiple models of the CFA measurement invariance were evaluated through groups defined by relevant variables (sex, age group, marital status, if they have a medical specialty, if they work in other institution, individual income per month, self-reported work-related illness and self-reported chronic disease). Thus, four measurement models with progressive restrictions were compared between categories of these groups (e.g. between females and males) [28, 29]. Change in the CFI (ΔCFI) was used as the main criterion for comparing models with more restrictions against models with fewer restrictions. Simulation evidence suggests that ΔCFI <.01 between successively more restricted models provides evidence for measurement invariance . Models first assumed configural invariance (i.e. similar factor structure across groups) as the baseline model, progressing then to metric invariance (i.e. similar factor loadings and factor structure across groups), strong invariance (i.e. similar thresholds, factor loadings and factor structure across groups), and strict invariance (i.e. similar residual item variances, thresholds, factor loadings and factor structure across groups). The ΔCFI was examined between each model to establish if the more restricted model was appropriate than the previous less restricted one. We preferred ΔCFI over χ2 comparisons, since it is not sensitive to big sample sizes [28, 29].
We evaluate reliability by internal consistency method taking as the optimal value a McDonald's omega coefficient (ω) and the alpha coefficient (α). In both cases, appropriate values are considered to be those that are > 0.70 [30-33].
We performed analysis according complex sampling in R Studio ®, specifically with the packages “lavaan” , “lavaan.survey” , “semTools” , and “semPlot” .
The survey was anonymous and there was no information in the database that could lead to participants identification. Hence, conducting this analysis did not represent an ethical hazard since there was no access to confidential data. Two authors (LBB and EMH) participated in the design process of the three scales at the time the survey was being designed.