Study Design
A cross-sectional study was conducted to analyze the reliability and validity of a measuring instrument between May 2016 and February 2017. The present study was carried out in ten public hospitals in the Andalusian Healthcare System (part of the Spanish National Healthcare System), which provides universal, free coverage to 8 million citizens. These ten hospitals serve 3 million citizens, representing approximately 40% of the total coverage of the Andalusian Healthcare System.
The main type of hospital’s units were medical (Internal Medicine, Hematology Unit, Respiratory Unit) and surgical wards (Traumatology, Cardiac surgery, General surgery). The average length of stay of the patients was 4–8 days.
Setting and Sample
The family caregiver sample was taken from the sample of patients whose inclusion criteria were over 16 years of age, of both sexes, and admitted in medical-surgical hospital units. No exclusion criteria were established.
To calculate the sample size, the minimum sample size for structural validity of an instrument is at least 7 patients by the number of items and a minimum of 100 assessments according to the COSMIN checklist [25]. A minimum of 10 patients per items was established. Starting from the original 39 items on the scale, the minimum sample required was 390 family caregivers.
Variables
The following variables were grouped into sociodemographic and clinical. Sociodemographic variables were considered for both patients and caregivers, whereas clinical variables were only studied in patients.
Sociodemographic patients and family caregivers’ variables: gender (male, female); age (years); relationship between patient and family caregivers (couple, son/daughter, other) and co-residence (yes, no).
Clinical variables for patients: Admission unit (surgical or medical unit), hospital admission: the hospitals were classified by volume of patients, number of beds and geographical location in three hospital categories according to their level of specialization and reference population: Primary (> 500 beds and large metropolitan areas), specialties (between 200 and 500 beds and small metropolitan areas), and tertiary hospitals (< 200 beds and rural areas).
Instruments: Barthel index, Pfeiffer’s test, INICIARE scale, that measures of patient acuity and dependency.
Barthel Index: The autonomy to perform activities of daily living was measured using the Barthel Index. It consists of 10 items with five scoring intervals (total dependency, severe dependency, moderate dependency, mild dependency, independence), between 0 and 100: the lower the score, the greater the dependency in patients [26].
Pfeiffer’s test
Pfeiffer’s test is widely used to assess cognitive status and consists of 10 items. Cognitive impairment is suspected when the error score is equal to 3 or more in people who can read and write, or to 4 or more in people who cannot [27, 28].
INICIARE scale
It was used to assess nursing care dependency in patients. INICIARE is a recently created scale designed to evaluate nursing care dependency in patients with excellent psychometric properties (Intraclass Correlation Coefficient = 0.830–0.964; Internal Consistency total Cronbach’s α = 0.98, and by subscales between Cronbach’s α = 0.92–0.98). An exploratory factorial analysis identified three factors (Physiological, Instrumental and Cognitive-behavioural), which explained 74% of the variance. It consists of 55 items measured on a five-point Likert scale (5 reflects the most desirable patient’s condition, and 1 reflects the least desirable). The scoring range is 55–275, with three cut-off points (four intervals) that indicate levels of dependency [20, 29].
Study Protocol: Instrument validation
The study had three distinct phases. A first phase of the conceptualization of instrument focused on a review of the literature, this resulted in a preliminary instrument. In the second phase, content validity by expert consensus and the preliminary instrument was tested in a sample of family caregivers, through an empirical validation process. The third phase consisted of fieldwork to validate the instrument in a hospital setting.
First phase: Conceptualization of instrument
The Roy Adaptation Model [17] was the conceptual foundation of the instrument, and the literature review was used to guide the selection of NOC indicators. The NOC taxonomy structure has five levels: domains, classes, outcomes, indicators and measures. Each outcome has a definition, a list of indicators that can be used to evaluate patient status in relation to the outcome, a target outcome rating and a scale to measure patient status. Likert scale is used; indicators present five levels of response depending on the adequacy of its state: a value of 5, the highest value, reflects the patient’s most desirable condition, whereas a value of 1 reflects the least desirable condition [19].
From this conceptualization, the research team built a scale based on 39 indicators that belonged to the Domain VI Family Health "Outcome that describe status, behavior, or functioning of the family as a whole or of an individual as a family member", specifically, Classes W and Z.
Classes W (Family Caregiver Performance): "outcomes that describe the adaptation and performance of a family member caring for a dependent child or adult". Classes Z (Family Member Health Status): "outcomes that describe the physical, psychological, social, and spiritual health of an individual family member".
The NOC chosen were 2507 Caregiver Physical Health, 2210 Caregiver Role endurance and 2208 Caregiver Stressors. The NOC 2507 Caregiver Physical Health is defined as the "Physical Well-being of a family care provider while caring for a family member", it includes 16 indicators. The NOC 2210 Caregiver Role endurance is defined as "Factors that promote family care provider´s capacity to sustain caregiving over an extended period of time", includes 10 indicators. The NOC 2208 Caregiver Stressors is defined as "Severity of biopsychosocial pressure on a family care provider caring for another over an extended period of time", with 13 indicators.
Second phase: Content validity
The second phase focused on expert consensus and was conducted with a panel of 30 experts. The experts were selected from among clinical nurses, academic and research nurses, all of them highly trained, with extensive experience in nursing processes and use of Standardized Nursing Languages (SNLs), and proposed by the Nursing Directorate of the hospitals participating in the study. The panelists were also asked to propose new indicators if they considered them necessary. Face validity was not tested, as the wording of the indicators used for the instrument was directly extracted from the list of nearly 10,000 indicators included in the official version of the NOC, without modification or adaptation [19].
The profiles of the experts were as follows: clinical nurses (46.7%), academic and research nurses (20%), and nurse managers (33.3%). The mean age was 46.7 years old (range: 35–57). Eighteen panelists had a master’s degree (54%), six panelists had a doctorate (20%), and the remaining experts were registered nurses (26%).
Each expert was asked to rate the relevance and clarity of each item, using a 5-point scale from 5 (reflects the patient’s most desirable condition) to 1 (reflects the least desirable condition).
Third phase: Data collection and fieldwork
The fieldwork was undertaken by interviewers trained by the principal investigator. The data collectors had a bachelor’s degree in Nursing and 10 years of clinical experience in medical/surgical care. Any problem identified in the study questionnaire or the data collection were resolved through discussions between the data collectors and the principal investigator.
Nursing staff collected the data after the patients’ admission, the family caregiver was taken from the sample of patients. The interviews were conducted by the nurses to respective patients and family caregivers Each data collector took between 5 to 10 minutes to complete the questionnaire.
The anonymity of the patients and caregivers was preserved at all times by using code numbers for their data, which were stored on a digital platform. A statistical analysis of the data was performed, confirming the validity and reliability of the instrument, which has been named CUIDARE.
Data Analyses
The SPSS statistical package for Windows (version 26.0) was used for statistical analysis (SPSS/IBM, Chicago, IL, USA). The results of the descriptive analysis are represented as measures of central tendency (mean, median) and dispersion (standard deviation) for quantitative variables, and as frequencies and percentages for qualitative variables.
An analysis of the normality of the data had been previously performed using the Kolmogorov-Smirnov test. The quantitative variables have not got a normal distribution. A bivariate analysis was carried out using the chi-square statistic for relationship between two qualitative variables (e.g. sex of family caregiver and co-residence). The Cramer’s V test was used to examine the magnitude of association; the Odds Ratio (OR) was calculated in 2 × 2 tables with 95% confidence intervals. In addition, non-parametric test was used such us Mann-Whitney U (e.g. sex of family caregiver and age of caregiver or patient) and Kruskal-Wallis test (e.g. dependency level according to INICIARE and CUIDARE score). The p-value significance of 0.05 was established.
For the psychometric analysis, internal consistency was calculated to determine whether the items of the same concept were homogeneous among them [27]. Using Cronbach’s alpha, internal consistency values can range from 0 to 1, where values close to 1 indicate increased instrument reliability. Structural validity was examined using an Exploratory Factor Analysis (EFA) after performing a Bartlett's Test of Sphericity (p < 0.05) and a Kaiser-Meyer-Olkin test (range from 0 to 1). To construct validity, an EFA was carried out using Principal Component Analysis and Varimax rotation [31].
To assess the unidimensional aspect of the scale, a Confirmatory Factor Analysis (CFA) was carried for obtaining the level of goodness-of-fit of a factor model. AMOS v26.0 was used. Chi-square, Adjusted Goodness-of-Fit Index (AGFI), Normed Fit Index (NFI), Comparative Fit Index (CFI), Related Fit Index (RFI), and Root-Mean-Square Error of Approximation (RMSEA) were used to test the goodness-of-fit of the model. The fit was acceptable if the CFI, AGFI, RFI, and NFI ≥ 0.95 and the RMSEA < 0.06 [31, 32].
Finally, a Multiple Correspondence Analysis (MCA) was carried out. This method allows to investigate the relation between different categorical variables. The final figure provides a general view of relationships between variables.