The development, validation, and reliability testing of the AP FoP-Q-SF (AP-FoP-Q-SF) were performed in a prospective cross-sectional study.
Participants
A convenience sample of 417 participants was recruited from August 2020 to February 2021 in the gastroenterology departments of three tertiary hospitals in the eastern region of China. The inclusion criteria for inpatients were as follows: (a) diagnosis of AP were based on the 2012 revised Atlanta criteria[12]; (b) Patients aged ≥ 18 years and < 80 years; and (c) agreement to participate in the study. The exclusion criteria were as follows: (a) diagnostic suspicion or a definitive diagnosis of severe illness, such as cancer or chronic failure of the heart, liver or other major organs; (b) pregnancy or breastfeeding; or (c) cognitive impairment or a history of psychiatric illness.
Patients completed the questionnaire after informed consent was obtained, and the completeness of the questionnaire was checked by the researcher after the patient finished. Patients with incomplete questionnaires were asked to complete the questionnaire on site. For sentences that were difficult to understand, standard guidelines were used to assist the patient.
Procedures
Preliminary Adjustment of Questionnaire Items
The FoP-Q-SF is a multidimensional self-report questionnaire comprising 12 items belonging to four categories (affective reactions, partnership/family issues, occupation, and loss of autonomy)[13]. Five Likert-style items were used to assess the degree of FoP (ranging from 1 (“never”) to 5 (“very often”)). In qualitative studies, FoP and special treatments, worry about work and the burden on family members were often mentioned by patients with AP[7, 14]. The items of the questionnaire and the results of qualitative studies were discussed among an internal expert group consisting of 3 pancreatic disease experts, 1 psychological expert and 1 scale expert, and the items and expressions of the scale were finally determined by consensus.
Determination of Questionnaire Items
To develop the AP-FoP-Q-SF, an item analysis comprising the following six guidelines was conducted:
- If the distribution of answers revealed floor or ceiling effects, i.e., a value of 1 or 5 accounted for more than 50% of the responses, respectively, questions were excluded because of insufficient measurement precision[15].
- If the answer distribution showed extreme skewness, the items with skewness coefficients > 2 were deleted[16].
- If the commonalities between the items indicated that the item could explain the variability of the same psychological trait, principal component analysis was used to extract one common factor to test the commonness among the items. If the factor loadings were less than 0.5, they were deleted[17].
- If the Pearson’s correlation coefficient between item scores and total scale scores was ≤0.50, the item was deleted[18].
- The t-value was calculated between the items of the upper 27% and lower 27% of cases to test the discriminability of the questionnaire. If the items showed poor discriminability (t<3), they were deleted.
- Cronbach’s alpha method was adopted by analyzing whether the Cronbach’s alpha of the instrument increased after deleting the current item.
Reliability and Validity Testing of the Questionnaire
Validation
Three types of validation were investigated.
- Content validity: The FOP-Q-SF is a simplified version of the FOP-Q, which has been widely used and verified in a variety of diseases, indicating that relevant experts affirm its content. In addition, the internal expert panel of this study also discussed the content of the questionnaire and ensured that all important aspects were covered in it through consensus.
- Construct validity: Construct validity refers to the degree to which the questionnaire can measure a certain psychological trait. We used the first half of the data to conduct exploratory factor analysis (EFA) to explore the structure of the questionnaire. We then conducted confirmatory factor analysis (CFA) using the second half of the data to verify the factor structure and assess factor loadings and model fit.
- Criterion validity: Based on previous studies[19], the Hospital Anxiety and Depression Scale (HADS) and the shortened version of the SF-36 Health Survey (SF-12) were used to verify the criterion validity of the AP-FOP-Q-SF.
Reliability
To verify the stability and heterogeneity of the questionnaire results, the Cronbach's alpha and split-half reliability methods were adopted in this study.
Statistical Analysis
The number of participants with no missing demographic or clinical characteristics, a completed AP-FoP-Q-SF and available HADS and SF-12 results was small (N=416). This was managed using pairwise deletion. Descriptive statistics are used to describe the demographic and clinical characteristics. Frequency and percentages are used to summarize categorical variables, and the mean±standard deviation is used to summarize quantitative measures.
To evaluate the construct validity of the AP-FoP-Q-SF, we randomly divided the sample into two datasets. The Kaiser-Meyer-Olkin test and Bartlett’s test of sphericity were used to confirm the sample and item adequacy in one subsample of the two datasets. If the data were suitable for exploratory factor analysis (EFA), a principal component analysis with varimax rotation was used to assess the main factors. An eigenvalue >1 was used to determine the underlying factor structure. The other subsample was used for confirmatory factor analysis (CFA). The maximum-likelihood-robust estimation method was applied to estimate the fit of the underlying structure. The goodness-of-fit was verified by using the following indexes: (1) the relative chi-square (χ2/df<3), (2) the root mean square error of approximation (RMSEA<0.08), (3) the normed fit index (NFI>0.90), (4) the incremental fit index (IFI>0.90), (5) the Tucker-Lewis index (TLI>0.90) and (6) the comparative fit index (CFI>0.90)[20]. Spearman’s correlation coefficients were calculated to test correlations between the FoP-Q-SF scores, the SF-12, and the HADS, and values higher than 0.50 indicated a strong correlation.
Internal consistency was examined using Cronbach’s alpha and split-half correlations with Spearman-Brown’s correction, where values above 0.7 were considered acceptable.
Identification of Patient Clusters and Characteristics Associated with Fear of Progression
A receiver operating characteristic (ROC) curve analysis was performed to identify the most appropriate cutoff value for high FoP, using the HADS anxiety subscale cutoff (≥11 points) as the gold standard to detect moderate or severe FoP[19]. The sensitivity and specificity were calculated. Frequencies and percentages are used to describe the degree to which moderate to severe FoP was present in AP patients.
Univariate and multivariate logistic regression analyses were used to identify patient variables (including demographic, social and economic characteristics, disease status, anxiety/depression and quality of life level) independently associated with high FoP.
All statistical analyses were performed using SPSS (version 26), AMOS (version 22) and Graph Prism 8. p values <0.05 were considered significant.