Around the world, the COVID-19 pandemic burdens the lives of people. As the duration of the pandemic increases, so does the importance of measuring the adverse effects of stressors on people’s mental health and wellbeing with valid measures. This research aimed to examine the factorial validity and reliability of a new scale, the Pandemic Stressor Scale (PaSS), that aims to assess the different stressors relevant during a pandemic or epidemic. First, an EFA was conducted in a German sample of participants. Second, the dimensional structure identified in the first analysis was examined using CFA in the second sample of Austrian participants to examine its replicability. Global and local goodness of fit indices and additional test quality criteria were evaluated in two models: A nine-factor model (Model 1) and a nine-factor model with a second-order general factor (Model 2).
Exploratory Factor Analysis
A nine-factor solution of the PaSS, including 30 items, showed the best fit with the data and sufficient interpretability of the factors. The factor loadings of the 30 items ranged between .409 and .949, suggesting meaningful and practically significant factor loadings [47]. The identified factors were ‘Restricted Face-to-Face Contact', 'Problems with Childcare', ‘Work-Related Problems’, ‘Fear of Infection’, ‘Burden of Infection’, ‘Restricted Activity’, ‘Crisis Management and Communication', ‘Restricted Access to Resources’, and ‘Difficult Housing Condition’. While five of the nine factors consistently showed low factor loadings on the other factors, four factors showed factors loadings greater than .40 on one additional factor. Three items of the factor ‘Restricted Face-to-Face Contact’ (‘Restricted face-to-face contact with others’, ‘Social isolation’, ‘Restricted physical closeness to loved ones’) showed factor loading greater than .40 on the ‘Restricted activity’ factor. The factor ‘Difficult housing conditions’ included one item (‘No place of retreat’) that showed loadings greater than .40 on the ‘Problems with Childcare’ factor; one item of the ‘Fear of Infection’ factor (‘Fear of getting infected with the coronavirus’) loaded greater than .40 on the ‘Burden of infection’ factor. Finally, two items of the ‘Restricted Face-to-Face Contact’ factor (‘Restricted leisure activity’, ‘Restricted everyday activity’) loaded greater than .40 on the ‘Restricted Activity’ factor. High cross-loading may indicate that the indicators measure both constructs. However, all items showed lower cross-loadings than 75%, as recommended [47].
All existing questionnaires on pandemic-related stressors were shorter, including between 13 and 25 [e.g., 22,23,35]. These questionnaires might only partly cover pandemic-related stressor domains, such as home-related stressors (e.g., restricted housing conditions, conflicts at home, lack of childcare), or work-related changes. By assessing nine stressor domains with 30 stressors, the PaSS might allow capturing a broader range of stressor domains.
Confirmatory Factor Analysis
Global Fit.
The obtained dimensional structure identified in the EFA could be replicated by Model 1. The SRMR, RMSEA global fit indices showed acceptable fit, as well as the TLI and CFI. The χ2 statistics of both models indicated significant differences between the theoretical model and the data; however, this measure tends to be overly sensitive in large samples [41]. The global fit of Model 2 was below the threshold we defined for acceptable fit. The comparison of the two models via χ2 difference test showed that Model 1 fitted better than Model 2. Consequently, a second-order global factor on which all first-order factors load seems not to be reflected in the data.
Local Fit.
On the level of the indicators, factor loadings of the items as indicators on the first-order factors were moderate to large. However, in Model 2, four of the nine first-order factors had small factor loadings on the second-order general factor. Only the factors ‘Restricted Face-to-Face Contact’, ‘Crisis Management and Communication’, ‘Fear of Infection’, ‘Restricted Access to Resources’ and ‘Restricted Activity’ showed large or moderate factor loadings on a general second-order factor. This result indicates, in line with the significant result of the chi-square difference test, that the computation of a total score might not be reasonable.
Future studies might test models which include more than one second-order factor. While most of the indicators showed acceptable commonalities, some showed low communalities, indicating that a part of the indicator variance remained unexplained by the respective factor.
On the level of the factors, all factors showed acceptable factor reliabilities, including the second-order general factor. The indicators of each factor shared a sufficient amount of variance within this factor, which indicates that they likely measure a similar construct. All factors except the factor ‘Restricted Access to Resources’ discriminated well from each other.
Item Difficulty.
Twelve items had high difficulties, indicating that the respective stressors measured were not considered a severe burden by many participants. This is reflected in the difficulty index of the second-order general factor which is acceptable but close to the lower bound (P = .30). The high item difficulties in some items resulted in reduced item variance. Most calculations of a CFA are based on variance-covariance matrices that are affected by reduced item variance, resulting in lower global and local fit indices. The restricted item variance in some of the items might have lowered global fit. However, global fit was acceptable in all assessed indices in Model 1. Items with the lowest difficulties might be removed from the questionnaire to increase item variance.
However, while some stressors were not perceived as stressful, it has to be considered that we assessed the data during the summer and autumn time of the first year of the pandemic. For this time period, the burden was reduced, as the lockdown measures were relaxed in both countries and people had the opportunity to spend time outside. Future studies need to reassess the items and their difficulties to a later timepoint of the pandemic, in which the burden of the stressors could be increased, e.g., during a subsequent lockdown period.
Item Discrimination and Internal Consistency.
Most item discrimination indices were considered acceptable to good, except for three indicators that did not discriminate well between high and low scores of the second-order general factor in Model 2. These results indicate that the items discriminated well between high and low scores of the first-order factor, the discriminated less well concerning the second-order general factor. Internal consistencies were acceptable for all factors except for ‘Restricted Access to Resources’. In Model 2, the second-order general factor showed a moderate internal consistency (α = .88).
Model Comparison.
The chi-square difference test showed a better model-fit of Model 1 compared to Model 2. The only difference between the models is the second-order general factor called ‘Pandemic Stressors’. Four of the first-order factors showed small factor loadings on the global second-order factor and less good item discrimination indices for the global factor than those concerning the first-order factors. These results indicate a less well fit of the nine-factor model that includes a second-order general factor, which is consistent with the results found on the level of the local fit indices.
Strength and Limitations
A strength of this study is the use of sufficiently sized samples and the combined use of EFA and CFA in two different large samples to replicate the results obtained in EFA. A limitation of the study is that we used a non-probability sample that was not representative of the general populations of Germany and Austria concerning gender, income, and education. Future studies need to examine the psychometric properties of the PaSS among representative samples. Furthermore, a comprehensive validity testing needs to include convergent and discriminant validity. As the focus of this study was to assess the factorial validity of the PaSS, we did not examine the convergent and discriminant validity of the measure. The measure’s convergent and discriminant validity with other stressor measures need to be examined in future studies. Finally, the psychometric properties of the measure should be examined in the general populations of other countries than Germany and Austria.