Participants and procedure
The data of Sample 1 was collected at two college campuses in the Eastern and Southwestern region of the USA. Participants were undergraduate introductory psychology students who contributed in return for course credit. The final pooled sample consisted of n = 501. They received a data protection declaration that is in agreement with the Helsinki Declaration. The study was approved by the institutional review boards of the involved university institutions and all participants provided written informed consent.
Participants of Sample 2 were recruited in Spring 2019 via Amazon's Mechanical Turk [19], a crowd-sourcing website. MTurk is an international online platform that allows researchers to post tasks or questionnaires that participants complete in return for payment. In the current study, participants signed up via MTurk and were then directed to the online survey to complete the questionnaire. This survey was only available to participants who were located in the USA and their MTurk approval rating greater than 95%. The questionnaire took approximately 5 min to complete and participants were compensated $0.50 for their time. Sample 2 was collected in order to evaluate the factorial structure in a study. The final sample consisted of N = 184 participants.
The study was approved by the ethic review boards of Landesärztekammer Rheinland-Pfalz, Germany, and all participants provided informed consent online by agreeing to take part in the study.
Table 1
Sample description
|
Sample 1 (n = 501)
|
Sample 2 (n = 184)
|
|
n
|
%
|
TICS
|
n
|
%
|
TICS
|
Gender
|
|
|
M (SD)
|
|
|
M (SD)
|
Female
|
366
|
73.1
|
2.47 (0.72)
|
96
|
52.2
|
2.44 (0.85)
|
Male
|
118
|
23.6
|
2.54 (0.70)
|
88
|
47.8
|
2.46 (0.89)
|
Missing
|
17
|
3.4
|
2.05 (0.84)
|
|
|
|
Age (in years)
|
M = 19.97, SD = 2.84
|
M = 37.86, SD = 11.35
|
≤20
|
366
|
73.1
|
2.50 (0.70)
|
2
|
1.1
|
3.44 (0.31)
|
21-25
|
116
|
23.2
|
2.42 (0.79)
|
16
|
8.7
|
2.78 (0.79)
|
≥26
|
19
|
3.8
|
2.37 (0.72)
|
166
|
90.2
|
2.41 (0.87)
|
Ethnicity
|
|
|
|
|
|
|
White
|
391
|
78.0
|
2.49 (0.67)
|
148
|
80.4
|
2.35 (0.81)
|
Black or African American
|
22
|
4.4
|
2.58 (0.69)
|
16
|
8.7
|
2.26 (0.81)
|
Asian or Pacific Islander
|
25
|
5.0
|
2.28 (0.81)
|
6
|
3.3
|
2.41 (0.87)
|
Hispanic or Latino
|
6
|
1.2
|
2.74 (0.44)
|
13
|
7.1
|
2.94 (0.98)
|
Multi-ethnic
|
6
|
1.2
|
3.67 (0.78)
|
3
|
1.6
|
2.70 (0.39)
|
Other
|
25
|
5.0
|
2.12 (0.97)
|
|
|
|
Missing
|
26
|
5.2
|
2.34 (0.91)
|
2
|
1.1
|
2.67 (0.63)
|
Note. TICS = Trier Inventory for Chronic Stress
Measures
The Trier Inventory for Chronic Stress (TICS) is a standardized German questionnaire that has been tested with respect to its factorial structure and psychometric properties, showing good to very good reliability [14]. Internal consistency (Cronbach’s Alpha, α) was good to very good with values ranging from .84 to .91 (mean of α = .87) [13]. Nine interrelated factors of chronic stress are assessed: Work Overload; Social Overload; Pressure to Perform; Work Discontent; Excessive Demands at Work; Lack of Social Recognition; Social Tensions; Social Isolation; Chronic Worrying. The nine factors were derived from 57 items rated on a five-point rating scale (1-5, labeled as: “never, “rarely”, “sometimes”, “frequently”, “always”). Participants rated the occurrence/frequency of specific situations with a recall period of the previous three months. The 12 items with the highest loadings constitute the short version by the original authors [13]. In addition, a new short version of the TICS was developed based on the alphamax algorithm representing the nine areas of chronic stress of the original TICS. The one-factor-model of this new short version provided a good fit for the latent construct and showed good reliability (α = .88) [17]. After the translation state-of-the-art (see Petrowski et al.[18]), the English version of the Trier Inventory of Chronic Stress (TICS-EN) with 57 items was used in the present study [18].
The Perceived Stress Scale (PSS-10) is the most widely used psychological instrument for measuring perceived stress [20]. Respondents report the degree to which situations in one’s life have been unpredictable, uncontrollable and overloaded in the past month on a 5-point scale (0=never, 1=almost never, 2=sometimes, 3=fairly often, 4=very often).
Statistical Analyses
We conducted the analyses in R, using the packages lavaan, lordif, MBESS, and semTools [21–24]. Participants with missing values on any of the TICS-9 items were excluded from the analysis: seven and eleven participants. In addition, we excluded participants who failed the attention checks utilized in Sample 2 (n = 28). Very few participants (less than 5% across all items in both samples) chose the highest response option, making the items essentially ordinally scaled. Previous research suggests that conventional maximum likelihood estimation tends to be inaccurate with four or fewer response categories [25,26]. Therefore, we used the robust diagonally weighted least squares estimation method [27].
To evaluate model fit we considered the following measures and cutoff values[28–30]: The χ²-statistic should ideally be non-significant and is calculated by χ² divided by the degrees of freedom of the model, which should < than 2 to indicate good, or < 3 to indicate acceptable, fit. The Comparative Fit Index (CFI) and the Tucker-Lewis Index (TLI), which should be > .95 for good, or > .90 for acceptable fit, and finally, the Root Mean Square Error of Approximation (RMSEA) and the Standardized Root Mean Square Residual (SRMR), which should be lower than .08 to indicate acceptable, or <.05 to indicate good fit. Additionally, we report the 90% confidence interval for the RMSEA. In line with Dunn et al. [31], we tested reliability using McDonald’s ω [32], accompanied by a 95% confidence interval.
In order to test for measurement invariance across gender groups, we used the approach of comparing increasingly constrained models as described by Milfont et al. [33]. Since we were dealing with ordered categorical data we modified the procedure in the way described by Wu et al.[34]: First, we compared the unconstrained (configural) model to a model with item thresholds fixed to be equal across groups. Second, the threshold-invariant model was compared to the metric model (item loadings constrained). Third, the metric model was compared to the scalar model (item intercepts constrained). To evaluate the model comparisons, we primarily used the differences in CFI and gamma hat (GH) between models – which should not exceed .01. Additionally, we tested for significant differences in χ². To avoid selecting a non-invariant marker variable we estimated all factor loadings freely and set the variance of the latent variable to 1 instead. In addition, we analyzed differential item functioning using a logistic ordinal regression framework to be able to pinpoint the origin of whatever instances of measurement non-invariance we encountered [35–37].