A total of 704 participants consented to take part in the study, with 702 completing the PANAS-C; 21 participants were removed from the analysis due to missing data and factor analysis requiring complete data patterns. Missingness was low. Of the total participants, 2% (n = 21) had one or more missing value; missing data was present for 2.5% of girls and 3.7% of boys. Thus, multiple imputation was deemed unnecessary, and listwise deletion of incomplete data was used. Attention check questions were included to ensure reliable responses were given. As a total of 13 participants incorrectly answered two attention check questions, their responses were removed from the analysis.
This resulted in a complete dataset of 668 participants, of which the average age was 14 years (SD = 0.91) and 58% were girls (n = 385). Follow-up data was collected on 181 participants at T2 for test-retest, carried out at a median of seven days from first data collection (range: 6-12 days). Of the sample, six had incomplete data and were removed from the analysis. Four participants selected incorrect responses on two attention check questions; these observations were removed from the analysis. Thus, test-retest was carried out on a sample of 171 participants. The average age of the follow-up sample was 14 years (SD = 0.89) and 51% were girls (n = 87).
Test-retest reliability
The stability of the PANAS-C items was first evaluated over a period of 6 to 12 days to ensure items were not problematic. The items were all found to have acceptable test-retest reliability, values of Psi ranged from 0.71 to 0.79, and the corresponding estimated ICC had a range between 0.83 and 0.85 (Table 1), thus indicating satisfactory agreement for all items between the two time points.
Table 1. Test-retest results for the PANAS-C at item level (n =171).
|
Label
|
Item
|
Psi
|
Lower CI
|
Upper CI
|
ICC
|
Interested
|
PAN1
|
0.73
|
0.71
|
1
|
0.83
|
Alert
|
PAN2
|
0.78
|
0.75
|
1
|
0.85
|
Excited
|
PAN3
|
0.71
|
0.69
|
1
|
0.83
|
Happy
|
PAN4
|
0.76
|
0.74
|
1
|
0.84
|
Strong
|
PAN5
|
0.76
|
0.73
|
1
|
0.84
|
Energetic
|
PAN6
|
0.74
|
0.71
|
1
|
0.83
|
Calm
|
PAN7
|
0.72
|
0.70
|
1
|
0.83
|
Cheerful
|
PAN8
|
0.76
|
0.73
|
1
|
0.84
|
Active
|
PAN9
|
0.74
|
0.72
|
1
|
0.84
|
Proud
|
PAN10
|
0.73
|
0.71
|
1
|
0.83
|
Joyful
|
PAN11
|
0.74
|
0.72
|
1
|
0.84
|
Fearless
|
PAN12
|
0.72
|
0.69
|
1
|
0.83
|
Delighted
|
PAN13
|
0.72
|
0.69
|
1
|
0.83
|
Daring
|
PAN14
|
0.76
|
0.74
|
1
|
0.84
|
Sad
|
PAN15
|
0.76
|
0.73
|
1
|
0.84
|
Frightened
|
PAN16
|
0.74
|
0.72
|
1
|
0.84
|
Ashamed
|
PAN17
|
0.79
|
0.76
|
1
|
0.85
|
Upset
|
PAN18
|
0.74
|
0.71
|
1
|
0.84
|
Nervous
|
PAN19
|
0.75
|
0.73
|
1
|
0.84
|
Guilty
|
PAN20
|
0.79
|
0.77
|
1
|
0.85
|
Scared
|
PAN21
|
0.74
|
0.71
|
1
|
0.84
|
Miserable
|
PAN22
|
0.75
|
0.73
|
1
|
0.84
|
Jittery-Jumpy
|
PAN23
|
0.73
|
0.71
|
1
|
0.83
|
Afraid
|
PAN24
|
0.74
|
0.71
|
1
|
0.83
|
Lonely
|
PAN25
|
0.75
|
0.72
|
1
|
0.84
|
Mad
|
PAN26
|
0.74
|
0.71
|
1
|
0.84
|
Disgusted
|
PAN27
|
0.77
|
0.75
|
1
|
0.85
|
Gloomy
|
PAN28
|
0.76
|
0.74
|
1
|
0.84
|
Factor analysis
The original two-factor structure of the PANAS-C was found to have satisfactory fit to the data, as indicated by the model fit indices: Relative χ2 = 4.36, root mean RMSEA = 0.071 (90% confidence interval: 0.067, 0.075), CFI = 0.92, TFI = 0.92, and SRMR = 0.063. However, the factor loadings suggested an unsatisfactory model structure as three items (PAN1, PAN2, and PAN12) had factor loadings below 0.3.
As the initially proposed model emerged as less than satisfactory, we proceeded to identify a better fitting model using EFA and CFA in separate random split halves of the sample.
We started our explorations using the complete set of the 28 items. First, we explored the unidimensional model. For the 28 items of the PANAS-C, the one-factor solution did not provide a suitable structure, with several items loading below 0.4. The two-factor structure solution was improved but the item PAN1 had a very low loading of 0.21, and PAN2 was found to cross load to factor 1 (loading of 0.36) and factor 2 (loading of 0.34). At three or more factors, the solutions were unsuitable, containing several cross-loaded items and factors of less than 3 items. Table 2 presents the goodness of fit values for all EFA models.
The two low-loading items in the two-factor structure, PAN1 and PAN2, were sequentially removed from the analysis. The 26-item PANAS-C was suggested to retain up to four factors as found by the eigenvalues (four eigenvalues above 1: 9.477, 3.506, 1.538, and 1.002) while the scree plot suggested that up to 2 factors should be retained (Figure 1). The one-factor solution was unsuitable, with several low-loading items and poor model fit (Table 2). The two-factor model fit was adequate in relation to model fit criteria, with a relative chi-square of 2.56, RMSEA = 0.068 (90% CI: 0.062, 0.075), CFI = 0.94, TLI = 0.93, and SRMR = 0.056. In considering three or more factors, the models were unsuitable in terms of the factor structure criteria. Therefore, the two-factor model was accepted as the final EFA-suggested solution.
Table 2. Goodness of fit for exploratory factor analysis models.
|
Items
|
Model
|
Relative χ2
|
RMSEA (90% CI)
|
CFI
|
TLI
|
SRMR
|
28
|
One factor
|
6.64
|
0.130 (0.125, 0.135)
|
0.72
|
0.70
|
0.139
|
|
Two factors
|
2.30
|
0.062 (0.056, 0.068)
|
0.94
|
0.93
|
0.056
|
|
Three factors
|
1.73
|
0.047 (0.040, 0.054)
|
0.97
|
0.96
|
0.043
|
26
|
One factor
|
7.38
|
0.138 (0.133, 0.144)
|
0.72
|
0.70
|
0.141
|
|
Two factors
|
2.56
|
0.068 (0.062, 0.075)
|
0.94
|
0.93
|
0.056
|
|
Three factors
|
1.92
|
0.053 (0.045, 0.060)
|
0.97
|
0.96
|
0.042
|
Table 3 shows the exploratory factor loadings of the PANAS-C items to the factors of PA and NA. One item had a factor loading of 0.4 (PAN12), suggesting this item may be problematic. This item was retained to be further evaluated in terms of CFA, measurement invariance, and internal consistency. CFA of the EFA-derived solution (26 items and 2 factors) was found to be of adequate fit; relative chi-square was 2.51, RMSEA = 0.067 (90% CI: 0.061, 0.073), CFI = 0.94, TLI = 0.94, SRMR = 0.062, with factor loadings shown in Table 3 (in parentheses).
The final model, suggested by factor analysis, was consistent with the originally proposed PANAS-C model. The items of the PA and NA subscales were also those proposed by Laurent et al. [40]; however, two items from the PA subscale were removed.
Table 3. Factor analysis and reliability statistics of the PANAS-C. Factor loadings are presented as EFA (CFA).
|
Label
|
Item
|
Positive affect
|
Negative affect
|
Average Inter-item correlation
|
Item-total correlations
|
Alpha if item deleted
|
Excited
|
PAN3
|
0.786 (0.740)
|
|
0.375
|
0.635
|
.868
|
Happy
|
PAN4
|
0.734 (0.847)
|
|
0.371
|
0.664
|
.867
|
Strong
|
PAN5
|
0.629 (0.675)
|
|
0.382
|
0.575
|
.872
|
Energetic
|
PAN6
|
0.769 (0.734)
|
|
0.372
|
0.658
|
.867
|
Calm
|
PAN7
|
0.503 (0.566)
|
|
0.396
|
0.468
|
.878
|
Cheerful
|
PAN8
|
0.755 (0.833)
|
|
0.371
|
0.669
|
.866
|
Active
|
PAN9
|
0.666 (0.717)
|
|
0.378
|
0.612
|
.870
|
Proud
|
PAN10
|
0.723 (0.716)
|
|
0.375
|
0.637
|
.868
|
Joyful
|
PAN11
|
0.639 (0.540)
|
|
0.388
|
0.525
|
.875
|
Fearless
|
PAN12
|
0.393 (0.411)
|
|
0.418
|
0.297
|
.888
|
Delighted
|
PAN13
|
0.757 (0.848)
|
|
0.368
|
0.695
|
.865
|
Daring
|
PAN14
|
0.600 (0.614)
|
|
0.390
|
0.509
|
.876
|
Sad
|
PAN15
|
|
0.634 (0.780)
|
0.375
|
0.663
|
.886
|
Frightened
|
PAN16
|
|
0.703 (0.433)
|
0.394
|
0.479
|
.894
|
Ashamed
|
PAN17
|
|
0.633 (0.651)
|
0.385
|
0.568
|
.891
|
Upset
|
PAN18
|
|
0.618 (0.651)
|
0.381
|
0.606
|
.889
|
Nervous
|
PAN19
|
|
0.720 (0.741)
|
0.378
|
0.631
|
.888
|
Guilty
|
PAN20
|
|
0.661 (0.708)
|
0.382
|
0.595
|
.889
|
Scared
|
PAN21
|
|
0.646 (0.667)
|
0.382
|
0.593
|
.889
|
Miserable
|
PAN22
|
|
0.530 (0.660)
|
0.394
|
0.476
|
.894
|
Jittery-Jumpy
|
PAN23
|
|
0.721 (0.575)
|
0.374
|
0.671
|
.886
|
Afraid
|
PAN24
|
|
0.764 (0.764)
|
0.378
|
0.640
|
.887
|
Lonely
|
PAN25
|
|
0.609 (0.691)
|
0.383
|
0.582
|
.890
|
Mad
|
PAN26
|
|
0.664 (0.658)
|
0.384
|
0.573
|
.890
|
Disgusted
|
PAN27
|
|
0.582 (0.383)
|
0.401
|
0.411
|
.897
|
Gloomy
|
PAN28
|
|
0.696 (0.810)
|
0.372
|
0.696
|
.885
|
Measurement invariance
Of the 26-item PANAS-C, the measurement invariance model found gender, adjusted for age, to have significant direct effects on three items. At the same level of affect the expectant score for boys, compared to girls, was higher for items PAN14 by –0.36 and PAN22 by –0.54, and lower for PAN8 by 0.40.
The direct effects of age, adjusted for gender, on three PANAS-C items were considered negligible (range –0.10 to 0.14). For a single point change on the 1-5 scale, a difference of age by 10 years would be required, which is beyond the age range of adolescents in this study.
The noninvariance for age and gender was negligible. Thus, the PANAS-C can be considered measurement invariant, with comparisons of subscale scores across ages of adolescence and between girls and boys to be fully justified.
Internal consistency
The PA scale of the PANAS-C had satisfactory internal consistency with α = .88, ITC between 0.30 and 0.70, and the deletion of PAN12 marginally increased the alpha to .89 (Table 3). The internal consistency of the NA scale of the PANAS-C was excellent, with α = .90, ITC between 0.41 and 0.70, and no increase of alpha by deletion of an item. The values of McDonald’s omega were equal to those of alpha (ω =0.88 and ω =0.89). Therefore, the decision to retain the item PAN12 was supported.
Validity and further analysis
To assess for convergent validity, Spearman’s correlation coefficients were evaluated. There was a strong positive correlation between the NA subscale of the PANAS-C and the K6 (Table 4). The PA subscale of the PANAS-C was significantly correlated with the CW-SWBS, with a moderate positive correlation (Table 4). These correlations provide evidence of convergent validity of the PANAS-C.
Age was found to be slightly significantly correlated with the PA subscale of the PANAS-C (Table 4).
Table 4. Spearman’s rho correlation coefficients between the positive and negative affect subscales and the K6 and CW-SWBS.
|
Measure
|
Positive affect subscale
|
Negative affect subscale
|
Age
|
–0.086*
|
0.018
|
K6
|
–0.159*
|
0.617**
|
Children’s Worlds Subjective Well-Being Scale
|
0.350**
|
–0.215**
|
Note. *p < 0.05 **p < 0.001
|
The difference between the scores for girls and boys on the PA subscale was significant (t = 2.84, df = 665, p = 0.005), with boys (M = 47.14, SD = 7.28) scoring higher than girls (M = 45.44, SD = 8.02). The NA subscale scores were significantly different based on gender (t = –7.20, df = 665, p < 0.001), with girls scoring higher than boys (girls: M = 37.89, SD = 9.38 and boys: M = 32.77, SD = 8.65).