Monitoring positive mental health in Swedish adolescents: psychometric evaluation of the Mental Health Continuum – Short Form 2

15 Background: The Mental Health Continuum – Short form (MHC-SF) is a self-report measure that has 16 been increasingly used to monitor mental well-being at the population level. The aim of this study 17 was to evaluate, for the first time, the psychometric properties of the MHC-SF in a Swedish 18 population, more specifically adolescents. 19 Methods: First, the evaluation was performed by examining face validity and test – retest reliability 20 obtained in a pre-study (n = 93). Then using data from the Survey of Adolescent Life in Vestmanland 21 2020 (n = 3880; participation rate = 71%; females = 51%; mean age = 16.23 years), we performed 22 confirmatory factor analysis on different factor structures based on theory and previous research. 23 Model-based estimates were calculated for assessing the internal reliability of the factor structure 24 with the best fit. Convergent validity was assessed by bivariate as well as model-based correlations, 25 and test – retest reliability was evaluated by intra-class correlation coefficients. 26 Results: This study on Swedish adolescents found that the MHC-SF is essentially unidimensional and 27 best described with a bifactor model consisting of a dominant general well-being factor and three 28 specific group factors of emotional, social and psychological well-being. Its overall reliability and the 29 reliability of the general well-being factor were good to excellent, while the reliability of its subscales 30 (specific group factor) was poor, and thus should not be used alone. Test – retest reliability of the total 31 scale was good, and convergent validity was supported by strong to very strong correlations with the 32 Short Warwick – Edinburg Mental Well-being Scale. 33 Conclusions: In conclusion, we consider the Swedish MHC-SF to be a psychometrically sound instrument for monitoring overall mental well-being in Swedish adolescents.


3
Background 37 Conventionally, epidemiological research and surveillance has focused mainly on diseases and their 38 risk factors [1]. This is perhaps particularly true for mental health [2] where it has been often tacitly 39 assumed that mental well-being would prevail in the absence of pathology [3]. Consequently, 40 surveillance has been largely limited to capturing mental problems and mental disorders with much 41 research and practice set on the treatment of pathologies and to some extent on their prevention. 42 However, a growing body of evidence shows that high levels of well-being⎯reflecting not merely the 43 absence of poor mental health⎯independently predict less subsequent mental illness and have a 44 range of positive effects on individuals and society [4][5][6][7]. Thus, there is a strong incentive to also 45 include measures of well-being in the monitoring of public mental health. 46 The Mental Health Continuum -Short form (MHC-SF) is a self-report measure that has been 47 6 The other source of data was the cross-sectional whole population study using Survey of 111 Adolescent Life in Vestmanland (SALVe) County in Sweden carried out during the early spring of 2020 112 prior to the Corona pandemic. In total, 5480 adolescents in the ninth grade of compulsory school or 113 second year of upper secondary school were invited to participate in the study by completing a 114 comprehensive health survey about living conditions, life habits and various aspects of health 115 including scales of mental well-being. The survey was web based and carried out during school hours 116 under exam-like conditions in a classroom context. Prior to logging on to the survey, all students 117 watched a video with information about the study and its purpose and that participation in the study 118 was voluntary and could be cancelled at any time for any reason. Consent to participate was given by 119 completing the questionnaire. All students participated anonymously; i.e., no names or personal 120 identification number was collected. Both municipal and private schools were included in the study. 121 In total, 3880 adolescents participated corresponding to a response rate of 71%, of which 5.4% did 122 not respond to all items of the MHC-SF. Since no data imputation for missing values was used, 123 analyses were based on total 3669 adolescents with a mean age of 16.23 years (range = 14-18 124 years), 51% of which were females. Ethical approval was acquired from the ethics review authority 125 (Dnr 2019-05620). 126

127
In the SALVe study, two well-being scales were used: MHC-SF of primary interest for this evaluation 128 [8] and the Short Warwick-Edinburgh Mental Well-being Scale (SWEMWBS) previously evaluated in 129 Scandinavian adolescents for assessing convergent validity [41,42]. The MHC-SF comprises 14 items 130 of hedonic and eudaimonic well-being. Respondents are asked to rate how often in the past month 131 they have experienced these signs of well-being on a 6-point Likert scale (never, once or twice a 132 month, about once a week, two or three times a week, almost every day or every day). The more 133 often the signs experienced, the higher the rating, and the better the well-being. The SWEMWBS 134 comprises seven items of hedonic and eudaimonic well-being. Similar to the MHC-SF, the SWEMWBS 135 and its longer version, the WEMWBS, have been widely used to measure public mental health [43].

7
Respondents are asked to rate how often in the past two weeks they have experienced the seven 137 signs of well-being on a 5-point Likert scale (none of the time, rarely, some of the time, often or all 138 the time). The more often the signs experienced, the higher the rating, and the better the well-being. 139 We used the Swedish version of the MHC-SF made publicly available by Keyes [44]. Based on the 140 input from the focus group interviews, three minor adjustments were made in the phrasing of some 141 items to better suit the adolescent context. First, examples of what the contribution to society in 142 item 4 might consist of were added; second, the last part of item 6, in English 'for people like you', 143 was deleted because it raised negative reactions in all focus groups; and third, item 10 was adjusted 144 to improve grammar. Beyond this, there was a strong consensus among participants that the 145 questions in the MHC-SF were interesting and relevant to the topic. No backtranslation of the MHC-146 SF was made. 147

Data analysis
148 Based on theory and previous studies, four CFA models were tested using maximum likelihood 149 estimation: a single-factor model consisting of all items; a two-factor correlated model testing the 150 concept of hedonic and eudaimonic well-being (i.e., items 1-3 and 4-14, respectively); a three-factor 151 correlated model of emotional, social and psychological well-being (i.e., items 1-3, 4-8 and 9-14, 152 respectively) as originally proposed by Keyes; and a bifactor model with one general and three 153 specific (i.e., not correlated) factors of emotional, social and psychological well-being. 154 Model fit was evaluated using three different fit indices: the root mean square error of 155 approximation (RMSEA), i.e., the standard deviation of the prediction errors; the comparative fit 156 index (CFI) as a measure of the discrepancy between the data and the hypothesized model while 157 adjusting for the issues of sample size inherent in the chi-squared test; and standardized root mean 158 square residual (SRMR) as an absolute measure of fit in the difference of the observed and predicted 159 correlations. RMSEA and SRMR values close to 0.06 were considered a close fit, and a CFI value above 160 0.90 was considered acceptable and above 0.95 satisfactory [45]. Configural, metric and scalar 161 8 invariance of the MHC-SF were evaluated for gender and age using the CFI, RMSEA and SRMR 162 with cut-off values of 0.010, 0.015 and 0.030, respectively, as indicators of pronounced 163 difference in fit between the nested and unconstrained models [46,47]. 164 Convergent validity, as the degree to which MHC-SF and SWEMWBS⎯both measures of hedonic 165 and eudaimonic well-being⎯are in fact related, was assessed correlating the total scale scores as 166 well as identified latent constructs from the CFA. A strong correlation with a coefficient above 0.7 167 supported convergent validity [48]. 168 Reliability, as a property of the data generated by the scale of interest here applied on Swedish 169 adolescents, was assessed by calculating the model-based estimate coefficient omega () and omega 170 hierarchical (H) for the general and the specific (group) factors (s). To further aid in the 171 interpretation of each factor's importance, and assessment of dimensionality, the explained common 172 variance (ECV) was calculated as previously detailed by Reise [27]. The conventional and more 173 frequently used coefficient alpha (that assumes tau-equivalence) was calculated for comparison with 174 coefficient omega estimates [49] and other studies that used alpha. Recognizing that all cut-off 175 values are arbitrary, the same recommendations were considered for omega as have been suggested 176 for alpha; the values above 0.7 were considered acceptable, while those above 0.8 and closer to or 177 above 0.9 indicated good to excellent reliability [50]. Finally, dimensionality was also assessed by 178 conducting exploratory factor analysis (EFA) using maximum likelihood estimation and the ratio of 179 the first and second eigenvalue, as suggested by Slocum-Gori and Zumbo [51]. A ratio of 4 was 180 considered indicative of essential unidimensionality. 181 Test-retest reliability of the MHC-SF total scale was assessed by calculating intra-class correlation 182 coefficient (ICC) estimates and their 95% confidence intervals for single and average measures based 183 on a two-way mixed-effects model where people effects were random and measure effects were 184 fixed. 185 9 All analyses that required the use of statistical software were carried out using SPSS and AMOS for 186 Structural validity

194
The four tested CFA models and their respective fit indices based on the SALVe data are summarized 195 in Table 1. The single-factor model yielded the poorest fit, followed by the two-and three-factor 196 correlated models, none of which showed an overall acceptable fit. The only model with a good fit 197 was the bifactor model, as illustrated by the AMOS input diagram in Fig. 1. The fit indices for the 198 bifactor model in Table 1 show an RMSEA close to 0.06, a CFI above 0.95 and an SRMR well under 199 0.06, indicating that the sample correlation matrix is well recovered. 200 The standardized factor loadings of the bifactor model are shown in Table 2  The degree to which item response data were unidimensional versus multidimensional is shown by 213 the ECV in Table 2 to be 0.73, and the degree to which raw scores reflect a common dimension, as 214 opposed to mostly error, is shown by the model-based reliability index H to be 0.79, thus indicating 215 good reliability of the general factor. The model-based sibling to H, the  index⎯including the 216 general as well as the specific group factors⎯and analogous to the popular coefficient , showed 217 good to excellent reliability of 0.88. Thus, when all sources of common variance are included in the 218 estimation of reliability, only a small difference is seen between  and . However, large differences 219 become evident as the viability of subscale score is evaluated by controlling for the variance due to 220 the general factor using the model-based subscale specific estimate omega hierarchical (S). In Table  221 2, for example, the coefficient  for the emotional well-being subscale is estimated to be as high as 222 0.9, whereas only 0.15 according to S⎯differences seen for all the subscales indicating poor 223 reliability of the group factors (EW, SW and PW). The ECV of 0.73 indicating that MHC-SF is 224 essentially unidimensional with a rather dominating general well-being (GW) factor was also 225 supported by EFA as the ratio of the first (6.2) and second (1.5) eigenvalue was above 4. 226 227   Table 3, no or only minor changes in fit are seen between the 242 different models indicating overall measurement invariance. The only notable decrease in fit is seen 243 for the CFI for gender between the models of scalar and configural invariance (CFI = 0.012). 244 However, no corresponding non-invariance in fit was seen in the other indices (RMSEA = 0.005; 245 SRMR = 0.003). 246 SWEMWBS. Since both these instruments are designed to measure hedonic and eudaimonic aspects 252 of mental well-being, the correlation between the two should be strong. Indeed, both Pearson's 253 correlation coefficient and Spearman's rho yielded an estimate of 0.766 in support of convergent 254 validity. Using AMOS, and the bifactor model in Figure 1, we also correlated the GW factor of MHC-SF 255 with a corresponding GW factor identified for SWEMWBS⎯first via EFA and then tested with single-256 14 factor CFA model that yielded an acceptable to good fit (RMSEA = 0.079; CFI = 0.975; SRMR = 257 0.0244). A correlation coefficient of 0.86 was found between the two modelled GW factors. 258

259
Given the increased use of measuring well-being, it is important to psychometrically evaluate these 260 instruments that cast light on how the concepts of well-being and positive mental health can be 261 described and understood. The aim of this study was therefore to evaluate the MHC-SF for the first 262 time in Swedish adolescents. 263 Regarding factor structure, the results indicate that positive mental health as measured by the 264 MHC-SF is best described by a bifactor model with a dominant general well-being factor and three 265 specific group factors of emotional, social and psychological well-being. The relatively high 266 proportion of ECV by the general well-being factor⎯73%⎯indicates that the MHC-SF, essentially, is 267 unidimensional [52]. The latter finding is also supported by EFA with a ratio of the first and second 268 eigenvalues above 4 [51]. Thus, the interpretation of the total scale score as an indicator of a single 269 construct should be valid. As a result, the specific group factors contribute 27% of the ECV in total. 270 Taken together, our results on factor structure are well in line with the only other study, to our 271 knowledge, that has been published thus far and included testing of the bifactor model in 272 adolescents [35]. The results are also in line with recent psychometric evaluations in adults that 273 compared other factor solutions to the bifactor model [28, 31-36, 39, 40]. These studies and our 274 study show that the bifactor has superior fit compared with previously tested models, including the 275 correlated three-factor model first proposed by Keyes [9,15,22]. 276 To verify that the bifactor structure is valid in different groups, measurement invariance of the 277 MHC-SF was evaluated by multigroup analysis. Overall, the results support the metric, scalar and 278 configural invariance of the model across gender and age groups. Similar multigroup invariance of 279 the bifactor model in adolescents was recently obtained by Reinhardt et al [35]. In addition, in a large 280 study on young adults, the metric invariance and cross-cultural replicability of the bifactor model 281 15 were supported by a multigroup confirmatory analysis of the bifactor structure between samples in 282 38 countries [37]. 283 An advantage of the bifactor model compared with, for example, single-or multi-factor correlated 284 models is that it allows for evaluation of the relevance and reliability of subscale scores after 285 controlling for the variance of the general factor [27]. In the present study, this was done by 286 calculating ECV for each latent factor along with the model-based reliability estimate omega 287 hierarchical (H)) for the general factor and the equivalent estimate for each subscale (S). While the 288 latter supported good reliability of the general factor alone (H) = 0.79) and excellent overall 289 reliability in the general factor together with the group factors ( = 0.88), it also revealed poor 290 reliability for the subscales of emotional, social and psychological well-being H) (0.15, 0.24 and 0.13, 291 respectively). The latter finding is well in line with previous studies that evaluated the bifactor model 292 on the MHC-SF and assessed model-based reliability with omega hierarchical [35,37]. Hence, the use 293 of subscale scores alone cannot be recommended. In addition, assessment of internal reliability of 294 these subscales by estimating coefficient  cannot be recommended since it tends to conflate 295 multiple sources of systematic variance when data are associated with a multidimensional model 296 [38]. Nevertheless, total scale scores of the MHC-SF yielded good to excellent internal consistency as 297 well as test-retest reliability as indicated by strong ICCs using data from our pre-study. 298 Regarding convergent validity, a strong correlation (0.76) was found in the present study between 299 total scale scores of MHC-SF and SWEMWBS, stronger than previously reported by Clarke et al (0.65), 300 also in adolescents [53]. In addition, using the modelled GW factors of the two scales, we filtered out 301 the error variances, which then amplified the correlation further (0.86), providing ample support for 302 convergent validity. 303

304
One of the strengths of our study was using the pre-study conducted to test the MHC-SF with respect 305 to face validity and test-retest validity. It gave us valuable qualitative input from adolescents in the 306 specific age groups that later participated in the population study on which the main analyses of this 307 evaluation were based. With input from the focus groups, we could make the necessary, yet small, 308 linguistic adjustments, and there was a strong consensus among the informants that the content of 309 the 14 items was relevant to them and the topic. Another strength of the present study is that its 310 main analyses are based on a whole population study with a relatively decent participation rate. The 311 data should thus be highly representative for adolescents at least in Region of Västmanland County, 312 but probably also to Swedish adolescents in general because of the size of the study. A limitation of 313 the study is that the sample consisted of ninth graders of compulsory school and second graders of 314 upper secondary school only. Thus, the analysis of measurement invariance across age was limited, 315 and the psychometric characteristics of MHC-SF for younger adolescents remain unknown. 316 Furthermore, this study was conducted on a general population. The factor structure might be 317 different in clinical populations, for example. Future research on adolescents would benefit from 318 including clinical populations too. In Sweden, there is also a need for a corresponding psychometric 319 evaluation in adults and the elderly. 320

321
This large population study on Swedish adolescents found that the MHC-SF is essentially 322 unidimensional and best described by a bifactor model with a dominant general well-being factor 323 and three specific group factors of emotional, social and psychological well-being. Its general well-324 being factor has good internal reliability, but the reliability of its subscales of the specific group factor 325 is poor, and thus should not be used alone. Test-retest reliability was good, and convergent validity 326 was supported. In conclusion, we consider the MHC-SF to be a psychometrically sound instrument 327 for overall mental well-being in Swedish adolescents. Availability of data and materials 349 Data is available on request from corresponding author. 350