The Swedish version of the MARS-E was translated from German by a translation company. Content validity was assessed by experts from the research project, and schoolteachers who participated in the two pilot tests. Corrections and clarifications on minor issues were made. As part of evaluating the content the instrument was piloted at three occasions in two schools in Sweden (N = 118). From the original 36 items, the Swedish MARS-E was reduced to 16 items, with 8 items originating from the cognitive component and 8 from the affective component. The target population for the current study were students in Swedish primary schools.
2.1 Participants and Procedure
The sample included grade 4 students from Sweden, selected from 18 schools across 6 different municipalities, spanning from the northern to the southern regions of the country. The participants were n = 429, 9–10-year-old students, 213 girls and 214 boys (2 preferred not to answer). The project underwent ethical review by the Swedish Ethical Review Authority and consent was collected by the student’s legal guardians. Participating teachers, students and guardians were informed about the project’s aim, procedure, and data handling.
Data was collected in autumn 2022, spring 2023 and autumn 2023. Two research assistants visited participating schools and carried out the measurements one class at a time (ca 18/class). The tests were split over two sessions with the questionnaire and math test administered on separate lessons, to a) reduce student fatigue, and b) reduce the risk of inducing awareness of anxiety. Verbal and written instructions were provided, with the option for students to request clarification or assistance. Though no specific guidance was given for solving math tasks. Each session was supervised by at least one research assistant in addition to their own teachers. All assessments were administered digitally.
2.2 Assessments
Previous research offers guidelines for how MA correlates with other anxieties, specifically test anxiety [3, 20]. This is also true for the relationship between math achievement and MA [5, 7, 11]. As part of assessing the validity of the Swedish MARS-E, criterion variables were employed to determine relations between the MA-measure and math achievement, as well as test anxiety, math interest, and math self-concept. A high positive correlation between math anxiety and test anxiety as well as a negative correlation with math self-concept would indicate criterion validity.
2.2.1 Swedish MARSE
The Swedish MARS-E was measured with 16 items where participants took a stand on statements describing scenarios about math on a 4-point likert scale. Out of the 16 items, 8 targeted the worry (cognitive) component of MA, where the answer options ranged from “Not at all worried” to “Very worried”. The remaining 8 items targeted the nervous (affective) component, with answer-options ranging from “Not at all nervous” to “Very nervous”. All items are presented in Appendix A.
2.2.2 Test anxiety
Test anxiety was assessed using the Swedish version of the STAS (School Test Anxiety Survey [25]), comprising 6 items rated on a 4-point likert scale. Participants responded to statements concerning their emotions in test-related situations. The answer options ranged from "Not at all nervous" to "Very nervous". The internal consistency across all 6 items was robust, with alpha levels ranging from αT1−T3 = 0.85–0.89, indicating good reliability.
2.2.3 Math self-concept and interest
In the past, self-beliefs have been a subject of debate regarding whether they have a significant relationship with actual academic performances. However, when isolating self-beliefs into domain-specific constructs and matching them to specific academic subjects, the relationship becomes clearer, indicating significant links [30]. Shavelson et al.’s [31] model of self-concept was established as a multidimensional hierarchy with a top tier consisting of a general self-concept. From the general self-concept, the model then branches out into academic- and non-academic self-concepts, and further into more domain-specific branches. Drawing on Shavelson et al.’s model, Marsh et al. [32] published the SDQ (I) containing 7 blocks of self-concept measures. The authors reported good reliability regarding the mathematics block (αpublic schools = 0.92; αprivate schools = 0.94). In the current study, the mathematics block measuring both math interest and math self-concept [33] are used. Each variable consisting of 3 statements on a 5-point likert scale ranging from 1 = False to 5 = True. Cronbach’s alpha for each variable indicated good reliability on all three measurement occasions (αmath self−concept = 0.83–0.85; αmath interest = 0.92–0.94).
2.2.4 Math performance
Math performance was measured with a digital test battery (FUNA-DB) which consists of six different sub-tests: (1) Number Comparison, (2) Digit Dot Matching, (3) Number Series, (4) Single Digit Addition, (5) Single Digit Subtraction, and (6) Multiple Digit Calculation. Räsänen et al. [34] found that the test battery was reliable and valid and was best described as a 2-factor model based on a large sample of 9–15-year-old Finnish students (invariant across Finnish speaking and Swedish speaking students). In the current study, four subtests were collapsed and used as a composite score of arithmetic fluency, providing a stable measurement (sub-test: 3, 4, 5 & 6, corresponding to one of Räsänen et al.’s factors). Response-consistency for each subtest was analysed on item-level with the KR-20 method, which resulted in acceptable estimates of 0.856, 0.957, 0.964, 0.934, for sub-tests 3, 4, 5 and 6, respectively. Prior to every actual sub-test, there were two practice rounds, one where the student received feedback on how he or she had succeeded and one that mimicked the actual test more closely.
2.3 Analysis
R (version 4.3.0) was used for computing descriptives, t-tests, reliability, and Little’s MCAR test, utilising the r-base package ‘stats’ [35], the ‘psych’-package [36], and the ‘naniar’-package [37]. Cronbach's alpha estimates were reported for all three occasions, demonstrating the consistent reliability of the instruments used in the current study. Beyond invariance and reliability analysis, the focus of the current study was on the T1 measurement occasion with respect to criterion validity. Mplus (version 8.5) was used to estimate latent measurement models of the Swedish MARS-E. Applying Confirmatory Factor Analysis (CFA), models were assessed based on a set of model fit indices, structural loadings between the latent factors and items, and previous theory. The final model underwent a series of constraints, effectively evaluating invariance across gender and time. This method (invariance testing) tests the hypothesis that the model with the constrained parameters is close to equal in the two samples [38]. Lastly, the measurement model of MA was extended by adding correlations to math performance, test anxiety, math self-concept, math interest and gender. The relations were assessed and juxtaposed to previous research.
Various patterns of missing data were identified in the dataset. To determine the nature of the missingness, Little’s MCAR test (Missing Completely At Random) was carried out on all the current study’s included variables. Little’s MCAR test examines if missing data are systematically missing [39, 40]. The resulting non-significant p-value suggested that the missingness was completely at random (χ2 = 77.4, df = 73, p = 0.341).
Table 1
Means, standard deviations and t-tests on sum score variables.
Variable | M (SD) Boys | M (SD) Girls | M (SD) Total | T-test | Ntotal (Nboys, Ngirls) |
---|
MARS-ET1 | 23.78 (7.47) | 26.30 (8.49) | 25.01 (8.07) | t = 2.98** | 360 (180, 178) |
MARS-ET2 | 22.11 (6.33) | 24.91 (8.05) | 23.50 (7.36) | t = 3.74** | 376 (190, 184) |
MARS-ET3 | 21.63 (5.95) | 25.99 (8.7) | 23.75 (7.73) | t = 5.63** | 375 (190, 183) |
TaxT1 | 11.05 (3.37) | 12.35 (3.68) | 11.70 (3.57) | t = 3.49** | 360 (178, 180) |
InterestT1 | 10.01 (3.88) | 10.10 (3.88) | 10.06 (3.88) | t = 0.22 | 357 (178, 177) |
SelfT1 | 11.96 (2.68) | 10.91 (2.73) | 11.43 (2.76) | t = 3.69** | 360 (178, 180) |
ArithT1 | 0.24 (1.10) | -0.22 (0.85) | 0.00 (1.00) | t = 4.31** | 367 (179, 186) |
**p < 0.001. Tax = test anxiety, Interest = math interest, Self = math self-concept, Arith = standardized arithmetic fluency. |
2.4 MARS-E Dimensionality and Invariance Testing
Two MA-models were tested on the data: the original 2-factor model, used in Henschel and Roick’s study [17], and a unidimensional model. The models were evaluated based on model fit indices and factor correlations. Model fit indices for all models were satisfactory by standard guidelines, recommended by Hu and Bentler [41]. CFI and TLI were close to 0.95, SRMR-values close to 0.08, and RMSEA-values close to 0.06. Since the data for the Swedish MARS-E was categorical, the models were estimated with the Weighted Least Squares estimator with adjusted means and variances (WLSMV).
A study of potential gender differences in MA was carried out. Ensuring an unbiased comparison between genders, necessitates that the scale used is invariant across groups. Accordingly, the MA invariant model was subjected to a series of constraints to rigorously test for invariance. These constraints were measured as (1) configural, (2) metric, (3) scalar and (4) strict invariance. The configural invariance is the least restrictive model, maintaining same model specification for each group. In the case of the Swedish MARS-E the number of factors was determined by the dimensionality investigation, with other model parameters freely estimated. The second model, metric invariance, assumes configural invariance is met, with factor loadings additionally constrained to equality across groups. The third model, scalar invariance, requires metric invariance in addition to equal intercepts across groups [38]. Given that the data for the Swedish MARS-E consisted of categorical ordered variables, scalar invariance tested the equality of thresholds across groups instead of intercepts. Lastly, strict invariance is achieved if the model remains stable even when residuals are constrained to equality in addition of scalar invariance [42].
A common cutoff value for assessing change in model fit during measurement invariance testing is a change in the CFI of less than or equal to 0.01 and a change in the RMSEA of less than or equal to 0.015 [43]. Accordingly, each model was evaluated both independently and juxtaposed against the preceding less restrictive model, to determine invariance.