Study design and setting
The current study employed a web-based cross-sectional design. We followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines for observational cross-sectional studies25. We distributed the survey weblink (URL) to all players in the Japan Rugby Players' Association through one or two team representatives participating in regular meetings. The potential participants were invited to complete the anonymous survey. They were informed about the survey process, including the purpose of the study, data collection procedures, and the consequences of participating or not participating via the cover page of the questionnaire. Consenting participants went on to complete the questionnaire, which took less than 10 minutes. The participants were provided with individual one-time access to the survey using IP address filtering access to a tablet or laptop computer to complete the survey. The cross-sectional data were collected shortly before the start of the off-season from December 2020 to February 2021.
We collected data from a total of 612 rugby players (565 Japanese players and 47 foreign players) registered with the Japan Rugby Players Association. The participants were all aged 18 years and over and belonged to the Japan Rugby Top League. The current survey was available in both Japanese and English. No exclusion criteria applied. Overall, 227 of the 612 players agreed to complete the survey (response rate: 37.1%). The response rate of this survey was consistent with other mental health surveys in Japan26. We analysed 219 out of 227 participants born in Japan as the target population in the current study. All investigators received the learning course on research ethics, and this study was approved and facilitated by the Research Ethics Committee at the National Center of Neurology and Psychiatry (approval number: A2020-058).
The Athlete Psychological Strain Questionnaire (APSQ)
The APSQ is a brief, self-report questionnaire specific to the elite sporting context, comprised of 10 items (e.g., ‘During the past 4 weeks, I could not stop worrying about injury or my performance’) scored on a 5-point scale (from ‘none of the time’ (1) to ‘all of the time’ (5)). A total score ranging from 10 to 50, with higher scores representing more psychological distress, is calculated by summing the answers on the 10 items22. A more recent study has shown that a scores of ≥15, ≥17 and ≥20 as representing moderate, high and very high levels of athlete-specific distress23. In the preliminary stage of developing the original version of APSQ22, a two-step approach was used in a sample of 1,007 currently competitive Australian elite male athletes from professional sports (M = 23.67, SD = 4.16). The exploratory and confirmatory factor analysis and tests of differential item functioning were conducted with the samples randomly partitioned into calibration (n = 497) and validation (n = 510) samples. Exploratory factor analysis, with parallel analysis, conducted on the calibration sample supported a second-order with three-factor model. The subscales included Self-Regulation, Performance, and External Coping domains, accounting for 50.44% of total scale variance. In the second-order model, the path coefficient from the upper factor "Athlete psychological strain" to each factor was 0.8 or more. In the confirmatory factor analysis, excellent model goodness-of-fit indicators were provided. The mean score for the 10 items among the Australian male athletes was 14.67 (SD = 5.47).
For this study, the original APSQ was translated into Japanese by the first author (Y.O.). The professional athletes (S.K. and K.H.) in this research team modified the terms and sentences to improve the readability for athletes. A bilingual English speaker then produced a back-translation. Finally, the back-translated version of the APSQ was confirmed and approved by the researchers who had originally developed the APSQ (S.M.R and R.P).
We used the Kessler-6 (K6) and the WHO-5 Well-Being Index, widely used measures of psychological distress and wellbeing to assess convergent validity with the APSQ-J. Previous studies have shown that the performance of the K6 in screening mood and anxiety disorders, as assessed by the areas under the receiver operating characteristic curves (AUCs), was excellent27, with values as high as 0.86, 0.89, and 0.94 from US28, Australian29, and Japanese general samples30, respectively. In the K-6, the scores are categorized to indicate the respondents’ mental health status over the previous 30 days. Responses to items are made on a 5-point scale. The K-6 was developed and validated based on many epidemiological surveys and is widely used as a screening tool in assessing treatment progress in common mental disorders such as anxiety and depression in people in the general community31. Cronbach’s α was 0.91 in the present sample for the K6. The WHO-5 Well-Being Index is a 5-item self-report scale that assesses positive aspects of mental health (i.e., “I have felt cheerful and in good spirits”) over the previous 2 weeks32,33. The raw score is calculated by totalling the figures of a 5-point scale (i.e., 0 = “at no time”; 5=“All of the time”). The score ranges from 0 to 25, with 0 representing the worst possible and 25 representing the best possible well-being. A score below 13 indicates poor wellbeing and is an indication for assessing for depression according to the ICD-10. Cronbach’s α was 0.90 in the present sample for the WHO-5.
The background information and demographic survey items included age, country of birth, educational attainments, marital status, the number (if any) of dependent children, residential status, the national team's experience, and playing status for the last season.
Descriptive statistics were used to characterise the sample. We conducted a two-stage process to validate the factor structure of the APSQ-J using exploratory factor analysis (EFA) in a randomly partitioned calibration sample and confirmatory factor analysis (CFA) in a separate validation sample. EFA was undertaken to determine the underlying factor structure of the APSQ-J. To determine which items belonged to each factor, we extracted items if they loaded ≥0.3. In addition, we examined the number of factors based on scree plots. We adopted the oblique rotation (promax) with principal factor method. In CFA, we evaluated the fit of the model with the data using the chi-squared statistic (CMIN), root mean square error of approximation (RMSEA), and comparative fit index (CFI). According to conventional criteria, an excellent fit would be indicated by CMIN/df<2, RMSEA<0.05, CFI>.97, while CMIN/df<3, RMSEA<0.08, CFI>.95 demonstrates an acceptable fit34. After confirming the factor structure, we examined the internal consistency and convergent and divergent validity. To evaluate internal consistency, the Item-test, Item-rest correlation and Cronbach’s alpha was calculated. In terms of convergent validity, Pearson product-moment correlation coefficients were calculated to determine if the APSQ-J correlated significantly with the K-6 and WHO-5. Therefore, we determined convergent and divergent validity through examining patterns of statistically significant correlations with the K-6 (convergent validity evidenced by significant positive associations) and the WHO-5 (divergent validity evidenced by significant negative associations). We conducted a one-sample t-test for comparing the current data to the means reported in developing the APSQ paper from the 1,007 currently competing Australian elite male athletes22. For the effect size of the mean difference between the two countries in each assessment point, through computing Cohen’s d, which was graded as 0.20 = small, 0.50 =medium and 0.80 = large. All analyses were conducted with Stata version 16 (StataCorp LLC, College Station, TX, US).