Study design
This study was a methodological approach to translate the Career Competencies Questionnaire into Japanese and validate its psychometric properties using a cross-sectional survey.
First, the English version of the CCQ was translated into Japanese. Second, we conducted an exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) with a separate sample. In the first survey, we conducted an item analysis and exploratory factor analysis of the CCQ-J. In the second survey, we conducted a confirmatory factor analysis and evaluated the construct validity and concurrent validity and reliability of the CCQ-J. All questionnaires distributed in this study were in Japanese.
Translation
Before we began the development of the CCQ-J, we obtained permission for translation from Dr. Akkermans, who developed the original Dutch version and confirmed that it was possible to translate an English version into a Japanese version. The English version of the CCQ was translated into Japanese by an English and Japanese bilingual translator. Next, we checked the translated questions, modified the wording to ensure it was easier for the respondents to understand, and created a 21-item draft scale. The draft scale was back translated by two other translators, not involved in the original translation. The draft scale was translated into English by a translator who did not see the original text and prepared a back-translated manuscript. Another translator compared the back-translated manuscript with the original English version and checked it for differences in terminology, expressions, and minor nuances to verify its accuracy. After the corrected draft scale was received, we reviewed these questions. We then asked four registered nurses to answer the corrected draft scale and check whether any items were difficult to understand and whether the contents were properly understood. Finally, we reviewed all the items and developed a preliminary Japanese version of the CCQ.
Sample Size
In general, larger sample sizes increase the generalizability of the results of an EFA [11,12]. Pett et al. (2003) argued that there should be least 10 to 15 subjects per initial item for factor analysis regarding sample size [13]. Hence, the required sample size was at least 210 nurses.Since COVID-19 put a strain on the medical system during the first survey, we limited the target facilities to one hospitalto minimize the burden of participation on the medical organization and target nurses.
Similarly, the larger the sample size for the CFA, the better. DeVellis(2016)explained that sample sizes of 100 were poor, 200 were normal, 300 were good, 500 were very good, and 1,000 were excellent [14]. Hence, the required sample size in the second survey was at least 500 nurses. Since the response rate was expected to be approximately 40%, we distributed the QR code to approximately 1300 nurses.
Participants and Data Collection
In this study, only registered nurses were included. Licensed practical nurses were excluded because their qualifications are different from those of registered nurses. Nursing assistants and students not licensed as registered nurses were excluded from the study. Furthermore, nurses on long-term leave were also excluded.
The first paper-and-pencil survey was conducted in June 2020. In total, 298 registered nurses from a Japanese hospital with more than 300 beds in a prefecture participated. We explained the contents and methods of the study to the nursing director and sub-director both in writing and orally and obtained their consent.A total of282 surveys were picked up by the researcher after completion; of which, six were blank or incomplete and were excluded. Therefore, 276 participants were included in the analysis (Collection rate: 94.6%, Effective response rate: 92.6%).
The second web survey was conducted between July and August 2021. The participants were registered nurses who worked in hospitals with more than 100 beds in Tohoku region, Japan. We mailed the research request letters to 80 surveyed hospitals selected by random sampling, and 24 hospitals returned consent forms for research cooperation (consent rate: 30%). A total of 50 copies with QR codes were mailed to the hospitals that provided consent. A hospital specified that only 22 copies of the survey needed to be mailed. Thus, a total of 1,172 copies were distributed. In total, 536 nurses responded to the web survey, whichincluded nine associate nurses whomet the exclusion criteriaand three that were returned blank or incomplete. Consequently, 522 nurses were included in the second survey. Table 1 lists the attributes of the participants (Collection rate: 45.7%, Effective response rate: 44.5%).
Measures
Career competencies were assessed using the preliminary 21-item Japanese version of the CCQ.All the items were rated on a 5-point Likert-type scale that ranged from 1 (completely disagree) to 5 (completely agree). They were evaluated using the total score on each subscale.
Work engagement was assessed using the Japanese version of the Utrecht Work Engagement Scale (UWES-J) [15]. The items were rated on a 7-point Likert scale that ranged from 0 (never) to 6 (always).
Life satisfaction was assessed using the Japanese version of the Satisfaction with Life Scale (SWLS-J) [16]. The items were rated on a 7-point Likert scale that ranged from 1 (strongly disagree) to 7 (strongly agree).
Data Analysis
Statistical analyses were performed using SPSS (version 23.0) and AMOS (version 23.0).Before the EFA, we evaluated the participants’ descriptive statistics. Skewness and kurtosis were calculated to determine the normality. We also evaluated ceiling effect, floor effect, and item-total (I-T) correlation. Bartlett’s sphericity test was performed for factor analysis compatibility and the Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy was calculated to determinesample validity.
In the EFA, the number of factorsweredetermined using the Scree plot, and factor structures with promax rotation were obtained. We assessed the factor pattern matrix to determine the factor structure. Since we used oblique rotation, we judged thatin a heywood case, the commonality exceeded 1.0. If the pattern loadings exceeded 1.0, it was confirmed that the commonality exceeded 1.0 [13, 17]. Crossloading judged the item if pattern loadingwas at 3.0 or higher in severaldimensions. Some researchers argue that weak loadings are those that are < |.30| [11, 18]. From a practical standpoint, we assessed whetherthe pattern scoresfor each factor were > |.30|. Furthermore, items with factor pattern scores > |.40| were retained. If we obtained crossloading items, we assessed Cronbach’sαforthe group of items that wereloaded on a given factor. According to Pett (2003), alphas can be used to evaluate a factor’s internal consistency and decide where to best place an item with strong loading on several factors [13].
In the CFA, structural equation modelling (SEM) was used to evaluate the construct validity of the factor structure provided by the EFA. We examined the chi-square value (CMIN), goodness-of-fit index (GFI), adjusted goodness-of-fit index (AGFI), comparative fit index (CFI), and root mean square error of approximation (RMSEA) as fit indices. To evaluate concurrent validity, Pearson’s correlation coefficients of the total score of the CCQ-J and the subscales between work engagement and life satisfaction were examined. Akkermans (2013) mentioned that career competencies acted similarly to personal resources consistent with the Job Demands-Resource model (JD-R model) [8]. Some studies showed that career competencies were positively related with work engagement and life satisfaction, consistent with the JD-R model [8, 9]. Thus, these positive correlations indicated the concurrent validity of the CCQ-J. The internal consistency of the entire scale and the subscales was examined using Cronbach's alpha.