Item finalization through correlation of preliminary items and iterative verification of reliability. Before analyzing the factor structure of the preliminary scale, we determined the degree of correlation between items by analyzing the overall correlation of each sub-factor and analyzed the internal consistency of the items. Analysis revealed that the correlation of 9 out of 55 items (11, 12, 13, 15, 22, 23, 24, 26, 55) was not statistically significant; thus, the nine items were removed. Afterwards, multiple factor analysis and item analysis were conducted to remove items with low factor loadings and reliability problems and that posed duplication problems in terms of content validity. Thus, 22 problematic items (7, 9, 10, 17, 18, 19, 20, 28, 29, 34, 35, 36, 41, 42, 43, 44, 45, 46, 47, 48, 49, 52) were eliminated based on these criteria. The final selection included 24 items.
Descriptive statistics of the scale items. Descriptive statistics were performed on the items to verify the validity of the final scale, and the results are shown in Table 1. Overall, the item scores were evenly distributed from one to five points with a mean at the three-point level. There were no items exceeding ± 3 in skewness, and kurtosis did not exceed ± 7, indicating that the items used in this study had no problem with normality [76, 77].
Exploratory factor analysis. . To check the factorability of the measure by testing sampling adequacy, we conducted a preliminary analysis. In the result, KMO was .87, indicating a satisfactory selection of variables for factor analysis [78]. In addition, Bartlett’s sphere formation test, which indicates the suitability of the factor analysis model, was χ2 = 4975.167 (p < .01). This indicates both that factor analysis is appropriate and that there is a common factor, meaning that the existing data for evaluating ego strength through game play attitude observation were suitable for factor analysis.
To identify the factor structure of the five factors of Coping Strategy, Cognitive Strategy, Ego Restriction, Interpersonal Functioning, and Frustration Tolerance that were theorized in process of the scale development, the principal axis factoring method was applied for EFA. The factorial rotation method used was the varimax technique, which is an orthogonal rotation method. A criterion for the number of factors was based on the Kaiser standard (Eigenvalue > 1.0) and the scree test standard [79, 80]. As shown in Figure 1 and the results from the EFA presented in Table 2, the items with eigenvalue of 1 or more among the 24 items of the scale were classified into five factors. Factor 1 accounted for 27.8% of the total variance with a factor loading ranging from .60 to .79, Factor 2 accounted for 13.3% with a factor loading ranging from .64 to .78, Factor 3 accounted for 8.6% with a factor loading ranging from .57 to .83, Factor 4 accounted for 6.8% with a factor loading of .60 to .82, and Factor 5 accounted for 4.6% with a factor loading ranging from .50 to .72. Thus, the five factors accounted for 61.2% of the total variance. Further, the five extracted factors were consistent with the theoretical concepts selected by the researcher at Stage 1 to measure ego strength through game play attitude observation.
Confirmatory factor analysis. Next, CFA was conducted to examine whether the items and factors obtained through EFA fit the theoretical model. All scores were greater than .90 were considered to indicate a good fit, χ2 = 1172.504, p <.01, goodness of fit index indicator (GFI) = 0.944, Tucker-Lewis Index (TLI) = 0.945, and comparative fit index (CFI) = 0.965 [81, 82]. The root mean square error of approximation (RMSEA) = 0.048 was less than .08; based on the fitness index criterion, this factor structure can be evaluated as having good fit [83, 84, 85, 86]. In addition, the average variance extraction value was above .5, and the conceptual reliability value was above .7, verifying the validity of all sub-factors. The factor structure model of this scale is shown in Figure 1.
Correlation and reliability of game play attitude observation-based ego strength evaluation scale. The correlation between the whole scale and each sub-factor and the reliability are shown in Table 3. Each of five factors had a correlation ranging between .20 and .74 (for all correlations, p < .01) and was found to have a statistically significant positive correlation. The correlation between the overall scale and each sub-factor ranged between .51 and .74 (for all correlations, p < .01). Thus, all showed statistically significant positive correlation. These results show that each sub-factor measures a different domain of ego strength while simultaneously measuring one overarching super-concept. In addition, the Cronbach’s alphas were .84, .86, .85, .82, .70, and .70 for the whole scale, Coping Strategy, Cognitive Strategy, Ego Restriction, Interpersonal Functioning, and Frustration Tolerance, respectively. These results show that this scale has strong internal consistency among the items. In addition, the correlation coefficients between the sub-factors (rs ranging from .20 to .56) were lower than those between the whole scale and sub-factors (rs ranging from .51 to .74). These results show that the sub-factors constituting the scale developed in this study satisfy the criteria that they should be homogeneous yet somewhat independent [87].
Stage 4: Verification of Criterion-Related Validity
Participants. Participants in Stage 4 included 25 play therapists and play therapist-supervisors who each had more than 10 years of clinical experience. To verify the criterion-related validity of this scale, 25 participants were each asked to select two children aged 7 to 9 years from their clinical cases, one should be judged to have strong ego strength, the other should be judged to have weak ego strength. These two children were either currently receiving counseling or had received counseling within the past six months. Of the 89 sessions with children that were analyzed, there were 24 game play therapy sessions of children evaluated to have strong ego strength and 63 game play therapy sessions of children evaluated to have weak ego strength.
In addition, 55 children aged between 7 and 9 years who received play therapy at two child counseling centers in Seoul area had their game playing attitudes analyzed to verify the scale’s concurrent validity. Children aged 7, 8, and 9 years were 30.9% (N = 17), 41.8% (N = 23), and 27.3% (N = 15), respectively, while 72.7% (N = 40) were boys and 27.3% (N = 15) were girls.
Measurements. The Rorschach test [88] measures the respondent’s cognitive style and psychodynamic construct based on the respondent’s response to ten inkblot cards. The evaluator directly assesses the child’s personality characteristics on the types of variables of the comprehensive system of this test, including resistance to stress, affect control, interpersonal skills, egocentricity, cognitive organization skills, copying deficit skill, presence of depression and perception disorder, excessive vigilance, and degree of compulsion [89]. The Rorschach test provides an optimal opportunity to measure ego impairment because it induces the respondent to use cognitive, affective, and human or representational resources to organize an ambiguous and complex task [90]. Thus, a number of researchers have verified the diagnostic efficacy and validity of the special indexes of the Rorschach CS [91, 92], and contributed to measuring ego strength by using certain variables of the Rorschach test or developing Rorschach Prognostic Scale (RPRS) and Ego Impairment Index (EII) [93, 94].
In our study, the reasons for using the Rorschach test as a criterion-related validity test are as follows. First, it was difficult to ensure the reliability of self-report test due to the age of child participants, thus a type of test conducted by the experimenter was needed. Unlike other self-report tests, the Rorschach test’s responses cannot be consciously manipulated by the test subjects. Second, the validity and reliability of the Rorschach test were scientifically assured with the introduction of Exner’s Comprehensive System (CS) [91, 92]. In addition, the Structural Summary in CS allows the evaluation of the overall domain of individual thinking, emotions, interpersonal relationships, frustration tolerance and coping resources, and cognitive processing and intervention [88], and it was most consistent with the construct of ego-strength that we attempted to evaluate. Past studies have created scales with some Rorschach variables [95], in the current study, however, we excluded random selection of variables and used all seven core sections and six special indexes to evaluate a broader and more comprehensive domain related to ego-strength.
Procedure and data analysis. The CESS was designed in a way such that higher scores are associated with weaker ego strength in order to improve clinical efficacy in detecting problematic behaviors. Thus, two positive items from F2: Cognitive Strategy, F4: Interpersonal Functioning, and F5: Frustration Tolerance were reverse scored. First, the participating children undergoing therapy were divided into strong ego strength and weak ego strength groups, and a t-test was performed to analyze differences in scores of the five factors of the CESS between the two groups in order to determine criterion-related validity of the CESS.
Next, concurrent validity of the CESS was determined by analyzing the correlations between the scores of the 5 factors of the CESS and scores of all variables of six special indexes of Rorschach CS (Perceptual Thinking Index; PTI, Depression Index; DEPI, Copying Deficit Index; CDI, Suicide Constellation; S-CON, Hypervigilance Index; HVI, Obsessive Style Index; OBS) and the seven sections of structural summary (Core Section, Ideation Section, Affect Section, Mediation Section, Processing Section, Interpersonal Section, and Self-Perception Section). Two clinical psychologists interpreted the Rorschach test of 55 children, and the inter-rater reliability was good, with .89.
Results.
Group Differences. As shown in Table 4, a significant difference between the two groups emerged through the analysis of the game play therapy sessions of children who were reported to have strong and weak ego strength to verify the criterion-related validity. As the CESS is designed in a way that higher scores represent weaker ego strength, a higher score on the CESS indicates weak ego strength. The children reported to have weak ego strength had significantly higher mean scores on the overall scale and in the sub-variables of Coping Strategy, Cognitive Strategy, Ego Restriction, Interpersonal Functioning, and Frustration Tolerance. This suggests that this scale can distinguish between groups with strong and with weak ego strength.
Concurrent Validity: Correlations between CESS and Rorschach. In order to determine concurrent validity of the CESS, we analyzed the correlations between the scores of the five factors of the CESS and scores of all variables in the seven sections of the comprehensive system of the Rorschach test (Core Section, Ideation Section, Affect Section, Mediation Section, Processing Section, Interpersonal Section, and Self Perception Section) and six special indexes (Perceptual Thinking Index, PTI; Depression Index, DEPI; Copying Deficit Index, CDI; Suicide Constellation, S-CON; Hypervigilance Index, HVI, and Obsessive Style Index, OBS).
The results are shown in Table 5. Mild to moderate correlations were obtained between the CESS factors and Rorschach variables. With regard to Rorschach index, the correlations between F5 (Frustration Tolerance) and F2 (Cognitive Strategy) factors and the CDI were significantly positive, r = .33 (p= .013) and r = .29 (p= .033), respectively. Rorschach CDI was calculated based on the individual’s stress tolerance and cognitive resources, mental activity (EA<6, Passive>Active+1) and coping resources (AdjustD<0), interpersonal skills (COP<2, AG<2), emotional expression (WSumC < 2.5, Afr < .46), and intimacy formation (SumT>1, Isolate/R > .24). An increase in CDI suggests that individuals have low stress tolerance, low coping skills, and passive cognitive performance in social settings [96]. Therefore, the positive correlation between F5 (Frustration Tolerance) and F2 (Cognitive Strategy) of the CESS and CDI indicates the validity of the CESS.
In addition, F3 (Ego Restriction) factor was significantly positively correlated to Depression index (DEPI), r =. 27 (p =.049). Considering that there are four areas out of 7 criteria of the Rorschach DEPI, egocentricity (3r+(2)/R>.44 and Fr + rF = 0), restriction of emotional expression (Afr<.46, Blends < 4), increase of isolation index ([Bt + 2×Cl + Ge + Ls + 2×Na] / R > .24), and restriction of cooperative interaction (COP<2) [88], this suggests a contraction and withdrawal of social activities due to energy decline. Thus, the positive correlation between DEPI and F3 (Ego Restriction) of the CESS seems to ensure the validity of social avoidance and withdrawal tendency for the CESS F3. These results indicate that weaker ego strength is associated with lower social skills and increased avoidance.
The F1 (Coping Strategy) factor and the other CS variables, Afr (degree of interest in affective stimuli), S (white space response), and S-% (proportion of distorted form that involve use of white space), were also significantly positively correlated, r = .48 (p = .000), r = .41 (p = .002), and r = .31 (p = .023), respectively. These results indicate that children who display greater number of inappropriate coping strategies are more likely to show emotional response, passively express frustration, and lose judgement ability so that they have a distorted perception of a situation when frustrated.
F2 (Cognitive Strategy) factor was significantly negatively correlated with DQ+ (Developmental quality; assessment of a person’s ability to analyze and synthesize information) r = -.27 (p =.048), showing more use of inappropriate cognitive strategies is indicative of lower organizing skills using cognitive resources.
F3 (Ego Restriction) factor was significantly correlated with number of response r = -.30 (p = .029), Pure C (number of pure color response) r = .35 (p =.009), a (number of flexibility responses) r =- .34, (p =.012), and Dd (unusual detail response) r =- .30, (p = .026). These results show children with high degree of ego restriction are unable to actively express the characteristics of their experience and are likely to have impulsive emotional response.
F4 (Interpersonal functioning) factor was significantly correlated with COP (cooperative movement) r = -.27, (p ≤ .043) and Dd (unusual detail response) r = -.28 (p = .042). This demonstrates that children who show behaviors that are more inappropriate in interpersonal functioning are less able to positively interpret interactions with others and have poor sensitivity, such as difficulty detecting small details. F5 (Frustration Tolerance) factor was significantly correlated with Lambda (crude index of responsiveness) r = .27 (p = .036) and COP (cooperative movement) r = -.27 (p = .044). This shows that children with lower frustration tolerance are less able to utilize various resources in the environment and experience positive interaction with others [88].