The Working Alliance Inventory - Short Revised: Psychometric properties of the youth and therapist form in youth mental health and addiction care

doi:10.21203/rs.3.rs-2462151/v1

Download PDF

Research Article

The Working Alliance Inventory - Short Revised: Psychometric properties of the youth and therapist form in youth mental health and addiction care

https://doi.org/10.21203/rs.3.rs-2462151/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

The therapeutic alliance is considered to play an important role in youth treatment. The commonly used versions of the Working Alliance Inventory (WAI) are based on Bordin’s three-dimensional alliance model. However, previous psychometric studies of the WAI did not find this three-dimensional structure in youth psychotherapy. These earlier findings may indicate different perceptions of the alliance by adolescent versus adult patients, but may also be due to methodological shortcomings. The current study aims to address previous study limitations by evaluating the factor structure of the most recent and refined version of the WAI in youth treatment and using multilevel analysis to address the hierarchical structure of the alliance data. We examined the psychometric properties of the youth (n = 203) and therapist (n = 62) versions of the short revised WAI in youth mental health and addiction care and tested the three multilevel models of alliance at start of treatment and 2-month follow-up. Our results suggests a two-factor model for youth and a three-dimensional model for their therapist at both time points. Since this is the first study that finds a best fit for a two-dimensional construct of alliance in youth, more research is needed to rule out whether the differences in alliance dimensions are due to measurement differences between the WAI-SR for youth and therapists or whether youth and their therapists truly differ in their perceptions of the concept of alliance.

In both adult and youth psychotherapy, the therapeutic alliance is considered to play an important role in therapeutic progress and treatment success. Recent meta-analyses among adults and youths found consistent, albeit moderate, associations between a strong therapeutic alliance and beneficial outcomes in mental health treatment (2–4). In the past decades, researchers have developed and adapted a wide range of instruments to assess the alliance, including versions for patients, therapists, parents and observers. Thus, the Working Alliance Inventory (WAI; Horvath and Greenberg, 1989) is the most extensively used questionnaire to assess therapeutic alliance. Specifically, more than two-thirds of the studies in the meta-analysis by Flückiger et al. (2) on adult psychotherapy used a version or adaptation of the WAI and in both meta-analyses on youth psychotherapy (3, 4) the WAI was also the most frequently used measure.

The WAI and its short forms the WAI-Short (WAI-S) (6) and WAI-Short Revised (WAI-SR) (7) are based on Bordin’s (1) conceptual model of the therapeutic alliance or 'working alliance'. Bordin (1, 8) describes the working alliance as the collaborative relationship between patient and therapist based on purposeful collaboration and an affective bond. Elements of the working alliance are the extent to which the patient-therapist dyad engages in purposeful collaboration, including the explicit negotiation of therapeutic tasks and goals, as well as the development of a sufficiently strong affective bond to engage in the therapeutic work during different treatment phases. In line with Bordin’s conceptualization, the working alliance is operationalized as a three-dimensional construct which incorporates the level of task collaboration (task), the degree of agreement on the treatment goals (goal) and the affective quality of the client-therapist relationship (bond). Both the WAI and WAI-S(R) are based on this three-dimensional structure with items that fall into a total scale and three subscales, i.e. task, goal and bond.

In adult populations the WAI has shown good internal, inter-rater and test-retest reliability indices (9) but empirical evidence for its underlying three-dimensional factor structure has been mixed. Some studies confirmed the 3-factor structure (5–7, 10, 11) whereas other studies demonstrated a 2- or single factor structure (12–15). Moreover, inter-correlations between the three subscales were high, especially the task and goal scale were highly correlated. More moderate correlations between the task and goal scales and improved model fit were found for the WAI-SR, the revised version of the WAI-S with negatively worded items excluded (7).

Distinguishing these different components of alliance has implications for research and practice. Some alliance studies focus primarily on the different components of alliance to determine which component has the highest priority at which stage of treatment (14, 15), while others see the value of alliance primarily as synergy of the different dimensions (5). According to Bordin (1) the demands placed on the different alliance dimensions differ by type of treatment. For example, cognitive behavioral therapy may focus more on collaboration (1, 14, 16–18) while interpersonal and humanistic therapies might place a greater emphasis on the bond (1, 14, 17). Although it is plausible that different types of treatments require different alliance `profiles`, most psychometric studies of the WAI do not differentiate between the specific alliance dimensions, especially in youth psychotherapy.

Although the WAI was originally designed for adult treatment and then adapted for use with children and adolescents (19, 20), its factor structure has not received much attention in the youth alliance literature. More research on this topic is particularly important since youth in psychotherapy may have a different understanding of the therapeutic alliance than adults. Some researchers propose that youth may view the alliance more as an affective instead of a cognitive construct (21). Others suggest that young people do not yet fully possess the cognitive skills needed to reflect on their own behavior and emotions, to formulate long-term treatment goals, to convert these abstract goals into specific tasks and to distinguish between treatment goals and tasks (19, 22, 23). Furthermore, it can be difficult to agree on treatment goals with young people who are often referred to treatment because of concerned parents or significant others, when they themselves may not recognize the symptoms of mental illness (24). Finally, typical characteristics of adolescents' developmental stage, such as the strong aspiration for independence and self-determination, may affect the formation of an affective and collaborative relationship between adolescents and their therapist (25, 26). In sum, due to (developmental) differences youth may experience the therapeutic alliance differently compared to adults which may imply that Bordin’s three-dimensional alliance model and the related factor structure of the WAI do not adequately reflect alliance in youth.

The few studies that have examined the factor structure of versions of the WAI questionnaire in youth have shown inconsistent findings and do not allow definite conclusions about the dimensional structure of the WAI in youth. The original WAI (5), with 36 items for both the patient and therapist version, was first investigated by DiGiuseppe et al. (19) among youth in psychotherapy. Their factor analysis yielded one single alliance factor for youth while for therapists besides the one single alliance factor the three factors for goal, task and bond were found. In addition, Diamond et al.(27) found a single alliance factor structure for both the therapist- and youth-rated WAI among patients in cannabis youth treatment (27).

The 12-item WAI-S (6), a shorter version of the WAI, was investigated in two youth studies showing a single factor solution for the client version in youths receiving treatment for anxiety (28) or depression (29). Although findings from Cirasola et al. (29) also provide some support for a two-factor alliance structure with collaboration (goal and task combined) and bond, the correlation between these two factors was very high (r = 0.91). Regarding the therapist version of the WAI-S, Cirasola et al. (29) demonstrated the best fit for the two-factor model and some evidence for the three-factor model. Unfortunately, factor analytic studies on the WAI-SR, the most refined version of the WAI, are lacking in youth.

Overall, prior studies on the factor structure of the WAI and WAI-S in youth treatment provide most empirical support for the general one-factor alliance model for both the youth and therapist versions. These results may indicate that the alliance structure is different in youth than in adults, but may also be explained by methodological limitations and the type of questionnaire that was investigated. A major limitation of previous studies is the lack of attention to the hierarchical ordering of the data, which may have led to biased parameter estimates, incorrectly and usually overestimated standard errors, and incorrect model fitting (30–34). Therefore, the use of multilevel confirmatory factor analyses (MCFA) to account for hierarchical data would be more appropriate for alliance data than the traditional confirmatory analyses.

Another explanation for previous findings may be the version of the WAI that was investigated. Both the WAI and WAI-S contained negatively worded items that could result in response bias among youth, a method effect found by Cirasola et al. (29). These earlier versions of the WAI may not be able to discern the different dimensions of the alliance in adolescents, whereas the WAI-SR may be more apt to identify the three-dimensional structure of the therapeutic alliance in adolescents.

In sum, therapeutic alliance is considered to play an important role in youth treatment and the emphasis on the three different alliance dimensions may vary by treatment type. To date, most psychometric studies that examined the factor structure of the WAI did not distinguish the three alliance dimensions but provided strongest support for a one-dimensional structure. These findings may indicate that the true alliance structure in youth consists of one general alliance dimension but may also be explained by methodological shortcomings, since most studies did not account for the clustering of the alliance data and examined WAI versions with negatively worded items which may lead to response bias.

In the present study we aim to address previous methodological limitations of psychometric studies on the WAI in youth by testing the factor structure of the latest adapted version of the WAI to avoid possible bias from negative items and by applying multilevel confirmatory factor analysis to account for clustered data. The main objective of the present study is to examine the psychometric properties of the WAI-SR in youth mental health and addiction treatment by (1) investigating whether the three-dimensional (task, goal and bond), or a one- or two-factor model of alliance adequately fits the construct of alliance in youth treatment; (2) examining whether the alliance structure is robust (a) across youths and therapists and (b) across time; and (3) taking into account the nested data structure, in which youth are nested in therapists.

Participants

The setting for this study is the Professional Alliance with Clients in Treatment (PACT) study – a multi-site prospective naturalistic clinical cohort study assessing the effects of therapeutic alliance and client feedback in outpatient youth mental health and addiction care. More details about the PACT study are reported in van Benthem et al. (35). The present study included 203 youth (aged between 13 and 23 years) and 62 therapists with at least one assessment of alliance completed by the youth or therapist at the start of the treatment. The therapists’ caseloads ranged from one till 16 youth (Median = 4, Mode = 2). At two-months follow-up 183 youth were still in treatment and treated by 56 therapists. Demographic information for youth and therapists at first session is displayed in Table 1.

Table 1

Demographic background for the first session WAI-SR youth and therapist samples
	WAI-SR youth sample (n = 203) %/mean(s.d.) median	WAI-SR therapist sample (n = 62) %/mean(s.d.) median
Demographic background
Age (years)	18.2 (2.6) 18.0	38.2 (8.8) 36.5
Male (%)	49.8	21.0
Cultural background Non-Dutch (%)	25.1	10.0

Instruments

Immediately after the first treatment session information on age, gender and demographic background was collected. Therapeutic alliance was assessed, immediately after the first treatment session and at two-months follow-up, with the adolescent and therapist version of the short and revised Working Alliance Inventory (WAI-SR) (Hatcher & Gillaspy, 2009; Dutch translation: WAV-12; Stinckens, Ulburghs & Claes, 2009). The WAI-SR consists of 12 items, reflecting Bordin's three dimensional conceptualization of alliance: Task, Goal and Bond (4 items each). Youths and therapists were required to rate each item on a 5-point Likert scale ranging from 'never' (1) to 'always' (5) with higher scores indicating better quality of the alliance. Data were collected by independent and trained research assistants.

Statistical Analyses

A series of multilevel confirmatory factor analysis (MCFA) was conducted to examine the factor structure of the youths’ and therapists’ ratings of the WAI-SR. Because in our study the youths were nested within therapists, we conducted multilevel CFA to estimate both within- and between therapist variances. In addition, to address the categorical nature of the alliance data, as all items were rated on 5-point Likert scales, the mean and variance adjusted weighted least squares (WLSMV) estimator was used. Three alliance models were tested and compared with each other: (1) one-factor model with all items loading into a general alliance factor; (2) two-factor model with both the task and goal items loading into one, combined collaboration factor and the bond items on the other, second factor; (3) three-factor model with task, goal and bond. To test the robustness of our model, across alliance rater and across time, each MCFA was run separately for youths' and therapists' alliance ratings and for each time point..

We evaluated the model fit of all the MCFA models, i.e. the discrepancy between model predictions and the observed data, by assessing the fit indices: Root Mean Square Error of Approximation (RMSEA) < .08, the Comparative Fit Index (CFI) > .90, the Standardized Root Mean Square Residual (SRMR) < .08. Given that the chi-square fit statistic is likely to be significant with larger sample sizes (38), the ratio of the chi-square statistic to the respective degrees of freedom is used to evaluate model fit, χ²/df less than 3 (39–41).

In order to improve model fit, dual loadings and residual correlation parameter constraints were freed based on the modification index (MI) with a maximum of two modifications at a time. MIs were re-estimated and this process was repeated up to three times or until the MI suggested no further improvements of the model fit. In each sample the whole cycles of modification indices was used. Cronbach’s alpha (.70 to .79 acceptable; .80 to .89 good; .90 to 1.00 excellent) (42) was computed to analyze internal consistency. The degree of nesting due to youths being treated by the same therapist was assessed with intraclass correlations (ICC’s), because in ‘real life’ cross-sectional studies the ICC is no higher than 0.20 (43, 44), we considered ICCs of 0.10–0.20 as moderate and > 0.20 as high. Missing data at first session were assumed to be missing completely at random and were handled using listwise deletion. All cases with missing data at first session were excluded from the analysis, resulting in a loss of 4 and 5 cases in the youth and therapist sample, respectively. Data processing, descriptive analyses, internal consistency and intraclass correlations were conducted with IBM SPSS Statistics for Windows, version 25.0 (IBM Corp., Armonk, N.Y., USA). Multilevel confirmatory factor analysis (MCFA) was conducted in Mplus version 8.7 (45).

Descriptive statistics

Table 2 shows means and standard deviations for each WAI-SR item in both samples at first session (for two-months follow-up, see the supplementary Table 1). Internal consistency of the WAI-SR was overall acceptable to good among youth and therapists (Table 3).

Table 2

Means and standard deviations of the first session WAI-SR items youth and therapist version
		mean(s.d.)		mean(s.d.)
WAI-SR	Youth version		Therapist version
Task
Item 2	What I do in this treatment gives me more insight into my problems	3.4 (1.1)	The youth and I have confidence in the usefulness or our current activities in the treatment	3.6 (0.8)
Item 8	The therapist and I agree about what is important for me to work on	4.2 (0.8)	The youth and I agree about what is important to work on	3.8 (0.7)
Item 10	I think that my contribution to this treatment will help me to achieve the changes I want	3.8 (1.0)	I am confident that the things we do in treatment will help the youth to achieve the changes he/she wanted	3.9 (0.7)
Item 12	I believe that the way we work on the problems is the right way	3.7 (1.0)	The youth believe(s) that the way we work on his/her problems is the right way	3.4 (0.8)
Goal
Item 1	One result of this treatment is that it is clearer for me how I can change	3.3 (1.0)	One result of this treatment is that it is clearer for the youth how he/she can change	3.8 (0.7)
Item 4	The therapist and I work together in determining the treatment goals	4.2 (0.9)	The youth and I worked together to determine treatment goals	4.0 (0.9)
Item 6	The therapist and I work on treatment goals we both agreed upon	4.4 (0.8)	The youth and I work on treatment goals we both agreed upon	4.2 (0.8)
Item 11	The therapist and I have formed a clear understanding of the kind of changes that would be good for me	3.8 (1.0)	The youth and I have formed a good understanding of the kind of changes that would be good for him/her	3.7 (0.8)
Bond
Item 3	I believe the therapist likes me	4.1 (0.9)	I believe that the youth likes me	3.6 (0.8)
Item 5	The therapist and I respect each other	4.7 (0.5)	The youth and I respect each other	4.4 (0.7)
Item 7	I feel appreciated by the therapist	4.3 (0.9)	I appreciate the youth as a person	4.6 (0.6)
Item 9	I feel the therapist cares for me, even if I do things he/she disapproves of	3.6 (1.1)	I respect the youth, even if he/she does things I don’t approve of	4.4 (0.7)

Table 3

Cronbach’s alpha’s of the first session WAI-SR youth and therapist version
	WAI-SR youth version (n = 203)					WAI-SR therapist version (n = 62)
	Task	Goal	Bond	Collaboration	Total	Task	Goal	Bond	Collaboration	Total
Cronbach’s alpha value	.73	.71	.76	.84	.88	.81	.75	.81	.87	.89

Factor structure WAI-SR youth. The fit statistics of the three models tested for the WAI-SR youth at both measurement times are presented in the upper part of Table 4.

First session. As can be seen from Table 4, the one-factor model did not fit well in terms of the RMSEA (0.130 > 0.08) and Chi-square/df (4.4 > 3) criteria, whereas an acceptable fit was suggested by the CFI (0.918 > 0.90) and SRMR (0.079 < 0.08) criteria. Similarly, the two-factor model did not fit well according to the RMSEA and Chi-square/df criteria, while the CFI and SRMR criteria suggested an acceptable fit. The three-factor model had a correlation greater than 1 (r = 1.04) between the scales Task and Goal, an indication that the model does not fit the data in a meaningful way (46). Hence, the three-factor model output was inadmissible and is not reported.

Inspection of the modification indices of the two-factor model showed that the residual correlation between item 1 (Goal factor; “clearer how to change”) and item 2 (Task factor; “insight in problems”) had an MI of 46.09 which was more than 3 times greater than the other MIs. Next, inspecting the modification indices for dual-loadings, we found that allowing item 6 (Goal factor; “treatment goals both agreed upon”) to load on the Bond significantly improves fit MI = 24.3. We therefore decided to modify the two-factor model, first freeing the largest residual correlation between item 1 and item 2 and second to allow item 6 to load on both the Goal and Bond factor. As seen in Table 4, the modified two-factor model had acceptable goodness-of-fit assessed by the CFI (0.966), SRMR (0.055) and the chi-square/df ratio (2.5) only the RMSEA (0.086) suggested a poor fit. To further improve model fit we inspected the MIs for the modified two-factor model and found one suggestion to improve model fit. Allowing item 4 (Goal; ‘work together in determining treatment goals’) to load also on the Bond factor would improve the two-factor modified model’s fit with MI 30.87. We re-ran the modified two-factor model, including one residual correlation and two dual-loadings and found a good goodness-of-fit given the absolute indices CFI (0.979), RMSEA (0.068), SRMR (0.048) and the chi-square/df ratio (2.0) (Table 4). The Bond and Collaboration factors in the modified two-factor model showed a latent variable correlation of r = 0.62. A visual representation of the WAI-SR youth modified two-factor model at first session is shown in figure S1 in the supplementary material.

Two-months. Consistent with the first session WAI-SR models, the one- and two-factor models did not fit well according to the RMSEA (0.116 and 0.111 respectively) and the chi-square/df ratio (3.5 and 3.3) whereas the CFI (0.952 and 0.957) and the SRMR (0.071 and 0.069) suggested an acceptable fit for both models (Table 4). Again, the three-factor model did not fit the data in a meaningful way and is not reported.

Inspection of the MIs of the two-factor model showed that freeing the residual correlation between item 6 (Goal factor; “treatment goals both agreed upon”) and item 10 (Task factor; “work on problems in right way”) would improve model fit. The modified two-factor model had acceptable goodness-of-fit assessed by the CFI (0.965), SRMR (0.066) and the chi-square/df ratio (2.9), but not according to the RMSEA (0.101). To further improve model fit we inspected the MIs for the modified two-factor model and found that freeing the residual correlation between item 2 (Task factor; “insight in problems”) and item 8 (“Task factor;

Table 4

Model Fit for WLSMV estimation in multilevel CFA of first-session and 2-month follow-up WAI-SR youth and therapist sample
WAI-SR sample	Model	χ2	df	CFI	RMSEA	SRMR	χ2 /df
Youth First session
	One-factor	238.205***	54	0.918	0.130	0.079	4.4
	Two-factor	187.738***	53	0.940	0.112	0.068	3.6
	Two-factor - modified with one residual correlation and one dual loading	127.685***	51	0.966	0.086	0.055	2.5
	Two-factor - modified with one residual correlation and two dual loadings	97.583***	50	0.979	0.068	0.048	2.0
2-month follow-up
	One-factor	188.115***	54	0.952	0.116	0.071	3.5
	Two-factor	172.816***	53	0.957	0.111	0.069	3.3
	Two-factor – modified with one residual correlation	149.263***	52	0.965	0.101	0.066	2.9
	Two-factor – modified with two residual correlations	127.242***	51	0.973	0.090	0.060	2.5
Therapist
First session
	One-factor	193.467***	54	0.946	0.113	0.078	3.6
	Two-factor	150.240***	53	0.962	0.095	0.064	2.8
	Three-factor	137.332***	51	0.966	0.091	0.060	2.7
	Three-factor - modified with one dual loading	94.911***	50	0.983	0.067	0.049	1.9
2-month follow-up
	One-factor	242.780***	54	0.952	0.136	0.084	4.5
	Two-factor	240.983***	53	0.952	0.137	0.084	4.6
	Two-factor modified with one residual correlation	200.334***	52	0.962	0.123	0.074	3.6
	Two-factor – modified with two residual correlations	179.707***	51	0.967	0.116	0.067	3.5
	Two-factor – modified with three residual correlations	165.417***	50	0.971	0.111	0.059	3.3
	Three-factor	238.391***	51	0.952	0.140	0.082	4.7
	Three-factor – modified with one residual correlation	192.711***	50	0.964	0.123	0.071	3.9
	Three-factor – modified with one residual correlation and one dual loading	170.982***	49	0.969	0.115	0.063	3.5
	Three-factor – modified with two residual correlations and one dual loading	153.530***	48	0.973	0.108	0.062	3.2

“agree what is important”) would improve model fit. The two-factor model, modified with two residual correlations, had acceptable goodness-of-fit assessed by the CFI (0.973), SRMR (0.060) and the chi-square/df ratio (2.5), but the RMSEA (0.090) suggested a poor fit. The Bond and Collaboration factors in the modified two-factor model did not show a very large correlation, with a latent variable correlation of r = 0.61. A visual representation of the modified two-factor model at month two is shown in figure S2 in the supplementary material.

Factor structure WAI-SR therapist. The fit statistics of the three models tested for the WAI-SR therapist are presented in the lower part of Table 4.

First session. As can be seen from Table 4, all three models did not fit well. Based on the MI’s we allowed item 11 to load also on the Task factor (MI = 37.2). The modified three-factor model revealed an acceptable fit according to the CFI, RMSEA, SRMR and the chi-square/df ratio. The latent variable correlation between Task and Goal was r = 0.72 and the correlations between Bond with Task and Goal was r = 0.56 and r = 0.67, respectively. A visual representation of the WAI-SR therapist modified three-factor model at first-session is shown in figure S3 in the supplementary material.

Two-months. All three models did not fit well according to the RMSEA, SRMR and chi-square/df ratio (Table 4). Because the differences between the fit measures of the two-factor and three-factor models were small, we examined the MIs of both models.

Regarding the two-factor model, we modified the model in three steps by freeing one residual correlation in each step (step 1) item 3 with item 6; (step 2) item 1 with item 6; and (step 3) item 1 with item 3. As can be seen in Table 4, the two-factor model modified with three residual correlations had acceptable goodness-of-fit assessed by the CFI and SRMR but a poor fit by the RMSEA and chi-square/df ratio. The latent variable correlation between Bond and Collaboration was moderate (r = 0.67).

Regarding the three-factor model, we modified the model in three steps by freeing in step 1 the residual correlation between item 3 and item 6, in step 2 to allow item 11 to load both on the Goal and Bond factors and in step 3 freeing the residual correlation between item 9 and item 8. The three-factor model, modified with two residual correlations and one dual loading, did not fit well according to the RMSEA and the chi-square/df ratio whereas the CFI and SRMR showed acceptable fit (Table 4). The latent variable correlation between Goal and Task was r = 0.83 and the correlations between Bond with Task and Goal was r = 0.60 and r = 0.68, respectively. A visual representation of the WAI-SR therapist modified three-factor model at month two is shown in figure S4 in the supplementary material.

Nested Data

Our data has a hierarchical structure in which youth are nested in therapists. Specifically, 7% of therapists treated one youth, 47% treated two-four youth and 46% treated five youth or more. The ICCs range was between .25 -and .41 for the WAI-SR youth and between .40 and .67 for the WAI-SR therapist (see supplementary material, Tables 2–6). Hence, for all items the dependency due to therapist was high or very high. Information about the fit statistics of the first session WAI-SR models, without considering nesting within therapists, are displayed in Table 6 in the supplementary material.

In our multilevel modeling we did find lower chi-square values, lower chi-square/df ratio values, lower RMSEA values, and higher CFI values than when the hierarchical data structure was ignored. Although the changes in the fit statistics were in the favorable direction, this did not lead to a different factor structure.

This study partly supports Bordin’s (1) Task-Goal-Bond model of alliance with the treatment of youth. From the therapists' perspective the three-dimensional model seems most appropriate, while for youth a two-dimensional model seems applicable. This is the first psychometric study of the WAI-SR in youth treatment that additionally considers clustering in the alliance data. In addition, the current study is the first to find a best fit for the two-dimensional construct of alliance in youth although a stricter distinction between the task and goal factor seems needed. If replicated in future studies, it could be that youth, like adults, do distinguish between the relational and collaborative aspects of the therapeutic alliance.

WAI

Working Alliance Inventory (WAI)

WAI-S

Working Alliance Inventory Short

WAI-SR

Working Alliance Inventory Short Revised

MCFA

Multilevel Confirmatory Factor Analyses

PACT

Professional Alliance with Clients in Treatment

WLSMV

Weighted Least Squares Mean and Variance adjusted

RMSEA

Root Mean Square Error of Approximation

CFI

Comparative Fit Index

SRMR

Standardized Root Mean Square Residual

Modification Index

ICC

IntraClass Correlations

Ethics approval and consent to participate
The study protocol was reviewed and approved by the Medical Ethical Board of the University Medical Center Leiden (P.15.001). All methods were carried out in accordance with relevant guidelines and regulations. Patients provided written informed consent. If minor, informed consent was obtained from parents/legal guardians.

Consent for publication
Not applicable

Availability of data and materials
The datasets analyzed during the current study are not publicly available due we did not request consent from study participants for using their coded (anonymous) study data for future research purposes but are available from the corresponding author on reasonable request.

Competing interests
The authors declare that they have no competing interests.

Funding
This research was funded by The Netherlands Organization for Health Research and Development (no. 729101014).

Authors' contributions
V.M.H. and R.S. designed the study with contribution of R.R.J.M.V.
V.M.H., R.S., A.L. and R.R.J.M.V supervised the project. P.v.B. collected the data and drafted the manuscript. P.v.B. and R.M.v.d.L. performed the analyses. P.v.B., R.M.v.d.L. and P.B. aided in interpreting the results. All authors discussed the results and commented on the manuscript.

Acknowledgements
The authors would like to thank all youths and therapists for their participation and the research assistants for their help with the data collection.

Bordin ES. The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory research & practice. 1979;16(3):252.
Flückiger C, Del Re AC, Wampold BE, Horvath AO. The alliance in adult psychotherapy: A meta-analytic synthesis. Psychother (Chicago III). 2018;55(4):316–40.
Karver M, De Nadai AS, Monahan M, Shirk SR. Meta-analysis of the prospective relation between alliance and outcome in child and adolescent psychotherapy. Psychother (Chicago III). 2018;55(4):341–55.
Murphy R, Hutton P, Practitioner, Review. Therapist variability, patient-reported therapeutic alliance, and clinical outcomes in adolescents undergoing mental health treatment - a systematic review and meta-analysis. J Child Psychol Psychiatry. 2018;59(1):5–19.
Horvath G. Development and validation of the Working Alliance Inventory. J Couns Psychol. 1989;36(2):223.
Tracey TJ, Kokotovic AM. Factor structure of the working alliance inventory. Psychol Assessment: J consulting Clin Psychol. 1989;1(3):207.
Hatcher RL, Gillaspy JA. Development and validation of a revised short version of the Working Alliance Inventory. Psychother Res. 2006;16(1):12–25.
Bordin ES. Counseling psychology in the year 2000: Prophecy of wish fulfillment? Couns Psychol. 1980;8(4):24–6.
Martin DJ, Garske JP, Davis MK. Relation of the therapeutic alliance with outcome and other variables: a meta-analytic review. J Consult Clin Psychol. 2000;68(3):438.
Busseri MA, Tyler JD. Interchangeability of the working alliance inventory and working alliance inventory, short form. Psychol Assess. 2003;15(2):193.
Munder T, Wilmers F, Leonhart R, Linster HW, Barth J. Working Alliance Inventory-Short Revised (WAI‐SR): psychometric properties in outpatients and inpatients. Clin Psychol Psychotherapy: Int J Theory Pract. 2010;17(3):231–9.
Andrusyna TP, Tang TZ, DeRubeis RJ, Luborsky L. The factor structure of the Working Alliance Inventory in cognitive-behavioral therapy. J Psychother Pract Res. 2001;10(3):173.
Hatcher RL, Barends AW. Patients' view of the alliance in psychotherapy: Exploratory factor analysis of three alliance measures. J Consult Clin Psychol. 1996;64(6):1326.
Webb CA, DeRubeis RJ, Amsterdam JD, Shelton RC, Hollon SD, Dimidjian S. Two aspects of the therapeutic alliance: differential relations with depressive symptom change. J Consult Clin Psychol. 2011;79(3):279.
Falkenström F, Hatcher RL, Holmqvist R. Confirmatory factor analysis of the patient version of the Working Alliance Inventory–Short Form Revised. Assessment. 2015;22(5):581–93.
Castonguay LG, Constantino MJ, Holtforth MG. The working alliance: Where are we and where should we go? Psychotherapy: Theory, Research, Practice, Training. 2006;43(3):271.
Muran JC, Barber JP. The therapeutic alliance: An evidence-based guide to practice. Guilford Press; 2011.
Raue PJ, Goldfried MR. Cognitive-Behavior Therapy. The working alliance: Theory, research, and practice. 1994;173:131.
DiGiuseppe R, Linscott J, Jilton R. Developing the therapeutic alliance in child—adolescent psychotherapy. Appl Prev Psychol. 1996;5(2):85–100.
Figueiredo B, Dias P, Lima VS, Lamela D. Working alliance Inventory for Children and adolescents (WAI-CA).European Journal of Psychological Assessment. 2016.
Ormhaug SM, Shirk S, Wentzel-Larsen T. Therapist and client perspectives on the alliance in the treatment of traumatized adolescents. Eur J Psychotraumatology. 2015;6:27705.
Shirk SR, Karver M. Prediction of treatment outcome from relationship variables in child and adolescent therapy: A meta-analytic review. J Consult Clin Psychol. 2003;71(3):452–64.
Zack SE, Castonguay LG, Boswell JF. Youth working alliance: A core clinical construct in need of empirical maturity. Harv Rev Psychiatry. 2007;15(6):278–88.
Gulliver A, Griffiths KM, Christensen H. Perceived barriers and facilitators to mental health help-seeking in young people: a systematic review. BMC Psychiatry. 2010;10(1):1–9.
Block AM, Greeno CG. Examining outpatient treatment dropout in adolescents: A literature review. Child Adolesc Soc Work J. 2011;28(5):393–420.
De Haan AM, Boon AE, de Jong JT, Hoeve M, Vermeiren RR. A meta-analytic review on treatment dropout in child and adolescent outpatient mental health care. Clin Psychol Rev. 2013;33(5):698–711.
Diamond G, Liddle HA, Wintersteen MB, Dennis ML, Godley SH, Tims F. Early therapeutic alliance as a predictor of treatment outcome for adolescent cannabis users in outpatient treatment. Am J Addictions. 2006;15(Suppl 1):26–33.
Anderson RE, Spence SH, Donovan CL, March S, Prosser S, Kenardy J. Working alliance in online cognitive behavior therapy for anxiety disorders in youth: comparison with clinic delivery and its role in predicting outcome. J Med Internet Res. 2012;14(3):e88.
Cirasola A, Midgley N, Fonagy P, Consortium I, Martin P. The factor structure of the Working Alliance Inventory short-form in youth psychotherapy: An empirical investigation. Psychother Res. 2021;31(4):535–47.
Julian MW. The consequences of ignoring multilevel data structures in nonhierarchical covariance modeling. Struct Equ Model. 2001;8(3):325–52.
Kaplan D, Elliott PR. A didactic example of multilevel structural equation modeling applicable to the study of organizations. Struct Equation Modeling: Multidisciplinary J. 1997;4(1):1–24.
Muthen BO, Satorra A. Complex sample data in structural equation modeling.Sociological methodology. 1995:267–316.
Pornprasertmanit S, Lee J, Preacher KJ. Ignoring clustering in confirmatory factor analysis: Some consequences for model fit and standardized parameter estimates. Multivar Behav Res. 2014;49(6):518–43.
Hox JJ, Moerbeek M, Van de Schoot R. Multilevel analysis: Techniques and applications:Routledge; 2017.
van Benthem P, Spijkerman R, Blanken P, Kleinjan M, Vermeiren RRJM, Hendriks VM. A dual perspective on first-session therapeutic alliance: strong predictor of youth mental health and addiction treatment outcome. Eur Child Adolesc Psychiatry. 2020;29(11):1593–601.
Vertommen H, Vervaeke GAC. Werkalliantievragenlijst (wav). Vertaling voor experimenteel gebruik van de WAI (Horvart & Greenberg, 1986) Departement Psychologie KU Leuven. 1990.
Stinckens N, Ulburghs A, Claes L. De werkalliantie als sleutelelement in het therapiegebeuren. Meting met behulp van de WAV-12: de Nederlandse vertaling van de Working Alliance. Inventory Tijdschr Klin Psychol. 2009;39:44–60.
Alavi M, Visentin DC, Thapa DK, Hunt GE, Watson R, Cleary ML. Chi-square for model fit in confirmatory factor analysis. 2020.
Brown TA. Confirmatory factor analysis for applied research. Guilford publications; 2015.
Kline RB. Principles and practice of structural equation modeling. Guilford publications; 2015.
Browne MW, Cudeck R. Alternative ways of assessing model fit. Sociol methods Res. 1992;21(2):230–58.
Gliem JA, Gliem RR, editors., editors. Calculating, interpreting, and reporting Cronbach’s alpha reliability coefficient for Likert-type scales2003: Midwest Research-to-Practice Conference in Adult, Continuing, and Community &#8230
Musca SC, Kamiejski R, Nugier A, Méot A, Er-Rafiy A, Brauer M. Data with hierarchical structure: impact of intraclass correlation and sample size on type-I error. Front Psychol. 2011;2:74.
Twisk JW. Applied multilevel analysis: a practical guide for medical researchers. Cambridge university press; 2006.
Muthén LK, Muthén B. Mplus user's guide: Statistical analysis with latent variables, user's guide. Muthén & Muthén; 2017.
Kenny DA. Measuring model fit. 2015.
Ross EC, Polaschek DL, Wilson M. Shifting perspectives: a confirmatory factor analysis of the working alliance inventory (short form) with high-risk violent offenders. Int J Offender Ther Comp Criminol. 2011;55(8):1308–23.
Hatcher RL, Lindqvist K, Falkenström F. Psychometric evaluation of the Working Alliance Inventory—Therapist version: Current and new short forms. Psychother Res. 2020;30(6):706–17.
van Benthem P, Spijkerman R, Blanken P, Kleinjan M, Vermeiren RR, Hendriks VM. A dual perspective on first-session therapeutic alliance: strong predictor of youth mental health and addiction treatment outcome. Eur Child Adolesc Psychiatry. 2020;29(11):1593–601.

No competing interests reported.

Download PDF

Editorial decision: Major revision
02 Apr, 2023
Reviews received at journal
29 Mar, 2023
Reviews received at journal
09 Feb, 2023
Reviews received at journal
28 Jan, 2023
Reviewers agreed at journal
19 Jan, 2023
Reviewers agreed at journal
17 Jan, 2023
Reviewers invited by journal
17 Jan, 2023
Editor assigned by journal
16 Jan, 2023
Editor invited by journal
16 Jan, 2023
Submission checks completed at journal
16 Jan, 2023
First submitted to journal
10 Jan, 2023

You are reading this latest preprint version

The Working Alliance Inventory - Short Revised: Psychometric properties of the youth and therapist form in youth mental health and addiction care

Status:

Version 1

Abstract

Background

Methods

Participants

Instruments

Statistical Analyses

Results

Descriptive statistics

Nested Data

Conclusions

Abbreviations

Declarations

References

Additional Declarations

Supplementary Files

Status:

Version 1