The role of working alliance in ACT teams: the building blocks retrieved

Background. In this study, we aimed to retrieve and operationalize the building blocks of the working alliance in multidisciplinary teams with shared caseloads, for use in daily practice and research to support (Flexible) Assertive Community Treatment teams. Methods. After reviewing literature, concept mapping with professionals and clients was used to dene working alliance in (F)ACT. The resulting concept maps formed the basis for the instrument. The instrument was pilot tested with professionals and clients by means of cognitive interviews with a think-aloud procedure. Results. The study led to development of a twenty-ve item instrument to assess Working Alliance in Multidisciplinary teams (WAM), comprised of three subscales: bond, task/goal and team, with a version for clients and professionals. Conclusions. The WAM was developed to determine the quality of the working alliance in ACT teams. Future research will focus on testing its psychometric properties and predictive value.


Introduction
In the last decades, the care for clients with severe mental illness (SMI) has increasingly moved from the institution to the community. Several interventions were developed to manage the care for clients within the community, among which Assertive Community Treatment (ACT) and Flexible Assertive Community Treatment 1,2 (FACT). Meta-analyses have shown that this type of community-based treatment programs is effective in symptomatic remission for homeless persons with SMI and improves psychosocial functioning [3][4][5][6][7] . Despite decades of research into the working alliance, it is unknown to what degree and in which ways working in multidisciplinary teams with shared caseloads affects the working alliance and subsequently, the outcomes.

Working alliance research
What is clear, is that the relationship therapists form with their clients has relevance for the effectiveness of treatment interventions. Meta-analyses show a consistent relationship between the quality of working alliance and the effect of psychotherapy on symptom severity [8][9][10][11][12][13][14][15][16][17] .
For clients with SMI, a strong working alliance with their case manager or individual caregiver was also linked to several positive outcomes, including greater treatment adherence, decrease in symptom severity, improved global functioning, greater quality of life and reduction in problem behaviours [18][19][20][21][22] . Working alliance in the ACT/FACT program The primary function of community-based treatment programs such as ACT and FACT is that they offer outpatient care by a multidisciplinary team with a caseload shared among its members. In multidisciplinary ways of working, the working alliance becomes more complex and concerns several single or group relations. The effects of these differences with traditional therapist-client relations were hypothesized upon by a number of authors. First, in (F)ACT the relationship between client and team might take the form of a more primary relationship, functioning as a hybrid of informal and formal functions 23 . Second, the community care might involve a wider and more concrete array of activities than regular care 24 . Third, there can be more disagreement and con ict in the relationship because the team serves both social control functions as well as therapeutic and supportive functions 25 . Finally, a characteristic of shared caseloads can be that the emotional and practical burden of treating clients with SMI can be shared with other caregivers 26 .
Several studies indicated that working alliance in these teams measured with traditional instruments show no or a small effect on treatment outcome [27][28][29][30][31][32] . The small results that have been found have not been replicated and meta-analyses have not been conducted.
It is unclear why ndings from studies involving clients and individual therapists do not appear to replicate to clients and their multidisciplinary teams. Some authors hypothesize that this is a function of the instruments used. Based on this hypothesis, Klinkenberg and colleagues decided to supplement the way they measured the working alliance for clients with SMI in a case manager relationship and added items such as the number of case manager contacts and several client variables to the often-used Working Alliance Inventory 31,33 . However, they still did not nd a relationship between the strength of the alliance and treatment outcome.
One problem in measuring the in uence of shared caseloads on the working alliance is the absence of an instrument tailored speci cally for use in multidisciplinary teams with shared caseloads. Therefore, in the current study we aimed to retrieve and operationalize the building blocks of the working alliance in multidisciplinary teams with shared caseloads, for use in daily practice and research to support (Flexible) Assertive Community Treatment teams. This led to the development of a measurement instrument.

Methods
The instrument was divided in four stages. In stage 1, a literature search took place. In stage 2, two concept map sessions were organised, on the basis of which a rst draft of the instrument was constructed in stage 3. In stage 4, pilot testing took place and the instrument was nalized. Ethical approval was granted by the research ethics committee of GGzE called the 'Wetenschapscommissie' and registered under the code IMBB/2017022.

Stage 1: Literature search
The literature was searched for working alliance instruments and the relationship between working alliance and treatment outcomes for clients with SMI in (F)ACT. The literature was organised by topic, target population and method of analysis, an overview of which can be obtained through the rst author.

Stage 2: Concept mapping
Two concept map sessions were organised to capture the essence of what constitutes the working alliance between clients and (F)ACT. Concept mapping is a mixed qualitative-quantitative participatory approach which results in a graphical representation of the concepts, the way these concepts are organised and their relative importance 34 . Concept mapping has been used in FACT research previously to measure quality indicators 35,36 .
The rst concept map session was conducted with team members of (F)ACT and experts in the eld and the second session was conducted with clients and former clients of (F)ACT and peer support workers who in the past received care from one of these teams. All participants were contacted in several ways: yers and information letters were made and distributed in the teams and clients were recruited via the professionals using client information letters. The lead researcher informed professionals of the study through oral presentations in team meetings. The experts from the eld were approached through email.
Information about the study was also provided to a client coordination point which clients, former clients and peer support workers attend, and was posted on a website which promotes research in the mental health care institution (www.ggzei.nl). Two experts and thirteen professionals who work for the mental health care institution where the study took place (GGzE) participated in the rst concept map session. Five current clients, one client in training to become a peer support worker and one former client participated in the second concept map session. Characteristics of both groups show diversity in gender, age, level of education and years of experience in or with outpatient, multidisciplinary teams with shared caseloads ( Table 1).
The sessions started with all participants lling in informed consent forms and providing demographic information about themselves. The sessions followed a written scenario, which can be obtained through the rst author, based on literature regarding concept mapping 34,37−39 . The goal of the study was explained, after which the generation of statements started.
After one hour, the participants were asked to sort the statements into categories and to rate them on 'importance' on a ve-point Likert scale, from not very important to very important. Ariadne was used to analyse the data 40 , rst by computing a binary symmetric similarity matrix per participant. Subsequently, the software calculates the similarity between any two statements in the same pile at 1, resulting in an aggregated group matrix. A high value in the group matrix indicates that many participants grouped those statements and implies that the statements are somehow similar conceptually. The aggregated similarity matrix was used as input for a principal component analysis. This translated the correlations between statements into coordinates in a multidimensional space. Subsequently, cluster analysis was used with the coordinates to further classify the statements and grouping statements that were similar into clusters 39,40 . During the nal phase of the concept mapping, the participants were involved in interpreting the concept maps.
The whole procedure took about 4 hours, which the professionals were allowed to register as work time, and all travel expenses were covered in both groups. All participants received a box of chocolates after the session.

Stage 3: Construction of the instrument
The two concept maps formed the basis for the construction of the instrument. The rst step in constructing the instrument was the identi cation of larger domains in both concept maps and integrating the concepts. The interpretation of the participants was used, and further built upon by the researchers.
The second step was searching for the speci c statements within the domains which were given highest priority by both clients and professionals. An overview was made of the domains, with all underlying concepts from each of the concept maps and with the matching statements, categorised by their priority score. When generating items for the instrument, a differentiation was made between statements from the concept map sessions scoring 3.5 to 4, 4 to 4.5 and 4.5 to 5 on priority. The statements which were scored 4.5 or higher were all included in the preliminary item list. The resulting statements were rewritten into items for the instrument.
The third step was the construction of the instrument. Visual Analogue Scales (VAS) were chosen as response scale, ranging from totally disagree to totally agree and very unimportant to very important. VAS scales were used because of their relative ease to use and their sensitivity to small differences in scores [41][42][43] . Participants were asked to rate degree of agreement as well as degree of importance for each item.
All items were formulated for both the client version and the professional version of the instrument. An introduction to the questionnaire was written and an open question was added which asked the participant if the questionnaire lacked important issues. The items were randomised to prevent items from the same domain being scored in a row and in uencing scores because of priming or bias.
In the fourth and last step, the instrument was peer reviewed by the second and third author and a registered nurse who was not involved in this study.

Stage 4: Pilot testing
The clients and professionals that participated in the concept map sessions and clients who volunteered for the concept map sessions but couldn't attend were contacted through email and asked if they wanted to participate in testing the instrument. Two gift certi cates (€10) were ra ed among the participants (one in each group). Participants differed in gender, age, level of education and years of experience in or with (F)ACT (Table 2). Participants were interviewed following a think-aloud procedure in which they were asked to state all their thoughts while reading and responding to the instrument 44,45 .
The written scenario for the interviews can be obtained through the rst author. Participants were asked about several topics, among which: the introduction, the separate items, the scoring of the items and the use of the VAS rating scale used in the instrument. All interviews were recorded and additional notes were made by the researcher. All feedback from the interviews was summarized per item and items were adjusted according to the feedback and in consultation with the second author.  (2) Mental health care psychologist (1) Casemanager (1) Social psychiatric nurse (2) Mean

Stage 1: Literature search
The literature study showed that most working alliance instruments differentiate between the alliance from the view of the client and from the view of the caregiver 46 . The experience of the relationship by the professional can differ from that of the client, which makes it fruitful to design two versions of the instrument, one for professionals and one for clients 23,47 .
The hypothesis that rose from the literature search was that clients of outpatient, multidisciplinary teams with shared caseloads value certain aspects of the working alliance more than other aspects, and that this difference is personal and matters. The rst draft of the instrument therefore focused both, on the degree to which professionals and clients agree with the statements, and on their importance rates per item of the instrument.
Stage 2: Concept mapping Figure 1 and 2 display both concept maps, based on respectively ninety-eight and sixty-one statements generated by professionals and experts and (former) clients. The thickness of the lines surrounding the concepts indicate the mean importance rates given during the sessions. The concepts were subsequently numbered on descending order, with 1 indicating the most important concept.
In the concept map made with the professionals, the item 'acknowledging the person as a whole' was found to be the single most important item. Its singularity shows that the participants did notstructurally -link this item to other items and found it to measure a single concept. The concept map made with the (former) clients in general shows more diversity in the items underlying the concepts, which made it more di cult to capture the concepts in one single word (Fig. 2). Clients and professionals both found the statements regarding treating clients humanely the most important.
In the concept map made with the professionals, the concepts move from team-based to more individually-based on the horizontal axis. On the left, concepts arise that comprise aspects of team functioning such as team culture, exibility and availability. Some of these concepts are tightly connected to the shared caseloads principle. On the right side of the concept map characteristics of the individual caregiver are displayed, varying from attitude to skills and professionalism. The vertical axis shows a continuum from the person of the caregiver to the professional role of the caregiver in both concept maps. On the top half, the concepts re ect personal characteristics, such as showing interest, and on the bottom half the concepts mirror professional aspects such as professional skills.
Both clients and professionals made an explicit distinction between the professional role of the caregiver and the personal role. Clients, however, did not make the team versus individual distinction in the way that the professionals did; the horizontal axis instead shows a continuum ranging from more formal to more informal rules (Fig. 2). On the formal rules side, there are elements such as shared decision making, involving the clients in treatment and sharing responsibility. On the informal side aspects of the relationship such as humour and sincerity were mentioned.

Stage 3: Construction of the instrument
Combing the clusters of both concept maps led to a number of concepts which are graphed in an overall concept model (Fig. 3). Although team and (F)ACT were two separate concepts in the concept model, closer study of the underlying data showed that both concepts were comprised of relatively few statements. The two separate concepts were therefore combined to one overarching domain ('team/(F)ACT'). Three large domains remained: 'bond', 'task/goal' and 'team/(F)ACT'. Since two of the three domains that remained showed conceptual overlap with the WAI, they were labelled in accordance to the WAI subscale names of bond and task/goal. The bond domain led to the formulation of nine items, and the task/goal and team/(F)ACT domain each resulted in eight items.
The informal test by the second and third author and an objective registered nurse led to small adjustments in items, mostly regarding complex words or sentences which seemed to be ambiguous or unclear. Eventually this led to a rst draft of the instrument, containing twenty-ve items in total.

Stage 4: Pilot testing
As a result of the pilot testing, rst, the layout of the introduction was clari ed, font size was changed to enhance the visual appeal, and boldface was employed to emphasize important elements. Second, the The importance rating was removed from the instrument. Almost all participants showed a tendency to judge the items by their importance in general instead of by their importance for themselves speci cally. Instead, a 25th question was added to the instrument which asks participants to rate the ve items which are most important in their opinion in the working alliance with this speci c caregiver (client version) or this speci c client (professional).

Discussion
This study aimed at retrieving and operationalizing the building blocks of working alliance in ACT teams. Concept map sessions were used to generate concepts and the resulting instrument was pilot tested. This led to the construction of a 25-item instrument to assess Working Alliance in Multidisciplinary teams (WAM).
The currently constructed instrument differs from existing instruments in several ways. The golden standard in working alliance research is the Working Alliance Inventory 33 , differentiating between goal, task and bond dimensions and focusing on the working alliance with one individual caregiver 48 . The current instrument measures dimensions similar to these three and shows overlap in items on the mentioned dimensions, but adds a new dimension speci cally measuring aspects of the team and shared caseloads. Also, the relatively new instrument especially designed for measuring the therapeutic relationship in community mental health care, STAR, does not take into account this team dimension 49 . Instruments designed to measure alliance with a team either measure alliance with clinical or residential care 50 or measure a different concept such as attachment 51 . No speci c instrument was found that measures the working alliance in multidisciplinary teams with shared caseloads 46 . Furthermore, the current instrument integrates the perspectives of both clients and professionals, which differ as is shown in the concept maps. Although both groups agree on the importance of a humane treatment and differentiate between more professional and personal characteristics of the caregiver, professionals add team-based factors explicitly as separate entity of the working alliance in (F)ACT. This makes sense since the added value of a community based treatment partly lies in the fact these teams are able to offer exible and continuous care. Professionals functioning in the team on a daily basis are confronted with these bene ts continuously, while clients are less aware of this organizational condition in their interaction with the team.

Strengths
The fact that the instrument takes into account the perspective of both the client and the professional is an obvious strength. Several studies have shown there is only a low to moderate correlation between the working alliance scores of clients and those of professionals 11,27,28,30,49,52−56 .The current study took both perspectives into account in the construction of the instrument and developed two versions of the instrument, one for the client and one for the professional.
A second strength is that the instrument is based on input from concept map sessions. The path of bottom-up development of the instrument was chosen to ensure an innovative perspective on the working alliance, speci cally for clients of (F)ACT. Almost all studies using existing instruments with small adjustments or additions found no or a small and marginally signi cant relationship between the working alliance and the results of community care. This asks for a new approach, which does not build on existing instruments but starts building from the bottom. That several concepts which are widely used in working alliance research did come up from the concept map sessions shows that a universal perspective on the working alliance is in place, but that this needs to be complemented with aspects concerning the team and the in uence of speci c team factors.

Limitations
Despite these strengths, limitations also need to be considered. First, due to privacy legislation, participants had to be recruited through their caregivers. This led to a convenience sample but also a possible selection bias. However, given that the concept map sessions, among other things, led to overarching concepts de ning the working alliance which for some part can also be found in existing instruments such as the Working Alliance Inventory 33 , some degree of generalizability can be presumed.
Second, as in most qualitative studies, the group of client participants was relatively small. It is possible that the small group sizes in uenced the extent of the brainstorming and led to narrower and fewer concepts than a larger group would have. However, it was noticed during the concept map session with clients and former clients that they found it di cult to voice their opinions freely. Possibly for this speci c client group a group of seven participants was su cient and more would have led to di culties in concentrating and participating. During the cognitive interviewing the same general themes kept emerging, which leads to believe that conducting more interviews would not necessarily have led to different input. Future studies will test the psychometric properties of the instrument in a larger sample, which will strengthen the current qualitative base of the instrument.
A nal general remark is that the cognitive interviewed showed a tendency to score to the 'totally agree' side of the VAS scale. Instruments measuring working alliance often show a ceiling effect 32,53,55 , which could partly be prevented by reversing a number of the items. This could counteract the ceiling effect, but also makes items more di cult to understand. Given the cognitive impairments often present in the group of clients with SMI, creating confusion in the wording of the items does not seem bene cial 57 .
Besides this, VAS scales are better able to detect small differences in scores than Likert scales [41][42][43] .

Conclusions
This study aimed at developing a Working Alliance Instrument that measures speci c characteristics of outpatient, multidisciplinary teams with shared caseloads for clients with SMI. Concept map sessions were used to generate concepts and the resulting instrument was pilot tested. This led to the construction of a 25-item instrument to assess Working Alliance in Multidisciplinary teams (WAM). The WAM is the rst instrument incorporating bond, task, goal and team aspects of the working alliance and can be used in a wide variety of settings where client care is organised through multidisciplinary teams with shared caseloads.
In routine care, caregivers can use the instrument to assess the working alliance with their clients and start a conversation with them about their experiences and what they nd important. It is not unthinkable that caregivers sometimes have different priorities than clients, without them knowing this of each other 58 . When clients and caregivers are more aware of their own perspective and the view of the other on their working alliance, both clients and caregivers can discuss possible pitfalls and ruptures. Also, it may not always be possible to in uence those things the client and / or the caregiver nd most important, but discussing this and discussing the limitations of care can strengthen the alliance.
In research, the instrument can be used to study if a qualitatively good working alliance leads to better results of mental health care by (F)ACT. Better results are for example self-reported and clinical recovery and a higher perceived quality of life, but can also be de ned as lower costs or a better balance in costs and bene ts for both clients and society. Also, if the instrument is able to measure reliable change, studies can focus on the question if the quality of the working alliance can be in uenced by targeting speci c elements of the working alliance with speci c interventions (e.g. supervision, training). The next step for future research is testing the psychometric properties of the instrument, by assessing its reliability, investigating its factor structure and establishing its predictive value and validity. This will lead to a new instrument which can be used to further improve care for clients with severe mental illness. Supplementary Files