The target group for the trial, as defined in the governmental commission of the study, were people in treatment for moderate to severe mental illness in secondary care. The diagnostic screening of participants at inclusion showed that 51% of the participants suffered from severe mental illness (psychosis or bipolar disorder), and 49% fulfilled criteria for moderate mental illness (primarily affective disorders). This indicates that the study population corresponds to the pre-defined target group.
An overview of study population characteristics is provided in table 2.
Table 2. Characteristics of the intervention group at baseline.
-------------------------INSERT TABLE 2-------------------------
The study population was relatively young (x = 35, SD 10.7) and education level was low. Nearly half of the participants had experienced violence, and one third had been involuntarily committed to a psychiatric hospital. Mean of previous years worked in main occupation was 7 (SD 7). The mean rating of health-related quality of life, measured by the EQ-5D visual analogue scale, was 58 (SD 18.3).
Barriers and facilitators to participate in IPS
Table 3 shows results from the barriers and facilitators to participation lists. Open-ended response categories were provided, but they did not generate additional barriers or facilitators.
Table 3. Percentage and number of respondents agreeing to the statements about facilitators and barriers.
--------------------------------------INSERT TABLE 3 HERE------------------------------------
Two of the most frequently cited facilitators among participants regarded the IPS specialist’s role: 94% of respondents agreed with the statement ‘Knowing that the IPS specialist was available for me was helpful’, while 81% agreed with the statement ‘The regular follow-up from the IPS specialist was helpful.’
Responding to a separate item regarding the IPS specialist, 78% reported to be satisfied, while 13% reported to be dissatisfied. 9% reported to be neither dissatisfied nor satisfied (n=78; x = 4 SD 1.11).
All over, participants were very happy with the role of the IPS specialist. Participant interviews gave further insight into the IPS specialist’s role, emphasizing their availability, support, and consistent job focus. When talking about availability, informants emphasized that the IPS specialist was quick to respond and to express their availability:
“She is really good, really efficient. Supportive, calls and asks me to call back, to call on her spare time. If it’s a good time it’s a good time, if it’s not a good time she calls me back up again. It’s been really nice. “
“I think he’s been really available, because even if he doesn’t answer my call immediately, he calls me back up, he is always there for me if anything comes up. So yes…I feel I have received very, very good follow-up from him. So I am very happy.”
Some participants said that the IPS specialist ‘pushed’ them to keep going with the job search, and some had been confronted with their lack of motivation. This resulted in taking a more active role in the job search:
“It wasn’t a threat, but they said we can’t help you if you’re not interested. They deserved an honest answer to that. It was the best question I could have received, instead of them saying ‘We’re not wasting time on this...’ I woke up.”
“It has been positive for me to start working, yes. But I do feel there is a small pressure and that I have to push myself to say yes to working. I am supposed to start working and not sit at home. And I did get a job, so maybe it’s good that they push a little.”
One participant described how being listened to in the process was important:
“If he has come with a job suggestion and I have said that this is not for me, because I will not function well there, he has just put it away immediately, he is very accommodating like that.”
There were a few exceptions to the positive descriptions of the IPS specialist’s role. Two of the informants who were less satisfied with the intervention, described the interaction with the IPS specialist as challenging.
Another facilitator for participation indicated by survey data was the freedom to disclose or not: 93% of respondents agreed that ‘Being able to choose whether or not to be open about my illness’ was helpful. The interviews indicated that for some participants, choosing to disclose expanded the possibilities for practical help in the job search:
“Yes, it’s very nice because the IPS specialist can call around for me, I have anxiety about talking on the phone sometimes. And she is with me in the conversations with employers so that I understand what is being said, and she can also inquire about salary.”
One participant reflected on the positive aspects of disclosing to potential employers:
“It might be that some employers think that they want to support it because it is kind of a good cause to help people get a job that maybe have a history of illness or have had problems, and because of that they can get a job.”
Most of the proposed barriers were not supported by participants. However, the most agreed-upon statement was ‘My illness was a barrier’ (46%, n=95). Considering the target group of the intervention, this number is not particularly high. Informants described illness factors mostly in the context of having the IPS follow-up tailored to their health condition. One in six participants agreed to the statement ‘IPS was not what I expected’ (17%, n=77). Examples of this were found in the interviews. While some participants were positively surprised by the intervention, others described being disappointed due to high expectations.
“I was promised employment within 6 weeks, and now I have waited for 13-14 months (…). I had expectations about follow-up from IPS specialist and close cooperation between my doctor, the District Psychiatric Center, and a permanent position with full salary. And none of it has happened.”
One informant stated that the follow-up was simply different from what he had expected regarding his own involvement:
“Uuuhm, but the only expectation I did have that turned out not to be correct was that I kind of thought they had some sort of obligation to help me find a job so I didn’t have to do such an effort myself. But that was totally wrong. (…) It’s not like I can nag them and say ‘Hey, find me a job’, it’s more like they come alongside and back me up on the things I manage to do.”
Two items measuring satisfaction and perceived usefulness were included as indicators of barriers and facilitators to participation, as low scores on these measures could indicate poor quality in intervention delivery, and/or low engagement with the intervention among participants (table 4). However, participants were overall satisfied with the intervention (n=95; x=3.95 SD 0.97), and also found it useful (n=96; x=3.96 SD 1.06).
--------------------------------------INSERT TABLE 4 HERE------------------------------------
Participant interviews provided further insight about this. Informants said that IPS had made them aware of their own competence and preferences. Most informants reported that IPS had increased the frequency of sending applications, and that they had learned more about going to job interviews. They emphasized that the focus on employment had been central in the follow-up:
“I haven’t got a job offer, but now I apply for jobs in a different way. I have been on many interviews, so that has improved as a result of this follow-up (…). I learned how to write an application, about motivation, qualities…”
The first thing we did was to go over what kind of jobs I wanted, then we got my CV sorted out, how to write an application, and have everything ready for sending the application. (…) And then we went out into the job market. We went step by step, one thing after another, in the right order.
Barriers and facilitators to implement IPS
Barriers and facilitators to implementation were examined using the IPS Fidelity Scale and focus group interviews with IPS teams. Results are presented in table 5. All six centers had reached fair or good fidelity at this point, with scores ranging between 89-108 (median 96), with 125 as the highest attainable score, and 74 as the cut-off score for being IPS. Single items were rated on a range from 1 to 5, where item scores 1-3 indicate no or poor implementation.
Table 5. Center scores on each item, mean scores of centers, mean of lowest and highest performing centres, total fidelity score and median for each centre.
--------------------------------------INSERT TABLE 5 HERE------------------------------------
Items with low scores across centers were ‘Community-based services’ (x = 1) and ‘Job development – frequency’ (x = 2). To receive a top score on ‘Community-based services’, the IPS specialists must spend 65% or more of their time outside the office, following up participants in their local area. This should be seen in relation to the item ‘Job development – frequency’, indicating frequency of contact with employers in order to develop a broad employer network. The IPS specialists reported in the interviews that it had taken quite some time to develop and understand their role, and that prioritizing tasks was demanding, as expressed in the following remarks:
“What we always have to challenge ourselves on is the use of time, considering time spent on internal meetings, participant meetings and employer contact. To obtain the optimal allocation of resources is pretty challenging.”
“Being an IPS specialist is a pretty complex and difficult role, where you are a seller on the one hand, selling the best manpower there is, next you’re a facilitator, an IPS specialist, and you can sometimes have a therapeutic approach at times when the therapist is not there. So, it is a very difficult role, and it takes time to be secure in it.”
Looking at the two items measuring ‘Integration with health services’, which is an important IPS principle, their mean score across centers is 3. According to the quality thresholds defined by the program developers, these two items barely pass the ‘Fair fidelity’ threshold . These items measure whether the IPS specialists are integrated in participant’s health treatment team, attend weekly meetings, and ensures focus on employment in coordinating services for the participant.
On the positive side, all six centers received top scores on the items ‘Caseload size’ and ‘Employer diversity’, and nearly top score on “Disclosure of disability to employers’. ‘Caseload size’ means that the caseload for each IPS specialist does not exceed 20 participants, in order to ensure close follow-up in all phases of the job search. ‘Employer diversity’ refers to the diversity of workplaces where participants get jobs. It is used as an indicator of whether IPS specialists are following participants’ own preferences, and not only working within the limits of their existing employer network. Disclosure measures whether IPS specialists provide information to participants about pros and cons of disclosing about their illness to an employer.
The effect evaluation showed that three of the six pilot centers performed particularly well on employment outcomes , though there were no obvious reasons for this. The top three centers did, however, differ from the rest on two particular fidelity items. They averaged 1.2 points above the average of the less performing centers on the item ‘Integration of IPS with treatment team’, which is considered a crucial intervention component . The challenges related to this topic was addressed frequently in the interviews, and are illustrated by the following remark:
“I think that it is the greatest success and the greatest challenge, that integration process (…), how we feel that they [treatment team] talk and feel concerning work.”
However, the most striking difference was found between scores on ‘Job development – quality’ (indicating quality of employer contact), where the top performing centers averaged 2.7 points above the average of the less performing centers. The issue of employer contact was addressed in the interviews, as exemplified in the following remarks:
“…the job development part, that’s something that for most of us, and definitely
for me, has been new and different, going out and being assertive both on the phone and in person (…) After a while I realized it’s been written in the manual the whole time, that we really need to have our main focus on job development. We’ve made some changes now this fall where we have set targets and try to reserve days and times to do that.”