Making implementation programmes better. Mixed-m ethods case study of an implementation process for two evidence-based brief psychotherapies


 Background

The Ostrobothnia Depression Programme (ODP) had a hybrid-design incorporating an implementation programme for two evidence-based treatments (EBTs), behavioural activation and motivational interviewing, and a study on their effectiveness. It was carried out regionally in secondary psychiatric care in a Finnish district. We evaluated the ODP through a mixed-methods analysis to extract organization- and programme-related knowledge to make future programmes more effective in terms of sustaining and scaling up the desired programme outcomes.
Methods

A cross-sectional mixed-methods survey was conducted with the programme addressees 4-5 months after the end of the ODP. The realization of case consultation groups analysed further by interviewing those responsible for them and making use of attendance lists.
Results

Out of 72 original programme addressees 33 completed the survey. The results showed that the ODP succeeded in initiating the desired change in clinical practices. Case consultations and training videos intended as reinforcers were underutilized. Deficits in the implementation plan and the hybrid design of the ODP jeopardized the maintenance of the implementation outcomes in the long term.
Conclusions

We discuss our results in light of the Normalization Process Theory (NPT) related sub-processes ‘implementation’, ‘embedding’ and ‘integration’. The complete training intervention in the target EBTs should comprise both workshops and non-optional case consultations. Access to case consultations should be made as convenient as possible. Means to decentralize the clinical support in everyday work should be elaborated in collaboration with the teams. Coaching team leaders to employ evidence-based active managerial practices that are connected e.g. to the concept of ‘transformational leadership’ would likely be fruitful. The original programme plan should be checked for means to sustain and scale up the implementation outcomes after the active programme phase.

We recommend merging workshops and case consultations as an integral training package since workshops only seem to be insu cient.
We recommend organizing low threshold clinical support locally available during everyday work, e.g. by nominating peer facilitators.
We recommend creating permanent organizational structures and policies, which enable new coming staff to be trained in the EBTs settled into the organization's permanent services. This to combat the detrimental effect of staff turnover.

Background
The power of transferring an evidence-based treatment (EBT) into routine patient care is a fundamental aspect of the success of an EBT implementation programme. Deploying means that ensure the long-term survival of the EBT is crucial right from the inception of programme design (1,2). The process for routinizing an EBT entails accomplishing the desired change in the treatment practices at organizational level, acquiring adequate skills in delivering the EBT at the individual level as well as sustaining and scaling up the implementation outcomes after an active programme phase (3,4).
Normalization Process Theory (NPT) introduces three different sub-processes to achieve and establish successful implementation programme outcomes: implementation, embedding and integration (1,5). In the context of psychiatric care, the sub-process of implementation includes e.g. the initial training in an EBT, embedding at least case consultations and other clinical support practices and the sub-process of integration includes all managerial processes that enable the staff to maintain the delivery of the EBT in their organization. High-quality training intervention in an EBT, augmented by case consultations and separate strategies for integrating the EBT into the organizational structures have been shown to have a signi cant impact on their long-term survival (1,4,(6)(7)(8)(9). A comprehensive implementation or programme plan comprises operative strategies from all three NPT sub-processes. All different activities can be put into operation simultaneously or consecutively regardless of which sub-process they are targeted at, for example, it may be appropriate to initiate the training of the frontline staff (an implementation strategy) and restructuring of the organization (an integration strategy) in tandem.
The attitudes and actions of management are considered the most decisive factors in promoting innovations in health care service organizations, thus paying special attention to them is crucial for the success of implementation programmes (10,11). For individual professionals, the readiness to adopt an innovation is in uenced by the attitudes and actions of peers (12). Furthermore, multiple identi ed factors related to the organization, individuals and the programme itself affect professionals' attitudes toward the innovation and their ultimate actions in adopting it (6, 7, 13-16).
In EBT implementation programmes for psychotherapy interventions, 'high-quality training intervention' implies the application of diverse training modalities, such as expert-led lectures accompanied by interactive and supervised simulations as well as self-study materials (4,6,17). Case consultations subsequent to brief workshops are a prerequisite for acquiring and sustaining adequate skills in the target EBTs (4,9). Case consultations individually or in groups have proven equally e cacious, group mode being more cost-effective (18). The ingredients of both training and case consultations varied across studies with regard to the extent and mode of delivery.
The regional Ostrobothnia Depression Programme (ODP) in Finland comprised simultaneously running implementation programmes for two EBTs and their effectiveness study, thus having an effectivenessimplementation hybrid design (19). The target EBTs were behavioural activation (BA) and motivational interviewing (MI) (20,21) and the programme addressees were therapists providing secondary psychiatric care. A previous longitudinal summative evaluation of the ODP showed that one third of the therapists could be regarded as active adopters a few months after the active programme phase, which was less than could have been expected (22). Thus, the implementation and embedding strategies were regarded as su cient to start the delivery of the target EBTs. However, the same summative evaluation revealed a lack of integration strategies, which jeopardize the sustainability and scaling up of the implementation outcomes in the long term. The ODP was concluded to represent a feasible base for future programmes but requiring some complementary enhancements.
With the present mixed-methods case study we sought to explain the ndings of the previous summative evaluation of the ODP. Our aim was to extract practical knowledge on how to ensure the sustainment and reproduction of the implementation outcomes after the end of the active programme phase.

Context
South Ostrobothnia Hospital District in Finland provides public specialized health care services to a population of 200,000. The Adult Psychiatry Department comprises twelve outpatient units and ve inpatient wards. The ODP was carried out for improving the treatment of depression with possible co-morbidities at six units, of which ve were outpatient clinics and one an 18-bed acute ward. The largest unit had two separate teams, and the rest one team each.

The ODP
The ODP was launched primarily to tackle several clinical challenges by increasing the use of EBTs. The challenges identi ed were heterogeneous treatment practices and congested patient ow as a result especially of a marked increase in depressive patients referred to specialized care and de cits in providing integrative treatment for patients with dual diagnoses, i.e. psychiatric patients with comorbid substance abuse. The design and execution of the implementation programme drew on the launchers' expertise in administration, clinical work and training of health care professionals. No explicit theoretical model or framework for implementation was used. The effectiveness study was designed and performed jointly with the implementation programme. The study protocol of the effectiveness study has been described in detail in the appropriate register (23).
The primary goal of the ODP was the implementation of an advanced systematic and regional model to address the treatment needs of non-psychotic patients. The patients treated for major depression and possible co-morbidities were used as a benchmarking group. This patient group is relatively large in psychiatric care, which makes it reasonable to focus on it. The aim was to reduce the negative gap between the resources and the increasing demands for treatment, and thus make the patient ow more uent (24,25).
A more detailed description of the ODP has been published previously (22).
The implementation plan comprised protocols for training and other programme supports aligned with earlier high-quality training programmes (6). It included workshops with both active and passive training modalities, case consultation groups, written and videotaped self-study material and regular research nurses' visits to the units involved. A more detailed description and analysis of the plan and the extent of the sustained use of the two EBTs after the active programme phase as well as associated therapist-and intervention-related factors have been reported elsewhere (22).

Setting and sample
A purposeful sampling strategy 'complete target population' was used to build the study sample (26). The total number of therapists regularly employed in the units involved in ODP was 72 and they comprised the target group. Enrolment in the training as well as responding the survey was voluntary for them. The study sample of present cross-sectional study comprised 33 ODP trained therapists who were still employed by the target units and were willing to respond to the survey. All members of the sample gave verbal informed consent to participate, see the section "Ethics review and consent to participate" for more detail. The study sample of 33 amounted to 46% of the original target group.
The rst author of this article (LHL) was the principal researcher and he gathered the data in March 2014, 4-5 months after the completion of the ODP. A survey was administered in each unit during their regular weekly meeting. In addition, LHL conducted a short semi-structured interview by telephone with the two programme executives responsible for the case consultations and information on the number of participants per session was collected from the list of participants. All authors but the second (JK) of this article were employed in the ODP managing organization. They and all participants knew each other prior to the study as well as the participants were aware about the authors' interests in terms of the study.

Instruments
The instruments were speci cally designed for the present study. A mixed-methods approach and method triangulation were used to collect and analyse the data. Quantitative data was predominant in the study, determining the magnitude and direction of the results, while the simultaneously and subsequently collected qualitative data was intended to help in explaining the quantitative results (i.e. complementary design, QUAN + qual) (27).
Four different types of measures suitable for each target were used in the questionnaire. These included the Visual Analogue Scale (VAS), school grade rating (SGR, 10 equals excellent, 9 very good, 8 good, 7 satisfactory, 6 moderate, 5 adequate and 4 fail), multiple choice and open-ended questions. The VAS with a continuum of 0-100 points is widely used for measuring different individual attitudes or perceptions (28). In addition, a focused brief semi-structured interview was conducted. The original instruments were administered in Finnish, the therapists' native language. In reporting the present study, we have adhered to the criteria of Good Reporting of a Mixed Methods Study (GRAMMS) (29) and Consolidated Criteria for Reporting Qualitative Studies (COREQ; Additional le 1) (30).

Quantitative assessment
Six clinical dimensions based on the core characteristics of BA and MI (20,21), were determined as the main clinical goals, and the therapists' perceptions of progress in each goal were assessed on the VAS scale. The therapists' overall appraisal of whether the ODP yielded a positive impact on the quality of their own work was examined with SGR (Impact-SGR). The therapists' overall perceptions of the degree of change in clinical practices that the ODP yielded at team level were also examined on the VAS scale (Change index). The therapists were asked to give SGR to the ODP training intervention. They were also asked to indicate whether they had watched the training videos and attended the case consultations, and in case of "yes" then to give the SGR for that item. Therapists' perceptions of the manifestation of four possible obstacles were examined on the VAS. Furthermore, six team-related factors, which in general may either promote or hamper the progress of a programme, were introduced to the therapists (Table 1), and they were asked to indicate the direction of each factor regarding the ODP. An option left blank was regarded as neutral.

Qualitative assessment
The rst survey question elicited therapists' general perceptions of the goals of the ODP and read: "Name the three most important goals that you perceive the ODP was intended for." Two open-ended questions were used to collect negative and positive feedback -'Censure and Praise' -on the ODP: a) "Name two major issues which should have been done in some other way during the ODP", and b) "Name two major issues which succeeded particularly well in executing the ODP".
The trainer-consultant and research nurse who were responsible for the case consultation groups were interviewed retrospectively using a semi-structured protocol to enhance the information on participation activity in the case consultations.

Analysis methods
The quantitative and qualitative items were analysed according to the respective methods as described in next chapters.

Analysis of quantitative variables
Frequencies were calculated for the total number of responses and multiple choices. Means and standard deviations (SD) were calculated for VASs and SGRs. Cronbach's alpha was calculated to test the reliability of the six-item set for the main clinical goals.
Spearman's correlations were calculated between the following means: Impact-SGR, perceived support from team leaders, the Change index and each four possible obstacles. Two hierarchical linear regression models were used to predict the Impact-SGR. The rst model included the support from team leaders and four possible obstacles as explanatory variables. The second model included the explanatory variables of the rst model and the Change index.
Activity rates were calculated for watching videos and attending case consultations.

Qualitative and triangulated analysis
Responses to both open-ended questions -the therapists' perceptions of the three most important goals of the ODP and the Censure and Praise -were both analysed using qualitative content analysis (31). Manifest expressions were objects of interest. The item Censure and Praise was further analysed by a typological method to form the respective model cases (32). The analysis methods are described in more detail in Additional le 2.
A report on the semi-structured interviews with those responsible for the case consultations was written immediately after the interview on the basis of the notes taken and elaborated after receipt of exact information on attendance rates by session. The rst author of this article (LHL) extracted possible explanatory factors pertaining to attendance at the case consultations from both the interview report and the therapists' responses to the Censure and Praise. The protocol and report on the interview in English translation are presented in Additional le 3.

Quantitative data
The therapists' perceptions of progress achieved (mean±SD) in the main clinical goals, each measured by VAS, varied between 58±24 and 71±20 ( Table 2). Addressing substance abuse earlier than before obtained the highest scores, followed by enhancement in the goal setting at the beginning of the therapy. The reliability of this six-item set was 0.827 (Cronbach's alpha).
[insert Table 2 here] The therapists appraised the positive impact of the ODP on their own work with a mean SGR of 6.8 (SD 1.1). Their perceptions of the degree of the Change index, the support from team leaders and four possible obstacles in being engaged in the ODP are presented in Table 3. Of these obstacles, "Personal exhaustion due to a variety of projects" manifested with the lowest scores of 40±30 (mean±SD) and the "Other current team related internal issue" came thereafter with scores of 44±33, meaning that those issues did indeed hamper the execution of the ODP.
[insert Table 3 here] The training programme obtained a mean SGR of 7.5 (SD 1.3, n=33). The training videos obtained a mean SGR of 7.9 (SD 0.64, n=21), and the case consultations groups 7.9 (SD 0.79, n=18). Twelve (36%) of the responding therapists had not watched the videos and 15 (45%) had not attended the case consultation groups.
In the linear regression models explaining the Impact-SGR, the support from team leaders was a signi cant explanatory variable in the rst model and Change index in the second model (the support from team leaders lost its signi cance in the second model) ( Table 4).
[insert Table 4 here] In the question on factors that may enhance or inhibit achieving progress (Figure 1), "opportunity to acquire new skills" was markedly stressed as enhancing and two items indicating support from the peers and the managers were stressed slightly positively. By contrast, the experiences of time resources available for rehearsing alone and together with the team both suggested a de cit, which inhibited progress.
[insert Figure 1  In the qualitative content analysis of the open-ended questions for feedback, the item itself provided the two main categories: Censure (C) and Praise (P). The C-and P-responses were analysed jointly to form the following three data driven subcategories: feedback on training and clinical support, feedback on the treatment model implemented and feedback on the simultaneously conducted implementation programme and clinical research. A few responses could not be classi ed, and were therefore labelled as non-categorized feedback. Each response was allocated to an appropriate subcategory. The data was further analysed by the typological method, which resulted in model cases of C-and P-types for each of the three subcategories. A concise summary of these is presented below. The process of the analysis and the model cases are presented in more detail in Additional le 2. Triangulated analysis on the case consultations Attendance at the case consultation groups was at its highest during the rst year and then declined slightly, until it fell considerably during the last year due to reasons related to both the organization and to individuals.
The following possible explanatory factors were identi ed: variation in geographical distances, differences in attitudes towards the ODP between the units, a change in the pro le of the clientele during the programme and that fact that some therapists became discouraged over time. Two units were located very close to the venue and the rest about an hour away by car, thus reaching the case consultations was quite timeconsuming for clearly more than half of the therapists. Of the two closest units, one achieved the highest attendance rate and the other the lowest rate. In the most active unit, the team leaders actively allocated time to the therapists to enable them to attend. By contrast, in the most inactive unit, the ODP had been subject to considerable doubts from the very beginning. The analysis of the attendance activity to the case consultations is presented in more detail in Additional le 3.

Discussion
We performed a comprehensive analysis of the ODP participating therapists' perceptions of the success with the main clinical goals of applying BA and MI, and practical issues that might potentially enable or inhibit their implementation. The present results complement the earlier summative evaluation (22). The main results of our study were twofold. First, moderate improvement was achieved in clinical goals pertaining to applying the implemented EBTs with improvement of the integrative treatment of dual-diagnosed patients as a In psychiatric care, the adoption of EBTs tends to penetrate the target group less during an active programme phase than the programme launchers might expect (14,17,37,38). Furthermore, without appropriate maintenance strategies their use tends to decline over time. Hence, deploying speci c strategies for integrating the new EBTs into an organization's routine structures and processes is a prerequisite for sustaining implementation outcomes and scaling them up after the active programme phase. (1, 2, 39).
Unfortunately, such strategies were lacking in the ODP implementation plan, as stated in our earlier summative evaluation (22).
Our present results showed that the case consultations and the training videos were both underutilized. Nearly half of the responding therapists had not attended the case consultation groups and about a third had not made use of the training videos. To our knowledge, there is a lack of studies addressing activity in utilizing such training videos in the context of implementation programmes. However, providing self-study material to facilitate rehearsal has been included among the best practices in contributing to the implementation of innovations (40).
The above raises some fundamental questions: How to address the justi ed call for more comprehensive training and clinical support as well as time resources for acquiring the necessary skills in the target EBT? In addition, what could be the appropriate and feasible integration strategies to ensure the long-term survival of the EBT implementation programme outcomes? In fact, these questions cannot be answered separately; they rather form an interconnected network of strategies and actions. We will next articulate some possible views on these critical questions drawing on our present results and earlier research in order to improve the effectiveness of future programmes.

Role of Case Consultations
Case consultations have been shown to be essential in implementing an EBT in terms of penetration and sustained adoption (9,14,35). They make it possible to enhance attendees' knowledge of the EBT introduced and strengthen their skills in applying it in everyday work. Organizing the case consultations in a more accessible way may be one option to satisfy the need for more robust clinical support the therapists called for in the ODP.
An implementation project at the Veterans' Health Administration (VHA) in the USA achieved a rate of 77% in adopting prolonged exposure therapy, an EBT for the treatment of post-traumatic stress disorder (14,35).
Attending the case consultations after the four-day workshop was mandatory, which it was not in the ODP.
Furthermore, the case consultations were carried out by telephone twice a week both in groups and individually and they were supported by audio recordings of the therapy sessions. A minimum of two cases treated with a good level of competency was required before the issue of a certi cate of having completed training. In another study, a cognitive behavioural therapy (CBT) application was implemented among community therapists by means of a one-day workshop followed by optional weekly case consultations for three months (4). The two-year follow-up showed that the time the therapists spent in the consultations during the active programme phase correlated positively with the sustained use of the CBT application (9).
In the ODP we identi ed factors that enabled and inhibited the attendance at the case consultations. The positive attitude of team leaders together with their active allocation of time enabled more frequent attendance at the case consultations. Instead, the considerable time needed to travel to the venue for most of the therapists and, in one unit, a sceptical attitude toward the programme were two obvious factors undermining the attendance.
The evidence reported above supports merging workshops and case consultations into one entity or training programme. The case consultations should not be optional for therapists enrolled on the programme.
However, attending the case consultations should be made convenient and easy. The example of the programme at the VHA encourages deploying remote access, e.g. feasible web applications are available today. Also, positive means of pinpointing an individual trainee's progress in skills would be fruitful. At the VHA, this was sought by the requirement to have completed treatment of two cases with a good level of competency before being issued with a certi cate of completed training.

Team Level Clinical Support and Time Allocation
In the ODP, the therapists called for the clinical support to be decentralized. We regard this as a request to enhance local or team level strategies to promote delivery of the target EBTs. In addition, in the therapists' experience, lack of time hampered the acquisition of the new skills during the everyday work, which is corroborated in other studies (35,36). Programmes come and go, organizations continue. That is why the clinical support practices should be incorporated into the organizations' routine structures (1). Team leaders are a formal and central in uential stakeholder group to collaborate with as early as in the pre-programme phase in order to achieve progress on this issue. Their actions are crucial to the ful lment of organizational strategies, fostering the implementation climate and sustaining implementation outcomes at the team level (10,11). However, it is not self-evident that team leaders, who usually act as both clinicians and leaders, are aware of all the means available to contribute to an implementation process.
Our results pinpoint two possible evidence-based means the team leaders could harness in addressing the need for more robust local clinical support in implementing an EBT: empowering programme champions and allocation of time resources needed to make good use of the programme facilities. In our case, a programme champion refers to a therapist who would be among the rst to take over a new EBT and so be able to provide peer support to colleagues in embracing the EBT (33,41). A part of these early adopters may be keen to acquire more thorough training to be nominated as formal peer facilitators (33), which we recommend taking into account in the original implementation plan. The team leaders could foster local support practices by sponsoring the rise of socially acknowledged peer facilitators and guiding their teams to prioritize the use of the target EBT (41). Managerial actions of this kind have been proven to promote to the penetration of the target EBT and sustain the delivery of it (13,15,35,42). Allocating time for studying the self-study material jointly with the team, e.g. watching the training videos, could be an easily harnessed strategy for the team leaders. This also gives them potential chances to render the climate more favourable towards the target EBT.
Beidas et al. (43) have described an advanced means of concerted behavioural rehearsal should some teams become more inspired by investing greater effort in learning together.

Managerial practices and building organizational structures as integration strategies
The NPT concept 'integration strategies' refers to all deliberately deployed processes intended to support the personnel in maintaining the adoption of the target EBT in their organizations (1). Lack of integration strategies emerged in the longitudinal summative evaluation of the ODP (22). The stability of leadership and the leadership style have been shown to be the main issues having a signi cant impact on the long-term survival of programme outcomes (11,38,44). Aarons et al. (44) stated that frontline transformational leadership predicted sustaining of implementation outcomes while passive-avoidant leadership predicted failure to sustain. In the case of ODP, a part of the therapists reported that their team leaders supported the EBTs' implementation. However, we do not know in detail about this support. Peterson et al. (11) de ned more precisely the relevant tasks the leaders should deploy to ensure the long-term vitality of the outcomes: eld mentoring, group supervision, training outcome monitoring and delity reviews. It would be highly recommended that the original programme plan should involve strategies that coach the leaders to establish those tasks. Proctor et al. (45) have reported a training intervention in implementation practices for leaders, which could provide evidence-based tools being deployed in health care organizations and thus enhance the quality of EBT implementation programmes. This would partly address the challenge of implementing evidence-based implementation strategies (46). Field mentoring and group supervision refer to practices that involve the team leaders spending time with their teams in regular meetings discussing successes and concerns in applying the target EBT and facilitating its delivery (45). These enable the leaders to be aware of the vitality of the target EBT and the level of competence in it among their teams. Awareness of this information is a prerequisite for being able to plan appropriate actions to further promote the implementation. These actions include not only addressing the need for complementary training after the active phase of a programme but need also for stable mechanisms that make it possible to guide newcomers to get training in the basic skills of a target EBT (45). Our results revealed a de cit in building permanent structures for addressing the continued need for training in the target EBTs. As a part of an EBT implementation programme, it should be checked that there are available permanent organizational structures and policies that enable the long-term maintenance of the EBT. If lacking, those should be built up.

Limitations
The modest response activity constitutes the major limitation to the study. No information was collected from the decliners and therefore their attitudes towards the ODP remain unknown. Bias of this kind was di cult to avoid in a real-world setting when anonymity was preserved and there was no explicit protocol for enrolling participants in the implementation study. In addition, lack of a separate engagement strategy for the team leaders may also have affected the response activity. However, the good quality of the responses obtained to the open-ended questions permits speculation on the likely experiences of the non-responders. A second limitation was the use of the measures developed for this particular study, which impedes the generalizability of the present results and also comparison with earlier studies. A similar challenge has also been common in earlier implementation studies (16, 46, 47).

Conclusions
Based on our results we highlight three important aspects in terms of an EBT implementation programme: the role of team leaders, the structure of the training intervention in the EBT and providing low threshold clinical support aa well as stable organizational structures that ensure the long-term maintenance of the implemented EBTs. The team leaders should be engaged with the implementation programme at an early stage to ensure its successful execution and long-term survival of its outcomes. This includes coaching the leaders to deploy evidence-based managerial practices, connected e.g. to 'transformational leadership', to promote the EBT implementation actively among their teams. We suggest combining workshops and case consultations as an integral training package. A completed training would entail accomplishing both components. Consultative support should be available also during the everyday work. The consultations should be made as easily accessed as possible from the staff's point of view. This need could be addressed for example by training and nominating local peer facilitators. To ensure the sustained delivery of the EBT despite staff turnover the organization should establish stable structures enabling newcomers to be trained in the EBT even after its initial implementation programme. The implications we presented above fall into NPT categories embedding and integration, which have a decisive role in terms of reproduction and longevity of EBT implementation programme outcomes. The ODP related implementation research was not medical research nor did not carry any experimental initiatives or interventions. Its participants were all staff members. The survey was conducted anonymously. When the ODP enrolled therapists were asked to respond to the survey, they were verbally informed about that the answers will be analysed and the results published, and that responding will serve as consent both to participate and for publication. Further, they were informed that responding is voluntary, and responding or not will not affect their status in any way. The ODP was managed and supported by the local administrative staff. According to the Finnish research regulation, the ODP related implementation research was exempt from ethical review (Finnish National Board of Research Integrity, TENK publications 3/2019). Thereby the present study was not subjected for ethics review.

Consent for publication
See the previous section.

Availability of data and materials
The original datasets (in Finnish) generated and analysed during the present study are available from the corresponding author on request. Only the results presented in the article have been translated into English.

Competing interests
Authors LHL, AL and OK have been employed in the target organization during the study. In addition, AL and OK have been in charge of designing and executing the programme under study. JK has no con icts of interest to disclose.

Funding
We gratefully acknowledge the study sponsors for partial nancial support: the South Ostrobothnia Hospital District Research Fund, the Finnish Psychiatric Association, the Gust. Rud. Idman Fund/Finnish Medical Foundation Duodecim and the South Ostrobothnia Fund/Finnish Cultural Foundation. The study sponsors had no role in the study design, collection, analysis, or interpretation of the data or made any other contribution.
Authors' contributions LHL and OK prepared the present study conception and design. LHL performed material preparation and data collection. All authors contributed the analysis and interpretation of the results. LHL wrote the rst draft of the manuscript and all authors commented on previous versions of it. All authors have read and approved the manuscript.

Acknowledgements
We thank all participants involved in the ODP and research nurses Susanna Ahola and Marja Koivumäki for their valuable contributions to this study as well as Virginia Mattila for reviewing the language of the manuscript.   f. Complicated problems are detected earlier in treatment leading to more comprehensive needs-based treatment.

19
a BA=Behavioural activation; MI=Motivational interviewing. b n=33 for all items in Visual Analogue Scale with extremities 0 = "not at all" and 100 = "as well as possible". The means were graded as <50 = failure, 50 = moderate and >50 success. Table 3. Data regarding perceived support from the team leaders, Change index and some possible obstacles.

Item Mean SD
To what extent you perceived the team leader supported your participation in the ODP a ?

29
The ODP has led to changes in the clinical practices followed by our team. (Change index) 41 24 To what extent have the following possibly hampered the execution of the ODP in your team?
a. Other simultaneous developmental tasks or projects 60 29 b. Personal exhaustion due to a variety of projects 40 30 c. Other current team related internal issue 44 33 d. Other current organizational issue external to own team 51 33 Note: n=33 for all items in Visual Analogue Scale with extremities 0 = "not at all" or "hampered very seriously" and 100 = "as well as possible" or "did not hamper at all". The means were graded as <50 = failure or serious, 50 = moderate and >50 success or easy. a Ostrobothnia Depression Programme.