Importance of congruence between communicating and executing implementation programmes: A qualitative study of focus-group interviews.​

doi:10.21203/rs.2.21353/v2

PDF

Research

Importance of congruence between communicating and executing implementation programmes: A qualitative study of focus-group interviews.

https://doi.org/10.21203/rs.2.21353/v2

This work is licensed under a CC BY 4.0 License

Journal Publication

published 27 Oct, 2020

Read the published version in Implementation Science Communications →

You are reading this older preprint version

Read the latest preprint version →

Background The Ostrobothnia Depression Programme (ODP) in Finland was intended to implement two evidence-based brief psychotherapy interventions, namely motivational interview and behavioural activation, in several regional psychiatric teams. A simultaneous effectiveness study was conducted. Considerable tension was encountered between these two arms, causing resistance to change. We conducted a qualitative case study to better understand this tension and to ascertain how managerial and executive practices may ensure the successful running of a hybrid design programme. Methods We conducted focus group interviews to evaluate the phases of preparation and practical execution of the ODP from the perspectives of management and the programme executives. To gather the data, we applied the revised Socratic approach for health technology assessment and the rapid assessment process model. We analysed the data deductively according to Normalization Process Theory. Results We identified two main critical issues: 1) Incongruence occurred between what the project group had explicitly communicated about being the spearhead of the ODP and what they then actually enacted. 2) The ODP programme plan ignored the team leaders' crucial role in influencing the implementation climate and mobilizing organizational strategies. The ODP had a simplistic top-down design with minimal and delayed collaboration with its target groups in the preparation phase. These caused tension between the implementation efforts and the effectiveness study as well as resistance to change among the staff. Conclusion Early, open collaboration with all prospective stakeholders towards a shared understanding about the programme is the first action the programme administrators should take. Agreement on goals and the means to achieve them would lower tension between the two arms of a hybrid design programme, thereby reducing resistance to change. Congruence between the goals communicated and the actual managerial and executive actions is of paramount importance in getting the programme recipients on board.

Health Economics & Outcomes Research

Evidence-based treatment

Implementation

Programme evaluation

effectiveness-implementation hybrid design

Normalization Process Theory

Focus group interview

Quality improvement is the main aim of a programme for implementing an evidence-based treatment (EBT) in the context of a health care organization (1). The ultimate intended beneficiaries are the patients. The key challenge for Programme administrators is to develop a programme plan encouraging frontline treatment providers to incorporate the EBT into their routine practices (2). Several theory-based implementation models or frameworks were constructed to facilitate the work (3–9).

Several determinants for the acceptance of an implementation programme have been identified (5,10), These include top-down vs. bottom-up programme design, early vs. late collaboration with each stakeholder group, the leaders’ reactions to various manifestations of readiness for change among the relevant personnel(11–13). The role and performance of leadership have been reported to be critical for the success of a programme and also for sustaining its outcomes (2,14–16). The factors above, in turn, have an influence on the implementation climate, by which we mean the shared receptivity of the staff involved (5). ‘Programme theory’ is a concept that refers to an individual idea about what might be achieved and by which interventions or operations in a given context (17). This theory, in turn, guides those responsible for the programme in designing the programme plan. They may accomplish this work either heuristically, relying on their previous experience and expertise, or then methodically, grounding their design in a theory-based framework or model, or then a combination of these (7).

Ensuring that an intervention continues to be effective throughout an implementation programme is a fundamental concern (18). Conducting effectiveness-implementation hybrid design studies is a rising and much advocated approach to address this concern (19,20). In such a study these two arms run concurrently. Hybrid design studies are susceptible to exposing the potential tensions inherent in real-world implementation processes of EBTs and their impacts on their application (21,22). For instance, some elements of the original intervention may require adjustment to the real-world setting, thereby risking impaired efficacy (21). This lends support to the call for increasing the application of hybrid designs to gather more clinically quality-controlled knowledge on implementation efforts (23,24). However, not enough is known about possible procedural tensions between effectiveness studies and implementation efforts in naturalistic settings and this gap needs to be addressed.

The administration of the psychiatric department of South Ostrobothnia Hospital District in Finland launched the Ostrobothnia Depression Programme (ODP) (25). The main goal was to bring about a change in the clinical practices to bridge the gap between the resources available and the increasing demand for treatment for depressive patients. The ODP carried out both the implementation programme and the effectiveness study for two evidence-based brief psychotherapy interventions, namely motivational interviewing (MI) and behavioural activation (BA) (26,27). A quantitative evaluation of the implementation programme showed that a third of the target group were active adopters of MI and BA (28). The effectiveness study yielded positive results (29).

Earlier evaluations of the ODP were conducted among the frontline therapists, the intended adopters of MI and BA. The summative evaluation revealed not only a modest implementation outcome but also that the ODP lacked strategies for sustaining implementation outcomes in the long term (28). This was attributed in part to the weak role of the team leaders in the programme execution and this was strengthened in the mixed-methods evaluation of the influence of different organizational- and programme-related factors (Lindholm et al., submitted). In addition, considerable resistance to change was encountered in some of the participating teams while others welcomed the ODP. These observations led us to augment the overall evaluation qualitatively with special focus on the perspective of management and the programme executives. Our ultimate purpose in conducting this case study was to scrutinise the social processes of designing and executing the ODP among those actors, and to increase our understanding of related phenomena. We hypothesized that these phenomena would explain the variation in the units’ willingness to accept the innovation. We also sought answers to the following questions: How do managerial and programme executive practices enable or inhibit the implementation of healthcare innovations or reforms in the clinical context? How can resistance to change be overcome?

The ODP

The hospital district in charge of the ODP is responsible for the provision of public specialized health care services to a population of 200,000. The ODP ran during the period 2009-2013. It was a regional programme comprising two integrated subprogrammes: the Ostrobothnia Depression Study (ODS) and related Implementation Programme (ODS-I). The ODP was aimed to encourage frontline therapists to implement MI and BA and to recruit patients in an effectiveness study of these interventions. Thus, the ODP had a hybrid effectiveness-implementation design, although the term was not used as the ODP was launched a few years before the term was introduced (19). Participation in the training in MI and BA and in applying them in everyday work did not constitute commitment to recruiting patients for the effectiveness study. However, this was intensely encouraged. The programme resources are presented in Figure 1 and the therapists’ tasks regarding the effectiveness study in Table 1.

[insert Figure 1 here]

Figure 1. Programme resources allocated to the Ostrobothnia Depression Programme. Attending training was the only prerequisite for a therapist to be regarded as ODP enrolled.

^aOne-day training workshops for both Behavioural Activation and Motivational Interviewing.

[insert Table 1 here]

The goal to develop clinical practices to meet the increasing flow of depressive patients was initiated by the hospital district administration. The clinical director of the psychiatric department assembled a project group to elaborate a programme for this purpose. In addition to the clinical director (MD, PhD), the project group comprised principal and associate programme executives (a professor of psychiatry and a registered psychologist respectively) and a senior consultant (MD, PhD), all of them permanently employed in the hospital district. They deemed an effectiveness study important for the quality control of the innovation. For the execution, the project group was reinforced with two assisting research nurses.

The unit external to the ODP managing organization was excluded from the present study in order to focus the evaluation on intra-organizational processes. We collected the present data in March 2015, 16 months after the end of the ODP. The time gap was because the analyses of the final summative inquiry and mixed-methods study, both administered to the frontline therapists in spring 2014, revealed a need to complement our understanding about the realization of the ODP. The present authors’ connections to the ODP and the present study are presented in Table 2: Their connections to the managing organization and their mutual professional relationships are presented in Additional File 1.

[insert Table 2 here]

Instruments

The Rapid Assessment Process model (RAP) (30) and the Revised Socratic Approach for Health Technology Assessment (HTA) (31) were used as technical guides while creating the data collection protocol and interview guides, which are presented below. Normalization Process Theory (NPT) (32,33) was applied for the deductive qualitative content analysis (34). These three instruments are presented and their use is explained in Additional File 2. In reporting the study, we adhered to the 32-item checklist of consolidated criteria for reporting qualitative studies (COREQ), which is presented in Additional File 3 (35).

Setting

Formation and description of the study group

We assembled the study group according to the purposeful sampling strategy ‘complete target population’ (36). We emailed the invitation to the whole ODP project group and all team leaders of the target units, 14 individuals in total. Only one recipient involved in the project group declined the invitation due to compelling personal reasons, thus resulting in a study group of 13 individuals. We informed the study group in advance about the purpose, setting and course of the study as well as the principles for handling the data. This included information about the videotaping of the interviews and the assurance that no interviews would be transcribed due to the sensitive nature of the material and further the assurance that each participant’s identity would be protected as far as possible during processing and utilization of the information obtained. Recipients were assured that participation in the study was voluntary and would in no way affect their status within the organization. All members of the study group gave verbal consent to participate.

The study group was divided into two Focus Groups (FG 1 and FG 2) according to each member’s relation to the ODP (for more detail, see Table 2). All members of the study group and the researchers had been permanently employed in the organization for years before the launching of the ODP, thus their relationship was established prior to the present evaluation.

Interview protocol and guides

We interviewed FG 1 twice (FGI 1.1 and FGI 1.2) and FG 2 once (FGI 2) (Figure 2). Each interview lasted three hours and was divided into two parts with a short break between them. Four members out of five in FG 1 and five out of eight in FG 2 attended the group interviews in person. Four individuals were unable to attend the group interviews in person due to pressure of work but provided the desired information in alternative ways: The FG 1-enrolled associate executive was interviewed separately immediately after FGI 1.1 and the information obtained was included in the respective report. One FG 2-enrolled person provided written feedback before the FGI 2, and this information was presented to FG 2 during the interview. The remaining two FG 2 -enrolled people who were unable to attend in person had discussed the issues beforehand with their attending colleague.

[insert Figure 2 here]

Figure 2. Setting for the iterative semi-structured Focus Group Interviews and gathering the raw data. Abbreviations: FG 1 = Focus Group 1; FG 2 = Focus Group 2; FGI 1.1 = the first interview with FG 1; FGI 2 = interview with FG 2; FGI 1.2 = second interview with FG 1.

The same interview guide was used for the interviews with FGI 1.1 and FGI 2. It comprised five topics and nine guiding questions relevant to the ODP. We had recognised five topics that would need to be evaluated in depth; these concerned the underlying motives and intentions of the ODP, its management, the perspectives of the participating units, the interests of the individuals conducting the present evaluation and creating a quick vision for future developments. To ensure that the interviews yielded sufficient of the information sought, we selected questions relevant to our topics from the Revised Socratic Approach for HTA (31). Finally, we reformulated the original questions to match the ODP and referred to them as guiding questions. The creation of the interview guide is written out in Additional File 4, Table A.

The FGI 1.2 interview guide (Additional File 4, Table B) was compiled according to the principles of the RAP. This includes FG 1’s involvement in reflexively appraising the description of ODP processes that had been outlined by the time of their second interview. The purpose of repeated data collection was to ensure richer and more accurate data.

While creating the interview protocol and guides, the first author had reflective discussions about the mission and procedure with the collaborating researcher, the clinical director and the principal programme executive in the spirit of RAP. Due to the setting, we had no opportunity to pilot the interview protocol and guides in practice.

Forming the raw data

As a base, we had the technical data on the ODP comprising the implementation plan of ODS-I (28), the research plan of the ODS including the protocol for data collection (Table 1) and total executive resources in ODP (Figure 1). We gathered the supplementary information through an iterative and collaborative process of FGIs and finally wrote one, rich narrative on the realization of the ODP that comprised the raw data (more detail in Figure 2).

Qualitative content analysis

The case of our study was the process of running the ODP all the way from its rationales to its completion, and the unit of analysis was the narrative formed by the raw data (36). We analysed the raw data through deductive qualitative content analysis (34) guided by NPT (see Additional File 5 for coding frame) (32,33). Our analysis and extracting the results progressed in four steps: First, we encoded the raw data using different colours and reorganized it according to the main categories. Second, we re-encoded and organized the data further according to the subcategories. We reviewed the relevance of the encoding during the two first steps and readjusted when needed. Third, we condensed and rewrote the information contained in the encoded text pieces into a fluent narrative in terms of each subcategory. Fourth, we answered the research questions on the basis of the data analysed, thereby providing the results of the present study.

The first author performed the coding and extracted the results in close consultation with the second author. Finally, we presented the results to the members of FG1 for appraisal and possible amendments. They suggested some refinements and, after these had been made, they accepted the results presented below. The analysis of the data was processed manually with assistance of Word for Mac 2011.

Two main critical issues emerged, which we interpreted to shed light on the friction encountered during the ODP: 1) Right from the outset there was tension between the simultaneously administered implementation programme and the effectiveness study. Although the programme executives explicitly articulated that implementation and quality improvement were the primary intention, the ambitions related to the effectiveness study practically outstripped those of the implementation programme. 2) The programme theory was grounded on the conception that the goals of the ODP were feasible by addressing programme strategies almost exclusively to frontline therapists (Figure 2). To the frontline therapists the focusing strategy was based on the idea of learning by doing. The programme theory was purely heuristic and implicit and was not tested against any formal implementation theory or model. Those who designed the ODP drew on their previous experience of administering developmental programmes and also on their pedagogical expertise and experience of serving as trainers. In addition, they had individual experience of their own training in psychotherapy having a positive impact on mastering clinical work (Figure 3). The results underlying these two issues are next presented in more detail.

[insert Figure 3 here]

Figure 3. Presumed cascade of the impacts of the Ostrobothnia Depression Programme at the level of individual therapist.

Coherence – communal specification.

The reasoning and determination of the ODP goals were constructed mostly at a high level in the organization (Table 3). Prior to the ODP, the clinical director was aware of increasing distress among the frontline personnel due to accelerating patient flow and the ODP was launched to tackle the problem. The project group included no lower level leaders or frontline therapists from the intended target group. The team leaders were not invited until the phase of finalizing the programme plan. The primary goal the project group had set for the ODP was to achieve quality improvement in clinical practices. However, in the experience of the team leaders the preparatory process had proceeded one-way, top-down, which they considered to be a deviation from the normal collaborative two-way managerial practices adhered to while preparing organizational strategies. They saw one-way preparation as a normal and acceptable practice for research programmes. In addition, in the name of the programme, the term “study” preceded the term “implementation”, which, they said, strengthened their perception that the research was accorded priority. In summary, the specification of the ODP was not a collaborative effort between various stakeholders, who thus achieved no shared understanding about the emphasis between the two endeavours.

[insert Table 3 here]

Cognitive participation – initiation

Participation in the ODP was originally voluntary for the units invited, at least in principle. Not all psychiatric units of the hospital district were invited. The invitations were targeted according to two criteria: 1) the clinical director’s impression of the positive readiness for change in the units and 2) the number of patients needed for the effectiveness study. The largest unit was invited according to the second criterion only, that is, to satisfy the needs of the research. Since the largest unit initially declined, they were persuaded to participate after a one-year delay. The other units accepted the invitation at the first step. In summary, involving the largest unit in the ODP was fundamentally incongruent with the first invitation criterion, i.e. readiness for change, and actual willingness to participate on the part of the staff.

Cognitive participation – enrolment

Most of the voluntarily participating units’ team leaders saw the ODP as an opportunity to learn something new and to review the prevailing treatment practices, although they still saw it primarily as a research programme. The reception of the ODP between units varied from welcoming it, through confusion, up to considerable resistance. The programme executives identified one team where the collaboration had been smoothest. The ideas in the ODP were congruent with the team’s own ideas, which they had already been working with. By contrast considerable resistance arose in the largest unit, which had initially been reluctant to participate. In addition, a previously unarticulated confusion about the ultimate intentions of the ODP was eventually articulated. Tackling the resistance greatly depleted the executives’ resources. The team leader of this unit deemed the goals for the ODP to be relevant but considered that the change aimed at was too ambitious to be loaded on one programme. Moreover, the team leader appraised merging programmes for implementation and clinical research as an improper setting to reach the goals.

Despite enrolment in the ODP at the level of units, the enrolment of the therapists in the training varied widely between the units. At best, all the therapists of one unit completed the training. At worst, only one or two therapists of a unit joined in, including one temporary substitute. Some of the units assigned more therapists to the training later on and some of them were motivated mainly by the hope of getting the ODP over and done with. In summary, despite the participation, collective inclination to work on the ODP varied widely across units between adherence and resistance. Two essential manifestations of this tension were identified: the therapists' enrolment in the training varied across units from poor to comprehensive and the motivation of some therapists joining at a later stages was dubious.

Collective action – skill set workability

Due to at least two reasons the number of patients to be recruited for the effectiveness study was accumulated more slowly than anticipated: 1) some of the initially keen therapists got tired in the course of the programme and withdrew and 2) staff turnover cut down the number of ODP-trained therapists. Recruiting patients began to accumulate on fewer shoulders, which caused stress. The question, “when will this be over?” arose among the therapists. In summary, the accumulation of workload biased progressively as the ODP proceeded, resulting in programme fatigue.

Reflexive monitoring.

Some positive experiences in the early phase encouraged the programme executives to think that the strategies applied in the ODP had the potential to bring about the desired cultural change in treatment practices at the level of the entire department. However, they became hesitant as the programme proceeded, partly because they noticed that patient recruitment for the effectiveness study occupied too large a role and the idea of implementation faded. The team leaders had a shared perception that the concurrent running of the implementation and clinical research programmes caused confusion among the therapists. The number of patients needed for the effectiveness study was intended not only to ensure the strength of the study but also a sufficient amount of practice needed to consolidate skills in BA and MI. The drive to satisfy the scientific interest escalated as the ODP proceeded and this exacerbated the therapists’ sense of pressure, which further increased their negative perception of the ODP. In summary, enthusiasm for the implementation declined and the effectiveness study gained in significance as the ODP progressed, which jeopardised achieving the original goal of extensive implementation of BA and MI.

Our analysis revealed two key factors and related phenomena, which helped to understand the course of the ODP: 1) The programme theory. This was based on the project group’s experience of previous developmental programmes and expertise in training. In addition, they assumed that the ODP goals would be feasible by addressing the programme strategies almost exclusively to the frontline therapists. This assumption in particular led to too narrow a programme theory, which ignored the team leaders’ crucial role in influencing the implementation climate and mobilizing organizational strategies (2,15). 2) Coherence between what was explicitly communicated and what was practically accomplished on a programme. The ODP was communicated primarily as an implementation programme for EBTs. However, the target teams perceived the scientific focus to be paramount. These two entry points provided us with material to answer our questions regarding the prospects for the managerial and programme executives to enable the progress of an implementation programme and overcome resistance to change. The two are inextricably linked; Failures in programme implementation led inevitably to failure to overcome resistance to change.

A programme theory is an individual compilation of beliefs as to what a programme might achieve and by what means (17). These beliefs determine the practical actions that the programme administrators will take. Hence it is crucial that the programme theory be evidence-based (37). This also has an obvious impact on the future progress of the programme itself. In the optimal case, the programme theory will be resilient and elaborated in early collaboration with the intended programme addressees (17). Such an approach inherently pursues the communal specification of a programme for creating a shared vision. This evidently has better potential for tackling resistance and, consequently, ensuring a smoothly-running programme (2,14,38). Determining and constructing the ODP programme were mainly accomplished at a high organizational level, with the team leaders only being brought in at a later stage. The project group included neither team leaders nor frontline therapists. The ODP programme theory ignored the crucial role of the team leaders (2,15) and underestimated the impacts of variation in readiness for change and thus also the implementation climate among the target group (5,13,39). Such an omission in the early stages carries a high risk of misunderstandings and, hence, of resistance among the programme addressees (11), and such was the case in the ODP. This had a negative effect on the coherence between the project group and the target group as well as on the willingness to espouse the innovation among the target group. Indeed, early collaboration may initially cause the start-up of a programme to be more complex and time consuming and, consequently, require more resources. However, such investments may be recouped later on in terms of a smoother-running programme.

Seeking a shared understanding is of paramount importance. This emerges or not in terms of accomplishing the programme and between the various stakeholders. The fundamental lack of a shared understanding emerged in terms of executive actions during the ODP. It was communicated as being essentially an EBT implementation programme. However, the mode of preparation caused the target teams to regard it primarily as an effectiveness study. Inducing the largest unit to participate primarily to ensure the strength of the effectiveness study and ignoring the first criterion of inviting the units, i.e. the readiness for change, conveyed a non-verbal message inconsistent with what had initially been articulated. Moreover, towards the end of the ODP, satisfying the patient count needed for the effectiveness study over-rode the idea of implementation. These phenomena also caused and exacerbated misunderstandings between the various stakeholders. In spite of this, one team found the ODP to be consistent with their own developmental efforts in the past, which was also apparent in their positive readiness for change. This led them to the conclusion that the ODP provided them with an opportunity to improve their professional capability, which, in turn, supported their cognitive participation in the ODP (32). The negative experience arose from the conviction that connecting the implementation of two EBTs and their effectiveness study was too much, which exacerbated an unreceptive climate. This fuelled the perception that the two arms of the ODP were in competition with each other, which culminated in a sense of administrative pressure. Furthermore, this caused frustration and rejection among the staff, which can be seen as negative manifestations of cognitive participation and collective action according to NPT (32).

Studying the effectiveness of an EBT in connection with its implementation programme serves as a clinical quality-control of both these arms and ensures movement in the right direction (1,10,18,21). This was also one reason for the hybrid design in the ODP. Additionally, the effectiveness study was assumed to prompt the therapists to actively adopt the EBTs and thus ensure the accumulation of a sufficient amount of clinical practice for acquiring skills in the EBTs. Consequently, in principle the implementation programme had priority. However, the ODP was inherently contradictory in terms of the priority order of the two objectives loaded on it, which caused confusion. Such a situation was likely to lead to a perception that the different objectives were actually competing against each other (10). Adjusting the ODP as a whole with respect to the varying reactions of the teams regarding the implementation climate would also have entailed adjustments in administering the effectiveness study.

Strengths and limitations

We reached all but one out of the intended informants since we accepted other ways of providing information than only individual attendance at the FGIs, which ensured obtaining a wide range of opinions. On the other hand, one more iteration with both FGs and inviting a third focus group from the frontline therapists would have provided us with richer data. Also, in not transcribing the interviews we deviated from the conduct of the conventional qualitative interview study. We made this decision as we were interested in the data verbally articulated, not the nonverbal data. These restrictions enabled us to keep the research process within our resources. In spite of these limitations, we consider that we reached our goal to scrutinise the social processes related to the ODP and thus identify the risks inherent in conducting an effectiveness study in connection with an implementation programme. We contend that the NPT was an appropriate tool for the purpose. In addition, we extended the existing knowledge about the need to ensure the early collaboration with every stakeholder group.

Fidelity of the data

Special attention was paid to the general atmosphere during the FGIs and to ensuring that the data articulated on the questions of interest was clearly expressed (Massey, 2011). During the interviews, a free and frank dialogue was achieved, where both disagreement and consensus within and between the groups were accepted. A report on each FGI was written only a few days after the interview and sent for confirmation to each participant in the FGI concerned. All reports were approved as such. In addition, the facilitator checked the reports by watching the videotaped interviews and no new substantive information was detected although some amplificatory and descriptive details, e.g. quotations, were indeed picked up. The foregoing serves to verify the true correspondence between the essential contents of the FGIs and the raw data. In addition, the members of the FG1 reviewed the present results section, which was amended according to the feedback.

Early, open collaboration with all intended stakeholders for pursuing a communal specification, i.e. a shared understanding, about the programme is the first action programme administrators should take on launching an EBT implementation programme. This has a direct link to the programme theory about what the programme has the potential to reach and how. Early collaboration would have improved the mutual understanding among the stakeholders and helped the administrators to take all relevant aspects into account. Congruence between what the programme administrators communicate and what they actually do is the second thing to be strictly adhered to throughout the programme. This is crucial to avoid confusion regarding the ultimate mission of the programme. Hybrid design programmes have potential to achieve quality-controlled outcomes in implementing healthcare innovations or reforms. However, they require careful attention to keeping the balance consistent between the programme’s primary mission and the effectiveness study. This and early collaboration are principles the clinical managers and programme executives should adopt to enable the implementation of healthcare innovations or reforms and to overcome resistance to change.

BA: Behavioural activation

COM-B framework: Capability, Opportunity, Motivation and Behaviour framework

COREQ: 32-item checklist of consolidated criteria for reporting qualitative studies

EBT: Evidence-based treatment

FG: Focus Group

FGI: Focus Group Interview

HTA: Health Technology Assessment

MI: Motivational interviewing

NPT: Normalization Process Theory

ODP: Ostrobothnia Depression Programme

ODS: Ostrobothnia Depression Study

ODS-I: Ostrobothnia Depression Study related Implementation Programme

RAP: Rapid Assessment Process model

FG 1 and 2: Focus Group 1 and 2

FGI: Focus Group Interview

FGI 1.1: first interview with FG 1

FGI 1.2: second interview with FG 1

FGI 2: interview with FG 2

Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Heal Ment Heal Serv Res. 2011;38(2):65–76.
Aarons GA, Green AE, Trott E, Willging CE, Torres EM, Ehrhart MG, et al. The Roles of System and Organizational Leadership in System-Wide Evidence-Based Intervention Sustainment: A Mixed-Method Study. Adm Policy Ment Heal Ment Heal Serv Res. 2016;43(6):991–1008.
Wensing M, Bosch M, Grol R. Selecting, tailoring, and implementing knowledge translation interventions. In: Straus S, Tetroe J, Graham ID, editors. Knowledge Translation in Health Care. 2nd ed. Oxford: Wiley-Blackwell BMJ Books; 2013. p. 150–62.
Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011;38(1):4–23.
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(50).
Michie S, Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6(42).
Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10(53).
Harvey G, Kitson A. PARIHS revisited : from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci. 2016;11(33).
May C. Towards a general theory of implementation. Implement Sci. 2013;8(18).
Vedung E. Public Policy and Program Evaluation. 1st ed. Abingdon, Oxon: Transaction Publishers; 1997.
Greenhalgh T, Roberts G, MacFarlane F, Bate P, Kyriakidou O. Diffusion of Innovations in Service Organizations: Systematic Review and Recommendations. Milkbank Q. 2004;82(4):581–629.
Engeström Y. From individual action to collective activity and back: developmental work research as an interventionist methodology. In: Luff P, Hindmarsh J, Heath C, editors. Work Place Studies. 1st ed. Cambridge: Cambridge University Press; 2000. p. 150–66.
Stanhope V, Ross A, Choy-Brown M, Jessell L. A Mixed Methods Study of Organizational Readiness for Change and Leadership During a Training Initiative Within Community Mental Health Clinics. Adm Policy Ment Heal Ment Heal Serv Res. 2019;46(5):678–87.
Beidas RS, Stewart RE, Adams DR, Fernandez T, Lustbader S, Powell BJ, et al. A Multi-Level Examination of Stakeholder Perspectives of Implementation of Evidence-Based Practices in a Large Urban Publicly-Funded Mental Health System. Adm Policy Ment Heal Ment Heal Serv Res. 2016;43(6):893–908.
Peterson AE, Bond GR, Drake RE, Mchugo GJ, Jones AM, Williams JR. Predicting the Long-Term Sustainability of Evidence-Based Practices in Mental Health Care : An 8-Year Longitudinal Analysis. J Behav Health Serv Res. 2014;41(3):337–346.
Bunger AC, Birken SA, Hoffman JA, MacDowell H, Choy-Brown M, Magier E. Elucidating the influence of supervisors’ roles on implementation climate. Implement Sci. 2019;14(1):1–12.
Oosthuizen C, Louw J. Developing program theory for purveyor programs. Implement Sci. 2013;8(23).
Kam C-M, Greenberg MT, Walls CT. Examining the role of implementation quality in school-based. Prev Sci [Internet]. 2003;4(1):55–63. Available from: papers3://publication/uuid/4E1D3845-11A2-4146-BA55-768FB6ABAFBD
Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: Combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217–26.
Landes SJ, McBain SA, Curran GM. Reprint of: An introduction to effectiveness-implementation hybrid designs. Psychiatry Res. 2020;283(November 2019).
Van Dijk-De Vries A, Bokhoven MA, Winkens B, Terluin B, Knottnerus JA, Van Der Weijden T, et al. Lessons learnt from a cluster-randomised trial evaluating the effectiveness of Self-Management Support (SMS) delivered by practice nurses in routine diabetes care. BMJ Open. 2015;5(6):1–11.
Mignogna J, Martin LA, Harik J, Hundt NE, Kauth M, Naik AD, et al. “I had to somehow still be flexible”: Exploring adaptations during implementation of brief cognitive behavioral therapy in primary care. Implement Sci. 2018;13(1):1–11.
Dzidowska M, Lee KSK, Wylie C, Bailie J, Percival N, Conigrave JH, et al. A systematic review of approaches to improve practice, detection and treatment of unhealthy alcohol use in primary health care: A role for continuous quality improvement. BMC Fam Pract. 2020;21(1).
Wolfenden L, Williams CM, Wiggers J, Nathan N, Yoong SL. Improving the translation of health promotion interventions using effectiveness–implementation hybrid designs in program evaluations. Heal Promot J Aust [Internet]. 2016 Dec 1;27(3):204–7. Available from: https://doi.org/10.1071/HE16056
ClinicalTrials.gov Identifier NCT02520271. Ostrobothnia Depression Study (ODS). A Naturalistic Follow-up Study on Depression and Related Substance Use Disorders. [Internet]. Available from: https://clinicaltrials.gov/ct2/show/NCT02520271
Lundahl B, Burke BL. The effectiveness and applicability of motivational interviewing: a practice-friendly review of four meta-analyses. J Clin Psychol. 2009 Nov;65(11):1232–45.
Cuijpers P, van Straten A, Warmerdam L. Behavioral activation treatments of depression: A meta-analysis. Clin Psychol Rev. 2007;27(3):318–26.
Lindholm LH, Koivukangas A, Lassila A, Kampman O. What is important for the sustained implementation of evidence-based brief psychotherapy interventions in psychiatric care? A quantitative evaluation of a real-world programme. Nord J Psychiatry. 2019;73(3):185–94.
Luoto KE, Lindholm LH, Paavonen V, Koivukangas A, Lassila A, Leinonen E. Behavioral activation versus treatment as usual in naturalistic sample of psychiatric patients with depressive symptoms : a benchmark controlled trial. BMC Psychiatry. 2018;18(238).
Beebe J. Rapid Assessment Process. In: Kempf-Leonard K, editor. Encyclopedia of Social Measurement. New York: Elsevier; 2005. p. 285–91.
Hofmann B, Droste S, Oortwijn W, Cleemput I, Sacchini D. Harmonization of ethics in health technology assessment: A revision of the socratic approach. Int J Technol Assess Health Care. 2014;30(1):3–9.
May C, Finch T. Implementing, embedding and integrating practices: an outline of Normalization Process Theory. Sociology. 2009;43(3):535–554.
May C, Rapley T, Mair FS, Treweek S, Murray E, Ballini L, et al. Normalization Process Theory On-line Users’ Manual, Toolkit and NoMAD instrument. [Internet]. 2015. Available from: http://www.normalizationprocess.org/
Schreier M. Qualitative Content Analysis. In: Flick U, editor. The SAGE Handbook of Qualitative Data Analysis. London: SAGE Publications Ltd; 2014. p. 170–84.
Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research: A 32-item checklist for interviews and focus groups. Int J Qual Heal Care. 2007;19(6):349–57.
Patton MQ. Qualitative research & evaluation methods: integrating theory and practice. 4th ed. California: Thousand Oaks, California: SAGE Publications, Inc; 2015.
Ogden T, Fixsen DL. Implementation Science. Z Psychol [Internet]. 2014 Jan [cited 2016 Feb 18];222(1):4–11. Available from: http://econtent.hogrefe.com/doi/abs/10.1027/2151-2604/a000160
Kitson AL. The need for systems change: reflections on knowledge translation and organizational change. J Adv Nurs. 2009;65(1):217–28.
Shea CM, Jacobs SR, Esserman DA, Bruce K, Weiner BJ. Organizational readiness for implementing change : a psychometric assessment of a new measure. Implement Sci. 2014;9(7).

Ethics approval

According to the principles agreed on in the South Ostrobothnia Hospital District, this case study of the ODP was exempt from ethical approval as the work was primarily intended to improve regional care through the implementation of EBTs and by using personnel members as informants.

Consent to participate and for publication

All members of the study group gave their verbal consent to participate in the study, which included consent to the publication of the analysed data.

Availability of data and materials

The original datasets generated and analysed during the present study are not publicly available due to the requirement to preserve confidentiality. However, the final narrative about the OPD, that is the raw data, is available from the corresponding author on reasonable request. The raw data is in Finnish. We made the English translation during the fourth step of the analysis or answering the research questions.

Competing interests

All authors were permanently employed in the main target organization of the ODP. Also, they had various professional relations with each other. These are disclosed in more detailed in Additional File 1.

Funding

The South Ostrobothnia Hospital District research fund supported the ODP but did not influence the course of the research at any stage or preparation of the manuscript.

Authors’ contributions

LHL designed the present study in close collaboration with OK and AL. In addition, LHL led the data collection, prepared the raw data, performed the qualitative analysis and also made the preliminary interpretation of the data. LHL also wrote the manuscript. ML was closely involved in the data collection, the raw data preparation and its analysis and interpretation. In addition, ML reviewed several versions of the manuscript. OK and AL were also informants in the study and according to the study protocol they contributed to the interpretation of the data. In addition, OK reviewed several versions of the manuscript. More detailed information on the authors’ professional relations to ODP, the organization and each other is presented in Additional File 1.

Acknowledgements

We thank all participants in the present study and especially research nurses Susanna Ahola and Marja Koivumäki for their contributions to the ODP. We also thank Virginia Mattila for checking the language.

Author’s information

Authors are introduced in Additional File 1: The authors’ professional relations to the ODP, the organization and each other.

Correspondence to Lars H. Lindholm ([email protected]).

All Additional files are in pdf-format.

Additional File 1. The authors’ professional relations to ODP, the organisation and each other.

Additional File 2. Descriptions and rationale behind of three instruments.

Additional File 3: The 32-item checklist of consolidated criteria for reporting qualitative studies (COREQ).

Additional File 4. The interview guide for the first interviews of Focus Groups 1 and 2 and the process of deriving of the guide (Table A). The interview guide for the second interview of Focus Group 1 (Table B).

Additional File 5. Coding frame according to Normalization Process Theory.

Table 1. Tasks for a therapist regarding the Ostrobothnia Depression Programme related effectiveness study. The doctors were responsible for diagnostics and medication. Patients referred to psychiatric secondary services because of depressive symptoms, anxiety, self-destructiveness, insomnia and alcohol or other substance related problems were screened for recruitment. The inclusion criterion was 17 points in BDI-21^a but patients with psychotic disorders or organic brain disease were excluded.
Patient’s first visit to a unit/therapist	Recruitment - Giving information about the study - Requesting a written informed consent After the consent - Filling in a three-page patient data form - Performing a structured patient assessment (BDI-21, AUDIT^b, alcohol dose counter form, GAF^c, MINI-C^d) - Filling in a referral to laboratory tests
During the treatment	Every two weeks - BDI-21 When necessary - CIWA-Ar^e - Patient follow-up form after detoxification - Study discontinuation form
^aBeck Depression Inventory, 21-item ^bAlcohol Use Disorder Identification ^cGlobal Assessment of Functioning ^dMini International Neuropsychiatric Interview, module C for assessment of suicidality ^eClinical Institute Withdrawal Assessment for Alcohol Scale

Table 2. Participants’ various relations to the study and manuscript.
Relation to the study	Role in the interviews	Relation to drawing the final results	Relation to the manuscript
Main researcher	Interviewer	Main	First author
Collaborating researcher	Secretary^a	Collaborating	Second author
Study Group
Focus Group 1/ the project group, five members
Clinical director of the psychiatric department	Informant	Acceptance	Third author
Principal designer and executive of the programme, prof. in psychiatry	Informant	Acceptance	Fourth author
Associate designer and executive, reg. psychologist	Informant	Acceptance	None
Two assisting research nurses	Informants	Acceptance	None
Focus Group 2/ team leaders, eight members
Eight people, both psychiatrists and registered nurse	Informants	None	None
^aThe collaborating researcher had to be excluded from the interview of Focus Group 2 due to her managerial relation to its nurse members, so the main researcher also took notes while interviewing. The notes were checked afterwards against the videotapes.

Table 3. Preparation of the ODP^aand stages of involving different stakeholder groups.
Stake-holder	Stage		Description
The project group^b	I	Identifying and analysing the problem to be tackled	Obstructed patient flow and difficulties in work management.
	II	Defining the goals	1. Speed up the treatment process by increasing delivery of brief therapies 2. Increase application of the integrated treatment model to make up for the deficit in the treatment of dual diagnosis patients 3. Measure the effectiveness of the treatment model 4. Improve the work well-being of the staff by strengthening their work management
	III	Preparation of the programme plan	a. Determining the criteria for selecting the interventions to implement b. Determining the criteria for inviting the units to participate c. Designing the treatment model d. Designing the implementation plan e. Designing the protocol for the effectiveness study
The project group and team leaders	IV	Finishing the programme plan.	a. The project group consulted the team leaders a few times for amendments b. The plan was modified slightly in terms of practical execution according to the comments
^aODP = Ostrobothnia Depression Programme. ^bThe project group = The clinical director of the department of psychiatry, the principal and associate executives of the ODP and a senior consultant, and for execution the group was reinforced with two assisting research nurses. They were all permanently employed in the main target organization.

PDF

Journal Publication

published 27 Oct, 2020

Read the published version in Implementation Science Communications →

Editorial decision: Minor revision
07 Sep, 2020
Review #1 received at journal
17 Aug, 2020
Review #2 received at journal
14 Aug, 2020
Reviewer #2 agreed at journal
23 Jul, 2020
Reviewer #1 agreed at journal
22 Jul, 2020
Reviewers invited by journal
19 Jul, 2020
Editor assigned by journal
26 Jun, 2020
Submission checks completed at journal
25 Jun, 2020
Editor invited by journal
25 Jun, 2020

You are reading this older preprint version

Read the latest preprint version →

Importance of congruence between communicating and executing implementation programmes: A qualitative study of focus-group interviews.

Status:

Journal Publication

Version 2

Abstract

Figures

Background

Methods

Results

Discussion

Conclusion

Abbreviations

References

Declarations

Additional File Information

Table

Supplementary Files

Status:

Journal Publication

Version 2

Importance of congruence between communicating and executing implementation programmes: A qualitative study of focus-group interviews.​

Status:

Journal Publication

Version 2

Abstract

Figures

Background

Methods

Results

Discussion

Conclusion

Abbreviations

References

Declarations

Additional File Information

Table

Supplementary Files

Status:

Journal Publication

Version 2

Importance of congruence between communicating and executing implementation programmes: A qualitative study of focus-group interviews.