In the present study, we found fidelity to be positively correlated with care providers’ perceptions of the implementation and that these perceptions can be affected by systematic implementation support. The correlations between fidelity and IPAT score were strongest late in the implementation process and were not present at baseline. The care providers perceived the implementation for the practice they received structured implementation support for more positively than the control practice.
A relationship between how care providers perceive the implementation effort and implementation outcome, as we found, is in line with well-known theories and frameworks such as Readiness for change, Stages of change and CFIR suggest [15, 18, 29, 33, 34]. However, we found the implementation factors and outcome to be associated later in the implementation process and not at onset. Our findings are in contrast to the literature emphasizing the investigation of facilitating factors only prior to an implementation effort [7, 18, 35]. They indicate that a more process-based approach to the readiness concept, as suggested by Stevens, among others , can be beneficial to understanding implementation. The changing IPAT scores during the period of 18 months also suggest that health professionals assess the situation during all phases of the implementation to decide if they will implement and sustain the new practice. This is in line with Nilsen, arguing that the progression of the implementation process and health professionals’ perceptions are mutually affected and that actions are mutually affected in a context [17, 36]. According to Stevens, the assumption that it is sufficient to establish readiness at baseline fails to understand the influence of context over time on individuals’ cognitive and affective evaluation and response . Furthermore, our results are in line with complex intervention theory highlighting mutual interactions between intervention and context factors  and with evidence for repeated feedback’s impact on maintaining engagement during the implementation process . Our results suggest that care providers’ perceptions and implementation outcomes are mutually affected, creating a positive or negative spiral of improvement success or failure, as described by Øvretveit .
The implementation literature states that systematically developed intervention bundles can affect health professionals’ engagement and implementation outcomes [29, 39]. The intervention in the present study was designed to affect the same construct of care providers’ individual and collective readiness and engagement as the IPAT questionnaire intend to measure. It combines several recommended interventions, such as continuous measurement and provision of feedback, clinical training, implementation training and team support [1, 39–41]. Our finding of a significantly higher score on the IPAT scale when implementation support was provided compared to when not was therefore expected. What we believe to be innovative is the measurement of and feedback on how the implementers experience the implementation effort. Responding to the questionnaire may introduce reflections about effective implementation among the responders, and the manager can gain insight into the facilitation needs. We combined this with guidance from implementation facilitators trained to support team activities to meet the needs revealed by the IPAT scores. Tailoring implementation support to the present needs of the implementers is emphasized in the literature ; however, we cannot state which part of the intervention was the most effective. Interestingly, we also found the IPAT score to be higher for the practices the units knew they would get support for before the onset of systematic implementation support. We believe this is best understood by readiness theories recognising the importance of self-induced initial preparation and collective tuning in on the forthcoming implementation effort .
Few implementation factors are tested for their associations with implementation outcomes. The review by Chaudoir et al. found only one study where explanations for variation in fidelity were investigated . In the present study, we employed a questionnaire based on a combination of theories and a framework for implementation . Given the complexity of implementation, we believe that the extraction of prominent factors from a set of theories is more likely to explain implementation than scales based on single theories. The significant correlation we found between the IPAT score and fidelity indicates that the questionnaire is well suited for investigating the “why” in implementation.
The present study used a cluster-randomized longitudinal design including a large sample of clinical units followed for 18 months. We gathered data four times on implementation factors and -outcome for both the intervention and the control arm. Most investigations of the implementation process have used a retrospective design and/or qualitative methods, implying a risk of bias related to the informants’ and researchers’ recollection and insight into the implementation process [2, 43]. The repeated measurement we conducted every sixth month enabled an investigation of associations between perceptions and actual change to practice in different phases of the process that qualitative or pre-post design cannot. The design, where each clinical unit was assigned to both intervention and control arms for two practices they had chosen to implement, reduced the risk of bias. However, a risk of spillover effects from the intervention to the control arm is present, which implies that we may underestimate the effect of the intervention.
Each clinical unit chose two of four practices and was drawn to receive support for only one. Logically, we expect the clinical units to choose practices they need to implement. The generally low score initially indicates that this was true. Our results and conclusions should be interpreted as true when implementing a practice that the clinical unit believes to be beneficial for them. We do not know to what degree our results are valid for situations where clinical units are obliged to implement a practice they have not chosen themselves.
The study was conducted in Norwegian specialist mental health care and regards the implementation of evidence-based practices for patients suffering from psychosis. The sample represents typical multiprofessional clinical units from urban and rural areas of Norway. The IPAT is based on international acknowledged literature and is expected to be valid for similar Western health services. It is developed for health care services in general but has only been tested within mental health care yet. The generalizability of the present study’s results and conclusions to health services for other patient groups or to other countries is unknown.