Evaluating Implementation Contexts, Mechanisms, and Outcomes: A Normalization Process Theory Coding Manual for Qualitative Research and Instrument Development

Carl May (  carl.may@lshtm.ac.uk ) London School of Hygiene and Tropical Medicine Faculty of Public Health and Policy https://orcid.org/0000-0002-0451-2690 Bianca Albers University of Zurich: Universitat Zurich Mike Bracher University of Southampton Tracy L Finch University of Northumbria at Newcastle: Northumbria University Anthony Gilbert Royal National Orthopaedic Hospital NHS Trust Melissa Girling Northumbria University Kathryn Greenwood University of Sussex Anne MacFarlane University of Limerick Frances Mair University of Glasgow Christine M May LSHTM: London School of Hygiene & Tropical Medicine Elizabeth Murray UCL: University College London Sebastian Potthoff Northumbria University Tim Rapley Northumbria University

and characterizing a set of concepts that are found to be salient to a research problem. The process of data transformation that follows from using a coding manual has two advantages. It enables researchers to manage the cognitive burden of searching for and handling multiple concepts, and thus enables them to manage a greater cognitive burden of interpretation. Researchers can therefore transparently share nuanced meanings and novel connections that are represented in any given corpus of data. Within research teams, coding manuals support the quality and rigor of coding by providing 'rules' that are employed by each team member and, in this way, can ensure the consistency of coding. Parsimony can be important too: more is not necessarily better in qualitative investigation and analysis. Reducing the number of codes to those that represent core concepts can be understood as what Adams et al [6], in a different context, have called subtractive transformation.
The purpose of this paper Despite their value, the process of creating rigorous and robust coding manuals for individual studies is rarely described, and generalizable coding manuals are rare. These would be particularly useful for applying theoretical constructs that are used by qualitative researchers. In this paper, we ll this gap. We describe the purposes, methods of development, and potential application of a coding manual that translates Normalization Process Theory (NPT) into an easily usable framework for qualitative analysis. NPT provides a set of conceptual tools that support understanding and evaluation of the adoption, implementation, and sustainment of socio-technical and organizational innovations [7][8][9][10][11]. It is supported by empirical studies using both qualitative and quantitative methods, and by systematic reviews that have explored its value in different research domains [12][13][14]. As a widely used theory, a generalizable coding manual would be of value to implementation researchers from a range of disciplines. This paper also serves to make the coding manual freely available to other researchers.
The coding manual performs four important functions. First, it identi es and de nes the core concepts of the theory, shows how they relate to each other, and demonstrates how they can be used to transparently categorize qualitative data. Second, it provides a clear and consistent set of de nitions of core concepts in NPT that can be used when QCA is being conducted by teams working together on large bodies of data and, sharing-and sometimes checking-each other's analytic work. Third, it offers a starting point for multimodal analysis, (for example, in evaluations employing different qualitative research techniques), and a framework for directed content analysis (for example, in qualitative evidence syntheses). Finally, as software for computational hermeneutics [15] becomes more widely available and practically workable, a coding framework such as this can also be integrated into the development of topic modelling instruments and algorithms.

Methods
NPT is one of the most widely used implementation theories, and is often used to inform qualitative research on the implementation of socio-technical and organizational innovations in healthcare and other domains. As it has developed, has moved through multiple iterations, and its conceptual foundations have become more complex.
These iterations have resulted from: a. Theory-building: the construction and interpretation of core concepts of NPT over time [7-9, 16, 17].
b. Theory-translation: the clari cation and formulation of core concepts by reference groups during methodological research leading to the development of toolkits [18,19] and survey instruments [20][21][22].
c. Theory-elaboration: development of new concepts during empirical studies and systematic reviews, that characterize and explain additional aspects of implementation processes [23][24][25].
Building a theory-informed framework for coding qualitative data involves a series of tasks that are, in themselves, a form of qualitative analysis. As we noted earlier, qualitative research focuses on the identi cation, characterization and interpretation of empirical regularities or deviant cases in natural language data. The process described here developed organically and opportunistically over a period of 18 months. The work of de ning key concepts of the theory, assembling these into a framework, and then transforming them into a manual that can be used in practice, resulted from practically working on the concepts described below and making them fully usable. To do this, we drew on procedures suggested by Schreier [2] in her account of QCA. The general structure of coding manual development is laid out in Figure 1.
First pass selection and structuring.
The result of the iterative development of NPY-like any theory-is a body of concepts representing the mechanisms that motivate and shape implementation processes, and also other features of interventions and the contexts in which their users make them workable and integrate them into practice. These core concepts of NPT were distributed over several papers that developed the theory, and in others that developed the means and methods of its application. In June 2020, these concepts were identi ed and assembled in a taxonomy of statements (n=149). They took ve main forms and are described in Table  1. This taxonomy consisted of concepts that de ned, identi ed and characterized observable features of the collective action and collaborative work of implementation. Once prepared, it could be translated into a coding manual.
Piloting the manual: rst pass The taxonomy of 149 statements assembled in selection and structuring work included multiple duplicates, along ambiguous and overlapping descriptions of concepts. (These are described in the Online Supplementary Materials.) Duplicate and overlapping concepts were amalgamated, ambiguous concepts were either disambiguated or eliminated. After this work was competed, 38 discrete concepts were retained to make up a ' rst pass' coding manual. (These are also described in the Online Supplementary Materials.) Earlier work to develop NPT, associated toolkits, and survey instruments had employed piloting of concepts and their de nitions [17,21,22,26], to make sure that they could be used by researchers, practitioners, and policy makers. This manual was then applied to coding two papers selected from an earlier NPT systematic review. These were comprehensively coded and checked amongst the authors of this paper, who critically commented on coding decisions. The same coding manual was then applied to two bodies of interview data collected in other studies that were organized around NPT.
A Randomized Controlled Trial. Transcripts of interviews (n=22, with community mental health professionals) conducted for the process evaluation of the EYE-2 Trial (an engagement intervention for rst episodes of psychosis employed in early intervention in the community [27], A 'real world' implementation program. Transcripts of interviews (n=55 with managers, practitioners and patients) conducted for an evaluation of the accelerated implementation of remote clinicianpatient interaction in a tertiary orthopedic center during the Covid-19 pandemic.
This pilot work demonstrated that the main elements of the coding manual were workable in practice. However, it also suggested that some concepts still overlapped or were redundant. For example, a code that sought to outline 'events in processes' [28] that confounded implementation work, thus: Intervention success: experienced workability and integration in contexts [29] In practice, this overlapped with Internal (endogenous) processes and events that shape the capacity of participants to implement and operationalize interventions and their components (e.g. staff shortages, strategic initiatives) [9] Intervention success was eliminated from the main body of the coding manual and moved to a 'holding' group of concepts, while Internal Processes was retained. Additionally, some concepts were insu ciently focused, and so were di cult to practically use. For example: Coupling: Relations of interdependence between people, interventions and their components, and the contexts in which they are working. [10,30] Less surprisingly, use of the coding frame suggested ways in which concepts were subject to different interpretations. Some of these were resolved by adjusting the wording in their de nitions, but it was also clear that some concepts could be eliminated without affecting the integrity of the whole.

Second pass selection and structuring
The coding manual did characterize the core mechanisms that motivate and shape the work of implementation, along with concepts that represented the effects of this work. The piloting exercise revealed that the rst pass coding manual was hard to use because it was over-complex and because it micro-managed the process of interpretation. This defeated attempts at nuanced interpretation. It was also clear that the shape of implementation processes themselves was much less visible in the coding manual. Here, the contexts or domains in which collective action and collaborative work, and their effects, are undertaken and experienced remained relatively invisible. This needed to be remedied.
In NPT, the contexts of implementation can be understood as both structures and processes. Following Hawe [28], we de ne contexts as events in systems that unfold over time within and between settings. To rectify the absence of a description of process domains, the taxonomy of NPT concepts was searched again for salient descriptors of context. Four main ways of framing implementation contexts were found: i. Statements about implementation processes as contexts for the translation of strategic intentions into new or modi ed practices.
ii. Statements about implementation processes as contexts for making practices workable through adaptive execution.
iii. Statements about implementation processes as contexts for the integration of strategic intentions.
iv. Statements about implementation processes as organizational logics that are reframed as the collective action and collaborative work that characterizes implementation are accomplished.
As a result of second pass selection and structuring, concepts that represented contexts for action were added to the manual. These concepts are fully described in Table 2.
The four 'new' process concepts were then piloted 'in use' on a set of transcripts collected for an already completed study [31] of the design and implementation of treatment escalation plans to manage care at end of life in British hospitals.
It was found that they characterized process contexts effectively. These concepts were then presented and discussed in a series of international webinars in February-April 2021. Discussion with participants in those webinars assisted in nalizing the process model.

Results: A Core Coding Manual For Normalisation Process Theory
The nal coding manual consisted of 12 primary NPT concepts, and 16 subsidiary concepts. Working through the procedures described above led to highly simpli ed form of NPT. Although it was not intended to do so, the nal structure of the coding manual conforms to the Context-Mechanism-Outcomes con guration of realist evaluation studies [32]. It is organized thus: Contexts-events in systems unfolding over time within and between settingsin which implementation work is done. (Concepts are described in Table 2: strategic intentions; adaptive execution; negotiating capability; reframing organizational logic.) Mechanisms-made visible through collaborative work and collective action-that motivate and shapethe work that people do when they participate in implementation processes. (Concepts are described in Table 3: coherence-building; cognitive participation; collective action; re exive monitoring.) Outcomes-the effects of implementation work in context-that make visible how things change as implementation processes proceed. (Concepts are fully described in Table 4: intervention performance; normative restructuring; relational restructuring; sustainment.) Coding is a centrally important procedure in qualitative analysis, but it needs to be emphasized that it is only one part of a whole bundle of cognitive processes through which researchers make meanings in the data. For this reason, we recommend a layered approach to applying the coding manual. The coding manual itself is presented in Tables 2, 3, 4 and 5. It can be used in the following way.

Familiarization and concept identi cation
Familiarization. All qualitative analysis calls on researchers to become completely familiar with that part of the data-set that they wish to analyze. This may consist of any combination of original video or audiorecordings, eldnotes, transcripts, social media posts, other documents, or already published texts.
Managing data. The organization of data analysis is of paramount importance. A set of agreed procedures needs to be developed for identifying, sharing and retrieving coded data. This may be done using software such as Atlas Ti, NVivo, or Dedoose); or by using a 'manual' method such as framework analysis on Microsoft Excel, or commentary using 'Track Changes' comments boxes in Microsoft Word. Both software and 'manual' methods can provide highly effective scaffolding for qualitative analysis.
Concept identi cation calls on the researcher to work through individual items in the data-set line by line and to link each item (which may be a phrase, sentence or paragraph in text, or an utterance or scene in untranscribed video or audio), either to one of the concepts included in the coding manual, or to a concept arising from some other source. In NPT studies these may be associated with constructs of the theory, and they may be related to the contexts, mechanisms, or outcomes of processes of adoption, implementation, and sustainment over time and between settings.

Characterization and category-building
Category-building calls on the researcher to use the de nitions in the coding manual to identify the speci c concepts or sub-concepts within each item in the data-set associated with Context, Mechanism, or Outcomes.
Characterization calls on the researcher to consider the relationship between, and signi cance of, concepts and sub-concepts. For example, are some constructs preconditions or barriers for others? Do others represent a temporal ow of events? Do the data suggest causal processes?
Identi cation of special cases. Within any qualitative data-set it is likely that there will be special cases. Coding outside the manual.
Familiarization and line by line analysis will inevitably reveal items in a data-set that cannot be easily categorized to the Context-Mechanism-Outcome model set out in the NPT Coding manual. Here, new categories can be also be created.
Associations between new and prior categories. Where new categories are associated with the categories already set out in the coding manual, it may be possible to treat them as sub-categories of these. For example, participants in implementation studies often talk about time as a personal or corporate resource, but these accounts are often closely associated to material already coded under constructs such as Cognitive Participation or Relational Restructuring.
Differences between topics. Where new categories relate to a different topic, an extended coding manual that includes new categories can be created. These may be driven by engagement with data (e.g. an implementation study shows how the implementation of an intervention leads to experiences of gender-inequity and this is made into a parallel analytic track); or they may be driven by theory, (e.g. constructs from other theories can be used to identify and explore the whole range of inequities that can be created through an implementation process).
Interpretation and explanation.
Four NPT concepts related to mechanisms (Coherence-building, Cognitive Participation, Collective Action, Re exive Monitoring), each possess four associated sub-concepts (described in Table 5). Users of NPT have often tended to treat these as discretionary in their coding and analysis [12][13][14], and their use is not mandatory. They do, however, add a layer of nuance to analysis of implementation mechanisms, and they remain highly usable and workable where the data support interpretation at that level of detail.
The act of coding is descriptive work that is a foundation for the interpretation of data, and is not a proxy for it. In each stage of coding, theoretical interpretations of the data can be recorded in what Glaser and Strauss [33] called Memos. These are commentaries on interpretations of the data and its contexts that the researcher makes as they perform coding.
It is not the purpose of a coding framework to verify the underpinning theory. The whole purpose of coding, and working abductively within theory, is to build and inform interpretation and understanding.
This is not a discrete stage in data analysis but is continuous throughout. Researchers can hold joint 'data clinics' to surface these interpretive understandings, make necessary adjustments to the coding rules to capture the speci cities of their data, work through their implications, and develop analytic propositions that add to, or extend, theory in relation to the speci c phenomena of interest that their studies focus upon.

Discussion
Faced with a problem, people tend to add rather than subtract elements to produce a solution [6]. The purpose of developing this coding manual was to clarify and simplify NPT in use: to make it more workable, and to make it more easily integrated into research on the adoption, implementation, and use of sociotechnical and organizational innovations. In QCA-as in other forms of qualitative analysisproliferating constructs can easily make the business of coding ever more microscopic and less analytically rewarding. Indeed, the more parsimonious a prescheduled theoretical structure is, the more space it provides for nuanced interpretation and the development of novel categories of data and the analytic concepts that can be derived from them. In the development of an NPT coding manual described here, rst pass selection and structuring sought to eliminate ambiguity and add workability at the outset, while second pass selection and structuring sought to eliminate overlapping or ambiguous constructs that were revealed in the analysis of qualitative data or in the coding of published articles. Taken together, these efforts yielded a set of 12 primary NPT concepts, and 16 sub-concepts. These concepts identify, characterize and explain the course of implementation processes through which strategic intentions are translated into practices, and enable understanding of how those practices can lead to different outcomes, and to varying degrees of sustainment.

Strengths and limitations
We describe a set of methods likely to be useful be useful to qualitative researchers in other areas of research who wish to consider developing such manuals for other theories (for example, Relational Inequalities Theory [35], or Event System Theory [36]). A strength of the work was that developing the coding manual was undertaken by an international multidisciplinary team working with three recent empirical studies and a qualitative evidence synthesis. Regrettably, because this work was undertaken during the Covid-19 pandemic, we could not meet in person, and were forced to use zoom conference calls and email instead.
This ensured that the analytic process was closely embedded to the actual use of NPT. An important limitation of this work, though, is that it was undertaken opportunistically and grew organically. Development of a coding manual thus developed cumulatively, and authors joined the project, in an ad hoc way rather than being designed to do so from the outset. There might have been better ways to do this work, but it turned out to be a strength because it involved a continuous process of problem identi cation and solution.

Conclusion
Coding manuals are useful tools to support analytic work in qualitative research. They reduce cognitive load and at the same time render the assumptions underpinning qualitative analysis transparent and easily shared amongst teams of researchers. In this paper we have done three things. We have shown how a coding manual for the application of concepts proposed by Normalization Process Theory was developed, piloted, and made into a usable product. The coding manual performs two important tasks. It makes the application of NPT simpler for the user, and it links important NPT concepts to the practice of Realist Evaluation methods.
To our knowledge, this is the rst qualitative coding manual for an implementation theory. It has been developed and used over a period of 18 months, and is already being used to inform practical implementation research in the eld. It adds value to qualitative research on the adoption, implementation and sustainment of innovations by providing a stable, workable, set of concepts located in a well-   'Individual intentions: agents' readiness to translate individual beliefs and attitudes into behaviors that are congruent, or not congruent, with system norms and roles. They frame individual motivation to participate in a complex intervention' [9].
Statements de ning the work of implementation.
'By normalization, we mean the work that actors do as they engage with some ensemble of activities (that may include new or changed ways of thinking, acting, and organizing) and by which means it becomes routinely embedded in the matrices of already existing, socially patterned, knowledge and practices ' [8].
Statements about relationships between actors, interventions, and contexts.
'The more tightly coupled intervention components are, the less discretion participants have in resource mobilisation and collective action, and the more they must do adaptive work to ensure intervention integration' [10].
'A complex intervention is disposed to normalization if it equals or improves accountability and con dence within networks' [7].
Statements about mechanisms of action.
'Collective action through which effort is invested in an intervention and its components: participants mobilize skills and resources and make a complex intervention workable. This work frames how participants realize and perform intervention components in practice' [11]. Contexts (patterns of social relations, environments, and events unfolding over time and across settings) in which an intervention can be formulated and plans for enacting it can be put in train [11].
Example: 'The analysis centres on English primary care and in particular on the issue of how healthcare professions are affected by, and in turn affect, the interpretation and adoption of new services. We use the case of the implementation of evidence-based approaches for managing patients with osteoarthritis. This musculoskeletal problem occurs in a high proportion of GP consultations, and is projected to increase due to a rapidly ageing population in the western world' [37].

Adaptive execution
Contexts (patterns of social relations, environments, and events unfolding over time and across settings) in which an intervention can be made workable [11].
Example: 'Huge effort was expended and continues to be required to implement and keep this technology in use. This innovation must be understood both as a computer technology and as a set of practices related to that technology, kept in place by a network of actors in particular contexts. While technologies can be 'made to work' in different settings, successful implementation has been achieved, and will only be maintained, through the efforts of those involved in the speci c settings and if the wider context continues to support the coherence, cognitive participation, and re ective monitoring processes that surround this collective action. Implementation is more than simply putting technologies in place -it requires new resources and considerable effort, perhaps on an on-going basis' [38].

Negotiating capacity
Contexts (patterns of social relations, environments, and events unfolding over time and across settings) in which an intervention can be integrated [11].
Example: 'Aligning IPC guidelines with local clinical context is an essential means to reduce the sense of dissonance and represents a critical step forward towards successful implementation. Some strategies described in the literature to promote alignment include: integration of IPC recommendations within other established programmes; and education and audit interventions acknowledging the positive and negative beliefs of staff on IPC practices [39].

Reframing organizational logics
Contexts (patterns of social relations, environments and events unfolding over time and across settings) that are expected to be changed by an intervention [11].
Example: 'The external and internal partnership building were key and also strategic, so as not to impose ERAS but to co-create it from the ground up. This relational work, as framed in the NPT, is deceptively complex as it involves convincing others that this is a legitimate improvement programme worth participating in without devaluing their current practice and beliefs. The interprofessional and interdepartmental relationships the champion teams established appeared to lay an important foundation for accepting changes and the data reports as meaningful and embedding ERAS into everyday practice' [40]. Coherence-building is collaborative work that makes interventions and their components meaningful. Participants work to understand how intervention components are different from other practices, and they work to make them a coherent proposition for action [11].
Example: 'Coherence was achieved around the CDSS despite local context variation. Across all three sites there was agreement that the CDSS was suitable for the (varied) tasks and that appropriate resources were in place to enable effective implementation, although these varied between settings. There were differences between settings where the CDSS replaced an established system with existing staff and where the service and/or the staff were new and the work of establishing coherence had to be altered to re ect this. It was clear that knowledge, experience and work identities built through doing call-handling work in uenced the coherence of the CDSS for staff in the different settings. What is especially interesting in the wider policy context -where this same CDSS is now being used to support a national '111' urgent care service (…) is that coherence was not just a local 'problem', it was necessarily underpinned by wider understandings and discourses for example about the necessity of rationing and the need to modify caller/patient behaviour and beyond that the very legitimacy of evidence based medicine and the kinds of expert knowledge which underpinned the CDSS' [38] Cognitive participation Cognitive participation forms commitment around an intervention and its components. This establishes its legitimacy and enrolls themselves and others into an implementation process. This work frames how participants become members of a speci c community of practice [11] Example: 'Cognitive participation relates to the work that participants undertake to build up and sustain a community of practice around an intervention. In terms of CST, participants identi ed training as an important factor in generating their own and their colleagues' interest in CST and thus ensuring all stakeholders were involved. Staff were further motivated to continue running the groups within their service through observing the direct bene cial effects of CST on clients' [41].

Collective Action
People work collaboratively to realize, perform, and operationalize and intervention and its components [11].
Example: 'The daily tasks involved in carrying out Point of Care (POC) testing were deciding which tests (if any) to take for each patient when they arrived; communicating this to others; taking the blood; running the tests; examining the results; communicating the results to others; and deciding what action to take accordingly. This work was allocated to different staff according to their skills and availability. Close teamwork appeared key to ensuring that each task was performed by an appropriate person at the necessary time' [42].
Re exive People work collaboratively to appraise the effects of an intervention and its monitoring components, and utilize that knowledge to recon gure social relations and action [34].
Example: 'Data provision by the laboratories proved to be di cult despite the standardized format. The database manager at the central level reported he had to put much effort in getting the data from the system administrator from the laboratories because they did not prioritize data delivery. It was reported by them that saving the data extraction queries, as the research group suggested, for use in the next time period was increasingly helpful in the course of the implementation period. By ne-tuning these queries after each extraction, the quality of the delivered data improved' [43]. Construct Source reference, and example text Sustainment Sustainment activities through which the normalization of an intervention and its embedding in everyday practice are carried forward as continuous translational action [11].
Example: 'At the end of the project period, the pathway was integrated in daily practice in two of the six municipalities. In these municipalities the care pathway was found to have the potential of structuring the provision of home care services and collaboration with the GPs, and serving as a management tool to effect change and improve knowledge and skills. (…) The generic care pathway for elderly patients has a potential of improving follow-up in primary care by meeting professional and managerial needs for improved quality of care, as well as more e cient organisation of home care services. However, implementation of this complex intervention in fulltime running organisations was demanding and required' [44].

Normative restructuring
Changes to norms, rules and resources that happen as a result of working with interventions and their components [10].
Example: 'The rst theme, trusting and embedding new relationships, is a reminder that while locally-led innovation is designed to address local problems, convincing others of its value is core work. This is particularly so when the innovation challenges professional norms and involves changes to traditional delivery models and renegotiation of professional roles (…). In this case, the ndings are consistent with previous research which has indicated that the success of such innovations is dependent on the trust of all involved and the credibility of

Relational restructuring
Changes to the ways that people are organized and relate to each other that happen as a result of working with interventions and their components [44].
Example: 'The CMs became "everyday representatives" for the secondary sector and were responsible for acting as bridge-builders between hospital psychiatry and general practice. Previous research on Nurse Practitioners/ Advanced Nurse Practitioners in general practice (…) has shown that if the clinics are not involved at an early stage and prepared thoroughly for the Nurse Practitioner's arrival, their integration in general practice is hampered. Preparation involves practical issues, a clearly de ned role for the nurse practitioner, and organizational leadership, meaning that the managers of the responsible organization must be involved in the process of de ning and supporting the role (…) The challenges also pointed towards a lack of managerial co-ordination of, and responsibility for, the practical issues associated with the CM's role in general practice. (…) This meant that on many occasions, the CMs had to take on the role of implementation ambassadors assuming responsibility for maintenance of the collaborative care model' [46].

Intervention
How interventions and their components are operationalized, enacted, reproduced, performance over time and across settings. The mobilization of these ensembles may be highly variable [6] Example: 'The bed-monitoring technologies were felt to be useful in helping staff identify patterns in resident behaviour and explore reasons for these behaviours. The bed sensors at Sycamore Lane were capable of recording clinical data such as heart rate, but the manager reported that "it's not something that we use readily", and this functionality was never observed in use during the present study. The location-based system at Conifer Gardens was similarly able to record data, including information about resident mobility activity. This functionality had initially been anticipated as potentially useful for enhancing clinical understanding, however, the Occupational Therapist re ected that the time needed to analyse and interpret these data had been "a job in itself" and thus has been di cult to integrate into daily practice. There were questions about the clinical utility of some of the data, which appeared to become more pronounced when considering the nancial expense of the technology' [47]. 'In order to invest in ERAS individuals needed to be able to differentiate its practices favourably with those enacted preimplementation. This required coherence work in understanding the potential patient bene ts allied to its introduction. Participants provided divergent accounts when they compared ERAS to previous practice. A number of participants asserted that the introduction of ERAS had brought about considerable changes to their day-to-day practice. These changes included positive adjustments in the management of patients and required patients to play a more active role in their own recovery' [40].  [7].
'The rural allied health team indicated that telehealth technology provided 'a whole range of other capabilities', and considered it 'safe and it's appropriate and it's an equivalent, if not better, sort of service that you can provide'. They were committed to the notion that telehealth could balance the unequal access to services across geographical locations, and were keen to pursue innovative ways of using telehealth technologies to allow them to provide complex distant therapy. In contrast to rural and experienced telehealth clinicians who were keen to utilise technology as part of their role and to deal with distance and isolation, urban clinicians with no exposure to telehealth reported more reservations about the safety and suitability of providing rehabilitation through telehealth. They generally felt that telehealth should be reserved for 'people who are more autonomous and more capable and … straightforward', rather than 'real' rehabilitation patients with complex issues. They felt that people who required rehabilitation often require a 'hands on' approach' [56].

Relational
Integration: This refers to the knowledge work that people do to build accountability and maintain con dence in interventions and their components and in each other as they use them [7].
'Enhanced collegial discussion about FV and adherence to the safety measures, such as the home visiting policy and procedures introduced in (…) model, were important for nurses to feel safe and undertake the FV work. As implementation progressed, intervention nurses felt safer than comparison nurses when attending home visits (…). Relationships within teams and with FV services varied across the MCH intervention teams. High workloads, time constraints and a lack of nursing staff or relievers in some centres impacted on the organisation of the FV work at times. The nurse mentor role to provide secondary consultation, linkage to FV services and support for other MCH nurses had varied success. Due to time constraints and the often solo nature of MCH practice, most nurses preferred to discuss clinical issues with a nurse friend or coworker at the time rather than try to contact the designated MOVE nurse mentor, with only 38% of nurses using the nurse mentor role early in the trial. This increased to 52% as time went on. If the nurse was not comfortable speaking and had insu cient time or access to the nurse mentor, then this aspect of the model was lost' [57].
Skill set Workability: This refers to the 'A key theme identi ed in the literature and through this study is the need for more training for practitioners. This includes training both in professional education and continuing allocation and training work that underpins that is built up around interventions and their components as it is operationalised in the real world [7].
educational opportunities for all practitioners. Medical, nursing and allied health education programs need to improve LGBT curriculum content [2,[64][65][66]. Providing education on general terminology, healthcare needs speci c to the transgender population, and practitioners' role in providing healthcare for this population will better prepare new practitioners for serving this community. Increased access to continuing education with LGBT content will help to increase the knowledge and skill of current practitioners. Embedding LGBT content within current programs of continuing education may increase awareness more than having speci c LGBT courses (…). Embedding it in current programs may bring awareness to the concepts and highlight the need for practitioners to seek out more speci c training to address their learning gaps' [58].

Contextual
Integration: This refers to the resource worksupporting interventions and their components through the allocation of different kinds of resources and the execution of protocols, policies and procedures [7].
'Since POs were able to self-select into the pilot, the alignment of PO priorities with participation in a pilot on care management was a good t. The leadership in all POs voiced interest in providing care management to patients within their PO as a means of improving patient outcomes, easing burden on providers of handling complex patients, and to meet health care standards and reimbursement policies such as patient-centered medical home recognition, accountable care, and meaningful use. Therefore, in this study overall organizational support was not found to be variant. Where organizational support emerged as an issue related more to resources and support for the care management program relative to the needs and goals of the program. The most common issue here was not having either enough care managers or enough care manager protected time to do care management for the number of patients needing it. So in well-normalized programs, there was a sense of "rationing" of the care manager. Because the program was being used so much more and there was a capacity constraint at the practice level with the practice-based care manager structure, the practices in these POs voiced more concern about lack of care manager capacity. Lack of resources was evident in other ways such as lack of space for patient visits or access to phone lines to make longer calls' [59].

Re exive Monitoring
Systematization: participants in interventions and their components may seek to determine how effective and useful it is for them and for others, and this involves the work of collecting structured information in a 'Feedback was never provided to staff on the effect of the AKI ealert "I haven't had any feedback since the new version (of the AKI e-alert) went in actually(...) I don't know whether there is a formal mechanism for that getting to anyone"' [60]. ONLINESUPPLEMENTARYMATERIAL.docx