After the preliminary development of the codebook (Phase 1), each phase of the development process informed revisions to the results of the previous phase. We thus describe below the results of all four phases for each of the i-PARIHS constructs and to the Codebook more generally.
Innovation
Because the characteristics of the Innovation construct were well described and/or grounded in theory in the core materials, subcode labels were based on characteristics identified by i-PARIHS developers with one exception. Based on literature and our experience, in Phase 1 we identified complexity as an important innovation characteristic and added a Complexity subcode to the codebook. In Phases 2 and 3, the expert panel and i-PARIHS developers concurred. Definitions of the Innovation subcodes were modified throughout all phases of development (see Table 1 for the names of the characteristics of the Innovation construct, the subcode labels/definitions, and modifications made during the development process).
Table 1
i-PARIHS Innovation construct characteristics, Innovation subcodes, and subcode definitions
Characteristics of the Innovation[7] | Innovation Subcodes | Innovation Subcode Definitions |
Underlying knowledge sources | Evidence: Research/ published guidelines | Presence or absence of findings from quantitative, qualitative, or mixed methods studies, as well as literature reviews, that show the efficacy, effectiveness, or other evidence for the innovation (e.g., its utility or acceptability). Also includes discussion about published guideline recommendations. |
Evidence: Clinical experience | Presence or absence of professional knowledge of or experience with the innovation which is embedded in or based upon clinical practice and is often tacit and intuitive. |
Evidence: Patient needs, preferences, and experiences | Presence or absence of patients' personal knowledge of and experiences with an innovation, including current or previous experiences with the innovation, the extent to which the innovation met/meets their needs and preferences. |
Evidence: Local practice information | Presence or absence of sources of evidence related to the innovation from the context of care, including, but not limited to, audit and performance data, report cards, progress reports, fidelity ratings, quality improvement and program evaluation data, and financial data/implications. |
Clarity | Clarity | Degree to which the innovation is understood, including specifics of what components of the innovation must be implemented (for fidelity) and/or what can be adapted or changed. |
Degree of fit | Degree of fit | Extent to which the innovation is compatible with 1) the values and norms of individuals implementing the innovation and/or 2) the existing practices and operations of the setting, including workflows, processes, roles, policies, etc. |
Degree of novelty | Degree of novelty | Extent to which the innovation or components of the innovation is/are new to or different from individuals’ current thinking, ways of relating to and interacting with each other, or practice. |
Usability | Usability | Degree of ease or difficulty with which the innovation can be, is, or was adopted and/or used, including the accessibility and availability of information/tools/guides regarding how to adopt/use the innovation. |
Relative advantage | Relative advantage | Comparison of the innovation with an existing program, practice, or alternative solution and the degree to which one is perceived and/or objectively observed to be more advantageous than the other in meeting patient, clinical, and/or organizational goals and needs. |
Trialability | Trialability | Whether the innovation can be or has been tested (or experimented with) on a small scale, including discussion about whether it is possible or not possible to conduct a pilot. |
Observable results | Observable results | Degree to which positive results/benefits of an innovation are directly observable/visible. |
N/A | Complexity1 | Ways in which the innovation itself is simple or complicated. Discussion may be about the number of innovation components and/or interaction between them, the number and difficulty of behaviors that those delivering or receiving the innovation must perform, the number of groups or organizational levels targeted by the innovation, and/or the number and variability of outcomes. |
1Subcode added during the Codebook development process |
Recipients
There were several challenges to identifying subcodes for the Recipients construct. Although the Innovation and Context constructs were part of the original PARIHS framework, the Recipients construct was new in i-PARIHS and not as well defined conceptually as the others had been. Also, the Recipients construct includes both individuals and teams, making it harder to differentiate some Recipient characteristics from characteristics of the Context. We thus made a number of modifications to subcode labels initially named after the characteristics of recipients identified by developers. First, although i-PARIHS identified motivation, values and beliefs, and goals as distinct Recipient characteristics, we felt it would be hard to parse these out in qualitative material. We therefore opted to combine these characteristics into one subcode, Personal attributes; individuals and teams collecting data specifically about the motivation, values and beliefs, and/or goals of study participants can add these for their project-specific codebook. Second, based on i-PARIHS developers’ feedback, we modified the labels for two Context characteristics (time, resources, and support and existing networks) to focus on how the presence or absence of these characteristics affect recipients. Third, at the recommendation of the expert panel, we added the word autonomy to the Power and authority subcode label to differentiate the ability to influence the actions of others from recipients’ own actions. Fourth, we did not include local opinion leaders as a subcode because this characteristic delineates a change agent role that may not be applicable for a specific implementation effort, and other types of change agents (e.g., champions, QI team) may be involved. In the codebook, we suggest that subcodes be added for local change agents if capturing change agent roles is important for a study. Finally, during the piloting process, we added a subcode for General attitude to account for statements study participants made about what they thought or felt about the innovation generally, e.g., they liked it or enjoyed using it. Definitions of the Recipients subcodes were modified across the phases of development (see Table 2 for the names of the characteristics of the Recipients construct, the subcode labels/definitions, and modifications made during the development process).
Table 2
I-PARIHS Recipients construct characteristics, Recipients subcodes, and subcode definitions
Characteristics of Recipients [7] | Recipients Subcodes | Recipients Subcode Definitions |
Motivation | Personal attributes1 | Personal traits or characteristics of any recipient(s). This can include tolerance of ambiguity, general intellectual ability, motivation to change, values, goals, competence, innovativeness, seniority or tenure, learning style, being self-aware, reliable, other personality traits, etc. |
Values and beliefs |
Goals |
Skills and knowledge | Skills and knowledge | What recipients know and understand about the innovation and/or whether recipients have the ability/expertise to perform the tasks required for implementation. |
Time, resources, support | How time, resources, and support affect recipients2 | How the presence or absence of sufficient time, resources and support is affecting/affected by the ability of a specific recipient (individual or team) to implement or receive the innovation. |
Collaboration and teamwork | Collaboration and teamwork | Group processes and team-related issues, including presence or absence of interprofessional collaboration, communication, and teamwork within teams, between teams and managers, and/or between individuals who work together toward a common goal; team building activities; areas of disagreement/conflict between team members or stakeholder groups; and available conflict management/resolution strategies. |
3Existing networks | How existing networks affect recipients2 | How formal or informal networks and/or relationships is affecting/affected the ability and/or motivation of a specific recipient (individual or team) to implement or receive the innovation. Networks/relationships may be professional, task-related, or social and may occur at any level or across levels of the context. Examples of formal networks/relationships include memberships, listservs, communities of practice, learning communities, learning collaboratives, practice-based research networks, etc. Examples of informal networks/relationships include social practices such as getting together with colleagues for lunch; regular hallway conversations with certain colleagues; friendships; ‘huddles” among clinical providers/teams; etc. |
Power and authority | Power, authority, and autonomy2 | The capacity or ability of an individual or team to direct or influence their own actions and/or the actions of others. Power and/or authority may be derived from organizational role (e.g., leadership), professional role (e.g., physician, nurse, etc.), expertise, relationships to powerful others, and/or ability to offer or deny rewards or use the threat of force to gain compliance. |
Presence of boundaries | Presence of boundaries | Experience with boundaries between groups (e.g., professions/occupations, work units, service lines, roles) that influence implementation. Examples include discussion about the lack of communication between primary care and mental health providers and how clinicians’ scopes of practice or discipline/unit-specific restrictions limit provision of/access to services. |
N/A | General attitude3 | How the interview participant thinks or feels about the innovation generally, e.g., that they like it or don’t like it, it is helpful, or they enjoy using it. |
Local opinion leaders4 | N/A | |
1Motivation, Values and beliefs, and Goals characteristics of Recipients combined under the subcode label Personal attitudes |
2Names of Recipients characteristics modified for sub-code label |
3Subcode added during the development process |
4Recipients characteristic not included as a subcode; codebook includes recommendations for using this and/or subcodes for other internal change agents as appropriate |
Context
There were also a number of challenges to identifying subcodes for the Context construct. First, i-PARIHS developers identified characteristics of context as unique to particular levels (e.g., policy drivers and priorities are considered characteristics of the outer context). However, in our collective experience, most characteristics of context can/might occur across organizational levels (e.g., local policies and priorities in the inner context might also impact implementation). To address this challenge, we developed two sets of context subcodes, one for characteristics of context and the other for levels of the implementation context (e.g., local, organizational; see Additional file 2 for the full codebook). To develop the context characteristics subcodes, we combined those delineated by the developers if they were similar in content but broken out into different organizational levels. For example, we combined i-PARIHS’s formal and informal leadership support (a characteristic of the inner/local context) and leadership and senior management support (a characteristic of the inner/organizational context) into a single Leadership support subcode (see Table 3 for names of the characteristics of the i-PARIHS Context construct, the subcode labels/definitions, and modifications made during the development process). In the codebook, we recommend that characteristics of context be co-coded with the level(s) at which they occur. For example, to code discussion of the influence of a broad organizational policy affecting multiple units in a large healthcare organization, coders would apply both the Policies and priorities code for the characteristic of the context and the Inner/Organizational code to identify the level of context.
Table 3
i-PARIHS Context construct characteristics, Context subcodes, and definitions
Characteristics of Context by Level [7] | Context Subcodes | Context Subcode Definitions |
Inner/Local: Formal and Informal Leadership Support Inner/Organizational: Leadership and Senior Management Support | Leadership Support | Characteristics or behaviors of formal or informal leaders that either support or interfere with the implementation or sustainment of the innovation. This could include discussion of leadership style, relationship building, role modeling, educating, planning-organizing-aligning, and communicating, encouraging, empowering, and/or concrete support, e.g., protected time, space, resources, training, etc. Alternatively, discussion may be about how leaders fail to provide such support or exhibit negative attitudes/behaviors toward innovation implementation/sustainment. |
Inner/Local: Culture Inner/Organizational: Culture | Culture and climate1 | Culture of the organization or organizational unit, including prevailing norms, values, beliefs, meanings, understandings, philosophies, way of life, and assumptions. Also includes discussion about the current climate of the organization or organizational unit, e.g., staff empowerment, morale, attitudes, job satisfaction, burnout, etc., as well as the degree of stability/instability of the environment in which implementation is occurring/will occur. |
Inner/Local: Past experiences w/ innovation/change Inner/Organizational: History of Innovations and change | History of innovation and change | How the organization or organizational unit has historically experienced, undertaken, and responded to past change initiatives and/or innovations. |
Inner/Local: Evaluation and Feedback processes | Evaluation, monitoring, and feedback1 | How the organization or organizational unit collects, assesses, monitors and disseminates data/information about clinical processes and outcomes, economic outcomes, user experiences, clinical performance, etc. Also includes discussion about data sources (e.g., data dashboards, medical records) and ways in which results are fed back to and used by individuals, teams, and services (e.g., through presentations and/or formal reports). This information may be used, e.g., to understand current ways of working or to improve processes. |
Inner/Organizational: Organizational Priorities Outer: Policy Drivers and Priorities and Regulatory Frameworks | Policies and priorities1 (includes mandates) | Organizational policies, policy drivers, mandates, and/or priorities; whether/how these are related to/support/hinder the innovation and/or its implementation; and the changes required. Policies are the decisions, plans, and actions that an organization, organizational unit, state or country take to achieve specific goals. They include statements of what needs to happen and how (e.g., legislation enacted by a government, regulations or rules issued to carry out the intent of laws or of regulatory bodies, regulatory frameworks or models for enacting regulations, and organizational policies and procedures). Policy drivers are forces that influence policy decisions, e.g., serious problems, i.e., high rates of suicide; legal or ethical concerns, i.e., lack of equity; and crisis events, i.e., hurricanes and forest fires. Mandates are formal orders/commands/requirements and may be a component of written policies. Organizational priorities are identified areas of focus, e.g., improving access to care and reducing medical errors in healthcare settings. |
Inner/Organizational: Learning Networks Outer: Interorganizational networks and relationships | Networks and relationships | Formal or informal networks and/or relationships that may be/have been leveraged to support or hinder implementation. Networks/relationships may be professional, task-related, or social and may occur at any level or across levels of the context. Examples of formal networks/relationships include memberships, listservs, communities of practice, learning communities, learning collaboratives, practice-based research networks, etc. Examples of informal networks/relationships include social practices such as getting together with colleagues for lunch; regular hallway conversations with certain colleagues; friendships; ‘huddles” among clinical providers/teams; etc. |
Inner/Organizational: Structures and Systems | Structures and systems | Formal and informal ways in which the organization or organizational unit is structured and managed and/or its processes for accomplishing work. Examples of structure include authority hierarchies (e.g., chain of command), service lines, matrices, specialized or functional units or departments, inter-/multi-disciplinary teams and task forces, and decision-making levels represented in organizational charts. Although structure and systems are not always distinct, systems generally are related to organizational routines and processes, e.g., for information sharing, learning, workflow, IT, etc. |
Inner/Organizational: Absorptive capacity | Absorptive capacity | How the organization or organizational unit (e.g., department or clinic) identifies, acquires, assimilates, transforms, and/or applies new, valuable knowledge (e.g., evidence, Guidelines, best practices). This includes analyzing, processing, interpreting, understanding, combining with existing knowledge, and applying/incorporating new knowledge into organizational competencies and routines. |
Outer: Incentives and Mandates | Incentives and rewards1 | Mechanisms/strategies that motivate/encourage/reinforce or that deter/discourage the implementation of the innovation and proposed changes, including incentives/rewards (e.g., casual dress day; pizza day; time off; recognition; financial incentives, i.e., pay for performance; etc.) and disincentives (e.g., negative performance reviews, reprimands, regulatory requirements, etc.). |
N/A | Infrastructure, resources, and support2 | Presence or absence of infrastructure (e.g., facilities, space, equipment, transportation), resources (e.g., funding, staffing, time, education, skills training, materials) and/or support (e.g., supervisory, clerical) for implementing the innovation. |
N/A | Political factors and dynamics2 | Organizational politics, i.e., how individuals or groups use political strategies to gain/use power and/or social influence in order to positively or negatively affect decisions and activities related to the adoption or implementation of an innovation. For example, they might create conflict, form alliances, bargain, use stalling tactics, discredit others, or compromise. If relevant, this code also includes discussion about the larger political environment (e.g., state or national government) and prevailing political ideology (e.g., nationalism, populism) as it relates to innovation implementation. |
Inner/Local: Mechanisms for embedding change3 | N/A | |
Outer: Environmental Stability3 | N/A | |
1Context characteristics modified for sub-code label |
2Subcode added during the development process |
3Context characteristic not given a subcode label; codebook includes recommendations for identifying these two higher level concepts during the data analysis process |
Second, the organizational scientist we consulted felt strongly that there were characteristics of context missing in the framework. To address this concern, we modified some subcodes and added several others. We modified the initial Culture subcode by adding the word ‘climate’; the initial Evaluation and feedback subcode by adding the word ‘monitoring’; and the initial Incentives subcode by adding the word ‘rewards. For conceptual clarity, we added the word ‘mandates’ to the initial Policies and priorities subcode. We also added two new subcodes: Infrastructure, resources, and support and Political factors and dynamics. As with the other constructs, we modified definitions of the context subcodes across the phases of development.
Finally, several characteristics of Context identified by i-PARIHS, mechanisms for embedding change and environmental stability are higher level concepts that can best be identified during the analysis process. For example, i-PARIHS developers suggest that mechanisms for change may include a variety of activities that fall under other context characteristics, such as regular team meetings and performance review systems (examples of Structures and systems) or audit and feedback processes (included in Evaluation, monitoring, and feedback). By co-coding characteristics of context subcodes with a generic enablers of implementation subcode, data analysts can identify mechanisms for embedding change during the analysis process. Similarly, environmental stability is related to multiple contextual events/circumstances, e.g., changes or lack of changes in structures and systems, leadership, or policies and procedures. We felt that ultimately determining environmental stability calls for judgement based on a full understanding of the context at multiple levels. In the codebook, we suggest that these two characteristics of context be explored during later stages of data analysis.
Facilitation Activities
We based the initial Facilitation Activities subcodes on 32 activities previously identified and defined [33, 34]; and we clustered these into 10 higher order facilitation activity subcodes. Based on expert panel recommendations, we standardized subcode labels, (e.g., Providing clinical education rather than Clinical education); made significant changes to many of the individual subcode definitions; and made minor changes to the clustered subcode definitions. See Table 4 for clustered subcodes and final definitions; see Additional File 2 for the 32 individual subcodes and definitions.
Table 4
Facilitation activities clustered subcodes, definitions, and individual subcodes
Clustered Subcodes | Facilitation Activities Clustered Subcode Definitions and Individual Subcodes1 |
Providing education/information | Educating stakeholders on clinical skills, the conduct of innovation marketing and/or organizational change processes and providing information to promote/publicize the innovation. This includes: 1) the content of education/information (e.g., information about the innovation and evidence for it, reasons for change, potential outcomes, clinical knowledge/skills needed, etc.); and/or 2) the process of providing education/information (e.g., teaching, training, mentoring, coaching, supervision, experiential/active learning, etc.). This is a cluster code that can be sub-coded with the following activity codes: Providing education on clinical skills, Providing education on marketing, Providing education on organizational change, and Marketing. |
Collecting data/providing feedback | Collecting data and other information to 1) assess and understand the local context, baseline performance, and implementation barriers/enablers; 2) collect/monitor implementation activities, progress, and outcomes; and 3) provide stakeholders with feedback on data and updates on implementation activities and relevant professional or system-level information. This is a cluster code which can be sub-coded with the following activity codes: Conducting ongoing monitoring of innovation implementation, Data collection to assess context and baseline performance, and Providing updates and feedback. |
Building relationships, teams, and networks | Engaging and building relationships with stakeholders, seeking their participation and buy-in, overcoming resistance to change, managing groups and team processes (including creating an atmosphere of mutual respect, empowering group members, and building relationships between them), and fostering stakeholder networking with peers and external experts/organizations. This is a cluster code that can be sub-coded with the following activity codes: Engaging stakeholders, obtaining buy-in, Fostering networking with experts, Fostering peer networking, Managing group/team processes, and Overcoming resistance to change. |
Enabling/fostering change | Encouraging, promoting and helping to support changes in the organization, including by interceding and liaising with leadership or other stakeholders and assisting with the development of strategies and policies. The target of change efforts may be the organizational structure or culture or the target of change may not be specified but the methods of fostering change are specified. (For example, discussion may be about assisting stakeholders with conducting quality improvement activities, helping them build capacity for sustainment, or guiding and supporting them during the implementation process.) This is a cluster code that can be sub-coded with the following activity codes: Fostering organizational change: cultural, Fostering organizational change: structural, Fostering change/unspecified, Interceding/liaising with others, and Strategy/policy development. |
Problem identification and resolution | Conducting or helping stakeholders 1) identify, become aware of, or clarify implementation challenges/barriers/problems and/or 2) generate potential solutions/countermeasures or select the one(s) most likely to address/solve implementation challenges/barriers/problems. This is a cluster code and can be sub-coded with the following activity codes: Problem identification, Problem solving. |
Planning/preparing for implementation | Helping stakeholders develop or refine Action/Implementation plans, come to consensus, adapt the innovation to the local context (structure, staffing, culture, and other initiatives), share a vision for change, and identify goals and priorities. This is a cluster code that can be sub-coded with the following activity codes: Action/implementation planning, Adapting innovation to local context, Developing shared vision/consensus building, and Goal/priority setting. |
Helping to define, identify, and fill stakeholder roles | Helping to identify and select local change agents (e.g., facilitators, QI team members, local champions, opinion leaders) and/or hire innovation providers, as well as establish, describe/clarify, and/or allocate facilitator and stakeholder roles and responsibilities. This is a cluster code and can be sub-coded with the following activity codes: Describing/clarifying roles and responsibilities, Helping to hire clinical program staff, Helping identify/select local change agents. |
Providing administrative/technical support | Conducting administrative tasks that support the operationalization of implementation activities and providing technical support, i.e., practical help and assistance to support implementation. Examples of administrative tasks include arranging calls, meetings, and implementation site visits; developing/preparing and disseminating minutes/reports and educational/marketing materials; and organizing innovation provider training. Examples of technical support include providing tools/sample materials; working with site stakeholders to co-create tools/materials, identifying/providing information about available resources for implementation, and working with relevant stakeholders to ensure that Information Technology (IT) systems accurately capture innovation activity and support implementation. This is a cluster code and may be sub-coded with the following activity codes: Administrative tasks, Technical support. |
Using interpersonal skills to create a supportive environment | Using positive, supportive behaviors and communications to create an open, supportive, and trusting environment conducive to change, including being generally helpful and available, communicating regularly, acknowledging ideas and efforts and celebrating achievements/success. This code also includes selectively reducing the level of facilitation support, including positive supportive behaviors, in order to allow the transfer of facilitation roles to site stakeholders. This is a cluster code that can be sub-coded with the following activity codes: Providing support, Pulling back/transferring roles. |
Obtaining/disseminating innovation or facilitation knowledge | Obtaining information about/developing skills needed for facilitating implementation of the innovation or fostering dissemination of knowledge about the innovation or facilitation other than at the implementation site(s). Facilitators may foster dissemination by attending, presenting at or organizing non-local meetings or by assisting with dissemination at sites not receiving facilitation. This is a cluster code that can be sub-coded with the following activity codes: Attending, presenting at, and/or organizing non-local meetings; Fostering spread of clinical innovation/facilitation methods; Obtaining training/continuing education. |
1See Additional File 2 for definitions of individual subcodes |
Instructions and Explanatory Material
In addition to i-PARIHS construct codes, subcodes for the characteristics of constructs, and definitions, the codebook (see Additional File 2) includes instructions for the application of individual codes and for the use of the codebook more generally. We initially developed instructions for applying some of the individual codes during Phase 1 and then modified them based on feedback from the expert panel, i-PARIHS experts, and the piloting process. Instructions expand on subcode definitions by providing inclusion and/or exclusion criteria, as well as guidance for differentiating between subcodes and/or co-coding when appropriate or subcodes are difficult to differentiate.
We also developed instructions for the use of the codebook, as well as explanatory material. For example, we encourage users to adapt the codebook to their project and to create their own examples. We also recommend that they co-code i-PARIHS subcodes, as applicable, to indicate factors that may impede implementation (barriers) and/or factors that may enhance or improve implementation (enablers). We added instructions and explanatory material based on feedback across the phases of development. For example, in response to i-PARIHS developers’ recommendations, we clarified that we developed the codebook based on our interpretation of the i-PARIHS framework.