Theory-Based Development of IPAF
Consistent with calls to apply theory to the development of interventions and implementation strategies [5, 6, 15], we sought to apply theories to develop a novel A&F strategy that promotes implementation of individual target behaviors, and to select relevant evaluation measures. One approach is to use the comprehensive Theoretical Domains Framework (TDF), which provides a lens through which to view behavior [16, 17]. However, as the TDF authors state, “the TDF is a broad theoretical framework rather than a theory and it does not include testable relationships [18] .” Among the 33 theories that informed TDF, we chose two theories for our purposes of clinic intervention development and evaluation. We chose to use middle-range theories because they do offer testable relationships and are readily applied in practice [19]. Specifically we chose Self-Regulation Theory (SRT) and Self-Determination Theory (SDT), which were classified as action and motivation theories, respectively, in the development of TDF [17]. We did not choose an organization theory because IPAF targets individual behaviors. We selected SRT and SDT, in particular, because these theories have been supported by evidence and for their clarity, utility, and parsimony [20, 21]. The following two sections describe how SRT and SDT theories guided our A&F development in a complementary manner: SRT involves identifying what goals to achieve and SDT involves how to interact to achieve goals.
How Self-Regulation Theory Guided IPAF Development
We applied SRT to the development of IPAF as an implementation strategy, consistent with recommendations by experts in SRT [22, 23] and A&F [4, 5]. As an illustration, assume a clinic goal is to increase rates of BP re-measurements to 80% among patients whose initial BPs were high. This clinic goal provides a standard against which we could compare staff’s rates of repeat BPs in an audit. Individual staff can iteratively compare their current behaviors against the clinic goal, whereby discrepancy can motivate behavior change.
The concept of discrepancy in SRT is critical to A&F at the individual level, as shown in Figure 1. When feedback to individuals indicates that their behaviors are inconsistent with reference points, then people likely experience dissatisfaction [6, 22, 23]. In the context of our applications, if staff receive feedback that their behaviors are inconsistent with intervention goals that they agreed to support, then they likely would experience discrepancy. As a result, they would be motivated to improve their target behaviors, have future behaviors align with the intervention goals, and experience less discrepancy in the future.
Consistent with SRT, specific behavioral strategies are helpful in adopting new behaviors. These strategies include barriers identification, goal setting, and action planning [24], as shown in Table 3. These are highly relevant to staff’s processes of adopting target behaviors, as demonstrated in Figure 2.
We also more broadly applied SRT to the interventions and overall implementation approach, as shown in Figure 1. Using SRT, researchers can describe, explain, and predict how people manage (i.e., regulate) themselves to reach their goals over time. At the intervention level, SRT applies to our choosing reference points for the patient population, that is, high BP or readiness to quit tobacco. At the staff and patient level, SRT applies to individuals; people need clear reference points against which to evaluate risk factors. In summary, SRT involved what goals to achieve.
How Self-Determination Theory Guided IPAF Development
We applied SDT, a theory regarding motivation and behavior [25-27], to guide the development of IPAF, as shown in Table 3. According to SDT, all people have three, inherent psychological needs relevant to behavior: relatedness, autonomy (i.e., choice), and perceived confidence (i.e., self-efficacy) [25, 27]. When these needs are met, people are more motivated to engage in relevant behaviors [25, 27]. Randomized, controlled studies, based on SDT, have demonstrated improvements in work behaviors [25, 27-29]. If staff’s psychological needs were met, then staff would be more motivated to adopt target behaviors than if their psychological needs were ignored or thwarted. Accordingly, we designed IPAF to meet staff’s psychological needs during individual feedback sessions. In summary, SDT involved how to interact to achieve goals.
Application of IPAF as an Implementation Strategy
Design
We evaluated implementation outcomes associated with IPAF in the context of a broader pre-, post-, quasi-experimental evaluation of CVD risk-reduction interventions [30]. That evaluation followed a Hybrid Type 2 effectiveness-implementation design [31], attending to both intervention and implementation outcomes. We reported elsewhere the intervention outcomes: a) timely patient follow-up in primary care after high BPs and b) referrals to the tobacco quit line after assessing readiness to quit [30, 32, 33]. In this paper, we evaluated the implementation outcomes of feasibility, acceptability, fidelity, and adoption [9]. focusing on measures relevant to IPAF. We delivered IPAF sessions to individuals synchronously (in-person or by phone) for six months after the beginning of implementation, and later, asynchronously by email for over 24 months. Table 1 shows the components of the CVD-risk reduction interventions and the implementation package within which IPAF was used to provide feedback.
Setting and Sample
We evaluated IPAF in a total of eight implementations, representing two interventions (BP Connect and Quit Connect) [30, 33-36] in four separate rheumatology clinics in two US health systems. Clinics A, B, and C were in a large, suburban, academic, multi-specialty practice; clinic D was a community clinic. Rheumatology clinics offer an ideal setting and specialty population to evaluate A&F with frontline staff as a strategy to implement CVD risk-reduction [10, 37, 38]. Our IPAF participants were all medical assistants and nurses who performed pre-visit rooming (i.e., vital signs, patient history, etc.) at the clinics. We collected mixed-methods data including their responses to questionnaires, (EHR) data, and team records such as IPAF worksheets.
Context
The components of our interventions and implementation package are shown in Table 1. This paper focuses on the development and evaluation of IPAF, the implementation component for providing feedback to staff. IPAF was used to improve staff’s target behaviors with two CVD risk-reduction interventions: BP Connect for high BP [30, 34] and Quit Connect for tobacco use [33, 35]. With a Check-Advise-Connect structure for both interventions, the target behaviors were to Check for addressable risk factors, confirming high BPs or readiness to quit tobacco; Advise patients on CVD risk, and Connect patients to relevant resources. Connecting consisted of offering follow-up arrangements for BP appointments with primary care or for quit-line phone calls for tobacco cessation counseling.
Pre-IPAF Staff Education
We held one-hour educational sessions with staff in small groups at the beginning of each implementation. We explained intervention rationale, principles, and components. We shared relevant evidence to address BP and tobacco, encouraged interactive discussion, and provided scenarios for staff role-plays regarding the Check-Advise-Connect behaviors. These staff had not previously had responsibilities for confirming or addressing CVD risk factors with patients (i.e., BP level or readiness to quit tobacco) or for referring patients to resources (i.e., primary care or quit line). The interactive educational sessions concluded with staff demonstrating mastery of role-play dialogue and navigation of the EHR, and receiving information about the monthly, individual IPAF feedback they would receive.
Interactive Participatory Audit & Feedback
We describe IPAF as interactive and participatory because our theory-based IPAF tool is a semi-structured worksheet that guides IPAF facilitators to address staff’s psychological needs in a collaborative approach. Facilitators systematically invite staff to interactively discuss barriers and action planning in order to improve their target behaviors. IPAF sessions consisted of three theory-based components:
- Providing feedback to individual staff about their actual rates of target behaviors and directing their attention to the ideal target behaviors for the intervention, based on SRT [22, 23].
- Interactive, one-on-one discussions of staff’s experiences, including barriers and goals for target behaviors, while simultaneously supporting staff’s psychological needs, based on SDT [25-27].
- Eliciting action plans with staff about how they could improve rates of target behaviors, based on SRT and evidence for behavior change strategies [24].
Consistent with best practices from the most recent Cochrane review of A&F [5], we delivered feedback monthly, individually, face-to-face when possible, by a respected colleague (not a supervisor), to improve staff’s target behaviors. The source of feedback (i.e., A&F facilitator) in clinics A, B, and C was a physician known to staff, a leader in the settings, not a direct supervisor of staff, and the project’s principal investigator (CB). In clinic D, the facilitator was a nurse researcher from another organization, with expertise in supporting nurses, known to staff only from engagement activities (AGB). The context for feedback with individual staff was synchronous for the first six months of implementation, in-person for clinics A, B, and C and by phone for clinic D. The IPAF facilitator met with individuals for up to 10 minutes, privately in a clinic room or by phone, at a mutually agreed time. After the first six months of each implementation, we shared feedback asynchronously by email, along with questions for staff to share their barriers, goals, and action steps regarding target behaviors. Staff sent their responses and goals to the facilitator by email. The frequency of feedback was monthly amounting to at least four synchronous sessions per individual between months one and six of each implementation, and over 375 monthly emails for up to four years thereafter (2016-2019).
Facilitators and participants collaboratively followed the IPAF tool as a guide for IPAF during the synchronous IPAF sessions, shown in Figure 2. The purpose of the tool was to support fidelity of delivery and receipt [8, 9]. Rather than being a rigid script, the tool consisted of a worksheet that was semi-structured to guide flexible discussions. It included what topics to address with staff, based on SRT, and how to support staff’s psychological needs, based on SDT. Table 3 presents the concepts, theories, and rationale for the IPAF components included in the tool.
Feedback sessions addressed the SRT concepts reflected in Figure 1. Guided by the IPAF tool (Figure 2), sessions addressed: a) individuals’ rates of target behaviors, b) range of peers’ rates of target behaviors, grouped by clinic; and c) the desired personal and organizational goals to minimize staff’s perceived discrepancy between their rates and intervention goals. The IPAF facilitator explained the altruistic, long-term goal of controlling CVD in the rheumatology population, to highlight potential discrepancy between actual and desired behaviors and stimulate staff motivation.
Feedback sessions also addressed SDT concepts [26]. Guided by the IPAF tool (Figure 2), the IPAF facilitator explained that the intent was to be collaborative, not judgmental. To respect individuals’ autonomy, the facilitator offered choices about the order of discussion topics, starting with either feedback on individual-level data or discussing how the individuals thought the intervention was going. The facilitator elicited from individuals their barriers to engaging in target behaviors, possible solutions, goals for target behaviors, and action steps for the upcoming month.
Evaluation of Implementation Outcomes Associated with IPAF
Data Collection
Feasibility
We evaluated feasibility by whether we were able to collect rates of individual staff’s target behaviors. Staff were to document these behaviors in new EHR data fields to enable monthly audits. Information technology staff were to report rates of these behaviors monthly to the implementation team. Additionally, we evaluated the degree of staff participation in IPAF sessions, based on team records including notes from facilitators and debriefing sessions facilitators had with other team members after each IPAF session. We assessed both staff’s attendance at feedback sessions and their engagement in planned steps for behavior change during these sessions.
Acceptability
We evaluated staff’s opinions with selected items from a 22-item written questionnaire. Staff answered anonymously at month six. Consistent with SDT concepts, participants reported the degree to which the project team listened to them, responded to their ideas, flexibly tailored protocols, addressed barriers, and collaborated in planning. Participants responded on a 5-point scale from 1, “not at all” to 5, “extremely”. We also asked staff for comments at the end of the questionnaires. Finally, team records, including facilitator notes and post-session debriefings, documented the degree of participants’ engagement during the one-to-one feedback sessions.
Fidelity
We evaluated fidelity based on monthly IPAF worksheets and staff questionnaire responses. Completion of all sections of the IPAF worksheet for each session indicated fidelity of delivery. We evaluated fidelity of receipt with relevant questionnaire items, including feedback based on SDT and SRT. These included the extent to which participants felt respected by staff, found problem-solving helpful, and experienced goal setting as motivational during the feedback sessions.
Adoption
We measured staff’s rates of target behaviors, consistent with the Check-Advise-Connect structure and SDT/SRT reinforced A&F for the two interventions. We defined adoption of the Check behavior as the rate of confirming the CVD risk factor: re-measuring high BPs or asking about readiness to quit tobacco. We defined adoption of the Connect behavior as the rate of offering follow-up: appointments with primary care for high BP, or electronic referrals for counseling calls from the tobacco quit line. Moreover, in addition to actual rates, our theories suggest that intentions and perceived confidence can predict behavior. On the monthly IPAF worksheets, we evaluated staff’s goals for future rates of target behaviors to reflect intentions at the first and last interactive session. In questionnaires, we asked staff to rate their perceived confidence in addressing CVD risk factors pre- and post-intervention, on a 5-point scale, retrospectively for clinics A, B, and C and prospectively for clinic D.
Analyses
We generated descriptive statistics, including frequencies and standard deviations, for responses to questionnaire items and target behaviors. Paired t-tests were used to compare staff’s pre- and post-questionnaire responses.
Protection of Human Participants
Our project received approval through the University of Wisconsin-Madison’s Health Sciences Institutional Review Board and the Gundersen Health System’s Institutional Review Board. According to policies covering research activities at both institutions, our project met exemption criteria for operational improvement activities, with permission to publish. We summarized participants’ data in aggregate and did not share individual audits with the supervisors of participants.