Fidelity methods for the REACH VA dementia caregiving implementation implemented in the VA health care system and the REACH II randomized clinical trial were identified and compared. The NIH treatment fidelity conceptual framework was used, which includes design, provider training, delivery, receipt, and enactment. Table 1 shows the NIH framework treatment fidelity strategies, using the strategies developed to assess the fidelity of the REACH I interventions that formed the basis for REACH II.10
To translate these strategies into practice, for each REACH VA session, core activities that measured delivery, receipt, and enactment were identified. Delivery, receipt, and enactment treatment fidelity data were then examined for a sample of 293 caregivers of veterans with dementia who received the REACH VA intervention from June 2012 to September 2014. This sample was pulled for an economic analysis of Veteran healthcare costs after caregiver participation in REACH. There were 73 VA staff nationwide who served as interventionists for the sample. REACH VA Program Documentation Forms for each session were scored for the number of delivery, receipt, and enactment core activities that should occur as specified in the protocol as part of the intervention. Additional activities could occur but were not counted. For example, during Session 2, safety material is presented. This is a core activity that must happen and was counted as present or absent. However, addressing a safety alert is an optional activity that might or might not occur, dependent on whether an alert was identified; this activity was not counted. Percentage of activities completed was computed for each session and across all sessions. Dosage, represented by the number of sessions caregivers completed, was also calculated.
Results
REACH Treatment Fidelity – Design. Treatment design encompasses decisions that are made as the treatment is developed that can facilitate or hinder fidelity. One important principle of treatment design is that the intervention’s components are clearly linked to the theoretical framework.2,4
The REACH intervention’s ability to reduce caregiving stress may best be understood through stress/health process theory. Caregivers experience stress if they perceive that the demands placed on them are greater than their resources and capacity to manage those demands.11 The Lazarus and Launier stress model12 expands this concept to focus on actions that caregivers can take to cope with stress. Effective coping depends in part on information and skills directed toward diminishing, tolerating, or mastering situational demands and cognitive and emotional responses. A major component of REACH is problem solving to manage caregiver concerns and patient concerns causing stress and burden. Caregivers are also taught strategies to manage stress and cognitive restructuring skills to reduce distress over behaviors and circumstances not amenable to change.13 The relationship of the REACH intervention components to the stress health process model is shown in Figure 1.
REACH Treatment Fidelity – Provider Training. Methods of enhancing fidelity through provider training include standardizing training, assessing skills acquisition, and developing strategies to help providers deal with diverse types of participants. Post training, care must be taken to prevent skills drift. Ideally, providers would be hired who are best suited to deliver the intervention.2,4
Although there are exceptions, for most VA providers, delivering REACH VA is not their exclusive job. Therefore, hiring decisions are not made based on suitability to deliver the intervention, as they would be for a research study such as REACH II. However, the clinical skills and empathy to develop rapport are part of the usual skill set of those who deliver REACH. For example, the 73 staff who delivered REACH VA for the assessed sample included 30 psychologists, 36 social workers, 3 nurses, and 4 other clinicians.
Although training for REACH II was longer and more rigorous, as is appropriate for a research intervention, the components were like REACH VA. Both REACH II and REACH VA had the challenge of making sure training was standardized across multiple sites. As is often the case for multi-site clinical trials, REACH II training was conducted centrally through a Coordinating Center, using slides and telephone and lasted 2 days. REACH VA training is also conducted centrally, through the aegis of VA’s Employee Education System. Staff register for a 3-hour webinar with slides and live lecture by the REACH Program Coordinator who delivers each training. The training is scripted but also involves participant interaction through poll questions, whiteboards for participant responses, and practice of skills such as problem solving. Continuing education credits are available.
Certification for both programs included a knowledge test and roleplay with a mock caregiver. The interventionist is rated on behavioral markers of specific procedural techniques (e.g., use of forms), clinical skills (e.g., active listening), components that are intended (positive), and plausible confounding parts that should not occur (negative). Suggestions for working with diverse types of caregivers, such as those with low literacy, are included during training and in the Program Coach Manual. For REACH II, a national certification committee provided feedback. For REACH VA, the program coordinator provides one-on-one feedback during the roleplay. In addition, coordinator and prospective interventionist have a one-hour consultation call to discuss the program, where and how the program will be delivered at the interventionist’s facility, and any factors that may help or hinder implementation.
REACH Treatment Fidelity –Delivery. To ensure that treatment is delivered as developed, content and dose must be specified,2,4 preferably through a treatment manual. There needs to be a mechanism to measure the provider’s adherence to the protocol’s content and dose.
Just as for REACH II, the REACH VA Coach Manual specifies each component that should be delivered during each session. The Manual includes an overview of the program, the risk assessment, and an array of where information to address each risk assessment question can be found in the Caregiver Notebook. Troubleshooting tips and all forms are also provided. Each session has a schedule with components to be addressed, descriptions of what should be covered, suggested times for each component, talking points, and scripts if needed.
For example, the description of the Signal Breath stress management technique includes six tasks/benchmarks to be completed.[1] These include: 1) introduce technique; 2) teach caregiver to rate level of tension; 3) describe technique; 4) practice with caregiver; 5) identify facilitators and barriers to practice; and 6) encourage use of technique. There is a script that covers each task. One difference between the two programs is in the length of time allotted to these tasks, reflecting the difference between a 12-session research study and a 4-session clinical program. For REACH II, 30 minutes are allotted; for REACH VA, 10 minutes.
For documentation, checklists include components of each session as a check off. However, for Signal Breath above, each task was listed for REACH II with places for notes; but for REACH VA, although each step is specified in the protocol, there is only the option to indicate present or absent for the entire teaching of Signal Breath. REACH VA interventionists have multiple means to document program delivery. Scanned paper or electronic fillable forms can be placed in the veteran’s or caregiver’s medical record. An electronic progress note mirrors the documentation form and is available in the electronic medical record.
Core delivery, receipt, and enactment activities include assessing the caregiver, reviewing materials and skills, introducing information and skills, practicing skills, developing problem solving plans, and rating the success of plans and strategies tried. Across all sessions the percentage of delivery activities that occurred was 91.9% (Table 2). Activities completed ranged from 97.2% of possible core activities completed in Session 1 to 85.7% of completed activities in Session 4. Mean dosage for the caregivers was 3.7 ± 1.7 sessions, with a range of 1 to 11 sessions. Of the sample, 67.2% of caregivers completed four or more sessions.
REACH Treatment Fidelity –Receipt
Assessing treatment receipt involves determining if participants have understood the intervention and if they can use the information and skills. Assessment methods can include pre-post tests, ensuring materials match health literacy level, and using multiple methods of presenting material, such as repeating information, queries about the material, and role-play with coaching and feedback.2,4
For delivery, activities that the interventionist delivered are marked on the documentation form, but for receipt there is neither direct observation of the interaction between interventionist and caregiver nor does the caregiver self-report to an outside observer. Instead, items marked by the interventionist are used to infer receipt, using the steps in the REACH protocol. As part of the intervention, participant and interventionist work though problem solving, cognitive reframing, and stress management exercises together to ensure receipt. For example, the caregiver practices the Signal Breath and stretching with the interventionist. For other stress management techniques there are forms to help the caregiver plan. Items that are reviewed by interventionist and caregiver in identifying pleasant events include the caregiver’s list of activities to try and a plan to ensure that an activity occurs. The plan includes the name of the activity, when it will occur, where it will occur, and what is needed to make it occur.
To help the caregiver learn the problem-solving process the caregiver answers questions about the chosen problem using the ABC process (antecedents, behavior, consequences) to pinpoint exactly what the problem is, who is present, and what is happening before and after the problem occurs. The caregiver then articulates a goal for the problem solving, generates possible solutions, and decides which is most likely to succeed. Caregiver and interventionist develop a plan to implement the solution, brainstorming barriers and enablers. The number of strategies is limited to no more than three, so the caregiver will not be overwhelmed. For REACH II, this joint plan was written up by the interventionist and then given to the caregiver at the next session. As an aid to the process, the REACH VA Caregiver Notebook, which is written at fifth grade reading level, has strategies listed in each chapter. Caregiver and interventionist highlight or otherwise identify strategies the caregiver will try. Finally, at the end of each session the caregiver is asked to make a written commitment that indicates what was discussed during the session and the specific strategy or strategies that the caregiver will try before the next session. Active learning strategies are encouraged; the interventionist may ask the caregiver to role-play any difficult solutions, practice asking for help, or try different communication strategies.
The percentage of core activities completed for receipt ranged from 97.3% in Session 1 to 88.4% in Session 4, and across all sessions the percentage of receipt activities that occurred was 92.2%.
REACH Treatment Fidelity – Enactment
Assessment of enactment involves determining if the participant has tried to implement the intervention.2,4 Assessment methods can include direct observation, self-report, or provider report.
The structure of the REACH intervention facilitates enactment. Each session builds on the previous session as part of the REACH protocol, and the topics that were discussed during the last session are reviewed. Modifications are then made if needed. Commitments – the strategies the caregiver tried since the last session – are reviewed, and barriers to implementation are assessed. Solutions are discussed, and commitment strategies modified, if necessary. Goal attainment for each problem is based on a five-point rating scale, from “a lot worse” to “a lot better.” If the problem is a lot better the caregiver has the option of choosing a different problem to work on next. If the problem is not a lot better, both the interventionist and caregiver review the last session’s strategies and what the caregiver tried, discuss success and barriers and the caregiver’s feedback and reflection, and select additional strategies to try.
Discharge from the program involves review of each section of the intervention, focusing on successes. The purpose is to encourage continued use of learned skills and the Caregiver Notebook as a resource for new challenges. The discharge session could occur during the final session or as a stand-alone session. After the final session is completed there is a program evaluation. Program evaluation includes a repeat of the Risk Appraisal to assess caregiver progress in safety and management of concerns for the care recipient, caregiver physical and emotional well-being, and social support. The caregiver is also asked about the usefulness of each section of the intervention and of the intervention itself.
In REACH II caregivers had forms to complete between sessions to document enactment. For REACH VA it is not possible to separate receipt and enactment because the interventionist and caregiver work on strategies and review homework during the session. Therefore, their percentage of core activities completed for enactment is the same as for receipt. For example, reviewing a problem solving plan can be evidence of receipt and enactment, because the caregiver understood the material and was able to make a plan.
---
[1] The Signal Breath technique was originally designed by Dr. Richard L. Hanso, Long Beach VA Medical Center, in his work with chronic pain patients and has been adapted for use with caregivers of persons with dementia by Jocelyn Shealy McGee, MSG, MA, Palo Alto VA Health Care System.