In 2016, the Precision Monitoring (PRIS-M) QUERI (Quality Enhancement Research Initiative) based at the Roudebush VA Medical Center in Indianapolis, Indiana was charged with supporting and studying the implementation of the VHA Tele-Stroke Robotic Rehabilitation program at four pilot sites around the United States. The 7-person implementation team based in Indianapolis included three doctoral-level implementation scientists who collectively had been working in implementation science for over 30 years in VHA, a senior physician-researcher, a masters-level program manager, and a research assistant. The implementation team was led by the senior implementation scientist and held regular weekly meetings to reflect on implementation progress.
As part of this work, the implementation support team developed general specifications of a new “State of Implementation” Progress Report (SIPREP) over a 6-month period in 2018. They drew upon multiple sources of information to map the implementation work: weekly national program phone calls, discussions with individual participants, site visits, notes from implementation team meetings, and online resources. Stage-specific grids organized the implementation work into columns, rows and cells, identifying specific implementation activities to be completed; who was responsible for completing each implementation activity; and the timing/level of each activity (early/basic, intermediate or late/advanced). Additional links provided access to specific tips and resources that could assist local staff in completing particular implementation activities.
The SIPREP was subsequently applied independently by two different teams working on two different and unrelated national VHA QI initiatives. The two teams were based in Indianapolis, Indiana and Ann Arbor, Michigan. The sections below present the development and use of the SIPREP for each project separately, as it was customized and applied differently in this cross-initiative pilot demonstration project.
2.1 Tele-Stroke Robotic Rehabilitation Project
Based at the Atlanta VA Medical Center, the Tele-Stroke Robotic Rehabilitation program provided rural Veterans who have had a stroke with an innovative, in-home solution for physical rehabilitation that improves access to care by mitigating transportation barriers for Veterans who live in rural areas at a distance from Veterans Health Administration medical centers. The Tele-robotics program was a quality improvement (QI) project funded by the VA Office of Rural Health as an Innovations Project to be implemented at the four pilot sites.
In FY17, the PRIS-M implementation support team assisted the Atlanta-based clinical team during its program expansion to incorporate additional Veterans Health Administration facilities, helped to develop and refine implementation strategies, attended in-person program kick-off meetings in Atlanta, Georgia and Birmingham, Alabama and conducted baseline interviews and surveys with participating Veterans Health Administration clinicians and Veterans. As Veterans’ access to the program continued to expand through implementation at additional VHA facilities, the PRIS-M implementation support team partnered with the Atlanta-based team and key stakeholders from the national VHA Office of Rural Health to formalize procedures, policies, practices and approaches to ensure successful implementation and larger-scale deployment across diverse VA medical centers at a national level.
Core implementation strategies used in the implementation of the Tele-robotics program were facilitation and education. In the Tele-robotics program, on-site kickoffs took place where members of the national program team travelled to the participating facilities to spend a half-day training local staff on how to use the new equipment, recruit new patients with an eye to both inclusion and exclusion criteria, conduct baseline assessments, and follow new protocols.
The PRIS-M implementation support team was specifically charged with providing ongoing feedback to the Tele-Stroke Robotic Rehabilitation program about implementation progress at the four participating sites, and to provide guidance for future scaling up of the program if at a later point it received approval for a larger-scale rollout. The implementation team also sought to gain new insights into the implementation process and generate actionable findings that could provide specific guidance to implementation leaders regarding implementation progress and the potential need for mid-course adjustments. With these aims in mind, the team developed the new “State of Implementation” Progress Report (SIPREP) approach.
Each of the four participating VA medical centers was given its own designated grid for each stage of implementation. Within each grid, item-level checkboxes were checked off and cells changed colors as particular activities were completed, offering a visual representation of implementation progress within and across sites across the various stages of implementation.
The SIPREP was hosted on a VA SharePoint platform: once staff accepted an initial, one-time invitation to access the SIPREP, they could then view it anytime from that time forward on any computer logged onto the VA system. The implementation support team created, maintained and updated the SIPREP for all four VA medical centers.
Two key concepts used to organize the SIPREP were “milestones” and “stages.” Milestones were significant implementation achievements that occurred in a chronological order. In the implementation of the Tele-robotics program, there were five milestones: Initial Agreement to Participate; Kickoff; Enrolling 1st Patient; Enrolling 10th Patient; and Adoption/Sustaining. Stages involved from getting from one milestone to the next, and there were thus four stages:
- Stage 1: Getting from Initial Agreement to Participate to Kickoff
- Stage 2: Getting from Kickoff to Enrolling 1st Patient
- Stage 3: Getting from Enrolling 1st Patient to Enrolling 10th Patient
- Stage 4: Getting from Enrolling 10th Patient to Adoption/Sustaining
“Grids” constituted another key organizing element for the SIPREP. Stage-specific grids organized the implementation work into columns, rows and cells, identifying specific implementation activities to be completed; who was responsible for completing each implementation activity; and the timing and level of each activity. Grid columns specifying who completed particular implementation activities included C for Champions/site leads; T for local implementation Team; and M for program-related Microsystem (e.g., local providers and staff not on the implementation team but who were nonetheless crucial to the success of program implementation: for example, local neurologists who could refer Veterans for participation in the Tele-Stroke Robotic Rehabilitation program). Rows indicated the timing/level of each activity, with 1 = Early/Basic (straightforward activities, like securing a room with video-conferencing capabilities for the day of the kickoff); 2 = Intermediate (nuanced activities, like recruiting potential staff to attend the project kickoff and consider joining the local implementation team); and 3 = Late/Advanced (capstone activities that build on prior early/basic and intermediate tasks, like resolving major concerns of stakeholders about participating in the project before the kickoff takes place).
Additional links provided access to specific tips and resources that could assist local staff in completing particular implementation activities. Each of the four participating VA medical centers was given its own designated grid for each stage of implementation. Within each grid, item-level checkboxes were checked off and cells changed colors as particular activities were completed, offering a visual representation of implementation progress within and across sites across the various stages of implementation.
2.2 Lung Decision Precision Tool Project
In addition to this implementation of the SIPREP with the VHA Tele-Stroke Robotic Rehabilitation project, a different VHA implementation support team working in another part of the country independently applied the SIPREP to evaluate implementation progress on another national VHA initiative. The PeRsonalizing Options for Veteran Engagement (PROVE) QUERI program based at the VA Ann Arbor Healthcare System began using the SIPREP as part of implementing and evaluating a web-based, provider-facing tool for enhancing shared-decision making with patients eligible for lung cancer screening.
The Lung Decision Precision (LDP) tool was initially implemented using a six-month, virtual quality improvement training approach in four VA medical centers beginning in 2017. Four other VA medical centers served as a control group, in which the tool was implemented using a one-time provider education approach.
Implementation of LDP proceeded in four steps. First a site lead was identified. This was someone involved in lung cancer screening (LCS) at the site, who would help champion implementation. This site lead was educated about the tool during a one-hour videoconference. Second, the Clinical Applications Coordinator (CAC) at the site was contacted and asked to add a link to the tool into electronic medical health record. The time required to complete this second step ranged from 1 week to 3 months. Third, the site lead notified primary care providers or LCS coordinators by e-mail about the tool, and explained where a link to the tool could be found in the medical health record. This email also included a one-page document describing the tool and how to use it. Time to complete this task depended on the ease of obtaining a provider listing and the site lead’s busy schedule, and ranged from a few minutes to three months.
Finally, a year to 18-months after the initial implementation strategies were completed, a second implementation strategy—academic detailing—was conducted at all sites in an effort to increase tool use among primary care providers. A team member in Ann Arbor received training in academic detailing, and then traveled to seven sites over the course of six months to meet individually and in groups with primary care providers to give them a brief tutorial.
In 2018, midway through implementation of LDP in the eight participating sites, the PROVE QUERI lead investigator learned about the SIPREP from a VHA webinar series and discussed potential use of the tool with the project manager in charge of the LDP implementation. The eight participating sites were all progressing with implementation at different rates, and had implemented lung cancer screening in different ways, which affected their interest in and ability to use LDP. As a result, it was becoming very challenging for the national team in Ann Arbor to keep track of site implementation status, including the unique barriers that were affecting the pace of implementation. The lead investigator thought the SIPREP would be an ideal mechanism for presenting the status of implementation at each site and clearly delineating next steps.
The project manager was also interested in using the tool for organizing the tremendous amount of documentation associated with the project, both for use by other sites wishing to implement LDP, as well as for documenting project history, needed for writing reports and manuscripts. The SIPREP offered a way of organizing project documents by the national coordinating team that was easier to understand and navigate than a standard electronic filing system.