Assessing the Sustainability of Compliance with Surgical Site Infection Prophylaxis After Discontinuation of Mandatory Active Reporting: Study Protocol

doi:10.21203/rs.3.rs-1408791/v1

PDF

Study protocol

Assessing the Sustainability of Compliance with Surgical Site Infection Prophylaxis After Discontinuation of Mandatory Active Reporting: Study Protocol

https://doi.org/10.21203/rs.3.rs-1408791/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 24 Apr, 2022

Read the published version in Implementation Science Communications →

You are reading this latest preprint version

Background

Surgical site infections are common. Risk can be reduced substantially with appropriate pre-operative antimicrobial administration. In 2005, the VA implemented the Surgical Care Improvement Project (SCIP) in the setting of high rates of non-compliance with antimicrobial prophylaxis guidelines. SCIP included public reporting of evidenced-based antimicrobial guideline compliance metrics in high-risk surgeries. SCIP was highly successful and led to high rates of adoption of pre-operative antimicrobials and early discontinuation of post-operative antimicrobials (> 95%). The program was retired in 2015, as the manual measurement and reporting process was costly with limited expected additional benefit. No studies have assessed whether the gains achieved by SCIP were sustained since discontinuation of active support for the program was discontinued. Furthermore, there has been no investigation of the spread of antimicrobial prophylaxis guideline adoption beyond the limited set of procedures that were included in the program.

Methods

Using a mixed methods sequential exploratory approach, this study will 1) quantitatively measure compliance with SCIP metrics over time and across all procedures in the five major surgical specialties targeted by SCIP, and 2) collect qualitative data from stakeholders to identify strategies that were effective for sustaining compliance. Diffusion of Innovation Theory will guide assessment of whether improvements achieved spread to procedures not included under the umbrella of the program. Electronic algorithms to measure SCIP antimicrobial use will be adapted from previously developed methodology. These highly novel data mining algorithms leverage the rich VA electronic health record and capture structured and text data and represent a substantial technological advancement over costly manual chart review or incomplete electronic surveillance based on pharmacy data. An interrupted time series analysis will be used to assess whether SCIP compliance was sustained following program discontinuation. Generalized linear models will be used to assess whether compliance with appropriate prophylaxis increased in all SCIP targeted and non-targeted procedures by specialty over the duration the program’s active reporting. The Dynamic Sustainability Framework will guide the qualitative methods to assess intervention, provider, facility, specialty, and contextual factors associated with sustainability over time. Barriers and facilitators to sustainability will be mapped to implementation strategies and the study will yield an implementation playbook to guide future sustainment efforts.

Relevance

Sustainability of practice change has been described as one of the most important, but least studied areas of clinical medicine. Learning how practices spread is also a critically important area of investigation. This study will use novel informatics strategies to evaluate factors associated with sustainability following removal of active policy surveillance and advance our understanding about these important, yet understudied, areas.

Registration

Not applicable.

Dynamic Sustainability Framework

sustainability

informatics

antimicrobial prophylaxis

surgical care improvement project

policy discontinuation

diffusion of innovation

scale up

spread

ERIC (Expert Recommendations for Implementation Change)

Sustainability is an important, yet understudied area of implementation science and clinical medicine. Factors that support long-term sustainability following discontinuation of active programmatic support and surveillance are poorly characterized.
This theory-driven study uses a mixed methods sequential exploratory approach and advanced informatics guided by the Dynamic Sustainability Framework and Theory of Diffusion of Innovations to identify factors associated with long-term practice change and spread of evidence-based practices.
This study will provide insight into factors that drive sustainability and diffusion of innovation, and can be used to design future late-stage implementation interventions to promote long-term practice improvement. Barriers and facilitators to sustainability will be mapped to implementation strategies to identify best practices for supporting guideline compliance after active support for a program is discontinued.

Surgical site infections (SSIs) are one of the most common types of healthcare-associated infections, accounting for substantial morbidity and mortality. Pre-incision antimicrobial prophylaxis, administered within one hour prior to incision, ishighly effective for reducing SSI. Post-operative antimicrobials given after skin closure do not reduce SSI and increase severe adverse events,including acute kidney injuries and Clostrideriodies difficile infections.^1-3 Beyond the direct impact on the patient, excess antimicrobial use contributes to the burden of antimicrobial resistance, a critical healthcare threat and a major target of VA and non-VA initiatives.⁴

In 2005the VA implemented the Surgical Care Improvement Project (SCIP)to increase compliance with SSI prevention measures.SCIP was a Joint Commission initiative, which included a set of publicly reported evidenced-based antimicrobial guideline compliance metrics targeting high-risk surgeries in five specialties, such as cardiac bypasses and orthopedic total joint replacements.⁵Public reporting of SCIP metrics required resource-intense manual review by a trained reviewer as part of the VA’s External Peer Review Program (EPRP). Following implementation of active reporting, VA compliance with guideline-concordant pre-operative antimicrobial use (SCIP INF-1) and prompt discontinuation of antimicrobials post-operatively (SCIP INF-3) exceeded 95%.^6-8 After this high level compliance was achieved, SCIP was retired in 2015, as the measurement and reporting process was felt to be costly with limited additional expected benefit.^{9, 10}Since the program’s retirement, no studies have examined if the practice changes achieved through the active SCIP program were sustained, or if improvements spread beyond the originally targeted surgeries to procedures not included under the umbrella of the original program.

Sustainability is defined as “the extent to which an evidence-based intervention can deliver its intended benefits over an extended period of time after external support from the donor agency is terminated.”^{11, 12} Sustainability of practice change has been called “one of the most significant translational research problems of our time.”¹³However, despite the critical importance of the topic, in 2004 Greenhalgh et al. noted a “near absence” of studies that aim to answer questions about factors driving high and low levels of sustainability.¹⁴ In the past 17 years, there has continued to be a dearth of research providing insights into this important area.¹⁵The proposed study aims to address this important scientific gap by examining whether guideline concordantpre- and post-operativeantimicrobial use adopted during SCIP were sustained, using the Dynamic Sustainability Framework¹¹ as a guide for the investigations. VA longitudinal databases, advanced informatics approaches, and qualitative data will be used to address the study aims. Assessment of program spread to uncovered specialties will be guided by the Diffusion of Innovations theory.¹⁶

Theoretical Basis and Frameworks:

The research in this study is guided by Chambers et al. 2013Dynamic Sustainability Framework (DSF, See Figure 1 for DSF adapted to evaluatelate-stage implementation outcomes for SCIP). The DSF emphasizes that sustainability is an ongoing aspect of implementation that requires continual adaptations and support. It highlights that successful sustainment of an intervention, such as evidence-based antimicrobial use practices, requires that the characteristics of the intervention be “consistently tracked, using valid, reliable, and relevant measures” and expects that the system will change and evolve overtime. Further, the framework underscores the importance of “ongoing assessment and quality improvement efforts” to “improve sustainment” and “identify opportunities for intervention improvement.” The DSF also highlights the importance of informatics advances to improve and enhance ongoing quality monitoring and improving efforts as part of a larger “learning healthcare system” model.^{11, 17}

Relevant to the proposed protocol, after discontinuation of active implementation support, the DSF theorizes that programs may undergo voltage drop, which refers to the extent to which providers return to practices that were in place prior to an active implementation strategy that promoted successful change (Figure 2).¹⁸Figure 1 shows the main constructs of the DSF, including characteristics of the intervention, the practice setting, and the ecological system, and how they interact to impact sustainability over time. In the case of peri-operative antimicrobial use and the SCIP program, there are two ways in which “voltage drop” may occur: 1) Providers stop appropriately administering pre-incision antimicrobial prophylaxis, i.e., non-compliance with SCIP INF-1, and 2) providers may inappropriately prolong antimicrobials after skin closure i.e., non-compliance with SCIP INF-3. The deviation from the intervention in the years SCIP was active (T₀) to post-retirement (T₁) may be due to features of the intervention itself, specialty-specific practice setting issues, and/or policy or other facility-level changes that make up the ecological system, such as surgical or infection control local guidelines.

The SCIP program targeted only a limited number of high-risk surgeries within five surgical specialties, such as coronary artery bypass grafting, cardiac valve replacements, and orthopedic arthroplasties (total joint replacements). Other types of common but less invasive orthopedic procedures, such as arthroscopyand others, were not specifically targeted by all elements of the SCIP program. This raises the question about whether evidence-based antimicrobial practices promoted by the policy may have diffused within specialties. In other words, once orthopedic surgeons adopted a practice change for hip and knee replacements, was it also adopted for less invasive procedures not specifically targeted by the program? As described in the Diffusion of Innovation Theory, provider-level increases in adoption depend on awareness, persuasion, decision, implementation, and continuation.¹⁹ Adoption of evidence-based practices mayspread to procedures not covered by SCIP, which was limited in scope and primarily targeted a sub-set of major procedures most frequently performed in the Medicare population (Figure 3).In other words, it is possible that practice change promoted by the program led to practice improvements for surgeries not included in its umbrella. For example, improvements may have been achieved through provider education or changes to pre-operative protocols and order sets that were then used for surgeries not targeted by the program (Figure 3). At the system level, the policies and practices may have changed for all surgical care and antimicrobial stewardship/infection control. For example, the Surgical Office or Infection Control may set policies or standards to effect change, such as surgical time outs that include a discussion of pre-operative antimicrobial prophylaxis, and we may see consistently high compliance across all specialties in the inpatient facility. This possible spread beyond the initial program scope is consistent with themes highlighted in the Theory of Diffusion of Innovation, and distinct from “program drift” highlighted in the DSF, which focuses on changes to a program that can occur over time, rather than how practice improvements may have impacts beyond their initial targets.¹⁹These considerations lead to the hypothesis that compliance with SCIP INF-1 and 3 Metrics Is Lower in SCIP-Excluded Procedures Compared to those Targeted by SCIP within a given specialty.

Gaps and Rationale:

Prior work demonstrates that while sustainability is an essential aspect of implementation science, few studies have measured factors associated with sustainability and how policy changes, such as the discontinuation of SCIP, impacted ongoing compliance with evidence-based practices. Guided by the DSF, this study will address these critically important translational research questions and will yield important insights about how to achieve sustained adoption of evidence-based practices after the active reporting program has ended. Informatics tools developed during the investigations will be adapted to measure compliance at the facility and specialty levels andwill be included in an implementation playbook. Another major gap that will be addressed is an evaluation of how practice changes spread after an initial implementation directed at a small segment of clinical care; these aspects of the study will be guided by the Diffusion of Innovations theory. SCIP targeted major inpatient surgeries, but more minor procedures were not included in the antimicrobial use metrics. Lessons learned about factors driving the spreadof practice change will provide critical insights for implementation science, and will be included in our implementation playbook, one of the final products of this study.

Overall Study Design:This studywill usemixed methods sequential exploratory approachto address the study’s aims, which are to assess the sustainability and spread of appropriate pre- and post-operative antimicrobial use achieved by the SCIP program (Figure 4). The research will adhere to the STROBE reporting checklist. Guided by the DSF, the final product of this study will yield an implementation playbook that will comprehensively describe for future implementation sites theneed for adaptingSCIP metrics and implementation strategies tofit local practice settings and the local environment (ecological system) (Figure 5) as well as the different evidence-basedimplementation strategies (ERIC)²⁰that can be used to support program sustainability.

Quantitative Data Collection and Analysis:

Study Population and Setting: VA surgical data from FY2005 – 2020from 70 complex VA inpatient facilities will be used to complete the quantitative analysis. This large, national dataset will include inpatient and outpatient clean or clean contaminated surgeries in cardiac, orthopedic, general, gynecology, and vascular specialties as defined by Current Procedural Terminology (CPT) codes.Surgical data will be combined with manually validated data about peri-operative antimicrobial use collected from the External Peer Reviewed Program (EPRP) dataset. Thus, the EPRP data can be used as the “gold standard” manual review for the purposes of developing, iteratively refining, and validating electronic measurement tools that do not require the time and resource intensive process the original SCIP program required.

Development Cohort: After obtaining EPRP data on SCIP compliance by facility and specialty, SCIP-eligible procedures will be identified in the VA electronic health record (EHR) from 2005-2015 to build a dataset for each specialty cohort with SCIP compliance information and structured/unstructured data relevant to our SCIP INF-1 and 3 algorithms.EPRP data contains manually reviewed surgeries targeted by SCIP, and based on previously published work, we anticipate that our sample will include many non-compliant cases, particularly in the early period following program implementation.

Algorithm Development: First, a list of appropriate antimicrobials for surgical care as specified in the original SCIP guidelines and in current multi-society SSI prevention guidelines will be reviewed and mapped for each of the included surgical specialties.^{5, 21, 22}Then, this list will be used to develop electronic measurement tools. Based on our prior work,²³ we will develop each algorithm to detect SCIP INF-1 and 3 antimicrobial prophylaxis iteratively over several stages, by varying 1) types of variables included in the tool (e.g., text note extraction only, orders only, administration only, or combinations of the three), 2) timing of the searches (e.g., including or excluding the procedure date), and 3) types of antimicrobials included in the tool. The list will be used to search clinical notes for documentation of antibiotic administration pre- and/or post-operatively to measure compliance with SCIP metrics. The list will also be mapped to structured data in the VA EHR and orders and administration of relevant medications will be extracted.

The two algorithms (one for pre-operative antimicrobial administration and one for post-operative administration) will be applied to half of all EPRP reviewed surgeries targeted by SCIP from 2005-2015. Algorithm performance, i.e., criterion validity,will be assessed as sensitivity (how many true positive cases were identified as positive by the algorithm) and specificity (how many true negative cases were identified as negative by the algorithm). Manually reviewed EPRP data will be used as the gold standard for algorithm development. To finalize the algorithm, all discordant cases (e.g., algorithm flagged positive but EPRP manual review was negative, or algorithm flagged negative but EPRP manual review was positive) will undergo a second round of manual review to identify reasons for the discordant flag and to qualitatively classify the reason for discordance.These findings will be used to adjust and adapt the algorithms for each SCIP metric and each specialtyprocedure and facility to improve accuracy. We will also conduct an analysis stratified by facility, to determine if facility-level effects, such as coding differences, impact algorithm performance and accuracy.

Algorithm Validation:We will validate the final SCIP INF-1 and 3 algorithms by applying the structured and unstructured data extracts to SCIP-eligible procedures in the validation half of the 2005-2015 EPRP data. We will measure and criterion validity: sensitivity, specificity and positive predictive validity for each algorithm per specialty (cardiac, orthopedics, general, gynecology and vascular surgeries).

Assessment of Voltage Drop: Analysis (Interrupted Time Series): After the algorithms for measuring pre-and post-operative antimicrobial use are validated, we will then apply them to the 2016-2020 period, e.g., after SCIP discontinuation and after discontinuation of the EPRP manual review. SCIP INF-1 and 3 compliance stratified by specialty will be assessed separately.

We will test the hypotheses that SCIP INF-1 and 3 compliance experienced “voltage drop,” defined as decreasing adherence to antimicrobial use guidelines over time,afterSCIP retirement in 2015 and an overall change in rate over time using an interrupted time series model. Observations will be at the procedure level with facility repeated measures; thus, we will control for facility-level random effects.The outcome measure will be a binary indicator of compliance for each procedure, and the models will include a binary indicator of pre- or post-SCIP retirement, a continuous variable for calendar time, and an interaction between these two variables. The beta coefficient for the interaction term will represent the difference in slopes between pre and post SCIP retirement, the coefficient for pre and post will indicate the “voltage drop,” or immediate drop after the SCIP retirement, and the coefficient for time will indicate the slope for the pre-period. For each outcome, SCIP INF-1 and 3, and for each surgical specialty, we will fit separate models. To adjust for multiple comparisons, we will perform false discovery rate (FDR) correction on the betas for the interaction terms for each of the 10 comparisons. For each of the 10 comparisons, setting alpha at 0.01 to be conservative (given that we will adjust for multiple comparisons), assuming there will be correlation among patients from the same facilities (0.02), we can detect at least a 5% difference in slopes between pre and post SCIP retirement for each surgical specialty with greater than 90% power. This assumes approximately 95% SCIP compliance between 2011-2015.⁹

Assessment of Diffusion of Practices to Surgeries not Covered by SCIP: SCIP active reporting demonstrably increased appropriate antimicrobial prophylaxis compliance in SCIP-targeted procedures, largely due to changes in provider behavior that may have been in response to local policies designed to facilitate adoption. With respect to diffusion, the provider behavior change for one type of surgery may have led to changes for all procedures, not just those specifically targeted by SCIP.

Diffusion of Practices Cohort: Allclean or clean/contaminated surgical procedures within the five specialties performed from 2016-2020will be identified and an expanded cohort will be created.Our analysis will be limited to surgeries where pre-operative antibiotics are recommended.

Statistical Analysis: We will use binomial generalized linear mixed models to estimate the association between SCIP-targeted versus excluded procedures and the two SCIP metrics, adjusting for correlation among observations within the same facility using facility random effects. The variance-covariance matrix will allow us to estimate the correlation coefficients for nesting so that we can make inferences on the strength of these sources of correlation. Separate models will be fit for each outcome (SCIP INF-1 and 3) and surgical specialty; a total of 10 models. We will again apply FDR correction on the p-values for the primary comparisons related to our hypothesis.

Power Calculation: For each of the 10 models, we will have greater than 90% power to detect at least a 10% difference in the two SCIP compliance outcome measures between the comparison groups for each hypothesis. This assumes correlation of 0.02 at the facility level and alpha set at 0.01. For example, if compliance is 85% for females and 75% for males, we can detect this difference with an effective sample size of 938 individuals.

Qualitative Data Collection and Analysis:

Overview of Qualitative Analysis: For the qualitative aspects of the study, based on current guidance,²⁴ we will interview up to six key stakeholders in different VA hospitals for each of the five specialties, for a total of 60 interviews in ten VA facilities. Interview participants will have work duties related to SSI prevention and antimicrobial stewardship: surgical staff (including anesthesiologists), infectious diseases staff, pharmacists, and surgical nurses.We will analyze interview data to identify facilitators and barriers to implementation sustainability and map findings to DSF constructs. Practice spread will also be assessed and evaluated using the Diffusion of Innovations Theory as a guide. As a final step, we will map facilitators and barriers to the ERIC implementation strategies and will develop an implementation playbook that can be used by VA hospitals to support long-term sustainability of the quality improvements in peri-operative antimicrobial use achieved by SCIP.

Key Stakeholders: Key stakeholders for peri-operative antimicrobial use include members of the surgical staff (surgeons,anesthesiologists, nurses), infectious diseases staff (physicians and infection control/antimicrobial stewardship team members) and inpatient pharmacists. These are the providers who will be targeted at each of the participating VA facilities.

Selection of Sites and Recruitment: We will purposefullyrecruit two facilities for each specialty (N=10 facilities) from the 70 high complexity VA facilities, targeting a range of sites, including regional diversity and urban, suburban, and rural variation. Through this selection process, we will recruit 60 stakeholders from 10 facilities (6 interview participants at each site).Our goal is to reach saturation within each of the five SCIP-targeted specialties across the sites; typical sample sizes for achieving this in implementation research range from 5 to 10 individuals in key roles.²⁵

Recruitment: Operational partners from the national patient safety office and the national antimicrobial stewardship office will co-sign a letter outlining support for the project; the PIs will then send out this letter to a purposefully sampled selection of sites and inform them of the study prior to sending recruitment emails to providers. Thereafter, the project manager will send out recruitment emails to providers. We will use an opt-in approach. If we do not get enough stakeholders who agree to participate, then we will identify additional stakeholders to recruit using the approach described above.

Data Collection:Two co-investigators will conduct semi-structured telephone or video interviews with key stakeholders; interviews are estimated to take approximately 30-minutes. All interviews will be digitally audio recordedfor transcription; informed consent will be obtained prior to starting the interview. Interviewers will follow a semi-structured interview guide, which will consist of both structured and open-ended questions. The interview guide will be revised with input from operational partners prior to pilot testing and data collection. Through this process, we will ensure that we develop a set of questions that will allow us to collect rich information about facilitators and barriers of sustainabilityand also about factors that lead to diffusion of the practice changes, or lack of spread.

Informed by DSF and the Theory of Diffusion of Innovations, interviews will elicit information about the intervention (SCIP), the practice setting/context (surgical specialty), and the ecological system (VA facility/VISN).The overall purpose of the interviews will be to understand, from the perspectives of surgical staff, infectious diseases staff, and inpatient pharmacists,what types of processes/practices have been implemented to help with peri-operative and post-operative antimicrobial use and compliance; whether and how those processes/practices were adapted and sustained after SCIP was retired in 2015; whether and how those antimicrobial use and compliance processes/practices spread to other settings; and the facilitators that helped with and the barriers that hindered implementation, maintenance, and spread of those processes/practices.Information will be collected about the contextual factors within the practice setting that affectimplementation and sustainment of antimicrobial use and compliance by asking about the culture of the practice settingas well as trainings and resources that are available to help with antimicrobial use and compliance.Additionally, interviews will probe to understand whether there are other metrics, policies, regulations or guidelines that are being used for peri-operative and post-operative antimicrobial compliance and how those have influenced processes/practices.

Qualitative Data Analysis:Coding of interview transcripts will be organized usingNVivo, a qualitative analytic software. Transcripts will be initially coded using a priori constructs consistent with the DSF and/or the Diffusion of Innovations theory. We will use a directed content analysis approach with allowance for new themes to emerge.²⁶ As coding proceeds, new emergent themes will iteratively be identified, elaborated on, and expanded based on team discussions, a process known as the constant comparative method.Inter-rater reliability will be established using the “check-coding” process.Coders will independently code the same interview transcripts and will then meet to compare their coding, discuss areas of difficulty, and reach consensus on the definitions and examples in the codebook. A new interview will then be independently coded by all, and the process will be repeated until a mutual understanding of the code definitions and when to apply the codes is achieved across all coders. After coding is complete, we will summarize the data by producing site-specific descriptive summaries, which will include key information (quotes and themes) about our findings for each of the DSF constructs - the intervention (SCIP), the practice setting/context, and the ecological system. Within the site-specific summaries, we will note any differences in perspectives between key stakeholders at the facility as well as differences by specialty. The site summaries will result in a rich description of each DSF constructs and the factors (e.g., facilitators and barriers) that affect implementation sustainability of SCIP.

Triangulation of Qualitative and Quantitative Data:When site-specific summaries are complete, we will triangulate our quantitative and qualitative findings. Utilizing Miles and Huberman’s analytical approaches,²⁷ we will triangulate the quantitative data elements from the facilities and specialties and the compliance rates with the qualitative findings to create a cross-site matrix. We will compare and contrast evidence to determine the key factors that may affect implementation sustainability of SCIP for sites with high or low SCIP complianceas well as for different specialties. We will then develop descriptive cross-site summaries based on our analysis of the integrated data.

Mapping of Findings to Expert Recommendations for Implementation Change (ERIC) Implementation Strategies: The data matrices will be used to map DSF-defined barriers and facilitators to the evidence-based list of implementation strategies developed by the Expert Recommendations for Implementation Change (ERIC) group.^{20, 28}Table 1 provides examples of implementation barriers we may find related to specific DSF constructs, and the selection and specification of ERIC implementation strategies that may be identified as a result of our mapping process (Figure 5). This process will identify and specify implementation strategies to address relevant challenges particular to pre- and post-operative antimicrobial use and compliance, and will inform the creation of an implementation playbook.

Following completion of the triangulation of qualitative and quantitative data, and mapping of findings to ERIC implementation strategies, we will create an implementation playbook: a document that comprehensively describes how to sustain SCIP best practices in sites with low compliance on either of the SCIP antimicrobial use metrics; different implementation strategies may be required depending upon whether sites have low sustainability with SCIP INF 1, 3, or both. The playbook will reflect the DSF in describing to sites the need to adapt SCIP antimicrobial use metrics and the ERIC implementation strategies to fit local practice settings, the local environment and Veteran populations (ecological system).Examples of sections that may be included in the implementation playbook are:1) evidence base for the SCIP metrics; 2) advice on communicating with facility leadership and staff about SCIP to generate interest; 3) assessment tools for sites to gauge their readiness; 4) planning tools, such as timelines to ensure appropriate rollout of relevant interventions; 5) operational tools, such as example policies, checklists, and example electronic ordersets that can be locally adapted; 6) training curricula and educational materials for SCIP site champions to train other staff; 7) a plan for supporting ongoing measurement through local adaptation and calibration of the informatics tools developed in our quantitative analyses, and measurement of clinical outcomes, including SSIs, and other adverse events. All of these elements are necessary to support future uptake and sustainment of guideline concordant antimicrobial use.

To ensure the implementation playbook is operationally useful, we will seek feedback from local and national stakeholders using a member checking process.²⁹Member checking, also known as participant or respondent validation, is a technique for exploring the credibility of research results. This willensure that the playbook has been developed as intended to increase the uptake and sustainability of SCIP antimicrobial use metrics at sites with low compliance with one or both of the INF metrics. We will interview a subset of our respondents fromthe qualitative interviews (n∼20) to assess the perceived acceptability and feasibility of elements of the draft implementation playbook, selected implementation strategies, and locations for targeting the sustainability, spread and diffusion of SCIP practices. Using data collection and analysis methods previously described, we will identify areas of the playbook that require further changes and updates.

We will use electronic data and health informatics tools (i.e., data mining through text queries and algorithm development/validation) to measure guideline compliance with surgical antimicrobial prophylaxis. Lessons learned about the sustainability and diffusion of evidence-based practices will have a direct impact on VA surgical care and will advance the field of implementation science through improving our knowledge and understanding about factors that influence sustainability, a historically understudied, yet critically important area. In addition to the contributions to the advancement of implementation science, healthcare informatics tools adapted and optimized will lead to expansion of antimicrobial stewardship efforts to include a broad range of surgical types and specialties and are likely to directly translate into improvements in clinical care delivery.

There are several limitations to our approach to assessing both sustainment and spread of SCIP practices. First, this study will gather retrospective data from the EHR; it is not a prospective or interventional study. However, for the purposes of evaluating the long-term impact of a policy change, this is an appropriate study design and the process of mapping barriers to strategies will provide valuable data to inform the development of implementation bundles and toolkits for testing in a future clinical trial. Second, a major hypothesis of our study is that voltage drop has occurred over time, but it is possible that we may not identify substantial variation and that all sites will have maintained outstanding compliance with SCIP metrics. If this occurs, the qualitative aspects of the proposal will be used to gather information about factors that contributed to the long-term sustainability of the practice change so that the effective strategies can be replicated or tested in other clinical settings. Data about why providers did not revert to outdated standards of careare important for advancing the field of implementation science and may promote similar improvements in other settings of care. Thus, we expect to still be able to collect rich and useful qualitative interview data, even if voltage drop has been minimal and/or if variation between sites is less than anticipated.

Our protocol presents an innovative approach to improving surgical care and antimicrobial stewardship by combining existing electronic data and health informatics tools (i.e., data mining through text queries and algorithm development/validation) to measure guideline compliance with surgical antimicrobial prophylaxis. Lessons learned about the sustainability and diffusion of evidence-based practices will have a direct impact on surgical care. Healthcare informatics tools adapted and optimized as part of the research will lead to expansion of antimicrobial stewardship efforts to include a broad range of surgical types and specialties.

Ethics Approval and Consent to Participate:

Not applicable.

Consent for Publication:

Not applicable.

Availability of Data and Materials:

Not applicable.

Competing interests:

WBE was the site PI of a COVID-19 therapeutics trial funded by Gilead Sciences (funds to institution). WBE is a paid expert witness for DLA Piper, LLC, Medtronic, and CRICO. All other authors report no conflicts of interest.

The views expressed are those of the authors and do not necessarily represent those of the US Federal Government or the Department of Veterans Affairs.

Funding:

This work was supported in part by Veterans Affairs Health Services Research and Development Investigator-Initiated Research award IIR 20-101.

Authors' contributions:

Conception or design: WBE, ARE, RL, MS, KC, MH, HJM.

Drafting and substantial revisions: WBE, ARE, RL, MS, RLL, KC, JR, JP, KH, MH, HJM.

Acknowledgements

None.

Berenguer CM, Ochsner MG, Jr., Lord SA, Senkowski CK. Improving surgical site infections: using National Surgical Quality Improvement Program data to institute Surgical Care Improvement Project protocols in improving surgical outcomes. J Am Coll Surg. May 2010;210(5):737-41, 741-3. doi:S1072-7515(10)00076-1 [pii] 10.1016/j.jamcollsurg.2010.01.029
Bratzler DW, Houck PM, Richards C, et al. Use of antimicrobial prophylaxis for major surgery: baseline results from the National Surgical Infection Prevention Project. Arch Surg. Feb 2005;140(2):174-82. doi:10.1001/archsurg.140.2.174
Carey K, Stefos T, Zhao S, Borzecki AM, Rosen AK. Excess Costs Attributable to Postoperative Complications. Med Care Res Rev. May 2 2011;doi:1077558710396378 [pii] 10.1177/1077558710396378
Centers for Disease Control (CDC). 2019 Antimicrobial Resistance Report: Biggest Threats and Data. Accessed November 20, 2019. https://www.cdc.gov/drugresistance/biggest-threats.html?deliveryName=USCDC_426-DHQP-DM13106
The Joint Commission. Surgical Care Improvement Project (SCIP) Specifications Manual Version 2010A1. Oct 10, 2019. https://manual.jointcommission.org/releases/archive/TJC2010B/SurgicalCareImprovementProject.html
Munday GS, Deveaux P, Roberts H, Fry DE, Polk HC. Impact of implementation of the Surgical Care Improvement Project and future strategies for improving quality in surgery. Am J Surg. Nov 2014;208(5):835-40. doi:10.1016/j.amjsurg.2014.05.005
Stulberg JJ, Delaney CP, Neuhauser DV, Aron DC, Fu P, Koroukian SM. Adherence to Surgical Care Improvement Project Measures and the Association With Postoperative Infections. JAMA. 2010;303(24):2479-2485. doi:10.1001/jama.2010.841
Healthcare Inspection Surgical Care Improvement Project , Report No. 07-00773-106 (2008).
Hawn MT, Vick CC, Richman J, et al. Surgical site infection prevention: time to move beyond the surgical care improvement program. Ann Surg. Sep 2011;254(3):494-9; discussion 499-501. doi:10.1097/SLA.0b013e31822c6929
Itani KM. Fifteen years of the National Surgical Quality Improvement Program in review. Am J Surg. Nov 2009;198(5 Suppl):S9-S18.
Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. Oct 2 2013;8:117. doi:10.1186/1748-5908-8-117
Rabin BA, Brownson RC, Haire-Joshu D, Kreuter MW, Weaver NL. A glossary for dissemination and implementation research in health. Journal of public health management and practice : JPHMP. Mar-Apr 2008;14(2):117-23. doi:10.1097/01.PHH.0000311888.06252.bb
Proctor E, Luke D, Calhoun A, et al. Sustainability of evidence-based healthcare: research agenda, methodological advances, and infrastructure support. Implementation science : IS. 2015;10:88-88. doi:10.1186/s13012-015-0274-5
Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581-629. doi:MILQ325 [pii] 10.1111/j.0887-378X.2004.00325.x
Moore JE, Mascarenhas A, Bain J, Straus SE. Developing a comprehensive definition of sustainability. Implementation Science. 2017/09/02 2017;12(1):110. doi:10.1186/s13012-017-0637-1
Berwick DM. Disseminating innovations in health care. JAMA. Apr 16 2003;289(15):1969-75. doi:10.1001/jama.289.15.1969
Agency for Healthcare Research and Quality (AHRQ). About Learning Health Systems. Updated May 2019. Accessed August 9, 2019. https://www.ahrq.gov/learning-health-systems/about.html
Kilbourne AM, Neumann MS, Pincus HA, Bauer MS, Stall R. Implementing evidence-based interventions in health care: application of the replicating effective programs framework. Implement Sci. Dec 9 2007;2:42. doi:10.1186/1748-5908-2-42
Rogers EM. Diffusion of innovations Vol. 5. Free Press; 2003.
Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. Feb 12 2015;10:21. doi:10.1186/s13012-015-0209-1
Berrios-Torres SI, Umscheid CA, Bratzler DW, et al. Centers for Disease Control and Prevention Guideline for the Prevention of Surgical Site Infection, 2017. JAMA surgery. Aug 1 2017;152(8):784-791. doi:10.1001/jamasurg.2017.0904
Bratzler DW, Dellinger EP, Olsen KM, et al. Clinical practice guidelines for antimicrobial prophylaxis in surgery. Am J Health Syst Pharm. Feb 1 2013;70(3):195-283. doi:10.2146/ajhp120568
Mull HJ, Stolzmann K, Kalver E, et al. Novel methodology to measure pre-procedure antimicrobial prophylaxis: integrating text searches with structured data from the Veterans Health Administration's electronic medical record. BMC Med Inform Decis Mak. Jan 30 2020;20(1):15. doi:10.1186/s12911-020-1031-5
Hamilton AB, Finley EP. Reprint of: Qualitative methods in implementation research: An introduction. Psychiatry Res. Jan 2020;283:112629. doi:10.1016/j.psychres.2019.112629
Hamilton AB, Finley EP. Qualitative methods in implementation research: An introduction. Psychiatry Res. Oct 2019;280:112516. doi:10.1016/j.psychres.2019.112516
Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. Nov 2005;15(9):1277-88. doi:10.1177/1049732305276687
Miles MB, Huberman AM. Qualitative data analysis. Sage; 1994.
Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. Dec 1 2013;8:139. doi:10.1186/1748-5908-8-139
Birt L, Scott S, Cavers D, Campbell C, Walter F. Member Checking: A Tool to Enhance Trustworthiness or Merely a Nod to Validation? Qual Health Res. Nov 2016;26(13):1802-1811. doi:10.1177/1049732316654870

Table 1: Process for Mapping Sustainability Barriers and Facilitators to ERIC Implementation Strategies for Inclusion in Implementation Playbook
Formative Evaluation Barrier/ Facilitator	DSF construct	Example ERIC Implementation Strategy	Definition of Implementation Strategy	Example Tool or Action to be Included in Playbook
Lack of knowledge about pre-incision prophylaxis, and harms associated with prolonged prophylaxis	Providers involved in intervention (intervention)	Conduct educational meeting	Hold meetings targeted toward different stakeholder groups to teach them about the clinical innovation	Educational sessions to focus on developing these skills
Concern about “standard of care” and lack of resources to comply with established practices	Staffing, training (practice setting/context)	Conduct ongoing training; Change clinic system	Plan for and conduct training in the innovation in an ongoing way; Change clinic systems to allow better assessment of implementation or clinical outcomes	Development of webinar and in-person trainings, technology platform to provide new method of collaborating with surgical and infectious disease colleagues across sites
Concern about lack of national-level policy and surveillance program to detect infections	Population characteristics (ecological system)	Develop and distribute educational material	Develop and format manuals, toolkits, and other supporting materials in ways that make it easier for stakeholders to learn about the innovation	FAQ sheets for patients, family members, staff, clinicians prior to Hybrid Type III testing of implementation strategies for sustaining SCIP
Availability of Surgical Checklists, Protocols, and Order sets (facilitator)	Components (intervention)	Capture and share local knowledge	Capture local knowledge from sites on how clinicians made something work in their setting and share it with other sites	Example protocols that can be locally adapted and disseminated; Sharepoint site with resources for facilities

Note: ERIC=Expert Recommendations for Implementation Change (ERIC)²⁰

STROBEChecklist.pdf

PDF

Journal Publication

published 24 Apr, 2022

Read the published version in Implementation Science Communications →

Editorial decision: Accept
22 Mar, 2022
Submission checks completed at journal
05 Mar, 2022
Editor invited by journal
05 Mar, 2022
Editor assigned by journal
03 Mar, 2022
First submitted to journal
01 Mar, 2022

You are reading this latest preprint version

Assessing the Sustainability of Compliance with Surgical Site Infection Prophylaxis After Discontinuation of Mandatory Active Reporting: Study Protocol

Status:

Journal Publication

Version 1

Abstract

Figures

Contributions To The Literature:

Background:

Methods And Scientific Approach:

Discussion And Limitations Of The Approach:

Conclusions:

Declarations

References

Tables

Supplementary Files

Status:

Journal Publication

Version 1