This systematic review protocol was registered with PROSPERO on 20/08/2022 (Registration ID: CRD42022352333) and is reported in accordance with the Preferred Reporting Items for Systematic review and Meta-Analysis Protocols checklist (PRISMA-P) (28).
Eligibility Criteria
Types of studies
We will include any randomised study with a control group that aims to assess the effect of a strategy or group of strategies to sustain the implementation of a chronic disease prevention EBI in a clinical or community setting. We will include the following types of studies:
• randomised controlled trial (RCT) (with a parallel control group)
• cluster randomised controlled trial (C-RCT) (with a parallel control group and at least two clusters randomised to each group)
• stepped-wedge trial
• cross over (only data prior to crossover will be used in the analysis)
We will restrict the review to this set of designs for pragmatic reasons due to the size of this review. Further, these designs are considered as gold standard for assessing casual effects, so are most appropriate for addressing the research questions. We will only include studies that compare a strategy or group of strategies to improve sustainment of a physical activity, healthy eating, obesity prevention, smoking cessation or harmful alcohol use EBI (also termed as policy, practice, or program) with no sustainment intervention or ‘usual practice'. There will be no restriction on the length of the study follow-up period due to the varied definitions of sustainment within the literature. There will also be no restriction on country of origin or language. However, we will exclude studies that are not focused on assessing the effect of a sustainment strategy on the sustained implementation of a policy, practice, or program as a specific aim.
Types of participants:
We will include managers, policy makers, staff, clinicians or volunteers delivering, or supporting the delivery of, EBIs to patients in clinical settings including hospitals, GP surgeries, community health centres and charity-based health programs or initiatives (e.g., charity run smoking cessation and healthy eating interventions in low socioeconomic countries/areas).
We will also include managers, policy makers, staff or volunteers delivering, or supporting the delivery of, EBIs to end users in community settings including educational settings (i.e., primary and secondary schools, colleges and universities), childcare services (long day care, family day care, preschools and nurseries), elite or non-elite sports organisations and clubs (professional and amateur sports clubs, sporting governing bodies), and community centres (youth centres, community outreach centres).
Types of interventions:
We will include any study that employs a strategy or group of strategies with the explicit aim of sustaining the implementation of a smoking cessation, healthy eating, physical activity, alcohol or obesity prevention policy, practice, or program by usual staff, clinicians or volunteers within the setting, for example, managers, policy makers, nurses, doctors, teachers, and carers. Studies embedding principals of sustainability into strategies that have a primary aim of increasing adoption or implementation of EBIs will be excluded. Strategies designed to support sustainment will be classified based on the sustainability-explicit expert recommendations for implementing change (ERIC) glossary (29). To be eligible, strategies designed to support sustainment must be distinct from continuous quality improvement (CQI). Distinctions will be made between sustainment and CQI by recognising CQI as studies focused on making immediate improvements to an individual organisation (30). This is compared to sustainability trials which are typically designed based on theoretical frameworks or models and focused on making generalisable improvements, rather than being restricted on one individual organisation.
Types of outcome measures
Primary outcome measures
Studies that report on any objective or subjective measure of the sustainment of a health prevention policy, practice, or program within any of the eligible settings will be included. This may include the ongoing delivery of physical activity, dietary, alcohol or smoking cessation interventions in line with public health or clinical guidelines.
Sustained implementation must be a measure of usual staff or volunteer delivery of the policy, practice or program and not be externally supported by research personnel, except for the purposes of data collection. Individual outcomes such as sustained effects of patient’s participation in a program (e.g., their participation in a healthy eating program) are not considered sustainment outcomes.
Secondary outcome measures
Data on secondary outcomes will only be extracted for those studies that first meet the eligibility criteria for the primary review outcomes. For example, if a study aims to sustain the implementation of a physical activity policy practice, but reports on dietary outcomes and physical activity practices, only data regarding physical activity practices will the extracted.
Secondary outcomes include:
1. Health outcomes where an EBI or initiative is used to target modifiable health behaviour risks related to chronic disease. I.e., any objective or subjective measure of diet (e.g., fruit/vegetable intake), physical activity (e.g., minutes of physical activity during the school day), sedentary behaviour (e.g., daily minutes of sedentary time), weight status (e.g., BMI (body mass index)), alcohol consumption (e.g., number of standard drinks consumed on a typical drinking day) and smoking cessation (e.g., weekly number of cigarettes smoked). A hierarchy will be used to prioritise multiple measures of the same health outcome.
2. Cost outcomes relating to estimates of absolute costs, the assessment of the cost-effectiveness or budget impact of strategies designed to support sustainment.
3. Any reported adverse effects of strategies designed to support sustainment. This may include negative impact on health outcomes (e.g., an increase in injury rates following physical activity initiatives), disruption to service operation or staff attitudes (e.g., negative impact on staff motivation or cohesion), or negative consequences to other key programs or practices (e.g., lack of funding for other vital programs due to reallocation of funding).
Search methods for identification of studies
We will conduct searches for peer-reviewed articles in relevant electronic databases. We will also hand search relevant journals and reference lists of included studies and related reviews.
Electronic searches
We will conduct searches in the following electronic databases: the Cochrane Central Register of Controlled trials (CENTRAL) (2022) via Cochrane Library; MEDLINE (1946 to November, 2022), PsycINFO (1950 to November, 2022) and Embase (1947 to November, 2022) via OVID; CINAHL (November, 2022) via EBSCO; and SCOPUS (November, 2022) via SCOPUS; Education Research Complete (November, 2022) via EBSCO.
Search Strategy /Search Terms
Search terms will be developed based on reviews conducted by Shelton et al. (16) (maintenance/sustainability); Wolfenden et al. (9–11) (physical activity, nutrition, and obesity, implementation, and setting) and will cover the following four concepts:
1) Sustainability (other terms include maintenance, durability, continuation, institutionalization, routinization, normalization, integration, adherence)
2) Heath behaviours (e.g., physical activity, healthy eating, smoking cessation etc.)
3) Clinical settings (e.g., hospitals, general practice etc.)
4) Community settings (e.g., schools, workplaces, community centres etc.)
Data Collection and Analysis
Selection criteria
The search results from the electronic databases will be managed and duplicates identified using EndNote. The de-duplicated library will be imported into Covidence software, where article screening will occur. Both title and abstract and full-text screening will be conducted independently by two members of the research team, who will assess study eligibility according to the inclusion criteria. Any conflicts will be resolved by consensus. In instances where the study eligibility cannot be resolved via consensus, a third review author will make the final decision.
Data extraction and management
Two review authors unblinded to author and journal information will independently extract information from the included studies. We will record the information extracted from the included studies in a data extraction form, developed based on the recommendations of the Cochrane Public Health Group Guide for Developing a Cochrane Protocol (31). The data extraction form will be piloted before the initiation of the review. Data extraction discrepancies between review authors will be resolved by consensus or by a third review author if required.
We will extract the following information:
1. Study eligibility as well as the study design, date of publication, EBI, country, the demographic/socioeconomic characteristics of the program and participants, the number of experimental conditions, setting, overall study duration, and time points measured.
2. Characteristics of the strategy designed to support sustainment, including strategy description and duration of initial implementation support and length of time since withdrawn (if noted), duration of strategies (i.e., duration for which the sustainment strategy was in place), description of strategies, the theoretical underpinning of the strategy (if noted in the study), process evaluation measures (e.g., acceptability and appropriateness), information to allow classification against the sustainability-explicit ERIC glossary (29). Strategies will be described in line with Proctor et al. (14) recommendations for reporting implementation strategies.
3. Primary and secondary outcomes within each study, including the data collection method, validity of measures used, effect size and measures of outcome variability
4. Source(s) of research funding and potential conflicts of interest.
Assessment of risk of bias in included studies
Overall risk of bias
Two review authors will assess risk of bias independently for each review outcome using Version 2 of the Cochrane risk-of-bias tool for randomized trials (RoB 2) described by Sterne et al. (32). Signalling questions will be used for the following domains: Bias arising from the randomisation process, bias due to deviations from intended interventions, bias due to missing outcome data, bias in measurement of the outcome, bias in the selection of the reported result, and overall bias. The response options to the signalling questions will be: ‘yes’, ‘probably yes’, ‘probably no’, ‘no’ and ‘no information’. Once the signalling questions are answered, a risk-of-bias judgement and one of three levels (low risk of bias, some concerns, or high risk of bias) will be assigned to each domain. Stepped wedge trials will be assessed for risk of bias using RoB 2, with consideration given to time confounding. Crossover trials will be assessed using the RoB 2 extension for crossover designs, and only the initial segment prior to crossover will be used in the analysis. We will use the ROB2 extension for cluster trials for the assessment of the risk of bias for cluster RCTs, which includes consideration of the following additional domains: recruitment bias, baseline imbalances, loss of clusters, incorrect analysis, contamination, and compatibility with individually randomised trials. An overall risk of bias will be assigned to each study outcome giving consideration to all of the above domains. Overall risk of bias for study outcomes will be assessed against set criteria and judged as follows: ‘low risk of bias’ (“the trial is judged to be at low risk of bias for all domains”), ‘some concerns’ (the trial is judged to raise some concerns in at least one domain, but not be at high risk of bias for any domain), and high risk of bias (the trial is judged to be at high risk of bias in at least on domain OR the trial is judged to have some concerns for multiple domains in a way that substantially lowers confidence in the result) (Higgins et al., 2022). The risk of bias of the included studies will be documented in a 'Risk of Bias' table.
Synthesis methods
Study characteristics will be grouped as types of studies, participants (i.e., clinical or community) and strategies designed to support sustainment. Strategies designed to support sustainment will be classified using the sustainability-explicit ERIC glossary (29). The sustainability-explicit ERIC glossary is a taxonomy which categorises and defines strategies designed to support sustainment. It is an adapted version of the original ERIC (33), which has been extended with a specific focus on sustainability. The sustainability-explicit ERIC glossary will allow us to code the strategies in this review based on the standardised definitions included in the glossary. Deductive and inductive coding approaches will be used and any strategies that do not fit within the sustainability-explicit ERIC glossary will be added. The effect of interest will be intention to treat, and we will prioritise differences between groups at follow-up, rather than differences between groups in the change from baseline. Primary outcomes will be reported using odds ratios and any primary outcomes measured as means and standard deviations will be transformed into odds ratios (Higgins, et al., 2022). For secondary outcomes, the most appropriate effect type will be used, which will include odds ratios for dichotomous outcomes and means for continuous outcomes. Random effects meta-analyses will be undertaken to estimate a pooled treatment effect overall for the primary outcome and by health behaviour for secondary outcomes (i.e., physical activity, alcohol consumption, dietary outcomes, and tobacco use). If we are unable to conduct a meta-analysis due to insufficient or incomplete data (e.g., missing standard deviations) that cannot be estimated from the data reported by authors, we will synthesise results using vote counting based on the direction of effect (31), with such methods reporting in compliance with the synthesis without meta-analysis (SWIM) guidelines (34). For trials with multiple follow-up periods, we will use data from the final follow-up period reported. For studies that report multiple results for primary and secondary outcomes we will prioritise the most objectively measured. Results from cluster and individual-level RCTs will be combined. The standard error from cluster trials that do not adjust for clustering will be adjusted for unit of analysis errors following recommended procedures outlined by the Cochrane Handbook (31). Trials reporting multiple, relevant intervention arms will be combined into a single group following methods outlined in the Cochrane Handbook (31).
Sensitivity analyses
Where there are sufficient studies, a sensitivity analyses removing studies with high risk of bias will be undertaken. If imputation of intra-class correlation coefficient (ICC) values to adjust for clustered trials is required, a sensitivity analysis assessing different ICC values will also be conducted.
Assessing heterogeneity and subgroup analyses
Statistical heterogeneity will be assessed by reviewing the distribution of studies on the forest plots and assessing the I2 statistic. Pre-specified sub-group analyses will be undertaken to explore possible causes of statistical heterogeneity and will include time period classified as sustainability, and type of setting (i.e., clinical or community). Differences between sub-groups will be statistically compared following procedures recommended by the Cochrane Handbook; within subgroup differences will not be interpreted.