Strategies used to implement Hospital at Home: a description of an approach to scaling a complex healthcare intervention

Background Advances have been made in recent years to characterize facilitators and barriers to implementation of complex health care intervention and to classify the implementation strategies available to address these determinants. We study the implementation of a Hospital at Home (HaH) intervention in a multi-hospital health system to understand the selection and use of implementation strategies in its launch, sustainment, and scaling. Methods We report on the implementation portion of an effectiveness-implementation study of the hybrid type 1 design. First, we retrospectively identi�ed determinants of practice most relevant to the HaH intervention using of the Integrated Checklist of Determinants (TICD) assisted by review of archived documents. We also identi�ed implementation strategies using the listing created by the Expert Recommendations for Implementing Change (ERIC) that could potentially address each determinant. Second, we then identi�ed which of the ERIC strategies were actually employed using a modi�ed Delphi process to obtain consensus among HaH program leaders involved in the program implementation. Program leaders also rated the importance and effort expended on each strategy on 1-9 Likert scales. The most relevant implementation strategies identi�ed through these steps were detailed with respect to actors, targets, dosing and justi�cation, and associated with prospectively collected implementation outcomes. Results The majority of ERIC implementation strategies (57 of 73, 78%) were utilized; 7 strategies (10%) were not used. On the remaining 9 strategies (12%), program leaders did not reach consensus regarding utilization. For used strategies, mean importance was 6.87 and mean effort expended was 6.22. Implementation strategies rated most important by program leaders had a broad target of actions that included clinical staff, patients, leadership, external vendors, health plans, and government o�cials. The strategies varied in temporality and dosing. Over the course of the implementation, adoption, acceptance, and penetration increased over time, while measures of �delity remained stable. Conclusions Considerable effort and multiple strategies were required to implement Hospital at Home. While potentially daunting, use of existing implementation frameworks can help focus limited efforts and


Background
Advances have been made in understanding the strategies employed to implement complex health and social interventions.The variants and range of existing strategies, ranging from providing ongoing consultation to mandating change, have been categorized and de ned (1,2) .To enhance understanding of the use and effectiveness of these strategies, methods have been proposed for specifying (3) a strategy's actor, action, temporality, dose, expected target of an action, and justi cation, as well as the expected implementation outcomes that ultimately impact more distal service and patient or client outcomes (4) .
Other work has focused more proximally or upstream on identifying the determinants that either prevent or enable implementation as a precursor to identifying implementation strategies linked to the identi ed determinants.Flottorp and colleagues have categorized seven domains of practice and 57 speci c determinants to used in designing implementation interventions (5) .Each domain (i.e.guideline or intervention factors, individual health professional factors, incentives and resources, etc.) consists of several determinants of practice which could be the focus of speci c implementation strategies.In this way, implementation strategies can be selected and driven by the determinants of particular importance and concern.
To understand the selection and use of strategies used to implement a complex healthcare intervention, we studied the implementation of Hospital at Home (HaH) with 30-day post-acute care follow-up of patients in a seven-hospital system in New York City.For select patients with speci c diagnoses (e.g., pneumonia) who would otherwise be admitted to a hospital bed, acute hospitallevel services (e.g., intravenous antibiotics, uids, oxygen, etc.) and daily clinician visits were provided at home along with durable medical equipment, phlebotomy, and home x-ray as needed.HaH has been shown to be safe, high quality, and cost effective in multiple studies, but it has been neither widely adopted in the United States nor able to achieve substantial scale (6 -12) .We considered HaH a complex healthcare intervention to implement because successful implementation depended on addressing multiple implementation domains and constructs that included characteristics of the intervention, aspects of the inner practice setting, as well as external regulatory and payment concerns. (13)r implementation of HaH (14) was an opportunity to better understand the barriers and facilitators to adoption and how implementation strategies were selected and used to bring about adoption of a complex intervention.In this paper, we report on data collected on the implementation process.We examine what strategies were used, their importance and effort, the determinants of practice they were intended to address, and the implementation outcomes they were intended to impact.First, we identify the determinants of practice that prevent or enable the implementation of HaH.We then enumerate the strategies used by program leaders to implement the program and the principal determinant(s) they were intended to address.Further, for each strategy, we identi ed the phase (planning/implementation, sustainment, or scaling) during which each strategy was used, the relative importance and effort associated with each strategy, and we report on the implementation outcome it would most likely impact.For selected strategies, we examine how the same strategy will differ in its actors, actions, targets, and dosing depending on the stage of implementation.In so doing, we attempt to "connect the dots" from determinants to implementation strategy, to implementation outcomes to illustrate how theoretical frameworks from the implementation science literature can guide strategic and operational decision making in the setting of starting and sustaining a complex healthcare intervention.

Methods
We conducted an effectiveness-implementation study of the hybrid type 1 design (15) .The effectiveness portion of the study has been previously reported (in support of one of the implementation strategies reported in this paper).Using a quasi-experimental design, patient outcomes for those receiving HaH were compared to those for patients meeting the same inclusionary and exclusionary criteria but admitted to a traditional hospital unit.HaH was associated with reduced 30-day hospital readmissions and emergency department revisits, as well as improved patient experience. (14)For the implementation portion of the hybrid design reported here, facilitators, barriers, and implementation strategies used were determined retrospectively from a combination of participant reports, qualitative interviews with key participants, and review of archived documents including proposals and quarterly progress and nancial reports to the funder (16) .Implementation outcomes were prospectively collected by quarter and analyzed in a time-series design.
First, we identi ed determinants of practice most relevant to the HaH intervention using of the Integrated Checklist of Determinants (TICD) (5) .We also identi ed implementation strategies using the listing created by the Expert Recommendations for Implementing Change (ERIC) 1-2 that could potentially address each determinant.Second, we identi ed which of the ERIC strategies were actually employed using a modi ed Delphi process to obtain consensus among HaH program leaders involved in the implementation.The most relevant implementation strategies identi ed through these steps were detailed with respect to actors, targets, dosing and justi cation (4) , and linked to speci c implementation outcomes which are reported.

Patients, Settings, Core Components of HaH
Patients were enrolled in HaH starting in November 2014 for 33 months through August 2017.Patients were identi ed in the emergency departments of Mount Sinai Health System hospitals, or by referral from physicians in outpatient clinical practices or a home-based primary care practice.Inclusion criteria are described elsewhere (14) .Core components of the intervention included enrollment of patients who required hospitalization; delivery of hospital-level services at home instead of the hospital; daily visitation from registered nurses to the home; daily visitation from a HaH clinician (physician or nurse practitioner); and 24/7 availability to patients and family members.We adapted the core components of previously-described HaH models with the addition of 30 days of postacute follow up at the end of the acute hospitalization episode in HaH (17) .

Identifying Determinants of Practice
Determinants of practice speci c to the implementation of HaH were retrospectively identi ed [by RZ and ALS] using the Integrated Checklist of Determinants (TICD) (5) .Determinants were identi ed from driver diagrams originally formulated pre-implementation in 2014 as well as quarterly progress reports prepared for the funding agency over the course of implementation.Implementation strategies were identi ed that might address the speci c determinants identi ed.To achieve consensus on use and importance of individual strategies.We used a modi ed Delphi process with two rounds.The rst round surveyed program leaders regarding which implementation strategies were employed to deal with barriers to implementation.Participants were asked whether each of 73 implementation strategies previously de ned by ERIC were used in the planning and provision of Mount Sinai's HaH program, and if so, during what phase of program enactment (planning/implementation, sustainment, scaling) each strategy was used.Implementation was de ned as the period including all planning and six-months following launch of HaH.Sustainment was de ned as the period after the initial six months of HaH enactment.Scaling involved all activities related to the enactment of HaH at new sites throughout the Mount Sinai Health System as well as broader dissemination.

Implementation Strategies Actually Employed
For strategies identi ed as having been utilized, participants were also asked to evaluate how important each strategy was to further program goals, as well as how much effort was involved in utilizing each strategy using Likert scales.Participants were asked to consider rating importance between "1 -Not important to do, but there may be other reasons to do it" and "9 -So important that you should not bother if you cannot do this."Participants were similarly asked to consider rating effort along a scale between "1 -Discrete amount of effort by a few individuals within a de ned time frame" and "9 -Open-ended collaboration amongst many individuals with an unde ned time frame over at least months."Participants were also given the opportunity to include free text explanations of their votes regarding the use of each strategy.
The second round consisted of a structured discussion moderated by a non-voting member of the research team [RMZ] to reach consensus regarding the strategies.Prior to the discussion, participants were given anonymized survey responses, including used/not used votes, Likert ratings of importance and effort, as well as free text responses of each participant.Participants were also given instructions about the format of the moderated discussion.Strategies that reached consensus during the rst round of the modi ed Delphi process, de ned as all participants voting a strategy was "used" or "not used," were not included in the discussion.Strategies for which consensus was not reached after the rst round were discussed individually.Program leaders were given the opportunity to speak in favor of or against inclusion of a strategy in the nal round.Once discussion concluded, participants were asked to revote and were given the opportunity to revise importance and effort ratings following discussion.
Following the moderated discussion, individual strategies were determined either to have reached consensus or not using the same criteria as the rst round (all participants voting a strategy was "used" or "not used").For each strategy that all respondents determined were utilized, a mean score of importance and effort was calculated.For each strategy that was used, we noted the phase(s) of use that were indicated by at least a majority of raters.

Detailed Speci cation of Selected Strategies and Reporting of Implementation Outcomes
The most important implementation strategies identi ed through the analysis of determinants and consensus process were detailed with respect to actors, targets, dosing and justi cation (3) , and linked to speci c implementation outcomes.We collected information on measures linked to these implementation outcomes.We collected information on the volume of patients by quarter to assess the implementation outcomes of adoption, appropriateness and feasibility of HaH.Similarly, to assess the implementation outcome of delity to operational protocols, we measured the percentage of patients who met Milliman Care Guidelines (MCG) for inpatient admission, and those who subsequently received daily provider home visits.We measured subject consent to be admitted into HaH as a measure of the implementation outcome of acceptability to patients.As measures of the implementation outcome of penetration, we considered the percentage of patients referred directly into HaH (as opposed to being enrolled from emergency departments) and the percentage of patients referred from a hospital other than the hospital where HaH was rst implemented.
We present implementation outcome measures by quarter of HaH patient admission for the 295 receiving HaH during the study period.We used bivariate regression models to examine the relationship between each implementation outcome and the numerical quarter of enrollment after an initial six-month implementation pilot phase.Linear regression was used to model patient volume.
Logistic regression was used to model other outcomes.For the linear regression model, the coe cient is reported, and for the logistic regression models, odds ratios are reported.Models were estimated that included an independent variable for season; results were qualitatively similar, and we report the results for models without seasonal adjustment.

Determinants of Practice and Associated Implementation Strategies
For each domain of practice, relevant determinants or barriers to implementing HaH were identi ed along with potential implementation strategies that might address the determinant.Barriers were identi ed from driver diagrams formulated prior to implementation with respect to feasibility, patient acceptance, referral processes, regulations, and payment.Table 1 outlines select determinants of practice for each domain of practice in TICD.Table 1 illustrates the complexity of the HaH implementation with relevant barriers identi ed for all domains of practice.Plausible implementation strategies were identi ed for each of these determinants.

Modi ed Delphi Process to Identify Strategies Actually Deployed
After the initial round of surveying program leaders, 24 of 73 (33%) of all ERIC implementation strategies had reached consensus.18 as "Used" and 6 as "Not Used."The remaining 49 strategies (67%) did not reach consensus after the rst round.These 49 strategies were discussed in a structured format and subsequently re-scored by program participants in the second round of the modi ed Delphi process.Following a moderated discussion and re-scoring by study participants, 64 of 73 strategies (88%) reached consensus (Table 2).The vast majority of ERIC implementation strategies (57 of 73, 78%) were "Used."Another 7 strategies (10%) were "Not Used."The remaining 9 strategies (12%) did not reach consensus at the end of the Delphi process.
Among strategies that reached consensus by participants as having been used mean importance was 6.87 and mean effort was 6.22 (Table 2).Notably, no strategies were rated in the lower range of importance ratings (1-3), and 23 had mean ratings of relatively high importance (greater than 7).Informing local opinion leaders (mean rating of 3.2) and conducting educational outreach visits (mean rating of 3.8) were rated as involving relatively less effort; however, the remaining strategies were rated as having moderate or high effort (ratings greater than 4).This table also indicates the number of times each strategy was selected by program leaders as having been used during each phase of HaH (implementation, sustainment, and scaling).Almost all strategies were used in initial implementation and sustainment.Strategies in the nancial cluster tended to be more heavily identi ed as having been used in sustainment and scaling efforts.

Speci cation of Selected Strategies and Implementation Outcomes
The relevant actors, actions, targets, temporality, dose, targeted outcome, and justi cation were speci ed for selected strategies.Table 3 presents these speci cations for strategies linked and organized by important determinants.Almost all the strategies involved program leadership (e.g., medical director, program manager and supervisors) as actors.However, several strategies notably involved actors outside core program staff such as legal counsel and contracting o cer, highlighting the importance of being able to engage actors outside the core program staff who may have broader organizational responsibilities.Implementation strategies had a broad target of actions that included clinical staff, patients, leadership, external vendors, health plans, and government o cials.The strategies varied in temporality and dosing illustrating the dynamic, continuing and signi cant effort that needs to be devoted to implementation activities.
Table 4 provides two examples of strategies used across all phases and illustrates how, the speci cs of an implementation strategy may vary across stage.For instance, the creation of new clinical teams may involve different actors in the planning/implementation phase than during program scaling.Similarly, the target of the action and justi cation for developing new clinical teams may vary by phase.
Implementation Outcomes 295 patients received HaH services in lieu of an inpatient hospital admission.A median of 33 patients (range 11-44) received services each quarter during the study period.Patient volume increased by quarter (β = 3.15, SE 0.99, p = 0.013), indicating improved adoption, appropriateness and feasibility of HaH (see Figure 1).Patient acceptance of HaH increased over time (1.22,OR 1.13-1.32,p < 0.0001) indicating improved acceptability of HaH over the course of implementation (see fFg. 2).All but one patient (not graphed) met Milliman Care Guidelines (MCG) for inpatient admission as an indication of delity to the original evidence-based practice; however, there was a non-signi cant trend (OR 0.87, 0.76-1.00,p=0.056) of cases not having a daily home visit over time (seeFig.3).The odds (OR 1.16, 1.04-1,29, p=0.008).ofreferrals coming directly into HaH from home or o ce practice (as opposed to being enrolled from emergency departments) increased over time as did the odds (1.26, OR 1.14-1.40,p < 0.0001) (see Fig. 4) of patients being referred from a hospital other than the hospital where HaH was initially implemented (seeFig.5).These last two measures indicate growing penetration, adoption, and acceptability of HaH.

Discussion
The ndings from this study indicate that determinants that would pose barriers or enablers to an intervention can be linked to speci c implementation strategies.Additionally, implementation of a complex intervention such as HaH involved use of these strategies and many more, all of which were rated to be of moderate or great importance and most of which were perceived by program leaders to involve moderate or greater effort.Use of these strategies were associated with achieving improved implementation outcomes.
The considerable effort involved in implementing many strategies simultaneously could seem daunting and might dissuade many potential program adopters.Most strategies were employed from the initial planning phase through efforts to scale.As a result, opportunities would be limited to signi cantly stagger the introduction of many of these strategies over time.Further, several of these strategies differed in signi cant ways when used in different implementation phases.The relevant targets and actors changed at each phase along with the indicated actions and their dose.Thus, the actual number of strategies employed could be even larger when one considers variations in how a strategy may be executed.
Our ndings illustrate a possible approach to this daunting process by focusing initially and targeting implementation strategies addressing the most important barriers and enablers to implementation of the speci c intervention similar to what Powell has described (18) .. Starting with a review of the relevant determinants as others have proposed (5) , determinants may be prioritized using pilot data and key informant interviews to estimate their situational relevance and likely impact on implementation (TICD Worksheet 3).Implementation strategies could then be selected aided by compilations from the literature (ERIC) based on their likely impact and feasibility (TICD Worksheet 4).Our report indicates that, speci c to the context of what is being implemented, selected strategies may be identi ed as being particularly important and that the effort involved in their use may be estimated.Expected implementation outcomes may be tracked, and that information may be used to further select strategies to target as the implementation proceeds.These considerations may be used in selecting strategies to initially target and to deploy as the intervention proceeds.
Although selected strategies can be targeted, our report indicates that many more than a few targeted strategies may need to be used for complex interventions involving determinants across many domains of practice.In these cases, the selection of implementation strategies may need to consider that many of these discrete strategies are actually closely related conceptually (2) , as well as in the actors and efforts involved in their use.For example, a number of strategies related to training and education (conducting ongoing training, making training dynamic, using train-the-trainer methods are all discrete strategies) may share actors and targets of the action.Efforts to use these strategies may be coordinated to share staff and materials.Additionally, some of these discrete implementation strategies are actually overarching strategic approaches and could encompass a number of other strategies.For example, adaptability is an overarching strategic approach that could include adaptability in other speci c strategies such as conducting educational meetings or in auditing and providing feedback.An otherwise daunting implementation plan can be made less forbidding by careful targeting, staging the use of strategies within related clusters of strategies, and by recognizing overarching strategic approaches in the overall plan.
Our report has several limitations.First, the identi cation of determinants and strategies was performed retrospectively; however, the determinants relied heavily on driver diagrams formulated before implementation initiated, and the implementation strategies were documented in quarterly progress reports over the course of the implementation.Second, use of speci c strategies and ratings of importance and effort were based on report of program leaders, but we were able to achieve consensus on these ratings with multiple raters.Third, it was beyond the scope of this analysis to examine the association between speci c strategies and implementation outcomes.Indeed, such an analysis would be di cult to perform for this type of implementation.The need to employ multiple strategies simultaneously that might be associated with a given implementation outcome would make such an analysis di cult to design.

Conclusions
Implementation of complex interventions targeting multiple determinants of practice may involve using a large array of implementation strategies, and the effort involved in planning and executing these strategies may dissuade potential adopters.Our work suggests that strategies may be identi ed and prioritized for the most important determinants, and that formulating an implementation plan around clusters of related strategies and overarching strategic approaches may be useful for conceptualizing and prioritizing implementation resources.These efforts can lead to implementation outcomes that can be tracked and that are important to achieving improved patient outcomes expected from these complex interventions.Subjects (PPHS).All panelists involved provided written, informed consent under protocol IRB-14-00944.PPHS approved a waiver of informed consent under IRB-17-02565 for the retrospective analysis of patient data presented, as patients were no longer being followed for clinical purposes.These data were collected for grant reporting, quality assurance, and internal program monitoring purposes.
Consent For Publication: Not applicable Availability of Data and Material: The datasets generated and analyzed during the current study are not publicly available as they were measured for internal quality improvement purposes but may be available from the corresponding author on reasonable request.
Competing Interests: ALS, AF, LVD, BM, SL, and EC are full time employees of the Icahn School of Medicine, which in turn has an ownership interest in a joint venture with Contessa Health, a venture that manages acute care services provided to patients in their homes through prospective bundled payment arrangements.ALS, AF, LVD, BM, SL, and EC have no personal nancial interest in the joint venture.Authors RZ, MG, and BL have no competing interests.*Domains of practice and relevant determinants identified using the Integrated Checklist of Determinants (TICD). (5)portance of professional competencies in practice (19) Theory of activated and informed consumers (20)   Matching intervention components with inner setting of the CFIR (13) Matching intervention components to the aspects of the inner setting of the CFIR (13) Adapting intervention from the CFIR (13) Importance of margin in sustaining a healthcare program (21) * One implementation strategy was selected for impactful determinant each of the seven domains practice  (13) Expanded hours of service initiation to include greater night and weekend hours to meet payer expectations; most consistent with Type I Scale-out (population fixed; different delivery system) (21) Dynamic Sustainability Framework posits that intervention success depends on intervention fit with the practice setting and the larger ecological system over time (23) ; scaling effort approaches Type III Scale-out (different population; different delivery system) (22)  HaH trials treated only a handful of diagnoses and had very strict inclusion criteria limited; implementation methods consistent with Plan-Do-Study-Act cycle approach to improving care (24) Expanded hours of service initiation to include greater night and weekend hours meet payer expectations; effort most consistent with Type I Scale-out (population fixed; different delivery system) (22) Dynamic Sustainability Framework posits that intervention success depends on intervention fit with the practice setting and the larger ecological system over time (23) ; scaling effort approaches Type III Scale-out (different population; different delivery system) (22) Figures

Five
program leaders were selected to participate based on longitudinal knowledge of HaH and familiarity with the implementation process.Participants all held leadership or advisory positions inside Mount Sinai's HaH and were involved in the earliest stages of planning through scaling HaH to new sites.Participants also had extensive prior experience with implementation of home-based medical care and HaH, including growth of the largest academic home-based primary care practice in the United States [LDC, ALS, AW] and the rst implementation of HaH in the United States [BL].
AbbreviationsHaH: Hospital at Home TICD: Integrated Checklist of Determinants ERIC: Expert Recommendations for Implementing Change MCG: Milliman Care Guidelines

Table 1 .
Important Determinants of Hospital at Home (HaH) by Domain of Practice, Linked to Implementation Strategy Cluster

Table 2 .
Summary of Implementation Strategies used, Organized by Cluster with Mean Importance and Effort Ratings and Phase of Use * Raters reached consensus that the following strategies were not centralizing technical assistance, developing an implementation glossary, making training dynamic, preparing patients/consumers to be active participants, developing incentive, changing physical structure and equipment, and changing liability laws.No consensus was reached on following strategies: facilitation, identifying early adopters, capturing and sharing local knowledge, promoting network weaving, distributing educational material, facilitating relay of clinical data to providers, involving patients/consumers and family members, increasing demand, and accessing new funding.Implementation strategies used were organized by clusters described by Waltz et al

Table 3 .
Specification of Selected Implementation Strategies

Table 4 .
Illustration of How an Implementation Strategy Specification Varies by Implementation Phase

Table 4 ,
part 2. Implementation Strategy Deployment Varies by Phase