We conducted an effectiveness-implementation study of the hybrid type 1 design (15).The effectiveness portion of the study has been previously reported (in support of one of the implementation strategies reported in this paper). Using a quasi-experimental design, patient outcomes for those receiving HaH were compared to those for patients meeting the same inclusionary and exclusionary criteria but admitted to a traditional hospital unit. HaH was associated with reduced 30-day hospital readmissions and emergency department revisits, as well as improved patient experience. (14) For the implementation portion of the hybrid design reported here, facilitators, barriers, and implementation strategies used were determined retrospectively from a combination of participant reports, qualitative interviews with key participants, and review of archived documents including proposals and quarterly progress and financial reports to the funder (16). Implementation outcomes were prospectively collected by quarter and analyzed in a time-series design.
First, we identified determinants of practice most relevant to the HaH intervention using of the Integrated Checklist of Determinants (TICD) (5). We also identified implementation strategies using the listing created by the Expert Recommendations for Implementing Change (ERIC) 1-2 that could potentially address each determinant. Second, we identified which of the ERIC strategies were actually employed using a modified Delphi process to obtain consensus among HaH program leaders involved in the implementation. The most relevant implementation strategies identified through these steps were detailed with respect to actors, targets, dosing and justification (4), and linked to specific implementation outcomes which are reported.
Patients, Settings, Core Components of HaH
Patients were enrolled in HaH starting in November 2014 for 33 months through August 2017. Patients were identified in the emergency departments of Mount Sinai Health System hospitals, or by referral from physicians in outpatient clinical practices or a home-based primary care practice. Inclusion criteria are described elsewhere (14). Core components of the intervention included enrollment of patients who required hospitalization; delivery of hospital-level services at home instead of the hospital; daily visitation from registered nurses to the home; daily visitation from a HaH clinician (physician or nurse practitioner); and 24/7 availability to patients and family members. We adapted the core components of previously-described HaH models with the addition of 30 days of postacute follow up at the end of the acute hospitalization episode in HaH (17).
Identifying Determinants of Practice
Determinants of practice specific to the implementation of HaH were retrospectively identified [by RZ and ALS] using the Integrated Checklist of Determinants (TICD) (5). Determinants were identified from driver diagrams originally formulated pre-implementation in 2014 as well as quarterly progress reports prepared for the funding agency over the course of implementation. Implementation strategies were identified that might address the specific determinants identified.
Implementation Strategies Actually Employed
Five program leaders were selected to participate based on longitudinal knowledge of HaH and familiarity with the implementation process. Participants all held leadership or advisory positions inside Mount Sinai’s HaH and were involved in the earliest stages of planning through scaling HaH to new sites. Participants also had extensive prior experience with implementation of home-based medical care and HaH, including growth of the largest academic home-based primary care practice in the United States [LDC, ALS, AW] and the first implementation of HaH in the United States [BL].
To achieve consensus on use and importance of individual strategies. We used a modified Delphi process with two rounds. The first round surveyed program leaders regarding which implementation strategies were employed to deal with barriers to implementation. Participants were asked whether each of 73 implementation strategies previously defined by ERIC were used in the planning and provision of Mount Sinai’s HaH program, and if so, during what phase of program enactment (planning/implementation, sustainment, scaling) each strategy was used. Implementation was defined as the period including all planning and six-months following launch of HaH. Sustainment was defined as the period after the initial six months of HaH enactment. Scaling involved all activities related to the enactment of HaH at new sites throughout the Mount Sinai Health System as well as broader dissemination.
For strategies identified as having been utilized, participants were also asked to evaluate how important each strategy was to further program goals, as well as how much effort was involved in utilizing each strategy using Likert scales. Participants were asked to consider rating importance between “1 - Not important to do, but there may be other reasons to do it” and “9 - So important that you should not bother if you cannot do this.” Participants were similarly asked to consider rating effort along a scale between “1 - Discrete amount of effort by a few individuals within a defined time frame” and “9 - Open-ended collaboration amongst many individuals with an undefined time frame over at least months.” Participants were also given the opportunity to include free text explanations of their votes regarding the use of each strategy.
The second round consisted of a structured discussion moderated by a non-voting member of the research team [RMZ] to reach consensus regarding the strategies. Prior to the discussion, participants were given anonymized survey responses, including used/not used votes, Likert ratings of importance and effort, as well as free text responses of each participant. Participants were also given instructions about the format of the moderated discussion. Strategies that reached consensus during the first round of the modified Delphi process, defined as all participants voting a strategy was “used” or “not used,” were not included in the discussion. Strategies for which consensus was not reached after the first round were discussed individually. Program leaders were given the opportunity to speak in favor of or against inclusion of a strategy in the final round. Once discussion concluded, participants were asked to revote and were given the opportunity to revise importance and effort ratings following discussion.
Following the moderated discussion, individual strategies were determined either to have reached consensus or not using the same criteria as the first round (all participants voting a strategy was “used” or “not used”). For each strategy that all respondents determined were utilized, a mean score of importance and effort was calculated. For each strategy that was used, we noted the phase(s) of use that were indicated by at least a majority of raters.
Detailed Specification of Selected Strategies and Reporting of Implementation Outcomes
The most important implementation strategies identified through the analysis of determinants and consensus process were detailed with respect to actors, targets, dosing and justification (3), and linked to specific implementation outcomes. We collected information on measures linked to these implementation outcomes. We collected information on the volume of patients by quarter to assess the implementation outcomes of adoption, appropriateness and feasibility of HaH. Similarly, to assess the implementation outcome of fidelity to operational protocols, we measured the percentage of patients who met Milliman Care Guidelines (MCG) for inpatient admission, and those who subsequently received daily provider home visits. We measured subject consent to be admitted into HaH as a measure of the implementation outcome of acceptability to patients. As measures of the implementation outcome of penetration, we considered the percentage of patients referred directly into HaH (as opposed to being enrolled from emergency departments) and the percentage of patients referred from a hospital other than the hospital where HaH was first implemented.
We present implementation outcome measures by quarter of HaH patient admission for the 295 receiving HaH during the study period. We used bivariate regression models to examine the relationship between each implementation outcome and the numerical quarter of enrollment after an initial six-month implementation pilot phase. Linear regression was used to model patient volume. Logistic regression was used to model other outcomes. For the linear regression model, the coefficient is reported, and for the logistic regression models, odds ratios are reported. Models were estimated that included an independent variable for season; results were qualitatively similar, and we report the results for models without seasonal adjustment.