Two Quasi-Experimental Pilot Trials to Inform a Nationwide Embedded Pragmatic Clinical Trial on the Aliviado Dementia Care-Hospice Edition program

Background: Most investigators studying Alzheimer’s disease and related dementias (AD/ADRD) are unfamiliar with embedded pragmatic clinical trials (ePCTs) and pilot studies designed to inform ePCTs. This paper provides a much-needed example for the pilot phase proceeding a nationwide AD/ADRD ePCT, reporting feasibility, applicability, and delity results. It also discusses implementation barriers and additional infrastructure and implementation strategies developed following the pilot phase. Methods: Two iterative, quasi-experimental pilot trials were conducted in two hospice agencies sequentially to rene and test the Aliviado Dementia Care-Hospice Edition program, a quality improvement program consisting of dementia care training; a toolbox of assessment instruments, symptom management algorithms, care plans, caregiver education sheets; and clinical workow changes. Participants were 72 interdisciplinary team (IDT) members at the two hospices who met the eligibility criteria. To demonstrate readiness for the full ePCT, three milestones had to be met: ³80% training completion rate (“feasibility”), ³80% post-training survey respondents indicating intention for practice changes (“applicability”), and at least one Aliviado care plan/assessment instrument administered in ³75% of dementia patients admitted to home hospice within one-month post training (“delity”). Results: Sixty-six IDT members (92%) completed the dementia care training. Ninety-three percent of post-training survey respondents (response rate: 82%) reported that they will implement practice changes across pilots. Four patients with a primary diagnosis of advanced AD/ADRD and seven with AD/ADRD as a comorbidity were admitted to home hospice during the one-month post-training period in pilot two. All (100%) had at least one assessment instrument or care plan implemented. Main implementation challenges were related to hospice staff turnover, integration of the Aliviado toolbox materials within the electronic health record, and hospices’ limited research experience and infrastructure. Conclusions: Pilot testing established the feasibility of implementing and embedding Aliviado Dementia Care Hospice-Edition into hospice settings. The program content was applicable to hospice practices where IDT members were able to incorporate assessment instruments and dementia care plans into routine care. In proceeding to a full ePCT, we will automate implementation procedures, add mobile device point-of-care access to the toolbox materials, and incorporate strategies to strengthen long-term research partnerships with participating hospices.


Background
Dementia is a syndrome caused by disease of the brain and characterized by chronic or progressive disturbances of memory, thinking, orientation, comprehension, problem-solving, and other cognitive abilities severe enough to interfere with one's ability to perform daily activities (1,2). The most common causes of dementia are Alzheimer's disease (AD), cerebrovascular disease, Lewy body disease, and frontotemporal lobar degeneration (3). The greatest known risk factor of dementia is advanced age (3). As the population ages, it is estimated that 13.8 million Americans will be living with dementia by 2050 (4).
Hospice is a care model that provides quality compassionate care, symptom management, and emotional and spiritual support for persons facing a life-limiting illness and their family (5). Even though hospice was original conceived for terminal cancer, due to the aging population, AD and AD related dementias (AD/ADRD) have become prevalent diagnoses among hospice recipients (6). In the United States, approximately 46% of hospice recipients now have AD/ADRD either as a principal diagnosis or as a comorbidity (7). Nevertheless, the quality of hospice AD/ADRD care remain suboptimal (8). Pain and behavioral and psychological symptoms of dementia (BPSD) represent two common symptoms among advanced dementia hospice patients (9). However, due to decreased ability to communicate, these symptoms are often under-recognized and under-treated in this population (9). Although evidence-based interventions for managing BPSD exist (10), they have not been tested or systematically adopted in hospice settings (11). Given the scarce real-world interventional research in hospice generally, and especially among persons living with AD/ADRD and their formal and informal caregivers, there is a distinct need to perform embedded pragmatic clinical trials (ePCTs).
ePCTs are "trials that use data collected in the electronic health record (EHR) as part of routine care, or are 'embedded' in routine care, and are a foundational component of such a system " (12). ePCTs are systematically designed studies developed to test the effectiveness (i.e., Stage IV of the National Institutes of Health Stage Model (13)) of an intervention in a diverse array of healthcare organizations under usual care circumstances (14,15). Despite a growing evidence base for effective workforce training to improve dementia care practices (16), few studies have examined how to disseminate this research in an interprofessional manner, and none to our knowledge have been performed in hospice (16,17).
Therefore, an opportunity exists to substantially improve hospice AD/ADRD care by embedding an interdisciplinary intervention into the real-world clinical work ow of actual hospice clinicians and care staff. To facilitate high-quality hospice care including optimal symptom management for persons living with AD/ADRD and their caregivers, we adapted an evidence-based, interdisciplinary intervention for dementia home health care, Aliviado Dementia Care (formerly "Dementia Symptom Management at Home" (18)), for use in hospice. Aliviado is a Spanish word for "relieved".
To ensure successful conduct of an ePCT, a pilot phase preceding the ePCT is indispensable (19). However, in the eld of AD/ADRD research, most investigators lack knowledge in ePCTs and fail to distinguish the key differences between a pilot study for a subsequent e cacy trial and a pilot study for a subsequent ePCT (19). Therefore, to contribute to the current limited knowledge base on how to design and conduct pilot studies to inform a subsequent ePCT in the eld of AD/ADRD research, we share our experience piloting Aliviado Dementia Care-Hospice Edition in preparation for a full-scale nationwide ePCT.
This report aims to examine the feasibility, applicability, and delity of implementing Aliviado Dementia Care-Hospice Edition through two sequential pilot trials. These pilot trials constituted Phase I of a twophase National Institute on Aging (NIA)-funded trial. The a priori plan was that successful demonstration of the pre-speci ed program milestones for feasibility, applicability, and delity described herein would lead to approval of Phase II, a nationwide, ePCT of Aliviado Dementia Care-Hospice Edition in 25 hospice sites. Therefore, in addition to our tests for feasibility, applicability and delity, we utilized this pilot process to re ne our intervention design, identify hospice barriers to participate in ePCTs, understand electronic health record (EHR) and data collection capability, and inform the development of the infrastructure necessary for successful conduct of the nationwide ePCT. This paper adheres to the following reporting guidelines: CONSORT (Consolidated Standard of Reporting Trials) pilot and feasibility checklist (Additional le 1) and TIDieR (Template for Intervention Description and Replication) checklist (Additional le 2).

Methods
The objectives of this study were to: 1. Further re ne the Aliviado Dementia Care-Hospice Edition program for use by the real-world hospice agencies and care teams; 2. Examine the program feasibility, applicability, and delity; 3. Identify potential hospice barriers to participate in ePCTs; 4. Pilot the data collection processes of study variables exported from hospice EHRs; 5. Understand the capability of hospice EHRs to integrate Aliviado toolbox materials (e.g. care plans); and 6. Inform the development of additional infrastructure needed for the subsequent nationwide ePCT.

Design And Setting
This study consists of two sequential, quasi-experimental pilot trials (pre-post, single group design) to allow for iterative re nement of the intervention components between trials. The rst trial (Pilot 1) took place from January to March 2019 at a large hospice (Site 1) that served a diverse patient population across New York City and Long Island. Following integration of all necessary revisions informed by the rst trial, the second trial (Pilot 2) took place from April to June 2019 at a smaller hospice program (Site 2), which was part of a chain in a suburban area in southern California that had a high concentration of Spanish-speakers. We chose this second pilot site because it was culturally and geographically different from the rst site. Each pilot site also had a different EHR vendor. The average daily census at Site 1 was 756 and the average daily census at Site 2 was 150. This study was approved by the New York University Grossman School of Medicine Institutional Review Board (IRB).

Hospice staff
To be eligible, hospice staff participants had to be over 18-years old, English-speaking, members of the interdisciplinary team (IDT), and employed or contracted by the participating hospices for more than 50% of a full-time equivalent unit. IDT members included nurses, social workers, chaplains, ordering providers (physicians, nurse practitioners, physicians assistants), and home health aides. The following hospice staff were excluded: those who were per diem; held a non-clinical care position or did not supervise frontline care staff; left the hospices during the study period; were hired after commencement of the pilot; had extended leave of two or more weeks during the implementation period; or had prior exposure to the intervention. The hospice staff participants were identi ed from a complete staff list provided by the participating hospice sites.
Persons living with AD/ADRD Because the Aliviado intervention program was planned to be re ned after the rst pilot trial, only patients at Site 2 were included. All persons who had a primary or secondary ICD-10 code diagnosis of AD/ADRD [see Additional le 3 for eligible ICD-10 codes] in their hospice medical records and were admitted for home hospice at Site 2 within one month following implementation of the Aliviado program and over the age of 50 were included through a waiver of authorization and informed consent granted by the IRB.

Aliviado Model Of Care
Aliviado Dementia Care-Hospice Edition is a multi-component, interdisciplinary quality assurance and performance improvement (QAPI) program. The program includes: 1. Dementia care training; 2. A toolbox consisting of assessment instruments, symptom management algorithms, care plans, and caregiver education sheets; 3. Clinical work ow changes to integrate toolbox materials into hospices' EHRs; and 4. Mentorship through the Aliviado technical support center (see Table 1).  (28) Symptom management algorithms (n = 3) • Neuropsychological Symptoms Aliviado care plan templates (n = 7) Aliviado caregiver education sheets (n = 20, initially available in English only; Spanish versions created after Pilot 1)

Additional Practice Change Program Components
Mentorship available via Aliviado technical support center and the integration of the Aliviado toolbox materials into participating hospices' electronic health records and/or Intranet.
Abbreviations: BPSD = behavioral and psychological symptoms of dementia; IDT = interdisciplinary team; P-I-E-C-E-S = Physical needs-Intellectual needs-Emotional-Capabilities-Environmental-Social The dementia care training has two formats: 1. Two-day, in-person training for hospice champions (faceto-face group sessions) and 2. Online training for non-champion IDT members (asynchronous, self-paced individual sessions). Hospice champions are hospice employees selected by their agency to serve as the local leaders who foster and reinforce changes for improvement in the Aliviado QAPI initiative (20). The champion training emphasizes the assessment and management of pain and BPSD, as well as quality improvement processes such as the plan-do-study-act cycle.
The champion training was held at the two participating hospice sites. The champion training was led by the study principal investigator and three implementation specialists trained by the study principal investigator. The study principal investigator was a PhD-prepared researcher and experienced boardcerti ed palliative care and geriatric nurse practitioner. The three implementation specialists were PhDprepared researchers in nursing, gerontology, and music therapy. The online non-champion training is tailored for different learner groups: ordering providers; registered nurses; chaplains and social workers; and home health aides. The training length and topics for each learner group are provided in Table 1. The online ordering provider training and home health aide training were developed after the rst pilot trial, in response to feedback from the rst pilot site and the study steering committee. Therefore, they were developed and subsequently implemented as part of the second pilot trial only.
The research team instructed the pilot sites to use the toolbox materials with all their dementia patients but allowed them to decide which assessment instrument(s) or care plan(s) to administer, considering each patient's unique needs. The research team worked with the hospice leadership and their clinical informatics and information technology staff to embed the toolbox materials into their clinical work ow, considering the existing capabilities of their EHRs. Hospice IDT members were encouraged to email the Aliviado technical support center for any implementation or dementia care-related questions with patient identi ers removed, which would then be forwarded to an appropriate research team member or an external expert.

Implementation strategies
To achieve improved quality, the Aliviado program should ideally be implemented in accordance with a continuous quality improvement cycle (Fig. 1). Main strategies to aid implementation include 1.
Purposeful selection of hospice champions to re ect the most frequent skilled disciplines in the patient's home-nursing, social work, and spiritual care/chaplaincy, and the inclusion of team and executive leaders; 2. Hospice leadership involvement in communicating and preparing for implementation; 3. Weekly training progress reports to champions and hospice leadership to reinforce completion; 4. Continuing education credits to incentivize completion; and 5. Periodic check-ins with the champions via email or conference calls to informally survey implementation challenges and answer study-related questions.

Measures
Milestones for the pilot phase were pre-speci ed and approved by NIA and captured key activities that needed to be accomplished before transiting to the full ePCT phase. Milestones were developed using behavior change and implementation science principles (39,40) and a prior pilot for an ePCTs as a model (41). The feasibility milestone required at least 80% completion of training. The applicability milestone required at least 80% of the IDT members who completed the post-training survey to indicate they will implement changes in their practice. The delity milestone required at least 75% of advanced dementia patients admitted to home hospice within one month following implementation to have at least one Aliviado assessment instrument or care plan completed by their hospice care team members. See the Aliviado Model of Care section above and Table 1 for more information about the assessment instruments and care plans. The delity milestone was only applicable to Pilot 2 because the intervention components were expected to be revised after the rst pilot trial. The actual usage of the Aliviado care plans and assessment instruments was determined based on Pilot Site 2's EHR report.

Piloting Full-scale Epct Data Collection Processes
We also sought to test the feasibility of data collection processes that would be used in the full-scale ePCT. We therefore collected de-identi ed outcomes data from each pilot site to assess our ability to identify patients living with AD/ADRD from routinely collected hospice data, as well as to collect the proposed primary outcome (antipsychotic use) and additional variables of interest (Table 2).

Results
A total of 113 hospice staff were screened (Fig. 2)  Missing data: Thirteen participants in Pilot 1 and four participants in Pilot 2 did not report their age. Eight participants in Pilot 1 and three participants in Pilot 2 did not report their race/ethnicity. Two participants in Pilot 2 did not specify their gender. No additional missing data were found. With respect to applicability, 54 of the 66 training completers (82%) submitted the post-training survey (26 in Pilot 1 and 28 in Pilot 2). Of these, 96% of the post-training survey respondents in Pilot 1 and 89% of the respondents in Pilot 2 indicated that they will implement practice changes.
In terms of delity, we only examined results from Pilot 2, since the intervention components were planned to be revised after the rst pilot trial. Four patients with an ICD-10 primary diagnosis of advanced AD/ADRD and an additional seven patients with an ICD-10 secondary diagnosis of AD/ADRD were admitted to home hospice during the 1-month post-implementation period. Demographics are omitted here due to the small sample size. All 11 patients (100%) had one particular assessment instrument in the Aliviado toolbox administered during this speci ed period, the PAINAD. (28) Two of the four patients with a primary diagnosis of advanced dementia (50%) had a PAINAD score greater than two, indicating the potential presence of pain; both had pain medications ordered. In addition, two patients with a primary diagnosis of advanced dementia (50%) and three with a secondary diagnosis of dementia (43%) also had a care plan implemented; all were for anxiety/agitation management.

Iteration Of Intervention
Between the two pilot trials, the iteration of the intervention focused on the creation of the online ordering provider training and the online home health aide training. Three out of the six home health aides (50%) who received the training commented that 2 of the 14 training videos were too long (12 and 15 minutes long). In response, these two videos were each divided into two parts, resulting in a total of 16 videos. After the second trial, a 17th video entitled "Conclusion" was added to recap and reinforce the overall training purpose. Considering that some of the aides speak Spanish as their primary language, Spanish language versions of these 17 videos were created.

Pilot Testing Of Data Collection
We successfully received data from both pilot sites; however, the ability to obtain requested elements ( Table 2) was different in each system. For the Site 1 data, we were able to receive all outcomes metrics, demographic data, and care plan usage data requested in a csv format through direct data pulls from its EHR. However, the EHR had limited capacity to integrate new assessment instruments, resulting in hospice clinicians' use of hard copies of the assessment instruments and our incapability to obtain the assessment results e ciently. For Site 2, we were able to integrate both assessment instruments and care plans into its EHR and receive reports back that included the required data elements. However, the utilization reports for assessment instruments and care plans were produced through primary chart review conducted manually by the hospice staff. The EHR did potentially have the capability to capture these items but would have required a custom report. Encouragingly, feedback from this site suggested that it would be reasonable to develop and embed such a customized report into the EHR if the program were to be implemented for long-term use rather than just for pilot testing. The two pilot sites also provided feedback for our list of eligible ICD-10 dementia diagnoses to ensure alignment with the coding requirements for day-to-day hospice practices. The nalized ICD-10 list (Additional le 3) was then used to identify participants with AD/ADRD in this study.
Establishing the infrastructure needed to address identi ed barriers and to facilitate implementation of the future full-scale ePCT Some hospice staff participants reported di culty accessing the Aliviado toolbox materials integrated into their EHR at patients' residences due to limited Internet access. To address this barrier, an Aliviado mobile application (app) was developed after the pilot phase to allow point-of-care access to all toolbox materials, including completion of assessments and care plans that can then be transferred to the agencies EHR. Another important barrier to the intervention implementation that we learned during the pilot phase was hospice staff turnover. To e ciently track longitudinal hospice staff turnover and assign all IDT members to the accurate learner group in the subsequent ePCT of 25 sites, a full-time operations manager was hired and a customer relationship management software (salesforce™ lightning) was customized and integrated with the Aliviado mobile app and learning management system. This system allows for business process automation to ensure implementation delity across all 25 sites. Another major barrier identi ed was that hospices tend to have limited research experience and infrastructure. In response, we developed the following human subjects infrastructure: assisting hospices to obtain Federalwide Assurance (FWA) numbers required for research participation, created a streamlined contract and single IRB agreement for each hospice, developed a reporting protocol for ongoing hospice staff changes at the hospice leadership and staff levels, and developed additional communication strategies to strengthen long-term research partnerships (i.e. monthly newsletters to hospice leadership, quarterly champion surveys, monthly champion calls, and weekly implementation tips and training reminder mobile push noti cations to all IDT members).

Discussion
The results of this pilot phase support the feasibility of implementing Aliviado Dementia Care-Hospice Edition in hospice settings. The study ndings showed that: 1. Hospice IDT members perceive the training content to be highly applicable to hospice practices; and 2. Practicing IDT members are able to adhere to the program recommendation, adopting assessment instruments and dementia care plans into their routine patient care. Because pain management has been a strong focus of hospice care, it was not surprising that among all assessment instruments provided, PAINAD was the one that the pilot site chose to prioritize. Our follow-up conversation with the hospice leadership revealed that they made PAINAD a requirement, or a "forced" item in their EHR, that the IDT members had to complete. Encouragingly, we found that as a result, PAINAD was administered to all dementia patients and pain medications were ordered accordingly for those who were likely experiencing pain as indicated by their PAINAD score. This nding supports that embedding the Aliviado program into hospice usual care can affect real-world patient symptom management-the extent to which we will formally evaluate in the subsequent, fullypowered ePCT.
On the other hand, we encountered several implementation challenges. First, hospice employee turnover and extended leave was high, requiring weekly adjustments to eligibility and training lists. Moreover, IDT members had limited remote access to the toolbox materials integrated into their hospice's EHR at the patients' residences due to limited Internet access. In addition, even if they had internet access, research shows that clinicians' sitting in front of a computer screen can negatively affect patient-centered practice by diminishing dialogue and reducing their ability to maintain eye contact with the patient and the caregiver (42). This possibly led IDT members to primarily complete elements of the toolbox "forced" in the EHR (i.e. PAINAD) as a requirement versus exibly implementing all assessment instruments in the toolbox to address different patient needs. Furthermore, through the pilot process, we realized that most hospices have limited research experience and infrastructure, such as not having an FWA number or having an expired one. Understanding these implementation barriers not only enabled us to further re ne our intervention and develop the necessary infrastructure for the subsequent 25-site full ePCT, but also contributed to the current limited knowledge on translating evidence-based AD/ADRD interventions into usual care settings.
Recognizing that there is much need to translate existing AD/ADRD research to practice across settings, NIA has recently invested substantially to develop the NIA Imbedded Pragmatic AD/ADRD Clinical Trials (IMPACT) Collaboratory to assist investigators in designing and carrying out pilots in preparation for fullscale ePCTs on non-pharmacologic interventions in AD/ADRD (19). The current study and similar pilot trials can help inform intervention development and iteration, data collection procedures, and even what the primary scienti c outcome of a study can be and therefore, are essential in ensuring that ePCTs are designed with the rigor necessary to have the greatest chance of success. Performing such pilot studies along with stakeholder engagement and alignment can also help to: 1. Improve the readiness for an ePCT, as can be measured using the Readiness Assessment for Pragmatic Trials model (43); 2. Reduce the number of ePCTs that end in null results; and 3. Ultimately save substantial resources and unnecessary efforts, as several recent relatively well-designed trials had some readiness gaps that could have potentially been revealed during the piloting phase (44,45). However, few pilot studies for ePCTs in AD/ADRD research have been performed and limited consensus exists currently on what an "ideal" pilot study would encompass. This study adds to the evidence in this regard as we seek to move the eld of ePCTs forward.
Other strengths of the current study include: First, to the best of our knowledge, this is the rst study to test an agency-wide, interdisciplinary QAPI intervention for hospice AD/ADRD care, which involves active participation of a wide range of hospice staff (i.e. aides, clinicians, leadership, informatics and information technology staff) as well as real-world clinical work ow changes. Secondly, the sequential staggered design in two operationally different hospices allowed iterative re nement of the intervention between pilots incorporating stakeholder feedback. Finally, the pilot sites included contrasting geographical and cultural characteristics, providing preliminary evidence that the QAPI intervention can be adopted to different hospice settings.

Limitations
This study also has several limitations: Although this study purposefully chose two pilot sites to represent different geographical regions and organizational cultures, the study ndings may still not be generalizable to other hospices until similar ndings are successfully duplicated in larger samples of hospices. In addition, the study post-implementation follow-up period was limited to one month and thus only a small number of dementia patients were admitted during this timeframe. Therefore, to further validate the results of this study, the subsequent 36-month, fully powered ePCT of 25 sites is needed for testing effectiveness, sustainment, and generalizability of the intervention.

Conclusions
Although not free from implementation challenges, this study supported the feasibility, applicability, and delity of implementing the Aliviado Dementia Care-Hospice Edition program in hospice settings. It also found that due to the complexity of implementing complex interventions in non-academic, communitybased settings, when scaling up, it is important to automate some of the implementation processes to reduce potential barriers and closely monitor delity of implementation and sustainment. These ndings led to progression to the R33 phase of the award, where we will complete a 25-site stepped wedge ePCT (ClinicalTrials.gov Identi er: NCT04175977) to test both the effectiveness of the intervention in improving care quality for persons living with AD/ADRD receiving hospice as well as its scaling and sustainment.

Consent for publication
Not applicable.

Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Competing interests
The authors declare that they have no competing interests.  Aliviado Continuous Quality Improvement Cycle