Trial design
This study is a pragmatic parallel randomized controlled trial with a delayed intervention for control participants at six months. We will simultaneously assess effectiveness and implementation strategy in a real world setting (Type II hybrid) (32). The main group comparisons will be at six months. We report this study protocol in accordance with the SPIRIT guidelines (see Additional file 1 for the checklist, see Figure 1 for the SPIRIT Figure) (33) and TIDier checklist (see Additional file 2) (34). We will use mixed-methods data collection, mapping onto the Adoption, Implementation, and Maintenance domains in RE-AIM.
Study setting
To be an implementation site, a primary care practice must meet a list of inclusion and exclusion criteria (Table 1). Included in the criteria is having partnership or access to an organization responsible for oversight of volunteers to implement the home visits and data collection in Health TAPESTRY.
Table 1 Health TAPESTRY implementation site inclusion and exclusion criteria
Primary Care Practice Inclusion Criteria
|
Primary care Practice Exclusion Criteria
|
A primary care practice with a clearly identified practice champion for Health TAPESTRY model
|
No functional connections to an interdisciplinary primary care team to support individual patient assessments
|
Interprofessional primary care team available to provide core implementation components
|
Focused assessment on a single condition or disease
|
Team-based use of an electronic medical record system for documentation or willingness to engage in team-based use
|
|
Able and willing to use the Health TAPESTRY web-based application (TAP-App)
|
|
Partnership or access to a local organization with volunteer infrastructure with the capacity to recruit, train, sustain, and coordinate volunteers, and ensure volunteers have access to the digital health tools needed to fulfill role
|
|
The study will be conducted within Family Health Teams (FHTs) in six communities across Ontario, Canada. A FHT is a type of primary care delivery model within Ontario that formally connects physicians and other health care professionals (e.g., dietitians, social workers, pharmacists) to improve the quality and effectiveness of primary health care services (35). FHT team members are often co-located, but not always.
Participant inclusion and exclusion criteria
Participants must be 70 years of age or older and rostered to a participating primary care physician. Participants will be excluded if they: live in a long-term care facility, are likely to be unavailable for follow-up for the 6-month study period, participated in the first implementation of Health TAPESTRY, or have other conditions or circumstances that would prevent them from engaging with the volunteers or completing the surveys with volunteers (e.g., severe dementia, inability to comprehend English even with caregiver translation). The inclusion and exclusion criteria for participants replicate those of the initial trial of Health TAPESTRY (24,25).
[Insert Figure 1 Study timeline and data collection time points.
Recruitment
The main recruitment strategy is an EMR query run to identify all patients 70 years of age or older and rostered to physicians participating in Health TAPESTRY. Implementation sites (i.e., a participating primary care practice) will develop a recruitment strategy to identify eligible patients that fits the local context. Eligible patients will be mailed an information letter and consent form on behalf of participating physicians at each FHT inviting them to participate. Patients that return a signed consent form will be contacted by a volunteer coordinator to confirm eligibility and review all information within the consent. Secondary recruitment strategies may include providing information in waiting rooms and printed materials for health care providers to hand out.
Randomization and blinding
After informed consent is obtained, participants will be randomized by a volunteer coordinator into the intervention or control group with a ratio of 1:1 using blocked randomization with variable block sizes of 2,4 and 6 (36) through REDCap (Version 9.3.1, Vanderbilt University). Couples who are eligible and both decide to participate in the program will be allocated to the same group by randomizing one person and allocating the second person for the same group. For the main analysis, one person’s data will be randomly selected for inclusion. Health care providers and volunteers will not be blinded; however, there will be some masking of healthcare providers as they will only know a participant is receiving the intervention once a Health TAPESTRY report is received.
Intervention
Procedures and data collection
A participant (termed ‘client’ within the program), regardless of allocation, will receive a visit in their home from two volunteers at the start of the study and again six months later. The volunteers will collect data using structured surveys on the Health TAPESTRY web-based application (TAP-App) covering a variety of domains (i.e., physical activity, nutrition, mobility, quality of life; see Table 2). The volunteers will enter all data into the TAP-App using a tablet computer. The volunteers may return to the client’s home multiple times until all the surveys are complete. Participants may withdraw from the study at any point by informing the volunteer coordinator.
Intervention group
For the intervention group, once all baseline surveys are completed, the TAP-App will create a summary TAP-Report from the information provided. This summary includes the client’s reported goals, key information associated with their answers and/or specific survey scores (Table 2), and volunteer observations. The TAP-Report will be reviewed by a volunteer coordinator and then securely sent using the TAP-App to the TAP-Huddle at the client’s clinic. The TAP-Huddle is an interprofessional team of health care providers who meet approximately weekly. Each TAP-Huddle is responsible for reviewing the report and creating and implementing a plan of care for each intervention client based on this review. The plan of care can involve a number of actions including a clinic visit, telephone call with a health care provider, further assessment, sharing information about or a referral to community resources. The TAP-Huddle can deliver the plan of care themselves and/or request a volunteer to help (e.g., help clients sign up for a community program). The TAP-Huddle can share a patient-friendly report (in plain language) with the client containing the patient’s goals, open-ended responses, and next steps suggested by the TAP-Huddle. At the end of the six months, the Health TAPESTRY volunteers will complete another visit with clients to complete the surveys for a second time.
Control group
Once the baseline visit is complete, control clients will receive usual care. No report will go to the TAP-Huddle until the client received the intervention, nor will any volunteer follow-up take place. After, 6-month research outcomes are collected, and clients will be offered the Health TAPESTRY program.
Volunteers
To manage the incorporation of trained volunteers into the program across sites, we formed a partnership with the Canadian Red Cross and the Windsor Essex Compassion Care Community to recruit, train, manage, and retain the community volunteers. All volunteers will undergo a screening process. Volunteers meeting screening criteria will receive a blended-model training program of online and in-person training specific to the Health TAPESTRY program. A full description of the volunteer program has been published elsewhere (37).
[Insert Table 2 List of all TAP-App surveys here]
Strategies to improve adherence to protocol
The Health TAPESTRY research team will continually monitor study participant recruitment and timelines. Bi-weekly meetings with the study team and volunteer coordinators will be held to ensure fidelity to the trial protocol and huddle process. Volunteer coordinators will provide general oversight for the TAP-Reports and address any issues volunteers may experience related to the visits. Continuing education opportunities for volunteers will be provided to clarify procedures, refresh information from initial training, and provide new information as knowledge and self-efficacy gaps become apparent. Deviations from the protocol will be documented in meeting minutes.
Data collection
Outcome data collection will occur through the structured surveys via the TAP-App, from the electronic medical record (EMR), program records, and focus groups/interviews (Table 3). Health care utilization outcomes will be measured during the 6-month period prior to study enrollment (i.e., before baseline), and during the six-month study period. For the EMR data extraction, all researchers will undergo training and use a standardized data abstraction form that has been pilot tested. We will calculate agreement scores between auditors for a subset of the sample.
Study Outcomes
The outcomes of the study relate to the effectiveness of Health TAPESTRY to determine the replicability of the program, as well as the implementation in six primary care practices. All study outcomes are mapped onto the RE-AIM framework, along with the data source (EMR, self-report, or program records) in Table 3 (30).
Reach: to determine the reach of the study to the target population. Relevant client characteristics will be assessed. Additionally, the proportions of eligible patients who participate and number of volunteer visits will be reported. This data will be collected from the TAP-App and program records.
Effectiveness: the impact of Health TAPESTRY on patient outcomes. The two primary outcomes: the number of hospitalizations during the six-month study period and total physical activity per week (described below) at six months were chosen based on results from the initial RCT (24). Number of hospitalizations will be extracted from the EMR. The reason for each hospitalization at discharge will be categorized into ambulatory care sensitive conditions, or acute care conditions as used to understand the nature of hospitalizations (51-53). Physical activity will be measured using the short form version of the International Physical Activity Questionnaire (IPAQ), using the standard methods of calculating metabolic equivalent of task (MET) (Craig et al., 2003). Secondary outcomes will include time sitting, patient enablement, quality of life, treatment burden, disease burden, emergency room/urgent care visits, falls, medications, primary care visits, and negative effects (see Table 3 for measures).
Adoption: the proportion of physicians and health care providers who consent to participate and a description of volunteer involvement throughout the study will be reported. In addition, to assess how Health TAPESTRY is taken up as normal practice in each site, the validated Normalization Measure Development (NoMAD) survey will be used (54). TAP-Huddle members will answer questions related to implementation processes every three months for one year.
Implementation: three different ways to understand ‘implementation’ will be completed. First, a fidelity checklist specific to this study will be used and completed during later implementation. The two-part checklist was developed by the research team based on reviewing the fidelity literature, and a team discussion about the core program components. One part of the checklist specifically assesses the functioning of the TAP-Huddle. Items were generated from reviewing the literature on ‘best practices’ for team-based care applicable to the TAP-Huddle (communication, roles, organizational support/resources, as well as processes deemed critical to Health TAPESTRY. The second part contains items related to the 4 core parts of Health TAPESTRY. All questions are scored as yes or no. The second way we will understand implementation is by interpreting the qualitative data. Primary health care team members will be invited to participate in focus groups/interview (stratified by site and role in Health TAPESTRY) at least 6 months post-implementation. The question guide will be grounded in NPT (26,27) and centre on implementation barriers and facilitators, interprofessional teamwork, collaboration, and system navigation. All focus groups/interviews will be audio-recorded and transcribed into intelligent verbatim. Finally, the third way to understand implementation is to determine the program’s value for money in regards to costs and quality-adjusted life years (QALYs).
Maintenance: we will report the proportion of providers and patients who would suggest the program to others, each implementation site’s interest in continuing the program after the study has ended. The NoMAD survey results which will reflect the normalization of Health TAPESTRY into current practice by each provider and site. It will also be used as a reflection exercise for sites throughout the study by identifying areas for improvement based on collective survey responses.
[Insert Table 3 Summary of outcomes, measures and analysis plan using the RE-AIM framework here]
Sample Size
A total sample of 500 participants is estimated to be required using hospitalizations as a primary outcome for the power calculation. Of the two co-primary outcomes, a reduction in hospitalizations requires a larger sample size. This estimation is based on means and standard deviations from the initial trial of Health TAPESTRY (24,25), with a power of 0.80 and alpha set at .05. We will aim to recruit up to 600 participants to account for up to 15% attrition, which was found in the first trial. Having multiple recruiting sites will allow us to achieve this sample size.
Data Analysis
The results will be reported according to the CONSORT extension for pragmatic randomized trials (55) and non-pharmacological interventions (56). The mean (standard deviation) or median (first quartile, third quartile) for continuous variables, depending on the distribution, and count (percent) for categorical variables will be calculated. The description of the data analysis approach below is separated based on the RE-AIM framework for effectiveness and implementation. Note that for reach, adoption, and maintenance, the analysis approach is found in Table 3.
Effectiveness: the effectiveness of Health TAPESTRY will be assessed at 6 months. We will adopt intention-to-treat (ITT) as the primary analysis approach. Multiple imputation approach using prediction mean matching (PMM) (57) will be used to impute the missing data. In total, five datasets will be imputed and the pooled effect estimates along with 95% confidence intervals will be reported. The co-primary outcome of a number of hospital admission will be analyzed using the Poisson regression or negative binomial regression depending on the distribution. The incidence rate ratio along with 95% confidence interval will be reported. And the co-primary outcome physical activity will be analyzed using the multiple linear regression. The mean difference between intervention and control group along with 95% confidence intervals will be reported. The continuous outcomes (sitting, patient enablement, quality of life, treatment and disease burden scores) and the count secondary outcomes (number of falls, number of primary care visits, number of emergency room or urgent care visits, number of medications) will be analyzed using the multiple linear and Poisson regression or negative binomial regression, respectively. The mean difference and incidence rate ratio along with 95% confidence intervals will be reported for continuous and count outcomes respectively.
All analyses will be adjusted for baseline values and sites. All statistical tests will be two-sided and all p-values will be reported to three decimal places with those less than 0.001 reported as p < 0.001. The criterion for statistical significance will be set a priori at alpha = 0.05, and will be adjusted using the Bonferroni method for multiple testing for the co-primary outcomes. There will be no adjustment of alpha for secondary and subgroup analyses as these are exploratory. Analyses will be performed using R v3.6.1.
We will perform sensitivity analyses of the primary outcomes to assess the effectiveness of Health TAPESTRY. We will adopt per-protocol approach to assess the effect of Health TAPESTRY at 6-month. In addition, we will assess the effect of Health TAPESTRY without adjusting for sites and baseline values. We will also use zero inflated Poisson or negative binomial distribution for the count data, depending on the distribution, to assess the robustness of the results.
Implementation: as noted above, implementation is assessed in three ways. For fidelity to the program, sites will be described as either high, medium and low adherence to the program as intended. The qualitative data centred on implementation will be analyzed using a descriptive thematic analysis (58) and organized using NVivo 12 (QSR 2018) (59). Transcripts will be independently coded inductively by two reviewers with qualitative analysis experience using open coding. A third reviewer will complete random code checking and provide oversight to ensure trustworthiness of the data. The interview questions will serve as the coding guide for the first few transcripts. The three reviewers will discuss and create a formal coding structure and review it regularly as more transcripts are coded to begin to collapse codes into over-arching themes. The themes will be aligned with the NPT constructs (26,27).
To assess the third part of ‘implementation’ of RE-AIM, an economic evaluation of the trial will be conducted in accordance with Canadian and international guidelines for the conduct of economic evaluations of healthcare programs (60-62). This 6-month trial-based economic evaluation will compare Health TAPESTRY versus usual care in terms of costs and quality-adjusted life years (QALYs) from a public payer perspective. To calculate costs associated with the intervention, healthcare resource utilization (e.g., physician visits, emergency room visits, hospital visits) captured in the EMR (Table 3) will be multiplied by their respective unit costs. Cost associated with the delivery of Health TAPESTRY will be derived from trial data. The volunteer organizations will track front-line program costs (e.g., training, transportation costs for home visit) and research team will monitor all other costs (e.g., development of material, personnel time). Every site may not implement the program exactly the same in which case any cost differences will be accounted for. To measure the impact of Health TAPESTRY and usual care on health-related quality of life, all participants will answer the EQ-5D-5L at baseline and 6 months (Tables 2, 3). The Canadian algorithm will be used to derive the EQ-5D health utility scores (63). QALYs will be calculated by weighted the EQ-5D health utility scores by time spent in health state using an area under the curve approach.
Differences in costs and QALYs will be determined using parametric or non-parametric tests as appropriate, and bootstrap techniques will be used to deal with sampling uncertainty and generate 95% confidence intervals (64). Cost-effectiveness acceptability curves will be used to present the probability of Health TAPESTRY to be cost-effective at different willingness-to-pay thresholds (e.g. $50,000/QALY gained; $100,000/QALY gained) (60,62). Several sensitivity analyses will be conducted to explore the impact of certain assumptions (e.g., cost of implementing Health TAPESTRY) on the results. Missing data will be imputed using multiple imputations (65). The results will be reported as per the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) guidelines (66).
Data management and confidentiality
All electronic information will be password protected and stored on password protected computers in secure networks or on the TAP-App or REDCap software (36). A coding system will be used to protect identifiable information. Only the Health TAPESTRY research team and volunteer coordinators will have access to the master file containing the coding system. Any electronic transfer of data will be done using a secure HTTPS protocol to mitigate risks associated with transferring information over the internet.
Data monitoring, critical and adverse events
Any critical incidents that volunteers encounter during study visits will be reported and followed up by the appropriate personnel using a standard operating procedure. Critical incidents include situations that prevent the visit from proceeding such injury to a volunteer or client and emergency situations. Critical incidents identified by volunteers during the study will be immediately reported and monitored until the issue has been resolved.
Program related adverse events from any source will be recorded. Adverse events are defined as “Any event that requires in-patient hospitalization or prolongation of existing hospitalization, causes congenital malformation, results in persistent or significant disability or incapacity, is life-threatening or results in death” (67). The research team will monitor data conduct and oversee data quality and provide independent outcome adjudication of adverse events as potentially related to intervention or not, and to provide an assessment of the safety data at six months before the intervention is provided to the wait list control group.