Survey distribution and eligibility
Between January 1, 2015 and February 28, 2017, Project INSPIRE enrolled 2,775 patients across the two clinical sites, while care coordination and data collection were conducted through August 31, 2017. Enrollment eligibility into INSPIRE required individuals to be HCV-positive NYC residents over the age of 18 and beneficiaries of Medicaid and/or Medicare.
The baseline survey was distributed by the care coordinators between August 2016 and March 2017, and the post-intervention survey was distributed between November 2016 and September 2017. Survey participation was voluntary.
INSPIRE participants enrolled up to one month prior to the start of survey distribution were eligible for the baseline survey. INSPIRE participants who were discharged and re-enrolled into the program were considered eligible for the baseline survey if they had received no or minimal INSPIRE services (no more than one health promotion session and/or no initial assessment) prior to their first discharge. INSPIRE participants were eligible to receive a post-intervention survey within one month of their last visit with their care coordinator regardless of whether they had completed a baseline survey.
Survey and data collection
Each survey packet included an information flier, the survey in English (6 pages) or Spanish (8 pages), an informed consent form, and a stamped, pre-addressed envelope to return the survey to New York City Department of Health and Mental Hygiene (NYC DOHMH). All materials were available in English and Spanish and distributed according to patients’ language preference. Surveys and informed consent forms were to be completed by participants without the assistance of their care coordinator or clinician (the care team). Respondents were directed to contact the NYC DOHMH evaluation team using the phone number on the information flier with questions or for clarification in order to limit care team involvement. Participants also had the option to call the evaluation team and complete the survey over the phone. For each returned baseline or post-intervention survey, respondents were mailed a $25 gift card. Respondents were contacted by phone if response clarification was necessary or questions were unanswered. All survey packets contained the same information and all surveys asked the same questions, although during the first few weeks of survey deployment a few minor changes were made to the survey itself. These included adding graphics to the cover page to make it more appealing, changing the responses from fill in the blank to categorical for number of ED visits and hospital nights in the past 6 months and removing the patient satisfaction questions from the baseline survey since the questions were only intended for post survey respondents.
Secondary data source
INSPIRE clinical sites submitted program data to NYC DOHMH in a clinical database which included participant name, contact information, sociodemographic information, dates of enrollment, care coordination services, treatment initiation and completion, discharge and SVR/cure status. First and last name of respondents and enrollment site listed on the returned surveys were used to identify respondents in the clinical database to link the two datasets.
Measures of interest
The main goal of the analysis was to assess changes in overall self-reported health status and health-related behavior, HCV-related knowledge, and self-efficacy in regard to managing health between baseline and post-intervention. Seven outcomes of interest were collected in both the baseline and the post-intervention survey: self-reported overall health (measured using the single general self-rated health question (GSRH) (15)) (outcome 1); self-reported number of emergency department (ED) visits (outcome 2), number of nights hospitalized (outcome 3), any drug use (outcome 4), and any alcohol consumption (outcome 5) in the past 6 months; HCV knowledge (assessed using a 14-question HCV knowledge scale adapted from the validated Brief HCV Knowledge Scale) (16) (outcome 6); and respondent self-efficacy (assessed using a 13-question self-efficacy measure adapted from the validated Patient Activation Measure) (17). The self-efficacy measure was sufficiently different from the original Patient Activation Measure, so the original scoring guide could not be used to score the adapted measure. Exploratory factor analysis (EFA) and subsequent factor scoring was instead used to generate respondent scores (outcome 7). Both surveys also included questions on demographics, specifically education and marital status.
Secondary goals were to measure patient adherence to their HCV medication and patient satisfaction with their HCV care coordinator, HCV medical provider, and the INSPIRE health promotion sessions at the end of the intervention. HCV medication adherence was assessed via two adapted questions (18) asking about the respondent’s adherence during the preceding four weeks. Respondent satisfaction with their care coordinator and clinician was measured with questions from the Patient Reactions Assessment (19) regarding quality of information received, perceived quality of communication, perceived affection, and perceived respect. Satisfaction with health promotion sessions was assessed using ten questions developed by the INSPIRE evaluation team. Responses were recoded into Agree (agree + strongly agree) and Neutral/Disagree (neutral + disagree + strongly disagree), in keeping with patient satisfaction analysis convention (12). Responses of “not applicable” were coded as missing.
Factor analysis
To generate self-efficacy scores, we used EFA to identify latent constructs (“factors”) underlying responses to the self-efficacy questions in the survey and to score responses based on this factor structure (20). The data were checked and met the assumptions for EFA analysis (21–25). Factors were then extracted using principal axis factoring and oblique rotation (24), taking into account non-normal distribution of responses (26) and communalities (multiple questions defining the same factor) (20, 25). A one-factor structure was hypothesized, based on prior assessment of the unmodified Patient Activation Measure (17) and confirmed based on factor retention guidelines (24, 25). Each patient’s self-efficacy question responses were then scored and standardized based on this one-factor structure. To account for the subjective methodologies used to determine factor retention (27) and compute factor scores (20), a sensitivity analysis was performed using respondents’ mean self-efficacy score generated from the original Patient Activation measure scoring guide in place of factor score. Crude and adjusted generalized estimating equation (GEE) models for a respondent’s mean self-efficacy score were calculated and the results were compared with models using factor scores.
Statistical analysis
Response rates
Survey responses were classified as complete, partial, break-off, ineligible, and lack of response (“refusal”), per American Association for Public Opinion Research (AAPOR) guidance (28). Surveys with responses for > 80% of the questions were counted as complete, those with responses for 50–80% of the questions were counted as partial, and those with responses for < 50% of the questions were deemed too incomplete to process (break-off, ineligible, or lack of response) and subsequently excluded from analyses. Response rates were calculated using the AAPOR definition for mail surveys of specifically named persons (all surveys were returned via mail except for 2 conducted over the phone), both for only complete surveys and for complete and partial surveys. The AAPOR publicly available Survey Outcome Rate Calculator version 4.0 (mail_SN) was used to conduct the calculations (29).
Power limitations
Baseline and post-intervention surveys received from the same respondent were classified as “paired”; when only the baseline or post-intervention survey was available, it was classified as “unpaired.” Because of the limited number of paired surveys available, we included all eligible surveys. In order to maximize power, we avoided sub-analyses because of small numbers, and we only used covariates shown to be important for the outcome in existing literature rather than based on statistical significance (13, 30).
Summary analyses
Descriptive statistics, including statistical tests of association, were used to compare sociodemographic characteristics between individuals with paired and unpaired surveys and between baseline and post-intervention survey respondents within the unpaired group. We evaluated differences between (paired vs unpaired) and within groups (unpaired baseline vs unpaired post) to determine what results in subsequent analyses we could attribute to the effect of INSPIRE versus differences in response rates (unpaired group). Wilcoxon-Mann Whitney tests were used to assess continuous variables, and chi-square tests were used to assess categorical variables. By comparing the between and within groups together, rather than directly baseline and post, all observations were independent and we did not need to account for the overlapping nature of the sample. Prior to analyses, values of “unknown” and “refused” were coded as missing and excluded from all summary analyses. Following descriptive analyses, missing values for survey variables were imputed (see Missing Data below). Differences in HCV knowledge were assessed by comparing the percentage of correct responses among baseline and post-intervention questions using chi-square tests that accounted for the overlapping sample (31). For the summary analyses, data about alcohol and IDU use in the last 6 months were collected at time of enrollment during the comprehensive assessment while for the outcome analyses survey responses were used.
Missing data
To include all eligible surveys in the multivariable analyses, multiple imputation (MI) was used to preserve surveys with missing responses. Given that social desirability bias likely influenced the response rates to some questions, we assumed that the data were missing not at random (MNAR). However, we chose to proceed with MI because the technique can still produce unbiased results under the MNAR mechanism (32). After assessing associations between various demographic or survey variables and survey completeness using t- and chi-square significance tests, the final variables included in the imputation process were survey type, age, gender, race/ethnicity, marital status, social support, HIV status, mental health status, and all survey variables assessed as one of the seven primary outcomes specified above, except for self-efficacy. Missing values for the self-efficacy questions were not imputed to simplify subsequent EFA.
MI with chained equations, generating 20 imputed datasets, was performed (33) using all variables included as predictors for imputation for each variable with missing values and using marital status and survey type as auxiliary variables. MI was performed using the IVEware stand-alone SRCWare package (34). To account for the MI, PROC MIANALYZE was used in the outcome analyses to generate a single estimate for each regression coefficient by synthesizing the values from all imputed datasets.
Outcome analyses
The changes in the following outcomes before and after receiving the INSPIRE intervention were assessed using GEE regression models: (1) overall health, (2) number of ED visits in the preceding 6 months, (3) number of nights spent in the hospital in the preceding 6 months, (4) any alcohol and (5) any drug use in the preceding 6 months, (6) HCV knowledge, and (7) self-efficacy. The self-reported health outcomes (1–3) were analyzed as ordinal multinomial variables, while the self-reported behavioral variables (4 and 5) were analyzed as binary variables. The HCV knowledge score (range: 0–100%) and self-efficacy scores (range: 0–5) were analyzed as continuous variables (6 and 7).
GEE regression models, with exchangeable correlation structure, were used because of the non-independence of the paired baseline and post-intervention survey responses, and the non-normal distributions of some variables [35–38]. To account for the exchangeable correlation structure for binary and ordinal outcomes, the alternating logistic regression (ALR) method was used in the models for these outcomes. The ALR approach uses odds ratios (ORs) to model within-respondent associations and calculates the odds of a change in the response to a given measure in the post-intervention survey relative to the baseline survey (35).
The initial confounders used in all adjusted models were age at enrollment, gender, race/ethnicity, education level, social support, HIV status, and mental health status, selected based on preexisting literature (13, 30) since not all variables included in the summary analyses were included in the model due to over dispersion. To account for model non-convergence, variables found to not contribute to the model based on the literature, followed by testing the statistical significance with the outcome, were removed (36). Race/ethnicity was removed from the alcohol use model, and race/ethnicity and education level were removed from the drug use model.
All analyses were performed using SAS 9.4 (Cary, North Carolina).
Patient satisfaction with their care team and adherence to medication were summarized and the proportion of responses for each question were calculated. Patient satisfaction was grouped into categories of information, communication, and respect for each member of the care team.