Setting and conceptual model
The PCHOOSE study is situated in Los Angeles County (LAC), California, which has the nation’s largest unsheltered homeless population [12]. LAC also represents the country’s largest natural experiment of housing interventions to protect PEH [13] due to a major taxpayer-funded $1.2 billion homelessness initiative passed in 2016 that resulted in the opening of thousands of new PSH units in LAC during the pandemic [14]. The study followed a diverse sample of PEH placed in either PB-PSH or SS-PSH for 6 months; barriers and facilitators that may affect PSH implementation during the pandemic and its aftermath were also investigated. The study was guided by the Gelberg-Anderson behavioral model for vulnerable populations [15] that involves examining predisposing, enabling, and need factors that contribute to health behaviors and patient-centered outcomes (see Fig. 1). PSH is positioned as a key enabling factor for PEH with the hypothesis that PSH type (i.e., PB or SS) will affect quality of life (general life satisfaction; physical, mental, social, and environmental health including housing and neighborhood characteristics) and health behaviors including health care utilization (e.g., receipt of health care, mental health care, and social supports) and COVID-19-related prevention practices. We also expected that changes to our primary outcomes would affect predisposing, enabling, and need characteristics of the PEH population over time.
In describing the protocols of the PCHOOSE study, this paper also highlights the challenges of conducting research with this vulnerable population in the United States during the highly dynamic COVID-19 pandemic. This includes steps taken to protect the research team, PEH participants, and PSH staff members and the need to change survey instruments to reflect an evolving understanding of the COVID-19 virus, public health mitigation policies, and personal protective practices.
Design overview
This study leveraged LAC’s ongoing efforts to provide PSH to PEH during the COVID-19 pandemic using a mixed-methods, prospective longitudinal design. We recruited people moving into PSH and provided them with smartphones with paid data plans to stay connected with the study for a 6-month period. All study participants were recruited through organizations that provide housing or supportive services or both. Although PB-PSH is clearly defined in the homelessness services system, SS-PSH could have been officially categorized as a PSH program or a rapid-rehousing program. Although rapid-rehousing programs are typically short-term housing programs with time-limited and tailored assistance programs, during the pandemic, LAC expanded the use of rapid-rehousing programs coupled with supportive services to quickly provide access to housing. Thus, for this study, we considered anyone receiving rapid-rehousing assistance to be SS-PSH if they also received support services.
For the study’s quantitative arm, participants completed a baseline survey followed by five monthly follow-up surveys once in housing. For the qualitative arm, we recruited a purposive subsample of PEH from both PB-PSH and SS-PSH who participated in three semistructured interviews; the first was an in-depth interview when they initially moved into PSH, followed by two follow-up interviews conducted 3 and 6 months later. Focus groups with housing providers from the affiliated housing agencies were also conducted. Study protocols for each study component, which were approved by the institutional review board at the University of Southern California with a reliance agreement in place with the University of California, Los Angeles, are described separately in more detail.
To ensure that the study addressed questions important to PEH, providers, and policymakers and that data were correctly interpreted, we also established two stakeholder advisory boards. The first advisory board is a lived experience group (LEG), which consists of 11 individuals who have personally experienced homelessness. The LEG meets quarterly and uses its lived experience, knowledge of what is important to PEH, and perspectives on how PEH may respond to study questions to inform recruitment strategies, interpretation of study findings, and dissemination efforts. The study’s second advisory board is a stakeholder advisory board (SAB), which consists of 15 providers, administrators, policymakers, and researchers with expertise on homelessness and housing programs in LAC. Due to concerns about meeting in person during the pandemic, all LEG and SAB meetings have been conducted virtually via Zoom.
Recruitment of study participants
PEH were eligible to participate in the study if they were 18 years or older, had been approved for PSH, could be interviewed in English or Spanish, and were willing to provide informed consent. Enrollment included PEH who had been approved for PSH in LAC and had either been housed in the past 2 weeks or expected to be placed in housing in 30 days. PSH placement is determined through a county-run coordinated entry system, which identifies clients’ needs and matches them with available housing options. Because the system typically assigns individuals approved for PSH to specific nonprofit, community-based agencies that are ultimately charged with securing housing and supportive services, recruitment was conducted through 26 agencies. The study began recruitment in January 2021 amid a significant surge in the COVID-19 pandemic in LAC. Therefore, in-person recruitment was not a viable option and instead depended on case managers at each program who were already interacting with PEH as part of the housing placement process. The study team relied on these agencies to inform anyone approved for PSH (or rapid rehousing who would be receiving supportive services) about the study. Study staff members then coordinated a meeting with the eligible PEH interested in the study via phone or Zoom to complete the enrollment process. Although this process remained largely intact throughout the recruitment period, in-person recruitment occasionally occurred when COVID-19 rates were low and visitors were allowed at program sites.
Recruitment for the qualitative portion of the study involved two sources. The qualitative PEH sample was selected from those who enrolled in the quantitative portion of the study. A subsample of study participants (N = 40; 20 from each PSH model type) was selected for interviews using maximum variation sampling [16] based on demographic and health characteristics (i.e., Black or African American, women, individuals with chronic disease, individuals with serious mental illness, individuals with substance use disorders, individuals aged 60 or older) and were separately consented to participate in qualitative interviews. The provider sample (N = 48; 24 from each PSH model type) was selected from housing providers who worked at agencies that were study recruitment sites. Sites were asked to identify staff members who would be willing to participate in a focus group; staff members were asked to provide verbal consent at the outset of the focus group.
Enrollment and quantitative data collection procedures
To enroll, an agency staff member initiated the referral process for PEH by contacting the research recruitment team via phone, email, or text. A research recruitment team member then followed up to schedule an appointment with the potential participant to complete a screening to ensure eligibility, review the consent form, and answer any questions related to the study protocols. Case managers then distributed study phones that had been shipped to the recruitment site. Participants received a Samsung A01 Core smartphone with an unlimited data and calling plan, which allowed them to provide electronic informed consent and complete a self-administered questionnaire sent via text message. Participants were enrolled in the study once they provided informed consent by completing an online form that assessed study comprehension and documented their electronic signature.
Initially, participants completed a 20-minute survey upon enrollment in the study to capture basic demographic and historical information about their housing and health. Baseline outcome measures were then administered in a follow-up survey approximately 1 day later to help reduce the burden involved in completing a lengthy questionnaire at the time of enrollment. Participants received a $15 electronic gift card incentive for completing the baseline outcome measure survey. Participants who enrolled in the study prior to moving into a housing unit were asked to complete a second baseline outcome measures survey once they moved in and received another $15 electronic gift card incentive. After moving into PSH and completing the demographic and baseline surveys, participants received five monthly follow-up surveys and received a $15 electronic gift card incentive for each completed survey. Participants could also complete surveys over the phone if they preferred to speak with a surveyor rather than self-administer the survey. Electronic gift cards were sent to an email address provided by the participant. Survey links were all sent via text message. The study team, in consultation with the LEG, opted to allow “prefer not to answer” as a response option for all historical and monthly survey questions to reduce any frustration among those who were not sure how to answer and minimize the possibility that some questions could be triggering given high rates of past trauma.
Quantitative measures
Patient-centered quality-of-life outcomes in this study were based on the World Health Organization’s definition of health as “a state of complete physical, mental, and social well-being and not merely the absence of disease or infirmity” [17]. Table 1 describes the study’s constructs and measures with the exception of COVID-19-related outcomes, which discussed in more detail subsequently.
Table 1
Study constructs and measures
Construct | Variables and instruments |
PEH patient population characteristics: covariates measured at baseline only [18] |
Predisposing | Demographics (i.e., age, gender, marital status, veteran status); health beliefs (i.e., knowledge of disease, health services attitudes); social structure (i.e., race and ethnicity, education, employment, religion, trauma, homelessness history); systems involvement history (e.g., foster care, criminal justice, trauma); housing preference |
Enabling | Income, insurance, benefits, case management, food insecurity, social support |
Need | Chronic physical disease, serious mental illness, substance use disorder; baseline measures for physical and mental health status as noted in outcomes |
Patient-centered outcomes: quality of life including physical, mental, social, and environment health, measured monthly |
Life satisfaction | NIH Toolbox Item Bank v2.0 General Life Satisfaction (10 items, self-rated life satisfaction) [19] |
Physical health | PROMIS Global Health Scale Version 1.2 (PROMIS; 2 items, self-rated physical health and activities) [19] |
Mental health | PROMIS (2 items, self-rated mental health, frequency bothered by symptoms) [19] |
Social health | PROMIS (1 item, self-rated satisfaction with social activities and relationships) [19] |
Environmental health | Housing retention and Housing Environment Scale (4 items, residential satisfaction; 12 items, neighborhood quality and safety) [20] |
Health behaviors: measured monthly |
Substance use | Collaborating Consortium of Cohorts Producing NIDA Opportunities survey (12 items) [21] |
Health care utilization | COVID-19- and non-COVID-19-related past-180-day utilization of physical, mental, substance use, social services, and PSH support; medication adherence; ambulatory and emergency department visits and hospitalizations; barriers to care |
COVID-19-related outcomes
Given that the study project period began in October 2020 with initial recruitment starting in January 2021, our initial survey questions focused on COVID-19-related protective health behaviors that were salient at the time—namely, social distancing and handwashing. As science and the pandemic evolved, questions about COVID-19 testing and vaccines were introduced as key protective health behaviors. For COVID-19 testing on the monthly surveys, we first asked whether participants had been tested in the past 30 days and if so, the outcome of the test. For anyone who had not been tested in the past 30 days, we asked about barriers to testing (i.e., “Did you want to get tested for coronavirus [COVID-19] in the past 30 days but were unable to?). For those who replied affirmatively, possible reasons for not getting tested included: I didn’t know how to make an appointment to be tested; I didn’t know where to go; I don’t think that I can afford the cost of the test; I didn’t have time to get tested; I am unable to travel to a testing location; I am worried about bad things happening to me or my family if I get tested (including discrimination, government policies, or social stigma).
For vaccination status, questions evolved over time. When the study began and there was only one approved vaccine that was not readily available, participants were asked if they “have been offered the COVID-19 vaccine?” with branch logic on whether they said they received the vaccine or would receive the vaccine if offered. If participants responded “no” to having received the vaccine after being offered or to a hypothetical offer, they were prompted about the reasons for not receiving the vaccine to better understand their hesitancy. Six months into data collection, when there were multiple approved vaccines that were more readily available, the survey was revised to first ask if a participant had received the vaccine rather than if it had been offered. Questions about the type of vaccine (brand) and the number of doses received were also added to the monthly surveys. In June 2022, the survey was revised a final time to ask new questions about receipt of COVID-19 booster vaccines using the following three uniform questions: (1) “How many doses of the COVID-19 vaccine have you received (including booster shots)? Please indicate the brand of each dose you received, and the month and date when you received it”; (2) “To the best of your knowledge, have you received all COVID-19 doses/boosters that you’re eligible for?; and (3) “In the future, how likely are you to receive another dose/booster of the COVID-19 vaccine if it’s recommended?” Participants who had already completed the study and not been asked these questions were recontacted.
Qualitative interview procedures
Each member of the qualitative PEH sample was asked to complete three semistructured qualitative interviews. The first interview, conducted over the telephone and audio recorded, focused on questions concerning participants’ (a) prehousing homelessness experiences; (b) experience obtaining PSH; (c) health, social services, and social experiences prior to and after obtaining PSH; and (d) management of COVID-19-related safety and concerns. These interviews lasted approximately 30–60 minutes each. The second and third interviews were conducted 3 and 6 months after baseline interviews and used photo elicitation interviewing (PEI), a method that uses photographs taken by participants to guide interviews [22]. This method enhances the depth of each interview and increases the capacity to identify potential topics of interest that were not part of the study’s original data collection plan. PEI is also an effective tool for rapport building, gaining access to participants’ lived experiences, and relieving the strain of direct, extended verbal questioning, which can be particularly taxing for participants with cognitive or mental disabilities [23–25]. All client PEI interviews were conducted over the phone and lasted approximately 45 minutes. Each interview was recorded and transcribed for analysis, and participants received a $50 gift card incentive of their choice (Visa, Amazon, or Target) after each interview.
PEI methods
Participants were asked to use their study-issued cellphones to take photos that represent their daily life and experience in their housing placement for a 2-week period. There was no limit on the number of photos they could take, though they were asked to avoid taking any identifiable photos of any individuals to protect privacy. Participants were then asked to choose five to 10 of the photos that best captured their recent experiences and email or text them to the study coordinator, who then provided them to the interviewer. At the outset of each interview, interviewers asked participants to choose whichever photo they wanted to discuss first. For this photo and subsequent images, participants were asked first to describe each photo and then elaborate on how it reflects their recent experiences related to housing, health, service utilization, COVID-19 protection, safety, resources, relationships, personal growth, and daily functioning. Participants were also asked to come up with a title or theme that collectively described the photos they discussed, if there was anything they wanted to take a photo of and could or did not and why, and any recommendations for improvements in services for others experiencing homelessness based on the photos. If any participant did not feel comfortable using the phone camera, they were offered an alternative of taking “mental snapshots,” in which they made a list of five to 10 things they wished to discuss and shared this list with their interviewer. For the second PEI at the 6-month follow-up, participants were asked to go through the same procedures. However, during this PEI, participants were asked to focus on how things changed for them across the different domains and compare their photos from the two PEIs.
Focus groups
Provider group interviews concentrated on learning provider perspectives on the difference between PB-PSH and SS-PSH, the strengths and weaknesses of each model in improving outcomes for PEH, and the impacts of the COVID-19 pandemic on PSH service delivery. Each focus group met for approximately 60 minutes and was conducted either in person or via Zoom, depending on the housing agency’s preference. As with the client interviews, staff focus groups were recorded and transcribed for analysis, and participants received a $50 gift card incentive of their choice (Visa, Amazon, or Target).
Quantitative analysis
The statistical analysis will rely on bivariate and multivariate regression analysis to examine the relationship between PSH (SS-PSH vs. PB-PSH) and quality of life and COVID-19-related health behaviors. For outcomes, we will employ a pre–post design that treats Month 1 as baseline and Month 6 as endline and compares pre–post changes for SS-PSH and PB-PSH. For outcomes sensitive to monthly trajectory variation, we will employ fixed-effects and random-effects models of monthly outcomes that test for interaction between PSH model and months of exposure to each PSH type. These models will be informed by an analysis of baseline variation between SS-PSH and PB-PSH clients. We expect variations in the level of baseline morbidity. Thus, we will employ both multivariate models that account for baseline differences and propensity score matching models that establish an area of common support for SS-PSH and PB-PSH comparisons. We will use calendar month controls to account for historic changes in the ecology of health behavior, particularly in relation to rising and falling rates of COVID-19. We will also address the potential need for multilevel or mixed-effects models to account for potential shared experiences by housing and service providers. Additional models will test for differences in housing outcomes by race and ethnicity, sex, and gender via interaction effects and stratified models. Further analysis will entail the use of a path modeling framework to understand the pathways leading from PSH type to outcomes of interest. Due to the decision to include “prefer not to answer” as an option for all questions, all analyses will account for the frequent use of this response option. In cases where “prefer not to answer” is a common or meaningful response option, such as regarding partisan political affiliation or experiences of trauma, we will include it as a response option. In cases where “prefer not to answer” appears to reflect a random nonresponse pattern, we will treat the value as missing, use multiple imputation to maintain sample size, and conduct robustness checks comparing the imputed and raw samples.
Qualitative data analysis and integration
Client semistructured interview and focus group transcripts have been imported into Dedoose for analysis. Analysis was conducted using template analysis, with thematic codes defined in relation to the study questions (e.g., housing process and experiences, COVID-19 experiences, posthousing experiences), and inductively through coding the transcripts [26]. Individual team members co-coded the transcripts and discussed the codes iteratively, including collapsing, expanding, and combining codes, until consensus was reached on the codebook. Any discrepancies in codes and excerpts were discussed in team meetings and resolved in discussion with the qualitative team leaders [26] After initial co-coding, team members coded the remainder of the transcripts using Dedoose.
PEIs will be analyzed using two approaches—one for the interview text and one for the photographs. Interview text will be analyzed using template analysis as previously described. Photographs are analyzed through a process of initial categorization by theme, referencing interview texts to ensure proper interpretation, and group discussion to develop an initial visual code book. Members of the team will co-code photos from five interviews to test the codebook’s utility and identify codes that needed further definition or clarity. Once the codebook is established, each photo will be coded independently by two researchers, who discussed discrepancies until they reach consensus on all interpretations of images [22].
Mixed-methods analysis
As appropriate, qualitative and quantitative data will be merged to achieve several functions. First, merged data will be used to identify convergences—areas where qualitative data confirm quantitative findings, and vice versa. Second, the research team will use qualitative data complementarily to provide depth and understanding to findings that emerge from quantitative analyses. Third, qualitative data will be used to explain and expand on findings from quantitative data. The researchers will note areas where one data source generates insights but the other does not [27].