Pilot Testing a Tracking Tool for High Need, High Cost Patients Enrolled in Complex Care Management in Safety-Net Settings

Irene H. Yen (  iyen@ucmereced.edu ) University of California, Merced Tessa M. Nápoles University of California, San Francisco Sean Tan University of California, Merced Janet K. Shim University of California, San Francisco Leah Hellerstein University of California, Berkeley Mark D. Fleming University of California, Berkeley Ariana Thompson-Lastad University of California, San Francisco Nancy J. Burke University of California, Merced


Introduction
More than 20% of total U.S. healthcare expenditures are generated by only 1% of healthcare utilizers (Cohen and Yu 2012). These patients, sometimes referred to as high-need, high-cost (HNHC) patients typically live with a combination of chronic illnesses resulting in frequent hospitalizations and emergency department visits (Blumenthal et al. 2016; Cohen and Yu 2012; Gawande 2011). Complex Care Management (CCM) programs emerged to mitigate high medical care utilization and the costs associated with utilization. CCM programs are designed to be nite interventions. In other words, patients who are deemed to be HNHC (e.g., hospitalized three or more times in the last 12 months) are enrolled in CCM, where they are provided with coordinated team-based care and self-management coaching, with the intention to graduate them into routine care. Their progress is tracked throughout care delivery until they graduate to routine primary care when they stabilize (Gottlieb, Wing and Adler 2017; Napoles et al. 2017).
Routine care is check-in appointments at some interval, depending on diagnoses, where vital signs are monitored and lab results are reviewed, e.g., blood pressure for patients with hypertension or Hb1ac for patients with diabetes. Increasing patient activation or "the ability and willingness to manage one's health and healthcare," is a core goal of CCM programs (Hibbard et al. 2004; Napoles et al. 2017). Key strategies to increase activation include teaching patients to recognize and address triggers which cause acute events that could lead to hospitalizations (Gottlieb, Wing and Adler 2017). Addressing the triggers earlier could mean seeking a routine appointment, adjusting a medication dosage, or changing or adding a medication. When CCM programs increase patient activation, patients are more likely to experience better health outcomes and health care organizations are more likely to experience cost savings (Gottlieb, Wing and Adler 2017).
In safety-net care settings, HNHC patients are more likely to experience social challenges such as housing instability and food insecurity (Blumenthal et al. 2016;Gawande 2011). As HNHC patients in safety-net settings experience both medical and social challenges, CCM providers must attend to both sets of needs as they are aware that addressing social challenges supports their medical management. Existing tracking tools do not track the social challenges. Therefore, a tracking tool that documents both the medical and social circumstances for these patients in a way that can show change over time is needed (Gottlieb,  regardless of how long they were in the program and even if CCM staff saw progress. In addition, CCM staff indicated that PAM questions were overwhelming to their patients and that they could not determine whether a patient's social circumstances had changed using the tool. PAM assesses the extent to which patients can manage their conditions with items such as understanding of symptoms and triggers (e.g., what makes things worse) and a patient's relationship with their doctor. From the ethnography, we learned that relationship-building between CCM staff and patients was an ongoing process. For example, knowledge of a patient's symptoms or triggers was often insu cient for CCM staff to establish patient engagement and the patient's social and economic challenges further undermined patient engagement.
The PAM is the most commonly used measure for tracking patients. CCM staff were not aware of another option. PAM was developed and is generally used effectively with predominantly White, college-educated population, different from the HNHC patients seeking care in safety net settings (Napoles et al. 2017). In order to address the need for a tool that would be more suitable, we developed the Contextual Health Assessment of Social Stability (CHAOSS) tool to capture the experiences of HNHC patients in CCM programs in safety-net settings beyond tracking patient activation and progress, alone. In conceptualizing HNHC patients enrolled in CCM programs in safety-net settings as structurally vulnerable (Quesada, Hart and Bourgois 2011), the CHAOSS tool serves as a means to take into account patient experiences more holistically and more importantly, provide a greater level of information for CCM staff to determine the appropriate care plan for HNHC patients in safety-net CCM programs.
The concept of structural vulnerability addresses the link between poor health outcomes among poor and marginalized individuals and their chronic experiences of "economic exploitation." Structural vulnerability views the poor health of the socially marginalized being the outcome of physical and emotional suffering rooted in "economic exploitation and cultural, gender/sexual, and racial discrimination" (Quesada, Hart and Bourgois 2011). For example, in prior reports using our ethnographic research, we document the extent of the economic and housing challenges patients experience, and how these challenges exacerbate their health outcomes ; Thompson-Lastad et al. 2017; Van Natta et al. 2018). As a structurally vulnerable population, HNHC patients seeking care in safety-net settings require a different approach in managing their health and therefore different approaches to track their experiences of such care.
This study sought to achieve two objectives: 1) Examine the demographic and clinical characteristics of HNHC patients receiving care in CCM programs in two urban safety-net hospitals, including components/indicators of structural vulnerability, and 2) Describe the pilot testing of the CHAOSS tool designed to track patient activation, progress, and structural vulnerability using cognitive interview techniques. In these ways, we document who is receiving care in the safety net and how progress can be detected for these patients.

Methods
This study emerged from the Health Literacy Systems in the Safety Net (HEALSS) ethnographic study of two CCM programs situated in different public safety-net hospitals (Fleming et

Study Objective #1
Semi-structured Interviews & Survey. CCM patients were recruited for one-on-one interviews. Eligible patient participants were at least 21 years of age and pro cient in spoken English or Spanish. A study information sheet was presented during clinic observations and a verbal description of the study was also given to the patient participants. In-person semi-structured interviews focused on patient's experiences with the CCM program and their thoughts about their past, present, and future health. At the conclusion of the interview, patients completed a brief questionnaire that included items on race/ethnicity, highest level of education attained, income, and housing circumstances. Patients consented for their medical records to be abstracted to determine their medical diagnoses, medications, visits, and hospitalizations.
Medical Records Review. A manual chart review was conducted between January and August 2016. For each patient included in our sample at each site, our study gathered healthcare utilization data (i.e., emergency department visits and hospitalizations over 12 months after enrollment in CCM) and medical characteristics from three separate electronic health records (EHR) per site that were not synced with each other. At CCM program A, one EHR provided information on medical visits, while the second EHR included emergency department visits, hospitalizations, and specialist visits. At CCM program B, one EHR recorded medical/behavioral health visits, while the second EHR recorded emergency department visits and hospitalizations. Each CCM program site also had a speci c case management EHR system with detailed provider notes from which additional social characteristics were gleaned for each patient. To collect the relevant information, a chart reviewer reconciled data collected from all six EHR systems to generate a full picture of patient conditions, medications, and utilization overtime for each patient.
Completing a chart review required approximately three hours per patient. All chart review data were then entered into REDCap-a secure, web-based data capture application.
One of the objectives of the chart review was to record the medical complexities of HNHC patients enrolled in CCM programs and in urban safety-net settings. To do so, we categorized the medical diagnosis list in each patient's electronic medical chart into Elixhauser comorbidity categories and used the most up-to-date medications list to quantify the number of medications taken regularly by each patient (Elixhauser et al. 1998).Another objective of the chart review was to quantify healthcare utilization at the beginning of enrollment in a CCM program. To achieve this objective, dates and reasons for each emergency department visit and hospitalization were recorded and then aggregated.

Study Objective #2
CHAOSS is an 18-item questionnaire informed by conversations with the CCM directors and providers and the Patient Activation Measure (PAM) tool to capture the experiences of HNHC patients in CCM programs in safety-net settings including patient activation, progress, and social circumstances. CCM staff told us that progress could be re ected in development of rapport with the patient, the patient exhibiting a grasp of how to keep their conditions in check and having access to the resources to do so, and patient appearance at appointments. The question items were developed and circulated to CCM team members and the overall research team for feedback. Initially, questions had four response choices modelled after the PAM. CCM team and research team members all indicated that patients would be challenged by having four response options. If at all possible, questions were re-worded for yes or no responses. The rst question of the CHAOSS tool is open-ended, the remaining questions have two or four response choices. The questionnaire includes topics such as patient's health goals, patient's ability to manage their own health, patient's experiences with healthcare providers, and reliance on social support.
Cognitive Interviews. Our study utilized cognitive interviewing to test whether the CHAOSS questionnaire could accurately capture the experiences of HNHC patients in CCM programs in safety-net settings.
Cognitive interviewing is "an evidence-based, qualitative method speci cally designed to investigate whether a survey question, whether attitudinal, behavioral, or factual in nature, ful lls its intended purpose" (Willis and Artino 2013). To recruit cognitive interview participants, the study team collaborated with CCM staff who provided potential names and phone numbers of patients who might be interested in participating in the study. Fifty patients were contacted resulting in 20 patients consented for interviews.
Ten interviews were conducted in English and 10 in Spanish in various places such as patient's home, a coffee shop, a local park and the hospital where the study was being conducted.
The cognitive interviews were performed as semi-structured interviews wherein study participants responded verbally to the CHAOSS questionnaire (Washington Group on Disability Statistics 2005). Study participants were also asked exploratory prompts to understand their cognitive processes toward formulating a response to the CHAOSS questionnaire (Beatty and Willis 2007). We used several types of exploratory prompts including general probes (e.g., tell me more about (blank)), speci c probes (e.g., So, you wake up early and take your medications?), and comprehension/interpretation probes (e.g., what does (blank) mean to you?) to draw out patients' reactions, thoughts, interpretations and/or potential di culties understanding a particular question that had been asked (Willis and Artino 2013). These exploratory probes also enabled us to discern whether certain CHAOSS questionnaire items generated potential response errors or varying interpretations of the questionnaire.
Semi-structured interviews were recorded with a digital recorder and later, the conversations were transcribed. A review of responses from the cognitive interview was conducted using the CDC's cognitive model of question-response, which categorizes a respondent's cognitive process for determining an answer to a question into "four stages: comprehension (Stage 1), retrieval (Stage 2), judgement (Stage 3) and response (Stage 4)" (Washington Group on Disability Statistics 2005). In doing so, we determined if the CHAOSS questionnaire was worded properly and identi ed any instances when respondents did not understand the question being asked representing what the CDC's cognitive model describes as a "cognitive response error."

Results
Study Objective #1 -Examine the demographic and clinical characteristics of HNHC patients. Table 1 shows the demographic characteristics of HNHC patients enrolled in CCM programs in the safety-net based on the chart review (n = 60). Seventy percent of patients perceived their health as "fair" or "poor".
6 Ibid. 7 11 Ibid. 12 Ibid. 13 Ibid. 14   post-schizophrenic depression, bipolar affective disorder, and persistent mood affective disorders), were diabetic (57%), and a third of patients had substance abuse disorder. Furthermore, the median was diagnosed with four conditions (range 1 to 9). We also found that patients were prescribed a mean of 16 medications (median = 16; range 4 to 30). During the rst year that patients were enrolled in a CCM program, 61% were hospitalized and 87% visited the emergency department (ED). Of the 45 individuals with healthcare utilization data, patients were admitted to the hospital a mean of 2.3 times (median = 1; range 0 to 14) and visited an ED a mean of 6.5 times (median = 3; range 0 to 52) in the rst year of CCM enrollment. The average number of days spent in the hospital was 8.9 days (SD = 13.0) and the average length of stay per hospitalization was 2.6 days (SD = 3.6). Acute care utilization data includes 45 of the 60 patients due to patients having not yet completed the rst 12 months of enrollment in a CCM or with an unknown start date for enrollment in a CCM. Note: these conditions are from the Elixhauser comorbidity list Table 3 illustrates the demographic pro le of the patients who participated in the cognitive interviews (n = 20). Overall, 55% of respondents identi ed as male. Forty-ve percent were in the age range of 50-59, 35% were in the age range of 60-69, 5% were in the age range of 70-79, and the rest were below the age of 49. Our population was Latinos (60%), African Americans (20%) and Whites (20%). Seventy-ve percent were very low-income (i.e., making less than $1000 a month) and 55% of patients had a high school education or less. Study Objective #2 -cognitive interview pilot testing of CHAOSS. The CHAOSS questionnaire was designed to track medical and social experiences of HNHC patients enrolled in a safety-net CCM program.
The questionnaire went over a patient's health and health goals, a patient's interactions with CCM staff including their doctor, nurse, and social worker as well as their social circumstances such as the availability of food, quality of their living conditions, and transportation needs. (We provide the CHAOSS tool in the Appendix.) Below are examples of how we utilized the CDC's guidelines and the four cognitive stages to examine the appropriateness of CHAOSS as a tool for HNHC patients seeking care in safety-net CCM.

Patient's Health
We begin with an example based on a question that is commonly used in health surveys. On the questionnaire, patient-participants were asked, "Today, my health is…" (Question 3) to determine their perceived health status. The answer choices were Poor, Fair, Good, and Very good. After probing about their answer selection, patient-participants associated experiences of "bodily pain," "fatigue," "depression," and "side effects from medication" to their current health status. A patient-participant described being of "good" health by stating, "my breathing is right, right now. You don't hear me breathing. I don't have anxiety. I'm not overthinking nothing." Using CDC's model for evaluating questions and response, the patient-participant's answer was salient to the question. They described characteristics that make up their health in the current period (comprehension). The patient-participant also relied on memory to refer to time when they had di culty breathing (retrieval), gauged whether they were having di culty breathing at the current time or not (estimation/judgment) and provided an actual response (response). Other patient-participants also provided a similar thought process to determine an answer. In this case, we were able to determine the question was understood by the patient-participants.
The following are examples of when a possible "cognitive stage error" occurred when responding to CHAOSS in two areas: Patient health improvements, Patient self-management skills Patient Health Improvements Patient-participants were asked, "My health is improving" (Question 14) with the possible answers of "Yes, No, or Unsure." Patient-participants who said that their health had improved described, "avoiding hospitalization," "feeling good or better about their health," managing their medical condition properly or showing improvement, "taking new medications," sobriety, "feeling less fatigued," and "having a better diet" were part of their thought process for determining an answer. In contrast, patient-participants whose health did improve described "not feeling well," "feeling constant pain," "not sleeping well," or "unable to control blood pressure".
In one of the interviews, a patient-participant said, "I don't know. I'm getting tired of this. I just try to walk and eat right. Like the other day, I just take the opium and ate pea soup for lunch. If I eat, I feel better." In this example, the patient-participant seemed to have trouble answering the question. But they could also have been irritated by the length of the cognitive interview, which led them to state, "I don't know." Despite describing some uncertainty, the patient-participant demonstrated that they understood the question by providing an instance when they felt better (comprehension). The patient-participant also recalled the instance when they said, "the other day" (retrieval), determined how that particular activity on that day affected their health (judgement/estimation), and concluded with an answer (response).

Patient Self-Management Skills
Another possible cognitive stage error was observed when a patient-participant was asked, "I know how to stay out of the hospital" (Question 13). The patient-participant explained that "Well...Obviously I'm a patient at the hospital. There isn't anything for me to harm myself to come to the hospital for. For that question, I guess, there is no way for me not to be a patient at the hospital". In this case, the patientparticipant's explanation was ambiguous. While they did not perceive any condition that could lead them to future hospitalizations, they also suggested that they were not able to avoid being hospitalized. In this case, the patient-participant's response represents several cognitive errors. The patient-participant did not understand the question by providing an unexpected response (comprehension error) and had di culty gauging their ability to self-manage (judgement/estimation error). During the interview, the patientparticipant was asked a similar question, "how do you avoid being admitted in the hospital?" for clari cation. However, they did not provide a more reliable answer. In this case, the questionnaire item requires revisions to improve how the question is presented to the patients.

Discussion
We learned that CCM patients in safety net settings in our overall ethnographic study were seriously ill and structurally vulnerable. We learned from the ethnographic observations that patients would make progress and that much of the progress was undergirded by CCM staff addressing patients' social needs from the cognitive interview pilot testing, we determined that CHAOSS is a comprehensive measurement tool for capturing patient experiences enrolled in CCM programs in the safety net. Patients were also engaged with their health provider team such as getting advice about their medical conditions and medications, asking questions about social services as well as following prescribed care routines to manage and improve their health. In addition to tracking patient progress, CHAOSS documented whether a patient was experiencing a lack of access to food, homelessness or housing instability, and gauged their level of social connectedness with friends and family.
This study used cognitive interview methods to pilot test CHAOSS. Our cognitive interview patient participants were introduced to us by CCM program staff. Arranging to meet to conduct the interviews could have been an additional burden for the CCM patients. As we learned through the medical chart review, CCM patients had serious medical conditions and high social needs. Those who were able to pilot test CHAOSS with us were likely less medically compromised than typical CCM patients. Our cognitive interview sample population may also have consisted of patients who were more likely to seek care at the hospital or attend their scheduled appointments. The patients who participated are likely less medically and socially compromised than the overall CCM patient population. While this is a limitation of this study, the CDC's guide to cognitive interviewing stated that the primary objective of cognitive interviewing is to generate an "in-depth investigation of particular concepts, processes, and patterns of interpretation rather than produce a statistical and representative sample of the general population". 14 Therefore, using the current sample was su cient and appropriate for the purpose of pilot testing CHAOSS.
Language differences can also pose as a barrier to the process of cognitive interviewing. However, we found that the were only slight differences between English-and Spanish-speaking patient participant responses during the pilot testing of CHAOSS. For the cognitive interviews conducted in Spanish, respondents had emphasized notions of family such as getting support and resources from their family more than English-speaking respondents. Spanish-speaking participants talked about family support as part of their care more than English-speaking participants because they tend to live with their spouse, children or relatives while English-speaking respondents often lived alone. Aside from these characteristics, no other notable differences were found between the interviews conducted in English or Spanish.
In the future, we aim to investigate how CHAOSS measures HNHC patient's ability to graduate to routine care using a larger sample. Data collection will be conducted at two or more points in time to capture changes in a patient's health and social needs. With data collected at two or more points in time, we can examine the mean and variation in the score(s) as well as score changes over time. Also, this envisioned larger study presents an opportunity to conduct psychometric analysis of the CHAOSS items (e.g., whether responses can be broken down into different subscales). Ultimately, this future study aims to test the validity of the CHAOSS tool and enable CCM staff to use the tool to develop better care and services coordination for HNHC patients.

Declarations
Ethics approval and consent to participate: All protocols are carried out in accordance with relevant guidelines and regulations. The Institutional Review Boards of the two CCM program institutions, as well as of the Human Research Protection Program (HRPP) at the University of California, San Francisco, approved this study. Informed consent was obtained by all participants in the study.
Consent for publication: Not applicable.
Availability of data and materials: The datasets generated and/or analyzed during the current study are not publicly available because they include identi able data but are available from the corresponding author on reasonable request.