Classifying Patients Seen at an Urban Healthcare for the Homeless Site: A Clinically-Driven Latent Class Analysis

Introduction: Federally qualied health centers (FQHC) provide care to over 28 million people in the United States, primarily serving people with low incomes who are underinsured or not insured. FQHC patients have different patterns of illness than non-FQHC populations, which may require tailored interventions to support at the population level. Model based segmentation methods can identify patterns of need but may generate spurious results without clinical context. Here, we used stakeholder feedback combined with machine learning methods to identify subgroups of patients seen by a large FQHC in an urban setting. Methods: We used electronic health record and administrative hospital utilization data to identify subgroups of patients seen at the FQHC in 2017 using latent class analysis (LCA). We designed an activity to gather feedback from physicians, social service staff, and administrators during model design. We trained a nal model using a feed-forward neural network. Results: Using data from 5,985 primary care patients, we identied four candidate LCA models—with 26, 27, 28, and 29 classes—and integrated inter-professional feedback to develop a nal model with 19 clinically meaningful classes. Three classes had greater medical complexity and older age, 15 classes were separated primarily by behavioral health diagnoses, and a nal class had low complexity. Conclusions: Populations served by FQHCs are clinically heterogenous with varying levels of complexity. Use of LCA can provide insights into patterns among FQHC patients, which can be used to inform the development of interventions tailored to the needs of specic classes. This study used a novel quantitative and qualitative approach to classify patients at a single FQHC, synthesizing feedback from an inter-professional team with a data-driven latent class model. The nal neural network model revealed 19 clinically meaningful classes, which are characterized primarily by health needs, age, and hospital utilization patterns. The characteristics of these classes highlight both the complexity and heterogeneity of this FQHC population. Our ndings suggest that despite this heterogeneity, common patterns emerge nevertheless. The classication model provides detailed insight into patterns of patient characteristics, which can be used to inform the development of interventions tailored to the needs of specic classes.


Introduction
Federally quali ed health centers (FQHC) are a mainstay of care delivery for underserved patients across the United States. Since the passage of the Affordable Care Act, FQHC's have grown in capacity, lling a critical role in caring for underinsured and uninsured patients across the United States. In 2020, over 1,400 FQHCs and 14,000 FQHC-look-alike community health centers serve over 28 million patients (1 in 12 people), triple the number of people served in 2000. 1 Compared to non-FQHC patient populations, FQHC patients are disproportionately poor, uninsured, and publicly insured. Patients served by FQHCs have a unique burden of illness compared to non-FQHC populations that often require different healthcare interventions to help address. 2 Improved identi cation of the needs of FQHC patient populations can aid in resource allocation, intervention development, and improved health outcomes. However, characterizing this complexity is challenging. 3,4 Harnessing electronic health records (EHR) data to characterize populations is of high interest to program and policy leaders. However, traditional statistical methods were not designed to identify highly dynamic groups in data. Newer statistical methods, including machine learning techniques, offer opportunities to characterize complexity in heterogeneous datasets, but work is still evolving to fully translate these statistical methods to tools that are useful and meaningful in health services research. 5 Clustering methods (e.g., hierarchical cluster analysis and k-means clustering) may yield statistically signi cant results to previously unanswerable questions. However, the identi ed clusters may not be reproducible or clinically relevant. 6,7 Latent class analysis (LCA) is a model-based method that may improve classi cation but faces similar limitations of reproducibility and clinical applicability. 8 One way to improve the usefulness of latent class analysis is to incorporate stakeholder input into class identi cation and model validation; doing so may improve the applicability of LCA results for populations of interest. 9,10 This paper reports the outcomes of a single FQHC's experience integrating stakeholder feedback on LCA results into LCA model building and validation. The objective of this project was to build a model that identi ed to inform the clinically-meaningful subgroups of patients to guide FQHC strategic planning and policy.

Setting and participants
Old Town Clinic (OTC) is an FQHC operated in Portland, Oregon by Central City Concern, a communitybased nonpro t that provides comprehensive solutions to ending homelessness and achieving selfsu ciency. 11 OTC serves about 6,000 low-income adult patients with high rates of homelessness, mental health conditions, and substance use disorders across ve integrated primary care teams. 12 Overview of Approach This study arose as part of a broader strategic planning exercise to identify needs of OTC's population and guide decision-making to better tailor health services for patients. We integrated stakeholder feedback alongside clustering methods in development and identi cation of clinical subgroups.
Administrators (RS, MS), data analytics and research personnel (MM, BC), and clinicians met regularly to outline priorities for this project and provide feedback on model development. We used LCA to initially de ne groups, then presented these groups for stakeholder feedback and revision in an iterative process.
The results led to creation of a nal neural network-based classi cation model.

Data collection and measures
Patient data were derived from the EHR, including demographics, diagnoses, health program enrollments, and other clinical and billing data. We retrospectively identi ed all primary care patients enrolled at OTC on December 1, 2017, de ned as having at least one visit in the previous two years. We extracted hospitalization and emergency department visit data from Oregon's Emergency Department Information Exchange (EDIE) system. 13 EDIE is a web-based platform that provides access to real-time hospital utilization data and has been broadly adopted by hospitals and health systems in Oregon, Washington, and northern California. In partnership with clinicians, data analysts and researchers, we developed a list of potential variables that might separate patient groups (referred to as classifying variables). We abstracted data for 75 days after December 1, 2017 to ensure that documentation and records were complete.

Variable selection
We considered demographic, diagnostic and utilization variables in our analyses. Our continuous classifying variables-age, total medical hospitalizations, psychiatric hospitalizations, and emergency department visits in the prior 365 days-were grouped into bins. Age was categorized as: 18-29, 30-39, 40-49, 50-59, and 60 and above. Medical and psychiatric hospitalization were each categorized as: 0, 1, 2, and 3 or more. Emergency department visits were categorized as: 0, 1, 2-3, 4-6, and 7 or more. We included bivariate variables (yes/no) for diagnoses of asthma or chronic obstructive pulmonary disease, diabetes, congestive heart failure, hepatitis C, chronic kidney disease, chronic liver disease, severe head injury, psychotic disorders, bipolar disorders, depressive disorders, trauma-related disorders, alcohol use disorders, opioid use disorders, and stimulant use disorders. Using the ICD10 diagnoses recorded in the patient problem list at the beginning of the study period, we grouped physical health diagnosis according to hierarchical condition categories 14,15 and behavioral health diagnosis according to the Diagnostic and Statistical Manual. 16 We include a table of ICD-10 codes used for diagnosis identi cation in Appendix 1. We included binary variables for gender and the presence of any monthly income as well as categorical variables for race and housing status, each pulled from EHR registration data as of the date of each patients' last visit. We considered housing statuses of sleeping on the street or in emergency shelters to be homeless.

Data analysis
We used latent class analysis (LCA) to identify an initial set of subgroups of patients seen at OTC. LCA is an unsupervised machine learning tool that identi es unobserved subgroups of patients in a larger population. 17 We hypothesized that a useful model would have between 15 and 30 classes based on previous quality improvement efforts.
To evaluate class t for the latent class analysis, we compared models using the Akaike information criterion (AIC), Bayesian information criterion (BIC), Log-likelihood, G-squared, and entropy values for each model. We prioritized AIC values in selecting potentially appropriate models to describe our population. [18][19][20] We planned to select the four models with the lowest AIC for the stakeholder feedback phase of model creation. Data analysis was conducted in 2018.

Stakeholder Group Qualitative Validation
To improve chances of identifying clinical meaningful groups for population health management and strategic planning, we sought to incorporate feedback on our models from end-users. Over the course of a month, we engaged an inter-professional team composed of physicians, social workers, substance use counselors, social service staff, quality management professionals, and administrators in a codesign process. The team of 12 individuals represented a cross-section of disciplines, consisting of staff who would use the nal classi cation model in clinical practice, population health management, and strategic planning. In two initial design sessions, the team identi ed prospective variables for the analysis and envisioned possible applications of a classi cation model.
After performing the LCA, we developed a sorting activity to engage the inter-professional team in qualitative analysis of the four candidate models. For the activity, we listed the characteristics of each class of the candidate models on index cards, totaling 110 cards. In small group and individual activities, we asked team members to organize the cards on a table to represent subgroups that matched the patients they served. We asked team members to narrate as they arranged cards to give the research team insight into their thought processes. Once the cards were sorted at the end of the activities, we took photographs of the cards to record the results (Appendix 2). We integrated feedback from the activity to develop the nal classi cation model by identifying recurring clusters of cards and synthesizing key insights from the narration to create the nal set of classes. For example, some participants grouped a card from one model representing a class over age 50 with high prevalence of diabetes, heart disease, and kidney disease with a card from another model representing a class over age 50 with high prevalence of depression, COPD, diabetes, and heart disease. Despite the fact that the high prevalence conditions from these cards were somewhat different, participants identi ed the cards as representing the same realworld class. Although no participants grouped the cards identically, patterns emerged with common groupings by age, behavioral health conditions, and higher numbers of physical conditions. Multiple classes from the latent class analysis were deemed uninformative by the inter-professional team, which led to the consolidation of multiple classes. Based on common patterns and key insights from a few participants, the analysts (MM, MS) synthesized the results from the card sorting activity into a set of proposed classes, which were reviewed with the participants before the developing the nal model.
Based on the synthesis of the card sorting activity and feedback from a review with the inter-professional team, we identi ed a nal set of classes and created a simulated dataset for training a neural network model. The data simulation used observed distributions of patient characteristics as the starting point, and we adjusted the distributions based on the stakeholder feedback from the sorting activity. We included all of the variables from the latent class analysis in the simulated data except for race, gender, and housing status. After a preliminary review of the analysis, the inter-professional team decided that race, gender, and housing should not be used to determine class membership-instead these characteristics should be explicitly considered as a part of population health management for all patients at the FQHC. We trained the nal classi cation model on the simulated data using a feed-forward neural network with two hidden layers and L2 regularization, withholding 25% of the data as a testing set. We then predicted class membership of the original OTC population using the nal model. Although the initial model output is probabilistic, we report a single class for each patient based on the maximum posterior probability.
We used the R statistical programming language version 3.5 for all analyses. 21 The poLCA package 22 was used to conduct latent class analyses and keras was used to train the nal model. 23

Results
We identi ed 5,985 primary care patients enrolled in services. The majority of OTC patients were male (61%), white (71%), and homeless (23%) or living in transitional housing (18%). Behavioral health conditions were prevalent-60% had a substance use disorder and 74% had a mental illness. Of 5,128 people with a behavioral health condition, 55% had at least one co-occurring physical condition. For the entire clinic population, the rate of medical hospitalizations was 369 per 1000 patients in the prior year.
Before the inter-professional team's review, we identi ed four candidate models using LCA-those with 26, 27, 28, and 29 classes-which had the lowest AIC values (Appendix 3). After integrating feedback from the inter-professional team and training the neural network model, we identi ed 19 distinct, clinically meaningful classes of patients. Figure 1 illustrates the characteristics of each class. The classes can be described primarily by diagnoses, hospital and emergency department (ED) utilization, and patient age.
To convey the conceptual relationships among classes, we visualize the 19 classes in Fig. 2.
Of the 19 classes, three classes displayed greater medical complexity (44% have two or more chronic physical conditions) and older age (92% 50 years and older) and comprised 20% of the study population. These classes can be distinguished from one another primarily by hospital and ED utilization. For example, 62% of patients in the Medically Complex C class had 4 or more ED visits while only 2% in the Medically Complex A class had 4 or more ED visits. (Fig. 2) Fifteen of the classes, comprising 65% of the study population, were characterized largely by groups of behavioral health conditions. Two classes were characterized by psychotic disorders, with the Psychosis and Older class having older age (41% were 50 years and older compared to 21%) and lower hospital and ED utilization (6% compared to 68%) than the Psychosis class. Four classes were characterized by bipolar and trauma-related disorders. Feedback from our clinician team indicated that bipolar and trauma-related disorders can be di cult to distinguish clinically, which resulted in the diagnoses being intermingled in these classes. The four classes differed from one another by hospital and ED utilization and by the prevalence of physical conditions. For example, 73% in the Bipolar/Trauma and Medical B class had a co-occurring physical condition compared to just 53% in the Bipolar/Trauma B class. However, both "B" classes had high levels of ED utilization (52% and 91% with 4 or more ED visits, respectively) compared to the low ED utilization of their "A" class counterparts (4% and 0% with 4 or more ED visits, respectively).
Six classes followed a similar pattern: they occurred in pairs that shared a common behavioral health diagnosis (i.e., depression/trauma-related disorders, opioid use, and stimulant use) that were distinguished from each other by average age and the prevalence of physical conditions. For example, in the Depression/Trauma and Medical class, 20% have two or more chronic physical conditions, compared to only 4% in Depression/Trauma class. Three classes of patients with alcohol use disorder displayed a variant of this pattern: two of the classes had older age but comparable prevalence of physical conditions, and the two older classes differed primarily in their hospital and ED utilization rates.
A nal class was identi ed, which had low prevalence of health conditions. In this group 83% had no physical health diagnosis, 79% had no mental health diagnosis, and 79% had no substance use diagnosis.

Discussion
This study introduced a novel method to characterize the population of a single FQHC, highlighting both the complexity and the heterogeneity of patients served by this community resource. The method began with LCA and then used stakeholder feedback from an inter-professional team to aid the development of a nal neural network model. Insights from the inter-professional team informed the analytic process, resulting in the pruning and nuanced characterization of patient subgroups. The unique codesign process enabled the creation of a conceptually cohesive model-despite the large number of classes-by incorporating insights from clinicians during model development.
The methods we used address limitations of existing classi cation methods for identifying population subgroups. Traditional clustering methods can be overly sensitive to initial values, can be skewed by outliers and often rely on naïve distance-based measures. 24 Multiple LCA models with similar model t statistics can produce drastically different results, and some classes in LCA models can be uninformative in practice. When there are multiple candidate LCA models, there is no clear way to identify the "correct" model. 25 Incorporating stakeholder feedback into the model development process aided in increasing end-user acceptability. The inter-professional team provided feedback on the initial set of models, which allowed us to combine similar classes, discard uninformative classes, and draw on the strengths of multiple initial models. This codesign process led to a nal model that corresponded to clinicians' experiences in the real world, building trust in the model improving clinical relevance. Because the nal model was developed with interpretability in mind, it is well-suited for program development and for developing targeted interventions for speci c classes of patients. The nal model in this study has been used to support population health initiatives at the FQHC. We used the model to identify prospective patients for an ambulatory intensive caring unit, a complex care intervention at OTC. 26 We also used the model as a case nding tool for pilot outreach interventions and for recommending new referrals to existing behavioral health services. To aid strategic planning, we made growth projections for the 19 classes and mapped those against the capacity of existing services at the FQHC. This enabled a gap analysis, which informed capacity planning-especially for behavioral health services-and strategies for integrating service delivery. The model provides a ne-grained and clinically relevant characterization of the FQHC's patient population, which has proven useful for case nding, population health management, and strategic planning.

Limitations
Although both the method and model described in this study have promise, there are limitations to this approach. First, EHR data, including diagnoses from the problem list, may be incomplete. Not all data in Conclusions This study used a novel quantitative and qualitative approach to classify patients at a single FQHC, synthesizing feedback from an inter-professional team with a data-driven latent class model. The nal neural network model revealed 19 clinically meaningful classes, which are characterized primarily by health needs, age, and hospital utilization patterns. The characteristics of these classes highlight both the complexity and heterogeneity of this FQHC population. Our ndings suggest that despite this heterogeneity, common patterns emerge nevertheless. The classi cation model provides detailed insight into patterns of patient characteristics, which can be used to inform the development of interventions tailored to the needs of speci c classes. the EHR are well-maintained, and information captured at other service locations may not be available. The large number of patients in the low prevalence of health conditions group may indicate inconsistent use of the problem list in the EHR. Second, while the inclusion of stakeholder information improved applicability of LCA results to a single FQHC, there are trade-offs on replicability and generalizability. However, the methods used to develop the groups could be applicable to other settings and may aid FQHCs in clinical program design when there are limited resources. Future research is needed to determine the stability of class membership over time, to evaluate the effectiveness of classi cation models for guiding intervention development and assessing outcomes, and to assess the applicability of our approach to other populations.

Consent for publication
Not applicable

Availability of data and materials
The datasets analyzed during the current study are not publicly available because they contain personally identifying information and protected health information. However, the code for generating the simulated dataset used for training the nal model is available on Bitbucket. https://bitbucket.org/centralcityconcern/kpop/src/master/

Competing interests
The authors declare that they have no competing interests.

Funding
This study was not funded.
Authors' contributions MM conceptualized the study, developed the methodology, conducted the formal analysis, and contributed to the writing of the original draft. BC conceptualized the study and contributed to the writing of the original draft. CK conceptualized the study and contributed to the writing of the original draft. MS developed the methodology and participated in the investigation. DD supported the development of the methodology and contributed to the review and editing of the manuscript. RS conceptualized the study and contributed to the review and editing of the manuscript. All authors read and approved the nal manuscript.  Classes of patients seen at Old Town Clinic identi ed using Latent Class Analysis and Inter-Professional Codesign Process Note: This gure illustrates the conceptual relationships among classes. Complexity increases along the vertical axis, and similar diagnoses are indicated by proximity and color. Differences in hospital utilization among similar classes are indicated by arrows and noted with the letters A, B, and C, with higher utilization indicated by higher letters.