Mammogram Uptake from Social Determinants of Health Can Be Lost in Translation to Individual Patients

Purpose The objective of this study is to describe patterns in barriers to breast cancer screening uptake with the end goal of improving screening adherence and decreasing the burden of mortality due to breast cancer. This study looks at social determinants of health and their association to screening and mortality. It also investigates the extent that models trained on county data are generalizable to individuals. Methods County level screening uptake and age adjusted mortality due to breast cancer are combined with the Centers for Disease Controls Social Vulnerability Index (SVI) to train a model predicting screening uptake rates. Patterns learned are then applied to de-identified electronic medical records from individual patients to make predictions on mammogram screening follow through. Results Accurate predictions can be made about a county’s breast cancer screening uptake with the SVI. However, the association between increased screening, and decreased age adjusted mortality, doesn’t hold in areas with a high proportion of minority residents. It is also shown that patterns learned from county SVI data have little discriminative power at the patient level. Conclusion This study demonstrates that social determinants in the SVI can explain much of the variance in county breast cancer screening rates. However, these same patterns fail to discriminate which patients will have timely follow through of a mammogram screening test. This study also concludes that the core association between increased screening and decreased age adjusted mortality does not hold in high proportion minority areas.


Introduction
Cancer is the second leading cause of death in United States.According to the CDC, female breast cancer in 2019 was the second leading cause of death at 19.4/100,000 with 129.7/100,000 new cases per year 1 .Improving breast cancer screening uptake is a key strategy to reducing mortality by enabling early detection and intervention 1 .This study uses machine learning to detect and quantify patterns in the relationships between SODH, mammogram screening and follow-up, and age adjusted breast cancer mortality rates.We test the hypothesis that models developed using county-level SODH data improve screening predictions for individual patients.It is well documented that SODH measures are valid for identifying population health issues, however, the value of these measures for individual risk prediction useful for identifying individuals in need of special outreach and resources to complete screening has not been well described.

Background
The relationship between population level SODH and age adjusted breast cancer mortality have been well documented in literature.This was summarized by a systematic review published by Gerend et al. in 2008 that identi ed poverty, social justice and social factors as contributors to screening uptake differences between African American and White/ Caucasian patients 2 .However, sources of information indicating SDOH risk can include individual billed ICD10-CM codes, self-reported data or geographically attributed population-level data.Our research study will extend ndings published in 2018 by Heller et al 3 that showed variation in the ability of County Health Rankings data to identify the percentage of female Medicare enrollees 67-69 years old per county who had at least one mammogram and speci cally lower screening uptake rates in counties associated with higher poverty rates.The 2018 study also reported that screening uptake was positively correlated to the proportion of Medicare patients in a particular county with some college education 3 .Heller et al also showed that college education was negatively correlated with the age adjusted mortality per county 3 .Our study extends this work by utilizing additional data sources for counties (the CDC CVI), applying new statistical methods to address intercorrelations between variables, and tests the hypothesis that patterns associated with county screening uptake data can be used to discriminate between patients that will and will not have timely follow through on mammogram screening results.We include the CDC SVI which incorporates 15 social factors, including unemployment, minority status, and disability that are calculated by census tract and county.These data provide robust measures representing social risk factors.Tree Augmented Naive Bayesian networks are used to reduce the SVI to essential features associated with screening uptake and the age adjusted breast cancer mortality outcome.Testing these models not only on population-level data, but for individual patient prediction, will provide insights into the generalizability of the patterns learned.

Methods
This project was reviewed and approved by MUSC IRB (Protocol number Pro00101494).
Data for this study used four publicly available county level data sets to train a Tree Augmented Naive Bayes (TAN) network, and a fth dataset of individual patient data to learn screening uptake patterns and make predictions about a county's breast cancer screening uptake, and age adjusted mortality.This study also tested the strength of these learned pattern's ability to discriminate between patients that followed through with having a breast cancer screening test.The usage of the datasets, and the transfer of the model trained on county data to individual patients is shown in Figure 1.Continuum Codes (RUCC).The datasets were joined together using Federal Information Processing Standard County Code (FIPS).This combined data was discretized in the following way.Mammogram screening rates and age adjusted mortality due to breast cancer were binned in buckets with uniform width of 5%.CDC SVI Estimated Percentile (EPL) columns were rounded to the nearest 10 to create buckets with uniform width of 10%.Counties without age adjusted mortality, or screening rates not measured between 0 and 100% were masked from the data.
Bayesian networks are graph models that encode probabilistic uncertainty between nodes and have been used to represent relationships between variables and outcomes 7 .They consist of a directed acyclic graph (DAG) and a table of conditional probabilities between the nodes.
Bayesian networks require the assumption that all features are independent.given the class of the observation, which in this case would be the county's mammogram screening uptake rate.It is doubtful that features in the CDC SVI such as poverty, income, education, and vehicle ownership percentiles per county would be conditionally independent with respect to cancer screening rates.To address this, the TAN model is used to force all features to be dependent on the screening uptake, and only one other feature.TAN is a network structure learning algorithm that relaxes the independence requirement and imposes a tree structure where all nodes initially share an edge with the class node and contains the variables interaction with other variables, limiting them to two parents 8 .This method greatly reduces the computation complexity required to learn the network.Both Bayesian networks and TANs require underlying data to be discrete in order to learn underlying network structure.A subset set of a graph known as the Markov blankets or Markov boundary around a particular node is de ned as the subset of parents, children and parents of children of a particular node 9 .The Markov blanket is thought of as the minimal set of information about a node, however, it is not unique 10 .
The Bayesian network structure was learned using the BN Learn package with the TAN method, trained on the county level discretized data 11 .The network structure was then used to learn conditional probabilities between nodes and used independence testing to prune features with p-values greater than 0.2.The nal network represented county associations between age adjusted mortality due to breast cancer, mammogram screening and the CDC SVI.The Markov blanket of the screening variable was used to subset the network.The results were analyzed with the linear weighted kappa score to quantify the extent the network learning produced predictions that agreed with the discretized screening at mortality variables.The individual level validation cohort was derived from electronic medical record data of Medical University of South Carolina (MUSC), and included females aged 50-74 at the time of at least one billed visit during the 2016-2019 time period with at least one breast cancer screening test ordered.The dataset had a target task of predicting which patients would follow through and complete the breast cancer screening test within 180 days.Patients completing the mammogram on the initial date that it was order were masked.The dataset consisted of 1880 female patients that had at least one mammogram screening test ordered with features describing comorbidity, demographic, self-reported personal and family cancer history as well as geographically linked (at the census track level) social variability derived from the 2018 CDC/ATSDR Social Vulnerability Index 4 .
The patient level data from MUSC was discretized using the same method as the data for the county level model.SVI tract level features were used to predict the screening uptake rate, and age adjusted mortality due to breast cancer.The predictions were compared to PCT or ICD10CM codes that indicated the patient had completed a mammogram screening during the study time frame.Performance metrics were calculated on the overall cohort of patients, and area under the curve of the receiver operating characteristic (AUC) was used as a primary metric to describe the model's ability to discriminate between patients that completed screening vs. not after being ordered by a provider at different thresholds.An investigation was also conducted into how percentile percentage of minorities of a particular county an effect on the relationship between age adjusted mortality and screening.Mixed effects linear models were used with age adjusted mortality as a dependent variable, percent unscreened as an independent variable, and a ag indicating the county was at or above the 90 th percentile of the percentage minority having a random slope and intercept.

Results
The county level data (joining the SVI, screening uptake data and breast cancer mortality data) resulted in a dataset containing 2,270 counties in the United States with 13 features and two outcomes.Summary statistics of the data are shown in Table 1.The result of joining the individual MUSC patient data to tract level SVI area shown in with summary statistics in Table 2.The network structure trained on the county data resulted in a host of associations between SVI features and screening shown in Fig. 2.This shows the associations learned between the percent of patients unscreened, age adjusted mortality and the CDC SVI.Each edge in the graph contains conventional probabilities between edges.The EPL_POV node, representing percentiles of persons in poverty is shown to have associations with lower vehicle ownership (EPL_NOVEH), lower income (EPL_PCI) higher unemployment (EPL_UNEMP) and lower mammogram screening uptake (Pct Un Screened).Also, shown in Fig. 2 is the strongest association to age adjusted mortality are mammogram screening uptake and percentile minority (EPL_MNTRY).The network learning algorithm found associations related to age adjusted mortality and screening uptake was confounded by the estimated percentile of minorities in a county.The network also revealed that estimated percentile of age over 65 was a confounding factor in the association between rural-urban continuum code and the proportion of female Medicare patients aged 67-69 without a mammogram screening in the prior two years.The positive correlation between the rural-urban continuum code had a Pearson coe cient 0.21 with p-value < 0.000, however, counties in the 90th percentile of age over 65% consistently had a lower percentage of unscreened individual as shown in Fig. 3, at almost all values of Rural-Urban continuum levels shown tabulated in Table 4.  3. The task of predicting the proportion of patients that went unscreened resulted in a weighted kappa of 0.82 and accuracy 0.79 predicting the proportion persons that went unscreened.This demonstrated a high level of agreement between the model's predictions and the actual proportion of unscreened patients.The task of predicting age adjusted mortality due to breast cancer from the same network resulted in weighted kappa of 0.14 and accuracy of 0.57.This demonstrated relatively poor agreement between model predictions and age adjusted mortality, however since it was not the primary class node, the TAN network architecture limited the number of parent variables to two; percentile of minorities in a county (EPL_MINRTY) and proportion of unscreened patients.EPL_MINRTY was shown to be a confounding factor between screening uptake and age adjusted mortality due to breast cancer, additional regressions were conducted to quantify the associations.This experiment used mixed effects models with the county mammogram screening uptake rate as an independent variable, and the age adjusted mortality as the dependent, where random slope and intercepts were t for counties agged in of the 90th percentile of proportion minority.
For counties not in the 90th percentile of proportion minority, the resulting regression shows every 10% increase in a county's screening rate, a 1.3 to 1.7 person per 100,000 decrease in age adjusted mortality would be expected with a p-value less than 0.001, and r-squared of 0.082.This shows a clear effect of decreasing age adjusted mortality when due to breast cancer, when screening is increased, for counties not in the highest percentiles of minorities.
For counties in the 90th percentile of proportion of minorities, a 10% increase in a county's screening rate would be associated with a -0.9 to 1.5 per person 100,000 change to the age adjusted mortality rate with p-value 0.58 and r-squared 0.001.This shows effect of increasing screening on age adjusted mortality is uncertain on the 209 counties, agged as being the highest percentiles of minorities.Side by side regression results demonstrate the discrepancy in the screening and age adjusted mortality relationship in high proportion minority areas verses other areas are shown in Fig. 4.This breakdown of the relationship between screening and age adjusted mortality in high proportion minority areas suggests other unaccounted factors are in uencing age adjusted mortality.
Results of translating the network model to patient with SVI data collected at the track level and using the predictions to rank the likelihood of patient completing a screening test after ordered within 180 days resulted in AUC score of 0.532 (0.524-0.54) and ROC shown in Fig. 5.This suggests that the network trained on county level data had little discriminative ability in predicting which patients would complete the screening test.The model preformed even worse speci cally for the age 67-69 cohort (matching the age in county level CMS screening uptake metric) with an AUC of 0.42.

Discussion
There were two key ndings from this study.Firstly, county level social factors in the CDC SVI can predict patterns about county mammogram screening rates, however, they fail to make meaningful predictions about individuals.The county model's fail to translate to individuals.Thus, models trained on county-level SDOH measures should not be assumed to provide useful insights about individual patient behavior.This nding is an example of the well-known concept of an ecological fallacy, where there is a mistaken assumption that statistical patterns derived from groups represent the individuals comprising those groups 12 .The associations were learned by the TAN network by county and applied to induvial patient screening follow-through with SVI attribution through census tract.This demonstrates this fallacy can arise during machine learning model development and application.
The second signi cant nding was that the association between screening uptake and age adjusted mortality measured in most counties, fails to hold in high minority density areas.The prediction breaks down in the cancer screening and age adjusted mortality relationship in high minority areas is profound.This means that it should not be assumed that increasing screening rates for high minority areas will have the same positive impact as that observed for areas with few minority residents.The implication of this is that one of the primary population health improvement tools used to alleviate the burden of breast cancer mortality does not appear to be adequate for high minority density areas.
Limitations of this study must be considered.Chief among them is the difference between county and individual screening measure de nitions.The county level screening metric was measured for females aged 67-69 with at least one mammogram screening in the prior two years, whereas the individual patient measure assessed whether a patient completed a mammogram screening with 180 days of having one ordered by a provider.This choice was made to understand whether the model would be useful in clinical practice, where uptake in the short term would be more relevant.
The primary limitation to this work is that the predictions made about MUSC patients had CDC SVI attributes that were attributed to patients at the census tract level.This associates aggregated information from an entire census tract to an individual.This study provides some evidence in the mammogram screening rate, and individual predictions should not be made based on this level of information.
This study sourced patient screening follow-through from a single health system (MUSC) and it is unclear whether results would be applicable at other health systems.
The learning algorithm itself presented limitations.The CDC SVI is a robust measurement representing geography based social determinants of health, however the TAN structure learning algorithm limits the number of parents that a child of the class node can have to two, and only the strongest associations are returned.This leaves open the possibility that other associations between the SVI and age adjusted mortality are present but being pruned considering the strength between the association of percentile rank of proportion of minorities, screening uptake and age adjusted mortality.
Future studies are needed to investigate additional factors such as stage at diagnoses, aggressiveness of care, social stigma, and access to treatment that may be differentiating the screening and mortality relationship in high minority areas, vs other areas.

Conclusion
This study shows the ability to use the CDC SVI to understand a signi cant portion of the variance in county level mammogram screening uptake.However, models trained on these data were shown to be ineffective at discriminating between which patients would complete a mammogram screening within six months after having one ordered from a health care provider.This suggests the need to use multiple data sources when developing breast cancer screening initiatives, as county level factors and individual level factors may supplement each other.This study also demonstrated that the core association between increased screening and decreased age adjusted mortality does not hold in high proportion minority areas.This suggests additional barriers not being captured by CDC SVI are contributing to the age adjusted mortality rate.In those areas, screening increases alone may be insu cient to decrease the burden of mortality due to breast cancer.
This shows the difference in mammogram screening uptake between counties with high proportion of people over age 65 vs not, for each RUC code with the 25th, 50th, and 75 th percentiles marked as the box, and the whisker bars as 1.5x the interquartile range .This demonstrates consistently higher screening rates in counties with older populations at each Rural-Urban Continuum level.
This shows female age adjusted mortality due to breast cancer plotted with the percent of persons that went unscreened.
When the proportion of females that are un screened goes up, the age adjusted mortality also increases for areas not in the highest proportion on minority shown in the left panel.This association is no true for high proportion minority areas where the association screening and age adjusted mortality is uncertain.
The rst four data sets were used for initial model training include(1) the County level 2018 CDC/ATSDR Social Vulnerability Index (SVI) 4 , (2) the County Health Rankings data to identify the percentage of Medicare enrollees 67-69 years old per county who had at least one mammogram in 2015 sourced from the Dartmouth Atlas of health care 5 , (3) CDC WONDER Female Breast Cancer Mortality Rate, averaged 2010-2020 by county 6 and (4) the United States Department of Agriculture Rural

Figure 1 Study
Figure 1

Figure 2 Network
Figure 2

Table 1
Summary statistics for county level mammogram screening uptake.We evaluated the individual-level variables for association with mammogram completion within 180 days.Counts, proportions of and shers exact test results to show difference in proportion for each feature being true for case and controls are summarized in Table3.Depression, anxiety and menopause/premenopausal billed ICD10-CM diagnoses codes all had strong associations with increased odds of failing to follow through on screening.Medicaid insurance also had an association with lower screening completion rates.

Table 3
Differences in demographic and clinical characteristics of MUSC patients with a mammogram screening ordered.
2Cases are patients, who failed to complete the screening in 180 days, and controls completed the screening within 180 days (excluding same day completion).2Signicance values below 0.05 are shown in bold.