A very large body of research documents relationships between Adverse Childhood Experiences (ACEs) and adult health outcomes. Much of the data in which these inferences are based comes from cross-sectional surveys containing adults’ retrospective self-reports of their ACEs and concurrent reports of their health status. We refer to this type of design and data structure as the ‘ACEs Framework’ (1) and to questionnaire responses over a specific set of adversity events contained in these datasets as srACEs (self-reported ACEs). While this tradition arguably began with the landmark 1998 Felitti et al Kaiser ACE’s Study (2), versions of the Kaiser group srACE questions are now used in several other large-scale health surveys including the CDC’s Behavioral Risk Factor Surveillance System (BRFSS) survey (3).
Remarkably, these studies show a substantial degree of inconsistency in the operationalization of the srACEs as a predictor variable. While a ‘cumulative risk’ conceptual model guides most research, resulting in the operationalization of childhood adversity using the cumulative numeric ‘ACE Score’, there are differences in the way this cumulative score is calculated and used in statistical models. Additionally, alternative conceptual models to the cumulative risk model, which can yield different conclusions about the effects of adversity on outcomes, are rarely used and even more rarely directly contrasted to each other. Differences in operationalization may impede efforts to synthesize the literature and differences in conceptual models of how adversity impacts outcomes have high stakes as policy and intervention programming depend on this body of literature. There has been some recent criticism of the use of the ‘ACE Score’ (4–6), some of it from within the original Kaiser ACEs Study team (7). Such criticism tends to focus on using a ‘crude’ or oversimplified measure in policy-making. However the recent critical publications are conceptual reviews, not empirical reports. Large data sources such as the BRFSS survey represent a significant investment of research resources; the BRFSS effort specifically surveys over 450,000 individuals each year, with a yearly budget over $18 million (3). These observations were the rationale for the present study.
In this paper we analyze two conceptual models of the effects of ACEs on adult health, contrasting the cumulative risk model (using a cumulative ACE Score) with a ‘multiple individual risk’ model that allows for each ACE event to have its own relationship with the outcome in a multivariable model that includes all the ACE event predictors. In order to undertake this comparison, we also analyze differences arising from the practical decisions that have to be made about which specific ACE questions to include from among those available and how to code individuals as ‘exposed’ when response options include information about the frequency of events.
Models of the Effects of Adversity - Cumulative & Multiple Risks
Cumulative Risk - CR
In the ACEs framework literature, the dominant model of the effects of developmental adversity on later health is the cumulative risk model. This model holds that it is not so much specific events which are detrimental to health, but rather that it is an accumulation of events (regardless of which specific events they are) that confers risk for negative health effects (8). The cumulative risk model is a specific type ‘multiple risk’ model (8) where exposure to multiple risks are included in the same statistical model. In the ACEs literature this is widely implemented through the use of a cumulative numeric score (the ‘ACE Score’) that indicates the total number of exposures. This model answers the question ‘what is the impact of increasing numbers of events? (regardless of which events they were?’). There are multiple ways to implement this model, however. Here we contrast a continuous with a categorical variable approach.
Continuous Cumulative Risk
A cumulative ACE Score can be treated as an integer count variable (i.e. a continuous variable) in statistical models. This model answers the question ‘what is the impact of increasing numbers of events?’ (regardless of which events they were?) but with a restrictive assumption about linearity of the effect (that each additional ACE has an equal impact). An example of the logistic regression model is represented in equation (1) for the ith participant, assuming a binary outcome and continuous ACE Score (1-11), and with the same set of covariates (not represented in the model equation).
As an example in the BRFSS literature, Nurius, Logan-Green and Green used a total ACE Score (0-8) and reported significant coefficients of -.19 (healthy days) and .23 (mental health symptoms) (9), implying a constant decrease of .19 healthy days and constant increase of .23 mental health symptoms for each additional ACE reported. An ACE Score characterized in this way serves as the primary illustration of a dose-effect relationship: one in which the dose-effect relationship is constant across levels of the ACE score.
Categorical Cumulative Risk
The most common alternative characterization for the ACE Score is to use it as a categorical variable in prediction models. While the model answers the same question about ‘the effects associated with increasing numbers of events’, it does not assume linearity of the relationship and allows each specific ACE count to have its own relationship with the outcome. The counts are entered into prediction equations as categories, producing a separate coefficient for each, compared to a reference category (usually zero). An example of the logistic regression model is represented in equation (2) for the ith participant, assuming a binary outcome and categorized and reference cell coded ACE Score (1-11).
The most common approach is categories of 0, 1, 2, 3, and ≥ 4, (10,11) although a top category of ≥ 5 is also common (12–13). Other researchers have combined counts into other categories such as 0, 1-3, 4-6, 7-9 (14), or used a wider range of the variable (1-8 compared to zero) as a categorical predictor (15,16). Compared to a continuous ACE Score approach, this model is more flexible and yields a different inference about the dose-response relationship: that risk for the outcome increases monotonically, but not at the same rate for every additional ACE. This was the approach used in the first ACEs study publication (2), where a total of 17 individual questions were aggregated into 7 categories of events and a truncated categorical variable (0, 1, 2, 3, ≥ 4) was used in statistical modeling. Those results showed increasing odds across the levels of the categorical variable (compared to those in a zero ACEs reference category) of 1.1 - 2.2 for current smoking; 1.5 - 4.6 for two or more weeks of depressed mood in the previous year; and 1.1 - 1.6 for ‘severe obesity’ (2). Results arrived at using this treatment of the ACE predictor are not only exceedingly common in the literature, but have been translated into public policy (17) as well as public-facing internet sources that refer to the risks associated with ‘4 or more ACEs’ (18,19).
Multiple Individual Risks - MIR
In contrast to the cumulative risk model, the presence or absence of multiple separate ACE events can be included together as separate predictors in a single regression; we refer to this as a ‘multiple individual risk’ model. This model is not the same as analyzing univariate (unadjusted) associations between single ACEs and outcomes, which is a common feature in studies that use a cumulative risk model (ACE score) in their primary analysis. Instead, this model answers the question ‘what is the impact of the occurrence of each specific event (given the presence/absence of the other events)’?. The model yields coefficients for each event separately, controlling for the other events in the model. It is therefore expected to function well for events which are highly correlated, as has been extensively supported for ACEs (20). An example of the logistic regression model is represented in equation (3), assuming a binary outcome and entry of all 11 BRFSS ACE events in the model.
Despite the high degree of information contained in this type of model, it appears only rarely in the ACEs framework literature. Our review found only one instance in the BRFSS data, in a study examining individual and cumulative effects of ACEs on adult mental health. In that study, only the specific ACEs that had a significant univariate relationship with the outcome were included in the ‘multivariate’ models, and they found that different sets of ACE events had significant associations with the mental health outcomes under study (9).
We presume that the absence of this type of model in the literature is due to the fact that, while the model itself is additive with respect to the joint effects of the events on the outcome, this model does not contain general summary information about cumulative effects. That is, when effect estimates for specific individual ACEs are estimated in the same model, effect estimates for a specific number of events are not estimated and therefore the model does not produce specific information about a dose-response relationship. However, there are instances that it makes theoretical sense (either for certain types of adversity or for certain outcomes) to consider that an accumulation of adversity might not be the only model to consider. The review of Lacey and Minnnis provides an overview (6).
Choosing a predictor characterization directly impacts interpretations about the effects of adversity on outcomes, and the cumulative risk model and the multiple individual risk model yield different inferences. Only in the first case would we be able to infer that an accumulation of adverse events (regardless of which specific events) is related to outcomes in a dose-response manner. However, only in the second case are we able to infer that one or some specific ACE events are a strong predictor, compared to other ACE event types.
Similarly, treating the ACE score as continuous vs categorical in the CR framework also has implications for interpretation. Finding that each additional ACE event contributes in a constant linear way to risk for negative outcomes (e.g. Dube et al. 2001; Nurius 2012) is a very different conclusion than finding that change in risk for an outcome is smallest across intervals at the low end of ACE Scores, but that increases in risk for the outcome are accelerated across intervals at the higher end of the score range (e.g. Anda et al. 1999).
However, even though these modeling choices can result in different conclusions, there is only limited evidence that directly contrasts them. Some exceptions include comparison of a latent class predictor characterization (LCA) to the cumulative risk ACE Score in predicting outcomes in college students (23) which found that LCA performed similarly to the cumulative ACE Score. In contrast, Schilling et al found that a cumulative risk approach produced different predictions than treating the same data with a cluster analysis approach (24).
The cumulative risk model is a straightforward and easy to understand explanatory model that has helped to publicize the negative health effects of adversity, but it has both statistical and theoretical shortcomings (5, 26). In this study, we investigated differences in model fit and performance based on operationalization of an ACE predictor variable in a cumulative risk model (with ACE score as continuous or categorical) vs a multiple individual risk model when applied to three commonly studied health outcomes.