The following section uses a combination of simulated and real-world data to illustrate issues and considerations for the residual approach. In order to provide a logical progression through these concepts, we intermix descriptions of methods and results organized around each point.
Non-independence of residuals and cognitive performance
We can examine how the residual is typically calculated using the standard regression formula. Let yi denote individual i’s score on a cognitive test and xi an adverse factor such as level of amyloid or brain atrophy:
where β is the regression coefficient reflecting the strength of association between the adverse factor and cognitive score and δ1 denotes the residual, or error term, for the unexplained variability in yi by xi. We then solve for δ1 to obtain:
This residual is then used as an index of resilience in subsequent analyses. For example, it may be used as a predictor of progression to dementia, or it may be used as an outcome when the goal is to identify what factors contribute to or are associated with resilience.
Although δ1 is by definition uncorrelated with xi, our adverse factor, it will almost certainly have some correlation with y, our measure of cognition. The magnitude of this correlation is dependent on the strength of β, the association between cognition and the adverse factor. When the association between these two variables is small (and thus β approaches 0), the residual measure δ1 will be more highly correlated with our cognitive measure y. In the extreme case where y and x are completely unrelated, β will be 0 and the resulting residual score will simply be y, our cognitive score (or a mean-centered version of y, assuming intercept is included in the regression). At the other extreme, where x perfectly predicts y, the variance of our residual will shrink to 0, at which point it no longer provides any information whatsoever. However, error in our x measure will shrink β towards 0, known as regression dilution, or attenuation (9, 10). Thus, we are more likely to encounter the former scenario than the latter.
These concepts are illustrated in Figure 1 using simulated data. We generated 1,000 pairs of yi and xi from a bivariate normal distribution, with low, medium or strong correlations between the variables. We will consider these to represent a cognitive measure (y) and a brain measure (x) corresponding to some adverse factor such as hippocampal atrophy. The individual data points are colored red or blue for positive or negative residuals, with the intensity corresponding to how large the residual value is (i.e., deviation from the expected value along the regression line). In Figure 1A, the cognitive and brain measures are correlated at r=0.9. The resulting scenario is what we intuitively expect in that individuals with higher or lower than average cognitive performance (above or below the dashed line) show a mix of positive and negative residuals of varying magnitude. Figure 1B shows what would be considered a relatively strong association between cognitive and brain variables in real-world scenarios (r=.5). Here, there is still some mixture, but individuals with higher cognition tend to also have higher residuals. Figure 1C shows a scenario in which the cognitive and brain measures are completely uncorrelated (r=0). In this case, individuals with higher-than-average cognition all have positive residuals. It becomes clear that the magnitude of the residual is simply the deviation of an individual’s score from the mean cognitive score.
We can examine how the correlation between the residual (δ1) and the original cognitive score (y) varies as the correlation between cognitive score (x) and brain measure (y) varies. As before, we simulated sets of 1,000 paired scores from a bivariate normal distribution with the correlation varying from 0 to 1. For each specified correlation, we create a residual score as described above, and then calculate the correlation between this residual (δ1) and the original cognitive score (y). Figure 2 shows how Corr(δ1, y) varies as a function of Corr(x, y). We can see that when the correlation between cognition and brain is ~0.7, the correlation of cognition and the residual is also ~0.7. In other words, when the correlation between cognition and brain is less than r=0.7, as is almost always the case, cognition will explain greater than 50% of the variance in the residual (i.e., squared correlation between the two variables).
The above point is further demonstrated using real data from the Alzheimer’s disease neuroimaging initiative (ADNI; http://adni.loni.usc.edu/). We selected 839 individuals with a baseline diagnosis of cognitively normal (n=175), mild cognitive impairment (n=437) or AD dementia (n=227) with evidence of amyloid-β (Aβ) neuropathology (determined based on 11C–Pittsburgh compound B or 18F-florbetapir PET if available, or CSF Aβ42 otherwise), structural MRI and baseline neuropsychological assessment within 6 months from the MRI scan (total sample average age 73.9±7.2 years, 46% females, median education level 16 [range 14-18] years). We selected the baseline ADNI-MEM composite memory score (11) and hippocampal volume as our cognitive and brain variables of interest, the latter measured as SPM12 segmented gray matter volume in a bilateral hippocampal mask based on the AAL atlas and adjusted for total intracranial volume. All procedures were approved by the Institutional Review Board of participating institutions and informed consent was obtained from all participants. Baseline ADNI-MEM (cognitive score; y) was regressed on hippocampal volume (adverse factor; x), and no other (e.g., demographic) covariates were included in the calculation of the residuals (δ1). Despite a relatively strong association between ADNI-MEM and hippocampal volume (r=0.56; Figure 3A), the model residual (our “resilience measure”) retained a very strong correlation with ADNI-MEM (r=0.83; Figure 3B), falling precisely along the simulated curve from Figure 2 (Figure 3C). Therefore, we cannot determine if better memory than expected given one’s hippocampal volume predicts much of anything that would not already be predicted by simply looking at memory alone.
Unfortunately, our measures of cognition and adverse factor rarely approach correlations of r=0.7. This means the majority (often most) of the variance in residual-based resilience measures is usually shared with cognition. While this is not strictly problematic, it may not be appropriate in many scenarios, for example when a secondary variable of interest is correlated with or dependent on cognition. In such a circumstance, we cannot differentiate whether the association with the external variable is driven by a pre-existing association with cognition or a unique resilience factor without further modeling these terms. Longitudinal cognitive decline or clinical diagnosis (when diagnosis is partially or entirely based on cognitive scores) are two frequent examples of this type of external variable. In such a scenario, a model may reveal a strong relationship between “resilience” and cognitive decline that may actually be driven by the correlation between baseline cognition and cognitive decline.
In another example demonstrating this point in ADNI data, we compare results from three models, each using the following predictors: baseline memory only (Fig 4A); the residual score only (hippocampal volume regressed out of memory performance prior to entry into the model; Fig 4B); baseline memory and hippocampal volume both used as predictors in the same model (Fig 4C). We show that baseline cognitive performance (i.e., memory in this example) shows a high correlation with longitudinal memory change (Fig 4A), the latter estimated using linear mixed effect models with random intercepts and slopes per participant. As an illustration of the above point, we find a strong relationship between memory change and the pre-regressed memory residual (Fig 4B). One can appreciate the difficulty in determining the degree to which this strong relationship is driven by a pre-existing strong relationship between memory decline and the original memory measure from which the residual was derived (Fig 4A), because these two predictors are highly collinear.
When modeling cognition (memory score; previosly denoted y) and brain (hippocampal volume; previously denoted x) measures together in a multivariable regression of memory decline (Fig 4C), the resulting term for the memory score in this situation is statistically equivalent to using the pre-regressed residual (δ1). Accordingly, the regression coefficients (β) for the two modelling approaches illustrated in Figures 4B and 4C are identical. Importantly, this highlights the fact that one is essentially regressing longitudinal cognitive decline onto baseline cognition (controlling for hippocampal volume) rather than some unique entity. Extending the residual-based resilience measure to external variables with unknown relationship to y can also lead to ambiguous interpretations. Given the high collinearity with cognition, one would not be able to easily disambiguate whether resulting associations were specific to resilience or simply cognition-related. Unfortunately, as will be described below, approaches to correct for this, such as adding y as a covariate into the model, may not surmount the issues described here.
Similarities with the brain age literature
The issue described above has been discussed extensively in the brain age literature. These studies attempt to predict chronological age using a combination of brain features measured with MRI. The trained model is then used to predict age in a new sample with the same brain features. The deviation from predicted age is known as the brain-age gap or predicted brain age difference and has been proposed to reflect accelerated or decelerated brain aging. However, it is well-known that this brain age gap is correlated with chronological age (12-16). For example, younger individuals will tend to have a predicted brain age older than their chronological age and therefore an advanced brain age gap. The opposite is true for older adults. This occurs for statistically identical reasons as seen in the residual approach to cognitive resilience. If we substitute chronological age (y), brain features (x1, x2…xn), and brain age gap (δ1) into our previous formulas, we can see that this correlation becomes stronger when brain features do not strongly predict chronological age.
Calculating an independent residual
Solutions to this problem are primarily discussed in the context of brain age, where it has been considered in detail (12-14, 16-19). We refer readers to these papers for more details, but there are two primary approaches proposed to correct or adjust for correlation with our y variables. The first is that we can simply include y as a covariate when δ1 is used as a predictor. We can then consider the effect of our residual to be independent of our original measure (e.g., resilience independent of cognitive performance level). However, these variables are likely to be collinear; in the extreme case where y and x are unrelated, the predictors will be perfectly collinear. Collinearity between predictors can be problematic, resulting in unstable or imprecise estimates. It can also result in sign flipping (20). We can understand this sign flipping from a conceptual point of view by considering what happens when we include our original cognitive measure with the resilience measure in the model. The coefficient of the resilience measure would be interpreted as the effect when individuals are equated on cognitive performance. If individuals are equated on cognitive performance, then any variance in their resilience measure must be driven by variance on the adverse brain factor. Therefore, a higher resilience score in this model would simply reflect higher levels of the adverse factor (e.g., atrophy), and the effect of such a score interpreted in this way may be expected to have the opposite effect of our resilience score unadjusted for cognitive performance.
We once again demonstrate this phenomenon with the ADNI data described above (Table 1). Several models were run to predict decline on the ADNI-MEM score. When the residuals score is the only predictor, there is a positive association such that higher residuals (i.e., greater resilience) predicts less decline. However, when the original cognitive measure is included in the measure, the sign of the coefficient for the residuals becomes negative because higher values now reflect more hippocampal atrophy. Additionally, this approach does not produce a single residual measure, which many studies seek to use in subsequent analyses as an outcome variable or entry as one feature in a multivariate model.
An alternative approach that does produce a single adjusted score entails regressing the y variable out of the residual:
Solving for δ2 yields our adjusted residual score:
This new adjusted residual may be considered an index of resilience that is uncorrelated with cognitive performance. Although this appears to match our conceptual definition of resilience, we can explore further by replacing yi with yi = xiβ + δ1i:
Thus, our adjusted residual δ2i contains our brain measure xi, resulting in a negative correlation between the two. The magnitude of this correlation will be proportional to Corr(δ1, y), albeit with the reverse sign. This flip in sign occurs for the same reasons as described above. In other words, we shift from a measure of resilience that is correlated with our cognitive score to one that is correlated with our adverse factor. This is likely not the desired measure from a conceptual standpoint.