Tables 1 and 2 show the results for asymptomatic and symptomatic patients, when traditional risk factors are included as variables. Other variables registered in the dataset are explored, but found of low additional value in the risk prediction model.
Table 1 The table shows the results of a cox regression analysis of the asymptomatic patients, included after an incidental diagnosis of carotid stenosis. A traditional cox regression is performed by analysing time from diagnosis to the first stroke or censoring at the end of follow up. In the extended analysis, time runs until the end of follow up for all participants, and accounts for the occurrence of more than one stroke in some patients. The time to left and right sided strokes are analysed separately
Asymptomatic at diagnosis
|
Traditional Cox, time to first event
|
Extended Cox model, right sided events
|
Extended Cox model, left sided events
|
Variable
|
HR (95% CI)
|
p
|
HR (95% CI)
|
p
|
HR ( 95% CI)
|
p
|
Age, years (at time of diagnosis)
|
1.02 (0.98, 1.06)
|
0.35
|
1.02 (0.97, 1.07)
|
0.36
|
1.01 (0.95, 1.07)
|
0.71
|
Sex (f)
|
0.82 (0.46, 0.48)
|
0.51
|
0.79 (0.35, 1.77)
|
0.56
|
2.16 (1.05, 4.48)
|
0.04
|
Systolic blood pressure(at time of diagnosis), 10mmHg
|
1.10 (0.99, 1.02)
|
0.34
|
1.00 (0.98, 1.01)
|
0.87
|
1.10 (0.99, 1.03)
|
0.32
|
Diabetes mellitus
|
0.67 (0.32, 1.39)
|
0.28
|
0.72 (0.23, 2.24)
|
0.57
|
2.80 (1.26, 6.25)
|
0.01
|
Current smoker
|
0.82 (0.46, 1.45)
|
0.49
|
1.31 (0.65, 2.65)
|
0.45
|
1.67 (0.79, 3.64)
|
0.18
|
No statin therapy after diagnosis
|
1.27 (0.64, 2.50)
|
0.50
|
0.60 (0.23, 1.51)
|
0.27
|
0.81 (0.35, 1.86)
|
0.62
|
No anticoagulation after diagnosis
|
2.26(0.85, 6.06)
|
0.10
|
0.63 (0.21, 1.87)
|
0.40
|
0.23 (0.03, 1.81)
|
0.16
|
No antihypertensive treatment
|
0.52 (0.28, 0.98)
|
0.05
|
2.10 (0.80, 5.47)
|
0.13
|
1.06 (0.44, 2.55)
|
0.89
|
Carotid stenosis, right side* 0
|
|
|
ref
|
|
ref
|
|
< 50%
|
ref
|
0.06
|
1.95(0.57, 6.75)
|
0.29
|
1.02 (0.28, 3.65)
|
0.98
|
50–69%
|
1.24 (0.60, 2.55)
|
0.57
|
1.02 (0.26, 3.97)
|
0.03
|
1.27 (0.40, 4.04)
|
0.69
|
70–99%
|
2.40 (1.18, 4.89)
|
0.16
|
4.01 (1.14, 14.07)
|
0.03
|
1.97 (0.58 ,6.67)
|
0.28
|
100%
|
2.4 (0.96, 5.88)
|
0.62
|
4.98 (1.39, 17.80)
|
0.01
|
0.35 (0.05, 2.27)
|
0.27
|
Carotid stenosis, left side
0
|
|
|
ref
|
|
ref
|
|
< 50%
|
ref
|
0.16
|
0.72 (0.22, 2.36)
|
0.59
|
3.60 (1.29, 10.08)
|
0.04
|
50–69%
|
2.00 (0.97, 4.12)
|
0.06
|
1.15 (0.41, 3.24)
|
0.79
|
4.18 (1.40, 12.48)
|
0.01
|
70–99%
|
2.08 (1.01, 4.28)
|
0.46
|
1.18 (0.38, 3.64)
|
0.77
|
5.23 (2.04, 13.38)
|
0.00
|
100%
|
1.89 (0.67, 5.34)
|
0.23
|
3.07 (0.86, 10.94)
|
0.08
|
1.96 (0.58, 6.68)
|
0.28
|
Right sided TIA
|
|
|
1.89 (0.50 ,7.19)
|
0.35
|
1.25 (0.25, 6.25)
|
0.78
|
Left sided TIA
|
|
|
0.90 (0.28, 2.93)
|
0.86
|
2.02 (1.08, 3.78)
|
0.03
|
TIA, unknown side
|
|
|
1.03 (0.24, 4.42)
|
0.96
|
Err (few events)
|
err
|
CEA asymptomatic
|
2.04 (0.75, 5.55)
|
0.16
|
|
|
|
|
*The reference category for the simple Cox regression is set to <50% (including 0) to mimic other traditional analyses. This is a very common categorization, as small plaques are considered of no hemodynamic significance. Still, a plaque of any size is considered to have the potential to progress over time, and no stenosis (0) is used as the reference category in the time dependent analysis, with plaque <50% as a separate category.
Table 2
The table shows the results of a cox regression analysis of the patient group presenting with an ischemic event at the time of inclusion. A traditional cox regression is performed by analysing time to the first ipsilateral stroke or censoring at the end of follow up. In the extended analysis, in the same way as for the asymptomatic patients, time runs until the end of follow up for all participants, and accounts for more than one stroke occurring in some patients. The time to ipsilateral and contralateral strokes is analysed separately.
Included after symptoms of a cerebral ischemic event
|
Traditional cox, time to first ipsilateral event
|
Extended cox model, time to ipsilateral events
|
Extended cox model, time to contralateral events
|
Variable
|
HR (95% CI)
|
p
|
HR (95% CI)
|
p
|
HR (95% CI)
|
p
|
Age, years (at index event)
|
1.01 (0.98, 1.04)
|
0.64
|
1.01 (0.99, 1.03)
|
0.41
|
1.01 (0.95, 1.07)
|
0.82
|
Sex (f)
|
0.96 (0.51 ,1.82)
|
0.91
|
0.83 (0.47, 1.45)
|
0.50
|
0.48 (0.16, 1.46)
|
0.20
|
Systolic blood pressure (at time of diagnosis), 10 mmHg
|
1.10 (1.00, 1.02)
|
0.15
|
1.00 (0.99, 1.01)
|
0.70
|
1.22 (1.00, 1.04)
|
0.06
|
Diabetes mellitus
|
0.86 (0.42, 1.77)
|
0.68
|
0.97 (0.51, 1.82)
|
0.92
|
0.46 (0.08, 2.60)
|
0.38
|
Current smoker
|
1.34 (0.63, 2.85)
|
0.44
|
0.85 (0.50, 1.47)
|
0.57
|
0.99 (0.31, 3.17)
|
0.98
|
No statin therapy after diagnosis
|
1.02 (0.45, 2.32)
|
0.97
|
0.80 (0.35, 1.81)
|
0.59
|
0.41 (0.04, 3.97)
|
0.44
|
No anticoagulation after diagnosis
|
4.43 (1.04, 18.54)
|
0.04
|
3.95 (1.03 ,15.19)
|
0.05
|
-
|
-
|
No antihypertensive treatment
|
0.96 (0.47, 1.97)
|
0.92
|
0.99 (0.53, 1.84)
|
0.98
|
1.03 (0.32, 3.32)
|
0.96
|
Carotid stenosis. ipsilateral to index side 0
|
|
|
ref
|
|
ref
|
|
< 50%
|
ref
|
|
1.23 (0.46, 3.31)
|
0.68
|
0.73 (0.06, 8.51)
|
0.80
|
50–69%
|
0.80 (0.31, 2.10)
|
0.65
|
1.61 (0.70, 3.71)
|
0.27
|
1.29 (0.21, 7.96)
|
0.78
|
70–99%
|
1.85 (0.78, 4.37)
|
0.16
|
2.25 (1.03, 4.94)
|
0.04
|
0.52 (0.04, 7.16)
|
0.62
|
100%
|
1.59 (0.61, 4.15)
|
0.34
|
2.93 (1.45, 3.07)
|
0.00
|
3.67 (0.90, 14.90)
|
0.07
|
Carotid stenosis. contralateral to index side 0
|
|
|
ref
|
|
ref
|
|
< 50%
|
ref
|
|
1.48 (0.81, 2.73)
|
0.20
|
6.30 (0.74, 53.78)
|
0.09
|
50–69%
|
0.55 (0.21, 1.43)
|
0.22
|
0.66 (0.30, 1.46)
|
0.31
|
15.43 (2.22, 107.23)
|
0.01
|
70–99%
|
0.58 ( 0.22, 1.51)
|
0.26
|
0.55 (0.21, 1.43)
|
0.22
|
12.08 (1.30, 112.26)
|
0.03
|
100%
|
0.41 (0.06, 3.22)
|
0.41
|
0.49 (0.09, 2.68)
|
0.49
|
-
|
-
|
New ipsilateral TIA
|
|
|
1.13 (0.41, 3.07)
|
0.82
|
0.46 (0.17, 1.24)
|
0.12
|
New contralateral TIA
|
|
|
1.75 (0.62, 4.95)
|
0.29
|
4.05 (1.18, 13.88)
|
0.03
|
New TIA. unknown side
|
|
|
1.54 (0.57, 4.13)
|
0.40
|
20.86 (3.20, 136.19)
|
0.00
|
CEA after index
|
0.32 (0.14, 0.71)
|
0.01
|
|
|
|
|
When compared to the traditional Cox model, the extended model shows a clear and consistent risk association in relation to the degree of stenosis to events on the same side. In the analysis of the asymptomatic patients, the wide confidence interval for carotid stenosis is interpreted as model instability due to insufficient power (table 1). For the patients presenting with symptoms and the endpoint “ipsilateral stroke” the model stability seems better (Table 2).
In the traditional Cox analysis CEA is included as a variable, and no clear benefit is seen in the asymptomatic patients, whereas the risk is significantly reduced for the symptomatic patients after treatment. This is consistent with results from previous studies.
Model testing
The model shows a strong association between the degree of stenosis and the risk of stroke, which is less apparent in a traditional Cox analysis of the same historical data. This suggests that the model could provide improved prediction based on the selected variables, and thereby a possible way of applying regular patient record data in risk prediction. We would like to explore this further by model testing.
To evaluate the predictive value of the extended model in comparison to a traditional Cox regression concordance analysis is applied (13). The SAS concordance statistics function currently does not perform analyses with time intervals and stratification. In order to perform the analysis, we therefore simplify the extended model. Rather than expressing observation time in intervals as start time and stop time, we calculated a single observation time as the difference between the start and stop time. This means that each time difference will be interpreted as a separate survival time with variable values as for this given time interval, with the possibility of several survival times for each patient. If no events occur and the time dependent variables remain unchanged, the patient has one survival time and a single row in the matrix setup, with censoring at the end of follow up. The time dependent variables can leave each individual at a different risk for each survival time. Patients who experience an event will be under risk again in the following time interval with or without a change in time dependent variable. Unfortunately, no stratification can be performed in SAS’ concordance analysis which means that the time intervals cannot be expressed as dependent on previous intervals within the same patient. We are not familiar with other software providing this function. This is a drawback in the model test, as identification of each time interval belonging to the same patient, and allowing these to depend on each other, is considered a key feature of the model. Still, stratification is considered very likely to improve the model fit compared to an unstratified model, which makes is likely that the performed model test provides a “worst case” concordance estimate. If this is true, the full model will represent a better fit than the one simplified in order to perform the concordance analysis.
Concordance analysis will evaluate predictive performance as a value C. If C = 0.5 the model has no informative predictive value, whereas C = 1 equals a perfect association. Uno`s C using SAS is considered to be the best method(14), as Harrell’s C (15) will provide a shortcoming when there is a large variation in time to censoring, which is the case in our data material. Concordance analysis is first applied on a traditional Cox model of the symptomatic dataset (Fig. 1). The endpoint is here defined as a stroke caused by the carotid stenosis, ipsilateral to the index event. Carotid surgery after the index event is included as an independent variable.
As seen in Fig. 2, it seems like the early predictive value is fair, but after the first 1–2 years the predictive value is around 0.65, which is low. The Area Under the Curve (AUC) is initially high, which suggests good prediction for the first month, but falls after a year and stays low for the following years. This is well illustrated by Receiver Operating Characteristic (ROC) curves after 30 days, one year, two years, and up to five years, as illustrated in Fig. 3. These curves are plots of sensitivity against one minus specificity for the given time points. The higher the curve rises above the 45-degree line, the better the model.
The same analysis is performed on the multiple event and time dependent variable set-up for the symptomatic dataset, with the same endpoint definition (Fig. 4). The AUC curve in Fig. 4 suggests a better long-term predictive value, but the confidence interval is still large, which indicates a high degree of uncertainty. This is assumed possible to improve by an increased power.
This can similarly be illustrated by the ROC curves (Fig. 5), which show an opposite trend compared with the traditional Cox model. The initial time interval of 30 days seems better predicted by a traditional Cox, as can be expected, as time dependent variable changes and multiple endpoint analysis is unlikely to affect prediction short term. The lack of stratification in the concordance analysis could also contribute to the weak early predictive value. This will impact the predictive properties of the model if the start risk in each stratum is different, which is assumed to be the case in this patient group. In the simplified expanded model, each individual with more than one event will have the same baseline risk at the start of each stratum, as the risk associated with the previous event will not be carried on to the next time interval as is expressed in the model by stratification. The area over the 45-degree line increases after the first 30-day period for the extended model, and decreases for the traditional cox.
As described, SAS does not have a function for concordance analysis for time intervals and stratification, and we needed to simplify the model for this is analysis. When we compare the HRs in the simplified analysis with the original analysis, the estimates are only slightly different. The similar results are interpreted as an indication of a reliable concordance analysis in the simplified model. Results from the simplified model compared to the original extended model for symptomatic patients are shown in Table 3.
Table 3
To prepare the results from the extended cox regression model for concordance analysis, simplification is necessary. This is performed by calculation of time differences (number of days) rather than time intervals, and by elimination of the stratification function. The table shows that the results from the simplified model are only slightly different from those of the original extended model. The simplification is therefore considered acceptable for this analysis.
|
Simplified model for concordance analysis
|
Original extended model
|
Variable
|
HR (CI)
|
p
|
HR (CI)
|
p
|
Age, years (at index event)
|
1.01 (0.98, 1.04)
|
0.54
|
1.01 (0.99, 1.03)
|
0.50
|
Sex (f)
|
0.81 (0.47, 1.39)
|
0.45
|
0.83 (0.47, 1.45)
|
0.50
|
Systolic blood pressure (at time of diagnosis), 10 mmHg
|
1.00 (0.99, 1.01)
|
0.91
|
1.00 (0.99, 1.01)
|
0.70
|
Diabetes mellitus
|
0.99 (0.54, 1.80)
|
0.96
|
0.96 (0.51, 1.82)
|
0.92
|
Current smoker
|
0.89 (0.52, 1.52)
|
0.66
|
0.85 (0.50, 1.47)
|
0.57
|
No statin therapy after diagnosis
|
0.84 (0.39, 1.79)
|
0.64
|
0.80 (0.35, 1.81)
|
0.59
|
No anticoagulation after diagnosis
|
3.87 (0.96, 15.6)
|
0.057
|
3.95 (1.03, 15.2)
|
0.046
|
No antihypertensive treatment
|
0.95 (0.51, 1.77)
|
0.88
|
0.99 (0.53, 1.85)
|
0.98
|
Carotid stenosis, ipsilateral to index side 0
|
ref
|
|
ref
|
|
< 50%
|
1.45 (0.55, 3.85)
|
0.45
|
1.23 (0.46, 3.31)
|
0.68
|
50–69%
|
1.75 (0.78, 3.94)
|
0.18
|
1.61 (0.70, 3.71)
|
0.27
|
70–99%
|
2.39 (1.11, 5.15)
|
0.03
|
2.25 (1.03, 4.95)
|
0.04
|
100%
|
3.49 (1.72, 7.08)
|
0.00
|
2.92 (1.45, 5.89)
|
0.00
|
Carotid stenosis, contralateral to index side
0
|
ref
|
|
ref
|
|
< 50%
|
1.53 (0.84, 2.77)
|
0.16
|
1.48 (0.81, 2.73)
|
0.20
|
50–69%
|
0.72 (0.32, 1.60)
|
0.42
|
0.65 (0.30, 1.46)
|
0.31
|
70–99%
|
0.56 (0.21, 1.47)
|
0.24
|
0.55 (0.21, 1.43)
|
0.22
|
100%
|
0.56 (0.07, 4.19)
|
0.57
|
0.49 (0.09, 2.68)
|
0.41
|
New ipsilateral TIA
|
0.56 (0.21, 1.44)
|
0.23
|
1.13 (0.41, 3.07)
|
0.82
|
New contralateral TIA
|
0.96 (0.26, 3.56)
|
0.96
|
1.75 (0.62, 4.95)
|
0.29
|
New TIA. unknown side
|
0.63 (0.07, 5.42)
|
0.68
|
1.54 (0.57, 4.13)
|
0.40
|
Accepting these results, we perform the concordance analysis in the same way for the asymptomatic dataset.
As in the previous concordance analysis, the endpoint “carotid-caused stroke” is chosen. In the first analysis, any side is allowed as endpoint. The AUC plots for the regular Cox model and the extended Cox model are shown in Figs. 6 and 7, respectively. The difference between the two models is small and the confidence interval is wide for the regular model.
We performed the same analysis for the extended Cox model with sides defined. Figures 8 and 9 show the plots for right and left sided strokes, respectively.
Although the AUC strongly diverges for the sides after about 3500 days, the extended model seems to give a good 10-year prediction of risk if sidedness is defined. The overall C is 0.68 for the traditional cox model, 0.81 for the extended model for right sided events and 0.78 for left sided events.
For the asymptomatic patients 183 patients had more than ten years of follow up. Of these, ten events occurred during the follow up time after ten years or more. Four right sided and six left sided. Improved power is thus obviously desirable for a reliable result and clinically applicable prediction tool.