Individualized MRI-based stroke PRediction scOre using plaque Vulnerability for symptomatic carotid artEry disease patients (IMPROVE)

Abstract Objective: In TIA and stroke patients with carotid stenosis, estimations of future ipsilateral ischemic stroke risk and treatment decisions are currently primarily based on the degree of stenosis. Intraplaque hemorrhage (IPH), which can be readily visualized on carotid MRI, is increasingly established as an easy to assess and a very strong and independent predictor for ipsilateral stroke risk, stronger than any clinical risk factor. We developed a clinical prediction model (IMPROVE) incorporating IPH, degree of stenosis, and clinical risk factors to select patients with symptomatic carotid stenosis at high risk for stroke. Methods: IMPROVE was developed on pooled clinical and MRI data from five cohort studies of 760 recent TIA or minor stroke patients with carotid plaque who received optimal medical treatment. We used Cox proportional hazards models to determine the coefficients of IMPROVE. IMPROVE was internally validated using bootstrapping and converted to one- and three-year ipsilateral ischemic stroke risk. Results: The development dataset contained 65 ipsilateral incident ischemic strokes that occurred during a median follow-up of 1.2 years (IQR: 0.5-4.1). The IMPROVE model includes five predictors, which are in order of importance: degree of stenosis, presence of IPH on MRI, classification of last event (cerebral vs ocular), sex, and age. Internal validation revealed a good accuracy (C-statistic: 0.82; 95% CI: 0.77–0.87) and no evidence for miscalibration (calibration slope: 0.93). Interpretation: Using presence of IPH on MRI and only four conventional parameters, the IMPROVE model provides accurate individual stroke risk estimates, which may facilitate stratification for revascularization.


Introduction
Selection of symptomatic patients for carotid revascularization relies primarily on the degree of stenosis since patients with severe (70-99%) stenosis were identi ed to bene t most from this procedure in large clinical trials performed in the 1980s-1990s. 1Bene t of carotid endartectomy (CEA) was considered moderate for patients with 50-69% carotid stenosis and non-existent for mild (< 50%) stenosis. 1,2owever, the 3-year rate of ipsilateral stroke on modern medical management is still 7.4% for mild stenosis. 3With up to 17 CEAs needed to prevent one future ipsilateral stroke and a ~ 2.7% procedural stroke and death risk, the selection of patients to undergo CEA should be optimized. 1,4inical prediction models There are currently three prediction models for the long-term (> 90 days) prediction of ischemic stroke risk in patients with symptomatic carotid stenosis: the European Carotid Surgery Trial (ECST) medical model and its derivative the Carotid Artery Risk (CAR)-score, the Symptomatic Carotid Atheroma In ammation Lumen stenosis (SCAIL) score, and the CaroTID-VasC score 2,5,6 Critical appraisal showed that the predictive performance of the ECST model and SCAIL score were at high risk of bias. 5The models were developed based on either a small events per variable (EPV) far below the generally advised 10 outcomes per candidate predictor (SCAIL and CaroTID-VasC) 7 , or using data of large, though relatively, old trials from the1980s-1990s (ECST model).Since then signi cant improvements in medical management (e.g.statin and anti-platelet therapy) for stroke prevention were introduced, for which the ECST model was recalibrated resulting in the derivative CAR-score. 2,80][11] The CaroTID-VasC score was developed based on only 20 recurrent events (12 ipsilateral TIAs, 5 ipsilateral strokes, 3 cardiovascular deaths) in 99 symptomatic patients.While no external validation was performed, the small EPV and the majority of outcomes being TIAs and cardiovascular deaths, will make the CaroTID-VasC score less likely to predict ipsilateral ischemic strokes.Therefore, there is a clear need for a more valid tool for risk strati cation of patients with symptomatic carotid stenosis.

Rationale for developing the IMPROVE prediction model
Intraplaque hemorrhage (IPH), a key contributor to plaque vulnerability, is present in ~ 50% of carotid plaques in patients with symptomatic carotid stenosis. 12Carotid plaque MRI is a novel diagnostic tool that enables the identi cation of IPH 13 , which can be recognized as a hyper-intense signal in the bulk of the plaque on a dedicated hyper T1-weighted MR images. 14IPH is a strong independent predictor of ischemic stroke with hazard ratios (HR) between 8.1 (95% CI: 3.7-17.9)and 10.2 (95% CI: 4.6-22.4),stronger than all conventional clinical risk factors including degree of stenosis. 12,15 developed the 'Individualized MRI-based stroke PRediction scOre using plaque Vulnerability for patients with symptomatic carotid artEry disease (IMPROVE)'.This risk score includes IPH presence on carotid plaque MRI and traditional predictors, to enable improved risk strati cation for personalized clinical decision-making in patients with symptomatic carotid stenosis.

Methods
We followed the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines (Supplemental Table ). 16

Data and participants
Studies eligible to contribute to the development dataset were identi ed based on previously reported search criteria 12 , updated to include publications until January 2022.All identi ed studies received an invitation to participate.Individual patient data from ve cohorts, which performed carotid plaque MRI in symptomatic patients with carotid stenosis on optimal medical treatment (OMT) regimen, were pooled.Details of the included studies are presented in the Supplemental Methods.OMT entailed lifestyle advice and medication including statin, antiplatelet, and blood pressure control.The inclusion period ranged from 2002-2015 with follow-up until 2020.Index events included amaurosis fugax, TIA, or minor stroke < 6 months prior to inclusion.Patients had a carotid plaque of ≥ 2mm and up to 99% stenosis (North American Symptomatic Carotid Endarterectomy (NASCET) criteria).
This study utilized data from four previously published cohort studies included in the meta-analysis by Schindler et al., as well as data from the PARISK cohort. 9,12Ethics Approval Statements: PARISK: Institutional review board approval was obtained from the Medisch Ethische Commissie azM/UM (approval number NL29116.068.09/MEC09-2-082), and all patients provided written informed consent. 9hindler et al.: Corresponding, rst, and last authors of publications from eligible cohorts were invited to share anonymized individual patient-level data.No imaging raw data were collected.The local ethics committee waived the requirement for individual consent or speci c approval for this analysis. 12

Outcome
The IMPROVE-predicted outcome is the one-and three-year risk of ipsilateral ischemic stroke, de ned as a rapidly developing syndrome of focal cerebral dysfunction lasting > 24h or leading to earlier death, with no other apparent cause than cerebral ischemia.Researchers of all studies who conducted the follow-up patient interviews, were blinded from imaging data.Clinical events were veri ed by a local stroke or neurology physician and con rmed as ischemic by computed tomography or MRI.

Predictors
A preselection of candidate predictors was performed based on previously reported hazard ratios and/or odds ratios. 2,12,15Candidate predictors included age, sex, degree of carotid stenosis, time since the last event, classi cation of the last event (ocular vs cerebral), presence of IPH on MRI, diabetes, hypertension, and hypercholesterolemia.A panel of international experts (n = 17) was consulted on predictor prioritization and availability to ensure face validity.
The time since the last event was de ned as the number of days between the index event and carotid plaque MRI.The classi cation of the index event was categorized into cerebral (stroke/TIA) and ocular (amaurosis fugax).
The category of degree of stenosis (< 50%, 50-69%, 70-99%, based on NASCET criteria) was determined using computed tomography angiography, contrast-enhanced magnetic resonance angiography, or Doppler ultrasonography.On carotid plaque MRI, IPH presence was scored by local observers blinded to clinical data and patient outcome using T 1 -weighted 3D gradient echo images that were acquired on ≥ 1.5T scanners in all cohorts.IPH was de ned as a hyperintense region in the plaque compared to surrounding muscle tissue.The plaque was considered IPH-positive when the signal intensity (SI) ratio was > 1.5 17,18 or > 1.2 19 , while remaining studies did not enforce a minimum SI threshold. 9,20

Statistical analysis
Baseline patient characteristics were summarized using descriptive statistics after testing for normality using QQ-plots.Normally distributed values are presented as mean (± standard deviation), while otherwise values are presented as median (interquartile range).Missing predictor values were imputed using multiple imputations by chained equations (MICE).
Given the general rule-of-thumb, we allowed for a maximum of 1 predictor (including extra degrees of freedom) per 10 events. 7All selected predictors were entered into a Cox proportional hazards model with time-to-ipsilateral ischemic stroke as an outcome.The nal model was pooled from the multiple imputed datasets according to Rubin's rules.The proportional hazards assumption was tested with global and individual tests on the Schoenfeld residuals.
Internal validation and calibration was performed through bootstrapping with 200 samples.We corrected the regression coe cients for over tting and estimated optimism of model performance.Individual risk prediction was converted to one and three-year risk of ipsilateral stroke by means of the linear predictor using mean-centered predictor values and the cumulative baseline survival.Conditional survival was estimated using the time since the last event.Model performance was assessed by measures of discrimination and calibration using the concordance (C)-statistic and the calibration curve.The Cstatistic is an indication of discriminative performance of the model, for which a value of 0.5 indicates a model that functions no better than a coin toss, while a C-statistic of 1 implies a perfect model.Details on model development and validation are provided in the Supplemental Methods.

Risk groups
Patients were categorized into high or intermediate risk based on the rounded median ipsilateral ischemic stroke risk according to the IMPROVE model.Patients were considered to be at high risk when exceeding the median 3-year risk of ipsilateral ischemic stroke, while remaining patients were categorized at intermediate risk.Risk groups are presented in Kaplan-Meier plots accompanied by the log-rank test.The sensitivity, speci city, positive predictive value (PPV) and negative predictive value (NPV) of IMPROVE was determined.The performance was compared to a care-as-usual scenario where all patients with ≥ 50% carotid stenosis were considered at high risk as recommended in guidelines from the European Society for Vascular Surgery. 21

Participants
Data from 794 symptomatic patients with a carotid plaque of ≥ 2 mm thickness were pooled to form the development dataset (Fig. 1).Thirteen patients were excluded based on the index event (6 contralateral strokes or TIAs, 2 posterior circulation strokes) or occlusion of the ipsilateral carotid artery (n = 5).Patients (n = 2) who experienced simultaneous bilateral strokes during follow-up were excluded to prevent bias from potential cardioembolic strokes.Given that IPH presence was a crucial predictor, patients with missing IPH status were excluded (n = 17) in addition to patients with missing follow-up (n = 2).This resulted in a development dataset with in total 760 patients who had a total follow-up of 1825 years and a median follow-up of 13.8 (IQR: 6.0-48.6)months.
The pooled dataset contained 760 patients of whom 68% were males (Table 1).The degree of carotid stenosis was < 50%, 50-69%, or ≥ 70% in 48%, 27%, and 25% of the patients, respectively.Seventy percent of the patients were hypertensive and 54% used statins before the index event.One hundred forty-one patients experienced a total of 142 ipsilateral ischemic cerebrovascular events during a median follow-up duration of 13.8 (IQR: 6.0-48.6)months, of which 65 were strokes, 57 TIAs, and 20 amaurosis fugax (Table 2).Considering 65 primary outcomes (ipsilateral ischemic strokes), a maximum of six predictors could be selected for model development to prevent over tting. 7  Values are presented as n (%) or median (IQR) given that all continuous variables were non-normally distributed.The rst ipsilateral ischemic cerebrovascular event to occur during follow-up are presented.
*All patients had a one-year follow-up.TIA = transient ischemic attack.

Predictor selection
The following predictors were selected based on previously reported hazard ratios and expert opinion: age, sex, classi cation of the last event (ocular vs cerebral), degree of carotid stenosis, and the presence of IPH on MRI (Supplemental Results).Time since the last event was not included as a predictor in the IMPROVE model, however the variable was used to determine the conditional stroke probability given the stroke-free time between the last event and risk calculation.
Information on all predictors was complete for 678 patients (Supplemental Results).Only for degree of stenosis there were some absent values (n = 34).The predictors showed no strong correlations with each other and the proportional hazards assumption was not rejected (Supplemental Results).Consequently, all selected predictors were included in the model.Data of all 760 patients was used for model development after multiple imputation of missing values.

The IMPROVE model
The hazard ratios of the IMPROVE predictors are presented in Fig.

Model performance
The IMPROVE score exhibited good discriminative value in the development dataset with a C-statistic of 0.84 (95% CI: 0.78-0.88).After internal validation, model performance decreased only marginally to 0.82.
Overall, calibration of the model showed good alignment between observed and predicted ipsilateral ischemic stroke risk with a calibration slope of 0.93 (Supplemental Results).Time-dependent calibration showed a calibration slope of 0.93 and 0.94 for 1 and 3-year ipsilateral ischemic stroke-free survival, respectively (Fig. 3).

Estimating the risk of ipsilateral ischemic stroke using IMPROVE
To determine ipsilateral ischemic stroke risk in percentages for clinical use, cumulative baseline survival and formulae for survival predictions are provided (Supplemental Results).
Patients had a mean 1 and 3-year ipsilateral ischemic stroke risk of 2.13% and 8.16%, respectively.Several examples of individual IMPROVE risk estimations are presented in Table 3.A clear example of a high-risk individual is a 73-year old male patient with a recent cerebral stroke and an IPH-positive carotid plaque causing ≥ 70% stenosis, resulting in an IMPROVE-estimated ipsilateral ischemic stroke risk of 16.5% within one year and 51.6% within 3 years.An individual considered for carotid revascularization according to current guidelines, i.e. a 73-year old male with ≥ 70% stenosis, an ocular index event, and no IPH, has an estimated 3-year risk of only 4.2%.On the other hand, patients with < 50% carotid stenosis at relatively high risk of stroke recurrence can be identi ed.A 73-year old male patient with a cerebral index event who is IPH-positive has a 3-year risk of 10.6%.For an 80-year old patient with these characteristics, this risk would be 11.4%.

IMPROVE risk groups
The median IMPROVE-estimated 3-year ipsilateral stroke risk was 9.4% (IQR: 2.0-20.7%) in the development dataset.Thus, patients with an ipsilateral stroke risk of ≥ 9% were categorized into a highrisk pro le, while remaining patients were considered to be at intermediate risk.These risk groups showed a clear division in ipsilateral stroke-free survival (Fig. 4).When stratifying patients using a careas-usual scenario where all patients ≥ 50% were considered as high risk, 5.3% of intermediate risk patients with < 50% stenosis had a recurrent ipsilateral ischemic stroke within 3 years.In comparison, IMPROVE strati cation of patients resulted in a 3-year ipsilateral ischemic stroke recurrence on OMT for only 2.1% of the patients with an intermediate risk pro le.IMPROVE also increased the likelihood of identifying individuals at true high risk of ipsilateral stroke recurrence compared to care-as-usual.
Patients identi ed as high risk by IMPROVE had a larger percentage of ipsilateral ischemic strokes within 3 years (24.0% vs. 20.7%,respectively).
To investigate model stability, several alternative models with varying predictor selection methods, subpopulations of the development dataset, and outcomes of interest were explored and all showed similar performance to the IMPROVE model (Supplemental Results).

Discussion
We developed a novel clinical prediction tool (IMPROVE).IMPROVE is the rst su ciently powered clinical prediction model of ipsilateral ischemic stroke risk for patients with symptomatic carotid artery disease on current optimal medical treatment.IMPROVE exhibited good performance after internal validation.It demonstrates the importance to look beyond the degree of stenosis for risk strati cation, since the ipsilateral ischemic stroke risk can vary to a large extent between patients with similar degree of stenosis.
For the development of the score, individual patient data from ve cohort studies were pooled to generate a derivation cohort with a representative sample of patients with carotid artery stenosis who present with a recent amaurosis fugax, TIA, or minor stroke, comparable to the patient characteristics of previous large clinical trials (i.e.ECST) 1 .However, the median age in our cohort is considerably higher than the previous large trials, due to demographic changes since the 1980s-1990s when the ECST trial was performed.
The performance of IMPROVE in the derivation cohort with a C-statistic of 0.84 (95% CI: 0.78-0.88)was marginally superior to the performance of SCAIL (0.82 (95% CI: 0.66-0.97).While the SCAIL score already demonstrated the potential of using plaque vulnerability for improving risk prediction, external validation of SCAIL resulted in a C-statistic of 0.66 (95% CI: 0.51-0.80). 11The large decline in discriminative performance of SCAIL may be explained by the very low EPV of < 2, considerably lower than the recommended minimum of 10 EPV that was used to develop IMPROVE. 7The CaroTID-VasC score had a C-statistic of 0.88 (95% CI: 0.81-0.96) in the derivation cohort, which declined to 0.83 after internal validation. 6Note that the CaroTID-VasC score was developed in 99 patients with only 25% of included outcomes-of-interest consisting of ipsilateral ischemic strokes.In comparison, IMPROVE was based on ipsilateral ischemic stroke only with 65 events in 760 patients.In addition to the low EPV of 4 for CaroTID-VasC, an univariate analysis and stepwise selection of candidate predictors was used for the development of this score.This can lead to over tting so that the model is expected to show lower performance in other datasets.The ECST model/CAR score had only moderate discriminative performance (C-statistic: 0.67 [95% CI: 0.54-0.80]) in the Plaque At RISK (PARISK) study of 244 symptomatic patients with < 70% carotid stenosis. 9Additionally, in a study of 134 patients with severe carotid stenosis no association was found between the ECST model and recurrent cerebrovascular events (HR = 0.86; 95% CI: 0.45-1.65;P = 0.65). 10e SCAIL score is based on the degree of stenosis and the maximal standard uptake value of carotid plaque on 18 F-uorodeoxyglucose Positron Emission Tomography (PET), an indicator of plaque in ammation. 22The radioactive tracer typically needs to be ordered a few days in advance, making SCAIL practically challenging.In contrast, the presence of IPH for IMPROVE is much easier to obtain with a short 5-minute additional scan with a standard neurovascular coil during a carotid or brain MRI examination.MRI is much cheaper and more readily available compared to PET.The CaroTID-VasC score also includes plaque vulnerability, which is de ned as echolucency on ultrasound for this score. 6Duplex ultrasound is widely available and relatively cheap.However, the intraobserver agreement for scoring plaque echogenicity is known to be poor. 23A more objective and less observer-dependent measure for plaque echogenicity is the gray-scale median (GSM). 24The PARISK study investigated the association between the risk for ipsilateral ischemic symptoms for various imaging modalities in TIA and stroke patients with a carotid plaque.No association between the echogenicity (GSM) and the risk for recurrent ipsilateral cerebrovascular events was reported, while on the contrary IPH was an independent risk factor for ipsilateral ischemic events. 9r study further strengthens the importance of using plaque vulnerability imaging parameters to get insight into the individual risk of patients.The strong contribution of IPH presence on MRI in the IMPROVE tool con rms the importance of incorporating MRI-based carotid plaque features for individualized ipsilateral ischemic stroke risk estimation.While external validation of the IMPROVE model in an independent cohort is of high priority, the IMPROVE score can already be used in clinical practice in border-line cases to provide additional information to patients about their risk and to weigh in this risk on the choice of treatment.The exemplary IMPROVE-threshold of 9% has a high sensitivity of > 90% and a speci city that is also higher than care-as-usual strati cation of high-risk patients based on a degree of stenosis ≥ 50%.The optimal risk threshold will depend on the user's goal, such as strati cation for carotid revascularization or aggressive novel medication.
Due to the increased sensitivity and speci city, we expect that the IMPROVE model will be superior to care-as-usual for selecting patients for carotid revascularization that will bene t most.Decision-analytic studies are warranted to determine the optimal thresholds of IMPROVE-based personalized clinical decision-making.A randomized trial including a cost-effectiveness analysis is urgently needed to demonstrate whether IMPROVE-based risk strati cation for carotid revascularization or novel aggressive medication can lead to a reduction in stroke and/or less costs.
The IMPROVE model has important strengths.Foremost, IMPROVE is the rst su ciently powered model to predict ipsilateral ischemic stroke risk on current OMT incorporating information on clinical risk factors and carotid plaque composition.In contrast, the most common risk score, the ECST model, and its derivative, the CAR score, is based on robust, but outdated data.Second, contrary to the SCAIL and CaroTID-VasC models, IMPROVE complies to the recommended minimum events per variable (EPV) of 10. 25 .Considering the 65 ipsilateral ischemic recurrent strokes in our derivation cohort, the IMPROVE model with 5 predictors and 1 extra degree of freedom due to categorization, has an EPV of 11.Third, for the development of IMPROVE, we used ipsilateral ischemic stroke only and not TIA and amaurosis fugax as a clinical endpoint.Fourth, information of predictor values in the derivation cohort was near complete.Only the degree of stenosis was missing and this occurred in < 5% of the patients, which was comfortably under the threshold of 10% above which bias is believed to be introduced. 26Multiple imputation by the MICE procedure was performed, since it can reduce bias by up to 98% compared to complete case analysis even for only 5% of missing values. 27Fifth, we used preselected predictors based on reported predictor strength in literature and expert opinion.No stepwise selection procedures were used, since they may lead to in ated coe cients, the selection of nuisance variables, and/or exclusion of true variables that are not statistically signi cant but do have predictive value. 28The use of preselected predictors not only positively impacts external validity, IMPROVE is thereby also expected to have good face validity since expert opinion on predictor importance was consulted.Last, while the ECST and CaroTID-VasC models were developed only for patients with ≥ 50% stenosis, outward remodeling of the carotid artery frequently occurs and it is also considered as a feature of plaque vulnerability.Therefore, vulnerable plaques can also be present in patients with < 50% stenosis.IMPROVE is the only model that facilitates ipsilateral ischemic stroke prediction for all patients with a carotid plaque ≥ 2 mm.Indeed, we showed that the IMPROVE model can stratify a subgroup of patients with < 50% stenosis that still have a considerable (> 9% 3-year) ipsilateral ischemic stroke risk.
Presently, we present the internal validation of the IMPROVE score.While external validation in an independent sample should be a goal for future research before large-scale implementation of the risk score, the performance of the model is expected to be highly stable.The derivation cohort was based on ve cohort studies with patients from varying countries and ethnic backgrounds.Also in the ve cohorts, different MRI sequences, different eld strengths and MRI systems from different vendors were used and the presence of IPH was scored by local observers.Therefore, the cohort already exhibits great variability and thus the IMPROVE score has been developed on the building blocks of exibility.We have assessed a number of alternative models developed with backwards selection and in a subset of the development dataset, which showed similar performance to the IMPROVE model, indicating a strong stability of an MRI-based ipsilateral ischemic stroke risk model.Internal validation of the IMPROVE model by means of bootstrapping has shown that the performance of the model decreased only marginally (C-statistic decreased from 0.84 to 0.82), further strengthening our expectation that the model performance will remain of high quality.
The IMPROVE model, utilizing information from novel carotid plaque MRI techniques and clinical risk factors, shows good performance for estimating ipsilateral ischemic stroke risk in symptomatic patients with carotid artery disease.IMPROVE can aid in risk strati cation for secondary stroke prevention strategies.We have demonstrated that IMPROVE is able to identify subpopulations of high-risk patients who are conventionally considered at intermediate risk in care as usual, and similarly, intermediate-risk patients with 70-99% stenosis can be identi ed.External validation and clinical impact analysis are urgently needed towards clinical implementation of IMPROVE.Ipsilateral ischemic stroke-free survival of IMPROVE risk categories.Patients were strati ed using a threshold of ≥9% (high-risk) versus <9% (intermediate-risk) 3-year IMPROVE-based ipsilateral stroke risk.It can be clearly observed that these two categories demonstrate a signi cant difference in cumulative ipsilateral ischemic stroke-free survival (p<0.0001).

Table 1
Patient characteristics of the pooled development dataset

Table 2
Ipsilateral ischemic cerebrovascular event recurrence during follow-up

Table 3
Examples of IMPROVE-based ipsilateral ischemic stroke risk