CT Angiography-Based Radiomics as a Tool for Carotid Plaque Characterization and Surgery Patient Selection: A Pilot Study

Objectives: Radiomics is a quantitative method able to analyze a high-throughput extraction of minable imaging features. Herein, we aim to develop a CT angiography-based radiomics analysis and machine learning model for carotid plaques to discriminate vulnerable from no vulnerable plaques. Methods: Thirty consecutive patients with carotid atherosclerosis were enrolled in this pilot study. At surgery, a binary classication of plaques was adopted (“hard” vs “soft”). Feature extraction was performed using the R software package Moddicom. Pairwise feature interdependencies were evaluated using the Spearman rank correlation coecient. A univariate analysis was performed to assess the association between each feature and the plaque classication and chose top-ranked features. The feature predictive value was investigated using binary logistic regression. A stepwise backward elimination procedure was performed to minimize the Akaike information criterion (AIC). The nal signicant features were used to build the models for binary classication of carotid plaques, including logistic regression (LR), support vector machine (SVM), and classication and regression tree analysis (CART). All models were cross-validated using 5-fold cross validation. Class-specic accuracy, precision, recall and F-measure evaluation metrics were used to quantify classier output quality. Results: A total of 230 radiomics features were extracted from each plaque. Pairwise Spearman correlation between features reported a high level of correlations, with more than 80% correlating with at least one other feature at |ρ| > 0.8. After a stepwise backward elimination procedure, the entropy and volume features were found to be the most signicantly associated with the two plaque groups (p< 0.001), with AUCs of 0.92 and 0.96, respectively. The best performance was registered by the SVM classier with the RBF kernel, with accuracy, precision, recall and F-score equal to 86.7, 92.9, 81.3 and 86.7%, respectively. The CART classication tree model for the entropy and volume features model achieved 86.7% well-classied plaques and an AUC of 0.987. Conclusion: This pilot study highlighted the potential of CTA-based radiomics and machine learning to discriminate plaque composition. This new approach has the potential to provide a reliable method to improve risk stratication in patients with carotid atherosclerosis.


Introduction
Carotid atherosclerotic plaque ulceration and rupture are one of the main causes of major adverse cardiac events such as ischemic stroke, which account for approximately one-fourth of all strokes (1).
Luminal stenosis degree has been historically considered the most important parameter to grade the severity of atherosclerotic disease and risk-stratify patients and treatment decision-making (2). However, new pathological studies have demonstrated major differences in the composition and risk of rupture for plaques with identical degrees of stenosis (3). The technological advancement of vascular imaging techniques such as CT and MR angiography allowed a routine high-resolution characterization of carotid plaque features. Recent literature has highlighted the critical role of plaque composition as a predictive factor of future cerebrovascular events (4)(5)(6)(7)(8). Plaque ulceration, intraplaque hemorrhage, thin or ruptured brous cap, lipid-rich necrotic core and the presence of calci cation have been proposed as structural plaque characteristics to distinguish vulnerable from nonvulnerable plaques (9)(10)(11)(12). In particular, vulnerable plaques reported a high risk of rupture, resulting in thrombus formation and embolization of plaque material and/or thrombus migrating into the intracranial circulation. Among advanced imaging modalities, CT angiography (CTA) has been established as an effective robust noninvasive imaging technique to evaluate carotid atherosclerotic plaque-related stenosis and to provide information on the composition of the plaque (13). Indeed, in addition to the stenosis degree assessment, other major relevant applications included the ability of CTA to identify speci c imaging markers associated with cerebrovascular ischemia, including plaque thickness (14), ulceration (15), and calci cation (16). However, despite its reliability, CTA imaging is a complex procedure requiring high expertise in assessing plaque structure components, translating into a qualitative and subjective task (17).
Radiological images, including those obtained by CTA imaging, are multidimensional data sets in which each voxel value represents a speci c metric based on some physical characteristics (18). This is the speci c eld of interest of radiomics, a complex computing process of obtaining a large number of quantitative parameters, called "radiomic features", from these multidimensional spatial datasets. These features aim to mathematically quantify the morphological characteristics of a speci c tissue, often di cult to interpret by visual assessment. In recent years, this approach has proven to be an effective tool in oncology, reporting a major ability in tumor staging and grading, response assessment to treatments and prediction of clinical outcomes (19).
Recently, a radiomics approach has emerged as a reliable tool for the assessment of cardiovascular diseases, particularly for the quantitative study of coronary and carotid atherosclerotic plaques (20,21). The rst attempts employed CT-based (22) and ultrasound-based (23) texture analysis to discriminate symptomatic carotid plaques reporting promising results. CT-and MRI-based machine learning algorithms were successfully implementedto accurately distinguish symptomatic from asymptomatic carotid plaques and proved to be superior to the traditional model in the identi cation of high-risk plaques (22,24). Le et al. (25) evaluated the ability of several robust features to identify culprit and nonculprit arteries using different machine learning algorithms and reported an average area under the curve of 0.73 and accuracy of 69%. Concerning risk assessment, Van Assen et al. (26) aimed to investigate an AI algorithm to predict major cardiac adverse events based on plaque characteristics. The authors reported that the model obtained combining both clinical risk factors and morphological features in a logistic regression analysis provided an accuracy of 87% with an AUC of 0.924. A recent study (27) investigated the potential of CTA texture analysis in identifying vulnerable patients with carotid artery atherosclerosis. The authors reported that texture analysis revealed a statistically signi cant difference in the skewness feature to distinguish symptomatic and asymptomatic patients, with an AUC of 0.68, sensitivity of 64% and speci city of 71%.
The aim of the study was to develop a CTA-based radiomics analysis model on carotid plaquesto discriminate vulnerable from nonvulnerable plaques and to identify surgical candidates in a timely manner. Different binary classi cation models were built using a machine-learning approach to improve the identi cation.

Study population
This study was conceived within the Gemelli Generator Molise (G2 M) group study and was approved by the Institutional Review Board of the Gemelli Molise Hospital (Campobasso, Italy). Thirty patients were retrospectively enrolled from October 2015 to October 2019. All patients aged >18 and <75 years, diabetic and nondiabetic, with indications to receive carotid endarterectomy (TEA) for extracranial high-grade (>70%) internal carotid artery stenosis were included. High-risk cardiological patient candidates for carotid revascularization by stent angioplasty or myocardial and carotid revascularization by combined aorto-coronary bypass surgery and carotid TEA were excluded.
Patients entering the study signed informed consent for treatment and use of their clinical data for research or educational purposes. Patient data were anonymized, managed according to the existing legislation for the protection of privacy and uploaded to the segmentation and analysis platforms.
Plaques classi cation at surgery At surgery, carotid artery plaques were considered "hard" when their composition was predominantly calci ed or with collagen-rich sclerotic tissue and "soft" when they contained atheromatous debris or in case of intraplaque hemorrhage (28,29). This schematic classi cation was based on the common assumptions that the sclerotic component, which is brous tissue, in "hard" plaques should stabilize these plaques and protect them against disruption, while the "soft" plaques should be considered "at-risk" because the atheromatous gruel destabilizes the plaque, thus leaving it vulnerable to rupture (30). Moreover, this binary classi cation choice was usefulto counteract the small sample size.
Imaging data and segmentation CT angiography was performed with a 128-slice scanner (Brilliance 128, Philips Healthcare, Best, the Netherlands). An 18-gauge intravenous catheter was placed in the antecubital vein; 55 mL of contrast, iomeprol 400 mg/ml (Iomeron®, Bracco, Milan, Italy) was infused at 4 mL/s after an initial injection delay depending on an attenuation of 140 HU in the ascending aorta with a slice thickness of 0.9 mm. Curved multiplanar and volume rendering reconstructions were obtained by means of dedicated computer software.
CTA images were transferred by the Digital Imaging and Communications in Medicine (DICOM) protocol to a treatment planning system (Philips Pinnacle, Fichburg, USA) in our radiation oncology department, containing a dedicated module for image segmentation and analysis. The plaques were manually segmented on all CT slices by a senior radiologist (A.P.) and by a vascular surgeon (P.M.).

Radiomics analysis
Feature extraction was performed using Moddicom (31), an R software package composed of classes written using functions and the S3 objects paradigm, which was standardized through the Image Biomarker Standardization Initiative (IBSI) (32). Two hundred thirty features, including rst-order statistical features, shape-based features, gray-level cooccurrence matrix features (GLCM), gray-level run length matrix (GLRLM), gray-level size zone matrix (GLSZM), neighboring gray tone difference matrix (NGTDM), and gray level dependence matrix (GLDM) features, were extracted (32).

Features selection and modeling
Radiomic feature extraction usually leads to a highly redundant feature space. Since many of these features could be highly correlated with one another, a feature selection method is necessary to avoid collinearity and reduce dimensionality. Following the common practice known as "one in ten rule", i.e., for each feature, at least ten subjects should be available for each class, our aim was to select no more than two features to build reliable models. To address this problem, we used a multistage feature selection method. First, the pairwise feature interdependencies were evaluated using the Spearman rank correlation coe cient (ρ), aiming to identify complex functional dependencies between features. The redundant features, de ned as those having |ρ| ≥ 0.8, were eliminated. In the second stage, a univariate analysis was performed to assess the association between each feature and the plaque classi cation and chose the top-ranked features. Subsequently, we investigated the feature predictive value in a binary logistic regression. To determine the relative importance of features, the remaining features were included in a stepwise backward elimination approach. In each step of this method, a feature is considered for subtraction from the set of explanatory variables based on speci ed criteria, including the Akaike information criterion (AIC). In particular, we tested the impact of the deletion of each feature whose loss gives the most statistically insigni cant deterioration on the model. AIC is derived from information theory and is a model selection criterion that penalizes models for which adding new explanatory variables does not supply su cient information to the model. The aim is to minimize the AIC. It is de ned as: where logL(M) is the maximized log likelihood for the tted model, N is the sample size and K is the number of covariates including an intercept.
The nal signi cant features were nally used to build the models for binary classi cation of carotid plaques, including logistic regression (LR), support vector machine (SVM), and classi cation and regression tree analysis (CART).
LR is a classical machine learning algorithm that is usually used for binary classi cation tasks. This model provides the probability p(y = 1|x), that is, the probability of a positive result y = 1 under the given data x. LR has the advantage of fast training, and discrete and continuous variables can be used as inputs (33).
SVM-based classi cation models are widely used supervised classi cation algorithms. For a given set of feature data from two groups of patients, SVM attempts to determine the maximum hyperplane between the two classes to maximize its distance to the nearest data points on each side (the so-called support vectors). SVM represents a powerful method fable to obtain good classi cation results by using only a few data points (34). In particular, if the two samples are not linearly separable, some kernel functions can be used to map them to higher dimensional space where they become more separable (35). In this study, we used four kernel types, including linear, power, sigmoid and radial basis function (RBF) kernels. To maximize the margin distance between the hyperplane and the closest samples of both classes, the internal parameter of the training process, the cost C, was set to 1 after a tuning process to obtain the optimal classi cation results.
Finally, the nal features were used as input for a classi cation and regression tree analysis (CART) (36) to visually stratify patients into a risk group model to stratify patients into plaque risk groups. The representation for the CART model is a binary tree; each root node represents a single input feature and a split point on that feature. The leaf nodes of the tree contain an output variable that is used to make a prediction. The best splits were identi ed by the Gini impurity (GI) index: where p i is the fraction of items in the class i. The models were cross-validated using 5-fold cross validation (37). This is a resampling method that randomly partitions each feature dataset into ve equally sized subsets of samples (folds), maintaining a balanced amount of both classes in each fold. In this way, ve models were trained and tested; each of the ve folds was used once as the test set, while the four remaining folds were used to train the model. The process was repeated ten times aiming to reduce the variance of the cross validation results and to reduce the unlikely chance of obtaining overoptimistic results with just one run.
The performance of the models (i.e. the ability of radiomics features to discriminate and classify plaques was assessed using receiver operator characteristic curves (ROCs) and the area under the curve (AUC). In addition, class-speci c accuracy, precision, recall and F-measure evaluation metrics were also used for their ability to provide classi er output quality. The accuracy is de ned as the proportion of correct predictions (both true positives and true negatives) among the total number of cases examined. The precision is de ned as the number of true positive results divided by the number of all positive results, including those not identi ed correctly (it is also known as positive predictive value). It can be thought of as a measure of classi er exactness. A low precision indicates a large number of false positives. The recall is de ned as the number of true positive results divided by the number of all samples that should have been identi ed as positive (it is also known as sensitivity in binary classi cation or true positive rate). It can be thought of as a measure of classi er completeness. A low recall indicates many False Negatives. The F-score is de ned as the harmonic mean of the precision and recall. Similar to AUC, these measures range from 0 to 1, with higher values indicating better classi cation performance.

Statistical analysis
All statistical analyses were performed using the XLSTAT statistical packages (Addinsoft, New York, USA).

Patient characteristics
Thirty patients (19 males and 11 females, median age 73, range 50-86) were included in the study. Patients characteristics are listed in Table 1:

Features extraction and selection
A total of 230 radiomics features were extracted from each plaque to predict plaque composition. Pairwise Spearman correlation between features reported a high level of correlations, with more than 80% correlating with at least one other feature at |ρ| > 0.8. Furthermore, in univariate analysis, 10 features were found to be signi cantly associated with the two plaque groups (p< 0.05). Detailed diagnostic accuracy statistics for the ten highly correlated features are reported in Table 2. Out of these features, only two features were selected after the stepwise backward elimination procedure, namely, the entropy, de ned as: Entropy = ∑ n i = 1 p(i)log 2 (p(i)) which measures the average amount of information required to encode the image values, and the mesh volume is de ned as: V is calculated from the triangular mesh of the ROI; for each face i in the mesh, de ned by points ai, bi and ci, the volume V of the tetrahedron de ned by that face and the origin of the image (O) are calculated, and their sum provides the total volume of the ROI.
The box plots for the two top features found to be associated with plaque classi cation are shown in Figure 1; the diagnostic performance AUCs were 0.924 and 0.960 for and volume, respectively.
Machine learning modeling and performance     Figure 2 shows the CART classi cation tree for the two most informative radiomic features. For each node, it was reported the number of objects at each node, the corresponding percentage, the improvement which correspond to the number of observations in the node times, the reduction in impurity caused by the split of the node, the purity that indicates what is the percentage of objects that belong to the dominating category of the dependent variable at this node, the split variable and the value(s) or intervals of the latter, and the class predicted by the node.
Node 1 represents the initial distribution of the plaques before any criteria had been applied. Nodes 2 and 3 re ect the effect of introducing the volume feature as a rst decision criterion. The category volume ≤ 481.8 relocates most of the plaques to node 2 with a purity index of 100%, e.g., all plaques have a calci c composition. The plaques in node 3 are located in the soft group and will be subjected to the second criterion for further segregation, e.g., the entropy feature. At this second decision level, entropy values greater than 12.5 relocate all plaques in the soft group with 100% purity. Looking at Figure 2, the rule built by the algorithm can be expressed in natural language as "If Volume < 481.8 then plaque = 1 in 100% of cases". Similarly, at node 5, "If Entropy 12.5 and Volume 481.8, then plaque = 2 in 100% of cases".
The confusion matrix of the results is reported in Table 5 and was used to calculate the percentage of well-classi ed observations. The model achieved 86.7% well-classi ed plaques and an AUC of 0.987.

Discussion
In recent years, the assessment of carotid atherosclerosis and risk strati cation have undergone a paradigm shift moving from the usual measurement of carotid stenosis degree to a more complete characterization of plaque composition. Prospective multicenter trials, such as PARISK (38), con rmed this approach, highlighting the role of plaque morphology in patient outcome. However, plaque imaging techniques still do not representthe basics for clinical decision making and routine clinical practice (39).
Recent guidelines have been issued by the European Society for Vascular Surgery (ESVS), aiming to provide a more uni ed approach for patients with carotid disease (40) and highlighting the need for new clinical/imaging noninvasive reliable methods able to identify the morphological features of vulnerable plaques.
Recently, following this suggestion, several researchers have attempted a radiomics-based methodology to overcome the limitation of visual assessment and provide a more objective method to characterize carotid atherosclerotic plaques (41)(42)(43). Zaccagna et al. (41) assessed the potential role of CT texture analysis in identifying vulnerable patients with carotid artery atherosclerosis. The authors highlighted the feasibility of identifying vulnerable patients in stroke and TIA, reporting CT texture as a reliable risk strati cation tool in patients with carotid atherosclerosis. In particular, the authors identi edskewness as the most statistically signi cant metric able to differentiate between symptomatic and asymptomatic patients. Le et al (42) assessed the robustness of carotid artery CT angiography radiomics in the identi cation of culprit lesions in cerebrovascular events. The authors identi ed a set of radiomics features that are robust, nonredundant and have superior predictive performance for the classi cation of culprit versus nonculprit carotid arteries in patients with stroke and TIA. Finally, Zhang et al. (43) successfully implemented a high-risk plaque MRI-based model using radiomics features and machine learning for differentiating symptomatic from asymptomatic carotid plaques. This MRI-based radiomics model was found to be superior to the traditional model in the identi cation of high-risk plaques.
In this study, we sought to build a machine learning-based model using radiomics features for differentiating hard from soft carotid plaques using CTA imaging. In particular, after the feature extraction and selection processes, the volume and entropy were found to have signi cantly higher values in plaques considered "soft". Radiomics modeling presents a distinct advantage over "black-box" techniques such as deep learning because it allows a potential interpretability and explanation of the obtained results.In this way, we can attempt to provide enlightenment on the prominence of volume and entropy features in the present classi cation model. The volume ofcarotid artery plaque has already been proven to be an important determinant of plaque vulnerability and the risk of rupture, and some authors have suggested that plaque volume should be considered a better descriptor of the severity of atherosclerotic disease than the degree of stenosis (44).Saba et al. (45) reported that the lipid component volume is associated with the presence of plaque ulcerations, which represent a signi cant risk factor for the development of cerebrovascular events, and that there is an increase in the proportion of lipids and calci cations with increasing plaque volume. Recently, Lu et al. (46) reported that the progression of carotid plaque volume is independently associated with recurrent ischemic cerebrovascular events.
Therefore, the plaque volume assessment and the monitoring of its temporal changes may improve the ability to estimate risk. The entropy feature describes the complexity of a segmented ROI so that the larger the entropy is, the more complex the ROI. This feature was found to be promising in oncoradiomics, where it has been associated with tumor gene expression, tumor metabolism, tumor stage, patient prognosis, and treatment response (47,48). Since entropy is a measure of texture irregularity, this feature can be a reliable surrogate for the quanti cation of plaque spatial heterogeneity. In this study, we reported that entropy values were signi cantly higher for "soft" plaques than for "hard" plaques. This is an indication that soft plaques present a more random density distribution, probably due to the presence of various components with various low-density Houns eld numbers (HU). In particular, HU values less than 0 in the plaque are associated with the presence of intraplaque hemorrhage, while HU values ranging from 0 to 60 are usually associated with fatty plaques (49). Both situations are associated with an increased risk of stroke (50). In contrast, calci ed plaques are classi ed according to HU values greater than 130 (49), and the presence of calcium seems to represent a protective factor, providing stability against biomechanical stress and disruption (51).
Using these top two ranked features, we trained radiomics-based machine learning models using SVM and CART analysis methods. Among the various models, our results reported that the SVM classi er with the RBF kernel achieved the best performance and was able to successfully create the hyperplane and maximize the margins between the support vectors. It reported higher accuracy and F-score values, both equal to 86.7%, able to correctly classify 13 out 16hard plaques and 13 out 14 soft plaques. We also build a CART model as a knowledge-discovery tool because of its well-known interpretability in terms of variable thresholds. To obtain a simple and small decision tree, we chose to model our CART with only 5 nodes and two depth levels. CART analysis chose patients with a plaque volume greater than 481.8 to be associated with soft plaques, i.e., plaques at increased risk. The entropy feature then subsequently played a signi cant role in increasing the classi cation rate among these plaques. The model provided a very high diagnostic performance of AUC=0.987; due to the high classi cation accuracy, we believe that this approach could be used as a valuable adjunct tool for vascular surgeons in prescribing effective treatment options. In particular, for asymptomatic patients with stenosis degrees greater than 70%, the use of a decision tree could provide a valuable tool in discriminating patients eligible for elective surgery.
The authors are currently gathering more data to better stratify the plaque classi cation beyond the proposed binary classi cation.
Despite encouraging results, the present study has several limitations, and some caution must be employed regarding any robust conclusions. First, the sample size used in this study to train the radiomics analysis of carotid plaques was relatively small (n=30). For this reason, we choose a simple binary plaque classi cation (hard vs. soft) to allow a reasonable sample size for each group. Second, carotid plaques were from a single center, and radiomics results were based on CT images from one scanner and software analysis system; therefore, generalizability of the results beyond these settings must be proven. Third, although all the radiomics feature calculations were automated, plaque segmentation was manually performed by a senior radiologist, a procedure always prone to interoperator variability. The application of automated segmentation, as already implemented for the de nition of volumes in radiation oncology, will reduce the interoperator variability and improve the clinical feasibility of radiomics. Overall, the present study is a pilot study conducted on a smaller scale to assess the ability of a quantitative radiomics approach to differentiate potentially vulnerable plaques. Although reliable internal validation was performed to minimize sample bias with a k-fold cross validation process, the developed model has not been externally validated. An external validation of our results is ongoing with an independent dataset to determine the prediction model's reproducibility and generalizability to new and different patients.

Conclusions
A radiomics-based analysis of carotid plaques on CTA images can provide a more accurate plaque risk strati cation and distinguish between hard and soft plaques. This quantitative approach could be easily implemented in current standard clinical decision paths, increasing the reproducibility andaccuracy of CTA imaging. Prospective larger studies are needed to further ascertain the ability of radiomics to predict future stroke risk.

Declarations
Funding The authors received no speci c nancial support for the research, authorship, and/or publication of this article.
Con ict of interest All authors declare no competing interests.
Author Contributions SC, MM, VV and PM contributed to the conception and design of the study. GM, FD, AR and LB organized the database. Sc, JL, EHT amd LI performed the radiomics analysis. LP, MF and CS performed statistical analysis. AP performed radiological analysis. SC, GM and FD wrote the rst draft of the manuscript. AP, PM and CS wrote sections of the manuscript. CMDF, EC, VV and MM critically revised the manuscript. All authors contributed to manuscript revision and read and approved the submitted version.
Ethical standards All procedures performed in studies involving human participants or on human tissue were in accordance with the ethical standards of the institutional and/or national research committee and with the 1975 Helsinki declaration and its later amendments or comparable ethical standards.
Patients signed informed consent to the treatment procedure; the study was retrospective.

Figure 1
Diagnostic performance of the entropy and volume features for plaque composition prediction.