Diagnostic performance of deep learning algorithm for analysis of computed tomography myocardial perfusion

To evaluate the diagnostic accuracy of a deep learning (DL) algorithm predicting hemodynamically significant coronary artery disease (CAD) by using a rest dataset of myocardial computed tomography perfusion (CTP) as compared to invasive evaluation. One hundred and twelve consecutive symptomatic patients scheduled for clinically indicated invasive coronary angiography (ICA) underwent CCTA plus static stress CTP and ICA with invasive fractional flow reserve (FFR) for stenoses ranging between 30 and 80%. Subsequently, a DL algorithm for the prediction of significant CAD by using the rest dataset (CTP-DLrest) and stress dataset (CTP-DLstress) was developed. The diagnostic accuracy for identification of significant CAD using CCTA, CCTA + CTP stress, CCTA + CTP-DLrest, and CCTA + CTP-DLstress was measured and compared. The time of analysis for CTP stress, CTP-DLrest, and CTP-DLStress was recorded. Patient-specific sensitivity, specificity, NPV, PPV, accuracy, and area under the curve (AUC) of CCTA alone and CCTA + CTPStress were 100%, 33%, 100%, 54%, 63%, 67% and 86%, 89%, 89%, 86%, 88%, 87%, respectively. Patient-specific sensitivity, specificity, NPV, PPV, accuracy, and AUC of CCTA + DLrest and CCTA + DLstress were 100%, 72%, 100%, 74%, 84%, 96% and 93%, 83%, 94%, 81%, 88%, 98%, respectively. All CCTA + CTP stress, CCTA + CTP-DLRest, and CCTA + CTP-DLStress significantly improved detection of hemodynamically significant CAD compared to CCTA alone (p < 0.01). Time of CTP-DL was significantly lower as compared to human analysis (39.2 ± 3.2 vs. 379.6 ± 68.0 s, p < 0.001). Evaluation of myocardial ischemia using a DL approach on rest CTP datasets is feasible and accurate. This approach may be a useful gatekeeper prior to CTP stress..


Introduction
Coronary artery disease (CAD) represents one of the leading causes of mortality in the USA [1]. Traditionally, anatomical evaluation of coronaries was strictly confined to invasive coronary angiography (ICA); today, coronary computed tomography angiography (CCTA) represents an important tool to rule out CAD [2,3].
Novel techniques have developed such as CT myocardial perfusion under stress conditions (CTP Stress ) and fractional flow reserve computed tomography derived (FFR CT ) demonstrating the possibility to bridge the coronary anatomy with the physiology and evaluate myocardial ischemia [4,5]. The depiction of coronary plaques leading to myocardial ischemia is extremely helpful in clinical practice, increasing the diagnostic accuracy, particularly the positive predictive value of CT [6], allowing a reclassification of patients that need ICA [7].
The applications of FFR CT and CTP Stress are not widely applied in clinical practice. Ideally, CTP Stress uses image datasets acquired with last generation CT scanners [8] that are not available in all centers and the readers need specific training in order to avoid inaccurate interpretation of images [9,10]. In addition, stress CTP requires a double acquisition (rest and stress) with related iodine contrast agent administration, radiation exposure concerns, and the use of a stressor with potential side effects [4,11].
Application of artificial intelligence in cardiac imaging is ranging from image acquisition to image reporting and prognostic stratification [12][13][14][15]. An advantage of this approach was highlighted by the possibility to speed up the time of reporting [12].
The aim of this article is to show the diagnostic accuracy of a deep learning (DL) model trained on rest and stress CTP for the identification of reversible myocardial ischemia compared with CCTA + stress CTP using ICA plus clinically indicated invasive FFR as the reference standard.

Population
One hundred and forty-seven consecutive patients with suspected CAD enrolled in the PERFECTION study [16,17] were retrospectively analyzed for this study. Patients with a history of previous myocardial infarction, acute coronary syndrome, previous revascularization, impaired renal function or contraindication to administration of contrast agent, pregnancy, cardiac arrhythmias, inability to sustain breath hold, BMI > 35, and contraindication to administer B-blockers and nitrates were excluded from the study as previously described [16,17]. The institutional ethical committee approved the protocol and all patients signed an informed consent.

Patient preparation
Patients were asked to refrain from caffeine and smoking for 24 h and fasting 6 h prior the CCTA examination.
During rest CCTA, in patients with heart rate > 65 beats/ min (bpm), metoprolol with a titration dose up to 15 mg was administered intravenously in order to obtain a HR ≤ 65 bpm.
Before the rest scan, all patients received sublingual nitrates to ensure coronary vasodilatation.

CCTA and CTP rest
All examinations were performed using a Revolution CT scanner (GE Healthcare, Milwaukee, Wisconsin) following the guidelines of the Society of Cardiovascular Computed Tomography (SCCT) [18]. The Rest CCTA parameters have been described previously [5].
Based on HR, CCTA was acquired in 70-80% of the cardiac cycle in patients with HR ≤ 65 bpm, while in patients with > 65 bpm, CCTA were acquired between 40 and 80% of cardiac cycle. CCTA images were acquired after the injection of 70 ml of iodine contrast agent (Visipaque 320 mg/ ml) at 6.2 ml/s followed by 50 ml of saline at the same flow rate of contrast agent.
All scans were performed using the bolus tracking technique with visual assessment to determine the correct timing for acquisition.
All images were reconstructed using an adaptative statistical iterative reconstruction (ASIR-V, GE Healthcare, Milwaukee, Wisconsin) [19]. Datasets of CCTA were transferred to an image-processing workstation (Advantage Workstation 4.7, GE Healthcare, Milwaukee, Wisconsin) to perform quantitative coronary analysis in consensus, according to SCCT guidelines for reporting by two certified expert readers with more than 8 years of experience in cardiovascular imaging following the European Association Cardiovascular Imaging (EACVI) guidelines for training and certification [20]. In particular, the stenosis was calculated using as reference standard the healthy coronary area before or after the coronary stenosis. In case of disagreement, a third cardiac radiologist evaluated the images.
Acquisition of CTP rest images was the same of CCTA. The comparison of CTP rest with CTP stress will be described in the section of CCTA + CTP stress interpretation.

CTP stress
The patient underwent intravenous administration of adenosine (0.14 mg/kg/min) over 4 min. At the end of the third minute of adenosine administration, a CCTA was acquired using the same technical parameters previously described [5]. Subsequently, images were transferred to an offline workstation (Advantage Workstation Version 4.7, GE Healthcare), reconstructed on short axis and long axis with an average slice thickness between 4 and 8 mm using a narrow window width and level of 350 W and 150 L, respectively. Perfusion datasets were analyzed in consensus by two certified expert readers, blinded to clinical history and CCTA findings, with more than 8 years of experience in cardiovascular imaging following the EACVI guidelines for training and certification [20]. In case of disagreement, a third reader evaluated the images.

Combined CCTA + stress CTP interpretation
Coronary arteries at CCTA were segmented according to the American Heart Association (AHA) model [21]. Obstructive CAD was defined as coronary stenosis > 50%.
Stress CTP was evaluated according to the AHA myocardial segmentation [22] model. Stress positivity was defined as any subendocardial hypo-enhancement extending more than 25% of transmurality in a specific territory that was not present at rest CTP.
Matching between CCTA and CTP Stress findings was performed according to the algorithm previously described [16,22,23].
Briefly, the adjudication process was applied each time there was a coronary arterial lesion with > 50% diameter stenosis and at least 1 myocardial perfusion defect in the matched myocardial segment.
Time of image reporting analysis for stress CTP was recorded.

ICA and invasive FFR performance and interpretation
All patients underwent ICA following the guidelines of the American College of Cardiology/AHA Task Force on Practice Guidelines and the Society for Cardiac Angiography and Interventions [24]. The coronary arteries were classified according to the AHA Classification system [21]. An interventional cardiologist blinded to results of CCTA analyzed ICA images with quantitative coronary angiography (QCA) (QantCor QCA; Pie Medical Imaging, Maastricht, the Netherlands).
Coronary stenosis was evaluated calculating the percentage of narrowing using the minimum diameter and reference diameter. In the presence of stenosis ranging between 30 and 80%, invasive FFR was calculated [25]. In order to calculate FFR, the pressure wire (Certus Pressure Wire, St. Jude Medical Systems, St. Paul, MI) was equalized with the aortic pressure and subsequently was placed distal to the stenosis in the distal third of the coronary artery with stenosis. An injection of 100 mg of glyceryl trinitrate was injected intracoronary in order to prevent vasospasm while 140 mg/kg/ min of adenosine was administered intravenously in order to create an inducible stress. FFR was assessed at the peak of hyperemia using the RadiAnalyzer Xpress (Radi Medical Systems, Uppsala, Sweden) by dividing the mean coronary pressure, measured with the pressure sensor placed distal to the stenosis, by the mean aortic pressure measured through the guide catheter. Intermediate stenoses showing values of invasive FFR ≤ 0.8 or anatomical stenoses showing > 80% diameter reduction or total occlusions were considered functionally significant.

Dataset generation
We selected 112 patients from the 147 studied; 58 with invasive FFR ≤ 0.8 or stenosis > 80% (group 0) and 54 with invasive FFR > 0.8 (group 1). Thirty five patients were excluded due to poor image quality (23 patients) or an inconsistent dataset due to corrupted images (12 patients). First, we randomly split the dataset in two subsets: an "independent test set," generated selecting 18 and 14 samples from group 0 and group 1, respectively, and a "learning set" composed of the remaining 80. To limit the effect of overfitting during the training step, a data augmentation step was applied. At the end, the learning set consisted of 80 group 0 and 80 group 1 samples (Fig. 1). The learning set was used to train the algorithms and to identify its best configuration of parameters and hyper-parameters; conversely, the independent test set was used to assess the performance.

DICOM-to-image pre-processing
Each dataset (rest and stress) scan was composed of 256 or 224 slices (512 × 512 pixels) and stored as a DICOM. To focus our attention on the left ventricle, we manually defined a region of interest (a 250-by-250 px window) around the left ventricle, allowing a decrease of computational burden and improving the overall network performance. Finally, the resulting slices were joined in a 10-by-10 squared image, resized to 1280 × 1280 pixels.

Convolutional neural network (CNN) architecture
The keras package has been used to build and test the CNN. The convolutional section of the network consisted of four consecutive layer blocks, each one containing a single convolutional layer and a max-pooling layer. A densely connected network with 4 hidden layers (512, 256, 64, and 32 neurons) and the output layer followed the convolutional section. To handle overfitting, a dropout strategy was implemented before the first hidden layer (dropout rate: 0.2) [26]. The ReLU activation function [27] was used for each neuron (densely connected and convolutional ones), except for the output node (activation function: "sigmoid"). Finally, we set the "stochastic gradient descend" optimizer to minimize the "binary cross entropy" loss function, the number of max epochs = 500, and the batch size = 20.
The CNN was trained in order to predict the ischemic myocardium by using both rest (CTP-DL Rest ) and stress (CTP-DL Stress ) datasets based on the values of invasive FFR. The time of analysis for the evaluation of algorithms for both CTP-DL Rest and CTP-DL Stress was recorded.
A representative figure of the workflow is represented in Fig. 1.

Statistical analysis
Statistical analysis was performed using SPSS version 21.0 software (SPSS, Chicago, Illinois) and R version 4.0.2 (R Foundation for statistical Computing, Vienna, Austria).
Continuous variables are presented as means ± SD, while categorical data are reported as frequencies and percentages and compared with Student's t-test or chi-squared test, respectively. Association between variables was assessed by Pearson's correlation coefficient (R). Patient-based sensitivity, specificity, negative predictive value (NPV), positive predictive value (PPV), diagnostic accuracy, and area under the curve (AUC) were measured for CCTA, CCTA + stress CTP, CCTA + CTP-DL Rest and CCTA + CTP-DL Stress on the independent test set and DeLong method used to compare the differences in terms of AUC between all approaches.
Time of analysis between human and DL algorithm for both CTP-DL rest and CTP-DL Stress was compared.
Differences were deemed significant if the p-value was < 0.05.

Results
Baseline characteristics of the overall population are summarized in Table 1. Obstructive CAD was observed in 78 out of 112 patients (66%) while the prevalence of functionally significant CAD was detected in 54 out of 112 patients (49%). CTP stress, CTP-DL Rest , and CTP-DL Stress were successfully performed in all patients.
The DL algorithm was associated with a correct prediction regardless of the vessel involved in stress and rest datasets, while a significant but fair correlation observed with transmurality in the stress dataset. If the significant pathology was located in the LAD and RCA the algorithm showed better prediction when compared to the LCX ( Table 3).
The evaluation of myocardial perfusion using DL was significantly lower as compared to the standard human approach (39.2 ± 3.2 vs. 379.6 ± 68.0 s, p < 0.001) (Fig. 3). Representative cases are shown in Figs. 4 and 5.

Discussion
In this paper, we showed the possibility to provide information regarding ischemic myocardium using a fully automated DL algorithm with comparable diagnostic accuracy and shorter time to human analysis. The DL algorithm can be applied to both rest and stress datasets with similar high diagnostic accuracy. The clinical application of these findings can be further improved with the combination of anatomical evaluation and myocardial perfusion using the DL analysis with an AUC, diagnostic accuracy, and negative predictive value of 96%, 84%, and 100%, respectively. Irrespective of the vessel involved and transmurality, for stress datasets, the DL algorithm seems to provide good performance.
Another important finding was demonstrated by the short time of analysis of myocardium using a DL approach.
Few papers in the literature analyzed CTP using an artificial intelligence approach [28,29].
Xiong et al. developed a machine learning algorithm that was able to identify myocardial ischemia on rest images [28]. The authors developed an algorithm that was helpful in providing the myocardial contours and identifying the myocardial segments susceptible for myocardial ischemia when compared with invasive quantitative coronary angiography [28]. Despite the authors' lack of a gold standard for detection of myocardial ischemia, they developed an algorithm that was able to detect impairment of myocardial perfusion in patient with significant stenosis with an accuracy, sensitivity, and specificity of 70%, 79%, and 64%. However, it is important to consider that the authors did not evaluate the presence of ischemia with invasive FFR. Moreover, the analysis of rest perfusion using the DL approach was time consuming considering the post-processing of images.
Another technique able to identify a rest perfusion defect was developed by Van Hamersvelt et al. [29]. The authors developed a DL algorithm that was able to identify   TP  14  12  14  13  TN  6  16  13  15  FP  12  2  5  3  FN  0  2  0  1  Sens  100% (77-100%  the presence of myocardial ischemia assessed by invasive FFR from a rest CCTA. Similar to our article, the authors reported an increase in accuracy when the DL algorithm was combined with anatomical evaluation, AUC of 76% compared to 68% for anatomical evaluation alone [29]. Despite a robust gold standard represented by invasive FFR for the training of the DL algorithm, the authors did not evaluate the impact of their algorithm in a dataset of CTP stress. Furthermore, the analysis of perfusion using the DL approach required the segmentation of myocardium [29], while in our study, it was not performed. In our study, beyond the feasibility of a new DL approach, we showed that it is possible to provide better diagnostic accuracy compared to humans if the myocardium is analyzed with a DL technique in both rest and stress datasets. Furthermore, in our approach, it was possible to identify significant CAD with a fully automated technique. Importantly, the comparison between the human approach and the DL algorithm showed similar accuracy despite the time of analysis with the DL approach being < 1 min.
Following the ESC guidelines, we hypothesize that the application of CCTA will exponentially grow [3]. Therefore, the application of AI algorithms in clinical practice could speed up the time of acquisition, reporting, and interpretation of images. A fully automated approach would be feasible in clinical practice with the first step being the evaluation of coronary arteries as shown by Muscogiuri et al. [12] followed by evaluation of ischemia with a DL algorithm that could evaluate the myocardium. In particular, the evaluation of myocardium on rest CCTA is performed without the administration of a stressor agent allowing for a safe and cheap approach. Furthermore, the application of a DL algorithm that could identify myocardial ischemia on CCTA can potentially break down the costs of ischemia testing following CCTA. This algorithm could potentially run on a standard personal computer without any particular imaging skills.
Some limitations need to be addressed. First, the small sample size and the retrospective nature of the study represent a limitation of the study. The prevalence of CAD in the population examined is higher than expected in a normal population; furthermore, it is important to consider that a cut off value of stenosis > 70% in CCTA could potentially provide different results in terms of diagnostic accuracy for identification of hemodynamically significant CAD.
Second, we did not test the inter-reader variability; therefore, the diagnostic accuracy of CCTA and CCTA + CTP stress could be different based on the experience of the readers.
Thirdly, a prospective larger study is needed in order to evaluate the prognostic impact of a CCTA + CTP-DL rest approach.
In conclusion, evaluation of myocardial ischemia using a DL approach on rest CTP datasets is feasible and accurate. This approach may be a useful gatekeeper prior to further testing such as CTP stress.

Declarations
Informed consent Informed consent was obtained from all individual participants included in the study. Patients signed informed consent regarding publishing their data and photographs.

Conflict of interest
The authors declare no competing interests.