Real-Time Coronary Artery Stenosis Detection Based on Modern Neural Networks

doi:10.21203/rs.3.rs-130610/v1

Download PDF

Research Article

Real-Time Coronary Artery Stenosis Detection Based on Modern Neural Networks

https://doi.org/10.21203/rs.3.rs-130610/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 07 Apr, 2021

Read the published version in Scientific Reports →

You are reading this latest preprint version

Invasive coronary angiography remains the gold standard for diagnosing coronary artery disease, which may be complicated by both, patient-specific anatomy and image quality. Deep learning techniques aimed at detecting coronary artery stenoses may facilitate the diagnosis. However, previous studies have failed to achieve superior accuracy and performance for real-time labeling. Our study is aimed at confirming the feasibility of real-time coronary artery stenosis detection using deep learning methods. To reach this goal we trained and tested eight promising detectors based on different neural network architectures (MobileNet, ResNet-50, ResNet-101, Inception ResNet, NASNet) using clinical angiography data of 100 patients.

Three neural networks have demonstrated superior results. The network based on Faster-RCNN Inception ResNet V2 is the most accurate and it achieved the mean Average Precision of 0.954 and the prediction rate of 3 fps on the validation subset. The relatively lightweight SSD MobileNet V2 network proved itself as the fastest one with an mAP of 0.830 and a mean prediction rate of 38 fps. The model based on RFCN ResNet-101 V2 has demonstrated an optimal accuracy-to-speed ratio. Its mAP makes up 0.94, while the prediction speed is 10 fps.

The resultant performance-accuracy balance of the modern neural networks has confirmed the feasibility of real-time coronary artery stenosis detection supporting the decision-making process of the Heart Team interpreting coronary angiography findings.

Cardiac & Cardiovascular Systems

Applied Mathematics

stenosis detection

coronary artery disease

coronary angiography

deep learning

neural networks

Real-time coronary artery stenosis detection is feasible using modern neural networks
RFCN ResNet-101 V2 neural network has demonstrated the optimal accuracy/speed ratio.
The heaviest network Faster-RCNN Inception has demonstrated the most accurate detection.

Coronary artery disease (CAD) is the leading cause of death worldwide ¹, affecting over 120 million people ². A mismatch between myocardial oxygen supply and myocardial oxygen demand commonly results in ischemia. Chest pain is the most likely symptom that occurs during physical and/or emotional stress, relieved promptly with rest or by taking nitroglycerin. Atherosclerotic plaques, impeding coronary blood flow, are the most common cause of coronary artery disease ³. Despite novel imaging modalities (e.g. coronary CT angiography) have been developed, invasive coronary angiography is the preferred diagnostic tool to assess the extent and severity of complex coronary artery disease according to the 2019 guidelines of the European Society of Cardiology ^4,5. Multivessel coronary artery disease affecting two or more coronary arteries requires interpretive expertise on the assessment of multiple parameters (the number of affected major coronary arteries, the location of lesions, the severity of stenosis, the length of the stenotic segment, tortuosity, etc) during an intervention. The process of interpreting complex coronary vasculature, image noise, low contrast vessels, and non-uniform illumination is time-consuming ⁶, thereby posing certain challenges to the operator. Real-time automatic CAD detection and labeling may overcome the abovementioned difficulties by supporting the decision-making process.

A number of approaches for automatic or semi-automatic assessment of coronary artery diseases have been proposed by different research groups. These approaches follow the general scheme: (1) coronary artery tree extraction, (2) calculation of geometric dimensioning, and (3) analysis of the stenotic segment. The key stage that determines the speed and accuracy of such algorithms is based on the coronary artery tree extraction using the centerline extraction ^7,8; the graph-based method ^9–11; superpixel mapping ^12,13; and machine/deep learning ^14–16.

Recent studies commonly use the Dice Similarity Coefficient (some studies reported the Dice Similarity Coefficient of more than 0.75) ^11,12 and/or the Sensitivity metric (some studies reported the Sensitivity of more than 0.70) ¹⁷ to assess the quality of automatic CAD analysis. Image processing time is an important indicator for the applied use of these methods that can reach 1.1–11.87 seconds ⁹, 20 seconds ^9,12, and over 60 seconds ⁸. Taking into account the mean duration of angiography imaging series, usually consisting of 50–80 frames, the total processing time can become a significant factor, limiting the use of many methods. Slow data processing does not allow providing real-time support for the operator during the procedure and may be performed after diagnosis and data collection. Moreover, in some cases, the interventional cardiologist performs Percutaneous Coronary Intervention (PCI) immediately following by the diagnostic catheterization (ad-hoc PCI) ^18,19. In this regard, the decision on the location and severity of stenosis should be made quickly, especially in patients with the acute coronary syndrome.

Some researchers try to improve the performance of these algorithms by segmenting only large vessels of the coronary bed ²⁰. This approach allows achieving the inference time of 0.04 frames per second, but it does not take into account stenotic lesions in small branches. Another approach using convolutional neural networks to speed up the algorithm includes the extraction of individual regions of interest with stenotic sites without the entire coronary artery tree. A similar principle has been reported by Cong et al. ¹⁷ describing the Inception V3 neural network and Hong et al. ²¹ describing the M-net (improved version of U-net).

Our study presents a detailed analysis of available neural network architectures and their potential in terms of accuracy and performance to detect single-vessel disease. Some of the selected models will be modified and adapted for real-time detection and assessment of coronary artery stenosis.

Source Data

Initial angiographic imaging series of one hundred patients who underwent coronary angiography using Coroscop (Siemens) and Innova (GE Healthcare) at the Research Institute for Complex Problems of Cardiovascular Diseases (Kemerovo, Russia) were retrospectively enrolled in the study (Table 1). Hemodynamically significant stenoses over 70% were determined. Patients with multivessel CAD (two or more affected major coronary arteries) were excluded. The study design was approved by the Local Ethics Committee of the Research Institute for Complex Issues of Cardiovascular Diseases (approval letter No. 112 issued on May 11, 2018). All participants provided written informed consent to participate in the study. Coronary angiography was performed by the single operator according to the indications and recommendations stated in the 2018 ESC/EACTS Guidelines on myocardial revascularization. The presence or absence of coronary stenosis was confirmed by the same operator using angiography imaging series according to the 2018 ESC/EACTS Guidelines on myocardial revascularization.

Table 1

Clinical and demographic data of the study population
Parameter	Value
A total number of patients	100
Mean age ± SD, years	60.3 ± 13.8
Men, n (%)	68 (68%)
Women, n (%)	32 (32%)
Body mass index (kg/m²)	21.6 ± 5.1
Diagnosis	CAD
Class I NYHA	5 (5%)
Class II NYHA	84 (84%)
Class III NYHA	11 (11%)
Comorbidities
Arterial hypertension	53 (53%)
Diabetes mellitus	14 (14%)
Chronic heart failure, classes 1–2	36 (36%)
Coronary artery stenosis > 70% (n, %)	100 (100%)

Angiographic images of the radiopaque overlaid coronary arteries with stenotic segments were selected and converted into separate images. An interventional cardiologist rejected non-informative images and selected only those containing contrast passage through a stenotic vessel. A total of 8325 grayscale 1-channel images of 512 × 512 to 1000 × 1000 pixels were included for further study. Of them, 7492 (90%) images were used for training, and 833 (10%) images were used for validation. Data were labelled using the LabelBox, a free version of SaaS (Software as a Service). It allows joint data labelling and subsequent validation by several specialists. Typical data labelling of the source images is shown in Fig. 1.

To analyse the source dataset, we estimated the size of the stenotic region computing the area of the bounding box. Similarly to the Common Objects in Context (COCO) dataset, we divided objects by their area into three types: small (area < 32²), medium (32² ≤ area ≤ 96²), and large (area > 96²). 2509 small objects (30%), 5704 medium objects (69%), and 113 large objects (1%) were obtained in the input data. Since our data were unbalanced, we suppose that image analysis may be poorer on larger objects than on small and medium ones.

Figures 2 and 3 show the distributions of the absolute and relative stenotic areas. To generate the distribution of the absolute area, we estimated the absolute values of the bounding box stenotic areas in pixels. To generate the distribution of the relative area, we estimated the value of the area of the bounding box relative to the area of the entire image in percentages. The dashed lines represent the mean values and standard deviations of the area. Based on the input data, the absolute stenotic area was 1942 ± 1699 pixels (Fig. 2). Since the size of the images from the input dataset varied within a certain range of values, we calculated the relative stenotic area. We selected images with normalized X and Y coordinates in the range of values [0; 1]. As a result, the relative stenotic area was 0.34 ± 0.27% (Fig. 3). As seen, the stenotic area is quite small compared to the area of the whole image that may confuse some detectors typically applied to detect objects in an unconstrained environment.

To determine the location of stenosis accurately, we evaluated the distribution of the stenosis coordinates along the vessel in the input images. We estimated the normalized coordinates of the centre point of the bounding box around the stenotic lesion. Based on this assessment, a distribution map of the coordinates of the stenosis centres was generated and is shown in Fig. 4. Each hexagon on this map reflects a number of the stenosis centres of the bounding box around the stenotic lesion. Distributing the coordinates highlights two centres with relative coordinates (0.50; 0.20) and (0.27; 0.27) along the stenotic segment. The coordinates of the stenosis centres are evenly distributed without explicit outliers. The latter should have a positive effect on training regressors based on neural networks that predict the coordinates of the bounding boxes.

Models description

We applied machine learning algorithms to detect coronary artery stenosis on the coronary angiography imaging series. Machine learning has shown beneficial potential in computer vision and image processing. We used SSD ²², Faster-RCNN ²³, and RFCN ²⁴ object detectors from the Tensorflow Detection Model Zoo ²⁵ based on MobileNet ^26,27, ResNet ^28,29, Inception ResNet ³⁰ and NASNet ^31,32. We examined eight models with various architectures, network complexity, and a number of weights. The lightweight SSD MobileNet V1 and SSD MobileNet V2 enable real-time data processing. While Faster-RCNN Inception ResNet V2 and Faster-RCNN NASNet, with over 50 million weights, were the most complex models selected for the study. Table 2 shows a brief description of the models. Characteristics of neural networks, including mAP, are reported based on their training on the COCO dataset.

Table 2

– Brief characteristics of the use
Model	Inference time, ms	mAP@[0.5:0.95]	Weights, mln	Model size, Mb
SSD MobileNet V1	56	32	4.2	44
SSD MobileNet V2	31	22	6.1	19
SSD ResNet-50 V1	76	35	25.6	127
Faster-RCNN ResNet-50 V1	89	30	25.6	114
RFCN ResNet-101 V2	92	30	44.7	199
Faster-RCNN ResNet-101 V2	106	32	44.7	190
Faster-RCNN Inception ResNet V2	620	37	55.9	241
Faster-RCNN NASNet	540	-	88.9	416

Model training

When training neural network models, their base configuration is similar to that used to train on the COCO dataset. For the unambiguous comparison of the selected models, the total number of training steps was set to 100 equal to 100000 iterations of learning. Regarding the loss functions, Weighted Smooth L1 loss (see Eq. 3 in ³³) was the localization loss, and the Weighted Focal Loss was the classification loss ³⁴. The SSD-based models were trained using the cosine decay with the warm-up and exponential decay. When using these techniques, the learning rate gradually decreased depending on the learning step. It is also worth noting a distinctive feature of the SSD MobileNet V2 neural network, which is the use of the Hard Example Mining technique ^22,35. It allows getting additional samples of the negative class and then learns from them. Using additional samples often improves the accuracy of the stenosis location. To train models, we used P2 (Nvidia Tesla K80 12 Gb, 1.87 TFLOPS) and P3 instances (Nvidia Tesla V100 16 Gb, 7.8 TFLOPS) from Amazon Web Services. We also divided the models into 4 groups according to their complexity for further comparison. Table 3 summarises the main characteristics of the training.

Table 3

– Model training settings
Model	Input size	Augmentation	Batch size	Type of LR	LR
SSD MobileNet V1	640 × 640 × 3	Random horizontal flip Random crop image	4	Cosine decay with warm up	0.04
SSD MobileNet V2	300 × 300 × 3	Random horizontal flip SSD random crop	4	Exponential decay	0.004
SSD ResNet-50 V1	640 × 640 × 3	Random horizontal flip Random crop image	2	Cosine decay with warm up	0.04
Faster-RCNN ResNet-50 V1	600 × 600 × 3	Random horizontal flip	2	Constant LR	0.0003
RFCN ResNet-101 V2	600 × 600 × 3	Random horizontal flip	1	Constant LR	0.0003
Faster-RCNN ResNet-101 V2	600 × 600 × 3	Random horizontal flip	1	Constant LR	0.0003
Faster-RCNN Inception ResNet V2	600 × 600 × 3	Random horizontal flip	1	Constant LR	0.0003
Faster-RCNN NASNet	1200 × 1200 × 3	Random horizontal flip	1	Constant LR	0.0003

Serial changes in accuracy were obtained on the validation set during the training process. The mAP metric, as the metric of interest ³⁶, with a predefined threshold value for Intersection over Union equal to 0.5 was used. Figure 5 shows smooth changes in the mAP on the validation set during the training process. All models converge to a specific value of the asymptotic accuracy. SSD ResNet-50 V1 could achieve higher quality with longer training, but this would require more steps.

Comparative assessment

Table 4 presents the results of the comparative study of the neural networks. In addition to the absolute values of the metrics, the relative values are also reported. The metrics of SSD MobileNet V1 were used as a benchmark to compare with other models. Color scale formatting reflects the distribution of models by their accuracy, training and inference times, and a number of weights, where deep blue shows the best value, and white – the worst. Figures 6 and 7 show two basic metrics, mAP and the inference time for the prediction of the stenotic lesion bounding box on an image.

Table 4

– Comparative study of the selected models
Model	Weights, mln		Training time, hours		Inference time, ms		[email protected]
Model	Abs.	Rel.	Abs.	Rel.	Abs.	Rel.	Abs.	Rel.
SSD MobileNet V1	4.2	1.0x	16	1.0x	43	1.0x	0.69	1.00x
SSD MobileNet V2	6.1	1.4x	20	1.3x	26	0.6x	0.83	1.20x
SSD ResNet-50 V1	25.6	6.0x	47	3.0x	61	1.4x	0.76	1.09x
Faster-RCNN ResNet-50 V1	25.6	6.0x	28	1.8x	98	2.3x	0.92	1.33x
RFCN ResNet-101 V2	44.7	10.5x	55	3.6x	99	2.3x	0.94	1.36x
Faster-RCNN ResNet-101 V2	44.7	10.5x	55	3.5x	118	2.7x	0.94	1.35x
Faster-RCNN Inception ResNet V2	55.9	13.2x	93	6.0x	363	8.4x	0.95	1.38x
Faster-RCNN NASNet	88.9	21.0x	147	9.5x	880	20.4x	0.84	1.22x

The inference time was estimated using P3 instance (Nvidia Tesla V100 16 Gb, 7.8 TFLOPS) of Amazon Web Services. We concluded that the inference time directly depended on the complexity of the model and the total number of its weights. Thus, Faster-RCNN Inception ResNet v2 and Faster-RCNN NASNet were the slowest in predictions. Their mean processing times per one image were 363 and 880 milliseconds, respectively. While testing the lightweight models based on the MobileNet backbone, we found that MobileNet V2 with a larger number of weights (6.1 mln) demonstrated superior inference time than Mobile Net V1 (4.2 mln). In general, MobileNet V2 had the most superior inference time than other models. Thus, it may be used for predicting the location of stenosis in real-time.

Faster-RCNN Inception ResNet V2 was the most accurate model. The mean Average Precision of this model on the validation set was 0.95 with the inference time of 363 ms/image (≈ 3 frames per second). The fastest and relatively lightweight SSD MobileNet V2 had the mean Average Precision of 0.83 with an inference time of 26 ms/image (≈ 38 frames per second). Based on the obtained results, we concluded that RFCN ResNet-101 V2 is an optimal one to solve the set tasks. The mAP of this model is 0.94 and the inference time is 99 ms/image (≈ 10 frames per second).

Model testing

The capabilities of the selected neural networks are presented using the data of three patients with the referenced labelling (Fig. 8a-c). Detailed visualization for predictions is presented in Appendices E-F. The models with the best values of the loss function and mAP were used for testing. Table 5 reports the best steps with the model optimal weights. Such localization metrics as Intersection over Union (IoU) and Dice Similarity Coefficient (DSC) were also computed and shown.

Table 5

– Best steps with optimal model weights
Model	Best step
SSD MobileNet V1	24
SSD MobileNet V2	99
SSD ResNet-50 V1	100
Faster-RCNN ResNet-50 V1	84
RFCN ResNet-101 V2	97
Faster-RCNN ResNet-101 V2	94
Faster-RCNN Inception ResNet V2	83
Faster-RCNN NASNet	95

Almost all models may accurately detect the location of stenosis. However, we faced several false positives while testing the Faster-RCNN NASNet model. In all three cases, this model detected the location of false stenotic segments with a probability of more than 90% in the right coronary artery (Fig. 8d) and the anterior descending artery (Fig. 8e,f) besides the reference stenotic region. SSD MobileNet V1 and SSD ResNet-50 V1 models failed to detect the location of stenosis in patient 1. SSD MobileNet V2 model demonstrated one of the best results in predicting the location of stenosis (Fig. 9). Despite the DSC metric of 0.65 in patient 3, it had the highest DSC metric in patients 1 and 2 (0.93 and 0.98, respectively). Additionally, the detectors based on the ResNet architecture, Faster-RCNN ResNet-50 V1 and Faster-RCNN ResNet-101 V2, should be noted. The average DSC metric on the test data was 0.85 and 0.84, respectively.

The ultimate goal of our study is to develop a novel stenosis detection algorithm for patients with multivessel CAD, as they represent the most difficult group for diagnosis and interpretation. We believe that automatic detection and grading of multivessel CAD may facilitate the operator work by minimizing the risk of misinterpretation and accelerate the decision making regarding the proper treatment strategy. To date, the accuracy and certainty of interpreting coronary angiograms fully rely on the operator who needs to identify the location of the stenosis and describe individual coronary vasculature, including the diameter of the affected vessels, the length of the stenotic segments, the presence of any lateral branches, any shunts, tortuosity, etc. ³⁷. We have successfully tested our algorithm for detecting single-vessel CAD to assess its potential for the key task. Real-time detection of multivessel disease and its automatic grading is a more complex and multicomponent task. According to the obtained results, we concluded that the current version of our algorithm fully corresponds to the following key criteria — sufficient processing speed and detection accuracy.

Image processing speed

From the technical point of view, the speed of the algorithm for real-time detecting coronary artery stenosis and grading its severity is one of the key parameters empowering accurate CAD diagnosis and treatment. Coronary angiography is an invasive procedure that is associated with radiologic exposure, obviating repeated contrast injections, and limiting interventional cardiologists in their manipulations. In this respect, the ability to perform real-time detection of the stenotic lesions and their simultaneous grading in the cath-lab significantly increases the diagnostic efficiency (e.g. if the algorithm is sufficiently accurate, the operator may refuse additional contrast injection and proceed with stenting). Algorithms that generate predictions slowly (inference time of 600–800 s per angiography projection) are limited in use. They should be used separately, after coronary angiography, and may serve for off-line research descriptive tasks. Since the prolonged door-to-balloon time significantly affects the patient's outcome ¹⁹ and is directly associated with mortality ³⁸, the minimization of time spend on diagnosis will facilitate the decision-making process, especially for severe cases (e.g. myocardial infarction).

The existing research teams mainly focus on the accuracy of the algorithms rather than their speed. Most of them do not fit for routine medical image processing. Some of the recently reported image processing algorithms are generally perceived as slow with a high “cost” of frame analysis: Fang et al. reported the inference time varying from 1.1 to 11.87 sec. ⁹; M’Hiri et al. — 20 sec. ¹²; and Wan et al. — 63.3 sec. to build the skeleton of the artery and 70.9 sec. for the subsequent processing cycle ⁸. Other studies have demonstrated a faster data analysis, spending almost 1.8 seconds per each artery ¹⁶, and 32 ± 21 seconds per each stenotic segment ²¹. However, these algorithms use computed tomography imaging series, that are commonly obtained during routine preoperative management but not urgently. Therefore, they are spending much more time on the descriptive analysis, empowering the decision-making process. Yang et al. have recently reported the use of convolutional neural networks for segmenting major coronary arteries ²⁰. The algorithm spends 60 ms per each angiogram, but it does not predict stenotic lesions of other small vessels.

There are no strict requirements for the processing speed of the angiography imaging series. It depends mainly on individual application settings. Thus, algorithms developed to support diagnostic angiography, performed with the aim of subsequent emergent blood flow restoration, should correspond to the following requirements: input video frame rate of 7.5–15 frames per second ^39,40, the duration of the procedure less than 25 minutes, and individual preferences of the operator ⁴¹. We concluded that neural network architectures with an inference time of less than 66 milliseconds are suitable for this task (Table 4. SSD MobileNet V1, SSD MobileNet V2, and SSD ResNet-50 V1), as they process at least 15 frames per second. However, their performance was assessed on a relatively simple case requiring detecting the location of stenosis without calculating its quantitative parameters. Thus, we expect that a detailed analysis of multivessel CAD may require a much longer time. Neural network models with the inference time of 98–118 milliseconds per frame (Table 4. Faster-RCNN ResNet-50 V1, RFCN ResNet-101 V2, and Faster-RCNN ResNet-101 V2) may be assigned to the “grey zone”, processing 8–10 frames per second. Their resultant performance is insufficient, but they can be used in the cath-lab with the detection lag. The heavyweight models with the inference time of over 360 milliseconds per frame (Table 4. Faster-RCNN Inception ResNet V2 and Faster-RCNN NASNet), do not fully correspond to the needs of the real-time angiography analysis, as they will fail to provide adequate productivity in complex cases.

Accuracy

Detection accuracy is another important parameter indicative to the quality of the algorithm, particularly for borderline cases, when the treatment strategy is not clearly defined and false positives may mislead the Heart Team to choose a more invasive treatment option ¹⁹. Therefore, it seems necessary to discuss these two cases separately — false positives and false negatives in the detection of stenosis. A false positive is an error in data reporting when an algorithm detects incorrectly the presence of stenosis. It may result in choosing coronary artery bypass grafting (CABG) rather than PCI since the operator relies on the misinterpreted data regarding the multiple stenotic lesions that increase individual SYNTAX Score ^19,42. Thus, we should take seriously false positives produced by the Faster-RCNN NASNet network, that misinterpreted the clinical states of three control patients (Fig. 8d-f). Alternatively, a false negative is an error in data reporting when an algorithm reports the absence of the existing stenosis. However, false negatives are less serious than false positives, as they can be leveled out during stenting by repeated contrast injection that will visualize the missed stenosis. This type of error was encountered for the two selected neural networks, the lightweight SSD MobileNet V1 and SSD ResNet-50 V1. Both these models showed the worst mAP of 0.69 and 0.76, respectively. Other models demonstrated acceptable accuracy with the mAP of 0.94–0.95. The Dice Similarity Coefficient varied from 0.64 to 0.93 on the validation set. Our data are generally consistent with the previously reported studies (the Sensitivity metric varying from 0.59 to 0.72 in ¹⁷, the Dice Similarity Coefficient of 0.75 in ¹² and 0.74 to 0.79 in ¹¹.

Both, the accuracy and the number of errors, may be improved using traditional approaches, including an increase of the training set size and its heterogeneity in addition to the use of more scalable and efficient neural network architectures (e.g. EfficientDet or СenterNet detectors ^43,44). However, as the complexity of the neural network grows, the speed of the detection algorithm will inevitably drop, resulting in its inability to operate in real-time settings. Therefore, the accuracy-to-speed ratio, being a key parameter of algorithms developed for detecting CAD, should be maintained. From this point of view, Faster-RCNN ResNet-50 V1, RFCN ResNet-101 V2, Faster-RCNN ResNet-101 V2 with the inference time of 98–118 milliseconds per frame (grey zone) and with an accuracy of 0.92–0.94 mAP, are of beneficial potential for detecting the location coronary artery stenosis and grading it.

The imbalance between accuracy and computer performance has been previously limited to the introduction of an automatic CAD detection algorithm in clinical practice. We have demonstrated that the development of hardware performance and appearance of the recent neural network architectures may significantly reduce the labor-intensive process during conventional invasive coronary angiography. We trained eight promising detectors based on different neural network architectures (MobileNet, ResNet-50, ResNet-101, Inception ResNet, NASNet) to detect the location of stenotic lesions using angiography imaging series and assessed their performance. Out of them, three neural networks have demonstrated superior results. Faster-RCNN Inception ResNet V2 is the most accurate to detect single-vessel disease. It demonstrates the mean Average Precision of 0.954, and the prediction rate of 363 ms per image (≈ 3 frames per second) on the validation set. The relatively lightweight SSD MobileNet V2 model is the fastest with an mAP of 0.830 and a mean prediction rate of 26 ms per image (≈ 38 frames per second). RFCN ResNet-101 V2 has demonstrated an optimal accuracy-to-speed ratio. Its mAP is 0.94, and the prediction speed is 99 ms per image (≈ 10 frames per second). The resultant performance-accuracy balance using the described neural networks has confirmed the feasibility of real-time CAD tracking supporting the decision-making process of the Heart Team. Real-time automatic labeling has opened new horizons for diagnosis and treatment of complex coronary artery disease.

Author contributions

V.D., K.K., and E.O. conceived the idea of the study. V.D., O.G., and A.F. contributed to the methodology used in the field of stenosis detection; K.K. and E.O. collected the data. V.D. prepared the software and algorithms for data analysis. V.D., A.K., V.G., and E.O. performed an analysis of the current state of research in the field of detection and quantitative assessment of stenosis; V.D. and A.F. trained deep learning models using the Multi-X platform. V.D. and K.K. wrote the manuscript. A.K., V.G., O.G., A.F., and E.O. reviewed and edited the manuscript. K.K. and E.O. contributed critical discussions and revisions of the manuscript. A.F. and E.O. were supervising and administering the project.

Competing interests

The authors have no competing interests as defined by Nature Research, or other interests that might be perceived to influence the results and/or discussion reported in this paper

Additional information

Supplementary Information: The online version contains supplementary material available at https://doi.org/abc/xyz.

Correspondence and requests for materials should be addressed to V.D.

Reprints and permissions information is available at www.nature.com/reprints.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

GBD 2017 Causes of Death Collaborators. Global, regional, and national age-sex-specific mortality for 282 causes of death in 195 countries and territories, 1980–2017: a systematic analysis for the Global Burden of Disease Study 2017.Lancet (London, England)392,1736–1788(2018).
Virani, S. S. et al. Heart Disease and Stroke Statistics-2020 Update: A Report From the American Heart Association. Circulation. 141, e139–e596 (2020).
Dababneh, E. & Goldstein, S. Chronic Ischemic Heart Disease Selection of Treatment Modality. in (2020).
Saraste, A. & Knuuti, J. ESC 2019 guidelines for the diagnosis and management of chronic coronary syndromes: Recommendations for cardiovascular imaging. Herz vol. 45 409–420 (2020).
Collet, C. et al. Coronary computed tomography angiography for heart team decision-making in multivessel coronary artery disease. Eur. Heart J. 39, 3689–3698 (2018).
Janssen, J. P. et al. New approaches for the assessment of vessel sizes in quantitative (cardio-)vascular X-ray analysis. Int. J. Cardiovasc. Imaging. 26, 259–271 (2010).
Gao, Y. & Sundar, H. Coronary arteries motion modeling on 2D X-ray images. in Medical Imaging 2012: Image-Guided Procedures, Robotic Interventions, and Modeling (eds. Holmes III, D. R. & Wong, K. H.) vol. 8316 83161A (SPIE, 2012).
Wan, T., Feng, H., Tong, C., Li, D. & Qin, Z. Automated identification and grading of coronary artery stenoses with X-ray angiography. Comput. Methods Programs Biomed. 167, 13–22 (2018).
Fang, H. et al. Greedy Soft Matching for Vascular Tracking of Coronary Angiographic Image Sequences. IEEE Trans. Circuits Syst. Video Technol. https://doi.org/10.1109/TCSVT.2019.2903883 (2020).
M’Hiri, F., Duong, L., Desrosiers, C. & Cheriet, M. Coronary arteries segmentation using random walks and Hessian-based vesselness filter. in Proceedings - International Symposium on Biomedical Imaging 918–921 (2013). doi:10.1109/ISBI.2013.6556625.
M’Hiri, F., Le, T. H. N., Duong, L., Desrosiers, C. & Cherief, M. Hierarchical segmentation and tracking of coronary arteries in 2D X-ray Angiography sequences. in Proceedings - International Conference on Image Processing, ICIP vols 2015-December 1707–1711 (IEEE Computer Society, 2015).
M’Hiri, F. et al. A graph-based approach for spatio-temporal segmentation of coronary arteries in X-ray angiographic sequences. Comput. Biol. Med. 79, 45–58 (2016).
Ren, X. & Malik, J. Learning a classification model for segmentation. in Proceedings of the IEEE International Conference on Computer Vision vol. 1 10–17 (Institute of Electrical and Electronics Engineers Inc., 2003).
Jo, K., Kweon, J., Kim, Y. H. & Choi, J. Segmentation of the Main Vessel of the Left Anterior Descending Artery Using Selective Feature Mapping in Coronary Angiography. IEEE Access. 7, 919–930 (2019).
Nasr-Esfahani, E. et al. Segmentation of vessels in angiograms using convolutional neural networks. Biomed. Signal Process. Control. 40, 240–251 (2018).
Zreik, M. et al. A Recurrent CNN for Automatic Detection and Classification of Coronary Artery Plaque and Stenosis in Coronary CT Angiography. IEEE Trans. Med. Imaging. 38, 1588–1598 (2019).
Cong, C., Kato, Y., Vasconcellos, H. D., Lima, J. & Venkatesh, B. Automated Stenosis Detection and Classification in X-ray Angiography Using Deep Neural Network. in Proceedings – 2019 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2019 (2019). doi:10.1109/BIBM47256.2019.8983033.
Rahman, Z., Paul, G. K. & Choudhury, A. K. Ad-hoc percutaneous coronary intervention and staged percutaneous coronary intervention. Mymensingh Med. J. 20, 757–765 (2011).
Sousa-Uva, M. et al. 2018 ESC/EACTS Guidelines on myocardial revascularization. Eur. J. cardio-thoracic Surg. Off. J. Eur. Assoc. Cardio-thoracic Surg. 55, 4–90 (2019).
Yang, S. et al. Deep learning segmentation of major vessels in X-ray coronary angiography. Sci. Rep. 9, 1–11 (2019).
Hong, Y. et al. Deep learning-based stenosis quantification from coronary CT angiography. in Proceedings of SPIE-the International Society for Optical Engineering vol. 10949 88 (SPIE-Intl Soc Optical Eng, 2019).
Liu, W. et al. SSD: Single Shot MultiBox Detector. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) vol. 9905 LNCS 21–37 (Springer Verlag, 2016).
Ren, S., He, K., Girshick, R., Sun, J. & Faster, R. C. N. N. Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans. Pattern Anal. Mach. Intell. 39, 1137–1149 (2017).
Dai, J., Li, Y., He, K. & Sun, J. R. F. C. N. Object detection via region-based fully convolutional networks. inAdvances in Neural Information Processing Systems379–387(Neural information processing systems foundation, 2016).
Huang, J. et al. Speed/accuracy trade-offs for modern convolutional object detectors. in Proceedings – 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 vols 2017-Janua 3296–3305 (Institute of Electrical and Electronics Engineers Inc., 2017).
Howard, A. G. et al. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications (2017).
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A. & Chen, L. C. MobileNetV2: Inverted Residuals and Linear Bottlenecks. in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 4510–4520 (IEEE, 2018). doi:10.1109/CVPR.2018.00474.
He, K., Zhang, X., Ren, S. & Sun, J. Identity Mappings in Deep Residual Networks. Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics). 9908 LNCS, 630–645 (2016).
He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition vols 2016-Decem 770–778 (IEEE Computer Society, 2016).
Szegedy, C., Ioffe, S., Vanhoucke, V. & Alemi, A. A. Inception-v4, inception-ResNet and the impact of residual connections on learning. in 31st AAAI Conference on Artificial Intelligence, AAAI 2017 4278–4284 (AAAI press, 2017).
Zoph, B. & Le, Q. V. Neural Architecture Search with Reinforcement Learning. 5th Int. Conf. Learn. Represent. ICLR 2017 - Conf. Track Proc. (2016).
Zoph, B., Vasudevan, V., Shlens, J. & Le, Q. V. Learning Transferable Architectures for Scalable Image Recognition. in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 8697–8710 (IEEE, 2018). doi:10.1109/CVPR.2018.00907.
Girshick, R. & Fast, R. C. N. N. in 2015 IEEE International Conference on Computer Vision (ICCV) 1440–1448 (IEEE, 2015). doi:10.1109/ICCV.2015.169.
Lin, T. Y., Goyal, P., Girshick, R., He, K. & Dollar, P. Focal Loss for Dense Object Detection. in 2017 IEEE International Conference on Computer Vision (ICCV) vol. 42 2999–3007 (IEEE, 2017).
Shrivastava, A., Gupta, A. & Girshick, R. Training region-based object detectors with online hard example mining. in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition vols 2016-Decem 761–769 (IEEE Computer Society, 2016).
Henderson, P. & Ferrari, V. End-to-end training of object class detectors for mean average precision. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) vol. 10115 LNCS 198–213 (Springer Verlag, 2017).
Farooq, V., Brugaletta, S. & Serruys, P. W. The SYNTAX score and SYNTAX-based clinical risk scores. Semin. Thorac. Cardiovasc. Surg. 23, 99–105 (2011).
De Luca, G., Suryapranata, H., Ottervanger, J. P. & Antman, E. M. Time Delay to Treatment and Mortality in Primary Angioplasty for Acute Myocardial Infarction: Every Minute of Delay Counts. Circulation. https://doi.org/10.1161/01.CIR.0000121424.76486.20 (2004).
Abdelaal, E. et al. Effectiveness of low rate fluoroscopy at reducing operator and patient radiation dose during transradial coronary angiography and interventions. JACC Cardiovasc. Interv. 7, 567–574 (2014).
Badawy, M. K. et al. Feasibility of using ultra-low pulse rate fluoroscopy during routine diagnostic coronary angiography. J. Med. Radiat. Sci. 65, 252–258 (2018).
Ferreira-Neto, A. N. et al. Clinical and Technical Characteristics of Coronary Angiography and Percutaneous Coronary Interventions Performed before and after Transcatheter Aortic Valve Replacement with a Balloon-Expandable Valve. J. Interv. Cardiol. 2019, 3579671 (2019).
Capodanno, D. et al. Usefulness of SYNTAX score to select patients with left main coronary artery disease to be treated with coronary artery bypass graft. JACC. Cardiovasc. Interv. 2, 731–738 (2009).
Tan, M., Pang, R. & Le, Q. V.Scalable and Efficient Object Detection. (2019).
Duan, K. et al. CenterNet:Keypoint Triplets for Object Detection. (2019).

Appendices.docx

Download PDF

Journal Publication

published 07 Apr, 2021

Read the published version in Scientific Reports →

Editorial decision: Major revision
09 Feb, 2021
Reviews received at journal
12 Jan, 2021
Reviewers agreed at journal
09 Jan, 2021
Reviewers agreed at journal
09 Jan, 2021
Reviewers invited by journal
09 Jan, 2021
Editor assigned by journal
06 Jan, 2021
Editor invited by journal
28 Dec, 2020
Submission checks completed at journal
28 Dec, 2020
First submitted to journal
17 Dec, 2020

You are reading this latest preprint version

Real-Time Coronary Artery Stenosis Detection Based on Modern Neural Networks

Status:

Journal Publication

Version 1

Abstract

Figures

Research Highlights

Introduction

Source Data

Methods

Models description

Model training

Results

Comparative assessment

Model testing

Discussion

Image processing speed

Accuracy

Conclusion

Declarations

References

Supplementary Files

Status:

Journal Publication

Version 1