Deep Learning Radiomics Nomogram Based on Multiphase Computed Tomography for Predicting Axillary Lymph Node Metastasis in Breast Cancer

Abstract

Purpose

To develop and validate a deep learning radiomics nomogram (DLRN) for prediction of axillary lymph node metastasis (ALNM) in breast cancer (BC) patients.

Materials and Methods

We retrospectively enrolled 196 patients with non-specific invasive breast cancer confirmed by pathology, radiomics and deep learning features were extracted from unenhanced and biphasic (arterial and venous phase) contrast-enhanced computed tomography (CECT), and the non-linear support vector machine (SVM) was used to construct the radiomics signature and the deep learning signature respectively. Next, a DLRN was developed with independent predictors and evaluated the performance of models in terms of discrimination and clinical utility.

Results

Multivariate logistic regression analysis showed that the radiomics signature, deep learning signature, and clinical n stage were independent predictors. The DLRN accurately predicted ALNM, yielded an area under the receiver operator characteristic curve of 0.893(95% confidence interval, 0.814–0.972) in the validation set, with good calibration. Decision curve analysis confirmed that the DLRN had higher clinical utility than other predictors.

Conclusions

The DLRN had good predictive value for ALNM in breast cancer patients and provide valuable information for individual treatment.

1 Introduction

Breast cancer (BC) has become the second leading cause of cancer death in women of the world, accounting for more than 30% of all cancers in women[1, 2]. BC is often accompanied by axillary lymph node metastasis (ALNM) and involvement. Preoperative identification of axillary lymph node (ALN) status can provide an important role for treatment selection strategy and prognosis evaluation[3, 4]. With the development of BC surgery towards less invasive local precision treatment, sentinel lymph node biopsy (SLNB) has become the gold standard for identifying ALNM[5]. Literatures suggest that SLNB can provide good regional control in patients with BC[6, 7]. Although SLNB has fewer limitations than axillary lymph node dissection (ALND), SLNB is an invasive method and carries a risk of certain complications, such as upper extremity lymphedema, shoulder dysfunction and loss of shoulder muscle strength[8]. Therefore, it is crucial to evaluate preoperative ALN status in patients with BC, which may provide the best adjuvant treatment strategy preoperatively to extend the survival of patients.

At present, non-invasive imaging modalities such as ultrasonography (US), contrast-enhanced computed tomography (CECT), magnetic resonance imaging (MRI), and positron emission tomography (PET) were used for ALNM assessment. The diagnostic performance of these imaging modalities is insufficiently sensitive in detecting ALNM, which is limited by false negatives [9, 10]. Preoperative clinicopathological feature is also widely applied in predicting ALNM, but the prediction performance of clinicopathological features alone is insufficient [11]. Meanwhile, BC biomarkers are generally used to predict prognosis, and the predicting effect of lymph node metastasis is unknown [12]. Therefore, it is difficult to accurately predict ALNM using traditional radiological techniques and clinical features.

In recent years, with the gradual integration of artificial intelligence (AI) into medical development, radiomics-based research has attracted increasing attention[13]. At present, the research of CT-based radiomics in ALNM mainly focuses on the construction and evaluation of prediction models. However, there is a lack of attention to multiphase. Previous studies have indicated that fusion modeling of radiomics features, deep learning features and clinicopathological features could efficiently solve clinical problems [14, 15]. For BC patients, space-occupying lesions are mainly identified in the unenhanced phase, and organic lesions are mainly identified in the biphasic contrast-enhanced phases. Each phase contains valuable quantitative features, and it is of great clinical significance to establish a multiphase prediction model.

In this study, based on multiphase CT images, a DLRN was developed to achieve the purpose of efficiently predicting ALNM.

2 Materials And Methods

2.1 Study Population

This retrospective study was approved by the Institutional Review Board of the Southwestern Medical University Hospital (No. KY2022216), and the requirement for written informed consent was waived. All procedures were conducted in accordance with the 1975 Declaration of Helsinki and its later amendments[16].This study retrospectively analyzed patients who underwent pathologically con-firmed non-specific invasive BC at the Affiliated Hospital of Southwest Medical University (Luzhou, China) from February 2021 and February 2022. The entire patient records of this study were obtained from the Institutional Picture Archiving and Communication System (PACS, Carestream, Canada). All patients were desensitized to protect personal privacy and met the following inclusion and exclusion criteria.

Inclusion criteria: 1. pathologically diagnosed as non-specific invasive BC. 2.Received SLNB/ALND. 3. All patients performed unenhanced and biphasic (arterial and venous phase) contrast-enhanced CT before SLNB/ALND.

Exclusion criteria: 1. CT images were incomplete, or image quality was poor; 2. Patients were suffering from other tumors. 3. Incomplete clinicopathological data. 4. preoperative therapy (radiotherapy, chemotherapy, or other treatments).

A total of 780 patients with non-specific invasive BC were enrolled in the study, we excluded 584 patients, such as lack of complete three phase images (n = 539), clinicopathological incomplete data (n = 45). Finally, 196 patients met the criteria for inclusion and exclusion, including 97 with ALNM and 99 with non-ALNM. Patients were assessed for clinical node stage (cNx) by imaging and clinical examination. The primary outcome of this study was pathological node stage (pNx), which was determined based on SLNB or ALND. All patients were divided into training set (n = 136) and validation set (n = 60) in chronological order, patients who underwent CECT examination before November 31, 2021, were included in the training set, and the remaining patients were included in the validation set. The patient recruitment and workflow are shown in Fig. 1.

2.2 CECT Image Acquisition

All patients preoperatively received both unenhanced and biphasic (arterial and venous phase) contrast-enhanced chest CT examination (Netherlands, Philips Medical Systems). The scanning method was as follows: the contrast agent (Iohexol, 320mg/mL) was injected using a two-barrel high-pressure syringe through the median cubital vein of the patients (the dosage was 1.0 mL/kg, and the flow rate was 3.0 mL/s). The CT value of blood vessels at the level of the aortic arch was monitored after injection of contrast agent. Enhanced CT scans are automatically triggered when the CT value reaches around 250 HU.and venous phase scans were performed after a delay of 30 s. The scanning range extended from the level of the lower neck to the bottom of the thorax in the supine position.

2.3 Region of interest (ROI) Segmentation

The non-enhanced and biphasic (arterial and venous phase) contrast-enhanced images of study patients were obtained from Digital Imaging and Communications in Medicine (DICOM). For each CT phase, only one slice with the largest tumor area was chosen visually by the radiologist, The regions of interest (ROI) segmentation was performed from the original CT images using ITK-SNAP software (version 3.8.0, open source software; http://www.itksnap.org)[17]. All manual segmentation were performed by 2 practicing experienced radiologists (Y.W., with 11 years of CT imaging experience; Y.T., with 2 years of CECT imaging experience), followed by reader 1 (Y.W.) and reader 2 (Y.T.) repeated co-determination. They selected the mediastinal window (window width: 350, window level: 40) in ITK-SNAP to maintain approximately 1–2 mm from the tumor margin and delineate ROI along the tumor margin. Both radiologists were blinded to the patient's clinical data when they evaluated the multiphase CT images. Any discrepancies that arose were discussed to resolve the differences until they reached consensus. Then, 50 patients in the training set were randomly selected, and their ROIs were segmented again by reader 1 and reader 2 for the assessment of inter-reader reproducibility of radiomic features.

2.4 Features Extraction

The radiomics features and deep learning features were extracted from multiphase CT images of patients with BC, respectively. We used an open-source python-based package Pyradiomics (https://pyradiomics.readthedocs.io/en/latest/) for image preprocessing and radiomics feature extraction of ROI images[18]. To make features reproducible, an interclass correlation coefficient (ICC) higher than 0.75 was considered credible.

Resnet 50 was used to extract deep learning features, The cropped single ROI area was copied into three images to fit the RGB three channels of the model, the input area was 224x224 pixels, and the transfer learning took the pretrained weights of Resnet 50 on the ImageNet dataset as the initial weights of the model, The model was fine-tuned using our data. Resnet50 was adjusted from the original multi-classification task to a binary classification task, we extracted the deep learning features from the last layer of resnet50, and the PAC algorithm further compressed the deep learning features.

We normalized all variables, correlations between features were measured using Spearman algorithm, If the correlation coefficient between the two features was greater than 0.9, one of them was excluded. The least absolute shrinkage and selection operator (LASSO) algorithm with 10-fold cross-validation was utilized to select the optimal features.

2.5 Development and Validation of DLRN

RBF-SVM was used to develop radiomics signature and deep learning signature. DLRN was also developed by adding Clinical independent predictor features on the basis of radiomics signature and deep learning signature. Receiver operating characteristic (ROC) curves were used to calculate the area under the curve (AUC) with 95% confidence interval (CI), sensitivity (SEN), specificity (SPE), accuracy (ACC), positive predictive value (PPV) and negative predictive value (NPV) to evaluate the performance of these models. We used the Hosmer-lemeshow goodness-of-fit test for DLRN and draw the calibration curve[15]. The Decision curve analysis (DCA) of all models was performed to quantify the net benefit of patients under different threshold probabilities in the cohort to assess the clinical value of the predictive models in our study [20].

2.6 Statistical Analysis

All statistical works in our study were conducted in R studio (version 4.1.1). Univariate analysis was used to select statistically significant clinical characteristics (p < 0.05). The independent t-tests or Wilcoxon rank sum tests was used to compare the continuous data, chi-square cross-tabulation test or Fisher’s exact test were used to compare the differences of categorical variables. The Delong test was used to compare the AUC between the models. Hosmer-lemeshow performs a goodness-of-fit test for DLRN, p > 0.05 indicates that model has excellent fitting effect.

3 Results

3.1 Demographic and Clinicopathological Characteristics 

Note: Data in parentheses were percentages, *p values were derived from the univariate analysis between each of characteristic and ALN status.

The baseline characteristics of all patients are summarized in Table 1. A total of 196 patients with BC were enrolled in this study. All patients were divided randomly into training set (n = 136, 70%) and validation set (n = 60, 30%). 99 patients were diagnosed with ALNM, the remaining 97 patients were diagnosed with no ALNM. The findings on clinical t stage, clinical n stage was significantly correlated with the ALN status in both the training and validation set (p < 0.05).

3.2 Radiomics signature and Deep Learning signature Construction

The inter-observer reproducibility of the feature extraction was excellent, with inter-observer ICCs ranging from 0.758 to 0.974 for unenhanced phase, 0.786 to 0.942 for arterial phase and 0.804 to 0.946 for venous phase. A total of 1218 radiomics features were extracted from each ROI per phase, totaling 3654 features from the three ROIs per patient, spearman correlation coefficient analysis was carried out to exclude redundant features, 1018 radiomics features could be obtained for each patient.

Then, Resnet50 was adopted to extract the deep learning features of CECT images, which can extract 2048 features from each ROI per phase, totaling 6144 features from the three ROIs per patient. We used the principal components analysis (PCA) algorithm to perform Data dimensionality reduction for deep learning feature. Each ROI selects 32 features per phase, and each patient could obtain 96 deep learning features after dimensionality reduction. To validated deep learning features, heatmaps were generated using the Gradient weighted Class Activation Mapping (Grad-Cam) method (Fig. 2). Heatmaps indicated that the position of the feature on which the prediction was based in the image and used the color to represent the important feature. Blue area in the heatmap showed the more important features in this region.

20 radiomics optimal features, 6 deep learning optimal features were selected to construct two distinct single-scale prediction models by Support Vector Machine (SVM) method, respectively. The optimal regularization parameter C and Gamma (γ) for Gaussian Radial Basis Function (RBF) kernel were determined by cross validation and grid search. Each prediction model generated a prediction signature for each patient, named Radiomics signature, Deep Learning signature. The details of LASSO feature selection are shown in Fig. 3. The Raincloud plot shows the distribution position and interval density of signatures (Fig. 4).

4 Discussion

In this study, based on the evaluation criteria of discriminating ability and clinical practicability, we found that the DLRN model that integrated clinicopathologic factors, radiomics signature and deep learning signature had better predictive performance than the single-feature model. The DRLN model yielded satisfactory predictions on the validation set, with an area under AUC of 0.893, a sensitivity of 85.29%, and a specificity of 80.76%. Our results demonstrate the feasibility of using multi-phase features to predict whether breast cancer will metastasize, which significantly improves the current predictive methods for implementing individual subjective judgments using radiomics imaging alone.

Some studies had shown that the ALN status of patients with BC is closely related to the treatment strategy. The NCCN guidelines mentioned that the adjuvant systemic therapy for patients with early BC was beneficial to reduce the risk of recurrence[21]. For patients with positive clinical ALN status, the systemic chemotherapy was performed based on adjuvant endocrine therapy. For patients with negative clinical ALN status, endocrine therapy can be used alone. A recent Austrian study demonstrated that patients with positive ALN status had a worse prognosis than patients with negative ALN status[22]. Therefore, for patients with early BC, doctors can accurately and clearly distinguish the ALN status, which has strong clinical significance for BC treatment.

Many scholars had applied Ultrasound, MRI, and PET-CT images to carry out radiomics studies[23–25], involving radiomics and clinical features to lymph node metastasis of breast cancer. In particular, LNM radiomics studies of tumors (pancreatic ductal adenocarcinoma, gastric cancer, cervical cancer, etc.) based on CT images had achieved encouraging results[26–28].The proposed models were validated and achieved good performance of accuracy, specificity and sensitivity, which indicated combine models that integrate radiomics and clinical features together were developed, which had better predictive performance.

Radiomics could mine information in patients with BC, this method mainly relied on manual delineation of ROI regions, which was prone to subjective individual differences, and qualitative analysis might be biased. Recently, deep learning had made rapid progress in medical imaging, which had attracted the attention of researchers. Deep learning could achieve automatic learning and extract features that match the target task. Several studies had demonstrated that deep learning-based radiomics analysis applied to establish a direct link between medical image diagnosis and disease prediction, and the deep learning models achieved good results[14, 15, 28]. However, this method was limited to the number of data samples. The combination of radiomics features and deep learning features can comprehensively utilize the advantages of both to effectively solve this problem. Several studies had demonstrated that radiomics models and deep learning models based on CECT images had good predictive value for LNM or SLNM in breast cancer[29–31]. These models showed good discrimination of LNM numbers in internal and external validation cohorts, respectively, which provided baseline information for individualized treatment of breast cancer.

In this study, based on three-phase CT images, we respectively extracted radiomics features and deep learning features of ROIs in different phases to diversify the quantitative features of patients as much as possible. We developed a DRLN that is more accurate and reproducible for quantitative assessment of breast cancer imaging information than qualitative imaging diagnosis. Of the 21 (10.7%/196) patients who were misdiagnosed in clinical CT diagnosis, 13 patients (61.9%/21) were predicted to be correctly classified using DLRN. DLRN can quickly and easily predict the risk of ALNM in patients with BC and provide clinical decision-making strategies and individualized precision treatment plans to improve the effectiveness of treatment.

Our research is innovative, but there are also some limitations. Firstly, this study is a retrospective study. The sample data are all from a single medical institution. The external generalization and robustness of the model are lack of evaluation. The large sample study with different patient sets of multiple medical institutions is worthy of further research. Secondly, we only preliminarily explore the research of multi feature fusion model. The samples are relatively small and prone to underfitting, and larger sample sizes are required for machine learning training and deep learning tuning. Thirdly, ROI segmentation of tumor is not automatic, and the probability of error in artificial semi-automatic segmentation is large and difficult to find. This may be overcome by automatic segmentation artificial intelligence system in the future.

5 Conclusion

DLRN prediction model developed in this study based on radiomics features and deep learning features of three-phase CT images, and clinicopathological features provide a highly accurate, non-invasive, and convenient method for preoperative ALNM prediction in patients with BC. Therefore, this model is helpful for individual prediction of patients with BC, which could assist doctors to perform preoperative assessment and make better clinical decisions. Future research will construct more larger databases of BC with patient, integrate more clinical factors, further improve the predictive model, and carry out multi-center collaborative research.

Declarations

Ethical Approval and Consent to participate

The study was conducted in accordance with the Declaration of Helsinki and approved by the Ethics Committee of the Affiliated Hospital of Southwest Medical University (protocol code KY2022216, 20 June 2022).

Consent for publication

Not applicable.

Availability of supporting data

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Competing interests

The authors declare no conflict of interest.

Funding

This research was funded by Sichuan Science and Technology Program (Grant No. 2020YJ0151,2021YFQ0002).

Authors' contributions

Study design: JQ, XP.

Data analysis: JQ.

Data collection: YW, ZY.

Drafting the manuscript: JQ, XP.

Supervision of the manuscript: All authors.

References

1. Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, et al. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin. 2021;71(3):209–49.
2. Siegel RL, Miller KD, Fuchs HE, Jemal A. Cancer Statistics. 2021. CA Cancer J Clin. 2021;71(1):7–33.
3. Fillon M. Breast cancer recurrence risk can remain for 10 to 32 years. CA Cancer J Clin. 2022;72(3):197–9.
4. Giuliano AE, Connolly JL, Edge SB, Mittendorf EA, Rugo HS, Solin LJ, et al. Breast Cancer-Major changes in the American Joint Committee on Cancer eighth edition cancer staging manual. CA Cancer J Clin. 2017;67(4):290–303.
5. Noguchi M, Inokuchi M, Noguchi M, Morioka E, Ohno Y, Kurita T. Axillary surgery for breast cancer: past, present, and future. Breast Cancer. 2021;28(1):9–15.
6. Giuliano AE, Ballman K, McCall L, Beitsch P, Whitworth PW, Blumencranz P, et al. Locoregional Recurrence After Sentinel Lymph Node Dissection With or Without Axillary Dissection in Patients With Sentinel Lymph Node Metastases: Long-term Follow-up From the American College of Surgeons Oncology Group (Alliance) ACOSOG Z0011 Randomized Trial. Ann Surg. 2016;264(3):413–20.
7. Giuliano AE, Ballman KV, McCall L, Beitsch PD, Brennan MB, Kelemen PR, et al. Effect of Axillary Dissection vs No Axillary Dissection on 10-Year Overall Survival Among Women With Invasive Breast Cancer and Sentinel Node Metastasis: The ACOSOG Z0011 (Alliance) Randomized Clinical Trial. JAMA. 2017;318(10):918–26.
8. Belmonte R, Messaggi-Sartor M, Ferrer M, Pont A, Escalada F. Prospective study of shoulder strength, shoulder range of motion, and lymphedema in breast cancer patients from pre-surgery to 5 years after ALND or SLNB. Support Care Cancer. 2018;26(9):3277–87.
9. Zhang X, Liu Y, Luo H, Zhang J. PET/CT and MRI for Identifying Axillary Lymph Node Metastases in Breast Cancer Patients: Systematic Review and Meta-Analysis. J Magn Reson Imaging. 2020;52(6):1840–51.
10. Samiei S, de Mooij CM, Lobbes MBI, Keymeulen K, van Nijnatten TJA, Smidt ML. Diagnostic Performance of Noninvasive Imaging for Assessment of Axillary Response After Neoadjuvant Systemic Therapy in Clinically Node-positive Breast Cancer: A Systematic Review and Meta-analysis. Ann Surg. 2021;273(4):694–700.
11. Gan L, Ma M, Liu Y, Liu Q, Xin L, Cheng Y, et al. A Clinical-Radiomics Model for Predicting Axillary Pathologic Complete Response in Breast Cancer With Axillary Lymph Node Metastases. Front Oncol. 2021;11:786346.
12. Wu SG, He ZY, Ren HY, Yang LC, Sun JY, Li FY, et al. Use of CEA and CA15-3 to Predict Axillary Lymph Node Metastasis in Patients with Breast Cancer. J Cancer. 2016;7(1):37–41.
13. Mayerhoefer ME, Materka A, Langs G, Häggström I, Szczypiński P, Gibbs P, et al. Introduction to Radiomics. J Nucl Med. 2020;61(4):488–95.
14. Dong D, Fang MJ, Tang L, Shan XH, Gao JB, Giganti F, et al. Deep learning radiomic nomogram can predict the number of lymph node metastasis in locally advanced gastric cancer: an international multicenter study. Ann Oncol. 2020;31(7):912–20.
15. Ran J, Cao R, Cai J, Yu T, Zhao D, Wang Z. Development and Validation of a Nomogram for Preoperative Prediction of Lymph Node Metastasis in Lung Adenocarcinoma Based on Radiomics Signature and Deep Learning Signature. Front Oncol. 2021;11:585942.
16. Shephard DA. The 1975 Declaration of Helsinki and consent. Can Med Assoc J. 1976;115(12):1191–2.
17. Yushkevich PA, Piven J, Hazlett HC, Smith RG, Ho S, Gee JC, et al. User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. NeuroImage. 2006;31(3):1116–28.
18. van Griethuysen JJM, Fedorov A, Parmar C, Hosny A, Aucoin N, Narayan V, et al. Computational Radiomics System to Decode the Radiographic Phenotype. Cancer Res. 2017;77(21):e104-e7.
19. Paul P, Pennell ML, Lemeshow S. Standardizing the power of the Hosmer-Lemeshow goodness of fit test in large data sets. Stat Med. 2013;32(1):67–80.
20. Zhang Z, Rousson V, Lee WC, Ferdynus C, Chen M, Qian X, et al. Decision curve analysis: a technical note. Ann Transl Med. 2018;6(15):308.
21. Gradishar WJ, Moran MS, Abraham J, Aft R, Agnese D, Allison KH, et al. NCCN Guidelines® Insights: Breast Cancer, Version 4.2021. J Natl Compr Canc Netw. 2021;19(5):484–93.
22. Tausch C, Taucher S, Dubsky P, Seifert M, Reitsamer R, Kwasny W, et al. Prognostic value of number of removed lymph nodes, number of involved lymph nodes, and lymph node ratio in 7502 breast cancer patients enrolled onto trials of the Austrian Breast and Colorectal Cancer Study Group (ABCSG). Ann Surg Oncol. 2012;19(6):1808–17.
23. Zong Q, Deng J, Ge W, Chen J, Xu D. Establishment of Simple Nomograms for Predicting Axillary Lymph Node Involvement in Early Breast Cancer. Cancer Manag Res. 2020;12:2025–35.
24. Cheng J, Ren C, Liu G, Shui R, Zhang Y, Li J, et al. Development of High-Resolution Dedicated PET-Based Radiomics Machine Learning Model to Predict Axillary Lymph Node Status in Early-Stage Breast Cancer. Cancers (Basel). 2022;14(4).
25. Yu Y, He Z, Ouyang J, Tan Y, Chen Y, Gu Y, et al. Magnetic resonance imaging radiomics predicts preoperative axillary lymph node metastasis to support surgical decisions and is associated with tumor microenvironment in invasive breast cancer: A machine learning, multicenter study. EBioMedicine. 2021;69:103460.
26. Li K, Yao Q, Xiao J, Li M, Yang J, Hou W, et al. Contrast-enhanced CT radiomics for predicting lymph node metastasis in pancreatic ductal adenocarcinoma: a pilot study. Cancer Imaging. 2020;20(1):12.
27. Chen X, Liu W, Thai TC, Castellano T, Gunderson CC, Moore K, et al. Developing a new radiomics-based CT image marker to detect lymph node metastasis among cervical cancer patients. Comput Methods Programs Biomed. 2020;197:105759.
28. Li J, Dong D, Fang M, Wang R, Tian J, Li H, et al. Dual-energy CT-based deep learning radiomics can improve lymph node metastasis risk prediction for gastric cancer. Eur Radiol. 2020;30(4):2324–33.
29. Yang C, Dong J, Liu Z, Guo Q, Nie Y, Huang D, et al. Prediction of Metastasis in the Axillary Lymph Nodes of Patients With Breast Cancer: A Radiomics Method Based on Contrast-Enhanced Computed Tomography. Front Oncol. 2021;11:726240.
30. Liu Z, Ni S, Yang C, Sun W, Huang D, Su H, et al. Axillary lymph node metastasis prediction by contrast-enhanced computed tomography images for breast cancer patients based on deep learning. Comput Biol Med. 2021;136:104715.
31. Yang X, Wu L, Ye W, Zhao K, Wang Y, Liu W, et al. Deep Learning Signature Based on Staging CT for Preoperative Prediction of Sentinel Lymph Node Metastasis in Breast Cancer. Acad Radiol. 2020;27(9):1226–33.