Unsupervised anomaly detection with generative adversarial networks in mammography

doi:10.21203/rs.3.rs-1533364/v1

PDF

Article

Unsupervised anomaly detection with generative adversarial networks in mammography

https://doi.org/10.21203/rs.3.rs-1533364/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Breast cancer is a common cancer among women, and screening mammography is the primary tool for diagnosing this condition. Recent advancements in deep learning technologies have triggered the implementation for the research studies via mammography. In clinical situations, where abnormal data are significantly lacking, semisupervised or unsupervised methods are often used to overcome the limitations of supervised learning such as manpower and time for labeling. Accordingly, we proposed a generative model that uses a state-of-the-art generative network (StyleGAN2) to create high-quality synthetic mammographic images and the anomaly detection method to detect breast cancer on mammograms in unsupervised methods. The generation model trained via only normal mammograms and breast cancer classification was performed via anomaly detection using 50 each breast cancer and normal mammograms that did not overlap with the dataset for generative model learning. We developed an anomaly detection method that would evaluate this method in breast cancer classification. Our generative model has shown comparable fidelity to real images, and the anomaly detection method via this generative model showed high sensitivity, demonstrating its potential for breast cancer screening. This method could differentiate between normal and cancer-positive mammogram and help overcome the weakness of current supervised methods.

mammography

deep learning

GAN

anomaly detection

Breast cancer is the most commonly diagnosed cancer and the leading cause of cancer death among females [1, 2]. Mammography is regarded as the most effective screening tool for breast cancer detection and diagnosis. Based on the results of several randomized clinical trials, screening mammography has been shown to reduce the rate of death from breast cancer by 25% in women between the ages of 50 and 69 years [3–6]. Although the use of screening mammography remains controversial due to concerns regarding methodological limitations in some of the randomized trials, recent studies have observed breast cancer mortality reductions in service screening programs consistent with those observed in the randomized trials [7, 8]. Moreover, Kalager et al. investigated the availability of screening mammography via valid comparison groups instead of historical control subjects to consider chronologic trends associated with advances in breast cancer awareness and treatment [9]. The authors concluded that screening mammography was still associated with a reduction in the rate of death from breast cancer, but screening itself accounted for only about a third of the total reduction.

Low contrast between cancerous and normal tissues is one of the most significant challenges of mammography, which makes it difficult for radiologists to interpret the results. Computer aided diagnosis and computer aided detection of abnormalities in mammography have been introduced and played an important role in breast cancer screening [10, 11]. Furthermore, recent advances in deep learning (DL) networks have enabled automatic feature extraction from training data and have thus become powerful techniques in various fields, including medical imaging. A recent study using DL methods, specifically convolutional neural networks (CNN) [12], resulted in a high area under the curve (AUC) of 0.942 and achieved 92% specificity for the classification of benign versus malignant tissues. Current CNN models with supervised learning are designed to improve the ability of radiologists to detect even the smallest breast cancers at their earliest stages, thus alerting radiologists when further analysis is needed [13]. However, supervised learning renders it difficult for the trained model to evaluate data that are completely different from the data encountered during training. This is especially true in real clinical situations where abnormal data are usually very scarce compared with normal data. Additionally, labeling efforts for supervised learning require enormous manpower and time resources. To alleviate the burden of manual annotation, several studies have introduced unsupervised training [14].

DL networks can also be used for generating realistic images. Various generative adversarial networks (GANs) [15] have been developed and improved the synthetic performance of high-resolution images via two competing DL networks: the network that generates new samples from noise and the another network that discriminates samples as real or synthetic data [16, 17]. Certain studies have applied GANs to generate realistic medical images such as abdominal computed tomography images [18] and high-resolution lateral cephalometric radiographic images [19]. Additionally, GAN-derived synthesized images have been used for data augmentation [20] and unsupervised anomaly detection [21]. Anomaly detection is one of the most important issues across a range of domains, including manufacturing [22] and medical imaging [23].

In this study, we generated realistic synthetic mammographic images from a state-of-the-art generative network, the StyleGAN2 [24], using normal mammographic images, and developed an anomaly detection method for breast cancer.

Figure 1 shows the images generate from the styleGAN2 model. The best Frechet inception distance (FID) [25] score for the trained StyleGAN2 was 4.383. The overall breast morphologies and internal parenchymal structures of synthetic images were observed quite realistically. However, some unusual noise-like patterns were noticed inside the parenchymal structure of the breast in the magnified view, which were not identified in real mammographic images (Fig. 2). Figure 3 shows examples of anomaly detection from the StyleGAN2 model. For each of the four cases depicted, the images in each row represent the real image, one of the nine normal synthetic images that was most similar to the real image (projected image), and the difference map between the real image and the average of nine synthetic images.

Table1 shows breast cancer classification results via anomaly detection together with the respective accuracy, sensitivity, specificity, PPV (positive predictive value), NPV (negative predictive value), and AUC (the area under the receiver operating characteristic curve). We compared these results according to the number of synthetic image seeds created per image. The use of nine different seeds provided the best results with high sensitivity for breast cancer and an accuracy of 64%. Figures4,and 5 demonstrate a ROC (the receiver operating characteristic) curve and a histogram of anomaly scores between cancer and normal data for the best case, respectively.

Table1. Performance comparisons of breast cancer classification with the number of seeds per image in the anomaly detection manner.

Number of synthetic image seeds created per image	Accuracy(%)	Sensitivity(%)	Specificity(%)	PPV(%)	NPV(%)	AUC(%)
1	61.0	74.0	48.0	58.7	64.9	66.6
9	64.0	78.0	52.0	61.4	70.2	70.0
16	63.0	78.0-	48.0	60.0-	68.6	67.8

Note: PPV, positive predictive value; NPV, negative predictive value; AUC, the area under the receiver operating characteristic curve.

Figure 4. The ROC curve of breast cancer classification

Figure 5. Anomaly score histogram

In this study, we generated highly realistic synthetic mammographic images from a state-of-the-art generative network, the StyleGAN2, and developed an unsupervised anomaly detection method. The StyleGAN2 model generated efficient coarse breast structures from the beginning, and as training progressed, it generated complex parenchymal tissues. Although most generated images have shown comparable fidelity to real images, unusual noise-like patterns were present in some synthesized images. These artifacts might have occurred because of the StyleGAN2 architecture, wherein the generator and discriminator use the progressive training process, which means that networks are trained layer by layer, and geological features are learned from coarse to fine scales. Moreover, per-pixel noise was injected after each convolution, which could compensate for the loss of information compression so as to capture high-variance details [26]. In our model, complex and highly variable internal parenchymal structures might be too fine to be trained as styles and could be regulated only via injected noise.

Furthermore, we developed an anomaly detection method using this generation model trained via only normal mammograms and evaluated this method in breast cancer classification. Also, we compared the results according to the number of synthetic image seeds created per image. The best case was using nine different seeds. The AUC, sensitivity, and specificity of this method via anomaly scores were 70%, 78%, and 52%, respectively. Using one seed for a synthetic image could be relatively insufficient to remove the false positive part of the normal dataset. When 16 seeds were used, the sensitivity was the same, but there could be a decrease in performance when averaging the difference map because too many different images were created.

Although the classification performance was not high enough and the validation dataset was small, these preliminary results highlight the potential of this method to classify breast cancer in mammographic images in an unsupervised manner using the anomaly detection using GANs (AnoGAN) architecture. However, projected synthetic images that were the most similar to normal mammograms did strictly correspond to the real images, and thus increased the false- positive cases. Additionally, benign diseases were not excluded when collecting mammographic images for training the generation model, and this might have also increased the false-negative cases because some benign conditions look similar to cancer. To overcome these weaknesses, future studies could consider methods to improve the acquisition of corresponding projection images for test images, but also a staged model that can reduce the number of false-negative cases. A stage model could be implemented via training normal mammograms with and without benign conditions and filtering the normal cases and then the benign cases via staged projection and anomaly scoring.

Although the performance of breast cancer classification via a supervised method is well-suited [27–29], it might have some limitations. It can be considered as a learning method that only memorizes patterns of learning data. Therefore, classifying new unseen data that have never been learned during training is difficult, and a large amounts of labeled data is inevitably required. However, in clinical situations, abnormal data are usually very scarce compared with normal data, and annotation for a large number of datasets for supervised learning requires enormous manpower and time. Thus, semisupervised or unsupervised methods could be very useful alternatives for supervised methods[30, 31]. In a similar context, our anomaly detection method for breast cancer detection would be an additional screening tool or/and alarm system with the supervised method.

This study has several limitations. First, only craniocaudal views of mammograms with limited-resolution were used for the generation of images and the detection of anomalies in this study. Therefore, generation with mediolateral oblique views of mammograms and full high-resolution images (i.e. 2294\(\times\)1914 pixels), which are commonly used in the clinical field would allow for significantly more accurate anomaly detection. Second, we could not conduct the image Turing test by radiologists to evaluate the qualitative performance of the generated images. Especially in medical images, evaluation for qualitative performance might be more crucial than quantitative evaluation, which only measures differences in the density between two distributions in the high-dimensional feature space. Finally, our preliminary results of breast cancer detection showed insufficient performance for clinical application. Therefore, improvements on more similar projections for test images and a staged model using a large amount of high-quality dataset, including distinguished benign cases should be considered to investigate its potential as an additional screening tool.

In conclusion, this study proposes a generative model that uses StyleGAN2 for the generation of high-quality synthetic mammographic images and the anomaly detection method for the detection of breast cancer on mammograms in an unsupervised manner via AnoGANs. Our generative model has shown comparable fidelity to real images, and the anomaly detection method via this generative model trained with only normal mammograms could differentiate between normal and cancer-positive mammograms. This method could provide additional information and help overcome the weakness of current supervised methods for breast cancer detection.

Data collection

We retrospectively reviewed electronic medical records of patients with breast cancer who underwent mammography in Asan Medical Center between January 2008 and December 2017(institutional review board (IRB) No: 2017 − 1341). Normal mammograms were collected from mammograms of the normal breast contra-lateral to cancer and their follow-up mammograms. Mammograms containing surgical clips and mammograms of patients with synchronous bilateral breast cancer were excluded from this study. Mammograms with clips were excluded because they can have a significant adverse effect on the generation performance of GAN due to their high intensity and eccentricity. Finally, this study included 105,948 normal mammograms from 22,848 patients for training the generation model. Only craniocaudal views of mammographic images were used for model training. Additionally, we collected downstream datasets to evaluate the method of anomaly detection. Fifty mammograms of breast cancer, which were pathologically staged to T stage 1 to 4, according to the 8th edition of the American Joint Commission on Cancer Staging Manual [32], and fifty normal mammograms, which did not overlap with those used in training for the generative model, were obtained.

This retrospective study was conducted according to the principles of the Declaration of Helsinki and was performed in accordance with current scientific guidelines. The study protocol was approved by the IRB of the Korea National Institute for Bioethics Policy, Seoul, Korea. Informed consent was acquired from all the patients.

Mammography generationStyleGAN2 training

GANs [15] are known as powerful DL generative models with two neural networks, i.e., a generator network and a discriminator network. The generator produces synthetic images from random noise vectors and tries to fool the discriminator, whereas the discriminator tries to distinguish the fake samples from the real samples. The process can be described as a min-max game shown in the following function (1):

\({\text{m}\text{i}\text{n}}_{\text{G}}{\text{m}\text{a}\text{x}}_{\text{D}}\text{V}\left(\text{D},\text{G}\right)={\text{E}}_{\text{x}\tilde{\text{P}}_{\text{d}\text{a}\text{t}\text{a}\left(\text{x}\right)}}\left[\text{l}\text{o}\text{g}\text{D}\left(\text{x}\right)\right]+{\text{E}}_{\text{Z}\tilde{\text{P}}_{\text{z}\left(\text{Z}\right)}}\left[\text{l}\text{o}\text{g}(1-\text{D}\left(\text{G}\left(\text{z}\right)\right))\right]\) (1),

where\(\text{x}\) is a “real” sample from the actual dataset, represented by distribution \({\text{P}}_{\text{d}\text{a}\text{t}\text{a}\left(\text{x}\right)}\), and \(\text{z}\) is a ‘latent vector’ sampled from the distribution \({\text{P}}_{\text{z}\left(\text{Z}\right)}\), which is typically noise. Initially, GAN training was highly unstable, and many strategies have been proposed to tackle this problem [16, 33]. Kerras et al. demonstrated that the style-based generator architecture for GANs(StyleGAN) was very effective in generating high-resolution images by learning both global attributes and stochastic details [34]. The latent space Z is first mapped into the W space via a non-linear mapping network (an 8-layer MLP) and is then merged into the synthesis network via adaptive instance normalization (AdaIN)[35] at each convolutional layer. Gaussian noise is added after each convolutional layer, before the AdaIN layer. StyleGAN2 was proposed to improve the shortcomings of StyleGAN. Including improvement in the loss function, AdaIN was restructured to weight demodulation, and progressive growth was removed because it introduced small artifacts during image generation.

To train the StyleGAN2 with mammographic images, all mammograms of right-sided breast were aligned to left by flipping along their vertical axis to use only left-sided mammograms for training efficiency. The windowing level of each image was adjusted using the center and width values of each image from Digital Imaging and Communications in Medicine (DICOM) header information to set the environment that is most similar to the one that radiologists see in the viewer. The original 12-bit grayscale DICOM images were converted into 8-bit grayscale. Then, we changed the original image (2294\(\times\)1914 pixels) to a modified size (2294\(\times\)2294 pixels) by padding zeros on the right edge and downscaling it to 512\(\times\)512 pixels for training.

A publicly available official implementation of StyleGAN2 via Tensorflow in Python was used. The model was trained via the original NVIDIA implementation on a computer with a Linux operating system. The learning rate and batch size were set to 0.001, and 8, respectively, and other parameters were fixed as default values while training. Furthermore, a quantitative analysis was conducted to analyze the quality of the images generated. We used the FID that measures differences in density of between two distributions in the high-dimensional feature space of an InceptionV3 [36] classifier, which compares the activation of a pre-trained classification network on real and generated images. We monitored the training process (i.e., training losses, FID score, and generated images) with TensorBoard, including the intermediated images, to determine whether the StyleGAN2 was properly trained. The training took 64 h with two Tesla v100-sxm2-32 GB graphic processing units.

Cancer detection with AnoGAN

Anomaly detection using GANs (AnoGAN) is an unsupervised learning method using deep convolutional generative adversarial networks. AnoGAN aims to use a standard GAN, which was trained only on positive samples, to learn a mapping from the latent space representation z to the realistic sample \(\widehat{\text{x}}=\text{G}\left(\text{z}\right)\), and use this learned representation to map unseen samples back to the latent space. Training GAN with normal samples alone makes the generator learn the manifold X of normal samples. Given that the generator learns how to generate normal samples, the difference between the input and the reconstructed image will highlight the anomalies.

We yield normal synthetic images that were most similar to the test image with different seeds on the latent space of the StyleGAN2 by minimizing a perceptual loss. An anomaly score was calculated by summing the difference maps between real and test images, which were then divided by the area of each breast. Finally, a threshold for the anomaly score that could classify normal and cancer images was determined by the threshold of the Youden J index. Figure 6 illustrates the overall workflow.

Statistical Analyses

Breast cancer classification was performed via AnoGAN. We then calculated AUC, accuracy, sensitivity, specificity, PPV, and NPV of this classification model as quantitative metrics to evaluate the performance of the model.

ROC curves represent the relation between true-positive and false-positive rates. The AUC was obtained to reflect the overall accuracy of the model. Eq. (2)-(6) show the formulas of each metric used:

Accuracy \(=\) (true positive \(+\) true negative) \(∕\) (positive \(+\) negative) (2)

Sensitivity \(=\) true positive \(∕\) (true positive \(+\) false negative) (3)

Specificity \(=\) true negative \(∕\) (true negative \(+\) false positive) (4)

PPV \(=\) true positive(true positive \(+\) false positive) (5)

NPV \(=\) true negative/(true negative \(+\) false negative) (6)

Data availability

The datasets are not publicly available because of restrictions in the data-sharing agreements with the data sources. Ethics approval for using the de-identified slides in this study will be allowed upon request to the corresponding authors.

Author contributions

S.Park (Seungju Park) contributed to the data cleansing, processing, GAN training, classification task, interpretation of results, and draft of the manuscript. K.Lee (Kyung Hwa Lee) contributed to data acquisition, cleansing, interpretation of results, and draft of the manuscript. B.Ko contributed to conception, design, interpretation, and data acquisition. N.Kim contributed to conception, design, and draft of the manuscript. All authors reviewed the manuscript.

Competing interests

The authors declare no competing interests.

Ghoncheh, M., Z. Pournamdar, and H. Salehiniya, Incidence and mortality and epidemiology of breast cancer in the world. Asian Pacific journal of cancer prevention, 2016. 17(sup3): p. 43–46.
Sung, H., et al., Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: a cancer journal for clinicians, 2021. 71(3): p. 209–249.
Fletcher, S.W., et al., Report of the international workshop on screening for breast cancer. JNCI: Journal of the National Cancer Institute, 1993. 85(20): p. 1644–1656.
Chu, K.C., C.R. Smart, and R.E. Tarone, Analysis of breast cancer mortality and stage distribution by age for the Health Insurance Plan clinical trial. JNCI: Journal of the National Cancer Institute, 1988. 80(14): p. 1125–1132.
Miller, A.B., et al., Canadian National Breast Screening Study: 2. Breast cancer detection and death rates among women aged 50 to 59 years. Cmaj, 1992. 147(10): p. 1477–88.
Nystrom, L., et al., Breast cancer screening with mammography: overview of Swedish randomised trials. Lancet, 1993. 341(8851): p. 973–8.
Hendrick, R.E., et al., Benefit of screening mammography in women aged 40–49: a new meta-analysis of randomized controlled trials. JNCI Monographs, 1997. 1997(22): p. 87–92.
Gabe, R. and S. Duffy, Evaluation of service screening mammography in practice: the impact on breast cancer mortality. Annals of oncology, 2005. 16: p. ii153-ii162.
Kalager, M., et al., Effect of screening mammography on breast-cancer mortality in Norway. New England Journal of Medicine, 2010. 363(13): p. 1203–1210.
Dheeba, J., N.A. Singh, and S.T. Selvi, Computer-aided detection of breast cancer on mammograms: A swarm intelligence optimized wavelet neural network approach. Journal of biomedical informatics, 2014. 49: p. 45–52.
Roehrig, J. and R.A. Castellino, The promise of computer aided detection in digital mammography. European Journal of Radiology, 1999. 31(1): p. 35–39.
Albawi, S., T.A. Mohammed, and S. Al-Zawi. Understanding of a convolutional neural network. in 2017 international conference on engineering and technology (ICET). 2017. Ieee.
Kooi, T., et al. A comparison between a deep convolutional neural network and radiologists for classifying regions of interest in mammography. in International Workshop on Breast Imaging. 2016. Springer.
Cordeiro, F.R., W.P.d. Santos, and A.G. Silva-Filho, Analysis of supervised and semi-supervised GrowCut applied to segmentation of masses in mammography images. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, 2017. 5(4): p. 297–315.
Goodfellow, I., et al., Generative adversarial nets. Advances in neural information processing systems, 2014. 27.
Karras, T., et al., Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196, 2017.
Miyato, T., et al., Spectral normalization for generative adversarial networks. arXiv preprint arXiv:1802.05957, 2018.
Frid-Adar, M., et al., GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing, 2018. 321: p. 321–331.
Kim, M., et al., Realistic high-resolution lateral cephalometric radiography generated by progressive growing generative adversarial network and quality evaluations. Scientific reports, 2021. 11(1): p. 1–10.
Bowles, C., et al., Gan augmentation: Augmenting training data using generative adversarial networks. arXiv preprint arXiv:1810.10863, 2018.
Schlegl, T., et al. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. in International conference on information processing in medical imaging. 2017. Springer.
Schubert, E., A. Zimek, and H.-P. Kriegel, Local outlier detection reconsidered: a generalized view on locality with applications to spatial, video, and network outlier detection. Data mining and knowledge discovery, 2014. 28(1): p. 190–237.
Taboada-Crispi, A., et al., Anomaly detection in medical image analysis, in Handbook of research on advanced techniques in diagnostic imaging and biomedical applications. 2009, IGI Global. p. 426–446.
Karras, T., et al. Analyzing and improving the image quality of stylegan. in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.
Heusel, M., et al., Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 2017. 30.
Feng, R., D. Zhao, and Z.-J. Zha. Understanding noise injection in gans. in International Conference on Machine Learning. 2021. PMLR.
Ting, F.F., Y.J. Tan, and K.S. Sim, Convolutional neural network improvement for breast cancer classification. Expert Systems with Applications, 2019. 120: p. 103–115.
Zheng, Y., C. Yang, and A. Merkulov. Breast cancer screening using convolutional neural network and follow-up digital mammography. in Computational Imaging III. 2018. International Society for Optics and Photonics.
Nithya, R. and B. Santhi, Classification of normal and abnormal patterns in digital mammograms for diagnosis of breast cancer. International journal of computer applications, 2011. 28(6): p. 21–25.
Nakao, T., et al., Unsupervised deep anomaly detection in chest radiographs. Journal of Digital Imaging, 2021. 34(2): p. 418–427.
Pocevičiūtė, M., G. Eilertsen, and C. Lundström. Unsupervised anomaly detection in digital pathology using GANs. in 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI). 2021. IEEE.
Amin, M.B., et al., The eighth edition AJCC cancer staging manual: continuing to build a bridge from a population-based to a more “personalized” approach to cancer staging. CA: a cancer journal for clinicians, 2017. 67(2): p. 93–99.
Arjovsky, M., S. Chintala, and L. Bottou, Wasserstein gan. arXiv 2017. arXiv preprint arXiv:1701.07875, 2017. 30: p. 4.
Karras, T., S. Laine, and T. Aila. A style-based generator architecture for generative adversarial networks. in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.
Huang, X. and S. Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. in Proceedings of the IEEE international conference on computer vision. 2017.
Szegedy, C., et al. Inception-v4, inception-resnet and the impact of residual connections on learning. in Thirty-first AAAI conference on artificial intelligence. 2017.

No competing interests reported.

PDF

Editor invited by journal
11 Apr, 2022
Submission checks completed at journal
11 Apr, 2022
First submitted to journal
07 Apr, 2022

You are reading this latest preprint version

Unsupervised anomaly detection with generative adversarial networks in mammography

Status:

Version 1

Abstract

Figures

Introduction

Result

Discussion And Conclusions

Methods

Data collection

Mammography generationStyleGAN2 training

Cancer detection with AnoGAN

Statistical Analyses

Declarations

References

Competing Interests

Status:

Version 1