Detection of lung cancer using Ten Convolutional Neural Network Models based modified co-learning technique in PET/CT image

doi:10.21203/rs.3.rs-839821/v1

Download PDF

Research Article

Detection of lung cancer using Ten Convolutional Neural Network Models based modified co-learning technique in PET/CT image

https://doi.org/10.21203/rs.3.rs-839821/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted 23 Aug, 2021

You are reading this latest preprint version

Background: A proposed Lung Cancer Detection System (LCDS) faces a major issue due to low spatial resolution in Positron Emission Tomography (PET) and low contrast in Computed Tomography (CT). However, radiologists are unable to notice lung nodules during the beginning stage of lung cancer.

Method: Such an issue has been resolved by creating a modified co-learning technique which will be based on ten Convolutional Neural Network (CNN) models (Alexnet, VGG16, VGG19, Squeezenet, Googlenet, Inceptionv3, Mobilenetv2, Densenet201, Resnet18, Xception). This technique encodes modality specific features and utilizes them to acquire a spatially varying fusion map. These fusion maps are multiplied using a modality feature map for an utilization of image analysis.

Result: A proposed LCDS attained 90 % sensitivity, 99.25% accuracy and 2.4 false positives per scan. By utilizing modified co-learning technique, our proposed LCDS attained 94.165 % sensitivity, 99.40 % accuracy and 1.6 false positives per scan in PET/CT.

Conclusion: Our proposed LCDS attained tremendously minimum false positive rate and it is a promising technique in support of cancerous recognition due to improved sensitivity and accuracy.

Cancer Biology

lung cancer

Positron emission tomography/computed tomography

fusion

Lung cancer [1] [23] is a principal cause of cancer related deaths world wide. To identify this syndrome at a beginning phase, three images were utilized which are named as X-ray, Computed Tomography (CT) and Positron Emission Tomography (PET). Each image face different issues. The issues faced by x-ray image is unable to represent 3D object since it is a 2D image. CT and PET images are suitable for representing 3D objects. Radiologists are unable to notice lung cancer in early stage due to low spatial resolution in PET and low contrast in CT. PET and CT images were combined to improve diagnostic accuracy for lung cancer detection in beginning phase.

FDG PET (Fluorodeoxyglucose positron emission tomography) [2] is the most precise imaging

method to identify functional rather than anatomical data. It’s main issue was lack of spatial resolution. The arrangement of PET and CT in one process assists in overcoming this limitation. Manual interpretation of a PET-CT image is a time consuming task for radiologists due to the differences in inter and intra observer variations [3]. Hence it is very important for people in the high risk category undergone differentiation between primary lung tumor and regional lymph nodes to detect cancerous cells in their early stage of development. Lung tumor is a key of abnormality. It reaches to the lymph nodes before reaching to other parts of the body [4]–[7]. Ju et al. [8] utilized the context of PET and CT regions by integrating graph based and random walk segmentation method for tumor segmentation.

The role of PET-CT in cancer care has aggravated extensive research into methods to create the feature and fusion map. These techniques can be separated into two types: (i) process each image individually and then combine individual attributes of an image [9]–[18] and (ii) combine or fuse complementary attributes from each image [19–22]. Methods that process each image separately are inherently limited when the intent is to consider both the functional and anatomical extent of the syndrome. In contrast, methods that fuse information from two images often utilize a priori knowledge about characterizing different images to prioritize information from one of two images for varied tasks. Alternatively, they may fuse information using a representation that model relationships between the two images. The fusion is particularly necessary in cases where different images identify different attributes of the same region of interest (ROI), with no one image capturing the entire ROI.

Our objective is to reduce false positive in proposed LCDS. We present ten CNN models (Alexnet, VGG16, VGG19, Squeezenet, Googlenet, Inceptionv3, Mobilenetv2, Densenet201,Resnet18, XCeption) that learns to fuse anatomical and functional data from PET-CT images in a spatially varying manner. Our modified co-learning technique based on 10 CNN models is intended as a general approach for integrating feature and fusion maps in PET/CT image for the improvement in contrast and spatial resolution. It leads to improvement in accuracy, sensitivity and specificity.

The dataset for the study was collected from Anderson Diagnostics & Labs, Chennai and MGH Department of Radiology website. Each study comprised one CT volume and one PET volume: the CT resolution was 512 x 512 pixels at 0.98mm x 0.98mm, the PET resolution was 200 x 200 pixels at 4.07mm x 4.07mm, with a slice thickness and an interstice distance of 3mm. Both volumes were reconstructed with the same number of slices. Studies contained between 1 to 7 tumors (inclusive) in the thorax. The tumor locations included the different lung lobes, the mediastinum, and hilar nodes. The images were rescaled to a resolution of 256 x 256 pixels (x-y axes); rescaling the PET and CT volumes so that they share the same coordinate space is a standard process for analysis of PET-CT data [18] [24] [25] [26]. The PET images were normalized by a transformation to standard uptake values (SUVs).

Table 1

Specification of Ten CNN Models
Network	Depth	Size	Parameters (Million)	Image Input Size
Alexnet	8	227 MB	61.0	227-by-227
Vgg16	16	515 MB	138	224-by-224
Vgg19	19	535 MB	144	224-by-224
Squeezenet	18	4.6 MB	1.24	227 by 227
Googlenet	22	27 MB	7.0	224-by-224
Inceptionv3	48	89 MB	23.9	299-by-299
Mobilenetv2	53	13 MB	3.5	224-by-224
Densenet201	201	77 MB	20.0	224-by-224
Resnet18	18	44 MB	11.7	224-by-224
Xception	71	85 MB	22.9	299-by-299

Fig 2 shows the architecture of our modified co-learning technique based on ten Convolutional Neural Network (CNN) models. The Purpose of every ten CNN model is to derive the image features that are most relevant to each specific images. The inputs given to each CNN model are PET image and CT image. The modified co-learning component uses the modality specific Features produced by the ten CNN models to derive a spatially varying fusion map to weight the modality-specific features at different locations. Finally, the reconstruction component integrates the modality-specific fused features across multiple scales to produce the final prediction map.

A. Creation of modified co-learning technique

Let G = X * Y + c be the output feature map of a CNN model where * is the convolution operation, Y is a input to CNN model, X is the learned weights and c is the learned bias. A batch normalization layer has been utilized to normalize every output feature dimension G to a distribution with zero mean and unit variance. The Leaky rectified linear unit (Leaky ReLU) activation function was utilized after feature map normalization:

Α_i(g) = { g g > 0 (1)

{ i.g g <= 0

Where g is a normalized feature and i is a parameter controlling the ‘leakiness’ of the activation function with the constraint that 0 < i < 1. The Leaky ReLU activation avoids the dead neuron problem that can occur with the standard ReLU function where some weights in W can be updated to a value where their training gradients are forever stuck at 0, thus preventing the weights from being updated in the future. The parameter i enables the introduction of a small non zero gradient when g < 0, thereby preventing the weights from being stuck at an unrecoverable value. For simplicity of notion, the output of a convolutional layer has referred by G = Α_i(X * Y + c) which is a feature map generated from Y after convolution, batch normalization and activation.

B. Ten CNN model based modified feature co-learning and fusion

The modified co-learning component contains two parts: (i) a modified co-learning unit is a CNN which learns to derive spatially varying fusion maps (ii) fusion operation utilizes the fusion maps to prioritize different features. Fig 3 shows a conceptual example of the ten CNN model based modified feature co-learning and fusion unit. The inputs to the modified feature co- learning unit are two feature maps G_CTand G_PET(each CNN model) of size w x h x c with w width, h height and c channels. These feature maps are stacked to form Y_multi (w x h x m x c) with m = 2 number of modalities. The channels of Y_multiare then convolved with the channels of a learnable 3D kernel X_multiof size k x k x m, where k is the width and height of the kernel and m = 2 is the number of modalities.

By performing ten CNN models without padding the modality dimension, we obtain for a given channel c a feature map with a singleton third dimension where the value at location (a,b) is determined from the neighborhood of both G_CT(a,b) and G_PET(a,b):

(X_multi * Y_{multi )}(a,b) = ∑ ∑ ∑ X_multi(I,j,l) . Y_multi(x-I,y-j,l) (2)

I j l

We then squeeze the singleton third dimension to obtain an output feature map X_multi*Y_multi of size w x h x 2c, the same width and height as the two modality-specific input feature maps F_CT(a, b) and F_PET (a,b) and double the number of channels, which is important for the weighting of modality-specific feature maps by the modifying co-learned fusion maps.

Our objective is that the modifying co-learned fusion map controls the level of importance given to information from each modality at each location, in contrast to the global fusion ratio in PET-CT pixel intermixing [27]–[29]. Thus the modifying co-learned fusion maps directly affect the input distribution of the learnable layers that immediately follow the modified co-learning unit. Hence, we do not normalize the output of the CNN model within the modified co-learning unit. As with the CNN model, we utilized a Leaky ReLU activation function to obtain the multi modality modifying co-learned fusion map:

G_CNN= A (X_multi * Y_multi+ c_multi) (3)

Where c_multiare the learned biases. The multi modality fusion map G_CNNis obtained by the modified co-learning unit based on ten CNN models. The fusion operation integrates the modality-specific feature maps according to the values (coefficients) in the multi-modality fusion map, as follows:

G_{modco-learned}= G_CNN⨂ (G_CT⨁G_PET) (4)

where G_CNNis the modifying co-learned feature map, ⨁ is the stacking operation and ⨂ is an element wise multiplication. This process merges the two modality-specific feature maps. G_CT and G_PET weights them by the modifying co-learned fusion map similar to pixel intermixing. Our modified co-learning based ten CNN models generates fused feature maps, one for each PET and CT image.

Fig 4 compares ten CNN models based on lung, mediastinum and tumor. It is a visual comparison of the ROIs detected by modified co-learning method based on ten CNN models. In contrast, Squeezenet, Vgg16 and Inceptionv3 detected fewer pixels (as shown by the tumor region) while Alex net and Googlenet detected more pixels within the region.

Table 2 shows the comparison of ten CNN models using accuracy, sensitivity and specificity.

Table 2 Comparison of CNN Models on Detection of lung, mediastinum and tumor

S.no	CNN models	Detection
S.no	CNN models	Accuracy (%)	Sensitivity (%)	Specificity (%)
1	Alex net	94.2	94.8	99.2
2	Vgg16	89.5	88.4	99.2
3	Vgg19	92.4	90.8	99.1
4	Squeezenet	87.4	82.2	99.2
5	Googlenet	97.5	98.3	99.8
6	Inceptionv3	91.2	92.2	99.2
7	Mobilenetv2	96.5	95.4	99.4
8	Densenet201	98.2	97.3	97.2
9	Resnet18	97.2	97.5	99.2
10	Xception	97.55	97.8	98.8
\ Average		94.165	93.47	99.03

In Table 3, the segmentation experiment of our proposed modified co-learning technique based on ten CNN models has higher Dice score when compared to tumor segmentation baseline [13] [25]

Table 3 Comparison with tumor segmentation baseline

S.No	Method	Input	Mean Dice (%)
1	Li et al. [25]	PET-CT patch PET-CT slice	36.45 7.61
2	Zhong et al. [13]	CT patch PET patch combined patch CT slice PET slice combined slice	45.62 62.37 63.09 12.09 62.29 60.13
3	Ashnil Kumar et al	PET-CT slice	63.85
4	Our proposed modified co-learning technique	PET-CT slice	95.039

Our proposed LCDS using modified co-learning technique based ten CNN Models attained 99.40% accuracy and 94.165% sensitivity is better to recognize lung cancer in beginning phase for improving survival rates. It is a promising method for radiologists to recognize an abnormality by PET/CT images from Anderson Diagnostics & Labs image set. By utilizing this technique, false positive of the proposed LCDS have diminished to 1.6 which was lower than previous works

We will give our sincere thanks to Kamaraj College of Engineering and Technology, Virudhunagar, India for doing this research work in the Department of Electronics and Communication Engineering. Our special thanks to Dr. Anant Achary, M.Tech., Ph.D., Principal, Kamaraj college of Engineering and Technology, Virudhunagar, India for his encouragement and continuous support for this research work. Our regards to Dr. Joe Pradeep Kumar, MBBS, Indian MRI Diagnostic & Research Limited, Madurai for his guidance and validation of data. The authors would like to thank all anonymous reviewers for their advice.

[1] https://thetruthaboutcancer.com/lung-cancer-causes/

[2] S. Kligerman and S. Digumarthy, “Staging of non–small cell lung cancer using integrated PET/CT”, Am J Roentgenol(2009), vol. 193, no. 5, pp. 1203–1211.

[3] T. M. Blodgett, C. C. Meltzer, and D. W. Townsend, “PET/CT: Form and Function”, Radiology(2007), vol. 242, no. 2, pp. 360–385.

[4] F. C. Detterbeck, D. J. Boffa, L. T. Tanoue, “The new lung cancer staging system”, Chest(2009), vol. 136, no. 1, pp. 260–271.

[5] S. B. Edge, D. R. Byrd, C. C. Compton, A. G. Frtiz, F. L. Greene, and A. Trotti, Eds., AJCC Cancer Staging Manual (2010). Springer New York, pp. 1471 – 74.

[6] S. B. Edge , C. C. Compton, “The American Joint Committee on Cancer: the 7th Edition of the AJCC Cancer Staging Manual and the Future of TNM”, Ann Surg Oncol(2010), vol. 17, pp. 1471–1474.

[7] E. Tatci, O. Ozmen, Y. Dadali, I. U. Biner, A. Gokcek, F. Demirag, F. Incekara, and N. Arslan, “The role of FDG PET/CT in evaluation of mediastinal masses and neurogenic tumors of chest wall”, Int J Clin Exp Med(2015), vol. 8, no. 7, pp. 11 146–52.

[8] W. Ju, D. Xiang, B. Zhang, L. Wang, I. Kopriva, X. Chen, “Random Walk and Graph Cut for Co-Segmentation of Lung Tumor on PET-CT Images”, IEEE T Imag Process(2015), vol. 24, no. 12, pp. 5854–5867.

[9] A. Teramoto, H. Fujita, O. Yamamuro, and T. Tamaki, “Automated detection of pulmonary nodules in PET/CT images: Ensemble false positive reduction using a convolutional neural network technique”,Med Phys(2016), vol. 43, no. 6, pp. 2821–2827.

[10] L. Bi, J. Kim, A. Kumar, L. Wen, D. Feng, and M. Fulham, “Automatic detection and classification of regions of FDG uptake in whole-body PET-CT lymphoma studies”, Computerized Medical Imaging Graphics(2017), vol. 60, pp. 3–10.

[11] L. Xu, G. Tetteh, J. Lipkova, Y. Zhao, H. Li, P. Christ, M. Piraud, A. Buck, K. Shi, and B. H. Menze, “Automated whole-body bone lesion detection for multiple myeloma on 68Ga-Pentixafor PET/CT imaging using deep learning methods”, Contrast Media Mol I(2018), vol. 2018, p. 11.

[12] F. Milletari, N. Navab, and S. A. Ahmadi, “V-net: Fully convolutional neural networks for volumetric medical image segmentation”, in Fourth International Conference on 3D Vision (3DV) (2016), pp. 565–571.

[13] Z. Zhong, Y. Kim, L. Zhou, K. Plichta, B. Allen, J. Buatti, and X. Wu, “3D fully convolutional networks for co-segmentation of tumors on PETCT images”, in IEEE ISBI(2018), pp. 228–231.

[14] J. Zhao, G. Ji, Y. Qiang, X. Han, B. Pei, and Z. Shi, “A new method of detecting pulmonary nodules with PET/CT based on an improved watershed algorithm”, PLOS ONE(2015), vol. 10, no. 4, pp. 1–15.

[15] C. Lartizien, M. Rogez, E. Niaf, and F. Ricard, “Computer-aided staging of lymphoma patients with FDG PET/CT imaging based on textural information”, IEEE J Biomed Health(2014), vol. 18, no. 3, pp. 946–955.

[16] Y. Song, W. Cai, H. Huang, X. Wang, Y. Zhou, M. J. Fulham, and D. D. Feng, “Lesion detection and characterization with context driven approximation in thoracic FDG PET-CT images of NSCLC studies”, IEEE Transactions on Medical Imaging(2014), vol. 33, no. 2, pp. 408–421.

[17] Q. Song, J. Bai, D. Han, S. Bhatia, W. Sun, W. Rockey, J. E. Bayouth, J. M. Buatti, and X. Wu, “Optimal co-segmentation of tumor in PET-CT images with context information”, IEEE Transactions on Medical Imaging(2013), vol. 32, no. 9, pp. 1685–1697.

[18] D. Han, J. Bayouth, Q. Song, A. Taurani, M. Sonka, J. Buatti, and X.Wu,“Globally optimal tumor segmentation in PET-CT images: A graph based co-segmentation method”, in Information Processing in Medical Imaging. Springer Berlin Heidelberg(2011), pp. 245–256.

[19] T. Bradshaw, T. Perk, S. Chen, H.-J. Im, S. Cho, S. Perlman, and R. Jeraj, “Deep learning for classification of benign and malignant bone lesions in [F-18]NaF PET/CT images”, J Nucl Med(2018), vol. 59, no. S1, p. 327,

[20] Y. Song, W. Cai, J. Kim, and D. D. Feng, “A multistage discriminative model for tumor and lymph node detection in thoracic images”, IEEE T Med Imaging(2012), vol. 31, no. 5, pp. 1061–1075.

[21] L. Bi, J. Kim, D. Feng, and M. Fulham, “Multi-stage thresholded region classification for whole-body PET-CT lymphoma studies” , in MICCAI(2014),pp. 569–576.

[22] A. Kumar, J. Kim, L. Wen, M. Fulham, and D. Feng, “A graph-based approach for the retrieval of multi-modality medical images”,Med Image Anal(2014), vol. 18, no. 2, pp. 330–342.

[23] Nilendu C Purandare, Venkatesh Rangarajan, “Imaging of lung cancer: Implications on staging and management”, Indian Journal of Radiology and Imaging(2015), vol.25, no.2, pp 109-120.

[24] X. Zhao, L. Li, W. Lu, and S. Tan, “Tumor co-segmentation in PET/CT using multi-modality fully convolutional neural network”, Physics in Medicine and Biology (2018), vol. 64, no. 1, pp 1-29.

[25] L. Li, X. Zhao, W. Lu, and S. Tan, “Deep learning for variational multimodality tumor segmentation in PET/CT”, Neurocomputing, pp 1-19.

[26] Z. Zhong, Y. Kim, K. Plichta, B. G. Allen, L. Zhou, J. Buatti, and X. Wu, “Simultaneous cosegmentation of tumors in PET-CT images using deep fully convolutional networks”, Medical Physics(2019), vol. 46, no. 2, pp. 619–633.

[27] W. Cai and G. Sakas, “Data intermixing and multivolume rendering”, Comput Graph Forum(1999), vol. 18, no. 3, pp. 359–368.

[28] A. Quon, S. Napel, C. F. Beaulieu, and S. S. Gambhir, ““flying through” and “flying around” a PET/CT scan: Pilot study and development of 3D integrated 18F-FDG PET/CT for virtual bronchoscopy and colonoscopy,” J Nucl Med(2006), vol. 47, no. 7, pp. 1081–1087.

[29] R. Cheirsilp, R. Bascom, T. W. Allen, and W. E. Higgins, “Thoracic cavity definition for 3D PET/CT analysis and visualization”, Comput Biol Med(2015), vol. 62, pp. 222–238.

[30] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in ICML(2015), vol. 37, pp. 448–456.

[31] A. L. Maas, A. Y. Hannun, and A. Y. Ng, “Rectifier nonlinearities improve neural network acoustic models” , in ICML Workshop on Deep Learning for Audio, Speech and Language (2013), vol. 30, no. 1, p. 3.

[32] J. C. Fernandez Caballero, F. J. Martinez, C. Hervas, and P. A. Gutierrez, “Sensitivity versus accuracy in multiclass problems using memetic pareto evolutionary neural networks”, IEEE Transaction on Neural Network(2010), vol. 21, no. 5, pp. 750–770.

No competing interests reported.

Download PDF

Version 1

posted 23 Aug, 2021

You are reading this latest preprint version

Detection of lung cancer using Ten Convolutional Neural Network Models based modified co-learning technique in PET/CT image

Status:

Version 1

Abstract

Figures

1. Introduction

2. Materials

3. Specification Of Ten Cnn Models

4. Proposed Lung Cancer Detection Scheme

5. Result

6. Conclusion

Declarations

References

Additional Declarations

Status:

Version 1