Few-shot Learning on the Diagnosis of Lymphatic Metastasis of Lung Carcinoma

Background ： Lymph node metastasis of lung cancer plays an important part in lung cancer diagnosis since its close relationship with patients ’ prognosis and subsequent treatment, however, diagnosis of lymph node metastases often was tedious and takes too much time for pathologists. Automatized classification with few-shot learning algorithms is required for clinical situations that only unbalanced and small datasets can be collected. We developed a few-shot learning algorithms for automatized classification of lymphatic metastasis of lung carcinoma from Whole Slide Images (WSIs) through a deep convolutional neural network (DCNN). Material and methods: Lymph node slides of patients with lung cancer collected from July to December 2018 were reviewed and used for model development retrospectively. A total of 1701 lymph node slides from 453 lung cancer patients with different histological types were enrolled. Each slide could provide a different number of patches, in which 6, 8, 10, 20, and 30 positive patches and 12, 16, 20, 40, and 60 negative patches for training and validation, respectively. A total of 1577 WSIs were used. The receiver operating characteristic (ROC) analysis of the model was performed for estimating the performance and compared to the traditional deep learning approach Resnet34. Results: Deep learning methods, when trained with very few shots (6 positive/12 negative, 8 positive/16 negative, and 10 positive/20 negative patches), the traditional deep learning approach Resnet34 was outperformed by our Aitrox model. When the dataset was raised to 20 positive/40 negative patches, similar performance was achieved with AUC 0.5895 (95% CI, 0.5833-0.5910) for Resnet34 and 0.6124 (95% CI, 0.6077-0.6147) for our Aitrox model. Resnet34 over-performed our Aitrox model (AUC=0.6450 [95%CI, 0.6397-0.6470]) with an AUC of 0.7481 (95%CI, 0.7120-0.7485) when the training and validation dataset was raised to 30 positive/60 negative patches. Conclusions : This proposed few-shot method may address AI approaches to the diagnosis of lymphatic metastasis of lung carcinoma and other kinds of carcinoma in the future.


Introduction
Lung cancer is one of the most common cancers worldwide[1] and is the most common cancer in China [2]. Accurate lung cancer staging is an essential task performed by pathologists to make clinical management. Evaluation of the extent of lung cancer spread by histopathological examination of the lymph node is an important factor of lung cancer staging [3]. Nevertheless, the pathologic screen of positive lymph nodes is tedious and timeconsuming. There was a total of 5657 lung cancer patients who had been performance pulmonary surgeries at Shanghai Chest Hospital from July 2018 to December 2018, and 24060 lymph node groups have been removed, each lymph node group represents one whole slide, while only 671 lymph node groups were positive. The possibility of artificial intelligence aided diagnostics may be advantageous.
In recent years, the increased use of whole slide imaging coupled with advances in computation and deep learning methods has sparked much interest in applying AI to pathology [4][5][6]. Depending on the technical improvement of the WSIs scanning, the pathological diagnosis combining the AI recognition technology improving the efficiency of pathologist may come true, for example, with the algorithm assistance, the sensitivity and specificity of reviewing the lymph node metastasis of breast cancer were increased, and the average review time per image was reduced [7,8]. The algorithm design based on the dataset, datasets with different scales, and types need different strategies, usually, a large dataset was fit for weakly supervised deep learning, and a small dataset was fit for fully supervised deep learning. The works of Kanavati[9] and Wang[10] suggested that the weakly supervised deep learning may be more efficient than the fully supervised deep learning when classifying the lung cancer histological types with the large dataset. The weakly supervised deep learning needed the slide-level diagnoses and avoided the detailed cell-level annotations which means tons of workload. The clinical-grade pathology using was trained by weakly supervised deep learning with over thousands of slides dataset including prostate core biopsy, breast cancer metastatic lymph node, and skin lesion separately; and the area under the curve (AUC) can be 0.965 on the test of breast axillary lymph node metastasis [8]. The accuracy of the weakly supervised deep learning may be affected by the labels[9] and the size of the dataset which could be improved by upgrading the algorithm iteration [11][12][13]. Atsushi Teramoto improves 4% AUC by using a two-step supervised strategy for the classification of lung cytological images with 60 cases dataset [14].
However, to further enhance the performance of the AI model, it would be necessary to increase the number of images used to train the CNN model. Furthermore, in the real world, most lymph nodes are negative and few nodes have metastatic tumors, which means the negative and normal tissues can be easily acquired, while positive and pathological tissues are infrequent. Small number of positive samples and unbalanced proportion of positive and negative samples often cause overfitting when training with traditional deep convolutional neural networks (DCNN). The rational utilization of the normal datasets instead of discarding them for only balancing the proportion is also our consideration.
In this study, a few-shot learning algorithm [15][16][17] on automatizing the classification of malignant lung cells from Whole Slide Images (WSIs) though a DCNN was developed to solve the above-mentioned problems on mimicking the real-world environment. Meanwhile, a traditional network Resnet 34 was also enrolled in this investigation for comparison with our prototypical network-based few-shot learning model [18][19][20]. To the best of our knowledge, this study introduces the first application of few-shot learning methods for digital pathology image analysis in the diagnosis of lymphatic metastasis of lung carcinoma.

Study participants
For this study, a total of 1701 lymph node slides from 453 lung cancer patients with different histological types were enrolled. Slides were collected from the Shanghai Chest Hospital,  Table 1).

Image dataset
All the optimized H&E stained WSIs were digitalized by a bright-field scanner (DMS-10, D-Metrix, http://www.dmks.cn/), using a 40X magnification objective with a color CCD with a pixel resolution of 0.25 µm. To construct the image dataset for our DCNN, patch images of 224x224 pixels were cut from the original microscopic WSIs. Patches were extracted from each WSI. In region annotation, patches with vivid cytopathologic features for tumors were selected as positive patches by an experienced pathologist with the online digital slides viewer named MISS (Microscope Image Information System) from Shanghai Aitrox Technology Corporation Limited (Shanghai, China) (Fig 1).

Resnet 34
In this study, a traditional network Resnet34 was used to compare against other algorithms and applied to few-shot lymphatic metastasis of lung carcinoma diagnostics. This Resnet34 was unmodified from its original architecture [19][20][21]. Training and validation datasets were allocated according to 4:1 and 5-folder cross-validation was carried out in each experiment.
Adam was used as the optimizer, and the initial learning rate was 0.001, beta 1 0.9, and beta 2 0.99. Cross entropy was used as the value function.

Our Aitrox model
For comparison with the Resnet34, a strategy to training a 2-way classification model with few and unbalanced samples was designed based on Prototypical Network [15]. In our cases, we use a small set of M usual positive labeled patches and a large set of N negative labeled patches as our training set (N =2 M). For each training epoch, M positive and M (randomly selected from N) negative patches were enrolled (Fig 2). There are two main steps in our method. First, the prototypical network was trained on annotated patches as mentioned before. Second, the well-trained model was used to predict the Whole Slide Images (WSIs).
The Prototypical Network is a kind of metric learning network, which is not like the traditional classification networks that classify objects directly. The output of the prototypical network is a high-dimensional feature vector that was used to update our model weights.

Design Choice
We chose Euclidean distance [22][23][24] as our distance metric. Take one of the small datasets for example, the number of the labeled positive patch in training set M = 6. The number of the labeled negative patch in training set N = 12. Ns = 3, Nq = 4.
We chose Resnet34 as our backbone to embed the input into a 512-dimensional feature vector. Trained about 10,000 epochs. The process of each epoch in our training mainly consists of 3 steps (Fig 3): AUC on WSI for each model was calculated by varying the probability of the Kth highest probability of patch in WSI. In our case threshold K = 5. Sensitivity, specificity, and accuracy were calculated under the Threshold of 0.6/5 (Table 1) When both models were trained in the same training and validation dataset with 20pos/40neg patches, respectively, similar performance was achieved since AUC for Resnet34 was 0.5895 (95%CI, 0.5833-0.5910) and our Aitrox model was 0.6124 (95%CI, 0.6077-0.6147), respectively (Fig 5). Sensitivity, specificity and accuracy were also shown in Table 1.

Discussion
In this study, the evaluation of deep learning for automated lymphatic metastasis of lung carcinoma diagnosis with small training datasets suggests that the traditional network Resnet 34 over-performed our Aitrox model with 30pos/60neg patches at an AUC of o.7481, but it deteriorated substantially when training was relatively low with 20pos/40neg patches or fewer. Results from our Aitrox model suggested that the performance was acceptable when training with a relatively small size (10pos/20neg patches) and also has more graceful degradation in performance when using a very small number of training datasets with 6pos/12neg patches.
We believe that the performance of Resnet34 decreases as the training number decreases was because that it may be hard for the network to do fine-tuning since cross-entropy [25][26][27] was used as a loss function which could not extract enough features for model training. In our Aitrox model, metric-based [28,29]  These findings should support the investigation of few-shot learning methods for the clinical AI diagnosis which rely on small and unbalanced data sets. Application to lymphatic metastasis of lung carcinoma diagnosis is one such use, in which the small number of positive lymph nodes and the large number of negative lymph nodes are typically available. AI bias is also a concern for both medical applications and predicting needs for health care [30]. In most cases, the number of positive lymph nodes is about half that of negative lymph nodes. Fewshot learning could offer an option to address both the small training dataset and the balance problem.

Limitations
This study has some limitations. Only Prototypical Network and Resnet34 were examined.
Other DCNNs such as Siamese Network [31], vgg [32,33], Inception [34] and Alex-net [35] should be taken into consideration in future studies. Also, the largest dataset used in this study was 30pos/60neg patches, in which 124 slides were used for training and validation.
Additional samples are needed for further comparing between few-shot learning and traditional deep learning methods.

Conclusions
To our knowledge, this study is the first to explore the performance of few-shot learning networks in lymphatic metastasis of lung carcinoma. The results show that the performance of widely used deep learning methods relying on full supervision that typically require large training data sets degrades substantially when used with limited data sets while few-shot learning appears to perform better and have more graceful performance degradation as the number of training database decreased. This few-shot method may address AI approaches to the diagnosis of lymphatic metastasis of lung carcinoma and other kinds of carcinoma in the future.

Ethics approval and consent to participate
All patients gave written informed consent for their participation. This study has been approved by the Ethics Committees of Shanghai Chest Hospital, Shanghai Jiao Tong University, and it conforms to the principles outlined in the Declaration of Helsinki. The study was also conducted under an exemption approved by the Shanghai Chest Hospital review board.

Consent for publication
Not applicable.

Availability of data and materials
The data sets supporting the results of this article are included within the article and its supplementary information files.

Competing interests
Authors have no conflict of interest to declare. I would like to declare on behalf of my coauthors that the work described was original research that has not been published previously, and not under consideration for publication elsewhere, in whole or in part. All the authors listed have approved the manuscript that is enclosed.     Overview of each training epoch

Supplementary Files
This is a list of supplementary les associated with this preprint. Click to download.