Deep Skin Cancer Model based on Knowledge Distillation Technique for Skin Cancer Classification

doi:10.21203/rs.3.rs-1243348/v1

Download PDF

Research

Deep Skin Cancer Model based on Knowledge Distillation Technique for Skin Cancer Classification

https://doi.org/10.21203/rs.3.rs-1243348/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Skin cancer is more treatable in the early stages since it spreads slowly to other parts of the body. Early detection is essential due to the rising number of cases of skin cancer, as well as the high fatality rate and high expense of medical treatment. Deep Learning algorithms have lately been used to improve the performance of a variety of biomedical image processing modalities. Their structures have been developed to solve classification challenges that are suffering from a lack of training data in skin cancer infections (actinic keratosis, basal cell carcinoma, dermatofibroma, melanoma, nevus, pigmented benign keratosis, seborrheic keratosis, squamous cell carcinoma, and vascular lesions are skin infections). In this work, we propose a Deep Skin Cancer (DSC) model depending on the Knowledge Distillation technique and some optimization algorithms. The experimental results based on the used benchmark datasets from the International Skin Imaging Collaboration (ISIC) show superior performance in skin cancer diagnosis. According to the reported results, it has been empirically achieved that a sensitivity value of 99.16% has been empirically achieved, a specificity value of 99.57% with the Adamax optimizer, and a lack of labeled input image data. Furthermore, the results assist in diagnosing some COVID-19 cases due to the similarity between the skin cancer infection and the black fungus found in some COVID-19 survivors, particularly those with co-morbid conditions similar to skin cancer infection.

Deep learning

Skin Cancer

medical imaging

ISIC dataset

COVID-19

Given that the skin is the body's largest organ, it's natural that skin cancer is the most prevalent type of cancer in humans. Melanoma Skin Cancer (MSC) affects only 1% of all patients, according to the American Cancer Society (ACS), but it is associated with a higher death rate; it is similar to some COVID-19 survivors’ cases[1][2][3].

© The Author(s), 2021. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution, and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third-party material in this article are included in the article’s Creative Commons license unless indicated otherwise in a credit line to the mmaterial If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit HTPP creativecommons.org/licenses/by/4.0/.

It is a type of cancer that is spread by cells called melanocytes. Computer-based technology helps in diagnosing skin cancer infections in a more comfortable, less expensive, and faster manner[4][5]. Multiple noninvasive procedures are offered to analyze skin cancer symptoms and determine whether the patients have melanoma or not. The difficulty of classifying skin cancer using images has been vastly reduced after the deployment of the Deep Neural Network (DNN) models[6][7]. In this paper, a Deep Skin Cancer (DSC) model is proposed for detecting the visual characteristics of skin lesions. Knowledge Distillation (KD) is a form of model compression in which a small model is trained to resemble a larger model that has already been trained [8][9]. In this training stage, the giant model represents the teacher, and the little model represents the student. This training stage, sometimes referred to as the teacher-student[10][11]. The majority of DL models are computationally too expensive to run on mobile phones or embedded devices [12][13]. Because the KD model is a type of model compression, the student model is frequently smaller than the teacher model. The student normally learns significantly faster and more reliably as a result of the losses that have a greater effect on the performance of the DSC model as shown in Figure (1). The DSC model's main contribution could be summarized in the following points: (i) proposing a DNN model depends on KD techniques that are used to classify skin cancer infections. (ii) utilizes KD techniques for designing the proposed model, which results in an accurate model with high generalization ability. (iii) exploration of the effects of using different optimizers, including Adagrad, Adam, Adamax, Nagam, RMSprop, and SGD, that is totally guided by medical experts to ensure and validate the proposed model and the final results[14]. (iv) The proposed model was statistically compared with other optimization algorithms, which proves that our model is robust against overfitting[15]. This paper is organized as follows: Section 2 describes the methods. Section 3 explains the experimental. Lastly, Section 4 concludes the paper.

DL has had a lot of success in biomedical engineering recently. It reduces the need for feature engineering by learning and extracting meaningful features from raw data automatically. Many fields, particularly computer vision, have been transformed by DL. It has had a lot of success in biomedical engineering recently as shown in Table 1. While classifying skin lesions, Datta et al.[16] compare the performance of VGG(19), ResNet, InceptionResNetv2, and DenseNet architectures with and without the Soft-Attention technique. On the HAM10000 dataset, the original network outperforms the baseline by 4.7 percent while reaching a precision of 93.7 percent when combined with Soft-Attention. Mahboda et al.[17] created a baseline classifier as the reference model. Then, in both the training and test phases, we investigated the classification performances using either manually or automatically constructed segmentation masks in various settings. Hosny et al.[18] proposed a CAD system for skin lesions using the dataset ISIC2019. This dataset has a number of flaws, including unequal classes. The authors utilised a bootstrap weighted classifier with a multiclass SVM. This classifier modified the weights based on the image class. They provided GoogleNet a new class to train with, each with a varying quantity of unknown photos obtained from diverse sources. Hameed et al.[19] proposed a skin lesion categorization system based on a multiclass multilevel algorithm. Traditional machine learning and deep learning approaches were used with the proposed model. Hasan et al.[18] proposed DSNet, a skin lesions semantic segmentation network. They employed depth-wise separable convolution to minimize the number of parameters, resulting in a lightweight network.

Table 1

a general comparison of skin cancer-related works
Year	Authors	Model	Task	Metrics values
2021	Datta et al.[16]	Soft Attention	• Image Classification • Lesion Classification	• ACC=93.4% • AUC=0.984% • Precision=0.937%
2020	Nadipineni et al.[20]	Skin lesion classification using loss balancing	• Data Augmentation • Lesion Classification	N/A
2020	Mahbodet al.[17]	Classification Model with two path CNN model	• Skin Lesion Classification	N/A
2020	Hosny el al.[18]	Transfer learning model	• Skin Lesion Classification	• ACC=98.70% • AUC=95.6% • Precision=95.06%
2020	Hameed et al.[19]	K-means, transfer learning, Augmentation.	Classification of skin lesions	N/A
2020	Hasan el al.[21]	semantic skin lesion segmentation with parameters reducing	U-Net, DSNet, Augmentation	N/A 87.5 95.5

The transfer of features extracted from pre-trained models appears to be preferable in existing reports on skin cancer datasets. However, for skin cancer datasets, the trends in model compression, in which a larger pre-trained model is produced to allow a smaller model to learn complicated characteristics while minimizing computation and memory costs, have not yet been examined. A vast and complicated network or ensemble model, in particular, is first trained and pulls significant feature information from the given data, resulting in targeted predictions. This more complicated model is then used to train a small network. The little model can yield equivalent findings or mimic the outcomes of the larger model.

We propose a DSC model based on pre-trained models of the ImageNet dataset for medical imaging tasks. The model can overcome the challenge of a lack of training data, especially in the medical imaging domain. Knowledge is encoded and conveyed in the conventional KD model using the forms of softening class scores[22]. The student model's total training loss is provided by (1):

\({L}=\left(1-\propto \right){{L}}_{{C}{E}}\left({y},{\sigma }\left({{z}}_{{s}}\right)\right)+2{\alpha }{{T}}^{2}{{L}}_{{C}{E}}({\sigma }\left(\frac{{{z}}_{{s}}}{{T}}\right),{\sigma }\left(\frac{{{z}}_{{T}}}{{T}}\right))\)

where \({L}_{CE}\) represents the cross-entropy; \(y\) represents the one-hot vector of ground truths; \(\sigma\)is the soft-max function; \({z}_{s}\) and \({z}_{t}\) are the output logits of student and teacher models, respectively; \(\propto\) is a balancing hyperparameter, and \(T\) is the temperature hyper-parameter. The student model was trained utilizing the predictions of the teacher model as well as the ground truth hard labels, as illustrated in the standard KD from Fig. 1. However, it is widely accepted that reversing the KD operation will not considerably improve the teacher because the student model is incapable of learning and transferring relevant knowledge. In the training phase, the dataset is split into batches, which are created from the data. The input dataset with a size of (224,224) pixels begins with two standard convolutional layers that are applied insequential order. The first convolution has a kernel size of 2*2 and a filter number of 12823. The accuracy and cross-entropy of the DSC model are determined as follows. Accuracy refers to the number of cases in the overall dataset that were correctly identified; it's the true predictions that the model has made out of the overall predictions. Table 2 shows the primary values of model layers that are affected by batch normalization and layer flattening. Finally, completely connected layers with a single dropout layer between them are used. To finish the output, softmax by using multiclass SVM’s objective to train DSC. The lower layer weights are learned by back propagating the gradients from the SVM classifier26. To do this, we need to differentiate the SVM objective with respect to the activation of the penultimate layer. The training samples are iterated a number of times to discover the best results, using different optimization algorithms (Adamax, SGD, RMSprop, AdaGrad, Namax, and Adam) to detect distinct patterns among them for the skin cancer classification task. The DSC model consists mainly of a sequence of CNN layers that are organized to reduce the training time and solve the overfitting problem. 287 Table 1 illustrates the parameter values for each layer. The DSC model is based on a powerful DNN model that incorporates various innovative components to address a variety of issues, such as better feature extraction and overfitting. The accuracy and cross entropy of the DSC model are determined as follows. Accuracy refers to the number of cases in the overall dataset that were correctly identified; it's the true predictions that the model has made out of the overall predictions.

Figure (1): The proposed model (DSC) steps for skin cancer diagnosing

Utilizing a poorly-trained teacher model that has been trained on 50 initial epochs may also result in poorer results than using standard or reverse KD techniques. Finally, compared to all of the preceding ways, self-training the model may yield higher outcomes. When trained from itself with a 90% accuracy requirement, for example, the model would learn with a 10% error from its softened-class targets. To address these problems, we used the KD technique (Fig. 1) to conduct all trials related to the five key training strategies: standard KD teaching a teacher model to teach a student model; reversed KD training a student model to teach a teacher model; faulty KD poorly educating a teacher over the first 50 epochs to then teach a student model, and self-training KD training a model to teach itself. We chose six types of DNN models with identical input sizes to assess our proposed training methods in order to undertake all KD training approaches.

Table 2

The main DSC layers parameters values
Name of Layer	Filter Size (FS) and Stride (S)	Activation
Input layer	-	(224 ,224,3)
Convolutional layer1	FS= (2,2), S=1, Padding=same, filters=128	(224,224,64)
Convolutional layer2	FS= (2,2), S=1, Padding=same, filters=128	(224,224,64)
Max Pooling	PS= (2,2), S=2, Padding=valid	(112,112,64)
Convolutional layer3	FS= (2,2), S=1, Padding=same, filters=64	(112,112,32)
Convolutional layer3	FS= (2,2), S=1, Padding=same, filters=64	(112,112,32)
Batch normalization	Momentum=0.99, Epsilion=0.001, Axis=3	(112,112,32)
Max Pooling	PS= (2,2), S=2, Padding=valid	(56,56,32)
Convolutional layer3	FS= (2,2), S=1, Padding=same, filters=32	(56,56,32)
Convolutional layer3	FS= (2,2), S=1, Padding=same, filters=32	(56,56,32)
Max Pooling	PS= (2,2), S=2, Padding=valid	(28,28,32)
Flatten	-	(None,25088)
Dense	Units=2048, Activation=Linear	(None,2048)
Activation	Relu	(None,2048)
Dense	Units=1024, Activation=Linear	(None,1024)
Batch normalization	Momentum=0.99, Epsilion=0.001, Axis=1	(None,1024)
Activation	Relu	(None,1024)
Dropout	Rate=0.4	(None,1024)
Dense	Units=1024	(None,1024)
Activation	Relu	(None,1024)
Dense	Units=2, activation=softmax	(None,2)
Output		(None,8)

However, it is widely accepted that reversing the KD operation will not considerably improve the teacher because the student model is incapable of learning and transferring relevant knowledge. Utilizing a poorly-trained teacher model that has been trained on 50 initial epochs may also result in poorer results than using standard or reverse KD techniques. Finally, compared to all of the preceding ways, self-training the model may yield higher outcomes. We chose six types of DNN models to investigate our suggested training methods (ResNet-50 in order to conduct all KD training procedures feasible.

The results of our proposed method have ensured the effectiveness of it and the performance was compared with the performance of each pre-trained model individually.

4.1 Dataset

The ISIC dataset is used, which includes 2357 images of malignant and benign oncological diseases. All data were sorted according to the classification achieved with the ISIC dataset, and all subgroups were divided into the same number of images as shown in Figure 2, with the exception of melanomas and moles, whose images are slightly dominant (2). The illnesses included in the data collection include actinic keratosis, basal cell carcinoma, dermatofibroma, melanoma, nevus, pigmented benign keratosis, seborrheic keratosis, squamous cell carcinoma, and vascular lesion [23][24].

4.2 Evaluation of Results and Metrics

The DSC proposed model was developed and trained on the NVIDIA GeForce GTX 1080Ti. The DSC model is determined by the ten evaluation measures, which include accuracy, sensitivity, or True Positive Rate (TPR), specificity, or true negative rate (TNR), and precision. The sensitivity, specificity, precision, negative predictive value, false positive rate, false discovery rate, false negative rate, accuracy, F1-score, and Matthews Correlation Coefficient should be kept to a minimum to maintain a maximum value of fitness (F), and they should be evaluated using the proposed model Figure 1. The number of times the positive class is correctly classified (TP), the number of times the negative class is correctly classified (TN), the number of cases that forecasted the positives incorrectly (FP), and the number of cases that forecasted the negatives incorrectly (TN) (FN).

\(\text{S}\text{e}\text{n}\text{s}\text{i}\text{t}\text{i}\text{v}\text{i}\text{t}\text{y}\)\(= \frac{\text{T}\text{P} }{ (\text{T}\text{P} + \text{F}\text{N})}\)	\(\left(1\right)\)
\(\text{S}\text{p}\text{e}\text{c}\text{i}\text{f}\text{i}\text{c}\text{i}\text{t}\text{y}\)\(= \frac{\text{T}\text{N} }{(\text{F}\text{P} + \text{T}\text{N})}\)	\(\left(2\right)\)
\(\text{P}\text{r}\text{e}\text{c}\text{i}\text{s}\text{i}\text{o}\text{n}\)\(= \frac{\text{T}\text{P} }{\left(\text{T}\text{P} + \text{F}\text{P}\right) }\)	\(\left(3\right)\)
\(\text{N}\text{e}\text{g}\text{a}\text{t}\text{i}\text{v}\text{e} \text{P}\text{r}\text{e}\text{d}\text{i}\text{c}\text{t}\text{i}\text{v}\text{e} \text{V}\text{a}\text{l}\text{u}\text{e}\)\(\left(\text{N}\text{P}\text{V}\right) = \frac{\text{T}\text{N} }{(\text{T}\text{N} + \text{F}\text{N})}\)	\(\left(4\right)\)
\(\text{F}\text{a}\text{l}\text{s}\text{e} \text{P}\text{o}\text{s}\text{i}\text{t}\text{i}\text{v}\text{e} \text{R}\text{a}\text{t}\text{e}\)\(\left(\text{F}\text{P}\text{R}\right) =\frac{ \text{F}\text{P} }{ (\text{F}\text{P} + \text{T}\text{N})}\)	\(\left(5\right)\)
\(\text{F}\text{a}\text{l}\text{s}\text{e} \text{D}\text{i}\text{s}\text{c}\text{o}\text{v}\text{e}\text{r}\text{y} \text{R}\text{a}\text{t}\text{e}\)\(\left(\text{F}\text{D}\text{R}\right) = \frac{\text{F}\text{P}}{ (\text{F}\text{P} + \text{T}\text{P})}\)		\(\left(6\right)\)
\(\text{F}\text{a}\text{l}\text{s}\text{e} \text{N}\text{e}\text{g}\text{a}\text{t}\text{i}\text{v}\text{e} \text{R}\text{a}\text{t}\text{e}\)\(\left(\text{F}\text{N}\text{R}\right)= \frac{\text{F}\text{N} }{(\text{F}\text{N} + \text{T}\text{P})}\)		\(\left(7\right)\)
\(\text{A}\text{c}\text{c}\text{u}\text{r}\text{a}\text{c}\text{y} =\frac{TN+TP}{TP+FP+TN+FN}\)		\(\left(8\right)\)
\(\text{F}1-\text{s}\text{c}\text{o}\text{r}\text{e} =\frac{2\text{T}\text{P}}{2\text{T} \text{P} + \text{F} \text{P} + \text{F} \text{N}}\)		\(\left(9\right)\)
\(\text{M}\text{a}\text{t}\text{t}\text{h}\text{e}\text{w}\text{s} \text{C}\text{o}\text{r}\text{r}\text{e}\text{l}\text{a}\text{t}\text{i}\text{o}\text{n} \text{C}\text{o}\text{e}\text{f}\text{f}\text{i}\text{c}\text{i}\text{e}\text{n}\text{t}\)\(\left(\text{M}\text{C}\text{C}\right)\frac{\text{T}\text{P}\text{}\text{T}\text{N} - \text{F}\text{P}\text{}\text{F}\text{N} }{\text{s}\text{q}\text{r}\text{t}\left(\right(\text{T}\text{P}+\text{F}\text{P}\left)\text{}\right(\text{T}\text{P}+\text{F}\text{N}\left)\text{}\right(\text{T}\text{N}+\text{F}\text{P}\left)\text{*}\right(\text{T}\text{N}+\text{F}\text{N}\left)\right)}\)		\(\left(10\right)\)

The DSC model is based on the ResNet (50) model. It is also advised that some weights in the convolutional layers be allowed to be readjusted in order to adapt to the situation, and it has been applied to some optimization algorithms (Adagrad, Adam, adamax, Nadam, RMSprop, SGD). The learning rate has been modified in such a way that it automatically decreases because the summation of the previous gradient square is always kept on increasing after every time step. Figure 3 shows accuracy and cross entropy for correctly identified instances. Table 3 illustrates the general parameter values for the mentioned optimizer algorithms. The Adam optimizer with a learning rate of 0.05 and (64 x 64) image batches. Model evaluation is evaluated using accuracy and cross entropy for correctly identified instances, as shown in figure (4). The total (lr 0.05) and (64 x 64) image batches Each image batch requires 2K gradient updates. The Adamax optimizer achieves the best accuracy and loss metrics as shown in figure (5) with lr = 0.01, 1 = 0.9, and 2 = 0.999. Figure (6) shows loss and accuracy for the Nadam optimizer with lr = 0.01, 1 = 0.9, 2 = 0.999, and schedule decay = 0.004 RMSprop reaches a robust evaluation with lr = 0.01, 1 = 0.9, and Rho factor (the discounting factor for the coming gradient) = 0.9 as shown in Figure (7). The SGD optimizer attains great metrics as shown in Figure (8). Table 4 illustrates the all-metrics measures for all optimising algorithms with KD techniques.

Figure (4): (a) Model loss is dependent on Resnet (50); (b) Model average accuracy is dependent on Resnet (50) (Adam)

Figure (5): (a) Model loss is dependent on Resnet (50); (b) Model average accuracy is dependent on Resnet (50) (Adamax)

Figure (6): (a) Model loss is dependent on Resnet (50); (b) Model average accuracy is dependent on Resnet (50) (Nadam)

Figure (7): (a) Model loss is dependent on Resnet (50); (b) Model average accuracy is dependent on Resnet (50) (RMSprop)

Figure (8): (a) Model loss is dependent on Resnet (50); (b) Model average accuracy is dependent on Resnet (50) (SGD)

Table 3

The parameters values for common implementation optimizers
Optimizer	Parameter’s value
Adam	lr=0.01, β1= 0.9, β2 =0.999, Decay=0, AMSGrad=false
SGD	lr=0.01, Momentum=none, Decay=0, Nesterov=false
AdaGrad	lr=0.01, Epsilon=None,, Decay=0
Adamax	lr=0.01, β1= 0.9, β2 =0.999, Decay=0, Epsilon=None
Namax	lr=0.01, β1= 0.9, β2 =0.999, Decay=0, Schedule decay=0.004
Rmsprop	lr=0.01, β1= 0.9, epsilon=None,Rho=0.9, Decay=0

Table 4

The metrics evaluation values for optimizers.
AdaGrad Optimizer
Metrics		Sensitivity		Specificity		Precision		NPV		FPR		FDR		FNR	Accuracy		F-score		MCC
Values		0.9661		0.9686		0.9686		0.9661		0.0314		0.0314		0.0339	0.9673		0.9673		0.9347
SGD Optimizer
Metrics	Sensitivity		Specificity		Precision		NPV		FPR		FDR		FNR			Accuracy		F-score		MCC
Values	0.9915		0.9941		0.9941		0.9915		0.0059		0.0059		0.0085			0.9928		0.9928		0.9856
RMSprop Optimizer
Metrics	Sensitivity		Specificity		Precision		NPV		FPR		FDR		FNR			Accuracy		F-score		MCC
Values	0.9744		0.9620		0.9621		0.9744		0.0380		0.0379		0.0256			0.9682		0.9682		0.9364
Namax Optimizer
Metrics	Sensitivity		Specificity		Precision		NPV		FPR		FDR		FNR			Accuracy		F-score		MCC
value	0.9573		0.9620		0.9614		0.9580		0.0380		0.0386		0.0427s			0.9597		0.9594		0.9194
Adamax Optimizer
Metrics	Sensitivity		Specificity		Precision		NPV		FPR		FDR		FNR			Accuracy		F-score		MCC
value	0.9916		0.9957		0.9958		0.9915		0.0043		0.0042		0.0084			0.9936		0.9937		0.9873
Adam Optimizer
Metrics	Sensitivity		Specificity		Precision		NPV		FPR		FDR		FNR			Accuracy		F-score		MCC
value	0.9497		0.9451		0.9457		0.9492		0.0549		0.0543		0.0503			0.9474		0.9477		0.8948

Pre-trained models were shown to be suitable for perceiving skin cancer images as well as performing well in competitive knowledge distillation. To categorize eight skin cancer instances, knowledge is transferred from large, highly regularized models into smaller ones, as well as from the model into itself. First, despite our efforts to evaluate a comprehensive ISCI dataset, simulating the practical clinical challenges of handling over 2357 images, correctly visualizing and discriminating the 8 classes using a deep learning framework proved difficult when the database was unbalanced and poorly supervised. Our extensive experiments confirmed the utility of KD methods in the classification of skin cancer cases. Despite the fact that we established KD's superior performance in terms of categorization results. Without addressing the instance's relationship to the student models or the inference technique, the KD model collected instance features as distilled knowledge from specific layers of the teacher models. It's challenging for student models to directly suit all of the teacher's layer outputs. As a result, new KD designs are needed to help reduce intra-class variances while amplifying inter-class differences in the feature space, as well as to prevent major performance drops when both teachers and pupils have distinct architectures. Our experiments demonstrated the feasibility of implementing various KD training strategies, implying that the self-training KD method can improve the targeted models into which the distilled knowledge is transferred when selecting superior teachers is difficult or when computation resources are limited.

This paper proposes a DSC model using the concept of knowledge distillation and a variety of parameters to assess if a set of skin cancer images are any type of the eight types. The measurements show that the classifier is consistent. The ResNet (50) architecture consistently performs well, suggesting that it has perfectly generalized cancer classification in images. This architecture has a diagnostic accuracy of 0.9936 and works effectively. This architecture has the highest accuracy when compared to others. Despite its failure to correctly identify these images, the ResNet(50) architecture worked admirably. In the future, when the greatest number of high-resolution images have been obtained, this investigation will be conducted on a series of skin cancer images of patients.

DSC

Deep Skin Cancer

ISIC

International Skin Imaging Collaboration

Deep Learning

MSC

Melanoma Skin Cancer

ACS

American Cancer Society

DNN

Deep Neural Network ()

DSC

Deep Skin Cancer ()

Knowledge Distillation

SGD

Stochastic gradient descent

Availability of data and materials

The online version contains supplementary material available at https://www.kaggle.com/nodoubttome/skin-cancer9-classesisic.

Competing interests

The authors state no conflict of interest regarding the publication of this paper.

Funding

Science, Technology & Innovation Funding Authority (STDF)

Authors' contributions

Esraa Hassan, Noha A. Hikal and Mahmoud Y. Shams designed the research and performed the research; Samir Elmougy wrote the paper. All authors read and approved the final manuscript.

Acknowledgments

Authors gratefully acknowledge the valuable comments and suggestions of reviewers, which have improved the presentation.

[1] S. Elmuogy, N. A. Hikal, and E. Hassan, “An efficient technique for CT scan images classification of COVID-19,” vol. 40, pp. 5225–5238, 2021, doi: 10.3233/JIFS-201985.

[2] O. M. Elzeki, M. Shams, S. Sarhan, M. A. Elfattah, and A. E. Hassanien, “COVID-19: a new deep learning computer-aided model for classification,” PeerJ Comput. Sci., vol. 7, pp. 1–33, 2021, doi: 10.7717/peerj-cs.358.

[3] M. Y. Shams, S. H. Sarhan, and A. S. Tolba, “Adaptive deep learning vector quantisation for multimodal authentication,” J. Inf. Hiding Multimed. Signal Process., vol. 8, no. 3, pp. 702–722, 2017.

[4] L. Chen, G. Papandreou, S. Member, I. Kokkinos, K. Murphy, and A. L. Yuille, “DeepLab : Semantic Image Segmentation with Deep Convolutional Nets , Atrous Convolution , and Fully Connected CRFs,” pp. 1–14.

[5] H. Pham, Z. Dai, Q. Xie, M. Luong, and Q. V Le, “Meta Pseudo Labels,” 2012.

[6] H. Hirano, K. Koga, and K. Takemoto, “Vulnerability of deep neural networks for detecting COVID-19 cases from chest X-ray images to universal adversarial attacks,” PLoS One, vol. 15, no. 12, 2020, doi: 10.1371/journal.pone.0243963.

[7] “Deep learning for computer-aided medical diagnosis,” Multimed. Tools Appl., vol. 79, no. 21–22, pp. 15073–15073, 2020, doi: 10.1007/s11042-020-08940-4.

[8] K. Li, L. Yu, S. Wang, and P. A. Heng, “Towards cross-modality medical image segmentation with online mutual knowledge distillation,” AAAI 2020 - 34th AAAI Conf. Artif. Intell., pp. 775–783, 2020, doi: 10.1609/aaai.v34i01.5421.

[9] K. Gao et al., “Dual-branch combination network (DCN): towards accurate diagnosis and lesion segmentation of COVID-19 using CT images,” Med. Image Anal., vol. 67, p. 101836, 2020, doi: 10.1016/j.media.2020.101836.

[10] “Skin dataset.” .

[11] S. Motamed, P. Rogalla, and F. Khalvati, “RANDGAN: Randomized generative adversarial network for detection of COVID-19 in chest X-ray,” Sci. Rep., vol. 11, no. 1, pp. 1–10, 2021, doi: 10.1038/s41598-021-87994-2.

[12] Y. Kim, J. Park, M. Chang, J. Ryu, W. H. Lim, and S. Jung, “Influence of the Depth of the Convolutional Neural Networks on an Artificial Intelligence Model for Diagnosis of Orthognathic Surgery,” 2021.

[13] J. Qin, W. Pan, X. Xiang, Y. Tan, and G. Hou, “A biological image classification method based on improved CNN,” Ecol. Inform., vol. 58, no. January, p. 101093, 2020, doi: 10.1016/j.ecoinf.2020.101093.

[14] V. Noroozi, Y. Zhang, E. Bakhturina, and T. Kornuta, “A fast and robust BERT-based dialogue state tracker for schema-guided dialogue dataset,” CEUR Workshop Proc., vol. 2666, no. August, 2020.

[15] G. Van Horn et al., “The iNaturalist Species Classification and Detection Dataset,” 2017.

[16] S. K. Datta, M. A. Shaikh, S. N. Srihari, and M. Gao, “Soft Attention Improves Skin Cancer Classification Performance,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 12929 LNCS, pp. 13–23, 2021, doi: 10.1007/978-3-030-87444-5_2.

[17] A. Mahbod, P. Tschandl, G. Langs, and R. Ecker, “The Effects of Skin Lesion Segmentation on the Performance of Dermatoscopic Image Classification arXiv : 2008 . 12602v1 [ cs . CV ] 28 Aug 2020,” pp. 1–40, 2020.

[18] K. M. Hosny, M. A. Kassem, and M. M. Foaud, “Skin Cancer Classification using Deep Learning and Transfer Learning,” 2018 9th Cairo Int. Biomed. Eng. Conf. CIBEC 2018 - Proc., pp. 90–93, 2019, doi: 10.1109/CIBEC.2018.8641762.

[19] N. Hameed, A. M. Shabut, M. K. Ghosh, and M. A. Hossain, “Multi-class multi-level classification algorithm for skin lesions classification using machine learning techniques,” Expert Syst. Appl., vol. 141, p. 112961, 2020, doi: 10.1016/j.eswa.2019.112961.

[20] H. Nadipineni, “METHOD TO CLASSIFY SKIN LESIONS USING DERMOSCOPIC IMAGES,” 2020.

[21] R. Mart, “DSNet : Automatic Dermoscopic Skin Lesion Segmentation,” pp. 1–25.

[22] T. Kieu, K. Ho, and J. Gwak, “Utilizing Knowledge Distillation in Deep Learning for Classification of Chest X-ray Abnormalities,” 2020, doi: 10.1109/ACCESS.2020.3020802.

[23] J. Zhang, Y. Xie, Q. Wu, and Y. Xia, “Medical image classification using synergic deep learning,” Med. Image Anal., vol. 54, pp. 10–19, 2019, doi: 10.1016/j.media.2019.02.010.

[24] A. Sroka-oleksiak, D. Rymarczyk, A. Piekarczyk, and M. Brzychczy-wloch, “Deep learning approach to describe and classify fungi microscopic images,” 2020.

Download PDF

Version 1

posted

You are reading this latest preprint version

Deep Skin Cancer Model based on Knowledge Distillation Technique for Skin Cancer Classification

Status:

Version 1

Abstract

1. Introduction

2. Related Works

3. Methods

4. Experimental

4.1 Dataset

4.2 Evaluation of Results and Metrics

5. Discussion And Future Works

6. Conclusion

Abbreviations

Declarations

References

Status:

Version 1