Categorization of Common Pigmented Skin Lesions (CPSL) using Multi-Deep Features and Support Vector Machine

The skin is the main organ. It is approximately 8 pounds for the average adult. Our skin is a truly wonderful organ. It isolates us and shields our bodies from hazards. However, the skin is also vulnerable to damage and distracted from its original appearance: brown, black, or blue, or combinations of those colors, known as pigmented skin lesions. These common pigmented skin lesions (CPSL) are the leading factor of skin cancer, or can say these are the primary causes of skin cancer. In the healthcare sector, the categorization of CPSL is the main problem because of inaccurate outputs, overfitting, and higher computational costs. Hence, we proposed a classification model based on multi-deep feature and support vector machine (SVM) for the classification of CPSL. The proposed system comprises two phases: First, evaluate the 11 CNN model's performance in the deep feature extraction approach with SVM, and then, concatenate the top performed three CNN model's deep features and with the help of SVM to categorize the CPSL. In the second step, 8192 and 12,288 features are obtained by combining binary and triple networks of 4096 features from the top performed CNN model. These features are also given to the SVM classifiers. The SVM results are also evaluated with principal component analysis (PCA) algorithm to the combined feature of 8192 and 12,288. The highest results are obtained with 12,288 features. The experimentation results, the combination of the deep feature of Alexnet, VGG16 and VGG19, achieved the highest accuracy of 91.7% using SVM classifier. As a result, the results show that the proposed methods are a useful tool for CPSL classification.


Introduction
The largest organ in the human body is the skin. Various forms of inflammatory, bacterial, and viral infections spread throughout the skin, leading to several skin diseases and health problems [1]. "Melanoma is the most dangerous type of skin cancer, which has been one of the most important challenges of public health in recent years" [2].
According to the latest statistics, in 2018, 91,270 new melanoma cases are diagnosed and treated in the USA [1]. "Melanoma incidence rates and mortality from this disease are projected to increase over the next few decades" [3]. "The latest study shows that the diagnosis of new melanoma cases has risen by 53% annually from 2008 to 2018" [1,4]. If it is possible to detect this type of cancer at its early stage by selecting the right treatment [5,6], survival rates are very promising [7]. Otherwise, a patient's estimated 5-year survival rate would be decreased from 99 to 14% [8][9][10]. There was also a drastic rise in the diagnosis of new cases between 1994 and 2014, up to 77%, in the non-melanoma form of cancer. "The most common form of non-melanoma skin cancer is basal cell carcinoma, resulting in 3000 deaths per year" [11]. Therefore, this will give rise to a strong demand for primary inspection and identification of various skin cancer forms to prevent it from getting worse and provide a better prognosis [3]. The normal way to diagnose melanoma is by examining the low-precision visual information [12,13]. "Dermoscopy, on the other hand, is a non-invasive technique that can capture a high-resolution skin image that helps dermatologists to identify features invisible to the naked eye" [14].
Several meta-analyses have shown that melanoma diagnosis is improved and reliable in naked eye examination using dermoscopy [15][16][17]. Even for experienced clinicians, the manual approach is challenging and subjective [18,22,23]. Due to the similarity of visual characteristics between malignant tumors of the skin and benign lesions, dermatologists face difficulty distinguishing between them [19,21]. Even if highly trained dermatologists and physicians have the average sensitivity for recognition of melanoma, it is less than 80% [24,25]. Again, highly trained dermatologists are not available all over the world. Therefore, automatic skin lesion classification at the same level and accuracy as dermatologists or even higher is crucial in public health.
To solve the aforementioned issues, there have been plenty of publications on exploring methods to classify skin cancers by means of computer assistance. These algorithms use various approaches to the segmentation, detection, and classification of melanoma by integrating areas such as image processing, computer vision, and machine learning [20,[26][27][28][29]. "The main problem of this type of scientific paper is the scarcity of data. However, skin cancer has been categorized into different stages according to 'man-made' methods, which is a major flaw in the classification method" [30][31][32][33].
In most current studies, the three main skin disorders, including seborrheic keratosis (SK), squamous cell (SC), and melanoma, are considered. In the last few decades, several researchers have developed various approaches to diagnose skin diseases. "In addition, natural computing system, region of interest (ROI), gray level co-occurrence matrix (GLCM), fractal-based regional texture analysis (FRTA) feature extraction methods, various machine learning algorithms such as Artificial Neural Network (ANN), Convolutional Neural Network (CNN), Generative Adversarial Networks (GANs) with Support Vector Machine (SVM), Naive Bayes were implemented" [34][35][36]. These approaches have produced improved classification accuracy in terms of skin disease detection. Still, they have few deficiencies in higher-dimensional space computational complexity, more processing time, overfitting problems, a noisy environment, and so on. To overcome this, we have proposed a classification model that uses multi-deep features and SVM to categorize CPSL. The main contribution of the article is summarized as follows.
• The dataset images were collected from the HAM-10000 dataset belongs to seven classes of CPSL such as melanoma (mel), melanocytic nevi (nv), basal cell carcinoma (bcc), actinic keratosis and carcinoma benign keratosis (bkl), dermato fibroma (df), and vascular lesions (vasc). • The performance of 11 CNN models is evaluated to classify seven CPSL based on their deep feature and SVM. • High-dimensional features such as 8192 and 12,288 are obtained by combining binary and triplet networks of three top-performing CNN architectures and feed to the SVM for classification. • The performance is also evaluated by introducing dimensional reduction techniques like PCA. • The LR + is 49, which is greater than 10, and implicates the proposed method has a significant contribution to the diagnosis and an excellent diagnostic test. • Finally, the AUC value of the proposed method is 0.99, which is within the range of 0.9 to 1. It reveals an excellent diagnostic approach as per STARD (Standards for Reporting of Diagnostic Accuracy Studies) guidelines.
The remaining of the manuscript is organized as follows. "Related Works" discusses the related works. "About Dataset" details the dataset. The methodology is presented in "Methodology". Experimental results of the proposed method are given in "Experimental Analysis and Results". Finally, "Conclusion" shows the discussion and conclusion section where the results are discussed by comparing with the previous methods.

Related Works
Many approaches are reported toward the classification of different skin cancer lesions or CPSL, especially in machine learning. The integrated computer-aided mechanism [35] was proposed for the detection of skin diseases. They analyze the shape, color, texture, and irregular border of the skin lesion to create various quantitative features before classification. Textural information is quantified through a fractalbased regional texture analysis (FRTA) and a gray-level co-occurrence matrix (GLCM). The radial base function (RBF) and the support vector machine (SVM) evaluate the classification's efficiency. The experimental study delivers 99.65%, 97.54%, and 98.99% for BCC, dysplastic nevi, and melanoma classification, but the characteristic dimensional space is higher. The computer-aided classification system [36] was implemented to diagnose suspected pigmented lesions. Pre-selected single-lesion images train computeraided diagnostic tools for skin lesions. One hundred thirtythree patients with a multitude of skin lesions were selected for this study. The board-certified dermatologist investigates all lesions that are classified as normal or abnormal. The sensitivity of 100 percent of suspected pigmented lesions is achieved on the basis of the test set. At the population level, this computer-aided classification method is very useful for skin screening.
Dynamic graphics cut algorithm (DGCA) and naive Bayes (NB), a classifiers are is proposed for the segmentation and classification of skin diseases [37]. Experimental datasets are taken from the International Skin Imaging Partnership (ISIC 207) [44], which shows better results compared to state-of-the-art approaches such as SegNet and FCN. Here, there are three kinds of cancers, such as keratosis, melanoma, and benign cases. The detailed analysis reveals 92.9%, 91.2%, and 94.3% for keratosis, melanoma, and benign disease classification. It was proposed that the deep convolutional neural network (DCNN) classifier should be used to identify skin diseases [38]. DenseNet-201, InceptionResNet-v2, ResNet-50, and Inception-v3 are used in the convolutional neural network classifier. The full-resolution convolutional network (FrCN) segments the skin lesion boundary of the entire dermoscopy image. The ISIC 2016 [27], ISIC 2017 [44], and ISIC 2018 [45] datasets are used to evaluate the performance of convolutional neural network classifiers in the case of skin disease classification.
Multi-label classification techniques are proposed [39] to identify skin disorders such as SK, BCC, nevus, and melanoma. Cross-spectrum and cross-correlation approaches for the extraction of features have been adopted. The extraction of both spectral and spatial features is clarified by suitable kernel patches and related visual impacts. Classifications of epidermal, benign melanocytic, and benign melanocytic skin lesions are performed using a multi-label multi-class skin lesion classification mechanism. Generative adversarial networks (GANs) have been developed to identify skin lesions [40]. The GAN-based data increase process improves the efficiency of the skin lesion classification. High-quality skin lesion images are synthesized by changing both the discriminator and the generator. Quantitative assessment measures such as recall, accuracy, inception distance (FID), and inception score (IS) assess GAN-based skin lesion's efficiency. The experimental analysis, 96.6% average precision, 74.3% specificity, 83.2% sensitivity, and 95.2% accuracy are obtained. Present methodologies used in the classification of skin diseases are illustrated in Table 1.

About Dataset
We considered the image classification task of the most common lesions that should be distinguished in a skin cancer screening setting. About 90% of lesions found in the routine skin examination are covered by the following seven skin lesion classes: (1) actinic keratoses and intraepithelial carcinoma/Bowen's disease (akiec), (2) basal cell carcinoma (bcc), (3) benign keratosis-like lesions (solar lentigines/seborrheic keratoses and lichen planus like keratoses, (bkl), (4) dermatofibroma (df), (5) melanoma (mel), (6) melanocytic nevi (nv), and (7) vascular lesions (angiomas, angiokeratomas, pyogenic granulomas and hemorrhage (vasc). All images were obtained from the HAM10000 Dataset [41]. This dataset contains 10,015 dermatoscopic images, which can serve as a training set for academic machine learning purposes. There are different dermatoscopy dataset publicly available, illustrated in Table 2. In this study, the HAM10000 dataset is taken into consideration because (1) it is the only dataset which contains seven kinds of CPSL, (2) all seven kinds of CPSL have more than 100 images, and (3) 53.3% of images are pathologically verified.
To make the dataset balanced across all classes, we have horizontally and vertically flipped the images of df, akiec, and vasc class, and chosen a subset of images from bkl, mel, and nv class. The dataset distribution of CPSL is taken in this research is given in Table 3. Figure 1 illustrates the sample images of seven kinds of CPSL.

Methodology
The proposed method comprises two phases: First, the 11CNN models are evaluated for classification of CPSL based on the feature extraction approach. Here, the features are extracted from fully connected layer and feed to the SVM for classification. The linear SVM is used for classification. The linear SVM has great potential toward image classification. To train the SVM, the function "fit class errorcorrecting output codes (fitcecoc)" is used. This function returns fully trained multi-class error-correcting output of the model. The function "fitcecoc" uses K(K − 1)/2, binary SVM model, using one-vs-all coding design. When SVM is used as a classifier in CNN models, it increases accuracy  [42,43]. Eleven different CNN models are evaluated their performance for CPSL classification. Then, the top performed three CNN models are taken to combine their features to enhance the performance. The performance evaluation model of CNN for the classification of CPSL is shown in Fig. 2. In the second phase, the features taken from three top performed models are combined and applied to the same SVM classifiers. In the last step of the experiment, features are given to the classifiers after PCA. The architectural design of the proposed method is shown in Fig. 3. The strategy for evaluating all classification models in this study is based on a training and testing ratio of 80:20.

Experimental Analysis and Results
Evaluating a method is important in building a useful machine learning model. There are many performance criteria used to test the performance of the classification. The most commonly used classification evaluation metric is "accuracy." However, this criterion alone is not always correct and can be misleading in some cases. Therefore, it will be useful to provide an evaluation by looking at other criteria that support accuracy. The metrics we use to choose the top performed CNN model in this study are accuracy, sensitivity, specificity, precision, FPR, and F1 score. Table 4 shows that the Alexnet, VGG16, and VGG19 are the three top performed CNN model for classification of CPSL. The deep features of these three top performed models are now combined and feed to SVM for classification. The results are recorded in Table 5 in terms of Accuracy and AUC. The classification model is also evaluated with the employment of PCA, and corresponding results are recorded in Table 6. Tables 5 and 6 show the performance of classification models is degraded with the employment of PCA. In the combination of deep features of binary and triplet CNN architectures, the performance of classification models is increased compared to an individual. Among the combined deep feature, the effectiveness of the deep features of triplet CNN architecture is more than the deep feature of binary CNN architectures. The confusion matrix and ROC plot of classification models, i.e., based on triplet deep feature and SVM, are presented in Figs. 4 and 5, respectively. =49. So, as per STARD [46], the proposed method is a good diagnostic test. In addition, the AUC value of the proposed method is 0.99, which is within the range of 0.9 to 1.0. It reveals an excellent diagnostic approach as per STARD (Standards for Reporting of Diagnostic Accuracy Studies) guidelines [46]. Further, Table 7   positive rates (TPR) and positive predictive values (PPV) value of seven classes of CPSL. Table 7 shows that the TPR or sensitivity of akiec is 98%, bcc is 100%, bkl is 63%, mel is 100%, nv is 66%, df is 79%, and vasc is 100%. The TPR values of classes are more than 60%, and the PPV values are more than 70%. Hence, it indicates the proposed methodology is going on a positive approach toward the classification of CPSL.  Further, no research paper is published to classify seven kinds of CPSL, as illustrated in Table 8. Table 8 shows the proposed methodology is the only method that deals with seven kinds of CPSL. Again, even if more kinds of CPSL are considered, the proposed method achieved better performance than the existing work. Also, the sensitivity of each class of CPSL is competent enough with the existing work.
The method achieved diagnostic effectiveness (accuracy) of 91.7%, AUC is 0.99, F1 score is 91.8% and LR + is 49. As per STARD guidelines, the proposed diagnosis method is excellent (as AUC is within the range of 0.9 to 1) and an excellent diagnostic test (as LR + > 10). In terms of definitely solving skin cancer issues for the classification of CPSL, the findings obtained signify that we are in the right direction.
Funding Any organization or institution does not support this research.
Data Availability Statement Data sharing does not apply to this article, as no new data were created or analyzed in this study.

Declarations
Ethical Approval This article does not contain any studies with human participants or animals performed by any authors.