Automatic Diagnosis of Multiple Lung Diseases in Chest Radiographs Based on Ensemble CNN Models

doi:10.21203/rs.3.rs-2235065/v1

Download PDF

Research Article

Automatic Diagnosis of Multiple Lung Diseases in Chest Radiographs Based on Ensemble CNN Models

https://doi.org/10.21203/rs.3.rs-2235065/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Purpose

Lung diseases are one of the most common causes of death worldwide. These diseases can be diagnosed accurately and early with the help of Chest X-ray Radiographic (CXR) imaging. CXR images carry a significant amount of anatomical and physiological information. Still, the potential for error is high even for an experienced radiologist to identify the information about a particular disease because of the similarity with other lung conditions.

Methods

To address this issue, this paper proposes an effective method for supporting medical diagnosis and facilitating the decision-making process to detect multiple lung diseases from chest X-ray images using an ensemble approach. In this context, the stacking method aggregates that three Convolutional Neural Network (CNN) models that are MobileNetV3, EfficientNetV2B0, and ResNet50V2, chosen for the training using a new combined dataset. The new combined dataset involves three publicly available datasets, including PA chest radiography images, NIH, and TBX11K. The proposed models are trained and tested to classify four and five lung disease cases: pneumonia, COVID-19, pneumothorax, atelectasis, and normal.

Results

The outstanding performance was obtained on the testing set of four classes represented by a sensitivity of 97.25%, specificity of 99.15%, and accuracy of 98.77%, and for five classes, the sensitivity of 91.74%, specificity of 97.88%, and accuracy of 96.68%.

Conclusion

Using the proposed stacking method that combines the strengths of multiple models and pre-trained weights, the results of the stack-2 model show good classification accuracy in classifying multi-category lung diseases and can be used as a reliable automated diagnostic tool to assist radiologists in the accuracy and speed of their decision-making.

CNN

X-ray

EfficientNetV2B0

Deep learning

Transfer learning

Ensemble stacking

One of the leading causes of death across the globe is lung diseases, which encompasses a wide range of conditions such as pneumonia, COVID-19, pneumothorax, atelectasis, and others. For example, pneumonia is a lung infection that affects the air sacs; some symptoms include fever, cough, shortness of breath, loss of appetite, tingling chest pain, low energy, vomiting, and confusion. COVID-19, on the other hand, causes fever, cough, shortness of breath, and pneumonia, affecting the human heart, brain, liver, and a variety of other organs. This disease must be identified and treated without delay. Primarily, its detection relies on real-time reverse transcription-polymerase chain reaction (PCR), a time-consuming technique [1]. Pneumothoraxes are accumulations of air that are not inside the lungs but rather in the pleural cavity. Air gathers within the chest, between the parietal pleura and the visceral pleura, causing the condition. The lung may experience pressure from the air buildup, which could cause it to collapse. Pneumothoraces can also be divided into simple, tension, and open categories [2]. One of the most common breathing problems after surgery is called atelectasis. It can also result from other breathing problems, like cystic fibrosis, lung tumors, chest injuries, lung fluid, or weak breathing. Atelectasis happens when the tiny air sacs in the lungs (called alveoli) lose air or fill up with alveolar fluid. Atelectasis can make breathing hard, especially if a patient already has lung disease. Atelectasis can be caused by many different things, like a mucus plug, a foreign body, a tumor in the airway, chest trauma, pleural effusion, pneumonia, pneumothorax, or scarring of the lung tissue [3]. Although early symptoms are challenging to identify and may be missed by humans, human experts' diagnosis throughput is inferior to that of machines [4]. Modern medical imaging modalities that use CXR and CT are extremely potent tools that help doctors diagnose various conditions. These diagnostic tools enable radiologists to see inside the body without cutting it, whereas a professional radiologist currently identifies lung disease by reviewing CXR images. Additionally, x-ray imaging exposes the body to significantly less harmful radiation than CT and is faster, cheaper, and more accessible [5]. In addition, it is appropriate for follow-up examinations because variations in disease can be detected more easily and sooner. Nevertheless, because of the intricate anatomical structure of the chest, there is a common human error that can be caused by misreading a CXR image. This error is quite common. As a consequence of this, computer-aided diagnostic systems, also known as CADS, are utilized to assist radiologists in order to assist them in making better clinical decisions and reducing misreadings. The detection of lung disease on time with a high classification accuracy while using minimal data continues to be challenging. The most significant Deep Learning (DL) model for 2D image classification is the convolutional neural network, also known as CNN, which can learn and distinguish more precisely on a three-channel image than on a one-channel image. In this paper, a deep learning method is proposed that employs a transfer learning approach to ensemble two and three architectures to classify four and five infections from lung diseases using CXR images.

In the past few years, there has been much interest in how deep neural networks can be used to diagnose diseases, so that chest X-rays can be looked into and evaluated for lung infections. Deep learning algorithms like the CNN network are used to derive good-quality and deep features from images. These features are inputs to classifiers (such as SoftMax) to classify the images. This section presents the latest research on multiple lung disease diagnoses using chest X-rays. Yimer et al.[6] used the Xception deep learning method to develop a method for automatic classification of multiple lung diseases (tuberculosis, pneumonia, COPD, pneumothorax, lung cancer, and normal) from chest X-ray images. The dataset was obtained from the in-house and NIH chest X-ray dataset repository. Authors claimed that their method achieved an accuracy, sensitivity, and specificity of 97.3%, 97.2%, and 99.4%, respectively. Gupta et al. [7] proposed the InstaCovNet-19 that is integrated and stacked deep convolution network. The proposed model employs ResNet101, Xception, InceptionV3, MobileNet, and NASNet to compensate for a modest amount of training data. The model detects three classes based on X-ray images. Their model achieved 99.08% accuracy across three types (COVID-19, Pneumonia, and Normal). Rohit et al. [7] proposed the ET-NET framework, which uses a bagging ensemble of three transfer learning models (Inception v3, ResNet34, and DenseNet201) to improve the performance of the individual models. When tested on a publicly available dataset, it achieved 97.81% accuracy, 97.77 % pecision, 97.81% sensitivity, and 97.77 % secificity. Enes et al. [8] used chest X-ray images to develop ensemble CNN models to diagnose pneumonia in children. The ensemble models consist of seven CNN models (VGG-16, VGG-19, ResNet-50, Inception-V3, Xception, MobileNet, and SqueezeNet). With a precision of 95.21, a sensitivity of 97.76%, and an accuracy of 90.71%, it gave important results in chest X-rays as normal, viral pneumonia, and bacterial pneumonia. In order to classify four disease categories (normal, bacterial, viral, and COVID-19) using X-ray images, Fareed et al. [9] proposed a hybrid ensemble deep CNN model based on MobileNet and InceptionV3. The proposed model attained the highest F1-Score of 92.97% and classification accuracy of 96.49%. Li D and Li S [10] developed a deep learning method for Covid-19 pneumonia diagnosis to identify 10,182 chest X-ray images of normal, bacterial pneumonia, and viral pneumonia (Covid-19 and non-Covid-19). It has a diagnostic accuracy of 99.95% for viral pneumonia. Covid-19 pneumonia can be distinguished from bacterial pneumonia or normal lung images with high accuracy of 99.85%. Soufiane et al. [11] developed a system for detecting COVID-19 in chest X-ray images using a stacking technique that combines CNN models with transfer learning and the KNN algorithm. The employed data set includes four classes: COVID-19, tuberculosis, viral pneumonia, and normal cases. They were gathered from six sets of X-ray image data. The proposed system achieves a 99.23% accuracy rate, which is considered extremely high rate. On two TB CXR image datasets of ensemble CNN models (InceptionV3 and Xception) for TB detection, Erdal et al. [12] propose Bayesian weighted voting and the average of probabilities as a combination of rule methods. In addition to using the CLAHE technique and data augmentation as preprocessing, according to computational results, 97.50% and 97.69% accuracy rates are achieved by the proposed method on the Montgomery and Shenzhen datasets, respectively. Three different pre-trained models named EfficientNetB1, NASNetMobile, and MobileNetV2 were used to classify four classes, including COVID-19 using a deep learning-based technique proposed by Ejaz et al. [13] The augmented dataset is used to train deep learning models. In contrast, two distinct training strategies are used for classification. The proposed method accurately categorizes four classes, including COVID-19, viral pneumonia, lung opacity, and normal, with a 96.13% success rate. A. Kaur and P. Bhardwaj [14] proposed a deep-learning ensemble of four CNN models (Inceptionv3, DenseNet121, Xception, and InceptionResNetv2) to classify three classes using chest X-ray images collected from online resources, including 2,161 COVID-19, 2,022 pneumonia, and 5,863 normal images. They employed contrast enhancement and data enhancement techniques. The experimental results demonstrate that the proposed model has an accuracy of 98.33% for the binary class and 92.363% for the multiclass. To develop a diagnosis support system for COVID-19, Nigam B, Nigam A, Jain R, et al. [15] used a variety of deep learning architectures, including VGG16, DenseNet121, Xception, NASNet, and EfficientNet. The dataset that was used consists of three categories, and EfficientNet was able to achieve the highest accuracy, which was 93.48%. In another study published in [17], Jabra M, Koubaa A, Benjdira B, et al. proposed a diagnostic system based on the majority voting method and was determined by the results given by different classifiers. The classifiers were MobileNetV2, ResNet50V2, ResNet50V1, and ResNet11. This system was developed using an X-ray image dataset that included three classes: COVID-19, viral pneumonia, and normal. The maximum level of accuracy achieved by this model was 99.31%. M. Shorfuzzaman and M. Masud [16] developed a deep learning model based on pretrained models and used majority voting. To construct these models, they utilized open-source chest X-ray images of healthy, pneumonia, and COVID-19 lung cases. An accuracy of 99.26% was achieved using the suggested model.. M. Loey, S. El-Sappagh, and S. Mirjalili [17]. Create a new CNN model using Bayesian optimization to classify COVID-19, pneumonia, and normal chest X-ray images. The proposed model comprises two main parts: one that extracts features and another that tunes hyperparameters. The model had a 96 % accuracy rate. Ibrahim D, Elsennawy N, and Sarhan A [18]. Using a combination of chest x-ray and CT images, a multi-classification deep learning model is proposed for the diagnosis of COVID-19, pneumonia, and lung cancer. Using these X-ray and CT-scan images will increase the size of the data set, which will increase classification accuracy, as the authors claim. Results from four different architectures (VGG19-CNN, ResNet152V2, ResNet152V2 + Gated Recurrent Unit, and ResNet152V2 + Bidirectional GRU) are compared. With the input of X-ray and CT scan images, the VGG19 + CNN model achieved the highest accuracy of 98.05%. In this paper, the contributions are:

Development of six unique CNN model and the ensemble method to stack two and three most promising performance models in two stages with transfer learning techniques to improve the performance of multiple lung infection classifications.
Creation of two new datasets merged and preprocessed from the three previously and publicly available datasets (PA chest radiography images, NIH, and TBX11K) to be used in this work. The first dataset has four lung disease categories including pneumonia, COVID-19, pneumothorax, and normal. The second dataset has five categories, which is similar to the previous dataset with the addition of atelectasis symptoms.

The rest of the paper is structured as follows: The third section describes how the dataset was combined, split, and preprocessed. In addition, the clarification of the proposed methods and training process is in detail. Section 4 presents performance measures and experimental results for the proposed methods. Finally, Sections 5 and 6 describe the work's discussion, conclusion, and future research.

This section presents the benchmark datasets for four diseases: atelectasis, COVID-19, pneumonia, and pneumothorax, in addition to the normal cases. Next, the top layers of six pretrained models are modified so that the new modified models will be capable to classify with a higher level of accuracy. Finally, the performance metrics were written and plotted on a graph, making it much simpler to analyze the symptoms and establish the issue. Figure 1 illustrates four different lung diseases that were considered for this research.

4.1 Dataset

The images that were used in this research came from three different datasets that were available to the general public: PA Chest Radiography Images (X-Rays) v3 [19], NIH [20], and TBX11K [21]. The curated dataset provided the basis for the selection of the COVID-19 disease. In addition, bacterial and viral pneumonia have been lumped together under the umbrella term "pneumonia" to form its classification. After that, the NIH and TBX11K datasets were used to derive the normal class, whereas classes of pneumothorax were taken from the NIH dataset. For the first dataset, which we referred to as "CXR4" for brevity's sake, lung atelectasis was added to the first four categories, making a new set of five categories, which we abbreviated "CXR5". Figures 2 (a) and 2 (b) show the relevant information about the image distribution of the CXR4 and CXR5 datasets.

4.1 Dataset Preprocessing

Original dataset includes images of a wide range of file types and dimensions. To reduce the time spent loading images into the model and ensure that the model's requirements are met, the Area interpolation technique for scaling down images was used in the dataset so that they would fit into a standard square shape with dimensions of 224 by 224 before they were loaded. After that, it was converted from grayscale to three-channel images by repeatedly use the data three times (224×224×3), which was then saved in PNG format and split into three sets. The training set comprised 70%, while the validation and testing sets comprised 14% and 16%, respectively. It was necessary to move some of the images from the test set to the other sets before calculating the ratio of test images between COVID-19 and the other classes. The preliminary processing steps are shown in Fig. 3.

4.1 The Proposed Approach

As the significance of training efficiency for classification problems has increased, model and training data volumes have expanded, particularly when more classes are involved in the classification process. Since transfer learning makes it possible to store and apply the knowledge acquired from a previously-trained CNN to solve a new problem. Therefore, this technique is adopted here to improve the ability of target models to generalize. During the transfer learning process, all pre-trained parameters are copied without the header layers. Some parameters, like colors, curves, patterns, corners, and edges, are shown in the ImageNet dataset.

The proposed approach adopts the transfer learning technique to exploit several single-handedly models in the first phase including MobileNetV3 Large [22], NASNetMobile [23], EfficientNetV2b0 [24], InceptionV3 [25], ResNet50V2 [26], and VGG16 [27], as the backbone network for four lung disease classes classification. CXR4 dataset is used to train the models and analyze their performance. In addition, only three models that have best performance results among the six models were selected and implemented for classifying five classes based on the CXR5 dataset. In the second phase, the ensemble method (stacking) is adopted in two different scenarios to improve the performance. Ensemble learning enables the fusion of key attributes from the base classifiers. These types of models are reliable because the process of computing the ensemble model brings the predictions of the base models closer together, thereby reducing their spread (also known as their dispersion). This leads to a decrease in the spread of prediction errors and the collection of extra data. The first ensemble model is based on two fast models (MobileNetV3 Large and EfficientNetV2b0) to classify four classes utilizing the CX4 dataset. The second ensemble model is based on the best and fastest three models (MobileNetV3 Large, EfficientNetV2b0, and Res-Net50V2) that are trained in the first stage to improve the performance. Figure 4 depicts the developed architecture for the proposed individually implemented models and the two proposed ensemble models that encapsulate several stacked high-performance models.

The publicly available datasets suffer from unbalanced data in which each class had an unbalanced number of images. Assigning weights to the classes assisted in balancing the data samples by giving small weights to large samples and large weights to small samples [28]. The weight of each class in the training set is estimated using the scikit-learn library (See Fig. 5). In addition, several call-back functions are used, such as ReduceLROnPlateau to slow down learning when metric stops getting better, ModelCheckpoint to save the best weight, and EarlyStopping to stop training if the model is overfitting and bring back the best network weights saved.

The MobileNetV3 Large, NASNetMobile, EfficientNetV2b0, InceptionV3, ResNet50V2, and VGG16 models and their learned weights were used in the first training stage. MobileNetV3 Large, EfficientNetV2b0, and ResNet50V2 were chosen as the base models to build stack models due to their training speed and efficient parameter-utilization to classify four categories using the CXR4 dataset. Moreover, the ensemble method, where two fast models are stacked, was used as shown in Fig. 4. In the second stage of training, three models (MobileNetV3 Large, EfficientNetV2, and ResNet50V2) are chosen to classify four and five disease categories using the CXR4 and CXR5 datasets, respectively, as illustrated in Fig. 4 (c). As there are 1000 classes in the ImageNet dataset, the head layers of the proposed models were used to classify the 1000 classes. In the CXR4 dataset, however, the target classes were four (normal, COVID-19, pneumonia, and pneumothorax) and five in the CXR5 dataset (normal, atelectasis, COVID-19, pneumonia, and pneumothorax).

The training process in the first stage for the individual models using the CXR4 dataset is started by obtaining the features of the base model bottleneck. These features are passed then to the preprocess function to rescale the image according to the requirements of each model. The preprocess input function for the VGG16 will convert the input images from RGB to BGR. Next, zero-center each color channel concerning the ImageNet dataset without scaling. While the preprocess function scales input pixels between (-1 and 1) in ResNet50V2, InceptionV3, and NASNet, it is part of the model, as a rescaling layer, in Efficient-NetV2 and MobileNetV3. The based model is frozen to make the convolutional-base untrainable and keep the trained weights. After that a global average pooling 2D (GAP2d) layer is added to the classification task to aggregate all of the features and reduce overfitting by reducing the total number of parameters. Next, the new head layers were trained with two dense layers of 1024 units, including normal kernel initializer weight and LeakyRELU activation functions. The last layer, dense output layer, comprises four and five output units to classify the images into four and five target classes using the CXR4 and CXR5 datasets, respectively. A softmax activation function is used to predict each class's probability, as shown in Fig. 4. (a).

The use of hard and soft labeling at a rate of (0.1) was also tested using the MobileNetV3 model, which can result in a more generalizable model. After that, to improve the performance of diagnosing multi-class lung disease, the second phase of the training process is consisted by implementing the stack for only two models on the CXR4 dataset. The proposed method is called "stack-1" as a short name, and it was similar to the training of individual architectures except that the GAP2d layers are passed to the aggregation layers, as shown in Fig. 4 (b). The output is then into two dense layers of 1024 units before classifying classes in the last layer. Similarly, the implementation of the three-stack models was carried out using the CXR5 dataset, where the architecture was asymptotic to stack-1 except that both the outputs of the GAP2d layers and dense layers were fed to the aggregation layers. The outputs of GAP2d were combined separately for each of MobileNetV3 and Efficient-NetV2, in addition to ResNet50V2 and EfficientNetV2. Subsequently, the output of each aggregation layer passes to two dense layers of 1024 nodes for training. Figure 4 (c) shows how the output features are combined in the last aggregate layer so that the output layer can be classified into four or five classes based on the dataset used. This second proposed staking method is called "stack-2" as an abbreviation. Table 1 shows the trainable parameters that the proposed individual and stack models produce.

Table 1

Trainable parameters of the proposed individual and stack models for the four and five classifications
Model name	Trainable parameters for the four classes	Trainable parameters for the five classes
MobileNetV3 Large	2,037,764	2,038,789
NASNetMobile	2,136,068	–
EfficientNetV2 B0	2,365,444	2,369,029
InceptionV3	3,151,876	–
ResNet50V2	3,151,876	3,151,901
VGG16	1,579,012	–
Stack-1	3,352,964	–
Stack-2	7,819,652	7,821,701

A powerful optimizer, Ranger optimizer [29], is utilized to adapt the learning rate coefficient. It includes both LookAhead[30] and RAdam[31], [32] optimizers. This embedding helped the RAdam optimizer escape from local minima and reduce loss variance. In addition, the batch size was set to 32, the start learning rate (step size) was 0.01, and the warmup proportion was set to 0.1%.

4.1 Test platform

The work is carried out using the free Google Colab Notebook equipped with the GPU run-time type and a personal computer equipped with a Core i7 processor. Every piece of software and library used is written in Python, a free and open-source programming language. A few examples of these libraries include data preprocessing algorithms, NumPy, Scikit-learn, Computer Vision, and the TensorFlow/Keras deep learning toolbox.

4.2 Results

The proposed models were trained first using the training set and then validated using the validation set. On the other hand, the test set, which had images that weren't used during the training phase, was used to test the model's accuracy and how well it worked outside of its training set. As a consequence of this, several metrics from the testing set were used in order to evaluate how well the new model performed. These metrics included sensitivity (Sen) or recall (Rec), specificity (Spe), accuracy (ACC), precision (Pre), and F1-score (F1-sc). The following equations are used to find accuracy, and sensitivity and specificity when measuring positive and negative predictions, respectively:

$$\text{Acc}=\frac{\text{T}\text{P}+\text{T}\text{N}}{\text{ }\text{TP + TN + FP + FN}\text{ }}$$

$$\text{Sen/}\text{Re}\text{c}\text{ }=\frac{\text{T}\text{P}}{\text{T}\text{P}+\text{F}\text{N}}$$

$$\text{Spe }=\frac{\text{T}\text{N}}{\text{T}\text{N}+\text{F}\text{P}}$$

where TP, FP, TN, FN, and TR stand for true positive, false positive, true negative, and false negative.

The number of x-ray images in a testing set has been reduced to 200 for COVID-19 and 500 for other classes because there are not many images in the COVID-19 category, and therefore need to maintain a good ratio when looking at the metrics. As a result, in the first training phase, the achieved accuracy using the stack-2 model was as follows: 98.2% with normal, 99.1% with COVID-19, 99.8% with pneumonia, and 98% with pneumothorax during testing. While in the second training phase, the achieved accuracy using the stack-2 model is as follows: 97.3% with normal, 99% with COVID-19, 99.9% with pneumonia, 94% with pneumothorax, and 93.2% with atelectasis. Despite the imbalanced data, the proposed model still performed very well in accuracy, as seen from the provided results. Figure 6 depicts the performance plots and confusion matrices for the proposed models MobileNetV3 Large, NASNetMobile, EfficientNetV2b0, InceptionV3, ResNet50V2, and VGG16, as well as stack-1 and stack-2, that were implemented using the CXR4 dataset. Similarly, Fig. 7 depicts the performance plots and confusion matrices for the proposed models MobileNetV3 Large, EfficientNetV2b0, ResNet50V2, and stack-2, which were applied using the CXR5 dataset. These plots and matrices compare the accuracy of the models in classifying the data. The numerical values in the confusion matrix indicate the values for TP, FP, FN, and FP.

CXRs are diagnosed using a combination of correct positive, correct negative, incorrect positive, and incorrect negative results. TP indicates that the sample's actual class and recognition result are positive. A real positive sample class that the model assigns a "negative" status is identified by the letter FN, and a real negative sample class that the model consider it positive is denoted by the letter FP. TN indicates that the sample class and recognition result are both negative. Figure 7 presents the training and validation performances of the proposed models and their confusion matrices, which were applied using the CXR5 dataset of five classes (Normal, Atelectasis, COVID-19, Pneumonia, and Pneumothorax). This allowed for the analysis of the proportion of true positives, true negatives, false positives, and false negatives.

The Receiver Operating Characteristics (ROC) curve is another rating scale that clearly shows simulation results. The better a model works, the more area there is on the ROC curve. The area under the curve is represented by AUC. Figures 8 and 9 show resultant ROC curves and AUC of the proposed models using CXR4 and CXR5 datasets, respectively.

In addition to graphs, Table 2 summarizes how well the proposed individual models and stack-1 performed on the test set of the CXR4 dataset during the first phase, as well as the individual and stack-2 models performed on the test set of the CXR5 dataset during the second phase.

Table 2

Metrics results for the proposed models after working on the CXR4 and CXR5 datasets (%)
Dataset	Models	Normal			COVID-19			Pneumonia			Pneumothorax			Atelectasis
Dataset	Models	Acc	Sen	Sep	Acc	Sen	Sep	Acc	Sen	Sep	Acc	Sen	Sep	Acc	Sen	Sep
CXR4	Mobile NetV3	97.7	95.8	98.5	98.6	92	99.5	99.9	99.8	100	97.3	96.8	97	–	–	–
	NASNet Mobile.	96.5	94.4	97.4	96.4	97.5	98.7	99.7	99.8	99.7	94.7	92.8	99.5	–	–	–
	Efficient Net.V2	98.1	96.2	98.9	98.8	92.5	99.7	99.8	99.8	99.9	97.7	97.8	97.6	–	–	–
	Inception V3	97	95.4	97.7	95.7	73	98	99.7	99.8	99.6	99.7	92.6	94.5	–	–	–
	ResNet 50V2	97.5	95.2	98.5	97.5	82	99.6	99.7	99.6	99.7	96.2	97.2	95.9	–	–	–
	VGG16	98	96.4	98.6	98.6	90	99.8	99.8	99.8	99.8	97.6	97.8	97.5	–	–	–
	Stack-1	97.9	95.4	99	99.2	95.5	99.7	99.9	100	99.9	97.7	97.6	97.7	–	–	–
	Stack-2	98.2	96	99.2	99.1	95	99.6	99.8	99.8	99.8	98	98.2	98	–	–	–
CXR5	Mobile NetV3	97.2	93.2	98.6	98.8	93.5	99.4	99.9	99.8	99.9	93.1	88	94.7	92	75	95.3
	Efficient Net.V2	97.1	93.4	98.3	98.7	91	99.5	99.9	99.8	99.9	93	87.4	94.8	92.1	76.2	95.2
	ResNet 50V2	96.4	93.8	97.2	97.2	75.5	99.6	99.7	99.6	99.8	90.4	98.6	90.7	90.7	62.6	96.2
	Stack-2	97.3	93.8	98.5	99	96.8	99.6	99.9	100	99.9	94	91	95	93.2	77.1	96.4

The test set of the CXR5 dataset was used to perform the prediction for five classes using the proposed stack-2 model, as shown in Fig. 10. The predictions show promising results. The black color represents the real label. If the prediction is correct, it will still be black (true label equals predicted label). If the prediction is wrong, it will be highlighted in red. Figure 10 shows only two image that was predicted incorrectly (red color) and 23 images that were correctly predicted (black color).

The number of epochs that the model needs for training in soft labeling as compared to hard labeling was increased from 14 to 30 when using the proposed model based on the MobileNetV3 architecture. Also, the accuracy was higher in all categories except the COVID-19 category. At the same time, the loss percentage was higher than with hard labeling because the predictive power of the positive class was cut by 10% and given to the other classes (see Figs. 6.a and 6.b). The utilization of soft labeling demonstrated that it could result in a more generalizable model, as shown in Figs. 10.a and 10.b, to classify multiple lung diseases with high accuracy. The tests to detect and classify these diseases are performed in two stages using X-ray imaging. As mentioned earlier, a combined and formatted CXR4 dataset was utilized for training the proposed individual and stack models to recognize four different types of diseases. These diseases included COVID-19, normal, pneumonia, and pneumothorax. The CXR5 dataset was then used to train three individual models and one stack model to recognize five conditions: normal, atelectasis, COVID-19, pneumonia, and pneumothorax. Figure 6 represents the performance plots for four classes with the corresponding confusion matrix. Whereas Fig. 7 illustrates the confusion matrix for five class problems. The graphs show a significant increase in loss values during the initial stages of training. These loss values dramatically decrease as training progresses.

On the other hand, considering N is the number of target classes, the confusion matrix (CM) is an N-by-N matrix used to figure out how well the overall proposed models work. The matrix compares the predicted values to target values (actual label). The horizontal axis represents the true label's value. The predicted level for the four classes, COVID-19, normal, pneumonia, and pneumothorax, is shown on the vertical axis. For five classes, lung atelectasis is also considered in addition to the four classes. Predicated labels are used to make predictions, whereas true labels are the values already set to true; as a result, there will not be any changes to these during the classification task. In addition, various evaluation metrics for the suggested model are computed using these labels. According to what is shown in the confusion matrix for the four classes, the model misclassified some images between the two categories, where the highest ratio of mistakes was between pneumothorax and normal. In contrast, in the confusion matrix for the five classes, the misclassification was between atelectasis and pneumothorax and normal.

Furthermore, the ROC curves and AUC values showed that the models could learn to use image features well when classifying images into four or five disease groups. The results of the experiments, which classified lung diseases into four and five classes using X-ray images, are summarized in Table 2. The highest efficiency rate for all infections was obtained by the proposed stack-2 method.

Many chest diseases, particularly those identified as pandemics (COVID-19), put pressure and a burden on medical professionals to diagnose them and therefore require time. This is especially true when there are not enough doctors or medical equipment, making it hard to diagnose all possible patients promptly. In this study, using a combined dataset, a deep multiclass classification technique is proposed that uses ensemble learning and transfer learning to distinguish class types among four and five different classes of infections related to the chest. In order to overcome the weaknesses of individual models, ensemble learning helps combine the strengths of features of various models. At the same time, the requirement for a sizable amount of training data is eliminated by transfer learning. The proposed stack-2 model design consists of MobileNetV3, EfficientNetV2B0, and ResNet50V2 architectures, which generate better accuracy in the classification than the individual pre-trained models according to tests and obtained results. When using the proposed stack-2 model on the test set in the first training phase, the following average performance was achieved: Acc of 98.77%, Sen of 97.25%, and Sep of 99.15%. During the second training phase, the Stack-2 model had an average performance of 96.68% for Acc, 91.74% for Sen, and 97.88% for Sep. The proposed model can help radiologists and diagnostic staff identify COVID-19, pneumonia, pneumothorax, and atelectasis quickly with high accuracy using chest X-rays images. This, in turn, can save lives and reduce social and economic effects. The dataset, overall, is still not big enough and of low quality for highly accurate and useful deep learning results that could be used as a benchmark to find these infections of patients by looking at their X-rays. More high-quality images are needed to raise the average rating accuracy to a higher level in multiple class classifications. Such data is expected to grow over time, allowing for better classification results.

Funding

The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

Competing Interests

The authors have no relevant financial or non-financial interests to disclose.

Author Contributions

All authors contributed to the study conception and design. Material preparation, data collection, and analysis were performed by Muhannad K. Jalehi and Baraa M. Albaker. The first draft of the manuscript was written by Muhannad K. Jalehi, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Consent to participate

Informed consent was obtained from all individual participants included in the study.

Consent to publish

The images type of the used dataset in this paper is X-rays, and it's free public access.

S. Levine et al., “The Global Impact of Respiratory Disease THIRD EDITION 2 WRITING COMMITTEE.” Accessed: Jun. 24, 2022. [Online]. Available: https://www.firsnet.org/images/publications/FIRS_Master_09202021.pdf
C. L. McKnight and B. Burns, “Pneumothorax,” StatPearls, May 2022, Accessed: Jun. 24, 2022. [Online]. Available: https://www.ncbi.nlm.nih.gov/books/NBK441885/
“Atelectasis - Symptoms and causes - Mayo Clinic.” https://www.mayoclinic.org/diseases-conditions/atelectasis/symptoms-causes/syc-20369684 (accessed Jun. 24, 2022).
S. H. Wang, V. V. Govindaraj, J. M. Górriz, X. Zhang, and Y. D. Zhang, “Covid-19 classification by FGCNet with deep feature fusion from graph convolutional network and convolutional neural network,” Information Fusion, vol. 67, pp. 208–229, Mar. 2021, doi: 10.1016/J.INFFUS.2020.10.004.
D. J. Brenner, E. J. Hall, and D. Phil, “Computed Tomography — An Increasing Source of Radiation Exposure,” https://doi.org/10.1056/NEJMra072149, vol. 357, no. 22, pp. 2277–2284, Nov. 2007, doi: 10.1056/NEJMRA072149.
A. W. T. G. L. S. Fethyaseid Yimer, “Multiple Lung Diseases Classification from Chest X- Ray Images using Deep Learning approach,” International Journal of Advanced Trends in Computer Science and Engineering, vol. 10, no. 5, pp. 2936–2946, Oct. 2021, doi: 10.30534/ijatcse/2021/021052021.
R. Kundu, P. K. Singh, M. Ferrara, A. Ahmadian, and R. Sarkar, “ET-NET: an ensemble of transfer learning models for prediction of COVID-19 infection through chest CT-scan images,” Multimed Tools Appl, vol. 81, no. 1, pp. 31–50, Jan. 2022, doi: 10.1007/S11042-021-11319-8/TABLES/9.
E. Ayan, B. Karabulut, and H. M. Ünver, “Diagnosis of Pediatric Pneumonia with Ensemble of Deep Convolutional Neural Networks in Chest X-Ray Images,” Arab J Sci Eng, vol. 47, no. 2, pp. 2123–2139, Feb. 2022, doi: 10.1007/S13369-021-06127-Z/FIGURES/12.
F. Ahmad, A. Farooq, and M. U. Ghani, “Deep Ensemble Model for Classification of Novel Coronavirus in Chest X-Ray Images,” Comput Intell Neurosci, vol. 2021, 2021, doi: 10.1155/2021/8890226.
D. Li and S. Li, “An artificial intelligence deep learning platform achieves high diagnostic accuracy for Covid-19 pneumonia by reading chest X-ray images,” iScience, vol. 25, no. 4, p. 104031, Apr. 2022, doi: 10.1016/J.ISCI.2022.104031.
S. Hamida, O. el Gannour, B. Cherradi, A. Raihani, H. Moujahid, and H. Ouajji, “A novel covid-19 diagnosis support system using the stacking approach and transfer learning technique on chest X-ray images,” J Healthc Eng, vol. 2021, 2021, doi: 10.1155/2021/9437538.
E. Tasci, C. Uluturk, and A. Ugur, “A voting-based ensemble deep learning method focusing on image augmentation and preprocessing variations for tuberculosis detection,” Neural Comput Appl, vol. 33, no. 22, pp. 15541–15555, Nov. 2021, doi: 10.1007/S00521-021-06177-2/TABLES/5.
E. Khan, M. Z. U. Rehman, F. Ahmed, F. A. Alfouzan, N. M. Alzahrani, and J. Ahmad, “Chest X-ray Classification for the Detection of COVID-19 Using Deep Learning Techniques,” Sensors 2022, Vol. 22, Page 1211, vol. 22, no. 3, p. 1211, Feb. 2022, doi: 10.3390/S22031211.
P. Bhardwaj and A. Kaur, “A novel and efficient deep learning approach for COVID-19 detection using X-ray imaging modality,” Int J Imaging Syst Technol, vol. 31, no. 4, pp. 1775–1791, Dec. 2021, doi: 10.1002/IMA.22627.
B. Nigam, A. Nigam, R. Jain, S. Dodia, N. Arora, and B. Annappa, “COVID-19: Automatic detection from X-ray images by utilizing deep learning methods,” Expert Syst Appl, vol. 176, p. 114883, Aug. 2021, doi: 10.1016/J.ESWA.2021.114883.
M. Shorfuzzaman and M. Masud, “On the detection of covid-19 from chest x-ray images using cnn-based transfer learning,” Computers, Materials and Continua, vol. 64, no. 3, pp. 1359–1381, Jun. 2020, doi: 10.32604/CMC.2020.011326.
M. Loey, S. El-Sappagh, and S. Mirjalili, “Bayesian-based optimized deep learning model to detect COVID-19 patients using chest X-ray image data,” Comput Biol Med, vol. 142, p. 105213, Mar. 2022, doi: 10.1016/J.COMPBIOMED.2022.105213.
D. M. Ibrahim, N. M. Elshennawy, and A. M. Sarhan, “Deep-chest: Multi-classification deep learning model for diagnosing COVID-19, pneumonia, and lung cancer chest diseases,” Comput Biol Med, vol. 132, p. 104348, May 2021, doi: 10.1016/J.COMPBIOMED.2021.104348.
U. Sait et al., “Curated Dataset for COVID-19 Posterior-Anterior Chest Radiography Images (X-Rays).,” vol. 3, 2021, doi: 10.17632/9XKHGTS2S6.3.
“NIH Chest X-ray dataset | Cloud Healthcare API | Google Cloud.” https://cloud.google.com/healthcare-api/docs/resources/public-datasets/nih-chest (accessed May 30, 2022).
“TBX 11 | Kaggle.” https://www.kaggle.com/datasets/usmanshams/tbx-11 (accessed May 30, 2022).
A. Howard et al., “Searching for MobileNetV3,” Proceedings of the IEEE International Conference on Computer Vision, vol. 2019-October, pp. 1314–1324, May 2019, doi: 10.48550/arxiv.1905.02244.
B. Zoph, V. Vasudevan, J. Shlens, and Q. v. Le, “Learning Transferable Architectures for Scalable Image Recognition,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 8697–8710, Jul. 2017, doi: 10.1109/CVPR.2018.00907.
“EfficientNetV2: Smaller Models and Faster Training.” http://proceedings.mlr.press/v139/tan21a.html (accessed May 30, 2022).
C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the Inception Architecture for Computer Vision,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2016-December, pp. 2818–2826, Dec. 2015, doi: 10.48550/arxiv.1512.00567.
K. He, X. Zhang, S. Ren, and J. Sun, “Identity Mappings in Deep Residual Networks,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9908 LNCS, pp. 630–645, 2016, doi: 10.1007/978-3-319-46493-0_38.
K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, Sep. 2014, Accessed: Feb. 21, 2022. [Online]. Available: https://arxiv.org/abs/1409.1556v6
“sklearn.utils.class_weight.compute_class_weight — scikit-learn 1.1.1 documentation.” https://scikit-learn.org/stable/modules/generated/sklearn.utils.class_weight.compute_class_weight.html (accessed May 30, 2022).
L. Wright, “Ranger - a synergistic optimizer,” 2020. https://github.com/lessw2020/Ranger-Deep-Learning-Optimizer (accessed Jun. 25, 2022).
“Lookahead Optimizer: k steps forward, 1 step back.” https://proceedings.neurips.cc/paper/2019/hash/90fd4f88f588ae64038134f1eeaa023f-Abstract.html (accessed May 30, 2022).
L. Liu et al., “On the Variance of the Adaptive Learning Rate and Beyond,” Aug. 2019, doi: 10.48550/arxiv.1908.03265.
A. Garg, S. Salehi, M. la Rocca, R. Garner, and D. Duncan, “Efficient and visualizable convolutional neural networks for COVID-19 classification using Chest CT,” Expert Syst Appl, vol. 195, p. 116540, Jun. 2022, doi: 10.1016/J.ESWA.2022.116540.

Download PDF

Version 1

posted

You are reading this latest preprint version

Automatic Diagnosis of Multiple Lung Diseases in Chest Radiographs Based on Ensemble CNN Models

Status:

Version 1

Abstract

Purpose

Methods

Results

Conclusion

Figures

1. Introduction

2. Related Works

3. Materials And Methods

4.1 Dataset

4.1 Dataset Preprocessing

4.1 The Proposed Approach

4. Experimental Platform And Results

4.1 Test platform

4.2 Results

5. Discussion

6. Conclusions

Declarations

References

Status:

Version 1