A Deep Learning Framework for Coronavirus Disease (COVID-19) Detection in X-Ray Images

Coronavirus, a large family of viruses, causes illness in both humans and animals. The novel coronavirus (COVID-19) came up in Wuhan in December 2019. This deadly COVID-19 pandemic has become very fast-spreading and currently present in several countries worldwide. The timely detection of patients who have COVID-19 is vitally important. To this end, scientists are working on diﬀerent detection methods.In this paper, a grid search (GS) and pre-trained model aided convolutional neural network (CNN) model is proposed to detect COVID-19 in X-Ray images. In the proposed method, the GS method is employed to optimize the hyperparameters of CNN, which directly aﬀects classiﬁcation performance. Three pre-trained CNN models (GoogleNet, ResNet18 and ResNet50), which can be used for classiﬁcation, feature extraction and transfer learning purposes were used for transfer learning in this study. The proposed method was trained using the training and validation subdatasets of the collected dataset and detail evaluations are presented according to diﬀerent performance metrics. According to the experimental studies, the best results were obtained with the GS and ResNet50 aided model.


Introduction
COVID-19 is a fast-spreading lethal coronavirus. It has spread all over the world in a very short time since it was initially detected in December 2019. Symptoms of this deadly include fever, dry cough and weakness [15]. The rapid and timely detection of sick individuals is important for both the patient's health and the prevention the further spread of the virus. COVID-19 diagnoses can be made by swabs from the nose and throat, blood count, computed tomography and X-Ray images. Convolutional neural networks (CNNs), which have basic functions such as object recognition, classification and regression, has been popular in recent years. Some researchers have employed CNN for the detection of COVID-19. Wang and Wong [36] developed a COVID-NET with a deep CNN architecture for COVID-19 detection. They also presented the COVIDx dataset , which is adopted in this study. According to their experimental results, their proposed method produced good results. A transfer learning-based CNN model that classifies medical data into three categories (normal, pneumonia, and Covid-19) has been proposed by Apostolopoulos and Bessiana [3]. In their study, which was carried out using five different pre-trained CNN models, they reported an accuracy of 98.75 % for the binarry classification problem and 93.48 % for the multiclassification problem. Hemdan et al. [10] investigated COVIDX-19 classification with 7 different deep CNN architectures. In their experimental studies, they worked on two different categories, COVID-19 and non-COVID- 19. Their dataset consisted of a total of 50 images. They reportedly achieved better results with the VGG19 and DenseNet201 models. Ghoshal and Tucker [11] proposed a Dropweights based Bayesian Convolutional Neural Networks model and tested the proposed model for 4 different classes (normal, bacterial pneumonia, non-COVID-19 viral pneumonia and COVID -19). Sethy and Behera [32] extracted features from X-Ray images using pre-trained moldels. They collected their dataset from GitHub, Kaggle, and Open-I repository. According to their experimental results, features obtained from ResNet50 model and subsequently classified with SVM produced more successful results than the other models. A new deep learning aided anomaly detection model for COVID-19 detection from X-Ray images was proposed by Zhang et al. [39]. Their model achieved a sensitivity of 90.00% specificity of 87.84% when the threshold was set to 0.25. Narin et al. [21] proposed a transfer learning-based CNN model for detection of COVID-19. They used ResNet50, InceptionV3, and Inception-ResNetV2 pre-trained models for transfer learning. According to the results of their simulations, the ResNet50 based model recorded the best results.
As seen in the above literature review for COVID-19, the use of transfer learning based CNN is important to produce an effective model. Transfer learning supported CNN model has also used for different problems [24,22,26,23,6]. Similarly, pre-trained models have been used in the literature for different purposes besides transfer learning [1,2,4,12,35,37]. The optimization of hyperparameters is an effective way to obtain higher success rates for machine learning methods. For this purpose, there is a plethora of studies in the literature. In addition to heuristic optimization algorithms [30,17,38,8,14,25,26,24], the global search methods [31,14,29,7] and other methods [9,28] have also been used for hyperparameter optimization. This study carries out the classification of COVID-19 from X-Ray images by using transfer learning-based CNN, whose hyperparameters were optimized by the GS method.
In the first stage, a dataset was created using X-Ray images collected from different platforms. This dataset includes bacteria, COVID-19, normal and virus classes. In the next stage, the proposed classification method was evaluated with the dataset and the detailed results are presented according to different performance metrics. The main contributions of the paper are given below: -In this paper, a transfer learning based CNN is proposed for COVID-19 detection on X-Ray images. The hyperparameters of the generated CNN models are tuned by using the GS method.
To the best of our knowledge, this study is the first in the literature that utilizes this approach for COVID-19 detection. -The GS and ResNet18 aided method have the smallest computational time for parameter optimization. -The GS and ResNet50 aided method obtained the best performance based on all of the metrics (accuracy, sensvitiy, specifity, precision, and F1 score).
The remainder of this paper is organized as follows.
The methodology including CNN, pre-trained models, GS, and the proposed method is given in Section 2.
The experimental setup and results are provided in Section 3. Finally, Section 4 presents the conclusion and future works.

Methodology
In this paper, a deep learning model based unified method is presented for COVID-19 detection in X-Ray images. In the method section, convolutional neural networks (CNN) [18], which is a deep learning method, is used as the classifier and GoogleNet [34], ResNet18 [13], ResNet50 [13] pre-trained models are used for transfer learning. Additionally, a grid search method, which is a global search algorithm, is used for optimization of the the training settings of the generated CNN model. Al detailed description is presented below.

Convolutional Neural Networks
CNN, which is a multi-layer neural network [19], is a type of artificial neural network and it is specialized to process big data [18]. A basic CNN structure usually has convolution, pooling, flattening, and fully connected layers. An example of a CNN architecture is presented in Fig. 1.
The convolution layer contains a set of learnable filters and is one of the structures in which the learning skill of CNN takes place [26]. An example of convolution operation is given in Fig. 2. Here, I refers to input image, K refers to filter and I*K refers to feature map.
Another important structure that reduces the computational cost of a network is the pooling layer. Maximum, average and minimum pooling are the most frequently used pooling techniques [16]. An example of maxpooling operation is given in Fig. 3.  The fully connected layer comes after the both convolution and pooling layers of a CNN model [22]. Neurons in this layer have a full connection with the neurons in a previous layer. The data is flattened before coming to the fully connected layer. An example of a flattening operation is given in Fig. 4.
The training options of the CNN have an effect on its successful performance. Minibatch size, initial learning rate, 2 regularization parameter, and momentum value, which are optimized by GS in this paper, are some of the most important hyperparameters of a CNN model. The description of these hyperparameters are shown in Table 1.
Developing a new CNN model is a very challenging and time-consuming process. Using pre-trained models is a faster and more effective approach than developing a new CNN model [23]. Pre-trained models are typically used for classification, transfer learning, and feature extraction. GoogleNet, ResNet18, and ResNet50 models were selected for transfer learning in this study. Some features of related pre-trained models are as shown in Table 2.

Grid Search Algorithm
The grid search (GS) method used to optimize the parameters of a model is frequently preferred in applications where a short process of optimization is   Fig. 5) are fed to the relevant model and the optimization process is started. The parameter set that gives the best results is determined and given as the output of the GS method.

Proposed Method
In this paper, a method including GS and transfer learning supported CNN model is proposed. The proposed method is given in Fig. 6. For the first step, the collected dataset is randomly split into three sub-datasets, which are training, validation, and testing. The training, validation, and testing sub-datasets are selected as 50%, 20%, and 30% of the dataset, respectively. For the next phase, parameter optimization using GS is realized. In this phase, the optimally selected hyperparameters from the parameter set

Experimental Studies
In this section, the proposed method is applied on the collected dataset. The experimental setup is presented in Subsection 3.1 whereas the results of the proposed method are presented in Subsection 3.2.

Experimental Setup
The experimental setup phase was carried out before the main simulation stage. The data collection, parameter setup, environmental setup, and performance metrics are sub parts of this subsection.

Data Collection
Since the COVID-19 appeared in December 2019, it is quite difficult to obtain a public dataset for research work. Therefore, the data was obtained from different sources [5,27] and hospital [33] shares. The number of samples for each class is given in Table 3.

Parameter Setup
The initial learning rate (ILR), 2 regularization ( 2 R), momentum (Mom), and minibatch size (MiB) parameter values to be used in grid search are shown in Table 4.  Fig. 6: The GS and Pre-trained model aided method  The performance metrics, which are accuracy, sensitivity (recall), specificity, precision, and F1-score, are calculated using Eq. 2-6, respectively.
P recision = T P T P + F P F 1 score = 2 × P recision × Sensitivity P recision + Sensitivity

Parameter Optimization using GS for the Pre-trained Aided Model
In the proposed method, the GS algorithm is used to optimize the training parameters of the CNN model. When Table 8 is examined, all combinations of parameter sets and validation errors (fitness value, (fval)) of these parameter sets are presented for GoogleNet aided proposed method. The parameter values giving the error of 0,0278 are the best hyperparameters kept in the memory as a result of the GS approach. All combinations of parameter sets and fvals of these parameter sets are presented in Table 9 for ResNet18 aided proposed method. The parameter values giving the error of 0,0208 are the best hyperparameters kept in the memory. For the last method, ResNet50 aided proposed method, all combinations of parameter sets and fval values of these parameter sets are presented in Table  10. The parameter values giving the error of 0,0278 are the best hyperparameters.

General Results
The cost of computational time for parameter optimization using GS is given in Table 5. According to the table, less time is required for parameter optimization of the ResNet18 aided method. While the optimal hyperparameters obtained by GS are presented in Table 6, the performance comparison of the proposed methods on the whole system is given in Table 7. When the results in the table is analyzed, the ResNet50 aided proposed method is the most successful one according to the metrics.      The confusion matrices which are formed according to the accuracy performance of the proposed methods are presented in Fig. 8 -10. In addition, the training progress graphics for the final training processes of the proposed methods are given in Fig. 11 -13, respectively.   Additionally, the performance of deep learning methods is implemented by a hyperparameter selection. In this study, a classification system was developed by using both the GS and pre-trained CNN models. Here, while GS was used for hyperparameter optimization, the pre-trained models were used for transfer learning. With the proposed method, classification experiments were carried for 4 different classes on X-Ray images of the lungs. These categories of the dataset include bacteria, COVID-19, normal, and virus. In the proposed method, 3 different pre-trained models (GoogleNet, ResNet18, ResNet50) were used. The highest accuracy rate (97.69%) was obtained with the GS and ResNet50 aided proposed method. In future studies, different pre-trained models will be used for transfer learning. Additionally, different global search methods would be employed for hyperparameter optimization.

Conflict of interest
The author declares that there is no conflict of interests regarding the publication of this article. Tayyip Ozcan received no financial support for the research, authorship, and/or publication of this article.