Machine learning-based CT radiomics model Distinguishes COVID-19 from other viral pneumonia

Purpose To develop a machine learning-based CT radiomics model is critical for the accurate diagnosis of the rapid spread Coronavirus disease 2019 (COVID-19). Methods In this retrospective study, a machine learning-based CT radiomics model was developed to extract features from chest CT exams for the detection of COVID-19. Other viral-pneumonia CT exams of the corresponding period were also included. The radiomics features extracted from the region of interest (ROI), the radiological features evaluated by the radiologists, the quantity features calculated by the AI segmentation and evaluation, and the clinical parameters including clinical symptoms, epidemiology history and biochemical results were enrolled in this study. The SVM model was built and the performance on the testing cohort was evaluated by the area under the receiver operating characteristic curve (AUC), sensitivity and specicity. Results For the SVM model that built on the radiomics features only, it reached an AUC of 0.688(95% CI 0.496 to 0.881) on the testing cohort. After the radiological features were enrolled, the AUC achieved 0.696(95% CI 0.501 to 0.892), then the AUC reached 0.753(95% CI 0.596 to 0.910) after the quantity features were included. Our nal model employed all the features, reached the per-exam sensitivity and specicity for differentiating COVID-19 was 29 of 38 (0.763, 95% CI: 0.598 to 0.886]) and 12 of 13 (0.923, 95% CI: 0.640 to 0.998]), respectively, with an AUC of 0.968(95% CI 0.911 to 1.000). Conclusion The machine learning-based CT radiomics models may accurately detect COVID-19 and differentiate it from other viral pneumonia.


Introduction
The Coronavirus Disease 2019 (COVID-19) has widely and rapidly spread throughout the world since late December 2019 [1,2]. The newly emerging disease is highly contagious and may cause severe acute respiratory distress or multiple organ failure in severe cases [3][4][5][6]. The World Health Organization (WHO) declared the outbreak of COVID-19 as a "public health emergency of international concern" (PHEIC) on January 30, 2020.
At present, the gold standard for the diagnosis of COVID-19 is reverse-transcription polymerase chain reaction (RT-PCR). However, a high false negative rate [7] and the shortage of RT-PCR assay in the early stage of the outbreak limited the early detection and treatment of the presumptive patients [8,9]. This speeded up the spread of COVID-19. Therefore, fast diagnosis is important for controlling the spread of COVID- 19. Recent studies have demonstrated that computed tomography (CT), as a non-invasive imaging approach, is of great value in detecting lung lesions in patients with COVID-19 infection [2,10]. Besides, CT had much higher sensitivity than initial RT-PCR in diagnosing COVID-19 [8,9]. Consequently, CT could be used as an effective tool for early detection and diagnosis of COVID-19. We should not neglect the fact that COVID-19 may have certain similar CT imaging features with other types of pneumonia, thus making it hard to differentiate.
Current studies have demonstrated that arti cial intelligence could distinguish COVID-19 from other pneumonia [11,12], improve radiologists' performance in distinguishing COVID-19 from non-COVID-19 pneumonia on chest CT and provide clinical prognosis with good accuracy that can assist clinicians to timely adjust their clinical management and allocate resources appropriately [13][14][15][16][17][18][19]. However, COVID-19 is caused by SARS-CoV-2 virus, its CT manifestations resemble other types of viruses. Most of the published works of literature have not included the viral pneumonia as the comparison group. Additionally, the non-COVID-19 diseases included as a comparison group are long before the COVID-19 outbreak. The most di cult situation in clinical diagnosis and treatment is to identify other viral pneumonias that occurred in the same period as the epidemic of COVID-19.
In recent years, much attentions had been paid to radiomics in disease diagnosis and treatment outcome evaluation [20,21]. Speci cally, radiomics is of great value in medical imaging because of its ability to extract high throughput of quantitative descriptors from routine computed tomography (CT) studies [21]. Radiomics was applied to many elds of cancer, such as tumor detection, preoperative prediction of lymph node metastasis and therapeutic response assessment [20,22,23]. Recently, radiomics have been proved to be helpful in COVID-19 screening, diagnosis, prediction of hospital stay, assessing the imaging characteristics and risk factors associated with adverse composite endpoints in patients with COVID-19 pneumonia [24][25][26][27]. However, these studies were limited in small sample size. In the study of Qi et al., 31 patients were nally included in the study [25]. Some did not extract high-throughput imaging features [27]. Besides, no radiomics study of COVID-19 has been done compared with other viral pneumonia of the corresponding period, which is di cult to differentiate from COVID-19. The purpose of this study was to develop and test machine learning-based CT radiomics models for the detection of COVID-19. Other viral-pneumonia exams were also included to test the robustness of the model.

Study population
This retrospective study was waived by the ethics committees of the Hainan General hospital. In total, 74 patients con rmed with COVID-19 infection from January 20 to February 8, 2020, and 73 patients with other viral pneumonia of the corresponding period were collected. In the COVID-19 dataset, 66 patients were nally included who met the following inclusion criteria: (i) RT-PCR con rmed COVID-19; (ii) non-contrast CT at diagnosis time; (iii) positive CT ndings. Other viral pneumonias were included who met the following inclusion criteria: (i) RT-PCR excluded COVID-19; (ii) non-contrast CT at diagnosis time; (iii) viral pneumonia highly suspected with COVID-19 at CT. The exclusion criteria were as follows: 1) contrast CT exam; 2) exams without slice thickness of 1mm; 3) negative CT ndings. Finally, 326 chest CT exams from 137 patients were included in this study (Fig. 1). The average age is 47.0±15.4 years. Finally, we included 244 (75%) exams for COVID-19 and 82(25%) for other viral pneumonia in the study.
All the COVID-19 were con rmed as positive by RT PCR and were acquired from January 21, 2020, to Feb 8, 2020. The most common symptoms were fever (82%) and cough (77%). Each patient had one or multiple CT scans during the progression of the disease. The follow-up study was performed until February 19,2020.
Other viral-pneumonia patients were selected from the hospital of the corresponding period between January 23 to March 16, 2020. The admission distribution of the patients with other viral-pneumonia was: outpatient (86%, 61 of 71), inpatient (14%, 10 of 71). No one received laboratory con rmation of the etiology because of limited medical resources.
All the CT scans were split with a ratio of 85:15 into a training cohort and a testing cohort at the patient level according to the visiting time of the hospital. The features selection and model building were performed on the training cohort, and the testing cohort was not used for the training procedure. The patient demographic statistics are summarized in Table 1.

Imaging Protocol
CT examinations were performed on the NeuViz 128 CT (Neusoft, China) with automatic tube current (300 mA-496 mA). The pitch was set at 1.5 and breath-hold at full inspiration. The slice-thickness of each CT scan was 1mm. The reconstruction matrix was 512x512 pixels. The ow chart of data collection, ROI and features annotation, radiomics and quantity feature extraction, model building and evaluation are shown in Fig. 2.

Lesion segmentation and radiological evaluation
The anonymized thin-slice DICOM format non-enhanced CT images were imported into the Dr. Wise research platform, on which the pneumonia lesions were automatically delineated by deep-learning segmentation algorithms. Fifteen radiologists with more than 5 years of experience in chest imaging, blind to the knowledge of the pathological report and other clinical information, con rmed the segmentation result (Region of Interest, ROI) and evaluated the radiological characteristics. The segmentations and radiological characteristics were con rmed by two radiologists (F. C and Y.C) with 16 and more than 30 years of experience.
The 7 radiological characteristics included ground grass opacity, crazy paving pattern, halo sign, reversed halo sign, vascular perforating in the lesion, subpleural line and lesion locations (Fig. 3). For each series, the frequency of the radiological characteristics occurring was used for modeling.

Quantitative CT characteristics and radiomics features
The segmentation result was used to extract quantitative CT characteristics and radiomics features. The delineation of the pulmonary lobes was segmented by the Dr. Wise research platform and the lesion and lobe information were extracted to construct quantitative characteristics of pneumonia.
There were a total of 33 quantitative characteristics, including the volume of the lesions, mean and standard deviation of the CT values in lesions, consolidation lesions and ground glass lesions respectively, the number of the consolidation lesions and the number of the ground glass lesions, the ratio of the volume of these lesions in the entire pulmonary and the ve pulmonary lobes respectively, and the number of lesions that suffered bulla, emphysema, pleural thickening, reticular, and stripe.
Before the radiomics features were extracted, the pixel spacing of images was resampled to 1.0 mm per pixel by the BSpline algorithm. Besides the original images, the wavelet lters or Laplacian of Gaussian lters were performed to generate several ltered images. A total of 1218 radiomics features were extracted from the manual con rmed ROIs of the original images and the ltered images by PyRadiomics V2.

Development of predictive models
The features were stacked in the order of radiomics features, radiological features quantity features and clinical features, and 4 SVM models with radial basis function kernel were built. All numerical features were normalized to [0,1], and the categorical features were encoded by one-hot encoder. To avoid over tting, the feature selection methods were used to reduce the dimension of the features. The optimal parameters of the combination of the feature selection method and the model were found by grid search using a ten-run 5-fold cross validation procedure on the training cohort. After the optimal params were determined, the entire training cohort was used to build the model and the performance on the testing cohort was evaluated. After the cross-validation procedure, the threshold that maximum the Youden Index was used to cut-off the discriminative score to differentiate the COVID-19 from other virus pneumonia. The evaluation indicators include the area under the receiver operator characteristic curve (AUC), accuracy (ACC), sensitivity and speci city.
The feature selection method we used included the f-test based method and the L1 based method. The f-test based feature selection method with different preserve ratio (1%, 5%, 10%, 30%, 50% and 100%) were tested, while the L1 based method that used linear C-support vector classi cation model with different C value (0.01, 0.1, 1, and 10) as a base model was tested. Table 1 demonstrates the study population characteristics for the training and testing cohorts. There are slightly more male patients than female   The details of the performance are shown in table 2 and the ROC curve of the 4 SVM models was shown in Fig. 5. The ridgeline plot (Fig. 6) showed the distribution of the normalized features selected for RRQC by our feature selection strategy, only numerical features were shown in this plot. Two image features, 2 biochemical indexes, 5 shape-based radiomics features, 2 intensity-based features and 1 texture feature on images preprocessed by wavelet lter were preserved. The encouraging diagnostic performance of the machine learning-based CT radiomics model indicates that radiomics may be helpful for the detection of COVID-19. Radiomics features in our model included rst order features, shape-based features and the distribution, correlation and variance in gray level intensities. These radiomics features described the relationship between voxels and contained quantitative information on the spatial heterogeneity of pneumonia lesions. Importantly, when radiomics features were included alone, the model revealed the good Figure 2 The ow chart showed our study work ow, consisting of data collection, ROI and features annotation, radiomic and quantity features extraction, model building and evaluation.