This retrospective study was waived by the ethics committees of the Hainan General hospital. In total, 74 patients confirmed with COVID-19 infection from January 20 to February 8, 2020, and 82 patients with other types of pneumonia of the corresponding period were collected. In the COVID-19 dataset, 63 patients were finally included who met the following inclusion criteria: (i) RT-PCR confirmed COVID-19; (ii) non-contrast CT at diagnosis time; (iii) positive CT findings. 71 patients with Non-COVID-19 pneumonias were included who met the following inclusion criteria: (i) RT-PCR excluded COVID-19; (ii) non-contrast CT at diagnosis time; (iii) pneumonia highly suspected with COVID-19 at CT. The exclusion criteria were as follows: 1) contrast CT exam; 2) exams without slice thickness of 1mm; 3) negative CT findings. Finally, 326 chest CT exams from 134 patients were included in this study (Fig. 1). The average age is 47.0±15.4 years. Finally, we included 244 (75%) exams for COVID-19 and 82(25%) for non-COVID-19 pneumonia in the study.
All the COVID-19 were confirmed as positive by RT PCR and were acquired from January 21, 2020, to Feb 8, 2020. The most common symptoms were fever (82%) and cough (77%). Each patient had one or multiple CT scans during the progression of the disease. The follow-up study was performed until February 19,2020.
Other types of pneumonia patients were selected from the hospital of the corresponding period between January 23 to March 16, 2020. For 82 patients with negative RT-PCR results, pneumonia was diagnosed according to the Infectious Diseases Society of America/American Thoracic Society (IDSA/ATS) guidelines (28). Patients with at least one of the following clinical symptoms including cough, sputum, fever, dyspnea, and pleuritic chest pain, plus at least one finding of coarse crackles on auscultation or elevated inflammatory biomarkers, in addition to a new pulmonary infiltration on chest CT, would be diagnosed to be infected with pneumonia. The admission distribution of the patients with other types of pneumonia was: outpatient (86%, 61 of 71), inpatient (14%, 10 of 71). No one received laboratory confirmation of the etiology because of limited medical resources.
CT examinations were performed on the NeuViz 128 CT (Neusoft, China) with automatic tube current (300 mA-496 mA), tube voltage =120kV. The pitch was set at 1.5 and breath-hold at full inspiration. The slice-thickness of each CT scan was 1mm. The reconstruction matrix was 512x512 pixels.
All the CT scans were split with a ratio of 85:15 into a training cohort and a testing cohort at the patient level according to the visiting time of the hospital. The features selection and model building were performed on the training cohort, and the testing cohort was not used for the training procedure.
All subjects’ demographic characteristics and clinical data were retrospectively reviewed and collected, including age, gender, exposure history, diabetes, hypertension, chronic obstructive pulmonary disease(COPD),chronic liver disease, chronic kidney disease, cancer, cardiovasular disease, fever, cough, myalgia, fatigue, headache, nausea, diarrhea, bellyache, dyspnea, other symptoms, white blood cell count, number of neutrophils, lymphocyte count, hemoglobin and platelet count. The patient demographic statistics are summarized in Table 1. Imaging Protocol
The flow chart of data collection, ROI and features annotation, radiomics and quantity feature extraction, model building and evaluation are shown in Fig. 2.
Lesion segmentation and radiological evaluation
The pneumonia lesions were segmented semi-automatically. Firstly, the anonymized thin-slice DICOM format non-enhanced CT images were imported into an AI pneumonia assessment system, on which the pneumonia lesions were automatically detected and delineated. On the assessment platform, a MVP-Net (Multi-View FPN with Position-aware attention) which was trained on the NIH DeepLesion dataset and achieved state-of-the-art performance(29), was used to detected the abnormal pattern and classified them into consolidation and ground-glass opacity. Then a 2D U-Net model that trained on a local dataset of about 8,000 lung CT images was used to segment detected consolidation and ground-glass opacity lesions. Besides, pulmonary lobes were segmented by a pre-trained lobe segmentation model(30). Secondly, fifteen radiologists with more than 5 years of experience in chest imaging, blind to the knowledge of the pathological report and other clinical information, refined the segmentation result (Region of Interest, ROI) and evaluated the radiological characteristics. Each series was refined and evaluated by one of the fifteen radiologists. The segmentations and radiological characteristics were confirmed by two radiologists (F. C and Y.C) with 16 and more than 30 years of experience.
The 7 radiological characteristics included ground glass opacity, crazy paving pattern, halo sign, reversed halo sign, vascular perforating in the lesion, subpleural line and lesion locations (Fig. 3). For each series, the frequency of the radiological characteristics occurring was used for modeling.
Quantifying CT characteristics and radiomics features
The segmentation results were used to extract quantifying CT characteristics and radiomics features.
There were a total of 33 quantitative characteristics. Beside of the segmentation results, the AI pneumonia assessment system also provided the number of lesions that suffered bulla, emphysema, pleural thickening, reticular, and stripe, which were included as quantitative characteristics. From the segmentations, the mean and standard deviation of the CT values of the consolidation lesions, ground glass lesions and both of them were included. Then the volumes of the consolidation lesions, ground glass lesions and their sum were calculated. From the volumes, some ratios were calculated, including the volumes of the consolidation lesions versus the volumes of the entire pulmonary and the five pulmonary lobes respectively, the ground glass lesions versus the volumes of the entire pulmonary and the five pulmonary lobes respectively and the volumes of both type of lesions versus the volumes of the entire pulmonary and the five pulmonary lobes respectively.
Before the radiomics features were extracted, the pixel spacing of images was resampled to 1.0 mm per pixel by the BSpline algorithm. Besides the original images, the wavelet filters or Laplacian of Gaussian filters were performed to generate several filtered images. A total of 1218 radiomics features were extracted from the manual confirmed ROIs of the original images and the filtered images by PyRadiomics V2.1.0, including (1) 252 First-order features; (2) 14 Shape-based features; (3) 308 Gray Level Co-occurrence Matrix Features (GLCM); (4) 224 Gray Level Size Zone Matrix Features (GLSZM); (5) 224 Gray Level Run Length Matrix Features (GLRLM); (6) 196 Gray Level Dependence Matrix Features (GLD-ZM). The pre-processing methods and radiomic feature descriptions are detailed in Supplementary Information 1.1. and 1.2.
Development of predictive models
There were 4 groups of features that enrolled in model building: radiomics features, radiological features, quantity features and clinical features. The Support Vector Machine(SVM) model with radial basis function kernel were built on the 4 groups of features individually and on the combination of them.
Before model building, all numerical features were normalized by z-score method, and the categorical features were encoded by one-hot encoder. To avoid overfitting, the feature selection methods were used to reduce the number of features. The optimal parameters of the combination of the feature selection method and the model were found by grid search using a ten-run 5-fold cross validation procedure on the training cohort. After the optimal params were determined, the entire training cohort was used to build the model and the performance on the testing cohort was evaluated. After the cross-validation procedure, the threshold that maximum the Youden Index on the validation cohort was used to cut-off the discriminative score to differentiate the COVID-19 from other pneumonia.
The features were selected by a two-step method. (1)The Mann-Whitney U test was used and the p values were correct by Benjamini-Hochberg method. The feature that significantly different(p<0.05) between the COVID-19 cohort and non-COVID-19 cohort was preserved. (2)the minimum-redundancy maximum-relevancy(mRMR) method was used and the number of selected features was determined by the cross-validation procedure. Specially, for the radiological features, the mRMR procedure was removed because there were only 7 radiological features.
The discrimination performance of the model was evaluated by the area under the receiver operator characteristic curve (AUC), accuracy (ACC), sensitivity and specificity. The AUCs of the SVM model that built on the combined features and that on each individual feature group were compared by using the Delong test. Because the SVM model with radial basis function kernel is nonlinear, the feature importance cannot be derived directly. The permutation importance(31) was used to evaluate the feature importance and the AUC was used to measure the difference between the baseline and the model that built with permutated feature.