Development and validation of a radiomics nomogram for identi�cation of severity of patients with COVID-19

Background The coronavirus disease 2019 (COVID-19) is a pandemic now, and the severe COVID-19 determines the management and treatment, even prognosis. Thus, we aim to develop and validate a radiomics nomogram for identifying severe patients with COVID-19. Methods There were 156 and 104 patients with COVID-19 enrolled in primary and validation cohorts respectively. Radiomics features were extracted from chest CT images. Least absolute shrinkage and selection operator (LASSO) method was used for feature selection and radiomics signature building. Multivariable logistic regression analysis was used to develop a predictive model, and the radiomics signature, abnormal WBC counts, and comorbidity were incorporated and presented as a radiomics nomogram. The performance of the nomogram was assessed through its calibration, discrimination, and clinical usefulness. Results The radiomics signature consisting of 4 selected features was signi�cantly associated with clinical condition of patients with COVID-19 in the primary and validation cohorts (P < 0.001). The radiomics nomogram including radiomics signature, comorbidity and abnormal WBC counts, showed good discrimination of severe COVID-19, with an AUC of 0.972, and good calibration in the primary cohort. Application of the nomogram in the validation cohort still gave good discrimination with an AUC of 0.978 and good calibration. Decision curve analysis demonstrated that the radiomics nomogram was clinically useful to identify the severe COVID-19. Conclusions We present an easy-to-use radiomics nomogram to identify the severe patients with COVID-19 for better guiding a prompt management and treatment.


Introduction
The coronavirus disease 2019 (COVID- 19), which is caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has widely spread all over the world [1][2][3] due to person to person transmission [4] .On March 11, 2020, the World Health Organization declared that COVID-19 outbreak has reached the stage where it could be described as a pandemic.The epidemic of COVID-19 has attracted worldwide attention and caused a certain degree of social panic.
The incidence and mortality of COVID-19 varied in different countries or territories [1] .According to the Chinese Center for Disease Control and Prevention [5,6] ,81% of the patients with COVID-19 had mild symptoms, but it was the rest of the patients (19%) who were in severe and critical conditions that determined the mortality.Because the severe patients with COVID-19 directly in uences the clinical management and treatment [7][8][9] , it is crucial for clinicians to evaluate the condition of COVID-19.
So far, chest computed tomography (CT) has been an important modality to screen, diagnose and evaluate COVID-19 [10][11][12][13][14] .It was reported that the CT features of COVID-19 was manifested as patchy ground-glass opacities (GGOs) with or without consolidation distributed in subpleural areas of bilateral lungs [13,15] , and increased numbers, greater extent of consolidation on chest CT images were related to progression of COVID-19 [10,11] .However, these studies were limited to qualitative analysis, merely focusing on the manifestation of COVID-19 on chest CT images to screen potential new cases of COVID-19.The quantitative analysis of correlation between pulmonary abnormalities of COVID-19 on chest CT images and the clinical severity or condition of COVID-19 has not been investigated thoroughly, which may be promising for improving the management of COVID-19.
As the wide application of the arti cial intelligence technology for detection of pulmonary nodules has demonstrated great success [16,17] , computer-aided detection and analysis makes quanti cation and classi cation of COVID-19 possible.The arti cial intelligence evaluation system of COVID-19 was rapidly developed and applied to solve the insu cient expertise of radiologists and speed up screening potential new cases of COVID-19 in Wuhan city, China [18] .However, the current system has been con ned to detect pulmonary nodules [16,17] , its application in assessing the severity of COVID-19 has yet to be developed.
Radiomics, as an emerging technique involved with the extraction of high-throughput data from quantitative imaging features and the subsequent association of these parameters with clinical data, has been applied in various diseases.For instance, radiomics has been used to predict lymph node metastasis in patients with colorectal cancer [19,20] .As far as we know, no research has reported the application of radiomics for evaluation of the severity of COVID-19.Therefore, the purpose of this study was to apply the arti cial intelligence to quantitatively analyze the lung abnormalities associated with COVID-19, and to develop and validate a radiomics nomogram for identifying the severe COVID-19 who need better management in intensive care units.

Patients
The protocol for this study was approved by the Institutional Review Board.All patients or their legally authorized representatives were provided written informed consent prior to participation in this study.A total of 260 patients with COVID-19 were enrolled from 24 January, 2020 to 1 May, 2020.The patients were grouped into primary cohort and validation cohort using strati ed random resampling method with a ratio of 3:2.The inclusion criteria were (a) positive for RT-PCR test of SARS-CoV-2; (b) complete clinical data; (c) patients underwent a CT scan.The exclusion criteria were (a) poor images and (b) normal CT.
Their baseline clinical and image data were reviewed retrospectively.

Clinical information
Basic information including gender, age, comorbidity (hypertension, diabetes mellitus, cardiocerebrovascular disease, the history of surgery for important organs, etc.), laboratory examinations including C reactive protein (CRP), white blood cell (WBC) and lymphocytes were derived from medical records for all patients.A score system based on the number of comorbidities was used to evaluate the state of the patients: none has a score of 0, one has a score of 1, two has a score of 2, more than two has a score of 3.
According to the latest National Recommendations for Diagnosis and Treatment of COVID-19 (the 7th edition), 7 the clinical condition of the patients was classi ed into none-severe (mild and common) and severe (severe and critical) types.

CT image acquisition, segmentation, and quantitative analysis
All patients underwent chest CT scan using a multidetector scanner (16-MDCT, SOMATOM Emotion16, SIEMENS, Germany; 16-MDCT, De nition AS, SIEMENS, Germany; 64-MDCT, Optima CT680, GE, USA) with the following parameters: display eld of view (dFOV) 32 cm, 300 mAs and 120 Kv, slice thickness 5 mm.All CT images were acquired at deep inspiration in the supine position and reconstructed with the slice thickness of 0.625-1.25 mm.
Pleural thickening and pleural effusion were observed in mediastinal window with a window width of 350 Houns eld unit (HU) and a window level of 40 HU.
The owchart of radiomics procedure is shown in Fig. 1.All CT images of COVID-19 were segmented by a pre-trained Multi-task Unet network.
Multi-task Unet is a 2D Unet 21 based network with a single encoder and two parallel decoders, one decoder with attention block to learn the lesion segmentation and another decoder with stacked dilated convolutions to learn the lung segmentation, providing at the same time a more e cient feature encoding and a regularizing effect.At each decoder layer, the features from the corresponding encoder layer are concatenated which help in retaining multi-scale features.Speci cally, we concatenated encoder and decoder features, based on the attention block (integration of spatial attention and channel attention) 22 which was learnt for encoder feature and decoder feature separately.By facilitating joint primary of two tasks, not only the model size and inference time were greatly reduced, but also low-level features were effectively reused.The network primary and inference of this experiment were implemented based on Dr. pecker cloud platform (http://www.jianpeicn.com/category/yuepianjiqiren).It is available to public research institutions and is free now around the world for COVID-19 research analysis and prevention.
Primary samples with detailed delineation of each lesion and lung regions were required.Two cardiothoracic radiologists who had 5-15 year's experiences segmented the lesion and lung region using ITK-SNAP software (version 2.2.0; http://www.itksnap.org) in lung window with a width of 1500 HU and a level of -600 HU.The margin of the lesion was delineated for each axial slice (Fig. 2.A-C).Then, a 3D regions of interests (ROI) was obtained (Fig. 2.D-F).We split 650 annotated CT scans into 550 for primary and 100 for testing.We tested our model on a holdout 100 CT scans as well to illustrate the robustness of our proposed approach.The average Dice similarity coe cient was 0.973 for the right lung, 0.985 for left lung, and 0.864 for lesion segments.
After segmentation by multi-task Unet, all segmentation results were manually reviewed again in this experiment.Various metrics were computed to quantify the COVID-19 lesion, including volumes of lesion in the whole lung, and volumes of lesion in each lung segment.The GGO and consolidation were distinguished with a threshold value of -450HU.We used the SimpleITK software tools (http://www.simpleitk.org) to quantify the mean HU of lung and lesion, number and volume of lesions, volume of GGO and consolidation in double lungs, and the volume of the whole lung automatically.
Simultaneously, the ratio of volume of GGO and consolidation in bilateral lungs to total lung volume and to lesion volume in bilateral lungs were calculated respectively.Totally, there were 14 quantitative parameters acquired for feature selection and radiomics model construction.

Statistical Analysis
R software (version 3.0.1;http://www.Rproject.org)was used for statistical analysis.All the radiological features were normalized between 0 to 1.The 'caret' package was used to obtain the accuracy, sensitivity and speci city of model.'pROC' package, 'rms' package and 'rmda' package were used to perform receiver operating characteristic (ROC) analysis, calibration curve analysis and decision curve analysis, respectively.Two-sided P < 0.05 indicated statistical signi cance.

Radiological features selection and radiomics signature construction
The most useful predictive features were selected by using the least absolute shrinkage and selection operator (LASSO) method. 23Brie y, the optimized hyperparameter λ was rst determined by using 10fold cross validation with binomial deviance as a criterion.Then the features with non-zero coe cient were selected based on the determined optimal λ.Finally, LASSO regression was conducted to construct the radiomics signature and a radiomics score (Rad-score) was calculated for each patient via a linear combination of selected and weighted features by their corresponding coe cients.

Individualized prediction model construction
Besides the radiomics features, the clinical data (termed "clinical feature" later in this article) was also collected.Two clinical features including comorbidity and abnormal WBC counts, which were signi cantly different between severe and non-severe COVID-19 in univariate regression analysis, were combined with Rad-score to build the nomogram using multivariate logistic regression.The nomogram provides the clinicians with a quantitative tool to predict individual probability of severe or none-severe COVID-19.

Performance validation of the nomogram in the primary cohort
In validation cohort, the same logistic regression formula formed in the primary cohort was used to calculate total points for each patient.Total points were then used as a factor for logistic regression analysis in validation cohort.Finally, two methods including calibration curves analysis and ROC analysis were used to evaluate the performance of nomogram model.Calibration curves were plotted to assess the agreement between the predicted event probability and the observed event probability.The ROC analysis was performed to evaluate the performance of the nomogram.Accuracy, sensitivity and speci city were calculated in both primary cohort and validation cohort.

Clinical Use
Decision curve analysis was performed to determine the clinical practicability of the nomogram by quantifying the net bene ts at different threshold probabilities in both the primary and validation cohorts. 24 Results

3.
1 Clinical characteristics There were 156 and 104 patients with COVID-19 enrolled in primary and validation cohorts respectively.Characteristics of patients in the primary and validation cohorts were shown in Table 1.There were signi cant differences in comorbidity, presence of pleural thickening, CRP increase, abnormal WBC and lymphocytes counts between severe and none-severe patients with COVID-19 in both primary and validation cohorts.Age and presence of pleural effusion differed between severe and none-severe patients with COVID-19 in primary cohort, while they did not differ in validation cohort.A signi cantly higher proportion of male with severe condition was shown in validation cohort, however, it did not show a signi cant difference in primary cohort.Note that 9 severe patients were presented with increased WBC counts.There were no signi cant differences in presence of severe or critical condition between two cohorts (P > 0.05).The rate of severe or critical patients was 16.7% and 17.3% for the primary and validation cohorts, respectively.Moreover, there were no signi cant differences in the clinical characteristics between the primary and validation cohorts (P > 0.05).

Feature selection and radiomics signature building
After analysis of LASSO (Fig. 3.A-B), four factors were selected from quantitative parameters: pleural thickening, total volume of the lesion, ratio of consolidation volume to whole lung volume and ratio of lesion volume to whole lung volume.The radiomics signature was set up as Rad-score = 0.044408621 × pleural thickening + 0.424464103 × total volume of the lesion + 0.419327051 × ratio of consolidation volume to whole lung volume + 0.575642290 × ratio of lesion volume to whole lung volume + 0.006363664.

Diagnostic Validation of Radiomics Signature
The model score was signi cantly different between none-severe and severe patients in the primary cohort (P < 0.001), which was further con rmed in the validation cohort (P < 0.001).The area under the ROC curve (Fig. 3.C-D) for identifying the severe and critical patients based on the model was 0.943 and 0.941 in the primary and validation cohorts, respectively.In primary cohort, the accuracy, sensitivity and speci city for evaluation of the clinical condition were 0.885, 0.880 and 0.885, respectively.Correspondingly, the accuracy, sensitivity and speci city were 0.856, 0.842 and 0.859 in validation cohort.The calibration curve of the radiomics signature for the probability of severe and critical condition of COVID-19 patients indicated good agreement between prediction and observation in the primary cohort (Fig. 3.E), which was then con rmed in the validation cohorts (Fig. 3.F).2).The model combing the above three independent predictors were developed and presented as the nomogram (Fig. 4).

Validation of the Radiomics Nomogram
The calibration curve of the radiomics nomogram for the probability of severe and critical condition of COVID-19 patients indicated good agreement between prediction and observation models in the primary cohort (Fig. 5.A), which was then con rmed in the validation cohorts (Fig. 5.B).The area under the curve of ROC (Fig. 5.C-D) for identifying the severe and critical patients based on the radiomic nomogram was 0.972 and 0.978 in the primary and validation cohorts, respectively.In primary cohort, the accuracy, sensitivity and speci city for evaluation of the clinical condition were 0.897, 0.880 and 0.900, respectively.In validation cohort, the corresponding accuracy, sensitivity and speci city were 0.923, 0.894 and 0.929.

Clinical Use
The decision curve analysis for the radiomics nomogram was shown in Fig. 7.The decision curve showed that if the threshold probability of a patient is more than 3%, using the radiomics nomogram to identify severe patients adds more bene t than either treat all as severe patients or none severe patients.When radiomics signature was combined with clinical risk factors (radiomics nomogram), an improved bene t net was achieved.

Discussion
In this study, we developed and validated a radiomics nomogram based on the quantitation of lung abnormalities on CT images caused by COVID-19 to identify the severe patients for guiding a prompt management and treatment.The radiomics nomogram incorporates three items of the radiomics signature, comorbidity and abnormal WBC counts.The radiomics signature successfully strati ed patients according to their clinical conditions (severe or none-severe).The use of multi-task Unet network, which could segment the lesion or lung abnormalities related to COVID-19 automatically, increased the potential value of the radiomics nomogram in evaluating the clinical condition of patients with COVID-19.
Lung parenchymal is the main attacked and damaged organ by SARS-CoV-2.Previous studies were limited to qualitative analysis, merely focusing on the manifestation of COVID-19 on chest CT images to screen potential new cases of COVID-19 [13,15] .Nevertheless, it is of great necessity to assess the severity of patients with COVID-19 before treatment, which may greatly determine the clinical prognosis.We rstly assessed the lung abnormalities associated with COVID-19 by quantitative analysis, and then developed and validated a radiomics signature to identify severe COVID-19 patients.The results in present study uncovered that the radiomics signature could get a better performance in discriminating the severity of COVID-19 patients with an AUC of 0.943 in primary cohort, which was then further con rmed in validation cohort with an AUC of 0.941.Thus, the radiomics signature was effective for identi cation of severe and critical COVID-19 patients.Notably, when combined with clinical risk factors including comorbidity and abnormal WBC counts, the discrimination potency was improved with an AUC of 0.972 and 0.978 in the primary and validation cohorts, respectively.Thus, we think that the noninvasive radiomics signature, which makes the most of the chest CT images, may serve as a practical method for identi cation of severe and critical COVID-19 patients.
The radiomics signature includes four parameters of pleural thickening, total volume of the lesion, ratio of consolidation volume to whole lung volume and ratio of lesion volume to whole lung volume, which were obtained automatically by computer-aided system or Multi-task Unet network.Presently, COVID-19 has reached the stage of a pandemic, which contributed to an extreme shortage of clinicians and radiologists.The application of arti cial intelligence technology or computer-aided system, a noninvasive, fast, reproducible technique, to assess the COVID-19 could alleviate the insu ciency of radiologists to some extent.Furthermore, patients with COVID-19 would bene t from a timely and accurate assessment of the severity through radiomics signature before getting a prompt and proper treatment.
It is unexpected that increased total volume of the lesion, ratio of consolidation volume to whole lung volume and ratio of lesion volume to whole lung volume, are associated with severe COVID-19 patients.
The more extensive involvement of lung parenchymal, the more severe condition it would be.The appearance of GGO indicates that alveolar cavity is partially lled by uid and cells to the layer against the alveolar walls [25] , while the consolidation sign demonstrates that the disease progresses due to further accumulation of exudates in alveolar cavity and aggravation of interstitial edema [25] .The chest CT features of COVID-19 are manifested as multiple patchy GGOs with or without consolidation distributed in subpleural areas of bilateral lungs [15] .When the volume of consolidation increases, more alveolar cavities are lled completely with exudates, resulting in dysfunction of oxygen exchange and oxygenation.Then, a respiratory failure occurs, which is presented as a severe condition.Above all, our study was the rst to quantify the lesion of GGO and consolidation to investigate its value in identi cation of severe patients with COVID-19, and to build a useful radiomics signature for clinicians.
Additionally, clinical features including comorbidity and abnormal WBC counts were independent risk factors contributing to worse clinical condition of patients with COVID-19.According to a previous study, presence of comorbidity is an essential factor in determining the prognosis of several diseases, especially pneumonia [26] .Therefore, we also has taken comorbidity into consideration in the present study and found a positive correlation with the severity of COVID-19, which was consistent with the study of Wang D et al [27] .CRP is an important in ammatory index.Although a signi cant difference in CRP increase was indicated by univariate analysis in primary and validation cohorts, it was not an independent predictor for identi cation of clinical condition of COVID-19 in this study.The main reasons may be that (a) CRP is a common signal for responding to in ammation; (b) the change of CRP is analyzed as a categorical variable, which may lead a bias to subtle difference.Moreover, Viral infections in the human body primarily involve damage to the immune system, which presents as decrease in the absolute number of lymphocytes and leukocyte [28] .In this study, we found that leukocyte and lymphocytes differed between severe and none severe patients with COVID-19, which is consistent with the study of Wang D et al [27] .In addition, WBC (leukocyte) is an independent predictor for identi cation of clinical condition of COVID-19.Interestingly, 9 severe patients presented with an increased WBC counts, which may be ascribed to other infections, such as bacterium.Comprehensively, a severe and critical patient with COVID-19 may be caused by cytokine storm, comorbidity with various infections (9 patients with increased WBC counts) and immune dysfunction.In a word, incorporating clinical features into radiomics nomogram could improve its diagnostic value of severe and critical cases with COVID-19.
The most important application of the radiomics nomogram is to guide management and treatment of patients with COVID-19, especially for severe and critical cases who need additional treatment or care.
According to recent reports and recommendations, severe and critical patients with COVID-19 need hospitalized therapy.Besides antiviral therapy, some additional treatment should be added for severe and critical patients [27,29] .To block cytokine storm, a blood-purifying therapy including plasmapheresis, hemoperfusion is recommended, which can reduce the damage of in ammatory reaction to the body or lung [7] .If possible, convalescent plasma therapy could be a preferred scheme for treatment of severe and critical patients [7] .Using the nomogram, we can quickly and precisely identify the severe and critical patients with COVID-19, and prompt a timely additional treatment and care to improve prognosis.On the other hand, COVID-19 is a dynamic disease [14,30] , a quantitative radiomics nomogram is helpful to follow up the changes of patients after treatment.To justify the clinical practicability of radiomics nomogram, decision curve analysis was applied in this study.This novel method offers an insight into clinical consequences based on threshold probability, from which the net bene t could be derived (Net bene t is de ned as the proportion of true positives minus the proportion of false positives, weighted by the relative harm of false-positive and false-negative results).The decision curve in our study showed that if the threshold probability of a patient was more than 3%, using the radiomics nomogram to identify severe or critical patients added more bene t than either treat all as severe patients or none severe patients.
Admittedly, our study has several limitations.The sample size in our cohort is relatively small.The relationship of radiomics to prognosis has not been studied due to time limitation.Thus, a further study with more cases and prolonged period should be conducted in the future.
In conclusion, we present an easy-to-use radiomics nomogram to identify the severe patients of COVID-19 for guiding a prompt management and treatment.We believe that both clinicians and COVID-19 patients could greatly pro t from the use of the radiomics nomogram.For example, when total points are less than 17, the risk probability of severe COVID-19 is lower than 5%, and if the total points are higher than 37, the risk probability is greater than 95%.

Figures Figure 1 Flowchart
Figures

Figure 3 Feature
Figure 3

Table 1
Characteristics of patients in the primary and validation cohorts.

Table 2
Risk Factors for clinical condition of patients with COVID-19.