CT radiomics facilitates more accurate diagnosis of COVID-19 pneumonia: compared with CO-RADS

doi:10.21203/rs.3.rs-76981/v1

PDF

Research

CT radiomics facilitates more accurate diagnosis of COVID-19 pneumonia: compared with CO-RADS

https://doi.org/10.21203/rs.3.rs-76981/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 07 Jan, 2021

Read the published version in Journal of Translational Medicine →

You are reading this latest preprint version

Background

Limited data was available for rapid and accurate detection of COVID-19 using CT-based machine learning model. This study aimed to investigate the value of chest CT radiomics for diagnosing COVID-19 pneumonia compared with clinical model and COVID-19 reporting and data system (CO-RADS), and develop an open-source diagnostic tool with the constructed radiomics model.

Methods

This study enrolled 115 laboratory-confirmed COVID-19 and 435 non-COVID-19 pneunomia patients (training dataset, n = 379; validation dataset, n = 131; testing dataset, n = 40). Key radiomics features extracted from chest CT images were selected to build a radiomics signature using least absolute shrinkage and selection operator (LASSO) regression. Clinical and clinico-radiomics combined models were constructed. The combined model was further validated in the viral pneumonia cohort, and compared with performance of two radiologists with CO-RADS. The diagnostic performance was assessed by receiver operating characteristics curve (ROC) analysis, calibration curve, and decision curve analysis (DCA).

Results

Eight radiomics features and 5 clinical variables were selected to construct the combined radiomics model, which outperformed the clinical model in diagnosing COVID-19 pneumonia with an area under the ROC (AUC) of 0.98 and good calibration in the validation cohort. The combined model also performed better in distinguishing COVID-19 from other viral pneumonia with an AUC of 0.93 compared with 0.75 (P = 0.03) for clinical model, and 0.69 (P = 0.008) or 0.82 (P = 0.15) for two trained radiologists using CO-RADS. The sensitivity and specificity of the combined model can be achieved to 0.85 and 0.90. The DCA confirmed the clinical utility of the combined model. An easy-to-use open-source diagnostic tool was developed using the combined model.

Conclusions

The combined radiomics model outperformed clinical model and CO-RADS for diagnosing COVID-19 pneumonia, which can facilitate more rapid and accurate detection.

Critical Care & Emergency Medicine

COVID-19

Computed Tomography

Pneumonia

Radiomics

Machine learning

The ongoing pandemic of coronavirus disease 2019 (COVID-19) caused by a novel coronavirus "severe acute respiratory syndrome coronavirus 2" (SARS-CoV-2) has become a global threat [1, 2]. The high contagion of SARS-CoV-2 and the virulence to cause severe illness involving multiple organs has caused many countries into a dilemma for screening, diagnosing, and treatment with limited healthcare resources. As of September 5, a total of 26,654,344 worldwide confirmed cases and 875,400 deaths have been reported [3], and the numbers continue to grow. The nucleic acid test using reverse-transcription polymerase chain reaction (RT-PCR) for SARS-CoV-2 was regarded as the diagnostic gold standard but with various sensitivities ranging from 59–71% depending on viral load and test sample quality [4, 5]. Furthermore, the lengthy turnaround times for final diagnosis and shortage of RT-PCR kit will delay the treatment, which contributes to the dilemma.

Chest CT imaging is a widely available, time-saving, and non-invasive approach for detecting COVID-19 pneumonia. Previous studies revealed that chest CT could serve as an efficient tool for diagnosing COVID-19 pneumonia with high sensitivity and monitoring disease course [4, 6–8]. Recently, a multinational consensus statement from the Fleischner Society also declared that CT scanning can be a major method if symptoms worsen or there is a situation short of RT-PCR kit [9]. CT features including peripherally distributed ground-glass opacity (GGO), GGO with consolidation and/or reticulation were considered as typical imaging characteristics [6]. However, COVID-19 pneumonia shared similar imaging features with pneumonia caused by other pathogens, especially other viral pneumonia. The specificity was relatively low when compared to RT-PCR results [4], which meant CT could not fully exclude COVID-19 infection for suspected patients. Quarantine for those with final COVID-19 negative RT-PCR results increased stress on limited healthcare resources. As for distinguishing COVID-19 from other viral pneumonia on chest CT, high specificities but moderate sensitivities were reported among different international radiologists [10]. To facilitate the evaluation of COVID-19 pneumonia, a standardized assessment scheme for pulmonary involvement of COVID-19 named CO-RADS (COVID-19 reporting and data system) was developed to estimate the risk [11, 12]. The subjective CO-RADS classification demonstrated high discriminatory power but moderate to substantial agreement among observers. Hence, more measures should be taken for more rapid and accurate diagnosis of COVID-19 to combat the current pandemic.

Radiomics involved high-throughput extraction of a large number of quantitative features from medical images, thereby converting image data into high-dimensional data to objectively and quantitatively describe the characteristics of lesions that may not be perceptible by the naked eye. The potential benefits of radiomics had been highlighted in improving diagnostic, prognostic, and predictive accuracy for cancers such as lung cancer, rectal cancer, etc. as well as other non-neoplastic diseases [13–16]. To date, there are limited data about the value of chest CT-based radiomics in rapidly and accurately detecting COVID-19 pneumonia.

In the present study, we aimed to develop and validate a combined radiomics model including clinical characteristics and the radiomics signature for distinguishing COVID-19 from pneumonia with other etiologies by using real-world data during the COVID-19 outbreak period in China. Additionally, the predictive performance of the clinico-radiomics combined model was compared with the clinical model and CO-RADS grading approach by recruiting independent viral pneumonia cohort.

Patients

This study was approved by the Institutional Ethics Committee of Xinhua Hospital affiliated to Shanghai Jiao Tong University School of Medicine (No. XHEC-D-2020-090). The informed consent requirement was waived for this retrospective study using de-identified data. Clinical and non-contrast chest CT data of consecutive 115 patients with COVID-19 confirmed by RT-PCR from Bengbu City, Anhui Province (center I) as well as 1205 patients with respiratory symptoms from Xinhua Hospital (center II) were reviewed during the COVID-19 outbreak from December 20, 2019 to February 15, 2020. Patients with common pathogen confirmation and disease improvement on follow-up CT after treatment were grouped as non-COVID-19 pneumonia patients. The exclusion criteria were as follows: (a) lack of complete clinical records (blood test or pathogen confirmation); (b) normal or without acute pneumonia on CT images; (c) lack of follow-up CT images; (d) insufficient image quality due to the severe artifacts affecting the image assessment. Consequently, 95 COVID-19 and 415 non-COVID-19 pneumonia patients were recruited and semi-randomly allocated to the training and internal validation cohorts according to the recruitment time. Another 40 patients with viral pneumonia between February 16, 2020 and March 20, 2020 who met the inclusion and exclusion criteria as an independent and new cohort were included to further test the constructed models. Finally, 115 COVID-19 and 435 non-COVID-19 pneumonia patients were enrolled in this study. The workflow of this study was displayed in Fig. 1. Among the non-COVID-19 patients, 128 were confirmed viral infections, 195 mycoplasma infections, 5 chlamydia infections, 3 fungus infections, and 104 co-infections.

CT imaging acquisition and interpretation

All the patients underwent non-enhanced chest CT examinations for detecting pneumonia in the supine position during end-inspiration. The CT scans were performed with a 64-section multi-detector CT scanner (uCT780, United imaging or Somatom Definition Flash, Siemens Healthineers, or Light Speed VCT, GE Healthcare, or Acuilion, Toshiba Healthcare). The detailed imaging parameters for different scanners were demonstrated in Appendix E1.

Initial CT images before any treatment were performed by three experienced radiologists in consensus (HZ, LH and JL, with 9, 11 and 10 years of experience in thoracic imaging, respectively). The disputes between the radiologists were resolved by consulting another experienced radiologist (DW or ZX, with more than 20 years of experience in thoracic imaging, respectively). All of them were blinded to the results of laboratory tests.

The lesion number, distribution, density, extent, and other features were assessed. Lesion number included single or multiple lesions. Distribution included unilateral or bilateral lungs, peripheral or central or both of the peripheral and central sites. Density included pure GGO, GGO with consolidation, and pure consolidation. Other features consisted of reticulation (intralobular/interlobular septal thickening), air bronchogram, lymphadenopathy within the mediastinum or hilus, and pleural effusion. Lymphadenopathy was defined as the size of lymph node more than 10 mm in short-axis diameter.

The extent of pulmonary involvement was estimated using a semi-quantitative scoring system. Each of the 5 lung lobe involvements was scored from 0 to 5 as follows: 0 (0%), 1(< 5%), 2 (5%-25%), 3 (26%-49%), 4 (50%-75%), and 5 (> 75%) [17]. The total CT score was obtained by summing the scores of the five lobes ranging from 0 to 25.

Image segmentation and radiomics feature extraction

Three-dimensional (3D) segmentation of the entire volume of interest (VOI) of the pneumonia lesions were performed manually and independently by two experienced radiologists (radiologist 1 [HR] and radiologist 2 [RC], with 5 and 6 years of experience in thoracic imaging, respectively) via a free and widely used open-source software package (itk-SNAP, version 3.4.0, www.itksnap.org). The outline of the lesions was delineated along the border on thick-section images with lung window and excluded the intralesional vessels, bronchi, necrosis, and cavitation (Fig. 2). Both of them were blinded to theresults of laboratory tests.VOIs with a volume less than 125 mm³were excluded.

The interobserver and intraobserver reproducibility evaluation of radiomics feature extraction was performed using intraclass correlation coefficients (ICC). Totally 15 VOIs from each group in the training cohort were randomly chosen. The intraobserver ICC was calculated by comparing two segmentations of radiologist 1 (repeated 7-day interval). The interobserver ICC was calculated by comparing segmentation of radiologist 1 (first time) and radiologist 2. An ICC of 0.81 to 1.00 showed almost perfect agreement, 0.61 to 0.80 as substantial agreement, and 0.41 to 0.60 as moderate agreement [14].

Radiomics features were extracted from VOIs by using pyradiomics 3.0.0 version [18] (http://www.radiomics.io/pyradiomics.html). Images were preprocessed and processed using the proposed default setting. Six classes of radiomics features were extracted: 18 first order statistics features, 14 shape-based features (3D), 22 gray level cooccurrence matrix (GLCM) features, 16 gray level run length matrix (GLRLM) features, 16 gray level size zone matrix (GLSZM) features, and 14 gray level dependence matrix (GLDM) features. The radiomics feature details were shown in the pyradiomics documentation (https://pyradiomics.readthedocs.io/en/latest/ features. html). In addition, two image filters of wavelet and Laplacian of Gaussian were applied to the original image, respectively. Finally, 14 different image types were used for extracting radiomics features.

Development of clinical and clinico-radiomics combined models

For clinical model, univariate and multivariate logistic regression analysis were applied to select the independent predictors of clinical and radiological features for identifying COVID-19 pneumonia in the training cohort.

For clinico-radiomics model, minimum redundancy and maximum relevance (mRMR), and the least absolute shrinkage and selection operator (LASSO) logistic regression algorithm were used to select the best performed radiomics features in the training cohort. Then, Radscore was calculated for each lesion by using a linear combination of selected radiomics features and their weight coefficients. The mean Radscore (mRadscore) of lesions for each patient was used for predicting COVID-19 pneumonia. A clinico-radiomics combined nomogram was developed with the selected clinical variables and mRadscore by using multivariate logistic regression analysis.

Internal validation and clinical utility of clinical and combined radiomics models

The diagnostic performance of clinical and combined models was assessed by using the receiver operating characteristic curve (ROC) analysis, in which the areas under the curve (AUCs), accuracies, sensitivities, and specificities were established. Then, the diagnostic performance of the models was validated in the validation cohort. Calibration curves, obtained by plotting the actual COVID-19 pneumonia probability against developed model-predicted probability of COVID-19 pneumonia, were performed to assess the goodness-of-fit of the clinical and combined models.

Decision curve analysis (DCA) was implemented to evaluate the net benefits of the prediction models at different threshold probabilities in the validation cohort.

Predictive performance of combined radiomics model in distinguishing COVID-19 from other viral pneumonia compared with clinical model and CO-RADS

Another independent testing cohort including 20 patients with COVID-19 pneumonia and 20 patients with other viral pneumonia was used to test the discriminatory power for the clinical model, clinico-radiomics combined model, and CO-RADS category approach. The CO-RADS included 6 levels of suspicion for pulmonary involvement of COVID-19 besides CO-RADS 0, not interpretable (scan technically insufficient for assigning a score) as follows [11]: CO-RADS 1, very low (normal or non-infectious); CO-RADS 2, low (typical for other infection but not COVID-19); CO-RADS 3, equivocal/unsure (features compatible with COVID-19, but also other diseases); CO-RADS 4, high (suspicious for COVID-19); CO-RADS 5, very high (typical for COVID-19); CO-RADS 6, proven (RT-PCR positive for SARS-CoV-2). The detailed information for each level was demonstrated in Appendix E2.

The CO-RADS categories for the 40 patients were independently performed by twoexperienced radiologists who were familiar with the CO-RADS categories and blinded to laboratory results (HZ and JL, with 9 and 10 years of experience in thoracic imaging, respectively). The interobserveragreement was assessed by using Cohen kappa test, where 0-0.2 was slight agreement, 0.21-0.40 fair agreement, 0.41-0.60 moderate agreement, 0.61-0.80 substantial agreement, and 0.81-1.00 almost perfect agreement [19]. The discriminatory power for the three methods was compared.

Statistical analysis

Quantitative variables were described as mean ± standard deviation or median (inter-quartile range，IQR). The categorical data were expressed as the frequency (percentage). Comparisons of patient characteristics between COVID-19 and non- COVID-19 pneumonia groups were performed by independent two-sample t test, Mann-Whitney U test, and chi-squared test or Fisher’s exact test via SPSS 23.0 (IBM). Other statistical analyses were performed with R software (version 3.6.1, http://www.Rproject.org). Youden’s index was used to determine the optimal threshold that would maximize the sum of sensitivity and specificity for ROC analysis. The AUCs were compared by DeLong test [20]. A two-sided P<0.05 indicated a statistically significant difference.

Patient characteristics

The clinical and radiological features of the 550 patients in the training, validation, and testing cohorts were depicted in the Table 1 and Table 2. For clinical features, there were significant differences for age, cough symptom, white blood cell count, neutrophil ratio, and lymphocyte count in both of the training and validation cohorts. While compared the COVID-19 pneumonia with other viral pneumonia in the testing cohort, only C-reactive protein showed significant difference. For the radiological features, the lesion distribution was significantly different between the COVID-19 and non-COVID-19 groups for all the three cohorts.

Table 1

Clinical characteristics of patients with COVID-19 and non-COVID-19 pneumonia in the training and validation cohorts
Characteristics	Training cohort (n = 379)		P value	Validation cohort (n = 131)		P value	Testing cohort (n = 40)		P value
	non-COVID-19 group (n = 313)	COVID-19 group (n = 66)		non-COVID-19 group (n = 102)	COVID-19 group (n = 29)		non-COVID-19 group (n = 20)	COVID-19 group (n = 20)
Age, median (IQR), years	7.0 (26.0)	47.5 (20.5)	< 0.001	30.5 (49.2)	40.0 (20.5)	0.019	56.5 (30.5)	43.0 (13.0)	0.081
Gender (%)			0.006			0.587			0.749
Female	177 (56.5)	25 (37.9)		62 (60.8)	16 (55.2)		9 (45.0)	8 (40.0)
Male	136 (43.5)	41 (62.1)		40 (39.2)	13 (44.8)		11 (55.0)	12 (60.0)
Initial symptoms (%)
Snotty	6 (1.9)	3 (4.5)	0.407	0 (0.0)	1 (3.4)	0.221	0 (0.0)	1 (5.0)	0.500
Sore throat	7 (2.2)	4 (6.1)	0.201	9 (8.8)	6 (20.7)	0.15	0 (0.0)	2 (10.0)	0.468
Cough	292 (93.3)	52 (78.8)	< 0.001	89 (87.3)	17 (58.6)	0.001	15 (75.0)	16 (80.0)	> 0.999
Sputum	123 (39.3)	20 (30.3)	0.171	18 (17.6)	6 (20.7)	0.709	2 (10.0)	5 (25.0)	0.405
Fever	196 (62.6)	48 (72.7)	0.119	65 (63.7)	24 (82.8)	0.053	11 (55.0)	14 (70.0)	0.327
Dyspnoea	3 (1.0)	4 (6.1)	0.022	2 (2.0)	1 (3.4)	0.531	0 (0.0)	1 (5.0)	0.500
*Laboratory test (%)**
White blood cell count	127 (40.6) /17 (5.4)	9 (13.6) /15 (22.7)	< 0.001	22 (21.6)/ 5 (4.9)	0 (0.0)/ 6 (20.7)	0.014	7 (35.0)/ 0 (0.0)	3 (15.0)/ 3 (15.0)	0.098
Neutrophil count	70 (22.4)/ 16 (5.1)	10 (15.2)/ 3 (4.5)	0.402	28 (27.5)/ 2 (2.0)	8 (27.6)/ 1(3.4)	0.989	8 (40.0)/ 0 (0.0)	5 (25.0)/ 0 (0.0)	0.311
Neutrophil ratio	114 (36.4)/ 34 (10.9)	9 (13.6) / 4 (6.1)	< 0.001	43 (42.2)/ 12 (11.8)	5 (17.2)/ 2 (6.9)	0.017	9 (45.0)/ 0 (0.0)	5 (25.0)/ 0 (0.0)	0.185
Lymphocyte count	68 (21.7)/74 (23.6)	2 (3.0)/ 36 (54.5)	< 0.001	5 (4.9)/ 32 (31.4)	0 (0.0)/ 17 (58.6)	0.007	0 (0.0)/ 10 (50.0)	0 (0.0)/ 5 (25.0)	0.102
Lymphocyte ratio	38 (12.1)/ 107 (34.2)	2 (3.0)/ 34 (51.5)	0.009	10 (9.8)/ 43 (42.2)	1 (3.4)/ 15 (51.7)	0.449	0 (0.0)/ 9 (45.0)	0 (0.0)/ 5 (25.0)	0.185
C-reactive protein	165 (52.7)/ 0 (0.0)	43 (65.2)/ 0 (0.0)	0.065	72 (70.6)/ 0 (0.0)	22 (75.9)/ 0 (0.0)	0.578	18 (90.0)/ 0 (0.0)	11 (55.0)/ 0 (0.0)	0.013
IQR, inter-quartile range
*Data are shown in the order of elevated and decreased results for the laboratory tests.

Table 2

Radiological characteristics of patients with COVID-19 and non-COVID-19 pneumonia in the training and validation cohorts
Characteristics	Training cohort (n = 379)		P value	Validation cohort (n = 131)		P value	Testing cohort (n = 40)		P value
	non-COVID-19 group (n = 313)	COVID-19 group (n = 66)		non-COVID-19 group (n = 102)	COVID-19 group (n = 29)		non-COVID-19 group (n = 20)	COVID-19 group (n = 20)
Number (%)			< 0.001			0.001			0.025
Single	179 (57.2)	5 (7.6)		64 (62.7)	8 (27.6)		12 (60.0)	5 (25.0)
Multiple	134 (42.8)	61 (92.4)		38 (37.3)	21 (72.4)		8 (40.0)	15 (75.0)
Location (%)			0.131			> 0.999			0.339
Unilateral lung	33 (10.5)	3 (4.5)		6 (5.9)	2 (6.9)		4 (20.0)	1 (5.0)
Bilateral lungs	280 (89.5)	63 (95.5)		96 (94.1)	27 (93.1)		16 (80.0)	19 (95.0)
Distribution (%)			< 0.001			< 0.001			0.006
Peripheral	48 (15.3)	46 (69.7)		19 (18.6)	15 (51.7)		6 (30.0)	16 (80.0)
Central	24 (7.7)	1 (1.5)		3 (2.9)	1 (3.4)		1 (5.0)	0 (0.0)
Peripheral and Central	241 (77.0)	19 (28.8)		80 (78.4)	13 (44.8)		13 (65.0)	4 (20.0)
Density (%)			0.544			0.469			0.067
Pure GGO	76 (24.3)	13 (19.7)		20 (19.6)	8 (27.6)		11 (55.0)	4 (20.0)
Pure consolidation	75 (24.0)	14 (21.2)		23 (22.5)	4 (13.8)		3 (15.0)	4 (20.0)
GGO with consolidation	162 (51.8)	39 (59.1)		59 (57.8)	17 (58.6)		6 (30.0)	12 (60.0)
Accompanying features (%)
Reticulation	22 (7.0)	27 (40.9)	< 0.001	13 (12.7)	8 (27.6)	0.102	9 (45.0)	7 (35.0)	0.519
Air bronchogram	168 (53.7)	34 (51.5)	0.749	52 (51.0)	15 (51.7)	0.944	6 (30.0)	9 (45.0)	0.327
Other findings (%)
Pleural effusion	31 (9.9)	10 (15.2)	0.212	12 (11.8)	2 (6.9)	0.683	1 (5.0)	0 (0.0)	> 0.999
Lymphadenopathy	46 (14.7)	1 (1.5)	0.003	11 (10.8)	2 (6.9)	0.790	2 (10.0)	0 (0.0)	0.468
CT score, median (IQR)	4.0 (3.0)	5.0 (3.0)	< 0.001	3.0 (3.0)	5.0 (4.5)	0.042	2.5 (3.5)	4.5 (4.8)	0.157
IQR, inter-quartile range

Features Selection And Development Of Clinical And Clinico-radiomics Models

After univariate and multivariate logistic regression analysis in the training dataset, 8 clinico-radiological features were selected for building the clinical model, including age, gender, neutrophil ratio, lymphocyte count, location (lateral), distribution, reticulation, and CT score.

For clinico-radiomics model, a total of 783 lesions in 66 COVID-19 patients and 542 lesions in 313 non-COVID-19 patients were used for extracting radiomics features in the training dataset. Totally 1218 radiomics features were extracted for each lesion. The interobserver and intraobserver reproducibility of radiomics feature extraction was satisfactory with ICCs ranging from 0.7139 to 0.9999, and 0.7130 to 0.9999, respectively. After applying mRMR algorithm and LASSO logistic regression algorithm (Fig. S1), 8 best performed radiomics features were selected to calculate the Radscore including wavelet_LLH_glcm_ InverseVariance, wavelet_HHL_ firstorder_RootMeanSquared, wavelet_LHL_gldm_SmallDependenceHighGray LevelEmphasis, wavelet_LHL_glcm_ClusterProminence, log_sigma_1_0_mm_3D_ glrlm_LongRunLowGrayLevelEmphasis, wavelet_HLL_gldm_LargeDependence LowGrayLevelEmphasis, wavelet_LHL_gldm_LargeDependenceLowGrayLevel Emphasis, and wavelet_HHL_glrlm_LongRunLowGrayLevelEmphasis (Figure S2). A clinico-radiomics combined nomogram including the mRadscore and 5 selected clinical features was developed (Fig. 3).

Internal validation and clinical utility of clinical and clinico-radiomics models

The AUCs of clinical and clinico-radiomics model developed in the training cohort were 0.95 and 1.00. Favorable performance was observed in the validation cohort. The combined model outperformed clinical model in diagnosing COVID-19 pneumonia, with an AUC of 0.98 compared with 0.83. The sensitivity and specificity of combined model was improved to 0.94 and 0.93. The AUCs, accuracies, sensitivities, and specificities of clinical and combined models in the training and validation cohorts were depicted in Table 3. The ROC analysis results are displayed in Fig. 4. A visual open-source diagnostic tool transformed through the developed clinic-radiomics combined nomogram for diagnosing COVID-19 pneumonia can be achieved through the website (https://duansf.shinyapps.io/COVID-Model/). The detailed representations of the numbers for the clinical variables were demonstrated in Appendix E3.

Table 3

Predictive performance of clinical and clinico-radiomics combined models for diagnosing COVID-19 pneumonia in the training and validation cohorts
	Training cohort		Validation cohort
	Clinical model	Combined model	Clinical model	Combined model
AUC (95% CI)	0.95 (0.93–0.98)	1.00 (1.00–1.00)	0.83 (0.75–0.90)	0.98 (0.96-1.00)
Accuracy	0.92	0.94	0.79	0.93
Sensitivity	0.89	0.97	0.63	0.94
Specificity	0.93	0.99	0.84	0.93
AUC, area under the receiver operating characteristic curve; CI, confidence interval

Calibration curves showed that combined radiomics model demonstrated a better agreement between the predicted and actual probabilities of COVID-19 both in the training and internal validation datasets (Fig.S3). DCA revealed that the combinedradiomics prediction model was more beneficial than the clinical model, as well as the “treat-all-patients” or “treat-none” strategies when the threshold probability was from 0.0 to 1.0 (Fig. 5).

Predictive performance of clinical model, clinico-radiomics model, and CO-RADS category in distinguishing COVID-19 from other viral pneumonia

In the testing cohort, clinico-radiomics model outperformed clinical model in distinguishing COVID-19 from other viral pneumonia with an AUC of 0.93 compared with 0.75 (P = 0.03) (Fig. 6). In addition, the combined model also performed better than two trained radiologists by using CO-RADS. The AUC of radiomics model was higher than 0.69 for radiologist 1 (P = 0.008) and 0.82 for radiologist 2 (P = 0.15) (Fig. 6). The AUCs, accuracies, sensitivities, and specificities of clinical model, combined model, and CO-RADS in the testing cohort were demonstrated in Table 4. The interobserver agreement between the two radiologists was moderate with a kappa value of 0.53.

Table 4

Predictive performance of clinical model, clinico-radiomics combined model, and CO-RADS in distinguishing COVID-19 pneumonia from other viral pneumonia in the testing cohort
	Clinical model	Combined model	CO-RADS
	Clinical model	Combined model	Radiologist 1	Radiologist 2
AUC (95% CI)	0.75 (0.60–0.90)	0.93 (0.85-1.00)	0.69 (0.53–0.85)	0.82 (0.70–0.95)
Accuracy	0.65	0.88	0.68	0.78
Sensitivity	0.60	0.85	0.80	0.90
Specificity	0.70	0.90	0.55	0.65
AUC, area under the receiver operating characteristic curve; CI, confidence interval
CO-RADS, COVID-19 reporting and data system

In this study, we developed and validated a combined radiomics model for diagnosing COVID-19 pneumonia, and compared the diagnostic performance with clinical model as well as the performance of two trained radiologists by applying a recently recommended CO-RADS approach. Our results revealed that the combined radiomics model outperformed clinical model in diagnosing COVID-19 pneumonia in the training, validation, and testing cohorts, and not only for the common pathogens’ infection but also for the selective viral infection. The proposed combined model achieved favorable performances with AUC values of 1.00, 0.98, and 0.93 as well as a high sensitivity and specificity in the three cohorts. Furthermore, the combined model was also superior to CO-RADS in discriminating COVID-19 from other viral pneumonia with a sensitivity and specificity of 0.85 and 0.90.

Rapid and accurate diagnosis of COVID-19 is crucial for early intervention and healthcare allocation during the ongoing outbreak. Previous studies had explored the clinical and imaging features of COVID-19 for facilitating the diagnosis of COVID-19 pneumonia, revealing that fever and/or cough, normal or decreased white blood cells, and decreased lymphocyte count, GGO lesions in the peripheral and posterior lungs on CT images could aid in screening the highly suspicious patients [6, 21–23]. However, more common consolidation lesions could be detected due to the time interval from symptom onset and atypical features including fibrous stripes and irregular solid nodules were also presented in the subsequent studies, which complicated the diagnosis [8, 24]. Our study also found that older age, normal neutrophil ratio, decreased lymphocyte count, peripheral distribution on CT as well as higher CT score were independent predictors for distinguishing COVID-19 from non-COVID-19 pneumonia derived from the training cohort, which was in accordance with the above studies. Nevertheless, the predictive performance was not satisfactory with an AUC value of 0.83 and a sensitivity of 0.63 in the validation dataset. The various sensitivities and specificities of identifying COVID-19 subjectively with the clinical and radiological features were also found in the previous studies [4, 5, 10].

When evaluating the diagnostic performance of clinical model in discriminating COVID-19 from other viral pneumonia in the testing dataset, the discriminatory power further decreased with an AUC and sensitivity of 0.75 and 0.60. In the previous investigations conducting comparison between chest CT and RT-PCR results, the sensitivity of CT in identifying COVID-19 pneumonia can be estimated to 98%, but the specificity was only 25% by analyzing 1014 patients [4, 5]. Regarding the diagnostic performance among different radiologists from different countries in distinguishing COVID-19 from viral pneumonia on chest CT, the sensitivity, however, was reported to be moderate but the specificity was high [10]. Even by applying the recently recommended CO-RADS approach with reported high discriminatory power of AUC 0.91 in identifying COVID-19 [11], the AUC, sensitivity, and specificity in our study were not satisfactory with 0.69, 0.80, and 0.55, respectively for a trained radiologist familiar with CO-RADS approach, as well as 0.82, 0.90, and 0.65, respectively for another trained radiologist in distinguishing COVID-19 from other viral pneumonia. The moderate interobserver agreement with a kappa value of 0.53 was also not in favor of the accurate diagnosis of COVID-19. Therefore, it is urgent to develop a more objective approach for improving the current diagnostic accuracy of COVID-19 pneumonia.

Recently, artificial intelligence (AI) using deep learning technology has demonstrated good performance to improve the diagnosis of COVID-19, with sensitivities ranging from 0.67 to 0.90 and specificities from 0.83 to 0.96 [25–27]. However, deep learning requires a large amount of data to be trained, which limited its timely application based on the sporadic COVID-19 cases in most parts of China. More clinical implementations are warranted for the test of AI system and wide availability. Another machine learning approach radiomics rapidly developed in recent years can be widely available through open-source software and the radiomics signature is easily utilized. The potential for diagnosing and predicting outcomes of different lesions has been proven in the prior reproducible investigations [14, 15], as well as our previous studies in predicting preoperative synchronous distant metastasis in patients with rectal cancer [28, 29]. In this study, 8 radiomics features, mainly focus on the textural features, were selected to build the radiomics signature and the proposed combined radiomics model performed well not only in the training cohort but also in the validation and testing cohorts with AUCs of 1.00, 0.98, and 0.93, respectively. The high sensitivities and specificities with 0.97 and 0.99 in the training cohort as well as 0.94 and 0.93 in the validation cohort were observed.

It was reported that there were overlaps in imaging findings between COVID-19 and other viral infections, such as the coronavirus SARS-CoV and MERS-CoV pneumonia, as well as H1N1, H5N1, influenza, human parainfluenza virus, respiratory syncytial virus, rhinovirus, adenovirus, and so on [6, 23, 30]. Therefore, it is not difficult to understand that the textural features outperformed the other extracted morphological features or the first-order statistical features according to the histogram analysis. Textural features encoded the relationships between nearby voxels within VOIs, reflecting the intralesional heterogeneity. It is the advantage that radiomics can transform conventional medical images into quantitative and high-dimensional data visual analysis [31, 32]. To further test the robustness of the combined radiomics model, we enrolled an independent testing cohort including viral infection patients to assess the diagnostic performance. The AUC, accuracy, sensitivity, and specificity were satisfactory with values of 0.93, 0.88, 0.85, and 0.90, respectively. When compared with the clinical model and the CO-RADS for identifying COVID-19 pneumonia, the AUC value of combined radiomics model was significantly higher. The high sensitivity and specificity can not only facilitate to select the highly suspicious patients of COVID-19 for timely management, but also help to exclude the negative patients for relieving the stress of healthcare system. In addition, we further transform the clinico-radiomics nomogram into a visual open-resource diagnostic tool, which is easily used for rapid diagnosis of COVID-19.

Our study has several limitations. First, this was a retrospective study conducted in two centers. Prospective investigation with a larger sample size from more centers will be required to validate our proposed model. Second, since we enrolled the non-COVID pneumonia patients with blood laboratory pathogen-confirmation and pneumonia improvement after treatment by follow-up CT scans, limited bacterial infection cases were available due to the lack of bacterial culture. Third, center II were a general hospital with a strong pediatric medical center, thus many children with mycoplasma infections were included in our study. The median age was demonstrated significantly lower than that of the COVID-19 infection patients, where selection bias may exist. However, our non-COVID-19 pneumonia cases were consecutively enrolled from the real word data in our center, and the children was also proved to be susceptible for COVID-19, which definitely needed rapid and accurate differential diagnosis.

In summary, our preliminary study demonstrated that chest CT-based combined radiomics model outperformed clinical model and CO-RADS in diagnosing COVID-19 pneumonia. The useful quantitative radiomics signature can facilitate the rapid and more accurate diagnosis as well as timely management of COVID-19.

COVID-19 = Coronavirus disease 2019; SARS-CoV-2 = Severe acute respiratory syndrome coronavirus 2; CO-RADS = COVID-19 reporting and data system; RT-PCR = Reverse-transcription polymerase chain reaction; GGO = Ground-glass opacity; ROC = Receiver operating characteristic curve; AUC = Area under the receiver operating characteristic curve;

Ethics approval and consent to participate

The Institutional Ethics Committee of Xinhua Hospital affiliated to Shanghai Jiao Tong University School of Medicine approved this retrospective study and waived the informed consent requirement.

Consent for publication

Consent for publication was obtained

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Competing interests

One of the authors (S.F.D) is an employee of GE Healthcare. The remaining authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.

Funding

This work was supported by Special Research Program of Shanghai Municipal Commission of Heath and Family Planning on medical intelligence [No.2018ZHYL0108], National Natural Science Foundation of China [No. 81901695], and Shanghai Sailing Program [No.19YF1433100]. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Authors' contributions

D.B.W. and Z.Y.X. are the guarantors of the paper and take responsibility for the integrity of the work as a whole, from inception to published article. H.H.L., H.R, Z.B.W., H.X., S.H.Z., S.F.D., H.M.L., Z.Y.X., and D.B.W. conceived the study concept and design. H.H.L., H.R, J.N.L., L.H., R.M.C., H.Z., and Y.H.C. performed the acquisition of data. H.H.L., H.R., and S.F.D conducted data analysis and interpretation. H.H.L. and H.R. drafted the manuscript. Z.B.W., H.X., S.H.Z., J.N.L., L.H., R.M.C., H.Z., Y.H.C., S.F.D, H.M.L., Z.Y.X., and D.B.W contributed to critical revision of the manuscript. All the authors reviewed and approved the final manuscript.

Acknowledgements

We thank all the patients of this study for their participation.

World Health Organization. WHO Director-General's remarks at the media briefing on 2019-nCoV on 11 February 2020. 2020. at https://www.who.int/dg /speeches/detail/who-director-general-s-remarks-at-the-media-briefing-on-2019-ncov-on-11-february-2020. Accessed February 11, 2020.
World Health Organization. WHO Director-General's opening remarks at the media briefing on COVID-19–11 March 2020. at https://www.who.int/dg/speeches /detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19-11-march-2020, Accessed March 11, 2020.
Johns Hopkinds University &
Johns Hopkinds University &. Medicine CRC. (2020). https://coronavirus.jhu.edu (accessed September 5, 2020).
Ai T, Yang Z, Hou H, et al.Correlation of Chest CT and RT-PCR Testing in Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases. Radiology 2020; 200642.
Fang Y, Zhang H, Xie J, et al. Sensitivity of Chest CT for COVID-19: Comparison to RT-PCR. Radiology 2020; 200432.
Song F, Shi N, Shan F, et al. Emerging Coronavirus 2019-nCoV Pneumonia. Radiology. 2020;295:210–7.
Shi H, Han X, Jiang N,et al. Radiological findings from 81 patients with COVID-19 pneumonia in Wuhan, China: a descriptive study. Lancet Infect Dis. 2020;20:425–34.
Pan F, Ye T, Sun P, et al. Time Course of Lung Changes On Chest CT During Recovery From 2019 Novel Coronavirus (COVID-19) Pneumonia. Radiology. 2020;295:715–21.
Rubin GD, Ryerson CJ, Haramati LB, et al. The Role of Chest Imaging in Patient Management during the COVID-19 Pandemic: A Multinational Consensus Statement from the Fleischner Society. Radiology 2020; 201365.
Bai HX, Hsieh B, Xiong Z,et al. Performance of radiologists in differentiating COVID-19 from viral pneumonia on chest CT. Radiology 2020; 200823.
Prokop M, van Everdingen W, van Rees Vellinga T,et al. CO-RADS - A categorical CT assessment scheme for patients with suspected COVID-19: definition and evaluation. Radiology 2020; 201473.
Salehi S, Abedi A, Balakrishnan S, Gholamrezanezhad A. Coronavirus disease 2019 (COVID-19) imaging reporting and data system (COVID-RADS) and common lexicon: a proposal based on the imaging data of 37 studies. Eur Radiol. 2020. Doi:10.1007/s00330-020-06863-0. Online ahead of print.
Guiot J, Vaidyanathan A, Deprez L, et al. Development and validation of an automated radiomic CT signature for detecting COVID-19. medRxiv, 2020; 2020. 2004.2028.20082966.
Liu Z, Zhang XY, Shi YJ,et al. Radiomics Analysis for Evaluation of Pathological Complete Response to Neoadjuvant Chemoradiotherapy in Locally Advanced Rectal Cancer. Clin Cancer Res. 2017;23:7253–62.
Chen BT, Chen Z, Ye N,et al. Differentiating Peripherally-Located Small Cell Lung Cancer From Non-small Cell Lung Cancer Using a CT Radiomic Approach. Front Oncol. 2020;10:593.
Liu Y, Dong D, Zhang L, et al. Radiomics in multiple sclerosis and neuromyelitis optica spectrum disorder. Eur Radiol. 2019;29:4670–7.
Chang YC, Yu CJ, Chang SC,et al. Pulmonary sequelae in convalescent patients after severe acute respiratory syndrome: evaluation with thin-section CT. Radiology. 2005;236:1067–75.
van Griethuysen JJM, Fedorov A, Parmar C,et al. Computational Radiomics System to Decode the Radiographic Phenotype. Cancer Res. 2017;77:e104–7.
Chen X, Feng B, Li C,et al. A radiomics model to predict the invasiveness of thymic epithelial tumors based on contrastenhanced computed tomography. Oncol Rep. 2020;43:1256–66.
DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44:837–45.
Wang D, Hu B, Hu C,et al. Clinical Characteristics of 138 Hospitalized Patients With 2019 Novel Coronavirus-Infected Pneumonia in Wuhan, China. JAMA. 2020;323:1061–9.
Chen N, Zhou M, Dong X, et al. Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study. Lancet. 2020;395:507–13.
Chung M, Bernheim A, Mei X, et al. CT Imaging Features of 2019 Novel Coronavirus (2019-nCoV). Radiology 2020; 200230.
Pan Y, Guan H, Zhou S,et al. Initial CT findings and temporal changes in patients with the novel coronavirus pneumonia (2019-nCoV): a study of 63 patients in Wuhan, China. Eur Radiol. 2020;30:3306–9.
Bai HX, Wang R, Xiong Z,et al. AI Augmentation of Radiologist Performance in Distinguishing COVID-19 from Pneumonia of Other Etiology on Chest CT. Radiology 2020;201491.
Li L, Qin L, Xu Z,et al. Artificial Intelligence Distinguishes COVID-19 from Community Acquired Pneumonia on Chest CT. Radiology. 2020; 200905.
Wang S, Kang B, Ma J, et al. A deep learning algorithm using CT images to screen for Corona Virus Disease (COVID-19). medRxiv 2020; 2020.2002.2014. 20023028.
Cui Y, Liu H, Ren J, et al. Development and validation of a MRI-based radiomics signature for prediction of KRAS mutation in rectal cancer. Eur Radiol. 2020;30:1948–58.
Liu H, Zhang C, Wang L, et al. MRI radiomics analysis for predicting preoperative synchronous distant metastasis in patients with rectal cancer. Eur Radiol. 2019;29:4418–26.
Koo HJ, Lim S, Choe J, Choi SH, Sung H, Do KH. Radiographic and CT Features of Viral Pneumonia. Radiographics. 2018;38:719–39.
Gillies RJ, Kinahan PE, Hricak H. Radiomics: Images Are More than Pictures. They Are Data Radiology. 2016;278:563–77.
Lambin P, Leijenaar RTH, Deist TM,et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol. 2017;14:749–62.

OnlineFigureS1.Png
Radiomics feature selection using LASSO logistic regression model. (a) The hyper parameter (λ) was selected via ten-fold cross-validation based on minimum criteria. Log (λ) is plotted on the x-axis, and binomial deviance is plotted on the y-axis. The dotted vertical lines indicate optimal values determined using the minimum criteria and one standard error of the minimum criteria (1-SE). log (λ)=-6.71. (b) LASSO coefficient profiles of the radiomics features. Coefficient profiles are plotted against log (λ). The optimal 8 non-zero coefficients were generated at the value selected using ten-fold cross-validation.
OnlineFigureS2.Png
The selected radiomics features and their coefficients.
OnlineFigureS3.Png
Calibration curves of the clinical and combined radiomics model in the training cohort (a) and validation cohort (b).

PDF

Journal Publication

published 07 Jan, 2021

Read the published version in Journal of Translational Medicine →

Review #1 received at journal
05 Dec, 2020
Editorial decision: Major revision
05 Dec, 2020
Reviewer #2 agreed at journal
30 Nov, 2020
Review #2 received at journal
30 Nov, 2020
Reviewer #1 agreed at journal
12 Nov, 2020
Reviewers invited by journal
27 Sep, 2020
Editor assigned by journal
25 Sep, 2020
Editor invited by journal
24 Sep, 2020
Submission checks completed at journal
16 Sep, 2020
First submitted to journal
12 Sep, 2020

You are reading this latest preprint version

CT radiomics facilitates more accurate diagnosis of COVID-19 pneumonia: compared with CO-RADS

Status:

Journal Publication

Version 1

Abstract

Background

Methods

Results

Conclusions

Figures

Background

Materials And Methods

Results

Patient characteristics

Features Selection And Development Of Clinical And Clinico-radiomics Models

Discussion

Conclusion

Abbreviations

Declarations

Ethics approval and consent to participate

Funding

Authors' contributions

Acknowledgements

References

Supplementary Files

Status:

Journal Publication

Version 1