CT-Based Quantitative Assessment of Coronavirus Disease 2019 Using a Deep Learning-Based Segmentation System: A Longitudinal Study

Coronavirus (COVID-19) catastrophic disease that The key steps in ghting against COVID-19 involve early detection and tracking of the treatment effects. A large number of studies highlighted computed tomography (CT) as a reliable method for early diagnosis and follow-up monitoring of the disease. However, there are limited data on quantitative analysis of the follow-up images. In this study, we used a deep learning model using a neural network with high accuracy in automatic segmentation and quantication to analyze the infected lesions on chest CT images. Pipeline: Illustration of Two Cases.Radiology: Cardiothoracic

Due to rapid progress of the disease, follow-up CT scans every 3-5 days have been recommended by clinicians to evaluate the response to treatment. Moreover, serial chest CT imaging performed at different time points is also effective in estimating the evolution of the disease from initial diagnosis to discharge. It has been reported that follow-up CT examination can evaluate the progression during the early stage of hospitalized COVID-19 patients [8,9]. However, these CT examinations undoubtedly put tremendous work pressure on radiologists. Moreover, subtle changes in follow-up CT scans are often ignored due to lack of computerized tools to accurately quantify the infected regions of the lungs and their longitudinal changes. Jin et al. [10] proposed an arti cial intelligence (AI) system based on CT images for fast COVID-19 diagnosis. AI, a relatively mature technology in the eld of medical imaging [11,12], has actively contributed to the ght against COVID-19 [13]. A previous study showed that not only AI using deep learning (DL) in chest CT images could quickly identify and automatically contour infected regions, but it could also accurately estimate their shape, volume, and percentage of infection (POI), as well as accurately locate the infected lung lobe and bronchopulmonary segment [13]. Thus, the use of AI to support screening and clinical medical decisions is a useful tool in ghting against COVID-19.
Most of the current research has focused on the screening and differential diagnosis of COVID -19, whereas studies of COVID-19 follow-up remain limited. The use of DL-based quantitative analysis of chest CT can not only help radiologists in assessment of a rising number of examinations but also help clinicians to acquire results much faster, in order to rapidly apply appropriate treatment. In this study, we used a DL-based segmentation system for quantitative assessment of the follow-up CT images of the whole lung and lobes affected by COVID-19 to measure the severity of the disease and the distribution of infection within the lung, and to provide the tracking of longitudinal changes during the course of treatment. In addition, this study also applied DL-based quantitative analysis to chest CT scans to standardize the discharge criteria of hospitalized COVID-19 patients with two consecutive negative nucleic acid tests.

Materials And Methods
Patients and study design This study was approved by our local institutional review board (IRB), and necessity for further consent was waived because the study was retrospective research that would pose no potential risk to patients and would not adversely affect the patients' right of welfare. From the hospital medical record system, we included 37 patients with COVID-19 con rmed by RT-PCR (mean age, 45±14 years; age range, 12-74 years; 54.1% men and 45.9% women), who were admitted to the hospital between January 1 and May 7, 2020. COVID-19 manifested with non-respiratory symptoms as well as respiratory symptoms, which were nonspeci c and of variable severity, ranging from minimal (n=5) and mild (n=31) to severe and critical disease (n=1). The inclusion criteria were as follows: 1) a positive result of RT-PCR test con rmed by the Centers for Disease Control (CDC), or positive antibodies against new coronavirus nucleic acid; 2) mild disease (non-severe and non-critical patients); 3) age ≥ 18 years; 4) availability of complete laboratory examinations and CT data; 5) successfully completed high-resolution thin-layer CT examinations during the study period, with a layer thickness ≤ 1 mm; 6) lung infection in CT images; and 7) more than two follow-up records of CT. The exclusion criteria were as follows: 1) minimal disease (positive nucleic acid test, but no infection in chest CT images) and severe or critical disease [14]; 2) contrast chest CT examination; 3) CT examination with slice thickness > 1 mm; 4) large motion artifacts or pre-existing obvious signs of other lung disease on CT images; and 5) only one follow-up CT record. The following discharge criteria were followed: 1) normal body temperature for more than 3 days; 2) obviously improved respiratory symptoms; 3) signi cant improvement of infected lesions on chest CT; 4) two consecutive respiratory samples tested negative for SARS-CoV-2 nucleic acid; and 5) the sample collection times were at least 24 hours apart.

Clinical data
Electronic medical records of the patients were reviewed. Speci c clinical information, including fever, cough, sore throat, fatigue, and muscle soreness, was collected from the medical records of all patients. Fever was de ned as a body temperature > 37.3°C. To con rm COVID-19, nasopharyngeal and oropharyngeal swabs received at a time interval of 24 hours were used for RT-PCR testing in each patient.
The RT-PCR test kits used in this study were manufactured by Da An Gene Co. (Guangzhou, China). In addition, the RT-PCR test was considered negative when two consecutive results were negative. Mild type of COVID-19 was de ned as patients with slight clinical symptoms and no signs of pneumonia on CT imaging. Mild-type COVID-19 was de ned as symptoms of fever and/or signs of respiratory infection, and signs of pneumonia on CT imaging. All of the patients underwent follow-up CT scans after antiviral drug treatment. Patient demographic characteristics, epidemiological characteristics, clinical manifestations, and laboratory results were collected.
CT image data acquisition A high-resolution CT scan was performed in all patients using multi-slice spiral CT scanners (Philips Brilliance-16, Philips iCT-256, Cleveland, USA). The scanning followed the common chest protocol with standard dose, where all patients were placed in a supine position, with arms raised, and held their breath during acquisition. The scanning range included whole lung volume. The following scanning parameters were applied: tube voltage, 120 kV; automatic tube current, 180 mA; pitch, 1.06; rotation time, 0.5 s; slice thickness, 0.8 mm; slice increment, 0.4 mm; image matrix, 1024 × 1024; and window width/level 1500/-600 Houns eld units (HU) for lung window setting.

DL-based network and quantitative assessment
We used a DL-based network called VB-Net (based on United Imaging Intelligence, Shanghai, China), which is a modi ed three-dimensional convolutional neural network [15,16]. This DL-based VB-Net system was developed to automatically segment and quantify the infected regions in CT scans of COVID-19 patients. After original CT images had been acquired, these DICOM data were fed to the VB-Net system. Segmentation was an essential step in VB-Net for image processing and analysis. It delineated the regions of interest (ROIs) in CT images, including whole lungs, lung lobes, and bronchopulmonary segments, as well as infected regions or lesions. After segmentation, various metrics were computed to quantify the degree of COVID-19 infection. The results of quantitative evaluation showed high accuracy (up to 91.6%) for automatic segmentation of the infected region and determination of percentage of infection (POI) metrics (15). In this study, we used VB-Net to quantitatively evaluate the lung infection regions of the initial and follow-up CT images. The volumes and POIs of infection were quanti ed for the whole lung and for each lobe. The histogram of Houns eld units (HU) within the infected region was also calculated to evaluate the density variation of pneumonia detected by DL, including ground-glass opacities (GGOs), consolidation components, and brosis inside the infected lesions. The ranges of CT numbers for GGOs, consolidation components, and brosis were set at <−300 HU, −300 HU to 49 HU, and ≥50 HU, respectively. The quanti cation provided a basis for tracking longitudinal changes of COVID-19 during the course of treatment from a CT perspective.

Statistical analysis
Statistical analyses were performed using IBM SPSS Statistics (version 19.0, SPSS Inc., United States). The measured data are presented as mean ± standard deviation (mean ± SD) or median and interquartile range (IQR, the rst to the third quartile) when the continuous data did not follow a normal distribution. The differences between the baseline and follow-up measurements were evaluated by Friedman test, which is a nonparametric test for multiple paired samples. P < 0.05 was considered statistically signi cant.

Initial and follow-up CT ndings
Fourteen patients underwent at least two chest CT scans, as per inclusion criteria. The follow-up CT scans every 3-5 days were recommended by clinicians to evaluate the therapeutic response in the early stage of COVID-19. In our cases, the interval of repeated CT scanning was 3-17 days. All initial CT scans showed signs of pneumonia after achieving positive RT-PCR results. The initial CT images of all 14 patients showed abnormal pulmonary ndings with irregular patches of pure GGOs (17.4%), the mixed pattern of GGOs and consolidation components (47.8%), and consolidation components with brosis (34.8%). There were three cases in which only one lung lobe was involved. In contrast, in 11 cases, multiple lobes were bilaterally affected, including 10 cases in the left upper lobe, 11 cases in the left lower lobe, 11 cases in the right upper lobe, 8 cases in the right middle lobe, and 12 cases in the right lower lobe. Apparently, the lower lobes of both lungs were often affected. A gradual decrease in GGO regions and increased consolidation components with reticulations and stripes were the main ndings at the follow-up CT scans. Finally, the latest CT scans showed complete absorption of GGO and reduction of consolidation components after antiviral and supportive treatment.
Quantitative results of the initial and follow-up CT images Some of the VB-Net system's predictions proposed in this paper are shown in Figure 1. Quantitative results from the initial and follow-up CT images obtained using VB-Net are presented in Figures 2 and 3 and Table 2. Table 2 shows that the median POI in the whole lung, left lung, and right lung was 3.4% (IQR 0.5%-8.4%), 0.8% (IQR 0.2%-6.7%), and 5.8% (IQR 0.5%-9.7%), respectively. This indicates that the infection was more serious in the right than in the left lung, and that the infected region was mainly distributed in the lower lobes of both lungs, especially in the right lower lobe with a median POI of 7.5% (IQR 0.7%-19.4%). In addition, the median POI of the infected region was 3.1% (IQR 0.5%-7.0%), 0.4% (IQR 0.0%-1.3%), and 0.0% (IQR 0.0%-0.1%) for the CT number ranges of <−300 HU, −300 HU to 49 HU, and ≥50 HU, respectively, indicating that the GGOs were the most extensive component, while consolidation and brosis were formed in some lesions. The 3D display and POI of the lesions are also shown in Figure 3.
In 14 patients, the quantitative analysis revealed signi cantly decreased POIs (p < 0.001) at the follow-up compared with the baseline (Table 2), especially after being tested negative on RT-PCR test. This showed that after antiviral and supportive treatment, the infection area gradually decreased, and the lesions of the whole lung, left lung, and right lung in those patients improved accordingly (Table 2). Three to seven days after receiving antiviral and supportive treatment, quantitative results of the rst follow-up revealed that most lesions improved and were absorbed. However, two patients showed obviously disease progression at the rst follow-up, demonstrating an increased infected area and consolidation components in bilateral multiple lobes. In addition, six patients showed an increase in the consolidation components compared with the initial scan. Three patients showed a slight disease progression at the second and third followups. All patients showed improvement at the last follow-up ( Figure 2). By the last follow-up, the infected lesions had been almost completely absorbed in nine patients, and the POIs were lower than 3% in other four patients. The infected lesions in these patients contained less than 2% of consolidation components and less brosis compared with the initial CT scans. However, one patient showed different ndings. Namely, because the infection area was large, involving 40.1% of the whole lung, and more consolidation components and brosis formed locally inside the lesions, the absorption time of the lesion was longer than in other patients. However, the POIs of all 14 patients showed that the CT number range of GGO and consolidation signi cantly decreased (p < 0.001). These follow-up results suggested that clinically relevant supportive treatment can be re ected by changes in POIs.
In addition, the RT-PCR test results of six inpatients again turned positive ("re-positive") during the followup period. However, the quantitative follow-up CT showed that there was no increase in the infected lesions. Our DL-determined discharge standard of the follow-up CT showed that the POI was reduced more than 50% or the lesion was completely absorbed compared with the initial CT scan. All 14 patients underwent the last CT before discharge, which showed more than 50% POI reduction rate of the lesions, and even the infected lesions in 6 patients were completely absorbed. Meanwhile, the body temperature remained normal for more than 3 days. There were no obvious respiratory symptoms, and the nucleic acid results were negative. Then, they were allowed to be discharged and isolated at home, and kept under health observation for 14 days. None of the 14 discharged cases had a subsequent positive nucleic acid test or manifested progression of pneumonia.

Discussion
CT imaging has become an effective tool for screening COVID-19 patients and assessing the severity of COVID-19 infection [17][18][19][20][21][22]. Considering that there is still no effective medicine to treat COVID-19, clinicians use different degrees of supportive therapy to intervene disease and then to see how disease progresses. To evaluate the therapeutic response in con rmed COVID-19 inpatients, follow-up CT examinations are required every 3 to 5 days. However, radiologists and clinicians still do not have a computerized tool to accurately quantify the treatment response. Deep learning (DL) has become a popular method in medical image analysis [23,24], and it has been used to screen for and diagnose COVID-19 on CT images [15,25,26]. Li et al. [25] used a DL model to train a large chest CT dataset; their results showed sensitivity of 90%, speci city of 96%, and accuracy of 0.96 in identifying COVID-19. The accurate segmentation by the DL system is a basis for quantitative assessment of CT images [15,27,28], which is necessary to track the progression of the disease and analyze longitudinal changes of COVID-19 throughout the treatment period [29,30]. In this study, we included the data from 14 patients with mildtype COVID-19 and evaluated longitudinally and quantitatively the changes on chest CT during COVID-19 by using a tool of a DL-based network (VB-Net). We believe that this DL-based CT system for COVID-19 quanti cation can facilitate clinical decisions on treatment. VB-Net system is an application of image processing used for segmentation of the lung, lung lobes, and lung infection [15]. It can provide accurate quantitative data for medical research, including quantitative assessment of disease progression at follow-up, comprehensive evaluation of severity, visualization and quanti cation of the lesion distribution using percentage of infection (POI). Shi et al. [31] used the VB-Net system to compute chest CT images of 2685 patients, and showed sensitivity of 90.7%, speci city of 83.3%, and accuracy of 87.9%. We used the VB-Net system to quantify longitudinal changes between the initial and follow-up CT scans of COVID-19 patients. Figure 2 shows a case of a 57-year-old COVID-19 patient with ve follow-up CT scans. The changes in infection volume as well as GGOs and consolidation components were clearly visualized by using the infection region segmentation method and surface rendering technique.
Dynamic radiological features on chest CT images of COVID-19 have already been reported [32]. In this study, we found that the distribution of lesions at the initial CT scan was predominantly in the bilateral lower lobes in 14 patients. GGOs accounted for the largest volume of the lesions, and some lesions showed consolidation changes and brosis, consistent with the results reported in previous studies [33][34][35][36][37][38]. Early brosis may correlate with good prognosis [39]. Six patients showed an increase in consolidation component at the rst follow-up, and two of them showed progression as re ected in increased infected area of multiple lobes and denser consolidation components. Finally, consolidation components were gradually absorbed. We also showed that the components in the lesions could be identi ed by CT number. Namely, for all lesions, the VB-Net system can calculate the POI with a CT number in the prede ned range. In this study, we found the highest POI for the CT number in the range <−300 HU, indicating that GGOs were major components of the lesions. These chest CT radiological features were consistent with COVID-19. The overall POIs of 14 patients showed that the range of CT number gradually decreased. These results indicate that the CT features of COVID-19 infected lesions can be visually displayed by using the DL-based CT system, which might help clinicians to manage patients with COVID-19.
Shan et al. [15] indicated that the POI estimated from CT scans correlated with the severity of pneumonia. With the VB-Net system, the POIs of the whole lung and lung lobes can be automatically calculated, and then, the severity of COVID-19 infection in the whole lung and each lobe can be quanti ed. Table 2 shows that the POI of the right lower lobe was higher than that of the other lobes, which is in agreement with the ndings reported in previous studies [36][37][38]. In addition, the research works of DL also can be helpful in predicting COVID-19 disease progression. Cao et al. [29] have reported that the use of voxel-level DLbased CT segmentation of pulmonary opacities can improve their quanti cation and assess longitudinal progression of COVID-19. Huang et al. [30] collected CT images from 126 patients and calculated the percentage of GGOs in chest CT images; they found that the quanti cation of the lung infection could be further used for monitoring of the progression of COVID-19. In our study, the whole lung and lesions were segmented by VB-Net system, and the POIs were calculated from follow-up CT scans of 14 COVID-19 patients. Our results suggested that the POIs of most patients showed a gradual downward trend after antiviral and supportive treatment (Figure 1). These follow-up results suggest that clinically relevant supportive treatment can be re ected in changes in POIs. Therefore, our study illustrates the potential of DL-based CT system to provide objective quantitative assessment of pulmonary infection as well as of response to treatment in patients with COVID-19.
In summary, DL plays an important role in the delineation of infected lesions and quanti cation of COVID-19 applications. It helps radiologists in accurately identifying lung infection and prompting quantitative analysis and diagnosis of COVID-19.
In addition, the discharge criteria of common COVID-19 patients were managed well by DL; however, six cases in our study showed "re-positive" results during the follow-up nucleic acid tests. The quantitative CT follow-up results showed that the absorption of the infected lesions gradually decreased. Currently, there are research reports on COVID-19 "re-positive" patients without worsening symptoms and chest CT ndings. Additionally, the patients were isolated at the infection ward of the hospital and were not exposed to other con rmed or suspected patients, which indicated that these "re-positive" tests were not the result of a re-infection. After obtaining the "re-positive" results, in the six patients, the samples from multiple sites, including the nasopharynx, throat, and anal swabs, were tested for COVID-19 nucleic acid for more than 3 days, but all the test results were negative. The quantitative results of CT before discharge showed that the reduction rate of POI in the lung lesions was more than 50%, and even the infected lesions in those six patients with re-positive tests were completely absorbed. After discharge, all the patients underwent 14 days of isolation and health surveillance at home.
To ensure that patients are completely cured, analysis of IgM and IgG COVID-19-speci c antibodies should be carried out for all discharged patients. Additionally, the CT scans of "re-positive" patients still showed abnormal lesions in the lungs at the follow-up visit, suggesting that more rigorous criteria are needed to evaluate the CT results so as to reduce the possibility of "re-positive" test results.
This study had several limitations. First, although DL has become an effective method in ghting against COVID-19, the CT data in COVID-19 patients may be incomplete and inaccurate, which creates di culty in training an accurate segmentation and diagnostic network. Meanwhile, our current AI study for quanti cation of the lesions was based on a small sample, which may have led to over tting of the results. Multicenter prospective studies with larger samples need to be conducted to further verify the conclusions of the present study. Additionally, our study did not analyze the COVID-19-speci c IgG-and IgM antibodies due to the unavailability of such data. Finally, more datasets should be established to include clinically collected CT images from patients with COVID-19.

Conclusion
The pandemic of COVID-19 continues. Every day, hundreds of thousands of new patients are diagnosed, which not only increases the diagnostic pressure on radiologists but also restricts the treatment capabilities of hospitals and intensive care units. The DL-based CT model can help radiologists to assess a rising number of examinations as well as help clinicians to acquire results much faster.
CT quantitative analysis might be an effective and important method for assessing the severity of COVID-19, which could guide clinicians to rapidly apply the appropriate treatment. Meanwhile, the objective evaluation of the course of the infection gives the possibility to choose the correct treatment, which could guide clinical decisions during management of patients with COVID-19. Additionally, CT characteristics and quantitative analysis based on AI should be evaluated strictly, and the discharge standard should be speci ed depending on the baseline, disease status, and other patient-related factors.
The follow-up of COVID-19 remains a challenge. We believe that CT-based quantitative assessment using DL has great potential to facilitate both the diagnosis and tracking of the management of COVID-19 patients. Declarations validation of data. QL: substantial contribution to analysis and interpretation of data, including assessment of CT scans and quanti cation for COVID-19. GJ: substantial contribution to conception and design, with oversight and leadership responsibility for the research. Substantial contribution to revision of the manuscript. JY: substantial contribution to conception and design throughout evolution and realization of the study idea, and management and coordination responsibility for the research activity planning and execution. LL: substantial contribution to conception and design throughout evolution of overarching research goals and aims, management and coordination responsibility for the research activity planning and execution, and nal approval of the manuscript. All authors read and approved the nal manuscript.

Funding
This research was supported by the Guangdong Second Provincial General Hospital Youth Research Project (grant number, YQ2018-007).

Availability of data and materials
The datasets used and/or analyzed in this study are available from the corresponding author on reasonable request.

Ethics approval and consent to participate
Ethics Committee of Guangdong Second Provincial General Hospital approved this study.

Consent for publication
Consent for publication has been obtained.