Active/Secondary Pulmonary Tuberculosis Classication Model Based on Deep Learning of CT Images and Its Clinical Applications

Objective: To develop a CT image-based deep learning framework (3D-ResNet) as a new technology for the early warning of active and secondary pulmonary tuberculosis. Methods: Chest CT images of patients with active pulmonary tuberculosis (n=1,160) and secondary pulmonary tuberculosis (n=1,131) diagnosed via bacteriological examination were retrospectively collected to analyze differences between the clinical and imaging presentations of the two diseases. Lung eld regions were presegmented by using a 3D Nested UNet model, and a 3D-ResNet model was developed for training the classication model. The data were randomly grouped at a ratio of 7:2:1 for training, validation and testing. Area under the curve (AUC), accuracy (ACC), recall and F1 scores were used as model evaluation metrics, and class activation mapping were used to evaluate the activation regions of interest. Results: Patients with active pulmonary tuberculosis were older than those with secondary pulmonary tuberculosis. Additionally, patients with active pulmonary tuberculosis were observed to cough more, and patients with secondary pulmonary tuberculosis experienced more chest pain. The differences were statistically signicant (p<0.05). The AUC values of the model on the validation and test datasets were 0.948 and 0.945, respectively, with ACC values being 0.973 and 0.969, Recall values being 0.941 and 0.949 and F1 scores being 0.977 and 0.969, respectively. Conclusion: This study demonstrated that 3D-ResNet can be used as a rapid auxiliary diagnostic tool for differentiating active pulmonary tuberculosis and secondary pulmonary tuberculosis, which can help patients with active pulmonary tuberculosis in receiving timely treatment, in reducing transmission and in avoiding overtreatment. Additionally, this tool can aid secondary pulmonary tuberculosis patients who are misdiagnosed as having active pulmonary tuberculosis.


Introduction
Pulmonary tuberculosis (TB) is one of the top ten causes of death throughout the world and remains a major public health issue [1][2][3] . In 2020, 9.87 million new cases of TB were recorded globally, with an incidence rate of 127 per 100,000. The estimated number of new TB patients in China in 2020 was 842,000 (833,000 in 2019), and the estimated TB incidence rate was 59/100,000 (58/100,000 in 2019).
Among the 30 countries with a high TB burden, China ranks second in estimated pulmonary TB incidence, after India (2.59 million) [4] . TB is caused by a Mycobacterium tuberculosis infection. If the sputum smear test result is positive and indicates infection, then the case is considered active pulmonary TB. In contrast to aggravating respiratory diseases, active TB can affect the lung tissues and increase the risk of death of patients. In clinical diagnosis, active pulmonary TB can be divided into smear negative and smear positive types. A tuberculous sputum smear can identify the occurrence of pulmonary TB, but the low detection rate of active pulmonary TB (approximately 40%) may cause active pulmonary TB patients to be unable to receive timely treatments, thus leading to the aggravation of active pulmonary TB [5][6][7][8][9] . At present, imaging technologies (as commonly used tools) serve an important role in diagnosing pulmonary TB [10][11][12] . The clinical examination mainly utilizes X-ray lms. X-rays can observe lung lesions and artefacts; however, the X-ray resolution cannot be maintained. Therefore, there is the possibility of missed diagnoses and misdiagnoses of smear-negative active pulmonary TB. Due to features such as high de nition and easy operation, chest CT scans can better visualize pulmonary cavity lesions and their surrounding lesions, thus making it clinically important for the diagnosis and evaluation of active pulmonary TB. Despite its superior performance, con rming active pulmonary TB via CT remains a labour-and time-intensive task. In countries where medical resources and specialist radiologists are scarce, conventional CT still exhibits limitations. In the treatment of pulmonary TB, various imaging manifestations of TB can be interconverted. As a result, the imaging manifestations of active pulmonary TB patients mainly involve atypical pulmonary TB diagnoses, which, may have a high misdiagnosis rate.
For example, when a patient with secondary pulmonary TB develops a pulmonary infection, it is di cult to distinguish through the use of CT, and the pulmonary infection may be misdiagnosed as being active pulmonary TB. Clinically, if the imaging manifestations of chronic destructive pulmonary TB have residual cavities, they can be misjudged as being active lesions. Therefore, the early and precise prediction and diagnosis of active pulmonary TB is an important clinical issue, and it is necessary to establish a model based on images and clinical heterogeneous data. At present, although innovations in molecular diagnosis (such as the GeneXpert system and Truenat) have greatly improved the speed and accuracy of TB diagnosis, these methods exhibit increased costs [13] . Lambin et al. [14] proposed the concept of radiomics. However, this method requires the extraction of a large amount of feature information from the region of interest (ROI) of the image, as well as the use of various statistical analytic and data mining methods to extract and separate the key data from a massive amount of information, which involves a considerable amount of labour in preprocessing the data. Additionally, the ROI extraction requires consistency testing. In the process of applying machine learning to medical image analysis, meaningful feature extraction is crucial to the successful completion of the target task.
Traditional or task-related features are mostly based on the knowledge of human experts in the target eld, thus making it challenging for nonexperts to use machine learning technology for research. Deep learning technology can aid in overcoming such obstacles by incorporating the feature engineering steps in the learning process. Deep learning techniques transfer the burden of feature extraction engineering from humans to computers, thus enabling non-machine learning experts to effectively use deep learning techniques for research or applications in areas such as medical imaging. In this case, the automatic differential diagnosis of active pulmonary TB based on CT images will have great clinical practical value.
Moreover, arti cial intelligence is capable of processing a large number of high-dimensional information, which helps improve e ciency and reduce missed diagnoses and misdiagnoses.

Patients and dataset
This study retrospectively collected imaging and clinical data of patients with pulmonary infections who were admitted to the First People's Hospital of Kashgar from January 2019 to January 2021. This study was approved by the institutional review board of the participating hospitals, which waived the requirements of informed consent for each patient.
Patients meeting the following criteria were enrolled: (1) patients con rmed with active pulmonary TB and/or secondary pulmonary TB by using the Nissl staining method and microbial cultures of sputum smear microscopy or polymerase chain reaction tests; (2)  CT images and clinical presentations were collected within 1 month before collecting samples for pathogenic microbiological examinations. The complete owchart of the data collection is shown in Figure 1.
Image data CT scans were completed by using CT scanner (Siemens, Germany) with a conventional layer thickness of 10 mm, a layer spacing of 10 mm, a thin layer thickness of 2-3 mm, a layer spacing of 2-3 mm, a table feed speed of 5 mm/s, a voltage of 120 kV and a current of 180-240 mA. For the procedure, the patient was placed in the supine position with their arms raised, and the entire lung was scanned with a layer thickness and layer spacing of 5 mm each, as well as a post-reconstruction with a layer thickness of 0.625 mm. The images were observed through both the lung window and the mediastinal window.
Clinically, based solely on the CT image, it is challenging to distinguish patients with secondary pulmonary TB and patients with active pulmonary TB when patients are experiencing chest infections, dyspnea and fever, because all of their CT images show multiple nodules, funicular foci, patchy dense shadows and buds for both conditions (see Figure 2).

Construction of the deep learning model
The overall owchart of this study is shown in Figure 3. This study was divided into two parts: the rst part included data cleaning, image preprocessing and segmentation of the lung eld by using 3D Nested UNet; the second part used 3D ResNet to complete the classi cation of secondary and active pulmonary TB.

Data preprocessing
The standard storage format of the raw CT imaging data involved the use of DICOM. Before being input into the algorithm, the raw data were converted into a uniform format, and the dataset was cleaned to ensure that there were no dirty data in the dataset to interfere with the experimental results. Additionally, in order to obtain a more robust model, it was necessary to initially normalize the CT images by adjusting the window width and window position to [1600, -600] and by normalising the image intensity to [0,1].
The lung eld segmentation grid was segmented by using DICOM images of their original sizes. Due to the differences in the patients' body shapes, the volumes of lung elds that were segmented by different patients were quite different. When considering the computational constraints, all of the CT data were normalized to 256×256×64 and sent to the classi cation network for training.

Lung eld segmentation algorithm
This study adopted 3D Nested UNet architecture for segmentation. The network structure of this segmentation algorithm model is shown in Figure 4 below. It consists of encoder, decoder, dense skip path and deep supervision. The encoder is used to extract the image features. This project uses a residual block structure to implement feature extraction. The use of this structure can optimally solve the problem of gradient attenuation with the deepening of the network and can enable the deep network to be trained successfully. The decoder is used to fuse features and to restore the segmentation results. The dense connection is used to recover the features through skip connections to strengthen the learning of the features. Finally, the deep supervision structure can accelerate the convergence of the network. On this basis, in order to further improve the segmentation accuracy of the network, we improved the model from the perspective of the model itself by using weight attenuation for the convolutional layer, by adding a regularization term to the objective function and by limiting the number of weight parameters to prevent over tting. Dropout mechanism and Batch Normalization are used to make the data distribution more uniform and to suppress the activation of all of the neurons in the convolutional layer, thus further improving the model tting ability. We have implemented the segmentation of lung elds in a 3D Nested UNet network based on the pretrained public dataset (Finding and Measuring Lungs in CT Data, https://www.kaggle.com/kmader/ nding-lungs-in-ct -data). To verify the effectiveness of the pretrained lung eld segmentation model, we invited two radiologists with 5 years of diagnostic experience to double-blindly review the segmentation results and to modify them, in order to obtain the nal segmentation results.
The loss function in the model was set to cross-entropy + dice con dence.
Deep learning classi cation algorithm The ResNet model was rst applied to 2D image classi cation. However, for 3D medical image data such as CT data, spatial information is crucial in the classi cations of active pulmonary TB and secondary pulmonary TB. The 3D deep learning model can better identify anatomical locations and pathological features from CT images. To make the most of the context information in 3D CT images, this paper expanded on the original ability of ResNet to support volume data and used volume-level 3D-ResNet instead of slice-level 2D-ResNet. Speci cally, this paper improved the residual module of the original ResNet model and adjusted all of the 2D versions of the convolutional layer, batch normalization layer and pooling layer to the 3D version, as shown in Figure 5. Moreover, Figure 3a represents the original 2D residual module, and Figure 3b represents the 3D residual module that was proposed in this paper. This study aimed to train the classi cation model of active pulmonary TB and secondary pulmonary TB based on 3D-ResNet. With the size smaller z-direction of the input taken into account, ResNet50 was used as the underlying network framework. For model training, the batch size was set to 2, the initial learning rate was set to 0.1, and it attenuated by half every 1,000 iterations. We used the PyTorch framework to train the model on 2 GeForce RTX 1080 Ti GPUs for up to 4,500 iterations, and the training time was 3 hours 47 minutes 11 seconds. The loss curve of model training is shown in Figure 6. The loss curve showed that at 4,000 iterations, the model is in a tting state; conversely, once 4,500 iterations is reached (corresponding to an epoch value of 20), the model is in an ideal training state. The 3D-ResNet trained for 4,500 iterations was chosen as the nal classi er for testing. Figure 7 shows that as the number of epochs increased, the evaluation metrics also tended to stabilize.

Model visualization
To explain the prediction of the network, we used class activation mapping to visualize the most representative regions of the predicted disease on the CT scan. Figure 8 (a) shows the CT image lung eld region and class activation mapping of 4 active pulmonary TB patients, and Figure 8 (b) shows the CT image lung eld region and class activation mapping of 4 active pulmonary TB patients.
According to the results in Figure 8, the CAM activation region was concentrated in features such as solid lesions, ground glass shadows, nodules, bronchiectasis, tree buds, cavities, brous cords, lung destruction and pleural effusion, which had certain differences between active and secondary pulmonary TB.

Evaluation of model results
To quantitatively evaluate the performance of the deep learning model, ROC curves were plotted, and Area under the curve (AUC), accuracy (ACC), recall and F1-score values were calculated. The diagnostic performance of the selected classi er on the training set and the test set is shown in The Python version 3.0 sklearn package was used to plot the subject operating characteristic curve (ROC). The ROC curves of our model for the training set, validation set and test set are all plotted in Figure  9.

Discussion
Several deep learning methods that can distinguish active pulmonary TB from normal lungs on chest radiographs have been reported [15][16][17][18] . The present study is different from previous existing studies. In this study, we developed a deep learning method based on chest CT images, which demonstrated good accuracy in distinguishing active pulmonary TB from secondary pulmonary TB. The automatic activation region of the deep learning model was highly consistent with the region that was identi ed by the radiologist, which indicated that deep learning was able to automatically detect pulmonary abnormalities (solid lesions, bronchiectasis, tree buds, nodules, ground glass shadows and bre cords). The advantage of this diagnostic model is that it eliminates the need to manually plot the ROI. In addition, compared with radiologists, our model achieved higher diagnostic accuracy and performance.
The morphologies of the lung regions with pulmonary TB lesions are characterized by polymorphism, multisite, multinodular and cavity features, thus making it di cult to classify and distinguish secondary pulmonary TB from active pulmonary TB. At present, there are two main methods for identifying active pulmonary TB on CT images: manual identi cation and computer-aided identi cation. Manual identi cation is more labour-intensive and may lead to misdiagnoses under intense work pressure.
Computer-aided identi cation mostly uses traditional methods and has a poor identi cation performance. After studying a large number of relevant studies and methods, this paper conducted effective research on the classi cation of secondary pulmonary TB and active pulmonary TB and generated an effective classi cation model. Our deep learning model offers various advantages, such as stability, repeatability and easy adaptions.
According to previous studies, the use of deep learning models in pulmonary TB diagnosis has shown good results. Xukun L [19] used a state-of-the-art 3D convolutional neural network (CNN) model to collect 501 CT image datasets from 223 patients with active pulmonary TB and to collect 501 datasets as negative samples from healthy people. The detection recall rate and accuracy of this algorithm obtained values of 85.9% and 89.2%, respectively. Li Wang [20] retrospectively collected chest CT images from 301 cases of nontuberculous mycobacterial lung diseases and from 804 cases of tuberculous mycobacterial lung diseases, as con rmed by pathogenic microbial examinations. They then developed a 3D ResNet model and randomly extracted data for training, validation and testing. The AUC values of the model on the training, validation and test datasets were 0.90, 0.88 and 0.86, respectively. Hwang [21] developed a deep learning automatic detection (DAD) algorithm using 5,421 normal CXRs and 6,768 active lung MTB CXRs. The sensitivity and speci city values of this algorithm were 94.3% and 91%, respectively. Additionally, the classi cation performance was signi cantly higher than the performances of the 15 physicians who were included in the study. Seowoo Lee [18] retrospectively collected chest radiographs of successfully treated patients with pulmonary TB in a multicentre continuous cohort, as well as normal chest radiographs. These radiographs were used to train a neural network to calculate the probability of active pulmonary TB and cured pulmonary TB. The AUC value (0.83) of this model was higher than the scores of the pulmonologists. Moreover, An Yang [22] constructed a deep network model that combined UNet and fast RCNN, and they optimized it via the Res block and detected and recognized TB lesions in preprocessed CT images. The sensitivity, speci city and accuracy values of the model were 0.877%, 0.865% and 0.885%, respectively.
Our study was different from previous studies in some aspects. First, our objective was to classify secondary pulmonary TB and active pulmonary TB according to CT images, because their CT manifestations are similar and di cult to distinguish, even for senior radiologists. When compared with previous studies, our objective was relatively di cult to achieve; however, our model still performed well, with model AUCs of 0.948 and 0.945, ACCs of 0.973 and 0.969, Recall values of 0.941 and 0.949 and F1 scores of 0.977 and 0.969 on the validation and test datasets, respectively. These data showed that the model can distinguish between active pulmonary TB and secondary pulmonary TB, and it can also reduce the missed diagnoses and misdiagnoses of active pulmonary TB. Additionally, the model can reduce the chances of infection, can enable patients with active pulmonary TB to receive timely treatment, can reduce the misdiagnoses of patients with secondary pulmonary TB as active pulmonary TB and can protect these patients from using repeated courses of anti-TB drugs and avoid overtreatment. Moreover, we were able to easily collect 2,291 chest CT images, which showed that the model can be promoted and used in multiple institutions that are equipped with CT equipment. In addition, pathogenic microbial examinations are more time-consuming when used to diagnose disease. However, our model shortened the diagnosis time, which further demonstrates the advantages of using this model.
Our study also had certain limitations. First, patients receiving speci c medications were not excluded from the data collection. There are no studies showing whether clinical manifestations and chest CT manifestations are affected by speci c medications. However, the CT data of patients who did or did not receive speci c drug treatments are common in clinical scenarios, which shows that our model is suitable for real-world medical scenarios. Second, the training, testing and validation data were collected from a single institution and there were no external testing data collected for an external validation of the model.
Third, our cohort did not include TB patients with human immunode ciency virus; therefore, the network performance of these patients is unknown. Fourth, the CT samples in this study were collected from hospitalized pulmonary TB patients with relatively large lesion regions. Thus, the current model may be less sensitive to small pulmonary TB lesions. Fifth, due to the false positive rate, the model may misjudge certain lesions. Therefore, physicians will still need to review the complete CT scan to con rm the results.

Conclusion
This study developed a personalized deep learning framework that uses conventional chest CT images to classify secondary pulmonary TB and active pulmonary TB with high accuracy. The model has the potential to be a screening tool for patients with active pulmonary TB and will lead to the early detection and selection of appropriate clinical treatment strategies.

Declarations Ethical Approval and Consent to participate
This study was approved by the institutional review committee of participating hospitals, which exempted the requirement of informed consent of each patient.
MN, HJLand XGZ designed the study. RBZ, MD, AA, AK and LL collected the data. RBZ, MD, AA, AKand BXQ analysed the data. MN, HJLand XGZ and MD draft the manuscript. All authors interpreted the results and revised the manuscript. All authors read and approved the nal manuscript. Tables   Tables 1 and 2 are not available with   a shows a diagnosed patient with active pulmonary TB (male, 60-years-old) with chest pain and dyspnea for 3 days, multiple central lobular nodules in the right lower lobe, left upper lobe and left lower lobe, and with cords and surrounding patchy dense shadowing.
b shows a diagnosed secondary pulmonary TB patient (female, 50-years-old) who experienced cough and sputum for 6 days, with multiple patchy shadows observed in the middle lobe of the right lung and the lower lobes of both lungs, adjacent to the pulmonary nodules and surrounded by tree buds.

Figure 3
Page 14/15 Overall owchart of the experiment Figure 4 Nested UNet network structure  Comparison of ROC results of training set, validation set and test set

Supplementary Files
This is a list of supplementary les associated with this preprint. Click to download. graphicalabstractimage.tif