Patient Demographics
A total of 4,495 participants were prospectively enrolled in the study, with enrollment occurring from January 2018 to November 2023 at the 12 primary health care centers and a single tertiary care center. From the initial cohort of 4,495 participants, 132 individuals were excluded from the analysis for various reasons. Among these exclusions, 81 individuals reported an unproductive cough without an available bronchoalveolar lavage (BAL) sample, while 29 participants declined to participate in the study. Additionally, 4 individuals had a documented history of previous TB treatment, and 17 participants were pregnant. However, 1 individual's inclusion was deemed inconclusive due to uncertain results from either the CXR or GeneXpert MTB/RIF test (refer to Figure 1). A total of 4,363 individuals were ultimately included in the analysis. Among the 4,363 individuals included in the study, 680 had an unproductive cough, but their BAL fluid was accessible and available for analysis. The median age of the participants was 43.1 years, and 2,161 males (49.6%) and 2,202 females (50.4%) were included. Predominantly, fever symptoms were evident in the majority of participants, accounting for 2,565 individuals (58.8%) (refer to Table 1).
Hemoptysis and night sweats were reported by 437 (10%) and 306 (7%) participants, respectively. Among the individuals in the study cohort, 2,345 individuals (53.7%) were confirmed to be TB positive, while 2,018 individuals (46.3%) tested negative for TB (refer to Figure 1). Notably, within the subgroup with positive TB results, there were 2,161 (49.6%) males and 2,202 (50.4%) females (Table 1). Gender, fever, cough, hemoptysis, and night sweats exhibited significant associations (P-value <0.05) with the GeneXpert MTB/RIF test results (refer to Table 1). Moreover, gender, age, hemoptysis, night sweats and cough demonstrated significant associations (P-value <0.01) with the radiological and DecXpert results (refer to Table 1).
Table 1: Demographic and clinical characteristics of the study population stratified by GeneXpert, radiology, and DecXpert test results. The table presents the demographic data (gender and age groups) and clinical characteristics (symptoms and comorbidities) of the 4,363 individuals included in the study. The data is stratified based on the results of the GeneXpert MTB/RIF test (considered the gold standard), radiology interpretation, and the DecXpert Computer-Aided Detection software. Percentages are provided for each subgroup within the respective categories. The P-values indicate the statistical significance of the differences observed between the subgroups. CAD stands for coronary artery disease and CKD for chronic kidney disease.
|
Total
|
GeneXpert +ve
|
GeneXpert -ve
|
Radiology +ve
|
Radiology -ve
|
DecExpert +ve
|
DecExpert -ve
|
P value
|
Gender
|
|
|
|
|
|
|
|
*
|
Male
|
2161(49.6%)
|
991 (45.9%)
|
1170 (54.1%)
|
704 (32.5%)
|
983 (45.5%)
|
872 (40.3%)
|
995 (46%)
|
|
Female
|
2202 (50.4%)
|
1354 (61.5%)
|
848 (38.5%)
|
961 (43.6%)
|
712 (32.3%)
|
1192 (54.1%)
|
721 (32.7%)
|
Age
|
|
|
|
|
|
|
|
*
|
<20
|
349(8%)
|
161 (46.1%)
|
188 (53.9%)
|
114 (32.6%)
|
158 (45.2%)
|
154 (44.1%)
|
160 (45.8%)
|
|
21-40
|
1484(34%)
|
686 (46.2%)
|
798 (53.8%)
|
487 (32.8%)
|
601 (40.5%)
|
604 (40.7%)
|
677 (45.6%)
|
|
41-60
|
1571(36%)
|
727 (46.3%)
|
844 (53.7%)
|
516 (32.8%)
|
522 (33.2%)
|
640 (40.7%)
|
717 (45.6%)
|
|
>60
|
959(22%)
|
443 (46.2%)
|
516 (53.8%)
|
315 (32.8%)
|
301 (31.3%)
|
360 (37.5%)
|
439 (45.7%)
|
|
Symptoms
|
|
|
|
|
|
|
|
**
|
Fever
|
2565 (58.8%)
|
1374(53.6%)
|
1191(46.4%)
|
1093(42.6%)
|
1084(42.3%)
|
1354(52.8%)
|
1097(42.8)
|
|
Hemoptysis
|
437 (10%)
|
301 (68.9%)
|
136 (31.1%)
|
252 (57.6%)
|
114 (26.1%)
|
264 (60.4%)
|
115 (26.3%)
|
|
Night sweats
|
306 (7%)
|
104(34%)
|
202(66%)
|
56(18.3%)
|
82(26.8%)
|
86(28.1%)
|
171(55.9%)
|
|
Cough > 2 wks
|
3683(84.4%)
|
2946(80%)
|
737(20%)
|
2385(64.8%)
|
542(14.7%)
|
2593(70.4%)
|
353(9.6%)
|
|
Cough < 2 wks
|
680 (15.6%)
|
268(39.4%)
|
412(60.6%)
|
174(25.6%)
|
238(35%)
|
235(34.6%)
|
350(51.5%)
|
|
Comorbidities
|
|
|
|
|
|
|
|
|
Hypertension
|
306 (45%)
|
200 (65.4%)
|
106 (34.6%)
|
85 (27.8%)
|
72 (23.5%)
|
106 (34.6%)
|
73 (23.9%)
|
|
Diabetes
|
299 (44%)
|
104 (34.8%)
|
195 (65.2%)
|
74 (24.8%)
|
164 (54.9%)
|
92 (30.8%)
|
166 (55.5%)
|
|
Chronic lung disease
|
114 (17%)
|
22 (19.3%)
|
92 (80.7%)
|
16 (14%)
|
77 (67.5%)
|
19 (16.7%)
|
78 (68.4%)
|
|
Chronic liver disease
|
18 (2.6%)
|
2 (11.1%)
|
16 (88.9%)
|
1 (5.6%)
|
13 (72.2%)
|
1 (5.6%)
|
14 (77.8%)
|
|
Malignancy
|
9 (1.3%)
|
1(11.1%)
|
8 (88.9%)
|
0
|
7 (77.8%)
|
1 (11.1%)
|
7 (77.8%)
|
|
CAD
|
94 (13.8%)
|
42 (44.7%)
|
52 (55.3%)
|
30 (31.9%)
|
44 (46.8%)
|
37 (39.4%)
|
44 (46.8%)
|
|
CKD
|
96 (14.1%)
|
54 (56.3%)
|
42 (43.7%)
|
38 (39.6%)
|
35 (36.5%)
|
47 (49%)
|
36 (37.5%)
|
|
Radiologists demonstrated a positive predictive value (PPV) of 71%, suggesting that out of the 2,345 individuals identified as positive by GeneXpert MTB/RIF, 1,665 individuals were confirmed positive by radiologists (refer to Table 2). In contrast, DecXpert exhibited a higher PPV of 88%, where out of the 2,345 individuals identified as positive by GeneXpert MTB/RIF, 2,064 were subsequently confirmed as positive by DecXpert. Regarding the negative predictive value (NPV), radiologists achieved an 83.9% NPV. This indicates that among the 2,018 individuals classified as negative by GeneXpert MTB/RIF, 1,695 were confirmed as negative by radiologists. Conversely, DecXpert exhibited a higher negative predictive value (NPV) of 85%, suggesting that of the 2,018 individuals classified as negative for TB by GeneXpert MTB/RIF, 1,716 were confirmed as negative by DecXpert (refer to Table 2).
Table 2: Performance evaluation of DecXpert against GeneXpert MTB/RIF and radiologists. This table compares the performance of the DecXpert Computer-Aided Detection software against the gold standard GeneXpert MTB/RIF test and radiologists’ interpretations. The true positive, true negative, positive predictive value (PPV), and negative predictive value (NPV) are reported for each method. The PPV indicates the probability that a positive test result is truly positive, while the NPV represents the probability that a negative test result is truly negative. MTB stands for Mycobacterium tuberculosis, the causative agent of tuberculosis and RIF for Right iliac fossa.
|
Gene Xpert MTB/RIF
|
Radiologists
|
DecXpert
|
True positives
|
2,345
|
1,665
|
2,064
|
PPV (%)
|
100
|
71
|
88
|
True negatives
|
2,018
|
1,695
|
1,716
|
NPV (%)
|
100
|
83.9
|
85
|
Quantitative assessment
We evaluated the performance of different models for the identification of TB using the DecXpert score as a primary predictor (refer to Figure 2). Initially, Model 1, which relies solely on DecXpert scores for TB detection, demonstrated an area under the receiver operating characteristic (ROC) curve (AUC) of 0.85 (95% CI: 0.82-0.87), indicating a reasonably strong predictive ability (refer to Table 3 and Figure 2). Model 2, which incorporated both the DecXpert score and symptom information, displayed an improved AUC of 0.88 (95% CI: 0.83-0.92). When patient demographic information (specifically age and gender) was integrated with DecXpert scores in Model 3, the AUC further increased to 0.91 (95% CI: 0.88-0.94) (refer to Table 3 and Figure 2), indicating an enhanced predictive performance compared to that of earlier models. Furthermore, the development of a composite Model 4, which integrates the DecXpert score, symptom incidence, age, and gender, resulted in a greater AUC of 0.95 (95% CI: 0.90-0.97) (refer to Table 3 and Figure 2).
Table 3: Summarizes the performance of the DecXpert Computer-Aided Detection system with added patient demographics and clinical information. It shows the Area Under the Receiver Operating Characteristic Curve (AUC) and corresponding 95% confidence intervals (CI) for different combinations of patient demographics and clinical data. The table outlines the AUC values achieved by DecXpert scores alone, DecXpert scores combined with symptom information, DecXpert scores combined with age and gender information, and DecXpert scores combined with symptom information, age, and gender. Higher AUC values indicate better discrimination between individuals with and without active tuberculosis. The integration of additional clinical data progressively enhances DecXpert's diagnostic performance, as evidenced by increasing AUC values across the combinations.
Models
|
Components
|
AUC
|
95% CI
|
Model 1
|
DecXpert scores
|
0.85
|
(0.82–0.87)
|
Model 2
|
DecXpert scores + Symptom information
|
0.88
|
(0.83–0.92)
|
Model 3
|
DecXpert scores + Age + Gender
|
0.91
|
(0.88–0.94)
|
Model 4
|
DecXpert scores + Symptom information + Age + Gender
|
0.95
|
(0.90–0.97)
|
Performance of the DecXpert Algorithm Against the Gold Standard Molecular Reference GeneXpert MTB/RIF
Next, we assessed the performance of the proposed DecXpert model that has proven to be highly effective in the identification of TB cases from CXR images. The GeneXpert MTB/RIF test reports served as the established ground truth for evaluation. The DecXpert model, operating solely on CXR imaging data without incorporating patient symptoms, achieved an AUC of 0.85 (95% CI=0.82–0.87), indicated by the blue ROC curve in Figure 3. However, upon inclusion of age and gender, the performance of the DecXpert model improved, yielding an AUC of 0.91 (95% CI=0.88–0.94), indicated by the green ROC curve in Figure 3. These results indicate a high level of accuracy ranging between 85% and 91% relative to GeneXpert MTB/RIF test reports, which are considered the gold standard (refer to Figure 3), suggesting that DecXpert reports could potentially serve as a surrogate for GeneXpert MTB/RIF. Notably, when basic patient demographics, specifically age and gender, and patient symptoms such as cough, fever, hemoptysis and night sweats were omitted, the DecXpert model still demonstrated a strong 85% concordance with GeneXpert MTB/RIF testing.
Performance of the DecXpert Algorithm against 3 Board-Certified Radiologists
Analysing the overall cohort revealed that the DecXpert algorithm successfully identified 2,064 TB patients (88%) out of the total 2,345 GeneXpert MTB/RIF-confirmed positive TB patients, whereas the radiologists identified 1,665 TB patients (71%). Thus, using the DecXpert algorithm increased the overall TB case detection rate by approximately 1.23 times compared to radiologists. Examining the performance of the three board-certified radiologists as illustrated in Figure 4, the first radiologist achieved an AUC of 0.79 (95% CI: 0.74–0.84), the second radiologist achieved an AUC of 0.72 (95% CI: 0.67–0.76), and the third radiologist achieved an AUC of 0.75 (95% CI: 0.71–0.78). Notably, each radiologist's sensitivity/specificity point fell outside the 95% CI space of the ROC curve of the DecXpert model, indicating that their identification performance was inferior to that of the DecXpert model (Figure 4). Furthermore, within the unproductive cough and comorbidity subgroup, there was a considerable improvement in the TB case detection rate, according to DecXpert, which detected 303 patients (71.2%), whereas radiologists were able to detect only 244 patients (57.4%). This observation emphasises the enhanced performance of the DecXpert algorithm compared to that of radiologists, particularly within this subgroup, demonstrating its greater efficiency in identifying TB patients.
Qualitative assessment
DecXpert went through a validation process that focused on visualizing CXRs and highlighting specific areas in the image that are important for DecXpert to make decisions when classifying TB cases. The assessment highlights the algorithm's reliance on clinically relevant regions within the lung features extracted from CXRs of TB patients to guide its decision-making process. Figure 5 illustrates patient cases presenting highlighted important factors (identified regions) in patients with confirmed TB from the gold standard reference test GeneXpert MTB/RIF. The model relies on accurate visual information and does not consider misleading visual cues such as symbols, motion artifacts, embedded text or symbols, or imaging irregularities when making decisions. This finding demonstrated that DecXpert-related decision-making behaviour is primarily rooted in clinically relevant features.
Evaluation of DecXpert Algorithm's Suitability for Deployment in Remote Isolated Regions with Offline and Online Functionality and Minimal Hardware Needs
Subsequently, we evaluated the suitability of integrating DecXpert into the pre-existing CXR workflows within primary healthcare facilities and diagnostic centres, particularly for deployment in geographically isolated regions of the nation where computational hardware capabilities are limited. To this end, we examined both online and offline iterations of the DecXpert software and incorporated them into current TB CXR workflows for the purpose of TB screening and diagnosis at primary healthcare facilities and diagnostic centres. The investigation focused on the implementation of DecXpert across seven distinct providers of digital chest X-ray machines, including GE Healthcare™, Siemens™, Philips Healthcare™, FujiFilm Medical Systems™, Shimadzu Corporation™, Toshiba Medical Systems™, and Hitachi Healthcare™. This examination encompassed varying computational hardware setups, ranging from 500 MB to 16 GB of RAM, and spanning different versions of the Windows operating system (2000, 7, 8, and 10).
Additionally, the study investigated the compatibility of DecXpert with all five perspectives of CXR images—posteroanterior (PA), anteroposterior (AP), lateral, decubitus, and oblique views. This assessment was conducted at six remote and geographically dispersed locations within the northern region of India. Moreover, we aimed to ascertain the ease of use of DecXpert by the existing X-ray technicians at these sites within their current CXR workflow. DecXpert demonstrated seamless integration and compatibility with all CXR images from the seven vendors, supporting TB screening and diagnostic workflows. Notably, it functioned effectively with basic computational hardware, such as systems with 500 MB RAM and running Windows 2000. Furthermore, the on-site technicians at these healthcare facilities were easily trained on a simple four-step process for processing CXR images (refer to Figure 6-a,b,c,d). DecXpert was made available in both offline and online configurations which had the same functionality, and Figure 6 (a,b,c,d) depicts this straightforward process, wherein both the CXR images and patient demographics are uploaded to the DecXpert software, this process culminates in the creation of a probable diagnostic report in PDF format, which can be disseminated in both electronic and printed forms for assessment by a physician.