Artificial intelligence-based model for predicting pulmonary arterial hypertension on chest radiograph

doi:10.21203/rs.3.rs-3214956/v1

Download PDF

Research Article

Artificial intelligence-based model for predicting pulmonary arterial hypertension on chest radiograph

https://doi.org/10.21203/rs.3.rs-3214956/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 27 Feb, 2024

Read the published version in BMC Pulmonary Medicine →

You are reading this latest preprint version

Background

Pulmonary arterial hypertension is a severe condition. Despite advances in targeted treatments for it, diagnostic delays have yet to improve. Early referral to a specialized hospital is important because a longer diagnostic interval has been reported to be associated with decreased 5-year survival. Computer-aided detection and diagnosis (CAD) support detecting and diagnosing abnormalities and diseases. In this study, we developed and tested a deep-learning algorithm to detect pulmonary arterial hypertension using chest radiographs.

Methods

From the image archive of Chiba University Hospital, 259 chest radiographs from 145 patients with pulmonary arterial hypertension PAH and 260 radiographs from 260 control patients were identified; 418 were used for training, and 101 were used for testing. Using the testing dataset for each image, the algorithm outputted a numerical value from zero to one (the probability of the pulmonary arterial hypertension score). In addition, using the same testing dataset, the capability of the algorithm to identify pulmonary arterial hypertension was compared with that of professional doctors.

Results

The area under the curve of the receiver operating characteristic curve for the detection ability of the algorithm was 0.988. Using a score cutoff of 0.69, the sensitivity and specificity of the algorithm were 0.933 and 0.982, respectively. The performance of the algorithm was not inferior to that of the doctors.

Conclusion

We developed a deep-learning algorithm to detect pulmonary arterial hypertension using chest radiographs.

Pulmonary arterial hypertension

Artificial intelligence

Deep learning

Chest radiograph

Pulmonary arterial hypertension (PAH) is a subtype of pulmonary hypertension (PH) and characterized by elevated pulmonary arterial pressure due to increased pulmonary vascular resistance [1]. Symptoms of PAH are non-specific but commonly include exertional dyspnea and fatigue [2]. The estimated prevalence of PAH is 48–55 per 1 million adults in economically developed countries, and the typical course of PAH without treatment is death from right-sided heart failure. Currently, selective pulmonary vasodilators that act via three different pathways are available for treating PAH, and clinicians recommend an initial combination therapy [3].

Despite the establishment of treatment algorithms and reduced mortality due to PAH, the number of patients in the yellow and red zones, (intermediate- and high-risk groups) according to the risk stratification of PH by the European Society of Cardiology (ESC) and European Respiratory Society (ERS), remains high [4, 5].

One cause is the diagnostic delay in patients with PAH. The mean time from symptom onset to diagnosis is 47 ± 34 months [6]. Patients reported 5.3 ± 3.8 general practitioner visits before being seen at a PH center [6]. Although 95% of patients were classified according to the World Health Organization Functional Class (WHO-FC) II at symptom onset, the distribution of WHO-FC II, III, and IV in the same patients at diagnosis was 0%, 94%, and 6% [6].

Khou et al. reported that the mean time from symptom onset to diagnosis was 2.5 ± 4.1 years, and a longer diagnostic interval was associated with decreased 5-year survival. Although the treatment of PAH is advancing, delays in diagnosis have not improved [7]. General physicians need to suspect PAH on chest radiographs (CRs) in the early stages of the disease and refer patients to a PH center as appropriate [8]. Computer-aided detection/diagnosis (CAD) supports the detection and/or diagnosis of abnormalities and/or diseases. It can help physicians detect suspicious lesions that are easily overlooked, leading to improved detection accuracy [9].

CAD algorithms on CRs to detect pneumonia [10], lung nodules [11], pulmonary tuberculosis [12], and interstitial lung diseases [13] have been developed using deep-learning methods. Kusunose et al. developed and tested a deep learning algorithm to predict elevated pulmonary arterial pressure in patients with suspected PH using CRs. In their study, the area under the curve (AUC) of the artificial intelligence (AI) algorithm was 0.71, and the negative predictive value (NPV) of the AI algorithm for detecting PH was 95.0% [14]. However, a deep-learning algorithm for PAH has yet to be developed.

In this study, we developed and tested a deep learning algorithm using CRs of patients with PAH to help physicians detect PAH.

Study design

This retrospective study used data collected at Chiba University Hospital to develop and test a deep-learning algorithm using the CRs of patients with PAH to help physicians detect PAH. This study was performed in accordance with the Declaration of Helsinki. This clinical study was approved by Research Ethics Committee of the Graduate School of Medicine, Chiba University (approval number: 4203). All adult participants provided written informed consent to participate in this study.

Datasets

The dataset included 145 patients with PAH and 260 control patients who visited Chiba University Hospital between January 1, 2003, and December 31, 2020. Using a Swan-Ganz catheter, PAH was diagnosed based on a right heart catheter (RHC). The diagnosis of PAH was made using hemodynamic measurements according to the most recent World Symposium standards: mean pulmonary arterial pressure (PAP) > 20 mmHg, pulmonary arterial wedge pressure (PAWP) ≤ 15 mmHg, pulmonary vascular resistance (PVR) > 3 wood units [2, 15].

Because PAH is a rare disease, one to three CRs were used at different times during the clinical course of 145 patients with PAH (259 total images). The RHC-CR data pair from patients with PAH, performed within 3-day intervals, constituted the analysis dataset. The normal control group consisted of 260 patients who visited the Ophthalmology Department of Chiba University Hospital from January 1, 2015 to December 31, 2020 and had CRs without suspicion of PH (260 total images). Although 263 patients were initially included as normal controls, two respirologists checked the CRs and excluded three patients with obvious cardiac enlargement or interstitial pneumonia.

Development of a deep-learning method for PAH detection

The ResNet50 model pre-trained on ImageNet-1k was used for image classification. A fully connected layer was added to the final layer for binary classification [16]. A PyTorch framework was used for the ResNet50 model implementation [17]. The training was performed on a workstation using an NVIDIA Tesla T4 graphics processing unit ( Santa Clara, CA, USA). All CRs datasets were split in a 4:1 ratio for development and testing. The developed CRs dataset was separated into training and validation datasets using the K-fold cross-validation method (K = 4). The patient identification number was used to separate the CRs to ensure that multiple CRs from the same patient were not distributed across the datasets. Before training, the image contrasts were normalized to an intensity range of 0–255 and resized to 320 pixels × 320 pixels. We adopted common schemes for data augmentation, including flipping, rotating, and resizing. The training used binary cross-entropy loss function with stochastic gradient descent optimization (learning rate parameter, α = 0.01).

Testing the detecting capability of the algorithm

The algorithm outputted a numerical value from zero to one using the testing dataset (the probability of the PAH score) for each image. A receiver operating characteristic (ROC) curve, which plots the sensitivity and false positive rate (1 - specificity) for every score cut-off, was plotted, and the AUC was calculated.

Comparing the performance of the algorithm versus the performance of the doctors

The detecting capability of the algorithm was compared with that of doctors (respirologists [> 8 years of experience], n = 7 and radiologists [> 14 years of experience], n = 2). The distance from the doctor's eyes to the monitor was about 60 cm. The time to read each image was limited to ≤ 5 s.

Clinical characteristics

Participants were divided into two groups; those with PAH (PAH group: 145 patients; mean age, 51.9 ± 16.1 years; 34 males) and the control (control group: 260 patients; mean age, 67.3 ± 10.9 years; 131 males). The characteristics of the patients with PAH included in this study are shown in Table 1.

Table 1

Baseline characteristics of the study population with PAH.
Variable	Total population (n = 145)
Age	51.9 ± 16.1
Sex, n (%)
Male	34 (23.4)
Female	111 (76.6)
IPAH, n (%)	53 (36.6)
HPAH, n (%)	9 (6.2)
Other comorbid conditions, n (%)
Systemic sclerosis	26 (17.9)
Mixed connective tissue disease	9 (6.2)
Systemic lupus erythematosus	8 (5.5)
Rheumatoid arthritis	6 (4.1)
Dermatomyositis	1 (0.7)
Human immunodeficiency infection	1 (0.7)
Portal hypertension	11 (7.6)
Atrial septal defect	14 (9.7)
Ventricular septal defect	3 (2.1)
Partial anomalous pulmonary venous return	4 (2.8)
WHO functional class, n (%)
ClassⅠ	6 (4.1)
ClassⅡ	53 (36.6)
ClassⅢ	69 (47.6)
ClassⅣ	17 (11.7)
Right heart catheterization results
Mean PAP, mmHg	41.4 ± 13.4
Mean PAWP, mmHg	8.4 ± 3.7
PVR, Wood unit	7.9 ± 5.5
CI, L/min/m²	3.2 ± 1.3
*Data are presented as number of patients (percentage), mean ± SD.
Abbreviations: IPAH, idiopathic pulmonary arterial hypertension; HPAH, heritable pulmonary arterial hypertension; WHO, World Health Organization; PAP, pulmonary artery pressure; PAWP, pulmonary arterial wedge pressure; PVR, pulmonary vascular resistance; CI, cardiac index;

Detection capability of the algorithm

The distribution of CRs included in the testing dataset according to the algorithm-assigned scores for those confirmed to be PAH-positive or PAH-negative is shown in Fig. 1.

Figure 2 shows the ROC curve, which represents the detection capability of the algorithm. The AUC of the AI algorithm for the detection of PAH was 0.988. Using a score cutoff of 0.69, the sensitivity and specificity of the algorithm were 0.933 and 0.982, respectively.

Performance of the algorithm versus the performance of the doctors

Among the nine doctors who interpreted the CRs, the mean sensitivity and specificity were 0.829 and 0.906, respectively. The median (range) sensitivity and specificity were 0.822 (0.667–0.911) and 0.875 (0.821–0.982), respectively. All the doctors’ results were plotted in Fig. 2. All plotted dots were below the ROC curve. These findings demonstrate that the algorithm’s performance was not inferior to that of the doctors.

In this retrospective study, we developed and tested the CAD algorithms for analyzing CRs to diagnose pulmonary arterial hypertension. The sensitivity and specificity of the algorithm were 0.933 and 0.982, respectively, and those were not inferior to those of professional doctors.

CRs can be used as a simple and economical method that is available globally. Recently, deep-learning AI diagnostic systems have been developed for chest imaging. Early detection and diagnosis of PAH are important because a longer diagnostic interval is associated with decreased 5-year survival [7]. If CAD improves the detection rate of PAHs on CRs and makes this information available to non-specialist physicians, the time required to diagnose PAH can be reduced, which might lead to a potentially better prognosis.

In this study, we developed an algorithm to extract suspected patients with PAH using CRs in a population, such as a general medical examination, and achieved an AUC of 0.988. In a previous study, Kusunose et al. developed an algorithm to predict increased pulmonary arterial pressure using CRs in a population with suspected PH, with an AUC of 0.71 and a specificity of 0.95 [14]. The CAD task was more difficult than that in our study because the people in the control group (non-PH) had subjective symptoms and were suspected of having PH; therefore, the AUC of their algorithm was considered lower than that in our study. To help understand the AI assessment, we analyzed the AI-focused images. Our AI model focused on the pericardial areas of the CRs (Fig. 3.). In cases of false-positive results in the control group, heart enlargement could be considered PAH by the AI algorithm (Supplementary Fig. 1). In Kusunose’s study, AI focused on the right upper lung area and the area around the heart because the right upper pulmonary field is generally a common site of focal congestion [14].

Future studies should include diseases with CRs similar to PAH, such as cardiac hypertrophy, and collect data from diverse clinical settings. This study will be a precursor to developing algorithms for all PH groups by developing and validating not only one group of patients (those with PAH) but also other groups of patients, such as those with chronic thromboembolic PH. We believe this report will serve as a basis for future large-scale studies.

Our study had some limitations. First, the number of images used in the training dataset was relatively small because PAH is a rare disease. The larger the training dataset, the better the algorithm’s performance. In addition, this study successfully created an algorithm to detect PAH on CRs with high accuracy; however, further real-world clinical trials are needed to determine how this algorithm can contribute to PAH detection. Second, the X-ray machines used were different because the CRs used in this study included some old images. Undeniably, differences in image quality and other factors may have affected the study results. Third, due to ethical issues, RHC was not performed in the normal control group with no symptoms suggesting cardiovascular diseases. Therefore, we could not completely exclude PH in the control group. Fourth, learning and testing this algorithm were conducted at a single facility. It is unclear whether the same performance can be achieved using CRs under different conditions and with different equipment. Further clinical trials are required to validate this algorithm in a real-world setting.

In conclusion, we developed a deep-learning algorithm to detect PAH using CR images. The detection capability of the algorithm was not inferior to that of doctors. Analyzing CRs with our algorithm could provide general physicians and non-specialists for PAH with an opportunity to suspect PH, which might lead to early diagnosis.

PAH, pulmonary arterial hypertension; PH, pulmonary hypertension; ESC, European Society of Cardiology; ERS, European Respiratory Society; WHO-FC, World Health Organization Functional Class; CRs, chest radiographs; CAD, computer-aided detection/diagnosis; AI, artificial intelligence; AUC, area under the curve; NPV, negative predictive value; PAP, pulmonary arterial pressure; PAWP, pulmonary arterial wedge pressure; PVR, pulmonary vascular resistance; RHC, right heart catheter; ROC, receiver operating characteristics

Ethics approval and consent to participate: This study was performed in accordance with the Declaration of Helsinki. This study was approved by Research Ethics committee of the Graduate School of Medicine, Chiba University (approval number: 4203). All adult participants provided written informed consent to participate.

Consent for publication: Not applicable

Availability of data and materials: The data supporting this study's findings are available from the corresponding author, Shun Imai, upon reasonable request.

Competing Interest: TN, SH, and KO are employees of M3 Inc. And all other authors do not have competing interest.

Funding: This work was partly supported by a grant to The Intractable Respiratory Diseases and Pulmonary Hypertension Research Group, the Ministry of Health, Labor and Welfare, Japan. This study was supported by M3 Inc., Tokyo, Japan.

Authors’ contributions: SI, SS, JN, TS, TN, SH and KO were equally involved in this study conceptualization, study design, data analysis, and writing of the original draft. SI, SS, JN, AN, AS, TS, AS, AN, HY read chest radiographs as specialist doctors. NS, NT, TB, and TS critically revised the report and commented on drafts of this manuscript. All authors read and approved the final manuscript.

Acknowledgements: We would like to thank Editage (www.editage.com) for English language editing.

Badesch DB, Raskob GE, Elliott CG, Krichman AM, Farber HW, Frost AE, et al. Pulmonary arterial hypertension: Baseline characteristics from the REVEAL Registry. Chest. 2010;137:376–87.
Frost A, Badesch D, Gibbs JSR, Gopalan D, Khanna D, Manes A, et al. Diagnosis of pulmonary hypertension. Eur Respir J. 2019;53:1801904.
Humbert M, Kovacs G, Hoeper MM, Badagliacca R, Berger RMF, Brida M, et al. 2022 ESC/ERS Guidelines for the diagnosis and treatment of pulmonary hypertension. Eur Heart J. 2022;43:3618–731.
Galiè N, Channick RN, Frantz RP, Grünig E, Jing ZC, Moiseeva O, et al. Risk stratification and medical therapy of pulmonary arterial hypertension. Eur Respir J. 2019;53:1801889.
Nagata J, Sekine A, Tanabe N, Taniguchi Y, Ishida K, Shiko Y, et al. Mixed venous oxygen tension is a crucial prognostic factor in pulmonary hypertension: A retrospective cohort study. BMC Pulm Med. 2022;22:282.
Strange G, Gabbay E, Kermeen F, Williams T, Carrington M, Stewart S, et al. Time from symptoms to definitive diagnosis of idiopathic pulmonary arterial hypertension: The delay study. Pulm Circ. 2013;3:89–94.
Khou V, Anderson JJ, Strange G, Corrigan C, Collins N, Celermajer DS, et al. Diagnostic delay in pulmonary arterial hypertension: Insights from the Australian and New Zealand pulmonary hypertension registry. Respirology. 2020;25:863–71.
Weatherald J, Humbert M. The 'great wait' for diagnosis in pulmonary arterial hypertension. Respirology. 2020;25:790–2.
Qin C, Yao D, Shi Y, Song Z. Computer-aided detection in chest radiography based on artificial intelligence: A survey. Biomed Eng OnLine. 2018;17:113.
Parveen NR, Sathik MM. Detection of Pneumonia in chest X-ray images. J Xray Sci Technol. 2011;19:423–8.
Schilham AM, van Ginneken B, Loog M. A computer-aided diagnosis system for detection of lung nodules in chest radiographs with an evaluation on a public database. Med Image Anal. 2006;10:247–58.
Lakhani P, Sundaram B. Deep learning at chest radiography: Automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology. 2017;284:574–82.
Nishikiori H, Kuronuma K, Hirota K, Yama N, Suzuki T, Onodera M, et al. Deep learning algorithm to detect fibrosing interstitial lung disease on chest radiographs. Eur Respir J. 2023;61:2102269.
Kusunose K, Hirata Y, Tsuji T, Kotoku J, Sata M. Deep learning to predict elevated pulmonary artery pressure in patients with suspected pulmonary hypertension using standard chest X ray. Sci Rep. 2020;10:19311.
Vachiéry JL, Tedford RJ, Rosenkranz S, Palazzini M, Lang I, Guazzi M, et al. Pulmonary hypertension due to left heart disease. Eur Respir J. 2019;53:1801897.
He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016. p. 770–8.
Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, PyTorch: An imperative style, high-performance deep learning library. In: Adv Neural Inf Process Syst 32: Annual Conference on Neural Information Processing Systems, NeurIPS 2019. Vancouver, BC, Canada, 2019. Wallach HM, Larochelle H, Beygelzimer A et al. d'Alché-Buc F, Fox EA, Garnett R, editors; 2019.

Competing interest reported. TN, SH, and KO are employees of M3 Inc. And all other authors do not have competing interest.

Download PDF

Journal Publication

published 27 Feb, 2024

Read the published version in BMC Pulmonary Medicine →

Editorial decision: Revision requested
16 Nov, 2023
Reviews received at journal
13 Oct, 2023
Reviews received at journal
09 Oct, 2023
Reviewers agreed at journal
09 Oct, 2023
Reviewers agreed at journal
07 Oct, 2023
Reviews received at journal
15 Sep, 2023
Reviewers agreed at journal
07 Sep, 2023
Reviewers invited by journal
29 Aug, 2023
Editor assigned by journal
29 Aug, 2023
Editor invited by journal
22 Aug, 2023
Submission checks completed at journal
22 Aug, 2023
First submitted to journal
29 Jul, 2023

You are reading this latest preprint version

Artificial intelligence-based model for predicting pulmonary arterial hypertension on chest radiograph

Status:

Journal Publication

Version 1

Abstract

Background

Methods

Results

Conclusion

Figures

BACKGROUND

METHODS

RESULTS

DISCUSSION

Abbreviations

Declarations

References

Additional Declarations

Supplementary Files

Status:

Journal Publication

Version 1