Artificial Intelligence-based Point-of-care Lung Ultrasound for Screening COVID-19 Pneumoniae: Comparison with CT Scans

doi:10.21203/rs.3.rs-1628466/v1

Download PDF

Research Article

Artificial Intelligence-based Point-of-care Lung Ultrasound for Screening COVID-19 Pneumoniae: Comparison with CT Scans

https://doi.org/10.21203/rs.3.rs-1628466/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background: Although lung ultrasound has been reported to be a portable, cost-effective, and accurate method to detect pneumonia, it has not been widely used because of the difficulty in its interpretation. Here, we aimed to investigate the effectiveness of a novel artificial intelligence-based automated pneumonia detection method using point-of-care lung ultrasound (AI-POCUS) for the coronavirus disease 2019 (COVID-19).

Methods: We enrolled consecutive patients admitted with COVID-19 who underwent computed tomography (CT) in August and September 2021. A 12-zone AI-POCUS was performed by a novice observer using a pocket-size device within 24 h of the CT scan. Fifteen control subjects were also scanned. Additionally, the accuracy of the simplified 8-zone scan excluding the dorsal chest, was assessed. More than three B-lines detected in one lung zone were considered zone-level positive, and the presence of positive AI-POCUS in any lung zone was considered patient-level positive.

Results: A total of 577 lung zones from 56 subjects (59.4 ± 14.8 years, 23% female) were evaluated using AI-POCUS. The mean number of days from disease onset was 9, and 14% of patients were under mechanical ventilation. The patient-level accuracy of 12-zone AI-POCUS for detecting CT-validated pneumonia was 94.5%, sensitivity was 92.3%, and specificity was 100%. When simplified with 8-zone scan, the accuracy, sensitivity, and sensitivity were 83.9%, 77.5%, and 100%, respectively. The zone-level accuracy, sensitivity, and specificity of AI-POCUS were 65.3 %, 37.2%, and 97.8 %, respectively.

Interpretation: AI-POCUS using the novel pocket-size ultrasound system showed excellent agreement with CT-validated COVID-19 pneumonia, even when used by a novice observer.

lung ultrasound

point-of-care ultrasound

COVID-19

machine learning

artificial intelligence

The novel coronavirus disease 2019 (COVID-19) emerged in China in 2019, and has since spread rapidly worldwide [1]. This highly infectious disease, which often causes fatal pneumonia, has caused 6 million deaths as of March 2022 [2–4]. This rapid increase in the number of patients with COVID-19 has strained medical resources in many countries [5, 6]. In such situations, detailed screening in medical facilities for all patients is not realistic; as such, there is an increasing demand for portable, easy, and cost-effective screening tools.

Point-of-care lung ultrasound (POCUS) is a simple diagnostic test with high diagnostic accuracy for lung disease [7, 8]. A previous systematic review reported that the presence of multiple B-lines could identify the presence of pneumonia with a sensitivity of 90.4%, and a specificity of 88.4% [9]. Although this examination has the potential to serve as a useful screening tool in COVID-19 patients, lung POCUS is not sufficiently widespread, mainly because of technical difficulties in image interpretation.

Artificial intelligence, especially machine learning including deep learning, is an emerging computer technology that automatically recognizes images with excellent accuracy [10–12]. This technology may have the potential to help interpret medical images and enable such examinations, even by novice observers. In this study, we investigated the effectiveness of a novel artificial intelligence-based automated B-line counting system for detecting COVID-19 pneumonia using a pocket-size ultrasound device [13].

Patient enrollment

We enrolled consecutive patients with COVID-19 who were admitted to our hospital and underwent computed tomography (CT) between August and September 2021. Lung POCUS using a pocket-size ultrasound device (Lumify; Philips Ultrasound, Inc., Bothell, WA) with a novel machine learning-based application (guided B-line quantification; Philips Ultrasound, Inc., Bothell, WA) was performed in all patients within 24 h of the CT scan (artificial intelligence-based point-of-care lung ultrasound; AI-POCUS). Patients aged < 20 years and pregnant women were excluded. We further performed AI-POCUS in 15 consecutive subjects without lung diseases who underwent chest CT for the purpose of medical checkup, as controls.

AI-POCUS and CT protocols

AI-POCUS was performed in a 12-zone manner: eight zones for the ventral and four zones for the dorsal chest wall using a 1.0–4.0 Hz sector probe (S4-1; Philips Ultrasound, Inc., Bothell, WA), with a depth setting of 6–12 cm [8, 14, 15]. As shown in Fig. 1, the ventral chest wall was segmented by the midline and anterior axillary lines, and further divided into the upper and lower zones. The dorsal chest wall was divided into the left-right and upper-lower zones. For those who were too sick to change their positions (such as intubated patients), only ventral eight zones were scanned [15]. The AI-POCUS movies were automatically analyzed using a real-time automated B-line counting application. All AI-POCUS scans were performed by a blinded novice observer who had experienced fewer than 10 cases of lung POCUS before the start of the study. This machine-learning-based application automatically counts the number of B-lines in the movie, and records the maximum number of B-lines. Three or more B-lines in each zone were considered abnormal (Fig. 2).

Non-contrast chest CT scans were acquired using commercially available equipment (Aquilion ONE, Canon, Japan) at inspiration, and reconstructed as axial images with a 5-mm slice thickness, 5-mm interval, and 120 peak kilovoltage. A board-certified pulmonologist, who was blinded to patient characteristics and AI-POCUS findings, read the CT and diagnosed the presence of pneumonia in each of the 12 zones of the lung that corresponded with the lung POCUS segmentation.

Zone-level concordance was assessed between AI-POCUS and CT results for each zone. In contrast, patient-level AI-POCUS results were defined as negative when the patient had no sign of pneumonia in any of the acquired zones, and patient-level concordance was assessed for each patient.

Statistical analyses

Data are presented as the mean ± standard deviation or median [1st and 3rd quartile] for continuous variables, and as the frequency (%) for categorical variables. Differences in patient characteristics between groups were tested using the Welch t-test and Mann-Whitney test for normally and non-normally distributed variables, respectively, and Fisher’s exact test for categorical variables. The primary endpoint of the study was patient-level accuracy, sensitivity, and specificity of AI-POCUS in comparison with CT-validated pneumonia. All statistical analyses were performed with R version 4.0.2 (The R Foundation for Statistical Computing, Vienna, Austria). A two-tailed p value of < 0.05 indicated statistical significance.

Study population

A total of 56 subjects (41 patients and 15 controls) were enrolled in the study. The baseline patient characteristics are summarized in Table 1. Overall, the mean age was 59.4 ± 14.8 years, 23% were female, and the mean body mass index was 25.3 ± 4.7 kg/m². The patients were significantly younger than the controls. The median time from the onset to the CT scan was 9 [5–12] days, and 20% of the patients were maintained on mechanical ventilation at the time of CT and AI-POCUS.

Table 1

Patient characteristic
	Overall (N = 56)	Case (N = 41)	Control (N = 15)	p value
Age, years old	59.4 ± 14.8	56.3 ± 14.6	67.6 ± 12.5	0.008
Female	13 (23%)	8 (20%)	5 (33%)	0.30
Body mass index, kg/m²	25.3 ± 4.7	25.6 ± 5.1	24.7 ± 3.5	0.50
Days from onset		9.0 [5.0–12.0]
Mechanical Ventilation		8 (20%)

AI-POCUS and CT availability

CT evaluation was successful for all 12 zones for all subjects (492 zones for cases and 180 for controls), whereas lung POCUS was not available for the dorsal chest zones in 15 patients who were unable to change their positions. One patient was placed in the prone position, and only the dorsal zones were scanned. Clear images were not available in the left lower inside zone in the ventral chest of 26 patients because the heart was close to the chest wall, and the lung was therefore not visible from this position. Finally, lung POCUS movies of 397 zones for patients and 180 zones for controls were analyzed.

Concordance of pneumonia identification

A CT scan identified pneumonia in 395 zones (58.7%) of 40 subjects, whereas no pneumonia was detected in the other 16 subjects. Pneumonia was present almost evenly in each zone, with a slightly higher prevalence in the dorsal zones (Fig. 3).

Panel A in Fig. 4 demonstrates the patient-level concordance between the AI-POCUS and CT-validated pneumonia. One subject was excluded from this analysis because AI-POCUS was not possible for the dorsal zones, although he only had pneumonia there. AI-POCUS was able to diagnose the presence or absence of pneumonia in 52 of 55 subjects, while pneumonia was overlooked in three patients. The overall accuracy, sensitivity, and specificity of the AI-POCUS were 94.5%, 92.3%, and 100%, respectively.

The screening performance of AI-POCUS using only the ventral eight zones was evaluated as a simpler approach that can be used even for those who cannot change their position. With only 8-zone AI-POCUS, pneumonia was overlooked in 9 patients, whereas 31 of 40 patients with pneumonia were still classified as positive (Fig. 4, Panel B). The accuracy and sensitivity decreased to 83.9% and 77.5%, respectively; however, the specificity remained at 100%.

In the zone-level analysis, AI-POCUS was correct in 377 of 577 zones (Fig. 4, Panel C). The zone-level accuracy, sensitivity, and specificity of the AI-POCUS were 65.3%, 37.3%, and 97.8%, respectively.

The main findings of our study were as follows: (1) AI-POCUS had excellent performance at detecting patients with COVID-19 pneumonia (sensitivity 92.3% and specificity 100%), and (2) the zone-level sensitivity was moderate (37.3%), although specificity was very high (97.8%). Even with moderate sensitivity in each zone, pneumonia in COVID-19 usually spreads to multiple lung zones, leading to a high sensitivity for each patient. Notably, these excellent results were achieved with AI-POCUS by a novice observer who had minimal experience in lung-POCUS.

With the recent miniaturization of ultrasound devices and advancements in image quality, lung-POCUS is becoming a popular examination, especially in intensive care [8, 16]. Although the lung itself, which is filled with air, cannot usually be observed by ultrasound beams, the noise and artifacts generated by ultrasound beams provide useful clinical information in lung-POCUS. Ultrasound B-lines are one of the most useful artifacts that present with pulmonary congestion, either by pneumonia or cardiogenic pulmonary edema. These reverberation artifacts originate at the pleura, reflecting an air-fluid mixture, which occurs when the subpleural interlobular septa surrounded by subpleural air-filled alveoli become edematous. Previous studies have reported that three or more B-lines visible in a single ultrasound plane are fully sensitive and specific to demonstrate subpleural thickened interlobular septa and/or ground-glass areas with a CT scan as a reference [17].

Diagnosis of pneumonia using lung-POCUS techniques, including the B-line, is expected to be an effective tool in the ongoing COVID-19 pandemic. Reports have already shown that B-lines are not only sensitive, but also associated with disease severity, future deterioration, and treatment effects in COVID-19 [15, 18–22]. However, chest radiography and CT remain the leading examinations used in the management of COVID-19 worldwide, and lung-POCUS has not been sufficiently used. Technical difficulties and problems in interpreting images are major concerns when using lung-POCUS; the number of experts is not large enough to teach and supervise the use of lung-POCUS by clinicians, including general practitioners, although this technique is relatively new and may be difficult for novice observers.

In the present study, we demonstrated that with AI technology, lung-POCUS can be effective to diagnose pneumonia, with excellent accuracy, even by a novice observer. Recent advancements in AI technology, more specifically machine learning, have enabled automated image recognition with similar or even higher accuracy compared to expert clinicians, and the application of such technologies to POCUS has been enthusiastically studied [10, 11, 23, 24]. The automated B-line counting application that we used in this study was also developed using machine learning technology, and showed excellent agreement with expert readings [25, 26].

The moderate zone-level accuracy observed herein may be due to the limited ability to detect slight pneumonia in the central area of the lung far from the body surface. Such slight pneumonia in a deep area of the lung may not affect the pleura, and would therefore not generate a B line. However, pneumonia caused by COVID-19 usually spreads to multiple regions of the lungs, appearing in the form of acute respiratory distress syndrome [22]. Therefore, diffuse pneumonia caused by COVID-19 is a suitable target for AI-POCUS.

COVID-19 is expected to persist for some time. AI-POCUS allows non-specialist general practitioners to diagnose pneumonia, a problem in viral infections, in an access-free manner, which may be useful in the treatment of pneumonia during a pandemic, and is expected to become more widespread.

Limitations

This study has some limitations. First, this was a single-center study conducted in a university hospital, and included a small number of cases. Larger studies involving various settings are necessary before the current results can be widely implemented. Second, we did not check the accuracy of AI-POCUS compared with traditional lung POCUS by an expert. However, a previous study showed that AI-POCUS has over 90% sensitivity and specificity for detecting expert-annotated B-lines [25]. Next, we studied only hospitalized patients who were considered sicker than those who were cared for at home or at other facilities. The performance of the present application should be tested in less intensively sick patients to uncover it’s applicability for screening purposes.

AI-POCUS showed excellent performance in diagnosing patients with CT-validated COVID-19 pneumonia, even by a novice observer.

COVID-19, coronavirus disease 2019

POCUS, point-of-care ultrasound

AI-POCUS, artificial intelligence-based point-of-care lung ultrasound

CT, computed tomography

Ethics approval and consent to participate

The study protocol complied with the Declaration of Helsinki and was approved by the institutional review board at Juntendo University Hospital (#E21-0197). Written informed consent was waived due to its purely observational nature.

Data Availability

Data are available from the corresponding author upon an appropriate request.

Authors' contributions

Manuscript was written by Yumi Kuroda and Nobuyuki Kagiyama. Figures were made by Nobuyuki Kagiyama. Other authors were responsible for critical review and data collection.

Conflicts of Interest

Dr. Kagiyama and Dr. Daida are affiliated with a department funded by Philips Japan, Asahi KASEI Corporation, Inter Reha Co., Ltd, and Toho Holdings Co., Ltd., based on collaborative research agreements.

Funding

This work was partially supported by a JSPS KAKENHI Grant-in-Aid for Research Activity Start-up, Grant Number 21K18086.

WHO Coronavirus Disease (COVID-19) Dashboard. [https://covid19.who.int/]
Carvalho T, Krammer F, Iwasaki A: The first 12 months of COVID-19: a timeline of immunological insights. Nat Rev Immunol 2021, 21(4):245–256.
Hu B, Guo H, Zhou P, Shi ZL: Characteristics of SARS-CoV-2 and COVID-19. Nat Rev Microbiol 2021, 19(3):141–154.
Wiersinga WJ, Rhodes A, Cheng AC, Peacock SJ, Prescott HC: Pathophysiology, Transmission, Diagnosis, and Treatment of Coronavirus Disease 2019 (COVID-19): A Review. Jama 2020, 324(8):782–793.
Ji Y, Ma Z, Peppelenbosch MP, Pan Q: Potential association between COVID-19 mortality and health-care resource availability. Lancet Glob Health 2020, 8(4):e480.
Kavanagh MM, Erondu NA, Tomori O, Dzau VJ, Okiro EA, Maleche A, Aniebo IC, Rugege U, Holmes CB, Gostin LO: Access to lifesaving medical resources for African countries: COVID-19 testing and response, ethics, and politics. Lancet 2020, 395(10238):1735–1738.
Mojoli F, Bouhemad B, Mongodi S, Lichtenstein D: Lung Ultrasound for Critically Ill Patients. Am J Respir Crit Care Med 2019, 199(6):701–714.
Volpicelli G, Elbarbary M, Blaivas M, Lichtenstein DA, Mathis G, Kirkpatrick AW, Melniker L, Gargani L, Noble VE, Via G et al: International evidence-based recommendations for point-of-care lung ultrasound. Intensive Care Med 2012, 38(4):577–591.
Xia Y, Ying Y, Wang S, Li W, Shen H: Effectiveness of lung ultrasonography for diagnosis of pneumonia in adults: a systematic review and meta-analysis. Journal of Thoracic Disease 2016, 8(10):2822–2831.
Liu Y, Jain A, Eng C, Way DH, Lee K, Bui P, Kanada K, de Oliveira Marinho G, Gallegos J, Gabriele S et al: A deep learning system for differential diagnosis of skin diseases. Nat Med 2020, 26(6):900–908.
Kagiyama N, Shrestha S, Farjo PD, Sengupta PP: Artificial Intelligence: Practical Primer for Clinical Research in Cardiovascular Disease. J Am Heart Assoc 2019, 8(17):e012788.
Roy S, Menapace W, Oei S, Luijten B, Fini E, Saltori C, Huijben I, Chennakeshava N, Mento F, Sentelli A et al: Deep Learning for Classification and Localization of COVID-19 Markers in Point-of-Care Lung Ultrasound. IEEE Trans Med Imaging 2020, 39(8):2676–2687.
Baloescu C, Chen A, Raju B, Evans N, Moore CL: 19 Automated Quantification Of B-Lines in Lung Ultrasound On COVID-19 Patients. Ann Emerg Med 2021, 78(2):S9-S10.
Levy Adatto N, Preisler Y, Shetrit A, Shepshelovich D, Hershkoviz R, Isakov O: Rapid 8-Zone Lung Ultrasound Protocol is Comparable to a Full 12-Zone Protocol for Outcome Prediction in Hospitalized COVID-19 Patients. J Ultrasound Med 2021.
Haurylenka D, Damantsevich V, Filustsin A, Damantsevich A: Diagnostic value of 12-zone lung ultrasound protocol for diagnosing COVID-19-associated pneumonia in outpatients. J Ultrason 2021, 21(87):e271-e276.
Bhagra A, Tierney DM, Sekiguchi H, Soni NJ: Point-of-Care Ultrasonography for Primary Care Physicians and General Internists. Mayo Clin Proc 2016, 91(12):1811–1827.
Lichtenstein D, Mézière G, Biderman P, Gepner A, Barré O: The comet-tail artifact. An ultrasound sign of alveolar-interstitial syndrome. Am J Respir Crit Care Med 1997, 156(5):1640–1646.
Zieleskiewicz L, Markarian T, Lopez A, Taguet C, Mohammedi N, Boucekine M, Baumstarck K, Besch G, Mathon G, Duclos G et al: Comparative study of lung ultrasound and chest computed tomography scan in the assessment of severity of confirmed COVID-19 pneumonia. Intensive Care Med 2020, 46(9):1707–1713.
Rubio-Gracia J, Giménez-López I, Garcés-Horna V, López-Delgado D, Sierra-Monzón JL, Martínez-Lostao L, Josa-Laorden C, Ruiz-Laiglesia F, Pérez-Calvo JI, Crespo-Aznarez S et al: Point-of-care lung ultrasound assessment for risk stratification and therapy guiding in COVID-19 patients: a prospective noninterventional study. Eur Respir J 2021, 58(3).
Volpicelli G, Gargani L, Perlini S, Spinelli S, Barbieri G, Lanotte A, Casasola GG, Nogué-Bou R, Lamorte A, Agricola E et al: Lung ultrasound for the early diagnosis of COVID-19 pneumonia: an international multicenter study. Intensive Care Med 2021, 47(4):444–454.
Wang M, Luo X, Wang L, Estill J, Lv M, Zhu Y, Wang Q, Xiao X, Song Y, Lee MS et al: A Comparison of Lung Ultrasound and Computed Tomography in the Diagnosis of Patients with COVID-19: A Systematic Review and Meta-Analysis. Diagnostics (Basel) 2021, 11(8).
Zheng Y, Wang L, Ben S: Meta-analysis of chest CT features of patients with COVID-19 pneumonia. J Med Virol 2021, 93(1):241–249.
Sengupta PP, Shrestha S, Berthon B, Messas E, Donal E, Tison GH, Min JK, D'Hooge J, Voigt JU, Dudley J et al: Proposed Requirements for Cardiovascular Imaging-Related Machine Learning Evaluation (PRIME): A Checklist: Reviewed by the American College of Cardiology Healthcare Innovation Council. JACC Cardiovasc Imaging 2020, 13(9):2017–2035.
Seetharam K, Kagiyama N, Sengupta PP: Application of mobile health, telemedicine and artificial intelligence to echocardiography. Echo Res Pract 2019, 6(2):R41-r52.
Baloescu C, Toporek G, Kim S, McNamara K, Liu R, Shaw MM, McNamara RL, Raju BI, Moore CL: Automated Lung Ultrasound B-Line Assessment Using a Deep Learning Algorithm. IEEE Trans Ultrason Ferroelectr Freq Control 2020, 67(11):2312–2320.
Moore CL, Wang J, Battisti AJ, Chen A, Fincke J, Wang A, Wagner M, Raju B, Baloescu C: Interobserver Agreement and Correlation of an Automated Algorithm for B-Line Identification and Quantification With Expert Sonologist Review in a Handheld Ultrasound Device. J Ultrasound Med 2021.

No competing interests reported.

Download PDF

Version 1

posted

You are reading this latest preprint version

Artificial Intelligence-based Point-of-care Lung Ultrasound for Screening COVID-19 Pneumoniae: Comparison with CT Scans

Status:

Version 1

Abstract

Figures

Introduction

Methods

Patient enrollment

AI-POCUS and CT protocols

Statistical analyses

Results

Study population

AI-POCUS and CT availability

Concordance of pneumonia identification

Discussion

Limitations

Conclusions

Abbreviations

Declarations

Ethics approval and consent to participate

Data Availability

Authors' contributions

Conflicts of Interest

Funding

References

Additional Declarations

Status:

Version 1