3D Ultrasound Volume Quantification for Pediatric Urinary Tract Dilation: A Semi-Automated Segmentation Software Inter-Rater Analysis

doi:10.21203/rs.3.rs-4713233/v1

Download PDF

Research Article

3D Ultrasound Volume Quantification for Pediatric Urinary Tract Dilation: A Semi-Automated Segmentation Software Inter-Rater Analysis

https://doi.org/10.21203/rs.3.rs-4713233/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Objective

We determined the reliability of a three-dimensional (3D) US segmentation software for evaluating hydronephrosis index (HI) and renal parenchymal and pelvicalyceal volume in children with UTD.

Material and methods

From 1/2019 to 9/2023, children clinically scheduled for a renal imaging exam to assess UTD at a single center were prospectively enrolled. They underwent a dedicated 2D and 3D US renal exam. A UTD score was assigned per kidney from the 2D images based on the 2014 consensus classification by an experienced pediatric radiologist. From the 3D dataset, the renal parenchyma and collecting system were independently segmented by three trained raters using a semi-automated software (Philips Health Technology Innovation, Paris, France). From this segmentation, the kidney parenchymal and pelvicalyceal volume, dimensions, and HI values, were analyzed using intraclass correlation coefficient, grading inter-rater reliability.

Results

Forty-eight studies from 47 patients were included (65% male; median age: 24 months; IQR: 61 months). From these, 46 right and 40 left kidneys were chosen based on image quality. Twenty-nine (33.7%) kidneys had no dilation, 10 (11.6%) had UTD P1, 23 (26.7%) UTD P2, and 24 (27.9%) UTD P3. Inter-rater reliability was almost perfect across all parameters, with estimates ranging from 0.85 to 0.95. In kidneys with UTD P2 and UTD P3, HI had the lowest inter-rater agreement (0.75 and 0.66, respectively).

Conclusions

We demonstrated that semi-automated 3D US segmentation for kidneys with UTD can reliably assess renal dimensions, parenchymal and collecting system volumes, and HI among raters.

Urinary tract dilation

three-dimensional ultrasound

hydronephrosis index

renal parenchymal volume

pelvicalyceal volume

Urinary tract dilation (UTD) is the enlargement of the urinary collecting system that can be detected by ultrasound (US) antenatally or postnatally. It is found in 1 to 5% of all pregnancies, with almost 90% of cases resolving by the age of 3 years [1, 2]. However, in other instances, UTD can be a manifestation of congenital anomalies of the kidney and urinary tract, putting the child at risk of chronic kidney disease and end-stage renal disease [1]. UTD can be a sign of obstructive and non-obstructive conditions such as ureteropelvic junction obstruction and vesicoureteral reflux [3]. Furthermore, complications related to UTD may also arise, such as the development of urinary tract infection (UTI), kidney stones, or renal dysfunction [4]. US is especially useful for the prompt diagnosis and monitoring of UTD, having demonstrated safety and efficacy [5].

In 2014, the US UTD classification system was established by consensus as a standardized method for describing and reporting antenatal and postnatal dilation, aiming to improve outcome assessment and guide recommendations for clinical management and imaging exams [4]. There is an association between the extent of the dilation and an increased risk of obstructive uropathy as well as a link with various conditions, such as UTI, ureteropelvic junction obstruction, ureterocele, lower urinary tract obstruction, and chronic kidney disease [3, 4]. However, some of the criteria evaluated in the UTD classification are still subjective, such as the presence of parenchymal abnormalities and extent of pelvicalyceal dilation [3, 6]. The hydronephrosis index (HI) offers a numerical measure for UTD, indicating the collecting system volume relative to the capsular volume, with lower HI values corresponding to a greater severity of UTD. The capsular volume encompasses both the collecting system volume and the parenchymal volume [7].

3D US was studied for kidney size in hydronephrotic kidneys proving to be accurate [8]. It generates an image of the entire kidney rather than visualizing the kidney in serial 2D images [9]. It also eliminates the need for subjectively choosing a single 2D sagittal image for segmentation and enables improved continuous monitoring of the extent of UTD [10]. Moreover, fully-automated 3D US segmentation methods enabled accurate quantification of visible parenchyma and estimation of volume, further reducing inter-observer variability and eliminating time-consuming manual segmentation [11]. 3D US demonstrated high reliability in image quality for evaluating UTD. However, larger kidneys, such as those with UTD P3, were more susceptible to cut-off artifacts, motion artifacts and overall lower image quality [12]. Furthermore, parenchymal thinning is a key predictor of decreased differential function in mercaptoacetyltriglycine [MAG3] renogram and serves as the primary distinction between UTD P2 and P3, complicating their differentiation [13]. Hence, in this study, we determined the inter-rater reliability of a 3D US semi-automated segmentation software for the assessment of renal and pelvicalyceal volumes as well as the hydronephrosis index in children with all grades of UTD, and for UTD P2, and UTD P3 separately.

Study Population

This single-center prospective cohort study was approved by our institutional review board. Consent and assent, when applicable, were obtained. We enrolled children (< 18 years of age) with known UTD who were being imaged at our institution (with renal bladder US, MAG3 renogram or magnetic resonance urography [MRU]) between January 2019 and September 2023. Patients with UTD related to acute obstruction (e.g., kidney stones) or with renal anomalies (e.g., dysplastic or multicystic kidneys, horseshoe kidney, renal ectopia, partial or complete renal duplication, or posterior urethral valves) were excluded.

Imaging Technique

All recruited participants underwent dedicated 2D and 3D renal US following their scheduled imaging exam. Studies were conducted by three sonographers using a Philips EPIQ Elite 7G (Philips Ultrasound, Inc., Bothell, WA, 2016) system. For 2D US, the probe was selected according to the child’s size, and sonographers adhered to the institutional standard renal and bladder US protocol which included kidney length, width, and height measurements. 3D images were obtained with the X6-1 transducer positioned along the midline of the kidney in the long axis through a wide-angle sweep of 90 degrees. The focal zone depth was aligned with the renal hilum, employing a constant focal zone width of 3 cm. Up to 2 attempts were done to acquire 3D images in both the supine and prone positions.

Urinary tract dilation assessment

A UTD score according to the 2014 consensus classification was assigned to each kidney based on 2D images by an experienced pediatric radiologist [3, 4].

Kidney segmentation

The prototype (Philips Health Technology Innovation, Paris, France) is a semi-automatic 3D US segmentation software that determines measurements for kidney length, width, height, capsular volume (parenchyma and collecting system volume), and the hydronephrosis index (HI = {capsular volume – collecting system volume}/ {capsular volume}). In the same way, parenchymal volume is calculated as the capsular volume minus the collecting system volume. Initially, the 3D images were evaluated by an experienced pediatric radiologist. The image exclusion criteria were poor image quality, referring to images with poor visualization of the kidney due to motion artifacts and incomplete visualization of the kidney’s borders due to the parenchymal edges being cut off, such as in the case of severe kidney enlargement due to dilation, where the entire kidney could not be visualized in a single field of view. Subsequently, the prototype software was independently employed by one experienced rater and two trained raters to segment the renal parenchyma and collecting system.

The two novice raters underwent training by a software expert to optimize segmentation efficiency and feasibility, to be able to do so within 15 minutes per kidney. Raters practiced with five images each to become acquainted with the tool. Raters initialized the segmentation by placing the cursor on the kidney to provide the location and position to the software. The software then estimated the 3D kidney shape. Raters could then adjust the outline by clicking on the kidney borders resulting in a 3D automated depiction. Following this, the collecting system was marked using a semi-automatic region-growing feature (see Fig. 1).

The kidney measurements (length, width, and height), volumes (capsular volume, collecting system volume, and parenchymal volume), and HI were retrieved from the semi-automated software for each rater.

Statistical Analysis

The data were summarized utilizing descriptive statistics. Continuous parametric data were presented as mean and standard deviations, while non-parametric continuous data were summarized using medians and interquartile ranges. Categorical variables were represented as frequencies and percentages. The correlation between 2D and 3D kidney measurements was estimated using Pearson or Spearman correlation tests, depending on the data distribution. The inter-rater reliability of kidney measurements, volumes, and HI values were assessed through an intraclass correlation coefficient (ICC) analysis. This evaluation was conducted across all kidneys, with distinct analyses developed for the right and left kidneys separately. A second analysis was made to evaluate the inter-rater reliability in kidneys with the highest grade of dilation (UTD P3), and those without or with lesser grades of dilation (non-UTD P3). Lastly, a subset of kidneys with UTD P2 was also evaluated. Reliability was classified as follows: <0.1 (poor), 0.1–0.2 (slight), 0.2–0.4 (fair), 0.4–0.6 (moderate), 0.6–0.8 (substantial), 0.8–0.99 (almost perfect), and 1.0 (perfect). All analyses were performed using StataCorp. 2023 (Stata Statistical Software, Release 18. College Station, TX: StataCorp LLC).

Forty-seven patients were enrolled, and 48 studies were included. After excluding images due to insufficient quality and incomplete kidney visualization, 46 right and 40 left kidneys were segmented (Fig. 2). Table 1 shows the demographic characteristics of the study population and the number of kidneys with UTD.

Table 1

Demographic characteristics and urinary tract dilation grade
Number of participants	Total: 47
Male: n (%)	31 (64.58)
Age	1 month to 17 years
Median age in months (IQR)	24 (61)
Number of segmented kidneys	Total: 86
No dilation: n (%)	29 (33.72)
UTD P1: n (%)	10 (11.63)
UTD P2: n (%)	23 (26.74)
UTD P3: n (%)	24 (27.91)
UTD: urinary tract dilation

The Spearman rank correlation analysis revealed a strong significant positive correlation between 2D and 3D kidney measurements. The correlation coefficients were as follows: length, 0.92 (p = < .001); width, 0.66 (p = < .001); and height, 0.79 (p = < .001).

The inter-rater reliability of assessing kidney measurements, volumes and HI was almost perfect, ranging from 0.85 (95% confidence interval [CI] 0.77–0.90) for width to 0.95 (0.92–0.97) for length (Table 2).

Table 2

Inter-rater reliability demonstrated by ICC for kidney measurements, volumes, and HI
Variable	Mean (SD) n = 86	ICC (95%CI)
Width (cm)	4.4 (1.1)	0.85 (0.77–0.90)
Length (cm)	7.7 (1.7)	0.95 (0.92–0.97)
Height (cm)	3.1 (0.8)	0.92 (0.84–0.95)
Capsular volume (cm³)	66.5 (48.8)	0.93 (0.88–0.95)
Collecting system volume (cm³)	15.8 (24.4)	0.91 (0.87–0.94)
Parenchymal volume (cm³)	52.7 (35.4)	0.90 (0.82–0.94)
Hydronephrosis index	0.79 (0.18)	0.91 (0.85–0.94)
CI: confidence interval; ICC: intraclass correlation coefficient

The separate analyses for right and left kidneys also demonstrated almost perfect reliability across all measurements, volumes, and HI, with ICC estimates varying from 0.81 (left kidney width) to 0.97 (right kidney length), all with narrow 95% confidence intervals (Table 3).

Table 3

Inter-rater reliability demonstrated by ICC for kidney measurements, volumes, and HI by right and left kidneys
Variable	Mean (SD)	ICC (95%CI)
Right kidney: n = 46
Width (cm)	4.4 (1.2)	0.87 (0.79–0.93)
Length (cm)	7.6 (1.9)	0.97 (0.93–0.98)
Height (cm)	3 (0.9)	0.92 (0.83–0.96)
Capsular volume (cm³)	67.4 (57.3)	0.93 (0.87–0.96)
Collecting system volume (cm³)	16.7 (30.8)	0.91 (0.85–0.95)
Parenchymal volume (cm³)	52.7 (38.2)	0.88 (0.75–0.94)
Hydronephrosis index	0.81 (0.19)	0.91 (0.84–0.95)
Left Kidney: n = 40
Width (cm)	4.4 (1.1)	0.81 (0.67–0.89)
Length (cm)	7.8 (1.4)	0.93 (0.86–0.96)
Height (cm)	3.2 (0.7)	0.92 (0.83–0.96)
Capsular volume (cm³)	65.4 (36.8)	0.92 (0.86–0.96)
Collecting system volume (cm³)	14.8 (13.8)	0.92 (0.85–0.96)
Parenchymal volume (cm³)	52.7 (32.2)	0.93 (0.87–0.96)
Hydronephrosis index	0.77 (0.17)	0.91 (0.83–0.95)
CI: confidence interval; ICC: intraclass correlation coefficient

Analysis of kidneys with UTD P3 demonstrated lower inter-rater reliability for HI compared to non-UTD P3 kidneys, with ICC values of 0.66 and 0.83, respectively. In the subset of kidneys with UTD P2, the lowest ICC was also for HI (0.75). Table 4 provides the ICC values for all measurements.

Table 4

Inter-rater reliability demonstrated by ICC for kidney measurements, volumes, and HI by UTD P3, non-UTD P3 and UTD P2 kidneys
Variable	UTD P2 (n = 23)		UTD P3 (n = 24)		Non-UTD P3 (n = 62)
Variable	Mean (SD)	ICC (95%CI)	Mean (SD)	ICC (95%CI)	Mean (SD)	ICC (95%CI)
Width (cm)	4.5 (1.2)	0.78 (0.56–0.90)	4.7 (1.3)	0.90 (0.80–0.95)	4.3 (1.1)	0.82 (0.71–0.89)
Length (cm)	7.8 (1.5)	0.94 (0.87–0.97)	8.4 (2)	0.95 (0.82–0.98)	7.5 (1.5)	0.96 (0.93–0.97)
Height (cm)	3.0 (0.6)	0.88 (0.73–0.95)	3.4 (0.9)	0.90 (0.72–0.96)	3 (0.7)	0.93 (0.87–0.96)
Capsular volume (cm³)	62.8 (35.4)	0.90 (0.78–0.96)	85.2 (68.1)	0.92 (0.82–0.97)	59.2 (36.6)	0.93 (0.88–0.96)
Collecting system volume (cm³)	15.3 (14.9)	0.91 (0.82–0.96)	36.1 (35.5)	0.87 (0.72–0.94)	8.0 (1.1)	0.92 (0.88–0.95)
Parenchymal volume (cm³)	49.6 (25.4)	0.85 (0.68–0.94)	54.1 (43.1)	0.88 (0.71–0.95)	52.1 (32.1)	0.92 (0.85–0.95)
Hydronephrosis index	0.78 (0.14)	0.75 (0.51–0.88)	0.58 (0.14)	0.66 (0.30–0.85)	0.87 (0.12)	0.83 (0.73–0.89)
CI: confidence interval; ICC: intraclass correlation coefficient: UTD: urinary tract dilation; P: postnatal

In this study, we found almost perfect inter-rater reliability of kidney measurements, kidney volumes, and HI among three raters using a 3D semi-automated segmentation software in an 86-kidney sample. Higher ICC estimates were found for length and lower for width, which can be attributed to the ease of identifying the poles in contrast to the hilar border during the kidney segmentation process, resulting in a higher variability in width measurements.

High inter-rater reliability was also observed in parenchymal and collecting system volumes. This is significant because both variables are part of the criteria to assess UTD, and the collecting system volume could signal obstructive uropathy, which has potential implications for kidney function [14].

When analyzing inter-rater reliability among kidneys with UTD P3, only HI showed a lower ICC estimate compared to other variables. HI provides a value for the dilated collecting system volume relative to the capsular volume, which can be harder to discern while segmenting in the context of an overall enlarged kidney with a distorted shape due to dilation [7]. Nevertheless, the inter-rater reliability of HI in UTD P3 kidneys was substantial, and for non-UTD P3 kidneys, it was almost perfect, showing higher estimates when the parenchyma was not affected and/or the dilation was less severe. Similarly, inter-rater reliability for HI was lower for kidneys with UTD P2 compared to non-UTD P3, yet higher than for UTD P3. This observation supports the notion that reliability diminishes with increasing dilation grade.

3D US allows for volumetric evaluation and improved visualization of the collecting system. Coupled with the semi-automatic segmentation approach, it facilitates an objective, standardized, and quantitative method for categorizing patients with UTD [5]. We demonstrated that there was a strong positive correlation between 2D and 3D kidney dimensions, supporting the reliability of 3D US for assessing kidney size in the context of UTD. These attributes underscore the value of semi-automatic 3D segmentation values as clinical parameters in the follow-up of patients with UTD. Given that US is a noninvasive, easily accessible, and radiation-free primary imaging modality [5], the high inter-rater agreement among users is a favorable finding in the clinical implementation of this tool.

Various 3D kidney segmentation models have been proposed to analyze renal parenchyma with UTD [10, 11, 15]. The evolution of the models has progressed towards semi-automatic segmentation as a solution to the drawbacks associated with manual segmentation, which is time-consuming, labor intensive and susceptible to inter-operator variability. Additionally, fully automatic methods face challenges; for example, the diverse shapes of kidneys and variations in image intensity distributions could affect the automatic segmentation [16]. Moreover, to make segmentation more accurate, semi-automatic tools address the variability in the direction of US wave propagation by assigning weights to the calculated values based on the reference point of orientation relative to the position of the transducer [17]. One of these models has been tested to evaluate the performance of the segmentation algorithm, yielding promising outcomes of segmentation accuracy and HI estimation [10]. Nevertheless, the software’s reliability across multiple raters was not tested, making our study one of the initial efforts to examine the consistency of a semi-automatic segmentation software among raters [18].

This study has limitations, as raters were aware that their values were being assessed for reliability, potentially influencing their behavior, and increasing agreement. However, the assessments' quantitative nature provided a higher level of objectivity.

In conclusion, our study demonstrates that semi-automated 3D US segmentation for pediatric kidneys can be used to effectively assess renal dimensions, parenchymal volumes, and HI among raters of kidneys with varying degrees of UTD. Identifying these reliable parameters could help determine when more invasive exams, such as MAG3 renography or MRU, are necessary to assess renal function. Additionally, this study establishes the potential for future comparisons between 3D US parenchymal volume measurements and those obtained from established imaging modalities (MAG3 renography and MRU).

Competing Interests

This study was funded by Philips Healthcare and the Children’s Hospital of Philadelphia, Department of Radiology. Philips owns the proprietary software and sponsored this study.

Author Contribution

S.B. conceptualized and designed the study, coordinated data collection, and interpreted the sonographic examinations. T.M.T. drafted the initial manuscript. S.B. and T.A.M. coordinated the enrollment process, obtained informed consent, and enrolled the participants. T.A.M. performed the sonographic examinations. T.M.T., L.S., and L.R performed the segmentation analysis. T.M.T., L.S., L.R., J.J., M.M.E., and Anush S. collected the data. T.M.T. and W.L. carried out the analyses. S.B., K.D., Arun S., and H.O. conceptualized the study and critically reviewed and revised the manuscript. All authors reviewed and approved the final manuscript.

Data Availability

All collected data were anonymized and stored in a password-protected research-focused electronic web-based data capture system, REDCap (Vanderbilt University). The datasets generated and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Vincent K, Murphy HJ, Twombley KE. Urinary tract dilation in the fetus and neonate. Neoreviews. 2022;23:e159–74.
Green CA, Adams JC, Goodnight WH, Odibo AO, Bromley B, Jelovsek JE, et al. Frequency and prediction of persistent urinary tract dilation in third trimester and postnatal urinary tract dilation in infants following diagnosis in second trimester. Ultrasound Obstet Gynecol. 2022;59:522–31.
Nguyen HT, Benson CB, Bromley B, Campbell JB. Multidisciplinary consensus on theclassification of prenatal and postnatalurinary tract dilation (UTD classificationsystem). J Pediatr Urol. 2014;10:998–9.
Nguyen HT, Phelps A, Coley B, Darge K, Rhee A, Chow JS. 2021 update on the urinary tract dilation (UTD) classification system: clarifications, review of the literature, and practical suggestions. Pediatr Radiol. 2022;52:740–51.
Viteri B, Calle-Toro JS, Furth S, Darge K, Hartung EA, Otero H. State-of-the-Art Renal Imaging in Children. Pediatrics. 2020;145.
Nulens K, Lorenzo AJ, Dos Santos J, Ellul K, Rickard M. Fetal urinary tract dilation: What to tell the parents. Prenat Diagn. 2023;
Shapiro SR, Wahl EF, Silberstein MJ, Steinhardt G. Hydronephrosis index: a new method to track patients with hydronephrosis quantitatively. Urology. 2008;72:536–8; discussion 538.
Riccabona M, Fritz GA, Schöllnast H, Schwarz T, Deutschmann MJ, Mache CJ. Hydronephrotic kidney: pediatric three-dimensional US for relative renal size assessment--initial experience. Radiology. 2005;236:276–83.
Jagtap JM, Gregory AV, Homes HL, Wright DE, Edwards ME, Akkus Z, et al. Automated measurement of total kidney volume from 3D ultrasound images of patients affected by polycystic kidney disease and comparison to MR measurements. Abdom Radiol (NY). 2022;47:2408–19.
Cerrolaza JJ, Safdar N, Biggs E, Jago J, Peters CA, Linguraru MG. Renal Segmentation From 3D Ultrasound via Fuzzy Appearance Models and Patient-Specific Alpha Shapes. IEEE Trans Med Imaging. 2016;35:2393–402.
Yin S, Zhang Z, Li H, Peng Q, You X, Furth SL, et al. Fully-automatic segmentation of kidneys in clinical ultrasound images using a boundary distance regression network. Proc IEEE Int Symp Biomed Imaging. 2019;2019:1741–4.
Otero HJ, Cerrolaza JJ, Loomis J, George A, Biggs E, Jago J, et al. Feasibility and quality determinants of 3D sonography in children with hydronephrosis. J Diagn Med Sonogr. 2018;34:31–6.
Agard H, Massanyi E, Albertson M, Anderson M, Alam M, Lyden E, et al. The different elements of the Urinary Tract Dilation (UTD) Classification System and their capacity to predict findings on mercaptoacetyltriglycine (MAG3) diuretic renography. J Pediatr Urol. 2020;16:686.e1-686.e6.
Kaspar CDW, Lo M, Bunchman TE, Xiao N. The antenatal urinary tract dilation classification system accurately predicts severity of kidney and urinary tract abnormalities. J Pediatr Urol. 2017;13:485.e1-485.e7.
Tabrizi PR, Mansoor A, Cerrolaza JJ, Zember J, Pohl HG, Jago J, et al. Automatic segmentation of the renal collecting system in 3D pediatric ultrasound to assess the severity of hydronephrosis. 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). IEEE; 2019. p. 1717–20.
Yin S, Peng Q, Li H, Zhang Z, You X, Fischer K, et al. Automatic kidney segmentation in ultrasound images using subsequent boundary distance regression and pixelwise classification networks. Med Image Anal. 2020;60:101602.
Cerrolaza JJ, Grisan E, Safdar N. Quantification of Kidneys from 3D Ultrasound in Pediatric Hydronephrosis . Annu Int Conf IEEE Eng Med Biol Soc. 2015;157–60.
Anvari A, Halpern EF, Samir AE. Essentials of statistical methods for assessing reliability and agreement in quantitative imaging. Acad Radiol. 2018;25:391–6.

Competing interest reported. This study was funded by Philips Healthcare and the Children’s Hospital of Philadelphia, Department of Radiology. Philips owns the proprietary software and sponsored this study.

Download PDF

Reviews received at journal
07 Sep, 2024
Reviewers agreed at journal
16 Aug, 2024
Reviews received at journal
11 Aug, 2024
Reviews received at journal
07 Aug, 2024
Reviewers agreed at journal
07 Aug, 2024
Reviewers agreed at journal
20 Jul, 2024
Reviewers agreed at journal
15 Jul, 2024
Reviewers invited by journal
15 Jul, 2024
Editor assigned by journal
11 Jul, 2024
Submission checks completed at journal
11 Jul, 2024
First submitted to journal
09 Jul, 2024

You are reading this latest preprint version

3D Ultrasound Volume Quantification for Pediatric Urinary Tract Dilation: A Semi-Automated Segmentation Software Inter-Rater Analysis

Status:

Version 1

Abstract

Objective

Material and methods

Results

Conclusions

Figures

INTRODUCTION

METHODS

Study Population

Imaging Technique

Urinary tract dilation assessment

Kidney segmentation

Statistical Analysis

RESULTS

DISCUSSION

Declarations

Author Contribution

Data Availability

References

Additional Declarations

Status:

Version 1