2.1 Study Patients and Images
In this retrospective, non-interventional, case–controlled study, we collected ultrasound images of patients who had been diagnosed with uterine fibroids according to surgical and pathological findings and those with normal uteri. Using these ultrasound images, we developed a DCNN model to automatically detect uterine fibroids in the obtained images and to assist junior ultrasonographers in diagnosing uterine fibroids. To protect patients’ privacy, all identifying information such as name, sex, age, and ID on the ultrasound images were anonymized and omitted when the data were first acquired. This retrospective study was approved by the hospital ethics committee (Shunde Hospital of Southern Medical University). All experiments were performed in accordance with relevant guidelines and regulations. All data in the study were obtained without any conflicts of interest.
A total of 3870 ultrasound images (2020 abnormal with uterine fibroids and 1850 normal) from 667 patients (mean age: 42.45 years ± 6.23 [SD]) pathological diagnosed with uterine fibroids and 570 women (mean age: 39.24 years ± 5.32 [SD]) without uterine lesions were included in the analysis. The DCNN was trained and developed on 3382 ultrasound images, which were randomly divided into a training dataset (80%, including 2706 images) and an internal validation dataset (20%, including 676 images). The model performance was tested on an external validation dataset containing 488 ultrasound images (268 uterine fibroids and 220 normal uteruses). The medical record system of the hospital provided us with the patients' clinical data (such as their ages, surgery statuses, pathological findings, surgical modalities, and postoperative ultrasound review findings). The information and imaging data of the patients with uterine fibroids are summarized in Table 1 and Table 2, respectively.
Table 1
Clinical and pathological data summaries of the patients
Variables
|
Value (n = 667)
|
Age (years), mean ± SD
|
42.45 ± 6.23
|
Diameter of uterine fibroids (cm), mean ± SD
|
6.14 ± 3.74
|
Method of operation
|
|
Transabdominal
|
184 (27.6%)
|
Transvaginal
|
14 (2.1%)
|
Hysteroscopy
|
64 (9.6%)
|
Laparoscope
|
405 (60.7%)
|
Surgical method
|
|
Removal of uterine fibroids
|
544 (81.6%)
|
Hysterectomy
|
123 (18.4%)
|
Review
|
|
No
|
322 (48.3%)
|
Yes
|
345 (51.7%)
|
Postoperative recurrence
|
|
Yes
|
102 (29.6%)
|
No
|
243 (70.4%)
|
Pathology results
|
|
Non degeneration
|
576 (86.4%)
|
Red degeneration
|
18 (2.7%)
|
Glassy degeneration
|
61 (9.1%)
|
Cystic degeneration
|
1 (0.2%)
|
Sarcoma degeneration
|
0 (0.0%)
|
Fatty degeneration
|
0 (0.0%)
|
Calcification
|
11 (1.6%)
|
Except where indicated, data are numbers of patients, with percentages in parentheses. Abbreviations: SD = Standard Deviation. |
Table 2
Data summaries for the training, internal validation and internal validation test groups
Variables
|
Training dataset (n = 1416)
|
Internal validation dataset (n = 336)
|
External validation dataset (n = 268)
|
P value
|
Diameter of uterine fibroids (cm)
|
|
|
|
0.53
|
< 4 cm
|
180 (12.7%)
|
37 (11.0%)
|
30 (11.2%)
|
|
≥ 4 cm and < 8 cm
|
1038 (73.3%)
|
247 (73.5%)
|
191 (71.3%)
|
|
≥ 8 cm
|
198 (14.0%)
|
52 (15.5%)
|
47 (17.5%)
|
|
Number of fibroids
|
|
|
|
0.26
|
Single
|
680 (48.0%)
|
147 (43.8%)
|
134 (50%)
|
|
Multiple (\(\ge\)2)
|
736 (52.0%)
|
189 (56.2%)
|
134 (50%)
|
|
Type of fibroids
|
|
|
|
0.17
|
Sub serous fibroids
|
86 (6.1%)
|
13 (3.9%)
|
10 (3.7%)
|
|
Intramural fibroids
|
1214 (85.7%)
|
303 (90.2%)
|
237 (88.4%)
|
|
Submucosal fibroids
|
116 (8.2%)
|
20 (5.9%)
|
21 (7.8%)
|
|
Location of fibroids
|
|
|
|
0.005
|
Cervix
|
11 (0.8%)
|
4 (1.2%)
|
5 (1.9%)
|
|
Ante theca
|
377 (26.6%)
|
83 (24.7%)
|
58 (21.6%)
|
|
Posterior
|
420 (29.7%)
|
134 (39.9%)
|
101 (37.7%)
|
|
Fundus
|
136 (9.6%)
|
27 (8.0%)
|
25 (9.3%)
|
|
The left side
|
187 (13.2%)
|
31 (9.2%)
|
24 (9.0%)
|
|
The right side
|
169 (11.9%)
|
37 (11.0%)
|
34 (12.7%)
|
|
Uterine cavity
|
116 (8.2%)
|
20 (6.0%)
|
21 (7.8%)
|
|
Type of ultrasound
|
|
|
|
0.04
|
Transvaginal
|
1042 (73.6%)
|
267 (79.5%)
|
209 (78.0%)
|
|
Abdominal
|
374 (26.4%)
|
69 (20.5%)
|
59 (22.0%)
|
|
Except where indicated, data are numbers of ultrasound images, with percentages in parentheses. |
2.2 Inclusion and Exclusion Criteria
The ultrasound images and clinical information data of patients with uterine fibroids and normal uteri were collected from Shunde Hospital of Southern Medical University between 2015 and 2020. We controlled the quality of the ultrasound images based on the associated pathological findings. The inclusion criteria for the abnormal patient group were as follows: 1) their preoperative ultrasound suggested the presence of a uterine mass; 2) no other combined uterine masses were present; 3) the ultrasound images were in black and white; and 4) the patients were diagnosed with uterine fibroids by two senior ultrasonographers (more than 10 years of clinical experience; 10 years of seniority or more). The exclusion criteria were as follows: 1) the patient lacked ultrasound data from our institution; and 2) images showing mass locations that did not match the clinical data. Finally, a total of 3870 ultrasound images (2020 uterine fibroids and 1850 normal uteruses) were acquired in this study.
2.3 Image Acquisition
All ultrasound images were acquired in .jpg format using a color Doppler ultrasound machine. The models included APLI300 TUS-300, APLI400 TUS-400, APLI500 TUS-500, and LOGIQ S8. The operation routes were transabdominal or transvaginal, with the abdominal ultrasound probe set at 2–7 MHz and the vaginal ultrasound probe set at 5–7 MHz
2.4 Ground Truth Labeling
We randomly grouped the 3382 ultrasound images (1752 uterine fibroids and 1630 normal uteruses) according to a training dataset (80%, 2706/3382) and an internal validation dataset (20%, 676/3382) for model training and development. The ground truths (GTs) of the training and validation dataset were labeled with Visual Geometry Group Image Annotator software. Ultrasonographers with more than 10 years of experience labeled the ultrasound images of uterine fibroids through each patient's clinical information and ultrasound image reports. If data samples with excessive labeling biases were generated, the final results were voted on again by the three ultrasonographers to determine the GTs results. The system automatically generated json files, which included the image and information such as the size and location of the GTs. The flow diagram is in Fig. 1. Examples of original ultrasound data and sample data produced after labeling (including the GTs) are provided in Fig. 2.
2.5 DCNN-Based Detection Algorithm
In this study, we developed a two-stage DCNN model to detect uterine fibroids in ultrasound images. The network structure in this study consisted of two parts: 1) the YOLOv3 detection network used to detect lesion regions in the ultrasound images and 2) another ResNet50 [14] network that was used to classify the images as normal or abnormal.
The YOLOv3 is a state-of-the-art object detection network that uses features from the entire input image to predict a bounding box for each region of interest. ResNet50, a unique residual module, was used to replace the original network in YOLOv3 to learn more complex feature representations from the ultrasound images of uterine fibroids. The ResNet50 backbone were pre-trained on the ImageNet [15] classification task and fine-tuned on our training dataset. The original ultrasound images and the bounding boxes outlined by the radiologist (covering the entire lesion area) were used as input data for training the YOLOv3 detection network.
All training and testing procedures were developed with Paddle (version 2.0.2), CUDA (version 10.1) and Python (version 3.7) [16]. Four graphics processing units (GPUs, NVIDIA GeForce GTX 1080Ti) were used, and the total training time was 10 hours. The Adam [17] optimizer was initialized, and each mini-batch contained 12 images. The weight decay was set as 0.0005, and the momentum was set as 0.9. A summary of the layers and output sizes of the ultrasound images produced by the DCNN during training is illustrated in Supplementary Table 1 (online). Outline of the DCNN for uterine fibroid detection and some uterine fibroid detection results are shown in Fig. 3 and Fig. 4, respectively.
2.6 Reference Standard
Four junior ultrasonographers (with less than 5 years of experience) and four senior ultrasonographers (with more than 10 years of experience) were selected to participate in the determination of the validation dataset from Shunde Hospital of Southern Medical University. Each ultrasonographer independently interpreted the ultrasound image data contained in the external validation dataset and gave answers, and four junior ultrasonographers interpreted the ultrasound image data again with the assistance of DCNN model after interpreting the ultrasound image data individually.
2.7 Statistical Analysis
To compare the performance of DL methods with that of each ultrasonographer, we calculated their sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and accuracy metrics. Sensitivity was calculated as the percentage of correctly detected uterine fibroid images, and specificity was calculated as the percentage of correctly detected normal uterine images. The number of true positive (TP), false positive (FP), true negative (TN), and false negative (FN) findings yielded by the described methods were determined based on the reference standards. According to the TP rate (sensitivity) versus the FP rate (1-specificity), we depicted the receiver operating characteristic (ROC) curve and calculated the area under the ROC curve (AUC) for each method. Finally, a statistical analysis was performed by the chi-square test using SPSS 22.0, and P < 0.05 was considered a statistically significant difference.