Comparison of the diagnostic performances of the Assessment of Different NEoplasias in the adneXa model, Simple Rules model, and Risk of Malignancy Index for adnexal masses in a gynecological oncology centre  in China

DOI: https://doi.org/10.21203/rs.3.rs-21877/v1

Abstract

Background

This study aimed to compare different ultrasound-based International Ovarian Tumor Analysis (IOTA) prediction models, namely, the Simple Rules (SRs) the Assessment of Different NEoplasias in the adneXa (ADNEX) models, and the Risk of Malignancy Index (RMI), for the pre-operative diagnosis of adnexal mass.

Methods

This single-centre, retrospective study of diagnostic accuracy involved 486 patients. All ultrasound examinations were analyzed and the prediction models were applied. Pathology was the clinical reference standard. The diagnostic performances of prediction models were measured by evaluating receiver-operating characteristic (ROC) curves, sensitivities, specificities, positive and negative predictive values, positive and negative likelihood ratios, and diagnostic odds ratios.

Results

To discriminate benign and malignant tumours, areas under the ROC curves (AUCs) for ADNEX models were 0.94 (95% CI: 0.92–0.96) with CA125 and 0.94 (95% CI: 0.91–0.96) without CA125, which were significantly higher than the AUCs for RMI I-III: 0.87 (95% CI: 0.83–0.90), 0.83 (95% CI: 0.80–0.86), and 0.82 (95% CI: 0.78–0.86), (all P < 0.0001). At a cut-off of 10%, the ADNEX model with CA125 had the highest sensitivity (0.93; 95% CI: 0.87–0.97) compared with the other models. The SRs model achieved a sensitivity of 0.93 (95% CI: 0.86–0.97) and a specificity of 0.86 (95% CI: 0.82–0.89) when inconclusive diagnoses (11.7%) were classified as malignant.

Conclusion

ADNEX and SRs models were excellent at characterising adnexal masses which were superior to the RMI in Chinese patients.

Background

Ovarian cancer (OC) has the highest mortality rate and most unfavourable prognosis among the gynaecological malignancies; the average 5-year survival rate is < 50% [1, 2]. To optimise the treatment and improve the survival of patients with malignant ovarian tumours, and to avoid unnecessary interventions in and preserve the fertility of patients with benign ovarian tumours, accurately characterising benign and malignant ovarian masses through appropriate staging is essential. Currently, transvaginal ultrasound is the most commonly used imaging technique for pre-operative evaluations of adnexal masses [3, 4]. Subjective assessments of ultrasound findings by specialists in gynaecological ultrasonography are one of the best means of evaluating adnexal masses in clinical practice [59].

Several ultrasound-based prediction models have been developed to accurately discriminate between benign and malignant tumours, because the numbers of experienced examiners are insufficient and they are unavailable in some regions. The Risk of Malignancy Index (RMI), which accounts for the serum cancer antigen (CA) 125 levels, menopausal status, and the ultrasound findings, is a prediction model that is recommended by many national guidelines [1012]. However, the procedures used to calculate the RMI are time-consuming, and its diagnostic performance is unsatisfactory. The International Ovarian Tumour Analysis (IOTA) group presented a consensus statement about the ultrasound characteristics of adnexal tumours in 2000 [13], and other diagnostic models were subsequently developed and validated, including the Logistic Regression model 2 [14, 15], simple ultrasound-based rules or Simple Rules (SRs) model [1618], and the Assessment of Different NEoplasias in the adneXa (ADNEX) model [19]. The findings from previous external validation studies have shown that the SRs model is easy to use and its diagnostic performance is good, but it is not suitable for all adnexal masses [1618]. The ADNEX model is excellent at differentiating between malignant and benign tumours [5, 2022], and it indicates the stages of malignant tumours.

This study aimed to compare the ADNEX and SRs models, and the RMI regarding their abilities to discriminate between benign and malignant adnexal masses using data from a single oncology centre in China.

Materials And Methods

Study setting and design

This single-centre retrospective study was approved by the Institutional Ethics Committee of Ruijin Hospital at Shanghai Jiaotong University’s School of Medicine. The requirement for informed consent was waived.

Between June 2017 and June 2018, the study was carried out using data prospectively collected from consecutive patients. It evaluated the diagnostic performances of the ADNEX and SRs models, and variants of the RMI (I–III) within a population of women who underwent surgery to remove adnexal masses at the Department of Obstetrics and Gynaecology at Ruijin Hospital in Shanghai, China, which is a tertiary referral oncology centre. All of the patients underwent pre-operative transvaginal or transrectal ultrasonography examinations according to the IOTA protocol [13] to assess the morphology of the adnexal masses. Clinicians made the final decisions regarding surgery and clinical judgments.

Participants

The patients were prospectively and consecutively enrolled, and they presented with ≥ 1 ultrasound-diagnosed adnexal mass. The inclusion criteria were ≥ 1 adnexal mass detected by transvaginal or transrectal ultrasonography that was not a physiological cyst, patients who were prepared to undergo surgery based on a clinician’s recommendation, and a time interval of 30 days between ultrasound and surgery.

Participants were excluded from the study if they failed to undergo surgery, they were diagnosed with a recurrence of OC, they had undergone a bilateral adnexectomy previously, they had an ectopic pregnancy, or their clinical data were incomplete. A total of 486 patients were included in the final analysis.

Data collection

All patients underwent pre-operative transvaginal or transrectal ultrasonography using Voluson E10 (GE Healthcare) and iU22 (Philips Healthcare) ultrasound machines with 5.0–9.0 MHz and 4.0–8.0 MHz transvaginal probes, and 1.0–5.0 MHz transabdominal probes, and the findings were recorded. When a malignancy was suspected or a mass was too large to be evaluated using transvaginal ultrasonography alone, transabdominal ultrasonography was performed. Two expert ultrasonographers with ≥ 10 years of experience in gynaecological ultrasound assessed the tumours’ pre-operative sonographic morphologies using the IOTA protocol’s nomenclature and methodology [13]. After the ultrasound examinations and before the statistical analysis of the data, we applied the ADNEX model and three variants of the RMI to calculate the risk of malignancy without knowledge of the histological findings. When multiple adnexal masses were detected, we analysed the mass with the most complex ultrasonographic morphology, and when masses had similar morphological characteristics, we chose the largest mass [13, 19].

Before the ultrasound examinations, we collected clinical data describing the patients’ ages, menopausal statuses, previous malignancies, and family histories of OC. The patients’ pre-operative CA125 levels were measured using a chemoluminescence technique and an automatic analyser (i2000SN; Abbott AxSYM).

Prediction models

Three prediction models were used to differentiate between benign and malignant adnexal masses. The ADNEX model is available at no cost on the IOTA website (https://www.iotagroup.org/iota-models-software/adnex-risk-model) or it can be installed as a mobile phone application; it comprises nine predictors, including three clinical and six ultrasound variables [19]. After inputting all the predictors objectively, the probability ratios for a benign or a malignant mass are displayed both graphically and numerically. As it is the first multiclass prediction model for adnexal masses, the likelihoods of a mass being a borderline ovarian tumour (BOT), stage I OC, stages II–IV OC, or a metastasis are presented. The ADNEX model is available in versions that include and exclude the CA125 level, and we evaluated the predictive accuracy of the ADNEX model with and without CA125 in this study.

The SRs model comprises a set of rules based on five ultrasound features that indicate benignity (B-features) and five features that indicate malignancy (M-features) [1618]. A lesion is classified as benign if ≥ 1 B-feature is present in the absence of any M-features, and malignant if ≥ 1 M-feature is present in the absence of any B-features. If both B-features and M-features are present or if none of the features are present, the model yields an inconclusive result.

Three principal variants of the RMI scoring system (RMI-I, RMI-II, and RMI-III) were applied that combined the ultrasound findings, serum CA125 levels, and menopausal status [1012]. The points attributed to patients’ ultrasound findings and menopausal statuses differ for the RMI variants, and these points generate a score; a total score of ≥ 200 points was used as the cut-off for malignancy.

Reference standard

Pathology was the reference standard used for all patients in this study. Tissue specimens obtained during surgery were analysed by a team of pathologists who specialised in gynaecological pathology and were unaware of the ultrasound findings. The tumours were classified according to the World Health Organization’s guidelines for the classification of tumours [23]. The stages of the malignant tumours were defined using the International Federation of Gynecology and Obstetrics 2012 criteria [24].

Statistical analyses

Basic discrimination between benign and malignant adnexal masses by the ADNEX model with or without the CA125 levels and the three RMI variants was assessed using receiver-operating characteristic curves (ROCs) and summarised by calculating the areas under the curves (AUCs). The prediction methods’ AUCs were compared using the method described by DeLong et al [25]. As AUCs could not be calculated for the SRs model, which is based on categorical variables, the McNemar test was used to assess the model’s discrimination between benign and malignant adnexal masses. Diagnostic performance measures, including the sensitivities, specificities, positive and negative predictive values, positive and negative likelihood ratios, and the diagnostic odds ratios (DORs), were calculated to evaluate the models’ classifications of benign or malignant tumours using cut-off points proposed in previous publications [5, 1012, 19].

The ultrasonographic and clinical characteristics of, and the CA125 levels associated with the benign and malignant tumours were compared; the chi-square test and Fisher’s exact test were used to analyse the categorical data, and the Mann-Whitney U-test was used to analyse the continuous data. The statistical analyses were conducted using IBM®SPSS® software, version 22.0 (IBM Corporation) and MedCalc Statistical Software, version 15.2.2 (MedCalc Software bvba). BOTs were considered malignant for the purposes of the statistical analyses. All of the statistical calculations were performed using 95% confidence intervals (CIs), and a value of P < 0.05 was considered statistically significant.

Results

Clinical findings and pathologic diagnosis

Between June 2017 and June 2018, 591 consecutive women with adnexal tumours who underwent pre-operative ultrasound examinations were prospectively enrolled. The final cohort consisted of 486 women; 105 women met the exclusion criteria and were excluded from study. Figure 1 provides a detailed overview of the patients’ inclusion and exclusion from the study.

In the final analysis, 486 patients with 366 (75.3%) benign and 120 (24.7%) malignant adnexal masses were included. Table 1 presents the histological results. Endometriomas (19.8%, 96/486) and serous cystadenomas (13.4%, 65/486) were the most common benign diagnoses. Among the malignant masses, 7.0% (34/486) were BOTs, 4.9% (24/486) were stage I OCs, 9.7% (47/486) were stages II–IV OCs, and 3.1% (15/486) were metastases.

Table 1
Distributions of histology outcomes of 486 adnexal masses
Histological type of masses
n(%)
Benign
366 (75.3)
Endometrioma
96 (19.8)
Serous cystadenoma
65 (13.4)
Teratoma
46 (9.5)
Mucinous cystadenoma
34 (7.0)
Hydrosalpinx
32 (6.6)
Fibrothecoma
20 (4.1)
Mesosalpinx cyst
17 (3.5)
Parovarian cyst
14 (2.9)
Cystadenofibroma
6 (1.2)
Fibroma
6 (1.2)
Adenofibroma
4 (0.8)
Brenner tumour
4 (0.8)
Peritoneal mesothelioma
3 (0.6)
Sertoli-Leydig cell tumour
2 (0.4)
Sclerosing stromal tumour
1 (0.2)
Tuberculosis
1 (0.2)
Other ovarian benign lesion
15 (3.1)
Borderline
34 (7.0)
Serous
16 (3.3)
Mucinous
15 (3.1)
Endometrioid
3 (0.6)
Primary ovarian malignant
71(14.6)
Serous adenocarcinoma
42 (8.6)
Clear cell carcinoma
10 (2.1)
Endometrioid adenocarcinoma
6 (1.2)
Mucinous adenocarcinoma
3 (0.6)
Sertoli-Leydig cell tumour
2 (0.4)
Carcinosarcoma
2 (0.4)
Granulosa cell tumour
2 (0.4)
Seromucinous adenocarcinoma
1 (0.2)
Diffuse large B cell lymphoma of ovary
1 (0.2)
Small cell neuroendocrine carcinoma
1 (0.2)
Strumal carcinoid of ovary
1 (0.2)
Metastasis
15 (3.1)
Gastric cancer
6 (1.2)
Appendiceal adenocarcinoma
3 (0.6)
Cholangiocarcinoma
2 (0.4)
Breast cancer
2 (0.4)
Pancreatic cancer
2 (0.4)

Table 2 summarises the patients’ clinical characteristics and data describing the ultrasound findings from the benign and malignant tumours. The patients with malignancies were older, were more likely to be post-menopausal and to have a family history of OC, and had higher CA125 levels than those with benign tumours (all P < 0.05). Regarding the ultrasound findings, the malignant tumours had significantly greater diameters, more solid tissue, wider solid tissue components, > 10-cyst locules, more papillary projections, and more ascites compared with the benign masses (all P < 0.001). None of the patients with malignant tumours had acoustic shadows.

Table 2
Results with regard to clinical characteristics and ultrasound features for 486 patients with adnexal mass
Characteristic
Benign
(n = 366)
Malignant
(n = 120)
P
Age (years)
41 (31–51)
54 (42–63)
< 0.001a
Menopausal status
   
< 0.001b
Premenopausal
275 (75.1)
53 (44.2)
 
Postmenopausal
91 (24.9)
67 (55.8)
 
CA125 (U/mL)
10 (11–38)
54 (18–517)
< 0.001a
Family history of OC
0 (0.0)
2 (1.7)
0.013c
Maximal diameter of lesion (mm)
56 (44–72)
78 (50–126)
< 0.001a
Presence of solid tissue
108 (29.5)
107 (89.2)
< 0.001b
Proportion solid tissue if present (mm)
31 (17–46)
44 (20–65)
< 0.001a
Presence of papillary projections
39 (10.7)
46 (38.3)
< 0.001b
0
327 (89.3)
74 (61.7)
 
1
21 (5.7)
13 (10.8)
 
2
6 (1.6)
3 (2.5)
 
3
5 (1.4)
5 (4.2)
 
༞3
7 (1.9)
25 (20.8)
 
> 10-cyst locules
11 (3.0)
22 (18.3)
< 0.001b
Acoustic shadows
43 (11.7)
0 (0.0)
< 0.001c
Ascites
2 (0.5)
35 (29.2)
< 0.001b
Data are given as n (%) for categorical data and median (interquartile range) for continuous data. aMann-Whitney U-test for continuous data, bChi-square test and cFisher’s exact test for categorical data. OC, ovarian cancer.

Diagnostic performance of adnexal mass prediction models

Table 3 details the diagnostic performances of the adnexal mass prediction models regarding their discrimination between benign and malignant tumours. The AUCs for the ADNEX models for differentiating malignant tumours from benign tumours that did and did not account for the CA125 level were 0.94 (95% CI: 0.92–0.96) and 0.94 (95% CI: 0.91–0.96), respectively. At a cut-off of 10%, the performance of the prediction model that included CA125 was excellent, with a sensitivity of 0.93 (95% CI: 0.87–0.97), a specificity of 0.76 (95% CI: 0.72–0.81), and a DOR of 43.67, and the performance of the prediction model that did not include CA125 had a sensitivity of 0.93 (95% CI: 0.87–0.97), a specificity of 0.74 (95% CI: 0.69–0.79), and a DOR of 40.00.

Table 3
Diagnostic performance of the prediction models for discrimination between benign and malignant adnexal masses.
Assessment method
AUC
Sensitivity
Specificity
PPV
NPV
LR+
LR-
DOR
ADNEX125
0.94
(0.92–0.96)
0.93
(0.87–0.97)
0.76
(0.72–0.81)
0.80
(0.75–0.84)
0.92
(0.87–0.95)
3.93
(3.20–4.72)
0.09
(0.04–0.22)
43.67
ADNEXN125
0.94
(0.91–0.96)
0.93
(0.87–0.97)
0.74
(0.69–0.79)
0.78
(0.73–0.83)
0.92
(0.87–0.95)
3.60
(3.00-4.31)
0.09
(0.05–0.20)
40.00
SRs + BE
NA
0.69
(0.60–0.77)
0.96
(0.93–0.97)
0.94
(0.90–0.97)
0.76
(0.70–0.80)
15.82
(9.66–25.93)
0.32
(0.25–0.42)
49.44
SRs + MAL
NA
0.93
(0.86–0.97)
0.86
(0.82–0.89)
0.87
(0.82–0.91)
0.92
(0.88–0.95)
6.51
(5.04–8.42)
0.09
(0.05–0.16)
72.33
RMI-I
0.87(0.83–0.90)
0.55
(0.46–0.64)
0.93
(0.90–0.96)
0.89
(0.83–0.94)
0.67
(0.62–0.72)
8.05
(5.33–12.19)
0.48
(0.43–0.59)
16.77
RMI-II
0.83
(0.80–0.86)
0.61
(0.52–0.70)
0.92
(0.89–0.95)
0.89
(0.83–0.93)
0.70
(0.65–0.75)
7.95
(5.42–11.75)
0.42
(0.33–0.52)
18.93
RMI-III
0.82
(0.78–0.86)
0.53
(0.44–0.63)
0.94
(0.91–0.96)
0.90
(0.84–0.95)
0.67
(0.62–0.72)
9.30
(5.91–14.49)
0.50
(0.45–0.63)
18.60
Values in parentheses are 95% CI. Prediction models: ADNEX125, the Assessment of Different NEoplasias in the adneXa model with CA125 level; ADNEXN125, the Assessment of Different NEoplasias in the adneXa model without CA125 level; SRs + BE, International Ovarian Tumour Analysis simple ultrasound-based rules applied with inconclusive tumours (13.2%, 64/486 cases) being classified as benign; SRs + MAL, International Ovarian Tumour Analysis simple ultrasound-based rules applied with inconclusive results being categorised as malignant; RMI-I, RMI-II, RMI-III, three variants of the Risk of Malignancy Index. For ADNEX models, cut-off value of 10% was used and for the three variants of RMI model, cut-off value of 200 was used. AUC, area under receiver-operating characteristic curve; PPV, positive predictive value; NPV, negative predictive value; LR+, positive likelihood ratio; LR–, negative likelihood ratio; DOR, diagnostic odds ratio; NA, not applicable.

The SRs model was applicable to 422 (86.8%) patients with adnexal tumours. Of the tumours with inconclusive diagnoses, 56.3% (36/64) were benign tumours, 20.3% (13/64) were BOTs, 14.1% (9/64) were stage I OCs, 4.7% (3/64) were stages I–IV OCs, 4.7% (3/64) were metastases, approximately 43.8% were malignant histologically, and most of the benign masses (75.0%, 27/36) presented with a solid component. When the masses with inconclusive diagnoses were classified as benign, the SRs model’s diagnostic performance had a sensitivity of 0.69 (95% CI: 0.60–0.77), a specificity of 0.96 (95% CI: 0.93–0.97), and a DOR of 49.44, and when they were categorised as malignant, the SRs model’s diagnostic performance had a sensitivity of 0.93 (95% CI: 0.86–0.97), a specificity of 0.86 (95% CI: 0.82–0.89), and a DOR of 72.33.

The three RMI variants with cut-offs of 200 showed poor diagnostic performances in relation to the adnexal masses. Regarding RMI variants I, II, and III, the AUCs for differentiating malignant tumours from benign tumours were 0.87 (95% CI: 0.83–0.90), 0.83 (95% CI: 0.80–0.86), and 0.82 (95% CI: 0.78–0.86), respectively, with sensitivities of 0.55 (95% CI: 0.46–0.64), 0.61 (95% CI: 0.52–0.70), and 0.53 (95% CI: 0.44–0.63), respectively, and specificities of 0.93 (95% CI: 0.90–0.96), 0.92 (95% CI: 0.89–0.95), and 0. 94 (95% CI: 0.91–0.96), respectively.

Figure 2 shows the ROC curves for the ADNEX model and the RMI variants for differentiating malignant and benign tumours. The ADNEX model with or without CA125 was superior to the RMI variants regarding the diagnosis of malignant and benign tumours. When the SRs model yielded inconclusive results that were classified as malignancies, the model’s diagnostic performance was good. Figure 3 (a, b) and Fig. 4 (a, b) present two case examples of the prediction models using in the clinical pactice.

Table 4 summarises the pairwise ROC curve comparisons of the ADNEX model with or without CA125 and the RMI, which are expressed as differences in the AUCs. The difference between the AUCs for the ADNEX model with or without CA125 was not significant (AUC difference: 0.0002; 95% CI: 0.01–0.02). Comparisons of the ADNEX model with or without CA125 and the three RMI variants revealed significant differences in the AUCs that ranged from 0.074 to 0.118 (all P < 0.0001). Comparisons of the ADNEX model with and without CA125 with RMI variant I showed the greatest differences in the AUC (AUC difference: 0.074; 95% CIs: 0.039–0.109 and 0.040–0.108, respectively; P < 0.0001). The diagnostic performances of the three RMI variants remained statistically significant for the pre-operative diagnosis of adnexal masses (AUC differences: 0.010–0.044; all P < 0.05).

Table 4
Pairwise ROC curve comparisons expressed as differences in AUC and P-values for the study.
d-AUC ( Pa )
ADNEXN125
RMI-I
RMI-II
RMI-III
ADNEX125
0.0002 (-0.012-0.012)
P = 0.977
0.074 (0.039–0.109)
P < 0.0001
0.109 (0.066–0.151)
P < 0.0001
0.118 (0.076–0.160)
P < 0.0001
ADNEXN125
/
0.074 (0.040–0.108)
P < 0.0001
0.108 (0.064–0.153)
P < 0.0001
0.118 (0.073–0.163)
P < 0.0001
RMI-I
/
/
0.035 (0.010–0.060)
P = 0.007
0.044 (0.017–0.071)
P = 0.001
RMI-II
/
/
/
0.010 (0.003–0.016)
P = 0.002
aComparisons of area under the curves (AUCs) of prediction models using Delong’s test; methods in left column are used as reference standard for comparisons; d-AUC, differences in area under the curve. Prediction models: ADNEX125, the Assessment of Different NEoplasias in the adneXa model with CA125 level; ADNEXN125, the Assessment of Different NEoplasias in the adneXa model without CA125 level; RMI-I, RMI-II, RMI-III, three variants of the Risk of Malignancy Index. Values in parentheses are 95% CI.

Discussion

Correctly discriminating between benign and malignant adnexal masses is a crucial starting point for optimal treatment. We compared the diagnostic performances of the ADNEX and SRs models, and the RMI. The RMI was the first prediction model used clinically, and it is the most widely used model in many regions [4, 26, 27]. However, our study’s findings showed that the ADNEX model was superior to the three RMI variants at distinguishing between benign and malignant adnexal masses. The ADNEX model with and without CA125 had higher AUCs (both 0.94) than the AUCs generated for the RMI variants that ranged from 0.82 to 0.87. Like previous studies’ findings [5, 19], the ADNEX model showed a better diagnostic performance and a higher level of sensitivity than the RMI in our study. Hence, the ADNEX model might be favoured for pre-operatively differentiating adnexal masses in Chinese patients.

Pre-operative evaluations using the SRs model were robust, with a sensitivity of 0.93 (95% CI: 0.86–0.97) and a specificity of 0.86 (95% CI: 0.82–0.89) for adnexal masses with inconclusive diagnoses that were classified as malignant; these findings are similar to the results from previous studies [5, 17, 21, 28, 29]. The IOTA SRs model is widely accepted as an effective prediction model for adnexal masses by clinicians, and its use is recommended in the 2011 Green-top guidelines for the assessment and management of suspected ovarian masses in pre-menopausal women that were developed by the Royal College of Obstetricians and Gynaecologists in the United Kingdom [30]. Recently, the American College of Obstetricians and Gynecologists incorporated the SRs model into their clinical practice guidelines for the evaluation and management of adnexal masses [31]. Followed in the First International Consensus on Adnexal Masses the SRs model was recommended as the main diagnostic strategy [32]. The SRs model is easy to apply in clinical practice, and it can be used for approximately 76–89% of adnexal masses [17, 33]. The SRs model was applicable to about 86.8% of the patients in our study. When specialists in gynaecological ultrasonography are not available, classifying tumours as malignant is reasonable following inconclusive diagnoses using the SRs model [17, 34]. However, this approach could be biased by the prevalence of malignant tumours within the population, and approximately half of the patients with benign diagnoses might undergo unnecessary interventions [17, 34]. An accurate second-stage diagnostic strategy is needed in China, particularly in the most remote undeveloped regions where the numbers of specialists in gynaecological ultrasonography are inadequate.

Our analyses determined that 64 patients had tumours with inconclusive diagnoses following the application of the SRs protocol to the ADNEX model with or without CA125 and the three RMI variants. Compared with the three RMI variants, the AUC for the ADNEX model was higher (0.59 vs 0.73), the sensitivity was greater (0.29–0.36 vs 0.89), and the specificities were lower (0.86–0.89 vs 0.33–0.39) (Supplementary Tables 1 and 2). Regarding the tumours with inconclusive diagnoses, the prediction models’ AUCs did not differ, which may be attributable to the limited sample size. Nevertheless, regarding the identification of malignant tumours among the masses with inconclusive diagnoses, the ADNEX model yielded slightly higher AUCs and DORs than the three RMI variants. More studies are needed to determine whether the ADNEX model could become a second-stage diagnostic strategy for adnexal tumours with inconclusive diagnoses when specialists in gynaecological ultrasonography are not available.

This is one of the first studies to compare the ultrasound-based IOTA prediction models and the RMI in a population of Chinese patients in strict accordance with the IOTA consensus statement, which is a study strength. Additionally, we prospectively and consecutively enrolled unselected patients, and only patients whose data were complete were included. Moreover, our results were validated within a relatively large study population. The study’s weakness, namely, its single-centre design, may have caused a sampling bias and limited the applicability of the results to other regions. Moreover, the ultrasound examinations were not performed by those with different levels of training experience in our study. More studies in different diagnostic centres with different levels of ultrasound expertise in China are needed to further evaluate the prediction models.

Conclusions

In conclusion, our study’s findings showed that the ADNEX and SRs models performed well in relation to discriminating between benign and malignant adnexal masses, and that both models were superior to the RMI in a Chinese context.

List of Abbreviations

ADNEX Assessment of Different NEoplasias in the adneXa

AUC area under the curve

BOT borderline ovarian tumour

CA carbohydrate antigen

CIs confidence intervals

DOR diagnostic odds ratio

FIGO International Federation of Gynecology and Obstetrics

IOTA International Ovarian Tumor Analysis

LR2 Logistic Regression model 2

MHz megahertz

OC ovarian cancer

ROC receiver-operating characteristics

RMI Risk of Malignancy Index

SRs the Simple Rules

WHO World Health Organization

Declarations

Ethics approval and consent to participate

The study was approved by the Ruijin Hospital, Shanghai Jiaotong University School of Medicine institutional ethics (Grant No.2018-136).

Consent for publication

Consent for publication was obtained from patients in this study.

Availability of data and material

The datasets analysed during the current study available from the corresponding author on reasonable request.

Competing interests

The authors declare that they have no competing interests.

Funding

This work was sponsored by The ultrasonic model in diagnosis of ovarian tumors (KY20200033), Shanghai Sailing Program (18YF1414400) and National Natural Science Foundation of China (81901488).

Authors' contributions

QL: participated in the study development, data collection, data analysis and manuscript writing.

DQW: participated in data analysis and manuscript writing and revising.

JMJ: participated in data collection and manuscript writing.

YF: provided advice for the manuscript.

CH: conceptualized the study, appraised and revised the manuscript.

FWW: appraised and revised the manuscript.

All authors read and approved the final manuscript.

Le Qian, Qinwen Du contributed equally to this work.

Corresponding author

Correspondence to Hui Chen

Acknowledgements

The authors are grateful to the participants of this study.

References

  1. Jayson GC, Kohn EC, Kitchener HC, Ledermann JA. Ovarian cancer. Lancet. 2014;384(9951):1376–88.
  2. Siegel RL, Miller KD, Jemal A. Cancer statistics. CA Cancer J Clin. 2019;69(1):7–34.
  3. Piovano E, Cavallero C, Fuso L, Viora E, Ferrero A, Gregori G, Grillo C, Macchi C, Mengozzi G, Mitidieri M, Pagano E, Zola P. Diagnostic accuracy and cost-effectiveness of different strategies to triage women with adnexal masses: a prospective study. Ultrasound Obstet Gynecol. 2017;50(3):395–403.
  4. Menon U, Gentry-Maharaj A, Hallett R, Ryan A, Burnell M, Sharma A, Lewis S, Davies S, Philpott S, Lopes A, Godfrey K, Oram D, Herod J, Williamson K, Seif MW, Scott I, Mould T, Woolas R, Murdoch J, Dobbs S, Amso NN, Leeson S, Cruickshank D, McGuire A, Campbell S, Fallowfield L, Singh N, Dawnay A, Skates SJ, Parmar M, Jacobs I. Sensitivity and specificity of multimodal and ultrasound screening for ovarian cancer, and stage distribution of detected cancers: results of the prevalence screen of the UK Collaborative Trial of Ovarian Cancer Screening (UKCTOCS). Lancet Oncol. 2009;10(4):327–40.
  5. Meys EMJ, Jeelof LS, Achten NMJ, Slangen BFM, Lambrechts S, Kruitwagen RFPM, Van Gorp T. Estimating risk of malignancy in adnexal masses: external validation of the ADNEX model and comparison with other frequently used ultrasound methods. Ultrasound Obstet Gynecol. 2017;49(6):784–92.
  6. Meys EM, Kaijser J, Kruitwagen RF, Slangen BF, Van Calster B, Aertgeerts B, Verbakel JY, Timmerman D, Van Gorp T. Subjective assessment versus ultrasound models to diagnose ovarian cancer: A systematic review and meta-analysis. Eur J Cancer. 2016;58:17–29.
  7. Valentin L, Hagen B, Tingulstad S, Eik-Nes S. Comparison of 'pattern recognition' and logistic regression models for discrimination between benign and malignant pelvic masses: a prospective cross validation. Ultrasound Obstet Gynecol. 2001;18(4):357–65.
  8. Van Gorp T, Veldman J, Van Calster B, Cadron I, Leunen K, Amant F, Timmerman D, Vergote I. Subjective assessment by ultrasound is superior to the risk of malignancy index (RMI) or the risk of ovarian malignancy algorithm (ROMA) in discriminating benign from malignant adnexal masses. Eur J Cancer. 2012;48(11):1649–56.
  9. Timmerman D, Schwärzler P, Collins WP, Claerhout F, Coenen M, Amant F, Vergote I, Bourne TH. Subjective assessment of adnexal masses with the use of ultrasonography: an analysis of interobserver variability and experience. Ultrasound Obstet Gynecol. 1999;13(1):11–6.
  10. Jacobs I, Oram D, Fairbanks J, Turner J, Frost C, Grudzinskas J. A risk of malignancy index incorporating CA125, ultrasound and menopausal status for the accurate preoperative diagnosis of ovarian cancer. Br J Obstet Gynaecol. 1990;97(10):922–9.
  11. Tingulstad S, Hagen B, Skjeldestad FE, Onsrud M, Kiserud T, Halvorsen T, Nustad K. Evaluation of a risk of malignancy index based on serum CA125, ultrasound findings and menopausal status in the pre-operative diagnosis of pelvic masses. Br J Obstet Gynaecol. 1996;103(8):826–31.
  12. Tingulstad S, Hagen B, Skjeldestad FE, Halvorsen T, Nustad K, Onsrud M. The risk-of-malignancy index to evaluate potential ovarian cancers in local hospitals. Obstet Gynecol. 1999;93(3):448–52.
  13. Timmerman D, Valentin L, Bourne TH, Collins WP, Verrelst H, Vergote I. Terms, definitions and measurements to describe the sonographic features of adnexal tumors: a consensus opinion from the International Ovarian Tumor Analysis (IOTA) Group. Ultrasound Obstet Gynecol. 2000;16(5):500–5.
  14. Timmerman D, Testa AC, Bourne T, Ferrazzi E, Ameye L, Konstantinovic ML, Van CB, Collins WP, Vergote I, Van HS. Logistic regression model to distinguish between the benign and malignant adnexal mass before surgery: a multicenter study by the International Ovarian Tumor Analysis Group. J Clin Oncol. 2005;23(34):8794–801.
  15. Timmerman D, Van Calster B, Testa AC, Guerriero S, Fischerova D, Lissoni AA, Van Holsbeke C, Fruscio R, Czekierdowski A, Jurkovic D, Savelli L, Vergote I, Bourne T, Van Huffel S, Valentin L. Ovarian cancer prediction in adnexal masses using ultrasound-based logistic regression models: a temporal and external validation study by the IOTA group. Ultrasound Obstet Gynecol. 2010;36(2):226–34.
  16. Timmerman D, Testa AC, Bourne T, Ameye L, Jurkovic D, Van Holsbeke C, Paladini D, Van Calster B, Vergote I, Van Huffel S, Valentin L. Simple ultrasound-based rules for the diagnosis of ovarian cancer. Ultrasound Obstet Gynecol. 2008;31(3):681–90.
  17. Timmerman D, Ameye L, Fischerova D, Epstein E, Melis GB, Guerriero S, Van Holsbeke C, Savelli L, Fruscio R, Lissoni AA, Testa AC, Veldman J, Vergote I, Van Huffel S, Bourne T, Valentin L. Simple ultrasound rules to distinguish between benign and malignant adnexal masses before surgery: prospective validation by IOTA group. BMJ. 2010. doi:.
  18. Timmerman D, Van Calster B, Testa A, Savelli L, Fischerova D, Froyman W, Wynants L, Van Holsbeke C, Epstein E, Franchi D, Kaijser J, Czekierdowski A, Guerriero S, Fruscio R, Leone FPG, Rossi A, Landolfo C, Vergote I, Bourne T, Valentin L. Predicting the risk of malignancy in adnexal masses based on the Simple Rules from the International Ovarian Tumor Analysis group. Am J Obstet Gynecol. 2016;214(4):424–37.
  19. Van Calster B, Van Hoorde K, Valentin L, Testa AC, Fischerova D, Van Holsbeke C, Savelli, Franchi D, Epstein E, Kaijser J, Van Belle V, Czekierdowski A, Guerriero S, Fruscio R, Lanzani C, Scala F, Bourne T, Timmerman D. Evaluating the risk of ovarian cancer before surgery using the ADNEX model to differentiate between benign, borderline, early and advanced stage invasive, and secondary metastatic tumours: prospective multicentre diagnostic study.BMJ. 2014; doi:.
  20. Araujo KG, Jales RM, Pereira PN, Yoshida A, de Angelo Andrade L, Sarian LO, Derchain S. Performance of the IOTA ADNEX model in preoperative discrimination of adnexal masses in a gynecological oncology center. Ultrasound Obstet Gynecol. 2017;49(6):778–83.
  21. Sayasneh A, Wynants L, Preisler J, Kaijser J, Johnson S, Stalder C, Husicka R, Abdallah Y, Raslan F, Drought A, Smith AA, Ghaem-Maghami S, Epstein E, Van Calster B, Timmerman D, Bourne T. Multicentre external validation of IOTA prediction models and RMI by operators with varied training. Br. J Cancer. 2013;108(12):2448–54.
  22. Chen H, Qian L, Jiang M, Du Q, Yuan F, Feng W. Performance of IOTA ADNEX model in evaluating adnexal masses in a gynecological oncology center in China. Ultrasound Obstet Gynecol. 2019;54(6):815–22.
  23. Meinhold-Heerlein I, Fotopoulou C, Harter P, Kurzeder C, Mustea A, Wimberger P, Hauptmann S, Sehouli J. The new WHO classification of ovarian, fallopian tube, and primary peritoneal cancer and its clinical implications. Arch Gynecol Obstet. 2016;293(4):695–700.
  24. Prat J, FIGO Committee on Gynecologic Oncology. Staging classification for cancer of the ovary, fallopian tube, and peritoneum. Int J Gynaecol Obstet. 2014;124(1):1–5.
  25. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristics curves: a nonparametric approach. Biometrics. 1988;44(3):837–45.
  26. Andersen ES, Knudsen A, Rix P, Johansen D. Risk of malignancy index in the preoperative evaluation of patients with adnexal masses. Gynecol Oncol. 2003;90(1):109–12.
  27. Geomini P, Kruitwagen R, Bremer GL, Cnossen J, Mol BW. The accuracy of risk scores in predicting ovarian malignancy: a systematic review. Obstet Gynecol. 2009;113(2 Pt 1):384–94.
  28. Kaijser J, Bourne T, Valentin L, Sayasneh A, Van Holsbeke C, Vergote I, Testa AC, Franchi D, Van Calster B, Timmerman D. Improving strategies for diagnosing ovarian cancer: a summary of the International Ovarian Tumor Analysis (IOTA) studies. Obstet Gynecol. 2013;41(1):9–20.
  29. Testa A, Kaijser J, Wynants L, Fischerova D, Van Holsbeke C, Franchi D, Savelli L, Epstein E, Czekierdowski A, Guerriero S, Fruscio R, Leone FP, Vergote I, Bourne T, Valentin L, Van Calster B, Timmerman D. Strategies to diagnose ovarian cancer: new evidence from phase 3 of the multicentre international IOTA study. Br J Cancer. 2014;111(4):680–8.
  30. Royal College of Obstetricians and Gynecologists, British Society of Gynecological Endoscopy. Management of suspected ovarian masses in premenopausal women. In: Green-top guidelines 62. 2011. . . Accessed 02 Dec 2011.
  31. American College of Obstetricians and Gynecologists’ Committee on Practice Bulletins-Gynecology. Practice Bulletin No.174: Evaluation and Management of Adnexal Masses. Obstet Gynecol. 2016;128(4):e210–26.
  32. Glanc P, Benacerraf B, Bourne T, Brown D, Coleman BG, Crum C, Dodge J, Levine D, Pavlik E, Timmerman D, Ueland FR, Wolfman W, Goldstein SR. First International Consensus Report on Adnexal Masses: Management Recommendations. J Ultrasound Med. 2017;36(5):849–63.
  33. Nunes N, Ambler G, Foo X, Naftalin J, Widschwendter M, Jurkovic D. Use of IOTA simple rules for diagnosis of ovarian cancer: meta-analysis. Ultrasound Obstet Gynecol. 2014;44(5):503–14.
  34. Froyman W, Timmerman D. Methods of Assessing Ovarian Masses: International Ovarian Tumor Analysis Approach. Obstet Gyneco Clin North Am. 2019;46(4):625–41.