CT-Based Radiomics and Machine Learning for Differentiating Benign, Borderline, and Early-stage Malignant Ovarian Tumors: A Multi-Class Classification and Multi-Model Comparation Study

doi:10.21203/rs.3.rs-2233426/v1

Download PDF

Research Article

CT-Based Radiomics and Machine Learning for Differentiating Benign, Borderline, and Early-stage Malignant Ovarian Tumors: A Multi-Class Classification and Multi-Model Comparation Study

https://doi.org/10.21203/rs.3.rs-2233426/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background

To explore and evaluate value a preoperative diagnosis model with contrast-enhanced computed tomography (CECT) imaging-based radiomics analysis in differentiating benign ovarian tumors (BeOTs), borderline ovarian tumors (BOTs), and early-stage malignant ovarian tumors (eMOTs).

Results

The retrospective research was conducted with pathologically confirmed 258 ovarian tumors patients from January 2014 to February 2021. All patients underwent preoperative CECT examination. The patients were randomly allocated to a training cohort (n = 198) and a test cohort (n = 60). A summary of 4238 radiomic features were extracted per patient. By providing a 3D characterization of the regions of interest (ROI) with ITK SNAP software at the maximum level of enhanced CT image, radiomic features were extracted from the ROI with an in-house software written in Python. The Wilcoxon–Mann–Whitney (WMW) test, least absolute shrinkage and selection operator logistic regression (LASSO-LR) and support vector machine (SVM) were employed to select the radiomic features. Five machine learning (ML) algorithms were applied to construct three-class diagnostic models for characterizing ovarian tumors taking the selected radiomic features parameters. Leave-one-out cross-validation (LOOCV) that estimated performance in an ‘independent’ dataset was implemented to evaluate the performance of the radiomics models in the training cohort. An independent dataset, that is the test cohort, was used to verify the generalization ability of the radiomics models. The receiver operating characteristics (ROC) was used to evaluate diagnostic performance of radiomics model. Global diagnostic performance of five models were evaluated by average area under the ROC curve (AUC).

Conclusion

The average ROC indicated that random forest (RF) diagnostic model in training cohort demonstrated the best diagnostic performance (micro average AUC, 0.98; macro average AUC, 0.99), which was then confirmed with by internal cross-validation (LOOCV) (micro average AUC, 0.89; macro average AUC, 0.88) and external validation (test cohort) (micro average AUC, 0.81; macro average AUC, 0.79). Our proposed CECT image-based radiomics diagnostic models may effectively assist in preoperatively differentiating BeOTs, BOTs, and eMOTs.

Ovarian tumors

Computed tomography

Radiomics

Machine learning

Differential diagnosis

Adnexal masses derive from ovary, which is very common in gynecological imaging, can be classified as benign, borderline, and malignant(1). Benign ovarian tumors (BeOTs), borderline ovarian tumors (BOTs), which can be cashed early and cured by surgical treatment, have a favorable prognosis as it is associated with a low recurrence or metastasis rate. However, malignant ovarian tumors (MOTs), which has the highest lethal ratio in gynecologic malignancy, even is the number five cause of cancer-related death in women(2). The ovaries locate in the pelvic cavity and are very hidden. Not only that, but there is still no clinically significant symptomatology and effective treatment at earlier stages of the disease. Therefore, only 20–25 percent of early-stage malignant ovarian tumors (eMOTs) could be detected and the outcomes of patients at an advanced phase are poor (3, 4). Thus, early in the disease and preoperative diagnostic nature of adnexal mass is benefit for the prognosis of ovarian tumors.

Preoperative biopsy is not recommended because of the risk of extravasation of tumor cell into the abdominal cavity. The non-invasive preoperative diagnosis methods of adnexal masses located in the ovary mainly depend on serum tumor markers and imaging methods, such as ultrasound, CT, MRI, and PET/CT. While current biomarkers (such as CA125, HE4 and CA19-9) has been widely put into routine clinical use of ovarian tumors, they have poorly diagnostic specificity (4–6). On the other hand, CT is the first-line preoperative diagnostic tool of the ovarian tumors, which are of important clinical consequences for estimating the extent of ovarian tumors and rationalizing surgical planning(7). But in actual operation, differentiating BeOTs, BOTs, and early-stage MOTs (International Federation of Gynecology and Obstetrics [FIGO] stage I–II) using CT still face challenges, which is qualitative and likely to be influenced by subjective factors of experienced examiners. Furthermore, information about tumor heterogeneity is ignored, which leads to traditional CT imaging application in classification of ovarian tumors was limited because of lacking in accuracy, quantifiability and reproducibility (8).

Radiomics could abstract the higher-dimensional data from preprocessed images, provide a non-invasive and quantitative method for exploration heterogeneity and biological characteristics of tumors(9, 10). As a sub-domain of artificial intelligence (AI), machine learning(ML) is the most common way to mine data of radiomics, which establish machine learning models, such as logistic regression (LR) or standard neural network (SNN), using CT radiomics features, which, has provide a new and wide prospect in predicting tumorigenesis development(11). Radiomics has been expected to be used for judgement of biological behavior of ovarian tumors, such as BeOTs or MOTs(12, 13), abdominopelvic recurrence(14–16), reactions to medicines(17), molecular characterization(18–20), prediction of lymph node metastasis before surgery(21), surgical outcome (22) and prediction of prognosis(18, 20, 23–25).

Radiomics combines CT images with machine learning algorithms and quantitatively assesses for pixel and spatial distributions to catch details of tumor heterogeneity, which has become a hot topic of medicine area at present. To the best of our knowledge, there is no previous research have been done evaluating radiomics features of ovarian tumors to diagnosis BeOTs, BOTs, and eMOTs preoperatively, which is hard for radiologists to identify the patients only by CT imaging characteristics.

The purpose of this study is looking to combine radiomics with contrast-enhanced computed tomography (CECT)—the most widely used modality for imaging diagnostic in ovarian tumors, which may provide preoperative diagnosis radiomic models for identifying BeOTs, BOTs, and eMOTs.

2.1 Patient Clinical Factors

Altogether the study enrolled 258 ovarian tumors patients with an average age of 45.92 ± 15.59 years (range 11–89 years), including a training cohort of 75 BeOTs, 48 BOTs and 75 eMOTs, and a test cohort of 23 BeOTs, 15 BOTs and 22 MOTs. The main baseline demographic, clinical characteristics of the patients are summarized in Table 1 and Table 2. Table 1 shows no statistical difference in age, menstrual status and FIGO stage was found between the training and test cohorts. However, training and test cohorts significantly differed with respect to HE4 (p = 0.004). Table 2 shows the differences in clinical characteristics and tumor markers (CA125, HE4) among patients with BeOTs, BOTs and MOTs in the training and testing cohorts. BeOTs, BOTs and eMOTs significantly differed with respect to tumor markers (p < 0.01).

Table 1

Clinical characteristics and biochemical examinations among BeOTs, BOTs and MOTs patients.

	BeOT	BOT	MOT	P Value	BeOT(C)2	BOT		MOT(A)0	P Value
No. Patients	75(37.9)	48(24.2)	75(37.9)		23(38.3)	15(25.0)	22(36.7)
Age (years)	43.00 ± 15.93	46.06 ± 18.17	47.55 ± 13.52	0.123	44.17 ± 15.84	47.40 ± 18.71		50.86 ± 11.40	0.388
Postmenopausal				0.688					0.749
Yes	39(52.0)	24(50.0)	43(57.3)		11(47.8)	9(60.0)		11(50.0)
No	36(48.0)	24(50.0)	32(42.7)		12(52.2)	6(40.0)		11(50.0)
CA125(miu/ml)	19.47 ± 5.53	51.61 ± 25.40	418.63 ± 427.76	0.000	33.79 ± 72.60	80.42 ± 123.70		482.61 ± 711.44	0.000
HE 4	44.45 ± 18.8	104.12 ± 188.37	239.44 ± 356.07	0.000	48.14 ± 39.48	137.41 ± 255.83		72.22 ± 81.70	0.012
FIGO-stage				0.900					0.056
I	-	36(75.0)	57(76.0)		-	14(93.3)		14(63.6)
II	-	12(25.0)	18(24.0)		-	1(6.7)		8(36.4)
Histologya
Epithelial tumors
Serous	27(36.0)	18 (37.5)	23 (30.7)		8(34.8)	4(26.7)		7(31.8)
Mucinous	15(20.0)	27(56.3)	7 (9.3)		9(39.1)	5(33.3)		1(4.5)
Seromucinous	1(1.3)	2(4.2)	3 (4.0)		0(0)	2(13.3)		4(18.2)
Endometrioid	0(0)	1(2.0)	8 (10.67)		0(0)	2(13.3)		1(4.5)
Clear cell	0(0)	0(0)	11 (14.67)		0(0)	0(0)		3(13.6)
Brenner tumours	0(0)	0(0)	0(0)		0(0)	2(13.3)		0(0)
Mesenchymal tumors
Carcinosarcoma	0(0)	0(0)	2(2.7)		0(0)	0(0)		0(0)
Sex cord-stromal tumors
Granulosa cell	0(0)	0(0)	10(13.3)		0(0)	0(0)		6(27.3)
Germ cell tumors
Mature teratoma	32(42.7)	0(0)	0(0)		6(26.1)	0(0)		0(0)
Yolk sac tumor	0(0)	0(0)	5(6.7)		0(0)	0(0)		0(0)
Immature teratoma	0(0)	0(0)	2(2.7)		0(0)	0(0)		0(0)
Dysgerminoma	0(0)	0(0)	2(2.7)		0(0)	0(0)		0(0)
Mixed germ cell tumor	0(0)	0(0)	2(2.7)		0(0)	0(0)		0(0)
The number in parenthesis is the percentage

Table 2

Clinical characteristics and biochemical examinations of patients enrolled in this research
Characteristic	Full Cohort (n = 258)	Patients
Characteristic	Full Cohort (n = 258)	Training Cohort (n = 198)	Test Cohort (n = 60)	P Value
Age (years)	45.92 ± 15.59	45.46 ± 15.71	47.43 ± 15.21	0.382
Postmenopausal
Yes	137	106	31	0.799
No	121	92	29
CA125(miu/ml)	258.14 ± 655.85	242.93 ± 612.04	308.35 ± 787.12	0.623
HE 4	123.74 ± 237.12	137.69 ± 258.89	76.81 ± 132.55	0.004
FIGO-stage				0.668
I	121	94	27
II	39	29	10
Histologya
Epithelial tumors
Serous	87 (33.7)	67 (33.8)	20 (33.3)
Mucinous	64 (24.8)	49 (24.7)	15 (25.0)
Seromucinous	12 (4.6)	6 (3.0)	6 (10.0)
Endometrioid	12 (4.6)	10 (5.0)	2 (3.3)
Clear cell	14 (5.4)	11 (5.5)	3 (5.0)
Brenner tumours	2 (0.7)	0 (0)	2 (3.3)
Mesenchymal tumors
Carcinosarcoma	2 (0.7)	2 (1.0)	0 (0)
Sex cord-stromal tumors
Granulosa cell	16 (6.2)	10 (5.0)	6 (10.0)
Germ cell tumors
Mature teratoma	38 (14.7)	32 (16.1)	6 (10.0)
Yolk sac tumor	5 (1.9)	5 (2.5)	0 (0)
Immature teratoma	2 (0.7)	2 (1.0)	0 (0)
Dysgerminoma	2 (0.7)	2 (1.0)	0 (0)
Mixed germ cell tumor	2 (0.7)	2 (1.0)	0 (0)
The number in parenthesis is the percentage

2.2 Radiomic Feature Extraction and Selection

Totally, we extracted 4238 radiomic features from each patient. After dimensionality reduction conducted by WMW, 359 features were retained. Furthermore, the 359 features were selected by LASSO and SVM algorithm combination method considering the λ with the largest mean AUC (λ = 10), and then, 58 important features for differentiating BeOTs, BOTs, and MOTs were finally obtained to build the ML. The selected features are summarized in Fig. 1 and Supplementary Table 1.

2.3 Machine Learning Models Performance

Five algorithms of LR, SNN, RF, KNN and DT were utilized for three-class ML models construction. In training cohort, the five had excellent performance with macro/micro average AUC of 0.97, 0.98, 0.99/0.98, 0.91/0.92 and 0.90 for LR, SNN, RF, KNN and DT, respectively. In internal cross-validation (LOOCV), SNN had excellent performance with macro/micro average AUC of 0.91/0.92, while the other four models had very good performance with that of 0.89/0.91, 0.88/0.89, 0.84/0.86 and 0.83/0.85 for LR, RF, KNN and DT, respectively. In test cohort, models achieved good performance except for DT with macro/micro average AUC of 0.72/0.74, 0.72/0.74, 0.78/0.81 and 0.75/0.77 for LR, SNN, RF and KNN, respectively. In test cohort, only RF and KNN achieved 0.6 in discrimination AUC of BOTs (AUC = 0.60 and AUC = 0.62, respectively), which means the diagnostic power were sufficient. There was no significant difference in diagnostic performance of BOTs between RF and KNN (P = 0.61). And RF showed better discriminatory performances than those of the LR and SNN, with AUC for MOTs were 0.89 and 0.83 (p = 0.01 and p = 0.012, respectively). The discriminatory power of RF was also superior to that of DT in the identification of BeOTs (p = 0.003). While there was no significant difference in diagnostic performance between KNN and LR or SNN (P > 0.05). The discriminatory power of KNN was superior to that of DT in the identification of BeOTs (p = 0.015). Taken together, RF-based three-class model was considered the best. The ability of five ML models in each data cohort were displayed in Table 3 and Figs. 2.

Table 3

Predictive performance of five machine learning algorithms: LR, SNN, RF, KNN and DT
Model		Overall performance (macro/micro)				BeOT				BOT			MOT
Model		Training cohort	LOOCV	Test cohort	Training cohort		LOOCV	Test cohort	Training cohort	LOOCV	Test cohort	Training cohort	LOOCV	Test cohort
LR	Acc	0.87	0.73	0.55	0.95	0.89		0.78	0.88	0.78	0.65	0.91	0.79	0.67
	Pre	0.86/0.87	0.70/0.73	0.52/0.55	0.93	0.83		0.75	0.79	0.57	0.29	0.86	0.71	0.54
	SENS	0.85/0.87	0.70/0.73	0.52/0.55	0.95	0.89		0.65	0.71	0.44	0.27	0.91	0.76	0.64
	SPE	0.94/0.87	0.86/0.73	0.78/0.55	0.96	0.89		0.86	0.94	0.89	0.78	0.91	0.81	0.68
	AUC	0.97/0.97	0.89/0.91	0.72/0.74	0.99	0.96		0.85	0.95	0.82	0.52	0.96	0.89	0.74
SNN	Acc	0.89	0.77	0.57	0.96	0.90		0.78	0.90	0.80	0.65	0.92	0.83	0.70
	Pre	0.88/0.89	0.74/0.77	0.54/0.57	0.95	0.85		0.78	0.81	0.60	0.29	0.89	0.78	0.57
	SENS	0.88/0.89	0.74/0.77	0.53/0.57	0.96	0.91		0.61	0.79	0.52	0.27	0.89	0.79	0.73
	SPE	0.95/0.89	0.88/0.77	0.78/0.57	0.97	0.90		0.89	0.94	0.89	0.78	0.93	0.86	0.68
	AUC	0.98/0.98	0.91/0.92	0.72/0.74	0.99	0.96		0.86	0.97	0.85	0.51	0.97	0.90	0.75
RF	Acc	0.89	0.72	0.63	0.96	0.90		0.83	0.91	0.76	0.70	0.91	0.78	0.73
	Pre	0.89/0.89	0.68/0.72	0.61/0.63	0.95	0.85		0.84	0.88	0.51	0.38	0.85	0.68	0.61
	SENS	0.88/0.89	0.68/0.72	0.60/0.63	0.96	0.91		0.70	0.75	0.35	0.33	0.92	0.77	0.77
	SPE	0.95/0.89	0.86/0.72	0.82/0.63	0.97	0.90		0.92	0.97	0.89	0.82	0.90	0.78	0.71
	AUC	0.99/0.98	0.88/0.89	0.78/0.81	0.99	0.96		0.89	0.98	0.81	0.60	0.97	0.86	0.83
KNN	Acc	0.73	0.67	0.65	0.89	0.86		0.82	0.79	0.74	0.78	0.78	0.74	0.70
	Pre	0.71/0.73	0.62/0.67	0.66/0.65	0.85	0.80		0.80	0.60	0.43	0.60	0.68	0.63	0.57
	SENS	0.69/0.73	0.62/0.67	0.62/0.65	0.88	0.84		0.70	0.34	0.27	0.40	0.81	0.75	0.77
	SPE	0.86/0.73	0.83/0.67	0.82/0.65	0.90	0.87		0.89	0.92	0.89	0.91	0.76	0.73	0.66
	AUC	0.91/0.92	0.84/0.86	0.75/0.77	0.97	0.94		0.87	0.94	0.75	0.62	0.87	0.84	0.76
DT	Acc	0.89	0.70	0.58	0.93	0.88		0.78	0.93	0.75	0.68	0.93	0.77	0.70
	Pre	0.90/0.90	0.69/0.70	0.62/0.58	0.97	0.87		0.92	0.86	0.48	0.39	0.87	0.70	0.57
	SENS	0.90/0.90	0.68/0.70	0.57/0.58	0.95	0.81		0.48	0.88	0.54	0.47	0.96	0.69	0.77
	SPE	0.95/0.90	0.85/0.70	0.80/0.58	0.98	0.93		0.97	0.95	0.81	0.76	0.91	0.82	0.66
	AUC	0.97/0.98	0.83/0.85	0.69/0.69	0.98	0.92		0.70	0.96	0.72	0.61	0.98	0.83	0.75
LOOCV, Leave-one-out cross-validation. AUC, area under the receiver operating characteristic. SPE, Specificity. SENS, sensitivity. Acc, accuracy. Pre, precision. LR, logistic regression. SNN, standard neural network. RF, random forest. KNN, k-nearest neighbors. DT, decision tree.

BeOTs, BOTs and eMOTs are very different in terms of their treatment and prognosis. The fact that lack of techniques for early detecting characteristics of ovarian tumors is associated with an increased risk of women mortality. Especially because of BOT’s similarity in radiographic findings to BeOTs, the BOTs are frequently misdiagnosed, and its therapeutic regimens and prognoses is probably underestimated(26, 27). Not only that, BeOTs are also sometimes hard to distinguish from MOTs in some cases, such as ovarian cystadenoma (28). Therefore, accurately distinguish those three kinds of ovarian tumors is the essential link in promoting therapeutic effects and prolonging the survival duration. Currently, a CT report is fundamental to diagnosis and evaluation of ovarian disease and cannot be replaced by routine or random noninvasive and invasive testing which may be neither comprehensive nor precise enough. However, radiologists interpretate CT image with the naked eye may ignore some imaging features, introduce interobserver bias, or be too subjective. Hence, an accurate preprocedural diagnosis model that identify different kind of ovarian tumors when they found as adnexal masses at the first time would contribute to make personalized clinical decision. With that in mind, our goal of this study is to explore and verify a machine-aided diagnosis model based on CECT image features and machine learning algorithm, and different algorithms were compared by diagnostic performance. This diagnosis model, which compare with conventional diagnostic methods, is objective, convenient and noninvasive. In this study, we explore the feasibility of five three-class machine learning models constructed by utilizing CECT based radiomics features to differentiate nature of ovarian tumors. The diagnostic model based on RF algorithm exhibited the best diagnostic performance to identify different nature of ovarian tumors in the two datasets (including training and test cohorts).

Studies have shown that imaging technologies-based radiomic frameworks could provide valuable and truthful information on ovarian tumor nature. The ultrasound-based ML model has shown an accuracy that is over 85% in differentiating BeOTs from MOTs (29). MR is multiparameter and better than other imaging procedures in soft tissue resolution. An algorithmic, as an aid in MRI for distinguishing BeOTs from MOTs, been recommended by European Society of Urogenital Radiology (ESUR) (30). Zhang et al. extracted radiomics features radiomics extracted from MR multiple sequences exhibit favorable performance in differentiating BeOTs, BOTs, and MOTs, and superior to that of radiologists. More importantly, radiologist’s lack of distinguishing BeOTs from BOTs was balanced by MR radiomics (31). More recently, Song et al. also found that radiomics features form 7 different DCE-MR images have favorable discrimination function with AUCs of 0.89, 0.94, and 0.89 for BeOT, BOT, and MOT, respectively (32). MR radiomic shows good diagnosis performance not just in vivo environments, but in vitro tissue of human ovarian tumors (33). However, CT-based radiomic research is principally focuses on the relationship between survival and radiomic features (20, 34–39), and BOT is seldom investigated. In contrast to previous research, this study built a CECT-based radiomic three-class preoperative diagnosis model for analyzing the nature of primary ovarian tumors. Our results support the previous studies that radiomic have diagnostic power for discriminating among BeOTs, BOTs, and MOTs.

In this study, because the CT images of two cohorts were supplied by different multidetector CT scanners, gray-level normalization of each VOI was performed to limit those variabilities. CECT-based radiomic features that extracted through algorithms defined by IBSI were applied to wavelet transform and quantization to get effective features. And then, WMW, LASSO and SVM were employed to select the most suitable radiomics features. The result of multiple classifiers, including LR, SNN, RF, KNN and DT was combined for comparison to choose the best one in the test cohort. Average AUC and discrimination AUC were used to assess performances of the models. In training cohort, excellent performance can be achieved in the five models. In LOOCV, SNN also had excellent performance, the other four were very good performance. In test cohort, models achieved good performance except for DT. These results suggest that models not only exhibited excellent performance in the training cohorts but had good generalization in test cohort. The degradation of diagnostic performance in three cohorts (including training, LOOCV and test cohorts) mainly caused by uneven distribution of pathological type. The models constructed by the RF obtained the maximum average AUC (0.78/0.81) in test cohort and the average bias between training AUC and testing AUC (0.20) was smaller than that in LR, DT and SNN. Our results indicating that in addition to BOTs group, the one-vs-all ROC curves of the other two groups had very good or good distinguishing performance, which echoed previous studies that BOTs were more difficult to be distinguished from ovarian neoplasms(26, 27). The one-vs-all ROC curves of the RF and KNN had sufficient discriminative ability to separate BOTs that requires aggressive treatment to prevent further progression, which again proved that radiomics digged the link between images and ovarian neoplasms more deeply than that of visual interpretation(32). RF showed higher discrimination performances than those of the KNN, with AUC for BeOTs and MOTs were 0.89 and 0.83.

Varying levels of quantization on images to obtain high diagnosis performance is rarely seen in differentiating ovarian tumors. Moreover, accurate texture features were extracted by applying different quantization levels in which 7-bit and 8-bit are relatively rare. Vamvakas et al. applied 8-bit quantized images to acquire textural features that contribute to MR glioma grading (40). Takahiro et al. selected the top 5 features for MR glioma grading and found 3 of them came from 7-bit or 8-bit quantized images (41). In our study, 58 selected features were highly associated with the diagnosis of ovarian tumors, and 44 of them had to be processed for Image quantization. 17 of them came from 4-bit, 15 of them came from 5-bit, 10 of them came from 8-bit, and the remaining 2 selected features were from 6-bit and 7-bit quantized images. That means two points: first, it is necessary to set different quantization levels to acquire accurate texture features; second, effective features perhaps come from a few rare quantization levels.

Combining machine learning with multiple classification in radiomics can be viewed as a novel diagnosis system in ovarian tumors identification. Split a multiple classification model across several dichotomous model, which is an essential ritual for solving multiple classification problems. One-vs-all algorithm can effectively extend binary classification algorithm to multi-classification problems. The diagnosis performance of models was evaluated by ROC curve and AUC in training cohort, and then, generalization performance of the models was assessed in test cohort. Discrimination performances of one-vs-all AUC and average AUC were assessed finally. Among the five ML models, the RF-based model exhibits the best performance in differentiating BeOTs, BOTs, and MOTs, with a macro/micro average AUC of 0.99/0.98 and an accuracy of 0.89 in training cohorts. The test cohort achieved an AUC of 0.89 for the BeOT, 0.60 for the BOT, and 0.83 for the eMOTs (Fig. 4b). Although there are no previous studies combining multiple classification and ML to differentiate ovarian lesions in CT images, Song et al. applied MR radiomics features based 3-class classification to discriminate ovarian neoplasms, and the findings revealed that the test cohort with AUC of 0.82, 0.81, and 0.85 for BeOTs, BOTs, and MOTs groups, respectively, proving a more effective method (32). Perhaps three reasons for they had better result than we did: first, MR have higher resolution for soft tissue, which lead to information come from CECT features is much more limit than that from MR. Second, radiomics features from 7 different DCE-MR images were used to construct a 3-class classification model which owns more information for descripting lesions. Third, eMOTs is more difficult to identify than MOTs in most cases. Although maybe CT-based model is less efficient than MR-based model, the negatively effect of the result is likely to be small from a clinical perspective because CT is still the routine method for the preoperative staging (42, 43). Besides, portal venous phase that was only considered in this study is the most frequently used phase in diagnosing of ovarian tumors(44). Lastly, compared to MR-derived features, CT-derived features are generally considered more reliable(45). Chen et al. builted a three-class CT radiomic model to discriminate risk stratification of gastrointestinal stromal tumors, and the result indicated that the training cohort and internal validation cohort with average AUC of 0.84 and 0.83, respectively (46). From this, we can see three classification model can obtain efficient, reliable diagnostic performance not only in ovarian tumors but also in other tumors.

This study possesses several limitations. First, our patients are all from a single centre, and just 258 patients were recruited, so robustness and reliability of the ML model for differentiating BeOTs, BOTs, and MOTs needs to be deeply researched with multicenter prospective validation. Second, 3D tumor was manually segmented, which may lead to time-consuming, subjective, unstable VOI segmentation and the model’s performance is likely to be effective. Thus, an automatic image segmentation method that accurate and reliable should be applied to map the VOI. Third, we built a pure radiomics diagnosis model that without any consideration of clinical characteristics and subjective CT findings model. Although the two sets were often be considered inferior to radiomics model in previous studies, a comprehensive model with combination of radiomics, clinical characteristics and subjective CT findings may perform better.

In current study, we provided a quantitative diagnosis model for CT-based on radiomics, which selected the radiomic features by t-tests and LASSO + SVM, and the result of five multi-class classifiers were combined for comparison. RF-based model demonstrated excellent diagnostic performance, and through the internal and external verification, its generalization ability had also been verified. This RF-based multi-class diagnosis model may be a noninvasive, convenient and promising method to assist in differentiating among BeOTs, BOTs, and eMOTs.

This retrospective study was approved by the Institutional Review Board of Guangxi Medical University Affiliated Cancer Hospital (No. LW2022153), and the acquisition of informed patient consent was waived. Informed consent regarding knowledge of adverse reactions to iodinated contrast is signed by all study participants before CT examination.

5.1. Patients

Between January 2014 and February 2021, a total of 258 consecutive patients who underwent preoperative CECT from our picture archiving and communication system (PACS) database and were pathologically confirmed BeOTs (epithelial (serous, mucinous, seromucinous, etc.), non-epithelial (mature cystic teratoma, fibroma, thecoma, etc.)), BOTs, or eMOTs. In patients with bilateral tumors (n = 23), we select more complex tumors for statistical analysis in order to reduce the potential effect of intraindividual correlation.

Inclusive criteria: (1) Pathologically proven BeOTs, BOT, or eMOTs; (2) preoperative CECT images at our institution; (3) Without operation or radiochemotherapy before CT scans. Exclusion criteria: (1) Other malignancy besides ovarian cancer; (2) No available CECT images prior to surgery; (3) Undergoing neoadjuvant operation or radiochemotherapy before surgery. (4) Tumor volume too small to be analyzed. The flow chart of inclusion and exclusion criteria was shown in Fig. 3.

Comparison of characteristics among BeOTs, BOTs, and eMOTs are summarized in Table 1, including age, postmenopausal, histology, FIGO stage, and tumor markers (CA125, HE4). The patients included in the study were randomly split into two teams: training cohort (n = 189) and test cohort (n = 60). Clinical characteristics and biochemical examinations of the two cohorts are summarized in Table 2. Our study flow diagram is shown in Fig. 4.

5.2 CT Image Acquisition and Tumors Segmentation

The Digital Imaging and Communications in Medicine (DICOM) files of CT images were extracted from the picture archiving and communication system (PACS). We adopted portal-venous phase (PVP) of CECT (60-70s delay). Two high resolution CT scanners 64-slice scanner (Discovery CT750 HD) and 128-slice scanner (Discovery CT750, GE Healthcare) were used. Either 64 or 128 slices the scanning parameters were as follows: field of view, 25 cm; matrix, 512×512; tube voltage, 120 kVp; tube current, 200–400 mA; rotation, 0.5s, pitch, 0.984:1; interval, 1.3 mm. The contrast agent (iodine, 350 mg/ml) was administered at 2 ml/kg (but not exceeding 100 ml) and 2 ml/s. Slice thickness was 0.625mm, and image reconstruction to 5mm thickness was carried out.

Two radiologists (Jia Chen and Zheng Wang, with 7 and 10 years of experience in CT abdominal diagnosis, respectively) who blinded to pathology result manually determine the volume of interest (VOI) by applying ITK-SNAP imaging package (ITK-SNAP, version 3.6.1; www.itksnap.org). 3D segmentation of the VOI in PVP images which covered the whole primary ovarian tumors. The calculation of the intraclass correlation coefficient (ICC) is done for assessing the interobserver agreement between the two readers, which ensure accuracy and reproducibility of data. ICC values > 0.75 was considered to indicate good agreement (47).

First, we determine the VOI for 50 randomly selected patients in our study. Second, interobserver reliability in two readers was assessed by determining VOI with blind evaluation. Third, to assess the intrareader consistency, JC repeated the determination procedure after one month on 50 randomly selected patients.

5.3 Image preprocessing and Radiomic Feature Extraction

PyRadiomics, an open-source Python 3.6 (https://www.python.org) package, was used to extraction and analysis of radiomic features from VOIs on PVP images. The VOI were preprocessed according to the image biomarker standardization initiative(IBSI)that existed prior to the radiomics feature extraction to standardize this process(48). 111 3D standardized features within the IBSI were initially extracted from the VOI (summarized in Supplementary Table 2). We apply a 3D Coifet Wavelet Transform(eight decompositions), which is mathematical tool for improved textural inference, makes further reconstruction images possible. The quantization was applied within the range of µ ± 3σ (µ = gray-level mean; and σ = gray-level standard deviation) to reduce the influence of contrast and brightness that could affect the true image textures(41, 49). The quantization levels were set to 4, 5, 6, 7, and 8bit. A total of 4238 radiomic features extracted from the VOI, which standardized using Z-score(50) and clustered using Ward’s method, were provided as heat maps in training cohort and test cohort(Figure 5).

5.4 Selection and Machine learning of Radiomic features

Radiomics feature selection as follows: first, the ability of radiomics features in differentiating nature of ovarian tumors was assessed by the Wilcoxon–Mann–Whitney (WMW) test, which was used for preliminary dimensionality reduction (P < 0.001); second, the significant radiomic features were selected by Leave-one-out cross validation (LOOCV) and the least absolute shrinkage and selection operator (LASSO) logistic regression that performed by a 3-fold cross-validation. We apply an optimal parameter (β) to select radiomics features in LASSO, by this means, the non-zero coefficient will be reserved. In the training cohort, by changing the values of the tuning parameter (λ), which ranged from 10^− 6 to 10², the area under receiver operating characteristic (ROC) curve (AUC) were achieved(51) (Fig. 6).

Scikit-learn (version 0.18.1, http://scikit-learn.org/stable/), a ML library in the Python programming environment, was used to mine the radiomics data. We established five models for differentiating nature of ovarian tumors, including logistic regression (LR), standard neural network (SNN), random forest (RF), k-nearest neighbors (KNN) and decision tree (DT). The performances of the five ML models were evaluated by ROC in training cohort and test cohort. Sensitivity, specificity, accuracy, precision and AUC were used as performance metrics to evaluate the five ML models.

5.5 Statistics

Statistical analyses were conducted with R language (version 3.4.3; https://www.r-project.org) and BM SPSS Statistics version 25.0. When a two-tailed p < 0.05, the test was considered statistically significant. The represents of categorical variables were expressed as percentages (%) and counts (n), and that of continuous variables were expressed as mean values ± standard deviations or as median with interquartile range (IQR). Acrossing the three datasets, differences in characteristics were assessed by One-way ANOVA or one-way non-parametric ANOVA (Kruskal-Wallis test) for continuous variables in normally or non-normally distributed, then a post hoc test was applied in appropriate. Continuous variables between groups were assessed by Student’s t test or the Wilcoxon test. Categorical variables were assessed with Chi-square test or Fisher’s exact test. Kappa statistic was employed to evaluate concordance between the two radiologists who segmented images. ROC curves for the five ML model were calculated on both the training cohort and test cohort to define the AUC of radiomic features. The DeLong test was used as a nonparametric method for the comparison of AUC values between models. Classification models with AUC values of 1.00–0.90, 0.90–0.80, 0.80 − 0.70, 0.70 − 0.60, 0.60 − 0.50 and < 0.5 are assessed as excellent, very good, good, sufficient, bad and test not useful, respectively(52). Macro-average and micro-average values were used for defining diagnostic metrics, including accuracy, sensitivity, specificity, precision and AUC, as overall performance indicators of multi-class classification(53).

CECT: Contrast-enhanced computed tomography; BeOTs: Benign ovarian tumors; BOTs: Borderline ovarian tumors; eMOTs: early-stage Malignant ovarian tumors; ROI: Region of interest; WMW: Wilcoxon–Mann–Whitney; LASSO-LR: Least absolute shrinkage and selection operator logistic regression; SVM: Support vector machine; ML: Machine learning; LOOCV: Leave-one-out cross-validation; ROC: Receiver operating characteristics; AUC: Area under the ROC curve; AI; artificial intelligence; LR: logistic regression; SNN: standard neural network; ESUR: European society of urogenital radiology; PACS; Picture archiving and communication system; PVP: portal-venous phase; VOI: Volume of interest; ICC: Intraclass correlation coefficient; RF: Random forest; KNN: k-nearest neighbors; DT: Decision tree; IQR: Interquartile range; SPE, Specificity; SENS, Sensitivity; Acc, Accuracy; Pre, Precision.

Ethics approval and consent to participate

The experimental protocol was established, according to the ethical guidelines of the Helsinki Declaration and was approved by the Human Ethics Committee of Guangxi Medical University Cancer Hospital (LW2022153). Written informed consent was obtained from individual or guardian participants.

Consent for publication

Not applicable.

Availability of data and materials

All data generated or analyzed during this study are included in this published article.

Competing interests

No potential conflicts of interest were disclosed.

Funding

This study was funded in part by grants from the National Natural Science Foundation of China (Grant No. 81560425), Guangxi Clinical Research Center for Medical Imaging Construction (Grant No. Guike AD 20238096) and Beijing Medical Award Foundation (Grant No. YXJL-2022-0665-0210).

Author contributions

Conception and design: Danke Su, Chanzhen Liu

Development of methodology: Jia Chen, Lei Liu, Danke Su, Chanzhen Liu

Acquisition of data: Lei Liu, Ziying He

Analysis and interpretation of data: Jia Chen, Lei Liu, Ziying He

Writing, review, and/or revision of the manuscript: Jia Chen

Administrative, technical, or material support: Danke Su, Chanzhen Liu

Study supervision: Danke Su, Chanzhen Liu

Acknowledgements

We appreciated Dong Xie and Zheng Wang for providing medical guidance to the manuscript.

Practice Bulletin No. 174: Evaluation and Management of Adnexal Masses. Obstet Gynecol. 2016;128(5):e210-e26.
Siegel RL, Miller KD, Fuchs HE, Jemal A. Cancer Statistics. 2021. CA Cancer J Clin. 2021;71(1):7–33.
Menon U, Gentry-Maharaj A, Burnell M, Singh N, Ryan A, Karpinskyj C, et al. Ovarian cancer population screening and mortality after long-term follow-up in the UK Collaborative Trial of Ovarian Cancer Screening (UKCTOCS): a randomised controlled trial. Lancet. 2021;397(10290):2182–93.
Mathieu KB, Bedi DG, Thrower SL, Qayyum A, Bast RC. Jr. Screening for ovarian cancer: imaging challenges and opportunities for improvement. Ultrasound Obstet Gynecol. 2018;51(3):293–303.
Shipeng G, Yongning C, Yadi Z, Chanyuan LI, Qifan J. [Comparison of serum cancer antigen 125, human epididymis protein 4, ROMA, and CPH-I for diagnosis of ovarian cancer in Chinese patients with ovarian mass]. Nan Fang Yi Ke Da Xue Xue Bao. 2019;39(12):1393–401.
Cheng HY, Zeng L, Ye X, Ma RQ, Tang ZJ, Chu HL, et al. Age and menopausal status are important factors influencing the serum human epididymis secretory protein 4 level: a prospective cross-sectional study in healthy Chinese people. Chin Med J (Engl). 2020;133(11):1285–91.
Rizzo S, De Piano F, Buscarino V, Pagan E, Bagnardi V, Zanagnolo V, et al. Pre-operative evaluation of epithelial ovarian cancer patients: Role of whole body diffusion weighted imaging MR and CT scans in the selection of patients suitable for primary debulking surgery. A single-centre study. Eur J Radiol. 2020;123:108786.
Li W, Zhang L, Tian C, Song H, Fang M, Hu C, et al. Prognostic value of computed tomography radiomics features in patients with gastric cancer following curative resection. Eur Radiol. 2019;29(6):3079–89.
Papanikolaou N, Matos C, Koh DM. How to develop a meaningful radiomic signature for clinical use in oncologic patients. Cancer Imaging. 2020;20(1):33.
van Timmeren JE, Cester D, Tanadini-Lang S, Alkadhi H, Baessler B. Radiomics in medical imaging-"how-to" guide and critical reflection. Insights Imaging. 2020;11(1):91.
Cuocolo R, Caruso M, Perillo T, Ugga L, Petretta M. Machine Learning in oncology: A clinical appraisal. Cancer Lett. 2020;481:55–62.
Park H, Qin L, Guerra P, Bay CP, Shinagare AB. Decoding incidental ovarian lesions: use of texture analysis and machine learning for characterization and detection of malignancy. Abdom Radiol (NY). 2021;46(6):2376–83.
Li S, Liu J, Xiong Y, Han Y, Pang P, Luo P, et al. Application Values of 2D and 3D Radiomics Models Based on CT Plain Scan in Differentiating Benign from Malignant Ovarian Tumors. Biomed Res Int. 2022;2022:5952296.
Shinagare AB, Balthazar P, Ip IK, Lacson R, Liu J, Ramaiya N, et al. High-Grade Serous Ovarian Cancer: Use of Machine Learning to Predict Abdominopelvic Recurrence on CT on the Basis of Serial Cancer Antigen 125 Levels. J Am Coll Radiol. 2018;15(8):1133–8.
Chen HZ, Wang XR, Zhao FM, Chen XJ, Li XS, Ning G, et al. A CT-based radiomics nomogram for predicting early recurrence in patients with high-grade serous ovarian cancer. Eur J Radiol. 2021;145:110018.
Wang S, Liu Z, Rong Y, Zhou B, Bai Y, Wei W, et al. Deep learning provides a new computed tomography-based prognostic biomarker for recurrence prediction in high-grade serous ovarian cancer. Radiother Oncol. 2019;132:171–7.
Yi X, Liu Y, Zhou B, Xiang W, Deng A, Fu Y, et al. Incorporating SULF1 polymorphisms in a pretreatment CT-based radiomic model for predicting platinum resistance in ovarian cancer treatment. Biomed Pharmacother. 2021;133:111013.
Avesani G, Tran HE, Cammarata G, Botta F, Raimondi S, Russo L, et al. CT-Based Radiomics and Deep Learning for BRCA Mutation and Progression-Free Survival Prediction in Ovarian Cancer Using a Multicentric Dataset. Cancers (Basel). 2022;14(11).
Kassuhn W, Klein O, Darb-Esfahani S, Lammert H, Handzik S, Taube ET, et al. Classification of Molecular Subtypes of High-Grade Serous Ovarian Cancer by MALDI-Imaging. Cancers (Basel). 2021;13(7).
Lu H, Arshad M, Thornton A, Avesani G, Cunnea P, Curry E, et al. A mathematical-descriptor of tumor-mesoscopic-structure from computed-tomography images annotates prognostic- and molecular-phenotypes of epithelial ovarian cancer. Nat Commun. 2019;10(1):764.
Chen HZ, Wang XR, Zhao FM, Chen XJ, Li XS, Ning G, et al. The Development and Validation of a CT-Based Radiomics Nomogram to Preoperatively Predict Lymph Node Metastasis in High-Grade Serous Ovarian Cancer. Front Oncol. 2021;11:711648.
Fotopoulou C, Rockall A, Lu H, Lee P, Avesani G, Russo L, et al. Validation analysis of the novel imaging-based prognostic radiomic signature in patients undergoing primary surgery for advanced high-grade serous ovarian cancer (HGSOC). Br J Cancer. 2022;126(7):1047–54.
Veeraraghavan H, Vargas HA, Jimenez-Sanchez A, Micco M, Mema E, Lakhman Y, et al. Integrated Multi-Tumor Radio-Genomic Marker of Outcomes in Patients with High Serous Ovarian Carcinoma. Cancers (Basel). 2020;12(11).
Hong Y, Liu Z, Lin D, Peng J, Yuan Q, Zeng Y, et al. Development of a radiomic-clinical nomogram for prediction of survival in patients with serous ovarian cancer. Clin Radiol. 2022;77(5):352–9.
Hu J, Wang Z, Zuo R, Zheng C, Lu B, Cheng X, et al. Development of survival predictors for high-grade serous ovarian cancer based on stable radiomic features from computed tomography images. iScience. 2022;25(7):104628.
Pascual A, Guerriero S, Rams N, Juez L, Ajossa S, Graupera B, et al. Clinical and ultrasound features of benign, borderline, and malignant invasive mucinous ovarian tumors. Eur J Gynaecol Oncol. 2017;38(3):382–6.
Rosati A, Gueli Alletti S, Capozzi VA, Mirandola M, Vargiu V, Fedele C, et al. Role of ultrasound in the detection of recurrent ovarian cancer: a review of the literature. Gland Surg. 2020;9(4):1092–101.
Yang B, Zhong J, Zhong J, Ma L, Li A, Ji H, et al. Development and Validation of a Radiomics Nomogram Based on (18)F-Fluorodeoxyglucose Positron Emission Tomography/Computed Tomography and Clinicopathological Factors to Predict the Survival Outcomes of Patients With Non-Small Cell Lung Cancer. Front Oncol. 2020;10:1042.
Martínez-Más J, Bueno-Crespo A, Khazendar S, Remezal-Solano M, Martínez-Cendán JP, Jassim S, et al. Evaluation of machine learning methods with Fourier Transform features for classifying ovarian tumors based on ultrasound images. PLoS ONE. 2019;14(7):e0219388.
Forstner R, Thomassin-Naggara I, Cunha TM, Kinkel K, Masselli G, Kubik-Huch R, et al. ESUR recommendations for MR imaging of the sonographically indeterminate adnexal mass: an update. Eur Radiol. 2017;27(6):2248–57.
Zhang H, Mao Y, Chen X, Wu G, Liu X, Zhang P, et al. Magnetic resonance imaging radiomics in categorizing ovarian masses and predicting clinical outcome: a preliminary study. Eur Radiol. 2019;29(7):3358–71.
Song XL, Ren JL, Zhao D, Wang L, Ren H, Niu J. Radiomics derived from dynamic contrast-enhanced MRI pharmacokinetic protocol features: the value of precision diagnosis ovarian neoplasms. Eur Radiol. 2021;31(1):368–78.
Wang G, Sun Y, Chen Y, Gao Q, Peng D, Lin H, et al. Rapid identification of human ovarian cancer in second harmonic generation images using radiomics feature analyses and tree-based pipeline optimization tool. J Biophotonics. 2020;13(9):e202000050.
Wang X, Lu Z. Radiomics Analysis of PET and CT Components of (18)F-FDG PET/CT Imaging for Prediction of Progression-Free Survival in Advanced High-Grade Serous Ovarian Cancer. Front Oncol. 2021;11:638124.
Meier A, Veeraraghavan H, Nougaret S, Lakhman Y, Sosa R, Soslow RA, et al. Association between CT-texture-derived tumor heterogeneity, outcomes, and BRCA mutation status in patients with high-grade serous ovarian cancer. Abdom Radiol (NY). 2019;44(6):2040–7.
Rizzo S, Botta F, Raimondi S, Origgi D, Buscarino V, Colarieti A, et al. Radiomics of high-grade serous ovarian cancer: association between quantitative CT features, residual tumour and disease progression within 12 months. Eur Radiol. 2018;28(11):4849–59.
Vargas HA, Veeraraghavan H, Micco M, Nougaret S, Lakhman Y, Meier AA, et al. A novel representation of inter-site tumour heterogeneity from pre-treatment computed tomography textures classifies ovarian cancers by clinical outcome. Eur Radiol. 2017;27(9):3991–4001.
Zargari A, Du Y, Heidari M, Thai TC, Gunderson CC, Moore K, et al. Prediction of chemotherapy response in ovarian cancer patients using a new clustered quantitative image marker. Phys Med Biol. 2018;63(15):155020.
Wei W, Rong Y, Liu Z, Zhou B, Tang Z, Wang S, et al. Radiomics: a Novel CT-Based Method of Predicting Postoperative Recurrence in Ovarian Cancer. Annu Int Conf IEEE Eng Med Biol Soc. 2018;2018:4130–3.
Vamvakas A, Williams SC, Theodorou K, Kapsalaki E, Fountas K, Kappas C, et al. Imaging biomarker analysis of advanced multiparametric MRI for glioma grading. Phys Med. 2019;60:188–98.
Nakamoto T, Takahashi W, Haga A, Takahashi S, Kiryu S, Nawa K, et al. Prediction of malignant glioma grades using contrast-enhanced T1-weighted and T2-weighted magnetic resonance images based on a radiomic analysis. Sci Rep. 2019;9(1):19411.
Forstner R, Sala E, Kinkel K, Spencer JA. ESUR guidelines: ovarian cancer staging and follow-up. Eur Radiol. 2010;20(12):2773–80.
Engbersen MP, Van Driel W, Lambregts D, Lahaye M. The role of CT, PET-CT, and MRI in ovarian cancer. Br J Radiol. 2021;94(1125):20210117.
Shur JD, Doran SJ, Kumar S, Ap Dafydd D, Downey K, O'Connor JPB, et al. Radiomics in Oncology: A Practical Guide. Radiographics. 2021;41(6):1717–32.
Jensen LJ, Kim D, Elgeti T, Steffen IG, Hamm B, Nagel SN. Stability of Radiomic Features across Different Region of Interest Sizes-A CT and MR Phantom Study. Tomography. 2021;7(2):238–52.
Chen Z, Xu L, Zhang C, Huang C, Wang M, Feng Z, et al. CT Radiomics Model for Discriminating the Risk Stratification of Gastrointestinal Stromal Tumors: A Multi-Class Classification and Multi-Center Study. Front Oncol. 2021;11:654114.
Song J, Shi J, Dong D, Fang M, Zhong W, Wang K, et al. A New Approach to Predict Progression-free Survival in Stage IV EGFR-mutant NSCLC Patients with EGFR-TKI Therapy. Clin Cancer Res. 2018;24(15):3583–92.
Zwanenburg A, Vallières M, Abdalah MA, Aerts H, Andrearczyk V, Apte A, et al. The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping. Radiology. 2020;295(2):328–38.
Collewet G, Strzelecki M, Mariette F. Influence of MRI acquisition protocols and image intensity normalization methods on texture classification. Magn Reson Imaging. 2004;22(1):81–91.
Goodier J. Encyclopedia of Measurement and Statistics. Ref Reviews. 2007;21(8):40.
Bruce P, Bruce A. Practical statistics for data scientists: 50 essential concepts. 2017.
Šimundić AM. Measures of Diagnostic Accuracy: Basic Definitions. Ejifcc. 2009;19(4):203–11.
Takaya S, Marc R, Guy B. The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets. PLoS ONE. 2015;10(3):e0118432.

No competing interests reported.

Download PDF

Version 1

posted

You are reading this latest preprint version

CT-Based Radiomics and Machine Learning for Differentiating Benign, Borderline, and Early-stage Malignant Ovarian Tumors: A Multi-Class Classification and Multi-Model Comparation Study

Status:

Version 1

Abstract

Background

Results

Conclusion

Figures

1 Background

2 Results

2.1 Patient Clinical Factors

2.2 Radiomic Feature Extraction and Selection

2.3 Machine Learning Models Performance

3 Discussion

4 Conclusions

5 Materials And Methods

5.1. Patients

5.2 CT Image Acquisition and Tumors Segmentation

5.3 Image preprocessing and Radiomic Feature Extraction

5.4 Selection and Machine learning of Radiomic features

5.5 Statistics

Abbreviations

Declarations

References

Additional Declarations

Supplementary Files

Status:

Version 1