Development and Validation of Nomograms to Predict Long-term Oncological Outcomes in Serous Ovarian Cancer

Abstract

Background: Nomograms are statistics-based predictive tools that integrate predictive factors. We developed and validated a nomogram to predict overall survival (OS) in serous ovarian cancer (SOC).

Methods: In total, 6957 patients from the SEER database were included in the training group; the external validation group included 1244 SOC patients from two Chinese hospitals. The nomogram was structured on Cox regression analyses and was evaluated in both the training and validation groups using consistency index, area under the receiver operating characteristics curve (AUC), calibration plots, and risk subgroup classification. Kaplan–Meier curves were plotted to compare survival outcomes between subgroups. A decision-curve analysis was used to test the clinical value of the nomogram.

Results: The independent factors identified by multivariate analysis in the training cohort and selected for the nomogram included age, tumor grade, and FIGO stage. The consistency indexes for OS were 0.689 (95% confidence interval: 0.677–0.701) in the training cohort and 0.639 (95% confidence interval: 0.601–0.670) in the validation cohort; the AUCs were 0.675 and 0.661 in the validation cohorts, respectively. Calibration curves showed good consistency between predicted and actual 3- and 5-year OS. Significant differences were observed in the survival curves of different risk subgroups. The decision-curve analysis indicated our nomogram was superior to the AJCC staging system.

Conclusion: We constructed a nomogram to predict long-term OS in SOC and externally verified it in an Asian population. This nomogram showed more accurate survival predictions, which will help provide personalized treatments and follow-up strategies.

Introduction

Epithelial ovarian cancer (EOC) is one of the most common malignant tumors of the female genital organs. Although the prognosis of early-stage ovarian cancer (stage I and II) is favorable, approximately 70% of newly diagnosed cases are in advanced stages (stage III and IV), for which the 5-year overall survival (OS) rate is only 29%. The 5-year OS rate is 92% for early-stage disease. EOC is generally treated with cytoreductive surgery and platinum-based combination chemotherapy. Although treatment has greatly improved, the incidence rates and mortality rates have been increasing year by year [1]. As a subtype of EOC, serous ovarian cancer (SOC) presents a distinct biological profile from other histological types. In 2002, Singer proposed a two-level classification system for SOC that divided patients into two subtypes: low-grade ovarian serous cancer (LGOS) and high-grade ovarian serous cancer (HGOS) [2]. In 2004, the pathologists Kurman and Shih, put forward the binary theory of EOC based on an analysis of clinical data, pathomorphology, and molecular genetics from a large number of EOC patients [3]. According to the binary theory of ovarian cancer, these malignancies can be divided into two types: type I and type II. In SOC, the binary theory of low- and high-grade disease is widely accepted because of the obvious differences in tumorigenesis, malignant biological behaviors, and molecular characteristics.

HGOS is type II ovarian cancer and the most common type of ovarian cancer, accounting for approximately 90% of SOCs. There are no precancerous lesions; thus, it is often found with extensive pelvic spread and high malignancy[4]. LGOS is type I ovarian cancer, and accounts for approximately 10% of ovarian serous adenocarcinomas. These tumors grow slowly and are not sensitive to chemotherapy, but the clinical diagnosis is usually early and the prognosis is good.

In the process of clinical diagnosis and treatment, gynecological oncologists are often asked by patients: “If I have surgery and chemotherapy, how long will I live?” For a doctor, having the ability to predict the probability of a certain outcomes could change medical practice models and clinical decisions. As a tool for risk and benefit assessments, clinical prediction models can provide more objective and accurate information for doctors and patients, so they are becoming more widely used. Nomograms are statistics-based predictive tools that integrate pivotal predictive factors and have been widely used to quantify risks and evaluate the prognosis of many cancer types [4–6]. The use of nomograms has been compared to the traditional staging systems for many cancers; thus, it has been proposed as a new standard. However, to the best of our knowledge, no nomograms for patients with SOC have been developed. In this study, we aimed to construct nomograms using data extracted from the Surveillance, Epidemiology, and End Results (SEER)[7] database to predict the prognosis of patients with SOC. Then the prediction model was verified externally to determine whether it provides more-accurate predictions of patient survival compared with the currently available staging systems.

Materials And Methods

Results

Patient characteristics

Detailed patient characteristics are shown in Table 1. We identified 8201 women with SOC. As shown in Table 1, the majority of cases (> 85%) were diagnosed in women aged 50 years and older. More than half of the patients (84.4%) had HGOS, of which the majority (77.8%) were at advanced stages. Characteristics of patients in the training (n = 6957) and validation cohorts (n = 1124) are listed in Table 1.

Independent prognostic factors in the training cohort

Univariate analysis of the training cohort showed that age, grade, and AJCC stage were significant risk factors for OS (p<0.05) (Table 2). The independent prognostic factors in the final model were identified by multivariate analyses. In the multivariate analysis of these three factors, age, grade, and AJCC stage remained independent factors significantly associated with OS (Table 2).

Constructing nomograms for OS

Prognostic nomograms for predicting 3- and 5-year OS were constructed independently on the basis of the prognostic variables from the training cohort (Figure 1). The complex Cox regression formulas were transformed into visual graphics. The line segment corresponding to each variable was marked with a scale that represented the value range of the variable, and the length of the line segment reflected the contribution of the factor to the outcome event. The nomograms demonstrated that AJCC stage contributed most to OS for SOC patients. Nomograms are commonly used to estimate prognosis in oncology. Each variable was assigned a score ranging from 0 to 100. To easily use the nomograms, one can draw a vertical line upward from the specific points of each predictors to the ‘point’ lines, and then add up the corresponding points [12]. Then, drawing a straight line from the total points axis to the 3- and 5-year survival axis, surgeons can get the probability of 3- and 5-year survival.

For example, if a patient was 55-years-old (18-score), diagnosed with HGOS (20-score), had undergone satisfactory cytoreductive surgery, with AJCC stage III disease (65-score), and received chemotherapy after surgery, the total score of the patient would be 103. Thus, the probability of survival for 3 + years was 73%, and the probability of survival for 5 + years was 53%. This visual scoring system of nomograms can help doctors answer the question: “how long can I live?” This calculated value aims to help clinicians make rapid predictions when counseling patients.

Validation of the nomograms

The nomograms were then externally validated. We used the c-statistic, ROC curves, and calibration curves to validate the model. These are indexes used to evaluate the accuracy of prediction models [13].

Values of the C-statistic

The C-statistic ranged from 0.5, which indicates absence of discrimination, to 1.0, which indicates perfect discrimination. Generally speaking, if the C-statistic value is > 0.6, the model has good predictive value.

In the training cohort, the C-statistic value for predicting OS was 0.689 (95% CI 0.677–0.701). In the validation cohort, the C-statistic value for OS was 0.639 (95% CI: 0.601–0.670). Thus, the c-statistic values of the training and validation sets were very similar.

ROC curve analyses

The nomograms were then externally validated. In the training cohort, the ROC curve for OS was 0.675; in the validation cohort, the ROC curve for OS was 0.661 (Fig. 2). ROC curves ranged from 0.5, which indicates absence of discrimination, to 1.0, which indicates perfect discrimination. Generally speaking, if the ROC curve was > 0.6, the model had a medium prediction value. The ROC curves of the training and validation sets were very similar (Fig. 2).

Calibration curves

The calibration curves indicated excellent agreement between the nomogram predictions and the actual survival outcomes in the training and validation cohorts. The x-axis represents the nomogram-predicted probability of the 3- and 5-year OS rates of each patient, while the y-axis represents the actual 3- and 5- year survival rates of each patient. If the blue line completely coincided with the black dashed line, this would indicate that the model was the most ideal. The agreement between the predicted versus observed probabilities are indicated by the calibration curves for 3- and 5-year OS (Figure 3).

Risk stratification of SOC patients and Decision-curve analysis (DCA)

A risk score of each variable was generated from the nomogram, and total scores were calculated for all patients. The whole cohort was divided into low- and high-risk subgroups based on the median risk score. According to the survival curves shown in Figure 4A, significant differences were observed between the low- and high-risk groups for OS (P<0.001), which implied that the nomogram had better risk stratification ability.

DCA indicated that the nomogram models made better predictions and outperformed the AJCC staging system (Figure 4 B).

Discussion

In the field of medical research, tumor risk prediction models are used to predict the future incidence and prognosis of certain tumors. Specifically, nomograms represent a mean to establish a statistical model of the quantitative relationship between multiple risk factors and tumor occurrence and/or prognosis. The purpose of such models include: informing patients of the risk of onset or prognosis, screening high-risk groups, and helping doctors make clinical decisions. In 2003, van Zee et al. [14] first proposed a nomogram model to predict the risk of non-sentinel lymph node metastasis in sentinel node-positive breast cancer. This nomogram was based on a regression model to intuitively present the probability of outcomes. The advantage is that it can provide better individualized prognostic risk assessments in the form of intuitive graphics, which have definite value in clinical practice and can provide a reference for individualized clinical decision-making. Thus, such predictive nomograms are widely used in clinical oncology [15–17]. All of the nomograms presented showed better discriminatory capacity than did classical staging systems. However, a lack of external verification was the common limitation for these studies. External validation of cohorts from other countries or prospective randomized clinical trials are necessary to confirm a model’s performance.

A subtype of EOC, SOC accounts for approximately 85% of EOC diagnoses. Therefore, it is important to separately analyze the performance of nomograms with regards to their prognostic predictions for this EOC subtype. In this study, we used the external verification method. We analyzed a training cohort of 6957 SOC patients from the SEER database and a validation cohort of 1244 SOC patients from two tertiary institutional hospitals to develop and validate easy-to-use nomograms for predicting the OS at 3- and 5-years. Our study identified age, grade, and AJCC stage as independent predictors of OS. We observed that the important predictors of improved OS were younger age, early clinical stage, and well-differentiated grade, which is consistent with previous studies [18]. In this study, the majority of cases (> 85%) were diagnosed in women over 50-years-old; the older the patient was, the worse the prognosis was. Generally, older patients were more likely to present worse survival outcomes due to lower immune responses [19].

Many scholars believe that clinical staging is an important factor that affects the prognosis of ovarian cancer. In our study, > 70% of patients were diagnosed with advanced (III-IV) ovarian cancer. The later the clinical stage, the lower the 3- and 5-year OS rates. Patients with early FIGO staging could be more thoroughly removed after surgery, as the residual lesions were relatively small, chemotherapy-sensitive, and had a low risk of recurrence and metastasis; thus, the prognosis of these patients was good. Patients with late FIGO stage disease have tumor cells in the body that spread more widely, making it difficult to implement complete surgical treatment. Patients with a poor tolerance to chemotherapy have poor prognoses. Especially after stage IIIA, the later the stage, the higher the risk of death and the worse the prognosis [20].

HGOS is the most common subtype of EOC, and a majority of HGOS patients subsequently develop platinum-resistance with relapse; which demonstrates their overall poor prognosis [21]. In this study, 84.4% of patients were HGOS. As can be seen from the nomogram, the prognosis of these patients was worse than that of LGOS. It is generally believed that tumors with low histological grades have high degrees of malignancy, such as rapid disease progression and poor survival and prognosis due to adverse biological behaviors (rapid cell proliferation, diffusion, and strong invasion. The higher the degree of tissue differentiation, the slower the proliferation of tumor cells in the body. Weaker invasion of body tissues was associated with lower degrees of malignancy, slower disease progression, relative sensitivity to chemotherapy, longer survival times, and better prognoses.

We validated the accuracy of our nomograms using the C-index, ROC analysis, and calibration curves in both the training and validation cohorts. The C-index and ROC curves all exceeded 0.6 for OS in the external verification processes. Calibration curves also demonstrated good performance of the nomograms. These results show that our final nomogram exhibited good discriminatory performance and calibration. Previous studies have reported using the SEER database to establish nomograms to predict the prognosis of EOC. For example, Wang et al. developed and internally validated nomograms to predict OS and cancer-specific survival (CSS) of EOC patients, achieving a C-index of 0.733 for OS and 0.747 for CSS [22]. Chen et al. constructed and validated nomograms to predict OS and CSS in patients with ovarian clear cell carcinoma, achieving a C-index of 0.802 (95% CI, 0.773–0.831) for OS and 0.802 (95% CI, 0.769–0.835) for CSS. The nomograms were validated and found to be of satisfactory predictive value, which could aid in future clinical decision making [23].

The traditional AJCC staging system does not appear to be able to accurately assess tumor prognosis because it only includes a limited subset of important prognostic factors. Several studies have reported better performance of nomograms than conventional staging systems and have proposed such models as promising tools for evaluating prognosis. Shi et al. generated nomograms to predict OS and CSS in patients with gastric cardia cancer, achieving a C-index of 0.714 (95% CI, 0.705–0.723) for OS and 0.759 (95% CI, 0.746–0.772) for CSS. This study showed that these nomograms gave better predictions than nomograms based on TNM stages [24]. Xu et al. developed and validated a nomogram based on log of odds between the number of positive lymph nodes and the number of negative lymph nodes to predict OS and CSS for EOC patients, achieving a C-index of 0.757 (95% CI, 0.746–0.768) for OS and 0.770 (95% CI, 0.759–0.782) for CSS. Additionally, this model performed favorably compared with the currently used Federation of Gynecology and Obstetrics (FIGO) model, with concordance indices of 0.699 (95% CI, 0.688–0.710; P < 0.05) and 0.719 (95% CI, 0.709–0.730; P < 0.05) for OS and CSS, respectively. This study also suggested that log of odds between the number of positive lymph nodes and the number of negative lymph nodes works as an independent prognostic factor for survival in EOC patients regardless of tumor stage; thus, the nomogram may be superior to the currently used FIGO staging system for predicting OS and CSS among post-operative EOC patients [25].The nomogram developed in our study also showed better predictive accuracy for survival compared with the AJCC 7th staging system. This nomogram model enabled risk stratification of patients; thus facilitating personalized treatment plans and follow-up schedules.

Our study had several advantages. First, we developed and validated nomograms using clinically important long-term oncological OS outcomes, which reduced bias by scoring the model performance using high-quality data based on the large sample sizes from SEER. Second, although some studies have established nomograms to predict the prognosis of SOC, but the nomogram model of these studies only received internal validation. In contrast, this study adopted the method of external verification, using cohorts from other countries, which was necessary to confirm performance. Third, compared with the AJCC-stages, DCA curves in this study showed that our nomograms provided excellent clinical utility. All variables included in the SOC nomogram could be obtained easily, which could facilitate the application of nomograms in clinical practice. Furthermore, we aimed to establish a prediction model for general SOC patients with common characteristics, such that results were not affected by ethnic and regional differences. Although the SEER database contains information from the US population, our external validation population was from China. The C-statistic and AUC values of the training and validation sets were very similar, demonstrating the great discriminatory power of the nomogram. It also showed that this nomogram was applicable to regions other than the United States, which will facilitate the application of nomograms in clinical practice.

However, there were also several limitations to this study. First, the included variables were relatively simple and there were more detailed data that could have been included, such as family history of ovarian cancer, primary tumor diameter, positive lymph nodes, ascites cytological results, location of metastasis, chemotherapy regimens and cycles, sensitivity to chemotherapy, and genetic results. Second, this study was constructed using retrospective data, among which there may have been some undetected potential factors that introduced bias.

Conclusion

In conclusion, we used the SEER database to construct a nomogram of SOC prognosis by integrating a variety of prognostic factors into a simple and intuitive tool, and then externally verifying the model in an Asian population to obtain unbiased estimates that are universally applicable. Compared with the AJCC staging system, our nomogram improved the accuracy of survival predictions, which could help clinicians provide personalized treatment suggestions and follow-up strategies. In our next study, we will include more detailed variables into the construction of predictive models of SOC prognoses, which will make them more complete and accurate. Thus, we will collect more information to construct a predictive model of chemotherapy resistance.

Declarations

Ethics approval and consent to participate

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

This article does not contain any studies with animals performed by any of the authors.

The information was accessed from the online database, Informed consent was obtained from all individual participants included in the study.

Consent for publication

Not applicable.

Availability of data and materials

• The datasets generated and/or analysed during the current study are available in the SEER database repository.

• The external validation datasets used and analysed during the current study are available from the corresponding author on reasonable request.

Competing interests

The authors declare that they have no competing interests

Funding

This study was not funded .

Author contributions

LYY: Methodology; Validation; Writing - Original Draft; Writing - Review & Editing;

ZRF: Investigation; Resources; Data Curation

LC: Software; Formal analysis;

ZJ: Software; Formal analysis;

YL: Resources; Data Curation;

LL: Conceptualization; Supervision; Project administration.

All authors read and approved the final manuscript.

Acknowledgements

The authors wish to thank the following participants in this consensus (in alphabetical order): Li Li, Liu Chao, Yang Liu ,Zhao Ren-feng, Zhou Jie.

The author has obtained confirmation permission from all persons mentioned in the confirmation section.

References

1. Peres LC, Sinha S, Townsend MK et al. Predictors of survival trajectories among women with epithelial ovarian cancer. Gynecol. Oncol. 156(2), 459–466. doi: 10.1016/j.ygyno.2019.12.011.
2. Singer G,Kurman RJ,Chang HW,et al.Diverse tumorigenic pathways in ovarian serous carcinoma[J].The American Journal of  Pathology,2002,160:1223-1228.DOI: 10.1016/s0002-9440(10)62549-7.
3. Shih Ie M,Kurman RJ.Ovarian tumorigenesis: a proposed model based on morphological and molecular genetic analysis[J].The American Journal of Pathology,2004,164:1511-1518.doi: 10.1016/s0002-9440(10)63708-x.
4. Kurman RJ,Shih IM.Pathogenesis of ovarian cancer: lessons from morphology and molecular biology and their clinical implications.International journal of gynecological pathology:official journal of the international[J].Society of Gynecological Pathologists,2008,27:151-160.doi: 10.1097/PGP.0b013e318161e4f5.
5. Pan X, Yang W,Chen Y, Tong L,Li C,,Li H. Nomogram for predicting the overall survival of patients with inflammatory breast cancer: a SEER-based study. Breast. 2019;47:56–61. doi: 10.1016/j.breast.2019.05.015.
6. Hu JQ, Yu PC, Shi X, Liu WL,Zhang TT,Lei BW, et al.Prognostic Nomograms for predicting overall survival and Cancer-specific survival of patients with major salivary gland Mucoepidermoid carcinoma. J Cancer. 2019;10:4380–8. doi: 10.7150/jca.27992. eCollection 2019.
7. Surveillance, Epidemiology, and End Results (SEER) Program:Incidence- SEER18 Regs Custom Data (with additional treatment fields), Nov 2018 Sub(1975–2016 varying) National Cancer Institute, DCCPS, Surveillance Research Program, released April 2019, based on the November 2018 submission.https://seer.cancer.gov/.
8. Peres LC, Cushing-Haugen KL, Köbel M et al. Invasive epithelial ovarian cancer survival by histotype and disease stage. J. Natl Cancer Inst. 111(1), 60–68. doi: 10.1093/jnci/djy071.
9. Royston P, Moons KG, Altman DG, Vergouwe Y. Prognosis and prognostic research: developing a prognostic model. BMJ 338-604. doi: 10.1136/bmj.b604.
10. Huitzil-Melendez FD, Capanu M, O’Reilly EM,et al.Advanced hepatocellular carcinoma:Which staging systems best predict prognosis? J Clin Oncol 28:2889-2895. doi: 10.1200/JCO.2009.25.9895.
11. Yizhou Wang,Jun Li,Yong Xia,,et al.Prognostic Nomogram for Intrahepatic Cholangiocarcinoma After Partial Hepatectomy.J Clin Oncol 31:1188-1195.DOI: 10.1200/JCO.2012.41.5984.
12. Balachandran VP,Gonen M,Smith JJ,et al.Nomograms in oncology:more than meets the eye[J].Lancet Oncol,2015,16(4) :173-180.doi: 10.1016/S1470-2045(14)71116-7.
13. van ZEE KJ,MANASSEH DM,BEVILACQUA JL,et al.A nomogram for predicting the likelihood of additional nodal metastases in breast cancer patients with a positive sentinel node biopsy[J].Ann Surg Oncol,2003,10(10) :1140-1151.doi: 10.1245/aso.2003.03.015.
14. Moreau JT,Hankinson TC,Baillet S,Dudley RWR.Individual-patient prediction of meningioma malignancy and survival using the Surveillance, Epidemiology, and End Results database. NPJ Digit Med.2020 Jan 30;3:12.doi: 10.1038/s41746-020-0219-5.
15. Yan H,Wei X, Wu A,et al.Nomograms for predicting overall and cancer-specific survival in patients with papillary renal cell carcinoma:a population-based study using SEER database.Transl Androl Urol.2020 Jun;9(3):1146-1158.doi: 10.21037/tau-19-807.
16. Hao Zi,Lei Gao,Zhaohua Yu,et al.Nomograms for predicting long‑term overall survival and cancer‑specifc survival in patients with primary urethral carcinoma:a population‑based study.Int Urol Nephrol.2020 Feb;52(2):287-300.doi: 10.1007/s11255-019-02314-y.
17. Zhang W,Zhang W,Lin Z et al.Survival outcomes of patients with primary breast cancer following primary ovarian cancer.Med.Sci.Monit.2019 May 24;25:3869-3879.doi: 10.12659/MSM.914163.
18. Pawelec G.Does patient age influence anti-cancer immunity? Semin Immunopathol. 2019 Jan;41(1):125-131. doi: 10.1007/s00281-018-0697-6. Epub 2018 Jul 13.
19. Li Ge,Ning Li,Guang-Wen Yuan,et al.Nedaplatin and paclitaxel compared with carboplatin and paclitaxel for patients with platinum-sensitive recurrent ovarian cancer[J].Am J Cancer Res.2018,8(6):1074-1082. PMID: 30034944 PMCID: PMC6048404.
20. Van ZB, Tang D, Bowden NA. Biomarkers of platinum resistance in ovarian cancer:what can we use to improve treatment. Endocr Relat Cancer.2018;25(5):303–18.doi: 10.1530/ERC-17-0336.
21. Wang R, Xie G,et al.Development and validation of nomograms for epithelial ovarian cancer:a SEER population-based,real-world study. Future Oncol. 2021 Mar;17(8):893-906. doi: 10.2217/fon-2020-0531.
22. Chen Q,Wang S, Lang JH.Development and validation of Nomograms for predicting overall survival and Cancer-specific survival in patients with ovarian clear cell carcinoma.J Ovarian Res. 2020 Oct 17;13(1):123.doi: 10.1186/s13048-020-00727-3.
23. Shi X,Xu L,Ma B, Wang S.Development and validation of a nomogram to predict the prognosis of patients with gastric cardia cancer. Sci Rep. 2020 Aug 24;10(1):14143.doi: 10.1038/s41598-020-71146-z.
24. Xu XL, Cheng H,Tang MS,Zhang HL,et al. A novel nomogram based on LODDS to predict the prognosis of epithelial ovarian cancer.Oncotarget. 2017 Jan 31;8(5):8120-8130.doi: 10.18632/oncotarget.14100.

Tables

Table 1. Demographics and Clinicopathologic Characteristics of Patients With SOC

Abbreviations: LGOS: Low-grade ovarian serous cancer; HGOS: High-grade ovarian serous cancer; AJCC: American Joint Committee on Cancer staging system (7th edition);.

Table 2. Univariate and multivariate Cox regression analysis for OS in the training cohort.

Abbreviations: LGOS: Low-grade ovarian serous cancer; HGOS: High-grade ovarian serous cancer; AJCC: American Joint Committee on Cancer staging system (7th edition); HR: Hazard ratio.