Development and Validation of a Simple-to-Use Nomogram to Predict 3-Year Recurrence rate of Ovarian Cancer

Objective: To establish a reliable nomogram model to predict the recurrence rate of ovarian cancer after surgery. Methods: We retrospectively reviewed 216 patients diagnosed with ovarian cancer in our hospital, of which 164 cases were considered valid. Logistic regression model was used to analyze the possible predictors. After that, a nomogram model based on those signicantly related predictors was established. We used the bootstrap to internally validate the predictive ability of the nomogram model and used the decision curve analysis (DCA) to compare the performance of the FIGO stage with this model. Results: The nomogram included seven signicant recurrence predictors: age, histology type, FIGO stage, omentum involvement, lymphovascular space invasion (LVSI), liver metastasis, and serum CA125. The measurement values for accuracy were Brier score 0.131, correction slope 1.00, and c-index 0.870. which demonstrated this model had a good predictive ability. Compared with the FIGO stage, this hybrid model is more superior in predicting recurrence risk in ovarian cancer patients. Conclusions: We developed and validated a non-invasion and user-friendly nomogram model to predict the recurrence risk of patients with ovarian cancer after surgery.


Introduction
Ovarian cancer is the most fatal gynecological cancer, which is the fth leading cause of cancer death in women accounting for about 14,000 deaths per year [1]. More than 75% of women were diagnosed at advanced stage. After diagnosis, these patients rst underwent surgical removal of all visible intraperitoneal disease (if feasible), followed by six cycles of platinum-based chemotherapy [2]. Despite the initial remission rate is good, about 75% of these women experience relapse within the rst 20 months [3].
Nowadays, postoperative evaluation of ovarian cancer patients mainly consider histological factors, such as CA-125, including computed tomography imaging and physical examination [11][12][13]. However, previous studies have indicated that the current follow-up model may not be su cient to assess the risk strati cation, especially for those patients who may have residual invisible disease after treatment [4,5]. Meanwhile, these monitoring tests cannot provide constructive information for gynecologic oncologists to determine which patients are at highest risk for recurrence [6]. Due to the defect in previous mentioned methods in prediction of the recurrence risk, gynecologist often need to consider additional clinical data to decide whether or not to proceed further intervention for patients undergoing surgery.
Many models have been reported to predict the overall survial [7][8][9], however, the researches about the establish of recurrence model were rarely mentioned [10]. Development of a tool to predict those patients who underwent remission after surgery but still at high risk of recurrence is indispensable. Nomogram, an important decision-making component of modern medicine, is widely used as a tool to estimate prognosis in medicine and oncology. By integrating different prognostic and deterministic variables to generate individual numerical probabilities of clinical events, nomogram satis es our desire for integrated biological and clinical models and our pursuit of personalized medicine.
In this study, we aimed to establish a nomogram model to predict a 3-years recurrence rate of ovarian cancer after surgery, which can provide effective information for clinicians to individually deal with patients.

Patients selection
We retrospectively analyzed 216 patients treated in our hospital for ovarian cancer from January 2014 to June 2018. The inclusion criteria were the following: 1) patients were diagnosed with ovarian cancer and only experienced one operation for this tumor, 2) patients received tissue biopsy during the surgery, which con rmed lymphatic metastasis and omentum involvement, 3) patients had no history of other cancers, With these inclusion criteria, only 164 patients were adopted in our study.

Data collection
We collected age as demographic and comorbidity data. Pathology-related data, including histological type, FIGO grade, lymph vascular space invasion (LSVI), liver metastasis, omentum involvement, and serum CA125 were taken into consideration as well. This study was undertaken with the ethical approval of the Human Ethics Committee of Jinan University, which was in accord with the Declaration of Helsinki.

Recurrence
In this study, we referred to recurrence according to radiologic results or tissue biopsy within 3 years after surgery [14].

Statistical analysis
All statistical analyses were performed using SPSS version 21 and R package Regression Modeling Strategies. We compared the continuous variables and categorical variables of patients with and without postoperative recurrence by t-test and c 2 test, respectively. Univariate analysis was performed to identify variables associated with recurrence. Multivariate logistics regression was used to construct a risk model that included variables of statistical signi cance(P < 0.05) and clinically important criteria as predictors.
Before performing nomogram, it is essential to verify the predictive ability of the model. The prediction accuracy of model was commonly assessed using 3 measures: 1) c-index for discrimination, 2) Brier score for overall performance, and 3) calibration slope for calibration. We used the bootstrapping method of 1000 repetitions to study the internal veri cation of the model, and obtained the deviation correction prediction. Furthermore, decision curve analysis (DCA) is a new method for evaluating clinical prediction models to evaluate the impact of clinical bene ts and visualize these impacts. Finally, in addition to this digital measurements, we also used calibration graphs, receiver operating characteristic (ROC) curves, and DCA to show the calibration, identi cation, and clinical bene ts of the nal model.
A nomogram was developed to visualize the graphical representation of the nal model. There is a guide line at the top of the nomogram, which shows the score of each predictor from 0 to 100. The size of the predictors shows their effect size, visually shows the relative weight of each variable, and allows points to be assigned to each important clinical feature. The integral of each predictor and the correponding result of predicting the probability of postoperative recurrence can be read from the bottom 2 rows.

General information
After application of selection criteria, a total of 164 cases were enrolled in analysis: the total recurrence rate of ovarian cancer was 37.1% (61/164) within 3 years.A summary of demographic factors and clinical parameters of patients in this study are shown in Table 1., 92% of cases were epithelial ovarian cancer. According to previous studies, we divided our patients into two groups, which were epithelial ovarian cancer and non-epithelial ovarian cancer respectively. The mean age and pre-surgery serum CA125 was 49.7 years old and 735.3 U/L, and the majority of patients were epithelial ovarian (92.1%) , had an apparent FIGO stage I tumor (68.9%) , didn't have LSVI (68.9%), liver metastasis (53.7%), and omentum involvement (58.5%), with a statistical signi cance. Recurrent ovarian cancer patient had higher rates of epithelial ovarian cancer, more advanced FIGO stage, higher level of serum CA125, and presence of LVSI, liver metastasis, and omentum involvement. (all P < 0.05) The model was veri ed with 1000 replicates of bootstrapping, and the bias correction measurement values for accuracy were Brier score 0.131, correction slope 1.00, and c-index 0.870. The DCA curve revealed that compared with FIGO models, the hybrid model was more superior in predicting the recurrent rate of patients. And the calibration plot , ROC curves, and DCA curve were also plotted for graphical evaluation of calibration and discrimination, as shown in Figure 1.
C-index tests the discrimination ability of the model, or the ability to distinguish a woman who gives birth prematurely from a woman who gives birth normally. The value ranges from 0.5 to 1, and the closer the value is to 1, the stronger the discriminant ability is. Correction slope tests the consistency between the predicted value and the result with a perfect slope equal to 1. The Brier score is a measure of overall performance that covers both calibration and discrimination. It represents the difference between the predicted probability and the actual outcome. The score ranged from 0 to 1, and the closer the value was to 0, the better the prediction ability. In general, the values obtained by our measurements show a fairly good prediction accuracy.

LVSI: lymphovascular invasion
Creation and use of the nomogram A nomogram diagram is constructed based on the results of the logistic regression model and shows the predictive variables and corresponding point scales, as presented in Figure 2. The steps to use a nomogram are: 1) identify the status of each predictor for pregnant women, 2) draw a straight line from each predicted state upwards to the partial point reference, 3) sum the points corresponding to each predicted state, 4) locate the sum on the total points reference line, and 5) draw a line from the total point line to the bottom risk line to nd the probability of recurrence after surgery.
The use of the nomogram can be illustrated by a clinical example. (Figure.3) In the example, we calculate the predicted probability of recurrence for a 50-year-old patient with omentum involvement, preoperative serum CA125 level of 300U/L, and FIGO stage I epithelial ovarian cancer. Points are assigned for each feature: 22.5 for 50 years old, 48 for epithelial ovarian cancer histology, 65 for omentum involvement, 78 for preoperative serum CA125 level of 300U/L. The total of 223.5 points corresponds to a nearly 55% chance of recurrence for this patient.

Discussion
In the present study, we successfully developed and internally validated a nomogram to predict the recurrence of ovarian cancer patients after surgery.We found our model had good predictive accuracy. This tool was developed using age, histological type, FIGO grade, serum CA125 level, and presence of LVSI, liver metastasis, omentum involvement. These factors have been shown to have perfect predictive signi cance individually [14][15][16], but not all of them have been assembled to establish an usable clinical tool to predict recurrence risk. Several models and nomograms have been developed to predict surgical outcome, progression-free survival or overall survival [17][18][19], but rare models predict the risk of postoperative recurrence [20,21]. Hu et al [20] created a nomogram using FIGO stage, histological grade, histological type, lymph node metastasis status and serum CA125 level at diagnosis, that model was predominantly driven by stage and has a limited capacity to account for the clinical heterogeneity of epithelial ovarian cancer. Our model included more histological type of ovarian cancer, meanwhile serum CA125 was the main driver, which adds to the ability to explain clinical heterogeneity.
The undelying recurent mechanism of ovarian cancer is complex. Although FIGO stage is an essential factor in predicting recurrence, FIGO stage alone cannot accurately predict of postoperative recurrence risk in ovarian cancers patients with different stage, especially those with lower FIGO stage. Patients who did not underwent chemotherapy after surgery had increased risk of recurrence compared with patietns experieced chemotherapy. some patients choose to routinely administer therapy while others do not. Our nomogram can differentiate those patient who are susceptible to recurrence. According to our results, early-stage patients with spontaneous ruputred lesion and high CA125 are more vulnerable to recurrence. The recurrence rate is as high as 55%. Therefore, it is highly recommended for these patients to choose chemotherapy. In contrast, we recommend the stage II patients with non-epithelial ovarian cancer and low CA125 to reduce the cycle of chemotherapy and adjust the regime, because they are less likely to suffer recurrence. Our model can improve patients' subsequent treatment options after surgery by identifying those patients who most likely to bene t and reducing extra costs., which spare those patients with low recurrence risk to experience chemotherapy.
Our study has some limitations that merit discussion. First of all, this study is a single-center retrospective study. In addition, the main predictive factor CA125 is based on preoperative screening of patients with operable ovarian cancer, and its applicability to patients with interval debulking surgery has not been demonstrated. Finally, our model has not been externally validated. These limitations are balanced by the utility of the nomogram and the strengths of our study. We use bootstrapping, which has been shown to provide the best and smallest deviation estimates to internally validate our clinical model, which provides good predictive power for optimally adjusted estimates. We included potentially available and clinically relevant variables, such as FIGO grade, LVSI, preoperative CA125, which can be used individually. This easy to use tool can provide additional clinical information when FIGO can not provide accurate information to predict the recurrence. The probability of postoperative recurrence from the nomogram can be combined with other clinical data, which help for shared decision making or providing patients more choices to improve the quality of life.
We developed and validated a non-invasion and user-friendly nomogram model to predict the recurrence risk of patients with ovarian cancer after surgery, which can assist in preoperative diagnosis and treatment. And we found that patients with epithelial ovarian cancer with more advanced FIGO staging, higher serum CA125 levels, LVSI, liver metastases, and greater omentum involvement are more likely to relapse after surgery.   A nomogram for predicting the likelihood of recurrence in ovarian cancer patients who recieved surgery.
To use the nomogram, the value for each predictor is determined by drawing a line upward to the point reference line, the points are summed, and a line is drawn downward from the total points line to nd the predicted probability of recurrence.