Development and Validation of a Nomogram for Predicting Overall Survival in Patients with Second Primary Small Cell Lung Cancer After Non-Small Cell Lung Cancer: A SEER-Based Study

DOI: https://doi.org/10.21203/rs.3.rs-845089/v1

Abstract

Background: Non-small cell lung cancer (NSCLC) survivors are at an increased risk of developing second primary malignancies, such as small cell lung cancer. This paper sought to establish a prognostic nomogram to assess overall survival (OS) in patients with second primary small cell lung cancer (SPSCLC) after NSCLC.

Methods: 420 patients who developed SPSCLC after NSCLC were randomly split into the training and validation groups. A nomogram was established by stepwise regression. Area under the curve (AUC) and calibration plots were applied to assess the prognostic performance of the nomogram. Concordance index (C-index), integrated discrimination improvement (IDI), net reclassification index (NRI) and decision curve analysis (DCA) were performed to compare the nomogram with the American Joint Committee on Cancer (AJCC) 8th staging system. Survival risk classification was constructed based on the nomogram.

Results: Five variables were chosen to construct the nomogram. The AUC showed that it had a satisfactory discrimination ability. All calibration plots displayed good concordance between nomogram and observation. The C-index, IDI, NRI and DCA showed the nomogram was superior to the AJCC 8th staging system. The Kaplan-Meier curves suggested huge differences in prognosis among the three risk groups.

Conclusions: This study build a nomogram and risk stratification system for predicting probabilities of OS in patients with SPSCLC after NSCLC, which can help clinicians in individualized survival assessment and treatment decisions.

1. Introduction

Lung cancer is a common and deadly cancer, which contains two major histological classes, non-small cell lung cancer (NSCLC) and small cell lung cancer (SCLC)[1]. Notably, individuals suffering from NSCLC account for nearly 85% of all lung cancer patients [2]. Unfortunately, survivors with NSCLC are at high risk of the development of second primary malignancies including second primary small cell lung cancer (SPSCLC) [3, 4].

The American Joint Committee on Cancer (AJCC) is the most widely applied staging system for lung cancer. However, it is mainly used for the initial primary tumor and is relatively poorly in predicting the risk of individual survival [5, 6]. Additionally, the AJCC staging system only considers the size and extension of the tumor, lymph node involvement and distant metastasis but does not take into account other prognostic factors such as demographic characteristics, histological type and therapeutic measures [7]. Therefore, a better prediction model is needed to evaluate the prognosis of SPSCLC.

Nomogram is a useful tool that can quantify and predict the occurrence of a certain clinical event in an individual patient, so as to help clinicians in risk stratification and clinical decision-making [8]. Consequently, this paper used data from the Surveillance, Epidemiology, and End Results (SEER) database to construct a nomogram for predicting adverse outcomes in SPSCLS patients.

2. Methods

2.1 Selection of patients

Patients pathologically diagnosed with SPSCLC after NSCLC from January 2004 to December 2015 were initially identified from the SEER Multiple Primary Standardized Incidence Ratios (MP-SIR) session. Due to the general treatment and monitoring schedule for NSCLC, the minimum interval of 4 months was demanded between the initial primary lung cancer (IPLC) and SPSCLC. It is worth noting that no informed consent statement is needed in this paper as our data were extracted from a public database and all the information collected was anonymized. Exemption from ethical approval was granted by the ethics review committee of the First Affiliated Hospital of Chongqing Medical University. All the procedures in the study complied with the 1964 Declaration of Helsinki and its subsequent amendments.

The histology of tumors was classified according to the morphological code of the third edition of the International Classification of Diseases for Oncology (ICD-O-3). The screening and selection procedures are presented in Supplementary Fig. 1. The following inclusion criteria were used: (1) ICD-O-3 site codes: C340–C349 (lung and bronchus); (2) ICD-0-3 histology codes of SPSCLC: 8002, 8041–8045 (small cell lung cancer); (3) age ≥ 18 years at the time of NSCLC diagnosis; (4) SPSCLC diagnosed between January 2004 and December 2015. On the other hand, patients who met the following criteria were excluded: (1) the IPLC was small cell lung cancer; (2) the time interval time between two cancers was less than 4 months; (3) those lacking the AJCC stage information; (4) survival time was unknown or less than 1 month.

2.2 Patient’s characteristics and outcome definition

All patients with a surgical site-specific code from 10 to 90 were defined as the surgery group while the others were defined as the non-surgery group. Tumor stage was manually re-classified using the SEER variables according to the AJCC 8th edition. Age and marital status at the time of SPSCLC diagnosis were also included in analysis. The overall survival (OS) was considered as the primary endpoint, which referred to the time between SPSCLC diagnosis and all-cause mortality or the last follow-up. All the patients with SPSCLC who survived at the last follow-up period were reviewed.

The study population was randomly split into the training and validation groups at a ratio of 7:3, using the createDataPartition function in the caret package. Thereafter, the training group was used to filter variables and establish a nomogram while the validation group was employed to confirm the results.

2.3 Statistical analysis

Baseline differences between the training and validation cohorts were compared by Fisher exact test. The Cox regression model was implemented to conduct univariate analysis on all 17 variables. Following this, variables (p < 0.05) in univariate Cox regression analysis were incorporated into multivariate analysis. The stepwise regression employing the minimum Akaike information criterion (AIC) was performed to screen variables for the nomogram [9], which was then used to estimate the 1-/3-year OS.

The 1-/3-year area under the curves (AUC) were used to assess the discriminative ability of the nomogram. We also used the 1-/3-year calibration plots to evaluate the calibrating ability of the nomogram. Besides, various statistical methods, including concordance index (C-index), integrated discrimination improvement (IDI), continuous net reclassification index (NRI) and decision curve analysis (DCA), were implemented to assess the clinical significance of the nomogram compared with AJCC 8th tumor staging alone [1012]. The cut-off value for different risk stratifications was generated by the X-tile software (Yale University, 3.6.1) [13]. Furthermore, Kaplan-Meier survival curves and the univariate Cox regression analyses were implemented to describe and compare the OS of patients in the different risk stratifications.

According to the Harrell guideline, the covariates in the prediction model should be less than 1/10 of the number of events [9]. Besides, the variance inflation factor (VIF) values of all variables in the nomogram were less than 4, suggesting that no multicollinearity was present. Statistical significance was set at a two-tailed p-value < 0.05 and R version 3.6.1 was used for all the statistical analyses.

3. Results

A total of 420 subjects diagnosed with SPSCLC after NSCLC were randomized into the training (N = 296) and validation (N = 124) cohorts. The process of population screening is presented in Supplementary Fig. 1. The median follow-up time was 10.00 months (interquartile range (IQR): 4.00, 13.63) in the overall study population, 10.00 months (IQR: 4.00, 17.00) in the training cohort, and 10.00 months (IQR: 5.00, 18.00) in the validation cohort. The baseline demographic data of the patients are reported in Table 1. The results showed that a larger proportion of the population consisted of patients who had received surgery for the first but not the second primary tumor. White was the most common ethnicity in the two cohorts. Nonetheless, no significant difference was detected between the training and validation cohorts (P > 0.05).

Table 1

Baseline clinicopathological characteristics and treatment experience of all patients and those in the training and validation cohort.

 

All cohorts [cases (%)]

Training cohort [cases (%)]

Validation cohort [cases (%)]

P

Total

420

296

124

 

Age, years

     

0.420

< 75

258 (61.4)

186 (62.8)

72 (58.1)

 

≥ 75

162 (38.6)

110 (37.2)

52 (41.9)

 

Marital status

     

0.839

Married

207 (49.3)

145 (49.0)

62 (50.0)

 

Un-married

192 (45.7)

135 (45.6)

57 (46.0)

 

Unknown

21 (5.0)

16 (5.4)

5 (4.0)

 

Race

     

0.066

White

366 (87.1)

263 (88.9)

103 (83.1)

 

Black

37 (8.8)

20 (6.8)

17 (13.7)

 

Other

17 (4.0)

13 (4.4)

4 (3.2)

 

Gender

     

0.839

Female

239 (56.9)

167 (56.4)

72 (58.1)

 

Male

181 (43.1)

129 (43.6)

52 (41.9)

 

Interval

     

0.652

≤ 48 months

266 (63.3)

190 (64.2)

76 (61.3)

 

> 48 months

154 (36.7)

106 (35.8)

48 (38.7)

 

SPSCLC stage

     

0.269

I

89 (21.2)

70 (23.6)

19 (15.3)

 

II

22 (5.2)

16 (5.4)

6 (4.8)

 

III

137 (32.6)

92 (31.1)

45 (36.3)

 

IV

172 (41.0)

118 (39.9)

54 (43.5)

 

IPLC stage

     

0.792

I

283 (67.4)

195 (65.9)

88 (71.0)

 

II

34 (8.1)

25 (8.4)

9 (7.3)

 

III

77 (18.3)

57 (19.3)

20 (16.1)

 

IV

26 (6.2)

19 (6.4)

7 (5.6)

 

SPSCLC surgery

     

0.898

No

373 (88.8)

262 (88.5)

111 (89.5)

 

Yes

47 (11.2)

34 (11.5)

13 (10.5)

 

IPLC surgery

     

0.269

No

112 (26.7)

84 (28.4)

28 (22.6)

 

Yes

308 (73.3)

212 (71.6)

96 (77.4)

 

SPSCLC laterality

     

0.188

Left

180 (42.9)

130 (43.9)

50 (40.3)

 

Right

216 (51.4)

153 (51.7)

63 (50.8)

 

Unknown

24 (5.7)

13 (4.4)

11 (8.9)

 

IPLC laterality

     

0.524

Left

186 (44.3)

136 (45.9)

50 (40.3)

 

Right

229 (54.5)

157 (53.0)

72 (58.1)

 

Unknown

5 (1.2)

3 (1.0)

2 (1.6)

 

SPSCLC radiotherapy

     

0.439

No

213 (50.7)

146 (49.3)

67 (54.0)

 

Yes

207 (49.3)

150 (50.7)

57 (46.0)

 

IPLC radiotherapy

     

0.535

No

308 (73.3)

214 (72.3)

94 (75.8)

 

Yes

112 (26.7)

82 (27.7)

30 (24.2)

 

SPSCLC chemotherapy

     

0.658

No

114 (27.1)

78 (26.4)

36 (29.0)

 

Yes

306 (72.9)

218 (73.6)

88 (71.0)

 

IPLC chemotherapy

     

0.924

No

278 (66.2)

195 (65.9)

83 (66.9)

 

Yes

142 (33.8)

101 (34.1)

41 (33.1)

 

IPLC histology

     

0.391

Adenocarcinoma

157 (37.4)

114 (38.5)

43 (34.7)

 

SCC

194 (46.2)

138 (46.6)

56 (45.2)

 

Others

69 (16.4)

44 (14.9)

25 (20.2)

 

IPLC grade

     

1.000

G1/G2

180 (42.9)

127 (42.9)

53 (42.7)

 

G3/G4

169 (40.2)

119 (40.2)

50 (40.3)

 

Unknown

71 (16.9)

50 (16.9)

21 (16.9)

 
Abbreviations: IPLC, initial primary lung cancer; SPSCLC, second primary small cell lung cancer; G, nuclear grade; SCC, squamous cell carcinoma.

Univariate regression analysis indicated that age, SPSCLC stage, SPSCLC surgery, SPSCLC radiation and SPSCLC chemotherapy were significantly related to OS. However, the remaining variables had no significant association with OS (Table 2). Additionally, multivariate analysis revealed that a higher SPSCLC stage, no SPSCLC surgery, no SPSCLC radiation and no SPSCLC chemotherapy were independently adverse predictors of all-cause death (Table 2).

Table 2

Univariate and multivariate analyses of prognostic variables for overall survival in the training cohort.

Variables

Univariate Cox Regression

Multivariate Cox Regression

HR (95% CI)

P-value

HR (95% CI)

P-value

Marital status

       

Married

Reference

Reference

-

-

Un-married

0.97 (0.76, 1.24)

0.805

-

-

Unknown

0.87 (0.51, 1.50)

0.576

-

-

Race

       

White

Reference

Reference

-

-

Black

1.00 (0.62, 1.62)

0.997

-

-

Other

0.79 (0.42, 1.49)

0.469

-

-

Gender

       

Female

Reference

Reference

-

-

Male

1.11 (0.87, 1.41)

0.389

-

-

Age

       

< 75

Reference

Reference

Reference

Reference

≥ 75

1.34 (1.05, 1.73)

0.020

1.27 (0.98, 1.63)

0.067

Interval

       

≤ 48 months

Reference

Reference

-

-

> 48 months

0.99 (0.77, 1.28)

0.971

-

-

SPSCLC stage

       

I

Reference

Reference

Reference

Reference

II

1.23 (0.69, 2.18)

0.479

1.48 (0.83, 2.65)

0.185

III

1.71 (1.22, 2.42)

0.002

1.86 (1.28, 2.70)

0.001

IV

2.91 (2.09, 4.06)

< 0.001

2.61 (1.80, 3.78)

< 0.001

IPLC stage

       

I

Reference

Reference

-

-

II

1.12 (0.73, 1.73)

0.608

-

-

III

1.28 (0.94, 1.73)

0.118

-

-

IV

1.00 (0.59, 1.70)

0.994

-

-

SPSCLC surgery

       

No

Reference

Reference

Reference

Reference

Yes

0.50 (0.34, 0.75)

< 0.001

0.51 (0.32, 0.81)

0.004

IPLC surgery

       

No

Reference

Reference

-

-

Yes

0.86 (0.66, 1.13)

0.273

-

-

SPSCLC laterality

       

Left

Reference

Reference

-

-

Right

0.89 (0.70, 1.15)

0.386

-

-

Unknown

1.47 (0.81, 2.67)

0.207

-

-

IPLC laterality

       

Left

Reference

Reference

-

-

Right

0.96 (0.75,1.22)

0.737

-

-

Unknown

1.56 (0.49, 4.93)

0.446

-

-

SPSCLC radiotherapy

       

No

Reference

Reference

Reference

Reference

Yes

0.58 (0.45, 0.74)

< 0.001

0.55 (0.42, 0.71)

< 0.001

IPLC radiotherapy

       

No

Reference

Reference

-

-

Yes

1.14 (0.87, 1.49)

0.335

-

-

SPSCLC chemotherapy

       

No

Reference

Reference

Reference

Reference

Yes

0.62 (0.48, 0.82)

< 0.001

0.54 (0.40, 0.71)

< 0.001

IPLC chemotherapy

       

No

Reference

Reference

-

-

Yes

0.98 (0.76, 1.26)

0.857

-

-

IPLC histology

       

Adenocarcinoma

Reference

Reference

-

-

SCC

1.17 (0.90, 1.52)

0.251

-

-

Others

1.29 (0.89, 1.86)

0.178

-

-

IPLC grade

       

G1/G2

Reference

Reference

-

-

G3/G4

1.23 (0.95, 1.60)

0.123

-

-

Unknown

1.06 (0.75, 1.50)

0.751

-

-

Abbreviations: CI, confidence interval; HR, hazard ratio; IPLC, initial primary lung cancer; SPLC, second primary lung cancer; G, grade; SCC, Squamous cell carcinoma.

In addition, stepwise regression analysis in the training cohort showed that age, SPSCLC stage, SPSCLC surgery, SPSCLC radiation and SPSCLC chemotherapy had minimal AIC values and they were subsequently chosen to establish the nomogram (Fig. 1). The nomogram can be used to predict the 1- and 3-year OS of an individual patient, allowing clinicians to obtain the survival probability of patients. Every independent prognostic factor corresponds to a specific point and the total risk points can be acquired by adding up the individual points. In this study, the total risk points for most patients ranged from 120 to 280.

The findings also showed that both the 1- and 3-year AUCs of the nomogram in the training and validation cohorts were more than 0.75 (Fig. 2). In addition, the 1- and 3-year calibration curves for the two cohorts indicated good linearity between the predicted and observed survival probability (Fig. 2).

The bias-corrected C-index of the nomogram in the training and validation cohorts was 0.69 and 0.75, respectively. Comparatively, the bias-corrected C-index of the AJCC 8th tumor staging in the two cohorts was 0.62 and 0.65, respectively. Besides, The IDI and NRI were carried out to compare the prognostic discriminatory power of the nomogram and the AJCC 8th tumor staging alone (Table 3). Compared to the AJCC 8th tumor staging, the nomogram had significantly higher discrimination (IDI for the 1- and 3-year OS were 0.097 (p < 0.001) and 0.052 (p = 0.02), respectively) and reclassification ability (continuous NRI for 1- and 3-year OS were 0.345 and 0.377, respectively (both p < 0.001), Table 3) in the training group. Additionally, DCA curves from the training group indicated that the nomogram model had good clinical applicability in predicting the 1- and 3-year OS as shown by the marked increase in net benefit (Fig. 3). The validation cohort obtained similar results (Table 3 and Fig. 3).

Table 3

Discrimination ability of different predictive models for primary endpoint in training cohort and validation cohort.

 

Training cohort

Validation cohort

Estimate

95% CI

P

Estimate

95% CI

P

IDI (vs. the AJCC 8th tumor staging)

           

For 1-year OS

0.097

0.060–0.151

< 0.001

0.152

0.070–0.241

< 0.001

For 3-year OS

0.052

0.005–0.123

0.020

0.079

0.002–0.181

0.040

NRI (vs. the AJCC 8th tumor staging)

           

For 1-year OS

0.345

0.220–0.457

< 0.001

0.388

0.194–0.518

< 0.001

For 3-year OS

0.377

0.142–0.541

< 0.001

0.186

0.003–0.568

0.020

Abbreviations: 95% CI, 95% confidence interval; OS, overall survival; AJCC, American Joint Committee on Cancer; NRI, net reclassification improvement; IDI, integrated discrimination improvement. Vs, versus.

Patients were assigned into three groups based on the total risk point, namely: the low-risk group (total points ≤ 189), the middle-risk group (189 < total points ≤ 244), and the high-risk group (total points > 244). In the total population, the median OS of patients in the low-, middle-, and high-risk groups was 16 months (95% CI, 14–19), 7 months (95% CI, 6–8), and 2 months (95% CI, 1–3), respectively. The Kaplan-Meier curves suggested huge differences in prognosis among the three risk groups (Fig. 4, all Log-rank p < 0.001). Univariate Cox analysis also confirmed the Kaplan-Meier results (all p < 0.001). Compared with the low-risk group, the middle-risk and high-risk groups had a 2.75 and 8.02 times, respectively, higher risk of all-cause mortality in the total population. Similarly, results from the training and validation groups displayed that the risk of all-cause mortality was highest in the high-risk group while patients in the low-risk group had the lowest risk of mortality over time.

4. Discussion

SPSCLC after NSCLC is a relatively rare disease and therefore lacks an optimal management strategy. To our knowledge, no study has evaluated its prognosis in these patients. Present study showed that second primary tumor stage and treatment rather than the initial primary tumor were significantly associated with the prognosis of SPSCLC patients. Based on this, we build a nomogram to predict the outcome of individual patients with SPSCLC. Compared to the AJCC staging system, the nomogram achieves a more accurate prognosis prediction and better clinical applicability.

We found that the prognosis of SPCLC patients was related to the second primary tumor and the associated treatment rather than the initial primary tumor. Zhang R et al. showed that the tumor stage and treatment of the initial primary tumor were not independent prognostic factors in patients with early-stage second primary NSCLC after small cell lung cancer [14]. Another population-based study also reported similar results in patients with second primary tumors after prostate cancer [15]. Therefore, we did not incorporate the initial primary tumor and related treatments into the nomogram. When a patient develops SPSCLC, more attention should be paid to the SPSCLC stage and treatment measures.

The nomogram also showed that patients treated with surgery and radio-chemotherapy had the lowest risk score while those who did not receive any treatment had the highest. This implied that patients with SPSCLC could benefit from surgery, chemotherapy and radiotherapy. This finding corroborated with the results of previous researches, which showed that an anatomical removal of the second lesion was the first choice as long as the patient’s cardiopulmonary reserve allowed [16, 17]. Multiple studies have demonstrated that chemotherapy and radiotherapy can improve survival in patients with multiple primary lung cancers (MPLC) [1821]. But, it is still contradictory whether the time interval between IPLC and second primary lung cancer is correlated with the OS of the patients. For instance, Aziz, et al. argued that the longer the interval, the better the prognosis [22]. Nevertheless, other studies did not draw the same conclusion [23, 24]. A meta-analysis, consisting of 22 relevant studies with 1796 MPLC patients, demonstrated that the time interval had no effect on the OS of MPLC patients [25]. In this analysis, the time interval was not related to OS and was therefore not included in the nomogram.

Previous studies reported that the nomogram integrated multiple clinicopathologic and treatment factors into a mathematical model and was not inferior to the AJCC staging system in predicting prognosis and making clinical decisions on various types of cancer[2628]. Given that the AJCC staging system does not take into account age, treatment regimens and other clinicopathological data, patients with the same stage may have completely different prognoses. In this study, several factors were incorporated into the nomogram, including age, tumor stage and treatment. The results revealed that the nomogram we established worked better than the AJCC staging system in predicting the probability of survival in individual SPSCLC patients. A highly linear calibration curve and integrated AUC suggested that the nomogram had a powerful predictive ability. The continuous NRI and IDI index indicated that the nomogram had better discriminatory accuracy. Besides, DCA showed that the nomogram performed better than the AJCC staging system, within a major range of reasonable threshold probability. In addition, the cutoff of risk stratification based on the total points from the nomogram worked well in the training and validation cohorts. Kaplan-Meier curves presented that the gradual increase in the incidences of all-cause mortality was linked to an increase in risk stratification.

Although our nomogram performed well, some shortcomings of our study ought to be acknowledged. First, some important confounding factors are lacking in the SEER database, such as targeted therapy and immunotherapy. These factors may influence our findings [29, 30]. Second, we did not perform an external validation and therefore concerns on generalizability and robustness are warranted. Finally, as a retrospective study, selection bias is inevitable. Therefore, a multicenter prospective study is warranted to validate the results in the future.

5. Conclusions

The study establishes a nomogram and a risk stratification system for predicting probabilities of OS in patients with SPSCLC after NSCLC. Compared to the AJCC staging system, the nomogram achieves a more accurate prognosis prediction and better clinical applicability. It can therefore be used by clinicians to assist patient consultation and guide treatment decisions.

Declarations

Ethics approval and consent to participate

The study has been approved by the Ethics Committee of the First Affiliated Hospital of Chongqing Medical University.

Consent for publication

Not applicable.

Availability of data and materials

The data analyzed during this study are available from the corresponding author on reasonable request.

Competing interests

The authors declared no competing interests.

Funding Statement

This study was supported by the Chengdu Science and Technology Agency (NO. 2020-YF05-00184-SN).

Authors’ contributions 

Ju Zhu1, MD, Hao-yu Ran1, MD, Qian-cheng Lai2, MD, Liu Ye3*, MD, Yue Shao

JZ and YS made substantial contributions to conception and design. HR, QL, QL, and LY collected, analyzed, and interpreted the data. JZ, LY, and YS write and revised the manuscript. All authors read and approved the final manuscript.

Acknowledgements

The authors acknowledge the efforts of the Surveillance, Epidemiology, and End Results (SEER) Program tumor registries in the construction of this database. We also thank Dr. Jerry for English language editing.


 

References

  1. Bray F, Ferlay J, Soerjomataram I, et al. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2018;68(6):394–424.
  2. Torre LA, Siegel RL, Jemal A. Lung Cancer Statistics. Adv Exp Med Biol. 2016;893:1–19.
  3. Han SS, Rivera GA, Tammemagi MC, et al. Risk stratification for second primary lung cancer. J Clin Oncol. 2017;35(25):2893–9.
  4. Wozniak AJ, Schwartz AG. The risk of second primary lung cancer: an unsolved dilemma. Transl Lung Cancer Res. 2018;7(Suppl 1):54–6.
  5. Liang W, Zhang L, Jiang G, et al. Development and validation of a nomogram for predicting survival in patients with resected non–small-cell lung cancer. J Clin Oncol. 2015;33(8):861–9.
  6. Pan H, Shi X, Xiao D, et al. Nomogram prediction for the survival of the patients with small cell lung cancer. J Thorac Dis. 2017;9(3):507–18.
  7. Wu J, Zhou Q, Pan Z, et al. Development and validation of a nomogram for predicting long-term overall survival in nasopharyngeal carcinoma: A population-based study. Med (Baltim). 2020;99(4):e18974.
  8. Narita Y, Kadowaki S, Oze I, et al. Establishment and validation of prognostic nomograms in first-line metastatic gastric cancer patients. J Gastrointest Oncol. 2018;9(1):52–63.
  9. Balachandran VP, Gonen M, Smith JJ, et al. Nomograms in oncology: More than meets the eye. Lancet Oncol. 2015;16(4):e173–80.
  10. Uno H, Tian L, Cai T, et al. A unified inference procedure for a class of measures to assess improvement in risk prediction systems with survival data. Stat Med. 2013;32(14):2430–42.
  11. Pencina MJ, Sr D’Agostino RB, Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med. 2011;30(1):11–21.
  12. Vickers AJ, Elkin EB. Decision curve analysis: A novel method for evaluating prediction models. Med Decis Making. 2006;26(6):565–74.
  13. Camp RL, Dolled-Filhart M, Rimm DL. X-tile: A new bioinformatics tool for biomarker assessment and outcome-based cut-point optimization. Clin Cancer Res. 2004;10(21):7252–9.
  14. Zhang R, Cai L, Wang G, et al. Resection of Early-Stage Second Primary Non-small Cell Lung Cancer After Small Cell Lung Cancer: A Population-Based Study. Front Oncol. 2020;9:1552.
  15. Chattopadhyay S, Zheng G, Hemminki O, et al. Prostate cancer survivors: Risk and mortality in second primary cancers. Cancer Med. 2018;7(11):5752–9.
  16. Xue X, Xue Q, Wang N, et al. Early clinical diagnosis of synchronous multiple primary lung cancer. Oncol Lett. 2012;3(1):234–7.
  17. Riquet M, Cazes A, Pfeuty K, et al. Multiple lung cancers prognosis: what about histology? Ann Thorac Surg. 2008;86(3):921–6.
  18. Kocaturk CI, Gunluoglu MZ, Cansever L, et al. Survival and prognostic factors in surgically resected synchronous multiple primary lung cancers. Eur J Cardiothorac Surg. 2011;39(2):160–6.
  19. Ridge CA, Silk M, Petre EN, et al. Radiofrequency ablation of T1 lung carcinoma: comparison of outcomes for first primary, metachronous, and synchronous lung tumors. J Vasc Interv Radiol. 2014;25(7):989–96.
  20. Chang JY, Liu YH, Zhu Z, et al. Stereotactic ablative radiotherapy: a potentially curable approach to early stage multiple primary lung cancer. Cancer. 2013;119(18):3402–10.
  21. Creach KM, Bradley JD, Mahasittiwat P, et al. Stereotactic body radiation therapy in the treatment of multiple primary lung cancers. Radiother Oncol. 2012;104(1):19–22.
  22. Aziz TM, Saad RA, Glasser J, et al. The management of second primary lung cancers. A single centre experience in 15 years. Eur J Cardiothorac Surg. 2002;21:527–33.
  23. Rosengart TK, Martini N, Ghosn P, et al. Multiple primary lung carcinomas: prognosis and treatment. Ann Thorac Surg. 1991;52(3):773–9.
  24. Okada M, Tsubota N, Yoshimura M, et al. Operative approach for multiple primary lung carcinomas. J Thorac Cardiovasc Surg. 1998;115(4):836–40.
  25. Jiang L, He J, Shi X, et al. Prognosis of synchronous and metachronous multiple primary lung cancers: systematic review and meta-analysis. Lung Cancer. 2015;87(3):303–10.
  26. Song C, Yu D, Wang Y, et al. Dual Primary Cancer Patients With Lung Cancer as a Second Primary Malignancy: A Population-Based Study. Front Oncol. 2020;10:515606.
  27. Wang Y, Li J, Xia Y, et al. Prognostic nomogram for intrahepatic cholangiocarcinoma after partial hepatectomy. J Clin Oncol. 2013;31(9):1188–95.
  28. Wang S, Yang L, Ci B, et al. Development and Validation of a Nomogram Prognostic Model for SCLC Patients. J Thorac Oncol. 2018;13(9):1338–48.
  29. Yang S, Zhang Z, Wang Q. Emerging therapies for small cell lung cancer. J Hematol Oncol. 2019;12(1):1–11.
  30. Armstrong SA, Liu SV. Immune checkpoint inhibitors in small cell lung cancer: a partially realized potential. Adv Ther. 2019;36(8):1826–32.