A Clinical Nomogram For Predicting Preterm Birth in Women Who Conceived Through Assisted Reproductive Technology

DOI: https://doi.org/10.21203/rs.3.rs-1036505/v1

Abstract

Background: We used prepregnant and gestational characteristics as predictors to develop and validated a nomogram predicting the risk of preterm birth (PTB) in assisted reproductive technology (ART) treated women. 

Methods: The National Vital Statistics System (NVSS) was queried for singleton ART-treated pregnant women from 2015 to 2019. Multivariable cox regression was used to develop the early (< 32 weeks) or late (< 37 weeks) PTB risk model using both statistical significance and clinical importance criteria for variable selection. The predictive accuracy was assessed, and bootstrapping was used for validation. A nomogram was constructed for the presentation of the final model. 

Results: ART-treated women who were over 45 years old, black, obese, had a history of cesarean section and PTB, restarting ART within 3 months, prepregnant diabetes, chronic hypertension, gestational diabetes, gestational hypertension, and eclampsia, had the highest risk for late and early-stage PTB. The nomogram with these variables accurately predicted PTB in ART women with a singleton pregnancy. (Brier score:0.121, calibration slope: 0.99, c-index: 0.684). 

Conclusion: We created a nomogram predicting the risk of early or late PTB in ART women with a singleton pregnancy, which could identify potentially at-risk women who seeking ART treatment and inform appropriate preterm care.

Introduction

Assisted reproductive technology (ART), a practical way to achieve pregnancy by artificial or partially artificial means, has been becoming more and more popular in infertility patients.[1] Despite the satisfactory efficacy of ART in the treatment of infertility, ART pregnancies are known to be associated with an increased risk of various adverse perinatal outcomes [2, 3, 4], particularly for preterm birth (PTB), which was defined as any delivery before 37 weeks gestation. PTB is a complex syndrome, in which there are multiple attributable causes and has been confirmed previously to be associated with increased risk of adverse health outcomes for the baby in both short and long term [5, 6]. Accumulating studies have found that an increased risk of PTB was observed among ART-related women compared with those naturally conceived pregnancies [5, 7, 8]. Further, various infertile factors, such as older age and malnutrition, will indirectly affect the risk of PTB in women undergoing ART treatment [9, 10, 11]. The causes of these disparities are complex, thus, there is an urgent need to establish a prediction or early recognition model for ART women.

Currently, many studies have attempted to construct models to predict the occurrence of PTB in pregnant women who conceived naturally [12, 13, 14, 15], but little has been done in women who conceived by ART [16]. Yet, most of these models have not been validated clinically, as a result, that their accuracy might be limited. And in these models, cervical length (CL) and fetal fibronectin(fFN) are usually used as PTB indicators for pregnant women at high risk. Unfortunately, most PTB occurs in low-risk pregnancies, suggesting that CL and fFN have limited predictive power to detect PTB risk in a majority of the population. In addition, most of these indicators are only related to conditions during pregnancy, with little consideration of pre-pregnancy characteristics [17]. Nevertheless, those pre-pregnancy factors have the potential to adversely impact PTB. So, we do need a model to consider comprehensively the maternal demographic as well as medical characteristics.

Nomograms, an important decision-making component of modern medicine, are widely used as a tool to estimate prognosis in medicine [18]. By integrating different prognostic and deterministic variables to generate individual numerical probabilities of clinical events, nomogram satisfies our desire for integrated biological and clinical models and our pursuit of personalized medicine. The development of a simple and visual clinical tool for predicting would help clinicians to assess PTB probability for ART pregnant women more accurately.

Therefore, we aimed to develop a workable nomogram model in clinical practice for predicting the risk of PTB in ART pregnant women by using prepregnancy characteristics and gestational characteristics as predictors.

Materials And Methods

Data source

This study selected qualified ART women from the National Vital Statistics System (NVSS) Participant Use File. The NVSS natality data is a retrospective dataset from the Centers for Diseases Control and Prevention’s National Center for Health Statistics (NCHS) that captures all births data from all 50 United States (U.S.) states and the District of Columbia.

Population Selection

The ART women who had live-birth deliveries and available maternal demographic, as well as clinical characteristics from 2015-2019, were included in this study. The excluded criteria were the following: 1) pregnant women under 18 years old; 2) the infant died after birth; 3) multiparous women; 4) having missing data on prepregnancy BMI, history of PTB, parity history, cesarean history, birth interval, prepregnancy diabetes, prepregnancy hypertension, history of smoking before and during pregnancy, gestational diabetes, gestation hypertension, eclampsia, and fertility-enhancing drugs.

Outcomes

The major outcome was gestational age which was obtained on the birth certificate and determined from ultrasonography. We included only those gestational ages in the range of 17-47 weeks. Our outcomes were further divided into term births (TB: ≥ 37 weeks gestation), and preterm births (PTB: < 37 weeks gestation).

Development of the Prediction Model

We collected and classified the following demographic and medical variables: age (18-24, 25-34, 35-44, ≥ 45), race (white, black, American Indian or Alaskan Native (AIAN), Asian, Pacific islander, more than one race), pre-pregnancy BMI (underweight < 18.5, normal 18.5 - 24.9, overweight 25.0 - 29.9, obesity I 30.0 - 34.9, obesity II 35.0 - 39.9, obesity III ≥ 40.0), history of PTB, parity history, cesarean history, birth interval (no previous pregnancy, 0-3 months, 4-17 months, 18-35 months, 36-59 months, ≥ 60 months), prepregnancy diabetes, prepregnancy hypertension, history of smoking before and during pregnancy, gestational diabetes, gestational hypertension, eclampsia, and fertility-enhancing drugs.

The t-test and c2 test was used to compare continuous and categorical variables for TB and PTB women, respectively. Univariable analysis was performed to identify variables associated with PTB. A predictor with P < 0.05 in univariate analysis was included as a candidate variable for multivariable analysis. Then, we used Cox regression for multivariable analysis to develop a model. All statistical analyses were performed using SPSS version 21.0 and R Package Regression Modeling Strategies.

Evaluation of the predictive model

It was necessary to assess the predictive accuracy of the model before developing a nomogram. The common methods, used to verify the predictive power of the model, included c-index for discrimination, Brier score for overall performance, and calibration slope for calibration.

The c-index a was used to evaluate the discrimination which was the ability of the predictive model to distinguish between people who have experienced an event and those who have not. The c-index value is 1, which indicates that the model can be accurately discriminate, while the value equal to 0.5 indicates the random chance of correctly identifying the event. Calibration slope is another index to measure the performance of the prediction model, which checks the consistency between the predicted results and the actual results. A 45° calibration curve represents an ideal prognosis prediction. And the calibration slope value is closer to 1, the performance is better. The Brier score evaluates overall performance and evaluates the difference between observed and predicted values. The closer the value is to 0, the better the predictive power.

The bootstraps using 1000 repeats were used for internal validation of our model and to obtain a deviation correction prediction accuracy measure for the final model. [19] Finally, we used receiver operator characteristic curve (ROC), and calibration curve to evaluate the utility of our nomogram in the validation set.

Creation of the Nomogram

A nomogram was developed as the visualized graphical representation of our final model. [20] There is a guideline on the top of the nomogram that shows a score from 0 to 100 for each predictor. The predictor variables are shown below with a scale showing their effect size, visually showing the relative weight of each variable, and allowing points to be assigned to each significant clinical feature. The sum of the points for each predictor and the corresponding result for predicting the probability of premature birth can be read from the bottom 2 lines.

Results

Baseline Clinical Characteristics

After carefully selecting ART-treatment women according to the selection criteria, a total of 122430 cases were enrolled in analysis: 106856 (87.28%) was term birth and 15574 (12.72%) was PTB. A summary of demographic factors and clinical parameters of ART women in this study are shown in Table 1. The mean age was 35.6 years old, and the majority were white (79.12%), and had normal pre-pregnancy BMI (51.14%), with a statistical significance. But the history of cesarean and smoking during the pregnancy had no significant difference after adjusting for multiple comparisons. The incidence of PTB was higher when ART women had a history of PTB and cesarean, prepregnancy hypertension, prepregnancy diabetes, prepregnancy hypertension, gestational diabetes, eclampsia, and short birth interval.

Table 1

Demographic characteristics of ART-treated pregnant women, dichotomized by preterm or term birth

Maternal characteristics

Total

(n = 122430)

Term birth

(n = 106856)

Preterm birth

(n = 15574)

P-value

N

%

N

%

N

%

 

Pre-pregnancy characteristics

     

Age

     

< 0.001

mean ± SD (years)

35.6 ± 5.09

35.5 ± 5.03

36.0 ± 5.42

 

18 --24

1148

0.94

999

0.93

149

0.96

 

25 - 34

52528

42.90

46267

43.30

6261

40.20

 

35 - 44

62452

51.01

54422

50.93

8030

51.56

 

≥ 45

6302

5.15

5168

4.84

1134

7.28

 

Race

     

< 0.001

White

96862

79.12

84818

79.38

12044

77.33

 

Black

5883

4.81

4785

4.48

1098

7.05

 

AIAN

184

0.15

162

0.15

22

0.14

 

Asian

17172

14.03

15031

14.07

2141

13.75

 

Pacific islander

95

0.08

80

0.07

15

0.10

 

More than one race

2234

1.82

1980

1.85

254

1.63

 

Pre-pregnancy BMI

     

< 0.001

mean ± SD (kg/m2)

25.9 ± 5.83

25.8 ± 5.75

26.9 ± 6.28

 

Underweight < 18.5

2949

2.41

2634

2.46

315

2.02

 

Normal 18.5 - 24.9

62611

51.14

55683

52.11

6928

44.48

 

Overweight 25.0 - 29.9

31579

25.79

27387

22.37

4192

25.79

 

Obesity I 30.0 - 34.9

14962

12.22

12578

11.77

2384

15.31

 

Obesity II 35.0 - 39.9

6715

5.48

5614

5.25

1101

7.07

 

Obesity III ≥ 40.0

3614

2.95

2960

2.77

654

4.20

 

Obstetric history

       

Parity

     

< 0.001

Nulliparous

76267

62.29

66353

62.10

9914

63.66

 

Multiparous

46163

37.71

40503

37.90

5660

36.34

 

Previous Cesarean

     

< 0.001

No

105922

86.52

92675

86.73

13247

85.06

 

Yes

16508

13.48

14181

13.27

2327

14.94

 

Previous Preterm Birth

     

< 0.001

No

118775

97.01

104119

97.44

14656

94.11

 

Yes

3655

2.99

2737

2.56

918

5.89

 

Pre-pregnancy Diabetes

     

< 0.001

No

121173

98.97

105924

99.13

15249

97.91

 

Yes

1257

1.03

932

0.87

325

2.09

 

Chronic hypertension

     

< 0.001

No

118708

96.96

104044

97.37

14664

94.16

 

Yes

3722

3.04

2812

2.63

910

5.84

 

Cigarettes Before Pregnancy

     

0.082

No

121169

98.97

105776

98.99

15393

98.84

 

Yes

1260

1.03

1079

1.01

181

1.16

 

Birth interval

     

< 0.001

no previous pregnancy

57613

47.06

50309

47.08

7304

46.90

 

0 - 3 months

3748

3.06

3045

2.85

703

4.51

 

4 - 17 months

20164

16.47

17683

16.55

2481

15.93

 

18 -35 months

24769

36.4

21930

20.52

2839

18.23

 

36 - 59 months

7404

6.05

6466

6.05

938

6.02

 

≥ 60 months

8732

7.13

7423

6.95

1309

8.41

 

Pregnancy characteristics

       

Cigarettes During 1st Trimester

   

0.964

No

121940

99.60

106428

99.60

15512

99.60

 

Yes

490

0.40

428

0.40

62

0.40

 

Cigarettes During 2nd Trimester

   

0.974

No

122035

99.68

106511

99.68

15524

99.68

 

Yes

395

0.32

345

0.32

50

0.32

 

Cigarettes During 3rd Trimester

   

0.890

No

122044

99.68

106520

99.69

15524

99.68

 

Yes

386

0.32

336

0.31

50

0.32

 

Gestational Diabetes

     

< 0.001

No

109827

89.71

96301

90.12

13526

86.85

 

Yes

12603

10.29

10555

9.88

2048

13.15

 

Gestational Hypertension

     

< 0.001

No

109893

89.76

97425

91.17

12468

80.06

 

Yes

12537

10.24

9431

8.83

3106

19.94

 

Eclampsia

     

< 0.001

No

122038

99.68

106624

99.78

15414

98.97

 

Yes

392

0.32

232

0.22

160

1.03

 

Fertility-Enhancing Drugs

     

0.052

No

105431

86.12

91941

86.04

13490

86.62

 

Yes

16999

13.88

14915

13.96

2084

13.38

 
Abbreviation: American Indian or Alaskan Native (AIAN); BMI: body mass index

Development and Evaluation of the Predictive Model  

Univariate and multivariate cox regression analyses were performed on the variables related to PTB, and the results are summarized in Supplementary Table 1. The final model included 13 variables that were independently associated with PTB: age, race, pre-pregnancy BMI, parity, history of PTB and cesarean, birth interval, prepregnancy diabetes, prepregnancy hypertension, gestational diabetes, prepregnancy hypertension, eclampsia, and fertility-enhancing drugs. What’s more, the forest plot was also drawn to graphically display the results of the cox model for multivariate analysis, as displayed in Figure 1.

The model was verified with 1000 replicates of bootstrapping, and the bias correction measurement values for accuracy were Brier score 0.17, correction slope 0.99, and c-index 0.67. And the calibration plot and ROC curves were also plotted for graphical evaluation of calibration and discrimination, as shown in Figure 2.

C-index tests the discrimination ability of the model, or the ability to distinguish a woman who gives birth prematurely from a woman who gives birth normally. The value ranges from 0.5 to 1, and the closer the value is to 1, the stronger the discriminant ability is. Correction slope tests the consistency between the predicted value and the result with a perfect slope equal to 1. The Brier score is a measure of overall performance that covers both calibration and discrimination. It represents the difference between the predicted probability and the actual outcome. The score ranged from 0 to 1, and the closer the value was to 0, the better the prediction ability. In general, the values obtained by our measurements show a fairly good prediction accuracy.

Creation and use of the nomogram

A nomogram diagram is constructed based on the results of the cox regression model and shows the predictive variables and corresponding point scales, as presented in Figure 3. The steps to use a nomogram are: 1) identify the status of each predictor for pregnant women, 2) draw a straight line from each predicted state upwards to the partial point reference, 3) sum the points corresponding to each predicted state, 4) locate the sum on the total points reference line, and 5) draw a line from the total point line to the bottom risk line to find the probability of delivery at 32 weeks and 37 weeks for pregnant woman treated with ART.

The use of the nomogram can be illustrated by some clinical examples. In the first example (Supplementary Figure 1A), we calculated the predicted probability of PTB in a 23-year-old black woman with a prepregnancy BMI of 26 who was treated with ART and had pre-gestational diabetes and gestational diabetes and had no history of PTB, cesarean section, pregestational hypertension, gestational hypertension, eclampsia, or failure of fertility-enhancing drugs. Assign a score to each feature: 0 for no chronic hypertension, no gestational hypertension, no eclampsia, no history of PTB and cesarean, 2.5 for the failure of fertility-enhancing drugs, 3.75 for no birth interval, 7.5 for 23 years old, 8.75 for prepregnancy BMI of 26, 12.5 for chronic diabetes, 15 for nulliparous, 39.5 for black, and 42 for gestational diabetes. A total of 131.5 points corresponds to a nearly 94.5% chance of birth at 32 weeks and about 72.5% probability of delivery at 37 weeks. In the second example (Supplementary Figure 1B), we calculated an ART-treated pregnant woman who was 33 years old, Asian, multiparous, with 60 months birth interval and failure of fertility-enhancing drug therapy, with a pre-pregnancy BMI of 30, with a history of cesarean, and with chronic hypertension, gestational hypertension, gestational diabetes, but without a history of PTB, chronic diabetes, eclampsia. Points are again assigned for each feature: 0 for 33 years old, multiparous, no pre-gestational diabetes, no eclampsia, and no history of PTB, 2.5 for the failure of fertility, 17 for Asian and 60 months birth interval, 17.5 for pre-pregnancy BMI of 30, 12.5 for the presence of gestational diabetes, 56.5 for chronic hypertension, 67.5 for gestational hypertension, and 11.5 for a history of cesarean. The total of 202 points corresponds to an approximately 87.5% chance of birth at 32 weeks and an almost 48% rate of delivery at 37 weeks. The expected likelihood of PTB for individual patients based on features of their maternal feature can be used for treatment planning.

Discussion

In this study, we developed and internally validated a simple and intuitive nomogram for predicting and quantifying the risk of PTB in ART-treated pregnant women. We identified a series of maternal risk factors (age, race, pre-pregnancy BMI, history of PTB, parity history, history of cesarean, prepregnancy diabetes, chronic hypertension, gestational diabetes, gestational hypertension, eclampsia, use of fertility-enhancing drugs) and fully assembled them into nomogram model to personalize prediction of PTB in ART pregnant women. This can be used as a clinical tool to support clinicians in providing pregnancy management recommendations for ART pregnant women to reduce the risk of PTB.

Compared to naturally conceived pregnancies, ART pregnant women tend to have great risks for PTB, so it is necessary to develop a reliable tool for this predicting. Currently, several previous studies [12, 13, 14] have attempted to create a risk scoring system during pregnancy to predict PTB in naturally conceiving women, with mixed success. In particular, a scoring system created by Metha-Lee et al. that included only pre-pregnancy characteristics had a strong negative predictive value (NPV) of 76.7% and modest positive predictive value (PPV) of 51.5%. [14] Nevertheless, in clinical practice, a combination of fFN and CL is the most commonly used predictor for PTB. However, Esplin pointed that quantitative fFN and transvaginal CL had a poor predictive performance as screening tests for spontaneous preterm birth before 37 weeks in nulliparous women, with relatively low sensitivity and low PPV (only a high of 20.8%). [18] This suggests that fFN and CL test alone cannot accurately predict the risk of PTB, a new risk scoring system with demographic factors and clinical parameters should be incorporated into prediction models for improvement of accuracy. In addition, the PTB prediction model based on naturally conceived women may not be fully applicable to women receiving ART, as most women receiving ART are obese and older, which increases their risk of preterm birth. [9, 10, 11] Our study, therefore, fills an important gap in our understanding of PTB risk in women receiving ART.

In our study, we developed a nomogram targeting a series of potential risk factors, which may affect the structure and function of the cervix and increase the risk of cervical abnormalities, eventually leading to PTB. The prediction model we developed is based on widely used risk factors available at all maternity care centers.[21, 22, 23, 24, 25, 26, 27, 28] We used cox regression model to employ a combination of these risk factors, to calculate the contribution rate of each risk factor in developing PTB. We found that ART women who reported that they were over 45 years old, black, obese, had a history of cesarean section and PTB, restarting ART within 3 months, prepregnancy diabetes, prepregnancy hypertension, gestational diabetes, gestational hypertension, and eclampsia, had the highest risk for late and early-stage PTB. According to the U.S. Centers for Disease Control and Prevention, the PTB rate in the United States increased for the fifth consecutive year in 2019, and the rate of PTB in the ART population is higher relative to the overall rate of PTB, so reducing the rate of preterm birth is critical. Fortunately, most of these risk factors can be modified or controlled to reduce PTB risk. Above all, this tool is simple and practically costless. Even it can accurately identify ART women who are at low risk of delivery before 32 weeks but high risk of delivery before 37 weeks, providing the basis for early intervention for modifiable risk factors and implementation for non-modifiable risk factors.

Our study has some limitations that are worth discussing. First, the use of retrospective data might introduce selection bias, which is unable to identify those receiving PTB treatment. Second, we do not have access to specific ART treatment programs for those infertility women, making it impossible to discriminate the risk between them. Then, our model is not externally validated. However, we did use a broad, representative data set that captured most of ART pregnant women in the US to build an accurate model. More importantly, we also used bootstrapping (1000 repetitions, proven to provide best and least deviation estimates) to internally validate our clinical model, which provides good optimally adjusted estimates for its predictive ability.[29] This nomogram expands and improves previously published models for predicting PTB in women with natural pregnancies by using a robust sample size of more than120000 women receiving ART and incorporating multiple variables into this tool. This easy-to-use tool can provide additional clinical information, especially where CL and fFN settings are not fully known. This nomogram can be used in conjunction with other clinical data to share decisions or treatment options to reduce the probabilities of PTB for ART pregnant women.

Conclusion

We developed a simple risk score using maternal demographics and risk factors with a high predictivity for the risk of early or late PTB in ART women with a singleton pregnancy, which is helpful for clinical monitoring and early treatment decisions. The current nomogram model still needs to be verified by more institutional data.

Abbreviations

PTB: preterm birth

ART: assisted reproductive technology

NVSS: National Vital Statistics System

NCHS: National Center for Health Statistics

CL: cervical length

fFN: fetal fibronectin

ROC: receiver operator characteristic curve

Declarations

Ethics approval and consent to participate

All methods in this study were carried out in accordance with relevant guidelines and regulations. The study was approved by an ethics committee of JiNan University. We have obtained informed consent from during the treatment and got consent to publish the case from the study participant.

Consent for publication

All the authors are agreed to publish the research finding.

Availability of data and materials

The data presented in this study are available on request from the corresponding author. 

Competing interests

The authors declare that they have no competing interests

Funding

This research was funded by the National Natural Science Foundation of China (No. 81903294) and Natural Science Foundation of Guangdong Province (No. 2018030310412)

Author Contributions

C.L., J.H. and Y.L. brought the conception and design forward. The data required were acquired by R.G., T.L. and K.Z.. Y.L. and F.H. were responsible for data analysis and interpretation. C.L. and R.G. provided the database needed for the study. Y.L, F.H., C.L. and J.H. were the main contributor and organized the study. Y.L, F.H., R.G., T.L., K.Z., C.L. and J.H. participated in the draft or critical revise of the manuscript. All authors have read and agreed to the published version of the manuscript. 

Acknowledgements

We would like to appreciate the support from Jieying Li. 

References

  1. Zegers-Hochschild F, Adamson GD, Dyer S, Racowsky C, de Mouzon J, Sokol R, et al. The International Glossary on Infertility and Fertility Care, 2017. Fertil Steril. 2017 Sep;108(3):393–406.
  2. Practice Committee of American Society for Reproductive Medicine. Multiple gestation associated with infertility therapy: an American Society for Reproductive Medicine Practice Committee opinion. Fertil Steril. 2012 Apr;97(4):825–34.
  3. Sunderam S, Kissin DM, Crawford SB, Folger SG, Boulet SL, Warner L, et al. Assisted Reproductive Technology Surveillance - United States, 2015. MMWR Surveill Summ. 2018 Feb 16;67(3):1–28.
  4. Maheshwari A, Pandey S, Shetty A, Hamilton M, Bhattacharya S. Obstetric and perinatal outcomes in singleton pregnancies resulting from the transfer of frozen thawed versus fresh embryos generated through in vitro fertilization treatment: a systematic review and meta-analysis. Fertil Steril. 2012 Aug;98(2):368-77.e1-9.
  5. Goldenberg RL, Culhane JF, Iams JD, Romero R. Epidemiology and causes of preterm birth. Lancet. 2008 Jan 5;371(9606):75–84.
  6. Martin JN Jr, DʼAlton M, Jacobsson B, Norman JE. In Pursuit of Progress Toward Effective Preterm Birth Reduction. Obstet Gynecol. 2017 Apr;129(4):715–719.
  7. Schieve LA, Ferre C, Peterson HB, Macaluso M, Reynolds MA, Wright VC. Perinatal outcome among singleton infants conceived through assisted reproductive technology in the United States. Obstet Gynecol. 2004 Jun;103(6):1144–53.
  8. Dunietz GL, Holzman C, McKane P, Li C, Boulet SL, Todem D, et al. Assisted reproductive technology and the risk of preterm birth among primiparas. Fertil Steril. 2015 Apr;103(4):974-979.e1.
  9. Draper ES, Kurinczuk JJ, Abrams KR, Clarke M. Assessment of separate contributions to perinatal mortality of infertility history and treatment: a case-control analysis. Lancet. 1999 May 22;353(9166):1746-9.
  10. Romundstad LB, Romundstad PR, Sunde A, von Düring V, Skjaerven R, Gunnell D, et al. Effects of technology or maternal factors on perinatal outcome after assisted fertilisation: a population-based cohort study. Lancet. 2008 Aug 30;372(9640):737–43.
  11. Basso O, Baird DD. Infertility and preterm delivery, birthweight, and Caesarean section: a study within the Danish National Birth Cohort. Hum Reprod. 2003 Nov;18(11):2478–84.
  12. Elias D, Campaña H, Poletta FA, Heisecke SL, Gili JA, Ratowiecki J, et al. Preterm birth etiological pathways: a Bayesian networks and mediation analysis approach. Pediatr Res. 2021 Jul 19.
  13. Allouche M, Huissoud C, Guyard-Boileau B, Rouzier R, Parant O. Development and validation of nomograms for predicting preterm delivery. Am J Obstet Gynecol. 2011 Mar;204(3):242.e1-8.
  14. Mehta-Lee SS, Palma A, Bernstein PS, Lounsbury D, Schlecht NF. A Preconception Nomogram to Predict Preterm Delivery. Matern Child Health J. 2017 Jan;21(1):118–127.
  15. Lee KJ, Yoo J, Kim YH, Kim SH, Kim SC, Kim YH, et al. The Clinical Usefulness of Predictive Models for Preterm Birth with Potential Benefits: A KOrean Preterm collaboratE Network (KOPEN) Registry-Linked Data-Based Cohort Study. Int J Med Sci. 2020 Jan 1;17(1):1-12.
  16. Nelson SM, Lawlor DA. Predicting live birth, preterm delivery, and low birth weight in infants born from in vitro fertilisation: a prospective study of 144,018 treatment cycles. PLoS Med. 2011 Jan 4;8(1):e1000386.
  17. Esplin MS, Elovitz MA, Iams JD, Parker CB, Wapner RJ, Grobman WA, et al. Predictive Accuracy of Serial Transvaginal Cervical Lengths and Quantitative Vaginal Fetal Fibronectin Levels for Spontaneous Preterm Birth Among Nulliparous Women. JAMA. 2017 Mar 14;317(10):1047–1056.
  18. Grimes DA. The nomogram epidemic: resurgence of a medical relic. Ann Intern Med. 2008 Aug 19;149(4):273–5.
  19. Harrell F. Regression Modeling Strategies with applications to Linear Models, Logistic and Ordinal Regression, and Survival Analysis. 2nd ed, Switzerland: Springer; 2015.
  20. Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. 1st ed, New York: Springer; 2009.
  21. Schaaf JM, Mol BW, Abu-Hanna A, Ravelli AC. Ethnic disparities in the risk of adverse neonatal outcome after spontaneous preterm birth. Acta Obstet Gynecol Scand. 2012 Dec;91(12):1402–8.
  22. Cnattingius S, Villamor E, Johansson S, Edstedt Bonamy AK, Persson M, Wikström AK, et al. Maternal obesity and risk of preterm delivery. JAMA. 2013 Jun 12;309(22):2362–70.
  23. Ball SJ, Pereira G, Jacoby P, de Klerk N, Stanley FJ. Re-evaluation of link between interpregnancy interval and adverse birth outcomes: retrospective cohort study matching two intervals per mother. BMJ. 2014 Jul 23;349:g4333.
  24. Kazemier BM, Buijs PE, Mignini L, Limpens J, de Groot CJ, Mol BW, et al. Impact of obstetric history on the risk of spontaneous preterm birth in singleton and multiple pregnancies: a systematic review. BJOG. 2014 Sep;121(10):1197–208.
  25. Iams JD, Berghella V. Care for women with prior preterm birth. Am J Obstet Gynecol. 2010 Aug;203(2):89–100.
  26. Bertagnolli M, Luu TM, Lewandowski AJ, Leeson P, Nuyt AM. Preterm Birth and Hypertension: Is There a Link? Curr Hypertens Rep. 2016 Apr;18(4):28.
  27. Brown HK, Speechley KN, Macnab J, Natale R, Campbell MK. Biological determinants of spontaneous late preterm and early term birth: a retrospective cohort study. BJOG. 2015 Mar;122(4):491–9.
  28. Goldenberg RL, Culhane JF, Iams JD, Romero R. Epidemiology and causes of preterm birth. Lancet. 2008 Jan 5;371(9606):75–84.
  29. Steyerberg EW, Harrell FE Jr, Borsboom GJ, Eijkemans MJ, Vergouwe Y, Habbema JD. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol. 2001 Aug;54(8):774–81.