Prediction of pre-eclampsia in nulliparous women using routinely collected maternal characteristics: A model development and validation study

doi:10.21203/rs.2.16757/v1

Download PDF

Research article

Prediction of pre-eclampsia in nulliparous women using routinely collected maternal characteristics: A model development and validation study

https://doi.org/10.21203/rs.2.16757/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 06 Jan, 2020

Read the published version in BMC Pregnancy and Childbirth →

You are reading this older preprint version

Read the latest preprint version →

Background Guidelines recommend identifying early in pregnancy women at elevated risk of pre-eclampsia. The aim of this study was to develop and validate a pre-eclampsia risk prediction model for nulliparous women attending routine antenatal care “the Western Sydney (WS) model”; and to compare its performance with the National Institute of Health and Care Excellence (NICE) risk factor-list approach for classifying women as high-risk.

Methods This retrospective cohort study included all nulliparous women who gave birth in three public hospitals in the Western-Sydney-Local-Health-District, Australia 2011-2014. Using births from 2011-2012, multivariable logistic regression incorporated established maternal risk factors to develop and internally validate the WS model. The WS model was then externally validated using births from 2013-2014, assessing its discrimination and calibration. We fitted the final WS model for all births from 2011-2014, and compared its accuracy in predicting pre-eclampsia with the NICE approach.

Results Among 12,395 births to nulliparous women in 2011-2014, there were 293 (2.4%) pre-eclampsia events. The WS model included maternal age, body mass index, ethnicity, multiple pregnancy, family history of pre-eclampsia, autoimmune disease, chronic hypertension and chronic renal disease. In the validation sample (6201 births), the model c-statistic was 0.70 (95% confidence interval 0.65–0.75). The observed:expected ratio for pre-eclampsia was 0.91, with a Hosmer-Lemeshow goodness-of-fit test p-value of 0.20. In the entire study sample of 12,395 births, 374 (3.0%) women had a WS model-estimated pre-eclampsia risk ≥8%, the pre-specified risk-threshold for considering aspirin prophylaxis. Of these, 54 (14.4%) developed pre-eclampsia (sensitivity 18% (14–23), specificity 97% (97–98)). Using the NICE approach, 1173 (9.5%) women were classified as high-risk, of which 107 (9.1%) developed pre-eclampsia (sensitivity 37% (31-42), specificity 91% (91–92)). The final model showed similar accuracy to the NICE approach when using lower risk-threshold of ≥4% to classify women as high-risk for pre-eclampsia.

Conclusion The WS risk model that combines readily-available maternal characteristics achieved modest performance for prediction of pre-eclampsia in nulliparous women. The model did not outperform the NICE approach, but has the advantage of providing individualised absolute risk estimates, to assist with counselling, inform decisions for further testing, and consideration of aspirin prophylaxis.

Obstetrics & Gynecology

Antenatal care

Australia

Maternal Health

National Institute of Health and Care Excellence

Pre-eclampsia

Prediction

Risk assessment

Risk prediction model

Antenatal guidelines recommend routine risk assessment for pre-eclampsia in early pregnancy and low dose aspirin prophylaxis for women at elevated risk [1–4]. However, approaches for using established maternal risk factors to classify a woman’s risk of pre-eclampsia vary; and Australian guidelines do not offer an explicit approach [1,2].

The National Institute of Health and Care Excellence (NICE) in the United Kingdom lists moderate and high-risk factors for pre-eclampsia and recommends prophylaxis for women with one or more high-risk factors, or two or more moderate-risk factors. The listed risk factors are readily available maternal characteristics, so this approach has the practical advantage of being widely accessible, including in low-resource settings. However, its predictive performance has not yet been adequately validated in Australian women. Internationally, validation studies of the NICE approach have reported poor to moderate predictive performance [5–11]. One Australian validation study has reported good sensitivity for prediction of pre-eclampsia requiring delivery before 37 weeks, but was limited by a small sample size (n = 543, sensitivity 75% (95% confidence interval (CI) 35–97%), false positive rate 22%) [12].

A limitation of risk factor list approaches such as the NICE approach is that common risk factors such as older maternal age and higher body mass index (BMI) are dichotomised as ‘present’ or ‘absent’ ignoring any relationship between the level of these factors and pre-eclampsia risk. A risk prediction model that combines maternal risk factors, and includes all informative numerical values, has the potential advantage of providing an individualised estimate of pre-eclampsia risk that can assist patient counselling and inform clinical management decisions, rather than limiting categorisation to high-risk versus not high-risk. Although there has been a major research effort to develop risk prediction models for pre-eclampsia in early pregnancy, this research has largely focused on the use of specialised tests such as serum biomarkers and Doppler [13] which are not available in low-resource settings, nor routinely used in all public antenatal care clinics in Australia.

A woman’s prior pregnancy history of pre-eclampsia is one of the strongest risk factors for pre-eclampsia [4]. More challenging, is identifying women at high-risk of pre-eclampsia in their first pregnancy. Accurate identification and appropriate management of this group has the potential to provide clinical benefits for current and subsequent pregnancies. However, few studies of pre-eclampsia risk assessment tools have targeted this group.

The aims of this study are to develop and validate a pre-eclampsia risk prediction model for nulliparous women that can be used at the first antenatal visit using routinely collected maternal characteristics; and compare its performance with the NICE approach to inform the development of an Australian strategy for pre-eclampsia risk assessment and prevention.

Study design, setting and data source

We conducted a multi-hospital retrospective cohort study of nulliparous women giving birth between 1 January 2011 and 31 December 2014 at three public hospitals (Auburn, Blacktown/Mount-Druitt and Westmead) in the Western Sydney Local Health District (WSLHD). We included all nulliparous women with no previous pregnancies in the study sample. Study data were extracted from the ObstetriX database held by the hospital maternity units. The ObstetriX database collects information for all women attending their first antenatal visit to the discharge of mothers and their babies from the hospital [14].

We excluded women with missing information for pre-eclampsia, parity or candidate risk factors for pre-eclampsia. We also excluded women who were prescribed antiplatelet therapy in the first trimester given the effectiveness of these agents for preventing pre-eclampsia.

The primary outcome was the development of pre-eclampsia of any severity or timing. During the study period, pre-eclampsia was defined as hypertension with new onset of significant proteinuria ≥20 weeks’ gestation [15]. Secondary outcomes were early-onset pre-eclampsia (requiring delivery <34 weeks’ gestation) and preterm pre-eclampsia (requiring delivery <37 weeks’ gestation).

Study data

We extracted maternal socio-demographic characteristics (age, country-of-birth, primary language spoken at home and socioeconomic status classified from postcode using the Index of Relative Socio-economic Advantage and Disadvantage from the Socio-Economic Indexes for Areas (SEIFA)) [16], risk factors for pre-eclampsia (listed below); and study outcomes from the ObstetriX database.

For development of the Western Sydney (WS) risk model, we selected 12 candidate risk factors from antenatal guidelines [1-4]; published risk prediction models for pre-eclampsia [13], and our prior assessment of ethnicity [17] that were available in the ObstetriX database. These were: maternal age, body mass index (BMI), autoimmune disease, chronic hypertension, chronic renal disease, diabetes mellitus (type 1 or 2), multiple pregnancy, family history of pre-eclampsia, conception method, ethnicity, socio-economic status, and smoking status. We categorised ethnicity into two groups based on country of birth and primary language spoken at home (Australian/New Zealand-born English speakers; immigrants and non-English speakers). We categorised socioeconomic status into two groups using the SEIFA index (most disadvantaged SEIFA 1-2; most advantaged SEIFA 3-5).

For validation of NICE approach, we classified women with ≥1 high-risk factors or ≥2 moderate-risk factors as meeting the criteria of high-risk for aspirin prophylaxis [3], and refer to this group herein as “screen-positive”. All NICE listed risk factors relevant for nulliparous women are collected in the ObstetriX database.

Women with missing values for study variables were excluded from the analysis requiring that variable.

Statistical analysis

We assessed the distribution of risk factors measured as continuous variables (age, BMI) visually by plotting a probability distribution curve. We performed a descriptive analysis of maternal risk factors by assessing the frequency of categorical variables as a percentage in all women, then separately for women who developed pre-eclampsia and those that did not.

Model development and validation

We split the study sample into two groups for model development and temporal validation by year of infant birth (model development sample 2011-2012, validation sample 2013-2014). For model development, we used a two-stage approach. First, to optimize prediction of pre-eclampsia from age, BMI and other NICE-listed moderate risk factors we developed a WS ‘base’ model by excluding women with NICE-listed high-risk factors (autoimmune disease, chronic hypertension, chronic renal disease and diabetes (type 1 or 2)). The approach optimizes the model for use for the large majority of women who do not have high-risk factors; and would be sufficient in settings where women with high risk factors are already referred for further testing and management. Second, we developed a WS ‘full’ model for use in all women, by introducing women with high-risk factors into the development sample, retaining the base model risk score, and estimating coefficients for the high-risk factors. We internally validated the model in the development sample then externally validated it in the validation sample to assess the potential for model overfitting. If the model fit was satisfactory, we planned to refit the model predictors in the entire study sample to develop a WS ‘final’ model. Further details of these analyses are given below.

WS base model

To develop the WS base model, we included the following candidate predictors in a multivariable regression model: maternal age in years, BMI in kg/m², socioeconomic status (high vs low), conception method (assisted, by use of medications such as clomiphene or fertilization procedures including intrauterine insemination, in-vitro fertilization and intracytoplasmic sperm injection, vs natural conception), smoking status (current smokers vs non-smokers), multiple pregnancy (yes vs no); and family history of pre-eclampsia (yes vs no), and ethnicity (Australian/New Zealand-born English-speakers vs immigrants and non-English speakers).

To consider how to deal with the factors measured on a continuous scale (maternal age, BMI) in the model, we graphically examined their relationship with logit pre-eclampsia using a cubic splines approach. We assessed each factor and possible interactions between factors such as maternal age and multiple pregnancy by inspecting their effect size and p-value. We manually excluded factors that did not contribute to the model. We then developed the WS base model using the final predictors with no further stepwise procedures.

We performed internal validation of the model using the bootstrapping sampling technique to assess potential overfitting of the regression coefficients [18]. The mean c-statistic (corresponds to the area-under-the curve (AUC)) of the bootstrapping models was compared with the WS base model using the following formula: AUC=0.5*(Dxy+1), where: Dxy is Somer’s D. A well fitted model will report minimal optimism. We planned to adjust the regression coefficients by the resulting shrinkage factor if required [18].

To externally validate the base model, we applied the model algorithm in the validation sample after excluding women with high-risk factors. We calculated the predicted probability of pre-eclampsia for each individual woman by calculating the (log odds (Y)) and the odds ratio (ExpY) for pre-eclampsia and using the following equation: Probability=odds/1+odds and presented the distribution of predicted probabilities in a histogram. We assessed model discrimination by calculating the AUC and 95% CI. We assessed model calibration in this sample using the Hosmer-Lemeshow goodness-of-fit test, with p-value <0.05 indicating poor calibration [19]. We also calculated the ratio of observed: expected pre-eclampsia events and graphically assessed calibration by plotting observed risks on the y-axis against predicted risks on the x-axis for subgroups of patients categorized by their predicted probabilities (1-<2%, 2-<3%, 3-<4%, 4-<5%, 5-<8%, ≥8%) [20].

WS full model

To develop the WS full model, we introduced women with NICE-listed high-risk factors (autoimmune disease, chronic hypertension, chronic renal disease, diabetes) into the model development sample. We developed a multivariable regression model in this sample by retaining the WS base model risk score (Y) and adding the four additional predictors. We manually excluded high-risk factors that were not strongly or statistically significantly associated with pre-eclampsia. We followed the same approach outlined above for the base model to undertake internal and external validation of the full model to assess potential model overfitting.

WS final model

After assessment of over-fitting and calibration of the model in the validation sample, we refitted the WS base and full model in the entire study sample to develop the final WS model. First, we fitted the WS base model in women without high-risk factors. We retained the base model risk score and refitted the WS final model in the entire study sample to estimate the ß-coefficients for the high-risk factors and a new intercept. We presented the intercept and beta (log odd ratio) estimates and 95% CI for the intercept and each predictor.

Given the model is intended to be used to provide pre-eclampsia risk estimates to inform clinical decisions, we also assessed model sensitivity (95% CI), specificity (95% CI), positive predictive value (PPV, 95% CI), negative predictive value (NPV, 95% CI), positive likelihood ratio (LR) and negative LR to predict pre-eclampsia at specified risk thresholds for classifying high- vs low-risk. For our primary analysis, we used ≥8% as the risk threshold to classify high-risk as recommended by the United States Preventive Services Task Force (USPSTF) for commencing aspirin prophylaxis based on the prevalence of pre-eclampsia in trials demonstrating the effectiveness of aspirin [4] and from a publication recommending a 6-10% risk threshold for informing aspirin decisions [21]. We also examined the final model performance at 2%, 3%, 4%, 5% and 8% risk thresholds. We also reported model sensitivity at 5% and 10% fixed false positive rates (FPRs) to allow comparison with published models. For these analyses, we classified women as ‘true positive’ if they had a model-predicted risk above the cut-point and developed pre-eclampsia; false positive (predicted risk at/above cut-point and no pre-eclampsia); true negative (predicted risk below cut-point, no pre-eclampsia) or false negative (predicted risk below cut-point and pre-eclampsia).

In a secondary analysis, we assessed the discrimination of the WS final model to predict early-onset pre-eclampsia and preterm pre-eclampsia by estimating the AUC and sensitivity and specificity at ≥8% risk threshold in the entire study sample.

Model comparison with NICE approach

We compared the performance of the WS base and final models with the NICE approach by assessing the sensitivity and specificity, PPV, NPV, positive LR and negative LR of the NICE approach to predict pre-eclampsia in women without high-risk factors for comparison with the WS base model; and all women for comparison with the final model. For these comparisons, we assessed model sensitivity by fixing model specificity at the same level as the NICE approach. We report the model risk threshold that corresponds to this specificity level. For both the WS final model and NICE approach, we also calculated the number needed to treat (NNT) and the number needed to screen (NNS) [22] to avoid one pre-eclampsia event based on a RR reduction of 10% for aspirin reported from the Perinatal Antiplatelet Review of International Studies (PARIS) individual participant data meta-analysis of randomized controlled trials [23].

We performed a secondary analysis to assess the performance of the NICE approach for predicting preterm versus term pre-eclampsia (delivery ≥37 weeks’ gestation) and early-onset versus late-onset pre-eclampsia (delivery ≥34 weeks’ gestation) by estimating the OR and 95% CI using multinomial logistic regression and reporting a p-value for the Wald Chi Square test for the hypothesis of no difference in approach performance between the pre-eclampsia subgroups (preterm versus term; and early-onset versus late-onset pre-eclampsia).

We used SPSS version 25 and SAS version 9.3 statistical software and R for all analyses. The R rms package was used for model internal validation (bootstrapping). A p-value of <0.05 was regarded as statistically significant for all analyses.

We created an Excel spreadsheet to present the WS final model as a risk prediction calculator that can be used in the clinic to provide women with an individualised estimate of their probability of pre-eclampsia [24]. We followed the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines to report our methods and findings [20].

Participant characteristics

A total of 12,793 nulliparous women gave birth in the three hospital sites during the study period. After exclusions, 12,395 women were included for model development (n=6194) and validation (n= 6201) (Additional File 1: Fig. S1).

Participant characteristics are summarized in Table S1 (Additional File 2). Maternal age ranged from 14 to 46 years (mean 27.1, SD 4.9). Overall, 802 (6.5%) women were ≥35 years, 1494 (12.1%) were obese (BMI at first visit ≥30 kg/m²) and 308 (2.5%) had a multiple pregnancy. Two-thirds of women were immigrants or non-English speakers.

Pre-eclampsia incidence was 2.4% (n=293). Forty-six (0.4%) women had early-onset pre-eclampsia and 115 (0.9%) women had preterm pre-eclampsia.

WS base model

After exclusion of 133 women with high-risk factors, the WS base model was developed in 6061 women. Age was modelled as a continuous spline linear from 27 years; and BMI was modelled as a continuous spline linear from 26.3 kg/m² (Additional File 1: Fig. S2). After inspection of effect size and p-value for each factor, three factors (conception method, smoking and socioeconomic status) were manually removed. The five factors included in the base model were: maternal age, BMI, ethnicity, multiple pregnancy, and family history of pre-eclampsia (Additional File 2: Table S2).

Internal validation by bootstrapping sampling indicated very small optimism (Dxy 0.009) (Additional File 2: Table S3). The optimism corrected performance estimate of Dxy was 0.3149, which corresponds to an AUC of 0.66, which was similar to the apparent model performance (AUC 0.66).

After exclusion of 137 women with high-risk factors, 6064 women were included in the external validation sample for the base model. In this sample, the AUC was 0.68 (0.62-0.73) indicating modest discrimination. Base model predictions in the validation sample ranged between 1% and 49% (Interquartile range (IQR): 1.2-2.2%, Additional File 1: Fig. S3). Model calibration was good (Hosmer and Lemeshow goodness-of-fit test X²=6.87; p=0.44); observed:expected ratio of pre-eclampsia events=0.91. The calibration plot showed an acceptable level of calibration, however, at predicted probabilities for pre-eclampsia higher than 8%, it overestimated the risk of pre-eclampsia (Fig. 1).

WS full model

The full model was built in the entire model development sample (n= 6194). Of the four high-risk factors considered, diabetes (type 1 or 2) was eliminated from the model (Additional File 2: Table S2).

Internal validation of the full model by bootstrapping sampling indicated very small optimism (Dxy 0.0087) (Additional File 2: Table S3). The optimism corrected performance estimate of Dxy was 3.716000e-01, which corresponds to an AUC of 0.69 which is similar to the apparent model performance (AUC 0.69).

In the external validation sample of 6201 women, the AUC was 0.70 (0.65-0.75) indicating good discrimination. Full model predictions in the validation sample ranged between 1% and 86% (IQR: 1.3-2.3%, Additional File 1: Fig. S4). Model calibration was good (Hosmer and Lemeshow goodness-of-fit test X²=9.90; p=0.20); observed:expected ratio of pre-eclampsia events=0.91. The calibration plot showed an acceptable level of calibration, although as observed for the base model, at predicted probabilities for pre-eclampsia higher than 8%, it overestimated the risk of pre-eclampsia (Fig. 2).

WS final model

Given the model did not demonstrate over-fitting and was adequately calibrated in the validation sample, the model was refitted in the entire study sample to produce the WS final model.

When applied to the entire study sample, the AUC was 0.70 (0.66-0.73) indicating good discrimination (Additional File 1: Fig. S5). The WS final model classified 374 (3%) of women at ≥8% risk of pre-eclampsia (Table 1). At this risk threshold, 54 (14.4%) women develop pre-eclampsia with model sensitivity 18% (14-23%), specificity 97% (97-98%), PPV 14% (11-18%), and NPV 98.0% (97.8-98.3%) The NNT was 69 and NNS was 2295. The performance of the model at lower risk thresholds is shown in Table 1. At fixed 5% FPR, corresponding to a 5.3% risk threshold, the model classified 6% of women to be at high-risk of pre-eclampsia and the sensitivity was 30% (25-36). At a fixed 10% FPR, corresponding to a 3.5% risk threshold, the model classified 11% of women to be at high-risk of pre-eclampsia and the sensitivity was 40% (95% CI 35-46).

The WS final model can be used to calculate a woman’s probability of developing pre-eclampsia as follows:

Y = - 7.786 + 0.052 * maternal age in years from age 27 years + 0.078 * BMI in kg/m² from 26.3 kg/m² + 0.525 if Australian/New Zealand born English-speaker + 1.318 if multiple pregnancy + 1.740 if family history of pre-eclampsia + 1.512 if autoimmune disease + 1.545 if chronic hypertension + 1.494 if chronic renal disease

Odds = Exp^{Y (final prediction score)}

Pre-eclampsia probability = Odds / (1 + Odds)

A ‘WS pre-eclampsia risk prediction tool’ has been created as an Excel spreadsheet that can be used in the clinic to perform these calculations automatically using information entered about a woman’s risk factors (Additional File 3). To illustrate, if a 24 year-old Australian nulliparous woman presents for her first antenatal visit with a BMI of 25 kg/m² and no other risk factor, inputting this value into the Excel spreadsheet gives an estimate of the probability of pre-eclampsia of 1.7% (as shown in Fig. 3). If the same woman also has a family history of pre-eclampsia, addition of this information into the tool gives a revised probability of pre-eclampsia of 8.9%.

WS model prediction of early-onset and preterm pre-eclampsia

For prediction of early-onset pre-eclampsia, the WS final model had an AUC of 0.72 (95% CI 0.63-0.81). At the ≥8% risk threshold, 17 (4.5%) women developed pre-eclampsia with a model sensitivity of 37% (25-51%) and specificity 97.1% (96.8-97.4%) (Additional File 2: Table S4). At a fixed 5% FPR, corresponding to a 5.6% risk threshold, model sensitivity was 43% (95% CI 30-58). At a fixed 10% FPR, corresponding to a 3.7% risk threshold, model sensitivity was 48% (95% CI 34-62).

For prediction of preterm pre-eclampsia, the WS final model had an AUC of 0.74 (95% CI 0.68-0.79). At the ≥8% risk threshold, 36 (9.6%) women developed pre-eclampsia with a model sensitivity of 31% (24-40%) and specificity 97% (97-98%) (Additional File 2: Table S4). At a fixed 5% FPR, model sensitivity was 44% (95% CI 36-53). At a fixed 10% FPR, model sensitivity was 50% (95% CI 41-59).

The NICE approach

Using the NICE approach in the subset of women without NICE-listed high-risk factors (n=12,125, pre-eclampsia events=260), 903 (7.4%) of women were classified screen-positive with a sensitivity of 28% (23-34%) and specificity 93% (93-94%) (Table 2). Among women classified screen-positive, 74 (8.2%) developed pre-eclampsia. For comparison, at the same specificity, the WS base model predicted 7.5% of women in this sample at ≥3.9% risk and achieved similar sensitivity (29%; 24-35%) to the NICE approach. The NNT for women classified by the model at the ≥3.9% risk threshold was 120 and for NICE was 122. The NNS for the model was 1595, compared to 1639 for women using the NICE approach.

Using the NICE approach in the entire study sample including women with high-risk factors (n=12,395, pre-eclampsia events=293), 1173 (9.5%) of women were classified screen-positive with a sensitivity of 37% (31-42%), specificity 91% (91-92%) (Table 2). Among women classified screen-positive, 107 (9.1%) developed pre-eclampsia. For comparison, at the same specificity, the WS final model predicted 9.7% of women in this sample at ≥3.8% risk, and achieved a similar sensitivity (38%; 33-44%) to the NICE approach. The NNT for women classified by the model at the ≥3.8 risk threshold was 108 and for NICE was 110. The NNS for the model was 1107, compared to 1158 using the NICE approach.

The accuracy of the NICE approach was higher to predict early-onset and preterm pre-eclampsia (Additional File 2: Table S5).

Main findings

The WS risk model combines eight routinely collected maternal characteristics to estimate the probability of pre-eclampsia in early pregnancy for women attending antenatal care for their first pregnancy. The model demonstrated good calibration for risk prediction overall, but model accuracy was modest when using the pre-specified 8% risk threshold to predict which women will develop pre-eclampsia. While very few women who did not develop pre-eclampsia had a model-predicted risk above 8% (specificity 97%, 97–98%), only one in five women who developed pre-eclampsia had a risk estimate above this level (sensitivity 18%, 95% CI 14–23%). Model sensitivity was higher when assigning a lower risk threshold to classify women as high-risk. For example, one in three women who developed pre-eclampsia had a predicted risk ≥4% (sensitivity 36% (31–41%), specificity 92% (91–92%)). The observed risk of pre-eclampsia was 10% in this group of women.

In contrast to the WS model algorithm, the NICE approach classifies all nulliparous women as moderate-risk, and those with any of the eight additional moderate or high-risk factor/s as high-risk. Based on our study sample, nearly 10% of nulliparous women screened ‘positive’ using the NICE approach, with a sensitivity and specificity for prediction of pre-eclampsia similar to the WS model when at a 3.8% risk threshold (NICE sensitivity 37% (95% CI 31–42%), specificity 91% (91–92%)).

The sensitivity of the WS model was higher for prediction of early-onset and preterm pre-eclampsia than any-onset pre-eclampsia with similar specificity. While these findings might suggest some maternal risk factors are more strongly associated with early-onset and pre-term pre-eclampsia than late-onset pre-eclampsia, an alternative explanation may be that women with risk factors for pre-eclampsia may be more likely to have an induced birth before 40 weeks, thus removing the risk of pre-eclampsia at term. Our finding that the median gestational age at birth for women classified as high-risk (≥1 high-risk factors listed by NICE guideline) was 38.6 weeks’ gestation compared to 39.5 weeks’ gestation for those not classified as high-risk provides some support for this explanation.

The model includes all NICE-listed risk factors except diabetes (type 1 or 2) which did not contribute to the model with an adjusted odds ratio of 0.71 (95% CI 0.22–2.31). One possible interpretation of these findings is that women with well managed diabetes (type 1 or 2) are not at higher risk of pre-eclampsia. However, few women in our study sample (<1%) had diabetes recorded and the wide 95% confidence interval does not exclude the possibility of diabetes as a predictor of pre-eclampsia.

Comparison with existing evidence

Despite the potential clinical benefits of targeting nulliparous women for pre-eclampsia prevention, most validation studies of risk assessment tools do not report performance separately for nulliparous women [7–12]. Two pre-eclampsia risk prediction models based on routinely collected maternal factors identified from our previous systematic review [13] have been assessed in nulliparous women [6,25]. Both included more risk factors than the WS model and NICE approach. First, North et al (2011) recruited 3529 singleton nulliparous women from five centres in Australia, New Zealand, Ireland, and the United Kingdom (UK) and developed a model that included 11 clinical predictors and mean arterial pressure [25]. Model calibration was only reported in the development sample. Model accuracy was not assessed at a clinically-defined risk threshold. At the study-defined cut-off of 25% FPR, model sensitivity (61% (54–68%) [25] was higher than our WS model sensitivity (50% (44–56%) at the same cut-off. This study adds to evidence that combining blood pressure measurements to maternal factors improves prediction of pre-eclampsia in early pregnancy [26]. Unfortunately, blood pressure measurements were not available in the ObstetriX database to allow us to assess the impact on WS model performance.

The second study reported by Wright et al assessed the accuracy of a model based on nine maternal factors (maternal age, height, interpregnancy interval, method of conception, chronic hypertension, systemic lupus erythematosus or antiphospholipid syndrome, weight, family history of pre-eclampsia and diabetes mellitus) and ethnicity (white, Afro-Caribbean, South Asian, East Asian and mixed)) [6]. The study included 59,947 singleton nulliparous women recruited from two hospitals in UK. The Wright model was internally validated by 5-fold cross validation. The authors assessed model accuracy at the risk cut-off corresponding to the FPR estimated for the NICE approach and reported a model sensitivity of 31% (28.8–33.3%) at 11.5% FPR which compared favourably to the sensitivity of the NICE approach in the same sample (24.8%, 22.7–26.9%) [6]. The WS final model achieved higher sensitivity to predict pre-eclampsia at the same cut-off (sensitivity 40% (35–46) at 10% FPR).

The Wright model was recently validated in a sample of 4184 nulliparous women recruited from single hospital in the UK [27] to predict preterm pre-eclampsia (preterm pre-eclampsia rate = 0.7%, and compared to the NICE approach. The authors reported higher sensitivity than the original study to predict preterm pre-eclampsia (57.1%; 95% CI 37.5–74.8% vs 35.9; 31.5–40%), with slightly lower FPR (8.8% vs 11.5%), and similar performance to the NICE approach [27]. In comparison, the WS final model showed slightly lower sensitivity to predict preterm pre-eclampsia with narrower confidence interval than the validation study of the Wright model (50%; 41–59% vs 57.1%; 37.5–74.8%) at 8.8% FPR. The accuracy of the NICE approach to predict preterm pre-eclampsia was similar to our present study in Australian women.

One other Australian study has assessed the NICE approach, although the authors did not report results separately for nulliparous women and the sample size of 543 women did not allow precise estimates of performance [12]. We have previously assessed the NICE approach in nulliparous women enrolled in international trials of aspirin prophylaxis who had a relatively high-risk of pre-eclampsia (4.8%) [5]. In this trial sample, the NICE approach had a lower sensitivity (8.9%, 10.1%, respectively) and higher specificity (97.2%, 96.6%, respectively) than the present study [5]. However, trial data were not available for BMI and not adequately reported for family history which may have resulted in an underestimation of the NICE approach performance [5]. Our present study findings highlight the importance of BMI as a risk factor with obesity (BMI ≥30 kg/m²) recorded for 12% women; and we observed a high rate of pre-eclampsia (7%) for this group.

Two international studies have reported on the performance of the NICE approach in nulliparous women, although both excluded women with multiple pregnancies [6,8]. Both studies reported slightly lower estimates of sensitivity and specificity than the present study. First, a UK study reported the NICE approach classified 12% of nulliparous women as high-risk (similar to 10% in the present study) with a sensitivity of 25% (95% CI 23–27%) and specificity (88%) for prediction of any-onset pre-eclampsia (overall pre-eclampsia rate 2.8%) [6]. Second, a study with smaller sample that included women from the UK, Ireland, New Zealand and Australia reported a sensitivity of 31% and a specificity 88% (calculated from data tabulated) for prediction of preterm pre-eclampsia (overall pre-eclampsia rate 1.3%) [8]. Together with our findings, these results provide consistent evidence that the NICE approach correctly classifies approximately one-quarter to one-third of women who will develop pre-eclampsia as high-risk.

Limitations

The main limitation of our study is that we collected study data retrospectively which might lead to under-ascertainment of study variables [28]. Under-ascertainment of risk factors may result in underestimation of their predictive performance. Another limitation is that we did not include births from private hospitals to assess whether hospital setting, or maternal factors related to delivery in a private hospital, impacts the risk of developing pre-eclampsia which would reduce the applicability of the WS model to women giving birth outside public hospitals. Finally, no women in the study sample had more than three high-risk factors recorded, thus model predictions cannot be extended to such women.

Clinical implications

The main intended benefit of pre-eclampsia risk assessment is to correctly identify high-risk women who will benefit from prophylaxis and/or further testing and management by a specialist to prevent the condition or its complications. Although we demonstrate the predictive performance of the WS model and NICE approach is modest compared to that reported for strategies using tests such as uterine artery Doppler and serum biomarkers [13], our findings support the clinical value of these approaches in settings where such specialised tests are not routinely available. For example, the NICE approach can be recommended for nulliparous women, if one considers managing eleven women as high-risk for every one woman who will develop pre-eclampsia is acceptable (PPV 9% (8–11%)). Acceptability will depend on the level of concern about the potential adverse effects of prophylactic agents, such as aspirin [29]. In settings where specialised tests are available, the NICE approach may have a role to select high-risk women for further testing. Under this strategy, our estimate of NPV at 98% indicates that for every 50 women classified as low-risk using the NICE approach, one woman will develop pre-eclampsia and miss the opportunity for further testing.

Compared to the NICE approach, the WS model has the advantage of providing an estimate of the probability of pre-eclampsia to assist patient counselling and guide clinical practice decisions about prophylaxis and/or referral for further testing. In particular, the model allows risk prediction based on maternal age and BMI, which are more common than high-risk factors such as chronic hypertension, whereas the NICE approach is restricted to using a single different cut-off for these two factors. We provide the model algorithm in an Excel spreadsheet that can be readily incorporated into the first antenatal clinic visit. At a population level, setting the risk threshold of 4% to guide decisions such as aspirin prophylaxis appeared appropriate with a 10% observed risk of pre-eclampsia in this group of women.

Research implications

The WS model should be validated outside of the WSLHD before implementing for nulliparous women in other settings. For example, our findings that immigrant women are at lower risk of pre-eclampsia may not apply outside of Australia.

Further research is needed to investigate whether the inclusion of individual ethnic groups may improve model prediction in Australia’s highly culturally diverse antenatal population, and to investigate the value of including maternal blood pressure measurement in the model.

In settings such as Australia, where tests such as Doppler ultrasound and serum biomarkers are available for pre-eclampsia risk assessment but not yet widely used, broader implementation will depend on cost-effectiveness relative to risk assessment without these tests. Our findings of the accuracy of simple risk tools will be valuable to inform these cost-effectiveness analyses. The WS model can also be used as a foundation on which to build a specialised risk prediction model.

A risk prediction model that incorporates guideline-listed risk factors and ethnicity provides modest performance for predicting pre-eclampsia in Australian nulliparous women. The model did not outperform the NICE approach, but has the advantage of providing individualised risk estimates over the NICE guideline to assist patient counselling and inform decisions for further testing and prophylaxis.

aOR: Adjusted odds ratio; AUC: Area under the curve; BMI: Body mass index; CI: confidence interval; FPR: False positive rate; IQR: Interquartile range; LR: Likelihood ratio; NICE: National Institute for Health and Clinical Excellence; NNS: Number needed to screen; NNT: number needed to treat; NPV: Negative predictive value; PARIS: Perinatal Antiplatelet Review of International Studies; PE: Pre-eclampsia; PPV: Positive predictive value; ROC: Receiver operating characteristics; SD: Standard deviation; SEIFA: Socio-economic Advantage and Disadvantage from the Socio-Economic Indexes for Areas; SOMANZ: Society of Obstetric Medicine of Australia and New Zealand; TRIPOD: Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis; USPSTF: United States Preventive Services Taskforce; WSLHD: Western Sydney Local Health District; WS: Western Sydney

Acknowledgements

We thank the data custodians of the WSLHD ObstetriX database for their valuable help in collecting and providing the data.

Authors’ contributions

ZA and SL contributed to development of research question, data analysis interpretation of results and writing. HM H contributed to data analysis and interpretation of results. GJ, IM, JR, LA contributed to development of research question, interpretation of results and writing. All authors approved the final version of the manuscript.

Funding

Ziad TA Al-Rubaie is supported by an Australian Research Training Program and Collaborative Research Network PhD Scholarship.

Availability of data and materials

The data and analyses that support the findings of this study are available on request from the co-author SL; email: [email protected]. The data are not publicly available due to WSLHD regulations.

Ethics approval and consent to participate

Study data were provided de-identified with a waiver for patient consent approved by the University of Notre Dame Australia and WSLHD ethic committees.

(Notre Dame HREC approval number: 016071S; and WSLHD HREC approval number: LNR/16/WMEAD/239, LNR SSA/16/WMEAD/256.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

1. SOMANZ (Society of Obstetric Medicine of Australia and New Zealand) 2014. The SOMANZ Guidelines for the management of hypertensive disorders of pregnancy. Sydney: SOMANZ.

Department of Health (2018) Clinical Practice Guidelines: Pregnancy Care.

Canberra: Australian Government Department of Health.

National Institute for Health and Care Excellence. NICE quality standard 35. Hypertension in pregnancy. NICE, 2013.

LeFevre ML. Low–dose aspirin use for the prevention of morbidity and mortality from preeclampsia: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med 2014;161:819–26.

Al-Rubaie ZT, Askie LM, Hudson HM, Ray JG, Jenkins G, Lord SJ. Assessment of NICE and USPSTF guidelines for identifying women at high-risk of pre-eclampsia for tailoring aspirin prophylaxis in pregnancy: An individual participant data meta-analysis. European Journal of Obstetrics & Gynecology and Reproductive Biology. 2018 Oct 1;229:159-66.

Wright D, Syngelaki A, Akolekar R, Poon LC, Nicolaides KH. Competing risks model in screening for preeclampsia by maternal characteristics and medical history. Am J Obstet Gynecol 2015;213:62.e1–10.

O'Gorman N, Wright D, Poon LC, Rolnik DL, Syngelaki A, de ALVARADO M, et al. Multicenter screening for pre-eclampsia by maternal factors and biomarkers at 11-13 weeks' gestation: comparison with NICE guidelines and ACOG recommendations. Ultrasound in Obstetrics & Gynecology. 2017 Jun 1;49(6):756-60.

Myers JE, Kenny LC, McCowan LM, Chan EH, Dekker GA, Poston L, et al. Angiogenic factors combined with clinical risk factors to predict preterm pre‐eclampsia in nulliparous women: a predictive test accuracy study. BJOG: An International Journal of Obstetrics & Gynaecology. 2013 Sep 1;120(10):1215-23.

Tan MY, Wright D, Syngelaki A, Akolekar R, Cicero S, Janga D, Singh M, Greco E, Wright A, Maclagan K, Poon LC. Comparison of diagnostic accuracy of early screening for pre‐eclampsia by NICE guidelines and a method combining maternal factors and biomarkers: results of SPREE. Ultrasound in Obstetrics & Gynecology. 2018 Jun;51(6):743-50.

Poon LC, Rolnik DL, Tan MY, Delgado JL, Tsokaki T, Akolekar R, Singh M, Andrade W, Efeturk T, Jani JC, Plasencia W. ASPRE trial: incidence of preterm pre‐eclampsia in patients fulfilling ACOG and NICE criteria according to risk by FMF algorithm. Ultrasound in Obstetrics & Gynecology. 2018 Jun;51(6):738-42.

Rocha RS, Alves JA, Júnior EA, Martins WP, Vasconcelos CT, Costa FD, Oriá MO. Comparison of three algorithms for prediction preeclampsia in the first trimester of pregnancy. Pregnancy hypertension. 2017 Oct 1;10:113-7.

Al-Amin A, Rolnik DL, Black C, White A, Stolarek C, Brennecke S, da Silva Costa F. Accuracy of second trimester prediction of preterm preeclampsia by three different screening algorithms. Australian and New Zealand Journal of Obstetrics and Gynaecology. 2018 Apr 1;58(2):192-6.

Al-Rubaie ZT, Askie LM, Ray JG, Hudson HM, Lord SJ. The performance of risk prediction models for pre-eclampsia using routinely collected maternal characteristics and comparison with models that include specialised tests and with clinical guideline decision rules: a systematic review. BJOG: An International Journal of Obstetrics & Gynaecology. 2016 Aug 1;123(9):1441–52.

Centre for Epidemiology and Evidence. New South Wales Mothers and Babies 2017. Sydney: NSW Ministry of Health, 2018.

Brown MA, Lindheimer MD, de Swiet M, et al. The classification and diagnosis of the hypertensive disorders of pregnancy: statement from the International Society for the Study of Hypertension in Pregnancy (ISSHP). Hypertens Pregnancy 2001;20:IXeXIV.

Census of Population and Housing: Socio-Economic Indexes for Areas (SEIFA), Australia, 2016 is available at Australian Bureau of Statistics (ABS) websites:

http://www.abs.gov.au/ausstats/[email protected]/Lookup/2033.0.55.001main+features100042011

https://www.abs.gov.au/AUSSTATS/[email protected]/Lookup/2033.0.55.001Main+Features12016?OpenDocument

Al-Rubaie ZT, Hudson HM, Jenkins G, Mahmoud I, Ray JG, Askie LM, Lord SJ. The association between ethnicity and pre‐eclampsia in Australia: A multicentre retrospective cohort study. Australian and New Zealand Journal of Obstetrics and Gynaecology. 2019 Oct 3; Doi.org/10.1111/ajo.13069.

Steyerberg E. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating (Statistics for Biology and Health). New York: Springer, 2010.

Hosmer D, Lemeshow S, Sturdivant RX. Applied Logistic Regression. New York, NY: A Wiley-Interscience Publication.

Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Annals of internal medicine. 2015 Jan 6;162(1):W1-73.

Bartsch E, Park AL, Kingdom JC, Ray JG. Risk threshold for starting low dose aspirin in pregnancy to prevent preeclampsia: an opportunity at a low cost. PloS one. 2015;10(3):e0116296.

Rembold CM. Number needed to screen: development of a statistic for disease screening. Bmj. 1998 Aug 1;317(7154):307-12.

Askie L, Duley L, Henderson-Smart D, Stewart L, on behalf of the PARIS Collaborative Group. Antiplatelet agents for prevention of pre-eclampsia: a meta-analysis of individual patient data. The Lancet, 2007. 369(9575): p. 1791–1798.

Bonnett LJ, Snell KI, Collins GS, Riley RD. Guide to presenting clinical prediction models for use in clinical settings. bmj. 2019 Apr 17;365:l737.

North RA, McCowan LM, Dekker GA, Poston L, Chan EH, Stewart AW, Black MA, Taylor RS, Walker JJ, Baker PN, Kenny LC. Clinical risk prediction for pre-eclampsia in nulliparous women: development of model in international prospective cohort. Bmj. 2011 Apr 7;342:d1875.

Poon LC, Kametas NA, Valencia C, Chelemen T, Nicolaides KH. Hypertensive disorders in pregnancy: screening by systolic diastolic and mean arterial pressure at 11–13 weeks. Hypertension in Pregnancy. 2011 Feb 1;30(1):93-107.

Sovio U, Smith GC. Evaluation of a simple risk score to predict preterm pre‐eclampsia using maternal characteristics: a prospective cohort study. BJOG: An International Journal of Obstetrics & Gynaecology. 2019 Jul 1.

Chen JS, Roberts CL, Simpson JM, Ford JB. Prevalence of pre‐eclampsia, pregnancy hypertension and gestational diabetes in population‐based data: Impact of different ascertainment methods on outcomes. Australian and New Zealand Journal of Obstetrics and Gynaecology. 2012 Feb;52(1):91-5.

Roberge S, Bujold E, Nicolaides KH. Meta-analysis on the effect of aspirin use for prevention of preeclampsia on placental abruption and antepartum hemorrhage. American journal of obstetrics and gynecology. 2018 Jan 3; 3218:483.

Table 1 Performance of the WS final model at different risk thresholds, entire study sample (N=12,395)

Risk threshold	PE/n ≥threshold <threshold		Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)	Positive LR (95%CI)	Negative LR (95%CI)
2%	196/5180	97/7215	67% (61-72%)	59% (58-60%)	4% (3-4%)	99 % (98-99%)	1.62 (1.49-1.77)	0.56 (0.48-0.66)
3%	138/1731	155/10664	47% (41-53%)	87% (86-87%)	8% (7-9%)	99 % (98-99%)	3.58 (3.14-4.07)	0.61 (0.55-0.68)
4%	105/1098	188/11297	36% (31-41%)	92% (91-92%)	10% (8-11%)	98 % (98-99%)	4.37 (3.71-5.15)	0.70 (0.64-0.76)
5%	92/776	201/11619	31% (26-37%)	94% (94-95%)	12% (10-14%)	98.3 % (98.0-98.5%)	5.56 (4.62-6.68)	0.73 (0.67-0.79)
8%	54/374	239/12021	18% (14-23%)	97% (97-98%)	14% (11-18%)	98.0% (97.8-98.3%)	6.97 (5.35-9.08)	0.84 (0.79-0.89)

CI = Confidence interval, LR = Likelihood ratio, NPV = Negative predictive value, PE = Pre-eclampsia, PPV = Positive predictive value, WS = Western Sydney

Table 2 Comparison of the NICE approach versus the WS model for predicting pre-eclampsia in nulliparous women

Approach

Threshold

n(%)

≥threshold

Sensitivity

(95%CI)

Specificity

(95%CI)

PPV

(95%CI)

NPV

(95%CI)

Positive LR

(95%CI)

Negative LR

(95%CI)

NNT

NNS

Women without high-risk factors, n=12,125

NICE approach*

Screen-positive

903 (7.4)

28%

(23-34%)

93%

(93-94%)

(7-10%)

98%

(98-99%)

4.07

(3.32-4.99)

0.77

(0.71-0.83)

122

1639

WS base model

3.9%**

912 (7.5)

29%

(24-35%)

93%

(92-93%)

(7-10%)

98%

(98-99%)

4.15

(3.40-5.07)

0.76

(0.70-0.82)

120

1595

All women, n=12,395

NICE approach*

Screen-positive

1173 (9.5)

37%

(31-42%)

91%

(91-92%)

(8-11%)

98%

(98-99%)

4.15

(3.53-4.87)

0.70

(0.64-0.76)

110

1158

WS final model

3.8%**

1205 (9.7)

38%

(33-44%)

91%

(90-91%)

(8-11%)

98%

(98-99%)

4.23

(3.62-4.95)

0.68

(0.62-0.74)

108

1107

*NICE approach screen-positive if ≥1 high-risk factors or ≥2 moderate-risk factors. High-risk factors included in this analysis: renal disease, diabetes (type 1 or 2) and chronic hypertension and autoimmune disease. Moderate-risk factors included in this analysis: first pregnancy, age ≥40 year, body mass index ≥35 kg/m², family history of pre-eclampsia and multiple pregnancy.

**Model risk cut-off when the model specificity is fixed at the level of the NICE approach

CI = Confidence interval, LR = Likelihood ratio, NICE = National Institute for Health and Care Excellence, NNS = Number needed to screen, NNT = Number needed to treat, NPV = Negative predictive value. PPV = Positive predictive value, WS = Western Sydney

Download PDF

Journal Publication

published 06 Jan, 2020

Read the published version in BMC Pregnancy and Childbirth →

Editorial decision: Minor revision
03 Dec, 2019
Review #3 received at journal
16 Nov, 2019
Review #2 received at journal
16 Nov, 2019
Reviewer #3 agreed at journal
14 Nov, 2019
Reviewer #2 agreed at journal
10 Nov, 2019
Review #1 received at journal
08 Nov, 2019
Reviewers invited by journal
04 Nov, 2019
Reviewer #1 agreed at journal
04 Nov, 2019
Submission checks completed at journal
30 Oct, 2019
Editor assigned by journal
29 Oct, 2019
Editor invited by journal
28 Oct, 2019
First submitted to journal
26 Oct, 2019

You are reading this older preprint version

Read the latest preprint version →

Prediction of pre-eclampsia in nulliparous women using routinely collected maternal characteristics: A model development and validation study

Status:

Journal Publication

Version 1

Abstract

Figures

Introduction

Methods

Results

Discussion

Main findings

Comparison with existing evidence

Limitations

Clinical implications

Research implications

Conclusions

Abbreviations

Declarations

References

Tables

Supplementary Files

Status:

Journal Publication

Version 1