A Sequential Machine Learning Model for Identifying At-risk NASH by Combining Liver Stiffness Measurement and Protein Biomarkers

doi:10.21203/rs.3.rs-2118271/v1

Download PDF

Research Article

A Sequential Machine Learning Model for Identifying At-risk NASH by Combining Liver Stiffness Measurement and Protein Biomarkers

https://doi.org/10.21203/rs.3.rs-2118271/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background: A number of protein biomarkers have been proposed for the assessment of non-alcoholic fatty liver disease (NAFLD), but few have been externally validated and directly compared. The aim of this study was to compare the diagnostic accuracies of 13 protein biomarkers and develop a biomarker-based machine learning algorithm to predict at-risk non-alcoholic steatohepatitis (NASH) in patients with NAFLD.

Methods: 281 NAFLD patients had blood biomarker tested within one week before liver biopsy. We used three machine learning methods to select biomarkers in training (70%) and testing (30%) datasets, and then input selected features into a logistic regression model to predict at-risk NASH (NAFLD activity score ≥4 with at least 1 point in each component and fibrosis stage ≥2).

Results: Among 13 protein biomarkers tested, growth differentiation factor-15 (GDF-15) and Pro-C3 had the highest accuracy for at-risk NASH and advanced fibrosis (F3-4), respectively. All three machine learning models selected GDF-15, Pro-C3, and tissue inhibitor matrix metalloproteinase 1 as the best predictors of at-risk NASH, and the logistic regression FibNASH-3 model had an area under receiver-operating characteristics curve of 0.784. Using LSM ≥8kPa as the first step, the sequential model had a 79.2% positive predictive value for at-risk NASH. During a mean follow-up of 9.7 years, 3% and 16.7% of patients with low and high sequential model score developed hepatocellular carcinoma and cirrhotic complications, respectively (P<0.01).

Conclusions: The study provides a head-to-head comparison of 13 protein biomarkers. FibNASH-3 has the ability to accurately identify at-risk NASH in NAFLD patients and predict the development of liver-related events.

Growth differentiation factor-15

transient elastography

non-alcoholic steatohepatitis

at-risk non-alcoholic steatohepatitis

liver-related events

Non-alcoholic fatty liver disease (NAFLD) is the most common chronic liver disease and now considered a global epidemic affecting 25% of the population worldwide.¹. NAFLD is divided into the non-alcoholic fatty liver (NAFL) and non-alcoholic steatohepatitis (NASH). NASH is the more aggressive form of NAFLD. Compared to NAFL, people with NASH have a more rapid progression to advanced liver fibrosis.² With a persistent liver injury, patients with NASH can progress to cirrhosis, which is the leading risk factor for the development of hepatocellular carcinoma (HCC).³ In the United States, NASH cirrhosis is one of the leading causes of liver transplantation.⁴ The US Food and Drug Administration (FDA) guidance regarding drug development in non-cirrhotic NASH with liver fibrosis was published in 2018 with primary aims to facilitate clinical development.⁵ The target population for drug trials should have a histological diagnosis of NASH with liver fibrosis based on a NAFLD activity score (NAS) ≥ 4 with at least 1 point each in inflammation and ballooning in addition to a NASH Clinical Research Network fibrosis score of stage 2 or 3 prior to enrolment (referred to as at-risk NASH). Currently, liver biopsy is still the gold standard for diagnosing and staging of NASH. However, the risk of performing this invasive procedure is a hurdle for enrolment in many clinical studies. To address this challenge, the strategy is to use of non-invasive tools to pre-screen subjects to better predict those who are most likely to fulfil the histological criteria of a clinical study and therefore should proceed to a liver biopsy. Currently, there are few specific non-invasive tests developed for the detection of at-risk NASH.^6–9 Identifying and validating the novel non-invasive NASH biomarkers (single or combined markers) is an area of continued focus for the industry and an area of considerable unmet need.

In recent years, machine learning has been widely applied in biomarker discovery.^{10, 11} Machine learning is a branch of artificial intelligence to apply mathematical approaches to train a model to learn from data for a particular task. The relevant machine learning techniques for biomarker discovery are classification and feature selection. Classification is a form of supervised learning where the algorithm is trained with labelled samples each represented by a set of features. The task is to learn a function that can predict the label of a sample from its features. Feature selection is usually applied in many real-world applications of classification by directly selecting a subset of relevant features for the model construction.^{12, 13} It retains only useful features via removing irrelevant and redundant features. Feature selection produces multiple benefits including improving classification performance and reliability, speeding up the data mining algorithm, and a better model readability and interpretability.

In this study, we compared the performance of 13 protein biomarkers for the diagnosis of at-risk NASH in a well-characterized cohort of patients with biopsy-proven NAFLD and applied machine learning feature selection methods to identify the consensus protein biomarkers to classify at-risk NASH patients. We then developed a sequential model for screening at-risk NASH by combing liver stiffness measurement (LSM) and a machine learning model of protein biomarkers to improve the screening success rate and reduce the need for liver biopsy.

Study population and study design

This was a post hoc analysis of a prospective cohort study at the Prince of Wales Hospital, Hong Kong. Adult patients (age ≥ 18) with biopsy-proven NAFLD were recruited. Clinical events were censored on 1 Oct 2021. We excluded patients with other liver diseases (e.g., positive hepatitis B surface antigen or anti-hepatitis C virus antibody, histologic features of an alternative diagnosis), secondary causes of hepatic steatosis (e.g., use of systemic steroids or methotrexate), excessive alcohol consumption (> 30g/day in men or > 20g/day in women), and history of HCC or other cancers. No patients underwent liver transplantations. The study was approved by the Ethics Committee of the Chinese University of Hong Kong, and all patients provided informed written consent.

All patients underwent liver biopsy. Anthropometric and clinical data were recorded at the time of the liver biopsy, including age, sex, weight, height, waist circumference, and hip circumference. Body mass index (BMI) was calculated as body weight (kg) divided by body height (m) squared.

Histology

One experienced pathologist (AWHC) scored all histologic slides using the Non-alcoholic Steatohepatitis (NASH) Clinical Research Network system.¹⁴ The NAFLD activity score (NAS) was the numerical sum of steatosis (0–3), lobular inflammation (0–3), and hepatocellular ballooning (0–2). NASH was defined as at least 1 grade for each of the NAS components. Patients not fulfilling the histologic criteria of NASH were considered to have non-alcoholic fatty liver (NAFL). Liver fibrosis was defined as stage 0, absence of fibrosis; stage 1, perisinusoidal or periportal fibrosis; stage 2, perisinusoidal and portal/periportal fibrosis; stage 3, bridging fibrosis and stage 4, cirrhosis. Patients who had NAS ≥ 4 with at least 1 point in each component and fibrosis stage 2–4 were considered to have at-risk NASH. Patients with stage 3–4 fibrosis were considered to have advanced fibrosis.

Blood tests

The blood sample was taken after at least 8 hours of fasting for complete blood cell count, international normalized ratio (INR) of the prothrombin time, fasting plasma glucose, haemoglobin A_1c, total cholesterol, high-density lipoprotein (HDL) cholesterol, low-density lipoprotein (LDL) cholesterol, triglycerides, alanine aminotransferase (ALT), aspartate aminotransferase (AST), and gamma-glutamyltransferase (GGT).

Transient elastography and other non-invasive algorithms

Transient elastography examination was performed using the FibroScan 502 machine (Echosens, Paris, France) within one week before liver biopsy by experienced operators according to the training and instructions by the manufacturer.¹⁵ Patients with ≥ 10 valid liver stiffness measurements and an interquartile range-to-median ratio of the measurements of ≤ 0.3 were considered to have valid results. The median measurement was taken as reflective as the liver stiffness. Fibrosis-4 (FIB-4) index was calculated with age (years), platelet count (10⁹/L), and the serum level of AST (U/L) and ALT (U/L).¹⁶ NAFLD fibrosis score (NFS) was calculated with age (years), BMI (kg/m²), presence of impaired fasting glucose or diabetes (YES/NO), platelet count (10⁹/L), and the serum level of AST (U/L) and ALT (U/L).¹⁷ AST to platelet ratio index (APRI) was calculated with platelet count (10⁹/L) and the serum level of AST (IU/L) and ALT (IU/L).¹⁸

Protein biomarkers

Serum and plasma from biopsy-proven NASH or NAFLD patients, consisting of males and females, were used to measure exploratory and known biomarker levels of liver fibrosis, inflammation, and disease. Procollagen III N-Terminal Propeptide (PIIINP, catalogue # EHP0667, ABclonal, Woburn, MA), Osteopontin (OPN, catalogue # K151HJC, Meso Scale Discovery, Rockville, MD), Growth Differentiation Factor 15 (GDF-15, catalogue # DGD150, R&D Systems, Minneapolis, MN), Fibronectin (FN, catalogue # DFBN10, R&D Systems, Minneapolis, MN), Cytokeratin 18 (CK-18, catalogue # 10011, PEVIVA, Sundbyberg, Sweden), Tissue Inhibitor of Metalloproteinases 1 (TIMP-1, catalogue # DTM100, R&D Systems, Minneapolis, MN), Hyaluronan (HA, catalogue # DHYAL0, R&D Systems, Minneapolis, MN), Matrix Metalloproteinase 7 (MMP-7, catalogue # DMP700, R&D Systems, Minneapolis, MN), Chitinase 3-like 1 (YKL-40, catalogue # DC3L10, R&D Systems, Minneapolis, MN), N-terminal pro-peptide of type III collagen (ProC3, Nordic Bioscience, Herlev, Denmark), C-terminal pro-peptide of type V collagen (ProC5, Nordic Bioscience, Herlev, Denmark), C-terminal of released C5 domain of type VI collagen α3 chain (ProC6, Nordic Bioscience, Herlev, Denmark), and Neo-epitope of MMP9-mediated degradation of type III collagen (C3M, Nordic Bioscience, Herlev, Denmark) biomarker levels were determined by enzyme-linked immunosorbent assays (ELISA). The frozen samples were aliquoted for each assay on wet ice and run in ELISA assays in duplicate and averages were reported.

Statistical analysis

Based on previous studies, NAFLD and NASH prevalence in Asia ranges from 15–40% and 2–3% respectively,¹⁹ a sample size of 281 in NAFLD cohort will provide a 95% confidence interval of 2.78. Continuous variables were presented as mean ± standard deviation (SD) or median (interquartile range (IQR)) compared using student t test and Mann-Whitney U test in two groups. Categorical variables were compared using the Chi-squared test and Fisher’s exact test. The overall accuracy of the biomarkers in predicting at-risk NASH or advanced fibrosis was expressed by the area under the receiver-operating characteristics curve (AUROC). Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated to evaluate the diagnostic accuracy of the model. To compare the diagnostic accuracy of non-invasive tests, we used the published cut-off of the following tests: FIB-4 (low cut-off 1.63, high cut-off 2.78), NFS (low cut-off − 0.678, high cut-off 1.17), APRI (low cut-off 0.57, high cut-off 0.84), liver stiffness (8 kPa).^{8, 20–22} Kaplan-Meier curve was used to estimate the cumulative incidence of liver-related events from follow-up. P value < 0.05 was considered statistically significant for all tests. For missing values of biomarkers, stochastic regression imputation was performed in the model building stage. All data were analysed by IBM SPSS 26 for macOS and R software Version 4.0.0.

Protein biomarker selection by the machine learning algorithm

Improving the accuracy of predictions by identifying a subset of features on the grounds of statistics learning is known as ‘feature selection’. For a dataset D having d variables, feature set F can be expressed as:

F = {f₁, f₂, …., f_d}, (1)

where F stands for the feature set. The objective is to identify an optimum group of features F’, where F’ \(\in\) F. The objective function of variable selection is formulated to optimize two terms as follows: (1) the goodness-of-fit (to be maximized), and (2) the number of variables

(To be minimized). We used the embedded methods to perform the feature selection in the process of training and testing together with the bootstrap resampling technique to improve the stability of feature selection (Supplementary Fig. 1 showed the framework). The workflow was described as follows. First, we randomly split the whole dataset into a training dataset (70%) and a testing dataset (30%). Second, we built the predictive model from the training dataset using 5-fold cross validation. The model parameters were tuned to select the best model. Finally, we tested the model performance and rank the features included in the models. The whole process was repeated 200 times by bootstrap resampling datasets to aggregate the selected features. A single run of feature selection tends to select the local optimum, while this ensemble feature selection has more chance to find the better features through the aggregation of multiple feature selection processes.

Machine learning algorithms for multivariate analysis

To derive the final model, we employed the machine learning algorithms, XGBoost, Random Forest, and Support vector machines to create at-risk NASH classifiers that were trained from protein biomarkers used as the input features.^23–25 A classifier produces a probability that a given object belongs to a class (e.g. that at-risk NASH is true). The threshold (or “cut-off”) is the specified probability (often 0.5), beyond which the prediction is considered positive. XGBoost stands for Extreme Gradient Boosting. XGBoost is a tree-based ensemble machine learning algorithm which is a scalable machine learning system for tree boosting. It uses more accurate approximations to find the best tree model. Random forest, as the name implies, consists of a large number of individual decision trees that operate as an ensemble. Each individual tree in the random forest spits out a class prediction and the class with the most votes becomes our model’s prediction. Support vector machines have become popular in an increasingly wide variety of biological applications. The objective of the support vector machine algorithm is to find a hyperplane in N-dimensional space (N — the number of features) that distinctly classifies the data points. The open-source machine learning packages XGBoost, randomForest, and e1071 in R were used in the analysis.

Baseline characteristics of patients

From July 2006 to February 2015, a total of 288 patients were recruited in this study. After excluding 7 suboptimal liver biopsy samples, 281 NAFLD patients were included in the final analysis. The mean age of the whole cohort was 52 years and 55.2% were males (Table 1). The median NAFLD activity score was 4 (3–4) and the median liver stiffness measurement was 8.1 (6.1–12.8) kPa. 59 (21%) patients had at-risk NASH. Compared with patients without at-risk NASH, those with at-risk NASH were older, more likely to have diabetes and hypertension, and had higher BMI, waist circumference, fasting plasma glucose, glycated haemoglobin, and AST. Because of the use of statins, patients with at-risk NASH had lower total cholesterol and low-density lipoprotein (LDL)-cholesterol. The mean length of the liver biopsy sample was 20mm. 204 (72.6%) and 77 (27.4%) patients had F0-2 and F3-4 fibrosis, respectively (Supplementary Table 1). Compared with patients with F0-2 fibrosis, those with F3-4 fibrosis were more likely to have diabetes and hypertension and had higher fasting plasma glucose and glycated haemoglobin.

Table 1

Baseline characteristics of patients with and without at-risk non-alcoholic steatohepatitis (NASH).
Characteristics	All patients	At-risk NASH	Not at-risk NASH	P
N	281	59	222
Age (years)	52 ± 11	55 ± 11	51 ± 11	0.004
Male sex, n (%)	155 (55.2)	32 (54.2)	123 (55.4)	0.873
Diabetes, n (%)	157 (55.9)	45 (76.3)	113 (50.9)	< 0.001
Hypertension, n (%)	156 (55.5)	43 (72.9)	113 (50.9)	0.003
Body weight (kg)	76.6 ± 14.9	79.8 ± 14.6	75.7 ± 14.9	0.067
Body mass index (kg/m²)	27.9 ± 4.0	29.1 ± 3.8	27.6 ± 3.9	0.008
Waist circumference (cm)	95.0 ± 13.3	98.5 ± 14.3	94.1 ± 12.8	0.016
Fasting plasma glucose (mmol/L)	6.6 ± 2.1	7.0 ± 1.7	6.5 ± 2.2	0.003
Haemoglobin A_1c (%)	6.7 ± 1.3	7.0 ± 1.2	6.5 ± 1.3	0.002
Total cholesterol (mmol/L)	4.9 ± 1.1	4.6 ± 1.1	5.0 ± 1.0	0.007
HDL-cholesterol (mmol/L)	1.4 ± 0.7	1.4 ± 0.7	1.4 ± 0.7	0.580
LDL-cholesterol (mmol/L)	2.8 ± 0.9	2.4 ± 0.9	2.9 ± 0.9	< 0.001
Triglycerides (mmol/L)	1.8 ± 1.1	1.8 ± 0.9	1.7 ± 1.1	0.134
Alanine aminotransferase (U/L)	45.6 ± 46.7	70.1 ± 35.2	63.5 ± 44.3	0.047
Aspartate aminotransferase (U/L)	62.8 ± 42.6	47.3 ± 24.9	39.1 ± 30.5	0.002
PNPLA3, n (%)	281	59	222
CC	62 (22.1)	15 (25.4)	47 (21.2)	0.732
CG	134 (47.7)	28 (47.5)	106 (47.7)
GG	85 (30.2)	16 (27.1)	69 (31.1)
TM6SF2, n (%)	281	59	222
AA	4 (1.4)	2 (3.4)	2 (0.9)	0.342
AG	45 (16.0)	10 (16.9)	35 (15.8)
GG	232 (82.6)	47 (79.7)	185 (83.3)
Liver fat percentage (%)	44.5 ± 23.2	50.4 ± 20.4	42.9 ± 23.7	0.027
Steatosis grade	2 (1–3)	2 (2–3)	2 (1–3)	0.032
Lobular inflammation	1 (0–1)	1 (1–2)	1 (0–1)	< 0.001
Ballooning	1 (0–1)	1 (1–1)	1 (0–1)	< 0.001
NAFLD activity score	4 (3–4)	4 (4–5)	3 (3–4)	< 0.001
Fibrosis stage	1 (0–3)	3 (3–4)	1 (0–1)	< 0.001
Liver stiffness measurement (kPa)	8.1 (6.1–12.8)	13.6 (9.4–20.7)	7.0 (5.8–10.0)	< 0.001
Continuous variables are expressed as mean ± standard deviation or median (interquartile range) as appropriate.
HDL, high density lipoprotein; LDL, low density lipoprotein; PNPLA3, patatin-like phospholipase domain-containing protein 3; TM6SF2, transmembrane 6 superfamily 2

Diagnostic performance of protein biomarkers for at-risk NASH

Compared with patients without at-risk NASH, patients with at-risk NASH had higher serum levels of CK-18, GDF-15, HA, OPN, TIMP-1, YKL-40, C3M, Pro-C3, and Pro-C6 (Fig. 1, Supplementary Fig. 2). Compared with patients with F0-2 fibrosis, those with F3-4 fibrosis had higher serum levels of GDF-15, HA, MMP-7, YKL-40, Pro-C3, and Pro-C6 (Fig. 2, Supplementary Fig. 3). Table 2 shows the AUROC of baseline liver stiffness measurement and protein biomarkers for the assessment of NAFLD. For the diagnosis of at-risk NASH, the diagnostic accuracy of liver stiffness was significantly higher than that of the protein biomarkers (AUROC = 0.798). Among protein biomarkers, GDF-15 had AUROC > 0.75 for at-risk NASH and better diagnostic performance than other biomarkers. Pro-C3 and Pro-C6 also showed high diagnostic accuracy for at-risk NASH (AUROC = 0.746 and 0.726) (Fig. 3). These 3 biomarkers had a higher AUROC than FIB-4 (AUROC = 0.610), NFS (AUROC = 0.641), APRI (AUROC = 0.642) (Table 2). Single cut-offs of these biomarkers were calculated using the Youden index. Among three biomarkers, GDF-15 had the highest sensitivity of 79.7%, a specificity of 63.1%, and a negative predictive value (NPV) of 92.1% at a cut-off of 936.5 pg/mL. Pro-C3 had the highest overall accuracy of 82.2% at a cut-off of 19.2 ng/mL (Table 3). By applying the single cut-offs, individual protein biomarkers can accurately identify 50%-80% patients with at-risk NASH. Compared with the protein biomarkers, liver stiffness measurement had the best sensitivity of 82.1% at the published cut-off of 8 kPa, followed by GDF-15. However, the positive predictive values (PPVs) of these non-invasive tests were all below 60%. For the diagnosis of advanced fibrosis, liver stiffness measurement had the highest AUROC of 0.855, followed by GDF-15 (0.811), YKL-40 (0.765), and HA (0.755) (Supplementary Fig. 4).

Table 2

Area under the receiver-operating characteristics curves (AUROC) of baseline liver stiffness measurement and protein biomarkers for the assessment of NAFLD
Biomarker	At-risk NASH (n = 59) vs no at-risk NASH (n = 222)			F3-4 (n = 77) vs F0-2 (n = 204)
	AUROC	CI	P	AUROC	CI	P
Liver stiffness measurement (n = 258)	0.798	0.736, 0.859	< 0.001	0.855	0.803, 0.907	< 0.001
cytokeratin-18 fragments (CK-18)	0.660	0.574, 0.746	< 0.001	0.568	0.492, 0.645	0.082
Fibronectin	0.544	0.457, 0.630	0.320	0.438	0.359, 0.518	0.123
Growth/differentiation factor 15 (GDF-15)	0.755	0.688, 0.821	< 0.001	0.811	0.756, 0.866	< 0.001
Hyaluronic acid (HA)	0.621	0.526, 0.717	0.014	0.755	0.687, 0.823	< 0.001
Matrix metalloproteinase-7 (MMP-7)	0.602	0.513, 0.691	0.022	0.635	0.555, 0.715	0.001
Osteopontin (OPN)	0.610	0.530, 0.689	0.041	0.589	0.511, 0.667	0.028
N-terminal propeptide of type III collagen (PIIINP)	0.515	0.424, 0.606	0.741	0.592	0.512, 0.671	0.024
Tissue inhibitor matrix metalloproteinase 1 (TIMP-1)	0.659	0.579, 0.740	< 0.001	0.610	0.534, 0.686	0.005
Chitinase-3-like protein 1 (YKL-40)	0.711	0.636, 0.786	< 0.001	0.765	0.703, 0.826	< 0.001
Collagen III degradation fragments (C3M)	0.594	0.515, 0.672	0.027	0.578	0.505, 0.652	0.043
Pro-C3	0.746	0.673, 0.820	< 0.001	0.682	0.607, 0.757	< 0.001
Pro-C5	0.574	0.460, 0.634	0.044	0.531	0.453, 0.608	0.430
Pro-C6	0.726	0.661, 0.791	< 0.001	0.698	0.631, 0.764	< 0.001
Fibrosis-4 (FIB-4) index (n = 248)	0.610	0.516, 0.704	0.016	0.692	0.614, 0.771	< 0.001
NAFLD fibrosis score (NFS) (n = 245)	0.641	0.553, 0.728	0.003	0.723	0.647, 0.800	< 0.001
AST-to-platelet ratio index (APRI) (n = 248)	0.642	0.551, 0.733	0.002	0.666	0.590, 0.742	< 0.001
AST, aspartate aminotransferase; NAFLD, non-alcoholic fatty liver disease; NASH, non-alcoholic steatohepatitis

Table 3

Performance of non-invasive tests in the detection of at-risk NASH.
Methods	Cut-off	Sensitivity% (95% CI)	Specificity% (95% CI)	Positive Likelihood Ratio (95% CI)	Negative Likelihood Ratio (95% CI)	Positive Predictive Value % (95% CI)	Negative Predictive Value % (95% CI)	Overall Accuracy (95% CI)
Transient elastography (n = 258)	8 kPa	82.1 (69.6–91.1)	57.9 (50.8–64.8)	1.95 (1.59–2.39)	0.31 (0.17–0.55)	35.1 (30.7–39.9)	92.1 (86.8–95.4)	63.2 (57.0-69.1)
FibNASH-3 model (N = 281)	0.5	35.6 (23.6–49.1)	96.9 (93.6–98.7)	11.29 (5.04–25.27)	0.67 (0.55–0.81)	75.0 (57.3–87.0)	85.0 (82.4–87.3)	84.0 (79.2–88.1)
Sequential model (1. TE > 8;2. FibNASH-3 model > 0.5) (n = 258)	NA	33.9 (21.8–47.8)	97.5 (94.3–99.2)	13.71 (5.36–35.08)	0.68 (0.56–0.82)	79.2 (59.8–90.7)	84.2 (81.5–86.5)	83.7 (78.6–88.0)
Growth/differentiation factor 15 (GDF-15) (N = 281)	936.5 pg/ml	79.7 (67.2–89.0)	63.1 (56.4–69.4)	2.16 (1.74–2.67)	0.32 (0.19–0.54)	36.4 (31.6–41.5)	92.1 (87.5–95.1)	66.6 (60.7–72.0)
Pro-C3 (N = 281)	19.2 ng/ml	49.2 (35.9–62.5)	91.0 (86.4–94.4)	5.46 (3.34–8.92)	0.56 (0.43–0.72)	59.2 (47.0-70.3)	87.1 (83.9–89.7)	82.2 (77.2–86.5)
Tissue inhibitor matrix metalloproteinase 1 (TIMP-1) (N = 281)	201.16 ng/ml	42.4 (29.6–55.9)	84.2 (78.8–88.8)	2.69 (1.76–4.11)	0.68 (0.55–0.86)	41.7 (31.8–52.2)	84.6 (81.4–87.3)	75.4 (70.0-80.4)
Fibrosis-4 (FIB-4) index (N = 248)	1.63	32 (19.5–46.7)	84.4 (78.6–89.2)	2.05 (1.22–3.45)	0.81 (0.66–0.98)	34 (23.5–46.4)	83.2 (80.2–85.8)	73.9 (68-79.2)
Fibrosis-4 (FIB-4) index (N = 248)	2.78	12 (4.5–24.3)	96.0 (92.2–98.3)	2.99 (1.08–8.21)	0.92 (0.82–1.02)	42.9 (21.4–67.4)	81.3 (79.6–82.8)	79.1 (73.5–84)
NAFLD fibrosis score (NFS) (N = 245)	-0.678	33.3 (20.4–48.4)	85.3 (79.6–89.9)	2.26 (1.34–3.82)	0.78 (0.63–0.96)	35.6 (24.7–48.2)	84 (81-86.6)	75.1 (69.2–80.4)
NAFLD fibrosis score (NFS) (N = 245)	1.17	2.1 (0.1–11.1)	97.5 (94.2–99.2)	0.82 (0.1–6.86)	1 (0.96–1.05)	16.7 (2.3–62.6)	80.3 (79.6–81.1)	78.8 (73.1–83.7)
AST-to-platelet ratio index (APRI) (N = 248)	0.57	51.0 (36.3–65.6)	74.4 (67.7–80.3)	1.99 (1.39–2.86)	0.66 (0.49–0.89)	32.9 (25.4–41.3)	86.1 (82.1–89.3)	68.8 (63.6–75.4)
AST-to-platelet ratio index (APRI) (N = 248)	0.84	26.5 (15.0-41.1)	90.5 (85.5–94.2)	2.78 (1.48–5.23)	0.81 (0.68–0.97)	40.6 (26.7–56.3)	83.3 (80.8–85.6)	77.8 (72.1–82.8)
AST, aspartate aminotransferase; NAFLD, non-alcoholic fatty liver disease; NASH, non-alcoholic steatohepatitis.

Selection of protein biomarkers by machine learning algorithms

Supplementary Table 2 showed the selected top protein biomarkers from 13 biomarkers and the averaged AUROC values from three machine learning algorithms: XGboost, Random Forest, and Support vector machines. Each algorithm selected different protein biomarkers. The consensus biomarkers are Pro-C3, TIMP-1, and GDF-15.

A sequential model for screening at-risk NASH patients

We developed a sequential model by combining liver stiffness measurement (LSM) and protein biomarker machine learning model (Supplementary Fig. 5). The first step was the assessment of LSM using FibroScan. Patients with an LSM value < = 8.0 would not be selected for liver biopsy confirmation because of the very low likelihood of at-risk NASH. In the second step, subjects with LSM > 8.0 will have the three protein biomarker measured: Pro-C3, TIMP-1, and GDF-15. A logistic regression model named FibNASH-3 was used to predict the probability of at-risk NASH with the formula as follows:

The diagnostic accuracy of FibNASH-3 was higher than any individual biomarkers (AUROC = 0.784) (Supplementary Fig. 6). Using the default cut-off value of 0.5, only patients with the predicted probability > 0.5 would require liver biopsy for clinical enrolment. The diagnostic performance characteristics of sensitivity, specificity, PPV, and NPV for all potential model cut-offs are displayed in Fig. 4A. The potential impact of screen failure rate, defined as the proportion of individuals who pass through screening but do not meet the histological criteria after biopsy, was shown in Fig. 4B. The selection of the optimal cut-off for trial enrichment will ultimately depend on the required balance between screen fail and missed cases rate. Using 0.5 as the cut-off value, the sequential model could reduce the screen failure rate to about 20% the PPV was79.2% (95% CI 59.8–90.7). Table 3 compared the performance of the sequential model with LSM, protein biomarker model (FibNASH-3), the single component biomarker used individually, and commonly used clinical models (such as FIB-4, NFS, and APRI). FibNASH-3 and our sequential model had better sensitivity, specificity, PPV and NPV than FIB-4 and NFS at any low cut-offs or high-offs. The sequential model was the best one to predict the true positives and the positive likelihood ratio is 13.71, which demonstrates more than 10 folds improvement of the positive likelihood ratio (LR) compared to the other clinical models. The likelihood ratio is referred to as the Bayes factor and constitutes one of the best ways to measure and express diagnostic accuracy. (20) In the true statistical sense, it corresponds to the ratio of the likelihood of the observed test result in the diseased versus non-diseased populations. In this cohort, the prevalence of at-risk NASH patients was only 20.9%, and the sequential model could enrich the targeted patient population almost 4 folds.

Prognostication with the FibNASH-3 Model and sequential model

After excluding one patient with varices before biopsy, eleven patients developed liver-related events during a mean follow-up of 10.7 years. Among patients with liver-related events, 5 patients developed HCC, 3 patient developed hepatic encephalopathy, 1 patient developed hepatorenal syndrome, 4 patients with variceal bleeding and 6 patients developed ascites. The incidence of liver-related events was 6.8% (4/59) in patients with at-risk NASH and 3.2% (7/221) in patients without at-risk NASH by liver biopsy (P = 0.15) (Fig. 5A), 14.3% (4/28) in patients with FibNASH-3 ≥ 0.5 and 2.8% (7/252) in patients with FibNASH-3 < 0.5 (P < 0.01) (Fig. 5B), and 16.7% (4/24) in patients with high sequential model score group and 3% (7/233) in patients with low sequential model score group (P < 0.01) (Fig. 5C). The results were consistent with stratified patients by model values and biopsy-proven at-risk NASH. 26 patients died during the follow-up; 4 patients died of HCC, 1 patient died of hepatic failure, 1 patient died of renal failure, 11 patients had non-HCC cancers, 8 patients had cardiovascular deaths and 1 patient died of COVID-19.

Subgroup analysis

Subgroup analysis was conducted to assess the overall diagnostic performance of FibNASH-3 compared with other blood-based scores among specific subgroup populations of at-risk NASH. (Supplementary Table 3) The overall performance of FibNASH-3 was the highest among different non-invasive scores. FibNASH-3 was not significantly affected by the status of type 2 diabetes (Delong test p > 0.05) compared to APRI (P < 0.05). FibNASH-3, APRI had better performance in patients with BMI < 30 and females. FibNASH3, FIB-4 and NFS had significantly higher performance in patients 55 years or older than those younger patients.

In this study, we compared the performance of 13 serum protein biomarkers and showed that GDF-15, Pro-C3 had the highest AUROC for the diagnosis of at-risk NASH and advanced fibrosis, respectively. We also derived and validated a machine learning model FibNASH-3 and a sequential model to improve the diagnosis of at-risk NASH. Importantly, the FibNASH-3 model and sequential model correlated with not only histologic features but also liver-related outcomes.

Fibrosis is the common final pathway of all chronic liver diseases. It correlates with clinical outcomes and the risk of HCC. Patients with at-risk NASH are currently the target population for NASH clinical trials and will likely be the target for pharmacological treatment in the real world when registered drugs become available. In this study, GDF-15, Pro-C3 showed satisfactory results, and GDF-15 had the best performance among the 13 serum markers for diagnosis of at-risk NASH (AUROC = 0.755) and diagnosis of advanced fibrosis (AUROC = 0.811). So far, data on GDF-15 are limited. A previous Korean study showed that the AUROC of GDF-15 was 0.789 (95% CI, 0.698-0.880; P < .001) for the diagnosis of advanced fibrosis with 150 NAFLD patients.²⁶ Our study results are consistent with this Korean study and provide clinical meaning for GDF-15.

One of the biggest challenges facing pharmaceutical companies involved in NASH clinical trials is the cost-effective and timely enrolment of study subjects. This is partly due to the large number of trials that are actively recruiting subjects. Adding to this is the difficulty of finding patients with confirmed at-risk NASH from a required liver biopsy at baseline. About 60 percent or more of screened subjects will not meet the eligibility criteria. This high screen failure rate has been reported by several of the published NASH clinical studies, including PIVENS, FLINT, GOLDEN, and others ^{27, 28}. A non-invasive diagnosis tool of at-risk NASH can improve the screening success rate and reduce the need for liver biopsy. A modest reduction of screening failure rate will have a potentially massive effect on trial enrolment. To pre-screen and enrich the pool of at-risk NASH participants, we developed a machine learning model then a sequential model combining with LSM and the machine learning model. The machine learning model, named FibNASH-3, with three components including GDF-15, Pro-C3 and TIMP-1 showed high accuracy in identifying at-risk NASH (AUROC = 0.784). Each component of FibNASH-3 was significantly associated with at-risk NASH. FibNASH-3 had better diagnostic performance than the other blood-based biomarkers and had similar accuracy as transient elastography (AUROC = 0.798). By applying the cut-off of 0.5, FibNASH-3 and the sequential model had a PPV of 75.0% (95%CI 57.3–87.0) and 79.2 (95%CI 59.8–90.7) separately. Our models had superior diagnostic performance than traditional non-invasive scores (FIB-4 and NFS). The positive predictive value (PPV) and accuracy of the biomarker are influenced by the prevalence of the disease. A biomarker is likely to achieve a higher PPV in the setting of a higher disease prevalence. In this study cohort, the prevalence of at-risk NASH patients was only 20.9%. Moreover, the sequential model achieved an outstanding positive likelihood ratio of 13.71. Positive likelihood ratio (LR) is not dependent on disease prevalence, thus, is considered as a robust global measure of the diagnostic properties of a test. Experts usually consider that tests with a positive LR greater than 10 have the potential to alter clinical decisions. (21) Finally, the sequential model used the default value of 0.5 as the cut-off to make Go/No-Go decision. Single tests usually have a large indeterminate zone, which is undesirable in clinical practice. As a pre-screening tool for targeted population enrichment, the sequential model may significantly reduce the need for liver biopsy to fulfil the trial’s histological criteria. The trade-off of the sequential model is at the risk of decreased sensitivity. At-risk NASH patients might be missed. It is estimated that in the United States, 9.8 million persons are living with NASH and F0/F1 fibrosis stage, and over 5 million with NASH and F2/F3 fibrosis stage. (24) The sequential model, targeting to diagnose at-risk NASH patients in a precise and cost-effective manner, enables NAFLD management and assessment in the primary care setting. On the other hand, patient recruitment for NASH trials within the primary care population is increasing the screening pool to overcome the low sensitivity of the sequential model.

The true rates and types of clinical outcomes among the at-risk NASH patients are of vital importance for patient management and clinical trial design. The current knowledge remains largely unknown due to the lack of prospective cohort studies. In this cohort with a mean follow-up of 10.7 years, the incidence of liver-related events in at-risk NASH patients diagnosed by liver biopsy (a rate of 1.05 per 100 person-year) was in the range of the rates observed in a large prospective study involving 1773 patients with the mean follow-up of 4.6 years. (25) A useful biomarker should not only correlate with liver histology but also have prognostic implications. In our study, 14.3% of patients with high FibNASH-3 score and 16.7% of patients with high sequential model score developed hepatic events, compared with 6.8% among those with at-risk NASH diagnosed from liver histology, which warrants the further exploration that non-invasive biomarkers are even better surrogates for meaningful clinical outcomes A study with 2154 advanced NASH patients showed that patients with higher non-invasive tests scores (ELF, NFS, FIB-4, APRI, FibroTest and liver stiffness) had a higher risk of hepatic events and histologic progression after a median of 16 months of follow-up.²⁹ Our study also provided support evidence of non-invasive test in predicting cirrhotic complications and HCC of NAFLD patients.

Our study had some limitations. First, as our study was based on liver biopsy, the sample size was modest. Second, the prevalence of diabetes and hypertension is expected high in patients with biopsy-proven at-risk NASH in our cohort. Our model should be further validated to determine whether the use of these clinical parameters distinguishes patients with at-risk NASH. Third, all participants were ethnic Chinese. Machine learning approaches had an inherent risk of overfitting, and our findings need to be externally validated in separate cohorts. The wider use of these biomarkers would depend on the pricing in the future.

In conclusion, we identified several promising protein biomarkers for the diagnosis of at-risk NASH, and developed and validated a new machine learning FibNASH-3 model using three biomarkers. Our model has the ability to accurately identify at-risk NASH in NAFLD patients and predict the development of liver-related events.

ALT, alanine aminotransferase; AST, aspartate aminotransferase; AUROC, area under the receiver operating characteristic curve; BMI, body mass index; CI, confidence interval; CK-18, cytokeratin-18 fragments; GDF-15, growth differentiation factor-15; GGT, gamma-glutamyltransferase; HA, hyaluronic acid; HDL, high-density lipoprotein; INR, international normalized ratio; LDL, low-density lipoprotein; MMP-7, Matrix metalloproteinase-7; NPV, negative predictive value; PPV, positive predictive value; Pro-C3, procollagen type-III N-terminal peptide; PIIINP, N-terminal propeptide of type III collagen; ROC, receiver operating characteristic; TIMP-1, tissue inhibitor matrix metalloproteinase 1; YKL-40, Chitinase-3-like protein 1

Guarantors of the article: Vincent Wai-Sun Wong, Kevin L. Duffin, Yu Chen

Ethics approval and consent to participate: The study was approved by the Ethics Committee of the Chinese University of Hong Kong, and all patients provided informed written consent. All methods were carried out in accordance with American Association for the Study of Liver Diseases (AASLD) guidelines and regulations.

Consent for publication: Not applicable

Availability of data and materials: The datasets generated during the current study are available from the corresponding author on reasonable request.

Competing interests: Vincent Wong has served as a consultant or advisory board member for 3V-BIO, AbbVie, Allergan, Boehringer Ingelheim, Echosens, Gilead Sciences, Inventiva, Merck, Novartis, Novo Nordisk, Pfizer, ProSciento, Sagimet Biosciences, TARGET PharmaSolutions, and Terns; and a speaker for Abbott, AbbVie, Echosens, Gilead Sciences, and Novo Nordisk. He has received a research grant from Gilead Sciences and is a co-founder of Illuminatio Medical Technology Limited. Jonathan Wilson, Lenden Bowsman, Joseph Haas, Kevin Duffin, and Yu Chen are employees of Eli Lilly. Other authors have no competing interest.

Funding: No funding.

Specific author contributions: Vincent Wong, Kevin L. Duffin, and Yu Chen have full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Vincent Wong, Terry Yip and Yu Chen were responsible for the study concept and design. Guanlin Li, Grace Wong, Terry Yip, Huapeng Lin, Xinrong Zhang, and Vincent Wong were responsible for the acquisition of data. Jonathan Wilson, Lenden Bowsman, and Joseph Haas contributed to the biochemical assays, data interpretation and manuscript review. Sall Shu and Angel Chim contributed to histological assessment of the liver biopsy samples. Guanlin Li, Yu Chen, Qingxiong Tan, Terry Yip, Yee Kit Tse, Henry Chan and Vincent Wong were responsible for data analysis. Guanlin Li, Yu Chen, and Vincent Wong drafted the manuscript. All authors had a significant contribution to the intellectual content of the manuscript and approved its final version.

Younossi ZM, Koenig AB, Abdelatif D, Fazel Y, Henry L, Wymer M. Global epidemiology of nonalcoholic fatty liver disease-Meta-analytic assessment of prevalence, incidence, and outcomes. Hepatology. 2016;64:73–84.
Pais R, Charlotte F, Fedchuk L, et al. A systematic review of follow-up biopsies reveals disease progression in patients with non-alcoholic fatty liver. J Hepatol. 2013;59:550–6.
Powell EE, Wong VW, Rinella M. Non-alcoholic fatty liver disease. Lancet. 2021;397:2212–24.
Noureddin M, Vipani A, Bresee C, et al. NASH Leading Cause of Liver Transplant in Women: Updated Analysis of Indications For Liver Transplant and Ethnic and Gender Variances. Am J Gastroenterol. 2018;113:1649–59.
FDA-CDER. Noncirrhotic Nonalcoholic Steatohepatitis With Liver Fibrosis: Developing Drugs for Treatment: Guidance for Industry. i>https://wwwfdagov/media/119044/download</i.
Newsome PN, Sasso M, Deeks JJ, et al. FibroScan-AST (FAST) score for the non-invasive identification of patients with non-alcoholic steatohepatitis with significant activity and fibrosis: a prospective derivation and global validation study. Lancet Gastroenterol Hepatol. 2020;5:362–73.
Boursier J, Anty R, Vonghia L, et al. Screening for therapeutic trials and treatment indication in clinical practice: MACK-3, a new blood test for the diagnosis of fibrotic NASH. Aliment Pharmacol Ther. 2018;47:1387–96.
Noureddin M, Truong E, Gornbein JA, et al. MRI-based (MAST) score accurately identifies patients with NASH and significant fibrosis. J Hepatol. 2022;76:781–7.
Harrison SA, Ratziu V, Boursier J, et al. A blood-based biomarker panel (NIS4) for non-invasive diagnosis of non-alcoholic steatohepatitis and liver fibrosis: a prospective derivation and global validation study. Lancet Gastroenterol Hepatol. 2020;5:970–85.
Torres R, Judson-Torres RL. Research Techniques Made Simple: Feature Selection for Biomarker Discovery. J Invest Dermatol. 2019;139:2068-+.
Xie Y, Meng WY, Li RZ, et al. Early lung cancer diagnostic biomarker discovery by machine learning methods. Transl Oncol. 2021;14:100907.
Guyon I, Elisseeff A. An introduction to variable and feature selection. J Mach Learn Res. 2003;3:1157–82.
Al-Rajab M, Lu J, Xu Q. A framework model using multifilter feature selection to enhance colon cancer classification. Plos One. 2021; 16.
Kleiner DE, Brunt EM, Van Natta M, et al. Design and validation of a histological scoring system for nonalcoholic fatty liver disease. Hepatology. 2005;41:1313–21.
Wong VW-S, Irles M, Wong GL-H, et al. Unified interpretation of liver stiffness measurement by M and XL probes in non-alcoholic fatty liver disease. Gut. 2019;68:2057–64.
Vallet-Pichard A, Mallet V, Nalpas B, et al. FIB-4: an inexpensive and accurate marker of fibrosis in HCV infection. comparison with liver biopsy and fibrotest. Hepatology. 2007;46:32–6.
Angulo P, Hui JM, Marchesini G, et al. The NAFLD fibrosis score: a noninvasive system that identifies liver fibrosis in patients with NAFLD. Hepatology. 2007;45:846–54.
Wai CT, Greenson JK, Fontana RJ, et al. A simple noninvasive index can predict both significant fibrosis and cirrhosis in patients with chronic hepatitis C. Hepatology. 2003;38:518–26.
Younossi Z, Tacke F, Arrese M, et al. Global Perspectives on Nonalcoholic Fatty Liver Disease and Nonalcoholic Steatohepatitis. Hepatology. 2019;69:2672–82.
Anstee QM, Lawitz EJ, Alkhouri N, et al. Noninvasive Tests Accurately Identify Advanced Fibrosis due to NASH: Baseline Data From the STELLAR Trials. Hepatology. 2019;70:1521–30.
Lee HW, Park SY, Kim SU, et al. Discrimination of Nonalcoholic Steatohepatitis Using Transient Elastography in Patients with Nonalcoholic Fatty Liver Disease. PLoS ONE. 2016;11:e0157358.
Siddiqui MS, Yamada G, Vuppalanchi R, et al. Diagnostic Accuracy of Noninvasive Fibrosis Models to Detect Change in Fibrosis Stage. Clin Gastroenterol Hepatol. 2019;17:1877-85 e5.
Chen TQ, Guestrin C. XGBoost: A Scalable Tree Boosting System. Kdd'16: Proceedings of the 22nd Acm Sigkdd International Conference on Knowledge Discovery and Data Mining. 2016: 785 – 94.
Breiman L. Random forests. Mach Learn. 2001;45:5–32.
Noble WS. What is a support vector machine? Nat Biotechnol. 2006;24:1565–7.
Koo BK, Um SH, Seo DS, et al. Growth differentiation factor 15 predicts advanced fibrosis in biopsy-proven non‐alcoholic fatty liver disease. Liver Int. 2018;38:695–705.
Younossi ZM, Ratziu V, Loomba R, et al. Obeticholic acid for the treatment of non-alcoholic steatohepatitis: interim analysis from a multicentre, randomised, placebo-controlled phase 3 trial. Lancet. 2019;394:2184–96.
Harrison SA, Wong VW, Okanoue T, et al. Selonsertib for patients with bridging fibrosis or compensated cirrhosis due to NASH: Results from randomized phase III STELLAR trials. J Hepatol. 2020;73:26–39.
Younossi ZM, Anstee QM, Wai-Sun Wong V, et al. The Association of Histologic and Noninvasive Tests With Adverse Clinical and Patient-Reported Outcomes in Patients With Advanced Fibrosis Due to Nonalcoholic Steatohepatitis. Gastroenterology. 2021;160:1608–9.e13.

Competing interest reported. Vincent Wong has served as a consultant or advisory board member for 3V-BIO, AbbVie, Allergan, Boehringer Ingelheim, Echosens, Gilead Sciences, Inventiva, Merck, Novartis, Novo Nordisk, Pfizer, ProSciento, Sagimet Biosciences, TARGET PharmaSolutions, and Terns; and a speaker for Abbott, AbbVie, Echosens, Gilead Sciences, and Novo Nordisk. He has received a research grant from Gilead Sciences and is a co-founder of Illuminatio Medical Technology Limited. Jonathan Wilson, Lenden Bowsman, Joseph Haas, Kevin Duffin, and Yu Chen are employees of Eli Lilly. Other authors have no competing interest.

Supplementaryfile.docx

Download PDF

Version 1

posted

You are reading this latest preprint version

A Sequential Machine Learning Model for Identifying At-risk NASH by Combining Liver Stiffness Measurement and Protein Biomarkers

Status:

Version 1

Abstract

Figures

Introduction

Patients And Methods

Study population and study design

Histology

Blood tests

Transient elastography and other non-invasive algorithms

Protein biomarkers

Statistical analysis

Protein biomarker selection by the machine learning algorithm

Machine learning algorithms for multivariate analysis

Results

Baseline characteristics of patients

Diagnostic performance of protein biomarkers for at-risk NASH

Selection of protein biomarkers by machine learning algorithms

A sequential model for screening at-risk NASH patients

Prognostication with the FibNASH-3 Model and sequential model

Subgroup analysis

Discussion

Conclusion

Abbreviations

Declarations

References

Additional Declarations

Supplementary Files

Status:

Version 1