4.1 Site investigations
The results of in-situ tests showed that the soil types were predominantly classified as SP, SM, and SP-SM. These soil types are known to have a high risk of liquefaction, as supported by previous research conducted by Robertson and Campanella (1985) and Holzer et al. (1989), which have established a correlation between soil types and liquefaction potential based on the soil's characteristics during an earthquake. The results of direct shear tests have shown an average soil friction angle of approximately 30 degrees and an average soil cohesion of 2.15 kg/cm2. Sandy soils' shear strength is often characterised by the internal friction angle, which typically ranges from 15 to 25 degrees for low-strength soils and around 30 to 40 degrees for high-strength soils (Das & Sivakugan, 2016).
The SPT results suggest a high likelihood of soil liquefaction. Most of the recorded SPT values are below 30, with an average soil water content ranging between 20% and 23 (Fig. 3a). The average groundwater depth in the region is approximately 2.8 meters (IGS, 2023). The values of LL and PL from the boreholes are shown in Fig. 3b, with an LL range of 20 to 23 and a PL range of 2 to 7. Figure 4 presents the average SPT and water content values in a 3D format. These results support the idea that lower SPT values and higher water content increase the likelihood of soil liquefaction. Considering the high seismic activity in the region (IGS, 2023) and the relatively soft soil deposits, it is crucial to conduct a thorough evaluation of SLP at the site.
The correlation between SPT and liquefaction potential is generally considered better for loose, sandy soils than for dense, clayey soils. Besides, the correlation between the SPT and liquefaction potential also depends on the magnitude of cyclic loading, which can vary widely during an earthquake. Therefore, the correlation between the SPT and liquefaction potential is used under probabilistic analysis, where the uncertainties in the input parameters are considered (Das & Sivakugan, 2016).
4.2 Performance metric
When comparing ML models, using multiple evaluation metrics is essential instead of relying on a single metric. This is because a single evaluation metric can be misleading and may not accurately reflect the model's performance. For instance, a model may have a high accuracy score but may need help to generalise well to new data. Or a model may have high precision but low recall, meaning it can identify most of the relevant data but also includes a lot of irrelevant data. A more comprehensive approach would be to use a variety of evaluation metrics such as F1-score, accuracy, the area under the ROC curve precision and recall. These metrics provide a complete picture of the model's performance and make it easier to identify the strengths and weaknesses of each model. Additionally, depending on the task, some metrics are more critical than others (Fawcett, 2006).
Figures 5 and 6 present the performance metrics of several ML models. The results indicate that the lightgbm, rf, and gpc models demonstrate superior performance, as measured by a combination of accuracy, AUC and recall. Specifically, the lightgbm model had the highest accuracy (0.9462), AUC (0.9825), and recall (0.8909), the rf model had an accuracy of 0.9442, AUC of 0.9842, and recall of 0.8796, and the gpc model had an accuracy of 0.9381, AUC of 0.9809, and recall of 0.8755. It is worth noting that these metrics are not always sufficient to evaluate the performance of a model, especially when it comes to real-world problems; they give a good overview of the model's performance. Additionally, the models' training time was relatively fast, with the best-performing model (lightgbm) having a training time of 0.157 seconds. Based on the results of these various performance metrics and taking into account the fast training time, the lightgbm model was selected as the best-performing model. However, it is essential to consider the problem requirements and constraints before making a final decision. It's also good practice to test the model on unseen data and evaluate its generalisation capabilities.
4.3 The lightgbm model
The results proved that the lightgbm model performed the best in estimating SLP. The lightgbm is a gradient-boosting library developed by Microsoft that is widely used for machine learning tasks. It is known for its speed and efficiency and is often used for large-scale tasks such as training decision tree ensembles on large datasets (Ke et al., 2017). The primary function of lightgbm is to train decision tree ensembles to make predictions on various machine learning tasks, including classification, regression, and ranking (Seto et al., 2022). Lightgbm incorporates multiple techniques to enhance the efficacy and effectiveness of the training procedure. One of these techniques is called Exclusive Feature Bundling (EFB), which involves grouping features together and training a separate decision tree for each group. The process of EFB starts by first finding the features that occur the most frequently in the training set. Subsequently, it removes the least distinctive characteristics for each class iteratively until the desired number of features is attained. The EFB method finds a minimal set of features that can accurately classify instances by identifying and preserving the most class-discriminative features. It is a heuristic method which provides a trade-off between accuracy and interpretability (Zhang & Gong, 2020).
This can help reduce the overall number of decision trees that need to be trained, improving the training speed and efficiency. Another technique used by lightgbm is called Gradient-Based One-Side Sampling. This involves sampling the training data to focus on the samples with more significant gradients, which can help improve the model's ability to learn from the data. In addition to these techniques, lightgbm also includes support for hyperparameter tuning using bagging techniques. Bagging involves training multiple models with different combinations of hyperparameters and then averaging the predictions of these models to obtain a final prediction (Ke et al., 2017). This can help to improve the model's performance and reduce the risk of overfitting (Kramer & Kramer, 2016). The details of the developed and hyperparameter-tuned model are shown in Table 3.
In Table 3, the GBDT (Gradient Boosting Decision Tree) is an iterative method that starts by training a simple model, such as a single decision tree, on the training data. It then uses the negative gradient of a loss function to identify the areas of the training data that the model is most uncertain about. The algorithm then trains new decision tree models to focus on these specific areas, and these new models are added to the ensemble. The procedure is repeated for a predetermined number of iterations or until a particular stopping criterion is met. (Friedman, 2001). In this Table, importance_type = 'split' refers to the method used to calculate the feature importance. Specifically, the feature importance is calculated as the total reduction in the criterion (e.g., Gini impurity or entropy) brought about by splits over that feature (https://scikit-learn.org). The parameter min_child_samples, in lightgbm, determines the minimum number of samples needed in a child node for a split to be included in the training process. By preventing the formation of extremely small leaf nodes, this parameter can control the model's complexity and avoid overfitting (source: https://Pycaret.org). The remaining parameters were chosen from the default settings provided in the Pycaret library.
In Table 3, the parameters of the Tuned model are described. In Pycaret, the "Tuned Model" refers to the best-performing model selected by the tuning process, which is based on the performance metric specified in the compare_models() or tune_model() function. Pycaret uses a hyperparameter tuning library internally called Optuna, which runs a Bayesian optimisation internally to find the best hyperparameters and returns the best model after automatic machine learning (auto-ml).
Table 3
Description of the models
Parameters | Values | Models: Developed Model (DM) and Tuned Model (TM) |
boosting_type | Gbdt | DM |
importance_type | Split | DM |
bagging_fraction | 0.9 | TM |
bagging_freq | 3 | TM |
learning_rate | 0.1 | DM |
learning_rate | 0.6 | TM |
min_child_samples | 20 | DM and TM |
min_child_weight | 0.001 | DM and TM |
min_split_gain | 0.00 | DM |
min_split_gain | 0.3 | TM |
n_estimators | 100 | DM |
n_estimators | 20 | TM |
num_leaves | 31 | DM |
num_leaves | 150 | TM |
4.3 Performance indices of the lightgbm model
The ROC curve, threshold plot, precision-recall curve, and learning curve were utilised to illustrate the performance of the lightgbm model after tuning, as depicted in Figs. 7(a)-(d). The area beneath the ROC curve is frequently employed as a metric to assess the effectiveness of ML classifiers, with values exceeding 0.9 generally regarded as outstanding (Merghadi et al., 2020). Models with a low false positive rate and a high true positive rate exhibit better overall performance, as reflected by a higher AUC value on the ROC curve. The AUC values range from zero to one - with 0.5 indicating random performance occurring and 1 indicating flawless performance. In the case of the tuned lightgbm model, both class 0 and class 1 have an AUC value of 0.99, as shown in Fig. 7 (a), indicating high accuracy, as the AUC value is close to 1. However, to fully evaluate the model's performance, other metrics such as precision, recall, and F1 score should be considered alongside the AUC value (Fawcett, 2006).
Figure 7 (b) depicts the threshold used to determine the True and False positive rates of a binary classification model. This plot evaluates the model’s performance by exhibiting the relationship between the false positive rate (1-specificity) and the true positive rate as the threshold for classifying a positive instance is altered. The vertical line in this plot corresponds to the point that maximises the difference between the TPR and FPR and is usually the point on the ROC curve closest to the top-left corner. Figure 7 (c) illustrates the tradeoff between precision and recall for different thresholds. It is essential to consider precision and recall in addition to the AUC value to evaluate the model's behaviour comprehensively. A higher area under this curve indicates a more robust classifier. The red dots in Fig. 7 (c) show an average precision of 98%. In Pycaret, a learning curve is a visualisation used to understand how the performance of a model changes as more data is used to train. Finally, the high score of the learning rate (between 0.98 and 1) for the optimised lightgbm model is depicted in Fig. 7 (d).
The sensitivity analysis for the lightgbm model shown in Fig. 8 demonstrates how different soil properties influence the soil liquefaction potential (SLP). Previous research on this topic has found that different soil properties can have varying levels of influence on the SLP. For example, Goh (1996) and Ahmad et al. (2021) found that cone tip resistance (qc) had the most significant impact on the SLP, while Pirhadi et al. (2018) reported that the normalised cone penetration tip value was the most influential factor. However, the present study found that the water content and SPT numbers were the two most influential parameters on the SLP, while the depths of the soil and strength parameters ranked fourth and fifth. These results suggest that the water content and SPT numbers may be essential factors to consider when assessing the risk of soil liquefaction.