This section describes the environmental setup, results and metrics comparison of the proposed GBHT algorithm. A server machine with an Intel® i7 processor, 4 cores, and 2.9 GHz clock speed with 128 gb RAM. Python 3.7 installed on the server machine. The pandas, scikit-learn, and numpy are installed. The Dataset taken from github and it contains the CPU Usage data of Microsoft Azure, which is a cloud service, sampled every 5 minute. The dataset has three attributes i.e. max cpu utilization, average cpu utilization, minimum cpu utilization as shown in Fig. 2. The parameters used in the work are describes in the Table 2. The implementation code loads the dataset, split it into input features (X) and target variable (y), and then further split it into training and testing sets. It defines a parameter grid for hyperparameter tuning, including the number of estimators, learning rate, and maximum depth.
The code creates a gradient boosting regressor model and perform grid search to find the best combination of hyperparameters using cross-validation.
Table 2
Experimental set-up parameters and their values.
Parameter | Values |
Epochs | 100–500 |
Learning_rate | 0.01 − .9 |
Max_depth | 3–9 |
'n_estimators' | 100–500 |
Training data | 70% |
Test data | 30% |
min_samples_split | 2–10 |
min_samples_leaf | 1–6 |
subsample | 0.2-1 |
Search | Grid search |
Finally, it evaluated the model's performance on the test set and print the best hyperparameters, MAE, MAPE, RMSE, and R2. The comparison metrics used in the proposed work are describes as follow
Table 3
Model | MAPE | MAE | MSE | RMSE | R2 |
Machine Learning Models |
SVM | 3.08% | 38525.54 | 1994839288.27 | 44663.62 | 0.84 |
KNN | 1.35% | 17938.21 | 794784034.67 | 28191.91 | 0.94 |
Random Forest | 1.05% | 13629.01 | 342898580.64 | 18517.52 | 0.97 |
Gradient Boost | 1.09% | 14204.60 | 380139800.03 | 19497.17 | 0.97 |
Deep Learning Models |
LSTM | 1.35% | 17232.49 | 446265580.40 | 21125.00 | 0.96 |
RNN | 0.91% | 11692.12 | 245462958.54 | 15667.26 | 0.98 |
CNN | 1.17% | 15263.26 | 417714732.56 | 20438.07 | 0.97 |
Facebook Prophet model | 0.02% | 29479.49 | 1557694333.18 | 39467.64 | 0.87 |
Hybrid Model | | | | | |
Hybrid LSTM + Gradient Boost | 1.08% | 13961.64 | 359225395.53 | 18953.24 | 0.97 |
Hybrid Gradient Boost + SVM | 3.17% | 39763.68 | 2104915743.43 | 45879.36 | 0.83 |
Proposed GBHT Model | | | | | |
Gradient Boost with Grid Search based Hyper parameter tuning | 0.01% | 166.62 | 286635.90 | 535.38 | 1.00 |
Best Parameters: {'learning_rate': 0.3, 'max_depth': 5, 'n_estimators': 400} |
Mean Absolute Error (MAE), a measure of the average absolute difference between predicted and actual values. MAE provides a glimpse into the magnitude of errors committed by the prediction model, with each discrepancy contributing to the collective understanding of its performance.
Mean Absolute Percentage Error (MAPE), a relative metric that unveils the average percentage deviation between predicted and actual values. It expresses the prediction error as a percentage of the actual value, enabling us to comprehend the relative impact of errors in the context of the real-world domain.
Mean Squared Error (MSE), an metric that captures the average of squared differences between anticipated and actual values. Through MSE, we embrace the inherent variability of predictions and unlock deeper insights into the model's performance.
Root Mean Squared Error (RMSE). It emerges as the square root of MSE, possessing the remarkable ability to measure the average magnitude of errors in the same units as the original data.
$$RMSE = \surd \left(MSE\right)$$
Coefficient of Determination (R2), a force that unveils the proportion of variance in actual values explained by the predicted values. R2, known as R-squared. It shows how well the prediction model fits the data that was actually observed.
where SST is the total sum of squares and SSR is the sum of squared residuals, y is the predicted values and ŷ is the actual value.
In the aforementioned equations, n is the number of data points. Higher values of R2 signify a better fit of the model to the data, whereas lower values of other metrics signify better prediction accuracy [39]. The effectiveness and dependability of the CPU utilization prediction model in the cloud context are evaluated and proposed GBHT model performance compared with other popular models. Results shows that GBHT model exhibits the highest level of accuracy and reliability in CPU utilization prediction shown from Fig. 3 to Fig. 16.
The performance of the models was evaluated using several key evaluation metrics: MAE, MSE, RMSE, and R2 as shown in Table 3. Total 7 models selected for comparison, which further divided into 4 categories i.e. machine learning (SVM, KNN, Random Forest, Gradient Boost), deep learning (LSTM, RNN, CNN), time series model (Facebook Prophet) and as well as the hybrid models, combining LSTM with Gradient Boost and Gradient Boost with SVM., time series and hybrid models. Among the machine learning models, SVM had a MAPE of 3.08%, indicating relatively higher prediction errors. KNN performed better with a lower MAPE of 1.35%, followed closely by Random Forest and Gradient Boost, both achieving MAPE values below 1.1%. These models exhibited strong predictive accuracy with high R2 scores ranging from 0.94 to 0.97, indicating a good correlation between predicted and actual values. In the realm of deep learning, LSTM achieved a MAPE of 1.35%, while RNN demonstrated even lower error with a MAPE of 0.91%. CNN also performed well with a MAPE of 1.17%. These models showcased high R2 scores ranging from 0.96 to 0.98, indicating their ability to accurately predict target values. The time series model, Facebook Prophet, stood out with outstanding results, boasting an extremely low MAPE of 0.02%. This model demonstrated superior prediction accuracy with an R2 score of 0.87. Among the hybrid models, the Hybrid LSTM + Gradient Boost achieved a MAPE of 1.08% and maintained consistent accuracy. However, the Hybrid Gradient Boost + SVM exhibited a higher MAPE of 3.17%, indicating relatively higher prediction errors.
The proposed GBHT model fared better than all other models, with a remarkable MAPE of 0.01%. It achieved perfect predictive accuracy with an R2 score of 1.00, indicating a flawless correlation between predicted and actual values. The GBHT model demonstrated its superiority by significantly reducing errors and optimizing hyper parameters through grid search [40]. These results validate the effectiveness of the GBHT model in achieving highly accurate predictions and highlight the importance of hyperparameter tuning in optimizing model performance.