In the process of preparing data for machine learning algorithms, an essential step involves exploring, visualizing, and preprocessing the available features. This step is crucial as it gives a clear overview of the data that will be fed into the machine learning model, helping to identify any areas that may need further refinement before proceeding.
The dataset utilised in this process consists of a substantial 740 rows and 15 columns. Such a volume of data provides a rich resource for the machine learning algorithms to draw from, thus increasing the potential accuracy of the forecasts produced.
In order to examine the details of the available columns in the dataset, the pandas data analysis library is employed. This powerful tool provides an efficient and effective means of handling and exploring the data in detail, allowing for a thorough examination of the dataset in its entirety.
This article goes a step further and delves into the relationships between the data by studying the correlation between different variables. This is done in order to select the most suitable data for the forecasting model. In this instance, the Pearson correlation is considered a useful tool for identifying quantities that have a high correlation with the ROA value. The results of this analysis can be seen in Figure 3. This step is crucial as it ensures that only the most relevant and impactful data is included in the forecasting model, thus increasing the probability of accurate and meaningful results.
The correlation matrix reveals that:
- Variables DSO, DIO, LEV, GRO, ROE, EBIT, GROS, and RE have a strong correlation with ROA.
- Variables DPO, CCC, NWC, SIZ, CR, and QR have a weak correlation with ROA.
Even with these correlations, all variables are included in the machine learning model to guarantee accurate predictions, including non-linear ones.
The results of models predicting return on assets (ROA) are given in Table 2. Among these, the RF model yields the best results when cross-validated with five iterations (n_splits).
The results of the models applied to predict the return on assets are presented in (Table 2).
Table 2: Comparison results of models
No.
|
Model
|
R2
|
RMSE
|
1
|
Lasso
|
0.5908
|
2.001
|
2
|
RR
|
0.7194
|
2.0012
|
3
|
KNR
|
0.452
|
2.79
|
4
|
SVR
|
0.3968
|
2.9339
|
5
|
RF
|
0.9762
|
0.5826
|
6
|
GBR
|
0.97
|
0.6528
|
7
|
XGBoost
|
0.9822
|
0.504
|
8
|
MLP
|
0.838
|
1.517
|
The results of ranking models R2 and RMSE indexes are shown in (Figure 4).
The model will be evaluated and selected based on three criteria:
- R Score: An optimal score is closer to 1.
- RMSE Score: A lower score is better.
- Learning Curves Chart: This chart displays the number of training and test points. The closer these two lines are, the better the model generalizes.
In deciding between the RF and XGBoost models, both of which have commendable R2 and RMSE scores, we observe a larger gap between the training and testing lines in XGBoost (1 unit) compared to RF (approximately 0.7 units). As such, this document will use the RF model for computation.
Calibration of RF model
The GridSearchCV tool gives you the power to discover the optimal parameters for your machine learning model. It achieves this by testing a variety of parameters and determining the model's performance for each set, using cross-validation techniques to evaluate the model across diverse training and validation datasets.
In simple terms, GridSearchCV breaks down the parameters into separate values and creates various parameter sets by combining these values. The model is then trained using each parameter set, with its performance evaluated through methods like cross-validation. The best parameter set is then selected based on the model's performance on the validation dataset.
- max_depth = 10.
- min_samples_leaf = 2.
- min_samples_split = 5.
- n_estimators = 200.
Looking at the ranking of important features, it's evident that the ROE index has the most significant impact on the ROA. Additionally, the variables LEV, EBIT, and CR also have a substantial influence on the ROA index (Figure 5).
From the learning curves (refer to Figure 6), we can make the following observations:
- The training score and cross-validation score remain relatively stable. As more samples are used for training, both scores tend to converge or nearly converge. This suggests that the model has strong generalization capabilities.
3.5 Prediction:
The predictive model outlined in this article can assist business owners in the following ways:
- Enhancing business decisions: The model provides a more accurate prediction of ROA when specific profit indicators have not been calculated, enabling businesses to predict future scenarios and analyze "What-If" situations. This supports planning and risk management.
- Streamlining and automating the prediction process: The application of machine learning models standardizes the prediction process, making it automated, quick, and less reliant on manual intervention.
- Integrating data, knowledge, and context: While this model only considers 15 variables, integrating additional variables and contexts into the analysis process can better reflect the actual situation and the influences from the external business environment.
When applying these machine learning models to forecast the ROA for construction businesses in 2023, we obtained the following results: (Figure 7)
Despite the impressive results (error < 1%), this study has not yet been validated with a larger dataset. This would involve a larger number of feature variables and require further enhancement of the computational model to improve the accuracy of the prediction tool.