During the experimentation, we took the system outcome in two forms: one with a feature selection step and one without. Precision, Recall, F1 Score, Cross Validation, Train Score, and Test Score are the outcomes. The experiment was done using a Laptop where the operating system was Windows 10 Enterprise 64-bit, Processor was Intel(R) Core(TM) i3 CPU and Python programming language was used.
Table 1
Performance Evaluation with Feature Selection
Algorithm
|
Precision
|
Recall
|
F1 Score
|
Accuracy
|
LR
|
92.351
|
92.432
|
92.389
|
92.312
|
GNB
|
91.762
|
89.231
|
89.176
|
89.19
|
SVM
|
97.543
|
97.327
|
97.160
|
97.155
|
DT
|
99.520
|
99.500
|
99.054
|
99.404
|
ET
|
99.381
|
99.010
|
99.981
|
99.986
|
VC
|
92.043
|
98.061
|
94.119
|
94.112
|
RF
|
99.580
|
99.815
|
99.985
|
99.986
|
KNN
|
98.451
|
98.243
|
98.308
|
98.306
|
BC
|
99.031
|
99.053
|
99.987
|
99.986
|
GB
|
98.140
|
98.521
|
98.390
|
98.391
|
AB
|
97.054
|
98.021
|
98.912
|
98.915
|
XG
|
99.581
|
99.851
|
99.986
|
99.986
|
Table 1 shows the Performance Evaluation (With Feature Selection) with different machine learning algorithms. An experiment conducted with Feature selection in this for parameters Precision, Recall, and F1 Score Random Forest (RF) gives a 100% score Decision tree (DT) gives a 99% score for each parameter. Also for parameter Accuracy, Random Forest (RF) gives a 99% score and DT gives a 99% score. From the observations, we can state that RF and DT give more accurate results.
Table 2
Performance Evaluation without Feature Selection
Algorithm
|
Precision
|
Recall
|
F1 Score
|
Accuracy
|
LR
|
87.540
|
87.570
|
87.530
|
0.870
|
GNB
|
58.643
|
87.562
|
47.783
|
56.723
|
SVM
|
57.872
|
69.952
|
70.562
|
54.982
|
DT
|
99.340
|
99.351
|
99.302
|
99.370
|
ET
|
99.215
|
99.148
|
99.186
|
99.190
|
VC
|
89.061
|
97.023
|
93.017
|
93.201
|
RF
|
99.465
|
99.389
|
99.981
|
99.982
|
KNN
|
98.471
|
98.364
|
99.286
|
99.386
|
BC
|
99.641
|
99.582
|
99.981
|
99.986
|
GB
|
99.281
|
99.472
|
98.562
|
98.600
|
AB
|
98.532
|
99.190
|
99.863
|
99.806
|
XG
|
99.682
|
99.840
|
99.656
|
99.800
|
The Same Machine Learning Algorithms are executed without feature selection and then we check the performance of it. Table 4.2 shows the Performance Evaluation (Without Feature Selection) with different machine learning algorithms.Experiment conducted without Feature selection in this for parameters Precision, Recall, F1 Score and Accuracy Bagging Classifier (BC) and XgBoost (XB) Achieves 100% Score.
Decision Tree (DT), Extra Tree Classifier (ETC) Random Forest (RF) and Gradient Boosting (GB) achieves 99% Score for Precision and Recall parameter.
From the observations we can state that without feature selection Decision Tree (DT), Random Forest (RF), Bagging Classifier (BC), Xgboost (XB) and Gradient Boosting (GB) gives accurate results.
Figure2 shows the proposed system accuracy with & without feature selection. For Accuracy parameter Random Forest (RF), Bagging Classifier (BC), Extra Tree Classifier (ETC) and Xgboost (XB) achieves 100% score. Also Decision tree (DT) achieves 99% score.
With Feature selection Random Forest (RF) Precision value is 100%, Decision Tree (DT), Extra Tree Classifier (ETC), Bagging Classifier (BC) and Xgboost (XB) precision value is 99%.
Without Feature selection Decision Tree(DT),Extra Tree Classifier(ETC),Random Forest (RF),Bagging Classifier(BC),Gradient Boosting(GB),Adaboost Classifier(ABC)and Xgboost (XB) Precision value is 99%. K Nearest Neighbour (KNN) precision value is 0.98%.
Above mentioned Fig. 3 Shows a Comparison of Precision values with & without feature selection in each algorithm. With Feature selection Random Forest (RF) Precision value is 1, Gradient Boosting (GB) and K Nearest Neighbour (KNN) precision value is 0.98. Without Feature selection, Random Forest (RF) Precision value is 0.99, Gradient Boosting (GB) Precision value is 0.99, and K Nearest Neighbour (KNN) and Decision Tree (DT) precision value is 0.97.
Figure4 shows a Comparison of the Recall value with & without feature selection in each algorithm. With Feature selection Random Forest (RF) Recall value is 100\%. Decision Tree (DT), Extra Tree Classifier (ETC), Bagging Classifier (BC) and Xgboost (XB) Recall value is 99%.
Without Feature Decision Tree (DT), Extra Tree Classifier (ETC), Random Forest (RF), Bagging Classifier (BC), Gradient Boosting (GB), Adaboost Classifier (ABC) and Xgboost (XB) Recall value is 99%. K Nearest Neighbour (KNN) precision value is 98%.
Also, the Experiment conducted without feature selection the goal of CV is to evaluate a machine learning model's ability to predict new data. Experiment conducted with feature selection for parameter CV Score of Decision tree (DT) and Random Forest (RF) gives 99% score.
Also Experiment conducted without feature selection for parameter CV score of Decision Tree (DT) and Random Forest (RF) gives 97% score. In both the experimentation Random Forest (RF) gives highest score. So we can say that Random Forest (RF) achieves high prediction ability.
Table 3
Performance Evaluation with Feature Selection
Algorithm
|
Train Score
|
Test Score
|
CV Score
|
LR
|
92.877
|
92.312
|
92.849
|
GNB
|
89.48
|
89.191
|
0.892
|
SVM
|
96.931
|
97.155
|
96.756
|
DT
|
100.000
|
99.364
|
99.5
|
ET
|
99.996
|
99.986
|
99.0132
|
VC
|
94.395
|
94.112
|
94.38
|
RF
|
99.992
|
99.986
|
99.637
|
KNN
|
98.0945
|
98.121
|
96.665
|
BC
|
99.996
|
99.986
|
99.62
|
GB
|
99.448
|
99.391
|
99.296
|
AB
|
98.9004
|
98.915
|
98.758
|
XG
|
99.992
|
99.986
|
99.676
|
Table 3 shows the Train, Test, and CV Score with feature selection. Train Score of Decision Tress (DT) is 100%. Random Forest (RF),Extra Tree Classifier (ETC),Bagging Classifier(BC),Gradient Boosting(GB) and Xgboost (XB)algorithm has a Train, Test Score and CV Score is 99%.From Observations we can say that Random Forest (RF),Bagging Classifier(BC),Xgboost (XB) and Decision Tress (DT) algorithm achieves higher accuracy for classification and prediction of attacks.
Table 4
Train, Test, and CV Score without feature selection
Algorithm
|
Train Score
|
Test Score
|
CV Score
|
LR
|
92.877
|
92.312
|
92.849
|
GNB
|
89.48
|
89.191
|
0.892
|
SVM
|
96.931
|
97.155
|
96.756
|
DT
|
100.000
|
99.364
|
99.5
|
ET
|
99.996
|
99.986
|
99.0132
|
VC
|
94.395
|
94.112
|
94.38
|
RF
|
99.992
|
99.986
|
99.637
|
KNN
|
98.0945
|
98.121
|
96.665
|
BC
|
99.996
|
99.986
|
99.62
|
GB
|
99.448
|
99.391
|
99.296
|
AB
|
98.9004
|
98.915
|
98.758
|
XG
|
99.992
|
99.986
|
99.676
|
Table 4 shows the Train, Test, and CV Score without feature selection. Train Score of Decision Tress (DT) is 100\%.
Random Forest (RF),Extra Tree Classifier (ETC),Bagging Classifier(BC),Gradient Boosting(GB),Adaboost Classifier(AC) and Xgboost (XB)algorithm has a Train, Test Score and CV Score is 99%.From Observations we can say that Random Forest (RF) and Decision Tress (DT) algorithm achieves higher accuracy for classification and prediction of attacks.
From the Observations, we can state that Extra Tree Classifier (ETC), Random Forest (RF), Bagging Classifier (BC) and Xgboost (XB) Algorithms achieves higher accuracy in terms of classification and prediction in both models (With & without Feature selection) execution. Next, K Nearest Neighbour (KNN), Decision Tree (DT), Adaboost (AC) and Gradient Boosting (GB) Algorithms also achieve higher accuracy in terms of classification and prediction in both (With & without Feature selection) models execution.
After analyzing the performance of the system we came to know that there is a minor difference in results with feature selection and without. Results may vary depending on the dataset values.