In the experimental study, two different study groups were generated, and three different experimental works were performed for every group. Moreover, two different data sets were generated for every study group. In the first study group, the images obtained from Lorenz and Rossler chaotic systems were used. In the second group, the images obtained from Lorenz and Chen chaotic systems were used. In every group, the time series images of the x, y and z state variables of the chaotic systems were classified separately, and their classification performance was evaluated.
In the data set which contains the images of the time series of Lorenz and Rossler chaotic system, 600 images of the x state variable of Lorenz chaotic system are created for the first class and 600 images of the x state variable of Rossler chaotic system are created for the second class. By using various pre-trained networks, the classification tests were performed on these total 1200 images. In the data set which contains the images of the time series of Lorenz and Chen chaotic system, 720 images of the x state variable of Chen chaotic system are created for the first class and 600 images of the x state variable of Lorenz chaotic system are created for the second class. By using various pre-trained networks, the classification tests were performed on these total 1300 images. In the study, tests were conducted with numerous pre-trained networks and the 8 pre-trained networks with highest performance were used. These networks are Squeezenet, VGG-19, AlexNet, ResNet50, ResNet101, DenseNet201, ShuffleNet, and GoogLeNet.
In the study, the classification of the images of the time series of totally different systems like Lorenz-Rossler and the classification of the images of the time series of similar systems like Lorenz – Chen is performed by utilizing deep neural networks. The purpose here is evaluation of classification performance of different time series images and similar time series images and compare these performance results.
5.1 Evaluation of results and performance of Lorenz - Rossler systems
In the first experimental study of this group, the images obtained from the time series of x state variable of Lorenz and Rossler chaotic systems are classified. In Figure 12, the sample images of the time series of x state variable of Lorenz chaotic system are given.
In Figure 13, the sample images of the time series of \(x\) state variable of Rossler chaotic system are given.
As it is seen in Figures 12 and 13, the series of the two systems are very different from each other and this makes the performance of the classification to be very high. Thanks to the preferred pre-trained networks and the optimizations performed on these networks, higher classification performance results are observed in the study. In Table 4, the opposition matrices of the images shown in Figures 12 and 13 are given. From Table 4, the classification performance of the pre-trained networks is seen clearly.
Table 4
x-coordinate time series opposition matrices of the Lorenz and Rossler chaotic systems.
Network
|
True Positive (TP)
|
True Negative (TN)
|
False Positive (FP)
|
False Negative (FN)
|
SqueezeNet
|
180
|
169
|
0
|
11
|
VGG-19
|
153
|
179
|
27
|
1
|
AlexNet
|
170
|
180
|
10
|
0
|
ResNet50
|
179
|
163
|
1
|
17
|
ResNet101
|
166
|
177
|
14
|
3
|
DenseNet201
|
167
|
180
|
13
|
0
|
ShuffleNet
|
176
|
151
|
4
|
29
|
GoogLeNet
|
173
|
168
|
7
|
12
|
In Table 5, the performance of the pre-trained networks on the classification of images shown in Figure 12 and 13. As it is seen in Table 5, the highest classification performance is obtained with SqueezeNet and DenseNet201.
Table 5
x-coordinate time series performance metrics of the Lorenz and Rosler chaotic systems.
Network
|
Accuracy
|
Precision
|
Sensitivity
|
Specificity
|
SqueezeNet
|
0.969444
|
1.000000
|
0.942408
|
1.000000
|
VGG-19
|
0.922222
|
0.850000
|
0.993506
|
0.868932
|
AlexNet
|
0.972222
|
0.944444
|
1.000000
|
0.947368
|
ResNet50
|
0.950000
|
0.994444
|
0.913265
|
0.993902
|
ResNet101
|
0.952778
|
0.922222
|
0.982249
|
0.926702
|
DenseNet201
|
0.963889
|
0.927778
|
1.000000
|
0.932642
|
ShuffleNet
|
0.908333
|
0.977778
|
0.858537
|
0.974194
|
GoogLeNet
|
0.947222
|
0.961111
|
0.935135
|
0.960000
|
In the second experimental study of this group, the images obtained from the time series of y state variable of Lorenz and Rossler chaotic systems are classified. In Figure 14, the sample images of the time series of y state variable of Lorenz chaotic system are given.
In Figure 15, the sample images of the time series of y state variable of Rossler chaotic system are given.
As it is seen in Figures 14 and 15, the series of the two systems are very different from each other and this makes the performance of the classification to be very high. Thanks to the preferred pre-trained networks and the optimizations performed on these networks, higher classification performance results are observed in the study. In Table 6, the opposition matrices of the images shown in Figures 14 and 15 are given. From Table 6, the classification performance of the pre-trained networks is seen clearly.
Table 6
y-coordinate time series opposition matrices of the Lorenz and Rosler chaotic systems.
Network
|
True Positive (TP)
|
True Negative (TN)
|
False Positive (FP)
|
False Negative (FN)
|
SqueezeNet
|
178
|
164
|
2
|
16
|
VGG-19
|
177
|
180
|
3
|
0
|
AlexNet
|
166
|
167
|
14
|
13
|
ResNet50
|
176
|
180
|
4
|
0
|
ResNet101
|
164
|
180
|
16
|
0
|
DenseNet201
|
178
|
173
|
2
|
7
|
ShuffleNet
|
160
|
180
|
20
|
0
|
GoogLeNet
|
176
|
179
|
4
|
1
|
In Table 7, the performance of the pre-trained networks on the classification of images shown in Figure 14 and 15. As it is seen in Table 7, the highest classification performance is obtained with SqueezeNet and DenseNet201.
Table 7
y-coordinate time series performance metrics of the Lorenz and Rosler chaotic systems.
Network
|
Accuracy
|
Precision
|
Sensitivity
|
Specificity
|
SqueezeNet
|
0.950000
|
0.988889
|
0.917526
|
0.987952
|
VGG-19
|
0.991667
|
0.983333
|
1.000000
|
0.983607
|
AlexNet
|
0.925000
|
0.922222
|
0.927374
|
0.922652
|
ResNet50
|
0.988889
|
0.977778
|
1.000000
|
0.978261
|
ResNet101
|
0.955556
|
0.911111
|
1.000000
|
0.918367
|
DenseNet201
|
0.975000
|
0.988889
|
0.962162
|
0.988571
|
ShuffleNet
|
0.944444
|
0.888889
|
1.000000
|
0.900000
|
GoogLeNet
|
0.986111
|
0.977778
|
0.994350
|
0.978142
|
In the last experimental study of this group, the images obtained from the time series of z state variable of Lorenz and Rossler chaotic systems are classified. In Figure 16, the sample images of the time series of z state variable of Lorenz chaotic system are given.
In Figure 17, the sample images of the time series of z state variable of Rossler chaotic system are given.
As it is seen in Figures 16 and 17, the series of the two systems are very different from each other and this makes the performance of the classification to be very high. Thanks to the preferred pre-trained networks and the optimizations performed on these networks, higher classification performance results are observed in the study. In Table 8, the opposition matrices of the images shown in Figures 16 and 17 are given. From Table 8, the classification performance of the pre-trained networks is seen clearly.
Table 8
z-coordinate time series opposition matrices of the Chen and Rossler chaotic systems.
Network
|
True Positive (TP)
|
True Negative (TN)
|
False Positive (FP)
|
False Negative (FN)
|
SqueezeNet
|
180
|
176
|
0
|
4
|
VGG-19
|
173
|
180
|
7
|
0
|
AlexNet
|
180
|
163
|
0
|
17
|
ResNet50
|
162
|
180
|
18
|
0
|
ResNet101
|
176
|
180
|
4
|
0
|
DenseNet201
|
179
|
180
|
1
|
0
|
ShuffleNet
|
172
|
180
|
8
|
0
|
GoogLeNet
|
179
|
180
|
1
|
0
|
In Table 9, the performance of the pre-trained networks on the classification of images shown in Figure 16 and 17. As it is seen in Table 9, the highest classification performance is obtained with SqueezeNet and DenseNet201.
Tablo 9. z-coordinate time series performance metrics of the Lorenz and Rossler chaotic systems.
Network
|
Accuracy
|
Precision
|
Sensitivity
|
Specificity
|
SqueezeNet
|
0.988889
|
1.000000
|
0.978261
|
1.000000
|
VGG-19
|
0.980556
|
0.961111
|
1.000000
|
0.962567
|
AlexNet
|
0.952778
|
1.000000
|
0.913706
|
1.000000
|
ResNet50
|
0.950000
|
0.900000
|
1.000000
|
0.909091
|
ResNet101
|
0.988889
|
0.977778
|
1.000000
|
0.978261
|
DenseNet201
|
0.997222
|
0.994444
|
1.000000
|
0.994475
|
ShuffleNet
|
0.977778
|
0.955556
|
1.000000
|
0.957447
|
GoogLeNet
|
0.997222
|
0.994444
|
1.000000
|
0.994475
|
5.2 Evaluation of results and performance of Lorenz - Chen systems
In the first experimental study of this group, the images obtained from the time series of x state variable of Lorenz and Chen chaotic systems are classified. In Figure 12, the sample images of the time series of x state variable of Lorenz chaotic system are given. In Figure 18, the sample images of the time series of x state variable of Chen chaotic system are given.
The time series given in Figure 12 and 18 of Lorenz and Chen systems, respectively, are close to each other since Lorenz and Chen systems are similar systems. This makes the classification of these signals difficult. However, high classification performance results are observed in the study with the help of the preferred pre-trained networks and the optimizations performed on these networks. In Table 10, the opposition matrices of the images shown in Figures 12 and 18 are given. From Table 10, the classification performance of the pre-trained networks is seen clearly.
Table 10
x-coordinate time series opposition matrices of the Lorenz and Chen chaotic systems.
Network
|
True Positive (TP)
|
True Negative (TN)
|
False Positive (FP)
|
False Negative (FN)
|
SqueezeNet
|
175
|
209
|
5
|
7
|
VGG-19
|
178
|
186
|
2
|
30
|
AlexNet
|
144
|
213
|
36
|
3
|
ResNet50
|
173
|
200
|
7
|
16
|
ResNet101
|
176
|
216
|
4
|
0
|
DenseNet201
|
158
|
216
|
22
|
0
|
ShuffleNet
|
174
|
176
|
6
|
40
|
GoogLeNet
|
164
|
215
|
16
|
1
|
In Table 11, the performance of the pre-trained networks on the classification of images shown in Figure 12 and 18. As it is seen in Table 11, the highest classification performance is obtained with SqueezeNet and DenseNet201.
Table 11
x-coordinate time series performance metrics of the Lorenz and Chen chaotic systems.
Network
|
Accuracy
|
Precision
|
Sensitivity
|
Specificity
|
SqueezeNet
|
0.969697
|
0.972222
|
0.961538
|
0.976636
|
VGG-19
|
0.919192
|
0.988889
|
0.855769
|
0.989362
|
AlexNet
|
0.901515
|
0.800000
|
0.979592
|
0.855422
|
ResNet50
|
0.941919
|
0.961111
|
0.915344
|
0.966184
|
ResNet101
|
0.989899
|
0.977778
|
1.000000
|
0.981818
|
DenseNet201
|
0.944444
|
0.877778
|
1.000000
|
0.907563
|
ShuffleNet
|
0.883838
|
0.966667
|
0.813084
|
0.967033
|
GoogLeNet
|
0.957071
|
0.911111
|
0.993939
|
0.930736
|
In the second experimental study of this group, the images obtained from the time series of y state variable of Lorenz and Chen chaotic systems are classified. In Figure 14, the sample images of the time series of y state variable of Lorenz chaotic system are given. In Figure 19, the sample images of the time series of y state variable of Chen chaotic system are given.
The time series given in Figure 14 and 19 of Lorenz and Chen systems, respectively, are close to each other since Lorenz and Chen systems are similar systems. This makes the classification of these signals difficult. However, high classification performance results are observed in the study with the help of the preferred pre-trained networks and the optimizations performed on these networks. In Table 12, the opposition matrices of the images shown in Figures 14 and 19 are given. From Table 12, the classification performance of the pre-trained networks is seen clearly.
Table 12
y-coordinate time series opposition matrices of the Chen and Rossler chaotic systems.
Network
|
True Positive (TP)
|
True Negative (TN)
|
False Positive (FP)
|
False Negative (FN)
|
SqueezeNet
|
177
|
202
|
3
|
14
|
VGG-19
|
180
|
191
|
0
|
25
|
AlexNet
|
167
|
191
|
13
|
25
|
ResNet50
|
180
|
202
|
0
|
14
|
ResNet101
|
153
|
216
|
27
|
0
|
DenseNet201
|
165
|
215
|
15
|
1
|
ShuffleNet
|
176
|
176
|
4
|
40
|
GoogLeNet
|
175
|
210
|
5
|
6
|
In Table 13, the performance of the pre-trained networks on the classification of images shown in Figure 14 and 19. As it is seen in Table 13, the highest classification performance is obtained with SqueezeNet and DenseNet201.
Table 13
y-coordinate time series performance metrics of the Lorenz and Chen chaotic systems.
Network
|
Accuracy
|
Precision
|
Sensitivity
|
Specificity
|
SqueezeNet
|
0.957071
|
0.983333
|
0.926702
|
0.985366
|
VGG-19
|
0.936869
|
1.000000
|
0.878049
|
1.000000
|
AlexNet
|
0.904040
|
0.927778
|
0.869792
|
0.936275
|
ResNet50
|
0.964646
|
1.000000
|
0.927835
|
1.000000
|
ResNet101
|
0.931818
|
0.850000
|
1.000000
|
0.888889
|
DenseNet201
|
0.959596
|
0.916667
|
0.993976
|
0.934783
|
ShuffleNet
|
0.888889
|
0.977778
|
0.814815
|
0.977778
|
GoogLeNet
|
0.972222
|
0.972222
|
0.966851
|
0.976744
|
In the last experimental study of this group, the images obtained from the time series of z state variable of Lorenz and Chen chaotic systems are classified. In Figure 16, the sample images of the time series of z state variable of Lorenz chaotic system are given In Figure 20, the sample images of the time series of z state variable of Chen chaotic system are given.
The time series given in Figure 16 and 20 of Lorenz and Chen systems, respectively, are close to each other since Lorenz and Chen systems are similar systems. This makes the classification of these signals difficult. However, high classification performance results are observed in the study with the help of the preferred pre-trained networks and the optimizations performed on these networks. In Table 14, the opposition matrices of the images shown in Figures 16 and 20 are given. From Table 14, the classification performance of the pre-trained networks is seen clearly.
Table 14
z-coordinate time series opposition matrices of the Lorenz and Chen chaotic systems.
Network
|
True Positive (TP)
|
True Negative (TN)
|
False Positive (FP)
|
False Negative (FN)
|
SqueezeNet
|
170
|
209
|
10
|
7
|
VGG-19
|
167
|
194
|
13
|
22
|
AlexNet
|
150
|
212
|
30
|
4
|
ResNet50
|
175
|
205
|
5
|
11
|
ResNet101
|
171
|
202
|
9
|
14
|
DenseNet201
|
173
|
190
|
7
|
26
|
ShuffleNet
|
153
|
194
|
27
|
22
|
GoogLeNet
|
160
|
213
|
20
|
3
|
In Table 15, the performance of the pre-trained networks on the classification of images shown in Figure 16 and 20. As it is seen in Table 15, the highest classification performance is obtained with SqueezeNet and DenseNet201.
Table 15
z-coordinate time series performance metrics of the Lorenz and Chen chaotic systems.
Network
|
Accuracy
|
Precision
|
Sensitivity
|
Specificity
|
SqueezeNet
|
0.957071
|
0.944444
|
0.960452
|
0.954338
|
VGG-19
|
0.911616
|
0.927778
|
0.883598
|
0.937198
|
AlexNet
|
0.914141
|
0.833333
|
0.974026
|
0.876033
|
ResNet50
|
0.959596
|
0.972222
|
0.940860
|
0.976190
|
ResNet101
|
0.941919
|
0.950000
|
0.924324
|
0.957346
|
DenseNet201
|
0.916667
|
0.961111
|
0.869347
|
0.964467
|
ShuffleNet
|
0.876263
|
0.850000
|
0.874286
|
0.877828
|
GoogLeNet
|
0.941919
|
0.888889
|
0.981595
|
0.914163
|