5.2 Methodology
Due to the different results of the evolutionary algorithm each time, the GAEWHH algorithm is evaluated by the Monte Carlo method, which is a type of statistical method.
In the experiment, 8 algorithms are compared to analyze their performance. There have been many studies on large-scale combinatorial optimization problems, from which we select two representative algorithms. First, OR-Tools developed by Google is a stable and effective optimization algorithm, which has been verified by a large number of experiments. Second, PSO is a typical evolutionary algorithm that is a hotspot of current research. Finally, based on the four EWOs, six operator combinations are designed for the LL, and the performance of these combinations is analyzed.
5.3 Data Sets
This study conducts experiments on data sets of different scales to examine the computational speed and optimality of the algorithm. To illustrate the experimental results, three groups of typical experimental data are displayed, namely, Data Set 1 (5×5), Data Set 2 (10×10), and Data Set 3 (400×400). Data Set 1 and Data Set 2 are reserved for comparison with the existing literature, and Data Set 3 is the maximum data scale allowed by the experimental equipment of this study.
5.4 Results and Analysis
The results of the comparative experiment are presented in Fig. 2.
Figure 5 consists of 9 subgraphs in 3 rows and 3 columns. Each column presents the results of the different data sets. The first row of subgraphs presents the convergence rate of the different algorithms to solve the problem. The second row shows the optimization capability of the algorithms. The third row presents the running time of the algorithms, which is used to analyze the complexity. Therefore, the experiments are analyzed from three aspects, namely, the convergence rate, optimization capability and algorithm complexity.
5.4.1 Convergence Rate
Convergence speed is one of the indicators for evaluating evolutionary algorithms; here, OR-Tools is not considered. The first row of subgraphs shows that when the data scale is relatively small, all algorithms can converge quickly. As the data scale grows, evolutionary algorithms require more iterations to converge. The average number of iterations of the GAEWHH algorithm, which includes FBSR, FSR, F, B, S and R, is less than 50, and the average number of iterations of the PSO algorithm is more than 150. Therefore, the convergence speed of the GAEWHH algorithm is generally better than that of PSO.
The main reason for this phenomenon is that operators in the HH framework can be flexibly designed for the LL according to the problem domain without compromising for the HL. Thus, the HH framework can better search the problem space. In the design of traditional heuristics, it is necessary to consider both the search behavior characteristics and the coding method of the problem domain. Therefore, the convergence ability of the HH framework is not worse than that of traditional heuristic methods.
5.4.2 Optimization Capability
The optimization capability is the most important index to evaluate the optimization algorithm. The results of each running of the evolutionary algorithm are generally different. Experiments were repeated and analyzed by statistical methods. Table 2 shows the results of 100 repeated experiments.
In the table below, a represents the mean, B represents the maximum value, and C represents the standard deviation.
Table 2
| Data Set 1 | Data Set 2 | Data Set 3 |
| VA | VM | Vs | VA | VM | Vs | VA | VM | Vs |
PSO | 3.65 | 3.65 | 0.00 | 7.05 | 7.26 | 0.10 | 7063.46 | 7360.14 | 91.23 |
OR-Tools | 3.65 | 3.65 | 0.00 | 7.26 | 7.26 | 0.00 | 11947.90 | 11947.90 | 0.00 |
FBSR | 3.65 | 3.65 | 0.00 | 7.26 | 7.26 | 0.00 | 11876.86 | 11884.15 | 2.41 |
FSR | 3.65 | 3.65 | 0.00 | 7.15 | 7.26 | 0.08 | 11846.20 | 11854.49 | 4.06 |
F | 3.65 | 3.65 | 0.00 | 6.71 | 6.71 | 0.00 | 11812.36 | 11812.36 | 0.00 |
B | 3.51 | 3.51 | 0.00 | 6.95 | 6.95 | 0.00 | 11851.68 | 11851.68 | 0.00 |
S | 2.75 | 2.75 | 0.00 | 5.62 | 5.62 | 0.00 | 11650.63 | 11650.63 | 0.00 |
R | 3.65 | 3.65 | 0.00 | 7.03 | 7.26 | 0.10 | 6704.33 | 6840.56 | 47.52 |
Here, VA is the mean value, VM is the maximum value, and Vs is the standard deviation.
As illustrated in Table 2, the best solution is found each time on Data Set 1 by the six algorithms, namely, PSO, OR-Tools, FBSR, FSR, F, and R. Because of the problem of falling into the local optimum, the GAEWHH algorithm with the B or S operator does not obtain the best solution. In the test of Data Set 2, only the OR-Tools and FBSR algorithms can obtain the global optimal solution every time they run. The PSO, FSR, and R algorithms can find the optimal solution in multiple runs, and the results fluctuate. The mean value of the FSR algorithm is better than that of PSO and more stable. In the third group of large-scale test data, only OR-Tools can obtain the global optimal solution, the optimization performance of FBSR and FSR is slightly weaker, and the mean and stability of FBSR are better than those of FSR. The mean value of PSO is small, and the results are unstable. In the third group of large-scale test data, only OR-Tools can obtain the global optimal solution, the optimization performance of FBSR and FSR is slightly weaker, and the mean and stability of FBSR are better than those of FSR. The mean value of PSO is small, only slightly better than that of the R algorithm, and the solution of each run is not stable.
Three sets of experiments show that evolutionary algorithms can find optimal solutions in small-scale problems but generally cannot find optimal solutions in large-scale problems and use feasible solutions instead. In large-scale problem solving, the optimization ability of the GAEWHH algorithm is approximately 1.6 times that of PSO and can reach 99.4% that of OR-Tools. The repeatability of the GAEWHH algorithm is also much better than that of PSO.
Table 2 also shows that the design of LL heuristic operators has a great influence on the optimization capability. The optimization capability of combining the four EWOs is always better than that of using one EWO alone, which proves that the design of the EWO is reasonable. The B and F operators search the solution space from the global and local perspectives, respectively, which can search for excellent individuals from the population. The S and R operators can increase the diversity of the population.
5.4.3 Algorithm Complexity
The algorithm complexity is analyzed by counting the running time of different algorithms to repeatedly solve the optimization problem. The statistical results are presented in Table 3.
Table 3
Time consumption statistics
| Data Set 1 | Data Set 2 | Data Set 3 |
| VA | VM | Vs | VA | VM | Vs | VA | VM | Vs |
PSO | 0.464 | 0.636 | 0.019 | 0.468 | 0.717 | 0.026 | 8.250 | 8.400 | 0.047 |
OR-Tools | 0.001 | 0.003 | 0.000 | 0.003 | 0.008 | 0.001 | 130.050 | 139.392 | 1.032 |
FBSR | 0.055 | 0.157 | 0.016 | 0.101 | 0.234 | 0.019 | 1076.481 | 1096.387 | 8.277 |
FSR | 0.052 | 0.140 | 0.014 | 0.086 | 0.181 | 0.015 | 47.146 | 48.468 | 0.563 |
F | 0.047 | 0.145 | 0.014 | 0.073 | 0.156 | 0.014 | 43.669 | 46.486 | 0.317 |
B | 0.052 | 0.133 | 0.014 | 0.098 | 0.186 | 0.015 | 1460.057 | 1463.537 | 1.810 |
S | 0.052 | 0.220 | 0.021 | 0.080 | 0.171 | 0.016 | 58.394 | 58.927 | 0.175 |
R | 0.058 | 0.125 | 0.012 | 0.100 | 0.192 | 0.016 | 45.297 | 46.266 | 0.308 |
Table 3 and the third row in Fig. 2 show that the GAEWHH algorithm and PSO as evolutionary algorithms need population iteration operations, and the computation time is longer than that of OR-Tools in the small-scale data set. However, the speed advantage of the evolutionary algorithm emerges in the large-scale Data Set 3. Then, the difference in the time consumption between the GAEWHH algorithm and PSO is explained. The GAEWHH algorithm is associated with the HH framework and is problem independent. The PSO algorithm is dependent on the problem, and it uses decimal encoding specifically for the problem domain to adapt it to the PSO search mode. Therefore, in large-scale data sets, PSO takes less time than the GAEWHH algorithm with the same iterations. However, in small-scale data sets, the calculation work is low, and PSO mainly spends time in the data preparation stage, which takes longer than the GAEWHH algorithm.
The LL of the GAEWHH algorithm includes six EWO combinations, namely, FBSR, FSR, F, B, S and R. The time consumption of different operator combinations in the three experiments is consistent with the theoretical analysis of the EWO complexity. Taking the experiment of Data Set 3 as an example, the algorithm complexity of the B operator is the highest, which is O(n^2), and takes the longest time with a mean time of 1460.057 s. The algorithm complexity of the S operator is O(2n), which is much less time consuming at 58.394 s than the B operator. The algorithmic complexity of the F and R operators is O(n), and the corresponding running time is also smaller than (but similar to) that of the S operator but. FSR is a mixture of three operators, whose algorithm complexity is greater than O(n) but less than O(2n). The running time of FSR should also be shorter than that of the S operator and longer than that of the F or R operator. The experimental results support this conclusion. Similarly, the algorithm complexity of FBSR is greater than O(2n) but less than O(n^2), and the time consumption should be slightly less than that of the individual B operator. Because of the characteristics of the evolutionary algorithm, the solutions of each run are different; that is, the heuristic combinations are different. The algorithmic complexity of the B operator is much larger than that of other operators, which causes great fluctuations in the running time with a standard deviation of 8.277 s. The experiment also reflects this feature. Therefore, the experimental results are consistent with the algorithmic complexity of the theoretical analysis.
5.5 Experimental Summary
The GAEWHH algorithm can effectively solve the jamming resource allocation problem. The convergence rate of this algorithm is not worse than that of the common heuristic algorithms.
In this study, four LL heuristic EWOs, namely, F, B, S, and R, are first proposed for jamming resource allocation. The EWOs can efficiently search the problem domain, and the optimization capability of combined operators is better than that of the individual operators.
The running time of the GAEWHH algorithm is related to the algorithmic complexity of the LL operator combination. The complexity of the LL operator combination is between that of the minimum-complexity operator and the maximum-complexity operator. The appropriate operator combination is selected according to the problem demand. For example, in the large-scale Data Set 3, choosing a combination of FSR operators can well balance optimization capability and calculation speed.