Authors propose a new intelligent optimization algorithm. This algorithm tries to learn the cost function shape in order to decide which points must be evaluated, and how many optimization iterations are enough. As far as the authors know, there is no optimization algorithm that applies prediction with all the evaluated points. Authors have performed a comparison study of the error prediction made by both the proposed algorithm, and the best-known intelligent optimization algorithm: Particle Swarm Optimization. The results show that this new algorithm is able to learn different cost functions more accurately. The cost function set proposed in this article are continuous evaluated functions which have very diverse mathematical shapes. The authors have concluded that the proposed algorithm is able to choose the evaluation points more appropriately.
Article
Basque Optimization: a new cost function prediction based optimization algorithm
https://doi.org/10.21203/rs.3.rs-3869536/v1
This work is licensed under a CC BY 4.0 License
Version 1
posted
You are reading this latest preprint version
optimization
intelligent algorithm
particle swarm optimization (PSO)
cost function
Swarm intelligence based algorithms represent those that implement animal patterns, such as bird flocks, fish schools, animal herds or insect colonies and are used to solve optimization problems of any kind of branch. Therefore, there are agents performing collective behavior by interacting locally with their environment. These local rules and interactions between self-organized agents lead to the collective intelligence called swarm intelligence, see Karaboga et al. [1], Mavrovouniotis et al. [2] and Yang et al. [3]. A commonly used swarm intelligence based algorithm is particle swarm optimization (commonly referred to as PSO). At the present time, this algorithm is widely used to solve complex multi-objective and non-linear problems. However, at first, PSO was used to simulate the flocking process of birds, and it was with time that its capability as an optimizer was discovered, see the work of Eberhart et al. [4]. Nevertheless, PSO cannot ensure achieving an optimal solution, but it always improves with every iteration. PSO is similar to a genetic algorithm, where the system is initialized with a population of random solutions. For each solution, a randomized velocity is assigned, and the original solutions, which are called particles, are then flown through the problem space. Each particle keeps track of its coordinates in the problem space, which are associated with the best solution (fitness) it has achieved so far, see Eberhart et al. [4] and Zulueta et al. [5]. In the study of Poli [6], several works that have utilized PSO are investigated. Authors have successfully employed particle swarm optimization algorithms multiple times in Zulueta et al. [5], Martinez-Filgueira et al. [7] and Uriarte et al. [8].
Another widely used algorithm belonging to the swarm algorithms is the ant colony optimization proposed in M. Dorigo et al [9], where an optimization algorithm inspired by ant colony dynamics is proposed. Unlike the previous algorithm, the applicability of this algorithm is limited to transport optimization problems.
Evolutionary algorithms process a set of solutions in parallel, while they also exploit similarities of solutions via recombination [10]. A good example of evolutionary algorithms are genetic algorithms (GA). These algorithms are inspired by the evolution theories stating that the strongest species are more likely to survive and pass their genes to upcoming generations. Furthermore, during evolution, random gene changes can occur. If these changes are beneficial, new species evolve. However, unsuccessful changes are removed by natural selection. Following this process, GA operate with a collection of chromosomes which are known as population. This population is usually created randomly. With each iteration, the population achieves fitter solutions than before, until it eventually converges in a single solution [11]. Authors have previously worked with genetic algorithms in Sanchez-Chica et al. [12].
Another example of evolutionary algorithm is the differential evolution (DE). DE is a vector-based metaheuristic algorithm that is similar to genetic algorithms and pattern searching. As a matter of fact, it is possible to consider it a further development to GA. DE is a stochastic search algorithm with self-organizing tendency that does not utilize any derivative information. Therefore, it is a population-based and derivative-free method, as explained in Storn et al. [13] and Yang et al. [14]. Authors have applied this algorithm in some works, such as Centeno-Telleria et al. [15].
As stated by Wang et al. [16], Backtracking Search Optimization Algorithm (BSA) is one of the most recently proposed population-based evolutionary algorithms for global optimization. Unlike many search algorithms, BSA has a single control parameter. BSA has a simple structure that is effective, fast and capable of solving multimodal problems and that enables it to easily adapt to different numerical optimization problems, as the ones proposed by Civicioglu et al. [17] and Passos et al. [18].
Furthermore, there are other algorithms worth mentioning that represent a more classical view of the matter, such as Nelder–Mead simplex method or the Hillclimbing algorithm. On the one hand, the Nelder–Mead simplex algorithm, first published in 1965, is a popular direct search method for multidimensional unconstrained minimization, see Chang et al. [19] and Lagarias et al. [20]. On the other hand, the Hillclimbing algorithm, in which each point corresponds to a solution, and the height of the point corresponds to the fitness of the solution, aims to ascend to a peak by continuously moving to an adjacent state with a higher fitness, as shown by Muhlenbein et al. [21].
Authors have employed a trained neural network for this work, a multi-layer perceptron back-propagation (MLP-BP). A multilayer perceptron is a feed forward neural network that maps sets of input data onto a set of appropriate output [22]. On the other hand, backpropagation is the learning method that allows the MLP to iteratively adjust the weights in the network.
In typical swarm and evolutionary algorithms, the researcher has to do the work of knowing about the cost function in order to guide the algorithm to have a good optimal solution. This implies that if the researcher does not have a broad and deep prior knowledge of the problem on which is working, it is quite likely that an extensive series of trial and error runs of the algorithm will have to be performed. Which means spending a lot of time just to understand how to run the algorithm properly for that particular problem.
That the researcher having more knowledge about the problem is not a bad thing, it is interesting to develop tools that allow solving the problems while obtaining knowledge of the problem automatically during their solution.
In this work, the authors propose a new optimization algorithm called Basque Optimization that, in contrast to common used algorithms from the likes of swarm algorithms or evolutionary algorithms, does not stop when a solution is reached, but when the function is adequately known.
The proposed algorithm works with a number of possible solutions, which are distributed among three different solution types. The first type consists of the best positions (N1 solutions at each iteration), the second one covers the unknown positions (N2 solutions at each iteration), and the last one contains the known positions which the interpolation algorithm does not make a good prediction of (N3 solutions at each iteration). The process is repeated until the previously established stop conditions are fulfilled.
Finally, the achieved results are compared to the results of a PSO algorithm that works with the same data, in order to determine the difference between the proposed algorithm and a widely used one as PSO.
2.1. Algorithm design
Firstly, as explained before, the authors propose in a stochastic manner around three solution types. The best solutions N1, the unknown positions N2 and the positions that are not adequately predicted by the interpolation algorithm N3. The number of solution of each type is parametrized. The stop condition is established via a number of maximum iterations Nmaxiter.
Despite the goal of this optimization algorithm is to be able to work with real data, for this study, randomly generated data is employed. Therefore, a superior and inferior limit must be defined. This value is parametrized with the variable Limit. This variable represents both the superior and inferior limit. Being Limit the superior one, and – Limit the inferior. Thus, a matrix of Nvar rows and Ntotal columns is generated, where Nvar is the number of variables, and Ntotal is calculated following Eq. (1). This way, Ntotal number of positions are generated and ready to work with. These solutions are stored in a matrix called XVek.
$${N}_{total}={N}_{1}+{N}_{2}+{N}_{3}$$
1
Afterwards, the loss of every position must be acquired. For this matter, a function among a total of 37 selected optimization test functions is used. These functions can be found at [23]. In the end, all functions will be utilized so that the comparison study with the PSO algorithm is more meaningful. The calculated loss vector is used to sort the positions of the matrix from lowest to highest loss. They are some of the common functions and datasets used for testing optimization algorithms. They are grouped according to similarities in their significant physical properties and shapes. In Table 1 there is a list containing all the functions that have been used for this work. The functions are sorted based on their main visual characteristics.
| Many Local Minima | Bowl Shaped | Plate Shaped | Valley Shaped | Steep Ridges and Drops | Other functions |
|---|---|---|---|---|---|
| Ackley | Bohachevsky | Booth | Camel 3-Hump | De Jong N.5 | Beale |
| Bukin N.6 | Perm 0,d,β | Matyas | Camel 6-Hump | Michalewicz | Branin |
| Cross-in-Tray | Sum Squares | McCormick | Dixon-Price | Goldstein-Price | |
| Drop-Wave | Sphere | Zakharov | Rosenbrock | Perm d,β | |
| Eggholder | Trid | Styblinski-Tang | |||
| Griewank | Sum of Different Powers | ||||
| Holder Table | Rotated Hyper-Ellipsoid | ||||
| Langermann | |||||
| Levy | |||||
| Levy N.13 | |||||
| Rastrigin | |||||
| Schaffer N.2 | |||||
| Schaffer N.4 | |||||
| Schwefel | |||||
| Shubert |
Now that both the positions and their respective losses have been obtained, the optimization process begins. This process will be repeated as many times as the previously defined variable Nmaxiter iterations. To start with, the best solution is saved in a variable called Xbest. Using this solution, N1 number of solutions are randomly proposed. For the generation, the Xbest solution is taken, and a uniform distribution changes are (see Eq. (2)) added to generate a new solution. These solutions are stored in a variable named X1.
$$-\frac{Limit}{\varDelta }\le sum\le \frac{Limit}{\varDelta }$$
2
Where Δ is a parametrized value.
All the generated solutions are passed through a function that checks if all the coordinates of the solution fulfil the following condition (3–4):
$$-Limit<x<Limit$$
3
$$-Limit<y<Limit$$
4
Where x and y represent the coordinates of the solution. If any of these conditions is not achieved, the current solution is removed.
The next step consists of interpolating the loss from the recently generated solutions taking into account the solutions stored in XVek and their respective losses. For this issue, a Nearest-neighbor interpolation algorithm has been developed.
This algorithm uses a parametrized quantity of neighbors to interpolate the loss of the wanted solutions considering the loss of those selected neighbors [23, 24]. The nearest neighbors are determined by calculating the distance in the two-dimensional plane formed by the x and y-axes.
$${P}_{optimum}={{(Ɵ}^{t}·Ɵ)}^{-1}·{Ɵ}^{t}·C$$
5
Where C is vector that contains the loss of every selected neighbor. \(Ɵ\) is the matrix that contains the x coordinate in the first column, the y coordinate in the second column, and a third column of ones. For instance, in a 3-neighbor interpolation:
$$\left(\begin{array}{ccc}0.1702& 0.2154& 1\\ -0.1827& -0.2149& 1\\ 1.3765& -0.6122& 1\end{array}\right)$$
$$Loss=s·{P}_{optimum}$$
6
Where s is the solution whose loss is aimed to interpolate. Once again, in the case of a 2-variable interpolation (x and y), the structure of s must be filled with a third component in form of 1.
This process is repeated as many times as the number of solutions that must have their loss estimated. After ending the interpolation, the loss of all solutions is stored so that the optimization algorithm can work with it.
As stated before, using this interpolation algorithm, the loss from the generated solutions is estimated. To address this issue, the solutions from XVek are used as neighbors. Therefore, in this first interpolation, the wanted solutions are included among the neighbors. Consequently, the error made in the loss estimation is measured utilizing the following definition for each solution independently (see Eq. 7).
$$Error={({z}_{0}-{z}_{1})}^{2}$$
7
The calculated error is then used to sort the solutions from lowest to highest error. Afterwards, the worst solutions regarding their error are selected. The total amount of selected solutions equals N3, as these are the solutions that are known but not well predicted by interpolation the algorithm. To carry on, these solutions have then a random value added to each one of their variables (x, y). This random value is generated according to Eq. (2). The resulting solutions are once again checked through the function to prove their validation, and then stored in a variable named XVek3.
$$-\frac{Limit}{\varDelta }\le sum\le \frac{Limit}{\varDelta }$$
8
The next step consists of generating the unknown solutions, i.e. the N2 solutions. In order to do this, the method used to generate the initial solutions is mirrored. Therefore, random values contained in the interval [-Limit, Limit] are generated. The amount of solutions that are generated correspond to N2. These solutions are saved in a variable called XVek2.
Subsequently, all the solutions stored in XVek1, XVek2 and XVek3 are saved in the previously created variable XVek. Once again, the loss of every solution from XVek is determined via the optimization test functions from Table 1. Furthermore, the loss is also estimated with the interpolation algorithm, in order to foresee the results.
Therefore, the next Eq. (9) is used to compare the loss obtained via the optimization test function with the loss estimated by the interpolation algorithm. A variable named PredictionError is assigned to this comparison.
$$PredictionError =\sqrt{\frac{{\left|\right|LossFcn-LossInterp\left|\right|}^{2}}{N}}$$
9
Where LossFcn represents the loss obtained with the test function, and LossInterp is the loss estimated by the interpolation algorithm. N is the number of solutions.
This value is then compared with a limit established by PredictionErrorLimit, and as long as Eq. (10) is not fulfilled, the iteration process keeps going. If the equation is accomplished, the optimization process ends.
$$PredictionError <PredictionErrorLimit$$
10
2.2. Pseudocode
The following pseudocode explains the optimization algorithm so that it is easier to understand.
| Basque Optimization Algorithm: Optimization procedure | |||
|---|---|---|---|
| Input: Nmaxiter, N1, N2, N3, Nvar, Limit, Nneighbors, Δ, PredictionErrorLimit | |||
| Output: | |||
| Xhistory, Costhistory, | |||
| 1 | Ntotal = N1 + N2 + N3, Stop = 0 | ||
| 2 | Generate Ntotal solutions and store them in Xi | ||
| 3 | Evaluate Ntotal solutions and store them in Costi | ||
| 4 | Sort the solutions from lowest to highest loss | ||
| 5 | Store these sorted solutions in Xhistory, Costhistory | ||
| 6 | While Stop = 0 do | ||
| 7 | Generate N1 solutions around the best solution in Xhistory, Costhistory with a uniform distribution addition (U(± Limit/ Δ )). These all N1 solutions are checked in order to be in search space. | ||
| 8 | Evaluate and store these N1 solutions in X1, Cost1. | ||
| 9 | Predict the Cost function in whole solutions stored in Xhistory, Costhistory using a nearest neighbor interpolation algorithm with Nneighbors neighbors and select N3 solutions with the worse prediction euclidean error. | ||
| 10 | These N3 solutions are the worse predicted solutions so N3 solutions are generated around these solutions with uniform distribution addition (U(± Limit/Δ)) and stored in X3, Cost3. These all N3 solutions are checked in order to be in search space. | ||
| 11 | Generate N2 solutions in whole possible space, evaluate them and store in X2, Cost2. These all N2 solutions are checked in order to be in search space. | ||
| 12 | Predict the cost in X1, X2 and X3 solutions using a nearest neighbor interpolation algorithm with Nneighbors neighbors and calculte the cost prediction errors. Store the mean square error of cost prediction in MaxCostPredictionsError. | ||
| 13 | Store in Xhistory, Costhistory X1, Cost1, X2, Cost2 and X3, Cost3 | ||
| 14 | Sort Xhistory, Costhistory | ||
| 15 | if | MaxCostPredictionsError < PredictionErrorLimit or Nmaxiter reached | |
| 16 | Stop = 1 | ||
| 17 | end | ||
| 18 | end | ||
| 19 | End | ||
As has previously been stated, in order to examine the results with greater accuracy, a comparison with a particle swarm optimization algorithm has been performed. All the optimization test functions that have been selected from [23] and are distributed in Table 1 were employed to strengthen the truthfulness of the results.
Therefore, Table 2 shows the average mean squared error (commonly referred to as MSE) committed during both algorithm execution for every test function. Note that BO stands for Basque Optimization.
| Test function | Number of iterations | BO minimum loss | PSO minimum loss | BO mean MSE | PSO mean MSE |
|---|---|---|---|---|---|
| Ackley | 100 | 0.8387 | 0.0222 | 41.6762 | 2.3709e + 03 |
| Beale | 100 | 0.006 | 5.9890e-05 | 2.2200e + 08 | 1.1729e + 09 |
| Bohachevsky | 100 | 0.2118 | 0.0024 | 690.5768 | 1.2050e + 03 |
| Booth | 100 | 0.0874 | 1.5391e-05 | 4.6033e + 03 | 2.5949e + 04 |
| Branin | 100 | 0.4023 | 0.3979 | 4.1640e + 03 | 2.2188e + 05 |
| Bukin N.6 | 100 | 0.3537 | 1.3315 | 1.0969e + 03 | 2.1763e + 03 |
| Camel 3-Hump | 100 | 0.0381 | 2.3894e-05 | 4.7272e + 05 | 1.2420e + 08 |
| Camel 6-Hump | 100 | -0.8912 | -1.0316 | 9.2824e + 05 | 7.4803e + 05 |
| Cross-in-Tray | 100 | -2.0621 | -2.0626 | 3.1193 | 1.4063e + 04 |
| De Jong N.5 | 100 | 12.6705 | 12.6705 | 1.8380e + 03 | 4.6341e + 03 |
| Dixon-Price | 100 | 0.0841 | 1.7379e-05 | 2.4197e + 05 | 1.0763e + 08 |
| Drop-Wave | 100 | -0.9362 | -0.9984 | 2.0928 | 482.7915 |
| Eggholder | 100 | -62.2591 | -60.1347 | 237.2205 | 1.4073e + 03 |
| Goldstein-Price | 100 | 6.3003 | 3.0058 | 1.9475e + 11 | 2.1195e + 13 |
| Griewank | 100 | 0.0051 | 1.0975e-05 | 5.9273 | 989.4041 |
| Holder Table | 100 | -19.2001 | -18.0375 | 36.5787 | 6.3931e + 03 |
| Langermann | 100 | -4.1530 | -4.1355 | 11.4203 | 1.8012e + 04 |
| Levy N.13 | 100 | 0.1851 | 2.2998e-04 | 871.6266 | 1.7597e + 05 |
| Levy | 100 | 0.0041 | 9.0104e-06 | 174.9199 | 185.0842 |
| Matyas | 100 | 3.8385e-04 | 5.8242e-06 | 194.5511 | 2.9799e + 04 |
| McCormick | 100 | -9.6449 | -8.7052 | 809.8109 | 1.4922e + 04 |
| Michalewicz | 100 | -1.8805 | -1.7754 | 6.0271 | 1.0774e + 04 |
| Perm 0,d,β | 100 | 10.1585 | 5.6967e-04 | 9.7103e + 06 | 5.1126e + 08 |
| Perm d,β | 100 | 0.0890 | 5.4410e-04 | 1.2454e + 05 | 7.8286e + 06 |
| Rastrigin | 100 | 1.5553 | 1.1325 | 460.0511 | 4.3321e + 03 |
| Rosenbrock | 100 | 0.0132 | 9.0392e-04 | 3.0758e + 06 | 2.3209e + 08 |
| Rotated Hyper-Ellipsoid | 100 | 0.0447 | 3.5553e-05 | 690.0531 | 1.1263e + 03 |
| Schaffer N.2 | 100 | 2.2704e-04 | 1.7133e-07 | 5.8371 | 1.6348e + 06 |
| Schaffer N.4 | 100 | 0.2929 | 0.2927 | 4.0474 | 1.6824e + 03 |
| Schwefel | 100 | 830.0758 | 830.0752 | 43.1008 | 88.9351 |
| Shubert | 100 | -186.5017 | -185.6551 | 489.8506 | 1.4573e + 05 |
| Sphere | 100 | 0.0085 | 2.5386e-05 | 440.0505 | 778.9835 |
| Styblinski-Tang | 100 | -78.2053 | -78.3321 | 1.6615e + 04 | 2.5472e + 04 |
| Sum of Different Powers | 100 | 0.0095 | 2.0016e-06 | 3.0170e + 03 | 2.5958e + 04 |
| Sum Squares | 100 | 0.0357 | 6.9162e-05 | 692.3715 | 1.2262e + 03 |
| Trid | 100 | -1.9934 | -2.0000 | 574.1606 | 7.0040e + 04 |
| Zakharov | 100 | 0.0300 | 2.3336e-05 | 9.1370e + 04 | 5.3217e + 05 |
Having a look at the achieved results, it is easily appreciated that the proposed algorithm’s average mean squared error is far below the MSE of the PSO algorithm. In fact, except for the Camel 6-Hump function, the MSE of the BO is always lower.
The starting inspiration for this work was to design an optimization algorithm capable of ensuring that the function which the algorithm is working with is understood and sufficiently known by the algorithm. This is something that many of these days’ most used optimization algorithms do not achieve, as they usually stop iterating when a solutions is reached.
For this measure, the proposed algorithm does not only work with the best positions, but also with positions that are out of the known space, as well as with those positions whose prediction is not good enough. This process helps the algorithm understand the function better, as it does not just keep the best solutions for itself, which sometimes leads to bad results when trying to predict an unknown positions with other optimization algorithms.
Therefore, the results showed what was foreseen, as the proposed algorithm achieves much lower mean squared error than the particle swarm optimization algorithm which was used for the comparison. This once again proves that the Basque Optimization algorithm gets to know the function before stopping the iteration process.
Conflicts of Interest:
The authors declare no conflict of interest.
Funding:
The authors were supported by the government of the Basque Country through the research grant ELKARTEK KK-2023/00058 DEEPBASK (Creación de nuevos algoritmos de aprendizaje profundo aplicado a la industria).
Author Contribution
Conceptualization, A.Z. and E.Z.; methodology, A.Z., J.G.-O. and E.Z.; software, E.Z. and U.F.-G.; validation, J.M.L.-G. and U.F.-G.; formal analysis, A.Z. and E.Z.; investigation, J.G.-O. and D.A.I.-G.; resources, U.F.-G. and E.Z.; writing—original draft preparation, A.Z. and J.G.-O.; writ-ing—review and editing, D.A.I.-G. and J.M.L.-G. All authors have read and agreed to the pub-lished version of the manuscript.
Acknowledgments:
The authors are grateful for the support provided by SGIker of UPV/EHU..
Data Availability Statement:
Data is available upon reasonable request to the corresponding author.
- Karaboga, Dervis, and Bahriye Akay. ‘A Survey: Algorithms Simulating Bee Swarm Intelligence’. Artificial Intelligence Review 31, no. 1–4 (June 2009): 61–85. [CrossRef]
- Mavrovouniotis, Michalis, Changhe Li, and Shengxiang Yang. ‘A Survey of Swarm Intelligence for Dynamic Optimization: Algorithms and Applications’. Swarm and Evolutionary Computation 33 (April 2017): 1–17. [CrossRef]
- Yang, Xin-She, and Mehmet Karamanoglu. ‘Swarm Intelligence and Bio-Inspired Computation’. In Swarm Intelligence and Bio-Inspired Computation, 3–23. Elsevier, 2013. [CrossRef]
- Eberhart and Yuhui Shi. ‘Particle Swarm Optimization: Developments, Applications and Resources’. In Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546), 1:81–86. Seoul, South Korea: IEEE, 2001. [CrossRef]
- Zulueta, Asier, Decebal Aitor Ispas-Gil, Ekaitz Zulueta, Joseba Garcia-Ortega, and Unai Fernandez-Gamiz. ‘Battery Sizing Optimization in Power Smoothing Applications’. Energies 15, no. 3 (19 January 2022): 729. [CrossRef]
- Poli, Riccardo. ‘Analysis of the Publications on the Applications of Particle Swarm Optimisation’. Journal of Artificial Evolution and Applications 2008 (14 February 2008): 1–10. [CrossRef]
- Martínez-Filgueira, Pablo, Ekaitz Zulueta, Ander Sánchez-Chica, Unai Fernández-Gámiz, and Josu Soriano. ‘Multi-Objective Particle Swarm Based Optimization of an Air Jet Impingement System’. Energies 12, no. 9 (29 April 2019): 1627. [CrossRef]
- Uriarte, I., E. Zulueta, T. Guraya, M. Arsuaga, I. Garitaonandia, and A. Arriaga. ‘CHARACTERIZATION OF RECYCLED RUBBER USING PARTICLE SWARM OPTIMIZATION TECHNIQUES’. Rubber Chemistry and Technology 88, no. 3 (1 September 2015): 343–58. [CrossRef]
- M. Dorigo, V. Maniezzo, A. Colorni- ‘Ant system: optimization by a colony of cooperating agents’. IEEE Transactions on Systems Man and Cybernetics Parb B-Cybernetics (February 1996), 26 (1), 29–41. [CrossRef]
- Zitzler, E., and L. Thiele. ‘Multiobjective Evolutionary Algorithms: A Comparative Case Study and the Strength Pareto Approach’. IEEE Transactions on Evolutionary Computation 3, no. 4 (November 1999): 257–71. [CrossRef]
- Konak, Abdullah, David W. Coit, and Alice E. Smith. ‘Multi-Objective Optimization Using Genetic Algorithms: A Tutorial’. Reliability Engineering & System Safety 91, no. 9 (September 2006): 992–1007. [CrossRef]
- Sánchez-Chica, Ander, Ekaitz Zulueta, Daniel Teso-Fz-Betoño, Pablo Martínez-Filgueira, and Unai Fernandez-Gamiz. ‘ANN-Based Stop Criteria for a Genetic Algorithm Applied to Air Impingement Design’. Energies 13, no. 1 (19 December 2019): 16. [CrossRef]
- Storn, Rainer. ‘Differential Evolution – A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces’. Journal of Global Optimization, no. 11 (December 1997): 341–359. [CrossRef]
- Yang, Xin-She. ‘Differential Evolution’. In Nature-Inspired Optimization Algorithms, 89–97. Elsevier, 2014. [CrossRef]
- Centeno-Telleria, Manu, Ekaitz Zulueta, Unai Fernandez-Gamiz, Daniel Teso-Fz-Betoño, and Adrián Teso-Fz-Betoño. ‘Differential Evolution Optimal Parameters Tuning with Artificial Neural Network’. Mathematics 9, no. 4 (21 February 2021): 427. [CrossRef]
- Wang, Shu, Xinyu Da, Mudong Li, and Tong Han. ‘Adaptive Backtracking Search Optimization Algorithm with Pattern Search for Numerical Optimization’. Journal of Systems Engineering and Electronics 27, no. 2 (20 April 2016): 395–406. [CrossRef]
- Civicioglu, Pinar. ‘Backtracking Search Optimization Algorithm for Numerical Optimization Problems’. Applied Mathematics and Computation 219, no. 15 (April 2013): 8121–44. [CrossRef]
- Passos, Leandro Aparecido, Douglas Rodrigues, and Joao Paulo Papa. ‘Quaternion-Based Backtracking Search Optimization Algorithm’. In 2019 IEEE Congress on Evolutionary Computation (CEC), 3014–21. Wellington, New Zealand: IEEE, 2019. [CrossRef]
- Chang, Kuo-Hao. ‘Stochastic Nelder–Mead Simplex Method – A New Globally Convergent Direct Search Method for Simulation Optimization’. European Journal of Operational Research 220, no. 3 (August 2012): 684–94. [CrossRef]
- Lagarias, Jeffrey C., James A. Reeds, Margaret H. Wright, and Paul E. Wright. ‘Convergence Properties of the Nelder–Mead Simplex Method in Low Dimensions’. SIAM Journal on Optimization 9, no. 1 (January 1998): 112–47. [CrossRef]
- Muhlenbein, Heinz. ‘How Genetic Algorithms Really Work: Mutation and Hillclimbing’. Parallel problem solving from nature, no. 2 (September 1992): 15–25.
- Huda, A. S. Nazmul, and Soib Taib. ‘A Comparative Study of MLP Networks Using Backpropagation Algorithms in Electrical Equipment Thermography’. Arabian Journal for Science and Engineering 39, no. 5 (May 2014): 3873–85. [CrossRef]
- ‘Optimization Test Functions and Datasets’. Accessed 11 July 2022. http://www.sfu.ca/~ssurjano/optimization.html.
- Ni, K.S., and T.Q. Nguyen. ‘An Adaptable $k$-Nearest Neighbors Algorithm for MMSE Image Interpolation’. IEEE Transactions on Image Processing 18, no. 9 (September 2009): 1976–87. [CrossRef]
- Olivier, Rukundo, and Cao Hanqiang. ‘Nearest Neighbor Value Interpolation’. International Journal of Advanced Computer Science and Applications 3, no. 4 (2012). [CrossRef]
No competing interests reported.