Optimal Design of Adaptive Neuro-Fuzzy Inference System Using PSO And Ant Colony Optimization For Estimation of Uncertain Observed Values

This study employs a new method for regression model prediction in an uncertain environment and presents fuzzy parameter estimation of fuzzy regression models using triangular fuzzy numbers. These estimation methods are obtained by new learning algorithms in which linear programming is used. In this study, the new algorithm is a combination of a fuzzy rule-based system, on the basis of particle swarm optimization (PSO) and ant Colony Optimization AC 𝑂 𝑅 . In addition, a simulation and a practical example in the field of machining process are applied to indicate the performance of the proposed methods in dealing with problems where the observed variables have the nature of uncertainty and randomness. Finally, the results of the proposed algorithms are evaluated.


Introduction
For precise and reliable estimation, it is important to consider uncertainty in the construction of the manufacturing process models which are based on input/output data. As a result of uncertainty, imprecision and randomness associated with the measurement, materials, process parameters, and so on, in most experimental tests, the observed values of the repeated tests with the same inputs on the same samples are not an equal value. Besides, in some cases such as surface roughness resulted from machining, the obtained output values are inherently random. As a result, the construction of the model is based on the output variables consisting of an average group of the repeated observed values without any exact variation around the mean. Uncertainty was classified by Isukapalli and coauthors (1998) into the following types: Natural Uncertainty, Parameter Uncertainty and Data Uncertainty. Through input-output relations, these uncertainties are spread and combined. When uncertainty quantification techniques differ from each other, two types of general uncertainty can be derived: probabilistic uncertainty and possibilistic (non-probabilistic) uncertainty (Wang 2019). Probabilistic uncertainty usually describes the uncertain parameter when data information is sufficient. However, recent years have witnessed the emergence of the Possibilistic uncertainty approaches, specifically the interval theory and fuzzy set theory to address the issues (Wang 2020, Faes 2020). Fuzzy number approaches which are an extension to interval methods, assign a membership function to all parameter values which are related to a certain interval (Hanss 2005). Recently, the application of fuzzy logic has widely increased in uncertainty quantification for its successful performance. , Zhang 2020) . The linear model of fuzzy regression analysis, established by , *corresponding author has enabled the fuzzy system to give the fuzzy output. In the fuzzy linear programming problems, known as the FFLP problems, all the parameters, as well as the variables, are represented by fuzzy numbers (Kumar 2011). In the fuzzy regression model, a fuzzy functional relationship is given between explanatory variables and response variables. Several fuzzy regression techniques have been suggested based on fuzzy least squares (FLS) and mathematical programming methods, such as linear programming (LP) or quadratic programming (QP) that minimize the total spread of the output. The FLS and LP methods were suggested by Diamond (2011) and Tanaka et al. ( ,1988, respectively. Several variants of the FLS (Hong 2001) and LP methods (Razzaghnia 2011, Razzaghnia 2015, Fan 2013, Zhang 2018 have been applied for the fuzzy linear regression problem. Fuzzy inference system can express the uncertain situations in the form of rules. Therefore, it is applied in solving uncertain problems (Hemayatkar 2018, Hidalgo 2012. Neuro-fuzzy is a hybrid artificial intelligence technique with fuzzy logic and artificial neural networks. One of the most popular of hybrid algorithms is the ANFIS. Jang (1993) proposed an adaptive neural network based on fuzzy inference systems (ANFIS). In the ANFIS, learning ability and relational structure of the artificial neural networks are combined with the decision-making mechanism of the fuzzy logic. ANFIS has a flexible and strong structure. In order to optimize the consequent parameters in the hybrid algorithm of the adaptive neurofuzzy inference system, Danesh et al. (2016 suggested the new hybrid algorithms. Dong et al. present d an adaptive optimal fuzzy logic based energy management solution to develop appropriate day-ahead fuzzy rules for real-time energy dispatch adaptively in the presence of operational uncertainties. In order to reduce the error of the fuzzy regression model, the current study proposed new algorithms that use the fuzzy inference system and meta-heuristic algorithms such as ant colony and particle swarm algorithms to optimize the premise parameters. In addition, linear programming was used for the consequence parameters prediction. Moreover, these algorithms were compared with the method proposed by Danesh (2018) which was based on adaptive fuzzy inference system and linear programming (FWLP). The study showed that the proposed methods had fewer errors than the LP and FWLP methods, and were further verified by the predictions using simulation and practical examples. This paper includes five sections. Section 1 is the introduction and the Section 2 explains concepts and formulations of the different models. The methodology of the proposed methods is explained in Section 3. To illustrate the procedure more accurately in predicting uncertain outputs of the proposed methods, two examples are given in Section 4. The first example is a simulated example and the second example describes the application of the proposed methods for predicting surface roughness in machining GFRP composites by considering the uncertainty, irregularity and imprecision associated with surface roughness measurement. In section 5, the analysis of the obtained results is discussed.

Basic concepts and methods
In this section, we briefly review basic concepts of fuzzy regression, linear programming (LP) in fuzzy regression, adaptive neuro-fuzzy inference system, particle swarm algorithm, and ant colony algorithm respectively.

Fuzzy regression
The function (x) is a mapping from x to where x = ( 0 , 1 , … , ) ( = 1, … , ) is adimentional vector crisp independent variable and domain is assumed to be ⊆ . Consider the following the fuzzy regression model: represents the regression error with conditional mean zero and variance 2 (x) given x. In this paper, response variable has the symmetric triangular fuzzy structure that can be written as = ( , ) where and are the center and the spread of a symmetric triangular fuzzy number respectively and = − = − .

Linear Programming (LP) in fuzzy regression
In this study, we consider the following fuzzy regression model as: where n is the number of data points, x = (1, 1 , 2 , … , , ) is a p-dimensional input vector of the independent variables at the ℎ observation, also, = ( 0 , 1 , … , ) is a vector of unknown fuzzy parameters and is the ℎ observed value of the dependent variables. can be denoted in vector form as = { , } where = ( 0 , 1 , … , ), = ( 0 , 1 , … , ), is the center value and α is the spread value of , = 0, … , . Also, = ( , ) is symmetric triangular fuzzy number where and are the center and the spread this number, respectively. Also, the fuzzy regression parameters can be calculated by solving linear programming (LP) model as follows (Hidalgo 2012): So that, the following two constraints must be established: and ≥ 0, = 0, … , , = 1, … , , 0 < ℎ ≤ 1 .
In this model, the constrains assure that the support of the predicted values from the regression model includes the support of the observed values.

Particle Swarm Optimization (PSO)
The particle swarm optimization is a meta-heuristic algorithm that was introduced by Eberhart et al. (1995). It is based on the behavior of animals in a flock by mimicking the collective learning of the group, such as birds. Each bird is represented as a particle. To reach the optimal solution, they share the information among themselves. The number of particles and their positions inside the parameter's space is randomly defined to initialize the method. The PSO method has both global and local searching ability with fast convergence and parameter adjustment. It solves the problem of the sensitivity of the iterative method to the initial value and prevents the clustering algorithm from getting into local optimum. For the optimization problem, each particle in the PSO algorithm represents a possible solution. The best solution found by the particle is the best position experienced by the whole group. x , and x , are individual extremum and global extremum respectively. To create a new population, every particle constantly updates itself through , , and . Also, the whole population comprehensively searches the solution region. The position of the ℎ particle ( = 1, 2, . . . . , ) can be expressed as x and speed as v .
The positions and velocities of all the particles are iteratively updated. In order to refine the search around the global optima, the "adaptation" concept has been proposed by Eberhart (1995). Furthermore, velocity limit [ , ] and positions limit [ , ] were suggested to assure that the particle does not move beyond the search space boundaries. The new position and velocity at ( + 1) can be calculated by the following equations: F [ ] is objective function. So, w is the inertial weight whose value decreases rapidly in the first iteration. However, its weight decreases slowly after a number of iterations. 1 and 2 are constants and called as acceleration factor. Also, 1 and 2 are random numbers that change in the interval of (0, 1). Then the individual extremum of each particle and the global extremum of the whole particles can be updated by the equations as follows: , The general procedure for this scheme is summarized as follows: Step 1: t = 0 is inputted.
Step 2: The initial values of positions and velocities for all particles are randomly determined.
Step4: The objective function is computed and evaluated at each particle position.
Step 9: If t> maximum, the training of network terminates, otherwise go to 4.

Ant Colony Optimization (ACO)
Ant colony optimization is a meta-heuristic technique that uses artificial ants to find solutions to combinatorial optimization problems. It uses a discrete structure to determine the solutions. In a discrete structure, each of the decision variables is divided into a specified number of cases in a defined range. With discrete variable space, a limitation is produced for the algorithm which reduces the accuracy of optimality. If you divide the space between the large numbers of decision variables, the accuracy of the solutions will be increased accordingly. Also, such a complex system may reduce accuracy. To fix this problem, generalized ACO to space R was proposed (AC ). Thus, the fundamental idea underlying AC is the shift from using a discrete probability distribution to using a continuous one. In R algorithm, the shift to a continuous space is carried out using a probability density function. Soosha and Dorigo [30] proposed Gaussian function in ant colony optimization for a continuous domain. In order to create multi-maximums, the kernel of Gaussian function (the sum of the singular Gaussian functions) is defined as: parameters ، and should be determined during the execution of the algorithm and is the number of singular Gaussian functions. Decision variables related to ℎ solution are denoted by 1 , 2 , … , n . For solution and ℎ variable, is considered equal to j and is assigned as follows: is the average distance between the ℎ variable of the solution and the ℎ variable of the other solutions in the archive and is the size of the archive. is an adjustment parameter. Large value reduces convergence speed. The value affects the long-term memory causing the worse solutions to be deleted, and the new solutions are considered closer to known good solutions (Dastranj 2011).
is a weight associated with solution j, where for every , a weight is calculated as follows: that 1 ≥ 2 ≥ ⋯ ≥ .
The poor decisions will be minimized if a very small value is considered for . Yet with extreme reduction, the assigned value will be trapped in the local optimum. Also, selection probability of solution j can be expressed as follows: The general procedure of is summarized as follows: Step 1: Based on the answer solution for and for the ℎ variable, a normal random number is generated to create a new solution.
Step 2: It is repeated for variable and then for each of the ant.
Step 3: All of the solutions are sorted according to their quality, where saved k better solutions are retained and the rest of the solutions are discarded.
Step 4: Algorithm stop conditions are as follows: 1. The lack of progress. The new solutions to be the same with the previous solutions.
2. The repetitions number. When the number of repetitions reached to the predetermined repetitions number.
3. Reach to a maximum amount. The algorithm continue to converge to a certain amount. Otherwise go to step 1.

Adaptive Neuro-Fuzzy Inference System (ANFIS)
ANFIS is a famous hybrid technique that combines the adaptive learning capability of ANN along with the intuitive fuzzy logic of human reasoning (Jang 1993). The network structure of ANFIS consists of two parts. Utilizing an optimization algorithm, the premise and consequence parameters are determined. Also during training, the existing input-output data pairs are used (Jang 1993). The structure of ANFIS consists of five layers. To present ANFIS model architecture, we consider four fuzzy if-then rules with two-dimensional input and one output.
Where, x 1 , x 2 and y ∈ R are input and output variables, respectively. A k 's are fuzzy sets and f k represents system output due to rule R k (k = 1,2,3,4). In the following, the five layers of the system are explained.
In the first layer, all the nodes are adaptive nodes that generate membership grades of the inputs. The node functions are given by: o l,k is the output of the k th node of the layer l. In this paper, Gaussian membership function is considered. τ k and σ k parameters represent the center and the spread, respectively.
In the second layer, the nodes are also fixed. This layer is called as rule layer. In this layer, the outputs can be calculated as follows: o 2,k = w jk = µ A k (x j1 ). µ A k (x j2 ), k = 1,2,3,4, j = 1, … , n.
In the third layer, the nodes are fixed nodes. This layer is called normalized firing strength. The outputs of this layer can be calculated as: , k = 1,2,3,4, j = 1, … , n.
In the fourth layer, which is called defuzzification layer, the node is an adaptive one. The association node function in this layer is a linear one and the outputs can be represented as below: o 4,k = w ̅ jk f k = w ̅ jk p i k , k = 1,2,3,4, i = 0,1,2.
In this study, p i k will be assumed to be a symmetric triangular fuzzy number.
In the fifth layer, the single node carries out the sum of inputs of all the layers. It is called as the summation layer. The overall output of the structure is obtained as bellow:
In the proposed methods, the premise and consequence parameters are obtained by method ANFIS for only the first part of the Eq. (18). Then, the obtained premise and consequence parameters are put in a matrix P=X 0 . ∈ [10 − , 10 ]. In the following, the obtained parameters are optimized by meta-heuristic algorithms such as PSO and . In the finally, the consequence parameters are computed for center and spread value as follows: Step 1: The optimization premise parameters are put in Eq. (18).
In this paper, we show the proposed methods methodology in diagram (1) and use the MATLAB software tool for codding.
To evaluate the performance of the various methods, the error rate can be calculated using Eq.

Simulation and Numerical examples
In order to illustrate the applicability of the proposed algorithms, the following simulation and practical examples are considered. Also, the obtained results of the different methods are compared.
Suppose the domain of X = ( 1 , 2 ) is = [0,10] 2 . A set of data is generated the same way as in (Dastranj 2011) that summarized in the following manner.
The independent variables 1 and 2 are crisp inputs. They are randomly taken from 0 to 10. Let output = ( , ) ( j = 1,2, … , ) is a symmetric fuzzy number and so that Determine the initial values of the premise and the consequence parameters by ANFIS method for only centers.

3.
Construct candidate solutions in a probabilistic way using a probability distribution over the search space. 4.
Use the candidate solutions for modification the probability distribution in a way.

5.
Repeat for variable until a solution is made for all variables and N ants. 6.
Sort all of solution archive, save k better answer and clear the rest of the answers. 7.
Get premise parameters.

8.
Put in Eq. (18) and estimate consequence parameters for centers and spread by linear programming and Eq. (20). 9.
Determine the initial values of positions and velocities for all particles. 10. Update positions according to Eq. (7). 11. Compute and evaluate the objective function at each particle position. 12. Compute x l,tseb [t] and x g,tseb [t] for j = 1, … , n. 13. Update velocity according to Eq. (6). 14. The lack of progress, reach to the predetermined repetitions number or converge to a certain amount. 15. Calculate ERROR. 16. Evaluate the chosen net by using the test dataset. 17. Report errors of the different methods and results of net in test and train dataset. In the following, the obtained parameters of using the FWLP method and the proposed methods are respectively displayed in Tables 2-5. Also, the obtained regression model in LP method is as follows: ̂= (̂,̂) = (0.0744,2.7036) + (1.4498,0.2699) 1 + (0.4007,0.1173) 2 .
In Tables 5 and 6, the obtained results of the different methods are summarized. It can be observed that the error values of the proposed methods are lower than the error values the other ones. Thus, the proposed algorithms provide the better prediction than the LP and FWLP methods.

Example 2: Application of the Proposed Method to Predict Surface Roughness in Machining GFRP Composites
Surface roughness is an important factor in manufacturing engineering because a rough surface has negative effects on the fatigue strength, creep life, wear and corrosion resistance of various components. In turning operation, high quality turned surface eliminates the need for secondary operations. Under ideal cutting conditions, the texture of machined surface comprises regular traces of the tool nose (Fig. 2). The surface geometry of a workpiece depends upon a number of factors including machining variables, tool geometry, materials of the workpiece and tool, and the vibrations of the machine tool system (Boothroyd 2011. The deviations may be repetitive or random and may result from roughness, waviness, lay and flaws (Lu 2007). There are many different roughness parameters in use, but the arithmetical mean roughness (Ra) value is the most widely used parameter for the surface roughness assessment. Ra can be calculated using the following formula: Where A is the area between the roughness profile and its central line and L is the sampling length. Fig. 2  In the following, the obtained premise parameters the FWLP method and the proposed methods have been shown in Table 7. For fitting regression model, the different methods are applied and the obtained outputs are summarized in In order to evaluate the different methods, value ERROR in equation 21 is calculated and reported in Table 9. Like the previous examples based on the error values, we can be observed that the proposed algorithms have lower error than the LP and FWLP methods in the prediction.

Conclusion
The aim of this study is to develop ANFIS method by PSO and algorithms, and the linear programming. In this paper, we considered the fuzzy regression model with crisp inputs and triangular fuzzy output. So, the proposed algorithms are fuzzified to fit the fuzzy regression function with the linear programming. The effectiveness of various methods is demonstrated by different examples. By considering the obtained results of different methods and figures, it can be determined that the performance of the proposed methods is evidently better than the ANFIS method and linear programming.
Based on Examples, it should be pointed out that the performance of the prediction can be significantly enhanced by using the proposed algorithms in case two-dimensional input. Some conclusions are summarized as follows: 1) A comparison with ANFIS and linear programming methods verifies high computational precision and efficiency of the proposed methods. The results demonstrate that proposed methods are efficient methods for the in many practical engineering problems with uncertainty.
2) By using the obtained results and tables, it can be determined that the proposed methods are feasible.
Based on examples, it should be pointed out that the proposed methods have more accurate than the LP and FWLP methods. Also, they decrease the error values to a minimum level. 3) In the proposed methods, the used constrains assure that the support of the estimated values from the regression model includes the support of the observed values in h-level(0 < h ≤ 1). In linear programming, the width of the estimated value depends on the observation number. As the number increases, there will be a rise in the width of the estimated value which can, in turn, be weakened by the proposed methods. 4) In addition, the proposed methods have not more complicated than the FWLP method in computations but they have more accurate than it.
5) The obtained results from the simulated example and the case study in the field of prediction of surface roughness in machining GFRP composites show that the presented methods are especially useful for practical problems which involve output parameters that include some degree of uncertainty, inhomogeneity, randomness and imprecision in the output data. 6) The proposed approach can enrich the methods that offer a promising way for further research on the design and optimization of fuzzy neural networks.