Optimal robust control with cooperative game theory for lower limb exoskeleton robot

For achieving trajectory tracking issue of the lower limb exoskeleton robot, a novel optimal robust control with cooperative game theory is proposed. The uncertainties are considered (possible time-varying, bounded and fast), and the fuzzy set theory is creatively adopted to describe the boundary. From the view of analytical mechanics, the trajectory tracking is treated as the constraints control problem, including holonomic and nonholonomic constraints, which need to satisfy the conditions of human motion. Combining the robust control and optimal design, optimal robust control is formulated to satisfy both performances guaranteeing and optimal. The Pareto optimal solution is obtained to guarantee the minimum control cost. In the simulation, the adaptive robust control is chosen as a comparison. The existence of Pareto optimality and the effectiveness of optimal robust control have been verified via simulation results.


Introduction
The aging problem can no longer be ignored, which has brought a series of social issues. The strokes have struck numerous elder people, which is a predominant source of lower limb paralysis [1]. Medical theory and clinical medicine have proved that rehabilitation training process significantly effects on relieving the symptoms [2]. However, traditional rehabilitation training wastes lots of manpower and resources, and the training effect is poorer than that of lower limb exoskeleton robot (LLER) [3]. Depending on the degree of paralysis, the motion modes of LLER are primarily classified as active or passive mode [4]. In the passive mode, the LLER needs to follow a certain desired trajectory, which is usually derived from clinical gait database. Passive training is essential for further rehabilitation, and the trajectory tracking issue plays an indispensable role, which is involved in this article.
In recent years, lots of excellent efforts have been carried out, such as robust control [5], hierarchical control [6], adaptive control [7], fault-tolerant control [8], neural network control [9] and so on. Campbell et al. [10] develop the assistance-as-needed control to help the patient with rehabilitation which means the robot will provide auxiliary torque only when the patient deviates from the desired gait. In order to overcome the unstable phenomenon, Sun et al. [11] construct a reduced adaptive fuzzy control with compensation and verify its effectiveness on LLER system. Yang et al. [12] describe a repetitive learning control of LLER in the frame of backstepping control and demonstrate its convergence using Lyapunov method. An active disturbance rejection control is proposed to compensate for the disturbance and converge quickly with the help of fast terminal sliding mode control [13]. However, these control methods mentioned above are not available for nonholonomic constraints, which are common in reality. The optimal robust control (ORC) we propose is applicable not only to holonomic, but also to nonholonomic constraints. The reason is that we design the controller based on Udwadia-Kalaba (U-K) theory.
Professors Udwadia and Kalaba proposed a novel equation of motion from the view of analytical mechanics in 1992, called U-K theory [14,15]. U-K theory applies to both holonomic and nonholonomic constraints. In addition, Lagrange multipliers and quasi-variables are not required in U-K theory, which can lead to less demanding computations [16]. These are the reasons why we choose U-K theory. Liu et al. [17] obtain the dynamic equation of constrained systems through U-K theory and formulate a systematic control method, which applies to not only fully actuated but also under-actuated and redundantly actuated systems. Sun et al. [18] apply the U-K theory to solve tracking control problem of mobile robots. Sun et al. [19] design an adaptive robust control using U-K theory to carry out the dual avoidance-arrival problem. Cho et al. [20] develop new continuous sliding mode controllers for multi-input multi-output systems and conduct two numerical examples to demonstrate the accuracy and robustness of them. Mylapilli and Udwadia extend the U-K theory to the control of three-dimensional incompressible hyperelastic beams [21]. Furthermore, U-K theory can also be applied to satellite formation flight control to develop a nonlinear controller without making any linearization [22]. However, the uncertainties are not considered in U-K theory, which is a limitation on the way of development.
In fact, the uncertainties cannot be negligible in trajectory tracking control. The description and management of uncertainty are similarly important, and many control methods have done related work, such as torque control [23], self-adjusting leakagetype adaptive control [24], robust bounded control with inequality constraints [25] and terminal sliding mode control [26]. Unfortunately, the bounds on the uncertainties involved in the control methods presented above are all deterministic and constant, which is not reasonable in reality. So, a better method to describe and manage the uncertainties is necessary.
Probability theory has captured the attention of many scholars on describing the uncertainties by adopting lots of observed data, and numerous achievements have been obtained [27]. Stochastic dynamic system method combined the probability theory and dynamic model [28] and has a well performance in tracking, which also needs lots of observed data. However, not all data are easily available for observation or cannot be accurately repeatable in large amount, such as flood data. In 1965, Zadeh [29] firstly proposed the fuzzy set theory, which has the ability to describe the degree to which the event occurs rather than frequency. Zhen et al. [30][31][32] employ fuzzy set theory to describe the uncertainties in the dynamic model and call it fuzzy dynamic system (FDS), which is not based on IF-THEN rules. In order to avoid the shortcomings of lots of probability theory, the uncertainties in LLER are described by fuzzy set theory. Then, based on U-K theory and FDS, a novel robust control with two adjustable parameters is proposed. Different parameters can cause different system's control cost. How to select an optimal pair of two adjustable parameters is a question worth considering.
Remarkable studies have been conducted about optimal control of multi-parameters in recent years, such as ant colony [33], particle swarm optimization [34] and genetic algorithm [35]. Unfortunately, the realization of FDS is difficult and inconvenient. [36][37][38] have researched the optimization of FDS. But their research object is single-objective optimization. Referring to previous work of Chen et al. [39,40], we employ the cooperative game theory to solve the problem of optimal parameter selection, which was introduced by Pareto. The cooperative game theory involves two players, whose goal is to find the Pareto optimal solution, in order to minimize both of their cost. More important, for a better interpretation about our work, it should be noticed that the FDS is quite different with the fuzzy logic control, since the latter is based on IF-THEN rules.
In conclusion, our main contributions are listed as follows: 1. The dynamic model of LLER is formulated, and the fuzzy set theory is employed to describe the unknown boundary of uncertainties in LLER, which is within a threshold. The trajectory tracking problem is treated as constraints control problem, where the constraints can be holonomic and nonholonomic. 2. Based on the information of the uncertainties bounds and U-K theory, a robust control method is designed with adaptive law and two adjustable parameters. The control method is deterministic that the LLER system is guaranteed to meet the uniform boundedness and uniform ultimate boundedness regardless of the uncertainties. 3. An optimal design associated with cooperative game theory is formulated. The fuzzy performance indicators for k 1 and k 2 are constructed. Moreover, the Pareto optimal solution obtained can select the optimal solutions of k 1 and k 2 and minimize the system's control cost.
The rest of this article is arranged as follows: Section 2 introduces the preliminary knowledge about fuzzy set theory, U-K theory and cooperative game theory. Robust control with two adjustable parameters is designed in Sect. 3. Optimal control with cooperative game theory is constructed in Sect. 4. Section 5 conducts the simulation experiments to verify the existence of Pareto optimality and the effectiveness of optimal robust control. Section 6 is the summary of the full paper.

Fuzzy set theory
Some basic knowledge about fuzzy set theory needs to be reviewed for further development (see [41,42]).
Fuzzy number [41]: Let G be a fuzzy set in R, the set of real numbers. G is named a fuzzy number if: (1) G is normal and convex, (2) the support of G is bounded and (3) all a-cuts are closed intervals in R. Moreover, assume the universe of discourse of a fuzzy number to be its 0-cut.
Fuzzy arithmetic [41]: Let G and H be two fuzzy numbers and be their a-cuts, a 2 0; 1 ½ . Moreover, the four fundamental rules of G and H are listed as follows: Decomposition theorem [41]: Define a fuzzy setH a in U whose membership function is lH a ¼ aIH Then, the fuzzy set H is obtained as where [ is the union of the fuzzy sets (that is, sup over a 2 0; 1 ½ ). On the ground of decomposition theorem, we can formulate the membership function of the resulting fuzzy number with the operation between the two fuzzy numbers. D-Operation [42]: Assume a fuzzy set For any functionf : To some extent, the D-operation D f ð Þ ½ takes an average value of f ð Þ over l N ð Þ. In the special case that f ð Þ ¼ ; this is simplified to the commonly known center-of-gravity defuzzification method [38].

Udwadia-Kalaba theory
Establish the dynamic model of a mechanical system using Newtonian or Lagrangian method, which can be formulated as [43] where t 2 R is the time, x t ð Þ, _ x t ð Þ and € x t ð Þ 2 R n (n denotes dimension) stand for generalized coordinates, velocity and acceleration, separately. d t ð Þ 2 R denotes the uncertain parameters, which may be fast and timevarying. R represents the boundary of the uncertainties. M 2 R nÂn [ 0 denotes the inertial mass matrix; Q 2 R n denotes the known force imposed on the control system. s 2 R m m n ð Þstands for the control input, and Z represents the corresponding coefficient and its value depends on whether the mechanical system is a fully actuated, over-actuated or underactuated system. When the system is not constrained, s ¼ 0. Furthermore, M Á ð Þ and Q Á ð Þ are of appropriate dimensions, continuous and Lebesgue measurable.
The coordinates x t ð Þ can be chosen according to the specific situation, which do not necessarily to be generalized coordinates. Consider the mechanical system is subjected to the following constraints: where B li ð Þ and d l ð Þ are both C 1 and 1 m n. i represents the ith element of each row of B li ð Þ and (10) can denote the system constraints. Determining the first-order derivative of (10) with respect to time t, we can obtain where the role of k is to iterate over x n x 1 ; ð Á Á Á ; x n Þ: Rewrite the constraints (11) as a matrix form: which contains the first derivative of x, where A ¼ A li ½ mÂn and c ¼ c 1 ; c 2 ; Á Á Á ; c m ½ T . The second derivative of (10) with respect to time is obtained: (15) can be rewritten as the matrix form: The advantage of using the second-order constraint lies in the fact that it is linear in the acceleration. The details of (10)- (18) can be found in [44,45]. Almost all control problems can be written in the second-order constrained form, including stability, trajectory tracking and optimal control [46].

Assumption 1 The constraint in (18) is continuous and full rank, then
Remark 1 For a given constraint, A and b are determined. There is at least one solution for € x in (18), then (18) can be seen as continuous.
Theorem 1 (U-K theory): Assumption 1 is satisfied, an unconstrained mechanical system is subjected to constraints as shown in (11), then the constraint force Q c is shown as [47] As mentioned above, U-K theory is applicable to both holonomic and nonholonomic constraints. For a better explanation, the flowchart of U-K theory is shown in Fig. 1.
The mechanical system considered in U-K theory is free of uncertainties or the uncertainties are known, in this case s ¼ Q c . Nominal control, also called Udwadia control, can be designed based on U-K theory. However, the uncertainties are inevitable and undetermined in practical applications.

Cooperative game theory
This subsection reviews some fundamental theories of cooperative game theory as follows.
We shall consider a game with a number of players. Each player has a cost function that conforms to the rules of the game, and the cost function is affected by the decisions of all players, which is determined by the rules. Assume Y players are involved in the game, and the rules impose the following mappings: where J i Á ð Þ and b H i are, respectively, the cost function and decision set for player i.
Generally, for each player, the least cost in the game is his goal. Consequently, the best decision to all players is h

Determine the dynamic system
Decompose M and Q to nominal and uncertain parts according to (25) and (26) Apply the desired trajectories to the system and transform to the form of (13) Formulate w1, w2 and w3 based on (37) which indicates that the Y-tuple h À Ã is an ideal decision and ensures that everyone's cost is minimized. Unfortunately, the satisfactory decision is not available in the reality and how to reach the optimal decision is a thorny problem. According to economist Pareto, if the players collaborate with each other, that is if the behavior of each player is interdependent and each player communicates with each other to obtain his minimum cost, the game is called a cooperative game. Moreover, Pareto formulates the optimal decision as Pareto optimal solution: A decision is Pareto optimal solution if and only if taking other decisions increases the cost of at least one player or consumes no one at all. To be more specific, the definition of Pareto optimal solution is shown as follows.
or there is at least one i 2 1; 2; Á Á Á ; Y f gsuch that Generally speaking, the Pareto optimal decision may be more than one, and the set of cost outcomes Þunder different Pareto optimal decisions is called the Pareto frontier.
Based on Definition 1, the following lemma is proposed to find the Pareto optimal solution.
Remark 2 From the perspective of cooperative game theory, we can transform the parameters and evaluation indicators to the decisions and cost functions. The optimal solution can be obtained by solving the Pareto optimal solution. That is why we choose the cooperative game theory to solve the multi-parameter optimal design problem.

Robust control design
We propose an adaptive robust control to ensure the LLER can track the desired trajectory in this section. The control design procedure is shown in Fig. 2.
Decompose the matrices/vectors M and Q of Eq. (9) into two parts which are the ''nominal'' parts and the uncertain parts as follows: Assume that M is positive definite. M Á ð Þ, DM Á ð Þ; Q Á ð Þ and DQ Á ð Þ are all continuous and uninterrupted. In the ideal case, there is no uncertainty and the corresponding matrices are equal to the corresponding ''nominal'' parts.
Definition 2 For the uncertain parts DM and DQ, there exist fuzzy sets G DM and G DQ in universe of discourse U DM 2 R and U DQ 2 R characterized by membership functions l U DM :

Remark 3
The dynamic Eq. (9) with fuzzy set theory to describe the uncertainties like Definition 2 is considered as the FDS. Let Then, we can obtain Assumption 2 When Assumption 1 is satisfied, for a given P 2 R mÂm ; P is positive definite. Let There exists a constant q E [ À 1. For all x; t ð Þ 2 R n Â R, q E satisfies the (33): where k m denotes eigenvalues of matrix.

Remark 4
The value of q E , W and E depends on W, E and the uncertainty bound R, respectively. Since the bound of uncertainty R & R p is unknown, q E is unknown. In the ideal situation, E ¼ 0; W ¼ 0 and q E ¼ 0: (2) For each a; x; _ Þis a linearized function with respect to a; there exists a function Here, R; which is unknown, determines the value of a.
The adaptive law can be designed to govern the parameter b a as follows: Assume that Assumptions 1-3 are satisfied; under the system of (9), the adaptive robust control (36) renders the uniform boundedness and uniform ultimate boundedness: ( The proof of Theorem 2 is given in Appendix A.

Optimal design
According to Rayleigh's principle where k min P ð Þ [ 0, because P is positive definite. So If we letq :¼ min 2=k max P ð Þ ½ ; L 3 f g , we can obtain where V is the Lyapunov function and the same as that in (100).
First, consider the differential equation The right-hand side of (50) meets the global Lipschitz condition with Solve the differential (50) and obtain the result Then, based on the solution process of differential inequality, we can obtain for all t ! t 0 . Similarly, for any t s and C ! t s where Þ . t s is not necessary to be equal to zero; it is the time when control system starts.
To separate the variable, V s is rewritten as (46) and (56), we have Þkb a À ak 2 and the right-hand side of (55) provides an upper bound of k min P ð This can be an upper bound of kbk 2 . For each C ! t s , let Remark 5 The uncertainty is described by fuzzy set theory, which can perfectly be solved adopting the corresponding membership function. Compared with probability theory, fuzzy set theory does not need numerous observation data. [41] has a more detailed description of fuzzy set theory.
The performance indicators for k 1 and k 2 are formulated as Considering the D-operation, we have Details for L i ; i ¼ 1; 2; Á Á Á ; 10 f g are given in Appendix B.
Similarly, by D-operation, we can obtain Details for , i ; i ¼ 1; 2; Á Á Á ; 6 f g are given in Appendix B.
Then, J 1 and J 2 can be rewritten as The optimal design is obtained by the following cooperative game.
For any t s and given H 1;2 min : To obtain the solution, we apply a partial differential operator to J k 1 ; k 2 ð Þwith respect to k 1 Similarly, applying partial differential operator to J k 1 ; k 2 ð Þwith respect to k 2 gives Let By analyzing the second-order derivative of J, we have Suppose the candidate solutions k Ã 1 ; k Ã 2 can meet the sufficient condition Then, (68) can be minimized by the solutions of (72) and (73) and the solutions are Pareto optimal.
Remark 6 The k Ã 1 and k Ã 2 can be determined with suitable choices of P, q E and L i i ¼ 1; 2; 3 ð Þ . Based on the above analysis, the resulting minimal performance index is acquired by The design procedure of optimal design is displayed in Fig. 3. Figure 4 demonstrates the 3D prototype and simplified 2D model of LLER. We assume the LLER has two active degrees of freedom (DOFs) of one leg, containing the flexion and extension DOFs of hip and knee joints. This idea is inspired by [9,11,50].

Description of LLER
x ¼ h 1 h 2 ½ T is chosen as the generalized coordinates, which denotes the rotation angle of hip and knee, respectively. By employing the Lagrangian dynamics, the dynamic model of LLER is formulated where s wr is the interaction torque between the wearer and robot [23,49]. x d and _ x d are the desired trajectory and velocity, respectively. k p and k d are constants that amplify the differences of trajectories and velocities between the wearer and robot. Moreover, m 1 stands for the sum of the masses of the thigh of wearer and robot, and m 2 stands for that of the shank, similarly. g is the gravitational acceleration. l i ¼ 2d i i ¼ 1; 2 ð Þ denotes the length of thigh and shank. I i i ¼ 1; 2 ð Þrepresents the moment of inertia of thigh and shank, respectively.

Remark 7
Considering that our main purpose is to realize the trajectory tracking control of LLER and verify the effectiveness of ORC, only one human motion actuation scenario is considered in this paper. The desired constraints (87)-(88) are obtained from Clinical Gait Analysis data and holonomic. The proposed control method is also applicable to other actuation scenarios and nonholonomic constraints. People can calculate A and b according to Fig. 1 and then design the controller. As shown in Assumption 1, the scenarios and constraints should be continuous.
Due to the imperfect knowledge of the LLER system parameters and/or external disturbance, uncertainties are inevitable in the system model. As we know, different wearers and even the same wearer may have different weights, so the masses m i i ¼ 1; 2 ð Þare chosen as the uncertain parameters:

Simulation results
The software is MATLAB R2019b. Figures 6, 7, 8  To better illustrate the effect of l i (i ¼ 1; 2) in Table 2, Figs. 12, 13, 14 and 15 compare the trajectories of s 1 , s 2 , error1 and error2 under five groups of k i (i ¼ 1; 2) to explore its impact for system performance. A significant event is discovered by us: The smaller the peak value of s i (i ¼ 1; 2), the slower the error1 and error2's speed converges to zero. We need to endure the larger torque cost to achieve the faster error convergence, which means more expensive motors. In conclusion, the value of l i (i ¼ 1; 2) should be chosen according to the specific situation. The curves of error1 and error2 also prove that the ORC method has the uniform boundedness and uniform ultimate boundedness.
Adaptive robust control (ARC) of [18] is chosen as a comparative control method, and the corresponding simulation results are also shown in Figs. 12, 13, 14 and 15. The trajectories tracking errors are larger than that of ORC, which means the h 0 i s i ¼ 1; 2 ð Þtracking effect of [18] is worse. And the absolute torque value of hip is larger than that of ORC, although that of knee ARC of [18] Fig. 14 Trajectories of error1 with respect to time is smaller than that of ORC. Considering the overall system, the performance of ORC is better than that of ARC of Sun. Figures 16 and 17 illustrate the absolute cumulative error (ACE) of h i (i ¼ 1; 2) under five groups of k i (i ¼ 1; 2) and ARC of Sun. ACE of h i (i ¼ 1; 2) with ARC of [18] is significantly larger than that with ORC under five groups of k i (i ¼ 1; 2). In summary, the tracking performance of ORC proposed by us is better than that of ARC of [18], and people can choose the appropriate pair of l i (i ¼ 1; 2) to meet the practical requirements of LLER.

Conclusion
In this paper, the optimal robust control with cooperative game theory is designed to solve the trajectory tracking problem of LLER, including the uncertainties. The uncertainties are time-varying and bounded. For a better description and control of uncertainties, fuzzy set theory is employed, which is essentially different from the traditional fuzzy logic theory. Afterward, the robust control with two adjustable parameters is proposed in the frame of U-K theory, which guarantees the uniform boundedness and uniform ultimate boundedness. Furthermore, the optimal design with cooperative game theory is formulated to find the Pareto optimal solution of two adjustable parameters. Finally, the simulation results verify the existence of Pareto optimal solution and the   Fig. 17 Absolute cumulative error of h 2 effectiveness of the optimal robust control proposed in this paper. Last but not least, the fuzzy set theory describes the uncertainties first, which are embedded into the robust control. Then, the cooperative game theory is used to optimal design. Our work extends the application of U-K, and cooperative game theory has a great significance on trajectory tracking control of LLER with uncertainties. The experiment will be another important work in the next step.
Authors' contributions JT involved in writing-original draft, validation and software. LY and LH involved in writingreview and editing, and supervision. WX, TR and JZ took part in methodology, investigation and formal analysis.
Funding The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: The authors would like to thank the National Natural Science Foundation of China [U1813220, 62063033] and Fundamental Research Funds for the Central Universities [buctrc202105] for their support in this research.
Data availability All data are available upon request at the authors' email address.
Code availability Custom code is available upon request at Liang Yuan email address.

Declarations
Conflict of interest The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.