Discounted-cost linear quadratic regulation of switched linear systems

This paper investigates the design of discounted-cost linear quadratic regulator for a class of switched linear systems. The distinguishing feature of the proposed method is that the designed discounted-cost linear quadratic regulator achieves not only the desired optimisation index but also the exponentially convergent of the state trajectory of the closed-loop switched linear systems. First, the studied problem is transformed into a quadratic-programming problem by embedding transformation. Then, the bang-bang type solution of the embedded optimal control problem on a finite-time horizon is derived, which is the optimal solution to the original problems. The computable sufficient conditions on discounted-cost linear quadratic regulator are proposed for finite-time and infinite-time horizon cases, respectively. Finally, two examples are provided to demonstrate the effectiveness of the proposed method.


Introduction
A switched system is composed of a family of subsystems and the rules orchestrating the switching between the subsystems (Huang et al., 2021;Li et al., 2021;Liberzon, 2003;Liu et al., 2021;Miladi et al., 2021;Sathishkumar & Liu, 2020;Wang et al., 2015). Switched systems have a wide range of practical engineering applications, such as aerospace field, chemical, biology, economics, and so on. The research of switched systems mainly focused on the analysis of dynamic behaviours, such as stability (Liberzon, 2003;Wang et al., 2020Wang et al., , 2021Wang et al., , 2014 and controllability (Daafouz et al., 2002), and synthesis problems, such as stabilisation (Huang & Xiang, 2016;Jin et al., 2019), H ∞ control , and so on. Recently, the optimal control of switched systems has attracted the attention of many researchers (Jin et al., 2014;Riedinger, 2014;Zhu & Antsaklis, 2015). Switched linear system is an important classification of switched systems. It can not only use the relatively mature analysis tools and methods of linear systems but also has high modelling accuracy. Therefore, in recent years, this field has attracted the research of many scholars and produced many research results (Baldi & Xiang, 2018 Ma, Chen, et al., 2021;Zhang et al., 2020).
The linear quadratic optimal control is one of the most important optimisation synthesis problems of switched linear system, whose control laws are often derived by taking the maximum or minimum of the performance index (Sun & Ge, 2005). The necessary condition of linear quadratic optimal control problem is to maximise a Hamiltonian function. The linear quadratic optimal control problem has been studied on a finite-time horizon  and an infinite-time horizon (Fan & Yang, 2017), respectively. For the finite-time horizon case, huge efforts have been devoted to the study of linear quadratic optimal control by using the variational method, minimum principle and dynamic programming (Luus & Chen, 2004;Seatzu et al., 2006;Xu & Antsaklis, 2004). Luus and Chen (2004) demonstrated the optimal switching time instants and the optimal control policy can be solved readily by direct search optimisation for switched linear systems. Seatzu et al. (2006) proposed the optimal mode switching sequence and the optimal switching times, then determined the state space region where the optimal mode switching should occur and designed the state feedback control law last. Under the assumption that a series of active subsystems given in advance, Xu and Antsaklis (2004) transformed the optimal control problem into an equivalent problem parameterised by the switching instants, and then developed an effective method to obtain the optimal switching time. Zhang et al. (2012) developed an efficient algorithm to solve the infinitehorizon discrete-time switched linear quadratic regulator problem with guaranteed suboptimal performance. Branicky et al. (1998) designed the hybrid controllers for hybrid systems in an optimal control framework. Although some efforts have been done for the linear quadratic regulator (LQR) problem of switched systems, the results on the exact solution of a switched LQR problem of switched systems have been few reported in the literature since the mode sequence, switching instants and control input need to be designed simultaneously. Wu et al. (2018) utilised the embedding transformation method for the linear quadratic regulator problem of switched systems and proposed the sufficient conditions to design the linear quadratic regulator. It should be pointed out that the papers mentioned above do not take the discountedcost function into account when studying the optimal linear quadratic regulator problem of switched linear systems.
On the other hand, the stabilisation of an equilibrium point via optimal control techniques has long been used as a method for computing feedback stabilisers, particularly in the context of the infinitehorizon linear quadratic regulator problem. It is shown that the discounted-cost linear quadratic regulator not only achieves the prescribed optimisation index but also guarantees the exponential stability of the optimal control system (Anderson & Moore, 1989). Therefore, the discounted-cost linear quadratic regulation problem has attracted increasing attention in recent years, specially for non-switched systems (Bijl & Schon, 2017;Bijl et al., 2016). For the linear quadratic Gaussian systems, Bijl et al. (2016) presented a set of analytic expressions for the mean and variance of the discounted-cost function by Lyapunov equations. Bijl and Schon (2017) proposed the optimal controller and observer gains for a class of the linear quadratic Gaussian systems by solving the Riccati equation. For a class of descriptor systems, Mena et al. (2020) proposed an important result on the discounted-cost linear quadratic Gaussian control problem with a discounted-cost functional on the infinite-time horizon. However, to our best knowledge, there are few results on the discounted-cost linear quadratic regulation problem of switched systems in the literature. This is the motivation of the present paper.
This paper investigates the discounted-cost linear quadratic regulation problem of the switched linear system in finite-time horizon and infinite-time horizon, respectively. The conditions for the exact solution of the discounted-cost linear quadratic regulation problem are first proposed. Then, both the optimal linear quadratic regulator for each subsystem and the switching signal are simultaneously designed. Specifically, in order to achieve the control objective of this paper, the switched LQR problem is converted to the continuous optimal control problem by adopting the embedding transformation method, in which, the sequence of modes is viewed as a control variable. It is proved that computing the optimal switch input leads to a quadratic-programming problem. The minimisation of a concave function is solved and a bangbang type solution is obtained. Therefore, a closedform optimal switching condition of subsystems can be developed. The mode sequence and the switching instants are determined afterward. For the infinitetime horizon case, the condition is to solve the Riccati equation. Finally, two examples are presented to demonstrate the effectiveness of the proposed method.
The main contributions of this paper are summarised as follows: (1) The problem of discounted-cost linear quadratic regulation of the studied switched linear systems is studied for the first time, which can cover the normal linear quadratic regulator problem as a special case. (2) The exact optimal solution of the switching sequence and the control input to the studied problem in this paper is obtained.
The paper is organised as follows. In Section 2, the problem formulation and preliminaries are presented. Main results are given in Section 3. An example is shown to illustrate the validity of the theoretical results in Section 4. Finally, some conclusions are drawn in Section 5.

Problem statements and preliminaries
Consider the following switched linear systems: where x(t) ∈ R n is the system state, u(t) ∈ R n 1 is the control input, σ (t) : [0, ∞) → M = {1, 2, . . . , m} is the switching signal, which is assumed to be a piecewise continuous (from the right) function depending on time, m ≥ 2 is the number of modes (called subsystems) of the switched system, and the system matrices A i and B i , ∀ i ∈ M, are assumed to be known with appropriate dimensions. For the switched systems (1), this paper will study the discounted-cost linear quadratic regulation problem, that is, both the control input u(t) for each subsystem and the switching signal σ (t) will be co-designed to minimise the following discounted quadratic cost function: where Q ∈ R n×n is the positive semi-definite matrix, R ∈ R n 1 ×n 1 is the positive definite matrix, and S ∈ R n 1 ×n , with the assumptions that 2 ) is observable, t 0 is a fixed initial time and t f is a fixed final time, and ν > 0 is called a discount exponent which represents a prescribed degree of stability. In addition, we assume that the following holds: It should be noted that (3) is a n 1 dimensional row vector.
Remark 2.1: Note that, when m = 1, the optimal control problem of non-switched linear systems with the discounted quadratic cost function has been extensively studied in, for example, Bijl and Schon (2017) and the references therein. On the other hand, when ν ≡ 0, the discounted quadratic cost function (2) reduces to the quadratic cost function in Wu et al. (2018) where the optimal control problem of switched linear systems with linear quadratic (LQ) cost or multiple LQ cost was investigated. So, the problem to be studied in this paper is much more general than the one in Bijl and Schon (2017); Wu et al. (2018).

Finite-time switched linear quadratic regulator
Theorem 3.1: Consider switched system (1) with discounted quadratic cost function (2). For given matrices then, the switching signal and the optimal control input achieve the finite-time discounted-cost linear quadratic regulation (2) of switched system (1).
Proof: Firstly, we definē where ν is the discount exponent in (2). Calculating the derivative of (7) yieldṡ By using the embedding transformation (Wu et al., 2018), we obtain the embedded switched system: and the cost function where w i (t) ∈ {0, 1}. The problem of embedded switched linear quadratic regulator (ESLQR) can be defined as determining a switch input and a control input associated with a general LQ cost function for evaluating the system's performance quantitatively in a finite-time horizon (t 0 , t f ). That is, minimise the quadratic cost function J: s.tẋ where the time-varying vector w(t) = [w 1 (t), . . . , w m (t)] T belongs to a convex set W, Now, define a new vector functionλ(t) = λ(t) e νt . Then, we have thaṫ That is,λ(t) = [λ 1 , . . . ,λ n ] T is the solution oḟ From (6) and (16), it is clear that The Hamilton function is defined as Then, substituting (17) into (19) yields Therefore, to minimise H with respect to w(t) can be simplified to minimise i.e.
By substituting (23) into (22), the quadraticprogramming problem is where G(i, j), then, we know that the matrix G(t) is symmetric.
In order to clearly express G(t), a new matrix M st is defined, in which the ith row and jth column element of matrix M st is constructed by the sth row and tth column element ofB ij : Therefore, we have where matrix T is a linear combination of N s denoting as T = n t=1λ s N s . As R is positive definite in (2), one can obtain that −G(t) ≤ 0. Therefore, the quadratic-programming problem (24) can be viewed as a minimisation of a concave function. In this case, the global minimum point ofH is always attained at the extreme point of the convex set W, i.e. the optimal solution to the ESLQR problem is of bang-bang type. Thus, the global minimum point ofH is as follows: where w k = 1 and w i = 0, ∀ k = i. This completes the proof.
Remark 3.1: Unlike Wu et al. (2018) where the normal linear quadratic regulator problem of switched linear systems is studied, Theorem 3.1 provides a sufficient condition to achieve for the discounted-cost linear quadratic regulator for a class of switched systems via co-designing the switching signal and the subsystems' controllers. It should be noted that when ν = 0, the studied problem in this paper is reduced to the normal linear quadratic regulator problem of switched linear systems in Wu et al. (2018). Therefore, our obtained result extends the normal linear quadratic regulator to the discounted-cost linear quadratic regulators of switched linear systems.

Infinite-time switched linear quadratic regulator
For switched system (1), the infinite-time horizon case of the discounted quadratic cost function (2) with t f = ∞ as follows: and co-design the control input u(t) and the switching signal σ (t) to minimise (30). Now, the following theorem is stated.

Theorem 3.2: Consider switched system (1) with discounted quadratic cost function (30). For given matrices
then, the switching signal and the optimal control input achieve the infinite-time discounted-cost linear quadratic regulation (30) of switched system (1).
Remark 3.2: Theorem 3.2 provides a sufficient condition to achieve for the discounted-cost linear quadratic regulator for a class of switched systems in an infinitetime horizon. Note that if the discounted-cost linear quadratic regulator for a class of switched systems in an infinite-time horizon, the whole closed-loop switched system is asymptotically stable. Otherwise, the state of the closed-loop switched system will no longer converge, and then the controller of the subsystem will not converge. This means that the performance index J will no longer be a finite value, which is contrary to the finite value of J obtained in Theorem 3.2. Therefore, the closed-loop switched system is asymptotically stable.

Remark 3.3:
In order to apply our proposed method, the continuous-time algebraic Riccati equation (31) is first calculated to obtain P i , then the controller (33) is designed for each subsystem. Thus, the switching signal is designed by (32). Both the controller (33) and the switching signal (32) are designed by the unique solution P i of the continuous-time algebraic Riccati equation (31). The solution of the closedloop switched systems depends on the designed controller (33) for each subsystem and the switching signal (32). Once the controller (33) for each subsystem is designed, the solution of the active closed-loop subsystem is the existence and uniqueness. Then, the existence and uniqueness of solution of the closed-loop switched system is also guaranteed by the designed switching signal (32).

Remark 3.4:
The discounted-cost linear quadratic regulation of the closed-loop switched systems is obtained under the condition that each closed-loop subsystem is asymptotically stable. From (33) and (1), one has thaṫ Taking the time derivative of Since Q − S T R −1 S ≥ 0 and ν > 0 in (3), P i > 0, and P i B i R −1 B T i P i > 0, then one hasV(x) < 0. Thus, each closed-loop subsystem is globally asymptotically stable.

Example 1
Consider a switched system (1) borrowed from Wu et al. (2018) with the following subsystems: Note that both the eigenvalues of matrix A 1 are 0, the eigenvalues of matrix A 2 are 0 and 1, and the eigenvalues of matrix A 3 are 0 and 2. All open-loop subsystems without controllers are not asymptotically stable. However, all subsystems are controllable. Thence, each subsystem can be asymptotically stabilised by our proposed controller and all closed-loop subsystems are asymptotically stable. That is, we are trying to switch among the asymptotically stable subsystems. The cost function is defined as where initial time t 0 , initial state x(0), and the weighting matrices Q = R = 2I and S = I (I is an n × n identity matrix). Our purpose is minimising the overall performance index (47) by co-designing switching signals and controllers for each subsystem. Let ν = 1, t 0 = 0, t f = ∞, and x(0) = [2, 1] T . By solving (31), P 1 = 15.542997197757764 −9.595998398718718 −9.595998398718718 6.946998799039037 , Then the switching signal is designed: and the controllers for each subsystems: According to Theorem 3.2, the performance index (47) is minimised by (51) and (52). The state trajectories are shown in Figure 1. The optimal switching time is drawn in Figure 2. Note that the system switches frequently between modes 1 and 2 for 0 s ≤ t < 1.241 s, and switches frequently between modes 1 and 3 for 1.241 s ≤ t < 2 s. These illustrate the efficacy of our results. If ν = 0 in (47), the studied problem in this   paper is reduced to the non-discounted-cost function case in Wu et al. (2018), whose state trajectories are shown in Figure 3. It can be seen that our obtained the state trajectories have a faster convergence speed than the one obtained in Wu et al. (2018) since there exists a discounted function in the performance index. On the other hand, the optimal value of J in this paper is 1.587, while the optimal value of J in Wu et al. (2018) is J = 2.01. Thus, our obtained result is smaller than the one obtained by Wu et al. (2018).

Example 2
A buck converter is described as a switched system (1) borrowed from Deaecto et al. (2010): anḋ (54) The cost function is defined as where initial time t 0 , initial state x(0), and the weighting matrices R 0 = 2/25 ,R = 2 , L = 0.5H,C = 0.05F, Q = 10I and R = 2I (I is an n × n identity matrix). Our purpose is minimising the overall performance index (55) by co-designing switching signals and controllers for each subsystem.
Let t 0 = 0, t f = ∞, ν = 4, and x (0) Then the switching signal is designed: and the controllers for each subsystem: For the finite-horizon switched linear quadratic regulator, according to Theorem 3.2, the performance index (55) is minimised by (57) and (58). The state responses are depicted in Figure 5, and the switching signal is drawn in Figure 6. If ν = 0, the studied problem in this paper is reduced to the non-discountedcost function case, whose state trajectories are shown in Figure 7. One can see that our obtained the state trajectories have a faster convergence speed since there exists a discounted function in the performance index. The optimal value of J is 26.9. The buck converter can make the output voltage lower than the input voltage, Figure 5 show the startup from the initial condition x(0) = [12, 15] T to the reference x ref = [10, 10] T . These illustrate the efficacy of our results.

Conclusion
This paper has investigated the discounted-cost linear quadratic regulation of switched linear systems in the finite-time horizon and infinite-time horizon, respectively, which involves finding a mode sequence, switching times between the modes and an input for each mode. The distinguishing feature of the proposed method is that the designed discounted-cost linear quadratic regulator achieves not only the desired optimisation index but also the exponentially convergent of the state trajectory of the closed-loop switched linear systems. The studied problem is transformed into a quadratic-programming problem by embedding transformation. Then, the bang-bang-type solution of the embedded optimal control problem is derived, which is the optimal solution to the original problems. The bang-bang-type solutions of the embedded optimal control problem are to be shown as the optimisation solution of the studied problem. Then, the computable sufficient conditions on discounted-cost linear quadratic regulator are proposed for finite-time and infinite-time horizon cases, respectively. Finally, two examples have been shown to demonstrate the effectiveness of the proposed method. It should be pointed out that the proposed control approach is developed without considering input constraints. These input constraints can be seen in practice. Future works will focus on the topic of discounted-cost linear quadratic regulators with input constraints.

Data availability statement
Data sharing is not applicable to this article as no new data were created or analysed in this study.

Disclosure statement
No potential conflict of interest was reported by the author(s).