A family of quaternion-valued pipelined second-order Volterra adaptive filters for nonlinear system identification

This paper primarily proposes a family of quaternion Volterra filters based on the feedforward pipelined structure (QPSOVAFs) for nonlinear quaternion system identification to reduce the computational complexity. Then, the strictly nonlinear QPSOVAF (SNL-QPSOVAF), semi-widely nonlinear QPSOVAF(SWNL-QPSOVAF) and widely nonlinear QPSOVAF (WNL-QPSOVAF) are proposed. This architecture consists of several quaternion-valued second-order Volterra (SOV) modules. The structure’s nonlinear subsection executes a nonlinear mapping from the input space to an intermediate space using the feedforward SOV; the linear combiner subsection performs a linear mapping from the intermediate space to the output space. Moreover, the theoretical analysis expresses the effectiveness of the proposed QPSOVAFs in a specific condition. Finally, simulation results further prove that the proposed QPSOVAFs have good performance in identifying the quaternion-valued nonlinear system.


Introduction
The Volterra filter is widely applied to process nonlinear problems [1][2][3][4][5][6]. It was usually used in many nonlinear applications such as system identification [7,8], acoustic echo cancellation [9], speech prediction [10], biomedical engineering [11], channel equalization [12] and digital communication channels [13]. However, the computational complexity of the Volterra filter (VF) will exponentially increase with the order and memory (or delay) of the filter increasing [14], which significantly limits the practical applications of the VF. Then, the complex-valued SOV proposed to characterize the complex-valued nonlinearity, which has used to many applications such as nonlinear amplifier modelling [15], beam-forming [16,17], Nyquist subcarrier modulation [18] and transmitter modelling [19]. However, neither the real model nor the complex-valued model could be effectively used to model nonlinear 3-D and 4-D signals.
Recently, the quaternion-valued SOV adaptive filters (QSOVAFs) were provided [39], which can efficiently process the nonlinear signals. However, it has a high computational burden and a low convergence speed in the identification system. To solve the high computational cost of quaternion-valued SOV, we introduce a novel pipelined architecture. This idea is inspired by the efficient pipelined structure [40,41] and later was improved by other researcher [42] leading to the novel adaptive joint process filter using pipelined feedforward second-order Volterra architecture (JPPSOV). The merits of the pipelined architecture are that the pipelined realization modularity and such realizations use less computational cost to approximate the nonlinear system effectively. However, the traditional pipelined structure ignores the importance of the signals closer to the current moment, leading to the poor performance. Therefore, we present a novel pipelined structure and then propose a quaternion-valued secondorder Volterra filter family based on the novel pipelined structure (QPSOVAFs), which has better performance than the JPPSOV [42].
In this study, we mainly have the following contributions: 1. To overcome the drawback of the architecture of the JPPSOV [42], we proposed a novel structure of QPSOVAF by maximizing use of the information of the signals closer to the current moment, which leads to better performance. 2. Adaptive algorithms for QPSOV are derived via using the generalized Hamilton-real (GHR) calculus. For the strictly nonlinear (SNL), semi-widely nonlinear (SWNL) and widely nonlinear (WNL) QSOVAFs, we propose novel weight update equations of the SNL-QSOVAF, SWNL-QSOVAF and WNL-QSOVAF, respectively, and the simulation proves that QPSOVAFs could better deal with nonlinear identification system. Compared with the conventional JPPSOV which cannot be applied to the modelling nonlinearity of 3-D and 4-D signals, the proposed QSOVAFs enable the modelling of nonlinear 3-D and 4-D signals in the quaternion domain and have merit pipelined architecture. 3. This study provides the convergence analysis and the mean square steady-state analysis to obtain the stable value range of the step size and the steadystate performance of the proposed QPSOVAFs.
In a word, this paper tries to implement low computational complexity SNL-QPSOV algorithm (will be emphatically studied), SWNL-QPSOV algorithm and WNL-QPSOV algorithm in the quaternion domain. The rest of this article is listed as follows. Section 2 reviews the quaternion theory. Sections 3 and 4 derive the proposed QPSOVAFs. In sect. 5, we give the performance analysis. Section 6 introduces the simulations. Finally, Sect. 7 summarizes this work.

Basic properties of quaternion algebra
The quaternion is a non-commutative extension of the complex number, i.e. it does not conform to the commutative law. For example, define the quaternion variable s as follows s = s a + is b + js c + ks d (1) where let R(s) = s a represents the real part and I(s) = is b + js c + ks d denotes the imaginary part with s a , s b , s c and s d being belong to the real-valued. {1, i, j, k} denotes a basis of the quaternion variable s and satisfies Define the conjugate of s as s * = s a − is b − js c − ks d and satisfies (qs) * = s * q * . The quaternion variable s has the following properties: (1) Defining its modulus and inverse operations as |s| = √ ss * , |qs| = |q| |s| , s −1 = s * / |s| 2 , and (qs) −1 = s −1 q −1 , where s = 0; (2) The polar form : s = |s| (cosθ +ŝsinθ), where θ = arccos(R(s)/ |s|) ∈ R is a angle andŝ = I s / |I s | is a pure unit quaternion; (3) s has rotation and involution properties. For the property (3), we define the 3-D rotation of the vector part of s by an angle 2θ about the vector part of μ as [43] s μ μsμ −1 (2) where μ = |μ| (cosθ +μsinθ) represents a nonzero quaternion.
(2) becomes a quaternion involution for a pure unit quaternion μ [43,44]. Similarly, s i , s j and s k can be defined by The other properties of quaternion rotation is listed as follows where μ is not necessarily a unit quaternion. However, μ/ |μ| = 1 is an unit quaternion. For more properties and details about the quaternion algebra can be looked for [29,43].

GHR calculus
Recently, [29] proposed HR calculus to decrease the high computational complexity of classical quaternion pseudo-derivatives. However, it does not include the traditional product and chain rules. Therefore, [31] provided GHR with novel product and chain rules.
Product rules: Since the proposed series of QPSOVAF algorithms mainly use the left GHR calculus attributes above in this paper, for more details about GHR calculus, please refer to [29,31].

WL model and SWL model
Usually, R = E{xx H } is defined as the classical covariance matrix to evaluate the second-order circular signals, where x is a random vector, and (·) H denotes the Hermitian transpose. However, in practical application, we often encounter second-order non-circular signals, which are no longer sufficient to process by the rules mentioned above of circular signals. Thus, we also need to evaluate the pseudo-covariance matrices, i.e. P = E{xx i H }, S = E{xx j H } and T = E{xx k H }.
To this end, the quaternion-valued WL modelling is given by [26] where y denotes the filter output, x kT ] T are the augmented weight vector and the augmented input vector, respectively. Therefore, combining the three pseudocovariance matrices and covariance matrix, the augmented covariance matrix R a = E{x a x a H } can be expressed as [26] Note that the augmented covariance matrix applies to second-order non-circular signals. If the quaternionvalued signal is a circular signal, then P, S and T will disappear. In addition, simplifying WL modelling to SWL modelling to obtain the special quaternion-valued signals' second-order statistical characteristics, Eq. (9) will become where the augmented weight vector of semi-widely linear modelling is h a = [w T , g T ] T , and the augmented input vector is x a = [x T , x i T ] T In addition, the augmented covariance matrix of semi-widely linear modelling is [37,38] Note that the pseudo-covariance matrix P will disappear for a circular second-order quaternion-valued signal, just like widely linear modelling.

Review of the quaternion-valued SOV
In the quaternion-valued domain, the classic Volterra series can be written as where y(m) ∈ H and h p n 1 , . . . , n p ∈ H are the output and the set of pth-order kernel coefficients of the quaternion-valued Volterra, respectively. h 0 denotes the zeroth-order quaternion-valued kernel constant, and N denotes the system memory size. Considering the high computational complexity of the Volterra series model in (13), we mainly focus on the SOV architecture for identifying the nonlinear system in this paper. Therefore, define the output of quaternionvalued SOV as where h 1 (n 1 ) is the first-order quaternion-valued kernel coefficients and h 2 (n 1 , n 2 ) is the second-order kernel coefficients. Equation (14) can also be rewritten as a vector form, as follows where h(m) and x(m) are the expanded weight vector and the input vector of QSOV, respectively. They are defined as where the length of x(m) is expressed as [45]

The novel QPSOVAF structure
Though the quaternion-valued SOV have good performance, its implementation to treat nonlinear cases would be unfeasible, as it would require an additional exponential increase in the number of coefficients. Therefore, we study the novel QPSOV filter structure in this subsection. As demonstrated in Fig. 1, the nonlinear subsection of the proposed novel QPSOV filter consists of a certain number of M identical feedforward SOV modules. Moreover, the output of the nonlinear subsection constitutes the linear subsection input and use the transversal filter to represent the linear filter in this paper. This combination structure of nonlinear and linear processes can effectively extract the nonlinear and linear relationships. More details about the proposed novel QPSOV filter architecture will be discussed in the following.

Nonlinear subsection
We can also clearly see from Fig. 1 that each module has two input signals: One is an external input signal, and the other is the output signal of the previous module. Since applying the same modules in this structure, they have the same number of input signals and similar operations. Besides, the synaptic weight matrix of all the modules of the proposed QPSOVAFs are set the same. Figure 2 shows the details of the r th module.
First, define the input vector X r (m) of the r th module as where y r −1 (m) represents the previous module output, and 1 r < M. However, if r = 1, then X r (m) is expressed as Then, the r th module input vector X r (m) is extended to U r (m) by the SOV series and is written as where the length of U r (m) is given by Eq. (18), i.e. L = 1 + 2 + 2(2 + 1)/2 = 6. Since the weight and rules of all modules are identical, the synaptic weight vector of the SOV series of each module is expressed by H(m), whose length is also L, and written as Therefore, the r th module output can be represented by

Linear subsection
In Fig. 1, each module's output connects a conventional transversal filter and its weight vector can be expressed as where M represents the number of SOV modules in structure. The linear filter input consists of the present output y r (m) by each module, as followŝ The output of the M cascaded modules forms an M × 1 output vectorŷ(m) in (25). Therefore, the linear filter output y(m) can be written as where y(m) is a estimated value of the practical desired sample d(m − 1) or a predicted value of the practical desired sample d(m). Compared with the nonlinear part, the storage space of the linear filter is limited.

Comparison with the traditional structure of JPPSOV
The block diagram of the JPPSOV [42] is depicted in Fig. 3. First, Fig. 3 shows that the JPPSOV uses the modular output feedback as input in final modular, i.e. final modular in essence is bilinear modular, which results in slow convergence speed and latent instability as same as adaptive bilinear filters even if the measurement noise is the Gaussian process. Th proposed novel PSOV use previous modular output as input in final modular, which improve the stability of nonlinear adaptive filter. Second, the number of the sub-module of the proposed novel pipelined Volterra structure in Fig. 1 can easily vary while the conventional pipelined Volterra structure cannot. Therefore, the proposed novel pipelined Volterra structure is more flexible than the pipelined Volterra structure in [42], which is important for the practice. Lastly, from Fig

Adaptive algorithms of the proposed QPSOVAFs
This section uses the pipelined filter structure to effectively extract the nonlinear and linear information. To be more specific, the input signal is linearized in the nonlinear subsection. These linearized data then feed into a linear combiner used the corresponding adaptive algorithm to generate the estimated value of the original desired signal.
Here, the cost function of the algorithm is defined by where the quaternion-valued error signal e(m) is given by Then, substituting (30) and (31) into (29), we get According to Eq. (32) and LMS adaptive algorithm [25,46], we get the weight update formula of the proposed SNL-QPSOVAF, as follows where λ 1 denotes a linear step size and the constant 1/2 in (32) is absorbed into the step size.

Nonlinear subsection
We still use the minimize the cost function J (m) to update the nonlinear weight vector H(m) of the proposed SNL-QPSOVAF as follows where λ 2 represents the nonlinear step size, and In order to obtain the above ∇ H * J (m), the rule given in (8) should also be used as follows Then, similar to (29), process two partial derivations in (36) as follows and where the approximate item depends on Eq. (14) in [31]. Note that we only consider the case where v = 1 when using Eq. (14) in [31], since other items are difficult to calculate and have little effect on the result.

Nonlinear subsection
According to SWL modelling in (11), similar to (23), the r th augmented model output of the SWNL-QPSOVAF is given by where H a (m) and U a r (m) are the augmented weight and input vector of the r th module of SOV series, respectively. H a (m) and U a r (m) are given by where Similar to Eq. (34), minimizing the cost function J (m), the weight update function of the proposed SWNL-QPSOVAF is obtained as Processing ∇ H a * J (m) by using the same manner in section A, we get

Linear subsection
Since the weight update of the linear subsection of the proposed model is the same as the previous part, it is not derived here. Please refer to the previous section IV-A for details.

Nonlinear subsection
In this section, considering the WNL model given in (9), we define the r th augmented model output of the proposed WNL-QPSOVAF in the vector form as (50)

Linear subsection
Note that the update of the parameters of the linear subsection is unchanged with the last part. So please refer to the previous section A for details.

Convergence analysis
To analyse the proposed algorithms, we primarily process the following assumptions.

Linear subsection
Substituting (28) into (33), we can obtain Rearranging yields we have Taking the transpose of both sides of (53) yields Using (54) yields where I M represents the M × M unit matrix.
Using Assumptions 1-4, the expectation on both sides of (55) can be obtained by Simplifying the above formula according to the Wiener filter theory [47] yields Finally, we can gain the step size range where ζ max1 is the max eigenvalue of matrix Rŷŷ.

Nonlinear subsection
Similar to the linear subsection, the derivation of J (m) with respect to H(m) can be rewritten as The first derivative on the right side of (60) is derived as follows where The second derivative on the right side of (60) is derived as shown below Therefore, according to Eqs. (61) and (63), (60) can be rewritten as Substituting (64) in (40) can be obtained as shown below Then, taking the expectations on both sides of (65) and using Assumptions 1-4, we can obtain E H(m + 1) where Considering the Wiener filter theory [47], Eq. (66) can be simplified as From the above equation, we can get the range of step size λ 2 as follows where ζ max2 denotes the max eigenvalue of matrix R . (41), (45) and (68), we can obtain the weight expectation update function of the SWNL-QPSOV algorithm as follows E H a (m + 1)

Remark 1 Combine
Then, by analysing (70), which is similar to the convergence of the SNL-QPSOV algorithm, the step size range of the SWNL-QPSOV algorithm can be finally obtained as where ζ a max2 denotes the max eigenvalue of matrix R a , which only include R and P defined in (12). (47), (50), similar to the above proposed algorithm analysis, we can get the step size range of the WNL-QPSOV algorithm, as follows

Remark 2 Combining
where ζ a max2 is the max eigenvalue of R a , which include R, P, S and T defined in (10).
Note that the linear subsection analysis of the SWNL-QPSOVAF algorithm and the WNL-QPSOVAF algorithm are the same as the SNL-QPSOVAF algorithm. Therefore, there is no need to go into details here, and please refer to the linear subsection analysis of the SNL-QPSOV algorithm.

Mean square steady-state analysis
In this section, the mean square steady-state theoretical performance of SNL-QPSOVAF algorithm under the Gaussian input assumption is discussed.

Steady-state analysis of W(m)
Let W(m) = W o − W(m), where W o is the optimal weight vector. Then, subtracting both sides of (33) from Then, calculating the 2-Norm of both sides of (73) yields Considering E W(m + 1) 2 = E W(m) 2 for m → ∞ and using assumptions, so the expectation of (74) can be expressed as The a prior error (m) of the whole system is consist of the linear error w (m) and the nonlinear error where E{ ŷ * (m) 2 } can be calculated by using the approximation H(m) = H o . Then, rearranging (76), we have which leads to the EMSE of the linear section as m → ∞, i.e.

Steady-state analysis of H(m)
Similar

Steady-state EMSE for SNL-QPSOVAF
The whole EMSE of the SNL-QPSOVAF algorithm is evaluated by On the condition of system at steady state, Therefore, (85) becomes

Complexity analysis
For the convenience of observation, we show the complexity evaluation in table I. As can be seen from the Table 1, if N , M and K are, respectively, set to 10, 5 and 2, the SNL-QSOVAF requires 66 additions and 131 multiplications while SNL-QPSOV needs only 25 additions and 41 multiplications. Therefore, we can obtain that the computational complexity of the proposed QPSOVs, is significantly lower than that of the QSOVAFs.

Simulation
In this section, simulated experiments are carried out to illustrate the performance of the proposed filters and verify the correctness of the steady-sate analysis.

Case study 1
We use the following model of the unknown system as where d(m) represents the output of the nonlinear system, and x(m) denotes the input of the unknown system. d(m) is also interfered with the independent Gaussian white noise υ(m). In the following experiments, the input x(m) is assumed as a white Gaussian noise with zero-mean and unit variance. We use the mean square error (MSE) defined as 10log10[e(m)] 2 to evaluate the algorithm performance. The simulated results are gained by averaging the results over 30 independent runs. The modules number M is set to 5, and the number of an external signal of each module is set to 1.
In Fig. 6, the step sizes λ 1 and λ 2 are set to 0.003. Figure 6 clearly shows that the convergence speed of the proposed WNL-QPSOV is faster than those of other proposed algorithms under the same steady-state error condition. Figure 7 compares the performance of the proposed SNL-QPSOV algorithm with that of the conventional SNL-QSOV algorithm [39], where the step sizes λ 1 and λ 2 of the SNL-QSOVAF and the proposed SNL-QPSOV are all set to 0.002. The signal-to-noise ratio (SNR) for Figs. 7a and 7b is set to 30dB and 20dB, respectively. The simulation result shows that the proposed SNL-QPSOV has a better performance than the SNL-QSOVAF.

Case study 2
In this experiment, we compare the simulated EMSE with the derived theoretical EMSE, where the unknown nonlinear system is given by In order to calculate the theoretical EMSEs given in (78) and (84) Figure 8 gives the simulated and theoretical EMSE curves of the SNL-QPSOV adaptive filter where H o is available. Figure 9 compares the simulated and theoretical EMSE versus step size. Figures 8 and 9 show the proximity between the theoretical and simulated EMSEs for different step sizes, where the variance δ 2 v is set to 0.001. Figure 10 depicts the simulated and theoretical EMSE curves of the SNL-QPSOV adaptive filter where W o is available. Figure 11 compares the simulated and theoretical EMSE versus step size, where the variance δ 2 v is set to 0.002. Figures 10 and 11 clearly shows that the simulated EMSE is very close to the theoretical one.

Case study 3
In this experiment, we verify the advantage of the proposed pipelined architecture. To get a fair comparison, we assume that the imaginary part of all signals in the proposed SNL-QPSOVAF are zero, because the JPPSOV is only for real-valued signals. In the experiment, the model of the unknown system is The measurement noise belonged to zero-mean and white Gaussian sequence, and the input signal-tomeasurement noise ratio is chosen to be 30 dB. Figure 12 presents the simulated results. It is clearly seen   that the proposed SNL-QPSOVAF obtain better performance than the JPPSOV, which demonstrates the superiority of the novel pipelined structure.
In this experiment, we also test the effect of the number of modules. Figure 13 depicts the steady-state MSE of the SNL-QPSOVAF with M increasing from 1 to

Conclusion
This paper first proposes the QPSOVs and gives detailed derivations. Since the pipelined structure is used, these algorithms could significantly reduce the computational complexities of conventional adaptive quaternion-valued SOV filters. Then, the convergence analysis, the complexity analysis and the mean square steady-state analysis of the proposed QPSOVs are provided. Moreover, this paper focuses on the study of the SNL-QPSOV algorithm. Considering that the analysis methods of the other two proposed algorithms are similar to the SNL-QPSOV algorithm and the length of the paper is limited, we do not conduct the detailed studies on the WNL-QPSOV and SWNL-QPSOV algorithms here. The results of the SNL-QPSOVAF algorithm can be easily expanded to its SWNL and WNL versions. Finally, we further verify effectiveness of the proposed algorithms and the correctness of the theoretical analysis through the simulations.
Some potential future work contains: (1) the extension to constrained minimization problems; (2) the robust QPSOVAFs against the impulsive noise in quaternion domain; and (3) the variable step size version of QPSOVAFs to improve the convergence rate.