THEORY AND APPLICATIONS OF ADAPTIVE SECOND ORDER IIR VOLTERRA FILTERS E. Roy, R. W. Stewart and T.S. Durrani Department of Electronic and Electrical Engineering Signal Processing Division University of Strathclyde Glasgow G I IXW, Scotland, U.K. E-mail: [email protected] rat hcly de.ac.uk ABSTRACT In this paper, an adaptive nonlinear filter based on a second order Volterra series and on an IIR filter structure is presented. This filter is able to model higher than second order nonlinearities for systems where the nonlinearities are harmonically related. This solution represents an alternative to using higher than second order Volterra filters. We present a full derivation of this gradient search based adaptive nonlinear filter and also highlight the various assumptions and simplifications which require to be made in order to produce a practical algorithm. A comparison is made in terms of performance and computational complexity between an adaptive second order IIR Volterra filter and an adaptive second and third order Volterra filters.

Figure 1: SOV-IIR filter in a system idlentification configuration. z ( k ) is the input

2. ADAPTIVE SOV-IIR FILTER 1. INTRODUCTION Nonlinear filters based on the Volterra series [l], [2] have been increasingly studied over the past few years and some applications based on second order (FIR)' Volterra filters have been performed. However, second order Volterra filters cannot deal with higher than quadratic nonlinearities and therefore some current research tends to focus on increasing the Volterra filter order to tackle higher order nonlinear systems. In this paper, the adaptive SOV-IJR filter is considered, which is a nonlinear extension of adaptive linear IIR filters [3] with nonlinear feed-back [l]using second order Volterra filters. The filter derivation is based on a gradient search method where some approximations for practical applications are considered and ultimately two forms of adaptive IIR algorithms are presented. The adaptive SOV-IIR filter can be seen as an alternative solution to higher order nonlinear system modelling when the nonlinearities produced by the system are directly dependent on the lower ones, in keeping a second order computational cost. The work is supported by the U.K. EngineeringandPhysical Sciences Research Council Grant GR/J/88243. 'The term Volterra FIR filter is used and referenced as SOVFIR filter to dearly distinguish the traditional Volterra filters from the presented Volterra IIR filter referenced as SOV-IIR filter.

0-7803-3 192-3/96 $5.0001996 IEEE

1597

Figure 1 shows the block diagram of the adaptive SOV-IIR filter which can be seen as a nonlinear extension of the IIR filter [3] where the non-recursive and recursive filters are two SOV-FIR filters. Its discrete output y(k) at time k is defined as N-1

i=O M

N-1 N-1 i=O

j=i

M

M

1=l

3=1

where z ( k ) represents the input signall, N and M are the number of the non-recursive and recursive filter coefficients respectively. The values a, and in (1)are the linear and quadratic filter coefficients for the non-recursive filter respectively, and b, and & the equivalent for the recursive filter. Eq. (1)can be rewritten as a l x ( N ( N 3) M(M 3))/2 vector product

-+ +

+

and $,(k) the input vector composed of the linear and quadratic current and previous input and output samples, defined as

. ,.(k - N + 11,."k),.(k).(k - I), . . . , 2 ( k - N + l), y(k - l), . . . , (4) y(k - M ) , y2(k - l), . . . ,y2(k - M ) ] * .

2.1.i. Full Grudient

In assuming that the filter weights are varying slowly3 then we can generalise and assume:

$,(k) = [.(k),..

The adaptive algorithms to minimise the error signal power are based on a gradient search type update and in the following sections, a straight-forward derivation of the gradient is firstly derived and thereafter two approximations are made to reduce the computational cost.

w(k) % w(k - 1) M . . .

N(N + 3 ) + M ( M + 3 ) zw(k-

2

(9)

>.

Therefore we can rewrite (8) replacing the current weight values v ( k ) with past weight values v(k - m ) :

2.1. Gradient search update

In order to minimise the error power, a gradient approximation of the mean squared error performance surface is calculated from the differentiation of the instantaneous squared error, e(k) (see Figure l), with respect to the weights

where

By expanding the differentiation of the double summation part of (10) with respect to the previous past weight vector v(k - m ) , and further relying on the slowly varying weight assumption such that w(k - m ) x v(R - n) (see (9)) then T,(k) can be re-written as:

4f is the information vector:

By substituting (11) in (7), 4r(k) is then composed of recursive equations (the partial derivatives of y(k) with respect to their current weights as a function of their corresponding past partial derivatives). This leads to an important simplification of the gradient computation but adds ( N ( N + 3 ) + M ( M + 3 ) ) / 2 filtering operations. The level of computation required by this algorithm is therefore high.

The differentiation leads to

-ay@) aai (k)

- .(k-

2)

+ Ya*(k)

2.1.2. Approximate Gradient

A more decisive simplification of (10) can be made in an analogous style with what was done by Feintuch for linear IIR filters [ 4 ] . Following Feintuch's assumption that the derivatives of the past output values with respect to the current weight vector are zero, Tv(k) will be assumed zero in (7). This simplification is often referred to as the pseudolinear regression (PLR) and is equivalent to the substitution of the information vector 4 f ( k )by the input vector 4 , ( k ) in (5) which greatly reduces the level of computation required. 3. ADAPTIVE NONLINEAR IIR ALGORITHMS

In ( 8 ) w is ai,a i , j ,bi or pi,,. A straight-forward utilisation of the gradient would require a recalculation of every single past output sample with respect to the current weight vector (see (8)) and huge computational requirements. In order to reduce computational costs, some simplifications can be made based on two different approximations2 which lead to the full grudient solution and the approximate gradient solution.

In the following utilisation of the SOV-IIR filter, the NewtonGauss algorithm [3] used for linear IIR filters has been extended to nonlinear systems in order to update the coefficients of the adaptive SOV-IIR filter. Two particular cases have been chosen which lead to the LMS SOV-IIR filter algorithm and the RPE SOV-IIR filter algorithm which has

2The approximations are an extension of those made for the linear case [3], [4]

3This slow varying weight assumptioncan only be made under small step size condition.

1598

a faster convergence behaviour (to the detriment of computational requirements). The weight update equation is defmed as

+

1) is an estimate of the Hessian matrix inwhere R-'(k verse, used to improve the convergence of the algorithm and calculated with the matrix inversion lemma. The step size matrix, p , of the algorithm is defined as p = diag(pl;, . . . , P L , c ( Q , .

.. ,PQ,VL,.

.. , V L , V Q , . .. , V Q )

where p ~p~, are the step sizes for the linear and quadratic parts of the non-recursive filter and V L ,V Q are the step sizes for the recursive filter.

3.1. The SOV-IIR

Frequency (Hz)

Figure 2: Unknown system output resul!ting from a 100 Hz sinusoidal input

L M S Algorithm

To obtain the SOV-IIR LMS type algorithm, the estimate of the Hessian matrix inverse in (12) is set to the identity matrix. Either the full gradient or the approximate gradient solution for the SOV-IIR filter developed above can be substituted in (12). Despite a more complex information vector compared to the approximate gradient, but at the expense of higher computational costs, the SOV-IIR filter based on the full gradient solution does not lead to any real improvement over the SOV-IIR filter based on the approximate gradient solution in minimising the mean squared error or in identifying the coefficients of the unknown system. 3.2. T h e RPE Algorithm

The RPE algorithm for the SOV-IIR filter has been derived from [3] where the scalar convergence factor a in 131 must be replaced by the diagonal matrix ,U. Nonetheless, a straightforward derivation of calculating the estimate of the Hessian matrix (i.e. R(k)) using the inversion lemma and including the matrix p leads to another matrix inversion which is not desirable for computational and stability features. Due to the characteristics of the convergence factor matrix which is diagonal and in defining 4 q , p ( k )as the weighted vector of (b,(k)4 (i.e. 4 q , y ( k )= p(bq(k)),no matrix inversion is required. The Hessian matrix is then recursively defined as

3.3. Stability

LMS algorithm Under an LMS update algorithm and although a full analysis of the stability and convergence properties of an approximate gradient type SOV-IIR filter has not been made, we are satisfied from the evidence obtained from extensive simulation that by judicious (or cautiously small) choice of the convergence factors, the algorithm is both stable and convergent. RPE algorithm Under an RPE update algorithm, some instabilities can occur independently under a careful choice of the convergence factor. This appears to be due to illconditioning of R-'(k) and is currently under investigation. Nonetheless and with some similarities with linear filters, an intermediate solution between the jull and the approximate gradients has been found and is referred to as the simplified gradient which appears more robust and avoids instability due to ill-conditioning of R.-'(k). However important the approximations used in this simplified gradient version are, we currently have no mathematical justification which would acknowledge the quality of the simulation results, where under same conditions, the filter based on the two other gradient versions leads to instable or less accurate performances. 4. SIMULATIONS

where X is the forgetting factor and by using the matrix inversion lemma, we obtain

R-'(k

+ 1)=

The SOV-IIR filter updated by a RPE algorithm is obtained by substituting (14) in (12). 4 4 , ( k ) represents either @,(k) 4f(k) full gradient choice.

in an approximate gradient or

1599

To demonstrate the algorithm performance, a system identification example is presented where the coefficients of the unknown system have been arbitrarily chosen in the interval (0,l) for the non-recursive coefficients and in (-0.1,O) for the recursive coefficients. The output d!(k) of the unknown system used is:

d(k) = 0.819Co + 0.547Si

+ 0.4789~:+ 0 . 6 7 9 3 ~ 0 ~ 1 + 0.2347~:- 0.0919di - 0.0047d2 - 0.04789d3 - 0.06793d; - O.Og347did2 - 0.03835d31d3 - 0.05194d; - 0.08130dzd3 - 0.010346d;.

(15)

1

0.8 0.6 0.4

0.2 “0

0.1

0.2

0.3

0.4

0.5 0.6 Frequency

0.7

0.8

0.9

1

017

0:8

019

!

(b)

1 0.8

0.6 0.4

0.2 Number of samplss

10’

1‘,

Figure 3: MSE using a uniformly distributed noise input: (a) SOV-FIR filter, (b) third order Volterra FIR filter, (c) SOV-IIR LMS filter and (d) SOV-IIR RPE filter. where E ; = ~ ( -kz) and d; = d(k - i) for notational convenience. In order to illustrate the high nonlinearities produced by the unknown system, Figure 2 is shown where the chosen ) szn(27rf0/f, k) where fo =100Hz driving input is ~ ( k = and fs =4kHz. The unknown system produces a high nonlinear output (up to about 8th order harmonics). In the following simulation, the driving input z ( k ) is a stationary time-discrete, uniformly distributed signal in the interval (-1,l) with zero mean. The mean squared error (MSE), over an ensemble of 10 simulations and time averaged by a window 40 in length, produced by an LMS SOV-FIR filter, an LMS third order Volterra filter, a LMS SOV-IIR filter based on an approximate gradient and a RPE SOV-IIR filter based on a simplified gradient are shown in Figure 3. The LMS SOV-IIR and third order Volterra filters cannot model higher than second and third signal harmonics, respectively, where the SOV-IIR filter based on the LMS and on the RPE algorithm can.

To use the algorithm with real world data, rotation and vibration signals produced by an aero-engine were used, where the low pressure shaft speed and the vibrations produced by the front transducer were the input and desired signals, respectively. The coherence function was used to show to which extent the outputs generated by the LMS SOV-FIR and LMS SOV-IIR filters matched the desired signal (Figure 4). A possible improvement of the SOVIIR filter over the SOV-FIR filter was not as obvious as in Figure 3. The mean of the each coherence function lets appear the SOV-IIR filter matches the desired signal at 74.6% in the mean sense against 71.1% for the SOV-FIR filter. Nonetheless and due to the highly dynamical behaviour of aero-engines, the models obtained were also dynamic and the high background noise level might explain this result. This investigation of modelling of real world nonlinear systems using the SOV-IIR filter is currently pursued.

1600

011

012

b13

014

0:s Frequency

Ol6

Figure 4: Coherence estimate between the vibration front and (a) the SOV-FIR LMS filter and (b) the SOV-IIR LMS filter.

5. CONCLUSION An adaptive nonlinear filter called SOV-IIR filter has been presented with different gradient updates. The filter improved performance in certain situations, over a second and third order Volterra filters, has been shown. However, a difficulty is to link the presented filter structure to real world systems. Some limitations in the SOV-IIR filter applications can be caused by the instability of the recursive filter part or/and of the recursive update equation, by the nonlinear system characteristics where nonlinearities harmonically related is a necessary condition but might not be a sufficient one, and by the dynamical characteristics of most nonlinear systems. 6. REFERENCES

M. Schetzen. “The Volterra Wiener Theory of the Nonlinear Systems”. New York, Wiley, 1980. V. J. Mathews. “Adaptive Polynomial Filters”. IEEE Signal Processing Magazine, vol. :pp. 10-26, July 1991.

J. J. Shynk. “Adaptive IIR Filtering”. IEEE ASSP Magazine, Vol. ASSP-6(No. 2):pp. 4-21, April 1989.

P. L. Feintuch.

“An Adaptive Recursive LMS Filter”. Proc. of the IEEE, Vol. 64(No. 1l):pp. 1622-1624, November 1976.