Kalman Filtering and smoothing

I I

Input-Output and State space modelling State space modelling examples I I

Electrical circuit examples Radar tracking of object

I

Kalman filtering basics

I

Kalman Filtering as recursive Bayesian estimation

I

Kalman Filtering extensions: Adaptive signal processing

I

Kalman Filtering extensions: Fast Kalman filtering

I

Kalman Filtering for signal deconvolution

A. Mohammad-Djafari, Introduction to Communication, Control and Signal Processing, 2016, Huazhong, Wuhan, China. 1/47

Input-Output model I

I

Linear Systems I

Single Input Single Output (SISO) systems Z y (t) = h(t, τ ) u(τ ) dτ

I

Multi Input Multi Output (MIMO) systems Z y(t) = H(t, τ ) u(τ ) dτ

Linear Time Invariant System I

SISO Convolution Z y (t) = h(t) ∗ u(t) =

I

h(t − τ ) u(τ ) dτ

MIMO Convolution Z y(t) =

H(t − τ ) u(τ ) dτ 

I

 . . . Impulse response h(t) or H(t) =  . hij (t) .  . . .

A. Mohammad-Djafari, Introduction to Communication, Control and Signal Processing, 2016, Huazhong, Wuhan, China. 2/47

State space modelling I

A simple electric system — R —— — — | u(t) ↑ x(t) ↑ C | ————– — —

I

↑ y (t)

State space model u(t) = R i(t) + vc (t) = RC v˙ c (t) + vc (t) = RC x(t) ˙ + x(t)

 −1 1 x(t) ˙ = RC x(t) + RC u(t) y (t) = x(t)

I



RC = 1:

x(t)= ˙ −x(t) + u(t) → LT → y (t)= x(t)



pX (p) + X (p) = U(p) → X (p) = y (t) = e −t ∗ u(t) = h(t) ∗ u(t)

1 p+1 U(p)

A. Mohammad-Djafari, Introduction to Communication, Control and Signal Processing, 2016, Huazhong, Wuhan, China. 3/47

State space modelling I

A more complex electric system example — R —— — —— R —— — — | | u(t) ↑ x2 (t) ↑ C x1 (t) ↑ C | | ————– — —————— — —

I

State space model u(t) = RC x˙ 2 (t) + x2 (t),

I

↑ y (t)

x2 (t) = RC x˙ 1 (t) + x1 (t)

RC = 1:         x ˙ (t) −1 1 x (t) 0  1 1  = + u(t)  x˙ 2 (t) x2 (t) 1  0 −1 1   = x1 (t)  y (t) 0

A. Mohammad-Djafari, Introduction to Communication, Control and Signal Processing, 2016, Huazhong, Wuhan, China. 4/47

State space model: Continuous case Dynamic systems: I

Single Input Single Output (SISO) system:  x(t) ˙ = F x(t) + G u(t) State equation y (t) = C x(t) + D v (t) Observation equation

I

Multiple Input Multiple Output (MIMO) system:  ˙ x(t) = F x(t) + G u(t) State equation y(t) = C x(t) + D v(t) Observation equation

I

MIMO discrete-time system:  x(n + 1) = F x(n) + G u(n) State equation y(n) = C x(n) + D v(n) Observation equation F, G, C and D are the matrices of the system.

A. Mohammad-Djafari, Introduction to Communication, Control and Signal Processing, 2016, Huazhong, Wuhan, China. 5/47

State space modelling examples I

One-dimensional motion: Track-While-Scan (TWS) Radar Xt , Vt , At : Position, Speed, Acceleration  t Xn+1 ' Xn + T Vn Vt = ∂X ∂t −→ t V At = ∂V n+1 ' Vn + T An ∂t       X Xn      x=  xn = V Vn −→ u = A u = A   n     n Y =X +v yn = Xn + vn       = Fxn + Gun 1 T 0 with F = ,G = ,H = 1 0 = Hxn + vn 0 1 T  f n (xn , un ) = F xn + G un hn (xn , vn ) = H xn + vn 



xn+1 yn

A. Mohammad-Djafari, Introduction to Communication, Control and Signal Processing, 2016, Huazhong, Wuhan, China. 6/47

State space modelling examples I

1D motion of heavy targets Track-While-Scan (TWS) Radar with dependent acceleration sequences

I

heavy target: An+1 = ρAn + Wn ,

n = 0, 1, . . .

I

ρ near to 0: low inertia target

I

ρ near to 1: high inertia target.        Xn+1 1 T 0 Xn 0 Vn+1  = 0 1 T  Vn  + 0 Wn An+1 0 0 ρ An 1     Xn Yn = 1 0 0 Vn  + en An

A. Mohammad-Djafari, Introduction to Communication, Control and Signal Processing, 2016, Huazhong, Wuhan, China. 7/47

Kalman Filtering: Recursive Linear Filtering y (n) x(n) −→ Linear System −→ non observable observable I

Objective: Estimate x(n) from the observed values of {y (n), n = 1, . . . , k}.

I

b x (n) is then a function of the data {y (n), n = 1, . . . , k} 4 b x (n | y (1), y (2), . . . , y (k)) = b x (n | k)

I

b x (n + k|n) is called the k-th order prediction of y (n) and the estimation procedure is called prediction.

I

b x (n|n) is the filtered value of y (n) and the estimation procedure is called filtering.

I

b x (n|n + l) is the smoothed value of y (n) and the estimation procedure is called smoothing.

A. Mohammad-Djafari, Introduction to Communication, Control and Signal Processing, 2016, Huazhong, Wuhan, China. 8/47

State space and input-Output modelling Transmission of a AR1 signal through a perturbed channel

u(t) −→

AR1

−→ x(t)

x(t) Z

x(t) = a x(t − 1) + u(t)

h(t)

v(t) ? - +m- y(t)

t

h(τ ) x(t − τ ) dτ + v (t)

y (t) = 0

X (p) = a p X (p) + U(p) 1 X (p) = a−p U(p) 1 X (ω) = 1−a(jω) U(ω) x(n) = a x(n − 1) + u(n)

Y (p) = H(p) X (p) + V (p) Y (p) = H(p) X (p) + V (p) Y (ω) = H(ω) X (ω) + V (ω) p X y (n) = hk x(n − k) + v (n) k=0

    xn+1 = apxn + un xn+1 = F xn + Gun X →?→ y = h x + v yn = H xn + vn  n n−k n  n k=0

A. Mohammad-Djafari, Introduction to Communication, Control and Signal Processing, 2016, Huazhong, Wuhan, China. 9/47

State space modelling example u(n) −→

AR1

x(n) - [h0 , h1 ] - +m- y (n)

−→ x(n)

x(n) = a x(n − 1) + u(n)

v (n) ?

y (n) = h0 x(n) + h1 x(n − 1) + v (n)

xn = [x(n), x(n − 1)]0 ,

xn+1 = [x(n + 1), x(n)]0

       x(n + 1) a 0 x(n) 1 = + u(n) x(n) 1 0 x(n − 1) 0     x(n) y (n) = h0 h1 + v (n) x(n − 1)     a 0 1 F= , G= , H = [h0 , h1 ] −→ 1 0 0  xn+1 = F xn + Gun yn = H xn + vn A. Mohammad-Djafari, Introduction to Communication, Control and Signal Processing, 2016, Huazhong, Wuhan, China. 10/47

State space and input-Output modelling A FIR channel y (n) =

Pp

k=0 hk x(n

− k) + v (n)

xn = [x(n), x(n − 1), · · · , x(n − p)]0 xn+1 = [x(n + 1), x(n), · · · , x(n − p + 1)]0     x(n) 1 a 0 . . . . . . 0 x(n + 1)     x(n − 1)  0    1 0 . . . . . . 0 x(n)        x(n − 2)  0   x(n − 1)   1 0 . . . 0     ..     ..  . . . .    =  . . .. .. .. ..  .. + .  u(n)           . .  .. .. .. .. ..   ..     ..   . . . .       .  x(n − p + 1) 0  . . . .. .. .. x(n − p + 1) 1 0 x(n − p) 0 





G = [1, 0, · · · , 0]0 −→ H = [h0 , h1 , · · · , hp ]0



xn+1 = F xn + Gun yn = H xn + vn

A. Mohammad-Djafari, Introduction to Communication, Control and Signal Processing, 2016, Huazhong, Wuhan, China. 11/47

State space and input-Output modelling I

A FIR channel y (n) = 

Pp

k=0 hk x(n

xn+1 = F xn + Gun yn = H xn + vn

xn = [xn , xn−1 , · · · , xn−p+1 ]0 xn+1 = [xn+1 , xn , · · · , xn−p+2 ]0 G = [1, 0, · · · , 0]0 H = [h0 , h1 , · · · , hn ]0 I

− k) + v (n)  a 0 1 0  0 1   F =  ... ...   .. .. . .  .. .. . .

 ... ... 0 . . . . . . 0  0 . . . 0   .. ..  . .   .. ..  . .  .. . 1 0

A perfect but noisy channel h(t) = h0 δ(t) −→ p = 1  xn+1 = a xn + un yn = h0 xn + vn

A. Mohammad-Djafari, Introduction to Communication, Control and Signal Processing, 2016, Huazhong, Wuhan, China. 12/47

State space modelling: Examples 

xn+1 = a xn + un yn = h xn + vn

  un ∼ N (0, q) vn ∼ N (0, r )  x0 ∼ N (m0 , p0 ) I

Try to obtain b xn+1|n as a function of b xn|n recursively  b xn+1|n = a b xn|n    b xn|n = b xn|n−1 + kn (yn − h b xn|n−1 ) pn|n−1 p h 1 n|n−1   = =  kn 2 h pn|n−1 + r h pn|n−1 + r /h2   pn+1|n = a2 pn|n + q 1 pn|n−1 =  pn|n h hr2 pn|n−1+1

A. Mohammad-Djafari, Introduction to Communication, Control and Signal Processing, 2016, Huazhong, Wuhan, China. 13/47

State space modelling of the systems I

Time Varying systems:  xn+1 = f n (xn , un ) state equation yn = hn (xn , vn ) observation equation

I

Time Variying but Linear system  xn+1 = Fn xn + Gn un yn = Hn xn + vn

I

Time Invariant and Linear system  xn+1 = F xn + G un yn = H xn + vn

A. Mohammad-Djafari, Introduction to Communication, Control and Signal Processing, 2016, Huazhong, Wuhan, China. 14/47

State space modelling: General case 

I I I I I

I

xk+1 = Fk xk + Gk uk yk = Hk xk + vk

state equation, observation equation

xk N-dimensional state vector yk P-dimensional observations vector vk P-dimensional observations error vector uk M-dimensional state representation error Fk , Gk and Hk with respective dimensions of (N, N), (N, M) and (P, N) are the state transition, the state input and the observation matrices and are assumed to be known. The noise sequences {uk } and {vk } and the initial state x0 are assumed to be centered, white and jointly Gaussian.      Rk 0 0  vk    E  x0  vtl , xt0 , utl =  0 P0 0  δkl   uk 0 0 Qk

A. Mohammad-Djafari, Introduction to Communication, Control and Signal Processing, 2016, Huazhong, Wuhan, China. 15/47

Kalman Filtering: Prediction, Filtering and Smoothing Objective: Find the best estimate b xk|l of xk from the observations y1 , y2 , . . . , yl . 4 b xk (y1 , y2 , . . . , yl ) = b x(k | l) I

If

k >l

prediction. For example l = n, k = n + 1: 4 b xn+1 (y1 , y2 , . . . , yn ) = b x(n + 1 | n)

I

If

k =l

filtering. For example l = n, k = n: 4 b xn (y1 , y2 , . . . , yn ) = b x(n | n)

I

If

k