.

Detecting and estimating the shape of a periodic component in short duration signals Ali Mohammad-Djafari Laboratoire des Signaux et Syst`emes (L2S) UMR8506 CNRS-CentraleSup´elec-UNIV PARIS SUD SUPELEC, 91192 Gif-sur-Yvette, France http://lss.centralesupelec.fr Email: [email protected] http://djafari.free.fr http://publicationslist.org/djafari MaxEnt 2016 workshop, July 10-15, 2016, Gent, Belgium.

A. Mohammad-Djafari, Approximate Bayesian Computation for Big Data, Tutorial at MaxEnt 2016, July 10-15, Gent, Belgium. 1/16

Contents

1. Description of the problem 2. Putting the things in equations) 3. Classical Regularization approach 4. Bayesian approach 5. Simulation results 6. Conclusions

A. Mohammad-Djafari, Approximate Bayesian Computation for Big Data, Tutorial at MaxEnt 2016, July 10-15, Gent, Belgium. 2/16

Description of the problem

I

Detecting a periodic component in a short duration signal, estimating its period and its shape. Low noise example

I

High noise example

I

A. Mohammad-Djafari, Approximate Bayesian Computation for Big Data, Tutorial at MaxEnt 2016, July 10-15, Gent, Belgium. 3/16

Description of the problem

I

Detecting a periodic component in a short duration signal, estimating its period and its shape. Low noise example

I

High noise example

I

A. Mohammad-Djafari, Approximate Bayesian Computation for Big Data, Tutorial at MaxEnt 2016, July 10-15, Gent, Belgium. 4/16

Description of the problem I

g = [g 1 , · · · , g N , g N+1 , · · · , · · · , ...g KN+1 , · · · , g M ]0 = [f 1 , · · · , f N , f 1 , · · · , f N , · · · , f 1 , · · · · · · , f r ]0

I

where M = KN + r where K is the number of complete repetition of the periodic shape and r is the rest. This relation can be written as a linear relation: g = HN f where H has the following structure HN = [IN |IN |...|IN |I(:, 1 : r )] where IN is the unitary matrix of size N × N and I(:, 1 : r ) is its first r columns. g = HN f +  the vector  represents those errors.

A. Mohammad-Djafari, Approximate Bayesian Computation for Big Data, Tutorial at MaxEnt 2016, July 10-15, Gent, Belgium. 5/16

Description of the problem I

Also, we require some regularity in the shape of f. This regularity can be modelled as f = Df + ξ → (I − D)f = ξ → Cf = ξ where C can be of the form  1 −1 0 · · ·  .  0 1 −1 . .  . .. .. .. CN =   .. . . .   0 ··· 0 1 −1 0 · · · 0

I

 0 ..  .    0   −1  1

A criterion which measures the regularity can be kCfk2

A. Mohammad-Djafari, Approximate Bayesian Computation for Big Data, Tutorial at MaxEnt 2016, July 10-15, Gent, Belgium. 6/16

Deterministic regularization method g = HN f + ,

Cf = ξ

With these two equations, we have at least two possibilities: I Deterministic regularization: bf = arg min {J(f)} with J(f) = kg = HN fk2 + λkCN fk2 2 2 f The solution is given by: bf = [H0 HN + λC0 CN ]−1 H0 g N

I

N

N

The above criterion was given for a given value of N. We can now try to define a criterion which depends explicitly on N : J(N, f) = kg = HN fk22 + λkCN fk22 and try to optimize it to find both the seeked period N and the shape f: b bf) = arg min {J(N, f)} (N, (N,f )

A. Mohammad-Djafari, Approximate Bayesian Computation for Big Data, Tutorial at MaxEnt 2016, July 10-15, Gent, Belgium. 7/16

Bayesian approach g = HN f + , I

Cf = ξ

Likelihood p(g|f) = N (g|HN f, v I),

I

Prior p(f) = N (f|0, vξ (C0N CN )−1 )

I

Poserior b p(f|g) ∝ p(g|f)p(f) = N (f|bf, Σ) with bf = arg max {p(f|g)} = arg min {J(f)} f f which is equivalent to the quadratic regularization as before bf = [H0 HN + λC0 CN ]−1 H0 g N N N with λ = v /vξ .

A. Mohammad-Djafari, Approximate Bayesian Computation for Big Data, Tutorial at MaxEnt 2016, July 10-15, Gent, Belgium. 8/16

Bayesian approach I

Poserior b p(f|g) ∝ p(g|f)p(f) = N (f|bf, Σ) with bf = arg max {p(f|g)} = arg min {J(f)} f f and b = [H0 HN + λC0 CN ]−1 Σ N N which can be used to put error bars on the solution.

I

Joint posterior p(N, f|g) and and try to optimize it to find the seeked solution b bf) = arg max {p(N, f|g)} (N, (N,f )

I

We can do better.

A. Mohammad-Djafari, Approximate Bayesian Computation for Big Data, Tutorial at MaxEnt 2016, July 10-15, Gent, Belgium. 9/16

Bayesian approach with more appropriate priors I

Forward and prior model equations: g = HN f + ,

I

I

Cf = ξ

i and ξj are Gaussian but with unknown variances that we want to estimate to. p(i |vi ) = N (i |0, vi ),

p(vi |α0 , β0 ) = IG(vi |α0 , β0 )

p(ξi |v ξ i ) = N (ξi |0, v ξ i ),

p(v ξ i |αξ0 , βξ0 ) = IG(vi |αξ0 , βξ0 )

This can also be interpreted as a wish to model them by a heavier tailed probability laws such as Student-t: Z ∞ St(i |αi , βi ) = N (i |, 0, vi ) IG(vi |α0 , β0 ) dvi 0

Z St(ξ j |α, β) = 0

∞

N (ξ j |, 0, v ξ j ) IG(v ξ j |αξ0 , βξ0 ) dv ξ j

A. Mohammad-Djafari, Approximate Bayesian Computation for Big Data, Tutorial at MaxEnt 2016, July 10-15, Gent, Belgium. 10/16

Non stationary noise and sparsity enforcing model – Forward model: g = Hf+, i ∼ N (i |0, vi ) →  ∼ N (|0, V ), V = diag [v1 , · · · , vM ]) – Prior model:   CN f = ξ, ξj ∼ N (ξi |0, ξ j ) → ξ ∼ N (ξ|0, Vξ ), Vξ = diag v ξ 1 , · · · , v ξ N ) 

p(g|f, v ) = N (g|Hf, V ), V = diag [v ] p(f|vf ) = N (f|0, Vξ CC0 ), Vξ = diag [vξ ] Q  ?  ?  p(v ) = Qi IG(vi |α0 , β0 ) vξ v   p(vξ ) = i IG(v ξ j |αξ0 , βξ0 )

αξ0 , βξ0 α0 , β0

?  ? 

f



p(f, v , vξ |g) ∝ p(g|f, v ) p(f|vξ ) p(v ) p(vξ )

 

H

? 

g



Objective: Infer (f, v , vξ ) – VBA: Approximate p(f, v , vξ |g) by q1 (f) q2 (v ) q3 (vξ )

A. Mohammad-Djafari, Approximate Bayesian Computation for Big Data, Tutorial at MaxEnt 2016, July 10-15, Gent, Belgium. 11/16

Results N=96, period=29, I

By using Fourier Transform technic, no way to find the right value of period.

It is also difficult to estimate the shape of this repeating scheme. Low noise case: I

High noise case:

A. Mohammad-Djafari, Approximate Bayesian Computation for Big Data, Tutorial at MaxEnt 2016, July 10-15, Gent, Belgium. 12/16

Results N=96, period=29, Low noise case:

High noise case:

A. Mohammad-Djafari, Approximate Bayesian Computation for Big Data, Tutorial at MaxEnt 2016, July 10-15, Gent, Belgium. 13/16

Results

A. Mohammad-Djafari, Approximate Bayesian Computation for Big Data, Tutorial at MaxEnt 2016, July 10-15, Gent, Belgium. 14/16

Results

A. Mohammad-Djafari, Approximate Bayesian Computation for Big Data, Tutorial at MaxEnt 2016, July 10-15, Gent, Belgium. 15/16

Conclusions

I

the first step in any inference is to write down the relation between what you observe (data g) and the unknowns f.

I

The second step is to model and assign priors to account for all uncertainties

I

The third step is to use the Bayes rule to find the expression of the joint probability law of all the unknowns given the data and all the hyper parameters.

I

Do the Bayesian computation, show the results

I

Interpret your results

I

Enjoy

A. Mohammad-Djafari, Approximate Bayesian Computation for Big Data, Tutorial at MaxEnt 2016, July 10-15, Gent, Belgium. 16/16