Bayesian methods for Inverse problems of imaging systems Ali Mohammad-Djafari Laboratoire des Signaux et Syst`emes, UMR8506 CNRS-SUPELEC-UNIV PARIS SUD 11 SUPELEC, 91192 Gif-sur-Yvette, France http://lss.supelec.free.fr Email: [email protected] http://djafari.free.fr Seminar at EECE Dept of the Tehran University, December 8, 2014 A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 1/68

Content 1. Inverse problems examples in imaging science 2. Classical methods: Generalized inversion and Regularization 3. Bayesian approach for inverse problems 4. Prior modeling - Gaussian, Generalized Gaussian (GG), Gamma, Beta, - Gauss-Markov, GG-Marvov - Sparsity enforcing priors (Bernouilli-Gaussian, B-Gamma, Cauchy, Student-t, Laplace) 5. Full Bayesian approach (Estimation of hyperparameters) 6. Hierarchical prior models 7. Bayesian Computation and Algorithms for Hierarchical models 8. Gauss-Markov-Potts family of priors 9. Applications and case studies A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 2/68

Seeing outside of a body: Making an image with a camera, a microscope or a telescope ◮

f (x, y) real scene

◮

g(x, y) observed image

◮

Forward model: Convolution ZZ g(x, y) = f (x′ , y ′ ) h(x − x′ , y − y ′ ) dx′ dy ′ + ǫ(x, y) h(x, y): Point Spread Function (PSF) of the imaging system

◮

Inverse problem: Image restoration Given the forward model H (PSF h(x, y))) and a set of data g(xi , yi ), i = 1, · · · , M find f (x, y)

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 3/68

Making an image with an unfocused camera Forward model: 2D Convolution ZZ g(x, y) = f (x′ , y ′ ) h(x − x′ , y − y ′ ) dx′ dy ′ + ǫ(x, y) ǫ(x, y)

f (x, y) ✲ h(x, y)

❄ ✎☞ ✲ + ✲g(x, y) ✍✌

Inversion: Image Deconvolution or Restoration ? ⇐=

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 4/68

Making an image of the interior of a body r Incident wave ✲

r

r

r

r

r r ❅ r ✁ r object r ❍ r ❍ r r r r r r Active Imaging

Measurement Incident wave ❅ ✁ ✲ object ❍ ❍ Transmission

r

r

r r ✻ ❨ ❍ ❅ ✒ r ✁❍ object ✲ r ❍ ❍ ✠ ❅ ❘ r r r r r Passive Imaging

r r r r

Measurement Incident wave ✲

❅ ✁ object ❍ ❍

Reflection

Forward problem: Knowing the object predict the data Inverse problem: From measured data find the object A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 5/68

Seeing inside of a body: Computed Tomography ◮

f (x, y) a section of a real 3D body f (x, y, z)

◮

gφ (r) a line of observed radiographe gφ (r, z)

◮

Forward model: Line integrals or Radon Transform Z gφ (r) = f (x, y) dl + ǫφ (r) L

ZZ r,φ f (x, y) δ(r − x cos φ − y sin φ) dx dy + ǫφ (r) =

◮

Inverse problem: Image reconstruction Given the forward model H (Radon Transform) and a set of data gφi (r), i = 1, · · · , M find f (x, y)

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 6/68

Computed Tomography: Radon Transform

Forward: Inverse:

f (x, y) f (x, y)

−→ ←−

g(r, φ) g(r, φ)

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 7/68

Microwave or ultrasound imaging Measurs: diffracted wave by the object g(ri ) Unknown quantity: f (r) = k02 (n2 (r) − 1) Intermediate quantity : φ(r)

y

Object

ZZ

r'

Gm (ri , r ′ )φ(r ′ ) f (r ′ ) dr ′ , ri ∈ S D ZZ Go (r, r ′ )φ(r ′ ) f (r ′ ) dr ′ , r ∈ D φ(r) = φ0 (r) + g(ri ) =

Measurement

plane

Incident

plane Wave

D

Born approximation (φ(r ′ ) ≃ φ0 (r ′ )) ): ZZ Gm (ri , r ′ )φ0 (r ′ ) f (r ′ ) dr ′ , ri ∈ S g(ri ) = D

r x

z

r

r r ✦ ✦ ▲ r ✱ ❛❛ r ✱ ❊ r ✲ ❊ ❡ φ0r (φ, f )✪ r ✪ r r r r g r

Discretization : g = H(f ) g = Gm F φ −→ with F = diag(f ) φ= φ0 + Go F φ H(f ) = Gm F (I − Go F )−1 φ0

r

r

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 8/68

Fourier Synthesis in different imaging systems G(ωx , ωy ) = v

ZZ

f (x, y) exp [−j (ωx x + ωy y)] dx dy v

u

X ray Tomography

v

u

Diffraction

v

u

Eddy current

u

SAR & Radar

Forward problem: Given f (x, y) compute G(ωx , ωy ) Inverse problem : Given G(ωx , ωy ) on those algebraic lines, cercles or curves, estimate f (x, y) A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 9/68

Invers Problems: other examples and applications ◮

X ray, Gamma ray Computed Tomography (CT)

◮

Microwave and ultrasound tomography

◮

Positron emission tomography (PET)

◮

Magnetic resonance imaging (MRI)

◮

Photoacoustic imaging

◮

Radio astronomy

◮

Geophysical imaging

◮

Non Destructive Evaluation (NDE) and Testing (NDT) techniques in industry

◮

Hyperspectral imaging

◮

Earth observation methods (Radar, SAR, IR, ...)

◮

Survey and tracking in security systems

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 10/68

3. General formulation of inverse problems and classical methods ◮

General non linear inverse problems: g(s) = [Hf (r)](s) + ǫ(s),

◮

Linear models: g(s) =

◮

◮

◮

s∈S

f (r) h(r, s) dr + ǫ(s)

If h(r, s) = h(r − s) −→ Convolution. Discrete data:Z g(si ) =

◮

Z

r ∈ R,

h(si , r) f (r) dr + ǫ(si ),

i = 1, · · · , m

Inversion: Given the forward model H and the data g = {g(si ), i = 1, · · · , m)} estimate f (r) Well-posed and Ill-posed problems (Hadamard): existance, uniqueness and stability Need for prior information

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 11/68

Inverse problems: Z Discretization g(si ) =

◮

h(si , r) f (r) dr + ǫ(si ),

i = 1, · · · , M

f (r) is assumed to be well approximated by N X f (r) ≃ fj bj (r) j=1

with {bj (r)} a basis or any other set of known functions Z N X g(si ) = gi ≃ fj h(si , r) bj (r) dr, i = 1, · · · , M j=1

g = Hf + ǫ with Hij = ◮ ◮

Z

h(si , r) bj (r) dr

H is huge dimensional b LS solution P : f = arg 2minf {Q(f )} with Q(f ) = i |gi − [Hf ]i | = kg − Hf k2 does not give satisfactory result.

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 12/68

Inverse problems: Deterministic methods Data matching ◮

Observation model gi = hi (f ) + ǫi , i = 1, . . . , M −→ g = H(f ) + ǫ

◮

Misatch between data and output of the model ∆(g, H(f )) b = arg min {∆(g, H(f ))} f f

◮

Examples:

– LS

∆(g, H(f )) = kg − H(f )k2 =

X

|gi − hi (f )|2

i

– Lp – KL

p

∆(g, H(f )) = kg − H(f )k = ∆(g, H(f )) =

X i

◮

X

|gi − hi (f )|p ,

1tsh

8

6

4

4

2

2

0

0

−2

−2

−4

−4

−6

−6

−8

−10

0

−8

0

20

40

60

80

100

120

140

160

Quadratic Reg. (Gaussian)

180

200

−10

0

20

40

60

80

100

120

140

160

180

200

L1 Reg. (Laplace)

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 23/68

Main advantages of the Bayesian approach ◮

MAP = Regularization

◮

Posterior mean ? Marginal MAP ?

◮

More information in the posterior law than only its mode or its mean

◮

Meaning and tools for estimating hyper parameters

◮

Meaning and tools for model selection

◮

More specific and specialized priors, particularly through the hidden variables More computational tools:

◮

◮

◮ ◮

◮

Expectation-Maximization for computing the maximum likelihood parameters MCMC for posterior exploration Variational Bayes for analytical computation of the posterior marginals ...

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 24/68

Two main steps in the Bayesian approach ◮

Prior modeling ◮

◮

◮

◮

Separable: Gaussian, Gamma, Sparsity enforcing: Generalized Gaussian, mixture of Gaussians, mixture of Gammas, ... Markovian: Gauss-Markov, GGM, ... Markovian with hidden variables (contours, region labels)

Choice of the estimator and computational aspects ◮ ◮ ◮ ◮

◮

MAP, Posterior mean, Marginal MAP MAP needs optimization algorithms Posterior mean needs integration methods Marginal MAP and Hyperparameter estimation need integration and optimization Approximations: ◮ ◮ ◮

Gaussian approximation (Laplace) Numerical exploration MCMC Variational Bayes (Separable approximation)

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 25/68

5. Prior modeling of signals

Gaussian p(fj ) ∝ exp −α|fj |2

Generalized Gaussian p(fj ) ∝ exp [−α|fj |p ] , 1 ≤ p ≤ 2

Gamma p(fj ) ∝ fjα exp [−βfj ]

Beta p(fj ) ∝ fjα (1 − fj )β

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 26/68

Sparsity enforcing prior models ◮

Sparse signals: Direct sparsity 1

3

0.8 2.5 0.6

0.4 2 0.2

0

1.5

−0.2 1 −0.4

−0.6 0.5 −0.8

−1

◮

0

20

40

60

80

100

120

140

160

180

200

0

0

20

40

60

80

100

120

140

160

180

200

Sparse signals: Sparsity in a Transform domaine

1

6

1

0.8

0.8 4

0.6

0.6

0.4

0.4 2

0.2

0.2

0

0

0

−0.2

−0.2 −2

−0.4

−0.4

−0.6

−0.6 −4

−0.8

−1

−0.8

0

20

40

60

80

100

120

140

160

180

200

−6

0

1

200

0.8

180

0.6

160

0.4

140

0.2

120

0

100

−0.2

80

−0.4

60

−0.6

40

−0.8

20

20

40

60

80

100

120

140

160

180

−1

200

0

20

40

60

80

100

120

140

160

180

200

180

200

1

2

3

4

5

6

7

8 −1

0

20

40

60

80

100

120

140

160

180

200

0

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

20

40

60

80

100

120

140

160

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 27/68

Sparsity enforcing prior models ◮

Simple heavy tailed models: ◮ ◮ ◮

◮ ◮

◮

Generalized Gaussian (particular case: Double Exponential) Student-t (particular case: Cauchy) Elastic net Symmetric Weibull, Symmetric Rayleigh Generalized hyperbolic

Hierarchical mixture models: ◮ ◮

◮ ◮ ◮ ◮

Mixture of Gaussians Bernoulli-Gaussian Mixture of Gammas Bernoulli-Gamma Mixture of Dirichlet Bernoulli-Multinomial

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 28/68

Simple heavy tailed models • Generalized Gaussian ((particular case: Double Exponential) Y X p(f |γ, β) = GG(f j |γ, β) ∝ exp −γ |f j |β j

j

β = 1 Double exponential or Laplace. 0 < β ≤ 1 are of great interest for sparsity enforcing.

• Student-t ((particular case: Cauchy models) X Y ν+1 log 1 + f 2j /ν p(f |ν) = St(f j |ν) ∝ exp − 2 j

j

Cauchy model is obtained when ν = 1.

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 29/68

Mixture models • Mixture of two Gaussians (MoG2) model Y λN (f j |0, v1 ) + (1 − λ)N (f j |0, v0 ) p(f |λ, v1 , v0 ) = j

• Bernoulli-Gaussian (BG) model Y Y p(f |λ, v) = p(f j ) = λN (f j |0, v) + (1 − λ)δ(f j ) j

j

• Mixture of Gammas Y λG(f j |α1 , β1 ) + (1 − λ)G(f j |α2 , β2 ) p(f |λ, v1 , v0 ) = j

• Bernoulli-Gamma model Y p(f |λ, α, β) = λG(f j |α, β) + (1 − λ)δ(f j ) j

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 30/68

6. Full Bayesian approach ◮ ◮ ◮ ◮ ◮

M: g = Hf + ǫ Forward & errors model: −→ p(g|f , θ 1 ; M) Prior models −→ p(f |θ 2 ; M) Hyperparameters θ = (θ 1 , θ 2 ) −→ p(θ|M) p(f |θ;M) p(θ|M) Bayes: −→ p(f , θ|g; M) = p(g|f ,θ;M)p(g|M) b , θ) b = arg max {p(f , θ|g; M)} Joint MAP: (f (f ,θ)

◮

Marginalization 1:

p(f |g; M) = ◮

Marginalization 2: p(θ|g; M) =

◮

ZZ

ZZ

b p(f , θ|g; M) dθ −→ f b p(f , θ|g; M) df −→ θ

Approximate p(f , θ|g; M) by a separable one q(f , θ) = q1 (f )q2 (θ) and then use them separately b b and θ. to find f

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 31/68

Summary of Bayesian estimation 1 ◮

Simple Bayesian Model and Estimation θ2

θ1

❄

❄

p(f |θ 2 ) Prior ◮

⋄ p(g|f , θ 1 ) −→ Likelihood

p(f |g, θ) Posterior

b −→ f

Full Bayesian Model and Hyperparameter Estimation ↓ α, β

Hyperpraram. model p(θ|α, β) p(θ 2 ) ❄

p(f |θ 2 ) Prior

p(θ 1 )

❄ b −→ f ⋄ p(g|f , θ 1 ) −→p(f, θ|g, α, β) b −→ θ Likelihood Joint Posterior

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 32/68

Summary of Bayesian estimation 2 ◮

Marginalization 1 p(f , θ|g) −→

p(θ|g)

b −→ f

Joint Posterior Marginalize over θ Marginalization 2

◮

p(f , θ|g) −→

p(θ|g)

b −→ p(f |θ, b g) −→ f b −→ θ

Joint Posterior Marginalize over f ◮

Variational Bayesian Approximation

p(f , θ|g) −→

Variational Bayesian Approximation

b −→ q1 (f ) −→ f b −→ q2 (θ) −→ θ

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 33/68

Variational Bayesian Approximation ◮

Full Bayesian: p(f , θ|g) ∝ p(g|f , θ 1 ) p(f |θ 2 ) p(θ)

◮

Approximate p(f , θ|g) by q(f , θ|g) = q1 (f |g) q2 (θ|g) and then continue computations.

◮

Criterion KL(q(f , θ|g) : p(f , θ|g)) Z Z Z Z q1 q2 KL(q : p) = q ln q/p = q1 q2 ln p Iterative algorithm q1 −→ q2 −→ q1 −→ q2 , · · ·

◮ ◮

h i qb1 (f ) ∝ exp hln p(g, f , θ; M)i qb2 (θ) i h qb2 (θ) ∝ exp hln p(g, f , θ; M)i qb1 (f ) p(f , θ|g) −→

Variational Bayesian Approximation

b −→ q1 (f ) −→ f b −→ q2 (θ) −→ θ

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 34/68

7. Hierarchical models and hidden variables ◮

All the mixture models and some of simple models can be modeled via hidden variables z. K X p(f |z = k) = pk (f P ), p(f ) = αk pk (f ) −→ P (z = k) = αk , k αk = 1 k=1

◮

◮

Example 2: Student-t model ν+1 St(f |ν) ∝ exp − log 1 + f 2 /ν 2 Infinite mixture Z ∞ N (f |, 0, 1/z) G(z|α, β) dz, St(f |ν) ∝=

with α = β = ν/2

0

h i Q Q 1P 2 p(f |z) = p(f |z ) = N (f |0, 1/z ) ∝ exp − z f j j j j j j j j j 2 Q Q (α−1) p(z|α, β) = j G(z hPj |α, β) ∝ j zj iexp [−βzj ] ∝ exp j (α − 1) ln zj − βzj

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 35/68

Summary of Bayesian estimation 3 • Full Bayesian Hierarchical Model with Hyperparameter Estimation ↓ α, β, γ Hyper prior model p(θ|α, β, γ) p(θ 3 )

p(θ 2 )

p(θ 1 )

❄

❄

❄

⋄ p(f |z, θ 2 ) ⋄ p(g|f , θ 1 ) −→

p(z|θ 3 )

Hidden variable

Prior

Likelihood

p(f , z, θ|g) Joint Posterior

• Full Bayesian Hierarchical Model and Variational Approximation

b −→ f b −→ z b −→ θ

↓ α, β, γ Hyper prior model p(θ|α, β, γ) p(θ 3 ) ❄ p(z|θ3 )

⋄

Hidden variable

p(θ2 ) ❄ p(f |z, θ2 ) Prior

p(θ 1 ) VBA b ❄ q1 (f ) −→ f b ⋄ p(g|f , θ1 ) −→ p(f , z, θ|g) −→ q (z) −→ z 2 b −→ θ q3 (θ) Likelihood Joint Posterior

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 36/68

8. Bayesian Computation and Algorithms for Hierarchical models ◮

Often, the expression of p(f , z, θ|g) is complex.

◮

Its optimization (for Joint MAP) or its marginalization or integration (for Marginal MAP or PM) is not easy

◮

Two main techniques: MCMC and Variational Bayesian Approximation (VBA)

◮

MCMC: Needs the expressions of the conditionals p(f |z, θ, g), p(z|f , θ, g), and p(θ|f , z, g)

◮

VBA: Approximate p(f , z, θ|g) by a separable one q(f , z, θ|g) = q1 (f ) q2 (z) q3 (θ) and do any computations with these separable ones.

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 37/68

Which images I am looking for? 50 100 150 200 250 300 350 400 450 50

100

150

200

250

300

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 38/68

Which image I am looking for?

Gauss-Markov

Generalized GM

Piecewize Gaussian

Mixture of GM

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 39/68

9. Gauss-Markov-Potts prior models for images

f (r)

c(r) = 1 − δ(z(r) − z(r ′ ))

z(r)

p(f (r)|z(r) = k, mk , vk ) = N (mk , vk ) X p(f (r)) = P (z(r) = k) N (mk , vk ) Mixture of Gaussians k

◮ ◮

Separable iid hidden variables: Markovian hidden variables:

Q p(z) = r p(z(r)) p(z) Potts-Markov: X

δ(z(r) − z(r ′ )) p(z(r)|z(r ′ ), r ′ ∈ V(r)) ∝ exp γ r ′ ∈V(r) X X δ(z(r) − z(r ′ )) p(z) ∝ exp γ r∈R r ′ ∈V(r)

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 40/68

Four different cases To each pixel of the image is associated 2 variables f (r) and z(r) ◮

f |z Gaussian iid, z iid : Mixture of Gaussians

◮

f |z Gauss-Markov, z iid : Mixture of Gauss-Markov

◮

f |z Gaussian iid, z Potts-Markov : Mixture of Independent Gaussians (MIG with Hidden Potts)

◮

f |z Markov, z Potts-Markov : Mixture of Gauss-Markov (MGM with hidden Potts)

f (r)

z(r)

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 41/68

Application of CT in NDT Reconstruction from only 2 projections

g1 (x) = ◮

◮

Z

f (x, y) dy,

g2 (y) =

Z

f (x, y) dx

Given the marginals g1 (x) and g2 (y) find the joint distribution f (x, y). Infinite number of solutions : f (x, y) = g1 (x) g2 (y) Ω(x, y) Ω(x, y) is a Copula: Z Z Ω(x, y) dx = 1 and Ω(x, y) dy = 1

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 42/68

Application in CT

20

40

60

80

100

120 20

g|f

f |z

g = Hf + ǫ iid Gaussian 2 g|f ∼ N (Hf , σǫ I) or Gaussian Gauss-Markov

40

60

80

100

120

z

c

iid or Potts

c(r) ∈ {0, 1} 1 − δ(z(r) − z(r ′ )) binary

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 43/68

Proposed algorithm p(f , z, θ|g) ∝ p(g|f , z, θ) p(f |z, θ) p(θ) General scheme: b g) −→ zb ∼ p(z|fb, θ, b g) −→ θ b ∼ (θ|fb, zb, g) fb ∼ p(f |b z , θ,

Iterative algorithme: ◮

◮

b g) ∝ p(g|f , θ) p(f |b b Estimate f using p(f |b z , θ, z , θ) Needs optimisation of a quadratic criterion. b g) ∝ p(g|fb, zb, θ) b p(z) Estimate z using p(z|fb, θ,

Needs sampling of a Potts Markov field. ◮

Estimate θ using p(θ|fb, zb, g) ∝ p(g|fb, σǫ2 I) p(fb|b z , (mk , vk )) p(θ) Conjugate priors −→ analytical expressions.

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 44/68

Results

Original

Backprojection

Gauss-Markov+pos

Filtered BP

GM+Line process

LS

GM+Label process

20

20

20

40

40

40

60

60

60

80

80

80

100

100

100

120

120 20

40

60

80

100

120

c

120 20

40

60

80

100

120

z

20

40

60

80

100

120

c

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 45/68

Application in Microwave imaging (Linearized Spectral method: Fourier Synthesis) Z g(ω) = f (r) exp [−j(ω.r)] dr + ǫ(ω) g(u, v) =

ZZ

f (x, y) exp [−j(ux + vy)] dx dy + ǫ(u, v) g = Hf + ǫ

20

20

20

20

40

40

40

40

60

60

60

60

80

80

80

80

100

100

100

100

120

120 20

40

60

80

f (x, y)

100

120

120 20

40

60

80

g(u, v)

100

120

120 20

40

60

80

fb IFT

100

120

20

40

60

80

100

120

fb Proposed method

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 46/68

Color (Multi-spectral) image deconvolution ǫi (x, y)

fi (x, y)

✲

h(x, y)

Observation model :

❄ ✗✔ ✲ + ✲ gi (x, y) ✖✕

g i = Hfi + ǫi ,

i = 1, 2, 3

? ⇐=

Same segmentation z for the three components fi , i = 1, 2, 3 A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 47/68

Images fusion and joint segmentation (with O. F´eron) gi (r) = fi (r) + ǫi (r) p(fi (r)|z(r) = k) = N (mik , σ 2 ) ik Q p(f |z) p(f |z) = i hi i P ′ )) p(z) ∝ exp γ P δ(z(r) − z(r ′ r∈R r ∈V(r)

g1

g2

−→

b f 1 b2 f

b z

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 48/68

Data fusion in medical imaging (with O. F´eron) gi (r) = fi (r) + ǫi (r) p(fi (r)|z(r) = k) = N (mik , σ 2 ) ik Q p(f |z) = p(f |z) i hi i P ′ )) p(z) ∝ exp γ P δ(z(r) − z(r ′ r∈R r ∈V(r)

g1

g2

−→

b f 1

b f 2

b z

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 49/68

gi (r) = [DMBfi (r) + ǫi (r) 2 p(fi (r)|z(r) = k) = N (mik , σ ) (with F. Humblot) ik Q p(f |z) = hi p(fi |z) i P p(z) ∝ exp γ P δ(z(r) − z(r ′ )) ′

Super-Resolution

r∈R

r ∈V(r)

? =⇒

Low Resolution images

High Resolution image

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 50/68

Joint segmentation of hyper-spectral images (with N. Bali & A. Mohammadpour) gi (r) = fi (r) + ǫi (r) 2 p(fi (r)|z(r) Q = k) = N (mik , σi k ), k = 1, · · · , K p(f |z) = hi p(fi |z) i P P ′ )) p(z) ∝ exp γ δ(z(r) − z(r ′ r∈R r ∈V(r) mik follow a Markovian model along the index i

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 51/68

Segmentation of a video sequence of images (with P. Brault) gi (r) = fi (r) + ǫi (r) 2 p(fi (r)|zi (r) Q = k) = N (mik , σi k ), k = 1, · · · , K p(f |z) = hi p(fi |zi ) i P P ′ )) p(z) ∝ exp γ δ(z(r) − z(r ′ r∈R r ∈V(r) zi (r) follow a Markovian model along the index i

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 52/68

Source (with H. Snoussi & M. Ichir) separation: P gi (r) = N j=1 Aij fj (r) + ǫi (r) p(fj (r)|zj (r) = k) = N (mj , σ 2 ) k jk h P i P ′ )) p(z) ∝ exp γ δ(z(r) − z(r ′ r∈R r ∈V(r) p(A ) = N (A , σ 2 ) ij 0ij 0 ij

f

g

b f

b z

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 53/68

Microwave imaging for breast cancer detection (with L. Gharsally and B. Duchˆene) receivers

◮

Breast model built up from MRI scan [Zastrow et al. 2008]

◮

64 sources

◮

64 receivers

◮

6 frequencies in the band 0.5 − 3 GHz

tumor " D$ % 2 cm

skin " D) %

7.5 c m

S 10 cm

source

breast " D& % D'

D 12.2 cm

D = Nx × Ny

D1 (ǫ1 , σ1 (S/m))

D2 (ǫ2 , σ2 (S/m))

D3 (ǫ3 , σ3 (S/m))

D4 (ǫ3 , σ3 (S/m))

120 × 120

(10, 0.5)

(6.12, 0.11)

([2.46, 60.6], [0.01, 2.28])

(55.3, 1.57)

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 54/68

Microwave imaging for breast cancer detection CSI: Contrast Source Inversion, VBA: Variational Bayesian Approach, MGI: Independent Gaussian mixture, MGM: Gauss-Markov mixture

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 55/68

Optical Diffraction Tomographic imaging

AA AA AAAAAAAA AA AA A A AAAAAAA AAAAAAAA (with H. Ayasso and B. Duchˆene)

1 µm

observation

x

0.5 µm

0.14 µm

resin

θ

incident wave θ 1

D1 air

γ

12

y

0.5 µm

0.2 µm

D

Ω

12

D2

silicon

γ

12

silicon

O2

0.11 µm

resin

γ

O1

0.3 µm

silicon

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 56/68

Optical Diffraction Tomographic imaging (with H. Ayasso and B. Duchˆene) x (µm) 2.4

0.8

0.2 2.0 0.6 1.6

0.15

1.2

0.1

0.4

0.8 0.2 0.4

0.05 0

0

0 -1.5

-0.5

0.5

-1

1.5

AA AA AA AA AA AA AAA AAA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA -0.5

0

0.5

x (µm) 0.15

0.1

0.05

0

1

y (µm)

y (µm)

x (µm)

x (µm)

0.15 1.8

0.2

2.0

1.5

0.15

1.2 0.9

0.1

1.5

1.0

0.6 0.5 0.3

0.05

0

0

0 -1.5

-0.5

0.5

1.5

-1

AA AA AA AA AA AAA AAA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA -0.5

y (µm)

0

0.5

0.1

0.05

0

1

y (µm) x (µm)

x (µm)

AA AA AA AA AA AAA AAA AA AA AA AA AA AAA AAA AA AA AA AA AA AA AA AA AA AA AAAAAAAAAA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA 0.15

0.2

0.1

0.15

AAAAAAAAA x”

0.1

x”

0.05

0

0.05

0

-1.5

-0.5

0.5

-1

1.5

y (µm)

-0.5

0

0.5

1

y (µm)

2

2

χ / k1

χ / k1

AA AAA AAA AA AAA AAA AAA AA AA AA AA AAA AAA AA AA AA AA AA AA AA AAA AA AA AA AA AA AA AAA AA AA AA AA AA AA AA AA AA AA AAA AAA AA AAA AA AAAA AAA AA AA A 2

2

1.5

1.5

Real profile

1

Bayesian CSI

0.5

0

-1.5

-0.5

0.5

y (µm)

1.5

AAA AA AAA AA AA AAA AAA AA AAA AA AA AA AAA AA AAA AA AAA AA AAA AA AAA AA AA 1

0.5

0

-1

-0.5

0

0.5

1

y (µm)

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 57/68

Acoustic source localization [Ning CHU et al] Vehicle acoustic imaging at 2500Hz 0 −2 1 −4 −6

0.5

−8

Beamforming

0

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

−10

0 −2 1 −4 −6

0.5

−8

MAP

0

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

−10

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 0 58/68

Acoustic source localization (Simulation) 1.4

1.4

10

2 1.3

1.3

8

0 1.2

1.2 6 −2

1.1

1.1

y (m)

y (m)

4 −4 1

1 2

−6

0.9

0.9 0

−8

0.8

0.7

0.8

−10

0.6

(a)

−2

0.7

−4

0.6 −1.2

−1

−0.8

−0.6 x (m)

−0.4

−0.2

(b)

0

−1.2

1.4

−1

−0.8

−0.6 x (m)

−0.4

−0.2

0

1.4 2

2

1.3

1.3 0

0

1.2

1.2 −2

−2 1.1

−4

y (m)

y (m)

1.1

1

−4

1

−6 0.9 −8

0.8

−1.2

−1

−0.8

−0.6 x (m)

−0.4

−0.2

0

−10

0.7

−12

0.6

−8

0.8

−10

0.7

(c)

−6

0.9

−12

0.6

(d)

−1.2

−1

−0.8

−0.6 x (m)

−0.4

−0.2

0

Simulation at 2500Hz, 0dB SNR in colored noises, 14dB display: (a) Source powers (b) Beamforming powers (c) Bayesian MAP inversion and (d) Proposed VBA inversion

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 59/68

Conclusions ◮

Bayesian Inference for inverse problems

◮

Different prior modeling for signals and images: Separable, Markovian, without and with hidden variables

◮

Sprasity enforcing priors

◮

Gauss-Markov-Potts models for images incorporating hidden regions and contours

◮

Two main Bayesian computation tools: MCMC and VBA

◮

Application in different CT (X ray, Microwaves, PET, SPECT)

Current Projects and Perspectives : ◮

Efficient implementation in 2D and 3D cases

◮

Evaluation of performances and comparison between MCMC and VBA methods

◮

Application to other linear and non linear inverse problems: (PET, SPECT or ultrasound and microwave imaging)

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 60/68

Current Applications and Perspectives

We use these models for inverse problems in different signal and image processing applications such as: ◮

Period estimation in biological time series

◮

Signal deconvolution in Proteomic and molecular imaging

◮

X ray Computed Tomography

◮

Diffraction Optical Tomography

◮

Microwave Imaging, Acoustic imaging and sources localization

◮

Synthetic Aperture Radar (SAR) Imaging

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 61/68

Thanks to: Graduated PhD students: 1. C. Cai (2013: Multispectral X ray Tomography) 2. N. Chu (2013: Acoustic sources localization) 3. Th. Boulay (2013: Non Cooperative Radar Target Recognition) 4. R. Prenon (2013: Proteomic and Masse Spectrometry) 5. Sh. Zhu (2012: SAR Imaging) 6. D. Fall (2012: Emission Positon Tomography, Non Parametric Bayesian) 7. D. Pougaza (2011: Copula and Tomography) 8. H. Ayasso (2010: Optical Tomography, Variational Bayes) 9. S. F´ekih-Salem (2009: 3D X ray Tomography) 10. N. Bali (2007: Hyperspectral imaging) 11. O. F´eron (2006: Microwave imaging) 12. F. Humblot (2005: Super-resolution) 13. M. Ichir (2005: Image separation in Wavelet domain) 14. P. Brault (2005: Video segmentation using Wavelet domain) 15. H. Snoussi (2003: Sources separation) 16. Ch. Soussen (2000: Geometrical Tomography) 17. G. Mont´emont (2000: Detectors, Filtering) 18. H. Carfantan (1998: Microwave imaging) 19. S. Gautier (1996: Gamma ray imaging for NDT) 20. M. Nikolova (1994: Piecewise Gaussian models and GNC) 21. D. Pr´emel (1992: Eddy current imaging) A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 62/68

Thanks to: Current PhD students: ◮ L. Gharsali (Microwave imaging for Cancer detection) ◮ M. Dumitru (Multivariate time series analysis for biological signals) ◮ S. AlAli (Electrical imaging of CO2 stocking under the earth) Master students: ◮ A. Cai (Non-circular X ray Tomography) ◮ F. Fuc (Multi component signal analysis for biology applications) Post-Docs: ◮ J. Lapuyade (2011: Dimentionality Reduction and multivariate analysis) ◮ S. Su (2006: Color image separation) ◮ A. Mohammadpour (2004: HyperSpectral image segmentation)

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 63/68

Thanks my colleagues and collaborators ◮ ◮ ◮ ◮ ◮ ◮ ◮ ◮ ◮ ◮ ◮ ◮

B. Duchˆene & A. Joisel (L2S) (Inverse scattering and Microwave Imaging) N. Gac (L2S) (GPU Implementation) Th. Rodet (L2S) (Computed Tomography) —————– A. Vabre & S. Legoupil (CEA-LIST), (3D X ray Tomography) E. Barat (CEA-LIST) (Positon Emission Tomography, Non Parametric Bayesian) C. Comtat (SHFJ, CEA) (PET, Spatio-Temporal Brain activity) J. Picheral (SSE, Sup´elec) (Acoustic sources localization) D. Blacodon (ONERA) (Acoustic sources separation) J. Lagoutte (Thales Air Systems) (Non Cooperative Radar Target Recognition) P. Grangeat (LETI, CEA, Grenoble) (Proteomic and Masse Spectrometry) F. L´evi (CNRS-INSERM, Hopital Paul Brousse) (Biological rythms and Chronotherapy of Cancer)

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 64/68

References 1 1. A. Mohammad-Djafari, “Bayesian approach with prior models which enforce sparsity in signal and image processing,” EURASIP Journal on Advances in Signal Processing, vol. Special issue on Sparse Signal Processing, (2012). 2. A. Mohammad-Djafari (Ed.) Probl` emes inverses en imagerie et en vision (Vol. 1 et 2), Hermes-Lavoisier, Trait´ e Signal et Image, IC2, 2009, 3. A. Mohammad-Djafari (Ed.) Inverse Problems in Vision and 3D Tomography, ISTE, Wiley and sons, ISBN: 9781848211728, December 2009, Hardback, 480 pp. 4. A. Mohammad-Djafari, Gauss-Markov-Potts Priors for Images in Computer Tomography Resulting to Joint Optimal Reconstruction and segmentation, International Journal of Tomography & Statistics 11: W09. 76-92, 2008. 5. A Mohammad-Djafari, Super-Resolution : A short review, a new method based on hidden Markov modeling of HR image and future challenges, The Computer Journal doi:10,1093/comjnl/bxn005:, 2008. 6. H. Ayasso and Ali Mohammad-Djafari Joint NDT Image Restoration and Segmentation using Gauss-Markov-Potts Prior Models and Variational Bayesian Computation, IEEE Trans. on Image Processing, TIP-04815-2009.R2, 2010. 7. H. Ayasso, B. Duchene and A. Mohammad-Djafari, Bayesian Inversion for Optical Diffraction Tomography Journal of Modern Optics, 2008. 8. N. Bali and A. Mohammad-Djafari, “Bayesian Approach With Hidden Markov Modeling and Mean Field Approximation for Hyperspectral Data Analysis,” IEEE Trans. on Image Processing 17: 2. 217-225 Feb. (2008). 9. H. Snoussi and J. Idier., “Bayesian blind separation of generalized hyperbolic processes in noisy and underdeterminate mixtures,” IEEE Trans. on Signal Processing, 2006.

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 65/68

References 2 1. O. F´ eron, B. Duch` ene and A. Mohammad-Djafari, Microwave imaging of inhomogeneous objects made of a finite number of dielectric and conductive materials from experimental data, Inverse Problems, 21(6):95-115, Dec 2005. 2. M. Ichir and A. Mohammad-Djafari, Hidden markov models for blind source separation, IEEE Trans. on Signal Processing, 15(7):1887-1899, Jul 2006. 3. F. Humblot and A. Mohammad-Djafari, Super-Resolution using Hidden Markov Model and Bayesian Detection Estimation Framework, EURASIP Journal on Applied Signal Processing, Special number on Super-Resolution Imaging: Analysis, Algorithms, and Applications:ID 36971, 16 pages, 2006. 4. O. F´ eron and A. Mohammad-Djafari, Image fusion and joint segmentation using an MCMC algorithm, Journal of Electronic Imaging, 14(2):paper no. 023014, Apr 2005. 5. H. Snoussi and A. Mohammad-Djafari, Fast joint separation and segmentation of mixed images, Journal of Electronic Imaging, 13(2):349-361, April 2004. 6. A. Mohammad-Djafari, J.F. Giovannelli, G. Demoment and J. Idier, Regularization, maximum entropy and probabilistic methods in mass spectrometry data processing problems, Int. Journal of Mass Spectrometry, 215(1-3):175-193, April 2002. 7. H. Snoussi and A. Mohammad-Djafari, “Estimation of Structured Gaussian Mixtures: The Inverse EM Algorithm,” IEEE Trans. on Signal Processing 55: 7. 3185-3191 July (2007). 8. N. Bali and A. Mohammad-Djafari, “A variational Bayesian Algorithm for BSS Problem with Hidden Gauss-Markov Models for the Sources,” in: Independent Component Analysis and Signal Separation (ICA 2007) Edited by:M.E. Davies, Ch.J. James, S.A. Abdallah, M.D. Plumbley. 137-144 Springer (LNCS 4666) (2007).

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 66/68

References 3 1. N. Bali and A. Mohammad-Djafari, “Hierarchical Markovian Models for Joint Classification, Segmentation and Data Reduction of Hyperspectral Images” ESANN 2006, September 4-8, Belgium. (2006) 2. M. Ichir and A. Mohammad-Djafari, “Hidden Markov models for wavelet-based blind source separation,” IEEE Trans. on Image Processing 15: 7. 1887-1899 July (2005) 3. S. Moussaoui, C. Carteret, D. Brie and A Mohammad-Djafari, “Bayesian analysis of spectral mixture data using Markov Chain Monte Carlo methods sampling,” Chemometrics and Intelligent Laboratory Systems 81: 2. 137-148 (2005). 4. H. Snoussi and A. Mohammad-Djafari, “Fast joint separation and segmentation of mixed images” Journal of Electronic Imaging 13: 2. 349-361 April (2004) 5. H. Snoussi and A. Mohammad-Djafari, “Bayesian unsupervised learning for source separation with mixture of Gaussians prior,” Journal of VLSI Signal Processing Systems 37: 2/3. 263-279 June/July (2004) 6. F. Su and A. Mohammad-Djafari, “An Hierarchical Markov Random Field Model for Bayesian Blind Image Separation,” 27-30 May 2008, Sanya, Hainan, China: International Congress on Image and Signal Processing (CISP 2008). 7. N. Chu, J. Picheral and A. Mohammad-Djafari, “A robust super-resolution approach with sparsity constraint for near-field wideband acoustic imaging,” IEEE International Symposium on Signal Processing and Information Technology pp 286–289, Bilbao, Spain, Dec14-17,2011

A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 67/68

Current PhD’s and projects PhD’s: 1. Microwave imaging: PhD Leila Gharsalli (co-supervising B. Duchˆ ene) 2. Multivariate and multicomponents biological data processing: PhD Mircea Dumitru (co-supervising F. L´ evi), ERASYSBIO 3. ANR: HONTOMIN, PhD Safa AlAli, (CO2 stock supervising using electrical imaging) (B. Duchne & G. Perruson) 4. New methods for reducing dose in Computed Tomography, PhD Li Wang (N. Gac) 5. Information fusion for radar target recognition, starting PhD, May Abou Chahine, Thales Syst` emes A´ eroports Post-docs 1. ANR: SURMITO (Optical imaging), S. Mehrab, (B. Duchˆ ene) 2. 3D Tomography (SAFRAN), Th. Boulay, (N. Gac) A. Mohammad-Djafari, Bayesian methods for Inverse problems of imaging systems, Tehran Univ., EECE Dept., December 8, 2014, 68/68