A. Mohammad-Djafari '
MVIP05
Teheran University, 23-24 Feb. 2005
Probabilistic Methods for Inverse Problems in Computer Vision
$
Ali MOHAMMAD-DJAFARI Laboratoire des signaux et syst`emes (UMR 08506 CNRS-Suplec-UPS) Sup´elec, Plateau de Moulon 91192 Gif-sur-Yvette Cedex, FRANCE.
&
[email protected] http://djafari.free.fr http://www.lss.supelec.fr 1
%
A. Mohammad-Djafari '
MVIP05
Teheran University, 23-24 Feb. 2005
Contents
$
• Inverses problems in computer vision • Summary of different statistical methods • Basics of Bayesian approach • HMM modeling of images • Examples of applications – Single channel image restoration – Fourier synthesis in optical imaging – Multi channel data fusion and joint segmentation – Video movie segmentation with motion estimation – Blind source (image) separation (BSS) – Hyperspectral image segmentation • Bayesian image processing in wavelet domain • Conclusions
&
2
%
A. Mohammad-Djafari '
MVIP05
Teheran University, 23-24 Feb. 2005
Inverses problems
$
• General non linear inverse problem: g(s) = [Hf (r)](s) + ²(s), • Linear model: g(s) =
Z
r ∈ R,
s∈S
f (r)h(r, s) dr + ²(s) R
• Discretized version g = h(f ) + ²
or
g = Hf + ²
where g = {g(s), s ∈ S}, ² = {²(s), s ∈ S}
and
f = {f (r), r ∈ R}
• Multi sensor imaging gi =
N X
Aij Hj fj + ²i ,
i = 1, · · · , M
j=1
where A = {Aij , i = 1, · · · , M, j = 1, · · · , N } is an unknown mixing matrix. & 3
%
A. Mohammad-Djafari '
MVIP05
Teheran University, 23-24 Feb. 2005
Fourier synthesis in optical imaging Z £ ¤ t g(ω) = f (r) exp −jω r dr + ²(ω)
• Non coherent imaging:
G(g) = |g|
−→
g = h(f ) + ²
• Coherent imaging:
G(g) = g
−→
g = Hf + ²
g = {g(ω), ω ∈ Ω},
² = {²(ω), ω ∈ Ω}
?
20
40
f = {f (r), r ∈ R}
20
40
⇐=
60
60
80
80
100
100
120
120 20
&
and
$
40
60
80
100
120
20
4
40
60
80
100
120
%
A. Mohammad-Djafari '
MVIP05
Teheran University, 23-24 Feb. 2005
Single channel image restoration
$
²(x, y)
? f (x, y) -
h(x, y)
- +
Observation model :
- g(x, y) = h(x, y) ∗ f (x, y) + ²(x, y)
g = Hf + ²
? ⇐=
&
5
%
A. Mohammad-Djafari '
MVIP05
Teheran University, 23-24 Feb. 2005
Color (Multi-spectral) image deconvolution
$
²i (x, y) fi (x, y) -
h(x, y)
Observation model :
? - + - gi (x, y) = h(x, y) ∗ fi (x, y) + ²i (x, y)
g i = Hfi + ²i ,
i = 1, 2, 3
? ⇐= &
6
%
A. Mohammad-Djafari '
MVIP05
Teheran University, 23-24 Feb. 2005
Image fusion and joint segmentation
g1 (r)
$
Fusion ? =⇒
z
g2 (r) gi (r) = fi (r) + ²i (r), g(r) = {gi (r), i = 1, M }, &
i = 1, · · · , M
g i = {gi (r), r ∈ R},
g(r) = f (r) + ²(r), 7
g = {g i (r), i = 1, M }
g =f +²
%
A. Mohammad-Djafari '
MVIP05
Teheran University, 23-24 Feb. 2005
$
Blind image separation and joint segmentation
gi (r) = f1 (r)
j=1
Aij fj (r) + ²i (r)
g(r) = {gi (r), i = 1, M }
? f2 (r)
PN
g1 (r)
g(r) = Af (r) + ²(r), g = {g i (r), i = 1, M }
Separation
g i = {gi (r), r ∈ R},
⇐= g2 (r)
g = Af + ²
f3 (r) &
8
%
A. Mohammad-Djafari '
MVIP05
Teheran University, 23-24 Feb. 2005
X ray Tomography 3D
$
2D Projections
80
60 f(x,y)
y 40
20
0 x −20
−40
−60
−80 −80
gφ (r1 , r2 ) =
Z
f (x, y, z) dl Lr1 ,r2 ,φ
−60
−40
gφ (r) =
−20
Z
0
20
40
60
80
f (x, y) dl Lr,φ
Forward problem: f (x, y) or f (x, y, z) −→ gφ (r) or gθ,φ (r1 , r2 ) &
Inverse problem: gφ (r) or gφ,φ (r1 , r2 ) −→ f (x, y) or f (x, y, z) 9
%
A. Mohammad-Djafari '
MVIP05
Teheran University, 23-24 Feb. 2005
X ray Tomography and Radon Transform
$
150
100
y
f(x,y)
f (x, y) -
50
0
- g(r, φ)
TR
x
−50
g(r, φ) =
−100
−150
−150
−100
phi
−50
0
50
100
150
g(r, φ) =
ZZ
Z
f (x, y) dl
Lr,φ
f (x, y) δ(r − x cos φ − y sin φ) dx dy D 60
p(r,phi)
40 315
20 270
?
225
0
180
=⇒
135 90
−20
−40
45
−60
0
−60
r
&
10
−40
−20
0
20
40
60
%
A. Mohammad-Djafari '
MVIP05
Teheran University, 23-24 Feb. 2005
3D Computed Tomography / 3D Shape from shadows
3D Computed Tomography
&
$
3D Shape from shadows
11
%
A. Mohammad-Djafari '
MVIP05
Teheran University, 23-24 Feb. 2005
3D Computed Tomography / 3D Shape from shadows
3D Computed Tomography
3D Shape from shadows
z
z
y
y
x
&
$
x
12
%
A. Mohammad-Djafari '
MVIP05
Teheran University, 23-24 Feb. 2005
Deterministic methods
$
Data matching • Observation model
i = 1, . . . , M −→ g = H(f )²
gi = hi (f ) + ²i ,
• Misatch between data and output of the model ∆(g, H(f )) • Examples: – LS
b = arg min {∆(g, H(f ))} f f
2
∆(g, H(f )) = kg − H(f )k =
X
|gi − hi (f )|
2
i
– Lp – KL
p
∆(g, H(f )) = kg − H(f )k = ∆(g, H(f )) =
X i
X
p
|gi − hi (f )| ,
1