. Inverse Problems in Imaging systems and Computer Vision: From Regularization to Bayesian Inference Ali Mohammad-Djafari ` Groupe Problemes Inverses Laboratoire des Signaux et Syst`emes UMR 8506 CNRS - SUPELEC - Univ Paris Sud 11 ´ Supelec, Plateau de Moulon, 91192 Gif-sur-Yvette, FRANCE.
[email protected] http://djafari.free.fr http://www.lss.supelec.fr
´ inter-instituts d’Alembert/Farman Journee ` et du vivant : Imagerie de la matiere de l’instrument a` l’algorithme : 18 mars 2010, ENS Cachan 1 / 37
Content ◮
Invers problems : Examples and general formulation
◮
Inversion methods : Analytical and Algebraic methods
◮
Deterministic methods : Least squares and Regularization
◮
Probabilistic methods : Bayesian inference Three main steps in Bayesian inference :
◮
◮ ◮ ◮
Forward models and likelihood Prior models for images Bayesian computation
◮
Applications (Computed Tomography)
◮
Conclusions
◮
Questions and Discussion 2 / 37
Inverse problems in imaging systems ◮
Example 1 : Measuring variation of temperature with a thermometer ◮ ◮
◮
Example 2 : Making an image with a camera, a microscope or a telescope ◮ ◮
◮
f (x , y ) real scene g(x , y ) observed image
Example 3 : Making an image of the interior of a body ◮ ◮
◮
f (t) variation of temperature over time g(t) variation of length of the liquid in thermometer
f (x , y ) a section of a real 3D body f (x , y , z) gφ (r ) a line of observed radiographe gφ (r , z)
Example 4 : Microwave, ultrasound or optical imaging ◮ ◮
f (x , y ) a section of a real 3D body gφ (r ) a line of diffracted wave measured at a given angle
3 / 37
Measuring variation of temperature with a thermometer ◮
f (t) variation of temperature over time
◮
g(t) variation of length of the liquid in thermometer
◮
Forward model : Convolution Z g(t) = f (t ′ ) h(t − t ′ ) dt ′ + ǫ(t) h(t) : impulse response of the measurement system
◮
Inverse problem : Deconvolution Given the forward model H (impulse response h(t))) and a set of data g(ti ), i = 1, · · · , M find f (t)
4 / 37
Making an image with a camera, a microscope or a telescope ◮
f (x, y) real scene
◮
g(x, y) observed image
◮
Forward model : Convolution ZZ g(x, y) = f (x ′ , y ′ ) h(x − x ′ , y − y ′ ) dx ′ dy ′ + ǫ(x, y) h(x, y) : Point Spread Function (PSF) of the imaging system
◮
Inverse problem : Image restoration Given the forward model H (PSF h(x, y))) and a set of data g(xi , yi ), i = 1, · · · , M find f (x, y) 5 / 37
Making an image with an unfocused camera Forward model : 2D Convolution ZZ g(x, y) = f (x ′ , y ′ ) h(x − x ′ , y − y ′ ) dx ′ dy ′ + ǫ(x, y) ǫ(x, y) ?
f (x, y)- h(x, y) - +m-g(x, y)
Inversion : Deconvolution ? ⇐=
6 / 37
Making an image of the interior of a body ◮
f (x, y) a section of a real 3D body f (x, y, z)
◮
gφ (r ) a line of observed radiographe gφ (r , z)
◮
Forward model : Line integrals or Radon Transform Z gφ (r ) = f (x, y) dl + ǫφ (r ) L
ZZ r ,φ = f (x, y) δ(r − x cos φ − y sin φ) dx dy + ǫφ (r )
◮
Inverse problem : Image reconstruction Given the forward model H (Radon Transform) and a set of data gφi (rj ), i = 1, · · · , M, j = 1, · · · N find f (x, y) 7 / 37
2D and 3D Computed Tomography 3D
2D Projections
80
60 f(x,y)
y 40
20
0 x −20
−40
−60
−80 −80
gφ (r1 , r2 ) =
Z
f (x, y, z) dl Lr1 ,r2 ,φ
−60
gφ (r ) =
−40
Z
−20
0
20
40
60
80
f (x, y) dl Lr ,φ
Forward probelm : f (x, y) or f (x, y, z) −→ gφ (r ) or gφ (r1 , r2 ) Inverse problem : gφ (r ) or gφ (r1 , r2 ) −→ f (x, y) or f (x, y, z) 8 / 37
Microwave or ultrasound imaging Mesaurs : scattered wave by the object φd (ri ) Incident
plane Wave Unknown quantity : f (r) = k02 (n2 (r) − 1) Intermediate quantity : φ(r) (total field)
y
Measurement
plane
Object
r'
r x
TM : 2D Case ZZ :
Gm (ri , r ′ )φ(r ′ ) f (r ′ ) dr ′ , ri ∈ S z ZZ r r r r Go (r, r ′ )φ(r ′ ) f (r ′ ) dr ′ , r ∈ D r φ(r) = φ0 (r) + ! ! D L r r , a , φd (ri ) =
D
a
E - E r Born approximation (φ(r ′ ) ≃ φ0 (r ′ ))) : D e (φ, x ) φ0 ZZ r % r ′ ′ ′ ′ % r Gm (ri , r )φ0 (r ) f (r ) dr , ri ∈ S φd (ri ) = r D
r
r
r
S
(y )
9 / 37
General formulation of inverse problems ◮
General non linear inverse problems : g(s) = [Hf (r)](s) + ǫ(s),
◮
r ∈ R,
s∈S
Z Linear models : g(s) = f (r) h(r, s) dr + ǫ(s) If h(r, s) = h(r − s) −→ Convolution.
◮
Discrete dataZ: g(si ) =
h(si , r) f (r) dr + ǫ(si ),
i = 1, · · · , m
◮
Inversion : Given the forward model H and the data g = {g(si ), i = 1, · · · , m)} estimate f (r)
◮
Well-posed and Ill-posed problems (Hadamard) : existance, uniqueness and stability
◮
Need for prior information 10 / 37
Analytical methods in X ray Tomography y 6
S•
r
@ @ @ @ @ @ @ f (x, y) @ @ @ φ @ @ x HH @ H @ @ @ @ •D
g(r , φ) = Radon : g(r , φ) = f (x, y) =
ZZ
R
L
f (x, y) dl
f (x, y) δ(r − x cos φ − y sin φ) dx dy
D
1 2π 2
Z
0
π
Z
+∞
−∞
∂ ∂r g(r , φ)
(r − x cos φ − y sin φ)
dr dφ 11 / 37
Filtered Backprojection method f (x, y) =
1 2π 2
Z
π 0
Z
∂ ∂r g(r , φ)
+∞ −∞
(r − x cos φ − y sin φ)
dr dφ
∂g(r , φ) ∂r Z 1 ∞ g(r , φ) ′ dr Hilbert TransformH : g1 (r , φ) = π 0 (r − r ′ ) Z π 1 g1 (r ′ = x cos φ + y sin φ, φ) dφ Backprojection B : f (x, y) = 2π 0 Derivation D :
g(r , φ) =
f (x, y) = B H D g(r , φ) = B F1−1 |Ω| F1 g(r , φ) • Backprojection of filtered projections : g(r ,φ)
−→
FT
F1
−→
Filter
|Ω|
−→
IFT
F1−1
g1 (r ,φ)
−→
Backprojection B
f (x,y )
−→
12 / 37
Limitations : Limited angle or noisy data
60
60
60
60
40
40
40
40
20
20
20
20
0
0
0
0
−20
−20
−20
−20
−40
−40
−40
−40
−60 −60
−60 −40
−20
0
20
Original
40
60
−60
−60 −40
−20
0
20
40
60
64 proj.
−60
−60 −40
−20
0
20
16 proj.
40
60
−60
−40
−20
0
20
40
60
8 proj. [0, π/2]
◮
Limited angle or noisy data
◮
Accounting for detector size
◮
Other measurement geometries : fan beam, ...
13 / 37
Algebraic methods g(si ) = ◮
Z
h(si , r) f (r) dr + ǫ(si ),
i = 1, · · · , M
f (r) is assumed to be well approximated by N X f (r) ≃ fj bj (r) j=1
with {bj (r)} a basis or any other set of known functions Z N X g(si ) = gi ≃ fj h(si , r) bj (r) dr, j=1
g = Hf + ǫ with Hij = ◮
Z
i = 1, · · · , M
h(si , r) bj (r) dr
H is huge dimensional
14 / 37
Inversion : Deterministic methods Data matching ◮
◮
Observation model gi = hi (f ) + ǫi , i = 1, . . . , M −→ g = H(f ) + ǫ Misatch between data and output of the model ∆(g, H(f )) fb = arg min {∆(g, H(f ))} f
◮
LS :
Examples :
∆(g, H(f )) = kg − H(f )k2 =
X
|gi − hi (f )|2
i
Lp : KL :
p
∆(g, H(f )) = kg − H(f )k = ∆(g, H(f )) =
X i
◮
X
|gi − hi (f )|p ,
1
