. Image Reconstruction Methods in Medical Imaging Ali Mohammad-Djafari Groupe Probl` emes Inverses Laboratoire des Signaux et Syst` emes UMR 8506 CNRS - SUPELEC - Univ Paris Sud 11 Sup´ elec, Plateau de Moulon, 91192 Gif-sur-Yvette, FRANCE.
[email protected] http://djafari.free.fr http://www.lss.supelec.fr European School of Medical Physics, Oct.-Nov. 2012
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 1/56
Content ◮
Seeing outside of a body
◮
Seeing inside of a body: Image reconstruction in Computed Tomography
◮
Different Imaging systems
◮
Common Inverse problem
◮
Analytical Methods
◮
Algebraic Deterministic Methods
◮
Probabilistic Methods
◮
Bayesian approach
◮
Examples and case studies
◮
Questions and Discussion
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 2/56
Seeing outside of a body: Making an image with a camera, a microscope or a telescope ◮
f (x, y ) real scene
◮
g (x, y ) observed image
◮
Forward model: Convolution ZZ g (x, y ) = f (x ′ , y ′ ) h(x − x ′ , y − y ′ ) dx ′ dy ′ + ǫ(x, y ) h(x, y ): Point Spread Function (PSF) of the imaging system
◮
Inverse problem: Image restoration Given the forward model H (PSF h(x, y ))) and a set of data g (xi , yi ), i = 1, · · · , M find f (x, y )
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 3/56
Making an image with an unfocused camera Forward model: 2D Convolution ZZ g (x, y ) = f (x ′ , y ′ ) h(x − x ′ , y − y ′ ) dx ′ dy ′ + ǫ(x, y ) ǫ(x, y )
f (x, y ) - h(x, y )
? - + -g (x, y )
Inversion: Deconvolution ? ⇐=
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 4/56
Different ways to see inside of a body Incident wave object
-
Active Imaging
object Transmission
R
Passive Imaging
Measurement
Incident wave -
6 object -
Y
Measurement Incident wave -
object
Reflection
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 5/56
Seeing inside of a body: Computed Tomography ◮
f (x, y ) a section of a real 3D body f (x, y , z)
◮
gφ (r ) a line of observed radiographe gφ (r , z)
◮
Forward model: Line integrals or Radon Transform Z gφ (r ) = f (x, y ) dl + ǫφ (r ) L
ZZ r,φ f (x, y ) δ(r − x cos φ − y sin φ) dx dy + ǫφ (r ) =
◮
Inverse problem: Image reconstruction Given the forward model H (Radon Transform) and a set of data gφi (r ), i = 1, · · · , M find f (x, y )
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 6/56
2D and 3D Computed Tomography 3D
2D Projections
80
60 f(x,y)
y 40
20
0 x −20
−40
−60
−80 −80
gφ (r1 , r2 ) =
Z
f (x, y , z) dl Lr1 ,r2 ,φ
−60
gφ (r ) =
−40
Z
−20
0
20
40
60
80
f (x, y ) dl Lr,φ
Forward probelm: f (x, y ) or f (x, y , z) −→ gφ (r ) or gφ (r1 , r2 ) Inverse problem: gφ (r ) or gφ (r1 , r2 ) −→ f (x, y ) or f (x, y , z) A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 7/56
Microwave or ultrasound imaging Mesaurs: diffracted wave by the object φd (ri ) Unknown quantity: f (r) = k02 (n2 (r) − 1) Intermediate quantity : φ(r)
y
Object
ZZ
r'
Gm (ri , r′ )φ(r′ ) f (r′ ) dr′ , ri ∈ S D ZZ Go (r, r′ )φ(r′ ) f (r′ ) dr′ , r ∈ D φ(r) = φ0 (r) + φd (ri ) =
Measurement
plane
Incident
plane Wave
x
D
Born approximation (φ(r′ ) ≃ φ0 (r′ )) ): ZZ Gm (ri , r′ )φ0 (r′ ) f (r′ ) dr′ , ri ∈ S φd (ri ) = D
r
z
-
φ0 Discretization : φd = H(f) φd = Gm Fφ −→ with F = diag(f) φ = φ0 + Go Fφ H(f) = Gm F(I − Go F)−1 φ0
(φ, f ) g
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 8/56
Fourier Synthesis in X ray ZZ Tomography
f (x, y ) δ(r − x cos φ − y sin φ) dx dy
g (r , φ) =
G (Ω, φ) = F (ωx , ωy ) = F (ωx , ωy ) = P(Ω, φ) y 6 s I
Z
g (r , φ) exp {−jΩr } dr
ZZ
f (x, y ) exp {−jωx x, ωy y } dx dy
for
ωx = Ω cos φ and ωy = Ω sin φ ωy 6 α Ω
r
I
f (x, y ) φ
-
F (ωx , ωy )
x
φ
-
ωx
p(r , φ)–FT–P(Ω, φ)
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 9/56
Fourier Synthesis in X ray tomography
F (ωx , ωy ) =
ZZ
f (x, y ) exp {−jωx x, ωy y } dx dy
v 50 100
u
? =⇒
150 200 250 300 350 400 450 50
100
150
200
250
300
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 10/56
Fourier Synthesis in Diffraction tomography ωy
y ψ(r, φ)
^ f (ωx , ω y )
FT 1
2 2 1
f (x, y)
x
-k 0
k0
ωx
Incident plane wave Diffracted wave
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 11/56
Fourier Synthesis in Diffraction tomography
F (ωx , ωy ) =
ZZ
f (x, y ) exp {−jωx x, ωy y } dx dy
v 50
100
150
u
? =⇒
200
250
300 50
100
150
200
250
300
350
400
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 12/56
Fourier Synthesis in different imaging systems
F (ωx , ωy ) = v
ZZ
f (x, y ) exp {−jωx x, ωy y } dx dy
v
u
X ray Tomography
v
u
Diffraction
v
u
Eddy current
u
SAR & Radar
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 13/56
Invers Problems: other examples and applications ◮
X ray, Gamma ray Computed Tomography (CT)
◮
Microwave and ultrasound tomography
◮
Positron emission tomography (PET)
◮
Magnetic resonance imaging (MRI)
◮
Photoacoustic imaging
◮
Radio astronomy
◮
Geophysical imaging
◮
Non Destructive Evaluation (NDE) and Testing (NDT) techniques in industry
◮
Hyperspectral imaging
◮
Earth observation methods (Radar, SAR, IR, ...)
◮
Survey and tracking in security systems
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 14/56
Computed tomography (CT) A Multislice CT Scanner Fan beam X−ray Tomography −1
−0.5
0
0.5
g (si ) = 1
Source positions
−1
−0.5
0.5
f (r) dli + ǫ(si )
Li
Detector positions
0
Z
1
Discretization g = Hf + ǫ
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 15/56
Magnetic resonance imaging (MRI) Nuclear magnetic resonance imaging (NMRI), Para-sagittal MRI of the head
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 16/56
X ray Tomography Z I = g (r , φ) = − ln f (x, y ) dl I0 Lr ,φ ZZ
150
100
y
f(x,y)
f (x, y ) δ(r − x cos φ − y sin φ) dx dy
g (r , φ) =
50
D
0
x
−50
−100
f (x, y)
−150
−150
phi
−100
−50
0
50
100
g (r , φ)
RT
150
60
p(r,phi)
40 315
IRT ? =⇒
270 225 180 135 90 45
20
0
−20
−40
−60
0 r
−60
−40
−20
0
20
40
60
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 17/56
Analytical Inversion methods S•
y 6
r
f (x, y ) φ
-
x
•D g (r , φ) Radon:
ZZ
f (x, y ) δ(r − x cos φ − y sin φ) dx dy Z π Z +∞ ∂ 1 ∂r g (r , φ) f (x, y ) = − 2 dr dφ 2π 0 −∞ (r − x cos φ − y sin φ)
g (r , φ) =
D
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 18/56
Filtered Backprojection method f (x, y ) =
1 − 2 2π
Z
π
0
Z
∂ ∂r g (r , φ)
+∞ −∞
(r − x cos φ − y sin φ)
dr dφ
∂g (r , φ) ∂r Z ∞ 1 g (r , φ) ′ dr Hilbert TransformH : g1 (r , φ) = π (r − r ′ ) Z π 0 1 g1 (r ′ = x cos φ + y sin φ, φ) dφ Backprojection B : f (x, y ) = 2π 0 Derivation D :
g (r , φ) =
f (x, y ) = B H D g (r , φ) = B F1−1 |Ω| F1 g (r , φ) • Backprojection of filtered projections: g (r ,φ)
−→
FT
F1
−→
Filter
|Ω|
−→
IFT
F1−1
g1 (r ,φ)
−→
Backprojection B
f (x,y )
−→
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 19/56
Limitations : Limited angle or noisy data
60
60
60
60
40
40
40
40
20
20
20
20
0
0
0
0
−20
−20
−20
−20
−40
−40
−40
−40
−60
−60
−60
−60
−40
−20
0
20
Original
40
60
−60
−40
−20
0
20
40
64 proj.
60
−60
−60 −40
−20
0
20
40
16 proj.
◮
Limited angle or noisy data
◮
Accounting for detector size
◮
Other measurement geometries: fan beam, ...
60
−60
−40
−20
0
20
40
60
8 proj. [0, π/2]
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 20/56
CT as a linear inverse problem Fan beam X−ray Tomography −1
−0.5
0
0.5
1
Source positions
−1
g (si ) =
Z
−0.5
Detector positions
0
0.5
1
f (r) dli + ǫ(si ) −→ Discretization −→ g = Hf + ǫ
Li
◮
g, f and H are huge dimensional
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 21/56
Algebraic methods: Discretization S•
Hij
y 6
r
f1 fj
f (x, y )
gi
φ
-
fN
x
•D g (r , φ) g (r , φ) =
Z
P f b (x, y ) j j j 1 if (x, y ) ∈ pixel j bj (x, y ) = 0 else f (x, y ) =
f (x, y ) dl L
gi =
N X
Hij fj + ǫi
j=1
g = Hf + ǫ A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 22/56
Inversion: Deterministic methods Data matching ◮
◮
◮
Observation model gi = hi (f) + ǫi , i = 1, . . . , M −→ g = H(f) + ǫ Misatch between data and output of the model ∆(g, H(f))
Examples:
– LS
bf = arg min {∆(g, H(f))} f
∆(g, H(f)) = kg − H(f)k2 =
X
|gi − hi (f)|2
i
– Lp – KL
p
∆(g, H(f)) = kg − H(f)k = ∆(g, H(f)) =
X i
◮
X
|gi − hi (f)|p , 1 < p < 2
i
gi gi ln hi (f)
In general, does not give satisfactory results for inverse problems.
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 23/56
Deterministic Inversion Algorithms Least Squares Based Methods bf = arg min {J(f)} f
with J(f) = kg − Hfk2
∇J(f) = −2Ht (g − Hf)
Gradient based algorithms: ◮
Initialize:
f (0)
f (k+1) = f (k) − α∇J(f (k) ) At each iteration: f (k+1) = f (k) + αHt g − Hf (k) we have to do the following operations: ◮ Compute g b = Hf (Forward projection) ◮
Iterate:
◮
Compute
◮
Distribute δf = Ht δg (Backprojection of error)
◮
Update
δg = g − b g (Error or residual)
f (k+1) = f (k) + δf
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 24/56
Gradient based algorithms Operations at each iteration:
f (k+1) = f (k) + αHt g − Hf (k)
b g = Hf (Forward projection) b (Error or residual) δg = g − g
◮
Compute
◮
Compute
◮
Distribute δf = Ht δg (Backprojection of error)
◮
Update
f (k+1) = f (k) + δf
projections of Initial estimated Forward guess −→ image −→ projection −→ estimated image −→ H b g = Hf (k) f (0) f (k) ↑ update ↑ correction term Backprojection in image space ←− ←− Ht δf = Ht δg
–
Measured ← projections g
↓ compare ↓ correction term in projection space δg = g − b g
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 25/56
Gradient based algorithms ◮
Fixed step gradient: f (k+1) = f (k) + αHt g − Hf (k)
◮
Steepest descent gradient:
f (k+1) = f (k) + α(k) Ht g − Hf (k)
with α(k) = arg minα {J(f + αδf)} ◮
Conjugate Gradient f (k+1) = f (k) + α(k) d(k) The successive directions d(k) have to be conjugate to each other.
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 26/56
Algebraic Reconstruction Techniques
◮
Main idea: Use the data as they arrive f (k+1) = f (k) + α(k) [Ht ]i ∗ gi − [Hf (k) ]i
which can also be written as:
f (k+1) = f (k) +
gi − [Hf (k) ]i
hti∗ hti∗ hi ∗ P (k) gi − j Hij fj = f (k) + hti∗ P 2 Hij j
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 27/56
Algebraic Reconstruction Techniques ◮
Use the data as they arrive f (k+1) = f (k) +
◮
gi − [Hf (k) ]i
hti∗ hti∗ hi ∗ P (k) gi − j Hij fj hti∗ = f (k) + P 2 Hij j
Update each pixel at each time P (k) gi − j Hij fj (k) (k+1) = fj + fj Hij P 2 j Hij
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 28/56
Algebraic Reconstruction Techniques (ART)
f (k+1) or fj
(k+1)
P (k) gi − j Hij fj hti∗ = f (k) + P 2 Hij j
P (k) gi − j Hij fj (k) Hij = fj + P 2 j Hij
projections of Initial estimated Forward image guess −→ image −→ projection −→ estimated P (k) (0) (k) H bi = g f f j Hij f j ↑ update ↑
−→
–
Measured ← projections gi
↓ compare ↓
correction term in image space P δg P i δfj = i Hij H j
ij
←−
Backprojection ←− Ht
correction term in projection space δgi = g i − b gi
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 29/56
Algebraic Reconstruction using KL distance ◮
bf = arg min {J(f)} f fj
(k+1)
with fj =P
J(f) =
(k)
i
Hij
X i
P
i
gi ln P gHi ij fj
Hij P
j
gi j
Hij fj
(k)
Interestingly, this is the OSEM (Ordered subset Expectation-Maximization) algorithm which is based on Maximum Likelihood and proposed first by Shepp & Vardi. estimated Initial image f (k) guess −→ (k) f (k+1) f (0) fj = Pj H i
ij
projections of Forward image −→ projection −→ estimated P (k) H b gi = j Hij f j
↑ update ↑
–
−→
Measured ← projections gi
↓ compare ↓
correction term in image space P δfj = P 1H i Hij δgi j
ij
←−
Backprojection ←− Ht
correction term in projection space gi δgi = b g i
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 30/56
Inversion: Regularization theory Inverse problems = Ill posed problems −→ Need for prior information Functional space (Tikhonov): g = H(f ) + ǫ −→ J(f ) = ||g − H(f )||22 + λ||Df ||22 Finite dimensional space (Philips & Towmey): g = H(f) + ǫ • Minimum norme LS (MNLS): J(f) = ||g − H(f)||2 + λ||f||2 • Classical regularization: J(f) = ||g − H(f)||2 + λ||Df||2 • More general regularization: or
J(f) = Q(g − H(f)) + λΩ(Df)
J(f) = ∆1 (g, H(f)) + λ∆2 (f, f 0 ) Limitations: • Errors are considered implicitly white and Gaussian • Limited prior information on the solution • Lack of tools for the determination of the hyperparameters A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 31/56
Bayesian estimation approach M:
g = Hf + ǫ
◮
Observation model M + Hypothesis on the noise ǫ −→ p(g|f; M) = pǫ (g − Hf)
◮
A priori information
p(f|M)
◮
Bayes :
p(f|g; M) =
p(g|f; M) p(f|M) p(g|M)
Link with regularization : Maximum A Posteriori (MAP) : bf = arg max {p(f|g)} = arg max {p(g|f) p(f)} f f = arg min {− ln p(g|f) − ln p(f)} f with Q(g, Hf) = − ln p(g|f) and λΩ(f) = − ln p(f) But, Bayesian inference is not only limited to MAP A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 32/56
Case of linear models and Gaussian priors g = Hf + ǫ ◮
◮
◮
◮
Hypothesis on the noise: ǫ ∼ N (0, σǫ2 I) 1 p(g|f) ∝ exp − 2 kg − Hfk2 2σǫ Hypothesis on f : f ∼ N (0, σf2 I) 1 2 p(f) ∝ exp − 2 kfk 2σf A posteriori: 1 σǫ2 2 2 p(f|g) ∝ exp − 2 kg − Hfk + 2 kfk 2σǫ σf MAP : bf = arg maxf {p(f|g)} = arg minf {J(f)} with
◮
J(f) = kg − Hfk2 + λkfk2 ,
Advantage : characterization of the solution b with bf = PH b t g, f|g ∼ N (bf, P)
λ=
σǫ2 σf2
b = Ht H + λI −1 P
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 33/56
MAP estimation with other priors: bf = arg min {J(f)} with J(f) = kg − Hfk2 + λΩ(f) f Separable priors: P 2 ◮ Gaussian: p(fj ) ∝ exp −α|fj |2 −→ Ω(f) = kfk2 = α j |fj | P ◮ Gamma: p(fj ) ∝ f α exp {−βfj } −→ Ω(f) = α j j ln fj + βfj ◮
◮
Beta: P P p(fj ) ∝ fjα (1 − fj )β −→ Ω(f) = α j ln fj + β j ln(1 − fj ) Generalized Gaussian: p(fj ) ∝ exp {−α|fj |p } ,
1 < p < 2 −→
Markovian models: X p(fj |f) ∝ exp −α φ(fj , fi ) −→ i ∈Nj
Ω(f) = α
Ω(f) = α
P
XX j
j
|fj |p ,
φ(fj , fi ),
i ∈Nj
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 34/56
Main advantages of the Bayesian approach ◮
MAP = Regularization
◮
Posterior mean ? Marginal MAP ?
◮
More information in the posterior law than only its mode or its mean
◮
Meaning and tools for estimating hyper parameters
◮
Meaning and tools for model selection
◮
More specific and specialized priors, particularly through the hidden variables More computational tools:
◮
◮
◮ ◮
◮
Expectation-Maximization for computing the maximum likelihood parameters MCMC for posterior exploration Variational Bayes for analytical computation of the posterior marginals ...
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 35/56
MAP estimation and Compressed Sensing
g = Hf + ǫ f = Wz
◮
W a code book matrix, z coefficients
◮
Gaussian:
◮
o n P p(z) = N (0, σz2 I) ∝ exp − 2σ1 2 j |z j |2 z P J(z) = − ln p(z|g) = kg − HWzk2 + λ j |z j |2
Generalized Gaussian (sparsity, β = 1): o n P p(z) ∝ exp −λ j |z j |β
J(z) = − ln p(z|g) = kg − HWzk2 + λ
◮
z z = arg minz {J(z)} −→ bf = Wb
P
j
|z j |β
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 36/56
Full Bayesian approach
M:
◮ ◮ ◮ ◮ ◮
◮
◮
◮
g = Hf + ǫ
Forward & errors model: −→ p(g|f, θ 1 ; M) Prior models −→ p(f|θ 2 ; M) Hyperparameters θ = (θ 1 , θ 2 ) −→ p(θ|M) p(g |f ,θ ;M) p(f |θ ;M) p(θ |M) Bayes: −→ p(f, θ|g; M) = p(g |M) b = arg max {p(f, θ|g; M)} Joint MAP: (bf, θ) (f ,θ ) R p(f|g; M) = R p(f, θ|g; M) df Marginalization: p(θ|g; M) = p(f, θ|g; M) dθ ( R bf = f p(f, θ|g; M) df dθ R Posterior means: b = θ p(f, θ|g; M) df dθ θ
Evidence of the model: ZZ p(g|M) = p(g|f, θ; M)p(f|θ; M)p(θ|M) df dθ
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 37/56
Two main steps in the Bayesian approach ◮
Prior modeling ◮
◮ ◮
◮
Separable: Gaussian, Generalized Gaussian, Gamma, mixture of Gaussians, mixture of Gammas, ... Markovian: Gauss-Markov, GGM, ... Separable or Markovian with hidden variables (contours, region labels)
Choice of the estimator and computational aspects ◮ ◮ ◮ ◮ ◮
MAP, Posterior mean, Marginal MAP MAP needs optimization algorithms Posterior mean needs integration methods Marginal MAP needs integration and optimization Approximations: ◮ ◮ ◮
Gaussian approximation (Laplace) Numerical exploration MCMC Variational Bayes (Separable approximation)
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 38/56
Which images I am looking for? 50 100 150 200 250 300 350 400 450 50
100
150
200
250
300
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 39/56
Which image I am looking for?
Gaussian p(fj |fj−1 ) ∝ exp −α|fj − fj−1 |2
Generalized Gaussian p(fj |fj−1 ) ∝ exp {−α|fj − fj−1 |p }
Piecewize Gaussian p(fj |qj , fj−1 ) = N (1 − qj )fj−1 , σf2
Mixture of GM p(fj |zj = k) = N mk , σk2
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 40/56
Gauss-Markov-Potts prior models for images ”In NDT applications of CT, the objects are, in general, composed of a finite number of materials, and the voxels corresponding to each materials are grouped in compact regions”
How to model this prior information?
f (r)
z(r) ∈ {1, ..., K }
p(f (r)|z(r) = k, mk , vk ) = N (mk , vk ) X p(f (r)) = P(z(r) = k) N (m Mixture of Gaussians k , vk )X p(z(r)|z(rk′ ), r′ ∈ V(r)) ∝ exp γ δ(z(r) − z(r′ )) ′ r ∈V(r) A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 41/56
Four different cases To each pixel of the image is associated 2 variables f (r) and z(r) ◮
f|z Gaussian iid, z iid : Mixture of Gaussians
◮
f|z Gauss-Markov, z iid : Mixture of Gauss-Markov
◮
f|z Gaussian iid, z Potts-Markov : Mixture of Independent Gaussians (MIG with Hidden Potts)
◮
f|z Markov, z Potts-Markov : Mixture of Gauss-Markov (MGM with hidden Potts)
f (r)
z(r)
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 42/56
Four different cases
Case 1: Mixture of Gaussians
Case 2: Mixture of Gauss-Markov
Case 3: MIG with Hidden Potts
Case 4: MGM with hidden Potts
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 43/56
Summary of the two proposed models
f|z Gaussian iid z Potts-Markov
f|z Markov z Potts-Markov
(MIG with Hidden Potts)
(MGM with hidden Potts)
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 44/56
Bayesian Computation p(f, z, θ|g) ∝ p(g|f, z, vǫ ) p(f|z, m, v) p(z|γ, α) p(θ) θ = {vǫ , (αk , mk , vk ), k = 1, ·, K }
p(θ) Conjugate priors
◮
Direct computation and use of p(f, z, θ|g; M) is too complex
◮
Possible approximations : ◮ ◮ ◮
◮
Gauss-Laplace (Gaussian approximation) Exploration (Sampling) using MCMC methods Separable approximation (Variational techniques)
Main idea in Variational Bayesian methods: Approximate p(f, z, θ|g; M) by q(f, z, θ) = q1 (f) q2 (z) q3 (θ) ◮ ◮
Choice of approximation criterion : KL(q : p) Choice of appropriate families of probability laws for q1 (f), q2 (z) and q3 (θ)
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 45/56
MCMC based algorithm p(f, z, θ|g) ∝ p(g|f, z, θ) p(f|z, θ) p(z) p(θ) General scheme:
◮
◮
◮
bf ∼ p(f|b b g) −→ b b g) −→ θ b ∼ (θ|bf, b z, θ, z ∼ p(z|bf, θ, z, g)
b g) ∝ p(g|f, θ) p(f|b b Sample f from p(f|b z, θ, z, θ) Needs optimisation of a quadratic criterion. b g) ∝ p(g|bf, b b p(z) Sample z from p(z|bf, θ, z, θ) Needs sampling of a Potts Markov field.
z, (mk , vk )) p(θ) Sample θ from p(θ|bf, b z, g) ∝ p(g|bf, σǫ2 I) p(bf|b Conjugate priors −→ analytical expressions.
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 46/56
Application of CT in NDT Reconstruction from only 2 projections
◮
◮
g1 (x) =
Z
f (x, y ) dy
g2 (y ) =
Z
f (x, y ) dx
Given the marginals g1 (x) and g2 (y ) find the joint distribution f (x, y ). Infinite number of solutions : f (x, y ) = g1 (x) g2 (y ) Ω(x, y ) Ω(x, y ) is a Copula: Z Z Ω(x, y ) dx = 1 and Ω(x, y ) dy = 1
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 47/56
Application in CT 20
40
60
80
100
120 20
40
60
80
100
120
g|f f|z z q g = Hf + ǫ iid Gaussian iid q(r) ∈ {0, 1} g|f ∼ N (Hf, σǫ2 I) or or 1 − δ(z(r) − z(r′ )) Gaussian Gauss-Markov Potts binary Forward model Gauss-Markov-Potts Prior Model Auxilary Unsupervised Bayesian estimation: p(f, z, θ|g) ∝ p(g|f, z, θ) p(f|z, θ) p(θ) A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 48/56
Results: 2D case
Original
Backprojection
Gauss-Markov+pos
Filtered BP
GM+Line process
LS
GM+Label process
20
20
20
40
40
40
60
60
60
80
80
80
100
100
100
120
120 20
40
60
80
100
120
c
120 20
40
60
80
100
120
z
20
40
60
80
100
120
c
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 49/56
Some results in 3D case (Results obtained with collaboration with CEA)
M. Defrise
Phantom
FeldKamp
Proposed method
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 50/56
Some results in 3D case
FeldKamp
Proposed method
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 51/56
Some results in 3D case Experimental setup
A photograpy of metalique esponge
Reconstruction by proposed method
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 52/56
Application: liquid evaporation in metalic esponge
Time 0
Time 1
Time 2
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 53/56
Conclusions ◮
Gauss-Markov-Potts are useful prior models for images incorporating regions and contours
◮
Bayesian computation needs often pproximations (Laplace, MCMC, Variational Bayes)
◮
Application in different CT systems (X ray, Ultrasound, Microwave, PET, SPECT) as well as other inverse problems
Work in Progress and Perspectives : ◮
Efficient implementation in 2D and 3D cases using GPU
◮
Evaluation of performances and comparison with MCMC methods
◮
Application to other linear and non linear inverse problems: (PET, SPECT or ultrasound and microwave imaging)
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 54/56
Some references ◮ ◮ ◮ ◮ ◮ ◮ ◮ ◮ ◮
◮ ◮ ◮
A. Mohammad-Djafari (Ed.) Probl` emes inverses en imagerie et en vision (Vol. 1 et 2), Hermes-Lavoisier, Trait´ e Signal et Image, IC2, 2009, A. Mohammad-Djafari (Ed.) Inverse Problems in Vision and 3D Tomography, ISTE, Wiley and sons, ISBN: 9781848211728, December 2009, Hardback, 480 pp. H. Ayasso and Ali Mohammad-Djafari Joint NDT Image Restoration and Segmentation using Gauss-Markov-Potts Prior Models and Variational Bayesian Computation, To appear in IEEE Trans. on Image Processing, TIP-04815-2009.R2, 2010. H. Ayasso, B. Duchene and A. Mohammad-Djafari, Bayesian Inversion for Optical Diffraction Tomography Journal of Modern Optics, 2008. A. Mohammad-Djafari, Gauss-Markov-Potts Priors for Images in Computer Tomography Resulting to Joint Optimal Reconstruction and segmentation, International Journal of Tomography & Statistics 11: W09. 76-92, 2008. A Mohammad-Djafari, Super-Resolution : A short review, a new method based on hidden Markov modeling of HR image and future challenges, The Computer Journal doi:10,1093/comjnl/bxn005:, 2008. O. F´ eron, B. Duch` ene and A. Mohammad-Djafari, Microwave imaging of inhomogeneous objects made of a finite number of dielectric and conductive materials from experimental data, Inverse Problems, 21(6):95-115, Dec 2005. M. Ichir and A. Mohammad-Djafari, Hidden markov models for blind source separation, IEEE Trans. on Signal Processing, 15(7):1887-1899, Jul 2006. F. Humblot and A. Mohammad-Djafari, Super-Resolution using Hidden Markov Model and Bayesian Detection Estimation Framework, EURASIP Journal on Applied Signal Processing, Special number on Super-Resolution Imaging: Analysis, Algorithms, and Applications:ID 36971, 16 pages, 2006. O. F´ eron and A. Mohammad-Djafari, Image fusion and joint segmentation using an MCMC algorithm, Journal of Electronic Imaging, 14(2):paper no. 023014, Apr 2005. H. Snoussi and A. Mohammad-Djafari, Fast joint separation and segmentation of mixed images, Journal of Electronic Imaging, 13(2):349-361, April 2004. A. Mohammad-Djafari, J.F. Giovannelli, G. Demoment and J. Idier, Regularization, maximum entropy and probabilistic methods in mass spectrometry data processing problems, Int. Journal of Mass Spectrometry, 215(1-3):175-193, April 2002.
A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 55/56
Thanks, Questions and Discussions Thanks to:
My graduated PhD students:
◮ ◮ ◮ ◮
H. Snoussi, M. Ichir, (Sources separation) F. Humblot (Super-resolution) H. Carfantan, O. F´ eron (Microwave Tomography) S. F´ ekih-Salem (3D X ray Tomography)
My present PhD students:
◮ ◮ ◮ ◮ ◮
H. Ayasso (Optical Tomography, Variational Bayes) D. Pougaza (Tomography and Copula) —————– Sh. Zhu (SAR Imaging) D. Fall (Emission Positon Tomography, Non Parametric Bayesian)
My colleages in GPI (L2S) & collaborators in other instituts:
◮ ◮ ◮ ◮ ◮ ◮ ◮
B. Duchˆ ene & A. Joisel (Inverse scattering and Microwave Imaging) N. Gac & A. Rabanal (GPU Implementation) Th. Rodet (Tomography) —————– A. Vabre & S. Legoupil (CEA-LIST), (3D X ray Tomography) E. Barat (CEA-LIST) (Positon Emission Tomography, Non Parametric Bayesian) C. Comtat (SHFJ, CEA)(PET, Spatio-Temporal Brain activity)
Questions and Discussions A. Mohammad-Djafari, Image Reconstruction Methods in Medical Imaging, European School of Medical Physics, Nov. 2012, 56/56
