Telecom ParisTech
THE FRENCH AEROSPACE LAB
Introduction to inverse problems. Applications to high angular resolution Astronomy
L. Mugnier – 2016
ONERA / Dept Optique Théorique & Appliquée, HAR team
http://www.onera.fr/dota
[email protected]
1
Telecom ParisTech
Context ❏ Aim of this talk: introduction to methods for solving “inverse problems”, including image deconvolution;
❏ Common framework for problems and methods in a wide variety of fields: ☞ X-ray tomography; ☞ ultrasound echography (medical diagnosis, Non-Destructive Testing); ☞ computer vision (stereo, etc.); ☞ geophysics; L. Mugnier – 2016
☞ wavefront sensing (Hartmann-Shack, curvature, phase diversity); ☞ image restoration, e.g., for imaging through turbulence (w/ or without AO); ☞ image reconstruction in interferometry, ☞ etc.
THE FRENCH AEROSPACE LAB
2
Telecom ParisTech
Inverse problem: a contrario definition ❏ Estimation of physical parameters from indirect measurements parameters; unknowns
→
instrument
→ measurements; data
❏ Wavefront sensing by Shack-Hartmann Unknowns: wavefront (phase)
Measurements: local slopes
Lenslet array Sensor
L. Mugnier – 2016
❏ image restoration object → atmosphere+telescope+camera
→ image
❏ image reconstruction in long baseline optical interferometry object → atmosphere+interferometer → visibilities Physics → model the data, i.e., compute the direct problem. THE FRENCH AEROSPACE LAB
3
Telecom ParisTech
Considered “direct model”, i.e. model of the data ❏ linear model: y = Hx + n,
H linear operator (matrix in finite dimension), y, vector of measurements (image pixels, S-H slopes,...), x, vector of unknowns (object, wavefront, etc.), n measurement noise.
L. Mugnier – 2016
❏ convolution case: ☞ H linear and shift-invariant ⇔ Hx = h ⋆ x ☞ h : point spread function (PSF) : x = δ ⇒ h ⋆ x = h. ˜ = FT(h) : transfer function. ☞ h ☞ Direct model:
y =h⋆x+n
where
˜ x+n ˜ = h.˜ ˜ y
THE FRENCH AEROSPACE LAB
4
Telecom ParisTech
“Classical” resolution of an inverse problem
❏ One searches for
ˆ ; y = Hx ˆ x (
❏ Invertible (linear) problem ∀y:
existence uniqueness
❏ If dim(x) > dim(y), not enough measurements for a unique solution (under-determined pb)
❏ If dim(x) < dim(y), usually no solution: noise makes it impossible to fit all L. Mugnier – 2016
data (over-determined pb).
❏ existence and uniqueness ∀y ⇔ H square and invertible ❏ Not often the case + accomodating more data is desirable!
THE FRENCH AEROSPACE LAB
5
Telecom ParisTech
Least Squares method One searches for
ˆ x
that minimizes the Least Squares criterion:
ˆ k2 = inf ky − Hxk2 ky − H x x
(1)
Legendre (1805) and Gauss (between 1795 and 1809) (estimation of the Earth’s ellipticity from arc measurements → definition of the meter)
❏ existence guaranteed (in finite dimension): ˆ , where P projector on Im(H). (1) ⇔ P y = H x ❏ uniqueness guaranteed: ˆ onto N(H)⊥ if N(H) 6= {0} one projects x ˆ norm → LS solution of minimal norm x min , or Generalized Inverse.
L. Mugnier – 2016
ˆ = (H t H)−1 H t y ❏ analytical solution: x N(H)
^ x min norm N(H)
y
x
T
^ x
(generally)
Im H
H Py H
THE FRENCH AEROSPACE LAB
6
Telecom ParisTech
Hartmann-Shack Wave Front Sensor incident wavefront Microlens array
Sub−images
01 1010 10 1010 1010
Sensor
L. Mugnier – 2016
α
0 1 1 0 0 1 1 0 0 1 0 1 11 00 0 1 1 0 0 0 1 01 1 1 0 0 1 0 1 1 11 00 0 0 1 00 11 0 1 0 1 0 1 0 1 0 0 1 1 0 1 0 1 00 11 00 11
αf
f
❏ Principle: measurement of local slopes of wavefront; ❏ Works with point sources and extended objects, w/ large spectral bandwidth; ❏ Estimation of wavefront from local slopes. THE FRENCH AEROSPACE LAB
7
Telecom ParisTech
Polynômes de Zernike
❏ Polynômes orthonormés sur D(0, 1) ; ❏ Décomposition de toute phase sur cette base de « modes » : L. Mugnier – 2016
φ(u, v) =
P
j
φj Zj (u, v)
❏ Ordre radial n ր ⇒ Fréq. spatialeր
THE FRENCH AEROSPACE LAB
8
Telecom ParisTech
Hartmann-Shack: direct problem and LS solution
❏ Data: K sub-apertures → 2 K slopes yu/v,i = y Z 1 ∂φ(u, v) yu,i = du dv + noise S i Si ∂u ❏ Linear direct model: y = Hx + n with: x = {xj }, φ(u, v) =
jX max
xj Zj (u, v),
L. Mugnier – 2016
j=2
x ←j max −→ H = 2K . y .
❏ Problem: estimate the wavefront x from the measured slopes y ˆ = (H t H)−1 H t y ❏ LS solution: x
THE FRENCH AEROSPACE LAB
9
Telecom ParisTech
Hartmann-Shack simulation ❏ 12 × 12 sub-apertures, central obstruction d/D = 0.33 ; ❏ K = 100 illuminated sub-apertures ⇒ 2.K = 200 slope measurements; ❏ data model: y = Hx + n; ❏ unknowns: x, decomposed on jmax − 1
sub-apertures
True phase
Zernike polynomials;
L. Mugnier – 2016
❏ n zero-mean white Gaussian (and approximately stationary), SNRslopes
such that
= 100;
❏ H of size 2.K × jmax .
Data: slopes in x and in y
THE FRENCH AEROSPACE LAB
10
Telecom ParisTech
LS wavefront reconstructions
L. Mugnier – 2016
True phase
LS jmax
=3 = 2.6 rad2
σϕ2 = 3.5 rad2
2 σerr
LS jmax
LS jmax
2 σerr
= 55 = 0.8 rad2
= 105 = 1.0 rad2
LS jmax 2 σerr
= 21 = 1.1 rad2
LS jmax
= 153 = 210 rad2
2 σerr Reconstruction quality depends on jmax optimal jmax depends on SNRslopes . 2 σerr
THE FRENCH AEROSPACE LAB
11
Telecom ParisTech
The convolutive case: LS vs inverse filter ˜x ˜ = h.˜ ❏ y =h⋆x⇒y ˆ = FT−1 hy˜˜ : inverse filter x → or ˜ˆ ) (f ) = 0 for frequencies f such that h ˜ (f ) = 0 idem + (x ❏ This solution minimizes ky − h ⋆ xk2 ; L. Mugnier – 2016
⇒ it is a Least Squares solution; ❏ Moreover it is of minimal norm (“energy”).
THE FRENCH AEROSPACE LAB
12
Telecom ParisTech
Example of Least Squares restoration
original signal
noisy image
LS restoration
L. Mugnier – 2016
❏ interpretation of the failure: equivalence Least Squares/ inverse filter. ❏ LS: works if “there are much more data than unknowns”, i.e., dim x ≪ dim y.
THE FRENCH AEROSPACE LAB
13
Telecom ParisTech
Regularization: general principle ❏ Data alone is not enough: many very different solutions are compatible w/ the data.
❏ Regularization: introduction of prior knowledge on the solution ☞ parametric: we have a model for x as a function of a few parameters θ : ˆ = arg min ky − Hx(θ)k2 . θ θ
Example:
x=disc, θ =radius, position, brightness.
L. Mugnier – 2016
☞ non-parametric: search for the solution, compatible w/ data, that best fits our prior knowledge:
ˆ = arg min ky − Hxk2 + α R(x); ✓ the “deterministic way”: x x
✓ the “stochastic way”: Bayesian estimation.
THE FRENCH AEROSPACE LAB
14
Telecom ParisTech
Stochastic framework (1): the Maximum Likelihood method ❏ y = Hx + n, where n models uncertainties on the measurement (detection, transmission, imperfection of data model,...)
❏ Common statistical model for n : probability density function pn . −1 n) Ex 1 : Gaussian noise. pn (n) ∝ exp(− 12 nt Cn P 2 1 Ex 2 : stationary white Gaussian noise. pn (n) ∝ exp(− 2σ 2 k nk ) ❏ Likelihood of the data: proba. to measure y knowing that the true object is x: p(y|x) = pn (y − Hx) L. Mugnier – 2016
❏ Maximum Likelihood estimator:
ˆ M L = arg max p(y|x) x x
one searches for the object x that makes the data y the most likely
❏ ML for stationary Gaussian noise: 1 ˆ M V = arg max exp − 2 ky − Hxk2 = arg min ky − Hxk2 ! x 2σ x x THE FRENCH AEROSPACE LAB
15
Telecom ParisTech
Graphical interpretation of Maximum Likelihood
L. Mugnier – 2016
p(y | x )
p(y | x ) 2
1
y measured y one chooses the object x that makes the data y the most likely. THE FRENCH AEROSPACE LAB
16
Telecom ParisTech
Stochastic framework (2): Bayesian estimation ❏ Data (slopes, image, etc.): y = Hx + n ❏ Bayes’ theorem : p(y|x) p(x) p(x|y) = | {z } p(y) posterior law
❏ Maximum A Posteriori (MAP) :
∝ p(y|x) × | {z } likelihood
p(x) |{z}
prior law on x
L. Mugnier – 2016
ˆ given the data y one searches for the most likely x ❏ p(y|x) : instrument model + statistics of noise. 1 2 Ex. : exp − 2σ 2 ky − Hxk . Maximization ↔ fidelity to data ❏ p(x) : prior knowledge on solution
Ex. : statistical information (turbulence) / qualitative (image). Maximization ↔ fidelity to the prior ⇒ regularization
THE FRENCH AEROSPACE LAB
17
Telecom ParisTech
Statistical characterization of the turbulent phase ❏ Kolmogorov ⇒ x zero-mean Gaussian:
p(x) ∝ exp
L. Mugnier – 2016
❏ Cx (j, j) = σx2j , in the Zernike polynomial basis:
− 12 xt Cx−1 x ;
THE FRENCH AEROSPACE LAB
18
Telecom ParisTech
MAP estimation in the Gaussian case ❏ Gaussian noise, Gaussian a priori law for the phase; ❏ Posterior law given by Bayes’ theorem: p(x|y) ∝
p(y|x) | {z }
×
model + noise statistics
p(x) |{z}
prior on x
1 1 t −1 t −1 ∝ exp − (y − Hx) Cn (y − Hx) × exp − x Cx x 2 2
1 1 t −1 t −1 ˆ MAP = arg max p(x|y) = arg min (y − Hx) Cn (y − Hx) + x Cx x ❏ x x x 2 | {z } |2 {z } − ln p(y|x)
L. Mugnier – 2016
− ln p(x)
❏ MAP solution: ˆ MAP = H x
t
Cn−1 H
+
−1 −1 Cx
H t Cn−1 y , M y
❏ Linear solution: specific to the Gaussian case, M is called the reconstruction matrix. THE FRENCH AEROSPACE LAB
19
Telecom ParisTech
MAP wavefront reconstruction (Gaussian case) ❏ We search for the most likely phase given the data: × p(x) p(x|y) ∝ p(y|x) |{z} | {z } model + noise statistics
prior on x
L. Mugnier – 2016
❏ white Gaussian noise + Gaussian prior law p(x) for turbulence (Kolmogorov):
True phase
MAP
σϕ2 = 3.5 rad2
2 σerr = 0.3 rad2
best ML/LS jmax
= 55
2 σerr = 0.8 rad2
MAP ⇒ optimal estimation
❏ Extension for control in AO: Kalman filtering. THE FRENCH AEROSPACE LAB
20
Telecom ParisTech
Simulation of AO-corrected long exposure image
L. Mugnier – 2016
object x
PSF h
image y
=h⋆x+n
Partial correction ⇒ deconvolution necessary VLT-NAOS case:
Mv = 11 , λ = 0.5 µm , field of view 0.8 arcsec , D/ro ≈ 60,
seeing 0.73 arcsec ,
SR = 2.1% THE FRENCH AEROSPACE LAB
21
Telecom ParisTech
Maximum Likelihood restoration
L. Mugnier – 2016
Image
Object restored by Max. Likelihood
❏ Homogeneous whiter Gaussian noise ⇒ ML = inverse filter.
THE FRENCH AEROSPACE LAB
22
Telecom ParisTech
MAP in the Gaussian case: application to deconvolution ❏ Criterion to be minimized (Cn = σ 2 Id kHx − yk2 J(x) , − ln p(x|y) = + 2σ 2 ❏ In the Fourier domain: X |h(f ˜ )˜ x(f ) − y˜(f )|2 → J(x) = 2 σ2 f
and
¯ 6= 0): x
1 ¯ ), ¯ )t Cx−1 (x − x (x − x 2 2 ˜ 1 X |˜ x(f ) − x ¯(f )| + 2 Sx (f ) f
ˆ MAP = (H t H + σ 2 Cx−1 )−1 (H t y + σ 2 Cx−1 x ❏ Solution: x ¯) ❏ In the Fourier domain:
L. Mugnier – 2016
→
2 ˜ ⋆ (f ) h σ /Sx (f ) ˆ x ˜MAP (f ) = y˜(f ) + 2 2 ˜ ˜ )|2 + σ 2 /Sx (f ) |h(f )| + σ /Sx (f ) |h(f
˜ x ¯(f )
Inverse filter for high SNR’s, prior solution for very low SNR’s. Interpretation :
˜ˆLS (f ) + (1 − λ)(f ) x ˆ˜MAP (f ) = λ(f ) x ˜ x ¯(f )
❏ MAP under Gaussian stats (for x and n) ⇔ LS + quadratic regul. ⇔ Wiener. THE FRENCH AEROSPACE LAB
23
Telecom ParisTech
L. Mugnier – 2016
MAP under Gaussian statistics
Original object
simulated AO-corrected image
restored
THE FRENCH AEROSPACE LAB
24
Telecom ParisTech
MAP under Gaussian stats / Wiener on experimental AO-corrected image
L. Mugnier – 2016
AO-corrected (yes!) image
Wiener
❏ Ganymede : satellite of Jupiter. ❏ image taken on the BOA (Banc d’optique adaptative ONERA), at 1.52m telescope of Observatoire de Haute Provence, 1997. THE FRENCH AEROSPACE LAB
25
Telecom ParisTech
Maximum A Posteriori deconvolution (known PSF) ❏ Direct model: y = h ⋆ x + n ❏ Maximum A Posteriori estimator: ˆ MAP = arg max x x
p(y/x) | {z }
fidelity to the data
=
arg min [L(x/y, h) x
×
p(x) |{z}
a priori on object
+
αR(x)]
❏ No analytical solutions as soon as L or R is not quadratic, or constrained minimization (e.g. positivity) ⇒ numerical minimization.
L. Mugnier – 2016
❏ a priori info on object: positivity, possibly support, spatial structure. E.g.: smooth (PSD) / smooth with sharp edges / spikes / spikes + smooth
❏ noise model: photon noise, mixture photon + detector noises. ❏ extension : PSF imperfectly known (myopic deconvolution).
THE FRENCH AEROSPACE LAB
26
Telecom ParisTech
Edge-preserving prior ❏ GaussianX prior ⇐⇒ quadratic regularization (L2 ) R(x) = |(Dx)(k)|2 , D differential operator
ψ
k
Example :
(Dx)(k) = x(k) − x(k − 1)
⇒ noise smoothed out
Do
but large-amplitude jumps (edges) strongly penalized.
❏ L1 regularization: R(x) =
X
|(Dx)(k)| a
k
b
L. Mugnier – 2016
⇒ penalty independent of the jump’s stiffness. ψ ❏ L1 −L2 regularization: R(x) =
X k
ψ(|(Dx)(k)|)
⇒ noise smoothed out and edges preserved.
Do THE FRENCH AEROSPACE LAB
27
Telecom ParisTech
Edge-preserving restoration:
L. Mugnier – 2016
linear-quadratic regularization
Quadratic
Linear-Quadratic
regularization
regularization
Linear-quadratic regularization −→ good restoration of edges
THE FRENCH AEROSPACE LAB
28
Telecom ParisTech
L. Mugnier – 2016
Sensitivity of AO-corrected OTFs to seeing variations
0.65, 0.73, 0.79, 0.85 and 0.93 arcsec. Simulated images correspond to a seeing of 0.73 arcsec. FTO for 5 differents seeing values:
THE FRENCH AEROSPACE LAB
29
Telecom ParisTech
Classical deconvolution with incorrectly estimated PSF Image simulated with PSF @ seeing = 0.73 arcsec.
L. Mugnier – 2016
“reference” PSF = average PSF for seeing ∈
[0.65, 0.93] arcsec.
Deconvolution with
Deconvolution with
reference PSF
true PSF
“classical” deconvolution needs a good estimate of the PSF
THE FRENCH AEROSPACE LAB
30
Telecom ParisTech
Principle of “myopic” deconvolution ❏ Direct model: y = h ⋆ x + n ❏ Joint estimation of x and h by Maximum A Posteriori: ˆ [ˆ x, h]
=
arg max p(y/x, h) × | {z } x,h fidelity to data
=
×
a priori on object
arg min [L(x, h/y) + x,h
p(x) |{z}
R(x) +
p(h) |{z}
a priori on PSF
Rh (h)]
❏ L: (Weighted) Least Squares; R: quadratic or linear-quadratic regularization
L. Mugnier – 2016
❏ Rh : necessary. Quadratic regularization (long exposure image):
2 ˜ ˜ m (f ) h(f ) − h X 1 Rh (h) = , 2 Sh (f ) f 2 ˜ )−h ˜ m (f )| Sh (f ) = E |h(f
THE FRENCH AEROSPACE LAB
31
Telecom ParisTech
Myopic deconvolution Image simulated with PSF @ seeing = 0.73 arcsec.
L. Mugnier – 2016
“reference” PSF = average PSF for seeing ∈
[0.65, 0.93] arcsec.
Deconvolution with
Myopic
PSF=mean PSF
deconvolution
Myopic deconvolution −→ Good restoration of the object’s photometry
THE FRENCH AEROSPACE LAB
32
Telecom ParisTech
MISTRAL : myopic deconvolution of AO-corrected images MISTRAL = Myopic Iterative STep-preserving restoration algorithm : Fine model for mixture of noises + edge-preserving prior + myopic.
L. Mugnier – 2016
[Conan et al., Appl. Opt., 1998.
Fusco et al., A&A, 1999.
Mugnier et al., JOSA A, 2004.]
Double star GJ 263; angular separation 0.040 arcsec. THE FRENCH AEROSPACE LAB
33
Telecom ParisTech
MISTRAL on images from BOA @ OHP’s 1.52m telescope
(b)
(a)
L. Mugnier – 2016
AO-corrected image [1997/09/28; 20:18 UT]
MISTRAL
(d)
(c)
synthetic image JPL/NASA/Caltech
synthetic image + 1.52 telescope PSF
Ganymede: satellite of Jupiter. THE FRENCH AEROSPACE LAB
34
Telecom ParisTech
L. Mugnier – 2016
MISTRAL on NAOS-Conica images at VLT Active Galaxy Nucleus NGC1068
M-band image (4.8µm) obtained after recentering ∼
100 images. Images courtesy D. Gratadour. THE FRENCH AEROSPACE LAB
35
Telecom ParisTech
Further reading ❏ Introduction to inverse problems: Des données à la connaissance de l’objet : le problème inverse, L. Mugnier, Sect. 9.6 of L’observation en astrophysique, P. Léna et al., EDP Sciences, 2008;
❏ reference book on inverse problems: Bayesian Approach to Inverse Problems, edited by J. Idier, ISTE / Wiley, 2008. Chap. 1 to 3: basic theory. Chap. 10: Inversion in optical imaging through atmospheric turbulence;
❏ Selected papers: ☞ fundamental paper on inverse problems: Image Reconstruction and Restoration: Overview of Common Estimation Structures and Problems, G. Demoment, IEEE Trans. Acoust. Speech Signal Process., 37, 2024–2036, 1989;
☞ deconvolution of AO-corrected images: MISTRAL: a myopic edge-preserving image restoration method, with application to L. Mugnier – 2016
astronomical adaptive-optics-corrected long-exposure images, L. M. Mugnier, T. Fusco, and J.-M. Conan, J. Opt. Soc. Am. A, 21, 1841–1854, 2004.
☞ review on image reconstruction in interferometry: Advanced imaging methods for long-baseline optical interferometry, G. Le Besnerais, S. Lacour, L. M. Mugnier, E. Thiébaut, G. Perrin, and S. Meimon, IEEE Journal of Selected Topics in Signal Processing, 2, 767–780, 2008. THE FRENCH AEROSPACE LAB
36
