Spectral Design in Markov Random Fields
Jiao Wang Ken Sauer U. of Notre Dame
J-B Thibault Zhou Yu GE Healthcare
Charles Bouman Purdue University
July 9, 2010 Workshop on Bayesian Inference and Maximum Entropy Methods Chamonix, France
Outline Response of estimators under Gaussian MRF a priori models Spatial frequency view of MRF Examples of response control in image estimation Nonquadratic penalty functions
MAP in Inverse Problems Observations:
y = Hx + n
x� MAP = argmax[ p( y | x)g(x)] x
ﾠ T = argmin[( y - Hx) D( y - Hx) /2 + g x
ﾠ
￥w
i, j
r ( x i - x j )]
( i, j )ￎ C
Quadratic log-likelihood, Gibbs/Markov a priori model H = forward transformation D = inverse noise covariance
MRF Image Models 2-pixel cliques in C
R(x) = g
￥w
i, j
r ( x i - x j )]
(i, j )ￎ C
γ = e.g. inverse a priori variance
ﾠ
ρ = penalty function for local differences
ﾠ 2w0,4 = w1,4
ﾠ wi, j ﾠ0 ﾠ
typically (inverse to distance)
Linearized Analysis A priori:
−log g( x) = gx T Rx /2 + const .
Log-likelihood quadratic in x MAP estimate: ﾠ T -1 T x� = [H DH + gR] H Dy MAP
Response to signal:
ﾠ
T -1 T x� = [H DH + gR] H DHx MAP
Linearized Analysis x
[H T DH + gR] - 1 HT D
H
ﾠ
MAP Estimator ﾠ
ﾠ n
ﾠ
x�
[H T DH + gR] - 1 HT D ﾠ n�
Spatial invariance: ﾠ T H DH ﾠ H D (w)
R ﾠ R(w)
ﾠ
x�
(Local) Spectral Representation Signal response:
H D (w) X (w ) = FX (w) X(w) HD (w) + gR(w ) Bias toward zero X� (w ) =
For convolutional H, coincidence of MAP and MMSE, ﾠ and power spectral densities S:
S NN (w ) s N2 γR(w ) ￎ or SXX (w) SXX (w )
Conventional MRF H = 3x3 blur FX (w)
R(w )
H(w)
ﾠ
1 In 1-D: R(w ) = (1 - cos(w )) 2
ﾠ
Conventional MRF with Varying SNR FX (w)
ﾠ
0.2
0.7 σs
sn
2.0
Spectral Design of MRF Expand dimension and freedom of a priori R(w ) Example: 5x5 neighborhood π Uniform frequency sampling for ﾠ zero penalty up to 2
FX (w)
R(w )
ﾠ
ﾠ
ﾠ
Meaning of R(w ) T A priori: −log g( x) = gx Rx /2 + const .
-1 R = K Gaussian MRF implies , inverse correlation
ﾠ
ﾠNon-Gaussian: R = ?? –
Example: Constraint to subset of vector space ﾠ R lacks properties of inverse correlation ﾠ
Improper A Priori Densities •
R with negative eigenvalues
•
Most common “Gaussian” MRFs have zero eigenvalue and are improper (penalize only differences)
•
Many precedents in inference
A posteriori can be well-behaved:
P( Ai | B) =
P(B | Ai )P( Ai )
￥P(B | A )P( A ) j
j
j
Spectrally Focused A Priori Densities? • Conditioned on
image type, a priori knowledge may legitimately be focused • Sharper
transitions in frequency response demand larger
Frequency Boost via R(w ) 1 Estimator signal response: FX (w) = gR(w) 1+ ﾠ H D (w)
•
•
Require −H D (w ) < gR(w ) (else inverse unbounded)
•
For known HD (w ) , may boost frequency bands
ﾠ
ﾠ ﾠ
Example: 3x3 blur + noise (st. dev.= 10)
Photo courtesy of Dr. Ken Hanson
Design Example •
Goal: – – –
•
Reduce high frequency noise Deblur Emphasize frequencies near 0.5 pi
Nonuniform 1-D frequency sampling, McClellan transform wij R(w )
ﾠ
ﾠ
Focused Spectral Emphasis Origin al
Blurre d+nois e
Std Gaussia n MAP
Spectr al Design MAP
X-ray CT
“Groun d Truth”
EdgePreserve d MAP w/ q-GGMRF
Standar d FBP w/ Bone Filter
Spectral Design MAP w/ Quadrati c Penalty
Nonquadratic A Priori Models •
General form: R(x) =
ρ(D)
ﾠ
ﾠ
￥w ( i, j )ￎ C
i, j
r ( xi - x j )
ρ'(D)
ﾠ
•
Convexity is desirable
•
Spatial behavior similar to quadratic case?
Generalized Gaussian MRFs p
ρ(D) =| D |
p
vs.
D ρ(D) = 1+ | D/c |p - q
•
GGMRF has advantages in simplicity, norm properties ﾠ
•
q-GGMRF parameter c = “edge threshold”
•
q-GGMRF usually quadratic at origin (p=2), q=1.1 to 1.3
ﾠ
Nonquadratic A Priori Models •
Quadratic prior keeps convexity for
•
Hessian diagonal term j:
−H D (w ) < R(w )
2 ￎ ￥diHi,2 j + ￥wk , j ﾠﾠx 2 r ( x k - x j ) j i kￎ N j
Prior with unbounded second derivative loses convexity with negative coefficients ﾠ • Quadratic-at-origin (q-GGMRF) potential allows bounding of second derivative •
Nonquadratic A Priori Models •
Edge-preserving log-priors have lower curvature away from origin
•
Stability factor: Quadratic log-likelihood always dominates log-prior at large values
Nonquadratic MAP Corrupte d Original
Spectral Design Quadrati c MAP
Std Gaussia n MAP
Spectr al Design GGMRF MAP
X-ray CT
“Groun d Truth”
Spectral Design, Quadrati c MAP
q-GGMRF MAP, std. coefficien ts
Spectral Design MAP w/ q-GGMRF
Optimization Pitfalls
Spectral Design GGMRF MAP, q = 1.3
Spectral Design GGMRF MAP, q = 1.2
Optimization Costs of Spectral Design in MAP Estimation •
Greatly increased prior computation cost
•
Non-convex penalty may demand more selectivity in choice of algorithm
Conclusion •
Spectral focus in MRFs may pay off in selected applications
•
Optimization issues key, but surmountable(?) in non-quadratic a priori penalties
•
Alternative methods for selecting coefficients: •
Minimum-cost prediction
•
ML estimation
