Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
Application of K¨ahler manifold to signal processing and Bayesian inference Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1 1. Department of Applied Mathematics and Statistics 2. Department of Physics and Astronomy SUNY at Stony Brook
September 23, 2014 MaxEnt 2014, Amboise
Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
Application of K¨ ahler manifold to signal processing and Bayesia
Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
Table of contents
1
Review on K¨ahler manifold
2
K¨ahlerian information geometry for signal processing
3
Geometric shrinkage priors
4
Example: ARFIMA
5
Conclusion
Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
Application of K¨ ahler manifold to signal processing and Bayesia
Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
K¨ahler manifold and information geometry
Implications of K¨ahler manifold differential geometry, algebraic geometry
Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
Application of K¨ ahler manifold to signal processing and Bayesia
Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
K¨ahler manifold and information geometry
Implications of K¨ahler manifold differential geometry, algebraic geometry superstring theory and supergravity in theoretical physics
Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
Application of K¨ ahler manifold to signal processing and Bayesia
Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
K¨ahler manifold and information geometry
Implications of K¨ahler manifold differential geometry, algebraic geometry superstring theory and supergravity in theoretical physics Information Geometry
Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
Application of K¨ ahler manifold to signal processing and Bayesia
Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
K¨ahler manifold and information geometry
Implications of K¨ahler manifold differential geometry, algebraic geometry superstring theory and supergravity in theoretical physics Information Geometry Barbaresco (2006, 2012, 2014): K¨ahler manifold and Koszul information geometry
Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
Application of K¨ ahler manifold to signal processing and Bayesia
Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
K¨ahler manifold and information geometry
Implications of K¨ahler manifold differential geometry, algebraic geometry superstring theory and supergravity in theoretical physics Information Geometry Barbaresco (2006, 2012, 2014): K¨ahler manifold and Koszul information geometry Zhang and Li (2013): symplectic and K¨ahler structures in divergence function
Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
Application of K¨ ahler manifold to signal processing and Bayesia
Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
K¨ahler manifold
Definition The K¨ahler manifold is the Hermitian manifold with the closed K¨ahler two-form. In the metric expression, gij = g¯i¯j = 0 ∂i gj k¯ = ∂j gi k¯ = 0 Any advantages? Let’s discuss later.
Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
Application of K¨ ahler manifold to signal processing and Bayesia
Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
Linear systems and information geometry
Linear systems are described by the transfer function h(w ; ξ) y (w ) = h(w ; ξ)x(w ; ξ) where input x and output y . The metric tensor for the filter Z π 1 gµν (ξ) = (∂µ log S)(∂ν log S)dw 2π −π where S(w ; ξ) = |h(w ; ξ)|2 .
Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
Application of K¨ ahler manifold to signal processing and Bayesia
Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
z-transformation h(z; ξ) =
P∞
log h(z; ξ) = log h0 + log (1 +
r =0 hr (ξ)z ∞ X hr r =1
h0
−r
z −r ) = log h0 +
∞ X
ηr z −r
r =1
The metric tensor in terms of transfer function I dz 1 gµν = ∂µ log h + log h¯ ∂ν log h + log h¯ 2πi |z|=1 z where µ, ν run holomorphic and anti-holomorphic indices.
Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
Application of K¨ ahler manifold to signal processing and Bayesia
Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
The metric tensors in holomorphic and anti-holomorphic coordinates I dz 1 ∂i log h(z; ξ)∂j log h(z; ξ) gij (ξ) = 2πi |z|=1 z I 1 ¯ z ; ξ) ¯ dz gi¯j (ξ) = ∂i log h(z; ξ)∂¯j log h(¯ 2πi |z|=1 z The metric tensor gij = ∂i log h0 ∂j log h0 gi¯j = ∂i log h0 ∂¯j log h¯0 +
∞ X
∂i ηr ∂¯j η¯r
r =1
Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
Application of K¨ ahler manifold to signal processing and Bayesia
Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
K¨ahler manifold for signal processing Theorem Given a holomorphic transfer function h(z; ξ), the information geometry of a signal processing model is K¨ahler manifold if and only if h0 is a constant in ξ. (⇒) If the geometry is K¨ahler, it should be Hermitian imposing gij = ∂i log (h0 )∂j log (h0 ) = 0 → h0 constant in ξ (⇐) If h0 is a constant in ξ, the metric tensor is given in gij = 0 and gi¯j =
∞ X
∂i ηr ∂¯j η¯r → Hermitian
r =1
The K¨ahler two-form is closed : Ω = igi¯j dξ i ∧ d ξ¯j Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
Application of K¨ ahler manifold to signal processing and Bayesia
Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
K¨ahler potential for signal processing On the K¨ahler manifold, the metric tensor is gi¯j = ∂i ∂¯j K where the K¨ahler potential K. Corollary Given K¨ahler geometry, the K¨ahler potential of the geometry is the square of the Hardy norm of the log-transfer function.
K=
1 2πi
Z |z|=1
∗ dz log h(z; ξ) log h(z; ξ) z
= || log h(z; ξ)||2H 2 Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
Application of K¨ ahler manifold to signal processing and Bayesia
Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
Benefits of K¨ahlerian information geometry 1. Calculation of geometric objects is simplified. gi¯j = ∂i ∂¯j K, Γij,k¯ = ∂i ∂j ∂k¯ K i Rjimn ¯ Γjn , Ri¯j = −∂i ∂¯j log G ¯ = ∂m
2. Easy α-generalization and linear order correction in α Γ(α) = Γ + αT , R (α) = R + α∂T 3. Submanifolds of K¨ahler is K¨ahler. ¯ 4. Laplace-Beltrami operator:∆ = 2g i j ∂i ∂¯j
Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
Application of K¨ ahler manifold to signal processing and Bayesia
Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
Komaki’s shrinkage prior for Bayesian inference
Komaki (2006): The difference in risk functions is given by E(DKL (p(y |ξ)||pπJ (y |x (N) ))|ξ)) − E(DKL (p(y |ξ)||pπI (y |x (N) ))|ξ)) π π 1 ij 1 πJ πI I I = g ∂ log ∆ ∂ log − + o(N −2 ) i j 2N 2 πJ πJ N 2 πI πJ If ψ = πI /πJ is superharmonic, pπI outperforms pπJ . Superharmonic prior πI , Jeffreys prior πJ Superharmonicity of functions is hard to check. In particular, in high-dimensional curved geometry!
Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
Application of K¨ ahler manifold to signal processing and Bayesia
Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
Geometric priors Theorem ¯ is a On a K¨ahler manifold, a positive function ψ = Ψ(u ∗ − κ(ξ, ξ)) ¯ superharmonic prior function if κ(ξ, ξ) is (sub)harmonic, bounded above by u ∗ , and Ψ is concave decreasing: Ψ0 (τ ) > 0, Ψ00 (τ ) < 0. The ans¨atze for Ψ: Ψ1 (τ ) = τ a , Ψ2 (τ ) = log (1 + τ a )
(τ > 0, 0 < a ≤ 1)
The ans¨atze for κ: κ1 = K, κ2 =
∞ X
2
ar |hr (ξ)| , κ3 =
r =0 Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
n X
bi |ξ i |2
(ar > 0, bi > 0)
i=1
Application of K¨ ahler manifold to signal processing and Bayesia
Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
Algorithm for geometric priors
The algorithm for finding geometric priors is the following: 1
Check whether the geometry is K¨ahler.
2
Check the superharmonicity of prior function ψ.
3
If (sub)harmonic, plug it into the theorem to get superharmonic functions and move to the next step.
4
If superharmonic, multiply the Jeffreys prior and set it as the shrinkage prior.
5
Do Bayesian inference.
Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
Application of K¨ ahler manifold to signal processing and Bayesia
Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
ARFIMA The transfer function of ARFIMA: h(z; ξ) =
(1 − µ1 z −1 )(1 − µ2 z −1 ) · · · (1 − µq z −1 ) (1 − z −1 )d (1 − λ1 z −1 )(1 − λ2 z −1 ) · · · (1 − λp z −1 )
The K¨ahler potential: ∞ n n n n X d + (µ1 + · · · + µq ) − (λ1 + · · · + λp ) 2 K= n n=1
The metric tesnor of ARFIMA: 2 π 6
gi¯j =
1 λi log (1 − λi ) − µ1i log (1 − µi )
1 ¯j λ
Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
¯ j ) − 1 log (1 − µ log (1 − λ ¯j ) µ ¯j
1 ¯j 1−λi λ − 1−µ1 λ¯ i j
− 1−λ1 i µ¯j 1 1−µi µ ¯j
Application of K¨ ahler manifold to signal processing and Bayesia
Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
Conclusion
K¨ahler manifold: information geometry for signal processing K¨ahler potential: square of Hardy norm of log-transfer function Several computational benefits exist on the K¨ahler manifold. In particular, Komaki priors are easy to build. An algorithm and ans¨atze for Komaki priors are introduced.
Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
Application of K¨ ahler manifold to signal processing and Bayesia
Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
Choi, J. and Mullhaupt, A. P., K¨ahlerian information geometry for signal processing, arXiv:1404.2006 Choi, J. and Mullhaupt, A. P., Geometric shrinkage priors for K¨ahlerian signal filters, arXiv:1408.6800 Amari, S. and Nagaoka, H., Methods of information geometry, Oxford University Press (2000) Barbaresco, F., Information intrinsic geometric flows, AIP Conf. Proc. 872 (2006) 211-218 Barbaresco, F., Information Geometry of Covariance Matrix: Cartan-Siegel Homogeneous Bounded Domains, Mostow/Berger Fibration and Fr´echet Median, Matrix Information Geometry, Bhatia, R., Nielsen, F., Eds., Springer (2012) 199-256 Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
Application of K¨ ahler manifold to signal processing and Bayesia
Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
Barbaresco, F., Koszul Information Geometry and Souriau Geometric Temperature/Capacity of Lie Group Thermodynamics, Entropy 16 (2014) 4521-4565 Komaki, F., Shrinkage priors for Bayesian prediction, Ann. Statistics 34 (2006) 808-819 Ravishanker, N., Melnick, E. L., and Tsai, C., Differential geometry of ARMA models, Journal of Time Series Analysis 11 (1990) 259-274 Ravishanker, N., Differential geometry of ARFIMA processes, Communications in Statistics - Theory and Methods 30 (2001) 1889-1902 Tanaka, F. and Komaki, F., A superharmonic prior for the autoregressive process of the second order, Journal of Time Series Analysis 29 (2008) 444-452 Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
Application of K¨ ahler manifold to signal processing and Bayesia
Review on K¨ ahler manifold K¨ ahlerian information geometry for signal processing Geometric shrinkage priors Example: ARFIMA Conclusion
Tanaka, F., Superharmonic priors for autoregressive models, Mathematical Engineering Technical Reports, University of Tokyo (2009) Zhang, J. and Li, F., Symplectic and K¨ahler Structures on Statistical Manifolds Induced from Divergence Functions, Geometric Science of Information 8085 (2013) 595-603
Jaehyung Choi∗,1,2 Andrew P. Mullhaupt1
Application of K¨ ahler manifold to signal processing and Bayesia
