Entropy Computation in Partially Observed Markov Chains F. Desbouvries INT / CITI Dept. & CNRS UMR 5157 91011 Évry, France
1
1. Embedded Markovian models : HMC
PMC
TMC
- Hidden Markov Chains (HMC) - Pairwise Markov Chains (PMC) - Triplet Markov Chains (TMC) - Examples
2. Efficient algorithms in Partially observed Markov Chains - Bayesian restoration (filtering, smoothing...) - Parameter estimation - Entropy Computation
2
Hidden Markov Chains (HMC)
% $#
&
&
! ) % (
$#
% $# "!
!
+% . & -
$#
+%
&
() -
+%
-
"!
+,% ! ) %
&
"! %
(
! ) %
*
-
(
&
()
$#
+%
%
linear and Gaussian case :
Kalman filtering
-
general case :
extended Kalman , particle filtering
Filtering :
-
'
3
indep. and jointly indep. and
/
1
0 0 0 0
2
1
32
0 0 0 0
;
87
9
:9 6
6
5 4 2 2
2
=
4
indep. and jointly indep. and
is Markovian. moreover the pair
Pairwise Markov Chains (PMC) ? Extension HMC
? @
Bayesian restoration :
Markovian
5
Markovian
>
=
PMC
?
Modeling power :
and
Markovian not necessarily Markovian
and
Markovian
Pairwise Markov Chains (PMC) ?
/
Extension HMC
HMC
Triplet Markov Chains (TMC) ?
/
Extension PMC
A
A 0 0
auxiliary
E
C
the triplet
B
hidden
is Markovian
0 0
F
D
observed
G %
? @
Modeling power :
TMC
PMC
A
marginal of the MC
H I
0
1
A
Bayesian restoration :
we restore
>
=
J 0
is obtained by marginalization .
)
6
Triplet Markov Chains (TMC) ?
/
Extension PMC
A
A 0 0
auxiliary
E
C
the triplet
B
hidden
is Markovian
0 0
F
D
observed
G %
? @
Modeling power :
TMC
PMC
A
marginal of the MC
H I
0
1
A
Bayesian restoration :
we restore
>
=
J 0
is obtained by marginalization .
)
7
Triplet Markov Chains (TMC) ?
/
Extension PMC
A
A 0 0
auxiliary
E
C
the triplet
B
hidden
is Markovian
0 0
F
D
observed
G %
? @
Modeling power :
TMC
PMC
A
marginal of the MC
H I
0
1
A
Bayesian restoration :
we restore
>
=
J 0
is obtained by marginalization .
)
8
1. Embedded Markovian models : HMC
PMC
TMC
- Hidden Markov Chains (HMC) - Pairwise Markov Chains (PMC) - Triplet Markov Chains (TMC) - Examples
2. Efficient algorithms in Partially observed Markov Chains - Bayesian restoration (filtering, smoothing...) - Parameter estimation - Entropy Computation
9
L
K
Hidden semi-Markov Chains (e.g.[Yu and Kobayashi, 2003]) :
N
M
H PQ
) of state duration B O
probability distribution (on
-
remains in the same state the time during which
-
discrete, discrete Example 1 :
f
cd
\
-
%e
%
b
^ Y
%
^
_
f
cd
\
%
%
^ X Z % X W
Y
V`
f
cd
\
-
%e
%
^
\ W -
g
Y
+ %
a _
^
Y
_
f
cd
\
-
%
%
^
\ W h
Y
`
%
^
Wi
^
X Z %
_ V
% ]j[ % ]\[ %
Wi
X Z -
%
Y
Y
Y
Y
Y
VU
n
n
m
no
M
kl
G
%
Y
M
k
M
m
m
m
k
M
k
M
m
k
M
k
M
10
XW
a _
% %
]\[ % \ Z %
X Z -
%
\ W
Y
XW
R S S SV S S S S
T SV S S S S S S
Then
L
K
continuous, discrete Example 2 :
"Switching" or "Jumping" models :
p
q
M r
M
0 0
s
q
M t
M
n
n
n
M
G
%
Y
M
m
k
M
m
m
k
M
k
M
m
k
M
k
M
11
M
M
M
M
0 0
Then
L
K
continuous, continuous Exemple 3 :
p
r
q
s
t
q
w
w
y
w v x
z
E
} C
E q
{
{xv
}
F
D
F
D
y
z
D
F
D
F
C
E
C
C
{
E
u
|
%
-
%
%
~
Y
p
z
z
r
E
C E
C E
C
E
C
A
v
}
q
z
A
z
z
t
z
s
F
D F
D
F
D
F
D
12
[Sorenson 1966]
1. Embedded Markovian models : HMC
PMC
TMC
- Hidden Markov Chains (HMC) - Pairwise Markov Chains (PMC) - Triplet Markov Chains (TMC) - Examples
2. Efficient algorithms in Partially observed Markov Chains - Bayesian restoration (filtering, smoothing...) - Parameter estimation - Entropy Computation
13
Efficient algorithms for Partially Observed Markov Chains
m
l k
Bayesian restoration filtering : – Kalman Filter (Ait-El-Fquih & Desbouvries, IEEE tr. SP Aug. 2006) – Particle Filtering (Desbouvries & Pieczynski , NSIP’03) smoothing : – Forward Backward, Viterbi (Pieczynski 2000) – RTS, Two-Filter, Frazer-Potter, Bryson-Frazier ... (Ait-El-Fquih & Desbouvries,
m
k
l
SSP’05, MaxEnt’06)
– Particle filtering (Ait-El-Fquih & Desbouvries , NSSPW’06)
Parameter Estimation Gaussian case, EM algorithm (Ait-El-Fquih & Desbouvries, Icassp’06)
Entropy Computation 14
Entropy Computation in Partially Observed Markov Chains
$ #
X
HMC : Hernando, Crespi & Cybenko, IEEE Tr. IT July 2005.
;
m
m
kl
k
m
¡
q
m
¡
¡
¢
%
m
k
m
¡
k
¡
;
m
k
m
k
k
¡
q
¢
+%
m
¡
8£
m
¡
15
not available. OK ;
operations Brute force :
Entropy Computation in Partially Observed Markov Chains
$ #
X
HMC : Hernando, Crespi & Cybenko, IEEE Tr. IT July 2005.
PMC :
;
m
m
k
l
k
m
¡
q
m
¡
¡
¢
%
m
k
m
¡
k
¡
;
m
k
m
k
k
¡
q
¢
+%
m
¡
8£
m
¡
16
not available. OK ;
operations Brute force :
Entropy Computation in Partially Observed Markov Chains
$ #
X
;
m
k
k
m
k
l
;
m
m
kl
k
m
¡
q
m
¡
¡
¢
%
m
k
m
¡
k
¡
;
m
k
m
k
k
¡
q
¢
+%
m
¡
8£
m
¡
17
not available. OK ;
HMC : Hernando, Crespi & Cybenko, IEEE Tr. IT July 2005.
) , PMC : recursive alg. + FB (
operations Brute force :
Entropy Computation in Partially Observed Markov Chains
;
m
k
k
m
k
l
;
m
m
kl
k
m
¡
q
m
¡
¡
¢
%
m
k
m
¡
k ¡
;
m
k
m
k
k
¡
q
¢
+%
m
¡
not available.
$ #
X
£
m
¡
18
HMC : Hernando, Crespi & Cybenko, IEEE Tr. IT July 2005.
) , PMC : recursive alg. + FB (
OK ; TMC :
operations Brute force :
