MAXIMUM ENTROPIES COPULAS Doriano-Boris Pougaza
&
Ali Mohammad-Djafari
Groupe Probl`emes Inverses Laboratoire des Signaux et Syst`emes (UMR 8506 CNRS - SUPELEC - UNIV PARIS SUD) Sup´elec, Plateau de Moulon, 91192 Gif-sur-Yvette, France.
[email protected] http://users.aims.ac.za/~ doriano Chamonix, France, July 4-9, 2010
STATISTICS & TOMOGRAPHY
Tomography : Given two projections horizontal and vertical f1 (x) and f2 (y), find image f (x, y) Statistics : Given two marginals pdfs f1 (x) et f2 (y), find the joint distribution f (x, y)
Two equivalent ill-posed Inverse Problems : Infinite number of solutions
D-B. POUGAZA & al. ()
MAXIMUM & COPULA
MaxEnt 2010
2 / 21
Content
1
OVERVIEW ABOUT COPULAS IN STATISTICS
2
MAXIMUM ENTROPIES COPULAS
3
NEW FAMILIES OF COPULAS
D-B. POUGAZA & al. ()
MAXIMUM & COPULA
MaxEnt 2010
3 / 21
OVERVIEW ABOUT COPULAS IN STATISTICS
WHAT IS A COPULA ? 1
2
SIMPLE DEFINITION : A copula is a multivariate probability distribution function defined on [0, 1]n whose marginals are uniform. IN THE STATISTICS LITERATURE
copula is a tool to link a multivariate distribution function to its marginal distributions. f (x, y) = f1 (x)f2 (y)Ω(x, y).
3
POWERFUL TOOLS IN MODELING Mostly used in Finance and Environmental Sciences (C. Genest & MacKay, 1986 ; R.M Cooke, 1997 ; P. Embrechts, 2003) Offer several choices to model dependency between variables (H. Joe, 1997 ; R.B. Nelsen, 2006) D-B. POUGAZA & al. ()
MAXIMUM & COPULA
MaxEnt 2010
4 / 21
OVERVIEW ABOUT COPULAS IN STATISTICS
COPULA DEFINITION A bivariate copula C is a function C : [0, 1] × [0, 1] (u, v)
−→ −→
[0, 1] C(u, v)
with the following properties : 1
C(u, 0) = 0 = C(0, v),
2
C(u, 1) = u and C(1, v) = v,
3
C(u2 , v2 ) − C(u2 , v1 ) − C(u1 , v2 ) + C(u1 , v1 ) ≥ 0, for all 0 ≤ u1 ≤ u2 ≤ 1 and 0 ≤ v1 ≤ v2 ≤ 1 ,
4
C(u, v) ≤ min {u, v} ,
5
C(u, v) ≥ max {u + v − 1, 0} .
D-B. POUGAZA & al. ()
MAXIMUM & COPULA
MaxEnt 2010
5 / 21
OVERVIEW ABOUT COPULAS IN STATISTICS
SKLAR’s THEOREM (1959) Let F be a two-dimensional distribution function with marginal distributions functions F1 and F2 . Then there exists a copula C such that :
(1)
F (x, y) = C(F1 (x), F2 (y)).
If the marginal functions are continuous, then the copula C is unique, and is given by C(u, v) = F (F1−1 (u), F2−1 (v)).
(2)
Otherwise C is uniquely determined on Ran (F1 ) × Ran (F2 ) . Conversely for any univariate distribution functions F1 and F2 and any copula C, the function F is a two-dimensional distribution function with marginals F1 and F2 , given by (1). D-B. POUGAZA & al. ()
MAXIMUM & COPULA
MaxEnt 2010
6 / 21
OVERVIEW ABOUT COPULAS IN STATISTICS
DIRECT INVERSION METHOD
f (x, y) = f1 (x)f2 (y)c(F1 (x), F2 (y))
f F1−1 (u), F2−1 (v) c(u, v) = f1 F1−1 (u) f2 F2−1 (v)
D-B. POUGAZA & al. ()
MAXIMUM & COPULA
MaxEnt 2010
7 / 21
OVERVIEW ABOUT COPULAS IN STATISTICS
ELLIPTICAL COPULAS 1
Gaussian Copula : with one parameter ρ ΦΣ : cdf of bivariate standard Gaussian. Σ : covariance matrix, with correlation coefficientρ
Cρ (u, v) = ΦΣ Φ−1 (u), Φ−1 (v)
2
Student Copula with two parameters ρ and ν tΣ,ν : cdf of a bivariate Student distribution, Σ : covariance matrix with correlation coefficient ρ, ν : the degree of freedom −1 Cρ,ν (u, v) = tΣ,ν t−1 ν (u), tν (v)
D-B. POUGAZA & al. ()
MAXIMUM & COPULA
MaxEnt 2010
8 / 21
OVERVIEW ABOUT COPULAS IN STATISTICS
ARCHIMEDEAN COPULAS For ϕ a non increasing and convex function, where ϕ(0) = ∞, ϕ(1) = 0 :
C(u, v) = ϕ−1 (ϕ(u) + ϕ(v)) . Important : Multivariate dependence is captivated by an univariate function. Some examples : 1 Clayton Copula (1978) : with one non-null parameter α ∈ [−1, ∞) The generator ϕ(t) =
1 α
t−α − 1
h −1 i C(u, v; α) = u−α + v −α − 1 α + 2
Gumbel Copula (1960) : with one parameter 0 < α ≤ 1. The generator ϕ(t) = ln(1 − α ln(t))
Cα (u, v) = uv exp (−α ln u ln v) . D-B. POUGAZA & al. ()
MAXIMUM & COPULA
MaxEnt 2010
9 / 21
MAXIMUM ENTROPIES COPULAS
Maximum Entropies Copulas Problem : Given the two marginals f1 (x) and f2 (y) find the joint pdf f (x, y) Solution : Select the solution which maximizes an entropy Mathematics Z: ZMaximize J1 (f ) = −
f (x, y) ln f (x, y) dx dy,
subject to C 1 C2 C 3
D-B. POUGAZA & al. ()
Z f (x, y) dy = f1 (x),
∀x
f (x, y) dx = f2 (y), ZZ : f (x, y) dx dy = 1.
∀y
: Z :
MAXIMUM & COPULA
MaxEnt 2010
10 / 21
MAXIMUM ENTROPIES COPULAS
Differents Entropies Expressions
R´enyi, Burg and Tsallis entropies Z Z 1 ln f q (x, y) dx dy , q > 0 and q 6= 1, 1−q ZZ J3 (f ) = ln f (x, y) dx dy, ZZ 1 q J4 (f ) = 1− f (x, y) dx dy q > 0 and q 6= 1. 1−q J2 (f ) =
D-B. POUGAZA & al. ()
MAXIMUM & COPULA
MaxEnt 2010
11 / 21
MAXIMUM ENTROPIES COPULAS
Lagrange Multipliers Technique
ZZ L(f, λ0 , λ1 , λ2 ) = Ji (f ) + λ0 1 − f (x, y)dxdy Z Z + λ1 (x) f1 (x) − f (x, y)dy dx Z Z + λ2 (y) f2 (y) − f (x, y)dx dy, Solution ∂L/∂f = 0 ∂L/∂λ = 0 0 ∂L/∂λ 1 =0 ∂L/∂λ2 = 0.
D-B. POUGAZA & al. ()
MAXIMUM & COPULA
MaxEnt 2010
12 / 21
MAXIMUM ENTROPIES COPULAS
Shannon’s Entropy
f (x, y) = f1 (x)f2 (y) f (x1 , . . . , xn ) =
n Y
fi (xi )
i=1
Z F (x1 , . . . , xn ) =
Z ...
0
D-B. POUGAZA & al. ()
x1
0
n xn Y
fi (si )
i=1
MAXIMUM & COPULA
n Y
dsi ,
0 ≤ xi ≤ 1.
i=1
MaxEnt 2010
13 / 21
MAXIMUM ENTROPIES COPULAS
Tsallis’ entropy index q = 2 Pdf : f (x, y) = [f1 (x) + f2 (y) − 1]+ . " n # X f (x1 , . . . , xn ) = fi (xi ) − n + 1 i=1
+
Cdf : Z
x
Z
0
0
F (x, y) = y F1 (x) + x F2 (y) − x y, F (x1 , . . . , xn ) =
n X i=1
D-B. POUGAZA & al. ()
Fi (xi )
n Y
xj + (1 − n)
j=1 j6=i
MAXIMUM & COPULA
y
f (s, t) ds dt
F (x, y) =
0 ≤ x, y ≤ 1. n Y
xi ,
0 ≤ xi ≤ 1.
i=1
MaxEnt 2010
14 / 21
NEW FAMILIES OF COPULAS
Direct Inversion Method Given the expressions of F (x, y) and F1 (x), F2 (y), the expressions of copula becomes : C(u, v) = u F2−1 (v) + v F1−1 (u) − F1−1 (u) F2−1 (v) + Multivariate case :
n X
C(u1 , . . . , un ) =
D-B. POUGAZA & al. ()
i=1
ui
n Y
Fj−1 (uj ) + (1 − n)
j=1 j6=i
MAXIMUM & COPULA
n Y i=1
Fi−1 (ui ) +
MaxEnt 2010
15 / 21
NEW FAMILIES OF COPULAS
Beta Distribution Marginals One great family of distributions defined on [0, 1] : f1 (x) =
1 xa1 −1 (1 − x)b1 −1 B(a1 , b1 )
f2 (y) =
1 y a2 −1 (1 − y)b2 −1 , B(a2 , b2 )
where Z B(ai , bj ) =
1
tai −1 (1 − t)bj −1 dt, 0 ≤ x, y ≤ 1 and ai , bj > 0 .
0
Interesting particular cases : case 1 : ai > 0 , bj = 1 case 2 : ai = bj = 1/2 D-B. POUGAZA & al. ()
MAXIMUM & COPULA
MaxEnt 2010
16 / 21
NEW FAMILIES OF COPULAS
New Families of copulas case 1 : ai > 0 , bj = 1 ( 1 f1 (x) = a1 xa1 −1 → F1 (x) = xa1 → F1−1 (u) = u a1
)
1
f2 (y) = a2 y a2 −1 → F2 (y) = y a2 → F2−1 (v) = v a2 F (x, y; a1 , a2 ) = y xa1 + x y a2 − x y,
0 ≤ x, y ≤ 1.
The corresponding copula : 1
1
1
1
C(u, v; a1 , a2 ) = u v a2 + v u a1 − u a1 v a2 is well defined for appropriate values of a1 , a2 and for almost u, v in [0, 1]. 1 If a1 = a2 = , a C(u, v; a) = (u v)a (u1−a ⊗1 v 1−a ) 1
where u ⊗a v = [ua + v a − 1] a is the generalized product. D-B. POUGAZA & al. ()
MAXIMUM & COPULA
MaxEnt 2010
17 / 21
NEW FAMILIES OF COPULAS
New Families of copulas Case 2 : ai = bj = 1/2 √ 2 1 → F1 (x) = arcsin( x) → F1−1 (u) = sin2 ( π2 u) f1 (x) = p π π x(1 − x) 1 2 √ → F2 (y) = arcsin( y) → F2−1 (v) = sin2 ( π2 v) f2 (y) = p π π y(1 − y) √ 2x 2y √ arcsin( x) + arcsin( y) − x y, π π The corresponding copula : F (x, y) =
C(u, v) = u sin2 (
0 ≤ x, y ≤ 1.
πv πu πu πv ) + v sin2 ( ) − sin2 ( ) sin2 ( ) 2 2 2 2
is well defined for all u, v in [0, 1]. D-B. POUGAZA & al. ()
MAXIMUM & COPULA
MaxEnt 2010
18 / 21
CONCLUSION
New families of copula, other examples are in investigation Link betwen copula & Tomography Shannon : f (x, y) = f1 (x)f2 (y) : Multiplicative Backprojection Tsallis : f (x, y) = [f1 (x) + f2 (y) − 1]+ : Backprojection Method
D-B. POUGAZA & al. ()
MAXIMUM & COPULA
MaxEnt 2010
19 / 21
Thanks to Dr. Durante Fabrizio Pr. Christian Genest Dr. Jean-Fran¸cois Bercher Pr. Christophe Vignat
D-B. POUGAZA & al. ()
MAXIMUM & COPULA
MaxEnt 2010
20 / 21
BIBLIOGRAPHY
Link between Copula and Tomography Pattern Recognition Letters, Elsevier, 2010. D-B. Pougaza , A. Mohammad-Djafari & J-F. Bercher Utilisation de la notion de copule en tomographie GRETSI 2009, Dijon France. D-B. Pougaza , A. Mohammad-Djafari & J-F. Bercher Copula and Tomography VISSAP 2009 , Lisbon, Portugal. A. Mohammad-Djafari & D-B. Pougaza Fonctions de r´epartition ` a n dimensions et leurs marges Publications de l’Institut de Statistique de L’Universit´e de Paris 8, 1959 Abe Sklar
D-B. POUGAZA & al. ()
MAXIMUM & COPULA
MaxEnt 2010
21 / 21
