This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and education use, including for instruction at the authors institution and sharing with colleagues. Other uses, including reproduction and distribution, or selling or licensing copies, or posting to personal, institutional or third party websites are prohibited. In most cases authors are permitted to post their version of the article (e.g. in Word or Tex form) to their personal website or institutional repository. Authors requiring further information regarding Elsevier’s archiving and manuscript policies are encouraged to visit: http://www.elsevier.com/copyright

Author's personal copy

Pattern Recognition Letters 31 (2010) 2258–2264

Contents lists available at ScienceDirect

Pattern Recognition Letters journal homepage: www.elsevier.com/locate/patrec

Link between copula and tomography Doriano-Boris Pougaza a,*, Ali Mohammad-Djafari a, Jean-François Bercher a,b a b

Laboratoire des Signaux et Systèmes, UMR 8506 (CNRS-SUPELEC-Univ Paris Sud 11) SUPELEC, Plateau de Moulon, 3 rue Joliot Curie, 91192 Gif-sur-Yvette Cedex, France Laboratoire d’Informatique Gaspard Monge, UMR 8049, CNRS-ESIEE-Université Paris-Est, 5 bd Descartes, 77454 Marne-la-Vallée Cedex 2, France

a r t i c l e

i n f o

a b s t r a c t

Article history: Available online 7 May 2010 Keywords: Copula Tomography Joint and marginal distributions Image reconstruction Additive and multiplicative backprojection Maximum entropy

An important problem in statistics is to determine a joint probability distribution from its marginals and an important problem in Computed Tomography (CT) is to reconstruct an image from its projections. In the bivariate case, the marginal probability density functions f1(x) and f2(y) are related to their joint distribution f(x, y) via horizontal and vertical line integrals. Interestingly, this is also the case of a very limited angle X-ray CT problem where f(x, y) is an image representing the distribution of the material density and f1(x), f2(y) are the horizontal and vertical line integrals. The problem of determining f(x, y) from f1(x) and f2(y) is an ill-posed undetermined inverse problem. In statistics the notion of copula is exactly introduced to characterize all the possible solutions to the problem of reconstructing a bivariate density from its marginals. In this paper, we elaborate on the possible link between copula and CT and try to see whether we can use the methods used in one domain into the other.  2010 Elsevier B.V. All rights reserved.

forward problem given by (1), whereas Fig. 1(b) illustrates the inverse problem. As we will see later, all functions in the form of

1. Introduction The word copula originates from the Latin meaning link, chain, union. In statistical literature, according to the seminal result in the copula’s theory stated by Sklar (1959), a copula is a function that connects a multivariate distribution function to its univariate marginal distributions. There is an increasing interest concerning copulas, widely used in Financial Mathematics and in modelling of Environmental Data (Joe, 1994; Genest and Favre, 2007). Recently, in Computational Biology, copulas were used for DNA analysis (Kim et al., 2008). Copula appears to be a powerful tool to model the structure of dependence (Zhang et al., 2006; Kallenberg, 2008). Copulas are useful for constructing joint distributions, particularly with non-Gaussian random variables (Joe, 1997). In 2D case, interpreting the joint probability density function f(x, y) as an image and its marginal probability densities f1(x) and f2(y) as horizontal and vertical line integrals:

f1 ðxÞ ¼

Z

f ðx; yÞdy and f 2 ðyÞ ¼

Z

f ðx; yÞdx;

ð1Þ

we see that the problem of determining f(x, y) from f1(x) and f2(y) is an ill-posed (inverse) problem (Hadamard, 1902; Tarantola, 2004; Idier, 2008). It is a well known fact that while a distribution has a unique set of marginals, the converse is not necessarily true. That is, many distributions may share a common subset of marginals. In general, it is not possible to uniquely reconstruct a distribution from its marginals. This is illustrated in Fig. 1: Fig. 1(a) shows the * Corresponding author. Tel.: +33 1 69 85 17 43. E-mail address: [email protected] (D.-B. Pougaza). 0167-8655/$ - see front matter  2010 Elsevier B.V. All rights reserved. doi:10.1016/j.patrec.2010.05.001

f ðx; yÞ ¼ f1 ðxÞf2 ðyÞcðF 1 ðxÞ; F 2 ðyÞÞ;

ð2Þ

where F1(x), F2(y) are the marginal cumulative distributions functions (cdf’s) and c is any copula density function, are solutions of this problem. Interestingly, this is very similar to the probability density function (pdf) reconstruction problem considered in (Pan et al., 2001), where a special copula was designed. The approach in (Pan et al., 2001) could certainly be interpreted using the results presented here. In 1917, Johann Radon introduced the Radon transform (RT) (Radon, 1917, 1986) which was later used in CT (Cormack, 1963). Indeed, if we denote by f(x, y), the spatial distribution of the material density in a section of the body, a very simple model that relates a line of the radiography image p(r, h) at direction h to f(x, y) is given by the Radon transform:

pðr; hÞ ¼

Z

f ðx; yÞdl

Lr;h

¼

Z Z

f ðx; yÞdðr  x cos h  y sin hÞdx dy;

ð3Þ

R2

where Lr,h = {(x, y): r = x cosh + y sinh} and d is the Dirac’s delta function. The experimental setup is presented in Fig. 2. If now we consider only the horizontal h = 0 projection and the vertical h = p/2 projection, we see easily the connexion between these two problems. The main object of this paper is to explore in more details these relations, and exploit the similarity between the two problems as a new approach to image reconstruction in Computed Tomography.

Author's personal copy

2259

D.-B. Pougaza et al. / Pattern Recognition Letters 31 (2010) 2258–2264

Definition. Bivariate copula: A bivariate copula, or shortly a copula is a function C from [0, 1]2 to [0, 1] with the following properties:  "u, v 2 [0, 1], C(u, 0) = 0 = C(0, v),  "u, v 2 [0, 1], C(u, 1) = u and C(1, v) = v and  C(u2, v2)  C(u2, v1)  C(u1, v2) + C(u1, v1) P 0 for all u1, u2,

v1, v2 2 [0, 1] such that u1 6

u2,

v1 6 v2.

Theorem 2.1. Sklar’s Theorem (for proof, see Schweizer and Sklar, 1983): Let F be a two-dimensional distribution function with marginal distributions functions F1 and F2. Then there exists a copula C such that: Fig. 1. Forward and inverse problems.

Fðx; yÞ ¼ CðF 1 ðxÞ; F 2 ðyÞÞ:

ð4Þ

Conversely, for any univariate distribution functions F1 and F2 and any copula C, the function F is a two-dimensional distribution function with marginals F1 and F2, given by (4). Lemma 2.2. If the marginal functions are continuous, then the copula C is unique, and is given by

  1 Cðu; v Þ ¼ F F 1 1 ðuÞ; F 2 ðv Þ :

ð5Þ

Definition. Copula density: From (4) and differentiating (5) gives the density of a copula

  1 f F 1 1 ðuÞ; F 2 ðv Þ @2C   ; cðu; v Þ ¼ ¼  @u@ v f1 F 1 ðuÞ f2 F 1 ðv Þ 1 2

ð6Þ

and thus

f ðx; yÞ ¼ f1 ðxÞf2 ðyÞcðF 1 ðxÞ; F 2 ðyÞÞ: Fig. 2. X-ray Computed Tomography: 2D parallel geometry.

The rest of this paper is organized as follows: In Section 2, we present a summary of the necessary definitions and properties of copulas and highlight methods to generate a copula. In Section 3, we present the main tomographic image reconstruction methods based on the Radon inversion formula. In Section 4, we will be in the heart of the link and relations between the notions of these two previous sections. Sections 5 and 6 are devoted to details concerning our method. Some preliminary results from our CopulaTomography Matlab package are shown.

ð7Þ

An usual simple example is the product or independent copula:

Pðu; v Þ ¼ uv ! cðu; v Þ ¼ 1;

ðu; v Þ 2 ½0; 12 :

ð8Þ

Property 2.3. Any copula C(u, v), satisfies the inequality

Wðu; v Þ 6 Cðu; v Þ 6 Mðu; v Þ;

ð9Þ

where the Fréchet–Hoeffding upper bound copula M(u, v) (or comonotonicity copula) is:

Mðu; v Þ ¼ minðu; v Þ;

ð10Þ

2. Copula

and the Fréchet–Hoeffding lower bound W(u, v) (or countermonotonicity copula) is:

In this section, we give a few definitions and properties of copulas that we need in the rest of the paper. For more details about this section we refer to Nelsen (2006). First, we note by F(x, y) a bivariate cumulative distribution function (cdf), by f(x, y) its bivariate probability density function (pdf), by F1(x), F2(y) its marginal cdf’s and f1(x), f2(y) their corresponding pdf’s with their classical relations:

Wðu; v Þ ¼ maxfu þ v  1; 0g ðu; v Þ 2 ½0; 12 :

Z x Z y @ 2 Fðx; yÞ ; Fðx; yÞ ¼ f ðs; tÞds dt; f ðx; yÞ ¼ @x@y 1 1 Z x Z y F 1 ðxÞ ¼ f1 ðsÞds ¼ Fðx; 1Þ; F 2 ðyÞ ¼ f2 ðtÞdt ¼ Fð1; yÞ; 1 1 Z Z dF 1 ðxÞ dF 2 ðyÞ ¼ f ðx; yÞdy; f 2 ðyÞ ¼ ¼ f ðx; yÞdx: f1 ðxÞ ¼ dx dy

ð11Þ

Generating copulas by the inversion method: A straightforward method is based directly on Sklar’s theorem. Given F(x, y) the joint cdf of two random variables X, Y and F1(x) and F2(y) their marginal cdf’s, all assumed to be continuous. The corresponding copula can be constructed by using the unique inverse transformations (Quantile 1 transform) x ¼ F 1 1 ðuÞ; y ¼ F 2 ðv Þ, and the Eq. (5) where u, v are uniform on [0, 1]. 2.1. Archimedean copulas The Archimedean copulas (see Nelsen, 2006, p. 109) form an important class of copulas which generalise the usual copulas.

Author's personal copy

2260

D.-B. Pougaza et al. / Pattern Recognition Letters 31 (2010) 2258–2264

Theorem 2.4. Let u be a continuous, strictly decreasing function from [0, 1] to [0, 1] such that u(1) = 0, and let u[1] be the pseudo-inverse1 of u. Let C be the function from [0, 1]2 to [0, 1] given by

Cðu; v Þ ¼ u½1 ðuðuÞ þ uðv ÞÞ:

ð12Þ

Then C is a copula if and only if u is convex.

e X; hÞ ¼ sgnðXÞPðX; hÞ ¼ sgnðXÞXPðX; hÞ ¼ jXjPðX; hÞ: Pð Finally the filtered backprojection which is currently the most used reconstruction method is performed by the following formula:

f ðx; yÞ ¼ BHDpðr; hÞ ¼ BF 1 1 jXjF 1 pðr; hÞ;

ð18Þ

that is

Archimedean copulas are in the form (12) and the function u is called the generator of the copula. u is a strict generator if u(0) = 1, then u[1] = u1 and

Cðu; v Þ ¼ u1 ðuðuÞ þ uðv ÞÞ:

ð13Þ

Property 2.5. Any Archimedean copula C satisfies the following algebraic properties:  C(u, v) = C(v, u) meaning that C is symmetric;  C(C(u, v), w) = C(u, C(v, w));  If a > 0, then au is again a generator of C.

4. Link between copula and tomography

There are many families of Archimedean copulas constructed from different generators ua with a suitable parameter a. For example ua ðtÞ ¼ a1 ðta  1Þ and ua(t) = ln(1  alnt) yield successively to Clayton copula Ca(u, v) = [max (ua + va  1, 0)]1/a and Gumbel-Hougaard copula Ca(u, v) = uv exp (a ln u ln v). 3. Tomography In 2D, the mathematical problem of tomography is to determine the bivariate function f(x, y) from its line integrals p(r, h) (see Eq. (3)). Radon has shown that this problem has a unique solution if we know p(r, h) for all h 2 [0, p] and all r 2 R, then f(x, y) can be computed by the inverse Radon transform (for details, see Deans, 1983):

  Z p Z þ1 @pðr;hÞ 1 @r f ðx; yÞ ¼  2 dr dh: 2p 0 1 r  x cos h  y sin h

In X-ray CT, if we have a great number of projections uniformly distributed over the angles interval [0, p], the filtered backprojection (FBP) or even the simple backprojection (BP) image are good solutions to the inverse CT problem (Kak and Slaney, 1988). But, when we are restricted to only two projections, the FBP or BP images are not correct reconstruction (Natterer, 2001; Markoe, 2006; Herman et al., 2007).

ð14Þ

Now, let consider the particular case where we have only two projections h = 0 and h = p/2. Then

Z f ðx; yÞdðr  xÞdx dy ¼ f ðr; yÞdy; Z Z Z f ðx; yÞdðr  yÞdx dy ¼ f ðx; rÞdx pp=2 ðrÞ ¼

p0 ðrÞ ¼

Z Z

and if we let f1 = p0 and f2 = pp/2 we can deduce the following new methods, inspired by the reconstruction approaches in CT, for the inverse problem that consists in determining the probability density f(x, y) from its marginals f1(x) and f2(y): Backprojection:

f ðx; yÞ ¼

1 ðf1 ðxÞ þ f2 ðyÞÞ: 2

Filtered Backprojection:

However, if the number of projections is limited, then the problem is ill-posed and the problem has an infinite number of solutions. To present briefly the main classical methods in CT, we start by decomposing the inverse RT in the following parts:

1 f ðx; yÞ ¼ 2

@pðr; hÞ h ðrÞ ¼ ; Derivative D : p @r

f ðx; yÞ ¼

ð15Þ

~ðr 0 ; hÞ ¼ Hilbert Transform H : p

1

p

p:v:

Z

þ1

1

ðr; hÞ p dr r  r0

ð16Þ

ð19Þ

Z

@f1 ðx0 Þ 0 @x dx 0 x x

þ

Z

@f2 ðy0 Þ 0 @y dy 0 y y

! ;

ð20Þ

which can also be implemented in the Fourier domain as it follows:

Z  0 eþjnx jnj ejnx f1 ðx0 Þdx0 dn Z  Z 1 0 eþjmy jmj ejmy f2 ðy0 Þdy0 dm: þ 2 1 2

Z

where p.v. is the Cauchy principal value.

Backprojection B : f ðx; yÞ ¼

1 2p

Z p

5. How to use copula in Tomography

~ðr 0 ¼ x cos h þ y sin h; hÞdh: p

0

ð17Þ Then defining the one dimensional inverse Fourier transform F 1 1 by

Inverse Fourier F 1 1 : PðX; hÞ ¼

Z

pðr; hÞ exp½jXrdr:

Using the properties of the Fourier transform F 1 and the derivative D, from (15) we have:

PðX; hÞ ¼ XPðX; hÞ; the relation between H and F 1 yields: 1



1 u½1 ðtÞ ¼ u ðtÞ; 0 6 t 6 uð0Þ; uð0Þ 6 t 6 1: 0;

The definition and the notion of copula give us the possibility to propose new X-ray CT methods. Let first consider the case of two projections. In this case, immediately, we can propose a first use which corresponds to the case of independent copula, as given in (8). We call this method Multiplicative Backprojection (MBP) (see Pougaza et al., 2009) MBP:

f ðx; yÞ ¼ f1 ðxÞf2 ðyÞ:

ð21Þ

If we compare the Eqs. (19)–(21) instead of the classical BP which is an additive operation or Additive Backprojection, the name MBP comes naturally. In Fig. 3 we give comparisons of BP and MBP. As we can see on the image original 1, at least the image obtained by MBP is better than the one obtained by BP and it satisfies exactly the marginals.

Author's personal copy

2261

D.-B. Pougaza et al. / Pattern Recognition Letters 31 (2010) 2258–2264

We may still do better if we choose another copula rather than the independent copula, by proposing the following method that we call Copula Backprojection (CopBP). CopBP:

f ðx; yÞ ¼ f1 ðxÞf2 ðyÞcðF 1 ðxÞ; F 2 ðyÞÞ;

ð22Þ

where c(u, v) is a parametrized copula. Here the main question is how to choose an appropriate copula for the particular application. This problem can be thought as a

way to introduce some prior information, just enough to choose an appropriate family of copula. For example if we know that the joint density has only one mode, and can be approximated by a bivariate Gaussian, U1 denoting the inverse of the standard Gaussian cdf, then we can use a Gaussian copula whose expression is given by

C q ðu; v Þ ¼

A 2p

Z

U1 ðuÞ

1

Z

U1 ðv Þ

1

exp



 ðs2  2qst þ t2 Þ ds dt; 2ð1  q2 Þ

Fig. 3. Comparison between BP, FBP, MBP and CopBP on two synthetic examples. This shows the improvement obtained with MBP and CopBP methods compared to standard Back Projection (BP) or Filtered Back Projections (FBP). It is noted that marginals of the BP and FBP reconstructions differ from the original data while marginals of MBP/CopBP perfectly agree with initial data.

Author's personal copy

2262

D.-B. Pougaza et al. / Pattern Recognition Letters 31 (2010) 2258–2264

where A = (1  q2)1/2 and q = 0 correspond to copulas P(u, v) in Eq. (8) and where q = 1, + 1 give respectively the copulas W(u, v) and M(u, v) in Eqs. (11) and (10). The corresponding Gaussian copula density is:

6.1. Problem’s formulation

cq ðu; v Þ

^f :¼ maximize fJðf Þg subject to ð24Þ:

Finally, the function f(x, y) we are looking for, can be written as:

Since the constraints are linear, if we choose a criterion which is a concave function, then there is a unique solution to the problem. Many entropies functional can serve as an objective function, e.g. (Shannon, 1948; Renyi, 1961; Mohammad-Djafari and Idier, 1991; Kapur and Kesavan, 1992; Pasha and Mansoury, 2008; Yu et al., 2009):

( )  A2  2 2 1 1 1 1 : ¼ A exp ðqU ðuÞÞ  2qU ðuÞU ðv Þ þ ðqU ðv ÞÞ 2

   2 2 q x  2qxy þ q2 y2 f ðx; yÞ ¼ Af1 ðxÞf2 ðyÞ exp  ; 2ð1  q2 Þ

ð23Þ

where U1(u) = x and U1(v) = y. Fig. 3 presents CopBP reconstructions obtained using this Gaussian copula. We see the interest of such an approach compared to standard BP. The particular reconstruction (23) is parametrized by the correlation coefficient q which is an hyperparameter of the reconstruction process. With a value q = 0, that is with no correlations, the CopBP method reduces to the multiplicative MBP method. The specification of q corresponds to the encoding of some prior information in the reconstruction procedure which helps to improve the quality of the reconstruction. For example, from physical or physiological knowledge, or from the experimental setting, the general orientation of the underlying object is known. Another situation is the case where a mean template for the object is available, for example as a result of previous experiments. The hyperparameter q may also be estimated from additional data. For instance, using some additional measurements, e.g. a third (may be partial) projection, it is easy to select the best value of q which minimizes the distance between the actual projection and the one computed according to the model. The general incorporation of prior information or additional data, with the automatic determination of the hyperparameters is a work in progress which is out of the scope of this letter. What we want to emphasize through this simple example is the interest of the CopBP approach for including a such simple prior as the main orientation of the object, that leads to an noticeable improvement of the reconstruction. This suggests that copula-based approaches have a potential in the field of image reconstruction from projections.

Among all possible f(x, y) satisfying the constraints (24) choose the one which optimizes a criterion J(f), i.e.:

1. 2. 3. 4. 5.

RR J 1 ðf Þ ¼  jf ðx; yÞj2 dx dy (-Energy or L2-norm). RR J 2 ðf Þ ¼  f ðx; yÞ ln f ðx; yÞdx dy (Shannon entropy). RR J 3 ðf Þ ¼ ln yÞdx dy (Burg entropy).  f ðx; RR a J 4 ðf Þ ¼ 11 a 1  f ðx; yÞdx dy (Tsallis entropy). RR a J 5 ðf Þ ¼ 11 a ln f ðx; yÞdx dy, (Rényi entropy).

Our main contribution here is to find the generic expression for the solution of these criteria. The main tool is the classical Lagrange multipliers technique which consists in defining the Lagrangian functional

  Z Z Lg ðf ; k0 ; k1 ; k2 Þ ¼ Jðf Þ þ k0 1  f ðx; yÞdx dy   Z Z þ k1 ðxÞ f1 ðxÞ  f ðx; yÞdy dx   Z Z þ k2 ðyÞ f2 ðyÞ  f ðx; yÞdx dy; and find its stationnary point which is defined as the solution of the following system of equations:

8 < @Lg ðf ;k0 ;k1 ;k2 Þ ¼ 0; @f

: @Lg ðf ;k0 ;k1 ;k2 Þ ¼ 0: @ki

Here, we give the final expression, assuming that the integrals converge: 1. ^f ðx; yÞ ¼  12 ðk1 ðxÞ þ k2 ðyÞ þ k0 Þ (-Energy). 2. ^f ðx; yÞ ¼ expðk1 ðxÞ  k2 ðyÞ  k0 Þ (Shannon entropy). 1 (Burg entropy). 3. ^f ðx; yÞ ¼ k1 ðxÞþk2 ðyÞþk0

6. Maximum entropy copulas

1 4. ^f ðx; yÞ ¼ 1a a ðk1 ðxÞ þ k2 ðyÞ þ k0 Þa1 (Tsallis and Renyi entropies),

The selection of a particular copula is a difficult task. We propose here to look at this ill-posed inverse problem using the maximum entropy (ME) method. The principle of ME was first expounded by Jaynes in two seminal papers in 1957 (Jaynes, 1957a,b). It is the way to assign a probability distribution to a quantity on which we have partial information. The classical ME problem is to assign a probability law to a quantity on which we only know a few moments. Here, the problem is a bit different, because the partial information we have is not in terms of moments but in the form of the following constraints:

where k1(x), k2(y) and k0 are obtained by replacing these expressions in the constraints (24) and solving the resulting system of equations. When solving the Lagrangian functional equation which is concave in f, we assume that there exists a feasible f > 0 with finite entropy. The results for Tsallis and Renyi entropies leads to the same family of distribution depending on a due to the monotonicity property of the logarithm function. For the two criteria Energy and Shannon entropy, we can find analytical solutions for k1(x), k2(y) and k0. For -Energy, we obtain: R R k1 ðxÞ ¼ 2f 1 ðxÞ þ k1 ðxÞdx þ 2; k2 ðyÞ ¼ 2f 2 ðyÞ þ k2 ðyÞdy þ 2 R R and k0 ¼ 2  k1 ðxÞdx  k2 ðyÞdy, which finally gives:

8 > < C1 C2 > : C3

:

R R

f ðx; yÞdy ¼ f1 ðxÞ;

8x;

f ðx; yÞdx ¼ f2 ðyÞ; 8y; RR : f ðx; yÞdx dy ¼ 1: :

ð24Þ

Hence, the goal is to find the most general copula, in the ME sense, compatible with available information, that is, with the marginals/ projections at hands.

^f ðx; yÞ ¼ f1 ðxÞ þ f2 ðyÞ  1:

ð25Þ

This is nothing else but the standard Backprojection mechanism (up to scale factor and a constant). Hence, the Backprojection method can be easily interpreted as a minimum norm solution. For the Shannon entropy, we get:

Author's personal copy

2263

D.-B. Pougaza et al. / Pattern Recognition Letters 31 (2010) 2258–2264

R

R

k1 ðxÞ ¼  lnðf1 ðxÞ k1 ðxÞdxÞ; k2 ðyÞ ¼  lnðf2 ðyÞ k2 ðyÞdyÞ R R k0 ¼ lnð k1 ðxÞdx k2 ðyÞdyÞ which yields

and

Appendix A. Relation with Shannon entropy in high dimension

^f ðx; yÞ ¼ f1 ðxÞf2 ðyÞ:

ð26Þ

From the n-dimensional version of Sklar’s theorem (Sklar, 1959; Schweizer and Sklar, 1983), we have

This is now the MBP we obtained as associate to an independent copula. Unfortunately, in the cases of Burg, Tsallis and Renyi entropies, it is not possible to find analytical expressions for k0, k1, and k2 as functions of f1 and f2. Consequently a numerical approach is required, see for example (Mohammad-Djafari, 1991). Using Eq. (22) one can write all entropies in terms of copulas. For example, if we denote the Shannon entropy by H(x, y) and the copula entropy by Hc(u, v), then:

Hðx; yÞ ¼ HðxÞ þ HðyÞ þ Hc ðu; v Þ: The previous relation shows that the Shannon entropy of the bivariate distribution is the sum of the entropies provided by each marginal density and the copula entropy. In Appendix, we provide the proof of this result in the multivariate case, which is, to the best of our knowledge, original. This result shall be of interest for multidimensional tomography, especially 3D tomography. Therefore, maximizing the joint entropy, given the marginals, is equivalent to maximize the entropy of the copula Hc(u, v). Since we only have here a domain constraint- the copula is defined on [0, 1]2-, the Shannon maximum entropy copula is uniform, c(u, v) = 1, and we obtain the MBP reconstruction (26). Now, if we look for a Shannon maximum entropy copula with an additional correlation constraint – that is we fix the correlation of the underlying normalized random variables-, then we end with a Gaussian copula, which in turn, lead us to the CopBP method with a Gaussian copula (22). Along these lines, it seems possible to characterize the different families of copula as maximum entropy solutions, possibly incorporating more prior information. More generally, it will also be interesting to characterize the copulas corresponding to the Burg/Rényi ME solutions. Some simulations are reported Fig. 3. The aim of these simulations from our Copula-Tomography package (which can be downloaded from Pougaza and Mohammad-Djafari (2008)) is to illustrate the link between copula in tomography in the case of only two projections. The original 1 image simulated is a Gaussian and the original 2 image is formed by four Gaussians. We performed BP, FBP, MBP and CopBP on these images. We observe that for the MBP and the CopBP, the two projections on the reconstructed images match those from the original images which is not the case for the BP and the FBP. 7. Conclusion The main contribution of this paper is to highlight a link between the notion of copulas in statistics and X-ray CT for small number of projections. This link brings up possible new approaches for image reconstruction in CT. We first presented the bivariate copulas and the image reconstruction problem in CT. We highlight the connexion between the two problems that consist in (i) determining a joint bivariate pdf from its two marginals and (ii) the CT image reconstruction from only two horizontal and vertical projections. We emphasize that in both cases, we have the same inverse problem for the determination of a bivariate function (an image) from the line integrals. We have indicated the potential of copula-based reconstruction methods, introducing the MBP (Multiplicative Back Projection) and CopBP (Copula Back Projection) methods. Current work addresses the characterization of family of copulas as well as the estimation of copulas parameters in the reconstruction process. We also intend to improve the results by accounting for more projections in the method, while keeping the copula approach.

Fðx1 ; . . . ; xn Þ ¼ C ðF 1 ðx1 Þ; . . . ; F n ðxn ÞÞ:

ðA:1Þ

Now taking the partial derivative in Eq. (A.1), since ui = Fi(xi) it follows that the probability density function can be expressed by

f ðx1 ; . . . ; xn Þ ¼ cðu1 ; . . . ; un Þ

n Y

fi ðxi Þ:

ðA:2Þ

i¼1

Notice also that the differentials dui = dFi(xi) = fi(xi)dxi, and Q Q dx ¼ ni¼1 dxi . Hence du ¼ ni¼1 fi ðxi Þdxi , and we remark that

Z I

cðuÞ n1

n Y

duj ¼

j¼1 j–i

Z Rn1

n f ðx1 ; . . . ; xn Þ Y fi ðxi Þ dxj ¼ ¼ 1: fi ðxi Þ fi ðxi Þ j¼1 j–i

From the Shannon entropy and using the expression of f(x) in Eq. (A.2): Proof HðxÞ ¼  ¼

Z

cðuÞ Rn

cðuÞ

Z

n Z X Rn

B BcðuÞ @

n Y i¼1

n X

j¼1 j–i

n Y

dxi

i¼1

fi ðxi Þdxi

i¼1

Yn

!

ln fi ðxi Þ

i¼1 n Y

! fi ðxi Þ dx

1 C fj ðxj Þdxj C Afi ðxi Þ ln fi ðxi Þdxi 

Z

cðuÞ ln cðuÞdu In

1 Z  Yn C B duj C fi ðxi Þ ln fi ðxi Þdxi þ Hc ðuÞ @ n1 cðuÞ A

n BZ X i¼1

I

n Z X i¼1

¼

fi ðxi Þ

0

0

¼

!

cðuÞ ln cðuÞ

i¼1

¼

n Y i¼1

Rn

¼

! fi ðxi Þ ln cðuÞ

i¼1

Z Rn



n Y

n X

R

j¼1 j–i

fi ðxi Þ ln fi ðxi Þdxi þ Hc ðuÞ

R

Hðxi Þ þ Hc ðuÞ:



i¼1

R Eq. (A.3) shows that the entropy HðxÞ ¼  Rn f ðxÞ ln f ðxÞdx of the joint multivariate distribution is the sum of the entropies provide by each marginal density H(xi) and the copula entropy Hc(u). References Cormack, A.M., 1963. Representation of a function by its line integrals with some radiological application. J. Appl. Phys. 34, 2722–2727. Deans, S., 1983. The Radon Transform and Some of its Applications. A WileyInterscience Publication, New York. Genest, C., Favre, A.-C., 2007. Everything you always wanted to know about copula modeling but were afraid to ask. J. Hydrologic Eng. 12, 347–368. Hadamard, J., 1902. Sur les problèmes aux dérivées partielles et leur signification physique. Princeton Univ. Bull. 13, 49–52. Herman, G., Kuba, A., Service, S.O., 2007. Advances in Discrete Tomography and its Applications, Birkhäuser. Idier, J., 2008. Bayesian Approach to Inverse Problems, first ed. Wiley-ISTE. Jaynes, E., 1957a. Information theory and statistical mechanics. Phys. Rev. 106 (4), 620–630. Jaynes, E., 1957b. Information Theory and Statistical Mechanics. II. Phys. Rev. 108 (2), 171–190. Joe, H., 1994. Multivariate extreme-value distributions with applications to environmental data. The Can. J. Statist. 22, 47–64. Joe, H., 1997. Multivariate Models and Dependence Concepts. Chapman & Hall, London. Kak, A., Slaney, M., 1988. Principles of Computerized Tomographic Imaging. Society of Industrial and Applied Mathematics. Kallenberg, W.C., 2008. Modelling dependence. Insur. Math. Econ. 42, 127–146.

Author's personal copy

2264

D.-B. Pougaza et al. / Pattern Recognition Letters 31 (2010) 2258–2264

Kapur, J., Kesavan, H., 1992. Entropy Optimization Principles with Applications. Academic Press, Boston, Tokyo. Kim, J., Jung, Y., Sungur, E., Han, K., Park, C., Sohn, I., 2008. A copula method for modeling directional dependence of genes. BMC Bioinf. 9 (1), 225. Markoe, A., 2006. Analytic Tomography. Cambridge University Press. Mohammad-Djafari, A., 1991. A Matlab program to calculate the maximum entropy distributions. In: Grandy, T.W. (Ed.). Kluwer Academic Publ., pp. 221–233. Mohammad-Djafari, A., Idier, Jérôme, 1991. Maximum likelihood estimation of the Lagrange parameters of the maximum entropy distributions. In: Smith, C.R., Erikson, G.J., Neudorfer, P.O. (Eds.). Kluwer Academic Publ., pp. 131–140. Natterer, F., 2001. The Mathematics of Computerized Tomography. Society for Industrial Mathematics. Nelsen, R., 2006. An Introduction to Copulas. Springer-Verlag. Pan, H., Liang, Z.-P., Huang, T.S., 2001. Estimation of the joint probability of multisensory signals. Pattern Recognition Lett. 22 (13), 1431–1437. Pasha, E., Mansoury, S., 2008. Determination of maximum entropy multivariate probability distribution under some constraints. Appl. Math. Sci. 2 (57), 2843– 2849. Pougaza, D.-B., Mohammad-Djafari, A., 2008. Copula-Tomography Package, .

Pougaza, D.-B., Mohammad-Djafari, A., Bercher, J.-F., 2009. Utilisation de la notion de copule en tomographie, in: XXIIe colloque GRETSI, Dijon, France. Radon, J., 1917. Über die bestimmung von funktionen durch ihre integralwerte längs gewisser mannigfaltigkeiten. Ber. Verh. Säch. Akad. Wiss. Leipzig, Math. Nat. Kl69, 262–277. Radon, J., 1986. On the determination of functions from their integral values along certain manifolds. IEEE Trans. Med. Imaging 5 (4), 170–176. Renyi, A., 1961. On measures of entropy and information, in: Proc. 4th Berkeley Symposium on Mathematical Statist. Probability, vol. 1, pp. 547–561. Schweizer, B., Sklar, A., 1983. Probabilistic Metric Spaces. North Holland, New York. Shannon, C., 1948. A Mathematical Theory of Communication. Bell Syst. Tech. J. 27, 379–432. Sklar, A., 1959. Fonctions de repartition à n dimensions et leurs marges. Publications de l’Institut de Statistique de L’Université de Paris 8, 229–231. Tarantola, A., 2004. Inverse Problem Theory and Methods for Model Parameter Estimation, first ed. SIAM: Society for Industrial and Applied Mathematics. Yu, D., Deng, L., Acero, A., 2009. Using continuous features in the maximum entropy model. Pattern Recognition Lett. 30 (14), 1295–1300. Zhang, D., Martin, T., Peng, L., 2006. Nonparametric estimation of the dependence function for a multivariate extreme-value distribution. J. Multivar. Anal. 99, 577–588.