MINIMUM RISK EQUIVARIANT MINIMUM DIVERGENCE ESTIMATES FOR MOMENT CONDITION MODELS 2 µ MICHEL BRONIATOWSKI1 , JANA JURECKOVÁ AND AMOR KEZIOU3

1. Introduction The semiparametric moment condition models are de…ned through estimating equations E (fj (X;

T ))

= 0 for all j = 1; : : : ; `;

where E( ) denotes the mathematical expectation, X 2 Rm is a random vector, T 2 Rd > is the unknown true value of the parameter of interest, and f (x; ) := (f1 (x; ); : : : ; f` (x; )) is some speci…ed measurable R` -valued function de…ned on Rm . Such models are popular in statistics and econometrics, see e.g., Qin and Lawless (1994), Haberman (1984), Sheehy (1987), McCullagh and Nelder (1983), Owen (2001) and the references therein. Denoting PX ( ) the probability distribution of the random vector X, then the above estimating equations can be written as Z f (x; T ) dPX (x) = 0: Rm

Let M be the collection of all signed …nite measures (s.f.m.) Q on the Borel -…eld (Rm ; B(Rm )) such that Q(Rm ) = 1. The submodel M , associated to a given value 2 , consists of all s.f.m.’s Q 2 M 1 satisfying ` linear constraints induced by the vector valued function f ( ; ) := (f1 ( ; ); : : : ; f` ( ; ))> , namely, Z 1 M := Q 2 M such that fj (x; ) dQ(x) = 0; 8j = 1; : : : ; ` Rm Z 1 Q 2 M such that f (x; ) dQ(x) = 0 ; = 1

Rm

with `

d. The statistical model which we consider can be written as Z [ [ 1 M := M := Q 2 M such that f (x; ) dQ(x) = 0 : 2

(1)

Rm

2

Let X1 ; : : : ; Xn be an i.i.d. sample of the random vector X 2 Rm with unknown probability distribution PX ( ). The problems of testing the model H0 : PX 2 M, con…dence region and point estimations of T , have been widely investigated in the literature. Hansen (1982) considered generalized method of moments (GMM) in order to estimate T . Hansen et al. (1996) introduced the continuous updating (CU) estimate. Asymptotic con…dence regions for the parameter T have been obtained by Owen (1988) and Owen (1990), introducing the empirical Date: March 8, 2013. 1

2

2 µ MICHEL BRONIATOWSKI1 , JANA JURECKOVÁ AND AMOR KEZIOU3

likelihood (EL) approach. This appraoch has been used, in the context of model (1), by Qin and Lawless (1994) and Imbens (1997) introducing the EL estimate for the parameter T . The recent literature in econometrics focusses on such models; Smith (1997), Newey and Smith (2004) provided a class of estimates called generalized empirical likelihood (GEL) estimates which contains the EL and the CU ones. Among other results pertaining to EL, Newey and Smith (2004) stated that EL estimate enjoys asymptotic optimality properties in term of e¢ ciency when bias corrected among all GEL estimates including the GMM one. Broniatowski and Keziou (2012) proposed a general approach through empirical divergences and duality technique which includes the above methods in the general context of signed …nite measures under moment condition models (1). These approach allows the asymptotic study of the estimates and associated test statistics both under the models and under misspeci…cation, leading to new results for the EL approach. Note that all the proposed estimates including the EL one are generally biased, and that the problem of their …nite sample e¢ ciency have not yet been studied. The aim of the talk is to investigate the …nite-sample optimality property estimation in the context of semiparametric model (1). We will discuss the problem of constructing minimum risk equivariant estimates (MRE, or Pitman estimators) for the parameter T , as well as the problem of the numerical calculation of these estimates. Let Gn be a group of one-to-one transformation on the sample space Rmn . We will consider two kinds of transformation groups, additive or multiplicative, and we will assume that the model M given in (1) is invariant under the group of transformation Gn . Denote by G the induced group on the parameter space . When estimating

T

by an estimate bn , we consider the quadratic loss function L(bn ;

T)

:= L2 (bn ;

T)

:= E

bn

2

T

if the model is invariant with respect to additive type groups, and the loss function is taken to be relative quadratic 0 !2 1 bn L(bn ; T ) := Lr (bn ; T ) := E @ 1 A T

if the model is invariant with respect to the multiplicative type groups. We will show that the empirical minimum divergence estimates introduced in Broniatowski and Keziou (2012) are invariant for the above models. 2. D' divergences. Let ' be a closed convex function from R onto [0; +1] with '(1) = 0, and such that its domain, dom' := fx 2 R such that '(x) < 1g =: (a; b); is an interval, with endpoints satisfying a < 1 < b, which may be bounded or unbounded, open or not. For any s.f.m.

MINIMUM RISK EQUIVARIANT MINIMUM DIVERGENCE ESTIMATES FOR MOMENT CONDITION MODELS3

Q 2 M , the D' -divergence between Q and the p.m. P , when Q is absolutely continuous with respect to (a.c.w.r.t) P , is de…ned through Z dQ D' (Q; P ) := ' (x) dP (x); (2) dP Rm and if the function x 7! '(x) is strictly convex on a neighborhood of x = 1, then D' (Q; P ) = 0 if and only if Q = P:

(3)

All the above properties are presented in Csiszár (1963), Csiszár (1967) and in Chapter 1 of Liese and Vajda (1987), for D' divergences de…ned on the set of all p.m.’s M 1 . When the D' -divergences are extended to M , then the same arguments as developed on M 1 hold. When de…ned on M 1 , the Kullback-Leibler (KL), modi…ed Kullback-Leibler (KLm ), 2 , modi…ed 2 ( 2m ), Hellinger (H), and L1 divergences are respectively associated to the convex functions '(x) = x log x x + 1, '(x) = log x + x 1, '(x) = 21 (x 1)2 , '(x) = 21 (x 1)2 =x, p 2 '(x) = 2( x 1) and '(x) = jx 1j. 3. Minimum empirical divergence estimates Let X1 ; :::; Xn denote an i.i.d. sample of a random vector X 2 Rm with distribution PX . Let Pn ( ) be the associated empirical measure, namely, 1X Pn ( ) := n i=1 n

Xi (

);

where x ( ) denotes the Dirac measure at point x, for all x. For a given 2 , the “plug-in” estimate of D' (M ; PX ) is Z dQ e ' (M ; PX ) := inf D' (Q; Pn ) = inf (x) dPn (x): (4) D ' Q2M Q2M dPn Rm (n)

(n)

If the projection Q of Pn on M exists, then it is clear that Q is a s.f.m. (or possibly a p.m.) (n) a.c.w.r.t. Pn ; this means that the support of Q must be included in the set fX1 ; : : : ; Xn g. So, de…ne the set ( ) n n X X (n) M := Q 2 M j Q a.c.w.r.t. Pn ; Q(Xi ) = 1 and Q(Xi )f (Xi ; ) = 0 ; (5) i=1

i=1

which may be seen as a subset of Rn . Then, the plug-in estimate (4) can be written as 1X ' (nQ(Xi )) : n i=1 n

e ' (M ; PX ) := inf D' (Q; Pn ) = D Q2M

In the same way,

inf Q2M

(n)

D' (M; PX ) := inf inf D' (Q; PX ) 2

Q2M

(6)

2 µ MICHEL BRONIATOWSKI1 , JANA JURECKOVÁ AND AMOR KEZIOU3

4

can be estimated by 1X ' (nQ(Xi )) : n i=1 n

e ' (M; PX ) := inf D e ' (M ; PX ) = inf inf D' (Q; Pn ) = inf D 2

By uniqueness of arg inf T through

2

2

Q2M

2

inf Q2M

(n)

D' (M ; PX ) and since the in…mum is reached in =

T,

(7)

we estimate

1X ' (nQ(Xi )) : n i=1 n

e' := arg inf 2

inf Q2M

(n)

(8)

b The expression of the estimate D(M ; PX ), given in (6), is the solution of a convex optimization problem under convex constrained subset in Rn . In order to transform this problem to an unconstrained one, we will make use of the Fenchel-Lengendre transform of the convex function ', as well as some other duality arguments. It is de…ned by : t 2 R 7! (t) := sup ftx

(9)

'(x)g :

x2R

Using some duality arguments, see Broniatowski and Keziou (2012), we can show that, for any (n) 2 , if there exists Q0 in M such that a < Q0 (Xi ) < b; for all i = 1; : : : ; n; e ' (M ; PX ), D e ' (M; PX ) and e expressions to the estimates D ) ( n X 1 e ' (M ; PX ) := sup t0 + t> f (Xi ; ) ; D t0 n 1+` (t0 ;t)2R i=1 ) ( n X 1 e ' (M; PX ) := inf D e ' (M ; PX ) := inf sup t0 + t> f (Xi ; ) D t0 2 2 (t ;t)2R1+` n i=1 0

and

e' := arg inf 2

sup (t0

;t)2R1+`

(

t0

1X n i=1 n

t0 + t> f (Xi ; )

)

:

(10)

(11)

(12)

(13)

The empirical likelihood estimate is obtained for the particular choice of the modi…ed KullbackLeibler divergence, namely, when '(x) = log x + x 1. We will show that for any divergence D' , the estimate b' is invariant with respect to L2 loss for additive groups, and invariant with respect to Lr loss for multiplicative groups. 4. MRE estimate for additive groups

2 Let en := h(X1 ; : : : ; Xn ) any one of the estimates e' of T , and assume that E ken k < 1. Then the MRE estimate of T is given by, see e.g. Lehmann and Casella (1998),

where

bn = en

yi := Xi

E0 en j y1 ; : : : ; yn

Xn ; 8i = 1; : : : ; n

1

;

1;

(14)

MINIMUM RISK EQUIVARIANT MINIMUM DIVERGENCE ESTIMATES FOR MOMENT CONDITION MODELS5

and E0 en j y1 ; : : : ; yn 1 is the conditional expectation of en given y := (y1 ; : : : ; yn 1 )> , under the assumption that T = 0. We give in the following a statistical methodology for computing the conditional expectation E0 en j y1 ; : : : ; yn 1 in two di¤erent cases. Note that, by invariance, we have E0 en j y1 ; : : : ; yn

1

:= E0 (h(X1 ; : : : ; Xn ) j y1 ; : : : ; yn 1 ) = E = E =

(h(X1 T ; : : : ; Xn T ) j y1 ; : : : ; y n 1 ) (h(X1 ; : : : ; Xn ) T j y1 ; : : : ; y n 1 ) T T + E T (h(X1 ; : : : ; Xn ) j y1 ; : : : ; yn 1 ) T

(15)

Case 1 : Assume that we dispose of N i.i.d. copies (X1;1 ; : : : ; Xn;1 ); : : : ; (X1;N ; : : : ; Xn;N ) of the sample (X1 ; : : : ; Xn ): Then compute N i.i.d. copies Tn (1); : : : ; Tn (N ) of the estimate en , and N i.i.d. copies y(1) := (y1 (1); : : : ; yn 1 (1))> ; : : : ; y(N ) := (y1 (N ); : : : ; yn 1 (N ))> of the spacings y := (y1 ; : : : ; yn 1 ): Following Stute (1986), consider a kernel (a centered probability density) K : x 2 Rn 1 7! K(x) := K(x1 ; : : : ; xn 1 ) and put y y(j) 1 K ; j = 1; : : : ; N; KhN (y; y(j)) := hN hN where the bandwidth hN is such that N hnN hN ! 0; ! +1 as N ! +1; log N N r X log N and < 1; for some 1 < r < 2: n N h N j=1 Then it follows that, when N ! +1, with probability one, the estimate PN j=1 Tn (j)KhN (y; y(j)) PN j=1 KhN (y; y(j)) converges to E by

T

(h(X1 ; : : : ; Xn ) j y1 ; : : : ; yn 1 ). Hence, the MRE estimate (14) is approximated bn

2en

PN

j=1 Tn (j)KhN (y; y(j)) : PN j=1 KhN (y; y(j))

(16)

Case 2 : Denote by X1 (b); : : : ; Xn (b), b=1,. . . ,B, the bootstrapped samples generated from the empirical measure Pn associated to the sample (X1 ; : : : ; Xn ): Then compute B bootstrapped copies Tn (1); : : : ; Tn (B) of the estimate en , and B copies y (1) := (y1 (1); : : : ; yn 1 (1)); : : : ; y (B) := (y1 (B); : : : ; yn 1 (B))

6

2 µ MICHEL BRONIATOWSKI1 , JANA JURECKOVÁ AND AMOR KEZIOU3

of the spacings y := (y1 ; : : : ; yn 1 ): The MRE estimate (14) is then approximated by PB b=1 Tn (b)KhN (y; y (b)) bn 2en : PB K (y; y (b)) h N b=1

(17)

References

Broniatowski, M. and Keziou, A. (2012). Divergences and duality for estimation and test under moment condition models. J. Statist. Plann. Inference, 142(9), 2554–2573. Chen, J. H. and Qin, J. (1993). Empirical likelihood estimation for …nite populations and the e¤ective usage of auxiliary information. Biometrika, 80(1), 107–116. Cressie, N. and Read, T. R. C. (1984). Multinomial goodness-of-…t tests. J. Roy. Statist. Soc. Ser. B, 46(3), 440–464. Csiszár, I. (1963). Eine informationstheoretische Ungleichung und ihre Anwendung auf den Beweis der Ergodizität von Marko¤schen Ketten. Magyar Tud. Akad. Mat. Kutató Int. Közl., 8, 85–108. Csiszár, I. (1967). On topology properties of f -divergences. Studia Sci. Math. Hungar., 2, 329–339. Godambe, V. P. and Thompson, M. E. (1989). An extension of quasi-likelihood estimation. J. Statist. Plann. Inference, 22(2), 137–172. With discussion and a reply by the authors. Haberman, S. J. (1984). Adjustment by minimum discriminant information. Ann. Statist., 12(3), 971–988. Hansen, L., Heaton, J., and Yaron, A. (1996). Finite-sample properties of some alternative gmm estimators. Journal of Business and Economic Statistics, 14, 462–2800. Hansen, L. P. (1982). Large sample properties of generalized method of moments estimators. Econometrica, 50(4), 1029–1054. Imbens, G. W. (1997). One-step estimators for over-identi…ed generalized method of moments models. Rev. Econom. Stud., 64(3), 359–383. Kuk, A. Y. C. and Mak, T. K. (1989). Median estimation in the presence of auxiliary information. J. Roy. Statist. Soc. Ser. B, 51(2), 261–269. Lehmann, E. L. and Casella, G. (1998). Theory of point estimation. Springer Texts in Statistics. Springer-Verlag, New York, second edition. Liese, F. and Vajda, I. (1987). Convex statistical distances, volume 95. BSB B. G. Teubner Verlagsgesellschaft, Leipzig. McCullagh, P. and Nelder, J. A. (1983). Generalized linear models. Monographs on Statistics and Applied Probability. Chapman & Hall, London. Newey, W. K. and Smith, R. J. (2004). Higher order properties of GMM and generalized empirical likelihood estimators. Econometrica, 72(1), 219–255. Owen, A. (1990). Empirical likelihood ratio con…dence regions. Ann. Statist., 18(1), 90–120. Owen, A. B. (1988). Empirical likelihood ratio con…dence intervals for a single functional. Biometrika, 75(2), 237–249. Owen, A. B. (2001). Empirical Likelihood. Chapman and Hall, New York. Pardo, L. (2006). Statistical inference based on divergence measures, volume 185 of Statistics: Textbooks and Monographs. Chapman & Hall/CRC, Boca Raton, FL.

MINIMUM RISK EQUIVARIANT MINIMUM DIVERGENCE ESTIMATES FOR MOMENT CONDITION MODELS7

Qin, J. and Lawless, J. (1994). Empirical likelihood and general estimating equations. Ann. Statist., 22(1), 300–325. Rockafellar, R. T. (1970). Convex analysis. Princeton University Press, Princeton, N.J. Sheehy, A. (1987). Kullback-Leibler constrained estimation of probability measures. Report, Dept. Statistics, Stanford Univ. Smith, R. J. (1997). Alternative semi-parametric likelihood approches to generalized method of moments estimation. Economic Journal, 107, 503–519. Stute, W. (1986). On almost sure convergence of conditional empirical distribution functions. Ann. Probab., 14(3), 891–901. 1

LSTA-Paris 6, [email protected]

2

Charles University in Prague, [email protected]

3

Laboratoire de Mathématiques de Reims, [email protected]