DOI: 10.1111/j.1541-0420.2006.00531.x

Biometrics 62, 735–744 September 2006

Parametric and Nonparametric FDR Estimation Revisited Baolin Wu,1,∗ Zhong Guan,2,∗∗ and Hongyu Zhao3,∗∗∗ 1

Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota 55455, U.S.A. 2 Department of Mathematical Sciences, Indiana University South Bend, South Bend, Indiana 46634, U.S.A. 3 Department of Epidemiology and Public Health, Yale University, New Haven, Connecticut 06520, U.S.A. ∗ email: [email protected] ∗∗ email: [email protected] ∗∗∗ email: [email protected]

Summary. Nonparametric and parametric approaches have been proposed to estimate false discovery rate under the independent hypothesis testing assumption. The parametric approach has been shown to have better performance than the nonparametric approaches. In this article, we study the nonparametric approaches and quantify the underlying relations between parametric and nonparametric approaches. Our study reveals the conservative nature of the nonparametric approaches, and establishes the connections between the empirical Bayes method and p-value-based nonparametric methods. Based on our results, we advocate using the parametric approach, or directly modeling the test statistics using the empirical Bayes method. Key words: Empirical Bayes method; False discovery rate; Microarray; Multiple comparisons; Multiple hypothesis testing; Simultaneous inference.

1. Introduction For current large-scale genomic and proteomic data sets, there are usually hundreds of thousands of variables but limited sample size, which poses a unique challenge for statistical analysis. Variable selection serves two purposes in this context: (a) biological interpretation and (b) reduction of the impact of noise. In microarray data sets, we are often interested in identifying differentially expressed genes. It can be formulated as the following hypothesis testing problem,

imize the power (or minimize type II error β) at the same time, α0 = Pr(rk = 1 | hk = 1), β = Pr(rk = 0 | hk = 0). When we do multiple hypothesis testing we want to control the overall type I error to be very small. There are different definitions for overall type I error in multiple hypothesis testing. A natural extension of type I error to multiple hypothesis testing is the family-wise-error-rate (FWER), which is the probability of identifying any false positives, that is, FWER = Pr(V > 0).

(1)

Hi : μi = 0 (i = 1, . . . , m), where m is the total number of genes and μi is the mean log ratio of the expression levels for the ith gene. Here we are testing m genes simultaneously, which causes complications for error control. Multiple hypothesis testing for a testing procedure is summarized in Table 1, where V is the number of false positives and S is the number of true positives. For the convenience of the following discussion, define hk = I{kth hypothesis being true null}, rk = I{kth hypothesis being rejected}, h = (h1 , . . . , hm ), r = (r1 , . . . , rm ), v = (r1 h1 , . . . , rm hm ). Here we treat hk as random variables. The L1 norms of these vectors are |h| = m0 , |r| = R, and |v| = V. In single hypothesis testing, the commonly used approach is to control type I error at a pre-specified level α0 and to max C

2006, The International Biometric Society

The most commonly used approach for FWER control is the Bonferroni correction, which adjusts individual significance levels to be α0 /m. Generally, the Bonferroni correction is conservative, especially in the context of genomic and proteomic data sets where m is very large. There have been some developments in using resampling methods to improve power while controlling FWER (Westfall and Young, 1993; Ge, Dudoit, and Speed, 2003). False discovery rate (FDR), a philosophically different approach, was first proposed by Benjamini and Hochberg (1995). It is defined as E(V/R). When R = 0, there is no discovery, we define 0/0 = 0. We can also write FDR as

 FDR = E









 V  V  R > 0 Pr(R > 0) = E V > 0 Pr(V > 0).  R R (2) 735

Biometrics, September 2006

736

Note that usually, under the true null hypothesis, pk ∼U[0, 1]. When the rejection region is chosen as Γ = [0, τ ], we have

Table 1 Possible outcomes of multiple hypothesis testing Accepted

Rejected

Total

U T N

V S R

m0 m1 m

True null True alternative Total

Storey (2002b) defined pFDR as the following conditional expectation:



pFDR = E





V  R>0 R

=

FDR . Pr(R > 0)

(3)

Clearly, FWER = Pr(V > 0) =

so FWER is always a stronger control than FDR. We can formally define the FDR estimation problem as follows: Data: m test statistics, (T 1 , . . . , Tm ), one for each hypothesis Hk , where k = 1, . . . , m. Goal: Develop testing procedure and estimate the expectation E(V /R), where V and R are defined in Table 1. Here we assume that (T 1 , . . . , Tm ) are m i.i.d. random variables. First define π0 = Pr(hk = 1), α0 = Pr(rk = 1 | hk = 1), α = Pr(rk = 1), (4) where π 0 is the proportion of true null hypotheses, α0 is the rejection probability of the true null hypothesis, and α is the marginal rejection probability. Under the i.i.d. assumption, we can have the following intuitive formula for pFDR and FDR (Benjamini, Krieger, and Yekutieli, 2001; Storey, 2002a; Storey, Taylor, and Siegmund, 2004): π0 α0 FDR = Pr(hk = 1 | rk = 1) = , α π0 α0 {1 − (1 − α)m }−1 . pFDR = α

(5)

So the pFDR and FDR estimation problems just transform into our familiar framework of estimating parameters π 0 , α0 , and α. Previous research on FDR control includes the nonparametric method of Storey (2002a) and parametric method of Guan, Wu, and Zhao (2004). In this article, we further study the operating characteristics of general p-value-based nonparametric methods. Our study reveals the conservative nature of the nonparametric approaches, and we further theoretically quantify the relations between parametric and nonparametric approaches. The basic idea of the nonparametric approach in Storey (2002a) is to use the p-values (p1 , . . . , pm ) as the test statistics.

1 − Fm (λ) , 1−λ

(6)

where Fm is the empirical distribution function of the observed p-values and λ ∈ [0, 1]. The optimal λ can be chosen by minimizing the mean squared error MSE{ˆ π0 (λ)}. In the parametric approach of Guan et al. (2004), two parametric functions are introduced to model the distribution of the test statistic: F 0 (·, θ0 ) for the null distribution and F 1 (·, θ1 ) for the alternative distribution. The marginal distribution is F (·, π 0 , θ0 , θ1 ) = π 0 F 0 (·, θ0 ) + (1 − π 0 )F 1 (·, θ1 ). The expectation-maximization (EM) algorithm (Dempster, Laird, and Rubin, 1977) can be used to obtain maximum likelihood estimations (MLE) of the parameters π 0 and θ1 . Then for any given rejection region Γ, we have α ˆ = F (Γ, π ˆ0 , θ0 , θˆ1 )

FDR

  ≥ FDR, V  E V > 0 R 

α ˆ = Fm (τ ), α ˆ 0 = τ, π ˆ0 (λ) =

and

α ˆ 0 = F0 (Γ, θ0 ).

(7)

For simplicity we have used (F , F 0 , F 1 ) to represent both the cumulative distribution functions and the corresponding probability measures. 2. Rejection Region Construction and FDR Modeling For the convenience of the following discussion, we write f0 (·) for the test statistic density under the null hypothesis and f1 (·) for that of the alternative hypothesis. In single hypothesis testing, we focus on type I error and power, α0 = F 0 (Γ) and 1 − β = F 1 (Γ), where Γ is the rejection region. The central dogma of traditional single hypothesis testing is to control type I error α0 under a pre-specified level and at the same time try to maximize the power 1 − β. In practice we try to construct rejection regions which will have maximum power. According to the Neyman–Pearson lemma (Neyman and Pearson, 1933), this can be achieved using the likelihood ratio (LR) statistic LR(x) = f 1 (x)/f 0 (x) constructed from the observed data, from which we can construct the following uniformly most powerful LR rejection region



x:

f1 (x) >η f0 (x)



.

(8)

2.1 p-Value Calculation p-value is a well-accepted significance measure for rejecting/ accepting a hypothesis, and in some papers discussing multiple comparisons (Benjamini and Hochberg, 1995; Benjamini and Yekutieli, 2001; Storey, 2002b; Ge et al., 2003), p-value is used as a test statistic. The distribution of the p-values can be estimated using the empirical distribution function of the observed p-values. The p-value densities are closely related to the distributions of the test statistics and the construction of the rejection region Γ. For p-values we have the following results (see the Appendix for proofs; similar results appeared in Sackrowitz and Samuel-Cahn, 1999). Lemma 1: For hypothesis test H0 versus Ha with test statistic X, assume X has density f0 (x) under H0 and f1 (x) under Ha , and let P0 and P1 be the corresponding measures. We assume that the rejection regions are constructed as {x : W(x) > η}, where W(·) is a measurable function. Let Qk (x), qk (x), k = 0, 1 be the

Parametric and Nonparametric FDR Estimation distribution and density functions of W(X) under H0 and Ha , respectively. Furthermore, we assume that Q0 (x) is continuous and strictly increasing. For an observed test statistic value x0 , the p-value can be calculated as p = P0 {x : W (x) > W (x0 )} = 1 − Q0 {W (x0 )}.

(9)

Under H0 , the p-value has a uniform density, g 0 (p) = I{p ∈ [0, 1]}. Under Ha , the p-value has the following density and distribution functions: g1 (p) =

 

 , G1 (p) = 1 − Q1 Q−1 (10) 0 (1 − p) ,

q1 Q−1 0 (1 − p) q0

Q−1 0 (1

some other smoothing methods, for example, kernel density estimations. The poor performance of the nonparametric approach is mainly because π ˆ0 (λ) is only based on those p-values over [λ, 1). Note that when λ is small, π ˆ0 (λ) as an estimator itself is very stable. In principle we could borrow strength from small λ to extrapolate π ˆ0 (1). This motivates us to smooth π ˆ0 (λ) or gˆ(λ) as functions of λ. As discussed previously, it is reasonable to assume g1 (p) is nonincreasing. The theoretical value of π ˆ0 (λ) is



− p)

π0 (λ) =

where P1 {x : η < W (x) ≤ η1 } q1 (η) = lim , q0 (η) η1 →η P0 {x : η < W (x) ≤ η1 }

(11)

and hence g1 (p) ≥ inf x {f1 (x)/f0 (x)}.



f1 (x) >η x : LR(x) = f0 (x)



,

We have



g1 (p) = Q−1 0 (1 − p).

(12)

Therefore g1 (p) is a nonincreasing function in the interval [0, 1]. Furthermore, we have min g1 (p) = g1 (1) = Q−1 0 (0) = inf x

p∈[0,1]

f1 (x) . f0 (x)

(13)

This theorem reveals that the p-value based on the LR test has a monotone decreasing density. In the multiple hypothesis testing, if we assume that p-values from individual testings follow one common distribution, nonparametric estimation of π 0 can be based on the p-value density (to be discussed in Section 2.2). Theorem 1 then justifies the common practice of using the p-value density at the boundary 1 to approximate π 0 . For rejection regions not based on LR test region, it is possible to observe nonmonotone p-value density, and according to Lemma 1, the least conservative π 0 estimation will be the minimum of the p-value density, which is not necessarily at the boundary 1. 2.2 Smoothing Nonparametric Approach Suppose we use p-value as the test statistic. Its distribution is g(p) = π 0 + (1 − π 0 )g 1 (p), where π 0 is the proportion of true null hypotheses and g1 (p) is the density for the p-values under the alternative hypothesis. In the nonparametric approach, the key is the estimation of π 0 . We propose the following least conservative estimation for π 0 : min g(p) = π0 + (1 − π0 ) min g1 (p). p

1

g1 (p) dp λ

1−λ

.

(14)

The simplest density estimation method is the histogram approach, gˆ(p) = {Fm (λ2 ) − Fm (λ1 )}/(λ2 − λ1 ), λ1 ≤ p ≤ λ2 . The nonparametric estimator π ˆ0 (λ) in (6) is just the histogram density estimation over (λ, 1], and implicitly assumes that g(1) achieves the minimum value. We can also apply

(15)

1

g1 (p) dp − (1 − λ)g1 (λ) ≤ 0,

λ

(1 − λ)2

so π 0 (λ) and g(λ) = π 0 + (1 − π 0 )g 1 (λ) are both nonincreasing functions of λ. Hence, monotone smoothing methods can be used for extrapolation. Furthermore, we have π0 (1) = g(1) = π0 + (1 − π0 )g1 (1).

we have

p

1 − F (λ) = π0 + (1 − π0 ) 1−λ

dπ0 (λ) = (1 − π0 ) dλ

Theorem 1: For the uniformly most powerful LR test (8), where the rejection region is constructed by

737

(16)

In the following applications, we used the constrained B-splines (He and Ng, 1999) for monotone extrapolation. 2.3 Model Test Statistic versus p-Values Although the p-value has a uniform distribution under the null hypothesis, its alternative distribution is often unknown. An empirical Bayes method (Efron et al., 2001; Efron and Tibshirani, 2002; Efron, 2003) proposed to use the posterior probability of “being different,” π ˆ1 (x) = 1 − π0

f0 (x) , f (x)

(17)

as a test statistic, and it was pointed out that π 0 is not identifiable for the nonparametric approach. In addition, Efron (2003) proposed the most conservative estimation for π1 = 1 − π0 : π1,min = 1 − inf x {f (x)/f0 (x)}, and hence, the least conservative estimate for π0 : π0,max = inf x {f (x)/f0 (x)}. Under the i.i.d. assumption, we have π1,min = π1 − π1 inf x

f1 (x) , f0 (x)

π0,max = π0 + π1 inf x

f1 (x) . f0 (x)

(18)

According to (8), this empirical Bayes method is equivalent to the nonparametric version of the LR-based test, where densities f0 (x) and f(x) are estimated from the observed data. Furthermore, according to Lemma 1 and Theorem 1, this is equivalent to the p-value-based nonparametric FDR estimation where p-values are obtained using the LR statistics. 3. Simulation Studies 3.1 Finite Normal Mixture Example Here we discuss the parametric and nonparametric approaches for finite normal mixture distributions. We assume that Ti | Hi = 1 ∼ N (0, 1);

Ti | Hi = 0 ∼

 k

πk N (μk , 1),

Biometrics, September 2006

738



where πk ∈ (0, 1), k πk = 1, and μk = 0. We have LR(x) = f1 (x)/f0 (x) = k πk exp(xμk − μ2k /2). 1. If all the μk are positive (negative), then inf x LR(x) = 0, and the uniformly most powerful rejection region is {x ≥ x0 }({x ≤ x0 }). Therefore the nonparametric π 0 estimate can approach the true value. 2. If ∃i , j , μi < 0, μj > 0, then it is obvious that inf x LR(x) > 0, and f1 (0)/f0 (0) = k πk exp(−μ2k /2) > 0. Under this setting, the LR test rejection region {LR(x) > η} is equivalent to {|x| > x0 }, if and only if all the π k and μk satisfy the following condition (see the Appendix for proof): ∀i, ∃j, st. μi + μj = 0 and πi = πj .

(19)

Furthermore, arg minx LR(x) = 0 if and only if



 

πk μk exp −μ2k 2 = 0.

(20)

k

This is because

  dLR(x)  = πk μk exp xμk − μ2k 2 , dx k

  d2 LR(x)  = πk μ2k exp xμk − μ2k 2 > 0, dx2 k

4. Application to Microarray Data 4.1 Leukemia Gene Expression Data We apply the proposed FDR estimation procedure to the leukemia gene expression data reported in Golub et al. (1999), where mRNA levels of 7129 genes were measured for n = 72 patients; among them n1 = 47 patients had acute lymphoblastic leukemia (ALL) and n2 = 25 patients had acute myeloid leukemia (AML). The goal is to identify differentially expressed genes between these two groups. The gene expression data can be summarized in a matrix X = (xij ), where (xi,1 , . . . , xi,n1 ) are for ALL patients and

Symmetric LR

1

density 2

Rejection Threshold –2 0 2 4

3 4

so LR(x) is strictly convex. In particular, condition (19) is a special case of (20). Hence for the commonly used symmetric region the estimate of π 0 will approach π 0 + (1 − π 0 )f 1 (0)/f 0 (0). It will be larger than the estimate of LR test region π 0 + (1 − π 0 )minx {f 1 (x)/f 0 (x)}, unless the condition (20) is met.

3.2 Simulation Consider the following setup for the finite normal mixture models, π 1 = 0.2, μ1 = 2, π 2 = 0.8, μ2 = −1, with f1 (x) = 2 π N (μk , 1). Suppose we conduct m = 1000 hypothek=1 k sis tests with π 0 = 0.2 and f0 (x) = N(0, 1). The parametric normal mixture model, π 0 N (0, 1) + (1 − π 0 ){π 1 N (μ1 , 1) + π 2 N (μ2 , 1)} is fitted to obtain π 0 ’s MLE π ˆpm . p-values can be calculated as p = 2Φ(−|x|), then we can get nonparametric estimate π ˆnp of π 0 (Storey, 2002b). For the empirical Bayes method, we first estimate the density of the test statistic fˆ(x), then π ˆeb = inf x fˆ(x)/f0 (x), where f 0 (x) = φ(x). Figure 1 plots the LR and the symmetric rejection regions as functions of the rejection probability α0 (4). Also shown in the plot are the p-value densities for the two rejection regions. For symmetric rejection regions, the minimum p-value density is π np = π 0 + (1 − π 0 )LR(0) = 0.61, compared to π eb = π 0 + (1 − π 0 )minx LR(x) = 0.48 for the LR rejection regions. They both overestimate the true value π 0 = 0.2. In Figure 1, boxplots are used to summarize the simulation results. We can clearly see that the simulation results agree with the theoretical results very well.

–4

LR Symmetric 0.0

0.2

0.4

α0

0.6

0.8

1.0

0.0

0.2

0.4 0.6 p–value

0.8

1.0

^ pm π

^ eb π

^ np π

0

π0 = 0.2

πeb = 0.48

πnp = 0.61

Figure 1. Simulation study: the top two plots compare the LR and symmetric rejection regions; the bottom one compares the parametric (pm), empirical Bayes (eb), and nonparametric (np) estimations.

Parametric and Nonparametric FDR Estimation

739

We can use the Bayesian information criterion (BIC) to seˆ − lect the number of components, BIC(p) = 2 log Pr(Data | θ) p log(m), where θˆ is a vector representing the MLEs of the parameters, and p is the number of parameters in the model (Fraley and Raftery, 2002). In our model setup p = 2G − 2, where G is the number of normal distributions (we know the mean for the first component and there is one constraint on the proportions). For G = 1, 2, . . . , 12, we use the EM algorithm to fit the mixture models and select G = arg maxG BIC(p). The maximum of BIC was achieved at G = 8. The corresponding parameter estimates are π ˆ0 = 0.35, with three positive components

(xi,n1 +1 , . . . , xi,n ) are for AML patients. We follow the same preprocessing procedure as Dudoit, Fridlyand, and Speed (2002). We first cut gene expression levels between 100 and 16,000, then keep the ith gene if it satisfies two conditions: maxj xij /minj xij > 5 and maxj xij − minj xij > 500. After this filtering m = 3571 genes are left. We then take the logarithm of their measured intensities and calculate two-sample t-test statistics Ti = (¯ xi1 −x ¯i2 )/(ˆ σ12 /n1 + σ ˆ22 /n2 )1/2 , where x ¯i1 = n1 n n1 2 2 x /n , x ¯ = x /n , σ ˆ = (x − x ¯ ij 1 i2 ij 2 ij i1 ) / 1 j=1 j=1 nj=n1 +1 2 2 ˆ2 = j=n1 +1 (xij − x ¯i2 ) /(n2 − 1). (n1 − 1), and σ For this relatively large sample size (n = 72), we know that Ti asymptotically follows a normal distribution with variance 1. We use normal mixture model to fit the t-statistics by proposing the following three-component model to model genes:

(ˆ πU , θˆU ) = {(0.214, 2.42), (0.045, 5.22), (0.003, 9.57)}, and four negative components

Without difference: standard normal distribution N(μ0 = 0, 1); Up-regulated: normal mixture with positive means, N (μU > 2 0, σU = 1); Down-regulated: normal mixture with negative means, 2 N (μL < 0, σL = 1). The mixture distribution can be written as where π = 1. k k



k

(ˆ πL , θˆL ) = {(0.306, −1.57), (0.068, −3.88), (0.012, −6.82), (0.002, −11.64)}. Figure 2 compares the empirical distribution function (ECDF) to the mixture model fitting, and displays the quantile–quantile plot for the test statistics. Overall we can see that the mixture model provides a reasonable fit. Figure 2 also displays the FDR estimations for this data set, where we

πk N (μk , 1),

QQ Plot

–5

0 Test Statistics

10 5

10

–15

0.35

–10

–15

0.0

–10

0.2

–5

0.4

Quantiles 0

0.6

5

0.8

1.0

Distribution Function Estimation ECDF 3–Component Model

–5

0 Test Statistics

5

10

0

0.00

0.05

0.10

FDR 0.15 0.20

0.25

Number of Significant Genes 500 1000 2000 3000

0.30

π0

12

Figure 2.

–10

10 8 6 4 2 Threshold Γ ⎛⎝rejection region {|T| ≥ Γ}⎞⎠

0

0.00

0.05

0.10

0.15

0.20

0.25

0.30

π0

FDR

Three-component model fitting for the leukemia data and FDR estimation.

Biometrics, September 2006

740

Density 2 3

permutation density mixture density

0

0.2

1

0.4

0.449

^ 0(λ) π 0.6

0.8

4

1.0

5

Nonparametric Smoothing π0 Estimation

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4 0.6 p-value

λ

Figure 3.

1.0

Nonparametric versus parametric estimation for the leukemia data.

choose the rejection region as {|T | > t0 }. The maximum value of FDR is π ˆ0 = 0.35 when t0 = 0, where every gene is declared as significant. Also shown in the figure is the number of significant genes versus FDR estimations. When FDR = π ˆ0 , all genes are declared as significant. We can also apply the nonparametric approach to this leukemia gene expression data. We use permutation to get the p-values for the t-statistics based on 1000 permutations. The histogram for the permutation p-values is plotted in Figure 3; also shown is the monotone smoothing estimation of π 0 based on the constrained B-splines (He and Ng, 1999). The extrapolated value at boundary is π ˆ0 = 0.449.

There is a difference between parametric and nonparametric estimation of π 0 (0.35 vs. 0.449). If we assume that the fitted mixture model is correct, then the least conservative nonparametric estimation for π 0 is minλ∈[0,1] g(p) = g(1) = π 0 + (1 − π 0 )LR(0) = 0.451, very close to 0.449. If we use the empirical Bayes method, the least conservative estimate is π0 + (1 − π0 ) min LR(x) = π0 + (1 − π0 )LR(0.41) = 0.428. x

Figure 3 compares the permutation p-value density and the theoretical density from the fitted mixture models. They agree with each other very well. QQ Plot

Distribution Function Estimation 1.0

0.8

0.0

–5

0.2

0.4

Quantiles 0

0.6

5

0.8

ECDF 3–Component Model

–4

–2

0 2 Test Statistics

4

6

8

0 Test Statistics

5

0

0.0

0.1

FDR 0.2

0.3

Number of Significant Genes 500 1000 1500

0.4

π0

8

Figure 4.

–5

2000

–6

6 4 Threshold Γ ⎛⎝rejection region

2

{|T| ≥ Γ}⎞⎠

0

0.00

0.10

0.20

0.30

π0

FDR

Three-component model fitting for the colon cancer data and FDR estimation.

741

6

1.0

Parametric and Nonparametric FDR Estimation

5

permutation density mixture density

0

0.4

1

0.5

2

0.6

density 3 4

π0(λ) 0.7 0.8

0.9

parametric estimation nonparametric estimation

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

λ

Figure 5.

0.4 0.6 p–value

0.8

1.0

Nonparametric versus parametric estimation for the colon cancer data. π ˆ0 = 0.408,

(ˆ πL , θˆL ) = {(0.073, −3.72), (0.193, −1.81)},

(ˆ πU , θˆU ) = {(0.247, 1.37), (0.074, 3.36), (0.005, 6.38)}. Figure 4 shows some model fitting diagnostics and the FDR estimation for the colon cancer data. Using permutations we can estimate the p-value for each gene, which can be compared to the parametric approach. Figure 5 shows the p-value

0.10

0.15

π0

FDR

0.20

0.25

0.15 0.20 0.25 0.30 0.35 0.40 0.45

4.2 Colon Cancer Gene Expression Data The colon cancer gene expression data contained the expression values of 2000 genes from 40 tumor and 22 normal colon tissue samples reported by Alon et al. (1999). We apply the normal mixture model to estimate FDR for these data. With BIC we select six normal components with mean and probability estimations being

ρ= 0 0.10.10.10.30.30.30.50.50.50.90.90.9 K= 35 7014035 7014035 7014035 70140

FDR

ρ= 0 0.10.10.10.30.30.30.50.50.50.90.90.9 K= 35 7014035 7014035 7014035 70140

0.02

0.04

0.03

0.06

0.08

0.04

FDR

0.10

0.05

0.12

0.06

0.14

0.07

ρ= 0 0.10.10.10.30.30.30.50.50.50.90.90.9 K= 35 7014035 7014035 7014035 70140

ρ= 0 0.10.10.10.30.30.30.50.50.50.90.90.9 K= 35 7014035 7014035 7014035 70140

Figure 6. FDR estimation under local dependence: there are 13 simulations based on the combination of five different ρ’s and three different K’s, which are labeled at the bottom of each plot. The boxplots are based on 100 replicates, and the horizontal dashed black lines represent the true value estimated from 100 replicates. We can see that the pattern of FDR estimation is very similar to π 0 : the bigger the correlation ρ and the number of local clusters K, the more variable the estimations. But overall we can see that the proposed model gives very good estimates, even when the local correlation is as large as 0.5.

Biometrics, September 2006



θk >0

0.05

πk

= −2.19.

θk η} = {|x| > x0 }, where η = LR(x0 ). Now suppose {LR(x) = f 1 (x)/f 0 (x) > η} = {|x| > x0 }, we have ∀x, LR(x) = LR(−x). Suppose maxj μj = μJ > 0, we have LR(x)exp(−xμJ ) = LR(−x)exp(−xμJ ), that is, L1 = L2 , where

 

L1 = πJ exp −μ2J 2 +







πk exp x(μk − μJ ) − μ2k 2 ,

k =J

 

L2 = πJ exp −2xμJ − μ2J 2 +







πk exp −x(μk + μJ ) − μ2k 2 .

k =J

We know that limx→∞ L1 = π J exp(−μ2J /2). So there must exist a K, such that π K = π J and μK + μJ = 0, which will make limx→∞ L2 = limx→∞ L1 . From LR(x) − π J exp(xμJ − μ2J /2) = LR(−x) − π K exp(−xμK − μ2K /2), we can prove that the second largest μk satisfies the symmetric condition. So sequentially we can prove that ∀i, ∃j, such that μi + μj = 0, and π i = π j .