Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer

Nonparametric Bayesian Estimation of X/γ Ray Spectra using a Hierarchical Dirichlet Process–P´ olya Tree Model

Introduction Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Applications

´ Barat1 Eric 1

Thomas Dautremer1

Electronics and Signal Processing Laboratory CEA-Saclay Gif-sur-Yvette, France

Conclusion Further reading

MaxEnt 2006 July 13, 2006

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

1 / 43

Outline

Nonparametric Bayesian Spectrum Estimation

1

Introduction Motivation Bayesian approaches A discrete vs. continuous mixture model

2

Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´olya tree process mixtures

3

Hierarchical Model for Physical Spectra Physical motivations Hierarchical model

4

Applications

5

Conclusion

6

Further reading

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

2 / 43

Outline

Nonparametric Bayesian Spectrum Estimation

1

Introduction Motivation Bayesian approaches A discrete vs. continuous mixture model

2

Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´olya tree process mixtures

3

Hierarchical Model for Physical Spectra Physical motivations Hierarchical model

4

Applications

5

Conclusion

6

Further reading

´ Barat, E. T. Dautremer Introduction Motivation Bayesian approaches Discrete/continuous mixture model Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

3 / 43

Motivation

Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Motivation Bayesian approaches Discrete/continuous mixture model Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra

Framework Quantitative analysis of physical quantities. Data of interest vs. smooth background separation. Ad hoc methods Polynomial, semi-empirical fitting procedures. Unpredictable results. Strong human monitoring Order of polynomials ? Fitting range ? Data selection ?

Applications Conclusion

Toward a more rigorous framework...

Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

4 / 43

Motivation

Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Motivation Bayesian approaches Discrete/continuous mixture model Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra

Framework Quantitative analysis of physical quantities. Data of interest vs. smooth background separation. Ad hoc methods Polynomial, semi-empirical fitting procedures. Unpredictable results. Strong human monitoring Order of polynomials ? Fitting range ? Data selection ?

Applications Conclusion

Toward a more rigorous framework...

Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

4 / 43

Motivation

Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Motivation Bayesian approaches Discrete/continuous mixture model Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra

Framework Quantitative analysis of physical quantities. Data of interest vs. smooth background separation. Ad hoc methods Polynomial, semi-empirical fitting procedures. Unpredictable results. Strong human monitoring Order of polynomials ? Fitting range ? Data selection ?

Applications Conclusion

Toward a more rigorous framework...

Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

4 / 43

Motivation

Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Motivation Bayesian approaches Discrete/continuous mixture model Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra

Framework Quantitative analysis of physical quantities. Data of interest vs. smooth background separation. Ad hoc methods Polynomial, semi-empirical fitting procedures. Unpredictable results. Strong human monitoring Order of polynomials ? Fitting range ? Data selection ?

Applications Conclusion

Toward a more rigorous framework...

Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

4 / 43

Outline

Nonparametric Bayesian Spectrum Estimation

1

Introduction Motivation Bayesian approaches A discrete vs. continuous mixture model

2

Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´olya tree process mixtures

3

Hierarchical Model for Physical Spectra Physical motivations Hierarchical model

4

Applications

5

Conclusion

6

Further reading

´ Barat, E. T. Dautremer Introduction Motivation Bayesian approaches Discrete/continuous mixture model Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

5 / 43

Bayesian approaches Bayesian approach

Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Motivation Bayesian approaches Discrete/continuous mixture model Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Applications Conclusion Further reading

Von der Linden et al. (1999), Fisher et al. (2000). Background as a cubic or exponential spline. Better properties than any ad hoc method. Our contribution Restriction to the problem where observations are realizations of a r.v. whose distribution has to be estimated. Introduction of (direct) priors for physical quantity probability distribution (energy). 6= Bayesian spline on a (Poisson) noisy histogram. 6= Gaussian Process approach (prior over functions, not over measures)

Joined estimation of background and peaks spectrum. Deconvolution of instrumental kernel. Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

6 / 43

Outline

Nonparametric Bayesian Spectrum Estimation

1

Introduction Motivation Bayesian approaches A discrete vs. continuous mixture model

2

Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´olya tree process mixtures

3

Hierarchical Model for Physical Spectra Physical motivations Hierarchical model

4

Applications

5

Conclusion

6

Further reading

´ Barat, E. T. Dautremer Introduction Motivation Bayesian approaches Discrete/continuous mixture model Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

7 / 43

A Mixed Discrete–Continuous Model 1

Quantum mechanics. Preservation of discreteness in “pure” photoelectric absorption. Number of peaks often unknown and open-ended.

Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer

2

Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Applications

Backgrounds sit on continuous distribution. Compton effects produce continuous spectra. Complex (unrealistic?) physical parametrization. Interactions with environment : back-scattering, etc.

Introduction Motivation Bayesian approaches Discrete/continuous mixture model

Underlying discrete model for the observed peaks spectrum.

3

Detection devices Observed data sit on continuous distributions (even peaks). Noise with parametric kernel (assumed gaussian with energy dependent variance). Other interactions processes : binding, Bremsstrahlung, etc.

Conclusion Further reading

How to capture this complex data structure ?

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

8 / 43

A Mixed Discrete–Continuous Model 1

Quantum mechanics. Preservation of discreteness in “pure” photoelectric absorption. Number of peaks often unknown and open-ended.

Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer

2

Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Applications

Backgrounds sit on continuous distribution. Compton effects produce continuous spectra. Complex (unrealistic?) physical parametrization. Interactions with environment : back-scattering, etc.

Introduction Motivation Bayesian approaches Discrete/continuous mixture model

Underlying discrete model for the observed peaks spectrum.

3

Detection devices Observed data sit on continuous distributions (even peaks). Noise with parametric kernel (assumed gaussian with energy dependent variance). Other interactions processes : binding, Bremsstrahlung, etc.

Conclusion Further reading

How to capture this complex data structure ?

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

8 / 43

Experimental Spectrum

106

´ Barat, E. T. Dautremer

104 Counts

Nonparametric Bayesian Spectrum Estimation

104 102

102 Introduction Motivation Bayesian approaches Discrete/continuous mixture model Nonparametric Bayesian density estimation

100 2300

2400

100 0

1000

2000

3000

Energy (KeV) Hierarchical Model for Physical Spectra Applications Conclusion

Figure: Uranium oxide (UO2 ) experimental energy spectrum (histogram).

Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

9 / 43

Experimental Spectrum

104 Nonparametric Bayesian Spectrum Estimation

104

´ Barat, E. T. Dautremer Introduction Motivation Bayesian approaches Discrete/continuous mixture model

102

102

Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra

100

Applications Conclusion Further reading

2300

2400

100 Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

9 / 43

Outline

Nonparametric Bayesian Spectrum Estimation

1

Introduction Motivation Bayesian approaches A discrete vs. continuous mixture model

2

Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´olya tree process mixtures

3

Hierarchical Model for Physical Spectra Physical motivations Hierarchical model

4

Applications

5

Conclusion

6

Further reading

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

10 / 43

Priors on Probability Distributions

Priors on M (R) Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction

Let M (X ) the set of all probability measures over X . X = R : M (R) is an infinite dimensional space. Priors on infinite dimensional spaces lead to nonparametric / semi-parametric models.

Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures

Nonparametric model Nonparametric 6= no parameters. Number of parameters can increase with dataset length.

Hierarchical Model for Physical Spectra Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

11 / 43

Priors on Probability Distributions

Priors on M (R) Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction

Let M (X ) the set of all probability measures over X . X = R : M (R) is an infinite dimensional space. Priors on infinite dimensional spaces lead to nonparametric / semi-parametric models.

Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures

Nonparametric model Nonparametric 6= no parameters. Number of parameters can increase with dataset length.

Hierarchical Model for Physical Spectra Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

11 / 43

Bayesian Nonparametrics Nonparametric Bayesian Density Estimation

Bayesian nonparametric statistical model

Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra Applications Conclusion Further reading

Ability to capture complex data structure. Relevant alternative to parametric models. Suitable for estimation of probability measures. Relevance in physical sciences Ability to embed physical belief in the model. Inference can be achieved by means of posterior draws (functionals, moments). Difficulties Construction of priors in infinite dimensional space. Elicitation of hyperparameters. Need for original MCMC schemes. Consistency issues. Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

12 / 43

Outline

Nonparametric Bayesian Spectrum Estimation

1

Introduction Motivation Bayesian approaches A discrete vs. continuous mixture model

2

Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´olya tree process mixtures

3

Hierarchical Model for Physical Spectra Physical motivations Hierarchical model

4

Applications

5

Conclusion

6

Further reading

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

13 / 43

Dirichlet Processes Definition

Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra Applications Conclusion Further reading

G0 be a probability measure over (X , B) and α ∈ R+? . A Dirichlet process is the distribution of a random measure G over (X , B) s.t., for any finite partition (B1 , . . . , Br ) of X , (G (B1 ) , . . . , G (Br )) ∼ Dir (αG0 (B1 ) , . . . , αG0 (Br )) G0 is the mean distribution, α the concentration parameter. We write G ∼ DP (αG0 ). Representations of Dirichlet processes P´olya urns (DP arises here as the De Finetti measure). Chinese restaurant (prior on partitions), limit of Dirichlet distribution, normalized Gamma process, etc. Sethuraman (constructive representation). Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

14 / 43

Some Properties of Dirichlet Processes

Properties Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

Discrete generated random distributions. Mean and variance : ∀B ⊆ X , E [G (B)] = G0 (B) var (G (B)) =

G0 (B) (1 − G0 (B)) 1+α

Therefore, if α is large G is concentrated around G0 . i.i.d.

Conjugacy : Consider G ∼ DP (αG0 ) and X1 , . . . , Xn−1 ∼ G . ! n−1 X G (·) |X1 , . . . , Xn−1 ∼ DP αG0 (·) + δXi (·)

Applications

i=1

Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

15 / 43

Stick-Breaking Representation Stick-breaking representation (Sethuraman, 1994). i.i.d.

Z = (Z1 , Z2 , . . .) ∼ G0 i.i.d.

Nonparametric Bayesian Spectrum Estimation

V = (V1 , V2 , . . .) ∼ Beta (1, α) p = (p1 , p2 , . . .), s.t. p1 = V1 and pk = Vk

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

P (·) =

∞ X

Qk−1 i=1

(1 − Vi ).

pk δZk (·)

k=1

is a DP (αG0 ) random probability measure. Almost sure truncation (Ishwaran and James, 2001) : PN (·) =

Applications

N X

pk δZk (·)

k=1

Conclusion Further reading

converges a.s. to a DP (αG0 ) random probability measure. Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

16 / 43

Dirichlet Processes In Action Stick-breaking construction

1

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.75 Distribution

Nonparametric Bayesian Spectrum Estimation

0

0.5

1

Stick-breaking

0.5

0.25

0 −3

−2

−1

0

1

2

3

k=0

Figure: Dirichlet process sequence construction (gaussian mean).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

17 / 43

Dirichlet Processes In Action Stick-breaking construction

1

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.75 Distribution

Nonparametric Bayesian Spectrum Estimation

0

0.5

1

Stick-breaking

0.5

0.25

0 −3

−2

−1

0

1

2

3

k=1

Figure: Dirichlet process sequence construction (α = 3).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

17 / 43

Dirichlet Processes In Action Stick-breaking construction

1

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.75 Distribution

Nonparametric Bayesian Spectrum Estimation

0

0.5

1

Stick-breaking

0.5

0.25

0 −3

−2

−1

0

1

2

3

k=2

Figure: Dirichlet process sequence construction (α = 3).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

17 / 43

Dirichlet Processes In Action Stick-breaking construction

1

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.75 Distribution

Nonparametric Bayesian Spectrum Estimation

0

0.5

1

Stick-breaking

0.5

0.25

0 −3

−2

−1

0

1

2

3

k=3

Figure: Dirichlet process sequence construction (α = 3).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

17 / 43

Dirichlet Processes In Action Stick-breaking construction

1

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.75 Distribution

Nonparametric Bayesian Spectrum Estimation

0

0.5

1

Stick-breaking

0.5

0.25

0 −3

−2

−1

0

1

2

3

k=4

Figure: Dirichlet process sequence construction (α = 3).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

17 / 43

Dirichlet Processes In Action Stick-breaking construction

1

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.75 Distribution

Nonparametric Bayesian Spectrum Estimation

0

0.5

1

Stick-breaking

0.5

0.25

0 −3

−2

−1

0

1

2

3

k=5

Figure: Dirichlet process sequence construction (α = 3).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

17 / 43

Dirichlet Processes In Action Stick-breaking construction

1

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.75 Distribution

Nonparametric Bayesian Spectrum Estimation

0

0.5

1

Stick-breaking

0.5

0.25

0 −3

−2

−1

0

1

2

3

k=6

Figure: Dirichlet process sequence construction (α = 3).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

17 / 43

Dirichlet Processes In Action Stick-breaking construction

1

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.75 Distribution

Nonparametric Bayesian Spectrum Estimation

0

0.5

1

Stick-breaking

0.5

0.25

0 −3

−2

−1

0

1

2

3

k = 50

Figure: Dirichlet process sequence construction (α = 3).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

17 / 43

Dirichlet Processes In Action Random distributions generation

1

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.75 Distribution

Nonparametric Bayesian Spectrum Estimation

CDF

1 0.5 0

0.5

−2

0

2

0.25

0 −3

−2

−1

0

1

2

3

k = 50

Figure: Distribution generation (α = 3, gaussian mean).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

18 / 43

Dirichlet Processes In Action Random distributions generation

1

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.75 Distribution

Nonparametric Bayesian Spectrum Estimation

CDF

1 0.5 0

0.5

−2

0

2

0.25

0 −3

−2

−1

0

1

2

3

k = 50

Figure: Distribution generation (α = 3, gaussian mean).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

18 / 43

Dirichlet Processes In Action Random distributions generation

1

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.75 Distribution

Nonparametric Bayesian Spectrum Estimation

CDF

1 0.5 0

0.5

−2

0

2

0.25

0 −3

−2

−1

0

1

2

3

k = 50

Figure: Distribution generation (α = 3, gaussian mean).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

18 / 43

Dirichlet Processes In Action Random distributions generation

1

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.75 Distribution

Nonparametric Bayesian Spectrum Estimation

CDF

1 0.5 0

0.5

−2

0

2

0.25

0 −3

−2

−1

0

1

2

3

k = 50

Figure: Distribution generation (α = 3, gaussian mean).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

18 / 43

Dirichlet Processes In Action Random distributions generation

1

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.75 Distribution

Nonparametric Bayesian Spectrum Estimation

CDF

1 0.5 0

0.5

−2

0

2

0.25

0 −3

−2

−1

0

1

2

3

k = 50

Figure: Distribution generation (α = 3, gaussian mean).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

18 / 43

Dirichlet Processes In Action Random distributions generation

1

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.75 Distribution

Nonparametric Bayesian Spectrum Estimation

CDF

1 0.5 0

0.5

−2

0

2

0.25

0 −3

−2

−1

0

1

2

3

k = 50

Figure: Distribution generation (α = 3, gaussian mean).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

18 / 43

Dirichlet Processes In Action Random distributions generation

1

´ Barat, E. T. Dautremer Introduction

0.75 Distribution

Nonparametric Bayesian Spectrum Estimation

CDF

1 0.5 0

0.5

−2

0

2

0.25

Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0 −3

−2

−1

0

1

2

3

average (20 draws)

Figure: Draws average (α = 3, gaussian mean).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

18 / 43

Truncated Dirichlet Process Mixtures Discreteness of DP (αG0 ) generated measures Cannot be used for probability density functions estimation ! =⇒ hierarchical mixture model with distribution µ (Xi |·). Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

Truncated Dirichlet Mixture Observations X1 , . . . , Xn , ind

(Xi |Yi ) ∼ µ (Xi |Yi )

Alternate TDPM Observations X1 , . . . , Xn , ind

(Xi |Z, K) ∼ µ (Xi |ZKi )

i.i.d.

(Yi |G ) ∼ G G∼

N X

i.i.d.

(Ki |p) ∼ pk δZk (·)

k=1

N X

pk δk (·)

k=1 N

(p, Z) ∼ µ (p) × (G0 )

Applications Conclusion Further reading

with Y = (Y1 , Y2 , . . . , Yn ) the hidden location variables. Nonparametric Bayesian Spectrum Estimation

with K = (K1 , K2 , . . . , Kn ) the classification Ki = j if Yi = Zj . MaxEnt 2006

July 13, 2006

19 / 43

Outline

Nonparametric Bayesian Spectrum Estimation

1

Introduction Motivation Bayesian approaches A discrete vs. continuous mixture model

2

Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´olya tree process mixtures

3

Hierarchical Model for Physical Spectra Physical motivations Hierarchical model

4

Applications

5

Conclusion

6

Further reading

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

20 / 43

P´ olya Tree Processes Definition

Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

m Let E = {0, 1}, E m = E × · · · × E and E ? = ∪∞ m=0 E . m Let πm = {B :  ∈ E } be a partition of X and Π = ∪∞ m=0 πm . A probability distribution G on X has a P´ olya tree distribution PT(Π, A) if there are nonnegative numbers A = {α :  ∈ E ? } and r.v. W = {W :  ∈ E ? } s.t. W is a sequence of independent random variables, for all  in E ? , W ∼ Beta(α0 , α1 ), and for all integer m and  = 1 · · · m in E m ,

G (B1 ···m ) =

m Y

W1 ···j−1 ×

j=1 j =0

m Y

(1 − W1 ···j−1 )

j=1 j =1

Applications Conclusion

Note that for  ∈ E ? , W0 = G (B0 |B )

Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

21 / 43

Some Properties of P´ olya Tree Processes Properties

Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures

P´olya trees are tail free processes. Dirichlet processes are P´ olya trees s.t. α0 = α1 = α /2 PT(Π, A) can generate absolutely continuous distributions. Conjugacy : posterior of PT(Π, A) after observations X = (X1 , . . . , Xn ) is the P´ olya tree PT(Π, AX ) with αX = α + n n = # {i ∈ {1, . . . , n} : Xi ∈ B } Predictive density (conditional mean)

Hierarchical Model for Physical Spectra Applications Conclusion

Pr (Xn+1 ∈ B1 ···m |X) =

m Y k=1

α1 ···k + n1 ···k α1 ···k−1 0 + α1 ···k−1 1 + n1 ···k−1

Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

22 / 43

P´ olya Trees In Action Partition and sequence construction

1

´ Barat, E. T. Dautremer

0.75 Density

Nonparametric Bayesian Spectrum Estimation

0.5

Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.25

0 −3

−2

−1

0

1

2

3

m=0

Figure: P´ olya tree sequence construction (normal mean).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

23 / 43

P´ olya Trees In Action Partition and sequence construction

1

´ Barat, E. T. Dautremer

0.75 Density

Nonparametric Bayesian Spectrum Estimation

0.5

Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.25

0 −3

−2

−1

0

1

2

3

m=1

Figure: P´ olya tree sequence construction (A = {αm = 3m }).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

23 / 43

P´ olya Trees In Action Partition and sequence construction

1

´ Barat, E. T. Dautremer

0.75 Density

Nonparametric Bayesian Spectrum Estimation

0.5

Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.25

0 −3

−2

−1

0

1

2

3

m=2

Figure: P´ olya tree sequence construction (A = {αm = 3m }).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

23 / 43

P´ olya Trees In Action Partition and sequence construction

1

´ Barat, E. T. Dautremer

0.75 Density

Nonparametric Bayesian Spectrum Estimation

0.5

Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.25

0 −3

−2

−1

0

1

2

3

m=3

Figure: P´ olya tree sequence construction (A = {αm = 3m }).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

23 / 43

P´ olya Trees In Action Partition and sequence construction

1

´ Barat, E. T. Dautremer

0.75 Density

Nonparametric Bayesian Spectrum Estimation

0.5

Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.25

0 −3

−2

−1

0

1

2

3

m=4

Figure: P´ olya tree sequence construction (A = {αm = 3m }).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

23 / 43

P´ olya Trees In Action Partition and sequence construction

1

´ Barat, E. T. Dautremer

0.75 Density

Nonparametric Bayesian Spectrum Estimation

0.5

Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.25

0 −3

−2

−1

0

1

2

3

m=5

Figure: P´ olya tree sequence construction (A = {αm = 3m }).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

23 / 43

P´ olya Trees In Action Partition and sequence construction

1

´ Barat, E. T. Dautremer

0.75 Density

Nonparametric Bayesian Spectrum Estimation

0.5

Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.25

0 −3

−2

−1

0

1

2

3

m=6

Figure: P´ olya tree sequence construction (A = {αm = 3m }).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

23 / 43

P´ olya Trees In Action Partition and sequence construction

1

´ Barat, E. T. Dautremer

0.75 Density

Nonparametric Bayesian Spectrum Estimation

0.5

Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.25

0 −3

−2

−1

0

1

2

3

m = 12

Figure: P´ olya tree sequence construction (A = {αm = 3m }).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

23 / 43

P´ olya Trees In Action Random distributions generation

1

´ Barat, E. T. Dautremer

0.75 Density

Nonparametric Bayesian Spectrum Estimation

0.5

Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.25

0 −3

−2

−1

0

1

2

3

m = 12

Figure: Distribution generation (normal mean, A = {αm = 3m }).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

24 / 43

P´ olya Trees In Action Random distributions generation

1

´ Barat, E. T. Dautremer

0.75 Density

Nonparametric Bayesian Spectrum Estimation

0.5

Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.25

0 −3

−2

−1

0

1

2

3

m = 12

Figure: Distribution generation (normal mean, A = {αm = 3m }).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

24 / 43

P´ olya Trees In Action Random distributions generation

1

´ Barat, E. T. Dautremer

0.75 Density

Nonparametric Bayesian Spectrum Estimation

0.5

Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.25

0 −3

−2

−1

0

1

2

3

m = 12

Figure: Distribution generation (normal mean, A = {αm = 3m }).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

24 / 43

P´ olya Trees In Action Random distributions generation

1

´ Barat, E. T. Dautremer

0.75 Density

Nonparametric Bayesian Spectrum Estimation

0.5

Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.25

0 −3

−2

−1

0

1

2

3

m = 12

Figure: Distribution generation (normal mean, A = {αm = 3m }).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

24 / 43

P´ olya Trees In Action Random distributions generation

1

´ Barat, E. T. Dautremer

0.75 Density

Nonparametric Bayesian Spectrum Estimation

0.5

Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.25

0 −3

−2

−1

0

1

2

3

m = 12

Figure: Distribution generation (normal mean, A = {αm = 3m }).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

24 / 43

P´ olya Trees In Action Random distributions generation

1

´ Barat, E. T. Dautremer

0.75 Density

Nonparametric Bayesian Spectrum Estimation

0.5

Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0.25

0 −3

−2

−1

0

1

2

3

m = 12

Figure: Distribution generation (normal mean, A = {αm = 3m }).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

24 / 43

P´ olya Trees In Action Random distributions generation

1

´ Barat, E. T. Dautremer Introduction

0.75 Density

Nonparametric Bayesian Spectrum Estimation

0.5

0.25

Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra

0 −3

−2

−1

0

1

2

3

average (20 draws)

Figure: Draws average (normal mean, A = {αm = 3m }).

Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

24 / 43

Finite (partially specified) P´ olya Tree

Finite P´ olya tree Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures

Computational constraints. Partitions and parameters specified until level M < ∞. M-th level subsets Bj = B1 ···M . Random measures are uniform over Bj . for all X ∈ Bj , X ∼ UBj

Hierarchical Model for Physical Spectra Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

25 / 43

P´ olya Tree Mixtures P´ olya tree mixtures

Nonparametric Bayesian Spectrum Estimation

Partitions and parameters depend on an r.v. Ψ
(G |Ψ) ∼ PT ΠΨ , AΨ Ψ ∼ µ (Ψ)

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures

Shifted finite P´ olya tree (G |d) ∼ PT ΠdM , AdM ∆

d∼

1 X δl (d) ∆+1 l=0

Hierarchical Model for Physical Spectra Applications Conclusion Further reading




Partition shifts = λM · d with ∀ j Leb(Bj ) = λM . Mitigation of partitions endpoints discontinuities. Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

26 / 43

Finite P´ olya Tree Process Mixtures Second level mixture : kernel with distribution µ(Xi |·) R If µ(Xi |Yi ) has pdf f (Xi |Yi ), µ(Xi |Bj ) has pdf Bj f (Xi |y ) dy . Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´ olya tree process mixtures Hierarchical Model for Physical Spectra Applications Conclusion

Shifted P´ olya tree mixture

Alternate SPTM ind

ind

(Xi |K) ∼ µ (Xi |BKi )

(Xi |Yi ) ∼ µ (Xi |Yi ) i.i.d.

(Yi |G ) ∼ G

i.i.d

(G |d) ∼ PT ΠdM , AdM




∆

1 X d∼ δl (d) ∆+1 l=0

Define q = (q1 , . . . , q2M −∆ ) with qj = G (Bj )

Further reading

Nonparametric Bayesian Spectrum Estimation

(Ki |q) ∼

M 2X −∆

qk δk

k=1

(q|d) ∼ µ (q|d) ∆

d∼

1 X δl (d) ∆+1 l=0

with classif. Ki = j if Yi ∈ Bj MaxEnt 2006

July 13, 2006

27 / 43

Outline

Nonparametric Bayesian Spectrum Estimation

1

Introduction Motivation Bayesian approaches A discrete vs. continuous mixture model

2

Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´olya tree process mixtures

3

Hierarchical Model for Physical Spectra Physical motivations Hierarchical model

4

Applications

5

Conclusion

6

Further reading

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Physical motivations Hierarchical model Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

28 / 43

Physical motivations of Statistical Model Peaks spectrum as a Dirichlet mixture Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer

Underlying discrete distribution. Eventually infinitely many energy locations. Predictive densities may embrace complex phenomena which take away discreteness of energy distribution.

Introduction Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Physical motivations Hierarchical model Applications Conclusion Further reading

Background spectrum as a P´ olya tree mixture Background belonging relies on continuity of mixing distribution. Parameters of P´ olya trees may depend on peaks locations (hierarchical model). Many analytic expressions arise from parametric approximations of usually more complex phenomena.

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

29 / 43

Nonparametric Bayesian Paradigm Nonparametric Bayesian approach for Complex Physical Phenomena

Physics in the prior mean distributions Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Physical motivations Hierarchical model Applications Conclusion Further reading

The nonparametric Bayesian approach let the analyst rely upon his understanding of the phenomenon by means of an approximated analytical description embedded in the mean distribution of the physical quantity. Nonparametrics for data model complexity The data complexity, not represented by analytic models, will be tackled by nonparametric random distributions. Concentration around the mean distribution The prior hyperparameters balance deviations around the mean distribution according to our degree of belief.

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

30 / 43

Nonparametric Bayesian Paradigm Nonparametric Bayesian approach for Complex Physical Phenomena

Physics in the prior mean distributions Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Physical motivations Hierarchical model Applications Conclusion Further reading

The nonparametric Bayesian approach let the analyst rely upon his understanding of the phenomenon by means of an approximated analytical description embedded in the mean distribution of the physical quantity. Nonparametrics for data model complexity The data complexity, not represented by analytic models, will be tackled by nonparametric random distributions. Concentration around the mean distribution The prior hyperparameters balance deviations around the mean distribution according to our degree of belief.

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

30 / 43

Nonparametric Bayesian Paradigm Nonparametric Bayesian approach for Complex Physical Phenomena

Physics in the prior mean distributions Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Physical motivations Hierarchical model Applications Conclusion Further reading

The nonparametric Bayesian approach let the analyst rely upon his understanding of the phenomenon by means of an approximated analytical description embedded in the mean distribution of the physical quantity. Nonparametrics for data model complexity The data complexity, not represented by analytic models, will be tackled by nonparametric random distributions. Concentration around the mean distribution The prior hyperparameters balance deviations around the mean distribution according to our degree of belief.

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

30 / 43

Outline

Nonparametric Bayesian Spectrum Estimation

1

Introduction Motivation Bayesian approaches A discrete vs. continuous mixture model

2

Nonparametric Bayesian density estimation Priors on Probability Distributions Truncated Dirichlet process mixtures Finite P´olya tree process mixtures

3

Hierarchical Model for Physical Spectra Physical motivations Hierarchical model

4

Applications

5

Conclusion

6

Further reading

´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Physical motivations Hierarchical model Applications Conclusion Further reading

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

31 / 43

Basics of Hierarchical Model

Key idea Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Physical motivations Hierarchical model Applications Conclusion Further reading

Cast the problem in terms of a finite number of r.v. Update blocks of parameters. Combination of Dirichlet and P´ olya tree mixtures Classification vector K embraces either Dirichlet process components or P´ olya tree subsets. The balance between Dirichlet process and P´olya tree is Beta distributed. Hyperparameters are introduced for kernel characterization. P´olya tree mean distribution (A parameters) may depend on Dirichlet process generated random distribution.

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

32 / 43

Hierarchical Model for Physical Spectra Hierarchical model ( µ (Xi |ZKi , η) if Ki ≤ N (Xi |Z, K, η) ∼ µ (Xi |BKi −N , η) otherwise ind

Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer

i.i.d.

(Ki |p, q) ∼ w

Introduction Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Physical motivations Hierarchical model Applications Conclusion Further reading

N X

pk δk (·) + (1 − w )

k=1

M 2X −∆

qk δk+N (·)

k=1

  G |d, p, Z ∼ PT Ad,p,Z , ΠdM , qj = G (Bj ) M ∆

d∼

1 X δl (d) 1+∆ l=0

N

(p, Z) ∼ µ (p) × (G0 ) w ∼ Beta (νP , νB ) η∼H

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

33 / 43

Gibbs/MCMC sampler The Gibbs sampler will successively draw samples from : Blocked Gibbs sampler (with eventually MCMC steps) Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Physical motivations Hierarchical model Applications Conclusion Further reading

(Z|K, η, X) (K|Z, p, q, d, w , η, X) (q, d|K, p, Z) (p|K) (w |K) (η|K, Z, X) Posterior draws After convergence the blocked Gibbs sampler may generate draws from (K, Z, p, q, d, w , η|X).

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

34 / 43

Posterior Draws Peaks spectrum From draws (K∗ , Z∗ , p∗ , q∗ , d ∗ , w ∗ , η ∗ ) we build draws from PN |X Nonparametric Bayesian Spectrum Estimation

PN∗ (·) =

N X

pk∗ δZk∗ (·)

k=1

´ Barat, E. T. Dautremer Introduction

Generating PN∗ (·), we can estimate PN |X and its functionals.

Nonparametric Bayesian density estimation

Background spectrum

Hierarchical Model for Physical Spectra

From draws (K∗ , Z∗ , p∗ , q∗ , d ∗ , w ∗ , η ∗ ) we build draws from GM |X

Physical motivations Hierarchical model Applications Conclusion Further reading

∗ GM

(·) =

M 2X −∆

qk∗ UBk (·)

k=1 ∗ Generating GM (·), we can estimate GM |X and its functionals. Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

35 / 43

Binned Data

Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Physical motivations Hierarchical model

Binned data extension Model of energy distribution before binning. Detectors are necessary... not binning (storage). When only binned data are observed, the Gibbs sampler can be adapted with minor modifications. The densities of µ (Xi |BKi −N , η) and µ (Xi |Yi , η) are integrated over the bin-width and become probability mass functions corresponding to each observation bin. The allocation step of the Gibbs sampler becomes a multinomial distribution where we break down simultaneously all the counts of a given bin.

Applications Conclusion Further reading

Computational issues This binned version appears computationally attractive for huge datasets. Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

36 / 43

Applications Experimental spectrum

Uranium oxide γ-ray spectrum. Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer

Observed data : binned histogram [0 KeV, 2.5 MeV]. 2 regions of interest (ROI) R1 = [400 KeV, 630 KeV] : high background. R2 = [2.25 MeV, 2.47 MeV] : smaller dataset.

Introduction Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Applications Conclusion Further reading

Estimator. Approximated predictive mixing densities : Monte-Carlo average of the mixing distributions posterior draws (DP and PT). Posterior moments and functionals can be computed in the same way.

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

37 / 43

Priors and hyperparameters

Normal instrumental kernel with linear variance. Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Applications Conclusion Further reading

σ 2 (E ) = η1 + η2 · E with η1 and η2 Gamma distributed (MCMC step in Gibbs). Dirichlet process : α = 1, N = 100, uniform mean distribution. P´olya tree : M = 10, A = {am = 6m : m ≤ M}, uniform mean distribution with binary quantile partitions (canonical P´olya tree). Shift range : ∆ = 128. Same priors for both ROI.

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

38 / 43

Results : R1 Region 107 2 104 106

´ Barat, E. T. Dautremer

Density

1 104 Nonparametric Bayesian Spectrum Estimation

5 103 105 530

540

550

104 Introduction Nonparametric Bayesian density estimation

103 400

Hierarchical Model for Physical Spectra

450

500

550

600

Energy (KeV)

Applications Conclusion Further reading

Figure: UO2 experimental energy spectrum, (R1) region : histogram (green), background spectrum –PT predictive distribution– (black), peaks spectrum –DP predictive distribution– (red). MCMC algo. : 20000 iters, burn-in : 10000 iters.

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

39 / 43

Results : R1 Region

Nonparametric Bayesian Spectrum Estimation

2 104

´ Barat, E. T. Dautremer Introduction

1 104

Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra

5 103

Applications Conclusion Further reading

530 Nonparametric Bayesian Spectrum Estimation

540

550 MaxEnt 2006

July 13, 2006

39 / 43

Results : R2 Region 105 104 104

´ Barat, E. T. Dautremer

Density

102 Nonparametric Bayesian Spectrum Estimation

103 100 2440

Introduction

101

Nonparametric Bayesian density estimation

100 2250

Hierarchical Model for Physical Spectra

2450

2460

102

2300

2350

2400

2450

Energy (KeV)

Applications Conclusion Further reading

Figure: UO2 experimental energy spectrum, (R2) region : histogram (green), background spectrum –PT predictive distribution– (black), peaks spectrum –DP predictive distribution– (red). MCMC algo. : 20000 iters, burn-in : 10000 iters.

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

40 / 43

Results : R2 Region

104 Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction

102

Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Applications Conclusion Further reading

100 2440 Nonparametric Bayesian Spectrum Estimation

2450

2460 MaxEnt 2006

July 13, 2006

40 / 43

Wider Region of Interest 106

´ Barat, E. T. Dautremer

Density

104 Nonparametric Bayesian Spectrum Estimation

102 Histogram DP peaks PT background

Introduction Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra

100 1800

2000

2200

2400

Energy (KeV)

Applications Conclusion Further reading

Figure: UO2 experimental energy spectrum : histogram (green), background spectrum –PT predictive distribution– (black), peaks spectrum –DP predictive distribution– (red). MCMC algo. : 20000 iters, burn-in : 10000 iters, M = 12, A = {am = 4m : m ≤ M}.

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

41 / 43

Wider Region of Interest

Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra Applications Conclusion Further reading

ound Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

41 / 43

Conclusion Nonparametric Bayesian physical spectra estimation

Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra

Our hierarchical model for physical spectra allows efficient Gibbs sampling. Bayesian nonparametrics priors over probability distributions are able to tackle physical data complexity. Posterior draws of the random mixing distributions produce separated nonparametric deconvoluted spectra. Alternative posterior exploration methods are suited to speed-up the approach. Application area

Applications Conclusion Further reading

It is in authors belief that various problems relying on a discrete vs. continuous measure separation are addressable by the approach. Theoretical aspects Consistency of the estimator remains to be studied. Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

42 / 43

For Further Reading.

Nonparametric Bayesian Spectrum Estimation ´ Barat, E. T. Dautremer Introduction Nonparametric Bayesian density estimation Hierarchical Model for Physical Spectra

R. Fischer, K. M. Hanson, V. Dose, and W. von der Linden. Phys. Rev. E 61, 1152–1160 (2000), H. Ishwaran, and L. F. James. J. Am. Stat. Assoc. 96, 161–173 (2001). M. Lavine. Ann. Statist. 20, 1222–1235 (1992). Z. Ghahramani. Non-parametric Bayesian Methods., Tutorial UAI, 2005.

Applications Conclusion Further reading

J. K. Ghosh, and R. V. Ramamoorthi. Bayesian Nonparametrics, Springer, 2003.

Nonparametric Bayesian Spectrum Estimation

MaxEnt 2006

July 13, 2006

43 / 43