MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
1 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering Paris, France, July 8-13, 2006
Sponsored by: Edwin T. Jaynes International Center for Bayesian Methods and Maximum Entropy Centre national de la recherche scientique, France Universit´e de Paris-sud, Orsay, France Laboratoire des signaux et syst`emes, France Minist`ere de la d´efense, Direction de la recherche et d´eveloppement, France International Society for Bayesian Analysis
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
2 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
M. DATCU
A. CATICHA
break time
C. CAVES
A PHORORILLE
lunch time
B. LECOUTRE
A. M.-DJAFARI
Registration & Coffee
17:15
16:00 16:30
15:00
12:00 14:00
11:00
10:00
9:00
Monday 10/07
M. BABIE-ZADEH (82)
S. MOUSSAOUI (83)
S. HOSSEINI (58)
break time
F. VRINS (81)
Y. NISHIMORI (87)
C. F. CAIAFA (10)
lunch time Quantom Mech., Inverse Pblms. & Source Sep. (U. v. TOUSSAINT)
Poster Presentations
A. VOUDRAS (35)
break time C. M. CAVES invited talk (93)
L. F. Lemmens (57)
A. CATICHA (48/49)
OPENING & REGISTRATION A. POHORILLE invited talk (32)
Quantum Mechanics
Source Separation
Sunday 09/07 (tutorials)
C. M. CAVES
A. CATICHA
Poster Session
K. KNUTH
C. JUTTEN
18:00
17:30
16:30 17:00
16:00
15:30
15:00
13:00 14:00
12:30
12:00
10:45 11:15
10:15
9:45
8:30 9:00 Information Geometry & Bayesian Nets Inverse Problems
The "Diner Banket" (19h30 - 23h30)
V. MAZET (17)
R. PREUSS (76)
break time
U. v. TOUSSAINT (52)
R. FISCHER (66)
K. M. HANSON (72)
Bayesian Networks & Inverse Problems (R. FISCHER)
lunch time
Poster Presentations
E. BJÖRNEMO (63)
D. E. HOLMES (6)
N. CATICHA (15)
break time
A. RAMER (100)
C. C. RODRIGUEZ (37)
S. FIORI invited talk (92)
Tuesday 11/07
17:30
16:30 17:00
16:00
15:30
15:00
13:00 14:00
12:30
12:00
11:30
10:30 11:00
10:00
9:30
8:45
Committee Meeting
Group Photo
Applications & Image Processing (Z. GHAHRAMANI)
lunch time
Poster Presentations
M. SOCCORSI (12)
G. DELYON (18)
W. PIECZYNSKI (21)
break time
J. SKILLING (67)
K. KNUTH (55)
Z. GHAHRAMANI invited talk (74)
Wednesday 12/07 J. CENTER J. SKILLING
Bayesian Inference & Image Processing Poster Session
H. SNOUSSI C. RODRIGUEZ
Poster Session R. PREUSS K. M. HANSON
16:00
15:00
13:00 14:00
12:30
12:00
11:30
10:30 11:00
10:00
9:30
8:45
Entropy & Data Processing Applications
M. GRENDAR V. GIRARDIN E. BARAT
17:00
16:30
15:30 16:00
15:00
14:30
12:45 14:00
12:15
11:45
11:15
10:15 10:45
9:45
9:15
8:45
invited talk talk break lunch poster session
0:45 0:30 0:30 1:00 1:00
Allocated times (min)
F. DESBOUVRIES (39) Tj. R. BONTEKOE (29)
E. BARAT (24)
break time
M. JOHANSSON (51)
J. WELCH (71)
J. M. STERN (68)
lunch time
M. GRENDAR (85)
A. SOLANA-ORTEGA (89) P. L. N. INVERARDI (9)
A. ZARZO (75)
break time
E. V. VAKARIN (64)
J. F. BERCHER (23)
V. GIRARDIN (38)
Thursday 13/07
CNRS ,
J. M. STERN
MaxEnt 2006 — Paris, France, July 8-13, 2006 3 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
4 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 — CNRS, Paris, France, July 8-13, 2006 MaxEnt 2006 Program
5 / 205 file:///home/giovannelli/Tex/Art/MaxEnt06/DiversPDF/program.htm
MaxEnt 2006 Program Information Saturday July 8, 2006: Welcome reception (Cité Internationale de Paris) 14:00 16:00 Registration 16:00 18h00 Welcome reception
Sunday July 9, 2006, Tutorial day (Cité Internationale de Paris) 09h00 10h00 Registration and Coffee 10h00 11h00 Tutorial 1: Ali Mohammad-Djafari: Maximum Entropy and Bayesian inference: Where do we stand and where do we go ? 11h00 12h00 Tutorial 2: Bruno Lecoutre: And if you were a Bayesian without knowing it ? 12h00 14h00 Lunch Tutorial 3: Andrew Pohorille: Exploring the connection between 14h00 15h00 sampling problems in Bayesian inference and statistical mechanics 15h00 16h00 Tutorial 4: Carlton Caves: Introduction to quantum computation 16h00 16h15 Break 16h15 17h00 Tutorial 5: A. Caticha: Updating probabilities Tutorial 6: Mihai Datcu: Information theory based inference in the 17h00 17h45 Bayesian context: applications for semantic image coding Footbal World cup: Walk around Paris coffees and share the 18h00 22h00 footbal excitings.
Monday July 10, 2006, (CNRS, Paris) 8h00 8h30
8h30 9h00
Registration Opening and official talks
Oral session 1: Information, Probability, Quantum systems (Chair: C. Caves) Invited talk 1: A. Pohorille: A Bayesian approach to calculating 09h00 09h45 free energies of chemical and biological systems 09h45 10h15 A. Caticha From objective amplitudes to Bayesian probabilities L.F. Lemmens Probability assignment in a quantum statistical 10h15 10h45 model 10h45 11h15 Break Oral session 2: Bayesian Probability, Quantum systems (Chair: A. Caticha) Invited Talk 2: C. Caves Why We Should Think of Quantum 11h15 12h00 Probabilities as Bayesian Probabilities A. Vourdas : Phase space methods in continuous tensor products 12h00 12h30 of Hilbert spaces One minute Poster Session 1 presentation (Chair: U.V. 12h30 13h00 Toussaint) 13h00 14h00 Lunch
1 sur 4
27.06.2006 17:09
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 — CNRS, Paris, France, July 8-13, 2006 MaxEnt 2006 Program
6 / 205 file:///home/giovannelli/Tex/Art/MaxEnt06/DiversPDF/program.htm
Poster session 1: (Chair: U.V. Toussaint) 003_Bityukov, 004_Barkova, 042_Lin, 046_Gadjiev, 14h00 15h00 050_Bastami, 065_Yanez, 079_Amiri-Sazi, 095_Olmos, 096_Lopez-Rosa, 019_Mohtashami, 041_Finn, 053_Yari, 090_Pappalardo. Oral session 3: Source Separation (Chair: Ch. Jutten) 15h00 15h30 C.F. Caiafa : A minimax entropy method for blind separation of dependent components in astrophysical images 15h30 16h00 Y. Nishimori : Riemannian optimization method on the generalized flag manifold for complex and subspace ICA 16h00 16h30 F. Vrins : Electrode selection for non-invasive Fetal Electrocardiogram Extraction using Mutual Information Criteria 16h30 17h00 Break Oral session 4: Source Separation (Chair: K. Knuth) Sh. Hosseini: Maximum likelihood separation of spatially 17h00 17h30 auto-correlated images using a Markov model Moussaoui Mars Hyperspectral Data Processing using ICA and 17h30 18h00 S. Bayesian Positive Source Separation M. Babaie-zadeh A fast method for sparse component analysis 18h00 18h30 based on iterative detection-projection
Tuesday July 11, 2006: (CNRS, Paris) Oral session 5: Information Geometry and Bayesian nets (Chair: H. Snoussi) Invited talk 3: S. Fiori: Extrinsic geometrical methods for neural 8h30 9h15 blind deconvolution 9h15 9h45 C. Rodriguez : Data, Virtual Data, and Anti-Data 9h45 10h15 A. Ramer : GraphMaxEnt 10h15 10h45 Break Oral session 6: Information Geometry - Bayesian nets (Chair: C. Rodriguez) Caticha : The evolution of learning systems: to Bayes or not to 10h45 11h15 N. be D.E. Holmes : Determining Missing Constraint Values in Bayesian 11h15 11h45 Networks with Maximum Entropy: A First Step Towards a Generalized Bayesian Network E. Bjornemo : Sensor network node scheduling for estimation of a 11h45 12h15 continuous field 12h30 13h00 One minute Poster Session 2 presentation (Chair: R. Fischer) 13h00 14h00 Lunch Poster session 2: (Chair: R. Fischer) 002_Borges, 007_Dobrzynski, 011_Barbaresco, 016_Center, 022_Bercher, 025_Dautremer, 026_Niven, 031_Alamino, 14h00 15h00 034_Neisy, 043_Cafaro, 044_Dodt, 045_Dreier, 047_Krajsek, 061_Mohammad-Djafari, 063_Bjornemo, 073_Sahmoodi, 077_Costache, 078_Esmer, 080_Kiss, 084_Roy, 084_Karimi, 091_Snoussi.
2 sur 4
27.06.2006 17:09
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 — CNRS, Paris, France, July 8-13, 2006 MaxEnt 2006 Program
7 / 205 file:///home/giovannelli/Tex/Art/MaxEnt06/DiversPDF/program.htm
Oral session 7: Inverse problems (Chair: R. Preuss) 15h00 15h30 K.M. Hanson : Probing the covariance matrix 15h30 16h00 R. Fischer : Integrated data analysis: non-parametric profile gradient estimation 16h00 16h30 U.V. Toussaint : Parameter estimation of ellipsometry measurements 16h30 17h00 Break Oral session 8: Inverse problems (Chair: K. Hanson) 17h00 17h30 R. Preuss Bayesian analysis on plasma confinement data bases V. Mazet, D. Brie and J. Idier : Decomposition of a chemical 17h30 18h00 spectrum using a marked point process and a constant dimension model 19h30 23h30 Conference Dinner
Wednesday July 12, 2006, (CNRS, Paris) Oral session 9: Bayesian inference and Image processing (Chair: J. Center) Invited talk 4: Z. Ghahramani A Bayesian approach to information 8h30 9h15 retrieval from sets of items K. Knuth Clearing up the mysteries: computing on hypothesis 9h15 9h45 spaces 9h45 10h15 J. Skilling: Calibration and interpolation 10h15 10h45 Break Oral session 10: Bayesian inference - Image processing (Chair: J. Skilling) W. Pieczynski : Unsupervised segmentation of hidden 10h45 11h15 semi-Markov non stationary chains G. Deylon : Minimal stochastic complexity image partionning with 11h15 11h45 non parametric statistical model M. Soccorsi : Space-Variant Model Fitting and Selection for 11h45 12h15 Image Denoising and Information Extraction 12h15 13h00 One minute Poster Session 3 presentation (Chair: Z. Gharamani) 13h00 14h00 Lunch Poster session 3: (Chair: Z. Gharamani) 001_Kyo, 005_Khireddine, 008_Zarpak, 013_Gueguen, 014_Chaabouni, 028_Roemer, 030_Goggens, 14h00 15h00 040_Desbouvries,020_Jalobeanu, 054_Amintoosi, 062_Aronsson, 069_Aminghafari, 070_Mehmood, 071_Verdoolaege, 086_Abrishami, 097_Mohammadpour. 15h00 15h30 Group Photo (Chair: R. Bontekoe) 15h30 16h30 Committee meeting (Chair: K. Knuth) Free time
Thursday July 13, 2006, (CNRS, Paris) Oral session 11: Entropy and Data Processing (Chair: M. Grendar)
3 sur 4
27.06.2006 17:09
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 — CNRS, Paris, France, July 8-13, 2006 MaxEnt 2006 Program
8 / 205 file:///home/giovannelli/Tex/Art/MaxEnt06/DiversPDF/program.htm
8h45
9h15
V. Girardin : Entropy and semi-Markov processes
9h15
9h45
J.F. Bercher : An amended MaxEnt formulation for deriving Tsallis factors, and associated issues
9h45
10h15 E.V. Vakarin : Maximum entropy approach to characterization of random media
10h15 10h45 Break Oral session 12: Entropy and Data Processing (Chair: V. Girardin) 10h45 11h15 A. Zarzo : The minimum cross-entropy method: a general algorithm for one-dimensional problems 11h15 11h45 A. Solana-Ortega : Entropic inference for assigning probabilities: some difficulties in axiomatics and applications 11h45 12h15 M. Grendar : Empirical maximum entropy methods 12h15 12h45 P.L.N Inverardi : A New Bound for Discrete Distributions based on Maximum Entropy 13h00 14h00 Lunch Oral session 13: Entropy, Bayes and Applications (Chair: E. Barrat) Stern : The Full Bayesian Significance Test for Separate 14h00 14h30 J.M. Hypotheses 14h30 15h00 J. Welch : Comparing Class Scores in GCSE Modular Science 15h00 15h30 M. Johansson : Competitive bidding in a certain class of auctions 15h30 16h00 Break Oral session 14: Entropy, Bayes and Applications (Chair: J.M. Stern) 16h00 16h30 E. Barat : Nonparametric Bayesian estimation of x/gamma-ray spectra using a hierarchical Polya tree -- Dirichlet mixture model 16h30 17h00 F. Desbouvries : Entropy computation in partially observed Markov chains 17h00 17h30 Tj. R. Bontekoe : Scheduling of schools 17h30 18h30 Ending session (Chair: Local organizers)
4 sur 4
27.06.2006 17:09
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
List of authors B. Abbasi Bastami: p. 106 H. Abrishami: p. 174 B. Ait-el-Fquih: p. 86 S. Akaho: p. 176 R.C. Alamino: p. 70 S.A. Ali: p. 92 A.A. Amini: p. 166 M.G. Amin: p. 148 H. Amindavar: p. 106 M. Aminghafari: p. 140 M. Amiri-Sazi: p. 160 J.C. Angulo: p. 192 D. Aronsson: p. 128, 126
M. Datcu: p. 156, 38, 36, 34, 198 T. Dautremer: p. 60, 58 J.S. Dehesa: p. 132, 190, 192 V. Dehghan: p. 174 G. Delyon: p. 46 F. Desbouvries: p. 84, 86 Y. Deville: p. 120 A. Dinklage: p. 134, 154, 96, 94 L. Dobrzynski: p. 24 D. Dodt: p. 94 V. Dose: p. 134, 154 H. Dreier: p. 96 B. Duriez: p. 54
M. Babaizadeh: p. 166 B. Bakar: p. 118 E. Barat: p. 60, 58 F. Barbaresco: p. 32 E.A. Barkova: p. 18 K. Benmahammed: p. 20 J.-F. Bercher: p. 56, 54 W. Biel: p. 144 S.I. Bityukov: p. 18, 16 E. Bj¨ornemo: p. 126, 128 T. R. Bontekoe: p. 66 W. Borges: p. 14 D. Brie: p. 44
A.E. Ekimov: p. 142 O. Esmer: p. 158
C. Cafaro: p. 92 C. Caiafa: p. 30 A. Caticha: p. 104, 102 N. Caticha: p. 70, 40 C.M. Caves: p. 188, 186 J. Center: p. 42 H. Chaabouni: p. 38 Y. Chi: p. 68 M. Costache: p. 156 D. Cowling: p. 64 J.C. Cuchi: p. 152
S.R. de Faria Jr.: p. 138 R. Farnoosh: p. 26 L.S. Finn: p. 88 S. Fiori: p. 184 R. Fischer: p. 96, 134, 94 B.R. Gadgjiev: p. 98 F. Galland: p. 46 Z. Ghahramani: p. 150 A. Giffin: p. 102, 92 V. Girardin: p. 82 P.M. Goggans: p. 142, 68 S. Goodarzi: p. 48 M. Grendar: p. 172 L. Gueguen: p. 36 R. Guidara: p. 120 H. Gzyl: p. 28 K.M. Hanson: p. 146 H. Hauksdottir: p. 168 K.A. Heller: p. 150 C. H´erail: p. 164 M. Hirsch: p. 96
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
D.E. Holmes: p. 22 S. Hosseini: p. 120 J. Idier: p. 44 P.L.N. Inverardi: p. 28
10 / 205
M.D. Plumbley: p. 176 A. Pohorille: p. 72, 74 R. Preuss: p. 154 A. Proto: p. 30 M. Quartulli: p. 34
A. Jalobeanu: p. 50 P. Jardin: p. 54 R. Jaspers: p. 144 M. Johansson: p. 128, 126, 108 G. Judge: p. 172 C. Jutten: p. 120, 166, 168, 164 A. Khireddine: p. 20 E. Kiss: p. 162 K. Knuth: p. 114 K. Krajsek: p. 100 N.V. Krasnikov: p. 16 E. Kuruoglu: p. 30 K. Kyo: p. 12 J. Lapuyade-Lahorgue: p. 52 M.S. Lauretto: p. 138 B. Lecoutre: p. 116 C. Lemen: p. 36 L. F. Lemmens: p. 118 S.-K. Lin: p. 90 S. Lopez-Rosa: p. 192 H. Maitre: p. 156 V. Mazet: p. 44 A. Mehmood: p. 142 R. Mester: p. 100 A. Mohammad-Djafari: p. 182, 194, 196, 124 A. Mohammadpour: p. 194 G.R. Mohtashami Borzadaran: p. 48 G. Mohtashami: p. 112 S. Moussaoui: p. 168 A. Neisy: p. 76 Y. Nishimori: p. 176 R.K. Niven: p. 62 H. Noda: p. 12
A. Ramer: p. 200 P. R´efr´egi´e: p. 46 C. Rodriguez: p. 80 L. Roemer: p. 64 D. Roy: p. 170 J.M. Sabatier: p. 142 M. Sahmoodi: p. 148 R. Sameni: p. 164 D. Satua: p. 24 F. Schmidt: p. 168 J. Skilling: p. 136 V.V. Smirnova: p. 16 H. Snoussi: p. 182 M. Soccorsi: p. 34 A. Solana-Ortega: p. 178 V. Solana: p. 178 J.M. Stern: p. 138 J.M. Stern: p. 14 K. Szymaski: p. 24 A. Tagliani: p. 28 V.A. Taperechkina: p. 18 C. Tison: p. 38 U.v. Toussaint: p. 110 F. Tupin: p. 38 T. Trigano: p. 60 E.V. Vakarin: p. 130 G. Verdoolaege: p. 144 M. Verleysen: p. 164 V. Vigneron: p. 164 F. Vrins: p. 164 M.G. Von Hellermann: p. 144 A. Vourdas: p. 78 J. Welch: p. 122
B. Olmos: p. 190 M. Pappalardo: p. 180 F. Parmentier: p. 164 B.B. Pereira: p. 138 W. Pieczynski: p. 52
R.J. Yanez: p. 132, 190 G. Yari: p. 112, 26 B. Zarpak: p. 26 A. Zarzo: p. 152
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
11 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
12 / 205
BAYESIAN ANALYSIS OF CROSS-PREFECTURE PRODUCTION FUNCTION WITH TIME VARYING STRUCTURE IN JAPAN Koki Kyo∗ and Hideo Noda Asahikawa University 3-23 Nagayama, Asahikawa, Hokkaido 079-8501, Japan Abstract The objective of this paper is to examine the performance of post-war Japanese economy using a production function of economic growth model. The basic framework is a variation of aggregate production function used by Solow (1956), Mankiw, Romer, and Weil (1992), etc. We consider the Cobb=Douglas production function with private capital, public capital, human capital and labour as inputs, so production for prefecture i at time t is represented by Qi (t) = Ki (t)αi Gi (t)βi Hi (t)γi [Ai (t)Li (t)]1−αi −βi −γi
(i = 1, 2, . . . , m),
where Qi (t) is output, Ki (t) is the stock of private capital, Gi (t) is the stock of public capital, Hi (t) is the stock of human capital, Li (t) is the size of the labour force and Ai (t) is a productivity index which summarizes the level of technology. The above model can be expressed in a form of linear model under the logarithmic tranformation. A set of Bayesian models is constructed by using smoothness priors for values related to Ai (t) and non-informative priors for the parameters αi , βi and γi . Furthermore, Monte Carlo filter and smoother approach is applied to estimate the parameters. We show the effects of the private capital, the public capital and the human capital on output by analyzing the values of these parameters. The related result was firstly reported by Kyo and Noda (2005). References: [1] K. Kyo, and H. Noda (2005), Statistical analysis of cross-prefecture production function with dynamic structure in Japan, Paper Presented at International Symposium: Intersection, Fusion and Development of Multi-Fields, Chinese Academy of Science and Engineering in Japan. [2] Mankiw, N. G., D. Romer, and D. Weil (1992), A Contribution to the Empirics of Economic Growth, Quarterly Journal of Economics, Vol.107, pp.407-437. [3] Solow, R. M. (1956), A Contribution to the Theory of Economic Growth, Quarterly Journal of Economics, Vol.70, pp.65-94. Key Words: PRODUCTION FUNCTION, SMOOTHNESS PRIORS, MONTE CARLO FILTER AND SMOOTHER ∗
E-mail:
[email protected]
FAX: +81-166-488718
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
13 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
14 / 205
FBST: Compositionality Wagner Borges∗ and Julio M. Stern† ∗
Mackenzie Presbiterian University,
[email protected] † University of São Paulo,
[email protected]
Abstract. In this paper, the relationship between the credibility of a complex hypothesis, H, and those of its constituent elementary hypotheses, H j , j = 1 . . . k, is analyzed, in the independent setup, under the Full Bayesian Significance Testing (FBST) mathematical apparatus. Key words: Bayesian models; Complex hypotheses; Compositionality; Mellin convolution; Possibilistic and probabilistic reasoning; Significance tests; Truth values, functions and operations.
INTRODUCTION The Full Bayesian Significance Test (FBST) has been introduced by Pereira and Stern (1999), as a coherent Bayesian significance test for sharp hypotheses. For detailed definitions, interpretations, implementation and applications, see the authors’ previous articles, including two papers in this conference series, [9], [17]. In this paper we analyze the relationship between the credibility, or truth value, of a complex hypothesis, H, and those of its elementary constituents, H j , j = 1 . . . k. This problem is known as the question of Compositionality, which plays a central role in analytical philosophy, see [3]. According to Wittgenstein [22], (2.0201, 5.0, 5.32): - Every complex statement can be analyzed from its elementary constituents. - Truth values of elementary statement are the results of those statements’ truthfunctions (Wahrheitsfunktionen). - All truth-function are results of successive applications to elementary constituents of a finite number of truth-operations (Wahrheitsoperationen). The compositionality question also plays a central role in far more concrete contexts, like that of reliability engineering, see [1] and [2], (1.4): “One of the main purposes of a mathematical theory of reliability is to develop means by which one can evaluate the reliability of a structure when the reliability of its components are known. The present study will be concerned with this kind of mathematical development. It will be necessary for this purpose to rephrase our intuitive concepts of structure, component, reliability, etc. in more formal language, to restate carefully our assumptions, and to introduce an appropriate mathematical apparatus.” When brought into a parametric statistical hypothesis testing context, a complex hypothetical scenario or complex hypothesis is a statement, H, concerning θ = (θ 1 , . . . , θ k ) ∈ Θ = (Θ1 × . . . × Θk ) which is equivalent to a logical composition of statements, H 1 , . . . , H k , concerning the elementary components, θ 1 ∈ Θ1 , . . . , θ k ∈ Θk , respectively. Within this setting, means to evaluate the credibility of H, as well as that of
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
15 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
16 / 205
!#" $%!# !#" &('*),+-'*. /,02143657.989:= ?A@CB 36:pPO]O7 X~sw^w.lXd -pTl1F7Eu&Dl]0]b>p, W Nh
,g>U86Ne-0o2DFNe>%*>@S>U1^DF/2143VPNO-©t½P¬¤³¦L´t¤]¶f¶¡_§L¡»µ¼B·¨¬y¡_¥¥¦L©Y¡t°Ã·&°©Yª¬L¨¬§L¬F¨ª w¨ª¨ w¨Ù]¨ª«£&¡ä©Y¨ª`§w¤]ªV§¦©t¥`º Ä ·P¨°· °©t¦¦L¤]¬¥¯©Yª¶¬®§©:§L·«¤¬² ©tÁÀ§L·«¤Uªq² ±½¯¤³¦w©tÁÀ½P¨§L¬ ª«¤³¤]¶«¤]¶¾§©Ú¶«¤]¬L°¦¨½¯¤µ§L·«¤¶¡_§L¡¢¡tª¶¾§L·«¤¿ªq² ±½¯¤³¦.©tÁX½P¨§L¬Bª«¤³¤]¶«¤]¶¾§©Ú¶«¤]¬L°¦¨½¯¤µ§L·«¤ ®©¶«¤]¹Ç» å ¤D¥¦L©t¥¯©Y¬¤¢¡£t¤]ª«¤³¦¡t¹S¨ ¢¡_£t¤¢¥P¡_¦L§L¨§L¨©Yª¨ª«£Â w¤³§L·«©q¶:½P¡t¬¤]¶:©Yªh¡¬§L¡_§L¨¬§L¨°³¡t¹M¡t°§L¨´t¤ £t¦¨¶¾æÆÞ
ço»p¼B·¨¬X§¤]°·Pª¨Ìq²«¤.¡t¹¹© Ä ¬4²P¬O§©F¤]¬L§L¨ w¡_§¤.§L·«¤S¥¦L©t½¡_½P¨¹¨§1º¶«¤]ª¬L¨§§©¸§L¡_Ët¤À¨ª`§©±¡t°³°©Y²ª`§ §L·«¤µ¥P·`ºq¬L¨°³¡t¹P©t¦¨£Y¨ª¢©tÁX§L·«¤FéP²°§L²¡_§L¨©YªP¬É©tÁX§L·«¤F£t¦L¤³ºÚ¹¤³´t¤]¹¬ Ä ¨§L·«©Y²«§S w¡_˨ª«£¸·Vºq¥¯©t§L·«¤]¬¤]¬ ©Yªê§L·«¤Ú¥¶«ÁRÁè¡t w¨¹º Ä ·¨°·ê¶«¤]¬°¦¨½¯¤]¬¡_§¿½¯¤]¬§¿§L·«¤¢éP²P°§L²¡_§L¨©Yª¬³»Úå ¤¶¤] ®©Yª¬§¦¡_§¤Ú§L·¡_§ §L·«¤Ê¥¡_¦L§L¨§L¨©Yª¨ª«£°³¡tª¢½¯¤B©t½P§L¡t¨ª«¤]¶w½qºw w¨ªP¨ w¨Ù]¨ª£F¡±°¦¨§¤³¦¨©Yª Ä ¨§L·«©Y²§p¥P¡_¦¡t ®¤³§¤³¦S§©¿½¯¤ §L²ª«¤]¶D½qºD§L·«¤²¬¤³¦Ê¡tªP¶¾§L·¡_§Ê°©t¦L¦¤]¬¥¯©Yª¶¬S§©¢¡wª©YªD¥P¡_¦¡t ®¤³§¦¨°À¬§L¡_§L¨¬§L¨°³¡t¹X¡_¥¥¦L©Y¡t°Ã·» ÔB¤³ÁȤ³¦L¤]ª°¤]¬³ë æÆÞ
çFÕM».ϵ¡t¹¹¡tª¶ì³í±îtïÆðÒ(ñLÓ¨ª¨ \² ò¶«¤]¬°¦¨¥§L¨©Yª9¹¤]ª«£t§L· ¬ºªV§L·¤³§L¨°U¡_¥¯¤³¦L§L²«¦¤¾¦¡t¶¡_¦ ¨ w¡_£t¤¿¬L¤³£Y ®¤]ªV§L¡_§L¨©YªÒôóhõöÉöRöÑõ÷SÒ´t©Y¹Ç»BÞ]øqÒ"ª«©«»RßqÒ¥¥ »RßtßtùúÞ³ûtûYüqÒø_ûtûYýq» þÀ¤³ºå ©t¦¶¬³ë0 w¡_£t¤¿¥P¡_¦L§L¨§L¨©YªP¨ª«£«Ò Í §©q°Ã·¡t¬§L¨°\ÝM©Y ®¥P¹¤Î«¨§1ºtÒÿy¤]ªP¬L¨§ 0) with Q(t) the number of clients in the queue, we improve the method by using additional information from the distribution of the number of particles participating to a busy period conditionally to its length. References: [1] Y. Kim, Annals of statistics 27, 562-588 (1999). [2] M. Lavine, Annals of statistics 20, 1222-1235 (1992). [3] R. Pyke, Annals of Mathematical Statistics 29, 737-754 (1958). Key Words: Bayesian nonparametrics, Type-II counter, Mt /G/∞, Polya tree.
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
61 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
62 / 205
MaxEnt Velocity Profiles in Laminar to Turbulent Flow Robert K. Niven School of Aerospace, Civil and Mechanical Engineering The University of New South Wales at ADFA Northcott Drive, Canberra, ACT, 2600, Australia. Email:
[email protected] 28 February 2006 Abstract This work applies the differential equation method developed by Chiu and co-workers1 based on Jaynes’ maximum entropy method2 - to determine the “most probable” steady-state velocity profile u(y) in three systems of “classical” fluid mechanics: (i) axial flow in a cylindrical pipe (Poiseuille flow) (previously examined by Chiu1); (ii) flow between stationary parallel plates; and (iii) flow between moving parallel plates (Couette flow). In each case, the analysis yields an analytical solution for the velocity profile over the complete spectrum of laminar to turbulent flow. The profiles are obtained as functions of the maximum velocity um and parameter M = !um"1 , where !1 is the Lagrangian multiplier for the conservation of mass constraint. M can be interpreted as a “temperature of turbulence”, with M=0 indicating laminar flow and M ! " complete turbulence. The main elements of this analysis, which have been presented briefly3, are reproduced here. For the axial flow system, the predicted profiles and their moments reduce to the well-known laminar solution at M=0. For M>0, the resulting solution can be used in place of existing semi-empirical correlations for the velocity profile in axial flow1,4. For the plane parallel flows, in order to match both the laminar profiles and higher order moments at M=0, it is necessary to make use of the relative entropy (Kullback-Liebler cross-entropy) function, incorporating a different Bayesian prior (Jaynes’ invariant) distribution. A method to determine this prior distribution is described. The analysis is then used to derive a new maximum-entropy laminar-turbulent boundary layer theory, for the velocity profile in steady flow along a flat plate. For M=0, this reduces to the laminar boundary layer theory given in some texts4, which approximates the Prandtl-Blasius solution to the Navier-Stokes equation5. For turbulent flow, it yields a previously unreported solution. Keywords: MaxEnt; fluid mechanics; velocity profile; turbulent flow; boundary layers. References: 1 2 3 4 5
C.-L. Chiu (1987) J. Hydraul. Eng.-ASCE 113(5) 583; C.-L. Chiu (1988) J. Hydraul. Eng.-ASCE 114(7): 738; C.-L. Chiu, G.-F. Lin, J.-M. Lu, (1993) J. Hydraul. Eng.-ASCE 119(6): 742. E.T. Jaynes (1957) Phys. Rev. 106: 620. R.K. Niven (2005) 3rd Int. Conf.: NEXT-Sigma-Phi, 3-18 August 2005, Kolymbari, Crete, Greece. R.L. Street, G.Z. Watters, J.K. Vennard (1996) Elementary Fluid Mechanics, 7th ed., John Wiley, NY. H. Schlichting, K. Gersten (2001), Boundary Layer Theory, 8th ed., Springer, NY.
1
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
63 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
64 / 205
Bayesian Methods in Ground Penetrating Radar Louis Roemer and David Cowling Abstract: A low frequency interferometer is used to collect data on subsurface obstacles. A simple model for the near-field reflection allows computation of the probability of the target location. The antennas of the interferometer are orthogonally polarized, and a balancing mechanism allows minimizing direct transmission of signals. Thus, the interferometer shows, mainly, reflections from changes in the geometry. Applications tested were land mine location and utility pipe location. An added benefit is that reflections from layers of soil, due to the constant reflection from the layer, do not appear as distracting false targets.
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
65 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
66 / 205
Scheduling of schools Tj. Romke Bontekoe1 , Do Kester2 , John Skilling3 (1) Bontekoe Research, Rooseveltstraat 4-d, 2321 BM Leiden, The Netherlands (2) SRON, Postbus 800, 9700 AV Groningen, The Netherlands (3) Killaha East, Kenmare, Kerry, Ireland (e-mail:
[email protected]) Abstract The scheduling of schools where students, teachers, rooms and their lessons according the curriculum is a large combinatorial optimization problem. It appears that there many solutions for a medium sized Dutch secondary school (1200 students, 100 teachers, 50 rooms). Therefore one can optimize, i.e. select a better solution from the lot. We have developed a computational method to find such solutions. We distinguish between “hard wishes” and “soft wishes”. In order to have a valid schedule all hard wishes must be fulfilled, such as the lessons table, student clusterings, teacher availability, room restrictions, etc. Even if a only single lesson cannot be placed, the schedule is invalid. In fact, very many valid school schedules are computed and as many “soft wishes” as possible are fulfilled. The main soft wish is minimization of idle hours for students. There are many more soft wishes for which the relative importance can be adjusted. We find solutions which comply with all hard wishes and a balanced compromise between the soft wishes. Key Words: Combinatorics, Optimization, School, Schedule
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
67 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
68 / 205
Electromagnetic Induction Landmine Detection using Bayesian Model Comparison Paul M. Goggans and Ying Chi The University of Mississippi Electrical Engineering Department University, MS 38677 Electromagnetic induction (EMI) landmine detection can be cast as a Bayesian model comparison problem. The models used for low metallic-content mine detection are based on the equivalent electrical circuit representation of the EMI detection system. The EMI detection system is characterized and modeled by the impulse response of its equivalent circuit. The analytically derived transfer function between the transmitter coil and receiver coil demonstrates that the EMI detection system is a third order system in the absence of a mine and that the presence of a mine adds an additional pole that makes the detection system fourth order. The value of the additional pole is determined by the equivalent inductance and resistance of the mine and is unique for each mine. This change in system order suggests that measured system impulse responses can be used in conjunction with impulse response models to infer the presence or absence of a landmine. The difficulty of this techniques is that the amplitude of the term added to the the system impulse response by the landmine is small compared to the impulse response of the system alone. To test the feasibility of Bayesian inference based EMI landmine detection, an EMI detection system experiment was designed and built. In the experiment the EMI detection system was driven by a broadband maximal-length sequence (MLS) in order to obtain sufficient dynamic range in the measured impulse responses. This paper discusses the development of parameterized impulse response models for the detections system with and without a landmine present and the assignment of appropriate priors for the parameters of these models. This paper also presents the ratios of computed posterior probabilities for the mine and no mine models based on data obtained from the experimental EMI landmine detection system. These odds ratios demonstrate the potential of Bayesian EMI landmine detection.
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
69 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
70 / 205
Online Learning in Discrete Hidden Markov Models Roberto C. Alamino1 , Nestor Caticha2 (1) Aston University, Birmingham, UK (2) University of Sao Paulo, Sao Paulo, Brazil Abstract We present and analyze three different online algorithms for learning in discrete Hidden Markov Models (HMMs) and compare their performance with the BaldiChauvin Algorithm. Using the Kullback-Leibler divergence as a measure of the generalization error we draw learning curves in simplified situations and compare the results. The performance for learning drift concepts of one of the presented algorithms is analyzed and compared with the Baldi-Chauvin algorithm in the same situations. A brief discussion about learning and symmetry breaking based on our results is also presented. Key Words: HMM, Online Algorithm, Generalization Error, Bayesian Algorithm.
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
71 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
72 / 205
A BAYESIAN APPROACH TO CALCULATING FREE ENERGIES OF CHEMICAL AND BIOLOGICAL SYSTEMS Andrew Pohorille NASA-Ames Research Center, USA (e-mail:
[email protected], fax: 650-604-1088) Abstract A common objective of molecular simulations in chemistry and biology is to calculate the free energy difference, ∆A, between states of a system of interest. Important examples are protein-drug interactions, protein folding and ionization states of chemical groups. However, accurate determination of ∆A from simulations is not simple. This can be seen by representing ∆A in terms of a one-dimensional integral of exp(−∆E/kB T ) × P (∆E) over ∆E. In this expression, ∆E is the energy difference between two states of the system, P (∆E) is the probability distribution of ∆E, kB is the Boltzmann constant and T is temperature. For finite systems, P (∆E) is a distorted Gaussian. Note that the exponential factor weights heavily the low ∆E tail of P (∆E), which is usually known with low statistical precision. One way to improve estimates of ∆A is to model P (∆E). Generally, this approach is rarely successful. Here, however, we take advantage of the “Gaussianlike” shape of P (∆E). As is known in physics, such a function can be conveniently represented by the square of a “wave function” which is a linear combination of Gram-Charlier polynomials. The number of terms, N, in this expansion supported by the data must be determined separately. This is done by calculating the posterior probability, P (N/∆E), where ∆E stands for all sampled values of ∆E. In brief, the dependence of the likelihood function on the coefficients of the expansion, CN is marginalized by determining their optimal values using Lagrange multipliers, and then expanding P (∆E)/CN , N) around the optimal solution. Special care needs to be taken to ensure convergence of this expansion. As expected, the maximum likelihood solution consists of two terms. One is related to the optimal values of CN and always increases with N . The second term is an “Ockham’s Razor” penalty. It involves a multivariate Gaussian integral on the N-dimensional hypersphere, which arises due to mormalization. This integral cannot be calculated analytically, but accurate approximations, which properly account for problem symmetries, can be obtained. The method offers the largest improvements over conventional approaches when P (∆E) is broad and sample size is relatively small. This makes is particularly suitable for computer aided drug design, in which the goal is to screen rapidly a large number of potential drugs for binding with the protein target.
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
73 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
74 / 205
EXPLORING THE CONNECTION BETWEEN SAMPLING PROBLEMS IN BAYESIAN INFERENCE AND STATISTICAL MECHANICS Andrew Pohorille NASA-Ames Research Center, USA (e-mail:
[email protected], fax: 650-604-1088) Abstract The Bayesian and statistical mechanical communities often share the same objective in their work – estimating and integrating probability distribution functions (pdfs) describing stochastic systems, models or processes. Frequently, these pdfs are complex functions of random variables exhibiting multiple, well separated local minima. Conventional strategies for sampling such pdfs are inefficient, sometimes leading to an apparent non-ergodic behavior. Several recently developed techniques for hadling this problem have been successfully applied in statistical mechanics. In the multicanonical and Wang-Landau Monte Carlo (MC) methods, the correct pdfs are recovered from uniform sampling of the parameter space by iteratively establishing proper weighting factors connecting these distributions. Trivial generalizations allow for sampling from any chosen pdf. The closely related transition matrix method relies on estimating transition probabilities between different states. All these methods proved to generate estimates of pdfs with high statistical accuracy. In another MC technique, parallel tempering, several random walks, each corresponding to a different value of a parameter (e.g. “temperature”), are generated and occasionally exchanged using the Metropolis criterion. This method can be considered as a statistically correct version of simulated anneling. An alternative approach is to represent the set of independent variables as a Hamiltonian system. Considerable progress has been made in understanding how to ensure that the system obeys the equipartition theorem or, equivalently, that coupling between the variables is correctly described. Then a host of techniques developed for dynamical systems can be used. Among them, probably the most powerful is the Adaptive Biasing Force method, in which thermodynamic integration and biased sampling are combined to yield very efficient estimates of pdfs. The third class of methods deals with transitions between states described by rate constants. These problems are isomorphic with chemical kinetics problems. Recently, several efficient techniques for this purpose have been developed based on the approach originally proposed by Gillespie. Although the utility of the techniques mentioned above for Bayesian problems has not been determined, further research along these lines is warranted.
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
75 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
76 / 205
Inverse Problem For Estimating Heat Source A. Neisy Department of Mathematics and Statistics , Faculty of Economics, Allameh .Tabatabaie University. Dr. Beheshti Ave., Tehran, Iran
Abstract: This paper considers, a two-dimensional inverse heat conduction problem. The direct problem will be solved by an application of the heat fundamental solution, and the heat source to be estimated by using least-square .method
Key Words: two-dimensional problem, direct and inverse heat conduction .problem, overposed data
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
77 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
78 / 205
!" !#$ %& ('*) +-,./!& & 021&35476 89#:@? :@D A4 s GNijt > 8-A; ? : 4 ;Ov;-A > BwGj;L:x`b4 D AHG D 6 476 ;A >@D ;C478 ? 8C4v9 6 `yAz4RE{AHf >.s Gjijt > 8-A; ? :R` > ;z9 > ;`b8CGNt G D&I AHf > G D 9 G]MGN9 6#:y}~? 47;GNACGj4 D 4 ?|> 8H:RAC478/ Gj;/G D AC8C4v9 6 ` > 92f&47; > >y?|> `@AH:RAHGN4 D MR: 9 > ;`b8Gjt > ;uACf > ij4v`h:RACGj4 D 4 G]AHf G D AHf > `gf#: 8H:RAC478*eGj; : 9 f 47; >_>?|> `@Ag:
AHGj4 D MR: 9 > ;`b8CGNt > ;xACf > `gf#: D I7> 4 Bx49 >Q? 47;GNAHGN4 D GNACf AHGNB > 1S 4 4 ?> 8C:RAH4 8OY9 GN > 8 >bD ArEU8C47BAHf >? 4bD AH6&B 4 ?> 8C:RAH4@D AHGk:Rij;Q: 9 GN; ? ij: B >@D A 4 ?> 8C:RAH478;(f&Gj`gf ? 8C4 ? : I :RA > :$dv6 : D AH6 B;AH:RA > G D AHf > `b4 D ACG D 6 6 B4 O$:R8 > `b4 `@ACGNM > AH8H: D ;EU478B:
AHGj4 D ;: D 92:R8 > M > 8O9&GN > 8 >@D A EU8C47B9 Gj; ? ik:R` > B >bD AC; G D ACf >$? f#: ; ? : 4 `@478C8 > ij:RAHGN4 D ;2: D 9 >bD AH: D I i > B >bD At > AP >b>@D ACf > MR: 478CqGj; ? 8 > ; >@D A > 9G D :_de6#: D AC6 BcB > `gf : D Gj`b: Tt 6&AoG]AlGj;:Rij;C4: ? ? iNGj`b: AC4 8 > ik:RA > 9:R8 > :R;ijGN > ACGjB >y} EU8 > de6 >@D `@O: D :RiNOv;CGN;.G D ;CG I7D : ;C;G D I J : ? ? ijG > 9f : 8C:y AGNAo;-AH6 9 G > ; ? f#:R; >y} ; ? :R` > B > AHf 4v9 ;EU478 ? 847t i > Bx;o9 > ;`b8CGNt > 9 GNACfV:`@4 D AHG D 6&476 ;A >bD ;C478 ? 849 6&`@AL4ls GNijt > 8-Ar; ? :R` > ;@1
X
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
79 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
80 / 205
Title: Data, Virtual Data, and Anti-Data. Author: Carlos C. Rodriguez Abstract \delta-priors optimize natural notions of ignorance. When the likelihood is in the exponential family the 0-priors become the standard cojugate priors relative to the information volume. In this case prior information is equivalent to having \alpha > 0 extra virtual observations. On the other hand 1-priors are not conjugate and where the 0-priors add the \alpha virtual observations to the actual n sample points, the 1-priors subtract the \alpha from the n. I call this "anti-data" since \alpha of these points annihilate \alpha of the observations leaving us with a total of n-\alpha. Thus, 1-priors are more ignorant than 0-priors. True ignorance, that claims only the model and the observed data, has a price. To build the prior we must spend some of the information cash in hand. No free lunches.
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
81 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
82 / 205
ENTROPY AND SEMI-MARKOV PROCESSES Valerie Girardin, LMNO Universit´e de Caen, BP5186, 14032 Caen, France (e-mail:
[email protected]) Abstract Entropy and Markov processes are linked since the first version of the asymptotic equirepartition property (AEP) stated by Shannon in 1948 for Markov chains. We define explicitely the entropy rate for semi-Markov processes and extend the AEP or ergodic theorem of information theory to these nonstationary processes. Among a given collection of functions satisfying constraints, selecting the one with the maximum entropy is equivalent to adding the less of information possible to the considered problem. The definition of an explicit entropy rate for processes allows one to extend the maximum entropy method to this case. We study different problems for Markov and semi-Markov processes, illustrated in reliability, queueing theory, sismology... References: [1] V. Girardin (2004) Entropy Maximization for Markov and SemiMarkov Processes Methodology and Computing in Applied Probability, V6, 109– 127. [3] V. Girardin (2005) On the Different Extensions of the Ergodic Theorem of Information Theory Recent Advances in Applied Probability, R. Baeza-Yates, J. Glaz, H. Gzyl, J. H¨ usler & J. L. Palacios (Eds), Springer-Verlag, San Francisco, pp163–179. [3] V. Girardin & N. Limnios (2006). Entropy for Semi-Markov Processes with Borel State Spaces: Asymptotic Equirepartition Properties and Invariance Principles Bernoulli, to appear. Key Words: asymptotic equirepartition property, entropy rate, Markov chains, Markov processes, maximum of entropy, semi-Markov processes, Shannon-McMillanBreiman theorems.
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
83 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
84 / 205
ENTROPY COMPUTATION IN PARTIALLY OBSERVED MARKOV CHAINS Fran¸cois Desbouvries Institut National des T´el´ecommunications, Evry, France e-mail:
[email protected] Abstract Hidden Markov Chains (HMC) [1] are widely used in speech recognition, image processing or protein sequence analysis, due to early availability of efficient Bayesian restoration (Forward-Backward, Viterbi) or parameter estimation (Baum Welch) algorithms. More recently, the problem of computing in an HMC the entropy of the possible hidden state sequences that may have produced a given sequence of observations has been addressed, and an efficient (i.e., linear in the number of observations) algorithm has been proposed [2]. Among possible extensions of HMC, Pairwise (PMC) [3] and Triplet [4] Markov Chains (TMC) have been introduced recently. In a TMC we assume that t = (x, r, y), where x is the hidden process, y the observation and r a latent process, is a Markov chain (MC). So a TMC can be seen as a vector MC, in which one observes some component y and one wants to restore some part of the remaining components. In a TMC the marginal process (x, r) is not necessarily an MC, but the conditional law of (x, r) given the observations y is an MC; as in HMC, this key computational property enables the development of efficient restoration or parameter estimation algorithms. In this paper, we extend to TMC the entropy computation algorithm of [2]. The resulting algorithm remains linear in the number of observations. References: [1] Y. Ephraim and N. Merhav, ”Hidden Markov processes”, IEEE Transactions on Information Theory, vol. 48-6, pp. 1518-69, June 2002. [2] D. Hernando, V. Crespi and G. Cybenko, ”Efficient Computation of the Hidden Markov Model Entropy for a Given Observation Sequence”, IEEE tr. Info. Th., pp. 2681-85, July 2005 [3] W. Pieczynski, ”Pairwise Markov Chains”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 25-5, pp. 634-39, May 2003 [4] W. Pieczynski and F. Desbouvries, ”On Triplet Markov chains”, Proceedings of the International Symposium on Applied Stochastic Models and Data Analysis (ASMDA 2005), Brest, France, May 17-20, 2005 Key Words: Entropy, Hidden Markov Chains, Markovian models
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
85 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
86 / 205
BAYESIAN SMOOTHING ALGORITHMS IN PARTIALLY OBSERVED MARKOV CHAINS Boujemaa Ait-el-Fquih and Fran¸cois Desbouvries Institut National des T´el´ecommunications, Evry, France e-mail:
[email protected] Abstract An important problem in signal processing consists in estimating an unobservable process x from an observed process y. In Hidden Markov Chains (HMC), efficient Bayesian smoothing restoration algorithms have been proposed in the discrete [1] as well as in the Gaussian case [2] [3]. Among other extensions of HMC, Triplet Markov Chains (TMC) have been introduced recently (see e.g. [4]). In a TMC we assume that the triplet (x, r, y) (in which r is some additional process) is a Markov Chain (MC). So a TMC can be seen as a vector MC, in which one observes some components y and one wants to restore some part of the remaining components. In a TMC the marginal process (x, r) is not necessarily an MC, but the conditional law of (x, r) given the observations y is an MC; as in HMC, this key computational property enables the development of efficient restoration or parameter estimation algorithms. This paper addresses fixed-interval smoothing algorithms in TMC and is a continuation of the work of [5]. In particular, we extend to Gaussian TMC the Bryson and Frazier algorithm, the backward-forward RTS algorithm, the Fraser and Potter algorithm and the backward-forward RTS algorithm of Desai et al. References: [1] L. R. Rabiner, ”A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition, Proceedings of the IEEE, February 1989. [2] T. Kailath, A. H. Sayed and B. Hassibi, Linear estimation, Prentice Hall, Upper Saddle River, New Jersey 2000 [3] H. L. Weinert, Fixed interval smoothing for state-space models, Kluwer Academic Publishers, Boston 2001. [4] W. Pieczynski and F. Desbouvries, On Triplet Markov chains, Proceedings of the International Symposium on Applied Stochastic Models and Data Analysis (ASMDA 2005), Brest, France, May 17-20, 2005 [5] B. Ait-el-Fquih and F. Desbouvries, ”Bayesian smoothing algorithms in Pairwise and Triplet Markov chains”, Proceedings of the 2005 IEEE Workshop on Statistical Signal Processing, Bordeaux, France, July 2005. Key Words: Hidden Markov Chains, state-space models, Markovian models, smoothing algorithms.
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
87 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
88 / 205
Title: Applications of Maximum Entropy in Gravitational Wave Astronomy Author: Lee Samuel Finn Institution: Center for Gravitational Wave Physics, The Pennsylvania State University Abstract: Large gravitational wave detectors in the United States (LIGO) and Europe (GEO and Virgo) have just now reached a sensitivity that makes them sensitive to gravitational wave emission from astronomical phenomena. In the next decade, ESA and NASA plan to place an even more sensitive detector (LISA) into space. Sources that these detectors have the capability of observing include the formation of the black holes that are thought to power gamma-ray bursts, the stellar core collapses that power type II supernovae, the coalescence of supermassive black holes that follows the collision of their host galaxies, and the myriad of compact binary white dwarf binary systems that populate our galaxy. The first detection of gravitational waves by these detectors will usher in the era of gravitational wave astronomy: the use of gravitational waves as a tool of astronomical discovery. Gravitational wave detectors are not imaging instruments and individual gravitational wave detectors lack the ability to localize a source on the sky. From a network of detectors we can synthesize a beam and thus determine the position of a source and the radiation amplitude and phase in each gravitational wave polarization. Alternatively, the signal acquired from a detector that is moving with respect to a source will be phase and amplitude modulated in a manner that depends on the source's sky location, the signal in each polarization, and the detector's changing position and orientation with respect to the source. From this information and models of the radiation expected from different sources we can test general relativity and learn about the sources we are observing. Here we describe the development and use of maximum entropy based tools to recover the gravitational wave signal amplitude and phase in each polarization, and the sky location of one or more sources whose radiation is incident on an array of detectors or a moving detector. These tools are being used now for the analysis of data from the United States LIGO detector and will likely play an important role in the analysis of data from the LISA detector.
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
89 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
90 / 205
Gibbs Paradox and the Higher Similarity - Higher Entropy Relationship Shu-Kun Lin Molecular Diversity Preservation International (MDPI), Matthaeusstrasse 11, CH-4057 Basel, Switzerland (e-mail:
[email protected], http://www.mdpi.org/lin) Abstract There are three kinds of correlation of the entropy of mixing with similarity. The Gibbs paradox statement, which has been regarded as a very fundamental assumption in statistical mechanics, says that the entropy of mixing or assembling to form solid assemblages, liquid and gas mixtures or any other analogous assemblages such as quantum states, decreases discontinuously with the increase in the property similarity of the composing individuals. Most authors accept this relastionship (e.g. [1]). Some authors revised the Gibbs paradox statement and argued that the entropy of mixing decreases continuously with the increase in the property similarity of the individual components [2]. A higher similarity - higher entropy relationship and a new theory has been constructed: entropy of mixing or assembling increases continuously with the increase in the similarity. Similarity Z can be easily understood when two items A and B are compared: if A and B are distinguishable (minimal similarity), Z=0. If they are indistinguishable (maximal similarity), Z=1. References: [1] E. T. Jaynes, The Gibbs Paradox, In Maximum Entropy and Bayesian Methods; C. R. Smith, G. J. Erickson, P. O. Neudorfer, Eds.; Kluwer Academic: Dordrecht, 1992, p.1-22. [2] J. von. Neumann, Mathematical Foundations of Quantum Mechanics. Princeton, NJ: Princeton University Press, 1955. Key Words: Gibbs Paradox, Entropy, configurational entropy
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
91 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
92 / 205
AN APPLICATION OF ENTROPIC DYNAMICS ON CURVED STATISTICAL MANIFOLDS Carlo Cafaro, S. A. Ali, A. Gi¢ n Department of Physics, University at Albany-SUNY, 1400 Washington Avenue, Albany, NY 12222, USA (e-mail:
[email protected])
Abstract Any attempt to unify the classical theory of gravity with quantum theories of electromagnetic, weak and strong forces in a single uni…ed theory has been unsuccessful so far. Entropic Dynamics (ED), namely the combination of principles of inductive inference (Maximum Entropy Methods) and methods of Information Geometry (IG), is a theoretical framework constructed to explore the possibility that laws of physics, either classical or quantum, might be laws of inference rather than laws of nature. The ultimate goal of such an ED concerns the derivation of Einstein’s theory of gravity from an underlying “statistical geometrodynamics” [1]. Our objective here is to show explicitly all the steps needed to derive an ED model and to underline the most delicate aspects of it. The …rst step is to identify the appropriate variables describing the system, and thus the corresponding space of macrostates. This is by far the most di¢ cult step because there does not exist any systematic way to search for the right macro variables; it is a matter of taste and intuition, trial and error. In the ED model here presented we do not specify the nature of our system, it might be a thermal system or something else. We will make connections to conventional physical systems only later in the formulation of the ED model. We only assume that the space of microstates is 2D and that all the relevant information to study the dynamical evolution of such a system is contained in a 3D space of macrostates. The second step is to de…ne a quantitative measure of change from one macrostate to another. Maximum Entropy Methods lead to the assignment of a probability distribution to each macrostate, while methods of IG lead to the assignment of the Fisher-Rao information metric quantifying the extent to which one distribution can be distinguished from another. The ED is de…ned on the space of probability distributions Ms . The geometric structure of Ms is studied in detail. We show that Ms is a 3D pseudosphere with constant negative Ricci scalar curvature, R = 1. The …nal step concerns the study of irreversible and reversible aspects of such ED on Ms . In the former case, we study the evolution of the system from a given macrostate to an unknown …nal macrostate. This study is used to show that the microstates of the model undergo an irreversible di¤usion process. In the latter, we study the evolution of the system from a given initial macrostate to a given …nal state. The trajectories of the system are shown to be hyperbolic curves on Ms , and the surface of evolution of the statistical parameters describing Ms is plotted. Finally, similarities and possible connections between ED methods and established physics are highlighted. References: [1] A. Caticha: “The Information Geometry of Space and Time”, Presented at MaxEnt2005, the 25th International Workshop on Bayesian Inference and Maximum Entropy Methods (August 7-12, 2005, San Jose, California, USA).
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
93 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
94 / 205
Reconstruction of the Electron Energy Distribution Function from Optical Emission Spectroscopic Measurements Dirk Dodt1 , Andreas Dinklage1, Rainer Fischer2 (1) Institut f¨ ur Plasma Physik, Teilinstitut Greifswald (2) Institut f¨ ur Plasma Physik, Teilinstitut Garching (e-mail:
[email protected]) Abstract The properties of a low temperature plasma (as for example used in energy saving light bulbs) are mainly determined by the energy distribution of the free electrons. This distribution is described by the so-called electron energy distribution function (EEDF). A well established method to obtain the EEDF is to measure the currentvoltage characteristics of a plasma using a small wire in contact with the plasma (probe). The approach presented here is motivated by the idea to utilise the light emitted by excited gas atoms, in order to get rid of the perturbing probe brought into the plasma. The inference of the EEDF from the measured intensities is an example of an ill-posed inversion problem, because of the high sensitivity of the reconstruction on small errors of the line intensities. The forward calculation consists of a so-called stationary collisional-radiative model which is describing the interaction of atoms and ions with the free electrons and the discharge device. The systematic uncertainties in the model parameters, namely the different atomic data that enter the calculation, have to be considered with particular care. First results are shown for the spectrum of a neon discharge lamp. The radially averaged EEDF is reconstructed. The applicability of different functional forms of the EEDF is assessed. In a first step Maxwell and Druyvenstein distributions which are having only a small number of parameters are considered. Key Words: Plasma Physics, Applied Bayesian Data Analysis
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
95 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
96 / 205
OPTIMIZATION OF PLASMA DIAGNOSTICS USING BAYESIAN PROBABILITY THEORY H. Dreier1 , A. Dinklage1 , R. Fischer2 , M. Hirsch1 , P. Kornejew Max–Planck–Institut f¨ ur Plasmaphysik, EURATOM Association, 1 Teilinstitut Greifswald, D–17491 Greifswald, Germany 2 D–85748 Garching, Germany (e-mail:
[email protected]) Abstract The Wendelstein 7-X stellarator will be a magnetic fusion device and is presently under construction. Its diagnostic set-up is currently in the design process to optimize the outcome under given technical constraints. In general, the preparation of diagnostics for magnetic fusion devices requires a physical model of the measurement which relates the physical effect to the measured data (forward function), and a diagnostics model which describes the error statistics. The approach presented here bases on maximization of an information measure (Kullback-Leibler entropy, see ref. [1]). Bayesian probability theory allows one to link measures from information theory with the model for the fusion diagnostics. The approach can be considered as the implementation of a virtual diagnostic which generates data from a range of parameters. The virtual diagnostic employs the forward function and accounts for the error statistics. Then, optimization means maximization of the expected utility with respect to the design parameters. It allows for extensive design studies of effects due to physical input and possible benefits due to technical elements. Comparisons with other information measures and approximation methods for the prior predictive value are discussed. The reconstruction of density profiles by means of a multichannel infrared interferometer at W7-X is investigated in detail. The influence of different error statistics and the robustness of the result are discussed. In addition, the impact of technical boundary conditions is shown. References: [1] R.Fischer, AIP Conf. Proc. 735 (2004) 76-83 [2] H.Dreier et al., Fusion Science and Technologies, (accepted 2006) Key Words: diagnostic design, optimization, information measure
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
97 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
98 / 205
On the theory of phase transition Landau Gadjiev B.R. International University of Nature, Society and Man, 141980, 19 Universitetskaya str., Dubna, Moscow Region, Russia
[email protected]
We consider close-packed structure with defects, which undergoes structural phase transition, and we research dependence of critical exponents on concrete kind of defects distribution. The analysis shows, that the connectivity distribution function of defects in structure can be presented by the generalized Boltzmann factor, as it is done in superstatistics. Namely, the distribution of defects in close-packed structure can be considered as the process of homogeneous growth. Using a maximum entropy principle it is possible to show, that in this case we have an exponential distribution of defects connectivity. If the spatial distribution of defects in structure is random, then, generally, the number of entering links will be random variable. For example, if this distribution is gamma distribution we obtain the analogue of Tsallis distributions. Other spatial distributions of defects generate infinite number different distributions. For a statistical mechanical foundation we use a maximum entropy principle, which allows to obtain the generalized Boltzmann factor that allows to obtain the concrete distribution links of defects. Namely from a entropy functional on which the constraints we are imposed and we define the distribution function of defects in the system. The maximum entropy principle, which was used in no extensive statistical mechanics, allows derive the generalized Boltzmann factor. It allows to obtain various distribution functions connectivity of defects in structure by a unified way. For the analysis of defects distribution dependence critical exponents we introduce a free energy functional, which depends on an order parameter and on connectivity distribution of defects of structure. The symmetry of an order parameter is defined by an irreducible representation of a space group of structure. After the procedure an average of a free energy functional on connectivity of defects (practically it implies calculation a moments of the distribution) we obtain, that the critical behaviour strongly depends on the form of the distribution function and can essentially differ from of the mean-field behaviour. The analysis of experimental results shows, that such situation is characteristic for doped layered crystals (system with competing interaction).
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
99 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
100 / 205
MARGINALIZED MAXIMUM A POSTERIORI HYPER-PARAMETER ESTIMATION FOR GLOBAL OPTICAL FLOW TECHNIQUES Kai Krajsek, Rudolf Mester Institute for Computer Science,D-60054 Frankfurt, Germany (e-mail:
[email protected] http://www.vsi.cs.uni-frankfurt.de) Abstract Global optical flow estimation methods contain a regularization parameter (or prior and likelihood hyper-parameters if we consider the statistical point of view) which control the tradeoff between the different constraints on the optical flow field. Although experiments (see e.g. Ng et al. [2]) indicate the importance of the optimal choice of the hyper-parameters, only little attention has been focused on the optimal choice of these parameters in global motion estimation techniques in literature so far (the authors are only aware of one contribution [2] which attempts to estimate only the prior hyper-parameter whereas the likelihood hyper-parameter needs to be known). We adapted the marginalized maximum a posteriori (MMAP) estimator developed in [1] to simultaneously estimating hyper-parameters and optical flow for global motion estimation techniques. The optimal hyper-parameters are strongly determined by first order statistics in the image scene, i.e. the illumination distribution. Optimal values for the hyper-parameter of former image scenes could therefore be used to feed in the Bayesian hyper-parameter estimation framework. Furthermore, the resulting objective function is not convex with respect to the hyper-parameters, thus an appropriate starting point for the estimated parameters is essential for obtaining a reasonable estimate and not to stick into an unimportant local minimum. Experiments demonstrate the performance of this optimization technique and show that the choice of the regularization parameter/hyper-parameters is an essential key-point in order to obtain precise motion estimates. References: [1] A. Mohammad-Djafari, A full Bayesian approach for inverse problems, Presented at the 15th International Workshop on Maximum Entropy and Bayesian Methods (MaxEnt95), Santa Fe, New Mexico, USA (1995) [2] Lydia Ng and Victor Solo, A Data-driven Method for Choosing Smoothing Parameters in Optical Flow Problems, Proc. International Conference on Image Processing, Washington, DC, USA (1997), pp. 360-363 Key Words: optical flow, marginalized MAP estimation
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
101 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
102 / 205
UPDATING PROBABILITIES Ariel Caticha and Adom Giffin Department of Physics, University at Albany - SUNY, Albany, NY, USA (e-mail:
[email protected]) Abstract The Method of Maximum (relative) Entropy (ME) has been designed for updating from a prior distribution to a posterior distribution when the information being processed is in the form of a constraint on the family of allowed posteriors. This is in contrast with the usual MaxEnt which was designed as a method to assign, and not to update, probabilities. The objective of this paper is to strengthen the ME method in two ways. In [1] the axioms that define ME have been distilled down to three; here the design is improved by considerably weakening the axiom that refers to independent subsystems. Instead of the old axiom which read: “When a system is composed of subsystems that are believed to be independent it should not matter whether the inference procedure treats them separately or jointly” we now modify it by replacing the word ‘believed’ by the word ‘known’. As pointed out by Karbelkar and by Uffink the modified axiom is a much weaker consistency requirement, which, in their view, fails to single out the usual (logarithmic) relative entropy as the unique tool for updating. It merely restricts the form of the entropy to a one-dimensional continuum labeled by a parameter η; the resulting η-entropies are equivalent to the Renyi or the Tsallis entropies. We show that further applications of the same modified axiom select a unique, universal value for the parameter η and this value corresponds to the usual (logarithmic) relative entropy. The advantage of our new approach is that it shows precisely how it is that the other η-entropies are ruled out as tools for updating. Our second concern is mostly pedagogical. It concerns the relation between the ME method and Bayes’ rule. We start by drawing the distinction between Bayes’ theorem, which is a straightforward consequence of the product rule for probabilities, and Bayes’ rule, which is the actual updating rule. We show that Bayes’ rule can be derived as a special case of of the ME method. The virtue of our derivation, which hinges on translating the information in data into constraints that can be processed by ME, is that it is particularly clear. It throws light on Bayes’ rule and it shows the complete compatibility of Bayes’ updating with ME updating. References: [1] A. Caticha, “Relative Entropy and Inductive Inference,” in Bayesian Inference and Maximum Entropy Methods in Science and Engineering, ed. by G. Erickson and Y. Zhai, AIP Conf. Proc. 707, 75 (2004) (arXiv.org/abs/physics/0311093). Key Words: relative entropy, Bayes rule
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
103 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
104 / 205
FROM OBJECTIVE AMPLITUDES TO BAYESIAN PROBABILITIES Ariel Caticha Department of Physics, University at Albany - SUNY, Albany, NY, USA (e-mail:
[email protected]) Abstract Many discussions on the foundations of quantum theory start from the abstract mathematical formalism of Hilbert spaces and some ad hoc ”postulates” or rules prescribing how the formalism should be used. Their goal is to discover a suitable interpretation. The Consistent-Amplitude approach to Quantum Theory (CAQT) is different in that it proceeds in the opposite direction: one starts with the interpretation and then derives the mathematical formalism from a set of ”reasonable” assumptions. The overall objective is to predict the outcomes of certain idealized experiments on the basis of information about how complicated experimental setups are put together from simpler ones. The theory is, by design, a theory of inference from available information. The ”reasonable” assumptions are four. The first specifies the kind of setups about which we want to make predictions. The second assumption establishes what is the relevant information and how it is codified. It is at this stage that amplitudes and wave functions are introduced as tools for the consistent manipulation of information. The third and fourth assumptions provide the link between the formalism and the actual prediction of experimental outcomes. Although the assumptions do not refer to probabilities, all the elements of quantum theory, including indeterminism and the Born rule, Hilbert spaces, linear and unitary time evolution, are derived. Within the CAQT approach probabilities are completely Bayesian, and yet, there is nothing subjective about the wave function that conveys the relevant information about the (idealized) experimental setup. The situation is quite analogous to assigning Bayesian probabilities to outcomes of a die toss based on the objective information that the (idealized) die is a perfectly symmetric cube. Key Words: mechanics
quantum theory, quantum information theory, Bayesian quantum
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
105 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
106 / 205
BLIND SOURCE SEPARATION USING MAXIMUM ENTROPY PDF ESTIMATION BASED ON FRACTIONAL MOMENTS Babak Abbasi Bastami, Hamidreza Amindavar Amirkabir University of Technology, Department of Electrical Engineering, Tehran, Iran (e-mail: babak
[email protected],
[email protected]) Abstract Recovering a set of independent sources which are linearly mixed is the main task of the blind source separation. Utilizing different methods such as infomax principle, mutual information and maximum likelihood leads to simple iterative procedures such as natural gradient algorithms[1]. These algorithms depend on a nonlinear function (known as score or activation function) of source distributions. Since there is no prior knowledge of source distributions, the optimality of the algorithms is based on the choice of a suitable parametric density model. In this paper, we propose an adaptive optimal score function based on the fractional moments of the sources. In order to obtain a parametric model for the source distributions, we use a few sampled fractional moments to construct the maximum entropy probability density function (PDF) estimation [2]. By applying an optimization method we can obtain the optimal fractional moments that best fit the source distributions. Using the fractional moments instead of the integer moments causes the maximum entropy estimated PDF to converge to the true PDF much faster . The simulation results show that unlike the most previous proposed models [3] for the nonlinear score function, which are limited to some sorts of source families such as sub-gaussian and super-gaussian or some forms of source distribution models such as generalized gaussian distribution, our new model achieves better results for every source signal without any prior assumption for its randomness behavior. References: [1] J. F. Cardoso,“Blind signal separation : Statistical Principles,” Proc. IEEE, vol 9, no 10, October 1998. [2] H. Gzyl, P. N. Inveradi, A. Tagliani, M. Villasana, “Maxentropic solution of fractional moment problems,” Applied Mathematics and Computation, 2005. [3] A. Cichocki, S. Amari, “Adaptive blind signal and image processing ,” John Wiley and Sons, 2003. Key Words: Fractional Moments, Blind Source Separation, Score Function
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
107 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
108 / 205
COMPETITIVE BIDDING IN A CERTAIN CLASS OF AUCTIONS Mathias Johansson1,2 (1) Signals and Systems Group, Uppsala University, Sweden, e-mail:
[email protected] (2) Dirac Research AB, Sweden, e-mail:
[email protected] Abstract We consider a problem of determining the amount to bid in a certain type of auctions in which customers submit one sealed bid. The bid reflects the price a customer is willing to pay for one unit of the offered goods. The auction is repeated and at each auction each customer requests a certain amount of goods, an amount that we call the capacity of the customer and that varies among customers and over time. At each auction, only the customer with the largest bid-capacity product obtains any goods. The price paid by the winner equals his/her bid-capacity product, and the amount of goods obtained in return equals the winner’s capacity. The auction is repeated many times, with only limited information concerning winning bidcapacity products being announced to the customers. This situation is motivated in for example wireless communication networks in which a possible way of obtaining a desired service level is to use dynamic pricing and competitive bidding. In this application, the capacity is typically uncertain when the bid is made. We derive bidding rules and loss functions for a few typical service requirements. We assume that the auctioneer announces only some limited aggregate statistics from previous auctions. Consequently, we use the maximum entropy principle in assigning probabilities for other customers’ bids and capacities. Our approach is to minimize the expected loss, conditional on the limited information I available to the customer. Let a particular customer u’s probability that he or she will have the largest bid-capacity product of all customers be denoted by P (u | I). Then P (u | I) is equal to the probability that the customer v with the largest bid-capacity product of all other customers has a lower bid-capacity product than customer u. Let qv denote the bid of v, cv the corresponding capacity, and y = qv cv the largest bid-capacity product among all customers except u. We can then find the probability that u wins as follows: R cu qu first determine the probability that P (y | cu I)dy. Then multiply this y < cu qu assuming knowledge of cu , i.e. 0 with the probability distribution for cu given I to obtain the joint probability for cu and y < cu qu . Integrating the result over all possible capacities cu , we have Z Z c u qu P (u | I) = P (cu | I) P (y | cu I)dydcu . (1) 0
In the full paper, we compute this probability explicitly for some particular states of knowledge I and illustrate how customers behave using the suggested strategy.
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
109 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
110 / 205
PARAMETER ESTIMATION OF ELLIPSOMETRY MEASUREMENTS Udo v. Toussaint1 , Thomas Schwarz-Selinger1 (1) Institute for Plasma Physics, 85748 Garching, Germany (e-mail:
[email protected]) Abstract Ellipsometry is a unique technique of great sensitivity for in situ non-destructive characterization of surfaces utilizing the change in the state of polarization of a light-wave probe which is extensively used in the semi-conductor industry. To relate ellipsometric measurements to surface properties (as eg layer thickness changes in the range of nm or chemical composition), Bayesian probability theory is used. The parameter estimation process is complicated by the incomplete phase information of the measured data. Examples of 3-D surface reconstructions of samples after plasma exposure demonstrate the tremendous information gain due to the Bayesian analysis. References: Key Words: Parameter Estimation, Bayesian Probability Theory
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
111 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
112 / 205
Applied Probability Trust (19 May 2005)
ENTROPY FOR PARETO (IV), BURR, AND ITS ORDER //STATISTICS DISTRIBUTIONS GHOLAMHOSSEIN YARI,∗ Iran University of Science and Technology
GHOLAMREZA MOHTASHAMI,∗∗ Birjand University
Abstract Main result of this paper is to derive the exact analytical expressions of entropy for Pareto, Burr and related distributions. Entropy for kth order statistic corresponding to the random sample size n from these distributions is introduced. These distributions arise as tractable parametric models in reliability, actuarial science, economics, finance and telecommunications. We showed that all the calculations can be obtained from one main dimensional integral whose expression is obtained through some particular change of variables. Indeed, we consider that this calculus technique for that improper integral has its own importance. Keywords: Gamma and Beta functions; Polygamma functions; Entropy; Order Statistics; Pareto, Burr models.
∗
Postal address: Iran University of Science and Technology, Narmak, Tehran 16844, Iran.
email: ∗∗
[email protected]
Postal address: Birjand university email:
[email protected]
1
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
113 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
114 / 205
CLEARING UP THE MYSTERIES: COMPUTING ON HYPOTHESIS SPACES Kevin H. Knuth Department of Physics University at Albany (SUNY) Albany NY, 12222, USA (e-mail:
[email protected], http://www.huginn.com/kknuth) Abstract We all have become very comfortable with Bayesian probability theory and the interpretation of probabilities as real numbers representing degrees of belief. Indeed this level of comfort was necessary for these methods to become widely accepted. In keeping with Jaynes’ original goal of ‘Clearing up the Mysteries’, I aim to inject some healthy discomfort back into this meeting by closely examining hypothesis spaces and the computations we perform on them. For example, these spaces are not necessarily Boolean spaces. There are two different types of logical and operations, one which occurs within a lattice and the other which is induced by the lattice product. Clearly, these details do not upset the Bayesian inferential framework with which we have become so comfortable. Instead, they serve to highlight the fact that even today there remains uncharted territory in the foundation of probability theory. Key Words: Hypothesis Space, Bayes’ Theorem, Boolean Algebra, Lattice Theory, Associativity, Distributivity
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
115 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
116 / 205
AND IF YOU WERE A BAYESIAN WITHOUT KNOWING IT? Bruno Lecoutre C.N.R.S. et Universit´e de Rouen, France (e-mail:
[email protected] http://www.univ-rouen.fr/LMRS/Persopage/Lecoutre/Eris) Abstract Many statistical users misinterpret the p-values of significance tests as “inverse” probabilities (1 − p is “the probability that the alternative hypothesis is true”). As is the case with significance tests, the frequentist interpretation of a 95% confidence interval involves a long run repetition of the same experiment: in the long run 95% of computed confidence intervals will contain the “true value” of the parameter; each interval in isolation has either a 0 or 100% probability of containing it. Unfortunately treating the data as random even after observation is so strange this “correct” interpretation does not make sense for most users. Ironically it is the interpretation in (Bayesian) terms of “a fixed interval having a 95% chance of including the true value of interest” which is the appealing feature of confidence intervals. Moreover, these “heretic” misinterpretations of confidence intervals (and of significance tests) are encouraged by most statistical instructors who tolerate and even use them. For instance Pagano (1990, page 288), in a book which claims the goal of “understanding statistics”, describes a 95% confidence interval as “an interval such that the probability is 0.95 that the interval contains the population value”. The literature is full of Bayesian interpretations of frequentist p-values and confidence levels. All the attempts to rectify these interpretations have been a loosing battle. In fact such interpretations suggest that most users are likely to be Bayesian “without knowing it” [2] and really want to make a different kind of inference [3]. References: [1] Pagano, R. R., Understanding Statistics in the Behavioral Sciences (1990, 3rd edition), West, St. Paul, MN. [2] Lecoutre B., Et si vous ´etiez un bay´esien “qui s’ignore”? La Revue de Modulad 32 (2005) [http://www-rocq.inria.fr/axis/modulad/archives/numero-32/lecoutre-32 /lecoutre-32.pdf]. [3] Lecoutre, B., Lecoutre, M.-P. & Poitevineau, J., Uses, abuses and misuses of significance tests in the scientific community: won’t the Bayesian choice be unavoidable? International Statistical Review 69 (2001), 399-418. Key Words: Frequentist probabilities, Bayesian probabilities
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
117 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
118 / 205
PROBABILITY ASSIGNMENT IN A QUANTUM STATISTICAL MODEL L. F. Lemmens 1 , Burhan Bakar1 (1) Department of Physics, University of Antwerp, Belgium (e-mail:
[email protected]) Abstract In ref [1] the evolution of a quantum system, appropriate to describe nano-magnets, is mapped on a Markov process when the system is cooled, the adjoint heating process is obtained using Bayes theorem. Once the mapping is achieved a Markov representation for the evolution with respect to inverse temperature of the quantum system is obtained. The representation can be used to study the probability density of the magnetization. The PDF changes from unimodal to bimodal as a function of the temperature. The change occurs at the so called blocking temperature and depends critically on the initial probability. This probability encodes the multiplicity of the states. The transition from paramagnetic to super-paramagnetic behavior is of importance to enhance the sensitivity of the nano-magnet. Using the information entropy [2] one can calculate the same PDF without invoking a Markov process. Although the characteristics of the PDF for both calculations are resembling, the numerical values are different: implying that probabilities obtained using the trace and the diagonal elements i.e. the method leading to the information entropy, are not necessarily equal to those derived from the Markov process. Considering both approaches as a model to assign probabilities, one can use the maximum entropy principle to perform a model selection. A straight forward calculation shows that the entropy obtained in the Markov representation is larger than the information entropy. References: [1] Burhan Bakar, and L F Lemmens Phys. Rev. E 71, 046109 (2005) see also cond-mat / 0502277 [2] Alexander Stotland, Andrei A. Pomeransky, Eitan Bachmat, Doron Cohen, Europhysics Letters 67, 700 (2004) Key Words: Blocking Temperature, information entropy, Markov representation
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
119 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
120 / 205
Maximum likelihood separation of spatially auto-correlated images using a Markov model Shahram Hosseini1 , Rima Guidara1 , Yannick Deville1 , Christian Jutten2 (1) Laboratoire d’Astrophysique de Toulouse-Tarbes (LATT), France (2) Laboratoire des Images et des Signaux (LIS), Grenoble, France Abstract We recently proposed an efficient maximum likelihood approach for blindly separating markovian time series [1]. In the present paper, we extend this idea to bi-dimensional sources (in particular images), where the spatial correlation of each source is described using a 2nd-order Markov model. The idea of using Markov Random Fields (MRF) for image separation has recently been exploited by other authors [2], where the source Probability Density Functions (PDF) are supposed to be known, and are used to choose the Gibbs priors. In the present work, however, we make no assumption about the source PDF so that the method can be applied to any sources. Beginning with the joint PDF of all the observations, and supposing a unilateral 2nd-order Markov model for the sources, we can write down the likelihood function and show that the nondiagonal entries of the separating matrix can be estimated by solving the following estimating equations E[
1 X
k=0
1 X
ψs(k,l) (m, n)ˆ sj (m − k, n − l)] = 0 i (k,l)
where the conditional score functions ψsi ψs(k,l) (m, n) = i
i 6= j
l=−1,k+l6=−1
of the estimated sources sˆi are
si (m, n)|ˆ si (m − 1, n − 1), sˆi (m − 1, n), sˆi (m − 1, n + 1), sˆi (m, n − 1)) −∂ log Psi (ˆ ∂si (m − k, n − l)
In practice, these functions must be estimated from data in a 5-dimensional space. The nonparametric estimation algorithm used in [1] being very time consuming, we developed a new algorithm using polynomials as score function parametric models. The estimating equations were solved using Newton algorithm. The experiments proved the better performance of our method in comparison to some classical algorithms. The final version of the paper will contain the theoretical details and the experimental results with artificial and real data including astrophysical images. References: [1] S. Hosseini, C. Jutten, D.-T. Pham, Markovian source separation, IEEE Transactions on Signal Processing, vol. 51, no. 12, pp. 3009-3019, Dec. 2003. [2] E. E. Kuruoglu, A. Tonazzini and L. Bianchi, Source separation in noisy astrophysical images modelled by markov random fields, ICIP’04, pp. 2701-2704. Key Words: Blind source separation, Markov random fields, Maximum likelihood
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
121 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
122 / 205
Comparing Class Scores in GCSE Modular Science JASON WELCH, County High School, Leftwich, Cheshire, UK
Abstract
Multiple choice tests are used widely in education and elsewhere. The results of these tests contain information both about the students’ knowledge and their ability to guess the answers. This paper describes the use of Bayesian statistical techniques to attempt to ‘remove’ the guess-work from the results in order to obtain information about the students’ underlying knowledge based on our prior knowledge about the structure of the test. The resulting mathematical model allows fair comparisons of the levels of knowledge of groups of students in schools and highlights the flaws in the common practice of analysing these scores using simple averages. It also allows more specific comparisons to be made that are not possible using averages. These comparisons can then inform teaching practice.
Introduction
A multiple choice test provides the student with a number of options from which they are to select the correct answer e.g. The Milky Way is a … A
galaxy
B
solar system
C
universe
D
star
Using such a test to assess knowledge can be problematic not least because the person being tested could guess the correct answer without any understanding of the topic. The literature on multiplechoice testing is wide-ranging but can be broadly categorised into four areas: question writing, administration of tests (electronically), scoring systems and results analysis.
The work comes
predominantly from higher-education (especially in medicine, law, economics and IT) with contributions from statistics and psychology.
1
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
123 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
124 / 205
Dirichlet or Potts ? Ali Mohammad-Djafari Laboratoire des signaux et systmes (L2S) Suplec, Plateau de Moulon, 3 rue Joliot Curie 91192 Gif-sur-Yvette, France Abstract When modeling the distribution of a set of data {xi , i = 1, · · · , n} by a mixture of Gaussians (MoG), there are two possibilities: i) the classical one is using a set of parameters which are the proportions αk , the means µk and the variances σk2 ; ii) the second is to consider the proportions αk as the probabilities of a hidden variable z whith αk = P (z = k) and assignining a prior law for z. In the first case P a usual prior distribution for αk is the Dirichlet which account for the fact that k αk = 1. In the second case, to each data xi we associate a hidden variable zi . Then, we have two possibilities: either assuming the variables zi to be i.i.d. or assigning them a Potts distribution. In this paper we give some details on these models and different algorithms used for their simulation and the estimation of their parameters. More precisely, P in the first case, the assumption is that the data are i.i.d sam2 ples from p(x) = k=1 αk N (µk , σk ) and the objective is the estimation of θ = 2 {K, (αk , µk , σk ), k = 1, · · · , K}. In the second case, the assumption is that the data xi is a sample from p(xi |zi = k) = N (µk , σk2 ), ∀i where the zi can only take the values k = 1, · · · , K. P Then if we assume zi i.i.d., then the two models become equivalent with αk = n1 ni=1 δ(zi −k). But if we assume that there some structure in the hidden n P o variables, we can use the Potts model p(zi |zj , j 6= i) ∝ exp γ j∈V(i) δ(zi − zj ) where V(i) represents the neighboring elements of i, for example V(i) = i − 1 or V(i) = {i − 1, i + 1} or in cases where i represents the index of a pixel in an image, then V(i) represents the four nearest neigbors of that pixel. γ is the Potts parameter. These two models have been used in many data classification or image segmentation where the xi represents either the grey level or the color components of the pixel i and zi its class labels. The main objective of an image segmentation algorithm is the estimation of zi . When the hyperparameters K, θ = (αk , µk , σk2 ), k = 1, · · · , K and gamma are not known and have also to be estimated, we say that we are in totally unsupervised mode, when are known we are in totally supervised mode and we say that we are in partially supervised mode when some of those hyperparameters are fixed. In the following, we present some of these methods. Key Words: Mixture of Gaussians, Dirichlet, Potts, Classification, Segmentation.
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
125 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
126 / 205
ESTIMATION AND DETECTION OF A PERIODIC SIGNAL D. Aronsson, E. Bj¨ornemo, M. johansson Signals and Systems Group, Uppsala University, Sweden dar,eb,
[email protected] Abstract Detection and estimation of a periodic signal with an additive disturbance is considered. We study estimation of both the frequency and the shape of the waveform and develop a method based on Fourier series modelling. The method has an advantage over time domain methods such as epoch folding, in that the hypothesis space becomes continuous. Using uninformative priors, the noise variance and the signal shape can be marginalised analytically, and we show that this expression can be evaluated in real time when the data is evenly sampled and does not contain any low frequencies. We compare our method with other frequency domain methods. Although derived in various different ways, most of these, including our method, have in common that the cumulative periodogram plays a central role in the estimation. But there are important differences. Most notable are the different penalty terms on the number of harmonic frequencies. In our case, these enter the equations automatically through the use of probability theory, while in previous methods they need to be introduced in an ad hoc manner. The Bayesian approach in combination with the chosen model structure also allow us to build in prior information about the waveform shape, improving the accuracy of the estimate when such knowledge is available.
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
127 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
128 / 205
SENSOR NETWORK NODE SCHEDULING FOR ESTIMATION OF A CONTINUOUS FIELD E. Bj¨ornemo, M. Johansson, D. Aronsson Signals and Systems Group, Uppsala University, Sweden eb,mj,
[email protected] Abstract A wireless sensor network consists of radio-equipped sensors that are spread out in space to perform some network task, such as monitoring or estimating a field quantity. In many sensor networks, the main limitation is the scarce energy resources available at each sensor node. A major issue is therefore optimisation of the activity in the network with respect to energy consumption. We investigate such an energylimited sensor network, whose purpose is to estimate a continuous field over a certain spatial and temporal range. One way of reducing energy consumption is to utilise knowledge of the field variations to reduce the number of actual measurements and thus not waste energy on measuring quantities that can be inferred with knowledge of related parameters. We investigate the trade-off between estimation performance and resource cost in terms of energy consumption, and devise a general Bayesian estimation scheme to take advantage of (necessarily incomplete) knowledge of physical properties of the field, such as bounds on time and space variations. Each measurement is taken at a discrete point in space and time and our goal is to infer the entire field over a given time and space horizon. We assume that the position of each node is known and that there is a known node-specific cost associated with each sensor measurement. The central unit schedules sensor measurements according to cost and information gain. We assume simple sensors that only perform the assigned measurements and forward them to the central unit along pre-defined routeS. We illustrate how different states of uncertainty lead to interesting special cases of the general problem scenario, and discuss relations to Nyquist sampling of a time series of known bandwidth.
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
129 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
130 / 205
Maximum entropy approa h to hara terization of random media E. V. Vakarin
UMR 7575 LECA ENSCP-UPMC, 11 rue P. et M. Curie, 75231 Cedex 05, Paris, Fran e e-mail: vakarin
r.jussieu.fr
Chara terization of omplex disordered media (porous matri es, random networks, et .) is usually based on an analysis of indire t probes. This is realized through a onta t of the medium and a system with a well-de ned response fun tion (j ) onditional to the medium state . Thus one deals with an inversion of the following integral () =
Z
df ( )(j );
where () is an experimental result, f ( ) is the desired distribution of some relevant quantity, . In many ases the inversion of this integral with respe t to f ( ) is an "illposed" mathemati al problem. Therefore, urrent approa hes involve either sophisti ated regularization pro edures, or a tting with multiple adjustable parameters. The problem is ompli ated by the absen e of a unique solution and strong sensitivity to the input deviations. Based on a ombination of the statisti al thermodynami s and the maximum information prin iple [1℄ we propose a omplementary approa h to this problem. The distributions are al ulated through a maximization of the Shannon entropy fun tional onditioned by the available data (). The s heme is shown to provide an expli it solution and a systemati link between the distribution and the input ((), and (j )). The gained amount of information is shown to be dire tly related to the probe thermodynami state (). Several illustrative examples, relevant to adsorption probes are dis ussed.
[1℄ E. T. Jaynes, Phys. Rev. 106, 620 (1957)
1
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
131 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
132 / 205
INFORMATION-THEORETIC MEASURES OF SOME QUANTUM SYSTEMS R.J. Y´an ˜ez1,3 , J.S. Dehesa2,3 (1) Dpto. de Matem´atica Aplicada, Universidad de Granada, Spain (2) Dpto. de F´ısica Moderna, Universidad de Granada, Spain (3) Instituto Carlos I de F´ısica Te´orica y Computacional, Universidad de Granada, Spain. (email:
[email protected],
[email protected], Fax: 958242862) Abstract The distribution of a probability density function all over its domain of definition may be best measured by means of information-theoretic notions of both global (Shannon entropy) and local (Fisher information) characters. These quantities will be here computed for several classical and quantum systems directly from its wave equation. In this communication we shall make emphasis on the following single particle systems: atoms in a spherically symmetric potential, circular membrane and atoms in an external electric field. The extension to multidimensional systems will be also discussed. All these problems require an extensive use of the theory of special functions and orthogonal polynomials. References: [1] J. S. Dehesa, A. Mart´ınez-Finkelshtein & V.N. Sorokin, Short-wave asymptotics of the information entropy of a circular membrane, Int. J. Bifurcation and Chaos, 12 (2002) 2387. [2] J.S. Dehesa, S. L´ opez-Rosa, B. Olmos & R.J. Y´an ˜ez, The Fisher information of D-dimensional hydrogenic systems in position and momentum spaces, J. Mathematical Physics (2006). Accepted. Key Words: Shannon entropy, Fisher information, wave equation, circular membrane, quantum mechanics, atoms, atoms in external fields, special functions, orthogonal polynomials
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
133 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
134 / 205
INTEGRATED DATA ANALYSIS: NON-PARAMETRIC PROFILE GRADIENT ESTIMATION R. Fischer1 , A. Dinklage2 , V. Dose1 Max-Planck-Institut f¨ ur Plasmaphysik, EURATOM Association, (1) Boltzmannstr. 2, D-85748 Garching, Germany (2) Greifswald Branch, Wendelsteinstr. 1, D-17493 Greifswald, Germany (
[email protected]) Abstract The estimation of distributions and distribution gradients from pointwise measurements of profiles is frequently hampered by measurement errors and lack of information. A combination of measured profile data from heterogeneous experiments is suitable to provide a more reliable data base to decrease the estimation uncertainty by complementary measurements. The Integrated Data Analysis (IDA) concept allows to combine data from different experiments to obtain improved results [1]. Persisting missing information is usually regularized by applying parametric interpolation schemes to fit profiles and derive gradients at the expense of flexibility. The lack of flexibility affects in particular the estimation of profile gradients. The estimation of profile gradient uncertainties is usually not considered. The goal is to reconstruct profiles only from the significant information in the measured data and avoid noise fitting without restricting profiles using parametric functions. A flexible non-parametric distribution estimation is achieved by using exponential splines. Exponential splines adaptively allow for flexibility in regions where profile data provide detailed information as well as smoothness (cubic splines as limiting case) elsewhere. Regularization parameters as well as number of knots and knot positions are marginalized in the framework of Bayesian probability theory. The resulting posterior probability distribution allows to estimate profiles, profile gradients and their uncertainties in a natural way. An application of exponential splines will be shown on temperature and density profile gradient estimation from an integrated data set measured with different experiments for transport modeling at Wendelstein 7-AS and ASDEX Upgrade. References: [1] R. Fischer, A. Dinklage, and E. Pasch, Plasma Phys. Control. Fusion 45 (2003) 1095 [2] V. Dose and A. Menzel, Global Change Biology 10 (2004) 259 [3] V. Dose and R. Fischer, Bayesian Inference and Maximum Entropy Methods in Science and Engineering, ed. K. Knuth, AIP Conf. Proc. 803 (2005) 67
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
135 / 205
26-th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
MaxEnt 2006 —
CNRS ,
Paris, France, July 8-13, 2006
136 / 205
!#"%$'&)(+*,*+(,"./102.(+354#3 67"98;:='?A@B0 8C0DE "#FG4#*H8I0>"J8
