Min-Plus Probability Calculus J.P. Quadrat ∗† March 18, 1998

Abstract We study here the duality appearing between probability calculus and optimization by substituting the semifield (R ∪ {+∞}, min, +) for the standard semiring (R+ , +, ×).

1

Introduction

This lecture aid is not original, it is, mainly, a compilation and reorganization of some sections of the three papers [5, 48, 20] in which all the members of the Maxplus working group and in particular M. Viot and M. Akian have played a key role. The min-plus probability calculus, called decision calculus, is obtained from the probability calculus by substituting the idempotent semifield (R ∪ {+∞}, min, +) for the standard semiring (R+ , +, ×). To the probability of an event corresponds the cost of a set of decisions. To random variables correspond decision variables. Almost all concepts and tools of probabilities have an analogue. First, we give the counterparts of characteristic functions, weak convergence, tightness and limit theorems. The analogue of Markov chains are the so-called Bellman Chains. The asymptotic theorems for the Bellman chains are the general min-plus linear system ones. They can be seen in [10, 22, 23, 48] and will not discussed here. The min-plus product forms exist and correspond to computing geodesics on a Z-module. In some cases, explicit formulae dual of the standard product forms give explicitly the minimal distance between two states. The general problem can be reduced to a standard flow problem. ∗ J.P. Quadrat : INRIA Domaine de Voluceau Rocquencourt, BP 105, 78153, Le Chesnay (France). Email : [email protected]. † This work has been partly supported by the ALAPEDES project of the European TMR programme.

1

The Cramer transform used in the large deviation literature is defined as the composition of the Laplace transform by the logarithm by the Fenchel transform. It transforms convolution into inf-convolution. Probabilistic results about processes with independent increments are then transformed into similar results on dynamic programming equations for systems with instantaneous costs which do not depend of the states. By this way we obtain explicit solutions of some Hamilton Jacobi Bellman equations (HJB) called Hopf formulae. This Cramer transform is well known in statistical mechanics. We illustrate, on a simple example, called min-plus perfect gaz, how the Cramer transform appears in the computing of the correponding Gibbs distribution. Bibliographic notes are given at the end of the paper.

2

Cost Measures and Decision Variables

2.1

Cost measures

Le us denote Rmin the idempotent semifield (R ∪ {+∞}, min, +) and by extension the metric space R ∪ {+∞} endowed with the exponential distance d(x , y) = | exp(−x ) − exp(−y)|. We start by defining cost measures which can be seen as normalized idempotent measures of Maslov in Rmin [41] . We call a decision space the triplet (U, U , K) where U is a topological space, U the set of open sets of U and K a mapping from U to Rmin such that 1. K(U ) = 0, 2. K(∅) = +∞,
S 3. K n An = infn K(An ) for any An ∈ U . The mapping K is called a cost measure. A set of cost measures K is said tight if sup

inf K(C c ) = +∞ .

Ccompact⊂U K∈K

A mapping c : U → Rmin such that K(A) = infu∈A c(u) ∀A ⊂ U is called a cost density of the cost measure K. def The set Dc = {u ∈ U | c(u) 6= +∞} is called the domain of c. Theorem 1. Given a l.s.c. c with values in Rmin such that infu c(u) = 0, the mapping A ∈ U 7→ K(A) = infu∈A c(u) defines a cost measure on (U, U ). Conversely any cost measure defined on open sets of a second countable topological space1 1

i.e. a topological space with a countable basis of open sets.

2

admits a unique minimal extension K∗ to P(U ) (the set of subsets of U ) having a density c which is a l.s.c. function on U satisfying infu c(u) = 0. Proof. This precise result is proved in Akian [1]. See also Maslov [41] and Del Moral [31] for the first part and Maslov and Kolokoltsov [39] for the second part. This theorem shows that on second countable spaces there is a bijection between l.s.c. functions and cost measures. In this paper, we will consider cost measures on Rn , RN , separable Banach spaces and reflexive Banach separable spaces with the weak topology which are all second countable topological spaces. We will use very often the two following cost densities defined on Rn with k.k the euclidian norm.  +∞ for x 6= m. def 1. χm (x ) = 0 for x = m, p

def 1 −1 p kσ (x

2. Mm,σ (x ) =

p

def

− m)k p for p ≥ 1 with Mm,0 = χm .

By analogy with the conditional probability we define conditional cost excess to take the best decision in A knowing that it must be taken in B by def

K(A|B) = K(A ∩ B) − K(B) .

2.2

Decision Variables

By analogy with random variables we define decision variables and related notions. 1. A decision variable X on (U, U , K) is a mapping from U to E (a second countable topological space). It induces a cost measure K X on (E, B) (B denotes the set of open sets of E) defined by K X (A) = K∗ (X −1 (A)) for all A ∈ B. The cost measure K X has a l.s.c. density denoted c X . When E = R, we call X a real decision variable; when E = Rmin , we call it a cost variable. 2. Two decision variables X and Y are said independent when: c X,Y (x , y) = c X (x ) + cY (y).

3. The conditional cost excess of X knowing Y is defined by: def

c X|Y (x , y) = K∗ (X = x | Y = y) = c X,Y (x , y) − cY (y).

3

4. The optimum of a decision variable is defined by def

O(X ) = arg min conv(c X )(x ) x∈E

when the minimum exists, where conv denotes the l.s.c. convex hull and arg min the point where the minimum is reached. When a decision variable X with values in a linear space satisfies O(X ) = 0 we say that it is centered. 5. When the optimum of a decision variable X with values in Rn is unique and when near the optimum, we have conv(c X )(x ) =

1 −1 kσ (x − O(X ))k p + o(kx − O(X )k p ) , p

we say that X is of order pand we define its sensitivity of order p by def S p (X ) = σ . When S p (X ) = I (the identity matrix) we say that X is of order p and normalized. def

6. The value of a cost variable X is V(X ) = infx (x + c X (x )), the conditional def value is V(X | Y = y) = infx (x + c X|Y (x , y)). 7. The density cost of the sum Z of two independent variables X and Y is the inf-convolution of their cost densities c X and cY , denoted c X ? cY defined by c Z (z) = inf [c X (x ) + cY (y) | x + y = z] . x,y

p

For a real decision variable X of cost Mm,σ , p > 1, we have O(X ) = m, S p (X ) = σ, V(X ) = m −

2.3

1 p0 σ . p0

Vector Spaces of Decision Variables

We can introduce vector spaces of decision variables which are the analogue of the standard L p () spaces. Theorem 2. For p > 0, the numbers   1 def def |X | p = inf σ | c X (x ) ≥ |(x − O(X ))/σ | p and kX k p = |X | p + |O(X )| p define respectively a seminorm and a norm on the vector space L p of real decision variables having a unique optimum and such that kX k p is finite. 4

Proof. Let us denote X 0 = X − O(X ) and Y 0 = Y − O(Y ). We first remark that σ > |X | p implies c X (x ) ≥

1 1 (|x − O(X )|/σ ) p ∀x ⇔ V(− |X 0 /σ | p ) ≥ 0 . p p

(1)

If there exists σ > 0 and O(X ) such that (1) holds, then c X (x ) > 0 for any x 6= O(X ) and c X (x ) tends to 0 implies x tends to O(X ) therefore O(X ) is the unique optimum of X . Moreover |X | p is the smallest σ such that (1) holds. If X ∈ L p , λ ∈ R and σ > |X | p we have 1 1 V(− |λX 0 /λσ | p ) = V(− |X 0 /σ | p ) ≥ 0 , p p then λX ∈ L p , O(λX ) = λO(X ) and |λX | p = |λ||X | p . If X and Y ∈ L p , σ > |X | p and σ 0 > |Y | p , 1 1 1 V(− (max(|X 0 /σ | p , |Y 0 /σ 0 | p )) = min(V(− |X 0 /σ | p ), V(− |Y 0 /σ 0 | p )) ≥ 0 p p p and

|X 0 + Y 0 | σ |X 0 | |X 0 | |Y 0 | σ 0 |Y 0 | ≤ ≤ max( + , 0 ), σ + σ0 σ + σ0 σ σ + σ0 σ0 σ σ

then

1 V(− (|X 0 + Y 0 |/(σ + σ 0 )) p ) ≥ 0 . p

Therefore we have proved that X + Y ∈ L p with O(X + Y ) = O(X ) + O(Y ) and |X + Y | p ≤ |X | p + |Y | p . Then L p is a vector space, |.| p and k.k p are seminorms and O is a linear continuous operator from L p to R. Moreover, kX k p = 0 implies c X = χ thus X = 0 up to a set of infinite cost. Theorem 3. For two independent real decision variables X and Y and k ∈ R we have (as soon as the right and left hand sides exist) O(X + Y ) = O(X ) + O(Y ), O(k X ) = kO(X ), S p (k X ) = |k|S p (X ) , 0

0

0

0

0

0

[S p (X + Y )] p = [S p (X )] p + [S p (Y )] p , (|X + Y | p ) p ≤ (|X | p ) p + (|Y | p ) p .

5

Proof. Let us prove only the last inequality. Consider X and Y in L p and σ > |X | p 0 0 0 and σ 0 > |Y | p . Let us denote σ 00 = (σ p + σ 0 p )1/ p , X 0 = X − O(X ) and 0 0 0 Y 0 = Y − O(Y ). The H¨older inequality aα + bβ ≤ (a p + b p )1/ p (α p + β p )1/ p implies (|X 0 + Y 0 |/σ 00 ) p ≤ |X 0 /σ | p + |Y 0 /σ 0 | p , then by the independency of X and Y we get 1 V(− (|X 0 + Y 0 |/σ 00 ) p ) ≥ 0 , p and the inequality is proved. Theorem 4 (Chebyshev). For a decision variable belonging to L p we have K(|X − O(X )| ≥ a) ≥ K(|X | ≥ a) ≥

1 (a/|X | p ) p , p

1 ((a − kX k p )+ /kX k p ) p . p

Proof. The first part is a straightforward consequence of the inequality cY (y) ≥ (|y|/|Y | p ) p / p applied to centered decision variable Y . The second part comes from the non increasing property of the function x ∈ R+ 7→ (a − x )+/x .

2.4

Characteristic Functions and Fenchel Transform

The role of the Laplace or Fourier transforms in probability calculus is played by the Fenchel transform in decision calculus. Let c ∈ Cx , where Cx denotes the set of functions from E (a reflexive Banach space with dual E 0 ) to Rmin convex, l.s.c. and proper2 . Its Fenchel transform is the function from E 0 to Rmin defined by def

def

c(θ ˆ ) = [F (c)](θ ) = sup[hθ, x i − c(x )] . x

def

Then the characteristic function of a decision variable is defined by F(X ) = F (c X ). Important properties of the Fenchel transform are : for f, g ∈ Cx 2

not always equal to +∞

6

1. F ( f ) ∈ Cx , 2. F is an involution that is F (F ( f )) = f , 3. F ( f ? g) = F ( f ) + F (g), 4. F ( f + g) = F ( f ) ? F (g). Therefore, for two independent decision variables X and Y and k ∈ R, we have F(X + Y ) = F(X ) + F(Y ), [F(k X )](θ ) = [F(X )](kθ ) . Moreover, a decision variable with values in Rn is of order p if we have : F(X )(θ ) = hO(X ), θ i +

2.5

1 p 0 0 kS (X )θ k p + o(kθ k p . 0 p

Convergences of Decision Variables

The analogue of the topologies used in probability can be introduced. The standard limit theorems have a min-plus counterpart. A sequence of independent and identically costed (i.i.c.) real decision variables of cost c on (U, U , K) is an application X from U to RN which induces the density cost c X (x ) =

∞ X

c(x i ), ∀x = (x 0 , x 1 , . . . ) ∈ RN .

i=0

The cost density is finite only on minimizing sequences of c, elsewhere it is equal to +∞. We have defined a decision sequence by its density and not by its value on the open sets of RN because the density always exists and can be defined easily. In order to state limit theorems, we define several type of convergence of sequences of decision variables. Definition 5. For the sequence of real decision variables {X n , n ∈ N}, cost measures Kn and cn functions from U (a first countable topological space3 to Rmin we say that : Lp

1. X n ∈ L p converges in p-norm towards X ∈ L p denoted X n −→ X , if limn kX n − X k p = 0 ; 3

Each point admits a countable basis of neighbourhoods.

7

K

2. X n converges in cost towards X , denoted X n −→ X , if for all  > 0 we have limn K{u | |X n (u) − X (u)| ≥ } = +∞; a.s. 3. X n converges almost surely towards X , denoted X n −→ X , if we have K{u | limn X n (u) 6= X (u)} = +∞ . w

4. Kn converges weakly towards K, denoted Kn → K, if for all f in Cb (E) we have limn Kn ( f ) = K( f )5 .

4

epi

5. cn converges in the epigraph sense (epi-converges) towards c, denoted cn → c if ∀u, ∀u,

∀u n → u,

lim inf cn (u n ) ≥ c(u) ,

(2)

∃u n → u : lim sup cn (u n ) ≤ c(u) .

(3)

n

n

A sequence Kn of cost measures is said asymptoticaly tight if sup Ccompact⊂U

lim inf Kn (C c ) = +∞ . n

The different “weak” convergences have strong relations given in the following theorem. Theorem 6. Let Kn , K be cost measures on a metric space U. Then the three following conditions are equivalent w

1. Kn → K ; 2. lim inf Kn (F) ≥ K(F),

∀F closed ,

(4)

lim sup Kn (G) ≤ K(G),

∀G open ;

(5)

n

n

◦

¯ 3. limn Kn (A) = K(A) for any set A such that K( A ) = K( A). 4 5

Cb (E) denotes the set of continuous and lower bounded functions from E to Rmin . def K( f ) = infu ( f (u) + c(u)) where c is the density of K.

8

On asymptoticaly tight sequences Kn the weak convergence of Kn towards K is equivalent to (5) and lim inf Kn (C) ≥ K(C), n

∀C compact .

(6)

On a first countable topological space, the epi convergence of l.s.c. cost densities is equivalent to (6) and (5). Proof. See [5]. In a locally compact space conditions (6) (5) are equivalent to the condition limn Kn ( f ) = K( f ) for any continuous function with compact support. This is the definition of weak convergence used by Maslov and Samborski in [43]. These conditions are are also equivalent to the epigraph convergence of densities. This type of convergence does not insure that a weak-limit of cost measures is a cost measure (the infimum of the limit is not necessarily equal to zero). Denoting by K(U ) the set of cost measures on U (a metric space) endowed with the topology of the weak convergence, any tight set K of K(U ) is relatively sequentially compact6 . These different kinds of convergence are connected in a nonstandard way. Theorem 7. We have the implications : 1. Convergence in p-norm implies convergence in cost but the converse is false. 2. Convergence in cost implies almost sure convergence and the converse is false. 3. For tight sequences, the convergence in cost implies the weak convergence. Proof. See Akian [2] for points 1 and 2 and 3 and Del Moral [31] for point 2. We have the analogue of the law of large numbers. Theorem 8. Given a sequence {X n , n ∈ N} of i.i.c. decision variables belonging to L p , p ≥ 1, we have def

YN =

X 1 N−1 X n → O(X 0 ) , N n=0

where the limit can be taken in the sense of almost sure, cost and p-norm convergence. 6

that is any sequence of K contains a weakly convergent subsequence

9

Proof. We have only to estimate the convergence in p-norm. The result follows from simple computation of the p-seminorm of Y N . Thanks to Theorem 3 we have 0 0 0 (|Y N | p ) p ≤ N (|X 0 | p ) p /N p which tends to 0 as N tends to infinity. We have the analogue of the central limit theorem. Theorem 9. Given an i.i.c. sequence {X n , n ∈ N} centered of order p with l.s.c. convex cost, we have def

ZN =

N−1 X

1 N 1/ p

0

w

p

X n → M0,S p (X 0 ) .

n=0 0

Proof. We have limN [F(Z N )](θ ) = p10 kS p (X 0 )θ k p , where the convergence can be taken in the pointwise, uniform on any bounded set or epigraph sense. In order to obtain the weak convergence we have to prove the tightness of Z N . But as the convergence is uniform on B = {kθ k ≤ 1} we have for N ≥ N0 , F(Z N ) ≤ C on B where C is a constant. Therefore c Z N (x ) ≥ kx k − C for N ≥ N0 and Z N is asymptoticaly tight. The central limit theorem may be generalized to the case of non convex cost densities. We have the analogue of the large deviation theorem. Theorem 10. Given an i.i.c. sequence {X n , n ∈ N} of tight cost density c, we have : w 1/nc(X 1 +···+X n )/n → cˆ , where cˆ denotes the convex hull of c. Proof. Let us give only the ideas of the proof. The cost density of X 1 + · · · + X n is c?n . Therefore we want limn 1/nc?n (x /n). But the epigraph of the convolution of two functions is equal to the sum of their epigraphs. Therefore the epigraph of 1/nc?n (x /n) is equal to the vectorial mean of n identical convex sets which are epigraphs of c. The limit (in the epigraph sense), when n goes to infinity, of the vectorial mean converges towards the convex hull of the epigraph of c [25]. See also [3]. This last theorem is only a trivial case of law of large numbers for random sets studied in [25, 26]. The interpretation of this result as a min-plus large deviation theorem comes from by M. Akian.

10

3

Bellman Chains

We denote M p the dioid of ( p, p)-matrices with entries in the semiring K. The matrix product in M p is def

def

[AB]i j = [A ⊗ B]i j = min[Aik + Bkj ] . k

All the entries of the zero matrix of M p are +∞. The diagonal entries of the identity matrix of M p are 0, the other entries being +∞. With a matrix M in Mn (K), we associate a precedence graph G(M) = (N , P) with nodes N = {1, 2, · · · , n}, and arcs P = {x y | x , y ∈ N , Mxy 6= ε}. The number Mxy , when it is nonzero, is called the weight of the arc x y. A path π , of length l, with origin x and end y, is an ordered set of nodes π = π0 π1 · · · πl with π0 = x and πl = y, and πi πi+1 ∈ P for all i = 0, · · · , l − 1. The couple πi πi+1 are called the arcs of π and the πi its nodes. The length of the path π is denoted |π |. The couple x y of the ends of π is denoted hπ i. When the two ends of π are equal one says that π is a circuit. The weight of π , denoted π(M), is the ⊗-product of the weights of its arcs. For example we have x yz(M) = Mxy ⊗ M yz . l . The paths of The set of all paths with ends x y and length l is denoted Pxy ∗ length 0 are the nodes P 0 = N . Then, Pxy is the set of all paths with ends x y and ∗ P the set of all paths. We have : P∗ =

def

∞ [

Pl .

l=0

For ρ ⊂ P, hρi is the set of the ends of the paths of ρ. Then denoting P N the set of arcs of the graph associated to the matrix N we have the following trivial accessibility results : Proposition 11. For M ∈ Mn we have : P M k = hP k i, P M ∗ = hP ∗i. For ρ ⊂ P ∗ one define : def

ρ(M) =

M π∈ρ

π(M) ,

which is the infimum of the weights of all the paths belonging to ρ. We denote ∞ M def M∗ = Mi , i=0

which exists if we accept entries in Rmin . Then, we have the following interpretation of the matrix product in Mn . 11

Proposition 12. For M ∈ Mn we have l ∗ Pxy (M) = (M l )xy , P xy (M) = (M ∗ )xy .

(7)

The matrix M ∗ has no entries equal to −∞ iff there is no circuits with negative weight in P. The min-plus Markov chain is called a Bellman chain, it is defined by a transition cost matrix M ∈ Mn satisfying Me = e where e denotes the column of e of size n. Then, given an initial cost, which is a line vector c0 satisfying c0 e = e, we can define a cost c, on the set of paths P, by c(π ) = cπ0 0 π(M) for all π ∈ P l and l ∈ N. Then, the analogue of the forward Kolmogorov equation is the forward Bellman equation cn = cn−1 ⊗ M, c0 given. It gives the marginal cost, for the def Bellman chain X n (π ) = πn , to be in state (node) x ∈ N at time n. If a transition cost matrix satisfies Mxy = M yx > 0, Mxx ≥ 0, ∀y 6= x ∈ N , ∗ ∗ ∗ then the matrix Mxy defines a metric. Indeed, we have Mxx = 0 and Mxy ≤ ∗ ∗ ∗ ∗ Mxz + Mzy by definition of the matrix product and the fact that M M = M ∗ . A path from x to y in G(M) achieving the minimal cost among the paths of any length is called a geodesic joining x to y. We will still call a geodesic an optimal path when the matrix M is nonsymmetric.

3.1

Min-plus closed Jackson service networks

A closed Jackson network of queues is a set of n customers and m services. The customers wait for services in queues attached to each service. The customers are served in the order of arrival. The service is random and markovian. In discrete time situation, a (m, m) transition probability matrix r is given. The entry ri j is the probability that a customer, served at queue i, goes to queue j , if the queue i is not empty. If the queue is empty, this probability is 0. Such a system is a Markov chain with state space : def

Snm = {x ∈ Nm : 1.x = n} , where 1 denotes the vectors with all its entries equal to 1 with size adapted to the context (here m). It is clear that, if r is irreducible, the Markov chain describing the system is irreducible. Therefore it has a unique invariant measure p. This measure is explicitly computable : px = kθ1x1 · · · θmxm , 12

with θ any solution of θr = θ and k a normalizing constant such that p1 = 1. The best way to understand what is a min-plus closed Jackson service network is to consider the following problem. We consider a company renting cars. It has n cars and m parkings in which customers can rent cars. The customers can rent a car in a parking and leave the rented car in another parking. After some time the distribution of the cars in the parkings is not satisfactory and the company has to transport the cars to achieve a better distribution. Given r the (m, m) matrix of transportation cost from a parking to another, the problem is to determine the minimal cost of the transportation from a distribution x = (x 1 , · · · , x m ) of the cars in the parking to another one y = (y1 , · · · , ym ) and to compute the best plan of transportation. Therefore the precise transportation problem is the following. M IN - PLUS C LOSED JACKSON P ROBLEM (T RANSPORTATION P ROBLEM ). Given the (m, m) transition cost matrix r irreducible such that ri j > 0 if i 6= j = 1, · · · , m and rii = 0 for all i = 1, · · · , m, compute M ∗ for the the Bellman def chain on Snm of transition cost M defined by Mx,Ti j (x) = ri j and def

Ti j (x 1 , · · · , x m ) = (x 1 , · · · , x i − 1, · · · , x j + 1, · · · , x m ) , for i, j = 1, · · · , m. The operator Ti j corresponds to the transportation of a car from the parking i def

to the parking j . We denote T = {Ti j , i, j = 1, · · · , m}. If rii = e for all i = 1, · · · , m (the absence of transportation costs nothing) the previous problem corresponds to the computation of the largest invariant cost c satisfying c = cM, and cx = e. Indeed, in this case the left eigen semimodule has as many independent generators as states7 . Remarking that the diagonal entries of ∗ M = M ∗ . Then, from the fact that q = bM ∗ is the largest M ∗ are e, it is clear Mx. x. solution of q = q M ⊕ b, we can prove that the searched extremal left eigenvector ∗. is Mx.

3.2

Solution of the m-parkings transportation problem

Let us consider the m-parkings case. In this case a path π ∈ P is x T 1 (x )T 2 ◦ T 1 (x ) · · · y = T l ◦ T l−1 ◦ · · · T 1 (x ) , with T i ∈ T . Since the arcs Pr of r are x Ti j (x ) with x ∈ N and Ti j ∈ T we can code (') a path π ∈ P ∗ in a simpler way by the couple π ' x µ with x ∈ N a 7

Let us recall that in the min-plus context the irreducibility of the transition matrix assures the uniqueness of the eigenvalue but not the uniqueness of the generators of the eigen semimodule see [10] Section 3.7.

13

node of G(M) and µ ∈ Pr∗ a path of G(r). Clearly we have : π(M) = µ(r), ∀π ' x µ ∈ P ∗ . Remarking that the vector Ti j (x ) − x is independent of x let us call it γi j and denote 0 = {γi j , i, j = 1, · · · , m}. These vectors are not mutually independent indeed we have the relations : γik = γi j + γ j k , ∀i, j, k = 1, · · · , m . For a path µ ∈ Pr∗ , the evaluation µ(0) ∈ Zm is obtained by using the morphism which to the concatenation associates the vectorial sum and to the letters associate the corresponding vectors of 0. For example for the path i j kl ∈ Pr∗ we have : i j kl(0) = γi j + γ j k + γkl . Then, the constraint on the paths π : hπ i = x y with x , y ∈ Zm is equivalent to the constraint µ(0) = y − x for the path π ' x µ. The cost of a path µ(r) depends only of the number of times each arc appears in µ and not of the order of the arcs. Similarly the constraint µ(0) = y − x does not depend of the order of the arcs in the path µ, since the evaluation µ(0) corresponds to additions of vectors, and addition of vector is commutative. To take account of this symmetry of the problem we denote Prc the set of equivalent classes of paths (where two paths are equivalent if the arcs appear the same number Q of times). Therefore, for µ ∈ Prc we can take the representative µ = a∈Pr a na . For example the path µ = i j i j k belongs to the class of (i j )2 ( j i)( j k). It is clear that µ(r ∗ ) ≤ µ(r) because ri j ≥ ri∗j . Moreover for each µ it exists µ˜ such that µ(r ∗ ) = µ(r) ˜ and µ(0) = µ(0). ˜ The path µ˜ is obtained by substituting the arcs i j of µ by paths µi j such that µi j (ri j ) = ri∗j . Inside Snm this substitution is always possible. This is not always possible on the boundary of Snm because the path x µ may leave Snm . To avoid this difficulty we suppose that the costs on the boundary arcs are not ri j but ri∗j . We can summarize the previous considerations in the following proposition. Proposition 13. The optimal value of the transportation problem is : ∗ ∗ Mxy = P xy (M) = 8r ∗ (y − x ) ,

with

def

8r ∗ (z) =

M µ∈Prc∗ µ(0)=z

14

µ(r ∗ ) .

The mathematical program 8r ∗ (z) is a flow problem. Proposition 14. Denoting by J the incidence matrix nodes-arcs of the complete graph with m nodes we have : 8r ∗ (z) = inf φ.r ∗ , φ≥0 J φ=z

where φ.r =

P

i, j ri j φi j .

Corollary 15. We have for all y and x satisfying x j = 0 for j 6= i and x i = n O ∗ Mxy = (ri∗j ) y j , j, j 6=i

and for all x and y satisfying y j = 0 for j 6= i and yi = n O ∗ Mxy = (r ∗j i )x j . j, j 6=i

Proof. In these two cases the flow problems are trivial. The nonnul components are respectively φi j = y j and φ j i = x i , for j 6= i. This corollary gives the searched min-plus product form.

3.3

example

Let us consider the transportation system with 3 parkings and 6 cars, and transportation costs :     0 1 +∞ e 1  r = +∞ 0 1  =   e 1 . 1 +∞ 0 1  e We have :



 e 1 2 r ∗ = 2 e 1 . 1 2 e

Let us suppose that x = (0, 0, 6) and y = (2, 3, 1), we can apply the corollary, we have : ∗ ∗ 2 ∗ 3 Mxy = (r31 ) (r32 ) = 2 × 1 + 3 × 2 = 8 . The Geodesic is given in Fig. 1. 15

1

(0,6,0)

1

2

2 3

3 y

x y

1

1 2

31 (6,0,0)

2

23

12

2

x (0,0,6)

Figure 1: Transportation System (6 cars, 3 parkings)

4

Bellman Processes with independent Increments, InfConvolution and Cramer Transform

4.1

Bellman Processes

We can easily define continuous time decision processes which correspond to deterministic controlled processes. We discuss here only decision processes with continuous trajectories. Definition 16. 1. A continuous time Bellman process X t with continuous trajectories is a decision variable with values in C(R+ ) 8 having the cost density Z ∞ def c X (x (·)) = φ(x (0)) + c(t, x (t), x 0(t))dt , 0

with c(t, ·, ·) a family of transition costs (that is a function c from R3 to Rmin such that inf y c(t, x , y) = 0, ∀t, x ) and φ a cost density on R. When the integral is not defined the cost is by definition equal to +∞. 8

C(R+ ) denotes the set of continuous functions from R+ to R.

16

2. The Bellman process is said homogeneous if c does not depend on time t. 3. The Bellman process is said with independent increments if c does not depend on state x . Moreover if this process is homogeneous, c is reduced to the cost density of a decision variable. p

4. The p-Brownian decision process, denoted by Bt , is the process with independent increments and transition cost density c(t, x , y) = 1p |y| p for all x. As in the discrete time case, the marginal cost to be in state x at time t can be computed recursively using a forward Bellman equation. def

Theorem 17. The marginal cost v(t, x ) = K(X t = x ) is given by the Bellman equation: ∂t v + c(∂ ˆ x v) = 0, v(0, x ) = φ(x ) ,

(8)

def

where cˆ means here [c(∂ ˆ x v)](t, x ) = supy [y∂x v(t, x ) − c(t, x , y)] . p For the Brownian decision process Bt starting from 0, the marginal cost to be at time t in state x satisfies the Bellman equation 0

∂t v + (1/ p0 )|∂x v| p = 0, v(0, ·) = χ . p

Its solution can be computed explicitly, it is v(t, x ) = M0,t 1/ p0 (x ) therefore we have " # xp p V[ f (Bt )] = inf f (x ) + . (9) p x pt p0

4.2

Cramer Transform

Definition 18. The Cramer transform C is a function from M, the set of positive def measures on E = Rn , to Cx defined by C = F ◦ log ◦L, where L denotes the Laplace transform9 . From the definition and the properties of the Laplace and Fenchel transforms the following result is clear. Theorem 19. For µ, ν ∈ M we have C(µ ∗ ν) = C(µ) ? C(ν). 9

µ 7→

R E

e hθ,xi µ(d x).

17

Table 1: Properties of the Cramer transform. M µ 0 δa

log(L(M)) = F (C(M)) R cˆµ (θ) = log eθ x dµ(x) −∞ θa H (λ − θ) + log(λ/(λ − θ)) log( p + (1 − p)eθ )

mθ + 12 |σ θ|2 cˆµ + cˆν log(k) + cˆ cˆ convex l.s.c.

C(M) cµ (x) = supθ (θ x − cˆ(θ)) +∞ χa H (x) + λx − 1 − log(λx) x log( 1−x p ) +(1 − x) log( 1−x p ) +H (x) + H (1 − x) p c(x) = Mm,σ , x ≥ m c(x) = 0, x < m, 1/ p + 1/ p0 = 1 M2m,σ cµ ? cν c − log(k) c convex l.s.c.

cˆ(0) = log(m 0 ) cˆ(0) = 0

infx c(x) = − log(m 0 ) infx c(x) = 0

Sµ = cvx(supp(µ))

cˆ strictly convex in Dcˆ

Dc =S µ

m0 = 1

cˆ is C ∞ in Dcˆ

c is C 1 in Dc

λe−λx−H (x) pδ0 + (1 − p)δ1

mθ +

stable distrib.

Gauss distrib. µ∗ν kµ µ≥0 def R m0 = µ m0 = 1 def

def

R

def

R

m 0 = 1, m =

m 0 = 1, m 2 = x µ m 0 = 1, 1 < p0 < 2 0 0 cˆ = |σ θ| p / p0 + o(|θ| p ) +H (θ)

+ H (θ) 1< p