Estimation dans les modèles non linéaires à effets mixtes
Marc Lavielle le groupe MONOLIX
Marc Lavielle & le groupe MONOLIX
MONOLIX
Le groupe MONOLIX
Groupe de travail pluridisciplinaire, né en octobre 2003. Il regroupe des chercheurs intéressés l’étude et l’utilisation des modèles non linéaires à effets mixtes: statisticiens universitaires, chercheurs de l’INSERM-P7 (applications en pharmacologie) chercheurs de l’INRA (applications en agronomie, génétique animale et microbiologie), chercheurs de la faculté de médecine de Lyon-Sud (applications en oncologie).
Marc Lavielle & le groupe MONOLIX
MONOLIX
le groupe MONOLIX: 2 thèses co-dirigées (INSERM/Université et INRA/Université) articles (statistiques et applications) sessions invitées dans différents congrès (SMAI 2005, SFdS 2005, IBC 2008) participation à différentes conférences (Lyon 2004, INRA 2005, PAGE 2004, 2005 2006, 2007, PAGANZ 2007, . . . ) soutien de l’ANR (Agence National de la Recherche) depuis 2006, soutien de l’INRIA pour recruter des ingénieurs pour le développement du logiciel MONOLIX, soutien de J&J pour développer le logiciel MONOLIX depuis 2006, contrats industriels (Pfizer, Tibotec), collaborations (Roche, Novartis, Servier, Sanofi,UCB) Marc Lavielle & le groupe MONOLIX
MONOLIX
The model
yij = f (xij , φi ) + εij , 1 ≤ i ≤ N , 1 ≤ j ≤ ni
yij ∈ R is the jth observation of subject i, N is the number of subjects ni is the number of observations of subject i. The regression variables, or design variables, (xij ) are known, x = (Dose , Time) for PK and PKPD models, The individual parameters (φi ) are unknown,
Marc Lavielle & le groupe MONOLIX
MONOLIX
The model
yij = f (xij , φi ) + εij , 1 ≤ i ≤ N , 1 ≤ j ≤ ni
yij ∈ R is the jth observation of subject i, N is the number of subjects ni is the number of observations of subject i. The regression variables, or design variables, (xij ) are known, x = (Dose , Time) for PK and PKPD models, The individual parameters (φi ) are unknown,
Marc Lavielle & le groupe MONOLIX
MONOLIX
The model
yij = f (xij , φi ) + εij , 1 ≤ i ≤ N , 1 ≤ j ≤ ni
yij ∈ R is the jth observation of subject i, N is the number of subjects ni is the number of observations of subject i. The regression variables, or design variables, (xij ) are known, x = (Dose , Time) for PK and PKPD models, The individual parameters (φi ) are unknown,
Marc Lavielle & le groupe MONOLIX
MONOLIX
The model yij = f (xij , φi ) + εij , 1 ≤ i ≤ N , 1 ≤ j ≤ ni The vector φi of individual parameters is assumed to be Gaussian: φi = µ + η i
with
ηi ∼i.i.d. N (0, Γ)
µ: unknown vector of population parameters (the fixed effects), (ηi ): unknown random vectors (the random effects). The sequence (εij ) is assumed to be Gaussian: εij ∼i.i.d. N (0, σ 2 ) Marc Lavielle & le groupe MONOLIX
MONOLIX
The model yij = f (xij , φi ) + εij , 1 ≤ i ≤ N , 1 ≤ j ≤ ni The vector φi of individual parameters is assumed to be Gaussian: φi = µ + η i
with
ηi ∼i.i.d. N (0, Γ)
µ: unknown vector of population parameters (the fixed effects), (ηi ): unknown random vectors (the random effects). The sequence (εij ) is assumed to be Gaussian: εij ∼i.i.d. N (0, σ 2 ) Marc Lavielle & le groupe MONOLIX
MONOLIX
The incomplete data model yij = f (xij , φi ) + εij , 1 ≤ i ≤ N , 1 ≤ j ≤ ni We are in a classical framework of “incomplete data”: the mesurement y = (yij , 1 ≤ i ≤ N , 1 ≤ j ≤ ni ) are the “observed data” the individual random parameters φ = (φi , 1 ≤ i ≤ N), are the “non observed data”, the “complete data” of the model is (y , φ). Our purpose is to compute the maximum likelihood estimator of the unknown set of parameters θ = (µ, Γ, σ 2 ), by maximizing the likelihood of the observations `(y , θ), without any approximation on the model. Marc Lavielle & le groupe MONOLIX
MONOLIX
The incomplete data model yij = f (xij , φi ) + εij , 1 ≤ i ≤ N , 1 ≤ j ≤ ni We are in a classical framework of “incomplete data”: the mesurement y = (yij , 1 ≤ i ≤ N , 1 ≤ j ≤ ni ) are the “observed data” the individual random parameters φ = (φi , 1 ≤ i ≤ N), are the “non observed data”, the “complete data” of the model is (y , φ). Our purpose is to compute the maximum likelihood estimator of the unknown set of parameters θ = (µ, Γ, σ 2 ), by maximizing the likelihood of the observations `(y , θ), without any approximation on the model. Marc Lavielle & le groupe MONOLIX
MONOLIX
The Monolix software
Methodology and algorithms used in the Monolix software: Estimation of the population parameters: Maximum likelihood estimation with the SAEM algorithm (combined with MCMC and Simulated Annealing), Estimation of the individual parameters: Estimation of the conditional distributions with MCMC, Estimation of the objective (likelihood) function: Monte Carlo and minimum variance Importance Sampling, Model selection and assessment: Information criteria (AIC, BIC), Statistical Tests (LRT, Wald test), Goodness of fit plots, .
Marc Lavielle & le groupe MONOLIX
MONOLIX
The SAEM algorithm
(Stochastic Approximation of EM)
Delyon, Lavielle and Moulines (the Annals of Statistics, 1999)
Iteration k of the algorithm: step E : Simulation: draw the non observed data φ(k) with the conditional distribution pΦ|Y ( · |y ; θk−1 ) Stochastic approximation: h i Qk (θ) = Qk−1 (θ) + γk log p(y , φ(k) ; θ) − Qk−1 (θ) where (γk ) is a decreasing sequence.
step M: update the estimation of θ: θk = Argmax Qk (θ)
Marc Lavielle & le groupe MONOLIX
MONOLIX
The SAEM algorithm
(Stochastic Approximation of EM)
Delyon, Lavielle and Moulines (the Annals of Statistics, 1999)
Iteration k of the algorithm: step E : Simulation: draw the non observed data φ(k) with the conditional distribution pΦ|Y ( · |y ; θk−1 ) Stochastic approximation: h i Qk (θ) = Qk−1 (θ) + γk log p(y , φ(k) ; θ) − Qk−1 (θ) where (γk ) is a decreasing sequence.
step M: update the estimation of θ: θk = Argmax Qk (θ)
Marc Lavielle & le groupe MONOLIX
MONOLIX
The SAEM algorithm
(Stochastic Approximation of EM)
Delyon, Lavielle and Moulines (the Annals of Statistics, 1999)
Iteration k of the algorithm: step E : Simulation: draw the non observed data φ(k) with the conditional distribution pΦ|Y ( · |y ; θk−1 ) Stochastic approximation: h i Qk (θ) = Qk−1 (θ) + γk log p(y , φ(k) ; θ) − Qk−1 (θ) where (γk ) is a decreasing sequence.
step M: update the estimation of θ: θk = Argmax Qk (θ)
Marc Lavielle & le groupe MONOLIX
MONOLIX
The SAEM algorithm
When γk = 1: step E : draw the non observed data φ(k) with the conditional distribution pΦ|Y ( · |y ; θk−1 ) step M: update the estimation of θ: θk = Argmax p(y , φ(k) ; θ)
Marc Lavielle & le groupe MONOLIX
MONOLIX
The SAEM algorithm
When γk = 1/k: step E : draw the non observed data φ(k) with the conditional distribution pΦ|Y ( · |y ; θk−1 ) step M: update the estimation of θ: k 1X log p(y , φ(j) ; θ) θk = Argmax k j=1
Marc Lavielle & le groupe MONOLIX
MONOLIX
Coupling SAEM with MCMC Kuhn and Lavielle, ESAIM P&S, 2004
Let Πθ be the transition probability of an ergodic Markov Chain with limiting distribution pΦ|Y (·|y ; θ). Iteration k of the algorithm: Simulation : draw φ(k) according to the transition probability Πθk−1 (φ(k−1) , ·). Stochastic approximation: h i Qk (θ) = Qk−1 (θ) + γk log p(y , φ(k) ; θ) − Qk−1 (θ) Maximization: θk = Argmax Qk (θ) Marc Lavielle & le groupe MONOLIX
MONOLIX
Coupling SAEM with MCMC Kuhn and Lavielle, ESAIM P&S, 2004
Let Πθ be the transition probability of an ergodic Markov Chain with limiting distribution pΦ|Y (·|y ; θ). Iteration k of the algorithm: Simulation : draw φ(k) according to the transition probability Πθk−1 (φ(k−1) , ·). Stochastic approximation: h i Qk (θ) = Qk−1 (θ) + γk log p(y , φ(k) ; θ) − Qk−1 (θ) Maximization: θk = Argmax Qk (θ) Marc Lavielle & le groupe MONOLIX
MONOLIX
The SAEM algorithm Convergence of SAEM theoretically demonstrated: Delyon B., Lavielle M. and Moulines E., Convergence of a stochastic approximation version of the EM algorithm, The Annals of Statistics, vol. 27, no. 1, pp 94–128, 1999. Kuhn E., Lavielle M. Coupling a stochastic approximation version of EM with a MCMC procedure ESAIM P&S, vol.8, pp 115-131, 2004.
Intensive use of powerful and well-known algorithms: MCMC, Simulated annealing, Importance Sampling, . . . Very good practical properties: - SAEM converges successfully with complex models and provides accurate estimations of the parameters of the model, the standard errors, the likelihood function, - SAEM is few sensitive to the initialization, - SAEM is fast.
Marc Lavielle & le groupe MONOLIX
MONOLIX
The SAEM algorithm Convergence of SAEM theoretically demonstrated: Delyon B., Lavielle M. and Moulines E., Convergence of a stochastic approximation version of the EM algorithm, The Annals of Statistics, vol. 27, no. 1, pp 94–128, 1999. Kuhn E., Lavielle M. Coupling a stochastic approximation version of EM with a MCMC procedure ESAIM P&S, vol.8, pp 115-131, 2004.
Intensive use of powerful and well-known algorithms: MCMC, Simulated annealing, Importance Sampling, . . . Very good practical properties: - SAEM converges successfully with complex models and provides accurate estimations of the parameters of the model, the standard errors, the likelihood function, - SAEM is few sensitive to the initialization, - SAEM is fast.
Marc Lavielle & le groupe MONOLIX
MONOLIX
The SAEM algorithm Convergence of SAEM theoretically demonstrated: Delyon B., Lavielle M. and Moulines E., Convergence of a stochastic approximation version of the EM algorithm, The Annals of Statistics, vol. 27, no. 1, pp 94–128, 1999. Kuhn E., Lavielle M. Coupling a stochastic approximation version of EM with a MCMC procedure ESAIM P&S, vol.8, pp 115-131, 2004.
Intensive use of powerful and well-known algorithms: MCMC, Simulated annealing, Importance Sampling, . . . Very good practical properties: - SAEM converges successfully with complex models and provides accurate estimations of the parameters of the model, the standard errors, the likelihood function, - SAEM is few sensitive to the initialization, - SAEM is fast.
Marc Lavielle & le groupe MONOLIX
MONOLIX
The SAEM algorithm Convergence of SAEM theoretically demonstrated: Delyon B., Lavielle M. and Moulines E., Convergence of a stochastic approximation version of the EM algorithm, The Annals of Statistics, vol. 27, no. 1, pp 94–128, 1999. Kuhn E., Lavielle M. Coupling a stochastic approximation version of EM with a MCMC procedure ESAIM P&S, vol.8, pp 115-131, 2004.
Intensive use of powerful and well-known algorithms: MCMC, Simulated annealing, Importance Sampling, . . . Very good practical properties: - SAEM converges successfully with complex models and provides accurate estimations of the parameters of the model, the standard errors, the likelihood function, - SAEM is few sensitive to the initialization, - SAEM is fast.
Marc Lavielle & le groupe MONOLIX
MONOLIX
Some extensions Models with data below LOQ (left-censored data), Samson A., Lavielle M., Mentré F. Extension of the SAEM algorithm to left-censored data in non-linear mixed-effects model: application to HIV dynamics models, Computational Statistics and Data Analysis, vol. 51, pp. 1562–1574, 2006.
Models defined by Ordinary Differential Equations, Donnet S., Samson A. Estimation of parameters in incomplete data models defined by dynamical systems, Jour. of Stat. Planning and Inference, Vol. 137, n 9, pp. 2815-31, 2007.
Models defined by Stochastic Differential Equations, Donnet S., Samson A. Parametric inference for mixed models defined by stochastic differential equations, ESAIM P&S (to appear), 2008. Marc Lavielle & le groupe MONOLIX
MONOLIX
Some extensions REML estimation, Meza C., Jaffrézic F., Foulley J.L., REML estimation of variance parameters in nonlinear mixed effects models using the SAEM algorithm, Biometrical Journal, 49, 867-888, 2007.
Inter-occasion variability (IOV), Panhard X., Samson A. Extension of the SAEM algorithm for nonlinear models with two levels of random effects, Biostatistics, 2008 (accepted for publication).
Binary data, Meza C., Jaffrézic F., Foulley J.L., Estimation in the probit normal model for binary outcomes using the SAEM algorithm, 2008 (submitted).
Marc Lavielle & le groupe MONOLIX
MONOLIX
Marc Lavielle & le groupe MONOLIX
MONOLIX
