nips nips2009 nips2009-215 nips2009-215-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Pierre-arnaud Coquelin, Romain Deguest, Rémi Munos
Abstract: This paper considers a sensitivity analysis in Hidden Markov Models with continuous state and observation spaces. We propose an Infinitesimal Perturbation Analysis (IPA) on the filtering distribution with respect to some parameters of the model. We describe a methodology for using any algorithm that estimates the filtering density, such as Sequential Monte Carlo methods, to design an algorithm that estimates its gradient. The resulting IPA estimator is proven to be asymptotically unbiased, consistent and has computational complexity linear in the number of particles. We consider an application of this analysis to the problem of identifying unknown parameters of the model given a sequence of observations. We derive an IPA estimator for the gradient of the log-likelihood, which may be used in a gradient method for the purpose of likelihood maximization. We illustrate the method with several numerical experiments.
[CDM08] [CGN01] [CM05] [Del04] [DFG01] [DM01] [DM08] P.A. Coquelin, R. Deguest, and R. Munos. Particle filter-based policy gradient in POMDPs. In Neural Information Processing Systems, 2008. F. Cérou, F. Le Gland, and N. J. Newton. Stochastic particle methods for linear tangent filtering equations. In J.-L. Menaldi, E. Rofman, and A. Sulem, editors, Optimal Control and PDE’s - Innovations and Applications, in honor of Alain Bensoussan’s 60th anniversary, pages 231–240. IOS Press, 2001. O. Cappé and E. Moulines. On the use of particle filtering for maximum likelihood parameter estimation. European Signal Processing Conference, 2005. P. Del Moral. Feynman-Kac Formulae, Genealogical and Interacting Particle Systems with Applications. Springer, 2004. A. Doucet, N. De Freitas, and N. Gordon. Sequential Monte Carlo Methods in Practice. Springer, 2001. R. Douc and C. Matias. Asymptotics of the maximum likelihood estimator for general hidden markov models. Bernouilli, 7:381–420, 2001. R. Douc and E. Moulines. Limit theorems for weighted samples with applications to sequential monte carlo methods. Annals of Statistics, 36:5:2344–2376, 2008. [DMDP07] P. Del Moral, A. Doucet, and GW Peters. Sharp Propagation of Chaos Estimates for Feynman–Kac Particle Models. SIAM Theory of Probability and its Applications, 51 (3):459–485, 2007. [DT03] A. Doucet and V.B. Tadic. Parameter estimation in general state-space models using particle methods. Ann. Inst. Stat. Math, 2003. [FLM03] J. Fichoud, F. LeGland, and L. Mevel. Particle-based methods for parameter estimation and tracking : numerical experiments. Technical Report 1604, IRISA, 2003. [Gla91] [GSS93] [Kit96] [KY97] [LM00] [ME07] [Pap07] [PDS05] [Pfl96] [Poy06] [Sto02] P. Glasserman. Gradient estimation via perturbation analysis. Kluwer, 1991. N. Gordon, D. Salmond, and A. F. M. Smith. Novel approach to nonlinear and nongaussian bayesian state estimation. In Proceedings IEE-F, volume 140, pages 107–113, 1993. G. Kitagawa. Monte-Carlo filter and smoother for non-Gaussian nonlinear state space models. J. Comput. Graph. Stat., 5:1–25, 1996. H. J. Kushner and G. Yin. Stochastic Approximation Algorithms and Applications. Springer-Verlag, Berlin and New York, 1997. F. LeGland and L. Mevel. Exponential forgetting and geometric ergodicity in hidden markov models. mathematic and control sugnal and systems, 13:63–93, 2000. R. Mamon and R.J. Elliott. Hidden markov models in finance. International Series in Operations Research and Management Science, 104, 2007. A. Papavasiliou. A uniformly convergent adaptive particle filter. Journal of Applied Probability, 42 (4):1053–1068, 2007. G. Poyadjis, A. Doucet, and S.S. Singh. Particle methods for optimal filter derivative: Application to parameter estimation. In IEEE ICASSP, 2005. G. Pflug. Optimization of Stochastic Models: The Interface Between Simulation and Optimization. Kluwer Academic Publishers, 1996. G. Poyiadjis. Particle Method for Parameter Estimation in General State Space Models. PhD thesis, University of Cambridge, 2006. G. Storvik. Particle filters for state-space models with the presence of unknown static parameters. IEEE Transactions on Signal Processing, 50:281–289, 2002. 9