nips nips2010 nips2010-33 nips2010-33-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Manfred Opper, Andreas Ruttor, Guido Sanguinetti
Abstract: We present a novel approach to inference in conditionally Gaussian continuous time stochastic processes, where the latent process is a Markovian jump process. We first consider the case of jump-diffusion processes, where the drift of a linear stochastic differential equation can jump at arbitrary time points. We derive partial differential equations for exact inference and present a very efficient mean field approximation. By introducing a novel lower bound on the free energy, we then generalise our approach to Gaussian processes with arbitrary covariance, such as the non-Markovian RBF covariance. We present results on both simulated and real data, showing that the approach is very accurate in capturing latent dynamics and can be useful in a number of real data modelling tasks.
[1] Cedric Archambeau, Dan Cornford, Manfred Opper, and John Shawe-Taylor. Gaussian process approximations of stochastic differential equations. Journal of Machine Learning Research Workshop and Conference Proceedings, 1(1):1–16, 2007.
[2] Neil D. Lawrence, Guido Sanguinetti, and Magnus Rattray. Modelling transcriptional regulation using Gaussian processes. In Advances in Neural Information Processing Systems 19, 2006.
[3] Uri Nodelman, Christian R. Shelton, and Daphne Koller. Continuous time Bayesian networks. In Proceedings of the Eighteenth conference on Uncertainty in Artificial Intelligence (UAI), 2002.
[4] Manfred Opper and Guido Sanguinetti. Variational inference for Markov jump processes. In Advances in Neural Information Processing Systems 20, 2007.
[5] Ido Cohn, Tal El-Hay, Nir Friedman, and Raz Kupferman. Mean field variational approximation for continuous-time Bayesian networks. In Proceedings of the twenty-fifthth conference on Uncertainty in Artificial Intelligence (UAI), 2009.
[6] Guido Sanguinetti, Andreas Ruttor, Manfred Opper, and Cedric Archambeau. Switching regulatory models of cellular stress response. Bioinformatics, 25(10):1280–1286, 2009.
[7] Mauricio Alvarez, David Luengo, and Neil D. Lawrence. Latent force models. In Proceedings of the Twelfth Interhantional Conference on Artificial Intelligence and Statistics (AISTATS), 2009.
[8] Carl E. Rasmussen and Christopher K.I. Williams. Gaussian Processes for Machine Learning. MIT press, 2005.
[9] C. W. Gardiner. Handbook of Stochastic Methods. Springer, Berlin, second edition, 1996.
[10] Andreas Ruttor and Manfred Opper. Efficient statistical inference for stochastic reaction processes. Phys. Rev. Lett., 103(23), 2009.
[11] Cedric Archambeau and Manfred Opper. Approximate inference for continuous-time Markov processes. In David Barber, Taylan Cemgil, and Silvia Chiappa, editors, Inference and Learning in Dynamic Models. Cambridge University Press, 2010.
[12] M. A. Lifshits. Gaussian Random Functions. Kluwer, Dordrecht, second edition, 1995.
[13] Michael I. Jordan, Zoubin Ghahramani, Tommi S. Jaakkola, and Lawrence K. Saul. An introduction to variational methods for graphical models. Machine Learning, 37:183–233, 1999.
[14] Manfred Opper and Guido Sanguinetti. Learning combinatorial transcriptional dynamics from gene expression data. Bioinformatics, 26(13):1623–1629, 2010.
[15] David Barber. Expectation correction for smoothing in switching linear Gaussian state space models. Journal of Machine Learning Research, 7:2515–2540, 2006.
[16] James C. Liao, Riccardo Boscolo, Young-Lyeol Yang, Linh My Tran, Chiara Sabatti, and Vwani P. Roychowdhury. Network component analysis: Reconstruction of regulatory signals in biological systems. Proceedings of the National Academy of Sciences USA, 100(26):15522– 15527, 2003.
[17] Martino Barenco, Daniela Tomescu, David Brewer, Robin Callard, Jaroslav Stark, and Michael Hubank. Ranked prediction of p53 targets using hidden variable dynamical modelling. Genome Biology, 7(3), 2006.
[18] G¨ rol M. Su¨ l, Jordi Garcia-Ojalvo, Louisa M. Liberman, and Michael B. Elowitz. An exu e citable gene regulatory circuit induces transient cellular differentiation. Nature, 440:545–50, 2006. 9