nips nips2000 nips2000-106 nips2000-106-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Zoubin Ghahramani, Matthew J. Beal
Abstract: Variational approximations are becoming a widespread tool for Bayesian learning of graphical models. We provide some theoretical results for the variational updates in a very general family of conjugate-exponential graphical models. We show how the belief propagation and the junction tree algorithms can be used in the inference step of variational Bayesian learning. Applying these results to the Bayesian analysis of linear-Gaussian state-space models we obtain a learning procedure that exploits the Kalman smoothing propagation, while integrating over all model parameters. We demonstrate how this can be used to infer the hidden state dimensionality of the state-space model in a variety of synthetic problems and one real high-dimensional data set. 1
[1] H. Attias. A variational Bayesian framework for graphical models. In Advances in Neural Information Processing Systems 12. MIT Press, Cambridge, MA, 2000.
[2] M.J. Beal and Z. Ghahramani. The variational Kalman smoother. Technical report, Gatsby Computational Neuroscience Unit, University College London, 2000.
[3] C.M. Bishop. Variational PCA. In Proc. Ninth ICANN, 1999.
[4] S. Friiwirth-Schnatter. Bayesian model discrimination and Bayes factors for linear Gaussian state space models. J. Royal. Stat. Soc. B , 57:237-246, 1995.
[5] Z. Ghahramani and M.J. Beal. Variational inference for Bayesian mixtures of factor analysers. In Adv. Neur. Inf. Proc. Sys. 12. MIT Press, Cambridge, MA, 2000.
[6] G.E. Hinton and D. van Camp. Keeping neural networks simple by minimizing the description length ofthe weights. In Sixth ACM Conference on Computational Learning Theory, Santa Cruz, 1993.
[7] M.1. Jordan, Z. Ghahramani, T.S. Jaakkola, and L.K Saul. An introduction to variational methods in graphical models. Machine Learning, 37:183- 233, 1999.
[8] D.J .C. MacKay. Ensemble learning for hidden Markov models. Technical report, Cavendish Laboratory, University of Cambridge, 1997.
[9] J. Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Mateo, CA, 1988.
[10] R. H. Shumway and D. S. Stoffer. An approach to time series smoothing and forecasting using the EM algorithm. J . Time Series Analysis, 3(4) :253- 264, 1982.
[11] S. Waterhouse, D.J.C. Mackay, and T. Robinson. Bayesian methods for mixtures of experts. In Adv. Neur. Inf. Proc. Sys . 7. MIT Press, 1995.