nips nips2001 nips2001-178 nips2001-178-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Lehel Csató, Manfred Opper, Ole Winther
Abstract: The adaptive TAP Gibbs free energy for a general densely connected probabilistic model with quadratic interactions and arbritary single site constraints is derived. We show how a specific sequential minimization of the free energy leads to a generalization of Minka’s expectation propagation. Lastly, we derive a sparse representation version of the sequential algorithm. The usefulness of the approach is demonstrated on classification and density estimation with Gaussian processes and on an independent component analysis problem.
[1] T.P. Minka. Expectation propagation for approximate Bayesian inference. PhD thesis, Dep. of Electrical Eng. and Comp. Sci.; MIT, 2000.
[2] J. S. Yedidia, W. T. Freeman and Y. Weiss, Generalized Belief Propagation, to appear in Advances in Neural Information Processing Systems (NIPS’2000), MIT Press (2001).
[3] M. Opper and O. Winther, Tractable approximations for probabilistic models: The adaptive TAP approach, Phys. Rev. Lett. 86, 3695 (2001).
[4] L. Csat´ and M. Opper. Sparse Gaussian Processes. Neural Computation accepted (2001). o
[5] P.A.d.F.R. Højen-Sørensen, O. Winther, and L. K. Hansen, Mean Field Approaches to Independent Component Analysis, Neural Computation accepted (2001). Available from http://www.cbs.dtu.dk/winther/
[6] T. Plefka, Convergence condition of the TAP equations for the infinite-ranged Ising spin glass model, J. Phys. A 15, 1971 (1982).
[7] T. Tanaka, Mean-Field Theory of Boltzmann Machine Learning, Phys. Rev. E 58, 2302(1998).
[8] G. Parisi and M. Potters, Mean-Field Equations for Spin Models with Orthogonal Interaction Matrices, J. Phys. A (Math. Gen.) 28, 5267 (1995).
[9] L. Csat´ , E. Fokou´ , M. Opper, B. Schottky, and O. Winther. Efficient approaches to Gaussian o e process classification. In Advances in Neural Information Processing Systems, volume 12, (2000).
[10] D.M. Schmidt. Continuous probability distributions from finite data. arXiv:physics/9808005 (1998)