nips nips2003 nips2003-32 nips2003-32-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Tom Heskes, Onno Zoeter, Wim Wiegerinck
Abstract: We discuss the integration of the expectation-maximization (EM) algorithm for maximum likelihood learning of Bayesian networks with belief propagation algorithms for approximate inference. Specifically we propose to combine the outer-loop step of convergent belief propagation algorithms with the M-step of the EM algorithm. This then yields an approximate EM algorithm that is essentially still double loop, with the important advantage of an inner loop that is guaranteed to converge. Simulations illustrate the merits of such an approach. 1
[1] A. Dempster, N. Laird, and D. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B, 39:1-38, 1977.
[2] R. Neal and G. Hinton. A view of the EM algorithm that justifies incremental, sparse, and other variants. In M. Jordan, editor, Learning in Graphical Models, pages 355-368. Kluwer Academic Publishers, Dordrecht, 1998.
[3] K. Murphy, Y. Weiss, and M. Jordan. 'Loopy belief propagation for approximate inference: An empirical study. In Proceedings of the Fifteenth Conference on Uncertainty in Articial Intelligence, pages 467-475, San Francisco, CA, 1999. Morgan Kaufmann.
[4] J. Yedidia, W. Freeman, and Y. Weiss. Constructing free energy approximations and generalized belief propagation algorithms. Technical report, Mitsubishi Electric Research Laboratories, 2002.
[5] T. Minka. Expectation propagation for approximate Bayesian inference. In Uncertainty in Artificial Intelligence: Proceedings of the Seventeenth Conference (UAI-2001), pages 362-369, San Francisco, CA, 2001. Morgan Kaufmann Publishers.
[6] B. Frey and A. Kanna. Accumulator networks: Suitors of local probability propagation. In T. Leen, T. Dietterich, and V. Tresp, editors, Advances in Neural Information Processing Systems 13, pages 486-492. MIT Press, 2001.
[7] T. Minka and J. Lafferty. Expectation propagation for the generative aspect model. In Proceedings of UAI-2002, pages 352-359, 2002.
[8] A. Yuille. CCCP algorithms to minimize the Bethe and Kikuchi free energies: Convergent alternatives to belief propagation. Neural Computation, 14:16911722,2002.
[9] Y. Teh and M. Welling. The unified propagation and scaling algorithm. In NIPS 14, 2002.
[10] R. Salakhutdinov and S. Roweis. Adaptive overrelaxed bound optimization methods. In ICML-2003, 2003.
[11] T. Heskes, K. Albers, and B. Kappen. Approximate inference and constrained optimization. In UAI-2003, 2003.
[12] K. Murphy and Y. Weiss.. The factored frontier algorithm for approximate inference in DBNs. In UAI-2001, pages 378-385, 2001.
[13] M. Wainwright, T. Jaakkola, and A. WHIsky. Tree-reweighted belief propagation algorithms and approximate ML estimation via pseudo-moment matching. In AISTATS-2003, 2003.
[14] Y. Teh and M. Welling. On improving the efficiency of the iterative proportional fitting procedure. In AISTATS-2003, 2003.