nips nips2013 nips2013-161 nips2013-161-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Andreas Stuhlmüller, Jacob Taylor, Noah Goodman
Abstract: We describe a class of algorithms for amortized inference in Bayesian networks. In this setting, we invest computation upfront to support rapid online inference for a wide range of queries. Our approach is based on learning an inverse factorization of a model’s joint distribution: a factorization that turns observations into root nodes. Our algorithms accumulate information to estimate the local conditional distributions that constitute such a factorization. These stochastic inverses can be used to invert each of the computation steps leading to an observation, sampling backwards in order to quickly find a likely explanation. We show that estimated inverses converge asymptotically in number of (prior or posterior) training samples. To make use of inverses before convergence, we describe the Inverse MCMC algorithm, which uses stochastic inverses to make block proposals for a Metropolis-Hastings sampler. We explore the efficiency of this sampler for a variety of parameter regimes and Bayes nets. 1
J. Cheng and M. Druzdzel. AIS-BN: An adaptive importance sampling algorithm for evidential reasoning in large bayesian networks. Journal of Artificial Intelligence Research, 2000. H. Haario, M. Laine, A. Mira, and E. Saksman. DRAM: efficient adaptive MCMC. Statistics and Computing, 16(4):339–354, 2006. L. D. Hernandez, S. Moral, and A. Salmeron. A Monte Carlo algorithm for probabilistic propagation in belief networks based on importance sampling and stratified simulation techniques. International Journal of Approximate Reasoning, 18(1):53–91, 1998. B. K. Horn. Understanding image intensities. Artificial intelligence, 8(2):201–231, 1977. R. Mateescu, K. Kask, V. Gogate, and R. Dechter. Join-graph propagation algorithms. Journal of Artificial Intelligence Research, 37(1):279–328, 2010. Q. Morris. Recognition networks for approximate inference in BN20 networks. Morgan Kaufmann Publishers Inc., Aug. 2001. L. E. Ortiz and L. P. Kaelbling. Adaptive importance sampling for estimation in structured domains. In Proc. of the 16th Ann. Conf. on Uncertainty in A.I. (UAI-00), pages 446–454. Morgan Kaufmann Publishers, 2000. M. Plummer et al. Jags: A program for analysis of bayesian graphical models using gibbs sampling. URL http://citeseer. ist. psu. edu/plummer03jags. html, 2003. G. Roberts and J. Rosenthal. Examples of adaptive MCMC. Journal of Computational and Graphical Statistics, 18(2):349–367, 2009. A. Salmeron, A. Cano, and S. Moral. Importance sampling in Bayesian networks using probability trees. Computational Statistics and Data Analysis, 34(4):387–413, Oct. 2000. A. N. Sanborn, V. K. Mansinghka, and T. L. Griffiths. Reconciling intuitive physics and Newtonian mechanics for colliding objects. Psychological Review, 120(2):411, Apr. 2013. R. D. Shachter and M. A. Peot. Simulation approaches to general probabilistic inference on belief networks. In Proc. of the 5th Ann. Conf. on Uncertainty in A.I. (UAI-89), pages 311–318, New York, NY, 1989. Elsevier Science. K. Watanabe and S. Shimojo. When sound affects vision: effects of auditory grouping on visual motion perception. Psychological Science, 12(2):109–116, 2001. D. Wingate and T. Weber. Automated variational inference in probabilistic programming. arXiv preprint arXiv:1301.1299, 2013. H. Yu and R. A. Van Engelen. Refractor importance sampling. arXiv preprint arXiv:1206.3295, 2012. C. Yuan and M. J. Druzdzel. Importance sampling in Bayesian networks: An influence-based approximation strategy for importance functions. arXiv preprint arXiv:1207.1422, 2012. 9