nips nips2012 nips2012-248 nips2012-248-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Gal Elidan, Cobi Cario
Abstract: The empirical success of the belief propagation approximate inference algorithm has inspired numerous theoretical and algorithmic advances. Yet, for continuous non-Gaussian domains performing belief propagation remains a challenging task: recent innovations such as nonparametric or kernel belief propagation, while useful, come with a substantial computational cost and offer little theoretical guarantees, even for tree structured models. In this work we present Nonparanormal BP for performing efficient inference on distributions parameterized by a Gaussian copulas network and any univariate marginals. For tree structured networks, our approach is guaranteed to be exact for this powerful class of non-Gaussian models. Importantly, the method is as efficient as standard Gaussian BP, and its convergence properties do not depend on the complexity of the univariate marginals, even when a nonparametric representation is used. 1
D. Bickson. Gaussian Belief Propagation: Theory and Application. PhD thesis, The Hebrew University of Jerusalem, Jerusalem, Israel, 2008. A. Bowman and A. Azzalini. Applied Smoothing Techniques for Data Analysis. Oxford Press, 1997. C. K. Chow and C. N. Liu. Approximating discrete probability distributions with dependence trees. IEEE Trans. on Info. Theory, 14:462–467, 1968. P. Cortez, A. Cerdeira, F. Almeida, T. Matos, and J. Reis. Modeling wine preferences by data mining from physicochemical properties. Decision Support Systems, 47(4):547–553, 2009. G. Elidan. Copula bayesian networks. In Neural Information Processing Systems (NIPS), 2010. P. Embrechts, F. Lindskog, and A. McNeil. Modeling dependence with copulas and applications to risk management. Handbook of Heavy Tailed Distributions in Finance, 2003. A. Globerson and T. Jaakkola. Fixing max-product: Convergent message passing algorithms for map lprelaxations. In Neural Information Processing Systems (NIPS), 2007. T. Heskes. On the uniqueness of loopy belief propagation fixed points. Neural Comp, 16:2379–2413, 2004. A. Ihler and D. McAllester. Particle belief propagation. In Conf on AI and Statistics (AISTATS), 2009. S. Kirshner. Learning with tree-averaged densities and distributions. In Neural Info Proc Systems (NIPS), 2007. D. Koller and N. Friedman. Probabilistic Graphical Models: Principles and Techniques. The MIT Press, 2009. D. Kurowicka and R. M. Cooke. Distribution-free continuous bayesian belief nets. In Selected papers based on the presentation at the international conference on mathematical methods in reliability (MMR), 2005. H. Liu, J. Lafferty, and L. Wasserman. The nonparanormal: Semiparametric estimation of high dimensional undirected graphs. Journal of Machine Learning Research, 2010. D. Malioutov, J. Johnson, and A. Willsky. Walk-sums and belief propagation in gaussian graphical models. Journal of Machine Learning Research, 7:2031–2064, 2006. R. Marion, A. Regev, E. Segal, Y. Barash, D. Koller, N. Friedman, and E. O’Shea. Sfp1 is a stress- and nutrientsensitive regulator of ribosomal protein gene expression. Proc Natl Acad Sci U S A, 101(40):14315–22, 2004. R. McEliece, D. McKay, and J. Cheng. Turbo decoding as an instance of pearl’s belief propagation algorithm. IEEE Journal on Selected Areas in Communication, 16:140–152, 1998. T. P. Minka. Expectation propagation for approximate Bayesian inference. In Proc. Conference on Uncertainty in Artificial Intelligence (UAI), pages 362–369, 2001. J. Mooij and B. Kappen. Sufficient conditions for convergence of loopy belief propagation. In Proc. Conference on Uncertainty in Artificial Intelligence (UAI), 2005. K. Murphy and Y. Weiss. Loopy belief propagation for approximate inference: An empirical study. In Proc. Conference on Uncertainty in Artificial Intelligence (UAI), pages 467–475, 1999. R. Nelsen. An Introduction to Copulas. Springer, 2007. J. Pearl. Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, 1988. C. P. Robert and G. Casella. Monte Carlo Statistical Methods (Springer Texts in Statistics. Springer, 2005. P. Rusmevichientong and B. Van Roy. An analysis of belief propagation on the turbo decoding graph with gaussian densities. IEEE Transactions on Information Theory, 47:745–765, 2000. G. Schwarz. Estimating the dimension of a model. Annals of Statistics, 6:461–464, 1978. A. Sklar. Fonctions de repartition a n dimensions et leurs marges. Publications de l’Institut de Statistique de L’Universite de Paris, 8:229–231, 1959. L. Song, A. Gretton, D. Bickson, Y. Low, and C. Guestrin. Kernel belief propagation. In Conference on Artificial Intelligence and Statistics (AIStats), 2011. E.B. Sudderth, A.T. Ihler, M. Isard, W.T. Freeman, and A.S. Willsky. Nonparametric belief propagation. Communications of the ACM, 53(10):95–103, 2010a. Erik Sudderth, Alexander Ihler, Michael Isard, William Freeman, and Alan Willsky. Nonparametric belief propagation. Communications of the ACM, 53(10):95–103, October 2010b. Y. Weiss and W. Freeman. Correctness of belief propagation in gaussian graphical models of arbitrary topology. Neural Computation, 13:2173–2200, 2001. W. Wiegerinck and T. Heskes. Fractional belief propagation. In Neural Information Processing Systems 15, Cambridge, Mass., 2003. MIT Press. A. Wilson and Z. Ghahramani. Copula processes. In Neural Information Processing Systems (NIPS), 2010. J. S. Yedidia, W. T. Freeman, and Y. Weiss. Generalized belief propagation. In Neural Information Processing Systems 13, pages 689–695, Cambridge, Mass., 2001. MIT Press. L. Zhang and V. Singh. Trivariate flood frequency analysis using the Gumbel-Hougaard copula. Journal of Hydrologic Engineering, 12, 2007. 9