nips nips2013 nips2013-281 nips2013-281-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Le Song, Bo Dai
Abstract: Kernel embedding of distributions has led to many recent advances in machine learning. However, latent and low rank structures prevalent in real world distributions have rarely been taken into account in this setting. Furthermore, no prior work in kernel embedding literature has addressed the issue of robust embedding when the latent and low rank information are misspecified. In this paper, we propose a hierarchical low rank decomposition of kernels embeddings which can exploit such low rank structures in data while being robust to model misspecification. We also illustrate with empirical evidence that the estimated low rank embeddings lead to improved performance in density estimation. 1
[1] A. J. Smola, A. Gretton, L. Song, and B. Sch¨ lkopf. A Hilbert space embedding for diso tributions. In Proceedings of the International Conference on Algorithmic Learning Theory, volume 4754, pages 13–31. Springer, 2007.
[2] B. Sriperumbudur, A. Gretton, K. Fukumizu, G. Lanckriet, and B. Sch¨ lkopf. Injective Hilbert o space embeddings of probability measures. In Proc. Annual Conf. Computational Learning Theory, pages 111–122, 2008.
[3] A. Gretton, K. Fukumizu, C.-H. Teo, L. Song, B. Sch¨ lkopf, and A. J. Smola. A kernel o statistical test of independence. In Advances in Neural Information Processing Systems 20, pages 585–592, Cambridge, MA, 2008. MIT Press.
[4] L. Song, A. Gretton, D. Bickson, Y. Low, and C. Guestrin. Kernel belief propagation. In Proc. Intl. Conference on Artificial Intelligence and Statistics, volume 10 of JMLR workshop and conference proceedings, 2011.
[5] L. Song, B. Boots, S. Siddiqi, G. Gordon, and A. J. Smola. Hilbert space embeddings of hidden markov models. In International Conference on Machine Learning, 2010.
[6] L. Song, A. Parikh, and E.P. Xing. Kernel embeddings of latent tree graphical models. In Advances in Neural Information Processing Systems, volume 25, 2011.
[7] L. Song, M. Ishteva, H. Park, A. Parikh, and E. Xing. Hierarchical tensor decomposition of latent tree graphical models. In International Conference on Machine Learning (ICML), 2013.
[8] A. Berlinet and C. Thomas-Agnan. Reproducing Kernel Hilbert Spaces in Probability and Statistics. Kluwer, 2004.
[9] L. Song, J. Huang, A. J. Smola, and K. Fukumizu. Hilbert space embeddings of conditional distributions. In Proceedings of the International Conference on Machine Learning, 2009.
[10] Tamara. G. Kolda and Brett W. Bader. Tensor decompositions and applications. SIAM Review, 51(3):455–500, 2009.
[11] L. Grasedyck. Hierarchical singular value decomposition of tensors. SIAM Journal on Matrix Analysis and Applications, 31(4):2029–2054, 2010.
[12] I Oseledets. Tensor-train decomposition. SIAM Journal on Scientific Computing, 33(5):2295– 2317, 2011.
[13] L. Rosasco, M. Belkin, and E.D. Vito. On learning with integral operators. Journal of Machine Learning Research, 11:905–934, 2010. 9