nips nips2002 nips2002-113 nips2002-113-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Guy Lebanon, John D. Lafferty
Abstract: A new family of kernels for statistical learning is introduced that exploits the geometric structure of statistical models. Based on the heat equation on the Riemannian manifold defined by the Fisher information metric, information diffusion kernels generalize the Gaussian kernel of Euclidean space, and provide a natural way of combining generative statistical modeling with non-parametric discriminative learning. As a special case, the kernels give a new approach to applying kernel-based learning algorithms to discrete data. Bounds on covering numbers for the new kernels are proved using spectral theory in differential geometry, and experimental results are presented for text classification.
[1] S. Amari and H. Nagaoka. Methods of Information Geometry, volume 191 of Translations of Mathematical Monographs. American Mathematical Society, 2000.
[2] A. Grigor’yan and M. Noguchi. The heat kernel on hyperbolic space. Bulletin of the London Mathematical Society, 30:643–650, 1998.
[3] Y. Guo, P. L. Bartlett, J. Shawe-Taylor, and R. C. Williamson. Covering numbers for support vector machines. IEEE Trans. Information Theory, 48(1), January 2002.
[4] T. S. Jaakkola and D. Haussler. Exploiting generative models in discriminative classifiers. In Advances in Neural Information Processing Systems, volume 11, 1998.
[5] T. Joachims, N. Cristianini, and J. Shawe-Taylor. Composite kernels for hypertext categorisation. In Proceedings of the International Conference on Machine Learning (ICML), 2001.
[6] R. E. Kass and P. W. Vos. Geometrical Foundations of Asymptotic Inference. Wiley Series in Probability and Statistics. John Wiley & Sons, 1997.
[7] R. I. Kondor and J. Lafferty. Diffusion kernels on graphs and other discrete input spaces. In Proceedings of the International Conference on Machine Learning (ICML), 2002.
[8] P. Li and S.-T. Yau. Estimates of eigenvalues of a compact Riemannian manifold. In Geometry of the Laplace Operator, volume 36 of Proceedings of Symposia in Pure Mathematics, pages 205–239, 1980.
[9] R. Schoen and S.-T. Yau. Lectures on Differential Geometry, volume 1 of Conference Proceedings and Lecture Notes in Geometry and Topology. International Press, 1994.