nips nips2002 nips2002-125 nips2002-125-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Jaz Kandola, Nello Cristianini, John S. Shawe-taylor
Abstract: The standard representation of text documents as bags of words suffers from well known limitations, mostly due to its inability to exploit semantic similarity between terms. Attempts to incorporate some notion of term similarity include latent semantic indexing [8], the use of semantic networks [9], and probabilistic methods [5]. In this paper we propose two methods for inferring such similarity from a corpus. The first one defines word-similarity based on document-similarity and viceversa, giving rise to a system of equations whose equilibrium point we use to obtain a semantic similarity measure. The second method models semantic relations by means of a diffusion process on a graph defined by lexicon and co-occurrence information. Both approaches produce valid kernel functions parametrised by a real number. The paper shows how the alignment measure can be used to successfully perform model selection over this parameter. Combined with the use of support vector machines we obtain positive results. 1
[1] N. Cristianini and J. Shawe-Taylor. An Introduction to Support Vector Machines. Cambridge University Press, Cambridge, UK , 2000.
[2] Nello Cristianini, John Shawe-Taylor, and Jaz Kandola. On kernel target alignment. In Proceedings of the Neural Information Processing Systems, NIPS '01, 2002.
[3] Nello Cristianini, John Shawe-Taylor, and Huma Lodhi. Latent semantic kernels. Journal of Intelligent Information Systems, 18(2):127-152,2002.
[4] R. Ferrer and R.V. Sole. The small world of human language. Proceedings of the Royal Society of London Series B - Biological Sciences, pages 2261- 2265 , 200l.
[5] Thomas Hofmann. Probabilistic latent semantic indexing. In Research and Development in Information Retrieval, pages 50-57, 1999.
[6] T. Joachims. Text categorization with support vector machines. In Proceedings of European Conference on Machine Learning (ECML) , 1998.
[7] R.I. Kondor and J. Lafferty. Diffusion kernels on graphs and other discrete structures. In Proceedings of Intenational Conference on Machine Learning (ICML 2002), 2002.
[8] Todd A. Letsche and Michael W. Berry. Large-scale information retrieval with latent semantic indexing. Information Sciences, 100(1-4):105- 137,1997.
[9] G. Siolas and F. d 'Alch Buc. Support vector machines based on a semantic kernel for text categorization. In IEEE-IJCNN 2000), 2000.