emnlp emnlp2012 emnlp2012-107 emnlp2012-107-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Wen-tau Yih ; Geoffrey Zweig ; John Platt
Abstract: Existing vector space models typically map synonyms and antonyms to similar word vectors, and thus fail to represent antonymy. We introduce a new vector space representation where antonyms lie on opposite sides of a sphere: in the word vector space, synonyms have cosine similarities close to one, while antonyms are close to minus one. We derive this representation with the aid of a thesaurus and latent semantic analysis (LSA). Each entry in the thesaurus a word sense along with its synonyms and antonyms is treated as a “document,” and the resulting document collection is subjected to LSA. The key contribution of this work is to show how to assign signs to the entries in the co-occurrence matrix on which LSA operates, so as to induce a subspace with the desired property. – – We evaluate this procedure with the Graduate Record Examination questions of (Mohammed et al., 2008) and find that the method improves on the results of that study. Further improvements result from refining the subspace representation with discriminative training, and augmenting the training data with general newspaper text. Altogether, we improve on the best previous results by 11points absolute in F measure.
Eneko Agirre, Enrique Alfonseca, Keith Hall, Jana Kravalova, Marius Pas ¸ca, and Aitor Soroa. 2009. A study on similarity and relatedness using distributional and wordnet-based approaches. In Proceedings of HLT-NAACL, pages 19–27. Ricardo Baeza-Yates and Berthier Ribiero-Neto. 1999. Modern Information Retrieval. Addison-Wesley. J. Bellegarda. 2000. Exploiting latent semantic information in statistical language modeling. Proceedings of the IEEE, 88(8). Thorsten Brants and Alex Franz. 2006. Web 1T 5-gram Version 1. Linguistic Data Consortium. N. Coccaro and D. Jurafsky. 1998. Towards better integration of semantic predictors in statistical language modeling. In Proceedings, International Conference on Spoken Language Processing (ICSLP-98). D. A. Cruse. 1986. Lexical Semantics. Cambridge University Press. James R. Curran and Marc Moens. 2002. Improvements in automatic thesaurus extraction. In Proceedings of the ACL-02 workshop on Unsupervised lexical acquisition - Volume 9, pages 59–66. Association for Computational Linguistics. Thomas de Simone and Dimitar Kazakov. 2005. Using wordnet similarity and antonymy relations to aid document retrieval. In Recent Advances in Natural Language Processing (RANLP). S. Deerwester, S.T. Dumais, G.W. Furnas, T.K. Landauer, and R. Harshman. 1990. Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(96). E. Gabrilovich and S. Markovitch. 2007. Computing semantic relatedness using wikipedia-based explicit semantic analysis. In AAAI Conference on Artificial Intelligence (AAAI). Sanda Harabagiu, Andrew Hickl, and Finley Lacatusu. 2006. Negation, contrast and contradiction in text processing. In AAAI Conference on Artificial Intelligence (AAAI). Zelig Harris. 1954. Distributional structure. Word, 10(23): 146–162. Mario Jarmasz and Stan Szpakowicz. 2003. Rogets thesaurus and semantic similarity. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP-2003). Thomas Landauer and Susan Dumais. 1997. A solution to plato’s problem: The latent semantic analysis theory of the acquisition, induction, and representation of knowledge. Psychological Review, 104(2), pages 211– 240. T.K. Landauer and D. Laham. 1998. Learning humanlike knowledge by singular value decomposition: A progress report. In Neural Information Processing Systems (NIPS). Thomas K. Landauer, Peter W. Foltz, and Darrell Laham. 1998. An introduction to latent semantic analysis. Discourse Processes, 25, pages 259–284. T.K. Landauer. 2002. On the computational basis of learning and cognition: Arguments from lsa. Psychology of Learning and Motivation, 41:43–84. Dekang Lin, Shaojun Zhao, Lijuan Qin, and Ming Zhou. 2003. Identifying synonyms among distributionally similar words. In International Joint Conference on Artificial Intelligence (IJCAI). Dekang Lin. 1998. Automatic retrieval and clustering of similar words. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics - Volume 2, ACL ’98, pages 768– 774, Stroudsburg, PA, USA. Association for Computational Linguistics. David Milne and Ian H. Witten. 2008. An effective lowcost measure of semantic relatedness obtained from wikipedia links. In Proceedings of the AAAI 2008 Workshop on Wikipedia and Artificial Intelligence. G. Minnen, J. Carroll, and D. Pearce. 2001 . Applied morphological processing of english. Natural Language Engineering, 7(3):207–223. Saif Mohammed, Bonnie Dorr, and Graeme Hirst. 2008. Computing word pair antonymy. In Empirical Methods in Natural Language Processing (EMNLP). Saif M. Mohammed, Bonnie J. Dorr, Graeme Hirst, and Peter D. Turney. 2011. Measuring degrees of semantic opposition. Technical report, National Research Council Canada. Gregory L. Murphy and Jane M. Andrew. 1993. The conceptual basis of antonymy and synonymy in adjectives. Journal of Memory and Language, 32(3): 1–19. Jorge Nocedal and Stephen Wright. 2006. Numerical Optimization. Springer, 2nd edition. John Platt, Kristina Toutanova, and Wen-tau Yih. 2010. Translingual document representations from discriminative projections. In Proceedings of EMNLP, pages 251–261. Hoifung Poon and Pedro Domingos. 2009. Unsupervised semantic parsing. In Empirical Methods in Natural Language Processing (EMNLP). Martin F. Porter. 1980. An algorithm for suffix stripping. Program, 14(3): 130–137. 1222 Joseph Reisinger and Raymond J. Mooney. 2010. Multi- prototype vector-space models of word meaning. In Proceedings of HLT-NAACL, pages 109–1 17. Gerard Salton and Michael J. McGill. 1983. Introduction to Modern Information Retrieval. McGraw Hill. G. Salton, A. Wong, and C. S. Yang. 1975. A Vector Space Model for Automatic Indexing. Communications of the ACM, 18(1 1). D. Schwab, M. Lafourcade, and V. Prince. 2002. Antonymy and conceptual vectors. In International Conference on Computational Linguistics (COLING). Peter Turney and Michael Littman. 2005. Corpus-based learning of analogies and semantic relations. Machine Learning, 60 (1-3), pages 251–278. Peter D. Turney and Patrick Pantel. 2010. From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research, (37). Peter D. Turney, Michael L. Littman, Jeffrey Bigham, and Victor Shnayder. 2003. Combining independent modules to solve multiple-choice synonym and analogy problems. In Recent Advances in Natural Language Processing (RANLP). Peter D. Turney. 2001. Mining the web for synonyms: Pmi-ir versus lsa on toefl. In European Conference on Machine Learning (ECML). P. D. Turney. 2006. Similarity of semantic relations. Computational Linguistics, 32(3):379–416. Peter Turney. 2008. A uniform approach to analo- gies, synonyms, antonyms, and associations. In International Conference on Computational Linguistics (COLING). Lonneke van der Plas and Gosse Bouma. 2005. Syntactic contexts for finding semantically similar words. In Proceedings of the Meeting of Computational Linguistics in the Netherlands 2004 (CLIN). Lonneke van der Plas and J o¨rg Tiedemann. 2006. Finding synonyms using automatic word alignment and measures of distributional similarity. In Proceedings of the COLING/ACL on Main conference poster sessions, COLING-ACL ’06, pages 866–873. Association for Computational Linguistics. Wei Xu, Xin Liu, and Yihong Gong. 2003. Document clustering based on non-negative matrix factorization. In Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, pages 267–273, New York, NY, USA. ACM. Wen-tau Yih, Kristina Toutanova, John C. Platt, and Christopher Meek. 2011. Learning discriminative projections for text similarity measures. In Proceedings of the Fifteen Conference on Computational Natural Language Learning (CoNLL), pages 247–256, Portland, Oregon, USA.