emnlp emnlp2013 emnlp2013-137 emnlp2013-137-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Kai-Wei Chang ; Wen-tau Yih ; Christopher Meek
Abstract: We present Multi-Relational Latent Semantic Analysis (MRLSA) which generalizes Latent Semantic Analysis (LSA). MRLSA provides an elegant approach to combining multiple relations between words by constructing a 3-way tensor. Similar to LSA, a lowrank approximation of the tensor is derived using a tensor decomposition. Each word in the vocabulary is thus represented by a vector in the latent semantic space and each relation is captured by a latent square matrix. The degree of two words having a specific relation can then be measured through simple linear algebraic operations. We demonstrate that by integrating multiple relations from both homogeneous and heterogeneous information sources, MRLSA achieves state- of-the-art performance on existing benchmark datasets for two relations, antonymy and is-a.
E. Agirre, E. Alfonseca, K. Hall, J. Kravalova, M. Pas ¸ca and A. Soroa. 2009. A study on similarity and relatedness using distributional and WordNet-based approaches. In NAACL ’09, pages 19–27. Brett W. Bader, Tamara G. Kolda, et al. 2012. Matlab tensor toolbox version 2.5. Available online, January. David M. Blei, Andrew Y. Ng, Michael I. Jordan, and John Lafferty. 2003. Latent dirichlet allocation. Journal of Machine Learning Research, 3:993–1022. Jordan L Boyd-Graber, David M Blei, and Xiaojin Zhu. 2007. A topic model for word sense disambiguation. In EMNLP-CoNLL, pages 1024–1033. Shay B. Cohen, Giorgio Satta, and Michael Collins. 2013. Approximate PCFG parsing using tensor decomposition. In NAACL-HLT 2013, pages 487–496. S. Deerwester, S. Dumais, G. Furnas, T. Landauer, and R. Harshman. 1990. Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(96). S. Dumais, T. Letsche, M. Littman, and T. Landauer. 1997. Automatic cross-language retrieval using latent semantic indexing. In AAAI-97 Spring Symposium Series: Cross-Language Text and Speech Retrieval. Weiwei Guo and Mona Diab. 2012. Modeling sentences in the latent space. In ACL 2012, pages 864–872. Weiwei Guo and Mona Diab. 2013. Improving lexical semantics for sentential semantics: Modeling selectional preference and similar words in a latent variable model. In NAACL-HLT 2013, pages 739–745. Thomas Hofmann. 1999. Probabilistic latent semantic analysis. In Proceedings of Uncertainty in Artificial Intelligence, pages 289–296. D. Jurgens, S. Mohammad, P. Turney, and K. Holyoak. 2012. SemEval-2012 Task 2: Measuring degrees of relational similarity. In Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012), pages 356–364. Tamara G. Kolda and Brett W. Bader. 2009. Tensor decompositions and applications. SIAM Review, 5 1(3):455–500, September. Tamara G. Kolda and Jimeng Sun. 2008. Scalable tensor decompositions for multi-aspect data mining. In ICDM 2008, pages 363–372. T. Landauer and D. Laham. 1998. Learning humanlike knowledge by singular value decomposition: A progress report. In NIPS 1998. T. Landauer. 2002. On the computational basis of learning and cognition: Arguments from lsa. Psychology of Learning and Motivation, 41:43–84. Jordan J. Louviere and G. G. Woodworth. 1991. Bestworst scaling: A model for the largest difference judgments. Technical report, University of Alberta. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013. Linguistic regularities in continuous space word representations. In NAACL-HLT 2013. Saif Mohammad, Bonnie Dorr, and Graeme Hirst. 2008. Computing word pair antonymy. In Empirical Methods in Natural Language Processing (EMNLP). John Platt, Kristina Toutanova, and Wen-tau Yih. 2010. Translingual document representations from discriminative projections. In Proceedings of EMNLP, pages 251–261. Xipeng Qiu, Le Tian, and Xuanjing Huang. 2013. Latent semantic tensor indexing for community-based question answering. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 434–439, Sofia, Bulgaria, August. Association for Computational Linguistics. Joseph Reisinger and Raymond J. Mooney. 2010. Multiprototype vector-space models of word meaning. In Proceedings of HLT-NAACL, pages 109–1 17. Sebastian Riedel, Limin Yao, Andrew McCallum, and Benjamin M. Marlin. 2013. Relation extraction with matrix factorization and universal schemas. In NAACL-HLT 2013, pages 74–84. Bryan Rink and Sanda Harabagiu. 2012. UTD: Determining relational similarity using lexical patterns. In Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012), pages 413–418, Montr ´eal, Canada, 7-8 June. Association for Computational Linguistics. G. Salton, A. Wong, and C. S. Yang. 1975. A Vector Space Model for Automatic Indexing. Communications of the ACM, 18(1 1). Richard Socher, Cliff Chiung-Yu Lin, Andrew Y. Ng, and Christopher D. Manning. 2011. Parsing natural scenes and natural language with recursive neural networks. In ICML ’11. 1612 Richard Socher, John Bauer, Christopher D. Manning, and Andrew Y. Ng. 2013. Parsing with compositional vector grammars. In Annual Meeting of the Association for Computational Linguistics (ACL). Ledyard R Tucker. 1966. Some mathematical notes on three-mode factor analysis. Psychometrika, 31(3):279–31 1. Peter D. Turney and Patrick Pantel. 2010. From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research, 37(1): 141– 188. P. D. Turney. 2006. Similarity of semantic relations. Computational Linguistics, 32(3):379–416. Peter Turney. 2008. A uniform approach to analogies, synonyms, antonyms, and associations. In International Conference on Computational Linguistics (COLING). Tim Van de Cruys, Thierry Poibeau, and Anna Korhonen. 2013. A tensor-based factorization model of semantic compositionality. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1142–1 15 1, Atlanta, Georgia, June. Association for Computational Linguistics. Wei Xu, Xin Liu, and Yihong Gong. 2003. Document clustering based on non-negative matrix factorization. In Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, pages 267–273, New York, NY, USA. ACM. Wen-tau Yih and Vahed Qazvinian. 2012. Measur- ing word relatedness using heterogeneous vector space models. In Proceedings of NAACL-HLT, pages 616– 620, Montr ´eal, Canada, June. Wen-tau Yih, Kristina Toutanova, John C. Platt, and Christopher Meek. 2011. Learning discriminative projections for text similarity measures. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning, pages 247–256, Portland, Oregon, USA, June. Association for Computational Linguistics. Wen-tau Yih, Geoffrey Zweig, and John Platt. 2012. Polarity inducing latent semantic analysis. In Proceedings of NAACL-HLT, pages 1212–1222, Jeju Island, Korea, July. Alisa Zhila, Wen-tau Yih, Christopher Meek, Geoffrey Zweig, and Tomas Mikolov. 2013. Combining heterogeneous models for measuring relational similarity. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1000–1009, Atlanta, Georgia, June. Association for Computational Linguistics.