emnlp emnlp2010 emnlp2010-4 emnlp2010-4-reference knowledge-graph by maker-knowledge-mining

4 emnlp-2010-A Game-Theoretic Approach to Generating Spatial Descriptions


Source: pdf

Author: Dave Golland ; Percy Liang ; Dan Klein

Abstract: Language is sensitive to both semantic and pragmatic effects. To capture both effects, we model language use as a cooperative game between two players: a speaker, who generates an utterance, and a listener, who responds with an action. Specifically, we consider the task of generating spatial references to objects, wherein the listener must accurately identify an object described by the speaker. We show that a speaker model that acts optimally with respect to an explicit, embedded listener model substantially outperforms one that is trained to directly generate spatial descriptions.


reference text

J. L. Austin. 1962. How to do Things with Words: The William James Lectures delivered at Harvard University in 1955. Oxford, Clarendon, UK. S. Branavan, H. Chen, L. S. Zettlemoyer, and R. Barzilay. 2009. Reinforcement learning for mapping instructions to actions. InAssociationfor Computational Linguistics andInternational Joint Conference on Natural Language Processing (ACL-IJCNLP), Singapore. Association for Computational Linguistics. D. L. Chen and R. J. Mooney. 2008. Learning to sportscast: A test of grounded language acquisition. In International Conference on Machine Learning (ICML), pages 128–135. Omnipress. David DeVault and Matthew Stone. 2007. Managing ambiguities across utterances in dialogue. J. Eisenstein, J. Clarke, D. Goldwasser, and D. Roth. 2009. Reading to learn: Constructing features from semantic abstracts. In Empirical Methods in Natural Language Processing (EMNLP), Singapore. J. Feldman and S. Narayanan. 2004. Embodied meaning in a neural theory of language. Brain and Language, 89:385–392. M. Fleischman and D. Roy. 2007. Representing intentions in a cognitive model of language acquisition: Effects of phrase structure on situated verb learning. In Association for the Advancement of Artificial Intelligence (AAAI), Cambridge, MA. MIT Press. M. C. Frank, N. D. Goodman, and J. B. Tenenbaum. 2009. Using speakers’ referential intentions to model early cross-situational word learning. Psychological Science, 20(5):578–585. Peter Gorniak and Deb Roy. 2004. Grounded semantic composition for visual scenes. In Journal of Artificial Intelligence Research, volume 21, pages 429–470. H. P. Grice. 1975. Syntax and Semantics; Logic and Conversation. 3:Speech Acts:41–58. G. Hinton. 1999. Products of experts. In International Conference on Artificial Neural Networks (ICANN). G. J ¨ager. 2008. Game theory in semantics and pragmatics. Technical report, University of T ¨ubingen. T. Kollar, S. Tellex, D. Roy, and N. Roy. 2010. Toward understanding natural language directions. In HumanRobot Interaction, pages 259–266. Barbara Landau and Ray Jackendoff. 1993. ”what” and ”where” in spatial language and spatial cognition. Behavioral and Brain Sciences, 16(2spatial prepositions analysis, cross linguistic conceptual similarities; comments/response):217–238. P. Liang, M. I. Jordan, and D. Klein. 2009. Learning semantic correspondences with less supervision. In Association for Computational Linguistics and International Joint Conference on Natural Language Processing (ACL-IJCNLP), Singapore. Association for Computational Linguistics. 419 S. T. Piantadosi, N. D. Goodman, B. A. Ellis, and J. B. Tenenbaum. 2008. A Bayesian model of the acquisition of compositional semantics. In Proceedings of the Thirtieth Annual Conference of the Cognitive Science Society. T Regier and LA Carlson. 2001 . Journal of experimental psychology. general; grounding spatial language in perception: an empirical and computational investigation. 130(2):273–298. Stefanie Tellex and Deb Roy. 2009. Grounding spatial prepositions for video search. In ICMI. Y. W. Wong and R. J. Mooney. 2007. Learning synchronous grammars for semantic parsing with lambda calculus. In Association for Computational Linguistics (ACL), pages 960–967, Prague, Czech Republic. Association for Computational Linguistics. C. Yu and D. H. Ballard. 2004. On the integration of grounding language and learning objects. In Association for the Advancement of Artificial Intelligence (AAAI), pages 488–493, Cambridge, MA. MIT Press. L. S. Zettlemoyer and M. Collins. 2005. Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. In Uncertainty in Artificial Intelligence (UAI), pages 658–666.