emnlp emnlp2011 emnlp2011-38 emnlp2011-38-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Alan Ritter ; Colin Cherry ; William B. Dolan
Abstract: Ottawa, Ontario, K1A 0R6 Co l . Cherry@ nrc-cnrc . gc . ca in Redmond, WA 98052 bi l ldol @mi cro so ft . com large corpus of status-response pairs found on Twitter to create a system that responds to Twitter status We present a data-driven approach to generating responses to Twitter status posts, based on phrase-based Statistical Machine Translation. We find that mapping conversational stimuli onto responses is more difficult than translating between languages, due to the wider range of possible responses, the larger fraction of unaligned words/phrases, and the presence of large phrase pairs whose alignment cannot be further decomposed. After addressing these challenges, we compare approaches based on SMT and Information Retrieval in a human evaluation. We show that SMT outperforms IR on this task, and its output is preferred over actual human responses in 15% of cases. As far as we are aware, this is the first work to investigate the use of phrase-based SMT to directly translate a linguistic stimulus into an appropriate response.
Ron Artstein and Massimo Poesio. 2008. Inter-coder agreement for computational linguistics. Comput. Linguist., 34:555–596, December. Regina Barzilay and Mirella Lapata. 2005. Modeling local coherence: an entity-based approach. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, ACL ’05. E. M. Bennett, R. Alpert, and A. C. Goldstein. 1954. Communications through limited-response questioning. Public Opinion Quarterly, 18(3):303–308. Michael Bloodgood and Chris Callison-Burch. 2010. Using mechanical turk to build machine translation evaluation sets. In Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon ’s Mechanical Turk, CSLDAMT ’ 10, pages 208–21 1, Morristown, NJ, USA. Association for Computational Linguistics. Chris Brockett. 2006. Aligning the rte 2006 corpus. In Microsoft Research Techincal report MSR-TR-200777. Peter F. Brown, John Cocke, Stephen A. Della Pietra, Vincent J. Della Pietra, Fredrick Jelinek, John D. Lafferty, Robert L. Mercer, and Paul S. Roossin. 1990. A statistical approach to machine translation. Comput. Linguist., 16:79–85, June. Chris Callison-Burch, Philipp Koehn, Christof Monz, and Josh Schroeder. 2009. Findings of the 2009 workshop on statistical machine translation. In Proceedings of the Fourth Workshop on Statistical Machine Translation, StatMT ’09. Nathanael Chambers and James Allen. 2004. Stochastic language generation in a dialogue system: Toward a domain independent generator. In Michael Strube and Candy Sidner, editors, Proceedings of the 5th SIG- dial Workshop on Discourse and Dialogue, pages 9– 18, Cambridge, Massachusetts, USA, April 30 - May 1. Association for Computational Linguistics. Cristian Danescu-Niculescu-Mizil, Michael Gamon, and Susan Dumais. 2011. Mark my words! Linguistic style accommodation in social media. In Proceedings of WWW. Hal Daum e´ III and Daniel Marcu. 2009. Induction of word and phrase alignments for automatic document summarization. CoRR, abs/0907.0804. Bill Dolan, Chris Quirk, and Chris Brockett. 2004. Unsupervised construction of large paraphrase corpora: exploiting massively parallel news sources. In Proceedings of the 20th international conference on Computational Linguistics, COLING ’04, Morristown, NJ, USA. Association for Computational Linguistics. Abdessamad Echihabi and Daniel Marcu. 2003. A noisy-channel approach to question answering. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - Volume 1, ACL ’03, pages 16–23, Morristown, NJ, USA. Association for Computational Linguistics. Micha Elsner and Eugene Charniak. 2008. You talking to me? a corpus and algorithm for conversation disentanglement. In Proceedings of ACL-08: HLT, June. Michel Galley, Eric Fosler-Lussier, and Alexandros Potamianos. 2001 . Hybrid natural language generation for spoken dialogue systems. In Proceedings of the 7th European Conference on Speech Communication and Technology (EUROSPEECH–01), pages 1735–1738, Aalborg, Denmark, September. Jon Hasselgren, Erik Montnemery, Pierre Nugues, and Markus Svensson. 2003. Hms: a predictive text entry method using bigrams. In Proceedings of the 2003 EACL Workshop on Language Modeling for Text Entry Methods, TextEntry ’03. Jerry R. Hobbs. 1985. On the coherence and structure of discourse. Charles Lee Isbell, Jr., Michael J. Kearns, Dave Kormann, Satinder P. Singh, and Peter Stone. 2000. Cobot in lambdamoo: A social statistics agent. In Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence, pages 36–41. AAAI Press. Sina Jafarpour and Christopher J. C. Burges. 2010. Filter, rank, and transfer the knowledge: Learning to chat. Howard Johnson, Joel Martin, George Foster, and Roland Kuhn. 2007. Improving translation quality by discarding most of the phrasetable. In Proceedings of the 592 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 967– 975, Prague, Czech Republic, June. Association for Computational Linguistics. Sangkeun Jung, Cheongjae Lee, Kyungduk Kim, Minwoo Jeong, and Gary Geunbae Lee. 2009. Datadriven user simulation for automated evaluation of spoken dialog systems. Comput. Speech Lang. , 23:479–509, October. Kevin Knight and Vasileios Hatzivassiloglou. 1995. Two-level, many-paths generation. In Proceedings of the 33rd annual meeting on Association for Computational Linguistics, ACL ’95. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In ACL. The Association for Computer Linguistics. J R Landis and G G Koch. 1977. The measurement of observer agreement for categorical data. Biometrics. Brian Langner, Stephan Vogel, and Alan W. Black. 2010. Evaluating a dialog language generation system: comparing the mountain system to other nlg approaches. In INTERSPEECH. Anton Leuski and David R. Traum. 2010. Practical language processing for virtual humans. In TwentySecond Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-10). Bill MacCartney, Michel Galley, and Christopher D. Manning. 2008. A phrase-based alignment model for natural language inference. In Proceedings ofthe Conference on Empirical Methods in Natural Language Processing, EMNLP ’08, pages 802–81 1, Morristown, NJ, USA. Association for Computational Linguistics. Robert C. Moore. 2004. On log-likelihood-ratios and the significance of rare events. In EMNLP. Franz Josef Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1): 19–5 1. F. J. Och. 2003. Minimum error rate training for statistical machine translation. In ACL, pages 160–167. K. Papineni, S. Roukos, T. Ward, and W. J. Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In ACL, pages 3 11–3 18. Chris Quirk, Chris Brockett, and William Dolan. 2004. Monolingual machine translation for paraphrase generation. In In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pages 142–149. Owen Rambow, Srinivas Bangalore, and Marilyn Walker. 2001. Natural language generation in dialog systems. In Proceedings of the first international conference on Human language technology research, HLT ’01, pages 1–4, Stroudsburg, PA, USA. Association for Computational Linguistics. Adwait Ratnaparkhi. 2000. Trainable methods for sur- face natural language generation. In Proceedings of the 1st North American chapter of the Association for Computational Linguistics conference. Sujith Ravi, Andrei Broder, Evgeniy Gabrilovich, Vanja Josifovski, Sandeep Pandey, and Bo Pang. 2010. Automatic generation of bid phrases for online advertising. In Proceedings of the third ACM international conference on Web search and data mining, WSDM ’ 10. Stefan Riezler, Alexander Vasserman, Ioannis Tsochantaridis, Vibhu Mittal, and Yi Liu. 2007. Statistical machine translation for query expansion in answer retrieval. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 464–471, Prague, Czech Republic, June. Association for Computational Linguistics. Alan Ritter, Colin Cherry, and Bill Dolan. 2010. Unsupervised modeling of twitter conversations. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, HLT ’ 10, pages 172–180, Morristown, NJ, USA. Association for Computational Linguistics. Samira Shaikh, Tomek Strzalkowski, Sarah Taylor, and Nick Webb. 2010. Vca: an experiment with a multiparty virtual chat agent. In Proceedings of the 2010 Workshop on Companionable Dialogue Systems. Rion Snow, Brendan O’Connor, Daniel Jurafsky, and An- drew Y. Ng. 2008. Cheap and fast—but is it good?: evaluating non-expert annotations for natural language tasks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Radu Soricut and Daniel Marcu. 2006. Discourse generation using utility-trained coherence models. In Proceedings of the COLING/ACL on Main conference poster sessions, COLING-ACL ’06. Xu Sun, Jianfeng Gao, Daniel Micol, and Chris Quirk. 2010. Learning phrase-based spelling error models from clickthrough data. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, ACL ’ 10, pages 266–274, Morristown, NJ, USA. Association for Computational Linguistics. Reid Swanson and Andrew S. Gordon. 2008. Say anything: A massively collaborative open domain story writing companion. In Proceedings of the 1st Joint International Conference on Interactive Digital Story593 telling: Interactive Storytelling, ICIDS ’08, pages 32– 40, Berlin, Heidelberg. Springer-Verlag. Lidan Wang and Douglas W. Oard. 2009. Context-based message expansion for disentanglement of interleaved text conversations. In HLT-NAACL. Joseph Weizenbaum. 1966. Eliza: a computer program for the study of natural language communication between man and machine. Commun. ACM, 9:36–45, January. Yorick Wilks. 2006. Artificial companions as a new kind of interface to the future internet. In OII Research Report No. 13. Yuk Wah Wong and Raymond Mooney. 2006. Learning for semantic parsing with statistical machine translation. In Proceedings of the Human Language Technology Conference of the NAACL, Main Conference. Yuk Wah Wong and Raymond Mooney. 2007. Generation by inverting a semantic parser that uses statistical machine translation. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference.