emnlp emnlp2012 emnlp2012-102 emnlp2012-102-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Nina Dethlefs ; Helen Hastie ; Verena Rieser ; Oliver Lemon
Abstract: Incremental processing allows system designers to address several discourse phenomena that have previously been somewhat neglected in interactive systems, such as backchannels or barge-ins, but that can enhance the responsiveness and naturalness of systems. Unfortunately, prior work has focused largely on deterministic incremental decision making, rendering system behaviour less flexible and adaptive than is desirable. We present a novel approach to incremental decision making that is based on Hierarchical Reinforcement Learning to achieve an interactive optimisation of Information Presentation (IP) strategies, allowing the system to generate and comprehend backchannels and barge-ins, by employing the recent psycholinguistic hypothesis of information density (ID) (Jaeger, 2010). Results in terms of average rewards and a human rating study show that our learnt strategy outperforms several baselines that are | v not sensitive to ID by more than 23%.
Matthew Aylett and Alice Turk. 2004. The smooth signal redundancy hypothesis: A functional explanation for the relationships between redundancy, prosodic prominence, and duration in spontaneous speech. Language and Speech, 47(1):3 1–56. Timo Baumann, Okko Buss, and David Schlangen. 2011. Evaluation and Optimisation of Incremental Processors. Dialogue and Discourse, 2(1). Alan Bell, Dan Jurafsky, Eric Fossler-Lussier, Cynthia Girand, Michelle Gregory, and Daniel Gildea. 2003. Effects of disfluencies, predictability, and utterance position on word form variation in english conversation. Journal of the Acoustic Society of America, 113(2): 1001–1024. Okko Buss, Timo Baumann, and David Schlangen. 2010. Collaborating on Utterances with a Spoken Dialogue Systen Using an ISU-based Approach to Incremental Dialogue Management. In Proceedings of 11th Annual SIGdial Meeting on Discourse and Dialogue. Heriberto Cuay a´huitl and Nina Dethlefs. 2011. Spatially-aware Dialogue Control Using Hierarchical Reinforcement Learning. ACM Transactions on Speech and Language Processing (Special Issue on Machine Learning for Robust and Adaptive Spoken Dialogue System), 7(3). Heriberto Cuay a´huitl, Steve Renals, Oliver Lemon, and Hiroshi Shimodaira. 2010. Evaluation of a hierarchical reinforcement learning spoken dialogue system. Computer Speech and Language, 24(2):395–429. Heriberto Cuay a´huitl. 2009. Hierarchical Reinforcement Learning for Spoken Dialogue Systems. PhD Thesis, University of Edinburgh, School of Informatics. Nina Dethlefs and Heriberto Cuay a´huitl. 2011. Combining Hierarchical Reinforcement Learning and Bayesian Networks for Natural Language Generation in Situated Dialogue. In Proceedings of the 13th European Workshop on Natural Language Generation (ENLG), Nancy, France. Nina Dethlefs, Helen Hastie, Verena Rieser, and Oliver Lemon. 2012. Optimising Incremental Generation for Spoken Dialogue Systems: Reducing the Need for Fillers. In Proceedings of the International Conference on Natural Language Generation (INLG), Chicago, Illinois, USA. David DeVault, Kenji Sagae, and David Traum. 2009. Can I finish? Learning when to respond to incremental 92 interpretation result in interactive dialogue. In Proceedings of the 10th Annual SigDial Meeting on Discourse and Dialogue, Queen Mary University, UK. Thomas G. Dietterich. 1999. Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition. Journal of Artificial Intelligence Research, 13:227–303. Dmitriy Genzel and Eugene Charniak. 2002. Entropy Rate Constancy in Text. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pages 199–206. James Henderson, Oliver Lemon, and Kallirroi Georgila. 2008. Hybrid Reinforcement/Supervised Learning of Dialogue Policies from Fixed Data Sets. Computational Linguistics, 34(4):487–5 11. T. Florian Jaeger. 2010. Redundancy and reduction: Speakers manage syntactic information density. Cognitive Psychology, 61:23–62. Srini Janarthanam and Oliver Lemon. 2010. Learning to Adapt to Unknown Users: Referring Expression Generation in Spoken Dialogue Systems. In Proceed- ings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL), pages 69–78, July. Anne Kilger and Wolfgang Finkler. 1995. Incremental generation for real-time applications. Technical report, DFKI Saarbruecken, Germany. Oliver Lemon. 2011. Learning What to Say and How to Say It: Joint Optimization of Spoken Dialogue Management and Natural Language Generation. Esther Levin, Roberto Pieraccini, and Wieland Eckert. 2000. A Stochastic Model of Computer-Human Interaction for Learning Dialogue Strategies. IEEE Transactions on Speech and Audio Processing, 8: 11–23. Roger Levy and T. Florian Jaeger. 2007. Speakers optimize information density through syntactic reduction. Advances in Neural Information Processing Systems, 19. Olivier Pietquin and Dutoit. 2006. A Probabilistic Framework for Dialogue Simulation and Optimal Strategy Learning. IEEE Transactions on Speech and Audio Processing, 14(2):589–599. Matthew Purver and Masayuki Otsuka. 2003. Incremental Generation by Incremental Parsing. In Proceedings of the 6th UK Special-Interesting Group for Computational Linguistics (CLUK) Colloquium. Rajakrishnan Rajkumar and Michael White. 2011. Linguistically Motivated Complementizer Choice in Surface Realization. In Proceedings of the EMNLP-11 Workshop on Using Corpora in NLG, Edinburgh, Scotland. Antoine Raux and Maxine Eskenazi. 2009. A FiniteState Turn-Taking Model for Spoken Dialog Systems. In Proceedings of the 10th Conference of the North American Chapter of the Association for Computational Linguistics—Human Language Technologies (NAACL-HLT), Boulder, Colorado. Verena Rieser, Oliver Lemon, and Xingkun Liu. 2010. Optimising Information Presentation for Spoken Dialogue Systems. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL), Uppsala, Sweden. Verena Rieser, Simon Keizer, Xingkun Liu, and Oliver Lemon. 2011. Adaptive Information Presentation for Spoken Dialogue Systems: Evaluation with Human Subjects. In Proceedings of the 13th European Workshop on Natural Language Generation (ENLG), Nancy, France. David Schlangen and Gabriel Skantze. 2011. A General, Abstract Model of Incremental Dialogue Processing. Dialogue and Discourse, 2(1). Ethan Selfridge, Iker Arizmendi, Peter Heeman, and Jason Williams. 2011. Stability and Accuracy in Incremental Speech Recognition. In Proceedings of the 12th Annual SigDial Meeting on Discourse and Dialogue, Portland, Oregon. Claude Shannon. 1948. A Mathematical Theory of Communications. Bell Systems Technical Journal, 27(4):623–656. Satinder Singh, Diane Litman, Michael Kearns, and Marilyn Walker. 2002. Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun System. Journal of Artificial Intelligence Research, 16: 105–133. Gabriel Skantze and Anna Hjalmarsson. 2010. Towards Incremental Speech Generation in Dialogue Systems. In Proceedings of the 11th Annual SigDial Meeting on Discourse and Dialogue, Tokyo, Japan. Richard Sutton and Andrew Barto. 1998. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA. Blaise Thomson. 2009. Statistical Methods for Spoken Dialogue Management. Ph.D. thesis, University of Cambridge. Menno van Zaanen. 2000. Bootstrapping Syntax and Recursion using Alignment-Based Learning. In Proceedings of the Seventeenth International Conference on Machine Learning, ICML ’00, pages 1063–1070. Marilyn Walker. 2000. An Application ofReinforcement Learning to Dialogue Strategy Selection in a Spoken Dialogue System for Email. Journal of Artificial Intelligence Research (JAIR), 12:387–416. Steve Young, Milica Gasic, Simon Keizer, Francois Mairesse, Jost Schatzmann, Blaise Thomson, and Kai Yu. 2010. The Hidden Information State Model: A Practical Framework for POMDP-based Spoken Dialogue Management. Computer Speech and Language, Steve Young. 2000. Probabilistic Methods in Spoken Dialogue Systems. Philosophical Transactions of the Royal Society (Series A), 358(1769): 1389–1402. 24(2): 150–174. 93