acl acl2010 acl2010-187 acl2010-187-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Verena Rieser ; Oliver Lemon ; Xingkun Liu
Abstract: We present a novel approach to Information Presentation (IP) in Spoken Dialogue Systems (SDS) using a data-driven statistical optimisation framework for content planning and attribute selection. First we collect data in a Wizard-of-Oz (WoZ) experiment and use it to build a supervised model of human behaviour. This forms a baseline for measuring the performance of optimised policies, developed from this data using Reinforcement Learning (RL) methods. We show that the optimised policies significantly outperform the baselines in a variety of generation scenarios: while the supervised model is able to attain up to 87.6% of the possible reward on this task, the RL policies are significantly better in 5 out of 6 scenarios, gaining up to 91.5% of the total possible reward. The RL policies perform especially well in more complex scenarios. We are also the first to show that adding predictive “lower level” features (e.g. from the NLG realiser) is important for optimising IP strategies according to user preferences. This provides new insights into the nature of the IP problem for SDS.
Cedric Boidin, Verena Rieser, Lonneke van der Plas, Oliver Lemon, and Jonathan Chevelu. 2009. Predicting how it sounds: Re-ranking alternative inputs to TTS using latent variables (forthcoming). In Proc. of Interspeech/ICSLP, Special Session on Machine Learning for Adaptivity in Spoken Dialogue Systems. Grace Chung. 2004. Developing a flexible spoken dialog system using simulation. In Proc. of the Annual Meeting of the Association for Computational Linguistics (ACL). P.R. Clarkson and R. Rosenfeld. 1997. Statistical Language Modeling Using the CMU-Cambridge Toolkit. In Proc. of ESCA Eurospeech. William W. Cohen. 1995. Fast effective rule induction. In Proceedings of the 12th International Conference on Machine Learning (ICML). Heriberto Cuay a´huitl, Steve Renals, Oliver Lemon, and Hiroshi Shimodaira. 2005. Human-computer dialogue simulation using hidden markov models. In Proc. of the IEEE workshop on Automatic Speech Recognition and Understanding (ASRU). Vera Demberg and Johanna D. Moore. 2006. Information presentation in spoken dialogue systems. In Proceedings of EACL. 1017 W. Eckert, E. Levin, and R. Pieraccini. 1997. User modeling for spoken dialogue system evaluation. In Proc. of the IEEE workshop on Automatic Speech Recognition and Understanding (ASRU). M. Gasic, S. Keizer, F. Mairesse, J. Schatzmann, B. Thomson, and S. Young. 2008. Training and Evaluation of the HIS POMDP Dialogue System in Noise. In Proc. of SIGdial Workshop on Discourse and Dialogue. James Henderson and Oliver Lemon. 2008. Mixture Model POMDPs for Efficient Handling of Uncertainty in Dialogue Management. In Proc. of ACL. Srinivasan Janarthanam and Oliver Lemon. 2009. A Two-tier User Simulation Model for Reinforcement Learning of Adaptive Referring Expression Generation Policies. In Proc. of SIGdial. Srini Janarthanam and Oliver Lemon. 2010. Learning to adapt to unknown users: Referring expression generation in spoken dialogue systems. In Proceedings of ACL. Sangkeun Jung, Cheongjae Lee, Kyungduk Kim, Minwoo Jeong, and Gary Geunbae Lee. 2009. Datadriven user simulation for automated evaluation of spoken dialog systems. Computer, Speech & Language, 23:479–509. Alexander Koller and Ronald Petrick. 2008. Experi- ences with planning for natural language generation. In ICAPS. Oliver Lemon, Kallirroi Georgila, and James Henderson. 2006. Evaluating Effectiveness and Portability of Reinforcement Learned Dialogue Strategies with real users: the TALK TownInfo Evaluation. In IEEE/ACL Spoken Language Technology. Oliver Lemon. 2008. Adaptive Natural Language Generation in Dialogue using Reinforcement Learning. In Proceedings of SEMdial. Oliver Lemon. 2010. Learning what to say and how to say it: joint optimization of spoken dialogue management and Natural Language Generation. Computer, Speech & Language, to appear. Xingkun Liu, Verena Rieser, and Oliver Lemon. 2009. A wizard-of-oz interface to study information presentation strategies for spoken dialogue systems. In Proc. of the 1st International Workshop on Spoken Dialogue Systems. Crystal Nakatsu. 2008. Learning contrastive connectives in sentence realization ranking. In Proc. of SIGdial Workshop on Discourse and Dialogue. Joseph Polifroni and Marilyn Walker. 2006. Learning database content for spoken dialogue system design. In Proc. of the IEEE/ACL workshop on Spoken Language Technology (SLT). Joseph Polifroni and Marilyn Walker. 2008. Intensional Summaries as Cooperative Responses in Dialogue Automation and Evaluation. In Proceedings of ACL. Verena Rieser and Oliver Lemon. 2008. Learning Effective Multimodal Dialogue Strategies from Wizard-of-Oz data: Bootstrapping and Evaluation. In Proc. of ACL. Verena Rieser and Oliver Lemon. 2009. Natural Language Generation as Planning Under Uncertainty for Spoken Dialogue Systems. In Proc. of EACL. Verena Rieser, Xingkun Liu, and Oliver Lemon. 2009. Optimal Wizard NLG Behaviours in Context. Technical report, Deliverable 4.2, CLASSiC Project. Dan Shapiro and P. Langley. 2002. Separating skills from preference: Using learning to program by reward. In Proc. of the 19th International Conference on Machine Learning (ICML). Amanda Stent, Rashmi Prasad, and Marilyn Walker. 2004. Trainable sentence planning for complex information presentation in spoken dialog systems. In Association for Computational Linguistics. R. Sutton and A. Barto. 1998. Reinforcement Learning. MIT Press. Kees van Deemter. 2009. What game theory can do for NLG: the case of vague language. In 12th European Workshop on Natural Language Generation (ENLG). Marilyn A. Walker, Candace A. Kamm, and Diane J. Litman. 2000. Towards developing general models of usability with PARADISE. Natural Language Engineering, 6(3). M. Walker, R. Passonneau, and J. Boland. 2001. Quantitative and qualitative evaluation of DARPA Communicator spoken dialogue systems. In Proc. of the Annual Meeting of the Association for Computational Linguistics (ACL). Marilyn Walker, Amanda Stent, Fran ¸cois Mairesse, and Rashmi Prasad. 2007. Individual and domain adaptation in sentence planning for dialogue. Journal of Artificial Intelligence Research (JAIR), 30:413–456. Steve Whittaker, Marilyn Walker, and Johanna Moore. 2002. Fish or Fowl: A Wizard of Oz evaluation of dialogue strategies in the restaurant domain. In Proc. of the International Conference on Language Resources and Evaluation (LREC). Andi Winterboer, Jiang Hu, Johanna D. Moore, and Clifford Nass. 2007. The influence of user tailoring and cognitive load on user performance in spoken dialogue systems. In Proc. of the 10th International Conference of Spoken Language Processing (Interspeech/ICSLP). SJ Young, J Schatzmann, K Weilhammer, and H Ye. 2007. The Hidden Information State Approach to Dialog Management. In ICASSP 2007. 1018