acl acl2011 acl2011-53 acl2011-53-reference knowledge-graph by maker-knowledge-mining

53 acl-2011-Automatically Evaluating Text Coherence Using Discourse Relations


Source: pdf

Author: Ziheng Lin ; Hwee Tou Ng ; Min-Yen Kan

Abstract: We present a novel model to represent and assess the discourse coherence of text. Our model assumes that coherent text implicitly favors certain types of discourse relation transitions. We implement this model and apply it towards the text ordering ranking task, which aims to discern an original text from a permuted ordering of its sentences. The experimental results demonstrate that our model is able to significantly outperform the state-ofthe-art coherence model by Barzilay and Lapata (2005), reducing the error rate of the previous approach by an average of 29% over three data sets against human upperbounds. We further show that our model is synergistic with the previous approach, demonstrating an error reduction of 73% when the features from both models are combined for the task.


reference text

Regina Barzilay and Mirella Lapata. 2005. Modeling local coherence: an entity-based approach. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL 2005), pages 141–148, Morristown, NJ, USA. Association for Computational Linguistics. Regina Barzilay and Mirella Lapata. 2008. Modeling local coherence: An entity-based approach. Computational Linguistics, 34: 1–34, March. Regina Barzilay and Lillian Lee. 2004. Catching the drift: Probabilistic content models, with applications to generation and summarization. In Proceedings of the Human Language Technology Conference / North American Chapter of the Association for Computational Linguistics Annual Meeting 2004. Micha Elsner, Joseph Austerweil, and Eugene Charniak. 2007. A unified local and global model for discourse coherence. In Proceedings of the Conference on Human Language Technology and North American Chapter ofthe Associationfor ComputationalLinguistics (HLT-NAACL 2007), Rochester, New York, USA, April. Robert Elwell and Jason Baldridge. 2008. Discourse connective argument identification with connective specific rankers. In Proceedings of the IEEE International Conference on Semantic Computing (ICSC 2010), Washington, DC, USA. Barbara J. Grosz, Scott Weinstein, and Aravind K. Joshi. 1995. Centering: a framework for modeling the local coherence of discourse. Computational Linguistics, 21(2):203–225, June. Thorsten Joachims. 1999. Making large-scale support vector machine learning practical. In Bernhard Schlkopf, Christopher J. C. Burges, and Alexander J. Smola, editors, Advances in Kernel Methods Support Vector Learning, pages 169–184. MIT Press, Cambridge, MA, USA. Nikiforos Karamanis. 2007. Supplementing entity coherence with local rhetorical relations for information ordering. Journal of Logic, Language and Information, 16:445–464, October. Mirella Lapata and Regina Barzilay. 2005. Automatic evaluation of text coherence: Models and representations. In Leslie Pack Kaelbling and Alessandro Saffiotti, editors, Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence, Edinburgh, Scotland, UK. Ziheng Lin, Min-Yen Kan, and Hwee Tou Ng. 2009. Recognizing implicit discourse relations in the Penn Discourse Treebank. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (EMNLP 2009), Singapore. Ziheng Lin, Hwee Tou Ng, and Min-Yen Kan. 2010. A PDTB-styled end-to-end discourse parser. Technical Report TRB8/10, School of Computing, National University of Singapore, August. William C. Mann and Sandra A. Thompson. 1988. Rhetorical Structure Theory: Toward a functional theory of text organization. Text, 8(3):243–281 . Daniel Marcu. 1996. Distinguishing between coherent and incoherent texts. In The Proceedings of the Student Conference on Computational Linguistics in – Montreal, pages 136–143. Jane Morris and Graeme Hirst. 1991 . Lexical cohesion computed by thesaural relations as an indicator of the structure of text. Computational Linguistics, 17:21– 48, March. Emily Pitler and Ani Nenkova. 2009. Using syntax to disambiguate explicit discourse connectives in text. In Proceedings of the ACL-IJCNLP 2009 Conference Short Papers, Singapore. Emily Pitler, Mridhula Raghupathy, Hena Mehta, Ani Nenkova, Alan Lee, and Aravind Joshi. 2008. Easily identifiable discourse relations. In Proceedings of the 22nd International Conference on Computational Linguistics (COLING 2008) Short Papers, Manchester, UK. Emily Pitler, Annie Louis, and Ani Nenkova. 2009. Automatic sense prediction for implicit discourse relations in text. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP (ACL-IJCNLP 2009), Singapore. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi, and Bonnie 1006 Webber. 2008. The Penn Discourse Treebank 2.0. In Proceedings of the 6th International Conference on Language Resources and Evaluation (LREC 2008). Radu Soricut and Daniel Marcu. 2006. Discourse generation using utility-trained coherence models. In Proceedings ofthe COLING/ACL Main Conference Poster Sessions, pages 803–8 10, Morristown, NJ, USA. Association for Computational Linguistics. WenTing Wang, Jian Su, and Chew Lim Tan. 2010. Kernel based discourse relation recognition with temporal ordering information. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL 2010), Uppsala, Sweden, July. Bonnie Webber. 2004. D-LTAG: Extending lexicalized TAG to discourse. Cognitive Science, 28(5):751–779. Ben Wellner and James Pustejovsky. 2007. Automatically identifying the arguments of discourse connectives. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL 2007), Prague, Czech Republic.