acl acl2011 acl2011-47 acl2011-47-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Samuel Brody ; Paul Kantor
Abstract: Common approaches to assessing document quality look at shallow aspects, such as grammar and vocabulary. For many real-world applications, deeper notions of quality are needed. This work represents a first step in a project aimed at developing computational methods for deep assessment of quality in the domain of intelligence reports. We present an automated system for ranking intelligence reports with regard to coverage of relevant material. The system employs methodologies from the field of automatic summarization, and achieves performance on a par with human judges, even in the absence of the underlying information sources.
Blei, David M. , Andrew Y. Ng, and Michael I. Jordan. 2003. Latent dirichlet allocation. Journal of Machine Learning Research 3:993– 1022. Burstein, Jill, Martin Chodorow, and Claudia Leacock. 2004. Automated essay evaluation: the criterion online writing service. AI Mag. 25:27–36. Gillick, Dan and Benoit Favre. 2009. A scalable global model for summarization. In Proc. of the Workshop on Integer Linear Programming for Natural Language Processing. ACL, Stroudsburg, PA, USA, ILP ’09, pages 10–18. Gillick, Daniel, Benoit Favre, Dilek HakkaniTur, Berndt Bohnet, Yang Liu, and Shasha Xie. 2009. The ICSI/UTD Summarization System at TAC 2009. In Proc. of the Text Analysis Conference workshop, Gaithersburg, MD (USA). 495 Goldstein, Jade, Vibhu Mittal, Jaime Carbonell, and Mark Kantrowitz. 2000. Multi-document summarization by sentence extraction. In Proc. of the 2000 NAACL-ANLP Workshop on Automatic summarization - Volume 4 . Association for Computational Linguistics, Stroudsburg, PA, USA, NAACL-ANLPAutoSum ’00, pages 40–48. Haghighi, Aria and Lucy Vanderwende. 2009. Exploring content models for multi-document summarization. In Proc. of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics. ACL, Boulder, Colorado, pages 362–370. Jijkoun, Valentin and Katja Hofmann. 2009. Generating a non-english subjectivity lexicon: Relations that matter. In Proc. of the 12th Conference of the European Chapter of the A CL (EA CL 2009). ACL, Athens, Greece, pages 398–405. Larkey, Leah S. 1998. Automatic essay grading using text categorization techniques. In SIGIR ’98: Proceedings of the 21st annual international A CM SIGIR conference on Research and development in information retrieval. ACM, New York, NY, USA, pages 90– 95. Morse, Emile L., Jean Scholtz, Paul Kantor, Diane Kelly, and Ying Sun. 2004. An investigation of evaluation metrics for analytic question answering. Available by request from the first author. Nenkova, Ani, Lucy Vanderwende, and Kathleen McKeown. 2006. A compositional context sensitive multi-document summarizer: exploring the factors that influence summarization. In SIGIR. ACM, pages 573–580. Radev, Dragomir R., Hongyan Jing, Ma lgorzata Sty s´, and Daniel Tam. 2004. Centroid-based summarization of multiple documents. Inf. Process. Manage. 40:919–938. Shermis, Mark D. and Jill C. Burstein, editors. 2002. Automated Essay Scoring: A Crossdisciplinary Perspective. Routledge, 1edition.