acl acl2013 acl2013-324 acl2013-324-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Shu Cai ; Kevin Knight
Abstract: The evaluation of whole-sentence semantic structures plays an important role in semantic parsing and large-scale semantic structure annotation. However, there is no widely-used metric to evaluate wholesentence semantic structures. In this paper, we present smatch, a metric that calculates the degree of overlap between two semantic feature structures. We give an efficient algorithm to compute the metric and show the results of an inter-annotator agreement study.
J.F. Allen, M. Swift, and W. Beaumont. 2008. Deep Semantic Analysis of Text. In Proceedings of the 2008 Conference on Semantics in Text Processing. D. Davidson. 1969. The Individuation of Events. In Nicholas Rescher (ed.) Essays in Honor of Carl G. HempeL Dordrecht: D. Reidel. R. Dridan and S. Oepen. 2011. Parser Evaluation using Elementary Dependency Matching. In Proceedings of the 12th International Conference on Parsing Technologies. B. Jones, J. Andreas, D. Bauer, K. M. Hermann, and K. Knight. 2012. Semantics-Based Machine Translation with Hyperedge Replacement Grammars. In Proceedings of COLING. P. Kingsbury and M. Palmer. 2002. From Treebank to Propbank. In Proceedings of LREC. I. Langkilde and K. Knight. 1998. Generation that Exploits Corpus-based Statistical Knowledge. In Proceedings of COLING-ACL. I. Langkilde-Geary. 2002. An Empirical Verification of Coverage and Correctness for a GeneralPurpose Sentence Generator. In Proceedings of International Natural Language Generation Conference (INLG’02). V. Nagarajan and M. Sviridenko. 2009. On the Maximum Quadratic Assignment Problem. Mathematics of Operations Research, 34. K. Papineni, S. Roukos, T. Ward, and W. Zhu. 2002. BLEU: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th An- L. R. Tang and R. J. Mooney. 2001. Using Multiple Clause Constructors in Inductive Logic Programming for Semantic Parsing. In Proceedings of the 12th European Conference on Machine Learning. L. S. Zettlemoyer and M. Collins. 2005. Learning to Map to Logical Form: Structured fication with Probabilistic Categorial Grammars. In Proceedings of the 21st Conference in Uncertainty in Artificial Intelligence. Sentences Classi- nual Meeting on Association for Computational Linguistics. T. Parsons. 1990. Events in the Semantics of English. The MIT Press. S. S. Pradhan, E. Hovy, M. Marcus, M. Palmer, L. Ramshaw, and R. Weischedel. 2007. Ontonotes: A Unified Relational Semantic Representation. In Proceedings of the International Conference on Semantic Computing (ICSC ’07). M. Snover, B. Dorr, R. Schwartz, L. Micciulla, and J. Makhoul. 2006. A Study of Translation Edit Rate with Targeted Human Annotation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas (AMTA-2006). 752