acl acl2013 acl2013-265 acl2013-265-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Marco Fossati ; Claudio Giuliano ; Sara Tonelli
Abstract: We present the first attempt to perform full FrameNet annotation with crowdsourcing techniques. We compare two approaches: the first one is the standard annotation methodology of lexical units and frame elements in two steps, while the second is a novel approach aimed at acquiring frames in a bottom-up fashion, starting from frame element annotation. We show that our methodology, relying on a single annotation step and on simplified role definitions, outperforms the standard one both in terms of accuracy and time.
[Baker2012] Collin F Baker. 2012. Framenet, cur- rent collaborations and future goals. Language Resources and Evaluation, pages 1–18. [Burchardt et al.2006] Aljoscha Burchardt, Katrin Erk, Anette Frank, Andrea Kowalski, Sebastian Pado, and Manfred Pinkal. 2006. Salto–a versatile multilevel annotation tool. In Proceedings of LREC 2006, pages 517–520. Citeseer. [Chamberlain et al.2008] Jon Chamberlain, Massimo Poesio, and Udo Kruschwitz. 2008. Phrase detectives: A web-based collaborative annotation game. Proceedings of I-Semantics, Graz. [Chamberlain et al.2009] Jon Chamberlain, Udo Kruschwitz, and Massimo Poesio. 2009. Constructing an anaphorically annotated corpus with non-experts: Assessing the quality of collaborative annotations. In Proceedings of the 2009 Workshop on The People ’s Web Meets NLP: Collaboratively Constructed Semantic Resources, pages 57–62. Association for Computational Linguistics. [Fillmore et al.2002] Charles J. Fillmore, Collin F. Baker, and Hiroaki Sato. 2002. The FrameNet Database and Software Tools. In Proceedings of the Third International Conference on Language Resources and Evaluation (LREC 2002), pages 1157– 1160, Las Palmas, Spain. [Fillmore1976] Charles J. Fillmore. 1976. Frame Se- mantics and the nature of language. In Annals of the New York Academy of Sciences: Conference on the Origin and Development of Language, pages 20–32. Blackwell Publishing. [Heilman and Smith2010] Michael Heilman and Noah A. Smith. 2010. Extracting Simplified Statements for Factual Question Generation. In Proceedings of QG2010: The Third Workshop on Question Generation, Pittsburgh, PA, USA. [Hong and Baker201 1] Jisup Hong and Collin F Baker. 2011. How good is the crowd at “real” wsd? ACL HLT 2011, page 30. [Negri et al.201 1] Matteo Negri, Luisa Bentivogli, Yashar Mehdad, Danilo Giampiccolo, and Alessandro Marchetti. 2011. Divide and conquer: crowdsourcing the creation of cross-lingual textual entailment corpora. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP ’ 11, pages 670–679, Stroudsburg, PA, USA. Association for Computational Linguistics. [Pad o´ and Lapata2009] Sebastian Pad o´ and Mirella Lapata. 2009. Cross-lingual annotation projection for semantic roles. Journal of Artificial Intelligence Research, 36(1):307–340. [Palmer et al.2005] Martha Palmer, Dan Gildea, and Paul Kingsbury. 2005. The Proposition Bank: A Corpus Annotated with Semantic Roles. Computational Linguistics, 3 1(1). [Ruppenhofer et al.2006] Josef Ruppenhofer, Michael Ellsworth, Miriam R.L. Petruck, Christopher R. Johnson, and Jan Scheffczyk. 2006. FrameNet II: Extended Theory and Practice. Available at http://framenet.icsi.berkeley.edu/book/book.html. [Snow et al.2008] Rion Snow, Brendan O’Connor, Daniel Jurafsky, and Andrew Y Ng. 2008. Cheap and fast—but is it good?: evaluating non-expert annotations for natural language tasks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 254–263. Association for Computational Linguistics. [Von Ahn et al.2006] Luis Von Ahn, Mihir Kedia, and Manuel Blum. 2006. Verbosity: a game for collecting common-sense facts. In Proceedings of the SIGCHI conference on Human Factors in computing systems, pages 75–78. ACM. 747