emnlp emnlp2010 emnlp2010-115 emnlp2010-115-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Slav Petrov ; Pi-Chuan Chang ; Michael Ringgaard ; Hiyan Alshawi
Abstract: It is well known that parsing accuracies drop significantly on out-of-domain data. What is less known is that some parsers suffer more from domain shifts than others. We show that dependency parsers have more difficulty parsing questions than constituency parsers. In particular, deterministic shift-reduce dependency parsers, which are of highest interest for practical applications because of their linear running time, drop to 60% labeled accuracy on a question test set. We propose an uptraining procedure in which a deterministic parser is trained on the output of a more accurate, but slower, latent variable constituency parser (converted to dependencies). Uptraining with 100K unlabeled questions achieves results comparable to having 2K labeled questions for training. With 100K unlabeled and 2K labeled questions, uptraining is able to improve parsing accuracy to 84%, closing the gap between in-domain and out-of-domain performance.
J. Blitzer, R. McDonald, and F. Pereira. 2006. Domain adaptation with structural correspondence learning. In EMNLP ’06. A. Blum and T. Mitchell. 1998. Combining labeled and unlabeled data with co-training. In COLT ’98. T. Brants. 2000. TnT a statistical part-of-speech tagger. In ANLP ’00. P. Brown, V. Della Pietra, P. deSouza, J. Lai, and R. Mercer. 1992. Class-based n-gram models of natural language. Computational Linguistics. X. Carreras, M. Collins, and T. Koo. 2008. TAG, dynamic programming, and the perceptron for efficient, feature-rich parsing. In CoNLL ’08. E. Charniak and M. Johnson. 2005. Coarse-to-Fine NBest Parsing and MaxEnt Discriminative Reranking. In ACL’05. E. Charniak. 1997. Statistical parsing with a context-free grammar and word statistics. In AI ’97. E. Charniak. 2000. A maximum–entropy–inspired parser. In NAACL ’00. S. Clark, J. Curran, and M. Osborne. 2003. Bootstrapping pos-taggers using unlabelled data. In CoNLL ’03. M.-C. de Marneffe, B. MacCartney, and C. Manning. – 2006. Generating typed dependency parses from phrase structure parses. In LREC ’06. J. Eisner. 1996. Three new probabilistic models for dependency parsing: An exploration. In COLING ’96. J. Foster. 2010. “cba to check the spelling”: Investigating parser performance on discussion forum posts. In NAACL ’10. D. Gildea. 2001. Corpus variation and parser performance. In EMNLP ’01. Z. Huang and M. Harper. 2009. Self-training PCFG grammars with latent annotations across languages. In EMNLP ’09. J. Judge, A. Cahill, and J. v. Genabith. 2006. Questionbank: creating a corpus of parse-annotated questions. In ACL ’06. T. Koo and M. Collins. 2010. Efficient third-order dependency parsers. In ACL ’10. T. Koo, X. Carreras, and M. Collins. 2008. Simple semisupervised dependency parsing. In ACL ’08. 713 M. Marcus, B. Santorini, and M. Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. In Computational Linguistics. A.F.T. Martins, N.A. Smith, and E.P. Xing. 2009. Concise integer linear programming formulations for dependency parsing. In ACL ’09. D. McClosky, E. Charniak, and M. Johnson. 2006a. Effective self-training for parsing. In NAACL ’06. D. McClosky, E. Charniak, and M. Johnson. 2006b. Reranking and self-training for parser adaptation. In ACL ’06. R. McDonald and J. Nivre. 2007. Characterizing the errors of data-driven dependency parsing models. In EMNLP ’07. R. McDonald, K. Crammer, and F. Pereira. 2005. Online large-margin training of dependency parsers. In ACL ’05. R. McDonald, K. Lerman, and F. Pereira. 2006. Multilingual dependency analysis with a two-stage discriminative parser. In CoNLL ’06. J. Nivre, J. Hall, J. Nilsson, A. Chanev, G. Eryigit, S. Kbler, S. Marinov, and E. Marsi. 2007. Maltparser: A language-independent system for data-driven dependency parsing. Natural Language Engineering, 13(2). J. Nivre. 2008. Algorithms for deterministic incremental dependency parsing. Computational Linguistics, 34(4). S. Petrov, L. Barrett, R. Thibaux, and D. Klein. 2006. Learning accurate, compact, and interpretable tree annotation. In ACL ’06. S. Petrov. 2010. Products of random latent variable grammars. In NAACL ’10. S. Riedel and J. Clarke. 2006. Incremental integer linear programming for non-projective dependency parsing. In EMNLP ’06. K. Sagae and A. Lavie. 2006. Parser combination by reparsing. In NAACL ’06. K. Sagae and J. Tsujii. 2007. Dependency parsing and domain adaptation with lr models and parser ensembles. In CoNLL ’07. M. Steedman, M. Osborne, A. Sarkar, S. Clark, R. Hwa, J. Hockenmaier, P. Ruhlen, S. Baker, and J. Crim. 2003. Bootstrapping statistical parsers from small datasets. In EACL ’03. J. Suzuki, H. Isozaki, X. Carreras, and M. Collins. 2009. An empirical study of semi-supervised structured conditional models for dependency parsing. In EMNLP ’09. H. Yamada and Y. Matsumoto. 2003. Statistical dependency analysis with support vector machines. In IWPT ’03.