emnlp emnlp2011 emnlp2011-50 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Reut Tsarfaty ; Joakim Nivre ; Evelina Andersson
Abstract: unkown-abstract
Reference: text
sentIndex sentText sentNum sentScore
1 This paper develops a robust procedure for cross-experimental evaluation, based on deterministic unificationbased operations for harmonizing different representations and a refined notion of tree edit distance for evaluating parse hypotheses relative to multiple gold standards. [sent-2, score-0.81]
2 We demonstrate that, for different conversions of the Penn Treebank into dependencies, performance trends that are observed for parsing results in isolation change or dissolve completely when parse hypotheses are normalized and brought into the same common ground. [sent-3, score-0.249]
3 1 Introduction Data-driven dependency parsing has seen a considerable surge of interest in recent years. [sent-4, score-0.263]
4 The evaluation metric traditionally associated with dependency parsing is based on scoring labeled or unlabeled attachment decisions, whereby each correctly identified pair of head-dependent words is counted towards the success of the parser (Buchholz and Marsi, 2006). [sent-8, score-0.44]
5 385 Joakim Nivre Uppsala University Sweden Evelina ndersson Uppsala University Sweden Different annotation schemes often make differ- assumptions with respect to how linguistic conparsing tent is represented in a treebank (Rambow, 2010). [sent-10, score-0.354]
6 Different methods have been proposed for making dependency parsing results comparable across experiments. [sent-12, score-0.312]
7 , 2008; Buyko and Hahn, 2010), or neutralizing the arc direction in the native representation of dependency trees (Schwartz et al. [sent-16, score-0.541]
8 Transforming dependency trees to a set of pre-defined labeled dependencies, or into task-based features, reent quires the use of heuristic rules that run the risk of distorting correct information and introducing noise of their own. [sent-20, score-0.446]
9 This paper proposes a new three-step protocol for cross-experiment parser evaluation, and in particular for comparing parsing results across data sets that adhere to different annotation schemes. [sent-22, score-0.404]
10 ec th2o0d1s1 i Ans Nsoactuiartaioln La fonrg Cuaogmep Purtoatcieosnsainlg L,in pgaugies ti 3c8s5–396, first step all structures are brought into a single formal space of events that neutralizes representation peculiarities (for instance, arc directionality). [sent-25, score-0.233]
11 The last step computes the normalized distance from this common denominator to parse hypotheses, minus the cost of distances that reflect mere annotation idiosyncrasies. [sent-27, score-0.288]
12 We use the proposed procedure to compare de- pendency parsing results trained on Penn Treebank trees converted into dependency trees according to five different sets of linguistic assumptions. [sent-29, score-0.719]
13 We show that when starting off with the same set of sentences and the same parser, training on different conversion schemes yields apparently significant performance gaps. [sent-30, score-0.302]
14 When results across schemes are normalized and compared against the shared linguistic content, these performance gaps decrease or dissolve completely. [sent-31, score-0.37]
15 Even if the formal representation in a dependency treebank is well-defined according to current standards (K¨ ubler et al. [sent-35, score-0.451]
16 When multiple conversion algorithms are applied to the same data, we end up with different dependency trees for the same sentences (Johansson and Nugues, 2007; Choi and Palmer, 2010; de Marneffe et al. [sent-40, score-0.514]
17 In linguistics, there is a distinction between lexical heads and functional heads. [sent-45, score-0.388]
18 A lexical head carries the semantic gist of a phrase while a functional one marks its relation to other parts of the sentence. [sent-46, score-0.398]
19 This phrase has two possible analyses, one selects a lexical head (1a) and the other selects a functional one (1b), as depicted below. [sent-49, score-0.398]
20 (1a) Sunday (1b) on onprep Sundpaoybj Similar choices are found in phrases which contain functional elements such as determiners, coordination markers, subordinating elements, and so on. [sent-50, score-0.327]
21 We can choose between a functional head (1a) and a lexical head (2b, 2c). [sent-55, score-0.469]
22 (3a) would (3b) worked havvge workvegd havvge wouvgld In standard settings, an experiment that uses a data set which adheres to a certain annotation scheme reports results that are compared against the annotation standard that the parser was trained on. [sent-67, score-0.438]
23 If parse1 and parse2 are compared against gold2 using labeled attachment scores (LAS), then parse1 results are lower than the results of parse2, even though both parsers produced linguistically correct and perfectly useful output. [sent-70, score-0.219]
24 Existing methods for making parsing results comparable across experiments include heuristics for converting outputs into dependency trees of a predefined standard (Briscoe et al. [sent-71, score-0.514]
25 × In this section we first define functional trees as the common space of formal objects and define a deterministic conversion procedure from dependency trees to functional trees. [sent-89, score-1.535]
26 Next we define a set of formal operations on functional trees that compute, for every pair ofcorresponding trees of the same yield, a single gold tree that resolves inconsistencies among gold standard alternatives and combines the information that they share. [sent-90, score-1.337]
27 Finally, we define scores based on tree edit distance, refined to consider the distance from parses to the overall gold tree as well as the different annotation alternatives. [sent-91, score-0.717]
28 sA a dependency graph d g risa a dairteiccatled re graph which consists of nodes Vd and arcs Ad ⊆ Vd Vd. [sent-94, score-0.351]
29 , tn is any dependency graph that is a directed tree originating out of a node v0 labeled t0 = ROOT, and spans all terminals in the sentence, that is, for every ti ∈ S there exists vj ∈ Vd labeled labelV(vj) = ti. [sent-100, score-0.601]
30 F∈or simplicity we assume that every node vj is indexed according to the position of the terminal label, i. [sent-101, score-0.222]
31 In a labeled dependency tree, arcs in Ad are labeled by elements of L via a function labelA : Ad → L that encodes tohfe L grammatical orenl altaiobenl between→ →the L t tehramti ennaclos dlea-s beling the connected nodes. [sent-104, score-0.495]
32 We define two auxiliary functions on nodes in dependency trees. [sent-105, score-0.246]
33 1 Step 1: Functional Representation Our first goal is to define a representation format that keeps all functional relationships that are represented in the dependency trees intact, but remains neutral with respect to the directionality of the head-dependent relations. [sent-108, score-0.707]
34 To do so we define functional trees linearly-ordered labeled trees which, instead of head-to-head binary relations, represent the complete functional structure of a sentence. [sent-109, score-1.124]
35 All 1If a dependency tree d is projective, than for all v ∈ Vd the termiIfn aals d einp span(v) rfoeerm d a contiguous segment oalfl lS v. [sent-111, score-0.32]
36 388 terminal nodes in π are labeled with terminal symbols via a labelT : V → T function. [sent-114, score-0.28]
37 We obtain functional trees from dependency t vrieaes A using tohbet following procedure: • Initialize the set of nodes and arcs in the tree. [sent-117, score-0.88]
38 This gives us a constituency- = like representation of dependency trees labeled with functional information, tic assumptions which retains the linguis- reflected in the dependency trees. [sent-121, score-1.001]
39 In order to compare, combine or detect inconsistencies in the information inherent in different functional trees, we define a set of formal operations that are inspired by familiar notions from unification-based formalisms (Shieber (1986) and references therein). [sent-145, score-0.487]
40 A completely flat tree over a span is the most general structural description that can be given to it. [sent-147, score-0.298]
41 If an arc structure in one tree merely elaborates an existing flat span in another tree, the theories underlying the schemes are compatible, and their information can be combined. [sent-149, score-0.571]
42 , remove functional nodes, in order to obtain a coherent structure that contains the information on which they agree. [sent-152, score-0.327]
43 Let π1 , π2 be functional trees over the same yield t1, . [sent-153, score-0.529]
44 Let the function span(v) pick out the terminals labeling terminal nodes that are accessible via a node v ∈ V in the functional tree through the 389 relation A. [sent-156, score-0.697]
45 s that a tree π1 is consistent with and more general than tree π2. [sent-160, score-0.284]
46 Looking at the functional trees of (4a)–(4b) we see that their unlabeled skeletons mutually subsume each other. [sent-164, score-0.529]
47 In their labeled versions, however, each tree contains labeling information that is lacking in the other. [sent-165, score-0.252]
48 In the functional trees (5b)–(5c) a flat structure over a span in (5b) is more elaborated in (5c). [sent-166, score-0.685]
49 In order to combine information in trees with compatible arc structures, we define tree unification. [sent-167, score-0.449]
50 In case of an inconsistency, as is the case in the functional trees (6a) and (6b), we cannot unify the structures due to a conflict concerning the internal division of an expression into phrases. [sent-180, score-0.592]
51 However, we still want to generalize these two trees into one tree that contains all and only the information that they share. [sent-181, score-0.344]
52 For every pair of trees there exists a tree that is more general than both: in the extreme case, pick the completely flat structure over the yield, which is more general than any other structure. [sent-198, score-0.405]
53 The generalization of two functional trees provides us with one structure that reflects the common and consistent content of the two trees. [sent-202, score-0.578]
54 These structures thus provide us with a formally well-defined gold standard for cross-treebank evaluation. [sent-203, score-0.26]
55 Our functional trees superficially look like constituency-based trees, so a simple proposal would be to use Parse- ‘vg val measures (Black et al. [sent-205, score-0.529]
56 , 1991) for comparing the parsed trees against the new generalized gold trees. [sent-206, score-0.484]
57 Here we propose to adopt measures that are based on tree edit distance (TED) instead. [sent-211, score-0.344]
58 TEDbased measures are, in fact, an extension of attachment scores for dependency trees. [sent-212, score-0.228]
59 Consider, for instance, the following operations on dependency arcs. [sent-213, score-0.273]
60 3 Here we apply the idea of defining scores by TED costs normalized relative to the size of the tree and substracted from a unity, and extend it from fixed-size dependency trees to ordered trees of arbitrary size. [sent-216, score-0.724]
61 relabel-node change the label of node v in π delete-node delete a non-root node v in π with parent u, making the children of v the children of u, inserted in the place of v as a subsequence in the left-to-right order of the children of u. [sent-220, score-0.221]
62 An optimal edit script is an edit script between π1 and π2 of minimum cost. [sent-229, score-0.468]
63 (π1 cost(e) ,π2 ) The tree edit distance problem is defined to be the problem of finding the optimal edit script and computing the corresponding distance (Bille, 2005). [sent-232, score-0.629]
64 A simple way to calculate the error δ of a parse would be to define it as the edit distance between the parse hypothesis π1 and the gold standard π2. [sent-233, score-0.466]
65 To solve this, we refine the distance between a parse tree and the generalized gold tree to discard edit operations on nodes that are there in the native gold tree but are eliminated through generalization. [sent-235, score-1.225]
66 We compute the intersection of the edit script turning the parse tree into the generalize gold with the edit script turning the native gold tree into the generalized gold, and discard its cost. [sent-236, score-1.186]
67 Different versions of the treebank go into different experiments, resulting in different parse and gold files. [sent-241, score-0.265]
68 The different δ arcs represent the different tree distances used for calculating the TED-based scores. [sent-244, score-0.305]
69 In the worst case, we would have to remove all the internal nodes in the parse tree and add all the internal nodes of the generalized gold, so our normalization factor ι is defined as follows, where |π| is the size4 of π. [sent-250, score-0.408]
70 We start off with two versions of the treebank, TB 1 and TB2, which are parsed separately and provide their own gold standards and parse hypotheses in a labeled dependencies format. [sent-252, score-0.337]
71 All dependency trees 4Following common practice, we equate size |π| with the numFbeorl oofw niongdes c oinm π, discarding ,t whee ete erqmuiantaels s iazned | rπo|o tw intohde th. [sent-253, score-0.38]
72 391 are then converted into functional trees, and we compute the generalization of each pair of gold trees for each sentence in the data. [sent-255, score-0.73]
73 This provides the generalized gold standard for all experiments, here marked as gold3. [sent-256, score-0.226]
74 6 We finally compute the distances δnew (parse1,gold1,gold3) and δnew (parse2,gold2,gold3) using the different tree edit distances that are now available, and we repeat the procedure for each sentence in the test set. [sent-257, score-0.461]
75 |test-set| Alternatively we can globally average of all edit distance costs, normalized by the maximally possible edits on parse trees turned into generalized trees. [sent-261, score-0.534]
76 4 Experiments We demonstrate the application of our procedure to comparing dependency parsing results on different versions of the Penn Treebank (Marcus et al. [sent-266, score-0.371]
77 The Data We use data from the PTB, converted into dependency structures using the LTH software, a general purpose tool for constituency-todependency conversion (Johansson and Nugues, 2007). [sent-268, score-0.375]
78 9267† Table 1: Cross-experiment dependency parsing evaluation for MaltParser trained on multiple schemes. [sent-288, score-0.263]
79 9 41 86 31† Table 2: Cross-experiment dependency parsing evaluation for the MST parser trained on multiple schemes. [sent-306, score-0.324]
80 ,ao2ic0heo7iacx)esti Table 3: LTH conversion schemes used in the experiments. [sent-311, score-0.302]
81 The LTH conversion settings in terms of the complete feature-value pairs associated with the LTH parameters in different schemes are detailed in the supplementary material. [sent-312, score-0.302]
82 392 The Default, OldLTH and CoNLL schemes mainly differ in their coordination structure, and the Functional and Lexical schemes differ in their selection of a functional and a lexical head, respectively. [sent-313, score-0.663]
83 Both parsers were trained on the different instances of sections 2-21 of the PTB obeying the different annotation schemes in Table 3. [sent-319, score-0.303]
84 All non-projective dependencies in the training and gold sets were projectivized prior to training and parsing using the algorithm of Nivre and Nilsson (2005). [sent-321, score-0.237]
85 A more principled treatment of non-projective dependency trees is an important topic for future research. [sent-322, score-0.38]
86 , subjects in one scheme are marked as SUB and in the other marked as SBJ, it is possible define a a set ofzero-cost operation types — in such case, to the operation relabel(SUB,SBJ) — in order not to penalize string label discrepancies. [sent-332, score-0.217]
87 In each of the tables, the top three groups of four rows compare results of parsed dependency trees trained on a particular scheme against gold trees of the same and the other schemes. [sent-346, score-0.8]
88 The next three groups of two rows report the results for comparing pairwise sets of experiments against a generalized gold using our proposed procedure. [sent-347, score-0.282]
89 In the last group of two rows we compare all parsing results against a single gold obtained through a three-way generalization. [sent-348, score-0.237]
90 In fact, when evaluating the effect of linguistically disparate annotation variations such as Lexical and Functional on the performance of MaltParser, Table 1 shows that when using TEDEVAL and a generalized gold the performance gaps are small and statistically insignificant. [sent-355, score-0.503]
91 In evaluating against a three-way generalization, all the results obtained for different training schemes are on a par with one another, with minor gaps in performance, rarely statistically significant. [sent-359, score-0.319]
92 This suggests that apparent performance trends between experiments when evaluating with respect to the training schemes may be misleading. [sent-360, score-0.231]
93 In each of the tables, results obtained against the training schemes show significant differences whereas applying our cross-experimental procedure shows small to no gaps in performance across different schemes. [sent-362, score-0.357]
94 As it turns out, this problem arises in NLP in different shapes and forms; when evaluating a parser against different annotation schemes, when evaluating parsing performance across parsers and different formalisms, and when comparing parser performance across languages. [sent-366, score-0.622]
95 In the future we plan to use this procedure for comparing constituency and dependency parsers. [sent-371, score-0.286]
96 A conversion from constituency-based trees into functional trees 394 is straightforward to define: simply replace the node labels with the grammatical function of their dominating arc and the rest of the pipeline follows. [sent-372, score-1.137]
97 , annotating arcs in dependency trees or decorating nodes in constituency trees. [sent-375, score-0.553]
98 When applied to parsing results of different dependency schemes, dramatic gaps observed when comparing parsing results obtained in isolation decrease or dissolve completely when using our proposed pipeline. [sent-386, score-0.557]
99 Neutralizing linguistically problematic annotations in unsupervised dependency parsing evaluation. [sent-524, score-0.31]
100 Simple fast algorithms for the editing distance between trees and related problems. [sent-541, score-0.253]
wordName wordTfidf (topN-words)
[('functional', 0.327), ('vd', 0.202), ('trees', 0.202), ('dependency', 0.178), ('tedeval', 0.172), ('schemes', 0.168), ('gold', 0.152), ('edit', 0.151), ('tree', 0.142), ('conversion', 0.134), ('sunday', 0.134), ('nivre', 0.121), ('lth', 0.11), ('labelv', 0.108), ('oldlth', 0.108), ('vg', 0.108), ('arc', 0.105), ('arcs', 0.105), ('joakim', 0.101), ('span', 0.095), ('operations', 0.095), ('gaps', 0.088), ('ubler', 0.088), ('node', 0.087), ('labelnt', 0.086), ('parsing', 0.085), ('script', 0.083), ('grammatical', 0.08), ('annotation', 0.079), ('protocol', 0.074), ('generalized', 0.074), ('terminal', 0.073), ('sandra', 0.073), ('las', 0.073), ('head', 0.071), ('nodes', 0.068), ('worked', 0.067), ('scheme', 0.066), ('labeled', 0.066), ('formal', 0.065), ('buyko', 0.065), ('dissolve', 0.065), ('labelt', 0.065), ('ad', 0.064), ('structures', 0.063), ('evaluating', 0.063), ('standards', 0.063), ('conj', 0.063), ('parseval', 0.062), ('vj', 0.062), ('parser', 0.061), ('heads', 0.061), ('flat', 0.061), ('maltparser', 0.06), ('es', 0.059), ('briscoe', 0.058), ('johansson', 0.058), ('distances', 0.058), ('treebank', 0.057), ('parsers', 0.056), ('comparing', 0.056), ('cl', 0.056), ('ted', 0.056), ('parse', 0.056), ('daughter', 0.056), ('neutralizing', 0.056), ('operation', 0.052), ('procedure', 0.052), ('distance', 0.051), ('attachment', 0.05), ('wildcard', 0.05), ('assumptions', 0.05), ('generalization', 0.049), ('across', 0.049), ('deterministic', 0.048), ('label', 0.047), ('earth', 0.047), ('discrepancies', 0.047), ('schwartz', 0.047), ('linguistically', 0.047), ('treebanks', 0.045), ('formally', 0.045), ('mcdonald', 0.045), ('carroll', 0.045), ('cost', 0.044), ('lacking', 0.044), ('marneffe', 0.043), ('aed', 0.043), ('conversions', 0.043), ('designating', 0.043), ('endeavor', 0.043), ('havvge', 0.043), ('heike', 0.043), ('hinrichs', 0.043), ('labela', 0.043), ('nnotation', 0.043), ('periphrastic', 0.043), ('relabel', 0.043), ('tgraolidndefaultold', 0.043), ('thoughtthrough', 0.043)]
simIndex simValue paperId paperTitle
same-paper 1 0.99999869 50 emnlp-2011-Evaluating Dependency Parsing: Robust and Heuristics-Free Cross-Annotation Evaluation
Author: Reut Tsarfaty ; Joakim Nivre ; Evelina Andersson
Abstract: unkown-abstract
2 0.28880516 4 emnlp-2011-A Fast, Accurate, Non-Projective, Semantically-Enriched Parser
Author: Stephen Tratz ; Eduard Hovy
Abstract: Dependency parsers are critical components within many NLP systems. However, currently available dependency parsers each exhibit at least one of several weaknesses, including high running time, limited accuracy, vague dependency labels, and lack of nonprojectivity support. Furthermore, no commonly used parser provides additional shallow semantic interpretation, such as preposition sense disambiguation and noun compound interpretation. In this paper, we present a new dependency-tree conversion of the Penn Treebank along with its associated fine-grain dependency labels and a fast, accurate parser trained on it. We explain how a non-projective extension to shift-reduce parsing can be incorporated into non-directional easy-first parsing. The parser performs well when evaluated on the standard test section of the Penn Treebank, outperforming several popular open source dependency parsers; it is, to the best of our knowledge, the first dependency parser capable of parsing more than 75 sentences per second at over 93% accuracy.
3 0.15119846 103 emnlp-2011-Parser Evaluation over Local and Non-Local Deep Dependencies in a Large Corpus
Author: Emily M. Bender ; Dan Flickinger ; Stephan Oepen ; Yi Zhang
Abstract: In order to obtain a fine-grained evaluation of parser accuracy over naturally occurring text, we study 100 examples each of ten reasonably frequent linguistic phenomena, randomly selected from a parsed version of the English Wikipedia. We construct a corresponding set of gold-standard target dependencies for these 1000 sentences, operationalize mappings to these targets from seven state-of-theart parsers, and evaluate the parsers against this data to measure their level of success in identifying these dependencies.
4 0.15108024 108 emnlp-2011-Quasi-Synchronous Phrase Dependency Grammars for Machine Translation
Author: Kevin Gimpel ; Noah A. Smith
Abstract: We present a quasi-synchronous dependency grammar (Smith and Eisner, 2006) for machine translation in which the leaves of the tree are phrases rather than words as in previous work (Gimpel and Smith, 2009). This formulation allows us to combine structural components of phrase-based and syntax-based MT in a single model. We describe a method of extracting phrase dependencies from parallel text using a target-side dependency parser. For decoding, we describe a coarse-to-fine approach based on lattice dependency parsing of phrase lattices. We demonstrate performance improvements for Chinese-English and UrduEnglish translation over a phrase-based baseline. We also investigate the use of unsupervised dependency parsers, reporting encouraging preliminary results.
5 0.14593878 137 emnlp-2011-Training dependency parsers by jointly optimizing multiple objectives
Author: Keith Hall ; Ryan McDonald ; Jason Katz-Brown ; Michael Ringgaard
Abstract: We present an online learning algorithm for training parsers which allows for the inclusion of multiple objective functions. The primary example is the extension of a standard supervised parsing objective function with additional loss-functions, either based on intrinsic parsing quality or task-specific extrinsic measures of quality. Our empirical results show how this approach performs for two dependency parsing algorithms (graph-based and transition-based parsing) and how it achieves increased performance on multiple target tasks including reordering for machine translation and parser adaptation.
6 0.14337239 15 emnlp-2011-A novel dependency-to-string model for statistical machine translation
7 0.13964951 102 emnlp-2011-Parse Correction with Specialized Models for Difficult Attachment Types
8 0.13885891 141 emnlp-2011-Unsupervised Dependency Parsing without Gold Part-of-Speech Tags
9 0.12559542 95 emnlp-2011-Multi-Source Transfer of Delexicalized Dependency Parsers
10 0.11709438 75 emnlp-2011-Joint Models for Chinese POS Tagging and Dependency Parsing
11 0.1052681 136 emnlp-2011-Training a Parser for Machine Translation Reordering
12 0.10095574 131 emnlp-2011-Syntactic Decision Tree LMs: Random Selection or Intelligent Design?
13 0.097883195 52 emnlp-2011-Exact Inference for Generative Probabilistic Non-Projective Dependency Parsing
14 0.093755059 74 emnlp-2011-Inducing Sentence Structure from Parallel Corpora for Reordering
15 0.09249384 127 emnlp-2011-Structured Lexical Similarity via Convolution Kernels on Dependency Trees
16 0.088117473 96 emnlp-2011-Multilayer Sequence Labeling
17 0.083997689 145 emnlp-2011-Unsupervised Semantic Role Induction with Graph Partitioning
18 0.08213374 10 emnlp-2011-A Probabilistic Forest-to-String Model for Language Generation from Typed Lambda Calculus Expressions
19 0.075125791 16 emnlp-2011-Accurate Parsing with Compact Tree-Substitution Grammars: Double-DOP
20 0.072810858 115 emnlp-2011-Relaxed Cross-lingual Projection of Constituent Syntax
topicId topicWeight
[(0, 0.299), (1, 0.107), (2, -0.066), (3, 0.285), (4, -0.028), (5, 0.093), (6, -0.018), (7, -0.04), (8, 0.165), (9, -0.003), (10, 0.017), (11, -0.074), (12, 0.07), (13, 0.086), (14, 0.044), (15, 0.065), (16, 0.061), (17, 0.0), (18, 0.063), (19, -0.092), (20, 0.156), (21, 0.042), (22, -0.033), (23, 0.024), (24, -0.102), (25, -0.12), (26, -0.076), (27, 0.174), (28, -0.003), (29, -0.06), (30, 0.03), (31, -0.036), (32, -0.008), (33, -0.194), (34, 0.044), (35, -0.135), (36, 0.024), (37, 0.017), (38, -0.093), (39, 0.091), (40, 0.074), (41, 0.109), (42, 0.044), (43, -0.031), (44, 0.066), (45, 0.059), (46, 0.085), (47, -0.012), (48, 0.019), (49, 0.024)]
simIndex simValue paperId paperTitle
same-paper 1 0.97468078 50 emnlp-2011-Evaluating Dependency Parsing: Robust and Heuristics-Free Cross-Annotation Evaluation
Author: Reut Tsarfaty ; Joakim Nivre ; Evelina Andersson
Abstract: unkown-abstract
2 0.80833763 4 emnlp-2011-A Fast, Accurate, Non-Projective, Semantically-Enriched Parser
Author: Stephen Tratz ; Eduard Hovy
Abstract: Dependency parsers are critical components within many NLP systems. However, currently available dependency parsers each exhibit at least one of several weaknesses, including high running time, limited accuracy, vague dependency labels, and lack of nonprojectivity support. Furthermore, no commonly used parser provides additional shallow semantic interpretation, such as preposition sense disambiguation and noun compound interpretation. In this paper, we present a new dependency-tree conversion of the Penn Treebank along with its associated fine-grain dependency labels and a fast, accurate parser trained on it. We explain how a non-projective extension to shift-reduce parsing can be incorporated into non-directional easy-first parsing. The parser performs well when evaluated on the standard test section of the Penn Treebank, outperforming several popular open source dependency parsers; it is, to the best of our knowledge, the first dependency parser capable of parsing more than 75 sentences per second at over 93% accuracy.
3 0.71819884 103 emnlp-2011-Parser Evaluation over Local and Non-Local Deep Dependencies in a Large Corpus
Author: Emily M. Bender ; Dan Flickinger ; Stephan Oepen ; Yi Zhang
Abstract: In order to obtain a fine-grained evaluation of parser accuracy over naturally occurring text, we study 100 examples each of ten reasonably frequent linguistic phenomena, randomly selected from a parsed version of the English Wikipedia. We construct a corresponding set of gold-standard target dependencies for these 1000 sentences, operationalize mappings to these targets from seven state-of-theart parsers, and evaluate the parsers against this data to measure their level of success in identifying these dependencies.
4 0.64296103 102 emnlp-2011-Parse Correction with Specialized Models for Difficult Attachment Types
Author: Enrique Henestroza Anguiano ; Marie Candito
Abstract: This paper develops a framework for syntactic dependency parse correction. Dependencies in an input parse tree are revised by selecting, for a given dependent, the best governor from within a small set of candidates. We use a discriminative linear ranking model to select the best governor from a group of candidates for a dependent, and our model includes a rich feature set that encodes syntactic structure in the input parse tree. The parse correction framework is parser-agnostic, and can correct attachments using either a generic model or specialized models tailored to difficult attachment types like coordination and pp-attachment. Our experiments show that parse correction, combining a generic model with specialized models for difficult attachment types, can successfully improve the quality of predicted parse trees output by sev- eral representative state-of-the-art dependency parsers for French.
5 0.5253424 137 emnlp-2011-Training dependency parsers by jointly optimizing multiple objectives
Author: Keith Hall ; Ryan McDonald ; Jason Katz-Brown ; Michael Ringgaard
Abstract: We present an online learning algorithm for training parsers which allows for the inclusion of multiple objective functions. The primary example is the extension of a standard supervised parsing objective function with additional loss-functions, either based on intrinsic parsing quality or task-specific extrinsic measures of quality. Our empirical results show how this approach performs for two dependency parsing algorithms (graph-based and transition-based parsing) and how it achieves increased performance on multiple target tasks including reordering for machine translation and parser adaptation.
6 0.52111381 52 emnlp-2011-Exact Inference for Generative Probabilistic Non-Projective Dependency Parsing
7 0.51198119 108 emnlp-2011-Quasi-Synchronous Phrase Dependency Grammars for Machine Translation
8 0.47617736 47 emnlp-2011-Efficient retrieval of tree translation examples for Syntax-Based Machine Translation
9 0.47246391 15 emnlp-2011-A novel dependency-to-string model for statistical machine translation
10 0.46505046 131 emnlp-2011-Syntactic Decision Tree LMs: Random Selection or Intelligent Design?
11 0.45990908 141 emnlp-2011-Unsupervised Dependency Parsing without Gold Part-of-Speech Tags
12 0.41471913 95 emnlp-2011-Multi-Source Transfer of Delexicalized Dependency Parsers
13 0.40939429 16 emnlp-2011-Accurate Parsing with Compact Tree-Substitution Grammars: Double-DOP
14 0.40368772 127 emnlp-2011-Structured Lexical Similarity via Convolution Kernels on Dependency Trees
15 0.38739315 75 emnlp-2011-Joint Models for Chinese POS Tagging and Dependency Parsing
16 0.36326784 85 emnlp-2011-Learning to Simplify Sentences with Quasi-Synchronous Grammar and Integer Programming
17 0.35218626 2 emnlp-2011-A Cascaded Classification Approach to Semantic Head Recognition
18 0.34779325 134 emnlp-2011-Third-order Variational Reranking on Packed-Shared Dependency Forests
19 0.33780664 136 emnlp-2011-Training a Parser for Machine Translation Reordering
20 0.33415762 74 emnlp-2011-Inducing Sentence Structure from Parallel Corpora for Reordering
topicId topicWeight
[(23, 0.08), (36, 0.037), (37, 0.025), (45, 0.064), (53, 0.011), (54, 0.025), (57, 0.011), (62, 0.019), (64, 0.025), (66, 0.033), (69, 0.011), (79, 0.035), (82, 0.02), (87, 0.011), (90, 0.411), (96, 0.097), (98, 0.026)]
simIndex simValue paperId paperTitle
same-paper 1 0.83849019 50 emnlp-2011-Evaluating Dependency Parsing: Robust and Heuristics-Free Cross-Annotation Evaluation
Author: Reut Tsarfaty ; Joakim Nivre ; Evelina Andersson
Abstract: unkown-abstract
2 0.83529812 16 emnlp-2011-Accurate Parsing with Compact Tree-Substitution Grammars: Double-DOP
Author: Federico Sangati ; Willem Zuidema
Abstract: We present a novel approach to Data-Oriented Parsing (DOP). Like other DOP models, our parser utilizes syntactic fragments of arbitrary size from a treebank to analyze new sentences, but, crucially, it uses only those which are encountered at least twice. This criterion allows us to work with a relatively small but representative set of fragments, which can be employed as the symbolic backbone of several probabilistic generative models. For parsing we define a transform-backtransform approach that allows us to use standard PCFG technology, making our results easily replicable. According to standard Parseval metrics, our best model is on par with many state-ofthe-art parsers, while offering some complementary benefits: a simple generative probability model, and an explicit representation of the larger units of grammar.
3 0.43263319 127 emnlp-2011-Structured Lexical Similarity via Convolution Kernels on Dependency Trees
Author: Danilo Croce ; Alessandro Moschitti ; Roberto Basili
Abstract: Alessandro Moschitti DISI University of Trento 38123 Povo (TN), Italy mo s chitt i di s i @ .unit n . it Roberto Basili DII University of Tor Vergata 00133 Roma, Italy bas i i info .uni roma2 . it l@ over semantic networks, e.g. (Cowie et al., 1992; Wu and Palmer, 1994; Resnik, 1995; Jiang and Conrath, A central topic in natural language processing is the design of lexical and syntactic fea- tures suitable for the target application. In this paper, we study convolution dependency tree kernels for automatic engineering of syntactic and semantic patterns exploiting lexical similarities. We define efficient and powerful kernels for measuring the similarity between dependency structures, whose surface forms of the lexical nodes are in part or completely different. The experiments with such kernels for question classification show an unprecedented results, e.g. 41% of error reduction of the former state-of-the-art. Additionally, semantic role classification confirms the benefit of semantic smoothing for dependency kernels.
Author: Spence Green ; Marie-Catherine de Marneffe ; John Bauer ; Christopher D. Manning
Abstract: Multiword expressions (MWE), a known nuisance for both linguistics and NLP, blur the lines between syntax and semantics. Previous work on MWE identification has relied primarily on surface statistics, which perform poorly for longer MWEs and cannot model discontinuous expressions. To address these problems, we show that even the simplest parsing models can effectively identify MWEs of arbitrary length, and that Tree Substitution Grammars achieve the best results. Our experiments show a 36.4% F1 absolute improvement for French over an n-gram surface statistics baseline, currently the predominant method for MWE identification. Our models are useful for several NLP tasks in which MWE pre-grouping has improved accuracy. 1
5 0.39422402 111 emnlp-2011-Reducing Grounded Learning Tasks To Grammatical Inference
Author: Benjamin Borschinger ; Bevan K. Jones ; Mark Johnson
Abstract: It is often assumed that ‘grounded’ learning tasks are beyond the scope of grammatical inference techniques. In this paper, we show that the grounded task of learning a semantic parser from ambiguous training data as discussed in Kim and Mooney (2010) can be reduced to a Probabilistic Context-Free Grammar learning task in a way that gives state of the art results. We further show that additionally letting our model learn the language’s canonical word order improves its performance and leads to the highest semantic parsing f-scores previously reported in the literature.1
6 0.39410472 47 emnlp-2011-Efficient retrieval of tree translation examples for Syntax-Based Machine Translation
7 0.38169798 134 emnlp-2011-Third-order Variational Reranking on Packed-Shared Dependency Forests
8 0.38141486 15 emnlp-2011-A novel dependency-to-string model for statistical machine translation
9 0.37943199 85 emnlp-2011-Learning to Simplify Sentences with Quasi-Synchronous Grammar and Integer Programming
10 0.37787354 53 emnlp-2011-Experimental Support for a Categorical Compositional Distributional Model of Meaning
11 0.37495285 108 emnlp-2011-Quasi-Synchronous Phrase Dependency Grammars for Machine Translation
12 0.36706379 4 emnlp-2011-A Fast, Accurate, Non-Projective, Semantically-Enriched Parser
13 0.36391196 147 emnlp-2011-Using Syntactic and Semantic Structural Kernels for Classifying Definition Questions in Jeopardy!
14 0.35994554 54 emnlp-2011-Exploiting Parse Structures for Native Language Identification
15 0.35766074 79 emnlp-2011-Lateen EM: Unsupervised Training with Multiple Objectives, Applied to Dependency Grammar Induction
16 0.35708499 103 emnlp-2011-Parser Evaluation over Local and Non-Local Deep Dependencies in a Large Corpus
17 0.35390082 136 emnlp-2011-Training a Parser for Machine Translation Reordering
18 0.35228291 59 emnlp-2011-Fast and Robust Joint Models for Biomedical Event Extraction
19 0.35094076 112 emnlp-2011-Refining the Notions of Depth and Density in WordNet-based Semantic Similarity Measures
20 0.35014305 1 emnlp-2011-A Bayesian Mixture Model for PoS Induction Using Multiple Features