emnlp emnlp2011 emnlp2011-136 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Jason Katz-Brown ; Slav Petrov ; Ryan McDonald ; Franz Och ; David Talbot ; Hiroshi Ichikawa ; Masakazu Seno ; Hideto Kazawa
Abstract: We propose a simple training regime that can improve the extrinsic performance of a parser, given only a corpus of sentences and a way to automatically evaluate the extrinsic quality of a candidate parse. We apply our method to train parsers that excel when used as part of a reordering component in a statistical machine translation system. We use a corpus of weakly-labeled reference reorderings to guide parser training. Our best parsers contribute significant improvements in subjective translation quality while their intrinsic attachment scores typically regress.
Reference: text
sentIndex sentText sentNum sentScore
1 We apply our method to train parsers that excel when used as part of a reordering component in a statistical machine translation system. [sent-2, score-0.894]
2 We use a corpus of weakly-labeled reference reorderings to guide parser training. [sent-3, score-0.419]
3 Our best parsers contribute significant improvements in subjective translation quality while their intrinsic attachment scores typically regress. [sent-4, score-0.475]
4 All to the contrary, as parsing technology has become more mature, parsers have become efficient and accurate enough to be useful in many natural language processing systems, most notably in machine translation (Yamada and Knight, 2001 ; Galley et al. [sent-16, score-0.283]
5 While it has been repeatedly shown that using a parser can bring net gains on downstream application quality, it is often unclear how much intrinsic parsing accuracy actually matters. [sent-19, score-0.368]
6 We present experiments on translation from English to three Subject-ObjectVerb (SOV) languages,1 because those require extensive syntactic reordering to produce grammatical translations. [sent-21, score-0.695]
7 We evaluate parse quality on a number of extrinsic metrics, including word reordering accuracy, BLEU score and a human evaluation of final translation quality. [sent-22, score-1.084]
8 We show that while there is a good correlation between those extrinsic metrics, parsing quality as measured on the Penn Treebank is not a good indicator of the final downstream application quality. [sent-23, score-0.381]
9 Since the word reordering metric can be computed efficiently offline (i. [sent-24, score-0.7]
10 ec th2o0d1s1 i Ans Nsoactuiartaioln La fonrg Cuaogmep Purtoatcieosnsainlg L,in pgaugies ti 1c8s3–192, which we refer to as targeted self-training (Section 2). [sent-30, score-0.339]
11 The experiments in Section 5 show that this simple procedure noticeably improves our parsers for the task at hand, resulting in significant improvements in downstream translation quality, as measured in a human evaluation on web text. [sent-34, score-0.407]
12 (2010), except that we use an extrinsic quality metric instead of a second parsing model for making the selection. [sent-38, score-0.364]
13 It should be noted that our extrinsic metric is com- puted from data that has been manually annotated with reference word reorderings. [sent-41, score-0.355]
14 Details of the reordering metric and the annotated data we used are given in Sections 3 and 4. [sent-42, score-0.7]
15 In our experiments in Section 6 we show that we can obtain similar improvements on downstream translation quality by targeted self-training with weakly labeled data (in form of word reorderings), as with training on the fully labeled data (with full syntactic parse trees). [sent-44, score-0.749]
16 In standard parser selftraining, one uses the baseline parsing model to parse a corpus of sentences, and then adds the 1-best output of the baseline parser to the training data. [sent-46, score-0.561]
17 Instead of taking the 1best parse, we produce a ranked n-best list of predictions and select the parser which gives the best score according to an external evaluation function. [sent-48, score-0.277]
18 That is, instead of relying on the intrinsic model score, we use an extrinsic score to select the parse towards 184 Algorithm 1 Select parse that maximizes an extrinsic metric. [sent-49, score-0.671]
19 Input: baseline parser B Input: sentence S Input: function COMPUTEEXTRINSIC(parse P) Output: a parse for the input sentence Pn = {P1, . [sent-50, score-0.311]
20 The motivation of this selection step is that good performance on the downstream external task, measured by the extrinsic metric, should be predictive of an intrinsically good parse. [sent-55, score-0.363]
21 At the very least, even if the selected parse is not syntactically correct, or even if it goes against the original treebanking guidelines, it results in a higher extrinsic score and should therefore be preferred. [sent-56, score-0.33]
22 In particular, we explore one application of targeted self-training, where computing the extrinsic metric involves plugging the parse into an MT system’s reordering component and computing the accuracy of the reordering compared to a reference word order. [sent-61, score-2.038]
23 Doing reordering as a preprocessing step, separately from translation, makes it easy to evaluate reordering performance independently from the MT system. [sent-73, score-1.303]
24 (201 1) present a framework for evaluating the quality of reordering separately from the lexical choice involved in translation. [sent-75, score-0.709]
25 They propose a simple reordering metric based on METEOR’s reordering penalty (Lavie and Denkowski, 2009). [sent-76, score-1.322]
26 To compute it, one takes the candidate reordering of the input sentence and partitions it into a set C of contiguous spans wanhdos pea crtointiotennst appears contiguously ninti tghueo same order in the reference. [sent-78, score-0.654]
27 The reordering score is then computed as ρ(esys,eref) = 1 −||eC|| − − 11. [sent-79, score-0.664]
28 This metric assigns a score between 0 and 1where 1 185 indicates that the candidate reordering is identical to the reference and 0 indicates that no two words that are contiguous in the candidate reordering are contiguous in the reference. [sent-80, score-1.517]
29 For example, if a reference reordering is A B C D E, candidate reordering A B E C D would get score 1−(3−1)/(5−1) = 0. [sent-81, score-1.375]
30 (201 1) show that this reordering score is strongly correlated with human judgment of translation quality. [sent-84, score-0.737]
31 Furthermore, they propose to evaluate the reordering quality of an MT system by computing its reordering score on a test set consisting of source language sentences and their reference reorderings. [sent-85, score-1.434]
32 We describe in more detail how the reference reordering data was prepared in Section 4. [sent-87, score-0.711]
33 2 Reordering quality as predictor of parse quality Figure 1 gives concrete examples of good and bad reorderings of an English sentence into Japanese word order. [sent-90, score-0.422]
34 It shows that a bad parse leads to a bad reordering (lacking inversion of verb “wear” and object “sunscreen”) and a low reordering score. [sent-91, score-1.454]
35 Could we flip this causality around, and perhaps try to identify a good parse tree based on its reordering score? [sent-92, score-0.75]
36 With the experiments in this paper, we show that indeed a high reordering score is predictive of the underlying parse tree that was used to generate the reordering being a good parse (or, at least, being good enough for our purpose). [sent-93, score-1.54]
37 In the case of translating English to Japanese or another SOV language, there is a large amount of reordering required, but with a relatively small number of reordering rules one can cover a large proportion of reordering phenomena. [sent-94, score-1.891]
38 (2010b), for instance, were able to get impressive English→Japanese results with only a single reordering rule, given a ssuulittsab wlei dhef oinniltyio na o sifn a lheea red-. [sent-96, score-0.622]
39 Hence, the reordering task depends crucially on a correct syntactic analysis and is extremely sensitive to parser errors. [sent-97, score-0.803]
40 Though we experiment with both dependency parsers and phrase structure parsers, our MT system assumes dependency parses as input. [sent-105, score-0.304]
41 , 2006) to convert phrase structure parse trees to dependency parse trees (for both treebank trees and predicted trees). [sent-107, score-0.447]
42 2 Reference reordering data We aim to build an MT system that can accurately translate typical English text that one finds on the Internet to SOV langauges. [sent-109, score-0.622]
43 To this end, we randomly sampled 13595 English sentences from the web and created Japanese-word-order reference reorderings for them. [sent-110, score-0.275]
44 The good parse correctly identifies “Wear” as the main verb and moves it to the end of the sentence; the bad parse analyses “Wear sunscreen” as a noun phrase and does not reorder it. [sent-120, score-0.296]
45 3 Parsers The core dependency parser we use is an implementation of a transition-based dependency parser using an arc-eager transition strategy (Nivre, 2008). [sent-124, score-0.458]
46 In addition to general uppercase/lowercase noisiness of the web text negatively impacting scores, we found that the baseline casesensitive parsers are especially bad at parsing imperative sentences, as discussed in Section 5. [sent-139, score-0.439]
47 We use a superset of the reordering rules proposed by Xu et al. [sent-144, score-0.622]
48 Because the reordering accuracy metric can be computed for any source/target language pair, this would have made our approach language completely independent and applicable to any language pair. [sent-149, score-0.7]
49 In addition to the regular distance distortion model, we incorporate a maximum entropy based lexicalized phrase reordering model (Zens and Ney, 2006) as a feature used in decoding. [sent-155, score-0.622]
50 This potentially underestimates the improvements that can be obtained, but also eliminates MERT as a possible source of improvement, allowing us to trace back improvements in translation quality directly to parser changes. [sent-158, score-0.375]
51 Targeted self-training: trained on WSJ-Train and, for each sentence in Web-Train, the parse from the baseline parser’s 512-best list that when reordered gives the highest reordering score. [sent-167, score-0.809]
52 1 Standard self-training vs targeted self-training Table 1 shows that targeted self-training on WebTrain significantly improves Web-Test reordering score more than standard self-training for both the shift-reduce parser and for the BerkeleyParser. [sent-169, score-1.523]
53 Bolded reordering scores are different from WSJ-only baseline with confidence but are not significantly different from each other within the same group. [sent-181, score-0.652]
54 “WSJ-only” corresponds sponds to the Web-Train targeted self-training to the baseline WSJ-only shift-reduce parser; “Targeted” corre- 10x shift-reduce parser. [sent-189, score-0.369]
55 Interestingly, weighting the selftraining data more seems to have a negative effect on both One explanation for the drops in LAS is that some parts of the parse tree are important for downstream reordering quality while others are not (or only to a lesser extent). [sent-191, score-0.95]
56 Some distinctions between labels become less important; for example, arcs labeled “amod” and “advmod” are transformed identically by the reordering rules. [sent-192, score-0.674]
57 We then trained MT systems varying only the parser used for reordering in training and decoding. [sent-199, score-0.803]
58 Table 2 shows that targeted self-training data increases BLEU score for translation into all three languages. [sent-200, score-0.454]
59 We show how our method improves parsing performance and reordering performance on two examples: the trendy word “click” and imperative sentences. [sent-209, score-0.768]
60 The WSJ-only shiftreduce parser refuses to tag “click” as a verb even with targeted self-training, but BerkeleyParser does learn to tag “click” more often as a verb. [sent-215, score-0.52]
61 This causes the parser’s n-best lists to never hypothesize “click” as a verb, and self-training doesn’t click no matter how targeted it is. [sent-218, score-0.482]
62 This shows that the targeted selftraining approach heavily relies on the diversity of the baseline parser’s n-best lists. [sent-219, score-0.415]
63 The idea would be to use the BerkeleyParser to generate the n-best lists; perhaps we could call this targeted uptraining. [sent-222, score-0.339]
64 This way, the shift-reduce parser could benefit both from the generally higher quality of the parse trees produced by the BerkeleyParser, as well as from the information provided by the extrinsic scoring function. [sent-223, score-0.57]
65 As a result, parsers trained exclusively on the WSJ underproduce imperative parses, especially a case-sensitive version of the baseline. [sent-229, score-0.278]
66 Targeted self-training helps the parsers to predict imperative parses more often. [sent-230, score-0.315]
67 It is thus especially easy to know whether an imperative parse is correct or not by looking at the reference reordering. [sent-232, score-0.296]
68 Figure 1 gives an example: the bad (WSJ-only) parse doesn’t catch on to the imperativeness and gets a low reordering score. [sent-233, score-0.777]
69 6 Targeted Self-Training vs Training on Treebanks for Domain Adaptation If task-specific annotation is cheap, then it is reasonable to consider whether we could use targeted self-training to adapt a parser to a new domain as a cheaper alternative to making new treebanks. [sent-234, score-0.549]
70 Create reference reorderings for 2000 questions and then do targeted self-training. [sent-239, score-0.577]
71 To compare these approaches, we created reference reordering data for our train (2000 sentences) and test (1000 sentences) splits of the Question Treebank (Judge et al. [sent-240, score-0.739]
72 Table 4 shows that both ways of training on QTB-Train sentences give similarly large improvements in reordering score on QTB-Test. [sent-242, score-0.695]
73 12, while the targeted self-training translations received a score of 2. [sent-245, score-0.413]
74 In the shift-reduce parser, targeted self-training gives higher reordering scores than training on the treebank, and in BerkeleyParser, the opposite is true. [sent-248, score-0.961]
75 From a practical perspective, the advantage of targeted self-training depends on whether the extrinsic metric is cheaper to calculate than treebanking. [sent-250, score-0.634]
76 For MT reordering, making reference reorderings is cheap, so targeted self-training is relatively advantageous. [sent-251, score-0.577]
77 As before, we can examine whether labeled attachment score measured on the test set of the QTB is predictive of reordering quality. [sent-252, score-0.774]
78 Table 4 shows that targeted self-training raises LAS from 64. [sent-253, score-0.339]
79 We can conclude that high LAS is not necessary to achieve top reordering scores. [sent-258, score-0.622]
80 Perhaps our reordering rules are somehow deficient when it comes to reordering correctly-parsed questions, and as a result the targeted self-training process steers the parser towards producing pathological trees with little intrinsic meaning. [sent-259, score-1.859]
81 To explore this possibility, we computed reordering scores after reordering the QTB-Test treebank trees directly. [sent-260, score-1.359]
82 Table 4 shows that this gives reordering scores similar to those of our best parsers. [sent-261, score-0.622]
83 Therefore it is at least 190 possible that the targeted self-training process could have resulted in a parser that achieves high reordering score by producing parses that look like those in the QuestionBank. [sent-262, score-1.221]
84 7 Related Work Our approach to training parsers for reordering is closely related to self/up-training (McClosky. [sent-263, score-0.793]
85 (2006), except that we use an extrinsic metric (MT reordering score) to select a high quality parse tree, rather than a second, reranking model that has access to additional features. [sent-270, score-1.047]
86 (2010) in which they jointly parse unannotated bilingual text using a multiview learning objective, then retrain the monolingual parser models to include each side of the jointly parsed bitext as monolingual training data. [sent-272, score-0.335]
87 Our approach is different in that it doesn’t use a second parser and bitext to guide the creation of new training data, and instead relies on n-best lists and an extrinsic metric. [sent-273, score-0.395]
88 In our case this would mean using the reordering metric as a means to train better dependency parsers. [sent-285, score-0.776]
89 Our objective, on the other hand, is to use the extrinsic metric to train parsers that are specifically better at the reordering task, and, as a result, better suited for MT. [sent-286, score-1.087]
90 tl)corpus,varying Table 4: Reordering and labeled attachment scores on QTB-Test for treebank training and targeted self-training on QTB-Train. [sent-298, score-0.496]
91 only et5he: parser bsceotwreeseann systems beevtawlueaetnio tnhere WSJ-only nsghliifsth-r→edJuacpea parser aannsdla tthioen nQoTfBth-eTQraTinB targeted self-training 10x shift-reduce parser. [sent-299, score-0.701]
92 (201 1) who train a perceptron algorithm on multiple objective functions with the goal of producing parsers that are optimized for extrinsic metrics. [sent-302, score-0.438]
93 One could apply the targeted self-training technique we describe to optimize any of these parsers for the PPI task, similar to how we have optimized our parser for the MT reordering task. [sent-307, score-1.338]
94 8 Conclusion We introduced a variant of self-training that targets parser training towards an extrinsic evaluation metric. [sent-308, score-0.369]
95 We use this targeted self-training approach to train parsers that improve the accuracy of the word 191 reordering component of a machine translation system. [sent-309, score-1.233]
96 While the new parsers give improvements in these external evaluations, their intrinsic attachment scores go down overall compared to baseline parsers trained only on treebanks. [sent-311, score-0.568]
97 We conclude that when using a parser as a component of a larger external system, it can be advantageous to incorporate an extrinsic metric into parser training and evaluation, and that targeted self-training is an effective technique for incorporating an extrinsic metric into parser training. [sent-312, score-1.468]
98 LRscore for evaluating lexical and reordering quality in MT. [sent-317, score-0.681]
99 Automatically learning source-side reordering rules for large scale machine translation. [sent-410, score-0.622]
100 Head finalization: A simple reordering rule for SOV languages. [sent-445, score-0.622]
wordName wordTfidf (topN-words)
[('reordering', 0.622), ('targeted', 0.339), ('extrinsic', 0.188), ('parser', 0.181), ('parsers', 0.171), ('berkeleyparser', 0.163), ('reorderings', 0.149), ('click', 0.143), ('japanese', 0.116), ('imperative', 0.107), ('parse', 0.1), ('las', 0.1), ('sov', 0.1), ('mt', 0.099), ('downstream', 0.095), ('petrov', 0.093), ('reference', 0.089), ('metric', 0.078), ('talbot', 0.076), ('treebank', 0.073), ('translation', 0.073), ('bleu', 0.065), ('burkett', 0.064), ('bestparse', 0.063), ('extrinsicscore', 0.063), ('tnt', 0.063), ('stack', 0.062), ('quality', 0.059), ('attachment', 0.058), ('isozaki', 0.057), ('reordered', 0.057), ('bad', 0.055), ('wear', 0.054), ('ppi', 0.054), ('external', 0.054), ('intrinsic', 0.053), ('dependency', 0.048), ('selftraining', 0.046), ('korean', 0.046), ('uptraining', 0.046), ('mert', 0.044), ('score', 0.042), ('computeextrinsic', 0.042), ('headverb', 0.042), ('maxscore', 0.042), ('regime', 0.042), ('sunscreen', 0.042), ('chang', 0.042), ('trees', 0.042), ('reorder', 0.041), ('parsing', 0.039), ('lavie', 0.039), ('web', 0.037), ('parses', 0.037), ('sudoh', 0.036), ('denkowski', 0.036), ('ichikawa', 0.036), ('qtb', 0.036), ('seno', 0.036), ('wsj', 0.036), ('precedence', 0.033), ('genzel', 0.033), ('kazawa', 0.033), ('english', 0.033), ('contiguous', 0.032), ('translations', 0.032), ('preprocessing', 0.031), ('improvements', 0.031), ('doesn', 0.031), ('identities', 0.03), ('meteor', 0.03), ('goldwasser', 0.03), ('xu', 0.03), ('judge', 0.03), ('subjective', 0.03), ('baseline', 0.03), ('mcdonald', 0.029), ('penn', 0.029), ('retraining', 0.029), ('cheaper', 0.029), ('cheap', 0.029), ('buffer', 0.029), ('signals', 0.029), ('ringgaard', 0.029), ('train', 0.028), ('tree', 0.028), ('separately', 0.028), ('bilingual', 0.028), ('objective', 0.026), ('marcus', 0.026), ('predictive', 0.026), ('labeled', 0.026), ('distinctions', 0.026), ('pk', 0.026), ('xia', 0.026), ('miyao', 0.026), ('bitext', 0.026), ('ganchev', 0.026), ('translating', 0.025), ('optimized', 0.025)]
simIndex simValue paperId paperTitle
same-paper 1 0.99999851 136 emnlp-2011-Training a Parser for Machine Translation Reordering
Author: Jason Katz-Brown ; Slav Petrov ; Ryan McDonald ; Franz Och ; David Talbot ; Hiroshi Ichikawa ; Masakazu Seno ; Hideto Kazawa
Abstract: We propose a simple training regime that can improve the extrinsic performance of a parser, given only a corpus of sentences and a way to automatically evaluate the extrinsic quality of a candidate parse. We apply our method to train parsers that excel when used as part of a reordering component in a statistical machine translation system. We use a corpus of weakly-labeled reference reorderings to guide parser training. Our best parsers contribute significant improvements in subjective translation quality while their intrinsic attachment scores typically regress.
2 0.44558531 13 emnlp-2011-A Word Reordering Model for Improved Machine Translation
Author: Karthik Visweswariah ; Rajakrishnan Rajkumar ; Ankur Gandhe ; Ananthakrishnan Ramanathan ; Jiri Navratil
Abstract: Preordering of source side sentences has proved to be useful in improving statistical machine translation. Most work has used a parser in the source language along with rules to map the source language word order into the target language word order. The requirement to have a source language parser is a major drawback, which we seek to overcome in this paper. Instead of using a parser and then using rules to order the source side sentence we learn a model that can directly reorder source side sentences to match target word order using a small parallel corpus with highquality word alignments. Our model learns pairwise costs of a word immediately preced- ing another word. We use the Lin-Kernighan heuristic to find the best source reordering efficiently during training and testing and show that it suffices to provide good quality reordering. We show gains in translation performance based on our reordering model for translating from Hindi to English, Urdu to English (with a public dataset), and English to Hindi. For English to Hindi we show that our technique achieves better performance than a method that uses rules applied to the source side English parse.
3 0.392903 137 emnlp-2011-Training dependency parsers by jointly optimizing multiple objectives
Author: Keith Hall ; Ryan McDonald ; Jason Katz-Brown ; Michael Ringgaard
Abstract: We present an online learning algorithm for training parsers which allows for the inclusion of multiple objective functions. The primary example is the extension of a standard supervised parsing objective function with additional loss-functions, either based on intrinsic parsing quality or task-specific extrinsic measures of quality. Our empirical results show how this approach performs for two dependency parsing algorithms (graph-based and transition-based parsing) and how it achieves increased performance on multiple target tasks including reordering for machine translation and parser adaptation.
4 0.30552447 123 emnlp-2011-Soft Dependency Constraints for Reordering in Hierarchical Phrase-Based Translation
Author: Yang Gao ; Philipp Koehn ; Alexandra Birch
Abstract: Long-distance reordering remains one of the biggest challenges facing machine translation. We derive soft constraints from the source dependency parsing to directly address the reordering problem for the hierarchical phrasebased model. Our approach significantly improves Chinese–English machine translation on a large-scale task by 0.84 BLEU points on average. Moreover, when we switch the tuning function from BLEU to the LRscore which promotes reordering, we observe total improvements of 1.21 BLEU, 1.30 LRscore and 3.36 TER over the baseline. On average our approach improves reordering precision and recall by 6.9 and 0.3 absolute points, respectively, and is found to be especially effective for long-distance reodering.
5 0.29915503 74 emnlp-2011-Inducing Sentence Structure from Parallel Corpora for Reordering
Author: John DeNero ; Jakob Uszkoreit
Abstract: When translating among languages that differ substantially in word order, machine translation (MT) systems benefit from syntactic preordering—an approach that uses features from a syntactic parse to permute source words into a target-language-like order. This paper presents a method for inducing parse trees automatically from a parallel corpus, instead of using a supervised parser trained on a treebank. These induced parses are used to preorder source sentences. We demonstrate that our induced parser is effective: it not only improves a state-of-the-art phrase-based system with integrated reordering, but also approaches the performance of a recent preordering method based on a supervised parser. These results show that the syntactic structure which is relevant to MT pre-ordering can be learned automatically from parallel text, thus establishing a new application for unsupervised grammar induction.
6 0.18012381 108 emnlp-2011-Quasi-Synchronous Phrase Dependency Grammars for Machine Translation
7 0.15895531 4 emnlp-2011-A Fast, Accurate, Non-Projective, Semantically-Enriched Parser
8 0.15578555 95 emnlp-2011-Multi-Source Transfer of Delexicalized Dependency Parsers
9 0.13540407 125 emnlp-2011-Statistical Machine Translation with Local Language Models
10 0.1234337 22 emnlp-2011-Better Evaluation Metrics Lead to Better Machine Translation
11 0.11569548 15 emnlp-2011-A novel dependency-to-string model for statistical machine translation
12 0.11451517 102 emnlp-2011-Parse Correction with Specialized Models for Difficult Attachment Types
13 0.1052681 50 emnlp-2011-Evaluating Dependency Parsing: Robust and Heuristics-Free Cross-Annotation Evaluation
14 0.092280887 103 emnlp-2011-Parser Evaluation over Local and Non-Local Deep Dependencies in a Large Corpus
15 0.08802513 118 emnlp-2011-SMT Helps Bitext Dependency Parsing
16 0.08434701 44 emnlp-2011-Domain Adaptation via Pseudo In-Domain Data Selection
17 0.081647567 58 emnlp-2011-Fast Generation of Translation Forest for Large-Scale SMT Discriminative Training
18 0.073780648 3 emnlp-2011-A Correction Model for Word Alignments
19 0.072171226 20 emnlp-2011-Augmenting String-to-Tree Translation Models with Fuzzy Use of Source-side Syntax
20 0.066998221 115 emnlp-2011-Relaxed Cross-lingual Projection of Constituent Syntax
topicId topicWeight
[(0, 0.319), (1, 0.328), (2, 0.163), (3, 0.162), (4, -0.087), (5, 0.159), (6, 0.455), (7, 0.215), (8, -0.014), (9, 0.087), (10, -0.01), (11, -0.141), (12, -0.152), (13, -0.082), (14, -0.058), (15, -0.118), (16, -0.036), (17, 0.015), (18, -0.007), (19, 0.024), (20, 0.018), (21, -0.03), (22, 0.081), (23, -0.084), (24, 0.042), (25, -0.042), (26, -0.038), (27, -0.036), (28, 0.07), (29, 0.04), (30, -0.018), (31, -0.028), (32, -0.029), (33, 0.046), (34, -0.019), (35, 0.045), (36, 0.027), (37, -0.057), (38, -0.029), (39, -0.052), (40, 0.012), (41, -0.007), (42, 0.019), (43, 0.04), (44, -0.013), (45, 0.053), (46, 0.029), (47, -0.053), (48, 0.023), (49, 0.069)]
simIndex simValue paperId paperTitle
same-paper 1 0.96642303 136 emnlp-2011-Training a Parser for Machine Translation Reordering
Author: Jason Katz-Brown ; Slav Petrov ; Ryan McDonald ; Franz Och ; David Talbot ; Hiroshi Ichikawa ; Masakazu Seno ; Hideto Kazawa
Abstract: We propose a simple training regime that can improve the extrinsic performance of a parser, given only a corpus of sentences and a way to automatically evaluate the extrinsic quality of a candidate parse. We apply our method to train parsers that excel when used as part of a reordering component in a statistical machine translation system. We use a corpus of weakly-labeled reference reorderings to guide parser training. Our best parsers contribute significant improvements in subjective translation quality while their intrinsic attachment scores typically regress.
2 0.85104311 13 emnlp-2011-A Word Reordering Model for Improved Machine Translation
Author: Karthik Visweswariah ; Rajakrishnan Rajkumar ; Ankur Gandhe ; Ananthakrishnan Ramanathan ; Jiri Navratil
Abstract: Preordering of source side sentences has proved to be useful in improving statistical machine translation. Most work has used a parser in the source language along with rules to map the source language word order into the target language word order. The requirement to have a source language parser is a major drawback, which we seek to overcome in this paper. Instead of using a parser and then using rules to order the source side sentence we learn a model that can directly reorder source side sentences to match target word order using a small parallel corpus with highquality word alignments. Our model learns pairwise costs of a word immediately preced- ing another word. We use the Lin-Kernighan heuristic to find the best source reordering efficiently during training and testing and show that it suffices to provide good quality reordering. We show gains in translation performance based on our reordering model for translating from Hindi to English, Urdu to English (with a public dataset), and English to Hindi. For English to Hindi we show that our technique achieves better performance than a method that uses rules applied to the source side English parse.
3 0.81208682 74 emnlp-2011-Inducing Sentence Structure from Parallel Corpora for Reordering
Author: John DeNero ; Jakob Uszkoreit
Abstract: When translating among languages that differ substantially in word order, machine translation (MT) systems benefit from syntactic preordering—an approach that uses features from a syntactic parse to permute source words into a target-language-like order. This paper presents a method for inducing parse trees automatically from a parallel corpus, instead of using a supervised parser trained on a treebank. These induced parses are used to preorder source sentences. We demonstrate that our induced parser is effective: it not only improves a state-of-the-art phrase-based system with integrated reordering, but also approaches the performance of a recent preordering method based on a supervised parser. These results show that the syntactic structure which is relevant to MT pre-ordering can be learned automatically from parallel text, thus establishing a new application for unsupervised grammar induction.
4 0.73377949 137 emnlp-2011-Training dependency parsers by jointly optimizing multiple objectives
Author: Keith Hall ; Ryan McDonald ; Jason Katz-Brown ; Michael Ringgaard
Abstract: We present an online learning algorithm for training parsers which allows for the inclusion of multiple objective functions. The primary example is the extension of a standard supervised parsing objective function with additional loss-functions, either based on intrinsic parsing quality or task-specific extrinsic measures of quality. Our empirical results show how this approach performs for two dependency parsing algorithms (graph-based and transition-based parsing) and how it achieves increased performance on multiple target tasks including reordering for machine translation and parser adaptation.
5 0.64662266 123 emnlp-2011-Soft Dependency Constraints for Reordering in Hierarchical Phrase-Based Translation
Author: Yang Gao ; Philipp Koehn ; Alexandra Birch
Abstract: Long-distance reordering remains one of the biggest challenges facing machine translation. We derive soft constraints from the source dependency parsing to directly address the reordering problem for the hierarchical phrasebased model. Our approach significantly improves Chinese–English machine translation on a large-scale task by 0.84 BLEU points on average. Moreover, when we switch the tuning function from BLEU to the LRscore which promotes reordering, we observe total improvements of 1.21 BLEU, 1.30 LRscore and 3.36 TER over the baseline. On average our approach improves reordering precision and recall by 6.9 and 0.3 absolute points, respectively, and is found to be especially effective for long-distance reodering.
6 0.4077467 95 emnlp-2011-Multi-Source Transfer of Delexicalized Dependency Parsers
7 0.38781071 102 emnlp-2011-Parse Correction with Specialized Models for Difficult Attachment Types
8 0.38661692 4 emnlp-2011-A Fast, Accurate, Non-Projective, Semantically-Enriched Parser
9 0.38143742 108 emnlp-2011-Quasi-Synchronous Phrase Dependency Grammars for Machine Translation
10 0.35066748 103 emnlp-2011-Parser Evaluation over Local and Non-Local Deep Dependencies in a Large Corpus
11 0.30388302 50 emnlp-2011-Evaluating Dependency Parsing: Robust and Heuristics-Free Cross-Annotation Evaluation
12 0.2982116 15 emnlp-2011-A novel dependency-to-string model for statistical machine translation
13 0.28172746 118 emnlp-2011-SMT Helps Bitext Dependency Parsing
14 0.28056633 93 emnlp-2011-Minimum Imputed-Risk: Unsupervised Discriminative Training for Machine Translation
15 0.27932 22 emnlp-2011-Better Evaluation Metrics Lead to Better Machine Translation
16 0.26790747 125 emnlp-2011-Statistical Machine Translation with Local Language Models
17 0.26062068 111 emnlp-2011-Reducing Grounded Learning Tasks To Grammatical Inference
18 0.2537578 44 emnlp-2011-Domain Adaptation via Pseudo In-Domain Data Selection
19 0.24514358 115 emnlp-2011-Relaxed Cross-lingual Projection of Constituent Syntax
20 0.24369234 148 emnlp-2011-Watermarking the Outputs of Structured Prediction with an application in Statistical Machine Translation.
topicId topicWeight
[(11, 0.182), (23, 0.122), (36, 0.038), (37, 0.065), (45, 0.058), (47, 0.046), (53, 0.031), (54, 0.028), (57, 0.019), (62, 0.021), (64, 0.055), (66, 0.033), (69, 0.013), (79, 0.067), (82, 0.023), (85, 0.017), (90, 0.021), (96, 0.065), (98, 0.02)]
simIndex simValue paperId paperTitle
1 0.88173348 134 emnlp-2011-Third-order Variational Reranking on Packed-Shared Dependency Forests
Author: Katsuhiko Hayashi ; Taro Watanabe ; Masayuki Asahara ; Yuji Matsumoto
Abstract: We propose a novel forest reranking algorithm for discriminative dependency parsing based on a variant of Eisner’s generative model. In our framework, we define two kinds of generative model for reranking. One is learned from training data offline and the other from a forest generated by a baseline parser on the fly. The final prediction in the reranking stage is performed using linear interpolation of these models and discriminative model. In order to efficiently train the model from and decode on a hypergraph data structure representing a forest, we apply extended inside/outside and Viterbi algorithms. Experimental results show that our proposed forest reranking algorithm achieves significant improvement when compared with conventional approaches.
2 0.8775146 71 emnlp-2011-Identifying and Following Expert Investors in Stock Microblogs
Author: Roy Bar-Haim ; Elad Dinur ; Ronen Feldman ; Moshe Fresko ; Guy Goldstein
Abstract: Information published in online stock investment message boards, and more recently in stock microblogs, is considered highly valuable by many investors. Previous work focused on aggregation of sentiment from all users. However, in this work we show that it is beneficial to distinguish expert users from non-experts. We propose a general framework for identifying expert investors, and use it as a basis for several models that predict stock rise from stock microblogging messages (stock tweets). In particular, we present two methods that combine expert identification and per-user unsupervised learning. These methods were shown to achieve relatively high precision in predicting stock rise, and significantly outperform our baseline. In addition, our work provides an in-depth analysis of the content and potential usefulness of stock tweets.
same-paper 3 0.79227948 136 emnlp-2011-Training a Parser for Machine Translation Reordering
Author: Jason Katz-Brown ; Slav Petrov ; Ryan McDonald ; Franz Och ; David Talbot ; Hiroshi Ichikawa ; Masakazu Seno ; Hideto Kazawa
Abstract: We propose a simple training regime that can improve the extrinsic performance of a parser, given only a corpus of sentences and a way to automatically evaluate the extrinsic quality of a candidate parse. We apply our method to train parsers that excel when used as part of a reordering component in a statistical machine translation system. We use a corpus of weakly-labeled reference reorderings to guide parser training. Our best parsers contribute significant improvements in subjective translation quality while their intrinsic attachment scores typically regress.
4 0.68970054 108 emnlp-2011-Quasi-Synchronous Phrase Dependency Grammars for Machine Translation
Author: Kevin Gimpel ; Noah A. Smith
Abstract: We present a quasi-synchronous dependency grammar (Smith and Eisner, 2006) for machine translation in which the leaves of the tree are phrases rather than words as in previous work (Gimpel and Smith, 2009). This formulation allows us to combine structural components of phrase-based and syntax-based MT in a single model. We describe a method of extracting phrase dependencies from parallel text using a target-side dependency parser. For decoding, we describe a coarse-to-fine approach based on lattice dependency parsing of phrase lattices. We demonstrate performance improvements for Chinese-English and UrduEnglish translation over a phrase-based baseline. We also investigate the use of unsupervised dependency parsers, reporting encouraging preliminary results.
5 0.68571556 137 emnlp-2011-Training dependency parsers by jointly optimizing multiple objectives
Author: Keith Hall ; Ryan McDonald ; Jason Katz-Brown ; Michael Ringgaard
Abstract: We present an online learning algorithm for training parsers which allows for the inclusion of multiple objective functions. The primary example is the extension of a standard supervised parsing objective function with additional loss-functions, either based on intrinsic parsing quality or task-specific extrinsic measures of quality. Our empirical results show how this approach performs for two dependency parsing algorithms (graph-based and transition-based parsing) and how it achieves increased performance on multiple target tasks including reordering for machine translation and parser adaptation.
6 0.67827582 59 emnlp-2011-Fast and Robust Joint Models for Biomedical Event Extraction
7 0.66899645 1 emnlp-2011-A Bayesian Mixture Model for PoS Induction Using Multiple Features
8 0.66844928 123 emnlp-2011-Soft Dependency Constraints for Reordering in Hierarchical Phrase-Based Translation
9 0.66672909 68 emnlp-2011-Hypotheses Selection Criteria in a Reranking Framework for Spoken Language Understanding
10 0.65807515 20 emnlp-2011-Augmenting String-to-Tree Translation Models with Fuzzy Use of Source-side Syntax
11 0.6567421 8 emnlp-2011-A Model of Discourse Predictions in Human Sentence Processing
12 0.65623182 85 emnlp-2011-Learning to Simplify Sentences with Quasi-Synchronous Grammar and Integer Programming
13 0.65391248 28 emnlp-2011-Closing the Loop: Fast, Interactive Semi-Supervised Annotation With Queries on Features and Instances
14 0.65339071 13 emnlp-2011-A Word Reordering Model for Improved Machine Translation
15 0.65103638 132 emnlp-2011-Syntax-Based Grammaticality Improvement using CCG and Guided Search
16 0.64943433 87 emnlp-2011-Lexical Generalization in CCG Grammar Induction for Semantic Parsing
17 0.6492793 35 emnlp-2011-Correcting Semantic Collocation Errors with L1-induced Paraphrases
18 0.64840341 54 emnlp-2011-Exploiting Parse Structures for Native Language Identification
19 0.64797187 111 emnlp-2011-Reducing Grounded Learning Tasks To Grammatical Inference
20 0.64730853 15 emnlp-2011-A novel dependency-to-string model for statistical machine translation