emnlp emnlp2011 emnlp2011-13 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Karthik Visweswariah ; Rajakrishnan Rajkumar ; Ankur Gandhe ; Ananthakrishnan Ramanathan ; Jiri Navratil
Abstract: Preordering of source side sentences has proved to be useful in improving statistical machine translation. Most work has used a parser in the source language along with rules to map the source language word order into the target language word order. The requirement to have a source language parser is a major drawback, which we seek to overcome in this paper. Instead of using a parser and then using rules to order the source side sentence we learn a model that can directly reorder source side sentences to match target word order using a small parallel corpus with highquality word alignments. Our model learns pairwise costs of a word immediately preced- ing another word. We use the Lin-Kernighan heuristic to find the best source reordering efficiently during training and testing and show that it suffices to provide good quality reordering. We show gains in translation performance based on our reordering model for translating from Hindi to English, Urdu to English (with a public dataset), and English to Hindi. For English to Hindi we show that our technique achieves better performance than a method that uses rules applied to the source side English parse.
Reference: text
sentIndex sentText sentNum sentScore
1 aramana2 @ in ibm com Abstract Preordering of source side sentences has proved to be useful in improving statistical machine translation. [sent-10, score-0.247]
2 Most work has used a parser in the source language along with rules to map the source language word order into the target language word order. [sent-11, score-0.304]
3 Instead of using a parser and then using rules to order the source side sentence we learn a model that can directly reorder source side sentences to match target word order using a small parallel corpus with highquality word alignments. [sent-13, score-0.596]
4 We use the Lin-Kernighan heuristic to find the best source reordering efficiently during training and testing and show that it suffices to provide good quality reordering. [sent-15, score-0.679]
5 We show gains in translation performance based on our reordering model for translating from Hindi to English, Urdu to English (with a public dataset), and English to Hindi. [sent-16, score-0.631]
6 Machine translation systems need to reorder words in the source sentence to produce fluent output in the 486 Jiri Navratil IBM T. [sent-19, score-0.312]
7 Even the capturing of these local reordering phenomena is constrained by the amount of training data available. [sent-25, score-0.61]
8 For example, if adjectives precede nouns in the source language and follow nouns in the target language we still need to see a particular adjective noun pair in the parallel corpus to handle the reordering via the phrase table. [sent-26, score-0.734]
9 While these models are simple, and can be integrated with the decoder they are insufficient to capture long-range reordering phenomena especially for language pairs that differ significantly. [sent-32, score-0.646]
10 The weakness of these simple distortion models has been overcome using syntax of either the source or target sentence (Yamada and Knight, 2002; Galley et al. [sent-33, score-0.244]
11 Another approach that overcomes this weakness, is to to reorder the source sentence based on rules applied on the source parse (either hand written or learned from data) both when training and testing (Collins et al. [sent-39, score-0.391]
12 Words are the cities in the TSP and the objective is to learn the distance between words so that the shortest tour corresponds to the ordering of the words in the source sentence in the target language. [sent-45, score-0.245]
13 We show experimentally that our reordering model, even when used to reorder sentences for training and testing (rather than being used as an additional score in the decoder) improves machine translation performance for: Hindi → English, En- glish → Hindi, aonrdm aUnrcdeu → English. [sent-49, score-0.809]
14 reordering phenomena we include it in our experiments since there are publicly available datasets for Urdu-English. [sent-51, score-0.61]
15 For English → Hindi we obtained better machine translEantigolnis performance ew obithta our reordering imneod treal as compared to a method that uses reordering rules applied to the source side parse. [sent-52, score-1.337]
16 Section 3 outlines reordering issues due to syntactic differences between Hindi and English. [sent-55, score-0.57]
17 487 Section 4 presents our reordering model, Section 5 presents experimental results and Section 6 presents our conclusions and possible future work. [sent-56, score-0.57]
18 2 Related work There have been several studies demonstrating improved machine translation performance by reordering source side sentences based on rules applied to the source side parse during training and decoding. [sent-57, score-1.03]
19 , 2006) which uses source and/or target side syntax in a Context Free Grammar framework which results in machine translation decoding being considered as a parsing problem. [sent-69, score-0.35]
20 In this paper we propose a model that does not require either source or target side syntax while also preserving the efficiency of reordering techniques based on rules applied to the source side parse. [sent-70, score-1.026]
21 In work that is closely related to ours, (Tromble and Eisner, 2009) formulated word reordering as a Linear Ordering Problem (LOP), an NP-hard permutation problem. [sent-71, score-0.607]
22 the translation, distortion and language model probabilities) we learn model weights to reorder source sentences to match target word order using an informative feature set adapted from graph-based dependency parsing (McDonald et al. [sent-82, score-0.384]
23 3 Hindi-English reordering issues This section provides a brief survey of constructions that the two languages in question differ as well as have in common. [sent-84, score-0.57]
24 Our reordering model assigns costs to candidate permutations as: C(π|w) =Xc(πi−1,πi). [sent-102, score-0.625]
25 Given a source sentence w we reorder it according to the permutation π that minimizes the cost C(π|w) . [sent-105, score-0.288]
26 Thus, we would like our cost function C(π|w) to be such that the correct reordering π∗ has tChe(π πl|owwe)s tot c boest s uocfh ha tllh possible reorderings π. [sent-106, score-0.646]
27 2 we describe how we train the weights θ to obtain a good reordering model. [sent-109, score-0.57]
28 Consider the following example: English input: John eats apples Hindi: John seba(apples) khaataa hai(eats) Desired reordered English: John apples eats The ATSP that we need to solve is represented pictorially in Figure 1 with sample costs. [sent-111, score-0.489]
29 We start and end the tour at node 0, and this determines the first word in the reordered sentence. [sent-113, score-0.275]
30 In this example the minimum cost tour is: Start → John → apple → eats recovering hthne → right reordering for translation into Hindi. [sent-114, score-0.76]
31 Solving the ATSP (which is a well known NP hard problem) efficiently is crucial for the efficiency of our reordering model. [sent-115, score-0.57]
32 Following (Hornik and N0 Figure 1: Example of an ATSP for reordering the sentence: John eats apples. [sent-120, score-0.655]
33 Overall, this means that our reordering model is as fast as parsing and hence our model is compara- c0(A, =A0) ∞ c0(A, B0) c0(B0, ble in performance to techniques based on applying rules to the parse tree. [sent-123, score-0.601]
34 1 Features Since we would like to model reordering phenomena which are largely related to analyzing the syntax of the source sentence, we chose to use features based on those that have in the past been used for parsing (McDonald et al. [sent-125, score-0.757]
35 A subset of the features we use was also used for reordering in (Tromble and Eisner, 2009). [sent-127, score-0.57]
36 These features depend only on the identities of the word and POS tag of the two positions iand j and we call 489 Table 1: Bigram feature templates used to calculate the cost that word at position iimmediately precedes word at position j in the target word order. [sent-131, score-0.302]
37 Each of the templates is also conjoined with i-j the signed distance between the two words in the source sentence. [sent-133, score-0.244]
38 Since Hindi is verb final, in Hindi sentences with multiple verb groups it is rare for words with a verb in between to be placed together in the reordering to match English. [sent-141, score-0.606]
39 Each of the templates described in Table 1 and Table 2 is also conjoined with i-j the signed distance between the two words in the source sentence. [sent-143, score-0.244]
40 Each of the templates is also conjoined with i-j the signed distance between the two positions in the source sentence. [sent-151, score-0.299]
41 types for the task of reordering Hindi sentences to be in English word order. [sent-152, score-0.606]
42 2 Training To train the weights θ in our model, we need a collection of sentences, where we have the desired reference reordering π∗ (x) for each input sentence x. [sent-154, score-0.666]
43 The quality and consistency of these reference reorderings will depend on the quality of the word alignments that we use. [sent-156, score-0.263]
44 Given word aligned source and target sentences, we drop the source words that are not aligned. [sent-157, score-0.339]
45 Let mi be the mean of the target word positions that the source word at index iis aligned to. [sent-158, score-0.285]
46 πˆ = argπminC(π|x) is the best reordering based on the current parameter value and L is a loss function. [sent-166, score-0.57]
47 We take the loss of a reordering to be the number of words for which the preceding word is wrong relative to the reference target order. [sent-167, score-0.685]
48 5 Experiments In this section we report on experiments to evaluate our reordering model. [sent-169, score-0.57]
49 The first method we use for evaluation (monolingual BLEU) is by generating the desired reordering of the source sentence (as described in Section 4. [sent-170, score-0.715]
50 2) and compare the reordered output to this desired reordered sentence using the BLEU metric. [sent-171, score-0.498]
51 5) the reordering by its effect on eventual machine translation performance. [sent-173, score-0.631]
52 We note that our reordering techniques uses POS information for the input sentence. [sent-174, score-0.57]
53 1 Reordering model training data and alignment quality To train our reordering models we need training data where we have the input source language sentence and the desired reordering in the target language. [sent-180, score-1.34]
54 2 we derive the reference reordered sentence using word alignments. [sent-182, score-0.291]
55 Table 3 presents our monolingual BLEU results for Hindi to English reordering as the source of the word alignments is varied. [sent-183, score-0.847]
56 We have word alignments from three sources: A small set of hand aligned sentences, HMM alignments (Vogel et al. [sent-185, score-0.32]
57 We see that the quality of the alignments is an important determiner of reordering performance. [sent-188, score-0.697]
58 Row 1 shows the BLEU for unreordered (baseline) Hindi compared with the Hindi sentences reordered in English Order. [sent-189, score-0.301]
59 Although using the Maximum Entropy alignments is better than using HMM alignments, we do not im- prove upon a small number of hand alignments by using all the Maximum Entropy alignments. [sent-191, score-0.254]
60 The heuristic we used in the selection of snippets was to pick maximal snippets of at least 7 consecutive Hindi words with all Hindi words aligned to a consecutive span of English words, with no unaligned English words in the span and no English words aligned to Hindi words outside the span. [sent-194, score-0.27]
61 Adding snippets selected with this heuristic improves the reordering performance of our model as seen in the last row of Table 3. [sent-195, score-0.639]
62 830UEn- glish reordering using models trained with different feature sets and tested on a development set of 280 Hindi sentences (5590 tokens). [sent-199, score-0.606]
63 3 Monolingual reordering comparisons Table 5 compares our reordering model with a reimplementation of the reordering model proposed in (Tromble and Eisner, 2009). [sent-206, score-1.71]
64 To generate our training data, for Hindi to English and English to Hindi we use a set of 6000 hand aligned sentences, for Urdu to English we use a set of 8500 hand aligned sentences and for English to French we use a set of 10000 hand aligned sentences (a subset of Europarl and Hansards corpus). [sent-208, score-0.27]
65 P7r02ig- inal source order with desired target reorder without reordering, and reordering using our model (TSP) and the model proposed in (Tromble and Eisner, 2009) (LOP). [sent-212, score-0.912]
66 We include English-French here to compare on a fairly similar language pair with local reordering phenomena (the main difference being that in French adjectives generally follow nouns). [sent-214, score-0.61]
67 The following analysis is for Hindi to English reordering with the best model (this is also the model used for Machine Translation experiments reported on in Section 5. [sent-218, score-0.57]
68 52 sentences were reordered by the model to match the order of the corresponding reference. [sent-226, score-0.267]
69 For example, the following sentence with two subjects and objects corresponding to the verb wearing has not been reordered correctly. [sent-233, score-0.26]
70 po- ing) safeda3 (white) evama4 (and) mahilaaen5 (women) kesariyaa6 (saffron) English: All men were wearing white and the women saffron The model possibly needs more data with patterns that deviate from the standard SOV order to learn to reorder them correctly. [sent-235, score-0.351]
71 • Local reordering: To estimate the short range reordering performance, we tceon thsied sehro hrot wra nogfeten different POS bigrams in the input are reordered correctly. [sent-243, score-0.801]
72 Here, we expect the model to reorder prepositions correctly, and to avoid any reordering that moves apart nouns and their adjectival pre-modifiers or components of compound nouns (see Section 3). [sent-244, score-0.712]
73 Table 6 summarizes the reordering performance for these categories for a set of 280 sentences (same as the test set used in Section 5. [sent-245, score-0.606]
74 , the number of instances of the pair in the reference (column titled Total), the number of instances that already appear in the correct order in the input (column Input), and the number that are ordered correctly by the reordering model (column Reordered). [sent-249, score-0.63]
75 The same problem in the training data would also adversely impact the learning of the preposition reordering rule. [sent-255, score-0.57]
76 5 Machine translation results We now present experiments in incorporating the reordering model in machine translation systems. [sent-257, score-0.692]
77 For all results presented here, we reorder the training and test data using the single best reordering based on our reordering model for each sentence. [sent-258, score-1.282]
78 For each of the language pairs we evaluated, we trained Direct Translation Model 2 (DTM) systems (Ittycheriah and Roukos, 2007) with and without reordering and compared performance on test data. [sent-259, score-0.57]
79 We note that the DTM system includes features that allow it to model lexicalized reordering phenomena. [sent-260, score-0.57]
80 The reordering window size was set to +/-8 words for both the baseline and our reordered input. [sent-261, score-0.801]
81 e we reordered the existing word alignments rather than realigning the sentences after reordering. [sent-263, score-0.394]
82 To train our reordering model, we used roughly 6K alignments plus 17K snippets selected from MaxEnt alignments as described in Section 5. [sent-275, score-0.925]
83 The monolingual reordering BLEU (on the same data reported on in Section 5. [sent-277, score-0.611]
84 o587rde ing (baseline) compared with performance after preordering with our reordering model. [sent-283, score-0.614]
85 To train the reordering model we used 9K hand alignments and 11K snippets extracted from MaxEnt alignments as described in Section 5. [sent-288, score-0.893]
86 The monolingual reordering BLEU for the reordering model thus obtained (on the same data reported on in Section 5. [sent-290, score-1.181]
87 , 2010)) applied to a parse to reorder source side English sentences. [sent-296, score-0.308]
88 2, which is an improvement over the baseline, but our reordering model is better by 1. [sent-298, score-0.57]
89 An added benefit of our reordering model is that the decoder can be run with a smaller search space exploring only a small amount of reordering without losing accuracy but running substantially faster. [sent-300, score-1.176]
90 6 Conclusion and future work In this paper we presented a reordering model to reorder source language data to make it resemble the target language word order without using either a source or target parser. [sent-305, score-1.04]
91 We show better performance compared to syntax based reordering rules for English to Hindi translation. [sent-307, score-0.639]
92 Considering the fact that treebanks required to build high quality parsers are costly to obtain, we think that our reordering model is a viable alterna- tive to using syntax for reordering. [sent-309, score-0.608]
93 We also note, that with the preordering based on our reordering model we can achieve the best BLEU scores with a much tighter search space in the decoder. [sent-310, score-0.614]
94 Even accounting for the cost of finding the best reordering according to our model, this usually results in faster processing than if we did not have the reordering in place. [sent-311, score-1.14]
95 We would like to investigate the use of other loss functions and their effect on reordering performance. [sent-315, score-0.57]
96 We also would like to explore whether the use of scores from our reordering model directly in machine translation systems can improve performance relative to using just the single best reordering. [sent-316, score-0.631]
97 Automatically learning source-side reordering rules for large scale machine translation. [sent-373, score-0.601]
98 Constituent reordering and syntax models for Englishto-Japanese statistical machine translation. [sent-402, score-0.608]
99 Word reordering and a dynamic programming beam search algorithm for statistical machine translation. [sent-432, score-0.57]
100 Syntax based reordering with automatically derived rules for improved statistical machine translation. [sent-440, score-0.601]
wordName wordTfidf (topN-words)
[('reordering', 0.57), ('hindi', 0.439), ('reordered', 0.231), ('tsp', 0.213), ('reorder', 0.142), ('contextpos', 0.136), ('alignments', 0.127), ('atsp', 0.119), ('source', 0.109), ('contextword', 0.102), ('visweswariah', 0.102), ('tromble', 0.087), ('eats', 0.085), ('bleu', 0.078), ('reorderings', 0.076), ('urdu', 0.073), ('snippets', 0.069), ('india', 0.069), ('honourable', 0.068), ('english', 0.068), ('aligned', 0.066), ('ittycheriah', 0.066), ('clause', 0.066), ('templates', 0.063), ('translation', 0.061), ('reference', 0.06), ('ramanathan', 0.059), ('saffron', 0.059), ('traveling', 0.059), ('side', 0.057), ('costs', 0.055), ('target', 0.055), ('positions', 0.055), ('minister', 0.053), ('till', 0.053), ('concorde', 0.051), ('dtm', 0.051), ('electricity', 0.051), ('lop', 0.051), ('bigram', 0.051), ('pos', 0.049), ('women', 0.046), ('ibm', 0.045), ('clauses', 0.045), ('apples', 0.044), ('preordering', 0.044), ('salesman', 0.044), ('tour', 0.044), ('wear', 0.044), ('zaslavskiy', 0.044), ('eisner', 0.042), ('distortion', 0.042), ('promise', 0.042), ('monolingual', 0.041), ('phenomena', 0.04), ('men', 0.04), ('syntax', 0.038), ('prime', 0.038), ('government', 0.038), ('conjoined', 0.037), ('ordering', 0.037), ('permutation', 0.037), ('policy', 0.037), ('desired', 0.036), ('sentences', 0.036), ('hmm', 0.036), ('decoder', 0.036), ('mcdonald', 0.036), ('white', 0.035), ('mira', 0.035), ('signed', 0.035), ('restructuring', 0.035), ('tillmann', 0.035), ('ananthakrishnan', 0.034), ('applegate', 0.034), ('badr', 0.034), ('clothes', 0.034), ('dalal', 0.034), ('hornik', 0.034), ('iimmediately', 0.034), ('jiri', 0.034), ('karthik', 0.034), ('navratil', 0.034), ('quantized', 0.034), ('unreordered', 0.034), ('house', 0.033), ('iand', 0.033), ('roughly', 0.032), ('position', 0.031), ('rules', 0.031), ('christoph', 0.031), ('iin', 0.031), ('zollmann', 0.031), ('decoding', 0.03), ('bangalore', 0.03), ('abraham', 0.029), ('wearing', 0.029), ('eval', 0.029), ('fifty', 0.029), ('pushpak', 0.029)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000004 13 emnlp-2011-A Word Reordering Model for Improved Machine Translation
Author: Karthik Visweswariah ; Rajakrishnan Rajkumar ; Ankur Gandhe ; Ananthakrishnan Ramanathan ; Jiri Navratil
Abstract: Preordering of source side sentences has proved to be useful in improving statistical machine translation. Most work has used a parser in the source language along with rules to map the source language word order into the target language word order. The requirement to have a source language parser is a major drawback, which we seek to overcome in this paper. Instead of using a parser and then using rules to order the source side sentence we learn a model that can directly reorder source side sentences to match target word order using a small parallel corpus with highquality word alignments. Our model learns pairwise costs of a word immediately preced- ing another word. We use the Lin-Kernighan heuristic to find the best source reordering efficiently during training and testing and show that it suffices to provide good quality reordering. We show gains in translation performance based on our reordering model for translating from Hindi to English, Urdu to English (with a public dataset), and English to Hindi. For English to Hindi we show that our technique achieves better performance than a method that uses rules applied to the source side English parse.
2 0.44558531 136 emnlp-2011-Training a Parser for Machine Translation Reordering
Author: Jason Katz-Brown ; Slav Petrov ; Ryan McDonald ; Franz Och ; David Talbot ; Hiroshi Ichikawa ; Masakazu Seno ; Hideto Kazawa
Abstract: We propose a simple training regime that can improve the extrinsic performance of a parser, given only a corpus of sentences and a way to automatically evaluate the extrinsic quality of a candidate parse. We apply our method to train parsers that excel when used as part of a reordering component in a statistical machine translation system. We use a corpus of weakly-labeled reference reorderings to guide parser training. Our best parsers contribute significant improvements in subjective translation quality while their intrinsic attachment scores typically regress.
3 0.27683645 123 emnlp-2011-Soft Dependency Constraints for Reordering in Hierarchical Phrase-Based Translation
Author: Yang Gao ; Philipp Koehn ; Alexandra Birch
Abstract: Long-distance reordering remains one of the biggest challenges facing machine translation. We derive soft constraints from the source dependency parsing to directly address the reordering problem for the hierarchical phrasebased model. Our approach significantly improves Chinese–English machine translation on a large-scale task by 0.84 BLEU points on average. Moreover, when we switch the tuning function from BLEU to the LRscore which promotes reordering, we observe total improvements of 1.21 BLEU, 1.30 LRscore and 3.36 TER over the baseline. On average our approach improves reordering precision and recall by 6.9 and 0.3 absolute points, respectively, and is found to be especially effective for long-distance reodering.
4 0.25664777 74 emnlp-2011-Inducing Sentence Structure from Parallel Corpora for Reordering
Author: John DeNero ; Jakob Uszkoreit
Abstract: When translating among languages that differ substantially in word order, machine translation (MT) systems benefit from syntactic preordering—an approach that uses features from a syntactic parse to permute source words into a target-language-like order. This paper presents a method for inducing parse trees automatically from a parallel corpus, instead of using a supervised parser trained on a treebank. These induced parses are used to preorder source sentences. We demonstrate that our induced parser is effective: it not only improves a state-of-the-art phrase-based system with integrated reordering, but also approaches the performance of a recent preordering method based on a supervised parser. These results show that the syntactic structure which is relevant to MT pre-ordering can be learned automatically from parallel text, thus establishing a new application for unsupervised grammar induction.
5 0.16955915 137 emnlp-2011-Training dependency parsers by jointly optimizing multiple objectives
Author: Keith Hall ; Ryan McDonald ; Jason Katz-Brown ; Michael Ringgaard
Abstract: We present an online learning algorithm for training parsers which allows for the inclusion of multiple objective functions. The primary example is the extension of a standard supervised parsing objective function with additional loss-functions, either based on intrinsic parsing quality or task-specific extrinsic measures of quality. Our empirical results show how this approach performs for two dependency parsing algorithms (graph-based and transition-based parsing) and how it achieves increased performance on multiple target tasks including reordering for machine translation and parser adaptation.
6 0.15885092 108 emnlp-2011-Quasi-Synchronous Phrase Dependency Grammars for Machine Translation
7 0.15463465 125 emnlp-2011-Statistical Machine Translation with Local Language Models
8 0.14121906 3 emnlp-2011-A Correction Model for Word Alignments
9 0.11876018 15 emnlp-2011-A novel dependency-to-string model for statistical machine translation
10 0.090285487 20 emnlp-2011-Augmenting String-to-Tree Translation Models with Fuzzy Use of Source-side Syntax
11 0.079317674 22 emnlp-2011-Better Evaluation Metrics Lead to Better Machine Translation
12 0.077850528 58 emnlp-2011-Fast Generation of Translation Forest for Large-Scale SMT Discriminative Training
13 0.077610575 65 emnlp-2011-Heuristic Search for Non-Bottom-Up Tree Structure Prediction
14 0.076576874 51 emnlp-2011-Exact Decoding of Phrase-Based Translation Models through Lagrangian Relaxation
15 0.074833892 72 emnlp-2011-Improved Transliteration Mining Using Graph Reinforcement
17 0.069965266 4 emnlp-2011-A Fast, Accurate, Non-Projective, Semantically-Enriched Parser
18 0.069757976 95 emnlp-2011-Multi-Source Transfer of Delexicalized Dependency Parsers
19 0.068375759 44 emnlp-2011-Domain Adaptation via Pseudo In-Domain Data Selection
20 0.06772963 75 emnlp-2011-Joint Models for Chinese POS Tagging and Dependency Parsing
topicId topicWeight
[(0, 0.264), (1, 0.287), (2, 0.169), (3, 0.027), (4, -0.065), (5, 0.078), (6, 0.422), (7, 0.192), (8, -0.085), (9, 0.075), (10, 0.004), (11, -0.077), (12, -0.139), (13, -0.102), (14, -0.115), (15, -0.167), (16, -0.091), (17, 0.085), (18, -0.025), (19, 0.08), (20, -0.003), (21, 0.005), (22, 0.014), (23, -0.092), (24, -0.043), (25, 0.011), (26, 0.028), (27, -0.117), (28, 0.051), (29, 0.01), (30, 0.023), (31, 0.032), (32, -0.07), (33, 0.059), (34, 0.004), (35, 0.062), (36, -0.008), (37, 0.051), (38, 0.011), (39, 0.067), (40, -0.001), (41, -0.042), (42, -0.062), (43, 0.064), (44, 0.033), (45, -0.004), (46, 0.004), (47, -0.014), (48, -0.009), (49, -0.015)]
simIndex simValue paperId paperTitle
same-paper 1 0.94830179 13 emnlp-2011-A Word Reordering Model for Improved Machine Translation
Author: Karthik Visweswariah ; Rajakrishnan Rajkumar ; Ankur Gandhe ; Ananthakrishnan Ramanathan ; Jiri Navratil
Abstract: Preordering of source side sentences has proved to be useful in improving statistical machine translation. Most work has used a parser in the source language along with rules to map the source language word order into the target language word order. The requirement to have a source language parser is a major drawback, which we seek to overcome in this paper. Instead of using a parser and then using rules to order the source side sentence we learn a model that can directly reorder source side sentences to match target word order using a small parallel corpus with highquality word alignments. Our model learns pairwise costs of a word immediately preced- ing another word. We use the Lin-Kernighan heuristic to find the best source reordering efficiently during training and testing and show that it suffices to provide good quality reordering. We show gains in translation performance based on our reordering model for translating from Hindi to English, Urdu to English (with a public dataset), and English to Hindi. For English to Hindi we show that our technique achieves better performance than a method that uses rules applied to the source side English parse.
2 0.83117008 136 emnlp-2011-Training a Parser for Machine Translation Reordering
Author: Jason Katz-Brown ; Slav Petrov ; Ryan McDonald ; Franz Och ; David Talbot ; Hiroshi Ichikawa ; Masakazu Seno ; Hideto Kazawa
Abstract: We propose a simple training regime that can improve the extrinsic performance of a parser, given only a corpus of sentences and a way to automatically evaluate the extrinsic quality of a candidate parse. We apply our method to train parsers that excel when used as part of a reordering component in a statistical machine translation system. We use a corpus of weakly-labeled reference reorderings to guide parser training. Our best parsers contribute significant improvements in subjective translation quality while their intrinsic attachment scores typically regress.
3 0.82464987 74 emnlp-2011-Inducing Sentence Structure from Parallel Corpora for Reordering
Author: John DeNero ; Jakob Uszkoreit
Abstract: When translating among languages that differ substantially in word order, machine translation (MT) systems benefit from syntactic preordering—an approach that uses features from a syntactic parse to permute source words into a target-language-like order. This paper presents a method for inducing parse trees automatically from a parallel corpus, instead of using a supervised parser trained on a treebank. These induced parses are used to preorder source sentences. We demonstrate that our induced parser is effective: it not only improves a state-of-the-art phrase-based system with integrated reordering, but also approaches the performance of a recent preordering method based on a supervised parser. These results show that the syntactic structure which is relevant to MT pre-ordering can be learned automatically from parallel text, thus establishing a new application for unsupervised grammar induction.
4 0.69680208 123 emnlp-2011-Soft Dependency Constraints for Reordering in Hierarchical Phrase-Based Translation
Author: Yang Gao ; Philipp Koehn ; Alexandra Birch
Abstract: Long-distance reordering remains one of the biggest challenges facing machine translation. We derive soft constraints from the source dependency parsing to directly address the reordering problem for the hierarchical phrasebased model. Our approach significantly improves Chinese–English machine translation on a large-scale task by 0.84 BLEU points on average. Moreover, when we switch the tuning function from BLEU to the LRscore which promotes reordering, we observe total improvements of 1.21 BLEU, 1.30 LRscore and 3.36 TER over the baseline. On average our approach improves reordering precision and recall by 6.9 and 0.3 absolute points, respectively, and is found to be especially effective for long-distance reodering.
5 0.46966797 137 emnlp-2011-Training dependency parsers by jointly optimizing multiple objectives
Author: Keith Hall ; Ryan McDonald ; Jason Katz-Brown ; Michael Ringgaard
Abstract: We present an online learning algorithm for training parsers which allows for the inclusion of multiple objective functions. The primary example is the extension of a standard supervised parsing objective function with additional loss-functions, either based on intrinsic parsing quality or task-specific extrinsic measures of quality. Our empirical results show how this approach performs for two dependency parsing algorithms (graph-based and transition-based parsing) and how it achieves increased performance on multiple target tasks including reordering for machine translation and parser adaptation.
6 0.40345231 3 emnlp-2011-A Correction Model for Word Alignments
7 0.3478415 125 emnlp-2011-Statistical Machine Translation with Local Language Models
8 0.33417743 108 emnlp-2011-Quasi-Synchronous Phrase Dependency Grammars for Machine Translation
9 0.33054155 15 emnlp-2011-A novel dependency-to-string model for statistical machine translation
10 0.29144198 65 emnlp-2011-Heuristic Search for Non-Bottom-Up Tree Structure Prediction
11 0.27752861 20 emnlp-2011-Augmenting String-to-Tree Translation Models with Fuzzy Use of Source-side Syntax
12 0.2720165 72 emnlp-2011-Improved Transliteration Mining Using Graph Reinforcement
13 0.26748168 85 emnlp-2011-Learning to Simplify Sentences with Quasi-Synchronous Grammar and Integer Programming
14 0.26730129 22 emnlp-2011-Better Evaluation Metrics Lead to Better Machine Translation
15 0.26178893 44 emnlp-2011-Domain Adaptation via Pseudo In-Domain Data Selection
16 0.26170123 148 emnlp-2011-Watermarking the Outputs of Structured Prediction with an application in Statistical Machine Translation.
17 0.25073084 51 emnlp-2011-Exact Decoding of Phrase-Based Translation Models through Lagrangian Relaxation
18 0.24800786 95 emnlp-2011-Multi-Source Transfer of Delexicalized Dependency Parsers
19 0.24427722 58 emnlp-2011-Fast Generation of Translation Forest for Large-Scale SMT Discriminative Training
20 0.22541164 93 emnlp-2011-Minimum Imputed-Risk: Unsupervised Discriminative Training for Machine Translation
topicId topicWeight
[(15, 0.01), (23, 0.111), (36, 0.024), (37, 0.034), (45, 0.06), (53, 0.041), (54, 0.022), (57, 0.011), (62, 0.014), (64, 0.047), (66, 0.034), (69, 0.032), (79, 0.047), (82, 0.02), (85, 0.326), (90, 0.015), (94, 0.026), (96, 0.029), (98, 0.014)]
simIndex simValue paperId paperTitle
same-paper 1 0.69760793 13 emnlp-2011-A Word Reordering Model for Improved Machine Translation
Author: Karthik Visweswariah ; Rajakrishnan Rajkumar ; Ankur Gandhe ; Ananthakrishnan Ramanathan ; Jiri Navratil
Abstract: Preordering of source side sentences has proved to be useful in improving statistical machine translation. Most work has used a parser in the source language along with rules to map the source language word order into the target language word order. The requirement to have a source language parser is a major drawback, which we seek to overcome in this paper. Instead of using a parser and then using rules to order the source side sentence we learn a model that can directly reorder source side sentences to match target word order using a small parallel corpus with highquality word alignments. Our model learns pairwise costs of a word immediately preced- ing another word. We use the Lin-Kernighan heuristic to find the best source reordering efficiently during training and testing and show that it suffices to provide good quality reordering. We show gains in translation performance based on our reordering model for translating from Hindi to English, Urdu to English (with a public dataset), and English to Hindi. For English to Hindi we show that our technique achieves better performance than a method that uses rules applied to the source side English parse.
2 0.69129652 44 emnlp-2011-Domain Adaptation via Pseudo In-Domain Data Selection
Author: Amittai Axelrod ; Xiaodong He ; Jianfeng Gao
Abstract: Xiaodong He Microsoft Research Redmond, WA 98052 xiaohe @mi cro s o ft . com Jianfeng Gao Microsoft Research Redmond, WA 98052 j fgao @mi cro s o ft . com have its own argot, vocabulary or stylistic preferences, such that the corpus characteristics will necWe explore efficient domain adaptation for the task of statistical machine translation based on extracting sentences from a large generaldomain parallel corpus that are most relevant to the target domain. These sentences may be selected with simple cross-entropy based methods, of which we present three. As these sentences are not themselves identical to the in-domain data, we call them pseudo in-domain subcorpora. These subcorpora 1% the size of the original can then used to train small domain-adapted Statistical Machine Translation (SMT) systems which outperform systems trained on the entire corpus. Performance is further improved when we use these domain-adapted models in combination with a true in-domain model. The results show that more training data is not always better, and that best results are attained via proper domain-relevant data selection, as well as combining in- and general-domain systems during decoding. – –
3 0.46887982 123 emnlp-2011-Soft Dependency Constraints for Reordering in Hierarchical Phrase-Based Translation
Author: Yang Gao ; Philipp Koehn ; Alexandra Birch
Abstract: Long-distance reordering remains one of the biggest challenges facing machine translation. We derive soft constraints from the source dependency parsing to directly address the reordering problem for the hierarchical phrasebased model. Our approach significantly improves Chinese–English machine translation on a large-scale task by 0.84 BLEU points on average. Moreover, when we switch the tuning function from BLEU to the LRscore which promotes reordering, we observe total improvements of 1.21 BLEU, 1.30 LRscore and 3.36 TER over the baseline. On average our approach improves reordering precision and recall by 6.9 and 0.3 absolute points, respectively, and is found to be especially effective for long-distance reodering.
4 0.46689004 3 emnlp-2011-A Correction Model for Word Alignments
Author: J. Scott McCarley ; Abraham Ittycheriah ; Salim Roukos ; Bing Xiang ; Jian-ming Xu
Abstract: Models of word alignment built as sequences of links have limited expressive power, but are easy to decode. Word aligners that model the alignment matrix can express arbitrary alignments, but are difficult to decode. We propose an alignment matrix model as a correction algorithm to an underlying sequencebased aligner. Then a greedy decoding algorithm enables the full expressive power of the alignment matrix formulation. Improved alignment performance is shown for all nine language pairs tested. The improved alignments also improved translation quality from Chinese to English and English to Italian.
5 0.45511749 76 emnlp-2011-Language Models for Machine Translation: Original vs. Translated Texts
Author: Gennadi Lembersky ; Noam Ordan ; Shuly Wintner
Abstract: We investigate the differences between language models compiled from original target-language texts and those compiled from texts manually translated to the target language. Corroborating established observations of Translation Studies, we demonstrate that the latter are significantly better predictors of translated sentences than the former, and hence fit the reference set better. Furthermore, translated texts yield better language models for statistical machine translation than original texts.
6 0.44245747 38 emnlp-2011-Data-Driven Response Generation in Social Media
7 0.43700624 108 emnlp-2011-Quasi-Synchronous Phrase Dependency Grammars for Machine Translation
8 0.43445158 1 emnlp-2011-A Bayesian Mixture Model for PoS Induction Using Multiple Features
9 0.42883241 59 emnlp-2011-Fast and Robust Joint Models for Biomedical Event Extraction
10 0.42830616 136 emnlp-2011-Training a Parser for Machine Translation Reordering
11 0.42769185 133 emnlp-2011-The Imagination of Crowds: Conversational AAC Language Modeling using Crowdsourcing and Large Data Sources
12 0.42575651 66 emnlp-2011-Hierarchical Phrase-based Translation Representations
13 0.42336559 68 emnlp-2011-Hypotheses Selection Criteria in a Reranking Framework for Spoken Language Understanding
14 0.42288771 85 emnlp-2011-Learning to Simplify Sentences with Quasi-Synchronous Grammar and Integer Programming
15 0.42283842 35 emnlp-2011-Correcting Semantic Collocation Errors with L1-induced Paraphrases
16 0.42172232 8 emnlp-2011-A Model of Discourse Predictions in Human Sentence Processing
17 0.42130166 58 emnlp-2011-Fast Generation of Translation Forest for Large-Scale SMT Discriminative Training
18 0.42074454 72 emnlp-2011-Improved Transliteration Mining Using Graph Reinforcement
19 0.4205468 22 emnlp-2011-Better Evaluation Metrics Lead to Better Machine Translation
20 0.41943702 20 emnlp-2011-Augmenting String-to-Tree Translation Models with Fuzzy Use of Source-side Syntax