emnlp emnlp2012 emnlp2012-54 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Nan Duan ; Mu Li ; Ming Zhou
Abstract: A forced derivation tree (FDT) of a sentence pair {f, e} denotes a derivation tree that can tpraainrsl {afte, f} idnetono itetss a acc duerraivtea target etrea tnhsaltat cioann e. In this paper, we present an approach that leverages structured knowledge contained in FDTs to train component models for statistical machine translation (SMT) systems. We first describe how to generate different FDTs for each sentence pair in training corpus, and then present how to infer the optimal FDTs based on their derivation and alignment qualities. As the first step in this line of research, we verify the effectiveness of our approach in a BTGbased phrasal system, and propose four FDTbased component models. Experiments are carried out on large scale English-to-Japanese and Chinese-to-English translation tasks, and significant improvements are reported on both translation quality and alignment quality.
Reference: text
sentIndex sentText sentNum sentScore
1 com Abstract A forced derivation tree (FDT) of a sentence pair {f, e} denotes a derivation tree that can tpraainrsl {afte, f} idnetono itetss a acc duerraivtea target etrea tnhsaltat cioann e. [sent-2, score-0.436]
2 In this paper, we present an approach that leverages structured knowledge contained in FDTs to train component models for statistical machine translation (SMT) systems. [sent-3, score-0.23]
3 We first describe how to generate different FDTs for each sentence pair in training corpus, and then present how to infer the optimal FDTs based on their derivation and alignment qualities. [sent-4, score-0.308]
4 Experiments are carried out on large scale English-to-Japanese and Chinese-to-English translation tasks, and significant improvements are reported on both translation quality and alignment quality. [sent-6, score-0.526]
5 The descent of the word alignment quality will lead to inaccurate component models straightforwardly. [sent-11, score-0.201]
6 On the other hand, several component models are designed to supervise the decoding procedures, 445 Mu Li Ming Zhou Microsoft Research Asia Microsoft Research Asia mul i cro s o ft . [sent-12, score-0.149]
7 com @mi which usually rely on training examples extracted from word-aligned sentence pairs, such as distortion models (Tillman, 2004; Xiong et al. [sent-14, score-0.156]
8 Ideally, training examples of models are expected to match most of the situations that could be met in decoding procedures. [sent-18, score-0.146]
9 But actually, plain structures of word alignments are too coarse to provide enough knowledge to ensure this expectation. [sent-19, score-0.123]
10 An FDT of a sentence pair {f, e} denotes a derivation tree that can ttreanncsela ptea f {inft,oe i}ts d aecncoutreaste a target ttrioanns tlrateieon th e. [sent-21, score-0.28]
11 First, using alignments induced from the 1-best FDTs of all sentence pairs, the overall alignment quality of training corpus can be improved. [sent-23, score-0.384]
12 lc L2a0n1g2ua Agseso Pcrioactieosnsi fnogr a Cnodm Cpoumtaptiuotna tilo Lnianlg Nuaist uircasl As the first step in this line of research, our approach is verified in a phrase-based SMT system on both English-to-Japanese and Chinese-to-English translation tasks . [sent-29, score-0.185]
13 Significant improvements are reported on both translation quality (up to 1. [sent-30, score-0.228]
14 2 Forced Derivation Tree for SMT A forced derivation tree (FDT) of a sentence pair {f, e} can be defined as a pair G =< D, A >: • D denotes a derivation that can translate f into e accurately, using a snet t hoaft tt craanns tlaratinosnla rteule fs. [sent-33, score-0.493]
15 i • A denotes a set of word links (i, j) indicating tAha dt ei ∈ e aligns tfo w fj ∈ f. [sent-34, score-0.112]
16 In this section, we first describe how to generate FDTs for each sentence pair in training corpus, which is denoted as the generation step, and then present how to select the optimal FDT for each sentence pair, which is denoted as the inference step. [sent-35, score-0.215]
17 1 Generation We first describe how to generate multiple FDTs for each sentence pair in training corpus C based on the feoarcched se decoding (FD) technique, rwpuhsic Ch performs vthiae the following four steps: 1. [sent-38, score-0.172]
18 In step 3: (1) all partial hypotheses that do not match any sequence in e will be discarded; (2) derivations covering identical source and target words but with different alignments will be kept as different partial candidates, as they can produce different FDTs for 446 the same sentence pair. [sent-43, score-0.256]
19 hFeorre eψa(cGh) )se isnt theenc FeD pair, ddeiflfe scroenret alignment oc Gan. [sent-45, score-0.113]
20 didates can be induced from its different forced derivation trees generated in the generation step, because FD can use phrase pairs with different internal word links extracted from other sentence pairs to recon- struct the given sentence pair, which could lead to better word alignment candidates. [sent-46, score-0.575]
21 Here, memory means nth aen dwh λole translation history that happened in the generation step will be used as the evidence to help us compute features. [sent-49, score-0.185]
22 From the definition we can see that the quality of an FDT directly relates to two aspects: its derivation D and alignments A. [sent-50, score-0.258]
23 (I) The features in the first category measure the derivation quality of each FDT, including: • h(¯ e| f¯), source-to-target translation probability ohf(¯ e a| translation rule r = {f¯, e}. [sent-53, score-0.569]
24 h(f¯| ¯e), target-to-source translation probability oh(ff a | te¯r)a,n stalargteiot-nt oru-sleo r = t{rfa¯,n e}. [sent-56, score-0.185]
25 h(f¯| ¯e) =∑∑{f,e{}f∈,eC}∑∈Cff¯r′afrcaHc(Hf,(ef),(ef)¯(,f¯ e¯ )′, e¯ ) • (4) h# (r), smoothed usage count for translation rhule r = {f¯, e} in the whole generation step. [sent-57, score-0.185]
26 h#(r) =1 + e{−∑{f,e}∈1CfracH(f,e)(f¯, e¯)} (5) In this paper, the sigmoid function is used to make sure that the feature values of different translation rules are in a proper value range. [sent-58, score-0.185]
27 • • hr(G), number of translation rules used in G. [sent-59, score-0.185]
28 For FDTs generated by phrase-based paradigms, oitr can Tbes computed by distortion models; while for FDTs generated by syntax-based paradigms, it can be computed by either parsing models or syntactic LMs (Charniak et al. [sent-61, score-0.081]
29 (II) The features in the second category measure the alignment quality of each FDT, including: • 447 word pair translation probabilities trained from IwBoMrd m paoidre tlrsa (Brown petr al. [sent-65, score-0.398]
30 Due to the fact that all FDTs of each sentence pair share identical translation, we cannot use BLEU as the error criterion any more. [sent-72, score-0.103]
31 We will show in Section 5 that after the inference step, alignment quality can be improved by replacing original alignments of each sentence pair with alignments induced from its 1-best FDT. [sent-74, score-0.564]
32 Imt can , beR split i dneton otwteod child translation states, Sl, which is denoted by the lightschhialdded tr rectangle ptaatiers, a Snd Sr, which is denoted by the swhhaidtee rectangle pair. [sent-91, score-0.361]
33 D anadsh S lines within rectangle pairs denote their internal alignments and solid lines with rows denote BTG rules. [sent-92, score-0.208]
34 (a) uses [·] to combine two translation states, w BhTiGle (rub)l uses ⟨)·⟩ u . [sent-93, score-0.185]
35 • each non-terminal translation state S ∈ N is a 5-tuple {f¯[i,j), ¯e [i′,j′), An¯s,l m, mn s′,t aRte} S1. [sent-96, score-0.185]
36 2 FDT-based Translation Model First, an FDT-based translation model (FDT-TM) is presented for our BTG-based system. [sent-101, score-0.185]
37 2A [·] operation combines thaen dtr Rans floartio an ssi mofp two consecutive source spans [i, m) abninde [m, j) ainn a amtioonnsot oonfi tcw way; wsehciluea ⟨·⟩ operation combines them in an inverted way. [sent-103, score-0.131]
38 448 Given sentence pairs in training corpus with their corresponding FDT spaces, we train FDT-TM in two different ways: (1) The first only uses the 1-best FDT of each sentence pair. [sent-104, score-0.122]
39 Based on each alignment A induced from each 1-best FDT G, all possimblee bilingual phrases are ehx 1tr-abcetsetd. [sent-105, score-0.246]
40 For each sentence pair {f, e}, we first induce n alignments {A1, . [sent-110, score-0.226]
41 Then, all possible bilingual phrases are extr∑acted from the expanded training corpus built using n-best alignments for each sentence pair. [sent-118, score-0.243]
42 The count of each phrase pair is now computed as the sum of posterior probabilities, instead of the sum of absolute frequencies. [sent-119, score-0.085]
43 3 FDT-based Distortion Model In Xiong’s BTG system, training instances of the distortion model (DM) are pruned based on heuristic rules, aiming to keep the training size acceptable. [sent-122, score-0.123]
44 But this will cause the examples remained cannot cover all reordering cases that could be met in real decoding procedures. [sent-123, score-0.359]
45 Given the 1-best FDT G of a sentence pair {f, e}, all Gniovnen-te trhmei 1n-able strta FnDslTat iGon o fs taat seesn {S1, . [sent-125, score-0.103]
46 Sinto} tw aroe child translation Fstoarte esa Skl Sand Skr. [sent-131, score-0.185]
47 Last, the cFoDnTt-aDinMed i isn tr Sainaedn db aSsed on all training instances by a MaxEnt toolkit, which can cover both local and global reordering situations due to its training instance extraction mechanism. [sent-133, score-0.311]
48 Two instances can be extracted from this FDT, where 0 and 1 denote a [·] operation and a ⟨·⟩ operation respectively. [sent-136, score-0.153]
49 se Ind as a label, while boundary words are extracted from each instance’s two phrase pairs and used as lexical features. [sent-138, score-0.087]
50 source language Given the 1-best FDT G of a sentence pair {f, e}, we fiivrestn e txhter1a c-tb a t r FeoDrdTe Gred o source nwcoerd p sequence f′ = {f0′, . [sent-141, score-0.239]
51 Then, all reordered source sentences of training corpus are used to train a source LM. [sent-146, score-0.172]
52 During decoding, each time when a new hypothesis is generated, we obtain its reordered source word sequence as well, compute a LM score based on FDT-SLM and use it as a new feature: ∏J hSLM(f′) = ∏p(fk′|fk′−n+1,. [sent-147, score-0.161]
53 5 FDT-based Rule Sequence Model The last contribution in this section is an FDT-based rule sequence model (FDT-RSM). [sent-151, score-0.102]
54 Given the 1-best FDT G of a sentence pair {f, e}, we fiirvsetn nex thtrea 1c-tb a sequence ooff atr saennstleanticoen p rauirle { applications {r1, . [sent-152, score-0.141]
55 ,Sinal }tra renpsrleatsieonnt sst aante osr dwehroedse target phrases cover e from left to right orderly; 3 foreach S ∈ S¯ in the left-to-right order do 564 a exp ptre an cdt fa¯[i b,jl)afntrko sfmp′a;Sc;etof′; 7 end 8 return f′ as a reordered source word sequence. [sent-161, score-0.277]
56 Figure 3 gives an example of extracting a rule sequence from an FDT. [sent-163, score-0.102]
57 An FDT-RSM is trained based on all rule sequences extracted from training corpus. [sent-164, score-0.093]
58 During decoding, each time when a new hypothesis is generated, we compute an FDT-RSM score based on its rule sequence and use it as a new feature: rk ∏K hRSM(f,e) = ∏p(rk|rk−n+1, . [sent-165, score-0.157]
59 ,Sinal }tra renpsrleatsieonnt states dwehroedse target phrases cover e from left to right orderly; 3 foreach S ∈ S¯ in the left-to-right order do 54 aedxdtra rckt= a p (hf¯r[ai,sj)e, p ¯ ea[i ′r,j (′f)¯)[i, tjo), r e¯ ′;[i′,j′) from S; 6 end 7 return r′ as a rule sequence. [sent-174, score-0.251]
60 Although these two models are trained and computed in an LM style, they are used as reordering features, because they help SMT decoder find better decoding sequences. [sent-176, score-0.265]
61 Figure 3: An example of extracting a rule sequence from an FDT. [sent-178, score-0.102]
62 In order to generate the correct target translation, the desired rule sequence should be r2 ⇒ r3 ⇒ r1. [sent-179, score-0.102]
63 1 Forced Decoding/Alignment Schwartz (2008) used forced decoding to leverage multilingual corpus to improve translation quality; Shen et al. [sent-181, score-0.356]
64 (2008) used forced alignment to train a better phrase segmentation model; Wuebker et al. [sent-182, score-0.243]
65 (2010) used forced alignment to re-estimate translation probabilities using a leaving-one-out strategy. [sent-183, score-0.4]
66 , 2010) used features from syntactic parse trees to reorder source sentences at training and translation time. [sent-191, score-0.234]
67 Recently, DeNero and Uszkoreit (201 1) proposed an approach that induced parse trees automatically from wordaligned training corpus to perform PRO for a phrasebased SMT system, instead of relying on treebanks. [sent-193, score-0.092]
68 First, binary parse trees are induced from wordaligned training corpus. [sent-194, score-0.092]
69 Based on them, a monolingual parsing model and a tree reordering model are trained to pre-reorder source words into the targetlanguage-like order. [sent-195, score-0.245]
70 3 Distortion Models Lexicalized distortion models (Tillman, 2004; Zens and Ney, 2006; Xiong et al. [sent-198, score-0.081]
71 Training instances of these models are extracted from word-aligned sentence pairs. [sent-200, score-0.117]
72 Due to efficiency reasons, only parts of all instances are kept and used in DM training, which cannot cover all possible reordering situations that could be met in decoding. [sent-201, score-0.36]
73 Such instances take both local and global reordering cases into consideration. [sent-203, score-0.238]
74 They used it as a reordering feature in the sense that it helped the decoder to find correct decoding sequences. [sent-207, score-0.265]
75 The difference between their model and our FDT-SLM is that, in their work, the reordered source sequences are extracted based on word alignments only; while in our FDT-SLM, such sequences are obtained based on FDTs. [sent-208, score-0.275]
76 The key difference between FDTRSM and previous work is that the rule sequences are extracted from FDTs, and no parser is needed. [sent-211, score-0.093]
77 In all evaluation data sets, each source sentence has only one reference translation. [sent-219, score-0.095]
78 5M sentence pairs with high translation quality, including LDC2003E07, LDC2003E14, LDC2005T06, LDC2005T10, LDC2005E83, LDC2006E26, LDC2006E34, LDC2006E85 and LDC2006E92. [sent-221, score-0.261]
79 In all evaluation data sets, each source sentence has four reference translations. [sent-225, score-0.095]
80 Default word alignments for both SMT tasks are performed by GIZA++ with the intersect-diag-grow refinement. [sent-226, score-0.123]
81 (2006) is used as the baseline system, with a MaxEnt principle-based lexicalized reordering model integrated, which is used to handle reorderings in decoding. [sent-231, score-0.196]
82 3 Translation Quality on E-J Task We first evaluate the effectiveness of our FDT-based model training approach on E-J translation task, and present evaluation results in Table 1, in which BTG denotes the performance of the baseline system. [sent-235, score-0.232]
83 2, we tried different sizes of n-best FDTs to induce alignments for phrase extraction and found the optimal choice is 5. [sent-239, score-0.151]
84 Besides, in order to make full use of the training corpus, for those sentence pairs that are failed in FD, we just use their original word alignments to extract bilingual phrases. [sent-240, score-0.273]
85 This makes our new DM can cover both local and global reordering situations that might be met in decoding procedures. [sent-245, score-0.387]
86 We analyze decoding-logs and find that the reordered source sequences of n-best translations are very similar, which, we think, can explain why improvements of using this model are so limited. [sent-251, score-0.123]
87 4 Translation Quality on C-E Task We then evaluate the effectiveness of our FDT-based Table 3 we can see that the F-score improves on all language pairs when using alignments modeltrainingapproachonC-Etranslationtask, and induced from 1-best FDTs, rather than GIZA++. [sent-281, score-0.212]
88 This might be due to the fact that the word alignment quality in C-E task is more reliable than that in E-J task for TM and DM training. [sent-285, score-0.156]
89 5 Effect on Alignment Quality We compare the qualities of alignments predicted by GIZA++ and alignments induced from 1-best FDTs. [sent-287, score-0.349]
90 For E-J task, 575 English-Japanese sentence pairs are manually annotated with word alignments. [sent-288, score-0.076]
91 382 sentence pairs are used as the dev set, and the other 193 sentence pairs are used as the test set. [sent-289, score-0.207]
92 For C-E task, 491 Chinese-English sentence pairs are manually annotated with word alignments. [sent-290, score-0.076]
93 250 sentence pairs are used as the dev set, and the other 241 sentence pairs are used as the test set. [sent-291, score-0.207]
94 Comparing to C-E language pair (S-V-O), E-J language pair (S-O-V) has much lower F-scores, due to its very different word order. [sent-294, score-0.114]
95 9005%%)) Table 3: Comparison of alignment qualities predicted by GIZA++ and induced from 1-best FDTs. [sent-307, score-0.216]
96 This is because FDTs can provide more deterministic and structured knowledge for training instance extraction, which can cover both local and global reordering cases. [sent-310, score-0.241]
97 This is feasible in the sense that FDTs can provide both tree-like structures and reordering information. [sent-324, score-0.196]
98 Automatically learning sourceside reordering rules for large scale machine translation, In Proceedings of the Conference on Computational Linguistics. [sent-358, score-0.196]
99 A lightweight evaluation framework for machine translation reordering, In Proceedings of the Sixth Workshop on Statistical Machine Translation. [sent-405, score-0.185]
100 Maximum entropy based phrase reordering model for statistical machine translation, In Proceedings of the Association for Computational Linguistics. [sent-420, score-0.224]
wordName wordTfidf (topN-words)
[('fdts', 0.591), ('fdt', 0.438), ('reordering', 0.196), ('translation', 0.185), ('smt', 0.169), ('btg', 0.148), ('alignments', 0.123), ('alignment', 0.113), ('dm', 0.104), ('forced', 0.102), ('gk', 0.098), ('xiong', 0.098), ('derivation', 0.092), ('fd', 0.082), ('distortion', 0.081), ('bilingual', 0.074), ('reordered', 0.074), ('decoding', 0.069), ('pro', 0.069), ('rule', 0.064), ('induced', 0.059), ('frach', 0.057), ('ihi', 0.057), ('skl', 0.057), ('pair', 0.057), ('dev', 0.055), ('rk', 0.055), ('rectangle', 0.055), ('giza', 0.053), ('sk', 0.053), ('vaswani', 0.049), ('source', 0.049), ('met', 0.049), ('denotes', 0.047), ('bleu', 0.046), ('sentence', 0.046), ('cover', 0.045), ('component', 0.045), ('qualities', 0.044), ('sr', 0.044), ('qun', 0.044), ('quality', 0.043), ('instances', 0.042), ('moore', 0.042), ('asia', 0.041), ('operation', 0.041), ('athlle', 0.038), ('dwehroedse', 0.038), ('egn', 0.038), ('fdtbased', 0.038), ('ghm', 0.038), ('menezes', 0.038), ('mrm', 0.038), ('orderly', 0.038), ('renpsrleatsieonnt', 0.038), ('skr', 0.038), ('sof', 0.038), ('taha', 0.038), ('tbes', 0.038), ('tillman', 0.038), ('ttrioanns', 0.038), ('hermann', 0.038), ('sequence', 0.038), ('birch', 0.037), ('hf', 0.036), ('mt', 0.035), ('cro', 0.035), ('fj', 0.035), ('denoted', 0.033), ('states', 0.033), ('wordaligned', 0.033), ('banchs', 0.033), ('agr', 0.033), ('arne', 0.033), ('ede', 0.033), ('foreach', 0.033), ('fraser', 0.033), ('mauser', 0.033), ('shouxun', 0.033), ('wuebker', 0.033), ('galley', 0.032), ('sl', 0.032), ('pairs', 0.03), ('mi', 0.03), ('franz', 0.03), ('hr', 0.03), ('ph', 0.03), ('aligns', 0.03), ('genzel', 0.03), ('hfe', 0.03), ('uszkoreit', 0.03), ('venugopal', 0.03), ('denero', 0.029), ('terminal', 0.029), ('maxent', 0.029), ('ef', 0.029), ('extracted', 0.029), ('situations', 0.028), ('hi', 0.028), ('phrase', 0.028)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000002 54 emnlp-2012-Forced Derivation Tree based Model Training to Statistical Machine Translation
Author: Nan Duan ; Mu Li ; Ming Zhou
Abstract: A forced derivation tree (FDT) of a sentence pair {f, e} denotes a derivation tree that can tpraainrsl {afte, f} idnetono itetss a acc duerraivtea target etrea tnhsaltat cioann e. In this paper, we present an approach that leverages structured knowledge contained in FDTs to train component models for statistical machine translation (SMT) systems. We first describe how to generate different FDTs for each sentence pair in training corpus, and then present how to infer the optimal FDTs based on their derivation and alignment qualities. As the first step in this line of research, we verify the effectiveness of our approach in a BTGbased phrasal system, and propose four FDTbased component models. Experiments are carried out on large scale English-to-Japanese and Chinese-to-English translation tasks, and significant improvements are reported on both translation quality and alignment quality.
2 0.26790011 67 emnlp-2012-Inducing a Discriminative Parser to Optimize Machine Translation Reordering
Author: Graham Neubig ; Taro Watanabe ; Shinsuke Mori
Abstract: This paper proposes a method for learning a discriminative parser for machine translation reordering using only aligned parallel text. This is done by treating the parser’s derivation tree as a latent variable in a model that is trained to maximize reordering accuracy. We demonstrate that efficient large-margin training is possible by showing that two measures of reordering accuracy can be factored over the parse tree. Using this model in the pre-ordering framework results in significant gains in translation accuracy over standard phrasebased SMT and previously proposed unsupervised syntax induction methods.
3 0.17800505 31 emnlp-2012-Cross-Lingual Language Modeling with Syntactic Reordering for Low-Resource Speech Recognition
Author: Ping Xu ; Pascale Fung
Abstract: This paper proposes cross-lingual language modeling for transcribing source resourcepoor languages and translating them into target resource-rich languages if necessary. Our focus is to improve the speech recognition performance of low-resource languages by leveraging the language model statistics from resource-rich languages. The most challenging work of cross-lingual language modeling is to solve the syntactic discrepancies between the source and target languages. We therefore propose syntactic reordering for cross-lingual language modeling, and present a first result that compares inversion transduction grammar (ITG) reordering constraints to IBM and local constraints in an integrated speech transcription and translation system. Evaluations on resource-poor Cantonese speech transcription and Cantonese to resource-rich Mandarin translation tasks show that our proposed approach improves the system performance significantly, up to 3.4% relative WER reduction in Cantonese transcription and 13.3% relative bilingual evaluation understudy (BLEU) score improvement in Mandarin transcription compared with the system without reordering.
4 0.16867222 109 emnlp-2012-Re-training Monolingual Parser Bilingually for Syntactic SMT
Author: Shujie Liu ; Chi-Ho Li ; Mu Li ; Ming Zhou
Abstract: The training of most syntactic SMT approaches involves two essential components, word alignment and monolingual parser. In the current state of the art these two components are mutually independent, thus causing problems like lack of rule generalization, and violation of syntactic correspondence in translation rules. In this paper, we propose two ways of re-training monolingual parser with the target of maximizing the consistency between parse trees and alignment matrices. One is targeted self-training with a simple evaluation function; the other is based on training data selection from forced alignment of bilingual data. We also propose an auxiliary method for boosting alignment quality, by symmetrizing alignment matrices with respect to parse trees. The best combination of these novel methods achieves 3 Bleu point gain in an IWSLT task and more than 1 Bleu point gain in NIST tasks. 1
5 0.14529538 86 emnlp-2012-Locally Training the Log-Linear Model for SMT
Author: Lemao Liu ; Hailong Cao ; Taro Watanabe ; Tiejun Zhao ; Mo Yu ; Conghui Zhu
Abstract: In statistical machine translation, minimum error rate training (MERT) is a standard method for tuning a single weight with regard to a given development data. However, due to the diversity and uneven distribution of source sentences, there are two problems suffered by this method. First, its performance is highly dependent on the choice of a development set, which may lead to an unstable performance for testing. Second, translations become inconsistent at the sentence level since tuning is performed globally on a document level. In this paper, we propose a novel local training method to address these two problems. Unlike a global training method, such as MERT, in which a single weight is learned and used for all the input sentences, we perform training and testing in one step by learning a sentencewise weight for each input sentence. We pro- pose efficient incremental training methods to put the local training into practice. In NIST Chinese-to-English translation tasks, our local training method significantly outperforms MERT with the maximal improvements up to 2.0 BLEU points, meanwhile its efficiency is comparable to that of the global method.
6 0.1390187 1 emnlp-2012-A Bayesian Model for Learning SCFGs with Discontiguous Rules
7 0.13271898 35 emnlp-2012-Document-Wide Decoding for Phrase-Based Statistical Machine Translation
8 0.12312829 42 emnlp-2012-Entropy-based Pruning for Phrase-based Machine Translation
9 0.11281212 82 emnlp-2012-Left-to-Right Tree-to-String Decoding with Prediction
10 0.10283571 127 emnlp-2012-Transforming Trees to Improve Syntactic Convergence
11 0.10280627 11 emnlp-2012-A Systematic Comparison of Phrase Table Pruning Techniques
12 0.098203659 128 emnlp-2012-Translation Model Based Cross-Lingual Language Model Adaptation: from Word Models to Phrase Models
13 0.089539945 16 emnlp-2012-Aligning Predicates across Monolingual Comparable Texts using Graph-based Clustering
14 0.071531057 25 emnlp-2012-Bilingual Lexicon Extraction from Comparable Corpora Using Label Propagation
15 0.059000835 2 emnlp-2012-A Beam-Search Decoder for Grammatical Error Correction
16 0.058055095 108 emnlp-2012-Probabilistic Finite State Machines for Regression-based MT Evaluation
17 0.057285201 58 emnlp-2012-Generalizing Sub-sentential Paraphrase Acquisition across Original Signal Type of Text Pairs
18 0.055028453 39 emnlp-2012-Enlarging Paraphrase Collections through Generalization and Instantiation
19 0.054790128 74 emnlp-2012-Language Model Rest Costs and Space-Efficient Storage
20 0.053415697 135 emnlp-2012-Using Discourse Information for Paraphrase Extraction
topicId topicWeight
[(0, 0.219), (1, -0.228), (2, -0.277), (3, -0.027), (4, -0.101), (5, -0.119), (6, -0.14), (7, 0.071), (8, -0.01), (9, -0.103), (10, -0.111), (11, 0.02), (12, -0.059), (13, 0.103), (14, -0.137), (15, 0.083), (16, -0.005), (17, 0.077), (18, 0.225), (19, 0.073), (20, 0.038), (21, -0.121), (22, -0.099), (23, -0.015), (24, -0.015), (25, 0.019), (26, 0.05), (27, 0.065), (28, 0.023), (29, 0.029), (30, -0.032), (31, 0.062), (32, 0.043), (33, -0.027), (34, 0.066), (35, 0.069), (36, 0.029), (37, -0.04), (38, -0.052), (39, 0.011), (40, 0.051), (41, -0.005), (42, -0.042), (43, -0.006), (44, -0.02), (45, -0.057), (46, 0.066), (47, -0.009), (48, 0.046), (49, 0.045)]
simIndex simValue paperId paperTitle
same-paper 1 0.91913569 54 emnlp-2012-Forced Derivation Tree based Model Training to Statistical Machine Translation
Author: Nan Duan ; Mu Li ; Ming Zhou
Abstract: A forced derivation tree (FDT) of a sentence pair {f, e} denotes a derivation tree that can tpraainrsl {afte, f} idnetono itetss a acc duerraivtea target etrea tnhsaltat cioann e. In this paper, we present an approach that leverages structured knowledge contained in FDTs to train component models for statistical machine translation (SMT) systems. We first describe how to generate different FDTs for each sentence pair in training corpus, and then present how to infer the optimal FDTs based on their derivation and alignment qualities. As the first step in this line of research, we verify the effectiveness of our approach in a BTGbased phrasal system, and propose four FDTbased component models. Experiments are carried out on large scale English-to-Japanese and Chinese-to-English translation tasks, and significant improvements are reported on both translation quality and alignment quality.
2 0.84709561 67 emnlp-2012-Inducing a Discriminative Parser to Optimize Machine Translation Reordering
Author: Graham Neubig ; Taro Watanabe ; Shinsuke Mori
Abstract: This paper proposes a method for learning a discriminative parser for machine translation reordering using only aligned parallel text. This is done by treating the parser’s derivation tree as a latent variable in a model that is trained to maximize reordering accuracy. We demonstrate that efficient large-margin training is possible by showing that two measures of reordering accuracy can be factored over the parse tree. Using this model in the pre-ordering framework results in significant gains in translation accuracy over standard phrasebased SMT and previously proposed unsupervised syntax induction methods.
3 0.74687111 31 emnlp-2012-Cross-Lingual Language Modeling with Syntactic Reordering for Low-Resource Speech Recognition
Author: Ping Xu ; Pascale Fung
Abstract: This paper proposes cross-lingual language modeling for transcribing source resourcepoor languages and translating them into target resource-rich languages if necessary. Our focus is to improve the speech recognition performance of low-resource languages by leveraging the language model statistics from resource-rich languages. The most challenging work of cross-lingual language modeling is to solve the syntactic discrepancies between the source and target languages. We therefore propose syntactic reordering for cross-lingual language modeling, and present a first result that compares inversion transduction grammar (ITG) reordering constraints to IBM and local constraints in an integrated speech transcription and translation system. Evaluations on resource-poor Cantonese speech transcription and Cantonese to resource-rich Mandarin translation tasks show that our proposed approach improves the system performance significantly, up to 3.4% relative WER reduction in Cantonese transcription and 13.3% relative bilingual evaluation understudy (BLEU) score improvement in Mandarin transcription compared with the system without reordering.
4 0.63932413 109 emnlp-2012-Re-training Monolingual Parser Bilingually for Syntactic SMT
Author: Shujie Liu ; Chi-Ho Li ; Mu Li ; Ming Zhou
Abstract: The training of most syntactic SMT approaches involves two essential components, word alignment and monolingual parser. In the current state of the art these two components are mutually independent, thus causing problems like lack of rule generalization, and violation of syntactic correspondence in translation rules. In this paper, we propose two ways of re-training monolingual parser with the target of maximizing the consistency between parse trees and alignment matrices. One is targeted self-training with a simple evaluation function; the other is based on training data selection from forced alignment of bilingual data. We also propose an auxiliary method for boosting alignment quality, by symmetrizing alignment matrices with respect to parse trees. The best combination of these novel methods achieves 3 Bleu point gain in an IWSLT task and more than 1 Bleu point gain in NIST tasks. 1
5 0.63267183 1 emnlp-2012-A Bayesian Model for Learning SCFGs with Discontiguous Rules
Author: Abby Levenberg ; Chris Dyer ; Phil Blunsom
Abstract: We describe a nonparametric model and corresponding inference algorithm for learning Synchronous Context Free Grammar derivations for parallel text. The model employs a Pitman-Yor Process prior which uses a novel base distribution over synchronous grammar rules. Through both synthetic grammar induction and statistical machine translation experiments, we show that our model learns complex translational correspondences— including discontiguous, many-to-many alignments—and produces competitive translation results. Further, inference is efficient and we present results on significantly larger corpora than prior work.
6 0.62164617 86 emnlp-2012-Locally Training the Log-Linear Model for SMT
7 0.47721899 128 emnlp-2012-Translation Model Based Cross-Lingual Language Model Adaptation: from Word Models to Phrase Models
8 0.45743009 82 emnlp-2012-Left-to-Right Tree-to-String Decoding with Prediction
9 0.43299508 35 emnlp-2012-Document-Wide Decoding for Phrase-Based Statistical Machine Translation
10 0.40165088 127 emnlp-2012-Transforming Trees to Improve Syntactic Convergence
11 0.37066451 16 emnlp-2012-Aligning Predicates across Monolingual Comparable Texts using Graph-based Clustering
12 0.36211681 118 emnlp-2012-Source Language Adaptation for Resource-Poor Machine Translation
13 0.35732275 25 emnlp-2012-Bilingual Lexicon Extraction from Comparable Corpora Using Label Propagation
14 0.34475881 42 emnlp-2012-Entropy-based Pruning for Phrase-based Machine Translation
15 0.32323229 108 emnlp-2012-Probabilistic Finite State Machines for Regression-based MT Evaluation
16 0.253878 18 emnlp-2012-An Empirical Investigation of Statistical Significance in NLP
17 0.2515848 94 emnlp-2012-Multiple Aspect Summarization Using Integer Linear Programming
18 0.2503314 41 emnlp-2012-Entity based QA Retrieval
19 0.24848273 74 emnlp-2012-Language Model Rest Costs and Space-Efficient Storage
20 0.23331122 11 emnlp-2012-A Systematic Comparison of Phrase Table Pruning Techniques
topicId topicWeight
[(2, 0.011), (11, 0.011), (14, 0.013), (16, 0.034), (25, 0.011), (34, 0.112), (41, 0.014), (54, 0.277), (60, 0.095), (63, 0.034), (64, 0.01), (65, 0.029), (70, 0.021), (74, 0.08), (75, 0.044), (76, 0.04), (79, 0.03), (80, 0.013), (86, 0.015), (95, 0.018), (96, 0.011)]
simIndex simValue paperId paperTitle
same-paper 1 0.70472437 54 emnlp-2012-Forced Derivation Tree based Model Training to Statistical Machine Translation
Author: Nan Duan ; Mu Li ; Ming Zhou
Abstract: A forced derivation tree (FDT) of a sentence pair {f, e} denotes a derivation tree that can tpraainrsl {afte, f} idnetono itetss a acc duerraivtea target etrea tnhsaltat cioann e. In this paper, we present an approach that leverages structured knowledge contained in FDTs to train component models for statistical machine translation (SMT) systems. We first describe how to generate different FDTs for each sentence pair in training corpus, and then present how to infer the optimal FDTs based on their derivation and alignment qualities. As the first step in this line of research, we verify the effectiveness of our approach in a BTGbased phrasal system, and propose four FDTbased component models. Experiments are carried out on large scale English-to-Japanese and Chinese-to-English translation tasks, and significant improvements are reported on both translation quality and alignment quality.
2 0.54800534 67 emnlp-2012-Inducing a Discriminative Parser to Optimize Machine Translation Reordering
Author: Graham Neubig ; Taro Watanabe ; Shinsuke Mori
Abstract: This paper proposes a method for learning a discriminative parser for machine translation reordering using only aligned parallel text. This is done by treating the parser’s derivation tree as a latent variable in a model that is trained to maximize reordering accuracy. We demonstrate that efficient large-margin training is possible by showing that two measures of reordering accuracy can be factored over the parse tree. Using this model in the pre-ordering framework results in significant gains in translation accuracy over standard phrasebased SMT and previously proposed unsupervised syntax induction methods.
3 0.51557678 109 emnlp-2012-Re-training Monolingual Parser Bilingually for Syntactic SMT
Author: Shujie Liu ; Chi-Ho Li ; Mu Li ; Ming Zhou
Abstract: The training of most syntactic SMT approaches involves two essential components, word alignment and monolingual parser. In the current state of the art these two components are mutually independent, thus causing problems like lack of rule generalization, and violation of syntactic correspondence in translation rules. In this paper, we propose two ways of re-training monolingual parser with the target of maximizing the consistency between parse trees and alignment matrices. One is targeted self-training with a simple evaluation function; the other is based on training data selection from forced alignment of bilingual data. We also propose an auxiliary method for boosting alignment quality, by symmetrizing alignment matrices with respect to parse trees. The best combination of these novel methods achieves 3 Bleu point gain in an IWSLT task and more than 1 Bleu point gain in NIST tasks. 1
4 0.51282817 35 emnlp-2012-Document-Wide Decoding for Phrase-Based Statistical Machine Translation
Author: Christian Hardmeier ; Joakim Nivre ; Jorg Tiedemann
Abstract: Independence between sentences is an assumption deeply entrenched in the models and algorithms used for statistical machine translation (SMT), particularly in the popular dynamic programming beam search decoding algorithm. This restriction is an obstacle to research on more sophisticated discourse-level models for SMT. We propose a stochastic local search decoding method for phrase-based SMT, which permits free document-wide dependencies in the models. We explore the stability and the search parameters ofthis method and demonstrate that it can be successfully used to optimise a document-level semantic language model. 1 Motivation In the field oftranslation studies, it is undisputed that discourse-wide context must be considered care- fully for good translation results (Hatim and Mason, 1990). By contrast, the state of the art in statistical machine translation (SMT), despite significant advances in the last twenty years, still assumes that texts can be translated sentence by sentence under strict independence assumptions, even though it is well known that certain linguistic phenomena such as pronominal anaphora cannot be translated correctly without referring to extra-sentential context. This is true both for the phrase-based and the syntaxbased approach to SMT. In the rest of this paper, we shall concentrate on phrase-based SMT. One reason why it is difficult to experiment with document-wide models for phrase-based SMT is that the dynamic programming (DP) algorithm 1179 which has been used almost exclusively for decoding SMT models in the recent literature has very strong assumptions of locality built into it. DP beam search for phrase-based SMT was described by Koehn et al. (2003), extending earlier work on word-based SMT (Tillmann et al., 1997; Och et al., 2001 ; Tillmann and Ney, 2003). This algorithm con- structs output sentences by starting with an empty hypothesis and adding output words at the end until translations for all source words have been generated. The core models of phrase-based SMT, in particular the n-gram language model (LM), only depend on a constant number of output words to the left of the word being generated. This fact is exploited by the search algorithm with a DP technique called hypothesis recombination (Och et al., 2001), which permits the elimination of hypotheses from the search space if they coincide in a certain number of final words with a better hypothesis and no future expansion can possibly invert the relative ranking of the two hypotheses under the given models. Hypothesis recombination achieves a substantial reduction of the search space without affecting search optimality and makes it possible to use aggressive pruning techniques for fast search while still obtaining good results. The downside of this otherwise excellent approach is that it only works well with models that have a local dependency structure similar to that of an n-gram language model, so they only depend on a small context window for each target word. Sentence-local models with longer dependencies can be added, but doing so greatly increases the risk for search errors by inhibiting hypothesis recombination. Cross-sentence dependencies cannot be directly integrated into DP SMT decoding in LParnogcue agdein Lgesa ornf tihneg, 2 p0a1g2e Jso 1in17t C9–o1n1f9e0re,n Jce ju on Is Elanmdp,ir Kicoarlea M,e 1t2h–o1d4s J iunly N 2a0tu1r2a.l ? Lc a2n0g1u2ag Aes Psorcoicaetsiosin fgo arn Cdo Cmopmutpauti oantiaoln Lailn Ngautiustriacls any obvious way, especially if joint optimisation of a number of interdependent decisions over an entire document is required. Research into models with a more varied, non-local dependency structure is to some extent stifled by the difficulty of decoding such models effectively, as can be seen by the problems some researchers encountered when they attempted to solve discourse-level problems. Consider, for instance, the work on cache-based language models by Tiedemann (2010) and Gong et al. (201 1), where error propagation was a serious issue, or the works on pronominal anaphora by Le Nagard and Koehn (2010), who implemented cross-sentence dependencies with an ad-hoc two-pass decoding strategy, and Hardmeier and Federico (2010) with the use of an external decoder driver to manage backward-only dependencies between sentences. In this paper, we present a method for decoding complete documents in phrase-based SMT. Our decoder uses a local search approach whose state consists of a complete translation of an entire document at any time. The initial state is improved by the application of a series of operations using a hill climbing strategy to find a (local) maximum of the score function. This setup gives us complete freedom to define scoring functions over the entire document. Moreover, by optionally initialising the state with the output of a traditional DP decoder, we can ensure that the final hypothesis is no worse than what would have been found by DP search alone. We start by describing the decoding algorithm and the state operations used by our decoder, then we present empirical results demonstrating the effectiveness of our approach and its usability with a document-level semantic language model, and finally we discuss some related work. 2 SMT Decoding by Hill Climbing In this section, we formally describe the phrasebased SMT model implemented by our decoder as well as the decoding algorithm we use. 2.1 SMT Model Our decoder is based on local search, so its state at any time is a representation of a complete translation of the entire document. Even though the decoder operates at the document level, it is important to keep 1180 track of sentence boundaries, and the individual operations that are applied to the state are still confined to sentence scope, so it is useful to decompose the state of a document into the state of its sentences, and we define the overall state S as a sequence of sentence states: S = S1S2 . . .SN, (1) where N is the number of sentences. This implies that we constrain the decoder to emit exactly one output sentence per input sentence. Let ibe the number of a sentence and mi the number of input tokens of this sentence, p and q (with 1 ≤ p ≤ q ≤ mi) be positions in the input sentence a1n ≤d [p; q] qde ≤no mte the set ofpositions from p up to and including q. We say that [p; q] precedes [p0; q0], or [p; q] ≺ [p0; q0], if q < p0. Let Φi([p; q]) be the set of t[pra;nqs]l ≺atio [pns for the source phrase covering positions [p; q] in the input sentence ias given by the phrase table. We call A = h[p; q] ,φi an anchored phrase pair w.it Wh coverage C(A) = [p; q] nif a φ ∈ Φi([p; q]) sise a target phrase translating =th [ep source w∈o Φrds at positions [p; q] . Then a sequence of ni anchored phrase pairs Si = A1A2 . . .Ani (2) is a valid sentence state for sentence iif the following two conditions hold: 1. The coverage sets C(Aj) for j in 1, . . . , ni are mutually disjoint, and 2. the anchored phrase pairs jointly cover the complete input sentence, or [niC(Aj) = [1;mi]. (3) [j=1 Let f(S) be a scoring function mapping a state S to a real number. As usual in SMT, it is assumed that the scoring function can be decomposed into a linear combination of K feature functions hk(S), each with a constant weight λk, so f(S) =k∑K=1λkhk(S). (4) The problem addressed by the decoder is the search for the state with maximal score, such that Sˆ Sˆ = argSmaxf(S). (5) The feature functions implemented in our baseline system are identical to the ones found in the popular Moses SMT system (Koehn et al., 2007). In particular, our decoder has the following feature functions: 1. phrase translation scores provided by the phrase table including forward and backward conditional probabilities, lexical weights and a phrase penalty (Koehn et al., 2003), 2. n-gram language model scores implemented with the KenLM toolkit (Heafield, 2011), 3. a word penalty score, 4. a distortion model with geometric (Koehn et al., 2003), and decay 5. a feature indicating the number of times a given distortion limit is exceeded in the current state. In our experiments, the last feature is used with a fixed weight of negative infinity in order to limit the gaps between the coverage sets of adjacent anchored phrase pairs to a maximum value. In DP search, the distortion limit is usually enforced directly by the search algorithm and is not added as a feature. In our decoder, however, this restriction is not required to limit complexity, so we decided to add it among the scoring models. 2.2 Decoding Algorithm The decoding algorithm we use (algorithm 1) is very simple. It starts with a given initial document state. In the main loop, which extends from line 3 to line 12, it generates a successor state S0 for the current state S by calling the function Neighbour, which non-deterministically applies one of the operations described in section 3 of this paper to S. The score of the new state is compared to that of the previous one. If it meets a given acceptance criterion, S0 becomes the current state, else search continues from the previous state S. For the experiments in this paper, we use the hill climbing acceptance criterion, which simply accepts a new state if its score is higher than that of the current state. Other acceptance criteria are possible and could be used to endow the search algorithm with stochastic behaviour. 1181 The main loop is repeated until a maximum number of steps (step limit) is reached or until a maximum number of moves are rejected in a row (rejection limit). Algorithm 1 Decoding algorithm Input: an initial document state S; search parameters maxsteps and maxrejected Output: a modified document state 1: nsteps ← 0 2: nrejected ← 0 3: nwrhejileec nsteps < maxsteps and nrejected < maxrejected do 4: S0 ← Neighbour (S) 5: if Accept (f(S0) , f(S)) then 6: S ← S0 7: nrejected ← 0 8: elsner 9: nrejected ← nrejected + 1 10: enndr eifj 11: nsteps ← nsteps + 1 12: ennds wtephsile ← 13: return S A notable difference between this algorithm and other hill climbing algorithms that have been used for SMT decoding (Germann et al., 2004; Langlais et al., 2007) is its non-determinism. Previous work for sentence-level decoding employed a steepest ascent strategy which amounts to enumerating the complete neighbourhood of the current state as defined by the state operations and selecting the next state to be the best state found in the neighbourhood of the current one. Enumerating all neighbours of a given state, costly as it is, has the advantage that it makes it easy to prove local optimality of a state by recognising that all possible successor states have lower scores. It can be rather inefficient, since at every step only one modification will be adopted; many of the modifications that are discarded will very likely be generated anew in the next iteration. As we extend the decoder to the document level, the size of the neighbourhood that would have to be explored in this way increases considerably. Moreover, the inefficiency of the steepest ascent approach potentially increases as well. Very likely, a promising move in one sentence will remain promising after a modification has been applied to another sentence, even though this is not guaranteed to be true in the presence of cross-sentence models. We therefore adopt a first-choice hill climbing strategy that non-deterministically generates successor states and accepts the first one that meets the acceptance criterion. This frees us from the necessity of generating the full set of successors for each state. On the downside, if the full successor set is not known, it is no longer possible to prove local optimality of a state, so we are forced to use a different condition for halting the search. We use a combination of two limits: The step limit is a hard limit on the resources the user is willing to expend on the search problem. The value of the rejection limit determines how much of the neighbourhood is searched for better successors before a state is accepted as a solution; it is related to the probability that a state returned as a solution is in fact locally optimal. To simplify notations in the description of the individual state operations, we write Si −→ Si0 (6) to signify that a state operation, when presented with a document state as in equation 1 and acting on sentence i, returns a new document state of S0 = S1 . . .Si−1 Si0 Si+1 . . .SN. (7) Similarly, Si : Aj . . .Aj+h−1 −→ A01 . . .A0h0 (8) is equivalent to Si −→ A1 . . .Aj−1 A01 . . .A0h0 Aj+h . . .Ani (9) and indicates that the operation returns a state in which a sequence of h consecutive anchored phrase pairs has been replaced by another sequence of h0 anchored phrase pairs. 2.3 Efficiency Considerations When implementing the feature functions for the decoder, we have to exercise some care to avoid recomputing scores for the whole document at every iteration. To achieve this, the scores are computed completely only once, at the beginning of the decoding run. In subsequent iterations, scoring functions are presented with the scores of the previous 1182 iteration and a list of modifications produced by the state operation, a set of tuples hi, r, s,A01 . . .A0h0i, each indicating tthioant ,t ahe s edto ocfu tmupelnets s hhio,ru,sld, Abe modifii,e eda as described by Si :Ar . . .As −→ A01 . . .A0h0 . (10) If a feature function is decomposable in some way, as all the standard features developed under the constraints of DP search are, it can then update the state simply by subtracting and adding score components pertaining to the modified parts of the document. Feature functions have the possibility to store their own state information along with the document state to make sure the required information is available. Thus, the framework makes it possible to exploit decomposability for efficient scoring without impos- ing any particular decomposition on the features as beam search does. To make scoring even more efficient, scores are computed in two passes: First, every feature function is asked to provide an upper bound on the score that will be obtained for the new state. In some cases, it is possible to calculate reasonable upper bounds much more efficiently than computing the exact feature value. If the upper bound fails to meet the acceptance criterion, the new state is discarded right away; if not, the full score is computed and the acceptance criterion is tested again. Among the basic SMT models, this two-pass strategy is only used for the n-gram LM, which requires fairly expensive parameter lookups for scoring. The scores of all the other baseline models are fully computed during the first scoring pass. The n-gram model is more complex. In its state information, it keeps track of the LM score and LM library state for each word. The first scoring pass then identifies the words whose LM scores are affected by the current search step. This includes the words changed by the search operation as well as the words whose LM history is modified. The range of the history de- pendencies can be determined precisely by considering the “valid state length” information provided by the KenLM library. In the first pass, the LM scores of the affected words are subtracted from the total score. The model only looks up the new LM scores for the affected words and updates the total score if the new search state passes the first acceptance check. This two-pass scoring approach allows us to avoid LM lookups altogether for states that will be rejected anyhow because of low scores from the other models, e. g. because the distortion limit is violated. Model score updates become more complex and slower as the number of dependencies of a model increases. While our decoding algorithm does not impose any formal restrictions on the number or type of dependencies that can be handled, there will be practical limits beyond which decoding becomes unacceptably slow or the scoring code becomes very difficult to maintain. These limits are however fairly independent of the types of dependencies handled by a model, which permits the exploration of more varied model types than those handled by DP search. 2.4 State Initialisation Before the hill climbing decoding algorithm can be run, an initial state must be generated. The closer the initial state is to an optimum, the less work remains to be done for the algorithm. If the algorithm is to be self-contained, initialisation must be relatively uninformed and can only rely on some general prior assumptions about what might be a good initial guess. On the other hand, if optimal results are sought after, it pays off to invest some effort into a good starting point. One way to do this is to run DP search first. For uninformed initialisation, we chose to implement a very simple procedure based only on the observation that, at least for language pairs involving the major European languages, it is usually a good guess to keep the word order of the output very similar to that of the input. We therefore create the initial state by selecting, for each sentence in the document, a sequence of anchored phrase pairs covering the input sentence in monotonic order, that is, such that for all pairs of adjacent anchored phrase pairs Aj and Aj+1, we have that C(Aj) ≺ C(Aj+1 ). For initialisation with DP search, we first run the Moses decoder (Koehn et al., 2007) with default search parameters and the same models as those used by our decoder. Then we extract the best output hypothesis from the search graph of the decoder and map it into a sequence of anchored phrase pairs in the obvious way. When the document-level decoder is used with models that are incompatible with beam search, Moses can be run with a subset of the models in order to find an approximation of the solution 1183 which is then refined with the complete feature set. 3 State Operations Given a document state S, the decoder uses a neighbourhood function Neighbour to simulate a move in the state space. The neighbourhood function nondeterministically selects a type of state operation and a location in the document to apply it to and returns the resulting new state. We use a set of three operations that has the property that every possible document state can be reached from every other state in a sequence of moves. Designing operations for state transitions in local search for phrase-based SMT is a problem that has been addressed in the literature (Langlais et al., 2007; Arun et al., 2010). Our decoder’s first- choice hill climbing strategy never enumerates the full neighbourhood of a state. We therefore place less emphasis than previous work on defining a compact neighbourhood, but allow the decoder to make quite extensive changes to a state in a single step with a certain probability. Otherwise our operations are similar to those used by Arun et al. (2010). All of the operations described in this paper make changes to a single sentence only. Each time it is called, the Neighbour function selects a sentence in the document with a probability proportional to the number of input tokens in each sentence to ensure a fair distribution ofthe decoder’s attention over the words in the document regardless of varying sentence lengths. 3.1 Changing Phrase Translations The change-phrase-translation operation replaces the translation of a single phrase with a random translation with the same coverage taken from the phrase table. Formally, the operation selects an anchored phrase pair Aj by drawing uniformly from the elements of Si and then draws a new translation φ0 uniformly from the set Φi(C(Aj)). The new state is given by Si : Aj −→ hC(Aj), φ0i. (11) 3.2 Changing Word Order The swap-phrases operation affects the output word order without changing the phrase translations. It exchanges two anchored phrase pairs Aj and Aj+h, resulting in an output state of Si : Aj . . .Aj+h −→ Aj+h Aj+1 . . .Aj+h−1 Aj. (12) The start location j is drawn uniformly from the eligible sentence positions; the swap range h comes from a geometric distribution with configurable decay. Other word-order changes such as a one-way move operation that does not require another movement in exchange or more advanced permutations can easily be defined. 3.3 Resegmentation The most complex operation is resegment, which allows the decoder to modify the segmentation ofthe source phrase. It takes a number of anchored phrase pairs that form a contiguous block both in the input and in the output and replaces them with a new set of phrase pairs covering the same span of the input sentence. Formally, Si : Aj . . .Aj+h−1 −→ A01 . . .A0h0 (13) such that j+[h−1 [h0 [ C(Aj0) = [ C(A0j0) = [p;q] j[0=j (14) j[0=1 for some p and q, where, for j0 = 1, . . . ,h0, we have that A0j0 = h[pj0; qj0] , φj0i, all [pj0; qj0] are mutually disjoint =an hd[ peach φj0 isi randomly drawn from Φi([pj0;qj0]). Regardless of the ordering of Aj . . .Aj+h−1 , the resegment operation always generates a sequence of anchored phrase pairs in linear order, such that C(A0j0) ≺ C(A0j0+1 ) for j0 = 1, . . . ,h0 −1 . As )f o≺r Cth(eA other operations, j is− generated uniformly and h is drawn from a geometric distribution with a decay parameter. The new segmentation is generated by extending the sequence of anchored phrase pairs with random elements starting at the next free position, proceeding from left to right until the whole range [p; q] is covered. 4 Experimental Results In this section, we present the results of a series of experiments with our document decoder. The 1184 goal of our experiments is to demonstrate the behaviour of the decoder and characterise its response to changes in the fundamental search parameters. The SMT models for our experiments were created with a subset of the training data for the English-French shared task at the WMT 2011workshop (Callison-Burch et al., 2011). The phrase table was trained on Europarl, news-commentary and UN data. To reduce the training data to a manageable size, singleton phrase pairs were removed before the phrase scoring step. Significance-based filtering (Johnson et al., 2007) was applied to the resulting phrase table. The language model was a 5gram model with Kneser-Ney smoothing trained on the monolingual News corpus with IRSTLM (Federico et al., 2008). Feature weights were trained with Minimum Error-Rate Training (MERT) (Och, 2003) on the news-test2008 development set using the DP beam search decoder and the MERT implementation of the Moses toolkit (Koehn et al., 2007). Experimental results are reported for the newstest2009 test set, a corpus of 111 newswire documents totalling 2,525 sentences or 65,595 English input tokens. 4.1 Stability An important difference between our decoder and the classical DP decoder as well as previous work in SMT decoding with local search is that our decoder is inherently non-deterministic. This implies that repeated runs of the decoder with the same search parameters, input and models will not, in general, find the same local maximum of the score space. The first empirical question we ask is therefore how different the results are under repeated runs. The results in this and the next section were obtained with random state initialisation, i. e. without running the DP beam search decoder. Figure 1 shows the results of 7 decoder runs with the models described above, translating the newstest2009 test set, with a step limit of 227 and a rejection limit of 100,000. The x-axis of both plots shows the number of decoding steps on a logarithmic scale, so the number of steps is doubled between two adjacent points on the same curve. In the left plot, the y-axis indicates the model score optimised by the decoder summed over all 2525 sentences of the document. In the right plot, the case-sensitive BLEU score (Papineni et al., 2002) of the current decoder Figure 1: Score stability in repeated decoder runs state against a reference translation is displayed. We note, as expected, that the decoder achieves a considerable improvement of the initial state with diminishing returns as decoding continues. Between 28 and 214 steps, the score increases at a roughly logarithmic pace, then the curve flattens out, which is partly due to the fact that decoding for some documents effectively stopped when the maximum number of rejections was reached. The BLEU score curve shows a similar increase, from an initial score below 5 % to a maximum of around 21.5 %. This is below the score of 22.45 % achieved by the beam search decoder with the same models, which is not surprising considering that our decoder approximates a more difficult search problem, from which a number of strong independence assumptions have been lifted, without, at the moment, having any stronger models at its disposal to exploit this additional freedom for better translation. In terms of stability, there are no dramatic differences between the decoder runs. Indeed, the small differences that exist are hardly discernible in the plots. The model scores at the end of the decoding run range between −158767.9 and −158716.9, a g re rlautniv rea ndgieffe breetnwceee nof − only a6b7.o9ut a n0d.0 −3 %15.8 F1i6n.a9l, BLEU scores range from 21.41 % to 21.63 %, an interval that is not negligible, but comparable to the variance observed when, e. g., feature weights from repeated MERT runs are used with one and the same SMT system. Note that these results were obtained with random state initialisation. With DP initialisation, score differences between repeated runs rarely 1185 exceed 0.02 absolute BLEU percentage points. Overall, we conclude that the decoding results of our algorithm are reasonably stable despite the nondeterminism inherent in the procedure. In our subsequent experiments, the evaluation scores reported are calculated as the mean of three runs for each experiment. 4.2 Search Algorithm Parameters The hill climbing algorithm we use has two parameters which govern the trade-off between decoding time and the accuracy with which a local maximum is identified: The step limit stops the search process after a certain number of steps regardless of the search progress made or lack thereof. The rejection limit stops the search after a certain number of unsuccessful attempts to make a step, when continued search does not seem to be promising. In most of our experiments, we used a step limit of 227 ≈ 1.3 · 108 and a rejection limit of 105. In practice, decoding terminates by reaching the rejection limit for the vast majority of documents. We therefore examined the effect of different rejection limits on the learning curves. The results are shown in figure 2. The results show that continued search does pay off to a certain extent. Indeed, the curve for rejection limit 107 seems to indicate that the model score increases roughly logarithmically, albeit to a higher base, even after the curve has started to flatten out at 214 steps. At a certain point, however, the probability of finding a good successor state drops rather sharply by about two orders of magnitude, as Figure 2: Search performance at different rejection limits evidenced by the fact that a rejection limit of 106 does not give a large improvement over one of 105, while one of 107 does. The continued model score improvement also results in an increase in BLEU scores, and with a BLEU score of 22. 1% the system with rejection limit 107 is fairly close to the score of 22.45 % obtained by DP beam search. Obviously, more exact search comes at a cost, and in this case, it comes at a considerable cost, which is an explosion of the time required to decode the test set from 4 minutes at rejection limit 103 to 224 minutes at rejection limit 105 and 38 hours 45 minutes at limit 107. The DP decoder takes 3 1 minutes for the same task. We conclude that the rejection limit of 105 selected for our experiments, while technically suboptimal, realises a good trade-off between decoding time and accuracy. 4.3 A Semantic Document Language Model In this section, we present the results of the application of our decoder to an actual SMT model with cross-sentence features. Our model addresses the problem of lexical cohesion. In particular, it rewards the use of semantically related words in the translation output by the decoder, where semantic distance is measured with a word space model based on Latent Semantic Analysis (LSA). LSA has been applied to semantic language modelling in previous research with some success (Coccaro and Jurafsky, 1998; Bellegarda, 2000; Wandmacher and Antoine, 2007). In SMT, it has mostly been used for domain adaptation (Kim and Khudanpur, 2004; Tam et al., 1186 2007), or to measure sentence similarities (Banchs and Costa-juss a`, 2011). The model we use is inspired by Bellegarda (2000). It is a Markov model, similar to a standard n-gram model, and assigns to each content word a score given a history of n preceding content words, where n = 30 below. Scoring relies on a 30dimensional LSA word vector space trained with the S-Space software (Jurgens and Stevens, 2010). The score is defined based on the cosine similarity between the word vector of the predicted word and the mean word vector of the words in the history, which is converted to a probability by histogram lookup as suggested by Bellegarda (2000). The model is structurally different from a regular n-gram model in that word vector n-grams are defined over content words occurring in the word vector model only and can cross sentence boundaries. Stop words, identified by an extensive stop word list and amounting to around 60 % of the tokens, are scored by a different mechanism based on their relative frequency (undiscounted unigram probability) in the training corpus. In sum, the score produced by the semantic document LM has the following form: wh(er|h)α=is tεpαheuncipgors(wp)o|hrtinof w fci os nakutneskotn wpon w ,onerldse,in(ls1teh5) training corpus and ε is a small fixed probability. It is integrated into the decoder as an extra feature function. Since we lack an automatic method for training the feature weights of document-wide features, its weight was selected by grid search over a number of values, comparing translation performance for the newstest2009 test set. In these experiments, we used DP beam search to initialise the state of our local search decoder. Three results are presented (table 1): The first table row shows the baseline performance using DP beam search with standard sentence-local features only. The scores in the second row were obtained by running the hill climbing decoder with DP initialisation, but without adding any models. A marginal increase in scores for all three test sets demonstrates that the hill climbing decoder manages to fix some of the search errors made by the DP search. The last row contains the scores obtained by adding in the semantic language model. Scores are presented for three publicly available test sets from recent WMT Machine Translation shared tasks, of which one (newstest2009) was used to monitor progress during development and select the final model. Adding the semantic language model results in a small increase in NIST scores (Doddington, 2002) for all three test sets as well as a small BLEU score gain (Papineni et al., 2002) for two out of three corpora. We note that the NIST score turned out to react more sensitively to improvements due to the semantic LM in all our experiments, which is reasonable because the model specifically targets content words, which benefit from the information weighting done by the NIST score. While the results we present do not constitute compelling evidence in favour of our semantic LM in its current form, they do suggest that this model could be improved to realise higher gains from cross-sentence semantic information. They support our claim that cross- sentence models should be examined more closely and that existing methods should be adapted to deal with them, a problem addressed by our main contribution, the local search document decoder. 5 Related Work Even though DP beam search (Koehn et al., 2003) has been the dominant approach to SMT decoding in recent years, methods based on local search have been explored at various times. For word-based SMT, greedy hill-climbing techniques were advo1187 cated as a faster replacement for beam search (Germann et al., 2001 ; Germann, 2003; Germann et al., 2004), and a problem formulation specifically targeting word reordering with an efficient word reordering algorithm has been proposed (Eisner and Tromble, 2006). A local search decoder has been advanced as a faster alternative to beam search also for phrasebased SMT (Langlais et al., 2007; Langlais et al., 2008). That work anticipates many of the features found in our decoder, including the use of local search to refine an initial hypothesis produced by DP beam search. The possibility of using models that do not fit well into the beam search paradigm is mentioned and illustrated with the example of a reversed n-gram language model, which the authors claim would be difficult to implement in a beam search decoder. Similarly to the work by Germann et al. (2001), their decoder is deterministic and explores the entire neighbourhood of a state in order to identify the most promising step. Our main contribution with respect to the work by Langlais et al. (2007) is the introduction of the possibility of handling document-level models by lifting the assumption of sentence independence. As a consequence, enumerating the entire neighbourhood becomes too expensive, which is why we resort to a “first-choice” strategy that non-deterministically generates states and accepts the first one encountered that meets the acceptance criterion. More recently, Gibbs sampling was proposed as a way to generate samples from the posterior distribution of a phrase-based SMT decoder (Arun et al., 2009; Arun et al., 2010), a process that resembles local search in its use of a set of state-modifying operators to generate a sequence of decoder states. Where local search seeks for the best state attainable from a given initial state, Gibbs sampling produces a representative sample from the posterior. Like all work on SMT decoding that we know of, the Gibbs sampler presented by Arun et al. (2010) assumes independence of sentences and considers the complete neighbourhood of each state before taking a sample. 6 Conclusion In the last twenty years of SMT research, there has been a strong assumption that sentences in a text newstest2009 newstest2010 newstest201 1 BLEU NIST BLEU NIST BLEU NIST 22.56 6.513 27.27 7.034 24.94 7.170 + hill climbing 22.60 6.518 27.33 7.046 24.97 7.169 with semantic LM 22.71 6.549 27.53 7.087 24.90 7.199 DP search only DP Table 1: Experimental results with a cross-sentence semantic language model are independent of one another, and discourse context has been largely neglected. Several factors have contributed to this. Developing good discourse-level models is difficult, and considering the modest translation quality that has long been achieved by SMT, there have been more pressing problems to solve and lower hanging fruit to pick. However, we argue that the popular DP beam search algorithm, which delivers excellent decoding performance, but imposes a particular kind of local dependency structure on the feature models, has also had its share in driving researchers away from discourse-level problems. In this paper, we have presented a decoding procedure for phrase-based SMT that makes it possible to define feature models with cross-sentence dependencies. Our algorithm can be combined with DP beam search to leverage the quality of the traditional approach with increased flexibility for models at the discourse level. We have presented preliminary results on a cross-sentence semantic language model addressing the problem of lexical cohesion to demonstrate that this kind of models is worth exploring further. Besides lexical cohesion, cross-sentence models are relevant for other linguistic phenomena such as pronominal anaphora or verb tense selection. We believe that SMT research has reached a point of maturity where discourse phenomena should not be ignored any longer, and we consider our decoder to be a step towards this goal. References Abhishek Arun, Chris Dyer, Barry Haddow, Phil Blunsom, Adam Lopez, and Philipp Koehn. 2009. Monte carlo inference and maximization for phrase-based translation. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL-2009), pages 102–1 10, Boulder, Colorado, June. Association for Computational Linguistics. Abhishek Arun, Barry Haddow, Philipp Koehn, Adam Lopez, Chris Dyer, and Phil Blunsom. 2010. Monte 1188 Ma- Carlo techniques for phrase-based translation. chine translation, 24(2): 103–121 . Rafael E. Banchs and Marta R. Costa-juss a`. 2011. A semantic feature for Statistical Machine Translation. In Proceedings of Fifth Workshop on Syntax, Semantics and Structure in Statistical Translation, pages 126– 134, Portland, Oregon, USA, June. Association for Computational Linguistics. Jerome R. Bellegarda. 2000. Exploiting latent semantic information in statistical language modeling. Proceedings of the IEEE, 88(8): 1279–1296. Chris Callison-Burch, Philipp Koehn, Christof Monz, and Omar Zaidan. 2011. Findings of the 2011 Workshop on Statistical Machine Translation. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 22–64, Edinburgh, Scotland, July. Association for Computational Linguistics. Noah Coccaro and Daniel Jurafsky. 1998. Towards better integration of semantic predictors in statistical language modeling. In Proceedings of the 5th International Conference on Spoken Language Processing, Sydney. George Doddington. 2002. Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In Proceedings of the second International conference on Human Language Technology Research, pages 138–145, San Diego. Jason Eisner and Roy W. Tromble. 2006. Local search with very large-scale neighborhoods for optimal permutations in machine translation. In Proceedings of the HLT-NAACL Workshop on Computationally Hard Problems and Joint Inference in Speech and Language Processing, pages 57–75. Marcello Federico, Nicola Bertoldi, and Mauro Cettolo. 2008. IRSTLM: an open source toolkit for handling large scale language models. In Interspeech 2008, pages 1618–1621 . ISCA. Ulrich Germann, Michael Jahr, Kevin Knight, Daniel Marcu, and Kenji Yamada. 2001 . Fast decoding and optimal decoding for machine translation. In Proceedings of 39th Annual Meeting of the Association for Computational Linguistics, pages 228–235, Toulouse, France, July. Association for Computational Linguis- tics. Ulrich Germann, Michael Jahr, Kevin Knight, Daniel Marcu, and Kenji Yamada. 2004. Fast and optimal decoding for machine translation. Artificial Intelligence, 154(1–2): 127–143. Ulrich Germann. 2003. Greedy decoding for Statistical Machine Translation in almost linear time. In Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics. Zhengxian Gong, Min Zhang, and Guodong Zhou. 2011. Cache-based document-level Statistical Machine Translation. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 909–919, Edinburgh, Scotland, UK., July. Association for Computational Linguistics. Christian Hardmeier and Marcello Federico. 2010. Modelling Pronominal Anaphora in Statistical Machine Translation. In Proceedings of the seventh International Workshop on Spoken Language Translation (IWSLT), pages 283–289. Basil Hatim and Ian Mason. 1990. Discourse and the Translator. Language in Social Life Series. Longman, London. Kenneth Heafield. 2011. KenLM: faster and smaller language model queries. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 187–197, Edinburgh, Scotland, July. Association for Computational Linguistics. Howard Johnson, Joel Martin, George Foster, and Roland Kuhn. 2007. Improving translation quality by discarding most of the phrasetable. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 967– 975, Prague, Czech Republic, June. Association for Computational Linguistics. David Jurgens and Keith Stevens. 2010. The S-Space package: An open source package for word space models. In Proceedings of the ACL 2010 System Demonstrations, pages 30–35, Uppsala, Sweden, July. Association for Computational Linguistics. Woosung Kim and Sanjeev Khudanpur. 2004. Crosslingual latent semantic analysis for language modeling. In IEEE international conference on acoustics, speech, and signal processing (ICASSP), volume 1, pages 257–260, Montr ´eal. Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Proceedings of the 2003 conference of the North American chapter of the Association for Computational Linguistics on Human Language Technology, pages 48– 54, Edmonton. 1189 Philipp Koehn, Hieu Hoang, Alexandra Birch, et al. 2007. Moses: open source toolkit for Statistical Machine Translation. In Annual meeting of the Association for Computational Linguistics: Demonstration session, pages 177–180, Prague. Philippe Langlais, Alexandre Patry, and Fabrizio Gotti. 2007. A greedy decoder for phrase-based statistical machine translation. In TMI-2007: Proceedings of the 11th International Conference on Theoretical and Methodological Issues in Machine Translation, pages 104–1 13, Sk¨ ovde. Philippe Langlais, Alexandre Patry, and Fabrizio Gotti. 2008. Recherche locale pour la traduction statistique par segments. In TALN 2008, pages 119–128, Avignon, France, June. ATALA. Ronan Le Nagard and Philipp Koehn. 2010. Aiding pronoun translation with co-reference resolution. In Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR, pages 252–261, Uppsala, Sweden, July. Association for Computational Linguistics. Franz Josef Och, Nicola Ueffing, and Hermann Ney. 2001. An efficient A* search algorithm for Statistical Machine Translation. In Proceedings of the DataDriven Machine Translation Workshop, 39th Annual Meeting of the Association for Computational Linguistics (ACL), pages 55–62, Toulouse. Franz Josef Och. 2003. Minimum error rate training in Statistical Machine Translation. In Proceedings of the 41st annual meeting of the Association for Computational Linguistics, pages 160–167, Sapporo (Japan). Kishore Papineni, Salim Roukos, Todd Ward, and WeiJing Zhu. 2002. BLEU: a method for automatic evaluation of Machine Translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 3 11–3 18, Philadelphia. ACL. Yik-Cheung Tam, Ian Lane, and Tanja Schultz. 2007. Bilingual LSA-based adaptation for Statistical Machine Translation. Machine Translation, 21(4): 187– 207. J o¨rg Tiedemann. 2010. To cache or not to cache? Experiments with adaptive models in Statistical Machine Translation. In Proceedings of the ACL 2010 Joint Fifth Workshop on Statistical Machine Translation and Metrics MATR, pages 189–194, Uppsala, Sweden. Association for Computational Linguistics. Christoph Tillmann and Hermann Ney. 2003. Word reordering and a Dynamic Programming beam search algorithm for Statistical Machine Translation. Computational linguistics, 29(1):97–133. Christoph Tillmann, Stephan Vogel, Hermann Ney, and Alex Zubiaga. 1997. A DP-based search using monotone alignments in Statistical Translation. In Proceedings of the 35th Annual Meeting of the Association for Computational Spain, tics. Linguistics, July. Association for 289–296, Madrid, Computational Linguis- pages Tonio Wandmacher and Jean-Yves Antoine. 2007. Methods to integrate a language model with semantic information for a word prediction component. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLPCoNLL), pages 506–5 13, Prague, Czech Republic, June. Association for Computational Linguistics. 1190
5 0.50888926 70 emnlp-2012-Joint Chinese Word Segmentation, POS Tagging and Parsing
Author: Xian Qian ; Yang Liu
Abstract: In this paper, we propose a novel decoding algorithm for discriminative joint Chinese word segmentation, part-of-speech (POS) tagging, and parsing. Previous work often used a pipeline method Chinese word segmentation followed by POS tagging and parsing, which suffers from error propagation and is unable to leverage information in later modules for earlier components. In our approach, we train the three individual models separately during training, and incorporate them together in a unified framework during decoding. We extend the CYK parsing algorithm so that it can deal with word segmentation and POS tagging features. As far as we know, this is the first work on joint Chinese word segmentation, POS tagging and parsing. Our experimental results on Chinese Tree Bank 5 corpus show that our approach outperforms the state-of-the-art pipeline system. –
6 0.50886554 42 emnlp-2012-Entropy-based Pruning for Phrase-based Machine Translation
7 0.50752813 136 emnlp-2012-Weakly Supervised Training of Semantic Parsers
8 0.50436801 123 emnlp-2012-Syntactic Transfer Using a Bilingual Lexicon
9 0.50368524 18 emnlp-2012-An Empirical Investigation of Statistical Significance in NLP
10 0.50298768 11 emnlp-2012-A Systematic Comparison of Phrase Table Pruning Techniques
11 0.50214922 24 emnlp-2012-Biased Representation Learning for Domain Adaptation
12 0.50086504 45 emnlp-2012-Exploiting Chunk-level Features to Improve Phrase Chunking
13 0.49744079 127 emnlp-2012-Transforming Trees to Improve Syntactic Convergence
14 0.4961262 89 emnlp-2012-Mixed Membership Markov Models for Unsupervised Conversation Modeling
15 0.4957945 64 emnlp-2012-Improved Parsing and POS Tagging Using Inter-Sentence Consistency Constraints
16 0.49495095 111 emnlp-2012-Regularized Interlingual Projections: Evaluation on Multilingual Transliteration
17 0.4940657 130 emnlp-2012-Unambiguity Regularization for Unsupervised Learning of Probabilistic Grammars
18 0.49401048 108 emnlp-2012-Probabilistic Finite State Machines for Regression-based MT Evaluation
19 0.49252227 95 emnlp-2012-N-gram-based Tense Models for Statistical Machine Translation
20 0.49231514 5 emnlp-2012-A Discriminative Model for Query Spelling Correction with Latent Structural SVM