acl acl2011 acl2011-310 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Preslav Nakov ; Hwee Tou Ng
Abstract: We propose a novel approach to translating from a morphologically complex language. Unlike previous research, which has targeted word inflections and concatenations, we focus on the pairwise relationship between morphologically related words, which we treat as potential paraphrases and handle using paraphrasing techniques at the word, phrase, and sentence level. An important advantage of this framework is that it can cope with derivational morphology, which has so far remained largely beyond the capabilities of statistical machine translation systems. Our experiments translating from Malay, whose morphology is mostly derivational, into English show signif- icant improvements over rivaling approaches based on five automatic evaluation measures (for 320,000 sentence pairs; 9.5 million English word tokens).
Reference: text
sentIndex sentText sentNum sentScore
1 sg Abstract We propose a novel approach to translating from a morphologically complex language. [sent-4, score-0.157]
2 Unlike previous research, which has targeted word inflections and concatenations, we focus on the pairwise relationship between morphologically related words, which we treat as potential paraphrases and handle using paraphrasing techniques at the word, phrase, and sentence level. [sent-5, score-0.511]
3 An important advantage of this framework is that it can cope with derivational morphology, which has so far remained largely beyond the capabilities of statistical machine translation systems. [sent-6, score-0.207]
4 Our experiments translating from Malay, whose morphology is mostly derivational, into English show signif- icant improvements over rivaling approaches based on five automatic evaluation measures (for 320,000 sentence pairs; 9. [sent-7, score-0.144]
5 1 Introduction Traditionally, statistical machine translation (SMT) models have assumed that the word should be the basic token-unit of translation, thus ignoring any wordinternal morphological structure. [sent-9, score-0.174]
6 s g Ignoring morphology was fine as long as the main research interest remained focused on languages with limited (e. [sent-19, score-0.132]
7 Since the attention shifted to languages like Arabic, however, the importance of morphology became obvious and several approaches to handle it have been proposed. [sent-24, score-0.132]
8 However, derivational morphology has not been specifically targeted so far. [sent-30, score-0.237]
9 In this paper, we propose a paraphrase-based approach to translating from a morphologically complex language. [sent-31, score-0.157]
10 Unlike previous research, we focus on the pairwise relationship between morphologically related wordforms, which we treat as potential paraphrases, and which we handle using paraphrasing techniques at various levels: word, phrase, and sentence level. [sent-32, score-0.244]
11 An important advantage of this framework is that it can cope with various kinds of morphological wordforms, including derivational ones. [sent-33, score-0.229]
12 We demonstrate its potential on Malay, whose morphology is mostly derivational. [sent-34, score-0.106]
13 Unlike other agglutinative languages such as Finnish, Hungarian, and Turkish, which are rich in both inflectional and derivational forms, Malay morphology is mostly derivational. [sent-44, score-0.29]
14 For example, the politeness marker lah can be added to the command duduk/‘sit down’ to yield duduklah/‘please, sit down’, and the pronoun nya can attach to kereta to form keretanya/‘his car’ . [sent-87, score-0.131]
15 Note that clitics are not affixes, and clitic attachment is not a word derivation or a word inflection process. [sent-88, score-0.172]
16 Moreover, the predominantly derivational nature of Malay morphology limits the applicability of standard techniques such as (1) removing some/all of the source-language inflections, (2) segmenting affixes from the root, and (3) clustering words with the same target translation. [sent-90, score-0.276]
17 Finally, if affixes tend to change semantics so much, how likely are we to find morphologically related wordforms that share the same translation? [sent-97, score-0.334]
18 Consider affixation, which can yield words with similar semantics that can use each other’s translation options, e. [sent-99, score-0.105]
19 Looking at compounding, it is often the case that the semantics of a compound is a specialization of the semantics of its head, and thus the target language translations available for the head could be usable to translate the whole compound, e. [sent-108, score-0.11]
20 Thus, if we do not know how to translate pelajar-pelajar, it would be reasonable to consider the translation options for pelajar since it could potentially contain among its translation options the plural ‘students’ . [sent-120, score-0.285]
21 3 Method We propose a paraphrase-based approach to Malay morphology, where we use paraphrases at three different levels: word, phrase, and sentence level. [sent-126, score-0.227]
22 First, we transform each development/testing Malay sentence into a word lattice, where we add simplified word-level paraphrasing alternatives for each morphologically complex word. [sent-127, score-0.286]
23 In the lattice, each alternative w′ of an original word w is assigned the weight of Pr(w′ |w), which is estimated using pivoting over the English shidiceh o isf tehstei training i bnigtext. [sent-128, score-0.183]
24 Then, we generate sentence-level paraphrases of the training Malay sentences, in which exactly one morphologically complex word is substituted by a simpler alternative. [sent-129, score-0.371]
25 Finally, we extract additional Malay phrases from these sentences, which we use to augment the phrase table with additional translation options to match the alternative wordforms in the lattice. [sent-130, score-0.354]
26 We assign each such additional phrase p′ a probability maxp Pr(p′|p), where p is a Malay phrase that is found in the original training Malay text. [sent-131, score-0.189]
27 The probability is calculated usingphrase-level pivoting over the English side of the training bi-text. [sent-132, score-0.118]
28 1 Morphological Analysis Given a Malay word, we build a list of morphologically simpler words that could be derived from it; we also generate alternative word segmentations: (a) words obtainable by affix stripping e. [sent-134, score-0.199]
29 , pelajaran → pelajar, ajaran, ajar (b) words that are part of a compound word e. [sent-136, score-0.154]
30 , adik-beradik → adik, beradik (d) words without clitics e. [sent-140, score-0.114]
31 Strictly speaking, (f) does not necessarily model a morphological process: it proposes an alternative tokenization, but this could make morphological sense too. [sent-148, score-0.222]
32 , adik-beradik would be- come adik - beradik and then by (a) it would turn into adik - adik, which could cause the SMT system to generate two separate translations for the two instances of adik. [sent-151, score-0.276]
33 As an illustration, here are the wordforms we generate for adik-beradiknya/‘his siblings’ : adik, adik-beradiknya, adik-beradik nya, adik-beradik, beradiknya, beradik nya, adik nya, and beradik. [sent-165, score-0.307]
34 , they would return adik for adik-beradiknya, and ajar for berpelajaran. [sent-172, score-0.189]
35 2 Word-Level Paraphrasing We perform word-level paraphrasing of the Malay sides of the development and the testing bi-texts. [sent-176, score-0.125]
36 First, for each Malay word, we generate the above-described list of morphologically simpler words and alternative word segmentations; we think of the words in this list as word-level paraphrases. [sent-177, score-0.17]
37 Then, for each development/testing Malay sentence, we generate a lattice encoding all possible paraphrasing options for each individual word. [sent-178, score-0.336]
38 More precisely, we use pivoting to estimate the probability Pr(w′|w) as follows: Pr(w′ |w) = Pi Pr(w′|w, ei)Pr(ei |w) 1301 Then, following (Callison-Burch et al. [sent-186, score-0.118]
39 Thus, we think of w′ as a pseudoword that stands for the union of all Malay words in the training bi-text that are reducible to w′ by our morphological analysis procedure. [sent-191, score-0.142]
40 3 Sentence-Level ParaPphrasing In order for the word-level paraphrases to work, there should be phrases in the phrase table that could potentially match them. [sent-194, score-0.302]
41 Thus, we need to augment the phrase table with additional translation options. [sent-200, score-0.151]
42 This would be problematic since the phrase translation probabilities associated with these new 4Note that our paraphrasing process is directed: the paraphrases are morphologically simpler than the original word. [sent-204, score-0.686]
43 For example, the clitics, and even many of the intermediate morphological forms, would not exist as individual words in the training bi-text, which means that there would be no word alignments or lexical probabilities available for them. [sent-206, score-0.143]
44 Then, the two bi-texts and their word alignments would be concatenated and used to build a phrase table (Dyer, 2007; Dyer et al. [sent-208, score-0.165]
45 This would solve the problems with the word alignments and the phrase pair probabilities estimations in a principled manner, but it would require choosing for each word only one of the paraphrases available to it, while we would prefer to have a way to allow all options. [sent-210, score-0.347]
46 Moreover, the paraphrased and the original versions of the corpus would be given equal weights, which might not be desirable. [sent-211, score-0.184]
47 We avoid the above issues by adopting a sentencelevel paraphrasing approach. [sent-213, score-0.125]
48 Following the general framework proposed in (Nakov, 2008), we first create multiple paraphrased versions of the sourceside sentences of the training bi-text. [sent-214, score-0.145]
49 Then, each paraphrased source sentence is paired with its original translation. [sent-215, score-0.16]
50 This augmented bi-text is wordaligned and a phrase table T′ is built from it, which is merged with a phrase table T for the original bi- text. [sent-216, score-0.189]
51 The merged table contains all phrase entries from T, and the entries for the phrase pairs from T′ that are not in T. [sent-217, score-0.15]
52 1302 Each of our paraphrased sentences differs from its original sentence by a single word, which prevents combinatorial explosions: on average, we generate 14 paraphrased versions per input sentence. [sent-223, score-0.305]
53 It further ensures that the paraphrased parts of the sentences will not dominate the word alignments or the phrase pairs, and that there would be sufficient interaction at word alignment time between the original sentences and their paraphrased versions. [sent-224, score-0.401]
54 4 Phrase-Level Paraphrasing While our sentence-level paraphrasing informs the decoder about the origin of each phrase pair (original or paraphrased bi-text), it provides no indication about how good the phrase pairs from the paraphrased bi-text are likely to be. [sent-226, score-0.517]
55 (2006), we further augment the phrase table with one additional feature whose value is 1 for the phrase pairs coming from the original bi-text, and maxp Pr(p′|p) for the phrase pairs extracted from the paraphrased fboi-r text. [sent-228, score-0.385]
56 Here p is a Malay phrase from T, and p′ is a Malay phrase from T′ that does not exist in T but is obtainable from p by substituting one or more words in p with their derivationally related forms generated by morphological analysis. [sent-229, score-0.348]
57 When calculating Pr(p′|ei), we think of p′ as the set of all possible Malay phrases q in T that are reducible to p′ by morphological analysis of the words they contain. [sent-231, score-0.142]
58 This can be rewritten as follows: = Pr(p′|ei) Pq:p′∈par(q) Pr(q|ei) where par(q) is the sPet of all possible phrase-level paraphrases for the Malay phrase q. [sent-232, score-0.302]
59 Int uses 0de-1l weights in the lattice and only allows lemmata as alternative wordforms; it uses no sentence-level or phrase-level paraphrases. [sent-393, score-0.287]
60 42 Table 2: Detailed BLEU n-gram precision scores: in %, for different numbers of training sentence pairs, for baseline and lattice + sent-par + word-par + phrase-par. [sent-472, score-0.181]
61 Our full morphological paraphrasing system is lattice + sent-par + word-par + phrase-par. [sent-473, score-0.404]
62 lattice + sent-par + word-par excludes the additional feature from phrase-level paraphrasing. [sent-475, score-0.181]
63 lattice + sent-par has all the morphologically simpler derived forms in the lattice during decoding, but their weights are uniformly set to 0 rather than obtained using pivoting from word alignments. [sent-476, score-0.695]
64 Finally, in order to compare closely to the ‘noisier’ channel model, we further limited the morphological variants of lattice + sent-par in the lattice to lemmata only in lattice + sent-par (orig+lemma). [sent-477, score-0.734]
65 First, we can see that lemmatize all has a consistently disastrous effect on BLEU, which shows that Malay morphology does indeed contain information that is important when translating to English. [sent-479, score-0.182]
66 It performs worse than lattice + sent-par (orig+lemma), from which it differs in the phrase table only; this confirms the importance of our sentence-level paraphrasing. [sent-481, score-0.256]
67 Moving down to lattice + sent-par, we can see that using multiple morphological wordforms instead of just lemmata has a consistently positive impact on BLEU for datasets of all sizes. [sent-482, score-0.477]
68 7398 paraphrases Table 3: Results for different evaluation measures: for baseline and lattice + sent-par + word-par + phrase-par (in % for all measures except for NIST). [sent-574, score-0.408]
69 Adding weights obtained using word-level pivoting in lattice + sent-par + word-par helps a bit more, and also using phrase-level paraphrasing weights yields even bigger further improvements for lattice + sent-par + word-par + phrase-par. [sent-575, score-0.663]
70 Overall, our morphological paraphrases yield statistically significant improvements (p < 0. [sent-576, score-0.325]
71 It was not a problem for our system though, which first paraphrased it as bekalan and then translated it as supply. [sent-593, score-0.121]
72 src : Mercy Relief telah menghantar 17 khemah khas bernilai $5,000 setiap satu yang boleh menampung kelas seramai 30 pelajar, selain bekalan-bekalan lain seperti 500 khemah biasa, barang makanan dan ubat-ubatan untuk mangsa gempa Sichuan. [sent-595, score-0.116]
73 re f1: Mercy Relief has sent 17 special tents valued at $5,000 each, that can accommodate a class of 30 students, including other aid supplies such as 500 normal tents, food and medicine for the victims of Sichuan quake. [sent-596, score-0.167]
74 base : mercy relief has sent 17 special tents worth $5,000 each class could accommodate a total of 30 students, besides other bekalan-bekalan 500 tents as usual, foodstuff and medicines for sichuan quake relief. [sent-597, score-0.36]
75 para : mercy relief has sent 17 special tents worth $5,000 each class could accommodate a total of 30 students, besides other supply such as 500 tents, food and medicines for sichuan quake relief. [sent-598, score-0.344]
76 src : Walaupun hidup susah, kami tetap berusaha untuk menjalani kehidupan seperti biasa. [sent-599, score-0.175]
77 For each example, we show a source sentence (s rc), one of the three reference translations (re f1), and the outputs of baseline (base) and of lattice + sent-par + word-par + phrase-par (para). [sent-604, score-0.181]
78 Note that the words menjalani (‘go through’) and kehidupan (‘life/existence’) are derivational forms of jalan (‘go’) and hidup (‘life/living’), respectively. [sent-607, score-0.29]
79 Thus, in the paraphrasing system, they were involved in sentence-level paraphrasing, where the alignments were improved. [sent-608, score-0.17]
80 While the wrong phrase pair was still available, the system chose a better one from the paraphrased training bi-text. [sent-609, score-0.196]
81 6 Related Work Most research in SMT for a morphologically rich source language has focused on inflected forms of the same word. [sent-610, score-0.186]
82 For languages with 1305 more or less regular inflectional morphology like Arabic or Turkish, another good idea is to segment words into morpheme sequences, e. [sent-616, score-0.159]
83 This can be achieved using a lattice input to the translation system (Dyer et al. [sent-619, score-0.257]
84 Unfortunately, none of these general lines of research suits Malay well, whose compounds are rarely concatenated, clitics are not so frequent, and morphology is mostly derivational, and thus likely to generate words whose semantics substantially differs from the semantics of the original word. [sent-621, score-0.303]
85 Therefore, we cannot expect the existence of equivalence classes: it is only occasionally that two derivation- ally related wordforms would share the same target language translation. [sent-622, score-0.147]
86 Then, the two bi-texts and their word alignments are concatenated and used to build a phrase table. [sent-626, score-0.165]
87 The arc weights are set to 1 for the original wordforms and to 0 for the lemmata. [sent-628, score-0.215]
88 In contrast, we provide multiple paraphrasing alternatives for each morphologically complex word, including derivational forms that occupy intermediary positions between the original wordform and its lemma. [sent-629, score-0.498]
89 Note that some of those paraphrasing alternatives are multi-word, and thus we use a lattice instead of a confusion network. [sent-630, score-0.348]
90 (2008), who use a lattice to add a single alternative clitic-segmented version of the original word for Arabic. [sent-633, score-0.246]
91 We also include derivational forms in addition to clitic-segmented ones, and we give different weights to the different alternatives (instead of 0). [sent-635, score-0.244]
92 Third, our work is also related to that of Dyer (2009), who uses a lattice to add multiple alternative segmented versions ofthe original word for German, Hungarian, and Turkish. [sent-636, score-0.27]
93 However, we focus on derivational morphology rather than on clitics and inflections, add derivational forms in addition to clitic-segmented ones, and use cross-lingual word pivoting to estimate paraphrase probabilities. [sent-637, score-0.598]
94 (2006), who use cross-lingual pivoting to generate phrase-level paraphrases with corresponding probabilities. [sent-639, score-0.345]
95 However, our paraphrases are derived through morphological analysis; thus, we do not need corpora in additional languages. [sent-640, score-0.325]
96 7 Conclusion and Future Work We have presented a novel approach to trans- lating from a morphologically complex language, which uses paraphrases and paraphrasing techniques at three different levels of translation: wordlevel, phrase-level, and sentence-level. [sent-641, score-0.471]
97 Our experiments translating from Malay, whose morphology is mostly derivational, into English have shown significant improvements over rivaling approaches based on several automatic evaluation measures. [sent-642, score-0.144]
98 In future work, we want to improve the probability estimations for our paraphrasing models. [sent-643, score-0.125]
99 We also want to experiment with other morphologically complex languages and other SMT models. [sent-644, score-0.145]
100 The ’noisier channel’ : translation from morphologically complex languages. [sent-694, score-0.195]
wordName wordTfidf (topN-words)
[('malay', 0.682), ('paraphrases', 0.227), ('pr', 0.19), ('lattice', 0.181), ('ei', 0.163), ('wordforms', 0.147), ('derivational', 0.131), ('paraphrasing', 0.125), ('paraphrased', 0.121), ('morphologically', 0.119), ('pivoting', 0.118), ('adik', 0.116), ('morphology', 0.106), ('clitic', 0.102), ('morphological', 0.098), ('dyer', 0.086), ('reduplication', 0.077), ('tents', 0.077), ('translation', 0.076), ('phrase', 0.075), ('ajar', 0.073), ('nya', 0.073), ('pelajar', 0.073), ('clitics', 0.07), ('nakov', 0.06), ('noisier', 0.06), ('bahasa', 0.058), ('indonesia', 0.058), ('kereta', 0.058), ('malaysia', 0.058), ('mercy', 0.058), ('compound', 0.052), ('affixation', 0.051), ('lemmata', 0.051), ('relief', 0.051), ('students', 0.048), ('bleu', 0.045), ('concatenated', 0.045), ('alignments', 0.045), ('beradik', 0.044), ('kehidupan', 0.044), ('menjalani', 0.044), ('orig', 0.044), ('reducible', 0.044), ('sichuan', 0.044), ('tesla', 0.044), ('alternatives', 0.042), ('channel', 0.042), ('forms', 0.042), ('smt', 0.042), ('inflections', 0.04), ('compounding', 0.04), ('koehn', 0.039), ('affixes', 0.039), ('original', 0.039), ('lemmatize', 0.038), ('supplies', 0.038), ('translating', 0.038), ('life', 0.037), ('turkish', 0.036), ('lcs', 0.035), ('teach', 0.035), ('para', 0.033), ('preslav', 0.032), ('arabic', 0.03), ('compounds', 0.03), ('options', 0.03), ('segmentation', 0.029), ('adriani', 0.029), ('ajaran', 0.029), ('circumfix', 0.029), ('derivationally', 0.029), ('hidup', 0.029), ('indonesian', 0.029), ('keretanya', 0.029), ('kerja', 0.029), ('khemah', 0.029), ('obtainable', 0.029), ('pelajaran', 0.029), ('pmr', 0.029), ('quake', 0.029), ('seperti', 0.029), ('untuk', 0.029), ('finnish', 0.029), ('semantics', 0.029), ('weights', 0.029), ('food', 0.028), ('singapore', 0.028), ('inflectional', 0.027), ('philipp', 0.026), ('languages', 0.026), ('alternative', 0.026), ('cdec', 0.026), ('taught', 0.026), ('inflected', 0.025), ('simpler', 0.025), ('versions', 0.024), ('meteor', 0.024), ('accommodate', 0.024), ('segmentations', 0.024)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000002 310 acl-2011-Translating from Morphologically Complex Languages: A Paraphrase-Based Approach
Author: Preslav Nakov ; Hwee Tou Ng
Abstract: We propose a novel approach to translating from a morphologically complex language. Unlike previous research, which has targeted word inflections and concatenations, we focus on the pairwise relationship between morphologically related words, which we treat as potential paraphrases and handle using paraphrasing techniques at the word, phrase, and sentence level. An important advantage of this framework is that it can cope with derivational morphology, which has so far remained largely beyond the capabilities of statistical machine translation systems. Our experiments translating from Malay, whose morphology is mostly derivational, into English show signif- icant improvements over rivaling approaches based on five automatic evaluation measures (for 320,000 sentence pairs; 9.5 million English word tokens).
2 0.1713247 37 acl-2011-An Empirical Evaluation of Data-Driven Paraphrase Generation Techniques
Author: Donald Metzler ; Eduard Hovy ; Chunliang Zhang
Abstract: Paraphrase generation is an important task that has received a great deal of interest recently. Proposed data-driven solutions to the problem have ranged from simple approaches that make minimal use of NLP tools to more complex approaches that rely on numerous language-dependent resources. Despite all of the attention, there have been very few direct empirical evaluations comparing the merits of the different approaches. This paper empirically examines the tradeoffs between simple and sophisticated paraphrase harvesting approaches to help shed light on their strengths and weaknesses. Our evaluation reveals that very simple approaches fare surprisingly well and have a number of distinct advantages, including strong precision, good coverage, and low redundancy.
3 0.15049241 75 acl-2011-Combining Morpheme-based Machine Translation with Post-processing Morpheme Prediction
Author: Ann Clifton ; Anoop Sarkar
Abstract: This paper extends the training and tuning regime for phrase-based statistical machine translation to obtain fluent translations into morphologically complex languages (we build an English to Finnish translation system) . Our methods use unsupervised morphology induction. Unlike previous work we focus on morphologically productive phrase pairs – our decoder can combine morphemes across phrase boundaries. Morphemes in the target language may not have a corresponding morpheme or word in the source language. Therefore, we propose a novel combination of post-processing morphology prediction with morpheme-based translation. We show, using both automatic evaluation scores and linguistically motivated analyses of the output, that our methods outperform previously proposed ones and pro- vide the best known results on the EnglishFinnish Europarl translation task. Our methods are mostly language independent, so they should improve translation into other target languages with complex morphology. 1 Translation and Morphology Languages with rich morphological systems present significant hurdles for statistical machine translation (SMT) , most notably data sparsity, source-target asymmetry, and problems with automatic evaluation. In this work, we propose to address the problem of morphological complexity in an Englishto-Finnish MT task within a phrase-based translation framework. We focus on unsupervised segmentation methods to derive the morphological information supplied to the MT model in order to provide coverage on very large datasets and for languages with few hand-annotated 32 resources. In fact, in our experiments, unsupervised morphology always outperforms the use of a hand-built morphological analyzer. Rather than focusing on a few linguistically motivated aspects of Finnish morphological behaviour, we develop techniques for handling morphological complexity in general. We chose Finnish as our target language for this work, because it exemplifies many of the problems morphologically complex languages present for SMT. Among all the languages in the Europarl data-set, Finnish is the most difficult language to translate from and into, as was demonstrated in the MT Summit shared task (Koehn, 2005) . Another reason is the current lack of knowledge about how to apply SMT successfully to agglutinative languages like Turkish or Finnish. Our main contributions are: 1) the introduction of the notion of segmented translation where we explicitly allow phrase pairs that can end with a dangling morpheme, which can connect with other morphemes as part of the translation process, and 2) the use of a fully segmented translation model in combination with a post-processing morpheme prediction system, using unsupervised morphology induction. Both of these approaches beat the state of the art on the English-Finnish translation task. Morphology can express both content and function categories, and our experiments show that it is important to use morphology both within the translation model (for morphology with content) and outside it (for morphology contributing to fluency) . Automatic evaluation measures for MT, BLEU (Papineni et al., 2002), WER (Word Error Rate) and PER (Position Independent Word Error Rate) use the word as the basic unit rather than morphemes. In a word comProce dPinogrstla ofn tdh,e O 4r9etghon A,n Jnu nael 1 M9-e 2t4i,n2g 0 o1f1 t.he ?c A2s0s1o1ci Aatsiosonc fioartio Cno fmorpu Ctoamtiopnuatalt Lioin gauli Lsitnicgsu,i psatgices 32–42, prised of multiple morphemes, getting even a single morpheme wrong means the entire word is wrong. In addition to standard MT evaluation measures, we perform a detailed linguistic analysis of the output. Our proposed approaches are significantly better than the state of the art, achieving the highest reported BLEU scores on the English-Finnish Europarl version 3 data-set. Our linguistic analysis shows that our models have fewer morpho-syntactic errors compared to the word-based baseline. 2 2.1 Models Baseline Models We set up three baseline models for comparison in this work. The first is a basic wordbased model (called Baseline in the results) ; we trained this on the original unsegmented version of the text. Our second baseline is a factored translation model (Koehn and Hoang, 2007) (called Factored) , which used as factors the word, “stem” 1 and suffix. These are derived from the same unsupervised segmentation model used in other experiments. The results (Table 3) show that a factored model was unable to match the scores of a simple wordbased baseline. We hypothesize that this may be an inherently difficult representational form for a language with the degree of morphological complexity found in Finnish. Because the morphology generation must be precomputed, for languages with a high degree of morphological complexity, the combinatorial explosion makes it unmanageable to capture the full range of morphological productivity. In addition, because the morphological variants are generated on a per-word basis within a given phrase, it excludes productive morphological combination across phrase boundaries and makes it impossible for the model to take into account any longdistance dependencies between morphemes. We conclude from this result that it may be more useful for an agglutinative language to use morphology beyond the confines of the phrasal unit, and condition its generation on more than just the local target stem. In order to compare the 1see Section 2.2. 33 performance of unsupervised segmentation for translation, our third baseline is a segmented translation model based on a supervised segmentation model (called Sup) , using the hand-built Omorfi morphological analyzer (Pirinen and Listenmaa, 2007) , which provided slightly higher BLEU scores than the word-based baseline. 2.2 Segmented Translation For segmented translation models, it cannot be taken for granted that greater linguistic accuracy in segmentation yields improved translation (Chang et al. , 2008) . Rather, the goal in segmentation for translation is instead to maximize the amount of lexical content-carrying morphology, while generalizing over the information not helpful for improving the translation model. We therefore trained several different segmentation models, considering factors of granularity, coverage, and source-target symmetry. We performed unsupervised segmentation of the target data, using Morfessor (Creutz and Lagus, 2005) and Paramor (Monson, 2008) , two top systems from the Morpho Challenge 2008 (their combined output was the Morpho Challenge winner) . However, translation models based upon either Paramor alone or the combined systems output could not match the wordbased baseline, so we concentrated on Morfessor. Morfessor uses minimum description length criteria to train a HMM-based segmentation model. When tested against a human-annotated gold standard of linguistic morpheme segmentations for Finnish, this algorithm outperforms competing unsupervised methods, achieving an F-score of 67.0% on a 3 million sentence corpus (Creutz and Lagus, 2006) . Varying the perplexity threshold in Morfessor does not segment more word types, but rather over-segments the same word types. In order to get robust, common segmentations, we trained the segmenter on the 5000 most frequent words2 ; we then used this to segment the entire data set. In order to improve coverage, we then further segmented 2For the factored model baseline we also used the same setting perplexity = 30, 5,000 most frequent words, but with all but the last suffix collapsed and called the “stem” . TabHMleoat1nr:gplhiMngor phermphocTur631ae04in, 81c9ie03ns67gi,64n0S14e567theTp 2rsa51t, 29Se 3t168able and in translation. any word type that contained a match from the most frequent suffix set, looking for the longest matching suffix character string. We call this method Unsup L-match. After the segmentation, word-internal morpheme boundary markers were inserted into the segmented text to be used to reconstruct the surface forms in the MT output. We then trained the Moses phrase-based system (Koehn et al., 2007) on the segmented and marked text. After decoding, it was a simple matter to join together all adjacent morphemes with word-internal boundary markers to reconstruct the surface forms. Figure 1(a) gives the full model overview for all the variants of the segmented translation model (supervised/unsupervised; with and without the Unsup L-match procedure) . Table 1shows how morphemes are being used in the MT system. Of the phrases that included segmentations (‘Morph’ in Table 1) , roughly a third were ‘productive’, i.e. had a hanging morpheme (with a form such as stem+) that could be joined to a suffix (‘Hanging Morph’ in Table 1) . However, in phrases used while decoding the development and test data, roughly a quarter of the phrases that generated the translated output included segmentations, but of these, only a small fraction (6%) had a hanging morpheme; and while there are many possible reasons to account for this we were unable to find a single convincing cause. 2.3 Morphology Generation Morphology generation as a post-processing step allows major vocabulary reduction in the translation model, and allows the use of morphologically targeted features for modeling inflection. A possible disadvantage of this approach is that in this model there is no opportunity to con34 sider the morphology in translation since it is removed prior to training the translation model. Morphology generation models can use a variety of bilingual and contextual information to capture dependencies between morphemes, often more long-distance than what is possible using n-gram language models over morphemes in the segmented model. Similar to previous work (Minkov et al. , 2007; Toutanova et al. , 2008) , we model morphology generation as a sequence learning problem. Un- like previous work, we use unsupervised morphology induction and use automatically generated suffix classes as tags. The first phase of our morphology prediction model is to train a MT system that produces morphologically simplified word forms in the target language. The output word forms are complex stems (a stem and some suffixes) but still missing some important suffix morphemes. In the second phase, the output of the MT decoder is then tagged with a sequence of abstract suffix tags. In particular, the output of the MT decoder is a sequence of complex stems denoted by x and the output is a sequence of suffix class tags denoted by y. We use a list of parts from (x,y) and map to a d-dimensional feature vector Φ(x, y) , with each dimension being a real number. We infer the best sequence of tags using: F(x) = argymaxp(y | x,w) where F(x) returns the highest scoring output y∗ . A conditional random field (CRF) (Lafferty et al. , 2001) defines the conditional probability as a linear score for each candidate y and a global normalization term: logp(y | x, w) = Φ(x, y) · w − log Z where Z = Py0∈ exp(Φ(x, y0) · w) . We use stochastiPc gradient descent (using crfsgd3) to train the weight vector w. So far, this is all off-the-shelf sequence learning. However, the output y∗ from the CRF decoder is still only a sequence of abstract suffix tags. The third and final phase in our morphology prediction model GEN(x) 3 http://leon. bottou. org/projects/sgd English Training Data words Finnish Training Data words Morphological Pre-Processing stem+ +morph MT System Alignment: word word word stem+ +morph stem stem+ +morph Post-Process: Morph Re-Stitching Fully inflected surface form Evaluation against original reference (a) Segmented Translation Model English Training Data words Finnish Training Data Morphological Pre-Prowceosrdsisng 1 stem+ +morph1+ +morph2 Morphological Pre-Processing 2 stem+ +morph1+ MPosrpthe-mPRr+eo-+cSmetsio crhp1i:nhg+swteomrd+ MA+lTmigwnSomyrspdthen 1mt:+ wsotermd complex stem: stem+morph1+ MPo rpsht-oPlro gcyesGse2n:erCaRtioFnstem+morph1+ morph2sLuarnfagcueagfeorMmomdealp ing Fully inflected surface form Evaluation against original reference (b) Post-Processing Model Translation & Generation Figure 1: Training and testing pipelines for the SMT models. is to take the abstract suffix tag sequence y∗ and then map it into fully inflected word forms, and rank those outputs using a morphemic language model. The abstract suffix tags are extracted from the unsupervised morpheme learning process, and are carefully designed to enable CRF training and decoding. We call this model CRFLM for short. Figure 1(b) shows the full pipeline and Figure 2 shows a worked example of all the steps involved. We use the morphologically segmented training data (obtained using the segmented corpus described in Section 2.24) and remove selected suffixes to create a morphologically simplified version of the training data. The MT model is trained on the morphologically simplified training data. The output from the MT system is then used as input to the CRF model. The CRF model was trained on a ∼210,000 Finnish sentences, consisting noefd d∼ o1n.5 a am ∼il2li1o0n,0 tokens; tishhe 2,000 cseens,te cnoncse Europarl t.e5s tm isl eito nco tnoskiesntesd; hoef 41,434 stem tokens. The labels in the output sequence y were obtained by selecting the most productive 150 stems, and then collapsing certain vowels into equivalence classes corresponding to Finnish vowel harmony patterns. Thus 4Note that unlike Section 2.2 we do not use Unsup L-match because when evaluating the CRF model on the suffix prediction task it obtained 95.61% without using Unsup L-match and 82.99% when using Unsup L-match. 35 variants -k¨ o and -ko become vowel-generic enclitic particle -kO, and variants -ss ¨a and -ssa become the vowel-generic inessive case marker -ssA, etc. This is the only language-specific component of our translation model. However, we expect this approach to work for other agglutinative languages as well. For fusional languages like Spanish, another mapping from suffix to abstract tags might be needed. These suffix transformations to their equivalence classes prevent morphophonemic variants of the same morpheme from competing against each other in the prediction model. This resulted in 44 possible label outputs per stem which was a reasonable sized tag-set for CRF training. The CRF was trained on monolingual features of the segmented text for suffix prediction, where t is the current token: Word Stem st−n, .., st, .., st+n(n = 4) Morph Prediction yt−2 , yt−1 , yt With this simple feature set, we were able to use features over longer distances, resulting in a total of 1,110,075 model features. After CRF based recovery of the suffix tag sequence, we use a bigram language model trained on a full segmented version on the training data to recover the original vowels. We used bigrams only, because the suffix vowel harmony alternation depends only upon the preceding phonemes in the word from which it was segmented. original training koskevaa mietint o¨ ¨a data: k ¨asitell ¨a ¨an segmentation: koske+ +va+ +a mietint ¨o+ + a¨ k a¨si+ +te+ +ll a¨+ + a¨+ +n (train bigram language model with mapping A = { a , a }) map n fi bniaglr asmuff liaxn gtou agbest mraocdte tag-set: koske+ +va+ +A mietint ¨o+ +A k ¨asi+ +te+ +ll ¨a+ + ¨a+ +n (train CRF model to predict the final suffix) peeling of final suffix: koske+ +va+ mietint ¨o+ k a¨si+ +te+ +ll a¨+ + a¨+ (train SMT model on this transformation of training data) (a) Training decoder output: koske+ +va+ mietint o¨+ k a¨si+ +te+ +ll a¨+ + a¨+ decoder output stitched up: koskeva+ mietint o¨+ k ¨asitell ¨a ¨a+ CRF model prediction: x = ‘koskeva+ mietint ¨o+ k ¨asitell ¨a ¨a+’, y = ‘+A +A +n’ koskeva+ +A mietint ¨o+ +A k ¨asitell a¨ ¨a+ +n unstitch morphemes: koske+ +va+ +A mietint ¨o+ +A k ¨asi+ +te+ +ll ¨a+ + ¨a+ +n language model disambiguation: koske+ +va+ +a mietint ¨o+ + a¨ k a¨si+ +te+ +ll a¨+ + a¨+ +n final stitching: koskevaa mietint o¨ ¨a k ¨asitell ¨a ¨an (the output is then compared to the reference translation) (b) Decoding Figure 2: Worked example of all steps in the post-processing morphology prediction model. 3 Experimental Results used the Europarl version 3 corpus (Koehn, 2005) English-Finnish training data set, as well as the standard development and test data sets. Our parallel training data consists of ∼1 million senFor all of the models built in this paper, we tpeanrcaelsle lo tfr a4i0n nwgor ddast or less, sw ohfi ∼le 1t mhei development and test sets were each 2,000 sentences long. In all the experiments conducted in this paper, we used the Moses5 phrase-based translation system (Koehn et al. , 2007) , 2008 version. We trained all of the Moses systems herein using the standard features: language model, reordering model, translation model, and word penalty; in addition to these, the factored experiments called for additional translation and generation features for the added factors as noted above. We used in all experiments the following settings: a hypothesis stack size 100, distortion limit 6, phrase translations limit 20, and maximum phrase length 20. For the language models, we used SRILM 5-gram language models (Stolcke, 2002) for all factors. For our word-based Baseline system, we trained a word-based model using the same Moses system with identical settings. For evaluation against segmented translation systems in segmented forms before word reconstruction, we also segmented the baseline system’s word-based output. All the BLEU scores reported are for lowercase evaluation. We did an initial evaluation of the segmented output translation for each system using the no5http://www.statmt.org/moses/ 36 TabSlBUeuna2gps:meulSipengLmta-e nioatedchMo12dme804-.lB8S714cL±oEr0eUs.6 9 S8up19Nre.358ofe498rUs9ntoihe supervised segmentation baseline model. m-BLEU indicates that the segmented output was evaluated against a segmented version of the reference (this measure does not have the same correlation with human judgement as BLEU) . No Uni indicates the segmented BLEU score without unigrams. tion of m-BLEU score (Luong et al. , 2010) where the BLEU score is computed by comparing the segmented output with a segmented reference translation. Table 2 shows the m-BLEU scores for various systems. We also show the m-BLEU score without unigrams, since over-segmentation could lead to artificially high m-BLEU scores. In fact, if we compare the relative improvement of our m-BLEU scores for the Unsup L-match system we see a relative improvement of 39.75% over the baseline. Luong et. al. (2010) report an m-BLEU score of 55.64% but obtain a relative improvement of 0.6% over their baseline m-BLEU score. We find that when using a good segmentation model, segmentation of the morphologically complex target language improves model performance over an unsegmented baseline (the confidence scores come from bootstrap resampling) . Table 3 shows the evaluation scores for all the baselines and the methods introduced in this paper using standard wordbased lowercase BLEU, WER and PER. We do TSCMaFBU(LubanRolpcesdFotu3lne-ipLr:gMdeLT-tms.al,Stc2ho0r1es:)l 1wB54 Le.r682E90c 27a9Us∗eBL-7 W46E3. U659478R6,1WE-7 TR412E. 847Ra1528nd TER. The ∗ indicates a statistically significant improvement o∗f BndLiEcaUte score over tchalel yB saisgenli nfice mntod imel.The boldface scores are the best performing scores per evaluation measure. better than (Luong et al. , 2010) , the previous best score for this task. We also show a better relative improvement over our baseline when compared to (Luong et al., 2010) : a relative improvement of 4.86% for Unsup L-match compared to our baseline word-based model, compared to their 1.65% improvement over their baseline word-based model. Our best performing method used unsupervised morphology with L-match (see Section 2.2) and the improvement is significant: bootstrap resampling provides a confidence margin of ±0.77 and a t-test (Collins ceot nafli.d , 2005) sahrogwined o significance aw ti-thte p = 0o.0ll0in1s. 3.1 Morphological Fluency Analysis To see how well the models were doing at getting morphology right, we examined several patterns of morphological behavior. While we wish to explore minimally supervised morphological MT models, and use as little language specific information as possible, we do want to use linguistic analysis on the output of our system to see how well the models capture essential morphological information in the target language. So, we ran the word-based baseline system, the segmented model (Unsup L-match) , and the prediction model (CRF-LM) outputs, along with the reference translation through the supervised morphological analyzer Omorfi (Pirinen and Listenmaa, 2007) . Using this analysis, we looked at a variety of linguistic constructions that might reveal patterns in morphological behavior. These were: (a) explicitly marked 37 noun forms, (b) noun-adjective case agreement, (c) subject-verb person/number agreement, (d) transitive object case marking, (e) postpositions, and (f) possession. In each of these categories, we looked for construction matches on a per-sentence level between the models’ output and the reference translation. Table 4 shows the models’ performance on the constructions we examined. In all of the categories, the CRF-LM model achieves the best precision score, as we explain below, while the Unsup L-match model most frequently gets the highest recall score. A general pattern in the most prevalent of these constructions is that the baseline tends to prefer the least marked form for noun cases (corresponding to the nominative) more than the reference or the CRF-LM model. The baseline leaves nouns in the (unmarked) nominative far more than the reference, while the CRF-LM model comes much closer, so it seems to fare better at explicitly marking forms, rather than defaulting to the more frequent unmarked form. Finnish adjectives must be marked with the same case as their head noun, while verbs must agree in person and number with their subject. We saw that in both these categories, the CRFLM model outperforms for precision, while the segmented model gets the best recall. In addition, Finnish generally marks direct objects of verbs with the accusative or the partitive case; we observed more accusative/partitive-marked nouns following verbs in the CRF-LM output than in the baseline, as illustrated by example (1) in Fig. 3. While neither translation picks the same verb as in the reference for the input ‘clarify,’ the CRFLM-output paraphrases it by using a grammatical construction of the transitive verb followed by a noun phrase inflected with the accusative case, correctly capturing the transitive construction. The baseline translation instead follows ‘give’ with a direct object in the nominative case. To help clarify the constructions in question, we have used Google Translate6 to provide back6 http://translate.google. com/ of occurrences per sentence, recall and F-score. also averaged The constructions over the various translations. are listed in descending P, R and F stand for precision, order of their frequency in the texts. The highlighted value in each column is the most accurate with respect to the reference value. translations of our MT output into English; to contextualize these back-translations, we have provided Google’s back-translation of the reference. The use of postpositions shows another difference between the models. Finnish postpositions require the preceding noun to be in the genitive or sometimes partitive case, which occurs correctly more frequently in the CRF-LM than the baseline. In example (2) in Fig. 3, all three translations correspond to the English text, ‘with the basque nationalists. ’ However, the CRF-LM output is more grammatical than the baseline, because not only do the adjective and noun agree for case, but the noun ‘baskien’ to which the postposition ‘kanssa’ belongs is marked with the correct genitive case. However, this well-formedness is not rewarded by BLEU, because ‘baskien’ does not match the reference. In addition, while Finnish may express possession using case marking alone, it has another construction for possession; this can disambiguate an otherwise ambiguous clause. This alternate construction uses a pronoun in the genitive case followed by a possessive-marked noun; we see that the CRF-LM model correctly marks this construction more frequently than the baseline. As example (3) in Fig. 3 shows, while neither model correctly translates ‘matkan’ (‘trip’) , the baseline’s output attributes the inessive ‘yhteydess’ (‘connection’) as belonging to ‘tulokset’ (‘results’) , and misses marking the possession linking it to ‘Commissioner Fischler’. Our manual evaluation shows that the CRF38 LM model is producing output translations that are more morphologically fluent than the wordbased baseline and the segmented translation Unsup L-match system, even though the word choices lead to a lower BLEU score overall when compared to Unsup L-match. 4 Related Work The work on morphology in MT can be grouped into three categories, factored models, segmented translation, and morphology generation. Factored models (Koehn and Hoang, 2007) factor the phrase translation probabilities over additional information annotated to each word, allowing for text to be represented on multiple levels of analysis. We discussed the drawbacks of factored models for our task in Section 2. 1. While (Koehn and Hoang, 2007; Yang and Kirchhoff, 2006; Avramidis and Koehn, 2008) obtain improvements using factored models for translation into English, German, Spanish, and Czech, these models may be less useful for capturing long-distance dependencies in languages with much more complex morphological systems such as Finnish. In our experiments factored models did worse than the baseline. Segmented translation performs morphological analysis on the morphologically complex text for use in the translation model (Brown et al. , 1993; Goldwater and McClosky, 2005; de Gispert and Mari n˜o, 2008) . This method unpacks complex forms into simpler, more frequently occurring components, and may also increase the symmetry of the lexically realized content be(1) Input: ‘the charter we are to approve today both strengthens and gives visible shape to the common fundamental rights and values our community is to be based upon. ’ a. Reference: perusoikeuskirja , jonka t ¨an ¨a ¨an aiomme hyv a¨ksy ¨a , sek ¨a vahvistaa ett ¨a selvent a¨ a¨ (selvent ¨a a¨/VERB/ACT/INF/SG/LAT-clarify) niit a¨ (ne/PRONOUN/PL/PAR-them) yhteisi ¨a perusoikeuksia ja arvoja , joiden on oltava yhteis¨ omme perusta. Back-translation: ‘Charter of Fundamental Rights, which today we are going to accept that clarify and strengthen the common fundamental rights and values, which must be community based. ’ b. Baseline: perusoikeuskirja me hyv ¨aksymme t¨ an ¨a a¨n molemmat vahvistaa ja antaa (antaa/VERB/INF/SG/LATgive) n a¨kyv a¨ (n¨ aky a¨/VERB/ACT/PCP/SG/NOM-visible) muokata yhteist ¨a perusoikeuksia ja arvoja on perustuttava. Back-translation: ‘Charter today, we accept both confirm and modify to make a visible and common values, fundamental rights must be based. ’ c. CRF-LM: perusoikeuskirja on hyv a¨ksytty t ¨an ¨a ¨an , sek ¨a vahvistaa ja antaa (antaa/VERB/ACT/INF/SG/LAT-give) konkreettisen (konkreettinen/ADJECTIVE/SG/GEN,ACC-concrete) muodon (muoto/NOUN/SG/GEN,ACCshape) yhteisi ¨a perusoikeuksia ja perusarvoja , yhteis¨ on on perustuttava. Back-translation: ‘Charter has been approved today, and to strengthen and give concrete shape to the common basic rights and fundamental values, the Community must be based. ’ (2) Input: ‘with the basque nationalists’ a. Reference: baskimaan kansallismielisten kanssa basque-SG/NOM+land-SG/GEN,ACC nationalists-PL/GEN with-POST b. Baseline: baskimaan kansallismieliset kanssa basque-SG/NOM-+land-SG/GEN,ACC kansallismielinen-PL/NOM,ACC-nationalists POST-with c. CRF-LM: kansallismielisten baskien kanssa nationalists-PL/GEN basques-PL/GEN with-POST (3) Input: ‘and in this respect we should value the latest measures from commissioner fischler , the results of his trip to morocco on the 26th of last month and the high level meetings that took place, including the one with the king himself’ a. Reference: ja t ¨ass¨ a mieless ¨a osaamme my¨ os arvostaa komission j¨ asen fischlerin viimeisimpi ¨a toimia , jotka ovat h a¨nen (h¨ anen/GEN-his) marokkoon 26 lokakuuta tekemns (tekem¨ ans ¨a/POSS-his) matkan (matkan/GENtour) ja korkean tason kokousten jopa itsens¨ a kuninkaan kanssa tulosta Back-translation: ‘and in this sense we can also appreciate the Commissioner Fischler’s latest actions, which are his to Morocco 26 October trip to high-level meetings and even the king himself with the result b. Baseline: ja t ¨ass¨ a yhteydess a¨ olisi arvoa viimeisin toimia komission j¨ asen fischler , tulokset monitulkintaisia marokon yhteydess a¨ (yhteydess/INE-connection) , ja viime kuussa pidettiin korkean tason kokouksissa , mukaan luettuna kuninkaan kanssa Back-translation: ‘and in this context would be the value of the last act, Commissioner Fischler, the results of the Moroccan context, ambiguous, and last month held high level meetings, including with the king’ c. CRF-LM: ja t ¨ass¨ a yhteydess a¨ meid ¨an olisi lis ¨aarvoa viimeist ¨a toimenpiteit a¨ kuin komission j¨ asen fischler , ett a¨ h a¨nen (h¨ anen/GEN-his) kokemuksensa (kokemuksensa/POSS-experience) marokolle (marokolle-Moroccan) viime kuun 26 ja korkean tason tapaamiset j¨ arjestettiin, kuninkaan kanssa Back-translation: ‘and in this context, we should value the last measures as the Commissioner Fischler, that his experience in Morocco has on the 26th and high-level meetings took place, including with the king. ’ Figure 3: Morphological fluency analysis (see Section 3. 1) . tween source and target. In a somewhat orthogonal approach to ours, (Ma et al. , 2007) use alignment of a parallel text to pack together adjacent segments in the alignment output, which are then fed back to the word aligner to bootstrap an improved alignment, which is then used in the translation model. We compared our results against (Luong et al. , 2010) in Table 3 since their results are directly comparable to ours. They use a segmented phrase table and language model along with the word-based versions in the decoder and in tuning a Finnish target. Their approach requires segmented phrases 39 to match word boundaries, eliminating morphologically productive phrases. In their work a segmented language model can score a translation, but cannot insert morphology that does not show source-side reflexes. In order to perform a similar experiment that still allowed for morphologically productive phrases, we tried training a segmented translation model, the output of which we stitched up in tuning so as to tune to a word-based reference. The goal of this experiment was to control the segmented model’s tendency to overfit by rewarding it for using correct whole-word forms. However, we found that this approach was less successful than using the segmented reference in tuning, and could not meet the baseline (13.97% BLEU best tuning score, versus 14.93% BLEU for the baseline best tuning score) . Previous work in segmented translation has often used linguistically motivated morphological analysis selectively applied based on a language-specific heuristic. A typical approach is to select a highly inflecting class of words and segment them for particular morphology (de Gispert and Mari n˜o, 2008; Ramanathan et al. , 2009) . Popovi¸ c and Ney (2004) perform segmentation to reduce morphological complexity of the source to translate into an isolating target, reducing the translation error rate for the English target. For Czech-to-English, Goldwater and McClosky (2005) lemmatized the source text and inserted a set of ‘pseudowords’ expected to have lexical reflexes in English. Minkov et. al. (2007) and Toutanova et. al. (2008) use a Maximum Entropy Markov Model for morphology generation. The main drawback to this approach is that it removes morphological information from the translation model (which only uses stems) ; this can be a problem for languages in which morphology ex- presses lexical content. de Gispert (2008) uses a language-specific targeted morphological classifier for Spanish verbs to avoid this issue. Talbot and Osborne (2006) use clustering to group morphological variants of words for word alignments and for smoothing phrase translation tables. Habash (2007) provides various methods to incorporate morphological variants of words in the phrase table in order to help recognize out of vocabulary words in the source language. 5 Conclusion and Future Work We found that using a segmented translation model based on unsupervised morphology induction and a model that combined morpheme segments in the translation model with a postprocessing morphology prediction model gave us better BLEU scores than a word-based baseline. Using our proposed approach we obtain better scores than the state of the art on the EnglishFinnish translation task (Luong et al. , 2010) : from 14.82% BLEU to 15.09%, while using a 40 simpler model. We show that using morphological segmentation in the translation model can improve output translation scores. We also demonstrate that for Finnish (and possibly other agglutinative languages) , phrase-based MT benefits from allowing the translation model access to morphological segmentation yielding productive morphological phrases. Taking advantage of linguistic analysis of the output we show that using a post-processing morphology generation model can improve translation fluency on a sub-word level, in a manner that is not captured by the BLEU word-based evaluation measure. In order to help with replication of the results in this paper, we have run the various morphological analysis steps and created the necessary training, tuning and test data files needed in order to train, tune and test any phrase-based machine translation system with our data. The files can be downloaded from natlang. cs.sfu. ca. In future work we hope to explore the utility of phrases with productive morpheme boundaries and explore why they are not used more pervasively in the decoder. Evaluation measures for morphologically complex languages and tun- ing to those measures are also important future work directions. Also, we would like to explore a non-pipelined approach to morphological preand post-processing so that a globally trained model could be used to remove the target side morphemes that would improve the translation model and then predict those morphemes in the target language. Acknowledgements This research was partially supported by NSERC, Canada (RGPIN: 264905) and a Google Faculty Award. We would like to thank Christian Monson, Franz Och, Fred Popowich, Howard Johnson, Majid Razmara, Baskaran Sankaran and the anonymous reviewers for their valuable comments on this work. We would particularly like to thank the developers of the open-source Moses machine translation toolkit and the Omorfi morphological analyzer for Finnish which we used for our experiments. References Eleftherios Avramidis and Philipp Koehn. 2008. Enriching morphologically poor languages for statistical machine translation. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, page 763?770, Columbus, Ohio, USA. Association for Computational Linguistics. Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and R. L. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2) :263–31 1. Pi-Chuan Chang, Michel Galley, and Christopher D. Manning. 2008. Optimizing Chinese word segmentation for machine translation performance. In Proceedings of the Third Workshop on Statistical Machine Translation, pages 224–232, Columbus, Ohio, June. Association for Computational Linguistics. Michael Collins, Philipp Koehn, and Ivona Kucerova. 2005. Clause restructuring for statistical machine translation. In Proceedings of 43rd Annual Meeting of the Association for Computational Linguistics (A CL05). Association for Computational Linguistics. Mathias Creutz and Krista Lagus. 2005. Inducing the morphological lexicon of a natural language from unannotated text. In Proceedings of the International and Interdisciplinary Conference on Adaptive Knowledge Representation and Reason- ing (AKRR ’05), pages 106–113, Espoo, Finland. Mathias Creutz and Krista Lagus. 2006. Morfessor in the morpho challenge. In Proceedings of the PASCAL Challenge Workshop on Unsupervised Segmentation of Words into Morphemes. Adri ´a de Gispert and Jos e´ Mari n˜o. 2008. On the impact of morphology in English to Spanish statistical MT. Speech Communication, 50(11-12) . Sharon Goldwater and David McClosky. 2005. Improving statistical MT through morphological analysis. In Proceedings of the Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 676–683, Vancouver, B.C. , Canada. Association for Computational Linguistics. Philipp Koehn and Hieu Hoang. 2007. Factored translation models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 868–876, Prague, Czech Republic. Association for Computational Linguistics. 41 Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In A CL ‘07: Proceedings of the 45th Annual Meeting of the A CL on Interactive Poster and Demonstration Sessions, pages 177–108, Prague, Czech Republic. Association for Computational Linguistics. Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In Proceedings of Machine Translation Summit X, pages 79–86, Phuket, Thailand. Association for Computational Linguistics. John Lafferty, Andrew McCallum, and Fernando Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning, pages 282–289, San Francisco, California, USA. Association for Computing Machinery. Minh-Thang Luong, Preslav Nakov, and Min-Yen Kan. 2010. A hybrid morpheme-word representation for machine translation of morphologically rich languages. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 148–157, Cambridge, Massachusetts. Association for Computational Linguistics. Yanjun Ma, Nicolas Stroppa, and Andy Way. 2007. Bootstrapping word alignment via word packing. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 304–311, Prague, Czech Republic. Association for Computational Linguistics. Einat Minkov, Kristina Toutanova, and Hisami Suzuki. 2007. Generating complex morphology for machine translation. In In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (A CL07), pages 128–135, Prague, Czech Republic. Association for Computational Linguistics. Christian Monson. 2008. Paramor and morpho challenge 2008. In Lecture Notes in Computer Science: Workshop of the Cross-Language Evaluation Forum (CLEF 2008), Revised Selected Papers. Habash Nizar. 2007. Four techniques for online handling of out-of-vocabulary words in arabic-english statistical machine translation. In Proceedings of the 46th Annual Meeting of the Association of Computational Linguistics, Columbus, Ohio. Association for Computational Linguistics. Kishore Papineni, Salim Roukos, Todd Ward, and Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of 40th Annual Meeting of the Association for Computational Linguistics A CL, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. Tommi Pirinen and Inari Listenmaa. 2007. Omorfi morphological analzer. http://gna.org/projects/omorfi. Maja Popovi¸ c and Hermann Ney. 2004. Towards the use of word stems and suffixes for statistiWei jing cal machine translation. In Proceedings of the 4th International Conference on Language Resources and Evaluation (LREC), pages 1585–1588, Lisbon, Portugal. European Language Resources Association (ELRA) . Ananthakrishnan Ramanathan, Hansraj Choudhary, Avishek Ghosh, and Pushpak Bhattacharyya. 2009. Case markers and morphology: Addressing the crux of the fluency problem in EnglishHindi SMT. In Proceedings of the Joint Conference of the 4 7th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, pages 800–808, Suntec, Singapore. Association for Computational Linguistics. Andreas Stolcke. 2002. Srilm – an extensible language modeling toolkit. 7th International Conference on Spoken Language Processing, 3:901–904. David Talbot and Miles Osborne. 2006. Modelling lexical redundancy for machine translation. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 969–976, Sydney, Australia, July. Association for Computational Linguistics. Kristina Toutanova, Hisami Suzuki, and Achim Ruopp. 2008. Applying morphology generation models to machine translation. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 514–522, Columbus, Ohio, USA. Association for Computational Linguistics. Mei Yang and Katrin Kirchhoff. 2006. Phrase-based backoff models for machine translation of highly inflected languages. In Proceedings of the European Chapter of the Association for Computational Linguistics, pages 41–48, Trento, Italy. Association for Computational Linguistics. 42
4 0.13716841 184 acl-2011-Joint Hebrew Segmentation and Parsing using a PCFGLA Lattice Parser
Author: Yoav Goldberg ; Michael Elhadad
Abstract: We experiment with extending a lattice parsing methodology for parsing Hebrew (Goldberg and Tsarfaty, 2008; Golderg et al., 2009) to make use of a stronger syntactic model: the PCFG-LA Berkeley Parser. We show that the methodology is very effective: using a small training set of about 5500 trees, we construct a parser which parses and segments unsegmented Hebrew text with an F-score of almost 80%, an error reduction of over 20% over the best previous result for this task. This result indicates that lattice parsing with the Berkeley parser is an effective methodology for parsing over uncertain inputs.
5 0.12911786 132 acl-2011-Extracting Paraphrases from Definition Sentences on the Web
Author: Chikara Hashimoto ; Kentaro Torisawa ; Stijn De Saeger ; Jun'ichi Kazama ; Sadao Kurohashi
Abstract: ¶ kuro@i . We propose an automatic method of extracting paraphrases from definition sentences, which are also automatically acquired from the Web. We observe that a huge number of concepts are defined in Web documents, and that the sentences that define the same concept tend to convey mostly the same information using different expressions and thus contain many paraphrases. We show that a large number of paraphrases can be automatically extracted with high precision by regarding the sentences that define the same concept as parallel corpora. Experimental results indicated that with our method it was possible to extract about 300,000 paraphrases from 6 Web docu3m0e0n,t0s0 w0i ptha a precision oramte 6 6o ×f a 1b0out 94%. 108
6 0.12476856 72 acl-2011-Collecting Highly Parallel Data for Paraphrase Evaluation
7 0.11603864 225 acl-2011-Monolingual Alignment by Edit Rate Computation on Sentential Paraphrase Pairs
8 0.10090235 44 acl-2011-An exponential translation model for target language morphology
9 0.1007221 247 acl-2011-Pre- and Postprocessing for Statistical Machine Translation into Germanic Languages
10 0.10009994 193 acl-2011-Language-independent compound splitting with morphological operations
11 0.096839413 171 acl-2011-Incremental Syntactic Language Models for Phrase-based Translation
12 0.093743674 87 acl-2011-Corpus Expansion for Statistical Machine Translation with Semantic Role Label Substitution Rules
13 0.092760921 313 acl-2011-Two Easy Improvements to Lexical Weighting
14 0.081452765 43 acl-2011-An Unsupervised Model for Joint Phrase Alignment and Extraction
15 0.080676198 57 acl-2011-Bayesian Word Alignment for Statistical Machine Translation
16 0.079978414 10 acl-2011-A Discriminative Model for Joint Morphological Disambiguation and Dependency Parsing
17 0.079433389 318 acl-2011-Unsupervised Bilingual Morpheme Segmentation and Alignment with Context-rich Hidden Semi-Markov Models
18 0.077424809 29 acl-2011-A Word-Class Approach to Labeling PSCFG Rules for Machine Translation
19 0.074739359 327 acl-2011-Using Bilingual Parallel Corpora for Cross-Lingual Textual Entailment
20 0.073048487 164 acl-2011-Improving Arabic Dependency Parsing with Form-based and Functional Morphological Features
topicId topicWeight
[(0, 0.172), (1, -0.119), (2, 0.063), (3, 0.061), (4, -0.005), (5, 0.028), (6, 0.145), (7, -0.014), (8, 0.07), (9, -0.056), (10, -0.017), (11, 0.099), (12, -0.085), (13, -0.023), (14, 0.104), (15, -0.051), (16, -0.04), (17, 0.024), (18, -0.011), (19, 0.122), (20, 0.002), (21, 0.031), (22, 0.052), (23, -0.063), (24, 0.046), (25, -0.028), (26, 0.043), (27, -0.014), (28, -0.056), (29, -0.001), (30, -0.016), (31, 0.02), (32, -0.017), (33, 0.033), (34, -0.01), (35, -0.006), (36, -0.005), (37, -0.049), (38, -0.062), (39, 0.021), (40, -0.092), (41, -0.078), (42, -0.008), (43, -0.087), (44, 0.023), (45, 0.056), (46, -0.019), (47, -0.076), (48, -0.021), (49, 0.01)]
simIndex simValue paperId paperTitle
same-paper 1 0.88356149 310 acl-2011-Translating from Morphologically Complex Languages: A Paraphrase-Based Approach
Author: Preslav Nakov ; Hwee Tou Ng
Abstract: We propose a novel approach to translating from a morphologically complex language. Unlike previous research, which has targeted word inflections and concatenations, we focus on the pairwise relationship between morphologically related words, which we treat as potential paraphrases and handle using paraphrasing techniques at the word, phrase, and sentence level. An important advantage of this framework is that it can cope with derivational morphology, which has so far remained largely beyond the capabilities of statistical machine translation systems. Our experiments translating from Malay, whose morphology is mostly derivational, into English show signif- icant improvements over rivaling approaches based on five automatic evaluation measures (for 320,000 sentence pairs; 9.5 million English word tokens).
2 0.71963722 75 acl-2011-Combining Morpheme-based Machine Translation with Post-processing Morpheme Prediction
Author: Ann Clifton ; Anoop Sarkar
Abstract: This paper extends the training and tuning regime for phrase-based statistical machine translation to obtain fluent translations into morphologically complex languages (we build an English to Finnish translation system) . Our methods use unsupervised morphology induction. Unlike previous work we focus on morphologically productive phrase pairs – our decoder can combine morphemes across phrase boundaries. Morphemes in the target language may not have a corresponding morpheme or word in the source language. Therefore, we propose a novel combination of post-processing morphology prediction with morpheme-based translation. We show, using both automatic evaluation scores and linguistically motivated analyses of the output, that our methods outperform previously proposed ones and pro- vide the best known results on the EnglishFinnish Europarl translation task. Our methods are mostly language independent, so they should improve translation into other target languages with complex morphology. 1 Translation and Morphology Languages with rich morphological systems present significant hurdles for statistical machine translation (SMT) , most notably data sparsity, source-target asymmetry, and problems with automatic evaluation. In this work, we propose to address the problem of morphological complexity in an Englishto-Finnish MT task within a phrase-based translation framework. We focus on unsupervised segmentation methods to derive the morphological information supplied to the MT model in order to provide coverage on very large datasets and for languages with few hand-annotated 32 resources. In fact, in our experiments, unsupervised morphology always outperforms the use of a hand-built morphological analyzer. Rather than focusing on a few linguistically motivated aspects of Finnish morphological behaviour, we develop techniques for handling morphological complexity in general. We chose Finnish as our target language for this work, because it exemplifies many of the problems morphologically complex languages present for SMT. Among all the languages in the Europarl data-set, Finnish is the most difficult language to translate from and into, as was demonstrated in the MT Summit shared task (Koehn, 2005) . Another reason is the current lack of knowledge about how to apply SMT successfully to agglutinative languages like Turkish or Finnish. Our main contributions are: 1) the introduction of the notion of segmented translation where we explicitly allow phrase pairs that can end with a dangling morpheme, which can connect with other morphemes as part of the translation process, and 2) the use of a fully segmented translation model in combination with a post-processing morpheme prediction system, using unsupervised morphology induction. Both of these approaches beat the state of the art on the English-Finnish translation task. Morphology can express both content and function categories, and our experiments show that it is important to use morphology both within the translation model (for morphology with content) and outside it (for morphology contributing to fluency) . Automatic evaluation measures for MT, BLEU (Papineni et al., 2002), WER (Word Error Rate) and PER (Position Independent Word Error Rate) use the word as the basic unit rather than morphemes. In a word comProce dPinogrstla ofn tdh,e O 4r9etghon A,n Jnu nael 1 M9-e 2t4i,n2g 0 o1f1 t.he ?c A2s0s1o1ci Aatsiosonc fioartio Cno fmorpu Ctoamtiopnuatalt Lioin gauli Lsitnicgsu,i psatgices 32–42, prised of multiple morphemes, getting even a single morpheme wrong means the entire word is wrong. In addition to standard MT evaluation measures, we perform a detailed linguistic analysis of the output. Our proposed approaches are significantly better than the state of the art, achieving the highest reported BLEU scores on the English-Finnish Europarl version 3 data-set. Our linguistic analysis shows that our models have fewer morpho-syntactic errors compared to the word-based baseline. 2 2.1 Models Baseline Models We set up three baseline models for comparison in this work. The first is a basic wordbased model (called Baseline in the results) ; we trained this on the original unsegmented version of the text. Our second baseline is a factored translation model (Koehn and Hoang, 2007) (called Factored) , which used as factors the word, “stem” 1 and suffix. These are derived from the same unsupervised segmentation model used in other experiments. The results (Table 3) show that a factored model was unable to match the scores of a simple wordbased baseline. We hypothesize that this may be an inherently difficult representational form for a language with the degree of morphological complexity found in Finnish. Because the morphology generation must be precomputed, for languages with a high degree of morphological complexity, the combinatorial explosion makes it unmanageable to capture the full range of morphological productivity. In addition, because the morphological variants are generated on a per-word basis within a given phrase, it excludes productive morphological combination across phrase boundaries and makes it impossible for the model to take into account any longdistance dependencies between morphemes. We conclude from this result that it may be more useful for an agglutinative language to use morphology beyond the confines of the phrasal unit, and condition its generation on more than just the local target stem. In order to compare the 1see Section 2.2. 33 performance of unsupervised segmentation for translation, our third baseline is a segmented translation model based on a supervised segmentation model (called Sup) , using the hand-built Omorfi morphological analyzer (Pirinen and Listenmaa, 2007) , which provided slightly higher BLEU scores than the word-based baseline. 2.2 Segmented Translation For segmented translation models, it cannot be taken for granted that greater linguistic accuracy in segmentation yields improved translation (Chang et al. , 2008) . Rather, the goal in segmentation for translation is instead to maximize the amount of lexical content-carrying morphology, while generalizing over the information not helpful for improving the translation model. We therefore trained several different segmentation models, considering factors of granularity, coverage, and source-target symmetry. We performed unsupervised segmentation of the target data, using Morfessor (Creutz and Lagus, 2005) and Paramor (Monson, 2008) , two top systems from the Morpho Challenge 2008 (their combined output was the Morpho Challenge winner) . However, translation models based upon either Paramor alone or the combined systems output could not match the wordbased baseline, so we concentrated on Morfessor. Morfessor uses minimum description length criteria to train a HMM-based segmentation model. When tested against a human-annotated gold standard of linguistic morpheme segmentations for Finnish, this algorithm outperforms competing unsupervised methods, achieving an F-score of 67.0% on a 3 million sentence corpus (Creutz and Lagus, 2006) . Varying the perplexity threshold in Morfessor does not segment more word types, but rather over-segments the same word types. In order to get robust, common segmentations, we trained the segmenter on the 5000 most frequent words2 ; we then used this to segment the entire data set. In order to improve coverage, we then further segmented 2For the factored model baseline we also used the same setting perplexity = 30, 5,000 most frequent words, but with all but the last suffix collapsed and called the “stem” . TabHMleoat1nr:gplhiMngor phermphocTur631ae04in, 81c9ie03ns67gi,64n0S14e567theTp 2rsa51t, 29Se 3t168able and in translation. any word type that contained a match from the most frequent suffix set, looking for the longest matching suffix character string. We call this method Unsup L-match. After the segmentation, word-internal morpheme boundary markers were inserted into the segmented text to be used to reconstruct the surface forms in the MT output. We then trained the Moses phrase-based system (Koehn et al., 2007) on the segmented and marked text. After decoding, it was a simple matter to join together all adjacent morphemes with word-internal boundary markers to reconstruct the surface forms. Figure 1(a) gives the full model overview for all the variants of the segmented translation model (supervised/unsupervised; with and without the Unsup L-match procedure) . Table 1shows how morphemes are being used in the MT system. Of the phrases that included segmentations (‘Morph’ in Table 1) , roughly a third were ‘productive’, i.e. had a hanging morpheme (with a form such as stem+) that could be joined to a suffix (‘Hanging Morph’ in Table 1) . However, in phrases used while decoding the development and test data, roughly a quarter of the phrases that generated the translated output included segmentations, but of these, only a small fraction (6%) had a hanging morpheme; and while there are many possible reasons to account for this we were unable to find a single convincing cause. 2.3 Morphology Generation Morphology generation as a post-processing step allows major vocabulary reduction in the translation model, and allows the use of morphologically targeted features for modeling inflection. A possible disadvantage of this approach is that in this model there is no opportunity to con34 sider the morphology in translation since it is removed prior to training the translation model. Morphology generation models can use a variety of bilingual and contextual information to capture dependencies between morphemes, often more long-distance than what is possible using n-gram language models over morphemes in the segmented model. Similar to previous work (Minkov et al. , 2007; Toutanova et al. , 2008) , we model morphology generation as a sequence learning problem. Un- like previous work, we use unsupervised morphology induction and use automatically generated suffix classes as tags. The first phase of our morphology prediction model is to train a MT system that produces morphologically simplified word forms in the target language. The output word forms are complex stems (a stem and some suffixes) but still missing some important suffix morphemes. In the second phase, the output of the MT decoder is then tagged with a sequence of abstract suffix tags. In particular, the output of the MT decoder is a sequence of complex stems denoted by x and the output is a sequence of suffix class tags denoted by y. We use a list of parts from (x,y) and map to a d-dimensional feature vector Φ(x, y) , with each dimension being a real number. We infer the best sequence of tags using: F(x) = argymaxp(y | x,w) where F(x) returns the highest scoring output y∗ . A conditional random field (CRF) (Lafferty et al. , 2001) defines the conditional probability as a linear score for each candidate y and a global normalization term: logp(y | x, w) = Φ(x, y) · w − log Z where Z = Py0∈ exp(Φ(x, y0) · w) . We use stochastiPc gradient descent (using crfsgd3) to train the weight vector w. So far, this is all off-the-shelf sequence learning. However, the output y∗ from the CRF decoder is still only a sequence of abstract suffix tags. The third and final phase in our morphology prediction model GEN(x) 3 http://leon. bottou. org/projects/sgd English Training Data words Finnish Training Data words Morphological Pre-Processing stem+ +morph MT System Alignment: word word word stem+ +morph stem stem+ +morph Post-Process: Morph Re-Stitching Fully inflected surface form Evaluation against original reference (a) Segmented Translation Model English Training Data words Finnish Training Data Morphological Pre-Prowceosrdsisng 1 stem+ +morph1+ +morph2 Morphological Pre-Processing 2 stem+ +morph1+ MPosrpthe-mPRr+eo-+cSmetsio crhp1i:nhg+swteomrd+ MA+lTmigwnSomyrspdthen 1mt:+ wsotermd complex stem: stem+morph1+ MPo rpsht-oPlro gcyesGse2n:erCaRtioFnstem+morph1+ morph2sLuarnfagcueagfeorMmomdealp ing Fully inflected surface form Evaluation against original reference (b) Post-Processing Model Translation & Generation Figure 1: Training and testing pipelines for the SMT models. is to take the abstract suffix tag sequence y∗ and then map it into fully inflected word forms, and rank those outputs using a morphemic language model. The abstract suffix tags are extracted from the unsupervised morpheme learning process, and are carefully designed to enable CRF training and decoding. We call this model CRFLM for short. Figure 1(b) shows the full pipeline and Figure 2 shows a worked example of all the steps involved. We use the morphologically segmented training data (obtained using the segmented corpus described in Section 2.24) and remove selected suffixes to create a morphologically simplified version of the training data. The MT model is trained on the morphologically simplified training data. The output from the MT system is then used as input to the CRF model. The CRF model was trained on a ∼210,000 Finnish sentences, consisting noefd d∼ o1n.5 a am ∼il2li1o0n,0 tokens; tishhe 2,000 cseens,te cnoncse Europarl t.e5s tm isl eito nco tnoskiesntesd; hoef 41,434 stem tokens. The labels in the output sequence y were obtained by selecting the most productive 150 stems, and then collapsing certain vowels into equivalence classes corresponding to Finnish vowel harmony patterns. Thus 4Note that unlike Section 2.2 we do not use Unsup L-match because when evaluating the CRF model on the suffix prediction task it obtained 95.61% without using Unsup L-match and 82.99% when using Unsup L-match. 35 variants -k¨ o and -ko become vowel-generic enclitic particle -kO, and variants -ss ¨a and -ssa become the vowel-generic inessive case marker -ssA, etc. This is the only language-specific component of our translation model. However, we expect this approach to work for other agglutinative languages as well. For fusional languages like Spanish, another mapping from suffix to abstract tags might be needed. These suffix transformations to their equivalence classes prevent morphophonemic variants of the same morpheme from competing against each other in the prediction model. This resulted in 44 possible label outputs per stem which was a reasonable sized tag-set for CRF training. The CRF was trained on monolingual features of the segmented text for suffix prediction, where t is the current token: Word Stem st−n, .., st, .., st+n(n = 4) Morph Prediction yt−2 , yt−1 , yt With this simple feature set, we were able to use features over longer distances, resulting in a total of 1,110,075 model features. After CRF based recovery of the suffix tag sequence, we use a bigram language model trained on a full segmented version on the training data to recover the original vowels. We used bigrams only, because the suffix vowel harmony alternation depends only upon the preceding phonemes in the word from which it was segmented. original training koskevaa mietint o¨ ¨a data: k ¨asitell ¨a ¨an segmentation: koske+ +va+ +a mietint ¨o+ + a¨ k a¨si+ +te+ +ll a¨+ + a¨+ +n (train bigram language model with mapping A = { a , a }) map n fi bniaglr asmuff liaxn gtou agbest mraocdte tag-set: koske+ +va+ +A mietint ¨o+ +A k ¨asi+ +te+ +ll ¨a+ + ¨a+ +n (train CRF model to predict the final suffix) peeling of final suffix: koske+ +va+ mietint ¨o+ k a¨si+ +te+ +ll a¨+ + a¨+ (train SMT model on this transformation of training data) (a) Training decoder output: koske+ +va+ mietint o¨+ k a¨si+ +te+ +ll a¨+ + a¨+ decoder output stitched up: koskeva+ mietint o¨+ k ¨asitell ¨a ¨a+ CRF model prediction: x = ‘koskeva+ mietint ¨o+ k ¨asitell ¨a ¨a+’, y = ‘+A +A +n’ koskeva+ +A mietint ¨o+ +A k ¨asitell a¨ ¨a+ +n unstitch morphemes: koske+ +va+ +A mietint ¨o+ +A k ¨asi+ +te+ +ll ¨a+ + ¨a+ +n language model disambiguation: koske+ +va+ +a mietint ¨o+ + a¨ k a¨si+ +te+ +ll a¨+ + a¨+ +n final stitching: koskevaa mietint o¨ ¨a k ¨asitell ¨a ¨an (the output is then compared to the reference translation) (b) Decoding Figure 2: Worked example of all steps in the post-processing morphology prediction model. 3 Experimental Results used the Europarl version 3 corpus (Koehn, 2005) English-Finnish training data set, as well as the standard development and test data sets. Our parallel training data consists of ∼1 million senFor all of the models built in this paper, we tpeanrcaelsle lo tfr a4i0n nwgor ddast or less, sw ohfi ∼le 1t mhei development and test sets were each 2,000 sentences long. In all the experiments conducted in this paper, we used the Moses5 phrase-based translation system (Koehn et al. , 2007) , 2008 version. We trained all of the Moses systems herein using the standard features: language model, reordering model, translation model, and word penalty; in addition to these, the factored experiments called for additional translation and generation features for the added factors as noted above. We used in all experiments the following settings: a hypothesis stack size 100, distortion limit 6, phrase translations limit 20, and maximum phrase length 20. For the language models, we used SRILM 5-gram language models (Stolcke, 2002) for all factors. For our word-based Baseline system, we trained a word-based model using the same Moses system with identical settings. For evaluation against segmented translation systems in segmented forms before word reconstruction, we also segmented the baseline system’s word-based output. All the BLEU scores reported are for lowercase evaluation. We did an initial evaluation of the segmented output translation for each system using the no5http://www.statmt.org/moses/ 36 TabSlBUeuna2gps:meulSipengLmta-e nioatedchMo12dme804-.lB8S714cL±oEr0eUs.6 9 S8up19Nre.358ofe498rUs9ntoihe supervised segmentation baseline model. m-BLEU indicates that the segmented output was evaluated against a segmented version of the reference (this measure does not have the same correlation with human judgement as BLEU) . No Uni indicates the segmented BLEU score without unigrams. tion of m-BLEU score (Luong et al. , 2010) where the BLEU score is computed by comparing the segmented output with a segmented reference translation. Table 2 shows the m-BLEU scores for various systems. We also show the m-BLEU score without unigrams, since over-segmentation could lead to artificially high m-BLEU scores. In fact, if we compare the relative improvement of our m-BLEU scores for the Unsup L-match system we see a relative improvement of 39.75% over the baseline. Luong et. al. (2010) report an m-BLEU score of 55.64% but obtain a relative improvement of 0.6% over their baseline m-BLEU score. We find that when using a good segmentation model, segmentation of the morphologically complex target language improves model performance over an unsegmented baseline (the confidence scores come from bootstrap resampling) . Table 3 shows the evaluation scores for all the baselines and the methods introduced in this paper using standard wordbased lowercase BLEU, WER and PER. We do TSCMaFBU(LubanRolpcesdFotu3lne-ipLr:gMdeLT-tms.al,Stc2ho0r1es:)l 1wB54 Le.r682E90c 27a9Us∗eBL-7 W46E3. U659478R6,1WE-7 TR412E. 847Ra1528nd TER. The ∗ indicates a statistically significant improvement o∗f BndLiEcaUte score over tchalel yB saisgenli nfice mntod imel.The boldface scores are the best performing scores per evaluation measure. better than (Luong et al. , 2010) , the previous best score for this task. We also show a better relative improvement over our baseline when compared to (Luong et al., 2010) : a relative improvement of 4.86% for Unsup L-match compared to our baseline word-based model, compared to their 1.65% improvement over their baseline word-based model. Our best performing method used unsupervised morphology with L-match (see Section 2.2) and the improvement is significant: bootstrap resampling provides a confidence margin of ±0.77 and a t-test (Collins ceot nafli.d , 2005) sahrogwined o significance aw ti-thte p = 0o.0ll0in1s. 3.1 Morphological Fluency Analysis To see how well the models were doing at getting morphology right, we examined several patterns of morphological behavior. While we wish to explore minimally supervised morphological MT models, and use as little language specific information as possible, we do want to use linguistic analysis on the output of our system to see how well the models capture essential morphological information in the target language. So, we ran the word-based baseline system, the segmented model (Unsup L-match) , and the prediction model (CRF-LM) outputs, along with the reference translation through the supervised morphological analyzer Omorfi (Pirinen and Listenmaa, 2007) . Using this analysis, we looked at a variety of linguistic constructions that might reveal patterns in morphological behavior. These were: (a) explicitly marked 37 noun forms, (b) noun-adjective case agreement, (c) subject-verb person/number agreement, (d) transitive object case marking, (e) postpositions, and (f) possession. In each of these categories, we looked for construction matches on a per-sentence level between the models’ output and the reference translation. Table 4 shows the models’ performance on the constructions we examined. In all of the categories, the CRF-LM model achieves the best precision score, as we explain below, while the Unsup L-match model most frequently gets the highest recall score. A general pattern in the most prevalent of these constructions is that the baseline tends to prefer the least marked form for noun cases (corresponding to the nominative) more than the reference or the CRF-LM model. The baseline leaves nouns in the (unmarked) nominative far more than the reference, while the CRF-LM model comes much closer, so it seems to fare better at explicitly marking forms, rather than defaulting to the more frequent unmarked form. Finnish adjectives must be marked with the same case as their head noun, while verbs must agree in person and number with their subject. We saw that in both these categories, the CRFLM model outperforms for precision, while the segmented model gets the best recall. In addition, Finnish generally marks direct objects of verbs with the accusative or the partitive case; we observed more accusative/partitive-marked nouns following verbs in the CRF-LM output than in the baseline, as illustrated by example (1) in Fig. 3. While neither translation picks the same verb as in the reference for the input ‘clarify,’ the CRFLM-output paraphrases it by using a grammatical construction of the transitive verb followed by a noun phrase inflected with the accusative case, correctly capturing the transitive construction. The baseline translation instead follows ‘give’ with a direct object in the nominative case. To help clarify the constructions in question, we have used Google Translate6 to provide back6 http://translate.google. com/ of occurrences per sentence, recall and F-score. also averaged The constructions over the various translations. are listed in descending P, R and F stand for precision, order of their frequency in the texts. The highlighted value in each column is the most accurate with respect to the reference value. translations of our MT output into English; to contextualize these back-translations, we have provided Google’s back-translation of the reference. The use of postpositions shows another difference between the models. Finnish postpositions require the preceding noun to be in the genitive or sometimes partitive case, which occurs correctly more frequently in the CRF-LM than the baseline. In example (2) in Fig. 3, all three translations correspond to the English text, ‘with the basque nationalists. ’ However, the CRF-LM output is more grammatical than the baseline, because not only do the adjective and noun agree for case, but the noun ‘baskien’ to which the postposition ‘kanssa’ belongs is marked with the correct genitive case. However, this well-formedness is not rewarded by BLEU, because ‘baskien’ does not match the reference. In addition, while Finnish may express possession using case marking alone, it has another construction for possession; this can disambiguate an otherwise ambiguous clause. This alternate construction uses a pronoun in the genitive case followed by a possessive-marked noun; we see that the CRF-LM model correctly marks this construction more frequently than the baseline. As example (3) in Fig. 3 shows, while neither model correctly translates ‘matkan’ (‘trip’) , the baseline’s output attributes the inessive ‘yhteydess’ (‘connection’) as belonging to ‘tulokset’ (‘results’) , and misses marking the possession linking it to ‘Commissioner Fischler’. Our manual evaluation shows that the CRF38 LM model is producing output translations that are more morphologically fluent than the wordbased baseline and the segmented translation Unsup L-match system, even though the word choices lead to a lower BLEU score overall when compared to Unsup L-match. 4 Related Work The work on morphology in MT can be grouped into three categories, factored models, segmented translation, and morphology generation. Factored models (Koehn and Hoang, 2007) factor the phrase translation probabilities over additional information annotated to each word, allowing for text to be represented on multiple levels of analysis. We discussed the drawbacks of factored models for our task in Section 2. 1. While (Koehn and Hoang, 2007; Yang and Kirchhoff, 2006; Avramidis and Koehn, 2008) obtain improvements using factored models for translation into English, German, Spanish, and Czech, these models may be less useful for capturing long-distance dependencies in languages with much more complex morphological systems such as Finnish. In our experiments factored models did worse than the baseline. Segmented translation performs morphological analysis on the morphologically complex text for use in the translation model (Brown et al. , 1993; Goldwater and McClosky, 2005; de Gispert and Mari n˜o, 2008) . This method unpacks complex forms into simpler, more frequently occurring components, and may also increase the symmetry of the lexically realized content be(1) Input: ‘the charter we are to approve today both strengthens and gives visible shape to the common fundamental rights and values our community is to be based upon. ’ a. Reference: perusoikeuskirja , jonka t ¨an ¨a ¨an aiomme hyv a¨ksy ¨a , sek ¨a vahvistaa ett ¨a selvent a¨ a¨ (selvent ¨a a¨/VERB/ACT/INF/SG/LAT-clarify) niit a¨ (ne/PRONOUN/PL/PAR-them) yhteisi ¨a perusoikeuksia ja arvoja , joiden on oltava yhteis¨ omme perusta. Back-translation: ‘Charter of Fundamental Rights, which today we are going to accept that clarify and strengthen the common fundamental rights and values, which must be community based. ’ b. Baseline: perusoikeuskirja me hyv ¨aksymme t¨ an ¨a a¨n molemmat vahvistaa ja antaa (antaa/VERB/INF/SG/LATgive) n a¨kyv a¨ (n¨ aky a¨/VERB/ACT/PCP/SG/NOM-visible) muokata yhteist ¨a perusoikeuksia ja arvoja on perustuttava. Back-translation: ‘Charter today, we accept both confirm and modify to make a visible and common values, fundamental rights must be based. ’ c. CRF-LM: perusoikeuskirja on hyv a¨ksytty t ¨an ¨a ¨an , sek ¨a vahvistaa ja antaa (antaa/VERB/ACT/INF/SG/LAT-give) konkreettisen (konkreettinen/ADJECTIVE/SG/GEN,ACC-concrete) muodon (muoto/NOUN/SG/GEN,ACCshape) yhteisi ¨a perusoikeuksia ja perusarvoja , yhteis¨ on on perustuttava. Back-translation: ‘Charter has been approved today, and to strengthen and give concrete shape to the common basic rights and fundamental values, the Community must be based. ’ (2) Input: ‘with the basque nationalists’ a. Reference: baskimaan kansallismielisten kanssa basque-SG/NOM+land-SG/GEN,ACC nationalists-PL/GEN with-POST b. Baseline: baskimaan kansallismieliset kanssa basque-SG/NOM-+land-SG/GEN,ACC kansallismielinen-PL/NOM,ACC-nationalists POST-with c. CRF-LM: kansallismielisten baskien kanssa nationalists-PL/GEN basques-PL/GEN with-POST (3) Input: ‘and in this respect we should value the latest measures from commissioner fischler , the results of his trip to morocco on the 26th of last month and the high level meetings that took place, including the one with the king himself’ a. Reference: ja t ¨ass¨ a mieless ¨a osaamme my¨ os arvostaa komission j¨ asen fischlerin viimeisimpi ¨a toimia , jotka ovat h a¨nen (h¨ anen/GEN-his) marokkoon 26 lokakuuta tekemns (tekem¨ ans ¨a/POSS-his) matkan (matkan/GENtour) ja korkean tason kokousten jopa itsens¨ a kuninkaan kanssa tulosta Back-translation: ‘and in this sense we can also appreciate the Commissioner Fischler’s latest actions, which are his to Morocco 26 October trip to high-level meetings and even the king himself with the result b. Baseline: ja t ¨ass¨ a yhteydess a¨ olisi arvoa viimeisin toimia komission j¨ asen fischler , tulokset monitulkintaisia marokon yhteydess a¨ (yhteydess/INE-connection) , ja viime kuussa pidettiin korkean tason kokouksissa , mukaan luettuna kuninkaan kanssa Back-translation: ‘and in this context would be the value of the last act, Commissioner Fischler, the results of the Moroccan context, ambiguous, and last month held high level meetings, including with the king’ c. CRF-LM: ja t ¨ass¨ a yhteydess a¨ meid ¨an olisi lis ¨aarvoa viimeist ¨a toimenpiteit a¨ kuin komission j¨ asen fischler , ett a¨ h a¨nen (h¨ anen/GEN-his) kokemuksensa (kokemuksensa/POSS-experience) marokolle (marokolle-Moroccan) viime kuun 26 ja korkean tason tapaamiset j¨ arjestettiin, kuninkaan kanssa Back-translation: ‘and in this context, we should value the last measures as the Commissioner Fischler, that his experience in Morocco has on the 26th and high-level meetings took place, including with the king. ’ Figure 3: Morphological fluency analysis (see Section 3. 1) . tween source and target. In a somewhat orthogonal approach to ours, (Ma et al. , 2007) use alignment of a parallel text to pack together adjacent segments in the alignment output, which are then fed back to the word aligner to bootstrap an improved alignment, which is then used in the translation model. We compared our results against (Luong et al. , 2010) in Table 3 since their results are directly comparable to ours. They use a segmented phrase table and language model along with the word-based versions in the decoder and in tuning a Finnish target. Their approach requires segmented phrases 39 to match word boundaries, eliminating morphologically productive phrases. In their work a segmented language model can score a translation, but cannot insert morphology that does not show source-side reflexes. In order to perform a similar experiment that still allowed for morphologically productive phrases, we tried training a segmented translation model, the output of which we stitched up in tuning so as to tune to a word-based reference. The goal of this experiment was to control the segmented model’s tendency to overfit by rewarding it for using correct whole-word forms. However, we found that this approach was less successful than using the segmented reference in tuning, and could not meet the baseline (13.97% BLEU best tuning score, versus 14.93% BLEU for the baseline best tuning score) . Previous work in segmented translation has often used linguistically motivated morphological analysis selectively applied based on a language-specific heuristic. A typical approach is to select a highly inflecting class of words and segment them for particular morphology (de Gispert and Mari n˜o, 2008; Ramanathan et al. , 2009) . Popovi¸ c and Ney (2004) perform segmentation to reduce morphological complexity of the source to translate into an isolating target, reducing the translation error rate for the English target. For Czech-to-English, Goldwater and McClosky (2005) lemmatized the source text and inserted a set of ‘pseudowords’ expected to have lexical reflexes in English. Minkov et. al. (2007) and Toutanova et. al. (2008) use a Maximum Entropy Markov Model for morphology generation. The main drawback to this approach is that it removes morphological information from the translation model (which only uses stems) ; this can be a problem for languages in which morphology ex- presses lexical content. de Gispert (2008) uses a language-specific targeted morphological classifier for Spanish verbs to avoid this issue. Talbot and Osborne (2006) use clustering to group morphological variants of words for word alignments and for smoothing phrase translation tables. Habash (2007) provides various methods to incorporate morphological variants of words in the phrase table in order to help recognize out of vocabulary words in the source language. 5 Conclusion and Future Work We found that using a segmented translation model based on unsupervised morphology induction and a model that combined morpheme segments in the translation model with a postprocessing morphology prediction model gave us better BLEU scores than a word-based baseline. Using our proposed approach we obtain better scores than the state of the art on the EnglishFinnish translation task (Luong et al. , 2010) : from 14.82% BLEU to 15.09%, while using a 40 simpler model. We show that using morphological segmentation in the translation model can improve output translation scores. We also demonstrate that for Finnish (and possibly other agglutinative languages) , phrase-based MT benefits from allowing the translation model access to morphological segmentation yielding productive morphological phrases. Taking advantage of linguistic analysis of the output we show that using a post-processing morphology generation model can improve translation fluency on a sub-word level, in a manner that is not captured by the BLEU word-based evaluation measure. In order to help with replication of the results in this paper, we have run the various morphological analysis steps and created the necessary training, tuning and test data files needed in order to train, tune and test any phrase-based machine translation system with our data. The files can be downloaded from natlang. cs.sfu. ca. In future work we hope to explore the utility of phrases with productive morpheme boundaries and explore why they are not used more pervasively in the decoder. Evaluation measures for morphologically complex languages and tun- ing to those measures are also important future work directions. Also, we would like to explore a non-pipelined approach to morphological preand post-processing so that a globally trained model could be used to remove the target side morphemes that would improve the translation model and then predict those morphemes in the target language. Acknowledgements This research was partially supported by NSERC, Canada (RGPIN: 264905) and a Google Faculty Award. We would like to thank Christian Monson, Franz Och, Fred Popowich, Howard Johnson, Majid Razmara, Baskaran Sankaran and the anonymous reviewers for their valuable comments on this work. We would particularly like to thank the developers of the open-source Moses machine translation toolkit and the Omorfi morphological analyzer for Finnish which we used for our experiments. References Eleftherios Avramidis and Philipp Koehn. 2008. Enriching morphologically poor languages for statistical machine translation. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, page 763?770, Columbus, Ohio, USA. Association for Computational Linguistics. Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and R. L. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2) :263–31 1. Pi-Chuan Chang, Michel Galley, and Christopher D. Manning. 2008. Optimizing Chinese word segmentation for machine translation performance. In Proceedings of the Third Workshop on Statistical Machine Translation, pages 224–232, Columbus, Ohio, June. Association for Computational Linguistics. Michael Collins, Philipp Koehn, and Ivona Kucerova. 2005. Clause restructuring for statistical machine translation. In Proceedings of 43rd Annual Meeting of the Association for Computational Linguistics (A CL05). Association for Computational Linguistics. Mathias Creutz and Krista Lagus. 2005. Inducing the morphological lexicon of a natural language from unannotated text. In Proceedings of the International and Interdisciplinary Conference on Adaptive Knowledge Representation and Reason- ing (AKRR ’05), pages 106–113, Espoo, Finland. Mathias Creutz and Krista Lagus. 2006. Morfessor in the morpho challenge. In Proceedings of the PASCAL Challenge Workshop on Unsupervised Segmentation of Words into Morphemes. Adri ´a de Gispert and Jos e´ Mari n˜o. 2008. On the impact of morphology in English to Spanish statistical MT. Speech Communication, 50(11-12) . Sharon Goldwater and David McClosky. 2005. Improving statistical MT through morphological analysis. In Proceedings of the Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 676–683, Vancouver, B.C. , Canada. Association for Computational Linguistics. Philipp Koehn and Hieu Hoang. 2007. Factored translation models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 868–876, Prague, Czech Republic. Association for Computational Linguistics. 41 Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In A CL ‘07: Proceedings of the 45th Annual Meeting of the A CL on Interactive Poster and Demonstration Sessions, pages 177–108, Prague, Czech Republic. Association for Computational Linguistics. Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In Proceedings of Machine Translation Summit X, pages 79–86, Phuket, Thailand. Association for Computational Linguistics. John Lafferty, Andrew McCallum, and Fernando Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning, pages 282–289, San Francisco, California, USA. Association for Computing Machinery. Minh-Thang Luong, Preslav Nakov, and Min-Yen Kan. 2010. A hybrid morpheme-word representation for machine translation of morphologically rich languages. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 148–157, Cambridge, Massachusetts. Association for Computational Linguistics. Yanjun Ma, Nicolas Stroppa, and Andy Way. 2007. Bootstrapping word alignment via word packing. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 304–311, Prague, Czech Republic. Association for Computational Linguistics. Einat Minkov, Kristina Toutanova, and Hisami Suzuki. 2007. Generating complex morphology for machine translation. In In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (A CL07), pages 128–135, Prague, Czech Republic. Association for Computational Linguistics. Christian Monson. 2008. Paramor and morpho challenge 2008. In Lecture Notes in Computer Science: Workshop of the Cross-Language Evaluation Forum (CLEF 2008), Revised Selected Papers. Habash Nizar. 2007. Four techniques for online handling of out-of-vocabulary words in arabic-english statistical machine translation. In Proceedings of the 46th Annual Meeting of the Association of Computational Linguistics, Columbus, Ohio. Association for Computational Linguistics. Kishore Papineni, Salim Roukos, Todd Ward, and Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of 40th Annual Meeting of the Association for Computational Linguistics A CL, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. Tommi Pirinen and Inari Listenmaa. 2007. Omorfi morphological analzer. http://gna.org/projects/omorfi. Maja Popovi¸ c and Hermann Ney. 2004. Towards the use of word stems and suffixes for statistiWei jing cal machine translation. In Proceedings of the 4th International Conference on Language Resources and Evaluation (LREC), pages 1585–1588, Lisbon, Portugal. European Language Resources Association (ELRA) . Ananthakrishnan Ramanathan, Hansraj Choudhary, Avishek Ghosh, and Pushpak Bhattacharyya. 2009. Case markers and morphology: Addressing the crux of the fluency problem in EnglishHindi SMT. In Proceedings of the Joint Conference of the 4 7th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, pages 800–808, Suntec, Singapore. Association for Computational Linguistics. Andreas Stolcke. 2002. Srilm – an extensible language modeling toolkit. 7th International Conference on Spoken Language Processing, 3:901–904. David Talbot and Miles Osborne. 2006. Modelling lexical redundancy for machine translation. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 969–976, Sydney, Australia, July. Association for Computational Linguistics. Kristina Toutanova, Hisami Suzuki, and Achim Ruopp. 2008. Applying morphology generation models to machine translation. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 514–522, Columbus, Ohio, USA. Association for Computational Linguistics. Mei Yang and Katrin Kirchhoff. 2006. Phrase-based backoff models for machine translation of highly inflected languages. In Proceedings of the European Chapter of the Association for Computational Linguistics, pages 41–48, Trento, Italy. Association for Computational Linguistics. 42
3 0.63806742 193 acl-2011-Language-independent compound splitting with morphological operations
Author: Klaus Macherey ; Andrew Dai ; David Talbot ; Ashok Popat ; Franz Och
Abstract: Translating compounds is an important problem in machine translation. Since many compounds have not been observed during training, they pose a challenge for translation systems. Previous decompounding methods have often been restricted to a small set of languages as they cannot deal with more complex compound forming processes. We present a novel and unsupervised method to learn the compound parts and morphological operations needed to split compounds into their compound parts. The method uses a bilingual corpus to learn the morphological operations required to split a compound into its parts. Furthermore, monolingual corpora are used to learn and filter the set of compound part candidates. We evaluate our method within a machine translation task and show significant improvements for various languages to show the versatility of the approach.
4 0.62364876 37 acl-2011-An Empirical Evaluation of Data-Driven Paraphrase Generation Techniques
Author: Donald Metzler ; Eduard Hovy ; Chunliang Zhang
Abstract: Paraphrase generation is an important task that has received a great deal of interest recently. Proposed data-driven solutions to the problem have ranged from simple approaches that make minimal use of NLP tools to more complex approaches that rely on numerous language-dependent resources. Despite all of the attention, there have been very few direct empirical evaluations comparing the merits of the different approaches. This paper empirically examines the tradeoffs between simple and sophisticated paraphrase harvesting approaches to help shed light on their strengths and weaknesses. Our evaluation reveals that very simple approaches fare surprisingly well and have a number of distinct advantages, including strong precision, good coverage, and low redundancy.
5 0.61325186 225 acl-2011-Monolingual Alignment by Edit Rate Computation on Sentential Paraphrase Pairs
Author: Houda Bouamor ; Aurelien Max ; Anne Vilnat
Abstract: In this paper, we present a novel way of tackling the monolingual alignment problem on pairs of sentential paraphrases by means of edit rate computation. In order to inform the edit rate, information in the form of subsentential paraphrases is provided by a range of techniques built for different purposes. We show that the tunable TER-PLUS metric from Machine Translation evaluation can achieve good performance on this task and that it can effectively exploit information coming from complementary sources.
6 0.60176319 132 acl-2011-Extracting Paraphrases from Definition Sentences on the Web
7 0.58789623 247 acl-2011-Pre- and Postprocessing for Statistical Machine Translation into Germanic Languages
8 0.5808211 72 acl-2011-Collecting Highly Parallel Data for Paraphrase Evaluation
9 0.57738417 10 acl-2011-A Discriminative Model for Joint Morphological Disambiguation and Dependency Parsing
10 0.57541817 124 acl-2011-Exploiting Morphology in Turkish Named Entity Recognition System
11 0.53772968 318 acl-2011-Unsupervised Bilingual Morpheme Segmentation and Alignment with Context-rich Hidden Semi-Markov Models
12 0.51796889 313 acl-2011-Two Easy Improvements to Lexical Weighting
13 0.48606709 151 acl-2011-Hindi to Punjabi Machine Translation System
14 0.48445615 171 acl-2011-Incremental Syntactic Language Models for Phrase-based Translation
15 0.48428151 290 acl-2011-Syntax-based Statistical Machine Translation using Tree Automata and Tree Transducers
16 0.48294362 184 acl-2011-Joint Hebrew Segmentation and Parsing using a PCFGLA Lattice Parser
17 0.47878453 44 acl-2011-An exponential translation model for target language morphology
18 0.46952036 327 acl-2011-Using Bilingual Parallel Corpora for Cross-Lingual Textual Entailment
19 0.46715814 249 acl-2011-Predicting Relative Prominence in Noun-Noun Compounds
20 0.45257202 157 acl-2011-I Thou Thee, Thou Traitor: Predicting Formal vs. Informal Address in English Literature
topicId topicWeight
[(5, 0.031), (17, 0.045), (26, 0.027), (31, 0.031), (37, 0.075), (39, 0.053), (41, 0.041), (55, 0.029), (59, 0.045), (72, 0.027), (73, 0.257), (91, 0.034), (96, 0.182), (97, 0.011)]
simIndex simValue paperId paperTitle
1 0.83383179 160 acl-2011-Identifying Sarcasm in Twitter: A Closer Look
Author: Roberto Gonzalez-Ibanez ; Smaranda Muresan ; Nina Wacholder
Abstract: Sarcasm transforms the polarity of an apparently positive or negative utterance into its opposite. We report on a method for constructing a corpus of sarcastic Twitter messages in which determination of the sarcasm of each message has been made by its author. We use this reliable corpus to compare sarcastic utterances in Twitter to utterances that express positive or negative attitudes without sarcasm. We investigate the impact of lexical and pragmatic factors on machine learning effectiveness for identifying sarcastic utterances and we compare the performance of machine learning techniques and human judges on this task. Perhaps unsurprisingly, neither the human judges nor the machine learning techniques perform very well. 1
same-paper 2 0.78574604 310 acl-2011-Translating from Morphologically Complex Languages: A Paraphrase-Based Approach
Author: Preslav Nakov ; Hwee Tou Ng
Abstract: We propose a novel approach to translating from a morphologically complex language. Unlike previous research, which has targeted word inflections and concatenations, we focus on the pairwise relationship between morphologically related words, which we treat as potential paraphrases and handle using paraphrasing techniques at the word, phrase, and sentence level. An important advantage of this framework is that it can cope with derivational morphology, which has so far remained largely beyond the capabilities of statistical machine translation systems. Our experiments translating from Malay, whose morphology is mostly derivational, into English show signif- icant improvements over rivaling approaches based on five automatic evaluation measures (for 320,000 sentence pairs; 9.5 million English word tokens).
3 0.7503078 58 acl-2011-Beam-Width Prediction for Efficient Context-Free Parsing
Author: Nathan Bodenstab ; Aaron Dunlop ; Keith Hall ; Brian Roark
Abstract: Efficient decoding for syntactic parsing has become a necessary research area as statistical grammars grow in accuracy and size and as more NLP applications leverage syntactic analyses. We review prior methods for pruning and then present a new framework that unifies their strengths into a single approach. Using a log linear model, we learn the optimal beam-search pruning parameters for each CYK chart cell, effectively predicting the most promising areas of the model space to explore. We demonstrate that our method is faster than coarse-to-fine pruning, exemplified in both the Charniak and Berkeley parsers, by empirically comparing our parser to the Berkeley parser using the same grammar and under identical operating conditions.
4 0.73989481 37 acl-2011-An Empirical Evaluation of Data-Driven Paraphrase Generation Techniques
Author: Donald Metzler ; Eduard Hovy ; Chunliang Zhang
Abstract: Paraphrase generation is an important task that has received a great deal of interest recently. Proposed data-driven solutions to the problem have ranged from simple approaches that make minimal use of NLP tools to more complex approaches that rely on numerous language-dependent resources. Despite all of the attention, there have been very few direct empirical evaluations comparing the merits of the different approaches. This paper empirically examines the tradeoffs between simple and sophisticated paraphrase harvesting approaches to help shed light on their strengths and weaknesses. Our evaluation reveals that very simple approaches fare surprisingly well and have a number of distinct advantages, including strong precision, good coverage, and low redundancy.
5 0.71847713 81 acl-2011-Consistent Translation using Discriminative Learning - A Translation Memory-inspired Approach
Author: Yanjun Ma ; Yifan He ; Andy Way ; Josef van Genabith
Abstract: We present a discriminative learning method to improve the consistency of translations in phrase-based Statistical Machine Translation (SMT) systems. Our method is inspired by Translation Memory (TM) systems which are widely used by human translators in industrial settings. We constrain the translation of an input sentence using the most similar ‘translation example’ retrieved from the TM. Differently from previous research which used simple fuzzy match thresholds, these constraints are imposed using discriminative learning to optimise the translation performance. We observe that using this method can benefit the SMT system by not only producing consistent translations, but also improved translation outputs. We report a 0.9 point improvement in terms of BLEU score on English–Chinese technical documents.
6 0.65312141 137 acl-2011-Fine-Grained Class Label Markup of Search Queries
7 0.64843237 3 acl-2011-A Bayesian Model for Unsupervised Semantic Parsing
8 0.64788669 318 acl-2011-Unsupervised Bilingual Morpheme Segmentation and Alignment with Context-rich Hidden Semi-Markov Models
9 0.64694679 308 acl-2011-Towards a Framework for Abstractive Summarization of Multimodal Documents
10 0.64586467 241 acl-2011-Parsing the Internal Structure of Words: A New Paradigm for Chinese Word Segmentation
11 0.64578712 187 acl-2011-Jointly Learning to Extract and Compress
12 0.64557862 327 acl-2011-Using Bilingual Parallel Corpora for Cross-Lingual Textual Entailment
13 0.64556819 274 acl-2011-Semi-Supervised Frame-Semantic Parsing for Unknown Predicates
14 0.64547259 117 acl-2011-Entity Set Expansion using Topic information
15 0.64515293 207 acl-2011-Learning to Win by Reading Manuals in a Monte-Carlo Framework
16 0.6443339 178 acl-2011-Interactive Topic Modeling
17 0.64423311 324 acl-2011-Unsupervised Semantic Role Induction via Split-Merge Clustering
18 0.64401209 15 acl-2011-A Hierarchical Pitman-Yor Process HMM for Unsupervised Part of Speech Induction
19 0.64334798 28 acl-2011-A Statistical Tree Annotator and Its Applications
20 0.64319277 190 acl-2011-Knowledge-Based Weak Supervision for Information Extraction of Overlapping Relations