acl acl2013 acl2013-367 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Omri Abend ; Ari Rappoport
Abstract: Syntactic structures, by their nature, reflect first and foremost the formal constructions used for expressing meanings. This renders them sensitive to formal variation both within and across languages, and limits their value to semantic applications. We present UCCA, a novel multi-layered framework for semantic representation that aims to accommodate the semantic distinctions expressed through linguistic utterances. We demonstrate UCCA’s portability across domains and languages, and its relative insensitivity to meaning-preserving syntactic variation. We also show that UCCA can be effectively and quickly learned by annotators with no linguistic background, and describe the compilation of a UCCAannotated corpus.
Reference: text
sentIndex sentText sentNum sentScore
1 We present UCCA, a novel multi-layered framework for semantic representation that aims to accommodate the semantic distinctions expressed through linguistic utterances. [sent-6, score-0.096]
2 We demonstrate UCCA’s portability across domains and languages, and its relative insensitivity to meaning-preserving syntactic variation. [sent-7, score-0.076]
3 We also show that UCCA can be effectively and quickly learned by annotators with no linguistic background, and describe the compilation of a UCCAannotated corpus. [sent-8, score-0.07]
4 For instance, while virtually all syntactic annotation schemes are sensitive to the structural difference between (a) “John took a shower” and (b) “John showered”, they seldom distinguish between (a) and the markedly different (c) “John took my book”. [sent-10, score-0.207]
5 In fact, the annotations of (a) and (c) are identical under the most widely-used schemes for English, the Penn Treebank (PTB) (Marcus et al. [sent-11, score-0.074]
6 This paper presents a novel approach to grammatical representation that annotates semantic distinctions and aims to abstract away from specific syntactic constructions. [sent-24, score-0.196]
7 This paper focuses on the foundational layer of UCCA, a coarse-grained layer that represents some of the most important relations expressed through linguistic utterances, including argument structure of verbs, nouns and adjectives, and the inter-relations between them (Section 2). [sent-29, score-0.454]
8 UCCA is exceptional in (1) being a semantic scheme that abstracts away from specific syntactic forms and is not defined relative to a specific domain or language, (2) providing a coarse-grained representation which allows for open-ended extension, and (3) using cognitively-motivated categories. [sent-37, score-0.089]
9 An extensive comparison of UCCA to existing approaches to syntactic and semantic representation, focusing on the major resources available for English, is found in Section 5. [sent-38, score-0.084]
10 This is an advantage of UCCA over its syntactic counterparts that usually need annotators with extensive background in linguistics (see Section 3). [sent-42, score-0.12]
11 The atomic meaning-bearing units are placed at the leaves of the DAG and are called terminals. [sent-46, score-0.063]
12 In the foundational layer, terminals are words and multi-word chunks, although this definition can be extended to include arbitrary morphemes. [sent-47, score-0.181]
13 A unit may be either (i) a terminal or (ii) several elements jointly viewed as a single entity according to some semantic or cognitive consideration. [sent-49, score-0.116]
14 In many cases, a non-terminal unit is comprised of a single relation and the units it applies to (its arguments), although in some cases it may also contain secondary relations. [sent-50, score-0.144]
15 Hierarchy is formed by using units as arguments or relations in other units. [sent-51, score-0.088]
16 Therefore, the internal structure of a unit is represented by its outbound edges and their categories, while the roles a unit plays in the relations it participates in are represented by its inbound edges. [sent-53, score-0.127]
17 UCCA is a multi-layered formalism, where each layer specifies the relations it encodes. [sent-59, score-0.158]
18 The question of which relations will be annotated (equivalently, which units will be formed) is determined by the layer in question. [sent-60, score-0.247]
19 For example, consider “John kicked his ball”, and assume our current layer encodes the relations expressed by “kicked” and by “his”. [sent-61, score-0.202]
20 Therefore, the units of the sentence are the terminals (which are always units), “his ball” and “John kicked his ball”. [sent-63, score-0.125]
21 The latter two are units by virtue of expressing a relation along with its arguments. [sent-64, score-0.116]
22 2 The UCCA Foundational Layer The foundational layer is designed to cover the entire text, so that each word participates in at least one unit. [sent-68, score-0.296]
23 Argument structure phenomena are considered basic by many approaches to semantic and grammatical representation, and have a high applicative value, as demonstrated by their extensive use in NLP. [sent-70, score-0.073]
24 The foundational layer views the text as a collection of Scenes. [sent-71, score-0.296]
25 For example, the Scene “John loves bananas” is a schematized event, which refers to John’s disposition towards bananas without making any temporal or spatial 1The anaphoric aspects of “his” are not considered part of the current layer (see Section 2. [sent-74, score-0.171]
26 229 ROOT ROOT SBJNOMBJOD John took a shower (a) -ROOT- SBJROOT John showered (b) -ROOT- SBJONBMJOD John took my book -ROOT- (c) Figure 1: CoNLL-style dependency annotations. [sent-76, score-0.168]
27 o”) Table 1: The complete set of categories in UCCA’s foundational layer. [sent-96, score-0.187]
28 Table 1 provides a concise description of the categories used by the foundational layer3. [sent-99, score-0.187]
29 Every Scene contains one main relation, which is the anchor ofthe Scene, the most important relation it describes (similar to frameevoking lexical units in FrameNet (Baker et al. [sent-102, score-0.093]
30 The main relation receives the category State (S) in static and Process (P) in processual Scenes. [sent-105, score-0.093]
31 2As UCCA annotates categories on its edges, Scene nodes bear no special indication. [sent-108, score-0.093]
32 The foundational layer designates a separate set of categories to units that do not evoke a Scene. [sent-116, score-0.383]
33 Centers (C) are the sub-units of a non-Scene unit that are necessary for the unit to be conceptualized and determine its semantic type. [sent-117, score-0.132]
34 First, units that apply to a single C are annotated as Elaborators (E). [sent-120, score-0.089]
35 Some units do not introduce a new relation or entity into the Scene, and are only part of the formal pattern in which they are situated. [sent-133, score-0.093]
36 Already in the foundational layer of UCCA, the need arises for multiple parents. [sent-140, score-0.296]
37 UCCA annotates inter-Scene relations (linkage) and, following Basic Linguistic Theory, distinguishes between three major types of linkage. [sent-146, score-0.094]
38 In general, the coarse-grained foundational layer does not try to resolve fine scope issues. [sent-151, score-0.317]
39 JohnAkPickedAEC his (a) ball JohnCNandCAbMouagrPyhtaAECDsofatogether theECfilmEA AwePSsaw DaFsyCewstoenrd aeyrful (b) (c) Figure 2: Examples of UCCA annotation graphs. [sent-152, score-0.108]
40 The units specifying the relation between Hs are marked as Link- ers (L)6. [sent-158, score-0.093]
41 Unlike common practice in grammatical annotation, linkage relations in UCCA can cross sentence boundaries, as can relations represented in other layers (e. [sent-160, score-0.139]
42 UCCA therefore annotates texts comprised of several paragraphs and not individual sentences (see Section 3). [sent-163, score-0.069]
43 “Golf became a passion for his oldest daughter: she took daily lessons and became very good, reaching the Connecticut Golf Championship. [sent-166, score-0.148]
44 3 UCCA’s Multi-layered Structure Additional layers may refine existing relations or otherwise annotate a complementary set of distinctions. [sent-194, score-0.072]
45 For instance, a refinement layer can categorize linkage relations according to their semantic types (e. [sent-195, score-0.228]
46 Another immediate extension to UCCA’s foundational layer can be the annotation of coreference relations. [sent-198, score-0.377]
47 A coreference layer would annotate a relation between “John” and “his” by introducing a new node whose descendants are these two units. [sent-200, score-0.206]
48 The fact that this node represents a coreference relation would be represented by a label on the edge connecting them to the coreference node. [sent-201, score-0.07]
49 The first row presents the average F-score when comparing the annotations of the different annotators among themselves. [sent-222, score-0.07]
50 The bottom row shows the average F-score between an annotated passage of a trained annotator and its manual correction by an expert. [sent-224, score-0.077]
51 unit between a parent unit and some of its subunits. [sent-227, score-0.102]
52 A layer focusing on Adverbial scope may refine the flat Scene structure assigned by the foundational layer, expressing the scope of “foolishly” over the relation “replied” in the first case, and over the entire Scene in the second. [sent-229, score-0.391]
53 For instance, consider “gave up”, an expression which the foundational layer considers atomic. [sent-231, score-0.296]
54 A layer that annotates tense can break the expression into “gave” and “up”, in order to annotate “gave” as the tense-bearing unit. [sent-232, score-0.225]
55 Although a more complete discussion ofthe formalism is beyond the scope of this paper, we note that the formalism is designed to allow different annotation layers to be defined and annotated independently of one another, in order to facilitate UCCA’s construction through a community effort. [sent-233, score-0.188]
56 The annotation is therefore carried out in passages of 300-400 tokens. [sent-239, score-0.096]
57 The section of the corpus annotated thus far contains 56890 tokens in 148 annotated passages (average length of 385 tokens). [sent-245, score-0.087]
58 Each passage contains 450 units on average and 42. [sent-246, score-0.096]
59 We employ 4 annotators with varying levels of background in linguistics. [sent-259, score-0.066]
60 Two of the annotators have no background in linguistics, one took an introductory course and one holds a Bachelor’s degree in linguistics. [sent-260, score-0.096]
61 The four annotators were given the same passage in each of these cases. [sent-268, score-0.079]
62 In the first passage there was a substantial advantage to the annotators 8In cases a unit had multiple parents, we discarded all but one of its incoming edges. [sent-273, score-0.13]
63 This is an advantage of UCCA over other syntactic annotation schemes that normally require highly proficient annotators. [sent-287, score-0.147]
64 We view this as a major advantage of semantic annotation schemes over their syntactic counterparts, especially given the huge amount of manual labor required for large syntactic annotation projects. [sent-291, score-0.274]
65 An average taken over a sample of passages annotated by all four annotators yielded an F-score of 93. [sent-298, score-0.107]
66 First, many existing schemes are based on other annotation schemes or heavily rely on automatic tools for providing partial annotations. [sent-301, score-0.161]
67 A recent work that did report inter-annotator agreement in terms of bracketing F-score is an annotation project of the PTB’s noun phrases with more elaborate syntactic structure (Vadas and Cur233 ran, 2011). [sent-303, score-0.097]
68 Note that this task is much more limited in scope than UCCA (annotates noun phrases instead of complete passages in UCCA; uses 2 categories instead of 12 in UCCA). [sent-306, score-0.08]
69 However, the syntactic structure of this unit leads annotators at times into analyzing the subject as a non-Scene relation whose C is “portrayal”. [sent-324, score-0.163]
70 We note that all of these disagreements can be easily resolved by introducing an additional layer focusing on the construction in question. [sent-331, score-0.133]
71 4 UCCA’s Benefits to Semantic Tasks UCCA’s relative insensitivity to syntactic forms has potential benefits for a wide variety of semantic tasks. [sent-332, score-0.106]
72 UCCA annotates the sentence as a single Scene, with a single Participant and a processual main relation: “JohnA [tookF [aE showerC]C ]P”. [sent-335, score-0.099]
73 In all of these cases, UCCA annotates the example as a Scene with an A and a P, whose Center is a word expressing the notion of showering. [sent-338, score-0.092]
74 The foundational layer of UCCA annotates negation units as Ds, which yields the annotation “JohnA [doesF]S- notD [haveC]-S [anyE moneyC]A” (where “does . [sent-340, score-0.489]
75 UCCA reflects the similarity of these two sentences, as it annotates both cases as a single Scene which has two Participants and a negation. [sent-345, score-0.069]
76 A syntactic scheme would normally annotate “no” in the second sentence as a modifier of “money”, and “not” as a negation of “have”. [sent-346, score-0.082]
77 As a third example, consider the two sentences “There are children playing in the park” and “Children are playing in the park”. [sent-355, score-0.09]
78 The first contains two clauses, an existential main clause (headed by “there are”) 9The foundational layer places “not” in the Scene level to avoid resolving fine scope issues (see Section 2) 234 and a subordinate clause (“playing in the park”). [sent-357, score-0.317]
79 While the parse trees ofthese sentences are very different, their UCCA annotation in the foundational layer differ only in terms of Function units: “ChildrenA [areF playingC]P [inR theE parkC]A” and “ThereF areF childrenA [playing]P parkC]A”10. [sent-359, score-0.357]
80 [inR theE Aside from machine translation, a great variety of semantic tasks can benefit from a scheme that is relatively insensitive to syntactic variation. [sent-360, score-0.089]
81 Syntactic annotation schemes come in many forms, from lexical categories such as POS tags to intricate hierarchical structures. [sent-368, score-0.135]
82 UCCA diverges from these approaches in aiming to abstract away from specific syntactic forms and to only represent semantic distinctions. [sent-370, score-0.103]
83 Put differently, UCCA advocates an approach that treats syntax as a hidden layer when learning the mapping between form and meaning, while existing syntactic approaches aim to model it manually and explicitly. [sent-371, score-0.169]
84 UCCA does not build on any other annotation layers and therefore implicitly assumes that semantic annotation can be learned directly. [sent-372, score-0.176]
85 The most prominent annotation scheme in NLP for English syntax is the Penn Treebank. [sent-384, score-0.084]
86 Many syntactic schemes are built or derived from it. [sent-385, score-0.086]
87 Several annotated corpora offer a joint syntactic and semantic representation. [sent-392, score-0.092]
88 UCCA diverges from these projects in aiming to abstract away from syntactic variation, and is therefore less coupled with a specific syntactic theory. [sent-396, score-0.109]
89 Semantic role labeling (SRL) schemes bear similarity to the foundational layer, due to their focus on argument structure. [sent-407, score-0.213]
90 At this point, all these schemes provide a more finegrained set of categories than UCCA. [sent-412, score-0.074]
91 Indeed, the construction of the commonly used semantic dependencies derived from these schemes (Surdeanu et al. [sent-419, score-0.08]
92 , 2011), do not support multiple heads, and often reflect syntactic rather than semantic considerations (e. [sent-422, score-0.066]
93 Finally, unlike PropBank and NomBank, UCCA’s foundational layer annotates linkage relations. [sent-430, score-0.405]
94 The disagreement between the schemes reflects both annotation conventions and principle differences, some of which were discussed above. [sent-434, score-0.111]
95 UCCA’s foundational layer can be seen as a complementary effort to FrameNet, as it focuses on high-coverage, coarsegrained annotation, while FrameNet is more finegrained at the expense of coverage. [sent-444, score-0.296]
96 We described the foundational layer of UCCA and the compilation of a UCCA-annotated corpus. [sent-446, score-0.32]
97 We demonstrated UCCA’s relative insensitivity to paraphrases and cross-linguistic syntactic variation. [sent-447, score-0.076]
98 We also discussed UCCA’s accessibility to annotators with no background in linguistics, which can alleviate the almost prohibitive annotation costs of large syntactic annotation projects. [sent-448, score-0.224]
99 Interlingual annotation of parallel text corpora: A new framework for annotation and evaluation. [sent-526, score-0.122]
100 The CoNLL-2008 shared task on joint parsing of syntactic and semantic dependencies. [sent-610, score-0.066]
wordName wordTfidf (topN-words)
[('ucca', 0.858), ('scene', 0.214), ('foundational', 0.163), ('scenes', 0.149), ('layer', 0.133), ('annotates', 0.069), ('propbank', 0.068), ('units', 0.063), ('johna', 0.062), ('annotation', 0.061), ('shower', 0.053), ('unit', 0.051), ('john', 0.05), ('schemes', 0.05), ('nombank', 0.049), ('ball', 0.047), ('annotators', 0.046), ('playing', 0.045), ('kicked', 0.044), ('abend', 0.041), ('omri', 0.041), ('linkage', 0.04), ('insensitivity', 0.04), ('framenet', 0.038), ('distinctions', 0.036), ('syntactic', 0.036), ('conforming', 0.035), ('passages', 0.035), ('cognitive', 0.035), ('became', 0.033), ('passage', 0.033), ('static', 0.033), ('evoked', 0.031), ('semantic', 0.03), ('took', 0.03), ('relation', 0.03), ('foolishly', 0.03), ('passion', 0.03), ('processual', 0.03), ('showered', 0.03), ('formalism', 0.028), ('ptb', 0.028), ('eyn', 0.026), ('shea', 0.026), ('annotated', 0.026), ('participants', 0.026), ('conceptual', 0.025), ('grammatical', 0.025), ('book', 0.025), ('relations', 0.025), ('thee', 0.024), ('layers', 0.024), ('compilation', 0.024), ('categories', 0.024), ('hebrew', 0.024), ('demands', 0.024), ('oxford', 0.024), ('annotations', 0.024), ('scheme', 0.023), ('replied', 0.023), ('expressing', 0.023), ('annotate', 0.023), ('ari', 0.022), ('baker', 0.022), ('lessons', 0.022), ('oepen', 0.022), ('scope', 0.021), ('money', 0.021), ('ross', 0.021), ('dollars', 0.021), ('structures', 0.02), ('coreference', 0.02), ('background', 0.02), ('aref', 0.02), ('azrieli', 0.02), ('bananas', 0.02), ('banarescu', 0.02), ('becamee', 0.02), ('blt', 0.02), ('childrena', 0.02), ('duschte', 0.02), ('elaborators', 0.02), ('langacker', 0.02), ('linkers', 0.02), ('parkc', 0.02), ('portrayal', 0.02), ('sangati', 0.02), ('tomatoes', 0.02), ('tookf', 0.02), ('valin', 0.02), ('yesterday', 0.02), ('distinction', 0.019), ('aiming', 0.019), ('universal', 0.019), ('terminals', 0.018), ('highlighting', 0.018), ('extensive', 0.018), ('annotator', 0.018), ('schematized', 0.018), ('diverges', 0.018)]
simIndex simValue paperId paperTitle
same-paper 1 1.0 367 acl-2013-Universal Conceptual Cognitive Annotation (UCCA)
Author: Omri Abend ; Ari Rappoport
Abstract: Syntactic structures, by their nature, reflect first and foremost the formal constructions used for expressing meanings. This renders them sensitive to formal variation both within and across languages, and limits their value to semantic applications. We present UCCA, a novel multi-layered framework for semantic representation that aims to accommodate the semantic distinctions expressed through linguistic utterances. We demonstrate UCCA’s portability across domains and languages, and its relative insensitivity to meaning-preserving syntactic variation. We also show that UCCA can be effectively and quickly learned by annotators with no linguistic background, and describe the compilation of a UCCAannotated corpus.
2 0.064420469 368 acl-2013-Universal Dependency Annotation for Multilingual Parsing
Author: Ryan McDonald ; Joakim Nivre ; Yvonne Quirmbach-Brundage ; Yoav Goldberg ; Dipanjan Das ; Kuzman Ganchev ; Keith Hall ; Slav Petrov ; Hao Zhang ; Oscar Tackstrom ; Claudia Bedini ; Nuria Bertomeu Castello ; Jungmee Lee
Abstract: We present a new collection of treebanks with homogeneous syntactic dependency annotation for six languages: German, English, Swedish, Spanish, French and Korean. To show the usefulness of such a resource, we present a case study of crosslingual transfer parsing with more reliable evaluation than has been possible before. This ‘universal’ treebank is made freely available in order to facilitate research on multilingual dependency parsing.1
3 0.053878725 98 acl-2013-Cross-lingual Transfer of Semantic Role Labeling Models
Author: Mikhail Kozhevnikov ; Ivan Titov
Abstract: Semantic Role Labeling (SRL) has become one of the standard tasks of natural language processing and proven useful as a source of information for a number of other applications. We address the problem of transferring an SRL model from one language to another using a shared feature representation. This approach is then evaluated on three language pairs, demonstrating competitive performance as compared to a state-of-the-art unsupervised SRL system and a cross-lingual annotation projection baseline. We also consider the contribution of different aspects of the feature representation to the performance of the model and discuss practical applicability of this method. 1 Background and Motivation Semantic role labeling has proven useful in many natural language processing tasks, such as question answering (Shen and Lapata, 2007; Kaisser and Webber, 2007), textual entailment (Sammons et al., 2009), machine translation (Wu and Fung, 2009; Liu and Gildea, 2010; Gao and Vogel, 2011) and dialogue systems (Basili et al., 2009; van der Plas et al., 2009). Multiple models have been designed to automatically predict semantic roles, and a considerable amount of data has been annotated to train these models, if only for a few more popular languages. As the annotation is costly, one would like to leverage existing resources to minimize the human effort required to construct a model for a new language. A number of approaches to the construction of semantic role labeling models for new languages have been proposed. On one end of the scale is unsupervised SRL, such as Grenager and Manning (2006), which requires some expert knowledge, but no labeled data. It clusters together arguments that should bear the same semantic role, but does not assign a particular role to each cluster. On the other end is annotating a new dataset from scratch. There are also intermediate options, which often make use of similarities between languages. This way, if an accurate model exists for one language, it should help simplify the construction of a model for another, related language. The approaches in this third group often use parallel data to bridge the gap between languages. Cross-lingual annotation projection systems (Pad o´ and Lapata, 2009), for example, propagate information directly via word alignment links. However, they are very sensitive to the quality of parallel data, as well as the accuracy of a sourcelanguage model on it. An alternative approach, known as cross-lingual model transfer, or cross-lingual model adaptation, consists of modifying a source-language model to make it directly applicable to a new language. This usually involves constructing a shared feature representation across the two languages. McDonald et al. (201 1) successfully apply this idea to the transfer of dependency parsers, using part-of- speech tags as the shared representation of words. A later extension of T ¨ackstr o¨m et al. (2012) enriches this representation with cross-lingual word clusters, considerably improving the performance. In the case of SRL, a shared representation that is purely syntactic is likely to be insufficient, since structures with different semantics may be realized by the same syntactic construct, for example “in August” vs “in Britain”. However with the help of recently introduced cross-lingual word represen1190 Proce dingsS o f ita h,e B 5u1lgsta Arinan,u Aaulg Musete 4ti-n9g 2 o0f1 t3h.e ? Ac s2s0o1ci3a Atiosnso fcoirat Cio nm foprut Caotimonpaulta Lti nognuails Lti cnsg,u piasgteics 1 90–120 , tations, such as the cross-lingual clustering mentioned above or cross-lingual distributed word representations of Klementiev et al. (2012), we may be able to transfer models of shallow semantics in a similar fashion. In this work we construct a shared feature representation for a pair of languages, employing crosslingual representations of syntactic and lexical information, train a semantic role labeling model on one language and apply it to the other one. This approach yields an SRL model for a new language at a very low cost, effectively requiring only a source language model and parallel data. We evaluate on five (directed) language pairs EN-ZH, ZH-EN, EN-CZ, CZ-EN and EN-FR, where EN, FR, CZ and ZH denote English, French, Czech and Chinese, respectively. The transferred model is compared against two baselines: an unsupervised SRL system and a model trained on the output of a cross-lingual annotation projection system. In the next section we will describe our setup, then in section 3 present the shared feature representation we use, discuss the evaluation data and other technical aspects in section 4, present the results and conclude with an overview of related work. – 2 Setup The purpose of the study is not to develop a yet another semantic role labeling system any existing SRL system can (after some modification) be used in this setup but to assess the practical applicability of cross-lingual model transfer to this – – problem, compare it against the alternatives and identify its strong/weak points depending on a particular setup. 2.1 Semantic Role Labeling Model We consider the dependency-based version of semantic role labeling as described in Haji cˇ et al. (2009) and transfer an SRL model from one language to another. We only consider verbal predicates and ignore the predicate disambiguation stage. We also assume that the predicate identification information is available in most languages it can be obtained using a relatively simple heuristic based on part-of-speech tags. The model performs argument identification and classification (Johansson and Nugues, 2008) separately in a pipeline first each candidate is classified as being or not being a head of an argument phrase with respect to the predicate in question and then each of the arguments is assigned a role from a given inventory. The model is factorized over arguments the decisions regarding the classification of different arguments are made in– – – dependently of each other. With respect to the use of syntactic annotation we consider two options: using an existing dependency parser for the target language and obtaining one by means of cross-lingual transfer (see section 4.2). Following McDonald et al. (201 1), we assume that a part-of-speech tagger is available for the target language. 2.2 SRL in the Low-resource Setting Several approaches have been proposed to obtain an SRL model for a new language with little or no manual annotation. Unsupervised SRL models (Lang and Lapata, 2010) cluster the arguments of predicates in a given corpus according to their semantic roles. The performance of such models can be impressive, especially for those languages where semantic roles correlate strongly with syntactic relation of the argument to its predicate. However, assigning meaningful role labels to the resulting clusters requires additional effort and the model’s parameters generally need some adjustment for every language. If the necessary resources are already available for a closely related language, they can be utilized to facilitate the construction of a model for the target language. This can be achieved either by means of cross-lingual annotation projection (Yarowsky et al., 2001) or by cross-lingual model transfer (Zeman and Resnik, 2008). This last approach is the one we are considering in this work, and the other two options are treated as baselines. The unsupervised model will be further referred to as UNSUP and the projection baseline as PROJ. 2.3 Evaluation Measures We use the F1 measure as a metric for the argument identification stage and accuracy as an aggregate measure of argument classification performance. When comparing to the unsupervised SRL system the clustering evaluation measures are used instead. These are purity and collocation 1191 N1Ximajx|Gj∩ Ci| CO =N1Xjmiax|Gj∩ Ci|, PU = where Ci is the set of arguments in the i-th induced cluster, Gj is the set of arguments in the jth gold cluster and N is the total number of arguments. We report the harmonic mean ofthe two (Lang and Lapata, 2011) and denote it F1c to avoid confusing it with the supervised metric. 3 Model Transfer The idea of this work is to abstract the model away from the particular source language and apply it to a new one. This setup requires that we use the same feature representation for both languages, for example part-of-speech tags and dependency relation labels should be from the same inventory. Some features are not applicable to certain lan- guages because the corresponding phenomena are absent in them. For example, consider a strongly inflected language and an analytic one. While the latter can usually convey the information encoded in the word form in the former one (number, gender, etc.), finding a shared feature representation for such information is non-trivial. In this study we will confine ourselves to those features that are applicable to all languages in question, namely: part-of-speech tags, syntactic dependency structures and representations of the word’s identity. 3.1 Lexical Information We train a model on one language and apply it to a different one. In order for this to work, the words of the two languages have to be mapped into a common feature space. It is also desirable that closely related words from both languages have similar representations in this space. Word mapping. The first option is simply to use the source language words as the shared representation. Here every source language word would have itself as its representation and every target word would map into a source word that corresponds to it. In other words, we supply the model with a gloss of the target sentence. The mapping (bilingual dictionary) we use is derived from a word-aligned parallel corpus, by identifying, for each word in the target language, the word in the source language it is most often aligned to. Cross-lingual clusters. There is no guarantee that each of the words in the evaluation data is present in our dictionary, nor that the corresponding source-language word is present in the training data, so the model would benefit from the ability to generalize over closely related words. This can, for example, be achieved by using cross-lingual word clusters induced in T ¨ackstr o¨m et al. (2012). We incorporate these clusters as features into our model. 3.2 Syntactic Information Part-of-speech Tags. We map part-of-speech tags into the universal tagset following Petrov et al. (2012). This may have a negative effect on the performance of a monolingual model, since most part-of-speech tagsets are more fine-grained than the universal POS tags considered here. For example Penn Treebank inventory contains 36 tags and the universal POS tagset only 12. Since the finergrained POS tags often reflect more languagespecific phenomena, however, they would only be useful for very closely related languages in the cross-lingual setting. The universal part-of-speech tags used in evaluation are derived from gold-standard annotation for all languages except French, where predicted ones had to be used instead. Dependency Structure. Another important aspect of syntactic information is the dependency structure. Most dependency relation inventories are language-specific, and finding a shared representation for them is a challenging problem. One could map dependency relations into a simplified form that would be shared between languages, as it is done for part-of-speech tags in Petrov et al. (2012). The extent to which this would be useful, however, depends on the similarity of syntactic-semantic in– terfaces of the languages in question. In this work we discard the dependency relation labels where the inventories do not match and only consider the unlabeled syntactic dependency graph. Some discrepancies, such as variations in attachment order, may be present even there, but this does not appear to be the case with the datasets we use for evaluation. If a target language is poor in resources, one can obtain a dependency parser for the target language by means of cross-lingual model transfer (Zeman and Resnik, 2008). We 1192 take this into account and evaluate both using the original dependency structures and the ones obtained by means of cross-lingual model transfer. 3.3 The Model The model we use is based on that of Bj ¨orkelund et al. (2009). It is comprised of a set of linear classifiers trained using Liblinear (Fan et al., 2008). The feature model was modified to accommodate the cross-lingual cluster features and the reranker component was not used. We do not model the interaction between different argument roles in the same predicate. While this has been found useful, in the cross-lingual setup one has to be careful with the assumptions made. For example, modeling the sequence of roles using a Markov chain (Thompson et al., 2003) may not work well in the present setting, especially between distant languages, as the order or arguments is not necessarily preserved. Most constraints that prove useful for SRL (Chang et al., 2007) also require customization when applied to a new language, and some rely on languagespecific resources, such as a valency lexicon. Taking into account the interaction between different arguments of a predicate is likely to improve the performance of the transferred model, but this is outside the scope of this work. 3.4 Feature Selection Compatibility of feature representations is necessary but not sufficient for successful model transfer. We have to make sure that the features we use are predictive of similar outcomes in the two languages as well. Depending on the pair of languages in question, different aspects of the feature representation will retain or lose their predictive power. We can be reasonably certain that the identity of an argument word is predictive of its semantic role in any language, but it might or might not be true of, for example, the word directly preceding the argument word. It is therefore important to pre- SCPDGylOespoSntreslTabunc1lra:obsFel-daitnguplrdoaeusntpagd-elronwfu-dcsopeyrnsd c.eylafguhtorsia mepgnrhs vent the model from capturing overly specific aspects of the source language, which we do by confining the model to first-order features. We also avoid feature selection, which, performed on the source language, is unlikely to help the model to better generalize to the target one. The experiments confirm that feature selection and the use of second-order features degrade the performance of the transferred model. 3.5 Feature Groups For each word, we use its part-of-speech tag, cross-lingual cluster id, word identity (glossed, when evaluating on the target language) and its dependency relation to its parent. Features associated with an argument word include the attributes of the predicate word, the argument word, its parent, siblings and children, and the words directly preceding and following it. Also included are the sequences of part-of-speech tags and dependency relations on the path between the predicate and the argument. Since we are also interested in the impact of different aspects of the feature representation, we divide the features into groups as summarized in table 1 and evaluate their respective contributions to the performance of the model. If a feature group is enabled the model has access to the corre– sponding source of information. For example, if only POS group is enabled, the model relies on the part-of-speech tags of the argument, the predicate and the words to the right and left of the argument word. If Synt is enabled too, it also uses the POS tags of the argument’s parent, children and siblings. Word order information constitutes an implicit group that is always available. It includes the Pos it ion feature, which indicates whether the argument is located to the left or to the right of the predicate, and allows the model to look up the attributes of the words directly preceding and following the argument word. The model we compare against the baselines uses all applicable feature groups (Deprel is only used in EN-CZ and CZ-EN experiments with original syntax). 4 Evaluation 4.1 Datasets and Preprocessing Evaluation of the cross-lingual model transfer requires a rather specific kind of dataset. Namely, the data in both languages has to be annotated 1193 with the same set of semantic roles following the same (or compatible) guidelines, which is seldom the case. We have identified three language pairs for which such resources are available: EnglishChinese, English-Czech and English-French. The evaluation datasets for English and Chinese are those from the CoNLL Shared Task 2009 (Haji ˇc et al., 2009) (henceforth CoNLL-ST). Their annotation in the CoNLL-ST is not identical, but the guidelines for “core” semantic roles are similar (Kingsbury et al., 2004), so we evaluate only on core roles here. The data for the second language pair is drawn from the Prague Czech-English Dependency Treebank 2.0 (Haji ˇc et al., 2012), which we converted to a format similar to that of CoNLL-ST1 . The original annotation uses the tectogrammatical representation (Haji ˇc, 2002) and an inventory of semantic roles (or functors), most of which are interpretable across various predicates. Also note that the syntactic anno- tation of English and Czech in PCEDT 2.0 is quite similar (to the extent permitted by the difference in the structure of the two languages) and we can use the dependency relations in our experiments. For English-French, the English CoNLL-ST dataset was used as a source and the model was evaluated on the manually annotated dataset from van der Plas et al. (201 1). The latter contains one thousand sentences from the French part ofthe Europarl (Koehn, 2005) corpus, annotated with semantic roles following an adapted version of PropBank (Palmer et al., 2005) guidelines. The authors perform annotation projection from English to French, using a joint model of syntax and semantics and employing heuristics for filtering. We use a model trained on the output of this projection system as one of the baselines. The evaluation dataset is relatively small in this case, so we perform the transfer only one-way, from English to French. The part-of-speech tags in all datasets were replaced with the universal POS tags of Petrov et al. (2012). For Czech, we have augmented the map- pings to account for the tags that were not present in the datasets from which the original mappings were derived. Namely, tag “t” is mapped to “VERB” and “Y” to “PRON”. We use parallel data to construct a bilingual dictionary used in word mapping, as well as in the projection baseline. For English-Czech – 1see http://www.ml4nlp.de/code-and-data/treex2conll and English-French, the data is drawn from Europarl (Koehn, 2005), for English-Chinese from MultiUN (Eisele and Chen, 2010). The word alignments were obtained using GIZA++ (Och and Ney, 2003) and the intersection heuristic. – 4.2 Syntactic Transfer In the low-resource setting, we cannot always rely on the availability of an accurate dependency parser for the target language. If one is not available, the natural solution would be to use crosslingual model transfer to obtain it. Unfortunately, the models presented in the previous work, such as Zeman and Resnik (2008), McDonald et al. (201 1) and T ¨ackstr o¨m et al. (2012), were not made available, so we reproduced the direct transfer algorithm of McDonald et al. (201 1), using Malt parser (Nivre, 2008) and the same set of features. We did not reimplement the projected transfer algorithm, however, and used the default training procedure instead of perceptron-based learning. The dependency structure thus obtained is, of course, only a rough approximation even a much more sophisticated algorithm may not perform well when transferring syntax between such languages as Czech and English, given the inherent difference in their structure. The scores are shown in table 2. We will henceforth refer to the syntactic annotations that were provided with the datasets as original, as opposed to the annotations obtained by means of syntactic transfer. – 4.3 Baselines Unsupervised Baseline: We are using a version of the unsupervised semantic role induction system of Titov and Klementiev (2012a) adapted to SetupUAS, % Table2:SyntaciE C ZcN HNt- rE ZaCFnN HZRsfer34 692567acuracy,unlabe dat- tachment score (percent). Note that in case of French we evaluate against the output of a supervised system, since manual annotation is not available for this dataset. This score does not reflect the true performance of syntactic transfer. 1194 the shared feature representation considered in order to make the scores comparable with those of the transfer model and, more importantly, to enable evaluation on transferred syntax. Note that the original system, tailored to a more expressive language-specific syntactic representation and equipped with heuristics to identify active/passive voice and other phenomena, achieves higher scores than those we report here. Projection Baseline: The projection baseline we use for English-Czech and English-Chinese is a straightforward one: we label the source side of a parallel corpus using the source-language model, then identify those verbs on the target side that are aligned to a predicate, mark them as predicates and propagate the argument roles in the same fashion. A model is then trained on the resulting training data and applied to the test set. For English-French we instead use the output of a fully featured projection model of van der Plas et al. (201 1), published in the CLASSiC project. 5 Results In order to ensure that the results are consistent, the test sets, except for the French one, were partitioned into five equal parts (of 5 to 10 thousand sentences each, depending on the dataset) and the evaluation performed separately on each one. All evaluation figures for English, Czech or Chinese below are the average values over the five subsets. In case of French, the evaluation dataset is too small to split it further, so instead we ran the evaluation five times on a randomly selected 80% sample of the evaluation data and averaged over those. In both cases the results are consistent over the subsets, the standard deviation does not exceed 0.5% for the transfer system and projection baseline and 1% for the unsupervised system. 5.1 Argument Identification We summarize the results in table 3. Argument identification is known to rely heavily on syntactic information, so it is unsurprising that it proves inaccurate when transferred syntax is used. Our simple projection baseline suffers from the same problem. Even with original syntactic information available, the performance of argument identification is moderate. Note that the model of (van der Plas et al., 2011), though relying on more expressive syntax, only outperforms the transferred system by 3% (F1) on this task. SetupSyntaxTRANSPROJ ZEC NH Z- EFCZNRHt r a n s 3462 1. 536 142 35. 4269 Table3EZ C:N H- CFEZANHZRrgumeon rt ig identf56 7ic13 a. t27903ion,21569t10ra. 3976nsferd model vs. projection baseline, F1. Most unsupervised SRL approaches assume that the argument identification is performed by some external means, for example heuristically (Lang and Lapata, 2011). Such heuristics or unsupervised approaches to argument identification (Abend et al., 2009) can also be used in the present setup. 5.2 Argument Classification In the following tables, TRANS column contains the results for the transferred system, UNSUP for the unsupervised baseline and PROJ for projection baseline. We highlight in bold the higher score where the difference exceeds twice the maximum of the standard deviation estimates of the two results. Table 4 presents the unsupervised evaluation results. Note that the unsupervised model performs as well as the transferred one or better where the – – SetupSyntaxTRANSUNSUP ZEC NH Z- EFCZNRHt r a n s 768 93648. 34627 6 5873. 1769 TableEZ C4NHZ:- FCEZANHZRrgumoe nr itg clasi78 fi94 3c. a25136tion,8 7 r9a4263n. 07 sferd model vs. unsupervised baseline in terms of the clustering metric F1c (see section 2.3). 1195 SetupSyntaxTRANSPROJ ZEC NH Z- EFCZNRHt r a n s 657 053. 1 36456419. 372 Table5EZ C:N H- CFEZANHZRrgumeon rt ig clasif657ic1936a. t170 ion,65 9t3804ra. 20847nsferd model vs. projection baseline, accuracy. original syntactic dependencies are available. In the more realistic scenario with transferred syn- tax, however, the transferred model proves more accurate. In table 5 we compare the transferred system with the projection baseline. It is easy to see that the scores vary strongly depending on the language pair, due to both the difference in the annotation scheme used and the degree of relatedness between the languages. The drop in performance when transferring the model to another language is large in every case, though, see table 6. SetupTargetSource Table6:MoCEZdHeNZ l- FECaZNRcH urac67 y53169o. 017nthes87 o25670u. r1245ceandtrge language using original syntax. The source language scores for English vary between language pairs because of the difference in syntactic annotation and role subset used. We also include the individual F1 scores for the top-10 most frequent labels for EN-CZ transfer with original syntax in table 7. The model provides meaningful predictions here, despite low overall accuracy. Most of the labels2 are self-explanatory: Patient (PAT), Actor (ACT), Time (TWHEN), Effect (EFF), Location (LOC), Manner (MANN), Addressee (ADDR), Extent (EXT). CPHR marks the 2http://ufal.mff.cuni.cz/∼toman/pcedt/en/functors.html LabelFreq.F1Re.Pr. recall and precision for the top-10 most frequent roles. nominal part of a complex predicate, as in “to have [a plan]CPHR”, and DIR3 indicates destination. 5.3 Additional Experiments We now evaluate the contribution of different aspects of the feature representation to the performance of the model. Table 8 contains the results for English-French. FeaturesOrigTrans ferent feature subsets, using original and transferred syntactic information. The fact that the model performs slightly better with transferred syntax may be explained by two factors. Firstly, as we already mentioned, the original syntactic annotation is also produced automatically. Secondly, in the model transfer setup it is more important how closely the syntacticsemantic interface on the target side resembles that on the source side than how well it matches the “true” structure of the target language, and in this respect a transferred dependency parser may have an advantage over one trained on target-language data. The high impact of the Glos s features here 1196 may be partly attributed to the fact that the mapping is derived from the same corpus as the evaluation data Europarl (Koehn, 2005) and partly by the similarity between English and French in terms of word order, usage of articles and prepositions. The moderate contribution of the crosslingual cluster features are likely due to the insufficient granularity of the clustering for this task. For more distant language pairs, the contributions of individual feature groups are less interpretable, so we only highlight a few observations. First of all, both EN-CZ and CZ-EN benefit noticeably from the use of the original syntactic annotation, including dependency relations, but not from the transferred syntax, most likely due to the low syntactic transfer performance. Both perform better when lexical information is available, although – – the improvement is not as significant as in the case of French only up to 5%. The situation with Chinese is somewhat complicated in that adding lexical information here fails to yield an improvement in terms of the metric considered. This is likely due to the fact that we consider only the core roles, which can usually be predicted with high accuracy based on syntactic information alone. – 6 Related Work Development of robust statistical models for core NLP tasks is a challenging problem, and adaptation of existing models to new languages presents a viable alternative to exhaustive annotation for each language. Although the models thus obtained are generally imperfect, they can be further refined for a particular language and domain using techniques such as active learning (Settles, 2010; Chen et al., 2011). Cross-lingual annotation projection (Yarowsky et al., 2001) approaches have been applied ex- tensively to a variety of tasks, including POS tagging (Xi and Hwa, 2005; Das and Petrov, 2011), morphology segmentation (Snyder and Barzilay, 2008), verb classification (Merlo et al., 2002), mention detection (Zitouni and Florian, 2008), LFG parsing (Wr o´blewska and Frank, 2009), information extraction (Kim et al., 2010), SRL (Pad o´ and Lapata, 2009; van der Plas et al., 2011; Annesi and Basili, 2010; Tonelli and Pianta, 2008), dependency parsing (Naseem et al., 2012; Ganchev et al., 2009; Smith and Eisner, 2009; Hwa et al., 2005) or temporal relation prediction (Spreyer and Frank, 2008). Interestingly, it has also been used to propagate morphosyntactic information between old and modern versions of the same language (Meyer, 2011). Cross-lingual model transfer methods (McDonald et al., 2011; Zeman and Resnik, 2008; Durrett et al., 2012; Søgaard, 2011; Lopez et al., 2008) have also been receiving much attention recently. The basic idea behind model transfer is similar to that of cross-lingual annotation projection, as we can see from the way parallel data is used in, for example, McDonald et al. (201 1). A crucial component of direct transfer approaches is the unified feature representation. There are at least two such representations of lexical information (Klementiev et al., 2012; T ¨ackstr o¨m et al., 2012), but both work on word level. This makes it hard to account for phenomena that are expressed differently in the languages considered, for example the syntactic function of a certain word may be indicated by a preposition, inflection or word order, depending on the language. Accurate representation of such information would require an extra level of abstraction (Haji ˇc, 2002). A side-effect ofusing adaptation methods is that we are forced to use the same annotation scheme for the task in question (SRL, in our case), which in turn simplifies the development of cross-lingual tools for downstream tasks. Such representations are also likely to be useful in machine translation. Unsupervised semantic role labeling methods (Lang and Lapata, 2010; Lang and Lapata, 2011; Titov and Klementiev, 2012a; Lorenzo and Cerisara, 2012) also constitute an alternative to cross-lingual model transfer. For an overview of of semi-supervised approaches we refer the reader to Titov and Klementiev (2012b). 7 Conclusion We have considered the cross-lingual model transfer approach as applied to the task of semantic role labeling and observed that for closely related languages it performs comparably to annotation projection approaches. It allows one to quickly construct an SRL model for a new language without manual annotation or language-specific heuristics, provided an accurate model is available for one of the related languages along with a certain amount of parallel data for the two languages. While an1197 notation projection approaches require sentenceand word-aligned parallel data and crucially depend on the accuracy of the syntactic parsing and SRL on the source side of the parallel corpus, cross-lingual model transfer can be performed using only a bilingual dictionary. Unsupervised SRL approaches have their advantages, in particular when no annotated data is available for any of the related languages and there is a syntactic parser available for the target one, but the annotation they produce is not always sufficient. In applications such as Information Retrieval it is preferable to have precise labels, rather than just clusters of arguments, for example. Also note that when applying cross-lingual model transfer in practice, one can improve upon the performance of the simplistic model we use for evaluation, for example by picking the features manually, taking into account the properties of the target language. Domain adaptation techniques can also be employed to adjust the model to the target language. Acknowledgments The authors would like to thank Alexandre Klementiev and Ryan McDonald for useful suggestions and T ¨ackstr o¨m et al. (2012) for sharing the cross-lingual word representations. This research is supported by the MMCI Cluster of Excellence. References Omri Abend, Roi Reichart, and Ari Rappoport. 2009. Unsupervised argument identification for semantic role labeling. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, ACL ’09, pages 28–36, Stroudsburg, PA, USA. Association for Computational Linguistics. Paolo Annesi and Roberto Basili. 2010. Cross-lingual alignment of FrameNet annotations through hidden Markov models. In Proceedings of the 11th international conference on Computational Linguistics and Intelligent Text Processing, CICLing’ 10, pages 12– 25, Berlin, Heidelberg. Springer-Verlag. Roberto Basili, Diego De Cao, Danilo Croce, Bonaventura Coppola, and Alessandro Moschitti. 2009. Cross-language frame semantics transfer in bilingual corpora. In Alexander F. Gelbukh, editor, Proceedings of the 10th International Conference on Computational Linguistics and Intelligent Text Pro- cessing, pages 332–345. Anders Bj ¨orkelund, Love Hafdell, and Pierre Nugues. 2009. Multilingual semantic role labeling. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL 2009): Shared Task, pages 43–48, Boulder, Colorado, June. Association for Computational Linguistics. Ming-Wei Chang, Lev Ratinov, and Dan Roth. 2007. Guiding semi-supervision with constraint-driven learning. In ACL. Chenhua Chen, Alexis Palmer, and Caroline Sporleder. 2011. Enhancing active learning for semantic role labeling via compressed dependency trees. In Proceedings of 5th International Joint Conference on Natural Language Processing, pages 183–191, Chiang Mai, Thailand, November. Asian Federation of Natural Language Processing. Dipanjan Das and Slav Petrov. 2011. Unsupervised part-of-speech tagging with bilingual graph-based projections. Proceedings of the Association for Computational Linguistics. Greg Durrett, Adam Pauls, and Dan Klein. 2012. Syntactic transfer using a bilingual lexicon. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1–1 1, Jeju Island, Korea, July. Association for Computational Linguistics. Andreas Eisele and Yu Chen. 2010. MultiUN: A multilingual corpus from United Nation documents. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10). European Language Resources Association (ELRA). Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, XiangRui Wang, and Chih-Jen Lin. 2008. LIBLINEAR: A library for large linear classification. Journal of Machine Learning Research, 9: 1871–1874. Kuzman Ganchev, Jennifer Gillenwater, and Ben Taskar. 2009. Dependency grammar induction via bitext projection constraints. In Proceedings of the 47th Annual Meeting of the ACL, pages 369–377, Stroudsburg, PA, USA. Association for Computational Linguistics. Qin Gao and Stephan Vogel. 2011. Corpus expansion for statistical machine translation with semantic role label substitution rules. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 294–298, Portland, Oregon, USA. Trond Grenager and Christopher D. Manning. 2006. Unsupervised discovery of a statistical verb lexicon. In Proceedings of EMNLP. Jan Haji cˇ. 2002. Tectogrammatical representation: Towards a minimal transfer in machine translation. In Robert Frank, editor, Proceedings of the 6th International Workshop on Tree Adjoining Grammars 1198 and Related Frameworks (TAG+6), pages 216— 226, Venezia. Universita di Venezia. Jan Haji cˇ, Massimiliano Ciaramita, Richard Johansson, Daisuke Kawahara, Maria Ant o`nia Mart ı´, Llu ı´s M `arquez, Adam Meyers, Joakim Nivre, Sebastian Pad o´, Jan Sˇt eˇp a´nek, Pavel Stra nˇ a´k, Mihai Surdeanu, Nianwen Xue, and Yi Zhang. 2009. The CoNLL2009 shared task: Syntactic and semantic dependencies in multiple languages. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL 2009): Shared Task, pages 1–18, Boulder, Colorado. Jan Haji cˇ, Eva Haji cˇov a´, Jarmila Panevov a´, Petr Sgall, Ond ˇrej Bojar, Silvie Cinkov´ a, Eva Fuˇ c ´ıkov a´, Marie Mikulov a´, Petr Pajas, Jan Popelka, Ji ˇr´ ı Semeck´ y, Jana Sˇindlerov a´, Jan Sˇt eˇp a´nek, Josef Toman, Zde nˇka Ure sˇov a´, and Zden eˇk Zˇabokrtsk y´. 2012. Announcing Prague Czech-English dependency treebank 2.0. In Nicoletta Calzolari (Conference Chair), Khalid Choukri, Thierry Declerck, Mehmet U gˇur Doˇ gan, Bente Maegaard, Joseph Mariani, Jan Odijk, and Stelios Piperidis, editors, Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’12), Istanbul, Turkey, May. European Language Resources Association (ELRA). Rebecca Hwa, Philip Resnik, Amy Weinberg, Clara Cabezas, and Okan Kolak. 2005. Bootstrapping parsers via syntactic projection across parallel text. Natural Language Engineering, 11(3):3 11–325. Richard Johansson and Pierre Nugues. 2008. Dependency-based semantic role labeling of PropBank. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 69–78, Honolulu, Hawaii. Michael Kaisser and Bonnie Webber. 2007. Question answering based on semantic roles. In ACL Workshop on Deep Linguistic Processing. Seokhwan Kim, Minwoo Jeong, Jonghoon Lee, and Gary Geunbae Lee. 2010. A cross-lingual annotation projection approach for relation detection. In Proceedings of the 23rd International Conference on Computational Linguistics, COLING ’ 10, pages 564–571, Stroudsburg, PA, USA. Association for Computational Linguistics. Paul Kingsbury, Nianwen Xue, and Martha Palmer. 2004. Propbanking in parallel. In In Proceedings of the Workshop on the Amazing Utility of Parallel and Comparable Corpora, in conjunction with LREC’04. Alexandre Klementiev, Ivan Titov, and Binod Bhattarai. 2012. Inducing crosslingual distributed representations of words. In Proceedings of the International Conference on Computational Linguistics (COLING), Bombay, India. Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In Conference Proceedings: the tenth Machine Translation Summit, pages 79–86, Phuket, Thailand. AAMT. Joel Lang and Mirella Lapata. 2010. Unsupervised induction of semantic roles. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 939–947, Los Angeles, California, June. Association for Computational Linguistics. Joel Lang and Mirella Lapata. 2011. Unsupervised semantic role induction via split-merge clustering. In Proc. of Annual Meeting of the Association for Computational Linguistics (ACL). Ding Liu and Daniel Gildea. 2010. Semantic role features for machine translation. In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), Beijing, China. Adam Lopez, Daniel Zeman, Michael Nossal, Philip Resnik, and Rebecca Hwa. 2008. Cross-language parser adaptation between related languages. In IJCNLP-08 Workshop on NLP for Less Privileged Languages, pages 35–42, Hyderabad, India, January. Alejandra Lorenzo and Christophe Cerisara. 2012. Unsupervised frame based semantic role induction: application to French and English. In Proceedings of the ACL 2012 Joint Workshop on Statistical Parsing and Semantic Processing of Morphologically Rich Languages, pages 30–35, Jeju, Republic of Korea, July. Association for Computational Linguistics. Ryan McDonald, Slav Petrov, and Keith Hall. 2011. Multi-source transfer of delexicalized dependency parsers. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP ’ 11, pages 62–72, Stroudsburg, PA, USA. Association for Computational Linguistics. Paola Merlo, Suzanne Stevenson, Vivian Tsang, and Gianluca Allaria. 2002. A multi-lingual paradigm for automatic verb classification. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL’02), pages 207– 214, Philadelphia, PA. Roland Meyer. 2011. New wine in old wineskins?– Tagging old Russian via annotation projection from modern translations. Russian Linguistics, 35(2):267(15). Tahira Naseem, Regina Barzilay, and Amir Globerson. 2012. Selective sharing for multilingual dependency parsing. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, pages 629–637, Jeju Island, Korea, July. Association for Computational Linguistics. Joakim Nivre. 2008. Algorithms for deterministic incremental dependency parsing. Comput. Linguist., 34(4):513–553, December. 1199 Franz Josef Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1). Sebastian Pad o´ and Mirella Lapata. 2009. Crosslingual annotation projection for semantic roles. Journal of Artificial Intelligence Research, 36:307– 340. Martha Palmer, Daniel Gildea, and Paul Kingsbury. 2005. The Proposition Bank: An annotated corpus of semantic roles. Computational Linguistics, 31:71–105. Slav Petrov, Dipanjan Das, and Ryan McDonald. 2012. A universal part-of-speech tagset. In Proceedings of LREC, May. Mark Sammons, Vinod Vydiswaran, Tim Vieira, Nikhil Johri, Ming wei Chang, Dan Goldwasser, Vivek Srikumar, Gourab Kundu, Yuancheng Tu, Kevin Small, Joshua Rule, Quang Do, and Dan Roth. 2009. Relation alignment for textual entailment recognition. In Text Analysis Conference (TAC). Burr Settles. 2010. Active learning literature survey. Computer Sciences Technical Report, 1648. Dan Shen and Mirella Lapata. 2007. Using semantic roles to improve question answering. In EMNLP. David A Smith and Jason Eisner. 2009. Parser adaptation and projection with quasi-synchronous grammar features. In Proceedings of the 2009 Confer- ence on Empirical Methods in Natural Language Processing, pages 822–831. Association for Computational Linguistics. Benjamin Snyder and Regina Barzilay. 2008. Crosslingual propagation for morphological analysis. In Proceedings of the 23rd national conference on Artificial intelligence. Anders Søgaard. 2011. Data point selection for crosslanguage adaptation of dependency parsers. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, volume 2 of HLT ’11, pages 682–686, Stroudsburg, PA, USA. Association for Computational Linguistics. Kathrin Spreyer and Anette Frank. 2008. Projectionbased acquisition of a temporal labeller. Proceedings of IJCNLP 2008. Oscar T¨ ackstr o¨m, Ryan McDonald, and Jakob Uszkoreit. 2012. Cross-lingual word clusters for direct transfer of linguistic structure. In Proc. of the Annual Meeting of the North American Association of Computational Linguistics (NAACL), pages 477– 487, Montr ´eal, Canada. Cynthia A. Thompson, Roger Levy, and Christopher D. Manning. 2003. A generative model for seman- tic role labeling. In Proceedings of the 14th European Conference on Machine Learning, ECML 2003, pages 397–408, Dubrovnik, Croatia. Ivan Titov and Alexandre Klementiev. 2012a. A Bayesian approach to unsupervised semantic role induction. In Proc. of European Chapter of the Association for Computational Linguistics (EACL). Ivan Titov and Alexandre Klementiev. 2012b. Semisupervised semantic role labeling: Approaching from an unsupervised perspective. In Proceedings of the International Conference on Computational Linguistics (COLING), Bombay, India, December. Sara Tonelli and Emanuele Pianta. 2008. Frame information transfer from English to Italian. In Proceedings of LREC 2008. Lonneke van der Plas, James Henderson, and Paola Merlo. 2009. Domain adaptation with artificial data for semantic parsing of speech. In Proc. 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 125–128, Boulder, Colorado. Lonneke van der Plas, Paola Merlo, and James Henderson. 2011. Scaling up automatic cross-lingual semantic role annotation. In Proceedings of the 49th Annual Meeting of the Association for Computa- tional Linguistics: Human Language Technologies, HLT ’ 11, pages 299–304, Stroudsburg, PA, USA. Association for Computational Linguistics. Alina Wr o´blewska and Anette Frank. 2009. Crosslingual projection of LFG F-structures: Building an F-structure bank for Polish. In Eighth International Workshop on Treebanks and Linguistic Theories, page 209. Dekai Wu and Pascale Fung. 2009. Can semantic role labeling improve SMT? In Proceedings of 13th Annual Conference of the European Association for Machine Translation (EAMT 2009), Barcelona. Chenhai Xi and Rebecca Hwa. 2005. A backoff model for bootstrapping resources for non-English languages. In Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, pages 85 1–858, Stroudsburg, PA, USA. David Yarowsky, Grace Ngai, and Ricahrd Wicentowski. 2001. Inducing multilingual text analysis tools via robust projection across aligned corpora. In Proceedings of Human Language Technology Conference. Daniel Zeman and Philip Resnik. 2008. Crosslanguage parser adaptation between related lan- guages. In Proceedings of the IJCNLP-08 Workshop on NLP for Less Privileged Languages, pages 35– 42, Hyderabad, India, January. Asian Federation of Natural Language Processing. Imed Zitouni and Radu Florian. 2008. Mention detection crossing the language barrier. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 1200
4 0.052247897 228 acl-2013-Leveraging Domain-Independent Information in Semantic Parsing
Author: Dan Goldwasser ; Dan Roth
Abstract: Semantic parsing is a domain-dependent process by nature, as its output is defined over a set of domain symbols. Motivated by the observation that interpretation can be decomposed into domain-dependent and independent components, we suggest a novel interpretation model, which augments a domain dependent model with abstract information that can be shared by multiple domains. Our experiments show that this type of information is useful and can reduce the annotation effort significantly when moving between domains.
5 0.050529741 83 acl-2013-Collective Annotation of Linguistic Resources: Basic Principles and a Formal Model
Author: Ulle Endriss ; Raquel Fernandez
Abstract: Crowdsourcing, which offers new ways of cheaply and quickly gathering large amounts of information contributed by volunteers online, has revolutionised the collection of labelled data. Yet, to create annotated linguistic resources from this data, we face the challenge of having to combine the judgements of a potentially large group of annotators. In this paper we investigate how to aggregate individual annotations into a single collective annotation, taking inspiration from the field of social choice theory. We formulate a general formal model for collective annotation and propose several aggregation methods that go beyond the commonly used majority rule. We test some of our methods on data from a crowdsourcing experiment on textual entailment annotation.
6 0.04935753 216 acl-2013-Large tagset labeling using Feed Forward Neural Networks. Case study on Romanian Language
7 0.049294569 265 acl-2013-Outsourcing FrameNet to the Crowd
8 0.045482691 189 acl-2013-ImpAr: A Deterministic Algorithm for Implicit Semantic Role Labelling
9 0.04408896 388 acl-2013-Word Alignment Modeling with Context Dependent Deep Neural Network
10 0.042629901 61 acl-2013-Automatic Interpretation of the English Possessive
11 0.042108629 28 acl-2013-A Unified Morpho-Syntactic Scheme of Stanford Dependencies
12 0.041871238 162 acl-2013-FrameNet on the Way to Babel: Creating a Bilingual FrameNet Using Wiktionary as Interlingual Connection
13 0.041714001 219 acl-2013-Learning Entity Representation for Entity Disambiguation
14 0.041011371 52 acl-2013-Annotating named entities in clinical text by combining pre-annotation and active learning
15 0.039981738 85 acl-2013-Combining Intra- and Multi-sentential Rhetorical Parsing for Document-level Discourse Analysis
16 0.038110413 314 acl-2013-Semantic Roles for String to Tree Machine Translation
17 0.036647931 57 acl-2013-Arguments and Modifiers from the Learner's Perspective
18 0.036205065 306 acl-2013-SPred: Large-scale Harvesting of Semantic Predicates
19 0.035898466 270 acl-2013-ParGramBank: The ParGram Parallel Treebank
20 0.035568994 80 acl-2013-Chinese Parsing Exploiting Characters
topicId topicWeight
[(0, 0.108), (1, 0.012), (2, -0.038), (3, -0.05), (4, -0.044), (5, 0.019), (6, 0.004), (7, 0.003), (8, 0.046), (9, 0.031), (10, -0.015), (11, -0.005), (12, -0.042), (13, 0.024), (14, -0.055), (15, -0.025), (16, -0.005), (17, 0.018), (18, 0.006), (19, -0.035), (20, -0.024), (21, -0.001), (22, -0.064), (23, 0.007), (24, 0.015), (25, 0.002), (26, 0.01), (27, -0.024), (28, 0.001), (29, 0.037), (30, -0.007), (31, -0.009), (32, -0.001), (33, -0.0), (34, 0.009), (35, -0.038), (36, -0.018), (37, -0.041), (38, 0.05), (39, -0.044), (40, 0.051), (41, -0.007), (42, 0.038), (43, -0.029), (44, -0.03), (45, -0.039), (46, 0.024), (47, -0.013), (48, 0.01), (49, -0.004)]
simIndex simValue paperId paperTitle
same-paper 1 0.88814491 367 acl-2013-Universal Conceptual Cognitive Annotation (UCCA)
Author: Omri Abend ; Ari Rappoport
Abstract: Syntactic structures, by their nature, reflect first and foremost the formal constructions used for expressing meanings. This renders them sensitive to formal variation both within and across languages, and limits their value to semantic applications. We present UCCA, a novel multi-layered framework for semantic representation that aims to accommodate the semantic distinctions expressed through linguistic utterances. We demonstrate UCCA’s portability across domains and languages, and its relative insensitivity to meaning-preserving syntactic variation. We also show that UCCA can be effectively and quickly learned by annotators with no linguistic background, and describe the compilation of a UCCAannotated corpus.
2 0.6686067 83 acl-2013-Collective Annotation of Linguistic Resources: Basic Principles and a Formal Model
Author: Ulle Endriss ; Raquel Fernandez
Abstract: Crowdsourcing, which offers new ways of cheaply and quickly gathering large amounts of information contributed by volunteers online, has revolutionised the collection of labelled data. Yet, to create annotated linguistic resources from this data, we face the challenge of having to combine the judgements of a potentially large group of annotators. In this paper we investigate how to aggregate individual annotations into a single collective annotation, taking inspiration from the field of social choice theory. We formulate a general formal model for collective annotation and propose several aggregation methods that go beyond the commonly used majority rule. We test some of our methods on data from a crowdsourcing experiment on textual entailment annotation.
3 0.6580075 52 acl-2013-Annotating named entities in clinical text by combining pre-annotation and active learning
Author: Maria Skeppstedt
Abstract: For expanding a corpus of clinical text, annotated for named entities, a method that combines pre-tagging with a version of active learning is proposed. In order to facilitate annotation and to avoid bias, two alternative automatic pre-taggings are presented to the annotator, without revealing which of them is given a higher confidence by the pre-tagging system. The task of the annotator is to select the correct version among these two alternatives. To minimise the instances in which none of the presented pre-taggings is correct, the texts presented to the annotator are actively selected from a pool of unlabelled text, with the selection criterion that one of the presented pre-taggings should have a high probability of being correct, while still being useful for improving the result of an automatic classifier.
4 0.62382764 280 acl-2013-Plurality, Negation, and Quantification:Towards Comprehensive Quantifier Scope Disambiguation
Author: Mehdi Manshadi ; Daniel Gildea ; James Allen
Abstract: Recent work on statistical quantifier scope disambiguation (QSD) has improved upon earlier work by scoping an arbitrary number and type of noun phrases. No corpusbased method, however, has yet addressed QSD when incorporating the implicit universal of plurals and/or operators such as negation. In this paper we report early, though promising, results for automatic QSD when handling both phenomena. We also present a general model for learning to build partial orders from a set of pairwise preferences. We give an n log n algorithm for finding a guaranteed approximation of the optimal solution, which works very well in practice. Finally, we significantly improve the performance of the pre- vious model using a rich set of automatically generated features.
5 0.62265372 385 acl-2013-WebAnno: A Flexible, Web-based and Visually Supported System for Distributed Annotations
Author: Seid Muhie Yimam ; Iryna Gurevych ; Richard Eckart de Castilho ; Chris Biemann
Abstract: We present WebAnno, a general purpose web-based annotation tool for a wide range of linguistic annotations. WebAnno offers annotation project management, freely configurable tagsets and the management of users in different roles. WebAnno uses modern web technology for visualizing and editing annotations in a web browser. It supports arbitrarily large documents, pluggable import/export filters, the curation of annotations across various users, and an interface to farming out annotations to a crowdsourcing platform. Currently WebAnno allows part-ofspeech, named entity, dependency parsing and co-reference chain annotations. The architecture design allows adding additional modes of visualization and editing, when new kinds of annotations are to be supported.
6 0.62173635 28 acl-2013-A Unified Morpho-Syntactic Scheme of Stanford Dependencies
7 0.6201511 61 acl-2013-Automatic Interpretation of the English Possessive
8 0.61007899 161 acl-2013-Fluid Construction Grammar for Historical and Evolutionary Linguistics
9 0.60791445 270 acl-2013-ParGramBank: The ParGram Parallel Treebank
10 0.60270596 265 acl-2013-Outsourcing FrameNet to the Crowd
11 0.59025985 94 acl-2013-Coordination Structures in Dependency Treebanks
12 0.58348501 368 acl-2013-Universal Dependency Annotation for Multilingual Parsing
13 0.57668519 278 acl-2013-Patient Experience in Online Support Forums: Modeling Interpersonal Interactions and Medication Use
14 0.55174428 65 acl-2013-BRAINSUP: Brainstorming Support for Creative Sentence Generation
15 0.54490286 51 acl-2013-AnnoMarket: An Open Cloud Platform for NLP
16 0.54190332 227 acl-2013-Learning to lemmatise Polish noun phrases
17 0.54181105 183 acl-2013-ICARUS - An Extensible Graphical Search Tool for Dependency Treebanks
18 0.53940558 344 acl-2013-The Effects of Lexical Resource Quality on Preference Violation Detection
19 0.52681625 295 acl-2013-Real-World Semi-Supervised Learning of POS-Taggers for Low-Resource Languages
20 0.52677178 324 acl-2013-Smatch: an Evaluation Metric for Semantic Feature Structures
topicId topicWeight
[(0, 0.046), (6, 0.035), (11, 0.065), (14, 0.025), (15, 0.018), (24, 0.048), (26, 0.048), (28, 0.017), (31, 0.229), (35, 0.072), (42, 0.056), (48, 0.047), (61, 0.018), (70, 0.037), (88, 0.026), (90, 0.019), (95, 0.048)]
simIndex simValue paperId paperTitle
1 0.88352889 313 acl-2013-Semantic Parsing with Combinatory Categorial Grammars
Author: Yoav Artzi ; Nicholas FitzGerald ; Luke Zettlemoyer
Abstract: unkown-abstract
2 0.85547084 48 acl-2013-An Open Source Toolkit for Quantitative Historical Linguistics
Author: Johann-Mattis List ; Steven Moran
Abstract: Given the increasing interest and development of computational and quantitative methods in historical linguistics, it is important that scholars have a basis for documenting, testing, evaluating, and sharing complex workflows. We present a novel open-source toolkit for quantitative tasks in historical linguistics that offers these features. This toolkit also serves as an interface between existing software packages and frequently used data formats, and it provides implementations of new and existing algorithms within a homogeneous framework. We illustrate the toolkit’s functionality with an exemplary workflow that starts with raw language data and ends with automatically calculated phonetic alignments, cognates and borrowings. We then illustrate evaluation metrics on gold standard datasets that are provided with the toolkit.
same-paper 3 0.79729438 367 acl-2013-Universal Conceptual Cognitive Annotation (UCCA)
Author: Omri Abend ; Ari Rappoport
Abstract: Syntactic structures, by their nature, reflect first and foremost the formal constructions used for expressing meanings. This renders them sensitive to formal variation both within and across languages, and limits their value to semantic applications. We present UCCA, a novel multi-layered framework for semantic representation that aims to accommodate the semantic distinctions expressed through linguistic utterances. We demonstrate UCCA’s portability across domains and languages, and its relative insensitivity to meaning-preserving syntactic variation. We also show that UCCA can be effectively and quickly learned by annotators with no linguistic background, and describe the compilation of a UCCAannotated corpus.
4 0.76532447 234 acl-2013-Linking and Extending an Open Multilingual Wordnet
Author: Francis Bond ; Ryan Foster
Abstract: We create an open multilingual wordnet with large wordnets for over 26 languages and smaller ones for 57 languages. It is made by combining wordnets with open licences, data from Wiktionary and the Unicode Common Locale Data Repository. Overall there are over 2 million senses for over 100 thousand concepts, linking over 1.4 million words in hundreds of languages.
5 0.64481926 211 acl-2013-LABR: A Large Scale Arabic Book Reviews Dataset
Author: Mohamed Aly ; Amir Atiya
Abstract: We introduce LABR, the largest sentiment analysis dataset to-date for the Arabic language. It consists of over 63,000 book reviews, each rated on a scale of 1 to 5 stars. We investigate the properties of the the dataset, and present its statistics. We explore using the dataset for two tasks: sentiment polarity classification and rating classification. We provide standard splits of the dataset into training and testing, for both polarity and rating classification, in both balanced and unbalanced settings. We run baseline experiments on the dataset to establish a benchmark.
6 0.59244907 374 acl-2013-Using Context Vectors in Improving a Machine Translation System with Bridge Language
7 0.57615489 225 acl-2013-Learning to Order Natural Language Texts
8 0.57230878 382 acl-2013-Variational Inference for Structured NLP Models
9 0.56641489 318 acl-2013-Sentiment Relevance
10 0.56445247 83 acl-2013-Collective Annotation of Linguistic Resources: Basic Principles and a Formal Model
11 0.56415135 275 acl-2013-Parsing with Compositional Vector Grammars
12 0.56236744 185 acl-2013-Identifying Bad Semantic Neighbors for Improving Distributional Thesauri
13 0.55945933 82 acl-2013-Co-regularizing character-based and word-based models for semi-supervised Chinese word segmentation
14 0.55899388 155 acl-2013-Fast and Accurate Shift-Reduce Constituent Parsing
15 0.55717051 2 acl-2013-A Bayesian Model for Joint Unsupervised Induction of Sentiment, Aspect and Discourse Representations
16 0.55642116 276 acl-2013-Part-of-Speech Induction in Dependency Trees for Statistical Machine Translation
17 0.55625355 123 acl-2013-Discriminative Learning with Natural Annotations: Word Segmentation as a Case Study
18 0.55618441 17 acl-2013-A Random Walk Approach to Selectional Preferences Based on Preference Ranking and Propagation
19 0.55446154 169 acl-2013-Generating Synthetic Comparable Questions for News Articles
20 0.55407488 132 acl-2013-Easy-First POS Tagging and Dependency Parsing with Beam Search