emnlp emnlp2011 emnlp2011-87 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Tom Kwiatkowski ; Luke Zettlemoyer ; Sharon Goldwater ; Mark Steedman
Abstract: We consider the problem of learning factored probabilistic CCG grammars for semantic parsing from data containing sentences paired with logical-form meaning representations. Traditional CCG lexicons list lexical items that pair words and phrases with syntactic and semantic content. Such lexicons can be inefficient when words appear repeatedly with closely related lexical content. In this paper, we introduce factored lexicons, which include both lexemes to model word meaning and templates to model systematic variation in word usage. We also present an algorithm for learning factored CCG lexicons, along with a probabilistic parse-selection model. Evaluations on benchmark datasets demonstrate that the approach learns highly accurate parsers, whose generalization performance greatly from the lexical factoring. benefits
Reference: text
sentIndex sentText sentNum sentScore
1 edu ∗School of Informatics University of Edinburgh Edinburgh, EH8 9AB, UK Abstract We consider the problem of learning factored probabilistic CCG grammars for semantic parsing from data containing sentences paired with logical-form meaning representations. [sent-8, score-0.404]
2 Traditional CCG lexicons list lexical items that pair words and phrases with syntactic and semantic content. [sent-9, score-0.294]
3 In this paper, we introduce factored lexicons, which include both lexemes to model word meaning and templates to model systematic variation in word usage. [sent-11, score-0.852]
4 Recent work has focused on learning such parsers directly from corpora made up of sentences paired with logical meaning representations (Kate et al. [sent-15, score-0.274]
5 For example, in a flight booking domain we might have access to training examples such as: Sentence: Meaning: I want flights from Boston λx. [sent-19, score-0.291]
6 For example, a typical learned lexicon might include entries such as: (1) flight ‘ N:λx. [sent-31, score-0.285]
7 In this list, the word “flight” is paired with the predicate flight in three separate lexical items which are required for different syntactic contexts. [sent-41, score-0.389]
8 Representing these three lexical items separately is inefficient, since each word of this class (such as “fare”) will require three similarly structured lexical entries differing only in predicate name. [sent-45, score-0.325]
9 For example, in (6) “Boston” is paired with the constant bos that represents its meaning. [sent-47, score-0.296]
10 However, item (7) also adds the predicate from to the logical form. [sent-48, score-0.326]
11 This might be used to analyse somewhat elliptical, unedited sentences such as “Show me flights Boston to New York,” which can be challenging for semantic parsers (Zettlemoyer and Collins, 2007). [sent-49, score-0.289]
12 1 We develop a factored lexicon that captures this insight by distinguishing lexemes, which pair words with logical constants, from lexical templates, which map lexemes to full lexical items. [sent-52, score-1.001]
13 Each word or phrase will be associated with a few lexemes that can be combined with a shared set of general templates. [sent-54, score-0.35]
14 However, instead of constructing fully specified lexical items for the learned grammar, we automatically generate sets of lexemes and lexical templates to model each example. [sent-58, score-0.882]
15 Instead, we model them with hidden variables and develop an online learning approach that simultaneously estimates the parameters of a log-linear parsing model, while inducing the factored lexicon. [sent-63, score-0.254]
16 2 Related work There has been significant previous work on learning semantic parsers from training sentences labelled with logical form meaning representations. [sent-68, score-0.29]
17 Zettlemoyer and Collins (2005, 2007) presented approaches that use hand generated, English-language specific rules to generate lexical items from logical forms as well as English specific type-shifting rules and relaxations of the CCG combinators to model spontaneous, unedited sentences. [sent-70, score-0.515]
18 (2010) described an approach for language-independent learning that replaces the hand-specified templates with a higher-order-unification-based lexical induction method, but their approach does not scale well to challenging, unedited sentences. [sent-73, score-0.409]
19 The learning approach we develop for inducing factored lexicons is also language independent, but scales well to these challenging sentences. [sent-74, score-0.274]
20 Problem We will learn a semantic parser that takes a sentences x and returns a logical form z repre- senting its underlying meaning. [sent-96, score-0.261]
21 n} containing sentences xi pauntd d logical form)|is zi, f. [sent-100, score-0.242]
22 example xi n=g“S sehnotwen me flights to Boston” and zi = λx. [sent-103, score-0.288]
23 For example, Boston NP : bos represents the entry for 1514 the word “Boston” with syntactic category NP and meaning represented by the constant bos. [sent-107, score-0.336]
24 Where a lexicon would usually list lexical items such as this, we instead use a factored lexicon (L, T) containing: ‘ • • A list of lexemes L. [sent-108, score-0.921]
25 Each lexeme pairs a word or phrase ewxiethm a lsis Lt oflogical cemonest paanitrss st aha wt can be used to construct its meaning. [sent-109, score-0.314]
26 For example, one lexeme might be (Boston, [bos] ). [sent-110, score-0.314]
27 For example, there is a that can map the lexeme above ical entry Boston NP : bos. [sent-113, score-0.314]
28 ‘ Each template t oE a fhu tlle lmepxlicaatel single template to the final lex- We will make central use of this factored representation to provide a more compact representation of the lexicon that can be learned efficiently. [sent-114, score-0.619]
29 The factored PCCG will also contain a parameter vector, θ, that defines a log-linear distribution over the possible parses y, conditioned on the sentence x. [sent-115, score-0.252]
30 Specifically, we modify the lexical learning, to produce lexemes and templates, as well as the feature space of the model, but reuse the existing parameter estimation techniques and overall learning cycle, as described in Section 7. [sent-118, score-0.434]
31 We present the complete approach in three parts by describing the factored representation of the lexicon (Section 5), techniques for proposing potential new lexemes and templates (Section 6), and finally a complete learning algorithm (Section 7). [sent-119, score-0.926]
32 1 Lambda Calculus We represent the meanings of sentences, words and phrases with logical expressions that can contain constants, quantifiers, logical connectors and lambda abstractions. [sent-122, score-0.42]
33 A traditional CCG grammar includes a lexicon Λ with entries like the following: flights ‘ N:λx. [sent-130, score-0.268]
34 f(x) ∧ to(x,y) Boston ‘ NP :bos where each lexical item w‘X : h has words w, a syntwahceticre category X, a itnedm a logical hfo hrams w who. [sent-134, score-0.419]
35 For example, given the lexicon above, the phrase “flights to Boston” can be parsed to produce: flights to Boston λx. [sent-141, score-0.214]
36 We discriminate between competing parses using a log-linear model which has a feature vector φ and a parameter vector The probability of a parse y that returns logical form z, given a sentence x is defined as: θ. [sent-156, score-0.302]
37 5 Factored Lexicons A factored lexicon includes a set L of lexemes and a set T of lexical templates. [sent-176, score-0.733]
38 We will use a set of lexical items from our running example to discuss the details of how the following lexical items: (1) flight ‘ N : λx. [sent-178, score-0.408]
39 from(x, bos) ∧ f(x) are constructed from specific lexemes and templates. [sent-184, score-0.35]
40 1 Lexemes A lexeme (w, c) pairs a word sequence w with an ordered list of logical constants = [c1 . [sent-186, score-0.592]
41 For example, item (1) and (2) above would come from a single lexeme (flight, [flight] ). [sent-190, score-0.422]
42 Similar lexemes would be represented for other predicates, for example (fare, [cost] ). [sent-191, score-0.35]
43 2 Lexical Templates A lexical template takes a lexeme and produces a lexical item. [sent-194, score-0.628]
44 [cωte ‘d from the single lexeme (flight, [flight]) with the two different templates λ (ω, v) . [sent-200, score-0.552]
45 For example, lexical item (7) in our running example above can be constructed from the lexemes (Boston, [bos] ) and (Boston, [from, bos]), given appropriate templates. [sent-206, score-0.542]
46 Given the lexical item produced by the chosen lexeme and tem- c. [sent-208, score-0.506]
47 This direct integration allows for features that signal which lexemes and templates have been used while also allowing for well defined marginal probabilities, by summing over all ways of deriving a specific lexical item. [sent-211, score-0.672]
48 6 Learning Factored Lexicons To induce factored lexicons, we will make use oftwo procedures, presented in this section, that factor lexical items into lexemes and templates. [sent-212, score-0.741]
49 For example, the maximal factoring for the lexical item Boston ‘ NP : bmoasl i fsa tchtoer pair we saw ebxeifcoarle: i (Boston, [bos]) aPnd : λ (ω, v) . [sent-216, score-0.335]
50 When the original algorithm would have added an entry lto the lexicon, we can instead compute the factoring of land add the corresponding lexeme and template to the factored lexicon. [sent-227, score-0.756]
51 λ (Boston, [from, bos] ), w whoicuhld dis i suboptimal since each possible city would need a lexeme of this type, with the additional from constant included. [sent-233, score-0.314]
52 Instead, we would ideally like to learn the lexeme (Boston, [bos] ) and have a template that introduces the from constant. [sent-234, score-0.46]
53 This would model the desired generalization with a single lexeme per city. [sent-235, score-0.314]
54 In order to permit the introduction of extra constants into lexical items, we allow the creation of templates that contain logical constants through partial factorings. [sent-236, score-0.694]
55 f(x) ∧ from(x, )] v1 The use of templates to introduce extra semantic constants into a lexical item is similar to, but more general than, the English-specific type-shifting rules used in Zettlemoyer and Collins (2007), which were introduced to model spontaneous, unedited text. [sent-240, score-0.685]
56 ” To propose templates which introduce semantic content, during learning, we build on the intuition that we need to recover from missing words, such as in the example above. [sent-242, score-0.338]
57 ” We will also assume that we have learned a good factored lexicon for the complete example that could produce the parse: flights from 1517 Boston λx. [sent-244, score-0.451]
58 flight(x) ∧ from(x, bos) Given analyses of this form, we introduce new templates that will allow us to recover from missing words, for example if “from” was dropped. [sent-248, score-0.291]
59 We identify commonly occurring nodes in the best parse trees found during training, in this case the nonterminal spanning “from Boston,” and introduce templates that can produce the nonterminal, even if one of the words is missing. [sent-249, score-0.31]
60 Not all templates introduced this way will model valid generalizations. [sent-254, score-0.238]
61 , 2010) to induce factored lexicons, while also simultanously estimating the parameters of a loglinear CCG parsing model. [sent-258, score-0.254]
62 1 Background: NEW-LEX NEW-LEX generates lexical items by splitting and merging nodes in the best parse tree of each training example. [sent-261, score-0.258]
63 Thh ies splitting procedure is a two step process that first splits the logical form h, then splits the CCG syntactic category X and finally splits the string w. [sent-265, score-0.351]
64 The first step enumerates all possible splits of the logical form h into a pair of new expressions (f,g) that can be used to reconstruct h by either function application (h = f(g)) or composition (h = λx. [sent-266, score-0.215]
65 The next two steps enumerate all ways of splitting the syntactic category X and words w to introduce two new lexical items which can be recombined with CCG combinators (application or composition) to recreate the original parse node X :h spanning w. [sent-271, score-0.419]
66 The algorithm induces a factored PCCG, including the lexemes L, templates T, and parameters θ. [sent-290, score-0.797]
67 Initialization The model is initialized with a factored lexicon as follows. [sent-294, score-0.299]
68 MAX-FAC is a function that takes a lexical item land returns the maximal factoring of it, that is the unique, maximal (lexeme, template) pair that can be combined to construct l, as described in Section 6. [sent-295, score-0.421]
69 Definitions: NEW-LEX(y) returns a set of new lexical items from a parse y as described in Section 7. [sent-309, score-0.257]
70 PART-FAC(y) generates a set of templates from parse y. [sent-312, score-0.283]
71 • Let ∆ = Ep(y|xi,zi;θ,(L,T)) [φ(xi,y,zi)] −Ep(y,z|xi;θ,(L,T))[φ(xi,y,z)] • Set θ = θ + γ∆ Output: Lexemes L, templates T, and parameters θ. [sent-329, score-0.238]
72 The lexemes and templates created in this way provide the initial factored lexicon. [sent-332, score-0.797]
73 Step 1 The first step of the learning algorithm in Figure 1 adds lexemes and templates to the factored model given by performing manipulations on the highest scoring correct parse y∗ of the current training example (xi, zi). [sent-333, score-0.842]
74 We then use the function MAX-FAC to create the maximal factorings of each of these new lexical items as described in Section 6 and these are added to the factored representation of the lexicon. [sent-336, score-0.505]
75 New templates can also be in- troduced through partial factorings of internal parse nodes as described in Section 6. [sent-337, score-0.341]
76 These templates are generated by using the function PART-FAC to abstract over the wordspan and a subset of the constants contained in the internal parse nodes of y∗. [sent-339, score-0.406]
77 This step allows for templates that introduce new semantic content to model elliptical language, as described in Section 6. [sent-340, score-0.409]
78 3 Discussion The FUBL algorithm makes use of a direct online approach, where lexemes and templates are introduced in place while analyzing specific sentences. [sent-344, score-0.588]
79 In general, this will overgeneralize; not all ways of combining lexemes and templates will produce high quality lexical items. [sent-345, score-0.672]
80 8 Experimental setup Data Sets We evaluate on two benchmark semantic parsing datasets: GeoQuery, which is made up of natural language queries to a database of geographical information; and Atis, which contains natural language queries to a flight booking system. [sent-348, score-0.288]
81 Lexical features fire on the lexemes and templates used to build the lexical items used in a parse. [sent-356, score-0.77]
82 For each (lexeme,template) pair used to create a lexical item we have indicator features φl for the lexeme used, φt for the template used, and φ(l,t) for the pair that was used. [sent-357, score-0.652]
83 We assign the features on lexical templates a weight of 0. [sent-358, score-0.322]
84 1 to prevent them from swamping the far less frequent but equally informative lexeme features. [sent-359, score-0.314]
85 Initialization The weights for lexeme features are initialized according to coocurrance statistics between words and logical constants. [sent-365, score-0.498]
86 The initial weights for templates are set by adding −0. [sent-367, score-0.238]
87 words, FUBL returns a parse for all of the Atis test sentences, since the factored lexicons we are learning can produce a very large number oflexical items. [sent-417, score-0.349]
88 Without making any modification to the CCG grammars or parsing combinators, we are able to induce a lexicon that is general enough model the natural occurring variations in the data, for example due to sloppy, unedited sentences. [sent-421, score-0.266]
89 y )I)n, contrast, FUBL uses a lexeme learned from the same word in different contexts, along with a template learnt from similar words in a similar context, to learn to per1520 pittsburgh to atlanta the cheapest on july twentieth NpiPt (λSx∧|NλfyPrλo)m\z. [sent-430, score-0.579]
90 FUBL can learn this type of analysis with novel combinations templates at test time, even if the individual words, like “cheapest,” of lexemes and were never seen in similar syntactic constructions during training, as described in Section 10. [sent-446, score-0.618]
91 As well as providing a new way to search the lexicon during training, the factored lexicon provides a way of proposing new, unseen, lexical items at test time. [sent-448, score-0.61]
92 Interestingly, the addition of templates that introduce semantic content (as described in Section 6. [sent-450, score-0.337]
93 In practice, FUBL learns to model many elliptical constructions with lexemes and templates introduced through maximal factorings. [sent-454, score-0.746]
94 For example, the lexeme (to, [from, to] ) can be used with the correct lexical template to deal with our motivating example “flights Boston to New York”. [sent-455, score-0.544]
95 Table 5 shows a selection of lexemes and templates learned for Atis. [sent-457, score-0.616]
96 However, as these lexemes now share templates, the total number of lexical variants that must be learned is reduced. [sent-459, score-0.462]
97 11 Discussion We argued that factored CCG lexicons, which include both lexemes and lexical templates, provide a compact representation of lexical knowledge that can have advantages for learning. [sent-460, score-0.727]
98 The factored lexical representation also has significant potential for lexical transfer learning, where we would need to learn new lexemes for each target application, but much of the information in the templates could, potentially, be ported across domains. [sent-464, score-0.965]
99 Inducing probabilistic CCG grammars from logical form with higher-order unification. [sent-534, score-0.228]
100 Online learning of relaxed CCG grammars for parsing to logical form. [sent-652, score-0.273]
wordName wordTfidf (topN-words)
[('lexemes', 0.35), ('lexeme', 0.314), ('bos', 0.265), ('ccg', 0.259), ('fubl', 0.246), ('templates', 0.238), ('boston', 0.226), ('factored', 0.209), ('logical', 0.184), ('atis', 0.175), ('template', 0.146), ('flight', 0.142), ('flights', 0.124), ('ubl', 0.116), ('zettlemoyer', 0.114), ('item', 0.108), ('zi', 0.106), ('kwiatkowski', 0.105), ('np', 0.104), ('items', 0.098), ('constants', 0.094), ('lexicon', 0.09), ('factoring', 0.087), ('unedited', 0.087), ('lexical', 0.084), ('elliptical', 0.072), ('fare', 0.072), ('mooney', 0.066), ('lexicons', 0.065), ('cheapest', 0.062), ('combinators', 0.062), ('wong', 0.059), ('xi', 0.058), ('factorings', 0.058), ('geoquery', 0.057), ('kate', 0.057), ('maximal', 0.056), ('raymond', 0.055), ('luke', 0.055), ('steedman', 0.053), ('lambda', 0.052), ('semantic', 0.047), ('parse', 0.045), ('parsing', 0.045), ('grammars', 0.044), ('combinatory', 0.044), ('pccg', 0.043), ('category', 0.043), ('parses', 0.043), ('proposing', 0.039), ('wah', 0.037), ('yuk', 0.037), ('returned', 0.034), ('predicate', 0.034), ('spontaneous', 0.034), ('slash', 0.034), ('reviewed', 0.031), ('goldwasser', 0.031), ('splits', 0.031), ('splitting', 0.031), ('parsers', 0.031), ('paired', 0.031), ('day', 0.03), ('constructions', 0.03), ('returns', 0.03), ('backward', 0.029), ('poon', 0.029), ('learnt', 0.029), ('grammar', 0.029), ('benchmark', 0.029), ('categorial', 0.029), ('ep', 0.029), ('clark', 0.029), ('branavan', 0.029), ('ccgs', 0.029), ('jul', 0.029), ('lecun', 0.029), ('nbops', 0.029), ('nyc', 0.029), ('psla', 0.029), ('recombined', 0.029), ('rohit', 0.029), ('wordspan', 0.029), ('meaning', 0.028), ('learned', 0.028), ('forward', 0.028), ('introduce', 0.027), ('collins', 0.027), ('predicates', 0.027), ('recover', 0.026), ('initialization', 0.026), ('content', 0.025), ('entries', 0.025), ('booking', 0.025), ('pccgs', 0.025), ('ramaswamy', 0.025), ('dcs', 0.025), ('literals', 0.025), ('watkinson', 0.025), ('repeatedly', 0.024)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000002 87 emnlp-2011-Lexical Generalization in CCG Grammar Induction for Semantic Parsing
Author: Tom Kwiatkowski ; Luke Zettlemoyer ; Sharon Goldwater ; Mark Steedman
Abstract: We consider the problem of learning factored probabilistic CCG grammars for semantic parsing from data containing sentences paired with logical-form meaning representations. Traditional CCG lexicons list lexical items that pair words and phrases with syntactic and semantic content. Such lexicons can be inefficient when words appear repeatedly with closely related lexical content. In this paper, we introduce factored lexicons, which include both lexemes to model word meaning and templates to model systematic variation in word usage. We also present an algorithm for learning factored CCG lexicons, along with a probabilistic parse-selection model. Evaluations on benchmark datasets demonstrate that the approach learns highly accurate parsers, whose generalization performance greatly from the lexical factoring. benefits
2 0.23576793 39 emnlp-2011-Discovering Morphological Paradigms from Plain Text Using a Dirichlet Process Mixture Model
Author: Markus Dreyer ; Jason Eisner
Abstract: We present an inference algorithm that organizes observed words (tokens) into structured inflectional paradigms (types). It also naturally predicts the spelling of unobserved forms that are missing from these paradigms, and discovers inflectional principles (grammar) that generalize to wholly unobserved words. Our Bayesian generative model of the data explicitly represents tokens, types, inflections, paradigms, and locally conditioned string edits. It assumes that inflected word tokens are generated from an infinite mixture of inflectional paradigms (string tuples). Each paradigm is sampled all at once from a graphical model, whose potential functions are weighted finitestate transducers with language-specific parameters to be learned. These assumptions naturally lead to an elegant empirical Bayes inference procedure that exploits Monte Carlo EM, belief propagation, and dynamic programming. Given 50–100 seed paradigms, adding a 10million-word corpus reduces prediction error for morphological inflections by up to 10%.
3 0.19335645 10 emnlp-2011-A Probabilistic Forest-to-String Model for Language Generation from Typed Lambda Calculus Expressions
Author: Wei Lu ; Hwee Tou Ng
Abstract: This paper describes a novel probabilistic approach for generating natural language sentences from their underlying semantics in the form of typed lambda calculus. The approach is built on top of a novel reduction-based weighted synchronous context free grammar formalism, which facilitates the transformation process from typed lambda calculus into natural language sentences. Sentences can then be generated based on such grammar rules with a log-linear model. To acquire such grammar rules automatically in an unsupervised manner, we also propose a novel approach with a generative model, which maps from sub-expressions of logical forms to word sequences in natural language sentences. Experiments on benchmark datasets for both English and Chinese generation tasks yield significant improvements over results obtained by two state-of-the-art machine translation models, in terms of both automatic metrics and human evaluation.
4 0.17111504 121 emnlp-2011-Semi-supervised CCG Lexicon Extension
Author: Emily Thomforde ; Mark Steedman
Abstract: This paper introduces Chart Inference (CI), an algorithm for deriving a CCG category for an unknown word from a partial parse chart. It is shown to be faster and more precise than a baseline brute-force method, and to achieve wider coverage than a rule-based system. In addition, we show the application of CI to a domain adaptation task for question words, which are largely missing in the Penn Treebank. When used in combination with self-training, CI increases the precision of the baseline StatCCG parser over subjectextraction questions by 50%. An error analysis shows that CI contributes to the increase by expanding the number of category types available to the parser, while self-training adjusts the counts.
5 0.16057807 24 emnlp-2011-Bootstrapping Semantic Parsers from Conversations
Author: Yoav Artzi ; Luke Zettlemoyer
Abstract: Conversations provide rich opportunities for interactive, continuous learning. When something goes wrong, a system can ask for clarification, rewording, or otherwise redirect the interaction to achieve its goals. In this paper, we present an approach for using conversational interactions of this type to induce semantic parsers. We demonstrate learning without any explicit annotation of the meanings of user utterances. Instead, we model meaning with latent variables, and introduce a loss function to measure how well potential meanings match the conversation. This loss drives the overall learning approach, which induces a weighted CCG grammar that could be used to automatically bootstrap the semantic analysis component in a complete dialog system. Experiments on DARPA Communicator conversational logs demonstrate effective learning, despite requiring no explicit mean- . ing annotations.
6 0.12278558 132 emnlp-2011-Syntax-Based Grammaticality Improvement using CCG and Guided Search
7 0.09105207 129 emnlp-2011-Structured Sparsity in Structured Prediction
8 0.07755474 103 emnlp-2011-Parser Evaluation over Local and Non-Local Deep Dependencies in a Large Corpus
9 0.073869914 32 emnlp-2011-Computing Logical Form on Regulatory Texts
10 0.068532102 53 emnlp-2011-Experimental Support for a Categorical Compositional Distributional Model of Meaning
11 0.067155741 111 emnlp-2011-Reducing Grounded Learning Tasks To Grammatical Inference
12 0.061199509 144 emnlp-2011-Unsupervised Learning of Selectional Restrictions and Detection of Argument Coercions
13 0.056040123 54 emnlp-2011-Exploiting Parse Structures for Native Language Identification
14 0.051258799 8 emnlp-2011-A Model of Discourse Predictions in Human Sentence Processing
15 0.048285007 143 emnlp-2011-Unsupervised Information Extraction with Distributional Prior Knowledge
16 0.048029624 60 emnlp-2011-Feature-Rich Language-Independent Syntax-Based Alignment for Statistical Machine Translation
17 0.047406778 16 emnlp-2011-Accurate Parsing with Compact Tree-Substitution Grammars: Double-DOP
18 0.047260635 83 emnlp-2011-Learning Sentential Paraphrases from Bilingual Parallel Corpora for Text-to-Text Generation
19 0.045760237 4 emnlp-2011-A Fast, Accurate, Non-Projective, Semantically-Enriched Parser
20 0.044965632 20 emnlp-2011-Augmenting String-to-Tree Translation Models with Fuzzy Use of Source-side Syntax
topicId topicWeight
[(0, 0.198), (1, 0.011), (2, -0.042), (3, 0.028), (4, 0.066), (5, -0.024), (6, -0.193), (7, 0.051), (8, 0.108), (9, -0.008), (10, -0.228), (11, 0.113), (12, -0.363), (13, -0.272), (14, -0.079), (15, -0.011), (16, -0.046), (17, -0.172), (18, 0.034), (19, -0.064), (20, -0.223), (21, -0.105), (22, 0.107), (23, -0.004), (24, -0.096), (25, -0.092), (26, -0.004), (27, -0.059), (28, -0.031), (29, 0.017), (30, -0.029), (31, -0.101), (32, -0.069), (33, -0.044), (34, 0.068), (35, -0.074), (36, -0.007), (37, -0.012), (38, -0.064), (39, 0.123), (40, -0.002), (41, 0.045), (42, -0.047), (43, -0.07), (44, 0.06), (45, -0.017), (46, 0.05), (47, 0.074), (48, -0.119), (49, -0.05)]
simIndex simValue paperId paperTitle
same-paper 1 0.94676763 87 emnlp-2011-Lexical Generalization in CCG Grammar Induction for Semantic Parsing
Author: Tom Kwiatkowski ; Luke Zettlemoyer ; Sharon Goldwater ; Mark Steedman
Abstract: We consider the problem of learning factored probabilistic CCG grammars for semantic parsing from data containing sentences paired with logical-form meaning representations. Traditional CCG lexicons list lexical items that pair words and phrases with syntactic and semantic content. Such lexicons can be inefficient when words appear repeatedly with closely related lexical content. In this paper, we introduce factored lexicons, which include both lexemes to model word meaning and templates to model systematic variation in word usage. We also present an algorithm for learning factored CCG lexicons, along with a probabilistic parse-selection model. Evaluations on benchmark datasets demonstrate that the approach learns highly accurate parsers, whose generalization performance greatly from the lexical factoring. benefits
2 0.61285049 39 emnlp-2011-Discovering Morphological Paradigms from Plain Text Using a Dirichlet Process Mixture Model
Author: Markus Dreyer ; Jason Eisner
Abstract: We present an inference algorithm that organizes observed words (tokens) into structured inflectional paradigms (types). It also naturally predicts the spelling of unobserved forms that are missing from these paradigms, and discovers inflectional principles (grammar) that generalize to wholly unobserved words. Our Bayesian generative model of the data explicitly represents tokens, types, inflections, paradigms, and locally conditioned string edits. It assumes that inflected word tokens are generated from an infinite mixture of inflectional paradigms (string tuples). Each paradigm is sampled all at once from a graphical model, whose potential functions are weighted finitestate transducers with language-specific parameters to be learned. These assumptions naturally lead to an elegant empirical Bayes inference procedure that exploits Monte Carlo EM, belief propagation, and dynamic programming. Given 50–100 seed paradigms, adding a 10million-word corpus reduces prediction error for morphological inflections by up to 10%.
3 0.54540324 121 emnlp-2011-Semi-supervised CCG Lexicon Extension
Author: Emily Thomforde ; Mark Steedman
Abstract: This paper introduces Chart Inference (CI), an algorithm for deriving a CCG category for an unknown word from a partial parse chart. It is shown to be faster and more precise than a baseline brute-force method, and to achieve wider coverage than a rule-based system. In addition, we show the application of CI to a domain adaptation task for question words, which are largely missing in the Penn Treebank. When used in combination with self-training, CI increases the precision of the baseline StatCCG parser over subjectextraction questions by 50%. An error analysis shows that CI contributes to the increase by expanding the number of category types available to the parser, while self-training adjusts the counts.
4 0.53176302 24 emnlp-2011-Bootstrapping Semantic Parsers from Conversations
Author: Yoav Artzi ; Luke Zettlemoyer
Abstract: Conversations provide rich opportunities for interactive, continuous learning. When something goes wrong, a system can ask for clarification, rewording, or otherwise redirect the interaction to achieve its goals. In this paper, we present an approach for using conversational interactions of this type to induce semantic parsers. We demonstrate learning without any explicit annotation of the meanings of user utterances. Instead, we model meaning with latent variables, and introduce a loss function to measure how well potential meanings match the conversation. This loss drives the overall learning approach, which induces a weighted CCG grammar that could be used to automatically bootstrap the semantic analysis component in a complete dialog system. Experiments on DARPA Communicator conversational logs demonstrate effective learning, despite requiring no explicit mean- . ing annotations.
5 0.46169943 10 emnlp-2011-A Probabilistic Forest-to-String Model for Language Generation from Typed Lambda Calculus Expressions
Author: Wei Lu ; Hwee Tou Ng
Abstract: This paper describes a novel probabilistic approach for generating natural language sentences from their underlying semantics in the form of typed lambda calculus. The approach is built on top of a novel reduction-based weighted synchronous context free grammar formalism, which facilitates the transformation process from typed lambda calculus into natural language sentences. Sentences can then be generated based on such grammar rules with a log-linear model. To acquire such grammar rules automatically in an unsupervised manner, we also propose a novel approach with a generative model, which maps from sub-expressions of logical forms to word sequences in natural language sentences. Experiments on benchmark datasets for both English and Chinese generation tasks yield significant improvements over results obtained by two state-of-the-art machine translation models, in terms of both automatic metrics and human evaluation.
6 0.42168272 32 emnlp-2011-Computing Logical Form on Regulatory Texts
7 0.41284442 132 emnlp-2011-Syntax-Based Grammaticality Improvement using CCG and Guided Search
8 0.31865838 111 emnlp-2011-Reducing Grounded Learning Tasks To Grammatical Inference
9 0.3110289 143 emnlp-2011-Unsupervised Information Extraction with Distributional Prior Knowledge
10 0.30480912 103 emnlp-2011-Parser Evaluation over Local and Non-Local Deep Dependencies in a Large Corpus
11 0.30016735 129 emnlp-2011-Structured Sparsity in Structured Prediction
12 0.26518714 144 emnlp-2011-Unsupervised Learning of Selectional Restrictions and Detection of Argument Coercions
13 0.22862926 53 emnlp-2011-Experimental Support for a Categorical Compositional Distributional Model of Meaning
14 0.22786921 8 emnlp-2011-A Model of Discourse Predictions in Human Sentence Processing
15 0.20719637 81 emnlp-2011-Learning General Connotation of Words using Graph-based Algorithms
16 0.2057457 54 emnlp-2011-Exploiting Parse Structures for Native Language Identification
17 0.19790721 45 emnlp-2011-Dual Decomposition with Many Overlapping Components
18 0.1889089 96 emnlp-2011-Multilayer Sequence Labeling
19 0.18599893 85 emnlp-2011-Learning to Simplify Sentences with Quasi-Synchronous Grammar and Integer Programming
20 0.18472536 137 emnlp-2011-Training dependency parsers by jointly optimizing multiple objectives
topicId topicWeight
[(15, 0.012), (23, 0.087), (36, 0.016), (37, 0.023), (45, 0.063), (53, 0.016), (54, 0.075), (57, 0.016), (62, 0.017), (64, 0.027), (66, 0.034), (69, 0.016), (78, 0.263), (79, 0.1), (82, 0.048), (87, 0.042), (90, 0.018), (96, 0.045), (98, 0.015)]
simIndex simValue paperId paperTitle
same-paper 1 0.75002813 87 emnlp-2011-Lexical Generalization in CCG Grammar Induction for Semantic Parsing
Author: Tom Kwiatkowski ; Luke Zettlemoyer ; Sharon Goldwater ; Mark Steedman
Abstract: We consider the problem of learning factored probabilistic CCG grammars for semantic parsing from data containing sentences paired with logical-form meaning representations. Traditional CCG lexicons list lexical items that pair words and phrases with syntactic and semantic content. Such lexicons can be inefficient when words appear repeatedly with closely related lexical content. In this paper, we introduce factored lexicons, which include both lexemes to model word meaning and templates to model systematic variation in word usage. We also present an algorithm for learning factored CCG lexicons, along with a probabilistic parse-selection model. Evaluations on benchmark datasets demonstrate that the approach learns highly accurate parsers, whose generalization performance greatly from the lexical factoring. benefits
2 0.70384866 65 emnlp-2011-Heuristic Search for Non-Bottom-Up Tree Structure Prediction
Author: Andrea Gesmundo ; James Henderson
Abstract: State of the art Tree Structures Prediction techniques rely on bottom-up decoding. These approaches allow the use of context-free features and bottom-up features. We discuss the limitations of mainstream techniques in solving common Natural Language Processing tasks. Then we devise a new framework that goes beyond Bottom-up Decoding, and that allows a better integration of contextual features. Furthermore we design a system that addresses these issues and we test it on Hierarchical Machine Translation, a well known tree structure prediction problem. The structure of the proposed system allows the incorporation of non-bottom-up features and relies on a more sophisticated decoding approach. We show that the proposed approach can find bet- ter translations using a smaller portion of the search space.
3 0.50347245 34 emnlp-2011-Corpus-Guided Sentence Generation of Natural Images
Author: Yezhou Yang ; Ching Teo ; Hal Daume III ; Yiannis Aloimonos
Abstract: We propose a sentence generation strategy that describes images by predicting the most likely nouns, verbs, scenes and prepositions that make up the core sentence structure. The input are initial noisy estimates of the objects and scenes detected in the image using state of the art trained detectors. As predicting actions from still images directly is unreliable, we use a language model trained from the English Gigaword corpus to obtain their estimates; together with probabilities of co-located nouns, scenes and prepositions. We use these estimates as parameters on a HMM that models the sentence generation process, with hidden nodes as sentence components and image detections as the emissions. Experimental results show that our strategy of combining vision and language produces readable and de- , scriptive sentences compared to naive strategies that use vision alone.
4 0.50258243 36 emnlp-2011-Corroborating Text Evaluation Results with Heterogeneous Measures
Author: Enrique Amigo ; Julio Gonzalo ; Jesus Gimenez ; Felisa Verdejo
Abstract: Automatically produced texts (e.g. translations or summaries) are usually evaluated with n-gram based measures such as BLEU or ROUGE, while the wide set of more sophisticated measures that have been proposed in the last years remains largely ignored for practical purposes. In this paper we first present an indepth analysis of the state of the art in order to clarify this issue. After this, we formalize and verify empirically a set of properties that every text evaluation measure based on similarity to human-produced references satisfies. These properties imply that corroborating system improvements with additional measures always increases the overall reliability of the evaluation process. In addition, the greater the heterogeneity of the measures (which is measurable) the higher their combined reliability. These results support the use of heterogeneous measures in order to consolidate text evaluation results.
5 0.50070024 111 emnlp-2011-Reducing Grounded Learning Tasks To Grammatical Inference
Author: Benjamin Borschinger ; Bevan K. Jones ; Mark Johnson
Abstract: It is often assumed that ‘grounded’ learning tasks are beyond the scope of grammatical inference techniques. In this paper, we show that the grounded task of learning a semantic parser from ambiguous training data as discussed in Kim and Mooney (2010) can be reduced to a Probabilistic Context-Free Grammar learning task in a way that gives state of the art results. We further show that additionally letting our model learn the language’s canonical word order improves its performance and leads to the highest semantic parsing f-scores previously reported in the literature.1
6 0.49970347 67 emnlp-2011-Hierarchical Verb Clustering Using Graph Factorization
7 0.49827984 22 emnlp-2011-Better Evaluation Metrics Lead to Better Machine Translation
8 0.49772519 53 emnlp-2011-Experimental Support for a Categorical Compositional Distributional Model of Meaning
9 0.49601445 107 emnlp-2011-Probabilistic models of similarity in syntactic context
10 0.493783 8 emnlp-2011-A Model of Discourse Predictions in Human Sentence Processing
11 0.48410338 123 emnlp-2011-Soft Dependency Constraints for Reordering in Hierarchical Phrase-Based Translation
12 0.48396352 35 emnlp-2011-Correcting Semantic Collocation Errors with L1-induced Paraphrases
13 0.48379368 10 emnlp-2011-A Probabilistic Forest-to-String Model for Language Generation from Typed Lambda Calculus Expressions
14 0.4831982 20 emnlp-2011-Augmenting String-to-Tree Translation Models with Fuzzy Use of Source-side Syntax
15 0.48150191 108 emnlp-2011-Quasi-Synchronous Phrase Dependency Grammars for Machine Translation
16 0.48086205 83 emnlp-2011-Learning Sentential Paraphrases from Bilingual Parallel Corpora for Text-to-Text Generation
17 0.48060435 38 emnlp-2011-Data-Driven Response Generation in Social Media
18 0.48028001 115 emnlp-2011-Relaxed Cross-lingual Projection of Constituent Syntax
19 0.47963405 136 emnlp-2011-Training a Parser for Machine Translation Reordering
20 0.47887614 85 emnlp-2011-Learning to Simplify Sentences with Quasi-Synchronous Grammar and Integer Programming