acl acl2011 acl2011-79 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Dan Goldwasser ; Roi Reichart ; James Clarke ; Dan Roth
Abstract: Current approaches for semantic parsing take a supervised approach requiring a considerable amount of training data which is expensive and difficult to obtain. This supervision bottleneck is one of the major difficulties in scaling up semantic parsing. We argue that a semantic parser can be trained effectively without annotated data, and introduce an unsupervised learning algorithm. The algorithm takes a self training approach driven by confidence estimation. Evaluated over Geoquery, a standard dataset for this task, our system achieved 66% accuracy, compared to 80% of its fully supervised counterpart, demonstrating the promise of unsupervised approaches for this task.
Reference: text
sentIndex sentText sentNum sentScore
1 edu , , Abstract Current approaches for semantic parsing take a supervised approach requiring a considerable amount of training data which is expensive and difficult to obtain. [sent-4, score-0.254]
2 We argue that a semantic parser can be trained effectively without annotated data, and introduce an unsupervised learning algorithm. [sent-6, score-0.395]
3 The algorithm takes a self training approach driven by confidence estimation. [sent-7, score-0.834]
4 The term semantic parsing has been used ambiguously to refer to several semantic tasks (e. [sent-11, score-0.362]
5 Unlike shallow semantic analysis tasks, the output ofa semantic parser is complete and unambiguous to the extent it can be understood or even executed by a computer system. [sent-15, score-0.331]
6 Our model compensates for the lack of training data by employing a self training protocol based on identifying high confidence self labeled examples and using them to retrain the model. [sent-21, score-1.024]
7 We base our approach on a simple observation: semantic parsing is a difficult structured prediction task, which requires learning a complex model, however identifying good predictions can be done with a far simpler model capturing repeating patterns in the predicted data. [sent-22, score-0.77]
8 We present several simple, yet highly effective confidence measures capturing such patterns, and show how to use them to train a semantic parser without manually annotated sentences. [sent-23, score-0.786]
9 Our basic premise, that predictions with high confidence score are of high quality, is further used to improve the performance of the unsupervised trainProceedingPso orftla thned 4,9 Otrhe Agonnn,u Jauln Mee 1e9t-i2ng4, o 2f0 t1h1e. [sent-24, score-0.928]
10 Our learning algorithm takes an EMlike iterative approach, in which the predictions of the previous stage are used to bias the model. [sent-27, score-0.266]
11 We show that by using confidence estimation as a proxy for the model’s prediction quality, the learning algorithm can identify a better model compared to the default convergence criterion. [sent-29, score-0.962]
12 Our experimental results show that using our approach we are able to train a good semantic parser without annotated data, and that using a confidence score to identify good models results in a significant performance improve- ment. [sent-33, score-0.759]
13 2 Semantic Parsing We formulate semantic parsing as a structured prediction problem, mapping a NL input sentence (denoted x), to its highest ranking MR (denoted z). [sent-34, score-0.544]
14 The prediction function, mapping a sentence to its corresponding MR, is formalized as follows: z = Fw(x) = ayr∈gY,mz∈aZxwTΦ(x,y,z) (1) Where Φ is a feature function defined over an input sentence x, alignment y and output z. [sent-44, score-0.334]
15 y z count( state( traverse( river( const(colorado)))) Figure 1: Example of an input sentence (x), meaning representation (z) and the alignment between the two (y) for the Geoquery domain ference problem based on Φ and w is what compromises our semantic parser. [sent-48, score-0.268]
16 In practice the parsing decision is decomposed into smaller decisions (Sec. [sent-49, score-0.294]
17 1 Target Meaning Representation The output of the semantic parser is a logical formula, grounding the semantics of the input sentence in the domain language (i. [sent-58, score-0.442]
18 In practice this decision is decomposed into smaller decisions, capturing local mapping of input tokens to logical fragments and their composition into larger fragments. [sent-73, score-0.568]
19 The first type of decisions are encoded directly by the alignment (y) between the input tokens and their corresponding predicates. [sent-76, score-0.225]
20 1: river ( const ( colorado ) ) is a composition of two predicates river and const ( colorado ) . [sent-82, score-0.567]
21 We denote by si the i-th output predicate composition in z, by si−1 (si) the composition ofthe (i− 1)th predicate on the i-th) predicate asnitdio by y(si) (tih−e i1n)-put word corresponding to that predicate according to the alignment y. [sent-87, score-0.557]
22 3 Unsupervised Semantic Parsing Our learning framework takes a self training ap- proach in which the learner is iteratively trained over its own predictions. [sent-88, score-0.264]
23 Successful application of this approach depends heavily on two important factors - how to select high quality examples to train the model on, and how to define the learning objective so that learning can halt once a good model is found. [sent-89, score-0.406]
24 In this work we suggest to address both of the above concerns by approximating the quality of the model’s predictions using a confidence measure computed over the statistics of the self generated predictions. [sent-91, score-0.89]
25 Output structures which fall close to the center of mass of these statistics will receive a high confidence score. [sent-92, score-0.687]
26 The first issue is addressed by using examples assigned a high confidence score to train the model, acting as labeled examples. [sent-93, score-0.726]
27 We also note that since the confidence score pro1488 vides a good indication for the model’s prediction performance, it can be used to approximate the overall model performance, by observing the model’s total confidence score over all its predictions. [sent-94, score-1.403]
28 This allows us to set a performance driven goal for our learning process - return the model maximizing the confidence score over all predictions. [sent-95, score-0.82]
29 We describe the details of integrating the confidence score into the learning framework in Sec. [sent-96, score-0.706]
30 , wTΦ(x, y, z)) as an indication of correctness is a natural choice, we argue and show empirically, that unsupervised learning driven by confidence estimation results in a better performing model. [sent-101, score-1.066]
31 The statistics used for confidence estimation are different than those used by the model to create the output structures, and can therefore capture additional information unobserved by the prediction model. [sent-105, score-0.928]
32 The success of our learning procedure hinges on finding good confidence measures, whose confidence prediction correlates well with the true quality of the prediction. [sent-108, score-1.302]
33 The ability of unsupervised confidence estimation to provide high quality confidence predictions can be explained by the observation that prominent prediction patterns are more likely to be correct. [sent-109, score-1.677]
34 Our specific choice of confidence measures is guided by the intuition that unlike structure prediction (i. [sent-111, score-0.698]
35 , solving the inference problem) which requires taking statistics over complex and intricate patterns, identifying high quality predictions can be done using much simpler patterns that are significantly easier to capture. [sent-113, score-0.408]
36 1), we then explain the rational behind confidence estimation over self-generated data and introduce the confidence measures used in our experiments (Sec. [sent-120, score-1.229]
37 1 Unsupervised Confidence-Driven Learning Our learning framework works in an EM-like manner, iterating between two stages: making predictions based on its current set of parameters and then retraining the model using a subset of the predictions, assigned high confidence. [sent-127, score-0.324]
38 The learning process “discovers” new high confidence training examples to add to its training set over multiple iterations, and converges when the model no longer adds new training examples. [sent-128, score-0.819]
39 We follow the observation that confidence estimation can be used to approximate the performance of the entire model and return the model with the highest overall prediction confidence. [sent-132, score-0.867]
40 The set of top confidence examples (for either correct or incorrect prediction), at iteration iof the algorithm, is denoted The exact nature of the confidence computation is discussed in Sec. [sent-141, score-1.204]
41 The algorithm iterates between these two stages, at each iteration it adds more self-annotated examples to its training set, learning therefore converges when no new examples are added (line 11). [sent-144, score-0.319]
42 The algorithm keeps track of the models it trained at each stage throughout this process, and returns the one with the highest averaged overall confidence score Scionf. [sent-145, score-0.635]
43 At each stage, the overall confidence score is computed by averaging over all the confidence scores of the predictions made at that stage. [sent-147, score-1.262]
44 2 Unsupervised Confidence Estimation Confidence estimation is calculated over a batch of input (x) - output (z) pairs. [sent-149, score-0.236]
45 Confidence estimation is done by computing the statistics of these decisions, over the entire set of predicted structures. [sent-153, score-0.238]
46 In the rest of this section we introduce the confidence measures used by our system. [sent-154, score-0.58]
47 This can be considered as an abstraction of the prediction model: we collapse the intricate feature representation into 1Since we commit to the max-score output prediction, rather than summing over all possibilities, we require a reasonable initialization point. [sent-156, score-0.26]
48 high level decisions and take statistics over these decisions. [sent-159, score-0.212]
49 Since it takes statistics over considerably less variables than the actual prediction model, we expect this model to make reliable confidence predictions. [sent-160, score-0.798]
50 We consider two variations of this approach, the first constructs a unigram model over the first order decisions and the second a bigram model over the second order decisions. [sent-161, score-0.317]
51 Given a set of structure predictions S, we com- pute this proportion for each structure (denoted as Prop(x, z)) and calculate the average proportion over the entire set (denoted as AvProp(S)). [sent-164, score-0.292]
52 The algorithm decomposes each predicted formula and its corresponding input sentence into a feature vector Φ(x, y, z) normalized by the size of the input sentence |x|, and assigns a binary label to this vector2. [sent-174, score-0.404]
53 1, and use the confidence score to select the top ranking examples as positive examples, and the bottom ranking examples as negative examples. [sent-177, score-0.861]
54 The second is a structured learning algorithm which considers learning as a ranking problem, i. [sent-179, score-0.276]
55 The structured learning algorithm can directly use the top ranking predictions of the model (line 8 in Alg. [sent-182, score-0.389]
56 1 is an inference procedure selecting the top ranked output logical formula. [sent-187, score-0.303]
57 1) is decomposed into smaller decisions, capturing mapping of input to- kens to logical fragments (first order) and their composition into larger fragments (second order). [sent-195, score-0.562]
58 We encode a first-order decision as αcs, a binary variable indicating that constituent c is aligned with the logical symbol s . [sent-196, score-0.321]
59 A second-order decision βcs,dt, is encoded as a binary variable indicating that the symbol t (associated with constituent d) is an argument of a function s (associated with constituent c). [sent-197, score-0.213]
60 First-order decision features Φ1 Determining if a logical symbol is aligned with a specific constituent depends mostly on lexical information. [sent-206, score-0.266]
61 , (Zettlemoyer and Collins, 2005)) we create a small lexicon, mapping logical symbols to surface forms. [sent-209, score-0.213]
62 , 1990) and add features which measure the lexical similarity between a constituent and a logical symbol’s surface forms (as defined by the lexicon). [sent-212, score-0.212]
63 We compare several confidence measures and analyze their properties. [sent-222, score-0.58]
64 Our learning system had access only to the NL questions, and the logical forms were only used to evaluate the system’s performance. [sent-234, score-0.225]
65 SCORE: An unsupervised framework using the model’s internal prediction score (wTΦ(x, y, z)) for confidence estimation. [sent-243, score-0.882]
66 The reported score was obtained by selecting the model at the training iteration with the highest overall confidence score (see line 12 in Alg. [sent-247, score-0.803]
67 While our approach is based on assessing the correctness os the model’s predictions according to unsupervised confidence estimation, their framework is provided with external supervision for these decisions, indicating if the predicted structures are correct. [sent-251, score-1.015]
68 We compare three self-trained systems, ALL EXAMPLES, PREDICTIONSCORE and COMBINED, which differ 4While unsupervised learning for various semantic tasks has been widely discussed, this is the first attempt to tackle this task. [sent-261, score-0.306]
69 1492 in their sample selection strategy, but all use confidence estimation for selecting the final semantic parsing model. [sent-264, score-0.92]
70 These results show that training the model with training examples selected carefully will improve learning - as the best performance is achieved with perfect knowledge of the predictions correctness (RESPONSE BASED). [sent-274, score-0.331]
71 The low performance of the PREDICTIONSCORE model is also not surprising, and it demonstrates one of the key principles in confidence estimation - the score should be comparable across predictions done over different inputs, and not the same input, as done in PREDICTIONSCORE model. [sent-277, score-0.891]
72 (2) How does confidence driven sample selection contribute to the learning process? [sent-278, score-0.742]
73 Comparing the systems driven by confidence sample-selection to the ALL EXAMPLES approach uncovers an interesting tradeoff between training with more (noisy) data and selectively training the system with higher quality examples. [sent-279, score-0.69]
74 We argue that carefully selecting high quality training examples will result in better performance. [sent-280, score-0.264]
75 The empirical results indeed support our argument, as the best performing model (RESPONSE BASED) is achieved by sample selection with perfect knowledge of prediction correctness. [sent-281, score-0.243]
76 We argue that different confidence measures capture different properties of the data, and hypothesize that combining their scores will improve the resulting model. [sent-284, score-0.627]
77 Results show that using the COMBINED measure leads to an improved performance, better than any of the individual measures, suggesting that it can effectively exploit the properties ofeach confidence measure. [sent-288, score-0.535]
78 (3) Can confidence measures serve as a good proxy for the model’s performance? [sent-290, score-0.624]
79 We argue that by selecting the model that maximizes the averaged confidence score, a better model can be found. [sent-292, score-0.723]
80 We compare the performance of the model selected using the confidence score to the performance of the final model considered by the learning algorithm (see Sec. [sent-295, score-0.8]
81 Since these experiments required running the learning algorithm many times, we focused on the binary learning algorithm as it converges considerably faster. [sent-299, score-0.299]
82 SCORE and PROPORTION models we used both their confidence prediction, and the simple UNIGRAM confidence score to evaluate model performance (the latter appear in parentheses in Tab. [sent-304, score-1.184]
83 Results show that the over overall confidence score serves as a reliable proxy for the model performance - using UNIGRAM and BIGRAM the framework can select the best performing model, far better than the performance of the default model to which the system converged. [sent-306, score-0.824]
84 273482 Table 4: Using confidence to approximate model performance. [sent-327, score-0.585]
85 We compare the best result obtained in any of the learning algorithm iterations (Best), the result obtained by approximating the best result using the averaged prediction confidence (Conf. [sent-328, score-0.754]
86 Results in parentheses are the result of using the UNIGRAM confidence to approximate the model’s performance. [sent-331, score-0.535]
87 Other approaches to unsupervised semantic analysis (Poon and Domingos, 2009; Titov and Klementiev, 2011) take a different approach to semantic representation, by clustering semantically equivalent dependency tree fragments, and identifying their predicate-argument structure. [sent-344, score-0.359]
88 While these approaches have been applied successfully to semantic tasks such as question answering, they do not ground the input in a well defined output language, an essential component in our task. [sent-345, score-0.24]
89 Our unsupervised approach follows a self training protocol (Yarowsky, 1995; McClosky et al. [sent-346, score-0.315]
90 We estimate the quality of the predictions and use only high confidence examples for training. [sent-351, score-0.839]
91 In addition to sample selection we use confidence estimation as a way to approximate the overall quality of the model and use it for model selection. [sent-358, score-0.834]
92 This use of confidence estimation was explored in (Reichart et al. [sent-359, score-0.649]
93 In this work we integrate this estimation deeper into the learning process, thus allowing our training procedure to return the best performing model. [sent-361, score-0.218]
94 7 Conclusions We introduced an unsupervised learning algorithm for semantic parsing, the first for this task to the best of our knowledge. [sent-362, score-0.342]
95 To compensate for the lack of training data we use a self-training protocol, driven by unsupervised confidence estimation. [sent-363, score-0.764]
96 We demonstrate empirically that our approach results in a high preforming semantic parser and show that confidence estimation plays a vital role in this success, both by identifying good training examples as well as identifying good over all performance, used to improve the final model selection. [sent-364, score-0.986]
97 In future work we hope to further improve unsupervised semantic parsing performance. [sent-365, score-0.317]
98 Particularly, we intend to explore new approaches for confidence estimation and their usage in the unsupervised and semi-supervised versions of the task. [sent-366, score-0.772]
99 Speech recognition and utterance verification based on a generalized confidence score. [sent-460, score-0.535]
100 Online learning of relaxed CCG grammars for parsing to logical form. [sent-620, score-0.301]
wordName wordTfidf (topN-words)
[('confidence', 0.535), ('zettlemoyer', 0.165), ('logical', 0.16), ('nl', 0.152), ('mooney', 0.149), ('geoquery', 0.148), ('clarke', 0.135), ('predictions', 0.128), ('unsupervised', 0.123), ('si', 0.121), ('reichart', 0.12), ('self', 0.12), ('prediction', 0.118), ('semantic', 0.118), ('decisions', 0.115), ('estimation', 0.114), ('zelle', 0.109), ('driven', 0.106), ('const', 0.1), ('mr', 0.099), ('predictionscore', 0.099), ('composition', 0.089), ('examples', 0.088), ('proportion', 0.082), ('parsing', 0.076), ('river', 0.072), ('protocol', 0.072), ('tang', 0.072), ('input', 0.069), ('structured', 0.067), ('supervision', 0.066), ('predicted', 0.066), ('learning', 0.065), ('score', 0.064), ('collins', 0.063), ('supervised', 0.06), ('statistics', 0.058), ('combined', 0.058), ('titov', 0.057), ('goldwasser', 0.057), ('decomposes', 0.057), ('wt', 0.056), ('structures', 0.055), ('binary', 0.055), ('decision', 0.054), ('bigram', 0.054), ('output', 0.053), ('mapping', 0.053), ('constituent', 0.052), ('formula', 0.052), ('wong', 0.051), ('model', 0.05), ('refer', 0.05), ('avprop', 0.049), ('initialmodel', 0.049), ('intricate', 0.049), ('prolog', 0.049), ('scionf', 0.049), ('line', 0.049), ('inference', 0.049), ('quality', 0.049), ('decomposed', 0.049), ('unigram', 0.048), ('xl', 0.048), ('fragments', 0.048), ('response', 0.048), ('logic', 0.047), ('argue', 0.047), ('predicates', 0.046), ('blum', 0.046), ('denoted', 0.046), ('capturing', 0.046), ('measures', 0.045), ('proxy', 0.044), ('reader', 0.044), ('colorado', 0.044), ('ranking', 0.043), ('chang', 0.043), ('parser', 0.042), ('framework', 0.042), ('converges', 0.042), ('inductive', 0.042), ('alignment', 0.041), ('selecting', 0.041), ('predicate', 0.041), ('representation', 0.04), ('prop', 0.04), ('caruana', 0.04), ('roth', 0.04), ('performing', 0.039), ('high', 0.039), ('geography', 0.038), ('ueffing', 0.038), ('takes', 0.037), ('indication', 0.037), ('selection', 0.036), ('algorithm', 0.036), ('patterns', 0.036), ('kate', 0.036), ('branavan', 0.036)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000001 79 acl-2011-Confidence Driven Unsupervised Semantic Parsing
Author: Dan Goldwasser ; Roi Reichart ; James Clarke ; Dan Roth
Abstract: Current approaches for semantic parsing take a supervised approach requiring a considerable amount of training data which is expensive and difficult to obtain. This supervision bottleneck is one of the major difficulties in scaling up semantic parsing. We argue that a semantic parser can be trained effectively without annotated data, and introduce an unsupervised learning algorithm. The algorithm takes a self training approach driven by confidence estimation. Evaluated over Geoquery, a standard dataset for this task, our system achieved 66% accuracy, compared to 80% of its fully supervised counterpart, demonstrating the promise of unsupervised approaches for this task.
2 0.19393864 3 acl-2011-A Bayesian Model for Unsupervised Semantic Parsing
Author: Ivan Titov ; Alexandre Klementiev
Abstract: We propose a non-parametric Bayesian model for unsupervised semantic parsing. Following Poon and Domingos (2009), we consider a semantic parsing setting where the goal is to (1) decompose the syntactic dependency tree of a sentence into fragments, (2) assign each of these fragments to a cluster of semantically equivalent syntactic structures, and (3) predict predicate-argument relations between the fragments. We use hierarchical PitmanYor processes to model statistical dependencies between meaning representations of predicates and those of their arguments, as well as the clusters of their syntactic realizations. We develop a modification of the MetropolisHastings split-merge sampler, resulting in an efficient inference algorithm for the model. The method is experimentally evaluated by us- ing the induced semantic representation for the question answering task in the biomedical domain.
3 0.18409163 146 acl-2011-Goodness: A Method for Measuring Machine Translation Confidence
Author: Nguyen Bach ; Fei Huang ; Yaser Al-Onaizan
Abstract: State-of-the-art statistical machine translation (MT) systems have made significant progress towards producing user-acceptable translation output. However, there is still no efficient way for MT systems to inform users which words are likely translated correctly and how confident it is about the whole sentence. We propose a novel framework to predict wordlevel and sentence-level MT errors with a large number of novel features. Experimental results show that the MT error prediction accuracy is increased from 69.1 to 72.2 in F-score. The Pearson correlation between the proposed confidence measure and the human-targeted translation edit rate (HTER) is 0.6. Improve- ments between 0.4 and 0.9 TER reduction are obtained with the n-best list reranking task using the proposed confidence measure. Also, we present a visualization prototype of MT errors at the word and sentence levels with the objective to improve post-editor productivity.
4 0.1639501 200 acl-2011-Learning Dependency-Based Compositional Semantics
Author: Percy Liang ; Michael Jordan ; Dan Klein
Abstract: Compositional question answering begins by mapping questions to logical forms, but training a semantic parser to perform this mapping typically requires the costly annotation of the target logical forms. In this paper, we learn to map questions to answers via latent logical forms, which are induced automatically from question-answer pairs. In tackling this challenging learning problem, we introduce a new semantic representation which highlights a parallel between dependency syntax and efficient evaluation of logical forms. On two standard semantic parsing benchmarks (GEO and JOBS), our system obtains the highest published accuracies, despite requiring no annotated logical forms.
5 0.12958959 78 acl-2011-Confidence-Weighted Learning of Factored Discriminative Language Models
Author: Viet Ha Thuc ; Nicola Cancedda
Abstract: Language models based on word surface forms only are unable to benefit from available linguistic knowledge, and tend to suffer from poor estimates for rare features. We propose an approach to overcome these two limitations. We use factored features that can flexibly capture linguistic regularities, and we adopt confidence-weighted learning, a form of discriminative online learning that can better take advantage of a heavy tail of rare features. Finally, we extend the confidence-weighted learning to deal with label noise in training data, a common case with discriminative lan- guage modeling.
6 0.096068092 167 acl-2011-Improving Dependency Parsing with Semantic Classes
7 0.095686577 170 acl-2011-In-domain Relation Discovery with Meta-constraints via Posterior Regularization
8 0.094352692 25 acl-2011-A Simple Measure to Assess Non-response
9 0.090222768 190 acl-2011-Knowledge-Based Weak Supervision for Information Extraction of Overlapping Relations
10 0.08976052 81 acl-2011-Consistent Translation using Discriminative Learning - A Translation Memory-inspired Approach
11 0.08847817 279 acl-2011-Semi-supervised latent variable models for sentence-level sentiment analysis
12 0.085511841 339 acl-2011-Word Alignment Combination over Multiple Word Segmentation
13 0.084583364 269 acl-2011-Scaling up Automatic Cross-Lingual Semantic Role Annotation
14 0.083531223 324 acl-2011-Unsupervised Semantic Role Induction via Split-Merge Clustering
15 0.08136417 39 acl-2011-An Ensemble Model that Combines Syntactic and Semantic Clustering for Discriminative Dependency Parsing
16 0.081252657 144 acl-2011-Global Learning of Typed Entailment Rules
17 0.080973983 284 acl-2011-Simple Unsupervised Grammar Induction from Raw Text with Cascaded Finite State Models
18 0.079776973 103 acl-2011-Domain Adaptation by Constraining Inter-Domain Variability of Latent Feature Representation
19 0.079177119 2 acl-2011-AM-FM: A Semantic Framework for Translation Quality Assessment
20 0.078961492 304 acl-2011-Together We Can: Bilingual Bootstrapping for WSD
topicId topicWeight
[(0, 0.268), (1, 0.001), (2, -0.069), (3, -0.052), (4, 0.005), (5, -0.017), (6, 0.048), (7, 0.021), (8, -0.029), (9, 0.009), (10, 0.084), (11, -0.013), (12, 0.043), (13, 0.044), (14, -0.085), (15, -0.014), (16, -0.074), (17, -0.141), (18, 0.021), (19, -0.01), (20, 0.026), (21, 0.016), (22, -0.059), (23, 0.02), (24, 0.016), (25, -0.034), (26, -0.02), (27, -0.034), (28, 0.028), (29, 0.042), (30, -0.032), (31, 0.02), (32, -0.012), (33, 0.02), (34, 0.019), (35, 0.081), (36, 0.008), (37, -0.051), (38, 0.095), (39, -0.072), (40, 0.048), (41, -0.036), (42, -0.032), (43, -0.007), (44, -0.014), (45, -0.096), (46, 0.066), (47, -0.068), (48, -0.036), (49, -0.024)]
simIndex simValue paperId paperTitle
same-paper 1 0.96533012 79 acl-2011-Confidence Driven Unsupervised Semantic Parsing
Author: Dan Goldwasser ; Roi Reichart ; James Clarke ; Dan Roth
Abstract: Current approaches for semantic parsing take a supervised approach requiring a considerable amount of training data which is expensive and difficult to obtain. This supervision bottleneck is one of the major difficulties in scaling up semantic parsing. We argue that a semantic parser can be trained effectively without annotated data, and introduce an unsupervised learning algorithm. The algorithm takes a self training approach driven by confidence estimation. Evaluated over Geoquery, a standard dataset for this task, our system achieved 66% accuracy, compared to 80% of its fully supervised counterpart, demonstrating the promise of unsupervised approaches for this task.
2 0.76577264 200 acl-2011-Learning Dependency-Based Compositional Semantics
Author: Percy Liang ; Michael Jordan ; Dan Klein
Abstract: Compositional question answering begins by mapping questions to logical forms, but training a semantic parser to perform this mapping typically requires the costly annotation of the target logical forms. In this paper, we learn to map questions to answers via latent logical forms, which are induced automatically from question-answer pairs. In tackling this challenging learning problem, we introduce a new semantic representation which highlights a parallel between dependency syntax and efficient evaluation of logical forms. On two standard semantic parsing benchmarks (GEO and JOBS), our system obtains the highest published accuracies, despite requiring no annotated logical forms.
3 0.72014654 3 acl-2011-A Bayesian Model for Unsupervised Semantic Parsing
Author: Ivan Titov ; Alexandre Klementiev
Abstract: We propose a non-parametric Bayesian model for unsupervised semantic parsing. Following Poon and Domingos (2009), we consider a semantic parsing setting where the goal is to (1) decompose the syntactic dependency tree of a sentence into fragments, (2) assign each of these fragments to a cluster of semantically equivalent syntactic structures, and (3) predict predicate-argument relations between the fragments. We use hierarchical PitmanYor processes to model statistical dependencies between meaning representations of predicates and those of their arguments, as well as the clusters of their syntactic realizations. We develop a modification of the MetropolisHastings split-merge sampler, resulting in an efficient inference algorithm for the model. The method is experimentally evaluated by us- ing the induced semantic representation for the question answering task in the biomedical domain.
Author: Michael Mohler ; Razvan Bunescu ; Rada Mihalcea
Abstract: In this work we address the task of computerassisted assessment of short student answers. We combine several graph alignment features with lexical semantic similarity measures using machine learning techniques and show that the student answers can be more accurately graded than if the semantic measures were used in isolation. We also present a first attempt to align the dependency graphs of the student and the instructor answers in order to make use of a structural component in the automatic grading of student answers.
5 0.66931337 78 acl-2011-Confidence-Weighted Learning of Factored Discriminative Language Models
Author: Viet Ha Thuc ; Nicola Cancedda
Abstract: Language models based on word surface forms only are unable to benefit from available linguistic knowledge, and tend to suffer from poor estimates for rare features. We propose an approach to overcome these two limitations. We use factored features that can flexibly capture linguistic regularities, and we adopt confidence-weighted learning, a form of discriminative online learning that can better take advantage of a heavy tail of rare features. Finally, we extend the confidence-weighted learning to deal with label noise in training data, a common case with discriminative lan- guage modeling.
6 0.66875982 322 acl-2011-Unsupervised Learning of Semantic Relation Composition
7 0.65065813 320 acl-2011-Unsupervised Discovery of Domain-Specific Knowledge from Text
8 0.63050401 74 acl-2011-Combining Indicators of Allophony
9 0.62242591 25 acl-2011-A Simple Measure to Assess Non-response
10 0.60400802 342 acl-2011-full-for-print
11 0.59799379 190 acl-2011-Knowledge-Based Weak Supervision for Information Extraction of Overlapping Relations
12 0.59691578 267 acl-2011-Reversible Stochastic Attribute-Value Grammars
13 0.58652633 170 acl-2011-In-domain Relation Discovery with Meta-constraints via Posterior Regularization
14 0.57877874 295 acl-2011-Temporal Restricted Boltzmann Machines for Dependency Parsing
15 0.57792252 199 acl-2011-Learning Condensed Feature Representations from Large Unsupervised Data Sets for Supervised Learning
16 0.57657427 324 acl-2011-Unsupervised Semantic Role Induction via Split-Merge Clustering
17 0.57457536 102 acl-2011-Does Size Matter - How Much Data is Required to Train a REG Algorithm?
18 0.57388669 77 acl-2011-Computing and Evaluating Syntactic Complexity Features for Automated Scoring of Spontaneous Non-Native Speech
19 0.56882864 269 acl-2011-Scaling up Automatic Cross-Lingual Semantic Role Annotation
20 0.56815553 222 acl-2011-Model-Portability Experiments for Textual Temporal Analysis
topicId topicWeight
[(5, 0.019), (17, 0.044), (26, 0.015), (37, 0.101), (39, 0.028), (41, 0.055), (55, 0.035), (59, 0.06), (72, 0.032), (91, 0.391), (96, 0.152), (97, 0.01)]
simIndex simValue paperId paperTitle
1 0.94726646 80 acl-2011-ConsentCanvas: Automatic Texturing for Improved Readability in End-User License Agreements
Author: Oliver Schneider ; Alex Garnett
Abstract: We present ConsentCanvas, a system which structures and “texturizes” End-User License Agreement (EULA) documents to be more readable. The system aims to help users better understand the terms under which they are providing their informed consent. ConsentCanvas receives unstructured text documents as input and uses unsupervised natural language processing methods to embellish the source document using a linked stylesheet. Unlike similar usable security projects which employ summarization techniques, our system preserves the contents of the source document, minimizing the cognitive and legal burden for both the end user and the licensor. Our system does not require a corpus for training. 1
2 0.93508595 140 acl-2011-Fully Unsupervised Word Segmentation with BVE and MDL
Author: Daniel Hewlett ; Paul Cohen
Abstract: Several results in the word segmentation literature suggest that description length provides a useful estimate of segmentation quality in fully unsupervised settings. However, since the space of potential segmentations grows exponentially with the length of the corpus, no tractable algorithm follows directly from the Minimum Description Length (MDL) principle. Therefore, it is necessary to generate a set of candidate segmentations and select between them according to the MDL principle. We evaluate several algorithms for generating these candidate segmentations on a range of natural language corpora, and show that the Bootstrapped Voting Experts algorithm consistently outperforms other methods when paired with MDL.
3 0.93288845 148 acl-2011-HITS-based Seed Selection and Stop List Construction for Bootstrapping
Author: Tetsuo Kiso ; Masashi Shimbo ; Mamoru Komachi ; Yuji Matsumoto
Abstract: In bootstrapping (seed set expansion), selecting good seeds and creating stop lists are two effective ways to reduce semantic drift, but these methods generally need human supervision. In this paper, we propose a graphbased approach to helping editors choose effective seeds and stop list instances, applicable to Pantel and Pennacchiotti’s Espresso bootstrapping algorithm. The idea is to select seeds and create a stop list using the rankings of instances and patterns computed by Kleinberg’s HITS algorithm. Experimental results on a variation of the lexical sample task show the effectiveness of our method.
same-paper 4 0.91810793 79 acl-2011-Confidence Driven Unsupervised Semantic Parsing
Author: Dan Goldwasser ; Roi Reichart ; James Clarke ; Dan Roth
Abstract: Current approaches for semantic parsing take a supervised approach requiring a considerable amount of training data which is expensive and difficult to obtain. This supervision bottleneck is one of the major difficulties in scaling up semantic parsing. We argue that a semantic parser can be trained effectively without annotated data, and introduce an unsupervised learning algorithm. The algorithm takes a self training approach driven by confidence estimation. Evaluated over Geoquery, a standard dataset for this task, our system achieved 66% accuracy, compared to 80% of its fully supervised counterpart, demonstrating the promise of unsupervised approaches for this task.
5 0.86914432 313 acl-2011-Two Easy Improvements to Lexical Weighting
Author: David Chiang ; Steve DeNeefe ; Michael Pust
Abstract: We introduce two simple improvements to the lexical weighting features of Koehn, Och, and Marcu (2003) for machine translation: one which smooths the probability of translating word f to word e by simplifying English morphology, and one which conditions it on the kind of training data that f and e co-occurred in. These new variations lead to improvements of up to +0.8 BLEU, with an average improvement of +0.6 BLEU across two language pairs, two genres, and two translation systems.
6 0.78321582 108 acl-2011-EdIt: A Broad-Coverage Grammar Checker Using Pattern Grammar
7 0.71184635 145 acl-2011-Good Seed Makes a Good Crop: Accelerating Active Learning Using Language Modeling
8 0.69924593 86 acl-2011-Coreference for Learning to Extract Relations: Yes Virginia, Coreference Matters
9 0.69720519 262 acl-2011-Relation Guided Bootstrapping of Semantic Lexicons
10 0.66453147 304 acl-2011-Together We Can: Bilingual Bootstrapping for WSD
11 0.66012919 200 acl-2011-Learning Dependency-Based Compositional Semantics
12 0.65633404 239 acl-2011-P11-5002 k2opt.pdf
13 0.65055835 241 acl-2011-Parsing the Internal Structure of Words: A New Paradigm for Chinese Word Segmentation
14 0.64219332 119 acl-2011-Evaluating the Impact of Coder Errors on Active Learning
15 0.6386863 177 acl-2011-Interactive Group Suggesting for Twitter
16 0.61868823 222 acl-2011-Model-Portability Experiments for Textual Temporal Analysis
17 0.61552781 126 acl-2011-Exploiting Syntactico-Semantic Structures for Relation Extraction
18 0.61475748 147 acl-2011-Grammatical Error Correction with Alternating Structure Optimization
19 0.6109522 190 acl-2011-Knowledge-Based Weak Supervision for Information Extraction of Overlapping Relations
20 0.60798758 284 acl-2011-Simple Unsupervised Grammar Induction from Raw Text with Cascaded Finite State Models