acl acl2013 acl2013-159 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Wei Xu ; Raphael Hoffmann ; Le Zhao ; Ralph Grishman
Abstract: Distant supervision has attracted recent interest for training information extraction systems because it does not require any human annotation but rather employs existing knowledge bases to heuristically label a training corpus. However, previous work has failed to address the problem of false negative training examples mislabeled due to the incompleteness of knowledge bases. To tackle this problem, we propose a simple yet novel framework that combines a passage retrieval model using coarse features into a state-of-the-art relation extractor using multi-instance learning with fine features. We adapt the information retrieval technique of pseudo- relevance feedback to expand knowledge bases, assuming entity pairs in top-ranked passages are more likely to express a relation. Our proposed technique significantly improves the quality of distantly supervised relation extraction, boosting recall from 47.7% to 61.2% with a consistently high level of precision of around 93% in the experiments.
Reference: text
sentIndex sentText sentNum sentScore
1 Filling Knowledge Base Gaps for Distant Supervision of Relation Extraction Wei Xu+ Raphael Hoffmannˆ Le Zhao#,* Ralph Grishman+ +New York University, New York, NY, USA {xuwe i gri shman} @ c s . [sent-1, score-0.033]
2 com Abstract Distant supervision has attracted recent interest for training information extraction systems because it does not require any human annotation but rather employs existing knowledge bases to heuristically label a training corpus. [sent-9, score-0.57]
3 However, previous work has failed to address the problem of false negative training examples mislabeled due to the incompleteness of knowledge bases. [sent-10, score-0.295]
4 To tackle this problem, we propose a simple yet novel framework that combines a passage retrieval model using coarse features into a state-of-the-art relation extractor using multi-instance learning with fine features. [sent-11, score-0.89]
5 We adapt the information retrieval technique of pseudo- relevance feedback to expand knowledge bases, assuming entity pairs in top-ranked passages are more likely to express a relation. [sent-12, score-0.714]
6 Our proposed technique significantly improves the quality of distantly supervised relation extraction, boosting recall from 47. [sent-13, score-0.457]
7 1 Introduction A recent approach for training information extraction systems is distant supervision, which exploits existing knowledge bases instead of annotated texts as the source of supervision (Craven and Kumlien, 1999; Mintz et al. [sent-16, score-0.775]
8 To combat the noisy training data produced by heuristic labeling in distant supervision, researchers (Bunescu and Mooney, 2007; Riedel et al. [sent-18, score-0.313]
9 Only a few studies have directly examined the influence of the quality of the training data and attempted to enhance it (Sun et al. [sent-23, score-0.054]
10 However, their methods are handicapped by the built-in assumption that a sentence does not express a relation unless it mentions two entities which participate in the relation in the knowledge base, leading to false negatives. [sent-27, score-0.771]
11 l5gsae%t ives tmrue ntions Figure 1: Noisy training data in distant supervision In reality, knowledge bases are often incomplete, giving rise to numerous false negatives in the training data. [sent-31, score-0.934]
12 We sampled 1834 sentences that contain two entities in the New York Times 2006 corpus and manually evaluated whether they express any of a set of 50 common Freebase1 relations. [sent-32, score-0.238]
13 3%) sentences that truly express one of these relations, only 32 (1. [sent-34, score-0.125]
14 Even for one of the most complete relations in Freebase, Employee-of (with more than 100,000 entity pairs), 6 out of 27 sentences with the pattern ‘PERSON executive of ORGANIZATION’ contain a fact that is not included in Freebase and are thus mislabeled as negative. [sent-37, score-0.217]
15 These mislabelings dilute the discriminative capability of useful features and confuse the models. [sent-38, score-0.039]
16 In this paper, we will show how reducing this source of noise can significantly improve the performance of distant supervision. [sent-39, score-0.294]
17 In fact, our system corrects the relation labels of the above 6 sentences before training the relation extractor. [sent-40, score-0.603]
18 , 2010) and inspired by pseudo-relevance feedback (Xu and Croft, 1996; Lavrenko and Croft, 2001 ; Matveeva et al. [sent-47, score-0.145]
19 As shown in Figure 2, we expand an original knowledge base with possibly missing relation instances with information from the highest ranked sentences returned by a passage retrieval model (Xu et al. [sent-50, score-1.011]
20 We use coarse features for our passage retrieval model to aggressively expand the knowledge base for maximum recall; at the same time, we exploit a multi-instance learning model with fine features for relation extraction to handle the newly introduced false positives and maintain high precision. [sent-52, score-1.418]
21 Similar to iterative bootstrapping techniques (Yangarber, 2001), this mechanism uses the outputs of the first trained model to expand training data for the second model, but unlike bootstrapping it does not require iteration and avoids the problem of semantic drift. [sent-53, score-0.243]
22 We further note that iterative bootstrapping over a single distant supervision system is difficult, because state-of-the-art systems (Surdeanu et al. [sent-54, score-0.503]
23 , 2009), detect only few false negatives in the training data due to their high-precision low-recall features, which were originally proposed by Mintz et al. [sent-58, score-0.264]
24 The key to this success is the combination of two different views as in co-training (Blum and Mitchell, 1998): an information extraction technique with fine features for high precision and an information retrieval technique with coarse features for high recall. [sent-64, score-0.567]
25 1 Definitions A relation instance is an expression r(e1, e2) where r is a binary relation, and e1 and e2 are two entities having such a relation, for example CEO-of(Tim Cook, Apple). [sent-70, score-0.304]
26 The knowledge-based distant supervised learning problem takes as input (1) Σ, a training corpus, (2) E, a set of entities mentioned in that corpus, (3) R, a set of relation names, and (4) ∆, a set of ground facts of relations in R. [sent-71, score-0.617]
27 To generate our training data, we further assume (5) T, a set of entity types, as well as type signature r(E1, E2) for relations. [sent-72, score-0.135]
28 We define the positive data set POS(r) to be the set of sentences in which any related pair of entities of relation r (according to the knowledge base) is mentioned. [sent-73, score-0.433]
29 The negative data set RAW(r) is the rest of the training data, which contain two entities of the required types in the knowledge base, e. [sent-74, score-0.175]
30 one person and one organization for the CEO-of relation in Freebase. [sent-76, score-0.321]
31 Another negative data set with more conservative sense NEG(r) is defined as the set of sentences which contain the primary entity e1 (e. [sent-77, score-0.156]
32 person in any CEO-of relation in the knowledge base) and any secondary entity e2 of required type (e. [sent-79, score-0.405]
33 organization for the CEO-of relation) but the relation does not hold for this pair of entities in the knowledge base. [sent-81, score-0.409]
34 2 Distantly Supervised Passage Retrieval We extend the learning-to-rank techniques (Liu, 2011) to distant supervision setting (Xu et al. [sent-83, score-0.461]
35 While relation extraction systems exploit rich and complex features that are necessary to extract the exact relation (Mintz et al. [sent-85, score-0.652]
36 , 2011), passage retrieval components use coarse features in order to provide different and complementary feedback to informa- tion extraction models. [sent-88, score-0.81]
37 We exploit two types of lexical features: BagOf-Words and Word-Position. [sent-89, score-0.034]
38 The two types of simple binary features are shown in the following example: Sentence: Apple founder Steve Jobs died. [sent-90, score-0.139]
39 :+2 ’ For each relation r, we assume each sentence has a binary relevance label to form distantly supervised training data: sentences in POS(r) are relevant and sentences in NEG(r) are irrelevant. [sent-93, score-0.693]
40 As a pointwise learning-to-rank approach (Nallapati, 2004), the probabilities of relevance estimated by SVMs (Platt and others, 1999) are used for ranking all the sentences in the original training corpus for each relation respectively. [sent-94, score-0.441]
41 Analogously, our assumption is that entity pairs that appear in more relevant and more sentences are more likely to express the relation, and can be used to expand knowledge base and reduce false negative noise in the training data for information extraction. [sent-99, score-0.728]
42 We identify the most likely relevant entity pairs as follows: 2http : / /www . [sent-100, score-0.125]
43 3 The relation extraction model is then trained using (Σ, E, R, ∆0) with a more complete database than the original knowledge base ∆. [sent-105, score-0.5]
44 MULTIR is based on multi-instance learning, which assumes that at least one sentence of those matching a given entity-pair contains the relation of interest (Riedel et al. [sent-109, score-0.237]
45 , 2010) in the given knowledge base to tolerate false positive noise in the training data and superior than previous models (Riedel et al. [sent-110, score-0.406]
46 MULTIR uses features which are based on Mintz et al. [sent-113, score-0.039]
47 (2009) and consist of conjunctions of named entity tags, syntactic dependency paths between arguments, and lexical information. [sent-114, score-0.081]
48 3 Experiments For evaluating extraction accuracy, we follow the experimental setup of Hoffmann et al. [sent-115, score-0.105]
49 (201 1), and use their implementation of MULTIR4 with 50 training iterations as our baseline. [sent-116, score-0.054]
50 Our complete system, which we call IRMIE, combines our passage retrieval component with MULTIR. [sent-117, score-0.436]
51 The sentential extraction evaluation is performed on a small amount of manually annotated sentences, sampled from the union of matched sentences and 3We found p = 0. [sent-121, score-0.354]
52 edu / ˜raphae lh/mr / 667 Table1:OveralsMnI TRteUMesnLtITDEiaILRltaELXeSExXtrac89i401o. [sent-126, score-0.034]
53 (201 1) and our corrected test set: Our proposed relevance feedback technique yields a substantial increase in recall. [sent-135, score-0.261]
54 We define Se as the sentences where some system extracted a relation and SF as the sentences that match the arguments of a fact in ∆. [sent-137, score-0.387]
55 The sentential precision and recall is computed on a randomly sampled set of sentences from Se ∪ SF, in which each sentence is manually labeled w∪hSether it expresses any relation in R. [sent-138, score-0.488]
56 Figure 3 shows the precision/recall curves for MULTIR with and without pseudo-relevance feedback computed on the test dataset of 1000 sentence used by Hoffmann et al. [sent-139, score-0.18]
57 With the pseudo-relevance feedback from passage retrieval, IRMIE achieves significantly higher recall at a consistently high level of precision. [sent-141, score-0.464]
58 Because the two types of lexical features used in our passage retrieval models are not used in MULTIR, we created another baseline MULTIRLEX by adding these features into MULTIR in order to rule out the improvement from additional information. [sent-146, score-0.514]
59 Note that the sentences are sampled from the union of Freebase matches and sentences from which some systems in Hoffmann et al. [sent-147, score-0.281]
60 We therefore also created a new test set of 1000 sentences by sampling from the union of Freebase matches and sentences where MULTIRLEX or IRMIELEX extracted a relation. [sent-150, score-0.235]
61 Table 1 shows the overall precision and recall computed against these two test datasets, with and without adding lexical features into multi-instance learning models. [sent-151, score-0.085]
62 It greatly improves a state-of-the-art multi-instance learning model by correcting the most likely false negatives in the training data based on the ranking of a passage retrieval model. [sent-157, score-0.7]
63 In the future, we would like to more tightly integrate a coarser featured estimator of sentential relevance and a finer featured relation extractor, such that a single joint-model can be learned. [sent-158, score-0.518]
64 Acknowledgments Supported in part by NSF grant IIS-10183 17, the Air Force Research Laboratory (AFRL) under prime contract number FA8750-09-C0181 and the Intelligence Advanced Research Projects Activity (IARPA) via Department of Interior National Business Center contract number D11PC20154. [sent-159, score-0.068]
65 Constructing biological knowledge bases by extracting information from text sources. [sent-187, score-0.155]
66 Knowledge-based weak supervision for information extraction of overlapping relations. [sent-193, score-0.307]
67 Distant supervision for relation extraction with an incomplete knowledge base. [sent-210, score-0.649]
68 End-to-end relation extraction using distant supervision from external semantic repositories. [sent-222, score-0.803]
69 New york university 2011 system for kbp slot filling. [sent-239, score-0.052]
70 Reducing wrong labels in distant supervision for relation extraction. [sent-248, score-0.698]
71 Passage retrieval for information extraction using distant supervision. [sent-262, score-0.527]
wordName wordTfidf (topN-words)
[('hoffmann', 0.294), ('passage', 0.273), ('distant', 0.259), ('relation', 0.237), ('multir', 0.233), ('supervision', 0.202), ('mintz', 0.166), ('retrieval', 0.163), ('feedback', 0.145), ('distantly', 0.133), ('proceedigns', 0.133), ('riedel', 0.129), ('false', 0.126), ('sigir', 0.121), ('irmie', 0.113), ('freebase', 0.112), ('xu', 0.11), ('expand', 0.105), ('extraction', 0.105), ('base', 0.104), ('bases', 0.101), ('founder', 0.1), ('surdeanu', 0.094), ('apple', 0.092), ('croft', 0.086), ('coarse', 0.085), ('sentential', 0.084), ('negatives', 0.084), ('entity', 0.081), ('grishman', 0.079), ('lavrenko', 0.076), ('bonan', 0.075), ('multirlex', 0.075), ('raphae', 0.075), ('sentences', 0.075), ('relevance', 0.075), ('entities', 0.067), ('takamatsu', 0.066), ('ralph', 0.066), ('neg', 0.066), ('cao', 0.062), ('matveeva', 0.061), ('craven', 0.061), ('featured', 0.061), ('mislabeled', 0.061), ('died', 0.058), ('nallapati', 0.058), ('ramesh', 0.058), ('blum', 0.055), ('training', 0.054), ('knowledge', 0.054), ('fine', 0.054), ('raghunathan', 0.053), ('raphael', 0.053), ('york', 0.052), ('organization', 0.051), ('incomplete', 0.051), ('iarpa', 0.051), ('express', 0.05), ('jobs', 0.049), ('afrl', 0.049), ('bunescu', 0.048), ('chang', 0.047), ('sampled', 0.046), ('recall', 0.046), ('platt', 0.045), ('union', 0.044), ('relevant', 0.044), ('query', 0.043), ('libsvm', 0.043), ('bruce', 0.043), ('le', 0.043), ('bootstrapping', 0.042), ('zhao', 0.041), ('sf', 0.041), ('matches', 0.041), ('technique', 0.041), ('annual', 0.041), ('wei', 0.04), ('extractor', 0.039), ('features', 0.039), ('nguyen', 0.038), ('mihai', 0.037), ('acm', 0.037), ('noise', 0.035), ('curves', 0.035), ('exploit', 0.034), ('edu', 0.034), ('contract', 0.034), ('steve', 0.034), ('person', 0.033), ('harman', 0.033), ('tolerate', 0.033), ('gri', 0.033), ('pseudorelevance', 0.033), ('yangarber', 0.033), ('avrim', 0.033), ('truc', 0.033), ('burkard', 0.033), ('guihong', 0.033)]
simIndex simValue paperId paperTitle
same-paper 1 1.0 159 acl-2013-Filling Knowledge Base Gaps for Distant Supervision of Relation Extraction
Author: Wei Xu ; Raphael Hoffmann ; Le Zhao ; Ralph Grishman
Abstract: Distant supervision has attracted recent interest for training information extraction systems because it does not require any human annotation but rather employs existing knowledge bases to heuristically label a training corpus. However, previous work has failed to address the problem of false negative training examples mislabeled due to the incompleteness of knowledge bases. To tackle this problem, we propose a simple yet novel framework that combines a passage retrieval model using coarse features into a state-of-the-art relation extractor using multi-instance learning with fine features. We adapt the information retrieval technique of pseudo- relevance feedback to expand knowledge bases, assuming entity pairs in top-ranked passages are more likely to express a relation. Our proposed technique significantly improves the quality of distantly supervised relation extraction, boosting recall from 47.7% to 61.2% with a consistently high level of precision of around 93% in the experiments.
2 0.31174791 352 acl-2013-Towards Accurate Distant Supervision for Relational Facts Extraction
Author: Xingxing Zhang ; Jianwen Zhang ; Junyu Zeng ; Jun Yan ; Zheng Chen ; Zhifang Sui
Abstract: Distant supervision (DS) is an appealing learning method which learns from existing relational facts to extract more from a text corpus. However, the accuracy is still not satisfying. In this paper, we point out and analyze some critical factors in DS which have great impact on accuracy, including valid entity type detection, negative training examples construction and ensembles. We propose an approach to handle these factors. By experimenting on Wikipedia articles to extract the facts in Freebase (the top 92 relations), we show the impact of these three factors on the accuracy of DS and the remarkable improvement led by the proposed approach.
3 0.14568564 55 acl-2013-Are Semantically Coherent Topic Models Useful for Ad Hoc Information Retrieval?
Author: Romain Deveaud ; Eric SanJuan ; Patrice Bellot
Abstract: The current topic modeling approaches for Information Retrieval do not allow to explicitly model query-oriented latent topics. More, the semantic coherence of the topics has never been considered in this field. We propose a model-based feedback approach that learns Latent Dirichlet Allocation topic models on the top-ranked pseudo-relevant feedback, and we measure the semantic coherence of those topics. We perform a first experimental evaluation using two major TREC test collections. Results show that retrieval perfor- mances tend to be better when using topics with higher semantic coherence.
4 0.12804186 169 acl-2013-Generating Synthetic Comparable Questions for News Articles
Author: Oleg Rokhlenko ; Idan Szpektor
Abstract: We introduce the novel task of automatically generating questions that are relevant to a text but do not appear in it. One motivating example of its application is for increasing user engagement around news articles by suggesting relevant comparable questions, such as “is Beyonce a better singer than Madonna?”, for the user to answer. We present the first algorithm for the task, which consists of: (a) offline construction of a comparable question template database; (b) ranking of relevant templates to a given article; and (c) instantiation of templates only with entities in the article whose comparison under the template’s relation makes sense. We tested the suggestions generated by our algorithm via a Mechanical Turk experiment, which showed a significant improvement over the strongest baseline of more than 45% in all metrics.
5 0.11788942 272 acl-2013-Paraphrase-Driven Learning for Open Question Answering
Author: Anthony Fader ; Luke Zettlemoyer ; Oren Etzioni
Abstract: We study question answering as a machine learning problem, and induce a function that maps open-domain questions to queries over a database of web extractions. Given a large, community-authored, question-paraphrase corpus, we demonstrate that it is possible to learn a semantic lexicon and linear ranking function without manually annotating questions. Our approach automatically generalizes a seed lexicon and includes a scalable, parallelized perceptron parameter estimation scheme. Experiments show that our approach more than quadruples the recall of the seed lexicon, with only an 8% loss in precision.
6 0.11605618 60 acl-2013-Automatic Coupling of Answer Extraction and Information Retrieval
7 0.10632626 134 acl-2013-Embedding Semantic Similarity in Tree Kernels for Domain Adaptation of Relation Extraction
8 0.10629527 318 acl-2013-Sentiment Relevance
9 0.10127402 365 acl-2013-Understanding Tables in Context Using Standard NLP Toolkits
10 0.087501943 160 acl-2013-Fine-grained Semantic Typing of Emerging Entities
11 0.086693779 179 acl-2013-HYENA-live: Fine-Grained Online Entity Type Classification from Natural-language Text
12 0.082150653 71 acl-2013-Bootstrapping Entity Translation on Weakly Comparable Corpora
13 0.080932319 215 acl-2013-Large-scale Semantic Parsing via Schema Matching and Lexicon Extension
14 0.080133624 291 acl-2013-Question Answering Using Enhanced Lexical Semantic Models
15 0.079676792 206 acl-2013-Joint Event Extraction via Structured Prediction with Global Features
16 0.079110257 273 acl-2013-Paraphrasing Adaptation for Web Search Ranking
17 0.076110125 285 acl-2013-Propminer: A Workflow for Interactive Information Extraction and Exploration using Dependency Trees
18 0.07140518 41 acl-2013-Aggregated Word Pair Features for Implicit Discourse Relation Disambiguation
19 0.071281143 338 acl-2013-Task Alternation in Parallel Sentence Retrieval for Twitter Translation
20 0.07041385 252 acl-2013-Multigraph Clustering for Unsupervised Coreference Resolution
topicId topicWeight
[(0, 0.198), (1, 0.076), (2, -0.021), (3, -0.091), (4, 0.068), (5, 0.161), (6, -0.029), (7, -0.087), (8, 0.034), (9, 0.026), (10, 0.071), (11, -0.056), (12, -0.046), (13, 0.036), (14, -0.016), (15, 0.02), (16, -0.032), (17, 0.026), (18, -0.104), (19, 0.006), (20, 0.0), (21, 0.072), (22, -0.031), (23, 0.153), (24, 0.081), (25, 0.015), (26, 0.007), (27, -0.067), (28, -0.081), (29, 0.09), (30, 0.021), (31, 0.054), (32, -0.071), (33, 0.003), (34, -0.12), (35, -0.045), (36, 0.097), (37, -0.08), (38, -0.047), (39, 0.022), (40, 0.008), (41, -0.021), (42, -0.034), (43, 0.048), (44, 0.035), (45, 0.019), (46, -0.019), (47, 0.048), (48, -0.018), (49, 0.067)]
simIndex simValue paperId paperTitle
same-paper 1 0.94444835 159 acl-2013-Filling Knowledge Base Gaps for Distant Supervision of Relation Extraction
Author: Wei Xu ; Raphael Hoffmann ; Le Zhao ; Ralph Grishman
Abstract: Distant supervision has attracted recent interest for training information extraction systems because it does not require any human annotation but rather employs existing knowledge bases to heuristically label a training corpus. However, previous work has failed to address the problem of false negative training examples mislabeled due to the incompleteness of knowledge bases. To tackle this problem, we propose a simple yet novel framework that combines a passage retrieval model using coarse features into a state-of-the-art relation extractor using multi-instance learning with fine features. We adapt the information retrieval technique of pseudo- relevance feedback to expand knowledge bases, assuming entity pairs in top-ranked passages are more likely to express a relation. Our proposed technique significantly improves the quality of distantly supervised relation extraction, boosting recall from 47.7% to 61.2% with a consistently high level of precision of around 93% in the experiments.
2 0.87225914 352 acl-2013-Towards Accurate Distant Supervision for Relational Facts Extraction
Author: Xingxing Zhang ; Jianwen Zhang ; Junyu Zeng ; Jun Yan ; Zheng Chen ; Zhifang Sui
Abstract: Distant supervision (DS) is an appealing learning method which learns from existing relational facts to extract more from a text corpus. However, the accuracy is still not satisfying. In this paper, we point out and analyze some critical factors in DS which have great impact on accuracy, including valid entity type detection, negative training examples construction and ensembles. We propose an approach to handle these factors. By experimenting on Wikipedia articles to extract the facts in Freebase (the top 92 relations), we show the impact of these three factors on the accuracy of DS and the remarkable improvement led by the proposed approach.
3 0.7580784 160 acl-2013-Fine-grained Semantic Typing of Emerging Entities
Author: Ndapandula Nakashole ; Tomasz Tylenda ; Gerhard Weikum
Abstract: Methods for information extraction (IE) and knowledge base (KB) construction have been intensively studied. However, a largely under-explored case is tapping into highly dynamic sources like news streams and social media, where new entities are continuously emerging. In this paper, we present a method for discovering and semantically typing newly emerging out-ofKB entities, thus improving the freshness and recall of ontology-based IE and improving the precision and semantic rigor of open IE. Our method is based on a probabilistic model that feeds weights into integer linear programs that leverage type signatures of relational phrases and type correlation or disjointness constraints. Our experimental evaluation, based on crowdsourced user studies, show our method performing significantly better than prior work.
4 0.75296408 179 acl-2013-HYENA-live: Fine-Grained Online Entity Type Classification from Natural-language Text
Author: Mohamed Amir Yosef ; Sandro Bauer ; Johannes Hoffart ; Marc Spaniol ; Gerhard Weikum
Abstract: Recent research has shown progress in achieving high-quality, very fine-grained type classification in hierarchical taxonomies. Within such a multi-level type hierarchy with several hundreds of types at different levels, many entities naturally belong to multiple types. In order to achieve high-precision in type classification, current approaches are either limited to certain domains or require time consuming multistage computations. As a consequence, existing systems are incapable of performing ad-hoc type classification on arbitrary input texts. In this demo, we present a novel Webbased tool that is able to perform domain independent entity type classification under real time conditions. Thanks to its efficient implementation and compacted feature representation, the system is able to process text inputs on-the-fly while still achieving equally high precision as leading state-ofthe-art implementations. Our system offers an online interface where natural-language text can be inserted, which returns semantic type labels for entity mentions. Further more, the user interface allows users to explore the assigned types by visualizing and navigating along the type-hierarchy.
5 0.72744292 169 acl-2013-Generating Synthetic Comparable Questions for News Articles
Author: Oleg Rokhlenko ; Idan Szpektor
Abstract: We introduce the novel task of automatically generating questions that are relevant to a text but do not appear in it. One motivating example of its application is for increasing user engagement around news articles by suggesting relevant comparable questions, such as “is Beyonce a better singer than Madonna?”, for the user to answer. We present the first algorithm for the task, which consists of: (a) offline construction of a comparable question template database; (b) ranking of relevant templates to a given article; and (c) instantiation of templates only with entities in the article whose comparison under the template’s relation makes sense. We tested the suggestions generated by our algorithm via a Mechanical Turk experiment, which showed a significant improvement over the strongest baseline of more than 45% in all metrics.
6 0.69479793 365 acl-2013-Understanding Tables in Context Using Standard NLP Toolkits
7 0.64916641 215 acl-2013-Large-scale Semantic Parsing via Schema Matching and Lexicon Extension
8 0.64852375 272 acl-2013-Paraphrase-Driven Learning for Open Question Answering
9 0.6117965 178 acl-2013-HEADY: News headline abstraction through event pattern clustering
10 0.58221132 290 acl-2013-Question Analysis for Polish Question Answering
11 0.57826924 139 acl-2013-Entity Linking for Tweets
12 0.57554024 172 acl-2013-Graph-based Local Coherence Modeling
13 0.55599278 340 acl-2013-Text-Driven Toponym Resolution using Indirect Supervision
14 0.55574036 219 acl-2013-Learning Entity Representation for Entity Disambiguation
15 0.54409719 158 acl-2013-Feature-Based Selection of Dependency Paths in Ad Hoc Information Retrieval
16 0.54127228 61 acl-2013-Automatic Interpretation of the English Possessive
17 0.53845292 273 acl-2013-Paraphrasing Adaptation for Web Search Ranking
18 0.51256311 346 acl-2013-The Impact of Topic Bias on Quality Flaw Prediction in Wikipedia
19 0.50864327 134 acl-2013-Embedding Semantic Similarity in Tree Kernels for Domain Adaptation of Relation Extraction
20 0.5081135 356 acl-2013-Transfer Learning Based Cross-lingual Knowledge Extraction for Wikipedia
topicId topicWeight
[(0, 0.051), (6, 0.038), (8, 0.039), (11, 0.08), (14, 0.01), (15, 0.01), (21, 0.066), (24, 0.04), (26, 0.064), (28, 0.013), (35, 0.118), (42, 0.063), (48, 0.028), (64, 0.028), (70, 0.053), (78, 0.109), (88, 0.047), (90, 0.027), (95, 0.064)]
simIndex simValue paperId paperTitle
same-paper 1 0.88816106 159 acl-2013-Filling Knowledge Base Gaps for Distant Supervision of Relation Extraction
Author: Wei Xu ; Raphael Hoffmann ; Le Zhao ; Ralph Grishman
Abstract: Distant supervision has attracted recent interest for training information extraction systems because it does not require any human annotation but rather employs existing knowledge bases to heuristically label a training corpus. However, previous work has failed to address the problem of false negative training examples mislabeled due to the incompleteness of knowledge bases. To tackle this problem, we propose a simple yet novel framework that combines a passage retrieval model using coarse features into a state-of-the-art relation extractor using multi-instance learning with fine features. We adapt the information retrieval technique of pseudo- relevance feedback to expand knowledge bases, assuming entity pairs in top-ranked passages are more likely to express a relation. Our proposed technique significantly improves the quality of distantly supervised relation extraction, boosting recall from 47.7% to 61.2% with a consistently high level of precision of around 93% in the experiments.
2 0.85756987 378 acl-2013-Using subcategorization knowledge to improve case prediction for translation to German
Author: Marion Weller ; Alexander Fraser ; Sabine Schulte im Walde
Abstract: This paper demonstrates the need and impact of subcategorization information for SMT. We combine (i) features on sourceside syntactic subcategorization and (ii) an external knowledge base with quantitative, dependency-based information about target-side subcategorization frames. A manual evaluation of an English-toGerman translation task shows that the subcategorization information has a positive impact on translation quality through better prediction of case.
3 0.85409731 352 acl-2013-Towards Accurate Distant Supervision for Relational Facts Extraction
Author: Xingxing Zhang ; Jianwen Zhang ; Junyu Zeng ; Jun Yan ; Zheng Chen ; Zhifang Sui
Abstract: Distant supervision (DS) is an appealing learning method which learns from existing relational facts to extract more from a text corpus. However, the accuracy is still not satisfying. In this paper, we point out and analyze some critical factors in DS which have great impact on accuracy, including valid entity type detection, negative training examples construction and ensembles. We propose an approach to handle these factors. By experimenting on Wikipedia articles to extract the facts in Freebase (the top 92 relations), we show the impact of these three factors on the accuracy of DS and the remarkable improvement led by the proposed approach.
4 0.83748978 282 acl-2013-Predicting and Eliciting Addressee's Emotion in Online Dialogue
Author: Takayuki Hasegawa ; Nobuhiro Kaji ; Naoki Yoshinaga ; Masashi Toyoda
Abstract: While there have been many attempts to estimate the emotion of an addresser from her/his utterance, few studies have explored how her/his utterance affects the emotion of the addressee. This has motivated us to investigate two novel tasks: predicting the emotion of the addressee and generating a response that elicits a specific emotion in the addressee’s mind. We target Japanese Twitter posts as a source of dialogue data and automatically build training data for learning the predictors and generators. The feasibility of our approaches is assessed by using 1099 utterance-response pairs that are built by . five human workers.
5 0.83322227 293 acl-2013-Random Walk Factoid Annotation for Collective Discourse
Author: Ben King ; Rahul Jha ; Dragomir Radev ; Robert Mankoff
Abstract: In this paper, we study the problem of automatically annotating the factoids present in collective discourse. Factoids are information units that are shared between instances of collective discourse and may have many different ways ofbeing realized in words. Our approach divides this problem into two steps, using a graph-based approach for each step: (1) factoid discovery, finding groups of words that correspond to the same factoid, and (2) factoid assignment, using these groups of words to mark collective discourse units that contain the respective factoids. We study this on two novel data sets: the New Yorker caption contest data set, and the crossword clues data set.
6 0.83033198 175 acl-2013-Grounded Language Learning from Video Described with Sentences
7 0.82244802 318 acl-2013-Sentiment Relevance
8 0.81769156 276 acl-2013-Part-of-Speech Induction in Dependency Trees for Statistical Machine Translation
9 0.80075264 22 acl-2013-A Structured Distributional Semantic Model for Event Co-reference
10 0.80058885 158 acl-2013-Feature-Based Selection of Dependency Paths in Ad Hoc Information Retrieval
11 0.8004328 46 acl-2013-An Infinite Hierarchical Bayesian Model of Phrasal Translation
12 0.79858243 169 acl-2013-Generating Synthetic Comparable Questions for News Articles
13 0.7984916 4 acl-2013-A Context Free TAG Variant
14 0.79739988 172 acl-2013-Graph-based Local Coherence Modeling
15 0.79584199 83 acl-2013-Collective Annotation of Linguistic Resources: Basic Principles and a Formal Model
16 0.7954911 215 acl-2013-Large-scale Semantic Parsing via Schema Matching and Lexicon Extension
17 0.79414421 228 acl-2013-Leveraging Domain-Independent Information in Semantic Parsing
19 0.79175538 18 acl-2013-A Sentence Compression Based Framework to Query-Focused Multi-Document Summarization
20 0.79018354 265 acl-2013-Outsourcing FrameNet to the Crowd