acl acl2012 acl2012-159 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Enrique Alfonseca ; Katja Filippova ; Jean-Yves Delort ; Guillermo Garrido
Abstract: We describe the use of a hierarchical topic model for automatically identifying syntactic and lexical patterns that explicitly state ontological relations. We leverage distant supervision using relations from the knowledge base FreeBase, but do not require any manual heuristic nor manual seed list selections. Results show that the learned patterns can be used to extract new relations with good precision.
Reference: text
sentIndex sentText sentNum sentScore
1 Abstract We describe the use of a hierarchical topic model for automatically identifying syntactic and lexical patterns that explicitly state ontological relations. [sent-5, score-0.311]
2 We leverage distant supervision using relations from the knowledge base FreeBase, but do not require any manual heuristic nor manual seed list selections. [sent-6, score-0.726]
3 Results show that the learned patterns can be used to extract new relations with good precision. [sent-7, score-0.381]
4 1 Introduction The detection of relations between entities for the automatic population of knowledge bases is very useful for solving tasks such as Entity Disambiguation, Information Retrieval and Question Answering. [sent-8, score-0.56]
5 The availability of high-coverage, generalpurpose knowledge bases enable the automatic iden- tification and disambiguation of entities in text and its applications (Bunescu and Pasca, 2006; Cucerzan, 2007; McNamee and Dang, 2009; Kwok et al. [sent-9, score-0.371]
6 , 2010) started as an effort to approach relation extraction in ∗Work done during an internship at Google Zurich. [sent-20, score-0.379]
7 54 a completely unsupervised way, by learning regularities and patterns from the web. [sent-21, score-0.265]
8 These systems do not need any manual data or rules, but the relational facts they extract are not immediately disambiguated to entities and relations from a knowledge base. [sent-25, score-0.691]
9 A different family of unsupervised methods for relation extraction is unsupervised semantic parsing, which aims at clustering entity mentions and relation surface forms, thus generating a semantic representation of the texts on which inference may be used. [sent-26, score-0.918]
10 Some techniques that have been used are Markov Random Fields (Poon and Domingos, 2009) and Bayesian generative models (Titov and Klementiev, 2011). [sent-27, score-0.048]
11 A usual problem is that two related entities may co-occur in one sentence for many unrelated reasons. [sent-36, score-0.232]
12 For example, Barack Obama is the president of the United States, but not every sentence including the two entities supports and states this relation. [sent-37, score-0.342]
13 extracting sentences only from encyclopedic entries (Mintz et al. [sent-40, score-0.058]
14 , 2011), or syntactic restrictions on the sentences and the entity mentions (Wu and Weld, 2010). [sent-45, score-0.153]
15 The main contribution of this work is presenting a variant of distance supervision for relation extraction where we do not use heuristics in the selection of the training data. [sent-48, score-0.545]
16 Instead, we use topic models to discriminate between the patterns that are expressing the relation and those that are ambiguous and can be applied across relations. [sent-49, score-0.6]
17 In this way, high-precision extraction patterns can be learned without the need of any manual intervention. [sent-50, score-0.379]
18 2 Unsupervised relational pattern learning Similar to other distant supervision methods, our approach takes as input an existing knowledge base containing entities and relations, and a textual corpus. [sent-51, score-0.897]
19 In this work it is not necessary for the corpus to be related to the knowledge base. [sent-52, score-0.059]
20 In what follows we assume that all the relations studied are binary and hold between exactly two entities in the knowledge base. [sent-53, score-0.474]
21 We also assume a dependency parser is available, and that the entities have been automatically disambiguated using the knowledge base as sense inventory. [sent-54, score-0.512]
22 One of the most important problems to solve in distant supervision approaches is to be able to distinguish which of the textual examples that include two related entities, ei and ej, are supporting the relation. [sent-55, score-0.438]
23 This section describes a fully unsupervised solution to this problem, computing the probability that a pattern supports a given relation, which will allow us to determine the most likely relation expressed in any sentence. [sent-56, score-0.503]
24 Specifically, if a sentence contains two entities, ei and ej, connected through a pattern w, our model computes the probability that the pattern is expressing any relation –P(r|w)– for any praeltatetirnon i r dxepfrineessdi ning tahney knowledge b(ars|ew. [sent-57, score-0.68]
25 –N footer that we refer to patterns with the symbol w, as they are the words in our topic models. [sent-58, score-0.311]
26 Preprocessing As a first step, the textual corpus is processed and the data is transformed in the following way: (a) the input corpus is parsed and en55 Figure 1: Example of a generated set of document collections from a news corpus for relation extraction. [sent-59, score-0.535]
27 Larger boxes are document collections (relations), and inner boxes are documents (entity pairs). [sent-60, score-0.346]
28 Document contain dependency patterns, which are words in the topic model. [sent-61, score-0.164]
29 This context may be a complex structure, such as the dependency path joining the two entities, but it is considered for our purposes as a single term; (e) for each relation r relating ei with ej, document Dij is added to collection Cr. [sent-63, score-0.593]
30 Note that if the two entities are related in different ways at the same time, an identical copy of the document Dij will be added to the collection for all those relations. [sent-64, score-0.388]
31 Figure 1 shows a set of document collections gen- Figure 2: Plate diagram of the generative model used. [sent-65, score-0.288]
32 Each relation r has associated a different document collection, which contains one document associated to each entity pair from the knowledge base which is in relation r. [sent-67, score-0.965]
33 The words in each document can be, for example, all the dependency paths that have been observed in the input textual corpus between the two related entities. [sent-68, score-0.207]
34 Each document will contain some very generic paths (e. [sent-69, score-0.113]
35 the two entities consecutive in the text) and some more specific paths. [sent-71, score-0.232]
36 Generative model Once these collections are built, we use the generative model from Figure 2 to learn the probability that a dependency path is conveying some relation between the entities it connects. [sent-72, score-0.75]
37 w (the observed variable) represents a pattern between two entities. [sent-74, score-0.119]
38 The topic model φG captures general patterns that appear for all relations. [sent-75, score-0.311]
39 φD captures patterns that are specific about a certain entity pair, but which are not generalizable across all pairs with the same relation. [sent-76, score-0.299]
40 Finally φA contains the patterns that are observed across most pairs related with the same relation. [sent-77, score-0.198]
41 The topic assignments (for each pattern) that are the output of this process are used to estimate P(r|w) : when we observe pataterern u w, dth teo probability (thr|awt )it: conveys reel oabtiseornv r. [sent-80, score-0.158]
42 56 3 Experiments and results Settings We use Freebase as our knowledge base. [sent-81, score-0.059]
43 text corpus used contains 33 million English news articles that we downloaded between January 2004 and December 2011. [sent-83, score-0.037]
44 A random sample of 3M of them is used for building the document collections on which to train the topic models, and the remaining 30M is used for testing. [sent-84, score-0.353]
45 The corpus is preprocessed by identifying Freebase entity mentions, using an approach similar to (Milne and Witten, 2008), and parsing it with an inductive dependency parser (Nivre, 2006). [sent-85, score-0.21]
46 From the three million training documents, a set of document collections (one per relation) has been generated, by considering the sentences that contain two entities which are related in FreeBase through any binary relation and restricting to high-frequency 200 relations. [sent-86, score-0.761]
47 Two ways of extracting patterns have been used: (a) Syntactic, taking the dependency path between the two entities, and (b) Intertext, taking the text between the two. [sent-87, score-0.289]
48 In both cases, a topic model has been trained to learn the probability of a relation given a pattern w: p(r|w). [sent-88, score-0.484]
49 For λ we use symmetric Deniri ach plaettt priors λG = 0. [sent-89, score-0.036]
50 001, following the intuition that for the background the probability mass across patterns should be more evenly distributed. [sent-92, score-0.198]
51 γ is set as (15, 15, 1), indicating in the prior that we expect more patterns to belong to the background and entity-pairspecific distributions due to the very noisy nature of the input data. [sent-93, score-0.198]
52 As a baseline, using the same training corpus, we have calculated p(r|w) using the maximum likelihhoavode ecastlicmulaatete:d dth pe( rn|uwm)b uersi nofg t ithmees m thaxaitm a pattern w has been seen connecting two entities for which r holds divided by the total frequency of the pattern. [sent-95, score-0.435]
53 Extractions evaluation The patterns have been applied to the 30 million documents left for testing. [sent-96, score-0.235]
54 For each pair of entities disambiguated as FreeBase entities, if they are connected through a known pattern, they are assigned arg maxr p(r|w). [sent-97, score-0.327]
55 An extraction is to be judged correct if both it is correct in real life and the sentence from which it was extracted really supports it. [sent-100, score-0.192]
56 4% of the items on whether the sentence supports the relation, and for 20% of the items on whether the relation holds in the real world. [sent-105, score-0.317]
57 fAfesr can hb ere seen, tsh oef M p(Lr|Ew b)a aserelines (in red with syntactic patterns and green with intertext) perform consistently worse than the models learned using the topic models (in pink and blue). [sent-107, score-0.311]
58 The difference in precision, aggregated across all relations, is statistically significant at 95% confidence for most of the thresholds. [sent-108, score-0.053]
59 Extractions aggregation We can take advantage of redundancy on the web to calculate a support metric for the extractions. [sent-109, score-0.056]
60 In this experiment, for every extracted relation (r, e1, e2), for every occurrence of a pattern wi connecting e1 and e2, we add up p(r|wi). [sent-110, score-0.41]
61 Extractions that are obtained many times apn(rd| fwrom high-precision patterns will rank higher. [sent-111, score-0.198]
62 We have considered the top four highest-frequency relations for people. [sent-113, score-0.183]
63 For each of these, both syntactic patterns and intermediate-text patterns have been evaluated. [sent-115, score-0.396]
64 The results are very interesting: using syntax, Death place appears easy to extract new relations and to find support. [sent-116, score-0.243]
65 subj prep vv ** ARG1 died pobj ww at prep uu home pobj ww in ARG2 57 RelationUnknown relationsKnown relations Correct relation P@50 Syntax Intertext Sentence Syntax P@50 Intertext support Parent0. [sent-119, score-0.827]
66 On the other hand, birth place and nationality have very different results for new relation acquisition vs. [sent-136, score-0.592]
67 The reason is that these relations are very correlated to other relations that we did not have in our training set. [sent-138, score-0.366]
68 In the case of birth place, many relations refer to having an official position in the city, such as mayor; and for nationality, many of the patterns extract presidents or ministers. [sent-139, score-0.563]
69 Not having mayor or president in our initial collection (see Figure 1), the support for these patterns is incorrectly learned. [sent-140, score-0.425]
70 In the case of nationality, however, even though the extracted sentences do not support the relation (P@50 = 0. [sent-141, score-0.308]
71 34 for intertext), the new relations extracted are mostly correct (P@50 = 0. [sent-142, score-0.183]
72 86) as most presidents and ministers in the real world have the nationality of the country where they govern. [sent-143, score-0.305]
73 4 Conclusions We have described a new distant supervision model with which to learn patterns for relation extraction with no manual intervention. [sent-144, score-0.932]
74 Results are promising, we could obtain new relations that are not in FreeBase with a high precision for some relation types. [sent-145, score-0.435]
75 It is also useful to extract support sentences for known relations. [sent-146, score-0.056]
76 More work is needed in understanding which relations are compatible or overlapping and which ones can partially imply each other (such as president-country or born in-mayor). [sent-147, score-0.183]
77 Relational duality: Unsupervised extraction of semantic relations between entities on the web. [sent-166, score-0.542]
78 In Proceedings of the 19th international conference on World wide web, pages 15 1–160. [sent-167, score-0.052]
79 The automatic content extraction (ace) program–tasks, data, and evaluation. [sent-188, score-0.127]
80 In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 362–370. [sent-203, score-0.052]
81 In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 286–295. [sent-212, score-0.052]
82 Knowledge-based weak supervision for information extraction of overlapping relations. [sent-222, score-0.293]
83 In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 541–550. [sent-223, score-0.052]
84 Learning arguments and supertypes of semantic relations using recursive patterns. [sent-237, score-0.183]
85 In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, 58 pages 1482–1491. [sent-238, score-0.052]
86 A combination of topic models with max-margin learning for relation detection. [sent-253, score-0.365]
87 Overview of the tac 2009 knowledge base population track. [sent-259, score-0.23]
88 In Proceeding of the 1 ACM conference 7th on Information and knowledge management, pages 509–518. [sent-267, score-0.111]
89 In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2-Volume 2, pages 1003– 1011. [sent-276, score-0.052]
90 Organizing and searching the world wide web of facts-step one: the one-million fact extraction challenge. [sent-291, score-0.168]
91 In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1-Volume 1, pages 1–10. [sent-309, score-0.052]
92 Learning surface text patterns for a question answering system. [sent-315, score-0.243]
93 In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pages 41–47. [sent-316, score-0.052]
94 In Proceedings of the COLING/ACL on Main conference poster sessions, pages 73 1–738. [sent-322, score-0.052]
95 In Proceedings of the 23rd national conference on Artificial intelligence, pages 1609–1614. [sent-348, score-0.052]
96 In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 118–127. [sent-356, score-0.052]
97 In Proceedings of Human Language Technologies: The Annual Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 25–26. [sent-375, score-0.052]
wordName wordTfidf (topN-words)
[('relation', 0.252), ('freebase', 0.25), ('entities', 0.232), ('intertext', 0.208), ('patterns', 0.198), ('relations', 0.183), ('nationality', 0.181), ('supervision', 0.166), ('hoffmann', 0.16), ('dij', 0.145), ('distant', 0.135), ('collections', 0.127), ('extraction', 0.127), ('pattern', 0.119), ('document', 0.113), ('topic', 0.113), ('ej', 0.112), ('extractions', 0.106), ('entity', 0.101), ('birth', 0.099), ('weld', 0.095), ('disambiguated', 0.095), ('ei', 0.094), ('pasca', 0.088), ('mayor', 0.083), ('milne', 0.083), ('presidents', 0.083), ('banko', 0.08), ('base', 0.075), ('ace', 0.074), ('uned', 0.073), ('yao', 0.071), ('relational', 0.068), ('unsupervised', 0.067), ('mcnamee', 0.066), ('kasneci', 0.066), ('textrunner', 0.066), ('kozareva', 0.066), ('pobj', 0.066), ('kwok', 0.066), ('supports', 0.065), ('association', 0.064), ('doddington', 0.062), ('sundheim', 0.062), ('mintz', 0.062), ('place', 0.06), ('knowledge', 0.059), ('inductive', 0.058), ('fader', 0.058), ('encyclopedic', 0.058), ('bollegala', 0.058), ('haghighi', 0.057), ('support', 0.056), ('prep', 0.056), ('yates', 0.056), ('death', 0.056), ('manual', 0.054), ('broadhead', 0.053), ('boxes', 0.053), ('poon', 0.053), ('aggregated', 0.053), ('mentions', 0.052), ('pages', 0.052), ('cafarella', 0.051), ('tac', 0.051), ('dependency', 0.051), ('wu', 0.05), ('titov', 0.049), ('volume', 0.048), ('generative', 0.048), ('bunescu', 0.048), ('ravichandran', 0.048), ('ww', 0.046), ('president', 0.045), ('dth', 0.045), ('population', 0.045), ('answering', 0.045), ('soderland', 0.044), ('collection', 0.043), ('textual', 0.043), ('etzioni', 0.042), ('annual', 0.042), ('bases', 0.041), ('world', 0.041), ('path', 0.04), ('syntax', 0.04), ('pereira', 0.039), ('connecting', 0.039), ('open', 0.039), ('disambiguation', 0.039), ('meeting', 0.037), ('million', 0.037), ('expressing', 0.037), ('strassel', 0.036), ('ach', 0.036), ('ars', 0.036), ('amershi', 0.036), ('fogarty', 0.036), ('patel', 0.036), ('bigham', 0.036)]
simIndex simValue paperId paperTitle
same-paper 1 1.0 159 acl-2012-Pattern Learning for Relation Extraction with a Hierarchical Topic Model
Author: Enrique Alfonseca ; Katja Filippova ; Jean-Yves Delort ; Guillermo Garrido
Abstract: We describe the use of a hierarchical topic model for automatically identifying syntactic and lexical patterns that explicitly state ontological relations. We leverage distant supervision using relations from the knowledge base FreeBase, but do not require any manual heuristic nor manual seed list selections. Results show that the learned patterns can be used to extract new relations with good precision.
2 0.33757812 40 acl-2012-Big Data versus the Crowd: Looking for Relationships in All the Right Places
Author: Ce Zhang ; Feng Niu ; Christopher Re ; Jude Shavlik
Abstract: Classically, training relation extractors relies on high-quality, manually annotated training data, which can be expensive to obtain. To mitigate this cost, NLU researchers have considered two newly available sources of less expensive (but potentially lower quality) labeled data from distant supervision and crowd sourcing. There is, however, no study comparing the relative impact of these two sources on the precision and recall of post-learning answers. To fill this gap, we empirically study how state-of-the-art techniques are affected by scaling these two sources. We use corpus sizes of up to 100 million documents and tens of thousands of crowd-source labeled examples. Our experiments show that increasing the corpus size for distant supervision has a statistically significant, positive impact on quality (F1 score). In contrast, human feedback has a positive and statistically significant, but lower, impact on precision and recall.
3 0.2901493 208 acl-2012-Unsupervised Relation Discovery with Sense Disambiguation
Author: Limin Yao ; Sebastian Riedel ; Andrew McCallum
Abstract: To discover relation types from text, most methods cluster shallow or syntactic patterns of relation mentions, but consider only one possible sense per pattern. In practice this assumption is often violated. In this paper we overcome this issue by inducing clusters of pattern senses from feature representations of patterns. In particular, we employ a topic model to partition entity pairs associated with patterns into sense clusters using local and global features. We merge these sense clusters into semantic relations using hierarchical agglomerative clustering. We compare against several baselines: a generative latent-variable model, a clustering method that does not disambiguate between path senses, and our own approach but with only local features. Experimental results show our proposed approach discovers dramatically more accurate clusters than models without sense disambiguation, and that incorporating global features, such as the document theme, is crucial.
4 0.27539513 169 acl-2012-Reducing Wrong Labels in Distant Supervision for Relation Extraction
Author: Shingo Takamatsu ; Issei Sato ; Hiroshi Nakagawa
Abstract: In relation extraction, distant supervision seeks to extract relations between entities from text by using a knowledge base, such as Freebase, as a source of supervision. When a sentence and a knowledge base refer to the same entity pair, this approach heuristically labels the sentence with the corresponding relation in the knowledge base. However, this heuristic can fail with the result that some sentences are labeled wrongly. This noisy labeled data causes poor extraction performance. In this paper, we propose a method to reduce the number of wrong labels. We present a novel generative model that directly models the heuristic labeling process of distant supervision. The model predicts whether assigned labels are correct or wrong via its hidden variables. Our experimental results show that this model detected wrong labels with higher performance than baseline methods. In the ex- periment, we also found that our wrong label reduction boosted the performance of relation extraction.
5 0.20879154 191 acl-2012-Temporally Anchored Relation Extraction
Author: Guillermo Garrido ; Anselmo Penas ; Bernardo Cabaleiro ; Alvaro Rodrigo
Abstract: Although much work on relation extraction has aimed at obtaining static facts, many of the target relations are actually fluents, as their validity is naturally anchored to a certain time period. This paper proposes a methodological approach to temporally anchored relation extraction. Our proposal performs distant supervised learning to extract a set of relations from a natural language corpus, and anchors each of them to an interval of temporal validity, aggregating evidence from documents supporting the relation. We use a rich graphbased document-level representation to generate novel features for this task. Results show that our implementation for temporal anchoring is able to achieve a 69% of the upper bound performance imposed by the relation extraction step. Compared to the state of the art, the overall system achieves the highest precision reported.
6 0.17135753 12 acl-2012-A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relation Extraction
7 0.1465584 73 acl-2012-Discriminative Learning for Joint Template Filling
8 0.1360528 201 acl-2012-Towards the Unsupervised Acquisition of Discourse Relations
9 0.13190705 142 acl-2012-Mining Entity Types from Query Logs via User Intent Modeling
10 0.12288015 60 acl-2012-Coupling Label Propagation and Constraints for Temporal Fact Extraction
11 0.11150713 22 acl-2012-A Topic Similarity Model for Hierarchical Phrase-based Translation
12 0.10736392 18 acl-2012-A Probabilistic Model for Canonicalizing Named Entity Mentions
13 0.10257109 153 acl-2012-Named Entity Disambiguation in Streaming Data
14 0.10177377 126 acl-2012-Labeling Documents with Timestamps: Learning from their Time Expressions
15 0.099857926 10 acl-2012-A Discriminative Hierarchical Model for Fast Coreference at Large Scale
16 0.097153813 61 acl-2012-Cross-Domain Co-Extraction of Sentiment and Topic Lexicons
17 0.095798977 150 acl-2012-Multilingual Named Entity Recognition using Parallel Data and Metadata from Wikipedia
18 0.093656875 177 acl-2012-Sentence Dependency Tagging in Online Question Answering Forums
19 0.085468024 5 acl-2012-A Comparison of Chinese Parsers for Stanford Dependencies
20 0.084368289 199 acl-2012-Topic Models for Dynamic Translation Model Adaptation
topicId topicWeight
[(0, -0.271), (1, 0.217), (2, -0.036), (3, 0.215), (4, -0.007), (5, 0.043), (6, -0.196), (7, 0.016), (8, 0.038), (9, -0.079), (10, 0.299), (11, -0.048), (12, -0.143), (13, -0.138), (14, 0.106), (15, 0.108), (16, -0.215), (17, -0.162), (18, 0.08), (19, 0.01), (20, 0.09), (21, -0.049), (22, -0.014), (23, -0.017), (24, -0.011), (25, -0.037), (26, 0.016), (27, 0.031), (28, 0.09), (29, -0.051), (30, 0.107), (31, -0.045), (32, -0.053), (33, -0.031), (34, 0.018), (35, -0.037), (36, 0.022), (37, -0.04), (38, 0.014), (39, 0.007), (40, 0.003), (41, -0.025), (42, -0.015), (43, 0.02), (44, -0.049), (45, 0.014), (46, 0.022), (47, -0.012), (48, 0.003), (49, 0.02)]
simIndex simValue paperId paperTitle
same-paper 1 0.96119636 159 acl-2012-Pattern Learning for Relation Extraction with a Hierarchical Topic Model
Author: Enrique Alfonseca ; Katja Filippova ; Jean-Yves Delort ; Guillermo Garrido
Abstract: We describe the use of a hierarchical topic model for automatically identifying syntactic and lexical patterns that explicitly state ontological relations. We leverage distant supervision using relations from the knowledge base FreeBase, but do not require any manual heuristic nor manual seed list selections. Results show that the learned patterns can be used to extract new relations with good precision.
2 0.90760285 169 acl-2012-Reducing Wrong Labels in Distant Supervision for Relation Extraction
Author: Shingo Takamatsu ; Issei Sato ; Hiroshi Nakagawa
Abstract: In relation extraction, distant supervision seeks to extract relations between entities from text by using a knowledge base, such as Freebase, as a source of supervision. When a sentence and a knowledge base refer to the same entity pair, this approach heuristically labels the sentence with the corresponding relation in the knowledge base. However, this heuristic can fail with the result that some sentences are labeled wrongly. This noisy labeled data causes poor extraction performance. In this paper, we propose a method to reduce the number of wrong labels. We present a novel generative model that directly models the heuristic labeling process of distant supervision. The model predicts whether assigned labels are correct or wrong via its hidden variables. Our experimental results show that this model detected wrong labels with higher performance than baseline methods. In the ex- periment, we also found that our wrong label reduction boosted the performance of relation extraction.
3 0.8932789 40 acl-2012-Big Data versus the Crowd: Looking for Relationships in All the Right Places
Author: Ce Zhang ; Feng Niu ; Christopher Re ; Jude Shavlik
Abstract: Classically, training relation extractors relies on high-quality, manually annotated training data, which can be expensive to obtain. To mitigate this cost, NLU researchers have considered two newly available sources of less expensive (but potentially lower quality) labeled data from distant supervision and crowd sourcing. There is, however, no study comparing the relative impact of these two sources on the precision and recall of post-learning answers. To fill this gap, we empirically study how state-of-the-art techniques are affected by scaling these two sources. We use corpus sizes of up to 100 million documents and tens of thousands of crowd-source labeled examples. Our experiments show that increasing the corpus size for distant supervision has a statistically significant, positive impact on quality (F1 score). In contrast, human feedback has a positive and statistically significant, but lower, impact on precision and recall.
4 0.72277927 208 acl-2012-Unsupervised Relation Discovery with Sense Disambiguation
Author: Limin Yao ; Sebastian Riedel ; Andrew McCallum
Abstract: To discover relation types from text, most methods cluster shallow or syntactic patterns of relation mentions, but consider only one possible sense per pattern. In practice this assumption is often violated. In this paper we overcome this issue by inducing clusters of pattern senses from feature representations of patterns. In particular, we employ a topic model to partition entity pairs associated with patterns into sense clusters using local and global features. We merge these sense clusters into semantic relations using hierarchical agglomerative clustering. We compare against several baselines: a generative latent-variable model, a clustering method that does not disambiguate between path senses, and our own approach but with only local features. Experimental results show our proposed approach discovers dramatically more accurate clusters than models without sense disambiguation, and that incorporating global features, such as the document theme, is crucial.
5 0.59056896 73 acl-2012-Discriminative Learning for Joint Template Filling
Author: Einat Minkov ; Luke Zettlemoyer
Abstract: This paper presents a joint model for template filling, where the goal is to automatically specify the fields of target relations such as seminar announcements or corporate acquisition events. The approach models mention detection, unification and field extraction in a flexible, feature-rich model that allows for joint modeling of interdependencies at all levels and across fields. Such an approach can, for example, learn likely event durations and the fact that start times should come before end times. While the joint inference space is large, we demonstrate effective learning with a Perceptron-style approach that uses simple, greedy beam decoding. Empirical results in two benchmark domains demonstrate consistently strong performance on both mention de- tection and template filling tasks.
6 0.56251138 191 acl-2012-Temporally Anchored Relation Extraction
7 0.54264158 12 acl-2012-A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relation Extraction
8 0.49344873 129 acl-2012-Learning High-Level Planning from Text
9 0.49090424 133 acl-2012-Learning to "Read Between the Lines" using Bayesian Logic Programs
10 0.44532663 60 acl-2012-Coupling Label Propagation and Constraints for Temporal Fact Extraction
11 0.44064265 201 acl-2012-Towards the Unsupervised Acquisition of Discourse Relations
12 0.43414435 153 acl-2012-Named Entity Disambiguation in Streaming Data
13 0.41185129 142 acl-2012-Mining Entity Types from Query Logs via User Intent Modeling
14 0.40886202 14 acl-2012-A Joint Model for Discovery of Aspects in Utterances
15 0.40540618 18 acl-2012-A Probabilistic Model for Canonicalizing Named Entity Mentions
16 0.40313122 126 acl-2012-Labeling Documents with Timestamps: Learning from their Time Expressions
17 0.35012934 6 acl-2012-A Comprehensive Gold Standard for the Enron Organizational Hierarchy
18 0.34983182 124 acl-2012-Joint Inference of Named Entity Recognition and Normalization for Tweets
19 0.34370583 10 acl-2012-A Discriminative Hierarchical Model for Fast Coreference at Large Scale
20 0.3377603 150 acl-2012-Multilingual Named Entity Recognition using Parallel Data and Metadata from Wikipedia
topicId topicWeight
[(25, 0.025), (26, 0.054), (28, 0.026), (30, 0.027), (37, 0.032), (39, 0.086), (59, 0.021), (74, 0.015), (82, 0.026), (84, 0.025), (85, 0.024), (90, 0.119), (92, 0.081), (94, 0.016), (96, 0.22), (99, 0.144)]
simIndex simValue paperId paperTitle
1 0.85626453 50 acl-2012-Collective Classification for Fine-grained Information Status
Author: Katja Markert ; Yufang Hou ; Michael Strube
Abstract: Previous work on classifying information status (Nissim, 2006; Rahman and Ng, 2011) is restricted to coarse-grained classification and focuses on conversational dialogue. We here introduce the task of classifying finegrained information status and work on written text. We add a fine-grained information status layer to the Wall Street Journal portion of the OntoNotes corpus. We claim that the information status of a mention depends not only on the mention itself but also on other mentions in the vicinity and solve the task by collectively classifying the information status ofall mentions. Our approach strongly outperforms reimplementations of previous work.
same-paper 2 0.80552661 159 acl-2012-Pattern Learning for Relation Extraction with a Hierarchical Topic Model
Author: Enrique Alfonseca ; Katja Filippova ; Jean-Yves Delort ; Guillermo Garrido
Abstract: We describe the use of a hierarchical topic model for automatically identifying syntactic and lexical patterns that explicitly state ontological relations. We leverage distant supervision using relations from the knowledge base FreeBase, but do not require any manual heuristic nor manual seed list selections. Results show that the learned patterns can be used to extract new relations with good precision.
3 0.66657388 191 acl-2012-Temporally Anchored Relation Extraction
Author: Guillermo Garrido ; Anselmo Penas ; Bernardo Cabaleiro ; Alvaro Rodrigo
Abstract: Although much work on relation extraction has aimed at obtaining static facts, many of the target relations are actually fluents, as their validity is naturally anchored to a certain time period. This paper proposes a methodological approach to temporally anchored relation extraction. Our proposal performs distant supervised learning to extract a set of relations from a natural language corpus, and anchors each of them to an interval of temporal validity, aggregating evidence from documents supporting the relation. We use a rich graphbased document-level representation to generate novel features for this task. Results show that our implementation for temporal anchoring is able to achieve a 69% of the upper bound performance imposed by the relation extraction step. Compared to the state of the art, the overall system achieves the highest precision reported.
4 0.66590106 170 acl-2012-Robust Conversion of CCG Derivations to Phrase Structure Trees
Author: Jonathan K. Kummerfeld ; Dan Klein ; James R. Curran
Abstract: We propose an improved, bottom-up method for converting CCG derivations into PTB-style phrase structure trees. In contrast with past work (Clark and Curran, 2009), which used simple transductions on category pairs, our approach uses richer transductions attached to single categories. Our conversion preserves more sentences under round-trip conversion (5 1.1% vs. 39.6%) and is more robust. In particular, unlike past methods, ours does not require ad-hoc rules over non-local features, and so can be easily integrated into a parser.
5 0.66509247 101 acl-2012-Fully Abstractive Approach to Guided Summarization
Author: Pierre-Etienne Genest ; Guy Lapalme
Abstract: This paper shows that full abstraction can be accomplished in the context of guided summarization. We describe a work in progress that relies on Information Extraction, statistical content selection and Natural Language Generation. Early results already demonstrate the effectiveness of the approach.
6 0.65832889 21 acl-2012-A System for Real-time Twitter Sentiment Analysis of 2012 U.S. Presidential Election Cycle
7 0.65677136 84 acl-2012-Estimating Compact Yet Rich Tree Insertion Grammars
8 0.65674305 53 acl-2012-Combining Textual Entailment and Argumentation Theory for Supporting Online Debates Interactions
9 0.65572923 40 acl-2012-Big Data versus the Crowd: Looking for Relationships in All the Right Places
10 0.65493912 29 acl-2012-Assessing the Effect of Inconsistent Assessors on Summarization Evaluation
11 0.65428227 149 acl-2012-Movie-DiC: a Movie Dialogue Corpus for Research and Development
12 0.64703923 206 acl-2012-UWN: A Large Multilingual Lexical Knowledge Base
14 0.6393255 62 acl-2012-Cross-Lingual Mixture Model for Sentiment Classification
16 0.63334352 28 acl-2012-Aspect Extraction through Semi-Supervised Modeling
17 0.63016146 187 acl-2012-Subgroup Detection in Ideological Discussions
18 0.62945175 139 acl-2012-MIX Is Not a Tree-Adjoining Language
19 0.62754875 167 acl-2012-QuickView: NLP-based Tweet Search
20 0.62595034 156 acl-2012-Online Plagiarized Detection Through Exploiting Lexical, Syntax, and Semantic Information