acl acl2011 acl2011-294 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Naushad UzZaman ; James Allen
Abstract: In this paper we propose a new method for evaluating systems that extract temporal information from text. It uses temporal closure1 to reward relations that are equivalent but distinct. Our metric measures the overall performance of systems with a single score, making comparison between different systems straightforward. Our approach is easy to implement, intuitive, accurate, scalable and computationally inexpensive. 1
Reference: text
sentIndex sentText sentNum sentScore
1 Allen Computer Science Department University of Rochester Rochester, NY, USA { naushad, j ame s } @ cs . [sent-2, score-0.138]
2 edu Abstract In this paper we propose a new method for evaluating systems that extract temporal information from text. [sent-4, score-0.73]
3 It uses temporal closure1 to reward relations that are equivalent but distinct. [sent-5, score-0.874]
4 Our metric measures the overall performance of systems with a single score, making comparison between different systems straightforward. [sent-6, score-0.248]
5 Our approach is easy to implement, intuitive, accurate, scalable and computationally inexpensive. [sent-7, score-0.148]
6 1 Introduction The recent emergence of language processing applications like question answering, information extraction, and document summarization has motivated the need for temporally-aware systems. [sent-8, score-0.305]
7 This, along with the availability of the temporal annotation scheme TimeML (Pustejovsky et al. [sent-9, score-0.77]
8 , 2003), a temporally annotated corpus, TimeBank (Pustejovsky et al. [sent-10, score-0.12]
9 , 2003) and the temporal evaluation challenges TempEval-1 (Verhagen et al. [sent-11, score-0.674]
10 , 2007) and TempEval-2 (Pustejovsky and Verhagen, 2010), has led to an explosion of research on temporal information processing (TIP). [sent-12, score-0.78]
11 Prior evaluation methods (TempEval-1 , 2) for different TIP subtasks have borrowed precision and recall measures from the information retrieval community. [sent-13, score-0.364]
12 This has two problems: First, systems express temporal relations in different, yet equivalent, ways. [sent-14, score-0.821]
13 Consider a scenario where the 1 Temporal closure is a reasoning mechanism that derives new implied temporal relations, i. [sent-15, score-1.21]
14 For example, if we know A before B, B before C, then using temporal closure we can derive A before C. [sent-18, score-0.958]
15 Allen (1983) demonstrates the closure table for 13 Allen interval relations. [sent-19, score-0.391]
wordName wordTfidf (topN-words)
[('temporal', 0.61), ('pustejovsky', 0.308), ('naushad', 0.269), ('closure', 0.269), ('allen', 0.238), ('tip', 0.237), ('verhagen', 0.237), ('rochester', 0.123), ('roche', 0.119), ('timebank', 0.119), ('borrowed', 0.109), ('relations', 0.108), ('ame', 0.103), ('temporally', 0.103), ('emergence', 0.093), ('subtasks', 0.082), ('derives', 0.079), ('explosion', 0.079), ('reward', 0.075), ('equivalent', 0.071), ('interval', 0.071), ('implied', 0.069), ('reasoning', 0.063), ('led', 0.063), ('metric', 0.063), ('computationally', 0.058), ('mechanism', 0.057), ('availability', 0.055), ('scalable', 0.055), ('ter', 0.055), ('scenario', 0.055), ('answering', 0.051), ('intuitive', 0.05), ('measures', 0.05), ('identifies', 0.048), ('demonstrates', 0.047), ('implicit', 0.046), ('derive', 0.045), ('annotation', 0.045), ('ny', 0.042), ('express', 0.042), ('implement', 0.041), ('summarization', 0.04), ('challenges', 0.039), ('motivated', 0.037), ('james', 0.035), ('cs', 0.035), ('traditional', 0.035), ('know', 0.034), ('scheme', 0.033), ('retrieval', 0.033), ('yet', 0.033), ('easy', 0.032), ('evaluating', 0.03), ('usa', 0.03), ('reference', 0.03), ('systems', 0.028), ('prior', 0.028), ('along', 0.027), ('question', 0.026), ('evaluation', 0.025), ('accurate', 0.024), ('document', 0.024), ('making', 0.024), ('department', 0.022), ('problems', 0.022), ('precision', 0.022), ('recent', 0.021), ('recall', 0.021), ('extract', 0.02), ('applications', 0.02), ('makes', 0.019), ('extraction', 0.018), ('annotated', 0.017), ('overall', 0.017), ('propose', 0.016), ('comparison', 0.013), ('processing', 0.013), ('consider', 0.013), ('contains', 0.012), ('computer', 0.012), ('score', 0.012), ('need', 0.012), ('science', 0.012), ('single', 0.011), ('information', 0.01), ('uses', 0.01), ('like', 0.009), ('new', 0.008), ('different', 0.007), ('performance', 0.007), ('system', 0.006), ('corpus', 0.006), ('method', 0.005), ('methods', 0.005), ('research', 0.005), ('university', 0.004), ('table', 0.004), ('approach', 0.003), ('paper', 0.003)]
simIndex simValue paperId paperTitle
same-paper 1 1.0 294 acl-2011-Temporal Evaluation
Author: Naushad UzZaman ; James Allen
Abstract: In this paper we propose a new method for evaluating systems that extract temporal information from text. It uses temporal closure1 to reward relations that are equivalent but distinct. Our metric measures the overall performance of systems with a single score, making comparison between different systems straightforward. Our approach is easy to implement, intuitive, accurate, scalable and computationally inexpensive. 1
2 0.22941074 138 acl-2011-French TimeBank: An ISO-TimeML Annotated Reference Corpus
Author: Andre Bittar ; Pascal Amsili ; Pascal Denis ; Laurence Danlos
Abstract: This article presents the main points in the creation of the French TimeBank (Bittar, 2010), a reference corpus annotated according to the ISO-TimeML standard for temporal annotation. A number of improvements were made to the markup language to deal with linguistic phenomena not yet covered by ISO-TimeML, including cross-language modifications and others specific to French. An automatic preannotation system was used to speed up the annotation process. A preliminary evaluation of the methodology adopted for this project yields positive results in terms of data quality and annotation time.
3 0.085948862 96 acl-2011-Disambiguating temporal-contrastive connectives for machine translation
Author: Thomas Meyer
Abstract: Temporal–contrastive discourse connectives (although, while, since, etc.) signal various types ofrelations between clauses such as temporal, contrast, concession and cause. They are often ambiguous and therefore difficult to translate from one language to another. We discuss several new and translation-oriented experiments for the disambiguation of a specific subset of discourse connectives in order to correct some of the translation errors made by current statistical machine translation systems.
4 0.080304563 222 acl-2011-Model-Portability Experiments for Textual Temporal Analysis
Author: Oleksandr Kolomiyets ; Steven Bethard ; Marie-Francine Moens
Abstract: We explore a semi-supervised approach for improving the portability of time expression recognition to non-newswire domains: we generate additional training examples by substituting temporal expression words with potential synonyms. We explore using synonyms both from WordNet and from the Latent Words Language Model (LWLM), which predicts synonyms in context using an unsupervised approach. We evaluate a state-of-the-art time expression recognition system trained both with and without the additional training examples using data from TempEval 2010, Reuters and Wikipedia. We find that the LWLM provides substantial improvements on the Reuters corpus, and smaller improvements on the Wikipedia corpus. We find that WordNet alone never improves performance, though intersecting the examples from the LWLM and WordNet provides more stable results for Wikipedia. 1
5 0.07536862 295 acl-2011-Temporal Restricted Boltzmann Machines for Dependency Parsing
Author: Nikhil Garg ; James Henderson
Abstract: We propose a generative model based on Temporal Restricted Boltzmann Machines for transition based dependency parsing. The parse tree is built incrementally using a shiftreduce parse and an RBM is used to model each decision step. The RBM at the current time step induces latent features with the help of temporal connections to the relevant previous steps which provide context information. Our parser achieves labeled and unlabeled attachment scores of 88.72% and 91.65% respectively, which compare well with similar previous models and the state-of-the-art.
6 0.054389436 322 acl-2011-Unsupervised Learning of Semantic Relation Composition
7 0.046855159 58 acl-2011-Beam-Width Prediction for Efficient Context-Free Parsing
8 0.043808684 271 acl-2011-Search in the Lost Sense of "Query": Question Formulation in Web Search Queries and its Temporal Changes
9 0.036775015 215 acl-2011-MACAON An NLP Tool Suite for Processing Word Lattices
10 0.036013659 8 acl-2011-A Corpus of Scope-disambiguated English Text
11 0.035045039 53 acl-2011-Automatically Evaluating Text Coherence Using Discourse Relations
12 0.034333687 156 acl-2011-IMASS: An Intelligent Microblog Analysis and Summarization System
13 0.034061901 114 acl-2011-End-to-End Relation Extraction Using Distant Supervision from External Semantic Repositories
14 0.032435991 307 acl-2011-Towards Tracking Semantic Change by Visual Analytics
15 0.030941533 4 acl-2011-A Class of Submodular Functions for Document Summarization
16 0.029978253 94 acl-2011-Deciphering Foreign Language
17 0.029170461 190 acl-2011-Knowledge-Based Weak Supervision for Information Extraction of Overlapping Relations
18 0.028861487 149 acl-2011-Hierarchical Reinforcement Learning and Hidden Markov Models for Task-Oriented Natural Language Generation
19 0.027932072 25 acl-2011-A Simple Measure to Assess Non-response
20 0.026781054 262 acl-2011-Relation Guided Bootstrapping of Semantic Lexicons
topicId topicWeight
[(0, 0.058), (1, 0.018), (2, -0.03), (3, 0.021), (4, -0.012), (5, 0.004), (6, 0.001), (7, 0.002), (8, -0.003), (9, -0.028), (10, 0.018), (11, -0.033), (12, 0.005), (13, 0.004), (14, -0.065), (15, -0.031), (16, 0.026), (17, -0.016), (18, 0.005), (19, -0.006), (20, 0.04), (21, 0.061), (22, -0.001), (23, -0.028), (24, 0.01), (25, 0.008), (26, -0.0), (27, -0.02), (28, 0.009), (29, 0.002), (30, -0.047), (31, 0.073), (32, 0.065), (33, -0.011), (34, -0.006), (35, 0.002), (36, -0.163), (37, 0.046), (38, 0.093), (39, -0.123), (40, -0.042), (41, -0.005), (42, 0.008), (43, -0.126), (44, -0.09), (45, -0.013), (46, -0.008), (47, 0.003), (48, 0.007), (49, -0.039)]
simIndex simValue paperId paperTitle
same-paper 1 0.9650169 294 acl-2011-Temporal Evaluation
Author: Naushad UzZaman ; James Allen
Abstract: In this paper we propose a new method for evaluating systems that extract temporal information from text. It uses temporal closure1 to reward relations that are equivalent but distinct. Our metric measures the overall performance of systems with a single score, making comparison between different systems straightforward. Our approach is easy to implement, intuitive, accurate, scalable and computationally inexpensive. 1
2 0.76995683 138 acl-2011-French TimeBank: An ISO-TimeML Annotated Reference Corpus
Author: Andre Bittar ; Pascal Amsili ; Pascal Denis ; Laurence Danlos
Abstract: This article presents the main points in the creation of the French TimeBank (Bittar, 2010), a reference corpus annotated according to the ISO-TimeML standard for temporal annotation. A number of improvements were made to the markup language to deal with linguistic phenomena not yet covered by ISO-TimeML, including cross-language modifications and others specific to French. An automatic preannotation system was used to speed up the annotation process. A preliminary evaluation of the methodology adopted for this project yields positive results in terms of data quality and annotation time.
3 0.5201602 322 acl-2011-Unsupervised Learning of Semantic Relation Composition
Author: Eduardo Blanco ; Dan Moldovan
Abstract: This paper presents an unsupervised method for deriving inference axioms by composing semantic relations. The method is independent of any particular relation inventory. It relies on describing semantic relations using primitives and manipulating these primitives according to an algebra. The method was tested using a set of eight semantic relations yielding 78 inference axioms which were evaluated over PropBank.
4 0.48776808 96 acl-2011-Disambiguating temporal-contrastive connectives for machine translation
Author: Thomas Meyer
Abstract: Temporal–contrastive discourse connectives (although, while, since, etc.) signal various types ofrelations between clauses such as temporal, contrast, concession and cause. They are often ambiguous and therefore difficult to translate from one language to another. We discuss several new and translation-oriented experiments for the disambiguation of a specific subset of discourse connectives in order to correct some of the translation errors made by current statistical machine translation systems.
5 0.44673994 53 acl-2011-Automatically Evaluating Text Coherence Using Discourse Relations
Author: Ziheng Lin ; Hwee Tou Ng ; Min-Yen Kan
Abstract: We present a novel model to represent and assess the discourse coherence of text. Our model assumes that coherent text implicitly favors certain types of discourse relation transitions. We implement this model and apply it towards the text ordering ranking task, which aims to discern an original text from a permuted ordering of its sentences. The experimental results demonstrate that our model is able to significantly outperform the state-ofthe-art coherence model by Barzilay and Lapata (2005), reducing the error rate of the previous approach by an average of 29% over three data sets against human upperbounds. We further show that our model is synergistic with the previous approach, demonstrating an error reduction of 73% when the features from both models are combined for the task.
6 0.39068183 200 acl-2011-Learning Dependency-Based Compositional Semantics
7 0.37482488 8 acl-2011-A Corpus of Scope-disambiguated English Text
8 0.35773778 114 acl-2011-End-to-End Relation Extraction Using Distant Supervision from External Semantic Repositories
9 0.34828338 190 acl-2011-Knowledge-Based Weak Supervision for Information Extraction of Overlapping Relations
10 0.34703836 170 acl-2011-In-domain Relation Discovery with Meta-constraints via Posterior Regularization
11 0.33585718 262 acl-2011-Relation Guided Bootstrapping of Semantic Lexicons
12 0.33398563 288 acl-2011-Subjective Natural Language Problems: Motivations, Applications, Characterizations, and Implications
13 0.33064368 222 acl-2011-Model-Portability Experiments for Textual Temporal Analysis
14 0.31910825 273 acl-2011-Semantic Representation of Negation Using Focus Detection
15 0.31810853 215 acl-2011-MACAON An NLP Tool Suite for Processing Word Lattices
16 0.31317985 317 acl-2011-Underspecifying and Predicting Voice for Surface Realisation Ranking
17 0.29678497 25 acl-2011-A Simple Measure to Assess Non-response
18 0.29414779 149 acl-2011-Hierarchical Reinforcement Learning and Hidden Markov Models for Task-Oriented Natural Language Generation
19 0.28126192 207 acl-2011-Learning to Win by Reading Manuals in a Monte-Carlo Framework
20 0.27674642 120 acl-2011-Even the Abstract have Color: Consensus in Word-Colour Associations
topicId topicWeight
[(17, 0.04), (37, 0.024), (39, 0.029), (41, 0.055), (59, 0.057), (72, 0.017), (93, 0.496), (96, 0.144)]
simIndex simValue paperId paperTitle
same-paper 1 0.71064466 294 acl-2011-Temporal Evaluation
Author: Naushad UzZaman ; James Allen
Abstract: In this paper we propose a new method for evaluating systems that extract temporal information from text. It uses temporal closure1 to reward relations that are equivalent but distinct. Our metric measures the overall performance of systems with a single score, making comparison between different systems straightforward. Our approach is easy to implement, intuitive, accurate, scalable and computationally inexpensive. 1
2 0.55628449 8 acl-2011-A Corpus of Scope-disambiguated English Text
Author: Mehdi Manshadi ; James Allen ; Mary Swift
Abstract: Previous work on quantifier scope annotation focuses on scoping sentences with only two quantified noun phrases (NPs), where the quantifiers are restricted to a predefined list. It also ignores negation, modal/logical operators, and other sentential adverbials. We present a comprehensive scope annotation scheme. We annotate the scope interaction between all scopal terms in the sentence from quantifiers to scopal adverbials, without putting any restriction on the number of scopal terms in a sentence. In addition, all NPs, explicitly quantified or not, with no restriction on the type of quantification, are investigated for possible scope interactions. 1
3 0.5495137 191 acl-2011-Knowledge Base Population: Successful Approaches and Challenges
Author: Heng Ji ; Ralph Grishman
Abstract: In this paper we give an overview of the Knowledge Base Population (KBP) track at the 2010 Text Analysis Conference. The main goal of KBP is to promote research in discovering facts about entities and augmenting a knowledge base (KB) with these facts. This is done through two tasks, Entity Linking linking names in context to entities in the KB and Slot Filling – adding information about an entity to the KB. A large source collection of newswire and web documents is provided from which systems are to discover information. Attributes (“slots”) derived from Wikipedia infoboxes are used to create the reference KB. In this paper we provide an overview of the techniques which can serve as a basis for a good KBP system, lay out the – – remaining challenges by comparison with traditional Information Extraction (IE) and Question Answering (QA) tasks, and provide some suggestions to address these challenges. 1
4 0.31604898 51 acl-2011-Automatic Headline Generation using Character Cross-Correlation
Author: Fahad Alotaiby
Abstract: Arabic language is a morphologically complex language. Affixes and clitics are regularly attached to stems which make direct comparison between words not practical. In this paper we propose a new automatic headline generation technique that utilizes character cross-correlation to extract best headlines and to overcome the Arabic language complex morphology. The system that uses character cross-correlation achieves ROUGE-L score of 0. 19384 while the exact word matching scores only 0. 17252 for the same set of documents. 1
5 0.3144165 57 acl-2011-Bayesian Word Alignment for Statistical Machine Translation
Author: Coskun Mermer ; Murat Saraclar
Abstract: In this work, we compare the translation performance of word alignments obtained via Bayesian inference to those obtained via expectation-maximization (EM). We propose a Gibbs sampler for fully Bayesian inference in IBM Model 1, integrating over all possible parameter values in finding the alignment distribution. We show that Bayesian inference outperforms EM in all of the tested language pairs, domains and data set sizes, by up to 2.99 BLEU points. We also show that the proposed method effectively addresses the well-known rare word problem in EM-estimated models; and at the same time induces a much smaller dictionary of bilingual word-pairs. .t r
6 0.31265962 244 acl-2011-Peeling Back the Layers: Detecting Event Role Fillers in Secondary Contexts
7 0.31241015 251 acl-2011-Probabilistic Document Modeling for Syntax Removal in Text Summarization
8 0.31165409 4 acl-2011-A Class of Submodular Functions for Document Summarization
9 0.31148458 171 acl-2011-Incremental Syntactic Language Models for Phrase-based Translation
10 0.31076759 3 acl-2011-A Bayesian Model for Unsupervised Semantic Parsing
11 0.31062543 37 acl-2011-An Empirical Evaluation of Data-Driven Paraphrase Generation Techniques
12 0.31039062 11 acl-2011-A Fast and Accurate Method for Approximate String Search
13 0.31036454 26 acl-2011-A Speech-based Just-in-Time Retrieval System using Semantic Search
14 0.30962622 110 acl-2011-Effective Use of Function Words for Rule Generalization in Forest-Based Translation
15 0.30957302 94 acl-2011-Deciphering Foreign Language
16 0.30948013 308 acl-2011-Towards a Framework for Abstractive Summarization of Multimodal Documents
17 0.30921322 174 acl-2011-Insights from Network Structure for Text Mining
18 0.30904758 61 acl-2011-Binarized Forest to String Translation
19 0.30877203 324 acl-2011-Unsupervised Semantic Role Induction via Split-Merge Clustering
20 0.30866358 15 acl-2011-A Hierarchical Pitman-Yor Process HMM for Unsupervised Part of Speech Induction