emnlp emnlp2011 emnlp2011-135 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Rui Yan ; Liang Kong ; Congrui Huang ; Xiaojun Wan ; Xiaoming Li ; Yan Zhang
Abstract: We investigate an important and challenging problem in summary generation, i.e., Evolutionary Trans-Temporal Summarization (ETTS), which generates news timelines from massive data on the Internet. ETTS greatly facilitates fast news browsing and knowledge comprehension, and hence is a necessity. Given the collection oftime-stamped web documents related to the evolving news, ETTS aims to return news evolution along the timeline, consisting of individual but correlated summaries on each date. Existing summarization algorithms fail to utilize trans-temporal characteristics among these component summaries. We propose to model trans-temporal correlations among component summaries for timelines, using inter-date and intra-date sen- tence dependencies, and present a novel combination. We develop experimental systems to compare 5 rival algorithms on 6 instinctively different datasets which amount to 10251 documents. Evaluation results in ROUGE metrics indicate the effectiveness of the proposed approach based on trans-temporal information. 1
Reference: text
sentIndex sentText sentNum sentScore
1 , Evolutionary Trans-Temporal Summarization (ETTS), which generates news timelines from massive data on the Internet. [sent-9, score-0.507]
2 Given the collection oftime-stamped web documents related to the evolving news, ETTS aims to return news evolution along the timeline, consisting of individual but correlated summaries on each date. [sent-11, score-0.361]
3 Existing summarization algorithms fail to utilize trans-temporal characteristics among these component summaries. [sent-12, score-0.318]
4 We propose to model trans-temporal correlations among component summaries for timelines, using inter-date and intra-date sen- tence dependencies, and present a novel combination. [sent-13, score-0.235]
5 cn news webpages by relevance to a user specified aspect, i. [sent-23, score-0.212]
6 , a query such as “first relief effort for BP Oil Spill”, but search engines are not quite capable of ranking documents given the whole news subject without particular aspects. [sent-25, score-0.222]
7 However, traditional information retrieval techniques can only rank webpages according to their understanding of relevance, which is obviously insufficient (Jin et al. [sent-27, score-0.189]
8 Even if the ranked documents could be in a satisfying order to help users understand news evolution, readers prefer to monitor the evolutionary trajecto- ries by simply browsing rather than navigate every document in the overwhelming collection. [sent-29, score-0.297]
9 Particularly, a timeline (see Table 1) can summarize evolutionary news as a series of individual but correlated component summaries (items in Table 1) and offer an option to understand the big picture of evolution. [sent-31, score-0.841]
10 With unique characteristics, summarizing timelines is significantly different from traditional summarization methods which are awkward in such scenarios. [sent-32, score-0.67]
11 We first study a manual timeline of BP Oil Along with the rapid growth of the World Wide Web, document floods spread throughout the Internet. [sent-33, score-0.28]
12 Given a large document collection related to a news subject (for example, BP Oil Spill), readers get lost in the sea of articles, feeling confused and powerless. [sent-34, score-0.18]
13 General search engines can rank these Spill in Mexico Gulf in Table 1from Reuters News1 to understand why timelines generation is observably different from traditional summarization. [sent-35, score-0.477]
14 c 201 1Association for Linguistics Table 1: Part of human generated timeline about BP Oil Spill in 2010 from Reuters News website. [sent-42, score-0.28]
15 osnegdw,uilntbetahleocwaeudseinofetwheaDrease,pawsther temporal characteristics of component summaries from the handcrafted timeline. [sent-56, score-0.356]
16 The component summaries are summarized locally: the component item on date t is constituted by sentences with timestamp t. [sent-58, score-0.574]
17 The component summaries are correlative across dates, based on the global collection. [sent-60, score-0.378]
18 To the best of our knowledge, no traditional method has examined the rela- tionships among these timeline items. [sent-61, score-0.309]
19 Taking a collection relevant to a news subject as input, the system automatically outputs a timeline with items of component summaries 434 which represent evolutionary trajectories on specific dates. [sent-66, score-0.854]
20 Particularly, the inter-date dependency calculation includes temporal decays to project sentences from all dates onto the same time horizon (Figure 1 (a)). [sent-68, score-0.295]
21 Based on intra/inter-date sentence dependencies, we then model affinity and diversity to compute the saliency score of each sentence and merge local and global rankings into one unified ranking framework. [sent-69, score-0.464]
22 2 Related Work Multi-document summarization (MDS) aims to produce a summary delivering the majority of information content from a set of documents and has drawn much attention in recent years. [sent-72, score-0.28]
23 However, update summarization only dealt with a single update and we make a novel contribution with multi-step evolutionary updates. [sent-97, score-0.457]
24 Further related work includes similar timeline systems proposed by (Swan and Allan, 2000) using named entities, by (Allan et al. [sent-98, score-0.28]
25 We have proposed a timeline algorithm named “Evolutionary Timeline Summarization (ETS)” in (Yan et al. [sent-100, score-0.28]
26 , 2011b) but the refining process based on generated component summaries is time consuming. [sent-101, score-0.235]
27 To the best of our knowledge, neither update summarization nor traditional systems have considered the relationship among “component summaries”, or have utilized trans-temporal properties. [sent-103, score-0.294]
28 ETTS approach can also naturally and simultaneously take into account global/local summarization with biased information richness and information novelty, and combine both summarization in optimization. [sent-104, score-0.642]
29 3 Trans-temporal Summarization We conduct trans-temporal summarization based on the global biased graph using inter-date dependency and local biased graph using intra-date dependency. [sent-105, score-0.766]
30 1 Global Biased Summarization The intuition for global biased summarization is that the selected summary should be correlative with sentences from neighboring dates, especially with those informative ones. [sent-108, score-0.685]
31 To generate the component summary on date t, we project all sentences in the collection onto the time horizon of t to construct a global affinity graph, using temporal decaying kernels. [sent-109, score-0.716]
32 1 Temporal Proximity Based Projection Clearly, a major technical challenge in ETTS is how to define the temporal biased projection function Γ(∆t), where ∆t is the distance between the 435 Figure 1: Construct global/local biased graphs. [sent-112, score-0.53]
33 Solid circles denote intra-date sentences on the pending date t and dash ones represent inter-date sentences from other dates. [sent-113, score-0.231]
34 Window kernel Γ(∆t) =(01 iofth ∆erwt ≤ise σ All kernels have one parameter σ to tune, which controls the spread of kernel curves, i. [sent-125, score-0.24]
35 In general, the optimal setting of σ may vary according to the news set because sentences presumably would have wider semantic scope in certain news subjects, thus requiring a higher value of σ and vice versa. [sent-128, score-0.312]
36 , C|T|}, we obttaimine Cstat = {sti |1T Ci {|CCt |} where si is a s,e wneten ocbewith the= timestamp ti = tsi . [sent-134, score-0.194]
37 W} whehne we generate component summary on t, we project all sentences onto time horizon t. [sent-135, score-0.225]
38 We use an affinity ≤ ≤ matrix Mt with the entry of the inter-date transition probability on date t. [sent-137, score-0.257]
39 Note that for the global biased matrix, we measure the affinity between local sentences from t and global sentences from other dates. [sent-139, score-0.634]
40 Therefore, intradate transition probability between sentences with the timestamp t is set to 0 for local summarization. [sent-140, score-0.203]
41 Mit,j is the transition probability of si to sj based on the perspective of date t, i. [sent-141, score-0.311]
42 ) 0is a symmetric function, p(si → sj |t) is usually not equal to p(sj → si |t), depending on ti she u degrees ooft enqoudaels si ap(nds sj. [sent-151, score-0.288]
43 3 Modeling Diversity Diversity is to reflect both biased information richness and sentence novelty, which aims to reduce information redundancy. [sent-159, score-0.178]
44 Most recently diversity rank DivRank is another solution to diversity penalization in (Mei et al. [sent-164, score-0.189]
45 By incorporating DivRank, we obtain rank ri† and the global biased ranking score Gi for sentence si from date t to summarize Ct. [sent-170, score-0.667]
46 2 Local Biased Summarization Naturally, the component summary for date t should be informative within Ct. [sent-172, score-0.256]
47 Given the sentence collection Ct = {sti | 1 ≤ i ≤ |Ct |}, we build an affinity matrix f=or Figure ≤1 i (b), Cwit|}h, t whee entry aonf ia nftfrian-date transition probability calculated from standard cosine similarity. [sent-173, score-0.211]
48 We incorporate DivRank within local summarization and we obtain the local biased rank and ranking score for si, denoted as and Li. [sent-174, score-0.687]
49 3 Optimization of Global/Local Combination We do not directly add the global biased ranking score and local biased ranking score, as many previous works did (Wan et al. [sent-176, score-0.702]
50 , 2007a), because even the same ranking score gap may indicate different rank gaps in two ranking lists. [sent-178, score-0.217]
51 ,|Ct |), ri is the final ranking oleft si t=o estimate, optimize ,th re following objective cost function O(R), O(R)=α+i|XC=β1t|XiC=G1ti|kLΨrikΨ−riG−i†kLr2i‡k2 (6) where Gi is the global biased ranking score while Li iws htheree l Gocails bthieas geldo ranking score. [sent-182, score-0.734]
52 Among the two components in the objective function, the first component means that the refined rank should not deviate too much from the global biased rank. [sent-184, score-0.419]
53 The second component is similar by refining rank from local biased summarization. [sent-186, score-0.385]
54 ∂O∂(rRi)=Ψ2αi(GΨiiri− ri†) +Ψ2βi(ΨLiiri− ri‡) (7) 437 Let ∂O∂(riR) = 0, we get ri∗=αΨαiGrii†++β β ΨLiiri‡ (8) = Two special cases are that if (1) α = 0, β 0: we owbota isnp ri = indicating we only use th 0e: local ranking score. [sent-193, score-0.255]
55 (2) α 0, β = 0, indicating we ignore lnokcianlg ranking score =an 0d, only 0c,o innsdiidceart global biased summarization using inter-date dependency. [sent-194, score-0.6]
56 Here we define Ψi as the weighted combination of itself with ranking scores from global biased and local biased summarization: cΨasire‡is/ aLrei, = Ψ(iz)=αGi+α β +Li β+ + γ γΨi(z−1). [sent-196, score-0.618]
57 Ψi=αG1i+ − βγLi=αGαi ++ β βLi (10) Final Ψi is dependent only on original global/local biased ranking scores. [sent-202, score-0.262]
58 Equation (8) becomes more concise with no Ψ or γ: r∗ is a weighted combination of global and local ranks by βα (α 0, β 0): = 4Experi∗m=en1αts+ αa1β n/drαi†Er+†viaαluβ+1a βti1roαi‡n/βri‡ = (1 ) 4. [sent-203, score-0.178]
59 We randomly choose 6 news subjects with special coverage and handcrafted timelines by editors from 10 selected news websites: these 6 test sets consist of news datasets and golden standards to evaluate our proposed framework empirically, which amount to 1025 1 news articles. [sent-205, score-0.986]
60 We choose these sites because many of them provide timelines edited by professional editors, which serve as reference summaries. [sent-212, score-0.369]
61 After preprocessing, we aotebta ainnd numerous snippets with fine-grained timestamps, and then decompose them into temporally tagged sentences as the global collection C. [sent-228, score-0.216]
62 The sizes of component summaries are not necessarily equal, and moreover, not all dates may be represented, so date selection is also important. [sent-232, score-0.44]
63 We apply a simple mechanism that users specify the overall compression rate φ, and we extract more sentences for important dates while fewer sentences for others. [sent-233, score-0.195]
64 3 Evaluation Metrics The ROUGE measure is widely used for evaluation (Lin and Hovy, 2003): the DUC contests usually officially employ ROUGE for automatic summarization evaluation. [sent-238, score-0.232]
65 In ROUGE evaluation, the summa- rization quality is measured by counting the number of overlapping units, such as N-gram, word sequences, and word pairs between the candidate timelines CT and the reference timelines RT. [sent-239, score-0.738]
66 Countmatch(N-gram) is the maximum number of Ngram in the candidate timeline and in the set of reference timelines. [sent-242, score-0.28]
67 Count(N-gram) is the number of Ngrams in reference timelines or candidate timelines. [sent-243, score-0.369]
68 According to (Lin and Hovy, 2003), among all sub-metrics, unigram-based ROUGE (ROUGE-1) has been shown to agree with human judgment most and bigram-based ROUGE (ROUGE-2) fits summarization well. [sent-244, score-0.232]
69 Intuitively, the higher the ROUGE scores, the similar the two summaries are. [sent-249, score-0.149]
70 4 Algorithms for Comparison We implement the following widely used summarization algorithms as baseline systems. [sent-251, score-0.232]
71 They are designed for traditional summarization without trans-temporal dimension. [sent-252, score-0.261]
72 The first intuitive way to generate timelines by these methods is via a global summarization on collection C and then distribution of selected sentences to their source dates. [sent-253, score-0.785]
73 The other one is via an equal summarization on all local sub-collections. [sent-254, score-0.304]
74 Chieu: (Chieu and Lee, 2004) present a similar timeline system with different goals and frameworks, utilizing interest and burstiness ranking but neglecting trans-temporal news evolution. [sent-261, score-0.545]
75 ETTS: ETTS is an algorithm with optimized combination of global/local biased summarization. [sent-262, score-0.178]
76 RefTL: As we have used multiple human timelines as references, we not only provide ROUGE evaluations of the competing systems but also of the human timelines against each other, which provides a good indicator as to the upper bound ROUGE score that any system could achieve. [sent-263, score-0.738]
77 Traditional MDS only consider sentence selection from either the global or the local scope, and hence bias occurs. [sent-327, score-0.178]
78 only global summarization and then distribution to temporal subsets) is better than local priority methods (only local summarization). [sent-331, score-0.603]
79 tThahen reason may be that Chieu does not capture sufficient timeline attributes. [sent-338, score-0.28]
80 The “interest” modeled in the algorithms actually performs flat clusteringbased summarization which is proved to be less useful (Wang and Li, 2010). [sent-339, score-0.232]
81 • ETTS under our proposed framework outperfor•ms E baselines, indicating stehdat trahem properties we use for timeline generation are beneficial. [sent-341, score-0.31]
82 It is understandable that ETS refines timelines based on neighboring component summaries iteratively while for ETTS neighboring information is incorporated in temporal projection and hence there is no such procedure. [sent-350, score-0.874]
83 To identify how global and local biased summarization combine, we provide experiments on the performance of varying α/β in Figure 5. [sent-357, score-0.588]
84 Results indicate that a balance between global and local biased summarization is essential for timeline generation because the performance is best when βα ∈ [10, 100] and outperforms global and local summa∈riz [1a0ti,o1n0 0in] isolation, i. [sent-358, score-1.076]
85 Another key parameter σ measures the temporal projection influence from global collection to local collection and hence the size of neighboring sentence set. [sent-363, score-0.484]
86 The effect of σ varies on long news sets and 44 1 short news sets. [sent-366, score-0.276]
87 Generally, Gaussian kernel outperforms others and window kernel is the worst, probably because Gaussian kernel provides the best smoothing effect with no arbitrary to distinguish different by temporal proximity, pected. [sent-369, score-0.433]
88 Window kernel fails weights of neighboring sets so its performance is as exare comparable. [sent-371, score-0.152]
89 7 Sample Output and Case Study Sample output is presented in Table 7 and it shares major information similarity with the human timeline in Table 1. [sent-373, score-0.28]
90 We notice that humans have biases to generate timelines for they have (1) preference on local occurrences and (2) different writing styles. [sent-378, score-0.471]
91 For instance, news outlets from United States tend to summarize reactions by US government while UK websites tend to summarize British affairs. [sent-379, score-0.196]
92 Some editors favor statistical reports while others prefer narrative style, and some timelines have detailed explanations while others are extremely concise with no more than two sentences for each entry. [sent-380, score-0.405]
93 Our system-generated timelines have a large variance among all golden standards. [sent-381, score-0.369]
94 Proba- bly a new evaluation metric should be introduced to measure the quality of human generated timelines to mitigate the corresponding biases. [sent-382, score-0.369]
95 5 Conclusion We present a novel solution for the important web mining problem, Evolutionary Trans-Temporal Summarization (ETTS), which generates trajectory timelines for news subjects from massive data. [sent-387, score-0.538]
96 We formally formulate ETTS as a combination of global and local summarization, incorporating affinity and Table 7: Selected part of timeline generated by ETTS for BP Oil. [sent-388, score-0.558]
97 Through our experiment we notice that the combination plays an important role in timeline generation, and global optimization weights slightly higher (α/β ∈ [10, 100]), but auxiliary local informhigahtioenr (dαo/eβs help t0o, 1en0h0a]n),c beu performance icna ElT inTfoSr. [sent-391, score-0.488]
98 Event recognition from news webpages through latent ingredients extraction. [sent-480, score-0.212]
99 A fine-grained digestion of news webpages through event snippet extraction. [sent-484, score-0.241]
100 Evolutionary timeline summarization: a balanced optimization framework via iterative substitution. [sent-488, score-0.28]
wordName wordTfidf (topN-words)
[('etts', 0.369), ('timelines', 0.369), ('timeline', 0.28), ('summarization', 0.232), ('biased', 0.178), ('evolutionary', 0.159), ('wan', 0.15), ('summaries', 0.149), ('rouge', 0.142), ('news', 0.138), ('date', 0.122), ('temporal', 0.121), ('roi', 0.115), ('chieu', 0.111), ('divrank', 0.111), ('sigir', 0.106), ('global', 0.106), ('yan', 0.106), ('kernel', 0.104), ('affinity', 0.1), ('ri', 0.099), ('si', 0.099), ('timestamp', 0.095), ('sj', 0.09), ('component', 0.086), ('ranking', 0.084), ('bp', 0.083), ('dates', 0.083), ('oil', 0.08), ('erwt', 0.074), ('webpages', 0.074), ('mds', 0.072), ('local', 0.072), ('pagerank', 0.071), ('diversity', 0.07), ('rui', 0.067), ('timestamps', 0.064), ('xiaojun', 0.064), ('xiaoming', 0.064), ('ets', 0.064), ('gaussian', 0.061), ('spill', 0.058), ('allan', 0.058), ('countmatch', 0.055), ('gmds', 0.055), ('horizon', 0.055), ('iz', 0.053), ('projection', 0.053), ('rank', 0.049), ('summary', 0.048), ('neighboring', 0.048), ('centroid', 0.048), ('ise', 0.045), ('radev', 0.045), ('burstiness', 0.043), ('collection', 0.042), ('ct', 0.042), ('gi', 0.04), ('compression', 0.04), ('summarizing', 0.04), ('positional', 0.038), ('novelty', 0.038), ('acm', 0.037), ('abstractive', 0.037), ('bursty', 0.037), ('correlative', 0.037), ('correlativeness', 0.037), ('eigenvector', 0.037), ('icount', 0.037), ('liiri', 0.037), ('pending', 0.037), ('prestige', 0.037), ('rcg', 0.037), ('worthy', 0.037), ('retrieval', 0.037), ('neighbors', 0.036), ('sentences', 0.036), ('matrix', 0.035), ('li', 0.035), ('cosine', 0.034), ('datasets', 0.034), ('update', 0.033), ('iofth', 0.032), ('evolution', 0.032), ('circle', 0.032), ('kumaran', 0.032), ('temporally', 0.032), ('bfe', 0.032), ('lexpagerank', 0.032), ('peking', 0.032), ('saliency', 0.032), ('subjects', 0.031), ('kong', 0.031), ('generation', 0.03), ('notice', 0.03), ('traditional', 0.029), ('summarize', 0.029), ('swan', 0.029), ('hin', 0.029), ('snippet', 0.029)]
simIndex simValue paperId paperTitle
same-paper 1 0.99999923 135 emnlp-2011-Timeline Generation through Evolutionary Trans-Temporal Summarization
Author: Rui Yan ; Liang Kong ; Congrui Huang ; Xiaojun Wan ; Xiaoming Li ; Yan Zhang
Abstract: We investigate an important and challenging problem in summary generation, i.e., Evolutionary Trans-Temporal Summarization (ETTS), which generates news timelines from massive data on the Internet. ETTS greatly facilitates fast news browsing and knowledge comprehension, and hence is a necessity. Given the collection oftime-stamped web documents related to the evolving news, ETTS aims to return news evolution along the timeline, consisting of individual but correlated summaries on each date. Existing summarization algorithms fail to utilize trans-temporal characteristics among these component summaries. We propose to model trans-temporal correlations among component summaries for timelines, using inter-date and intra-date sen- tence dependencies, and present a novel combination. We develop experimental systems to compare 5 rival algorithms on 6 instinctively different datasets which amount to 10251 documents. Evaluation results in ROUGE metrics indicate the effectiveness of the proposed approach based on trans-temporal information. 1
Author: Rui Yan ; Jian-Yun Nie ; Xiaoming Li
Abstract: Most traditional summarization methods treat their outputs as static and plain texts, which fail to capture user interests during summarization because the generated summaries are the same for different users. However, users have individual preferences on a particular source document collection and obviously a universal summary for all users might not always be satisfactory. Hence we investigate an important and challenging problem in summary generation, i.e., Interactive Personalized Summarization (IPS), which generates summaries in an interactive and personalized manner. Given the source documents, IPS captures user interests by enabling interactive clicks and incorporates personalization by modeling captured reader preference. We develop . experimental systems to compare 5 rival algorithms on 4 instinctively different datasets which amount to 5197 documents. Evaluation results in ROUGE metrics indicate the comparable performance between IPS and the best competing system but IPS produces summaries with much more user satisfaction according to evaluator ratings. Besides, low ROUGE consistency among these user preferred summaries indicates the existence of personalization.
3 0.14683685 61 emnlp-2011-Generating Aspect-oriented Multi-Document Summarization with Event-aspect model
Author: Peng Li ; Yinglin Wang ; Wei Gao ; Jing Jiang
Abstract: In this paper, we propose a novel approach to automatic generation of aspect-oriented summaries from multiple documents. We first develop an event-aspect LDA model to cluster sentences into aspects. We then use extended LexRank algorithm to rank the sentences in each cluster. We use Integer Linear Programming for sentence selection. Key features of our method include automatic grouping of semantically related sentences and sentence ranking based on extension of random walk model. Also, we implement a new sentence compression algorithm which use dependency tree instead of parser tree. We compare our method with four baseline methods. Quantitative evaluation based on Rouge metric demonstrates the effectiveness and advantages of our method.
4 0.080840863 110 emnlp-2011-Ranking Human and Machine Summarization Systems
Author: Peter Rankel ; John Conroy ; Eric Slud ; Dianne O'Leary
Abstract: The Text Analysis Conference (TAC) ranks summarization systems by their average score over a collection of document sets. We investigate the statistical appropriateness of this score and propose an alternative that better distinguishes between human and machine evaluation systems.
5 0.068004891 132 emnlp-2011-Syntax-Based Grammaticality Improvement using CCG and Guided Search
Author: Yue Zhang ; Stephen Clark
Abstract: Machine-produced text often lacks grammaticality and fluency. This paper studies grammaticality improvement using a syntax-based algorithm based on CCG. The goal of the search problem is to find an optimal parse tree among all that can be constructed through selection and ordering of the input words. The search problem, which is significantly harder than parsing, is solved by guided learning for best-first search. In a standard word ordering task, our system gives a BLEU score of 40. 1, higher than the previous result of 33.7 achieved by a dependency-based system.
6 0.062462814 36 emnlp-2011-Corroborating Text Evaluation Results with Heterogeneous Measures
7 0.053197209 125 emnlp-2011-Statistical Machine Translation with Local Language Models
8 0.052948501 119 emnlp-2011-Semantic Topic Models: Combining Word Distributional Statistics and Dictionary Definitions
9 0.052115463 104 emnlp-2011-Personalized Recommendation of User Comments via Factor Models
10 0.051357135 147 emnlp-2011-Using Syntactic and Semantic Structural Kernels for Classifying Definition Questions in Jeopardy!
11 0.050355952 127 emnlp-2011-Structured Lexical Similarity via Convolution Kernels on Dependency Trees
12 0.048721969 29 emnlp-2011-Collaborative Ranking: A Case Study on Entity Linking
13 0.046184972 6 emnlp-2011-A Generate and Rank Approach to Sentence Paraphrasing
14 0.045980468 145 emnlp-2011-Unsupervised Semantic Role Induction with Graph Partitioning
15 0.045003828 5 emnlp-2011-A Fast Re-scoring Strategy to Capture Long-Distance Dependencies
16 0.044725668 88 emnlp-2011-Linear Text Segmentation Using Affinity Propagation
17 0.042398252 83 emnlp-2011-Learning Sentential Paraphrases from Bilingual Parallel Corpora for Text-to-Text Generation
18 0.039979838 148 emnlp-2011-Watermarking the Outputs of Structured Prediction with an application in Statistical Machine Translation.
19 0.039830659 89 emnlp-2011-Linguistic Redundancy in Twitter
20 0.039534412 22 emnlp-2011-Better Evaluation Metrics Lead to Better Machine Translation
topicId topicWeight
[(0, 0.161), (1, -0.068), (2, -0.009), (3, -0.13), (4, 0.007), (5, 0.013), (6, -0.017), (7, -0.091), (8, 0.073), (9, 0.014), (10, -0.082), (11, -0.266), (12, -0.118), (13, 0.105), (14, 0.308), (15, 0.042), (16, -0.241), (17, 0.096), (18, 0.042), (19, 0.109), (20, 0.04), (21, -0.056), (22, 0.043), (23, -0.096), (24, -0.044), (25, -0.014), (26, 0.076), (27, -0.055), (28, -0.079), (29, 0.084), (30, -0.023), (31, -0.022), (32, 0.176), (33, -0.012), (34, 0.09), (35, 0.04), (36, 0.014), (37, 0.164), (38, -0.039), (39, -0.078), (40, -0.021), (41, -0.109), (42, -0.144), (43, 0.007), (44, -0.097), (45, 0.121), (46, 0.104), (47, 0.081), (48, -0.049), (49, 0.078)]
simIndex simValue paperId paperTitle
same-paper 1 0.96374571 135 emnlp-2011-Timeline Generation through Evolutionary Trans-Temporal Summarization
Author: Rui Yan ; Liang Kong ; Congrui Huang ; Xiaojun Wan ; Xiaoming Li ; Yan Zhang
Abstract: We investigate an important and challenging problem in summary generation, i.e., Evolutionary Trans-Temporal Summarization (ETTS), which generates news timelines from massive data on the Internet. ETTS greatly facilitates fast news browsing and knowledge comprehension, and hence is a necessity. Given the collection oftime-stamped web documents related to the evolving news, ETTS aims to return news evolution along the timeline, consisting of individual but correlated summaries on each date. Existing summarization algorithms fail to utilize trans-temporal characteristics among these component summaries. We propose to model trans-temporal correlations among component summaries for timelines, using inter-date and intra-date sen- tence dependencies, and present a novel combination. We develop experimental systems to compare 5 rival algorithms on 6 instinctively different datasets which amount to 10251 documents. Evaluation results in ROUGE metrics indicate the effectiveness of the proposed approach based on trans-temporal information. 1
Author: Rui Yan ; Jian-Yun Nie ; Xiaoming Li
Abstract: Most traditional summarization methods treat their outputs as static and plain texts, which fail to capture user interests during summarization because the generated summaries are the same for different users. However, users have individual preferences on a particular source document collection and obviously a universal summary for all users might not always be satisfactory. Hence we investigate an important and challenging problem in summary generation, i.e., Interactive Personalized Summarization (IPS), which generates summaries in an interactive and personalized manner. Given the source documents, IPS captures user interests by enabling interactive clicks and incorporates personalization by modeling captured reader preference. We develop . experimental systems to compare 5 rival algorithms on 4 instinctively different datasets which amount to 5197 documents. Evaluation results in ROUGE metrics indicate the comparable performance between IPS and the best competing system but IPS produces summaries with much more user satisfaction according to evaluator ratings. Besides, low ROUGE consistency among these user preferred summaries indicates the existence of personalization.
3 0.56603432 110 emnlp-2011-Ranking Human and Machine Summarization Systems
Author: Peter Rankel ; John Conroy ; Eric Slud ; Dianne O'Leary
Abstract: The Text Analysis Conference (TAC) ranks summarization systems by their average score over a collection of document sets. We investigate the statistical appropriateness of this score and propose an alternative that better distinguishes between human and machine evaluation systems.
4 0.46962759 61 emnlp-2011-Generating Aspect-oriented Multi-Document Summarization with Event-aspect model
Author: Peng Li ; Yinglin Wang ; Wei Gao ; Jing Jiang
Abstract: In this paper, we propose a novel approach to automatic generation of aspect-oriented summaries from multiple documents. We first develop an event-aspect LDA model to cluster sentences into aspects. We then use extended LexRank algorithm to rank the sentences in each cluster. We use Integer Linear Programming for sentence selection. Key features of our method include automatic grouping of semantically related sentences and sentence ranking based on extension of random walk model. Also, we implement a new sentence compression algorithm which use dependency tree instead of parser tree. We compare our method with four baseline methods. Quantitative evaluation based on Rouge metric demonstrates the effectiveness and advantages of our method.
5 0.24958132 36 emnlp-2011-Corroborating Text Evaluation Results with Heterogeneous Measures
Author: Enrique Amigo ; Julio Gonzalo ; Jesus Gimenez ; Felisa Verdejo
Abstract: Automatically produced texts (e.g. translations or summaries) are usually evaluated with n-gram based measures such as BLEU or ROUGE, while the wide set of more sophisticated measures that have been proposed in the last years remains largely ignored for practical purposes. In this paper we first present an indepth analysis of the state of the art in order to clarify this issue. After this, we formalize and verify empirically a set of properties that every text evaluation measure based on similarity to human-produced references satisfies. These properties imply that corroborating system improvements with additional measures always increases the overall reliability of the evaluation process. In addition, the greater the heterogeneity of the measures (which is measurable) the higher their combined reliability. These results support the use of heterogeneous measures in order to consolidate text evaluation results.
6 0.24579482 104 emnlp-2011-Personalized Recommendation of User Comments via Factor Models
7 0.2454958 132 emnlp-2011-Syntax-Based Grammaticality Improvement using CCG and Guided Search
8 0.19808748 82 emnlp-2011-Learning Local Content Shift Detectors from Document-level Information
9 0.19611457 139 emnlp-2011-Twitter Catches The Flu: Detecting Influenza Epidemics using Twitter
10 0.18645471 148 emnlp-2011-Watermarking the Outputs of Structured Prediction with an application in Statistical Machine Translation.
11 0.18312879 147 emnlp-2011-Using Syntactic and Semantic Structural Kernels for Classifying Definition Questions in Jeopardy!
12 0.18050221 145 emnlp-2011-Unsupervised Semantic Role Induction with Graph Partitioning
13 0.17305607 29 emnlp-2011-Collaborative Ranking: A Case Study on Entity Linking
14 0.17293079 32 emnlp-2011-Computing Logical Form on Regulatory Texts
15 0.16767418 2 emnlp-2011-A Cascaded Classification Approach to Semantic Head Recognition
16 0.16543722 100 emnlp-2011-Optimal Search for Minimum Error Rate Training
17 0.1640953 88 emnlp-2011-Linear Text Segmentation Using Affinity Propagation
18 0.16254131 81 emnlp-2011-Learning General Connotation of Words using Graph-based Algorithms
19 0.15902409 127 emnlp-2011-Structured Lexical Similarity via Convolution Kernels on Dependency Trees
20 0.15640213 73 emnlp-2011-Improving Bilingual Projections via Sparse Covariance Matrices
topicId topicWeight
[(23, 0.592), (36, 0.023), (37, 0.017), (45, 0.064), (53, 0.012), (54, 0.016), (57, 0.054), (62, 0.012), (66, 0.011), (69, 0.013), (79, 0.032), (82, 0.013), (90, 0.01), (96, 0.027), (98, 0.025)]
simIndex simValue paperId paperTitle
1 0.99404013 42 emnlp-2011-Divide and Conquer: Crowdsourcing the Creation of Cross-Lingual Textual Entailment Corpora
Author: Matteo Negri ; Luisa Bentivogli ; Yashar Mehdad ; Danilo Giampiccolo ; Alessandro Marchetti
Abstract: We address the creation of cross-lingual textual entailment corpora by means of crowdsourcing. Our goal is to define a cheap and replicable data collection methodology that minimizes the manual work done by expert annotators, without resorting to preprocessing tools or already annotated monolingual datasets. In line with recent works emphasizing the need of large-scale annotation efforts for textual entailment, our work aims to: i) tackle the scarcity of data available to train and evaluate systems, and ii) promote the recourse to crowdsourcing as an effective way to reduce the costs of data collection without sacrificing quality. We show that a complex data creation task, for which even experts usually feature low agreement scores, can be effectively decomposed into simple subtasks assigned to non-expert annotators. The resulting dataset, obtained from a pipeline of different jobs routed to Amazon Mechanical Turk, contains more than 1,600 aligned pairs for each combination of texts-hypotheses in English, Italian and German.
same-paper 2 0.9936828 135 emnlp-2011-Timeline Generation through Evolutionary Trans-Temporal Summarization
Author: Rui Yan ; Liang Kong ; Congrui Huang ; Xiaojun Wan ; Xiaoming Li ; Yan Zhang
Abstract: We investigate an important and challenging problem in summary generation, i.e., Evolutionary Trans-Temporal Summarization (ETTS), which generates news timelines from massive data on the Internet. ETTS greatly facilitates fast news browsing and knowledge comprehension, and hence is a necessity. Given the collection oftime-stamped web documents related to the evolving news, ETTS aims to return news evolution along the timeline, consisting of individual but correlated summaries on each date. Existing summarization algorithms fail to utilize trans-temporal characteristics among these component summaries. We propose to model trans-temporal correlations among component summaries for timelines, using inter-date and intra-date sen- tence dependencies, and present a novel combination. We develop experimental systems to compare 5 rival algorithms on 6 instinctively different datasets which amount to 10251 documents. Evaluation results in ROUGE metrics indicate the effectiveness of the proposed approach based on trans-temporal information. 1
3 0.99257869 93 emnlp-2011-Minimum Imputed-Risk: Unsupervised Discriminative Training for Machine Translation
Author: Zhifei Li ; Ziyuan Wang ; Jason Eisner ; Sanjeev Khudanpur ; Brian Roark
Abstract: Discriminative training for machine translation has been well studied in the recent past. A limitation of the work to date is that it relies on the availability of high-quality in-domain bilingual text for supervised training. We present an unsupervised discriminative training framework to incorporate the usually plentiful target-language monolingual data by using a rough “reverse” translation system. Intuitively, our method strives to ensure that probabilistic “round-trip” translation from a target- language sentence to the source-language and back will have low expected loss. Theoretically, this may be justified as (discriminatively) minimizing an imputed empirical risk. Empirically, we demonstrate that augmenting supervised training with unsupervised data improves translation performance over the supervised case for both IWSLT and NIST tasks.
4 0.99163979 48 emnlp-2011-Enhancing Chinese Word Segmentation Using Unlabeled Data
Author: Weiwei Sun ; Jia Xu
Abstract: This paper investigates improving supervised word segmentation accuracy with unlabeled data. Both large-scale in-domain data and small-scale document text are considered. We present a unified solution to include features derived from unlabeled data to a discriminative learning model. For the large-scale data, we derive string statistics from Gigaword to assist a character-based segmenter. In addition, we introduce the idea about transductive, document-level segmentation, which is designed to improve the system recall for out-ofvocabulary (OOV) words which appear more than once inside a document. Novel features1 result in relative error reductions of 13.8% and 15.4% in terms of F-score and the recall of OOV words respectively.
5 0.99020112 7 emnlp-2011-A Joint Model for Extended Semantic Role Labeling
Author: Vivek Srikumar ; Dan Roth
Abstract: This paper presents a model that extends semantic role labeling. Existing approaches independently analyze relations expressed by verb predicates or those expressed as nominalizations. However, sentences express relations via other linguistic phenomena as well. Furthermore, these phenomena interact with each other, thus restricting the structures they articulate. In this paper, we use this intuition to define a joint inference model that captures the inter-dependencies between verb semantic role labeling and relations expressed using prepositions. The scarcity of jointly labeled data presents a crucial technical challenge for learning a joint model. The key strength of our model is that we use existing structure predictors as black boxes. By enforcing consistency constraints between their predictions, we show improvements in the performance of both tasks without retraining the individual models.
6 0.91239762 6 emnlp-2011-A Generate and Rank Approach to Sentence Paraphrasing
7 0.90826702 58 emnlp-2011-Fast Generation of Translation Forest for Large-Scale SMT Discriminative Training
8 0.90643191 137 emnlp-2011-Training dependency parsers by jointly optimizing multiple objectives
9 0.87859994 79 emnlp-2011-Lateen EM: Unsupervised Training with Multiple Objectives, Applied to Dependency Grammar Induction
10 0.87401915 61 emnlp-2011-Generating Aspect-oriented Multi-Document Summarization with Event-aspect model
11 0.86912453 44 emnlp-2011-Domain Adaptation via Pseudo In-Domain Data Selection
12 0.86784536 17 emnlp-2011-Active Learning with Amazon Mechanical Turk
13 0.86687905 124 emnlp-2011-Splitting Noun Compounds via Monolingual and Bilingual Paraphrasing: A Study on Japanese Katakana Words
14 0.86502272 23 emnlp-2011-Bootstrapped Named Entity Recognition for Product Attribute Extraction
15 0.86488533 136 emnlp-2011-Training a Parser for Machine Translation Reordering
16 0.86362106 126 emnlp-2011-Structural Opinion Mining for Graph-based Sentiment Representation
17 0.86297464 28 emnlp-2011-Closing the Loop: Fast, Interactive Semi-Supervised Annotation With Queries on Features and Instances
18 0.85749739 25 emnlp-2011-Cache-based Document-level Statistical Machine Translation
19 0.85745525 82 emnlp-2011-Learning Local Content Shift Detectors from Document-level Information
20 0.85638183 89 emnlp-2011-Linguistic Redundancy in Twitter