acl acl2010 acl2010-122 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Swati Tata ; Barbara Di Eugenio
Abstract: Music Recommendation Systems often recommend individual songs, as opposed to entire albums. The challenge is to generate reviews for each song, since only full album reviews are available on-line. We developed a summarizer that combines information extraction and generation techniques to produce summaries of reviews of individual songs. We present an intrinsic evaluation of the extraction components, and of the informativeness of the summaries; and a user study of the impact of the song review summaries on users’ decision making processes. Users were able to make quicker and more informed decisions when presented with the summary as compared to the full album review.
Reference: text
sentIndex sentText sentNum sentScore
1 The challenge is to generate reviews for each song, since only full album reviews are available on-line. [sent-3, score-1.164]
2 We developed a summarizer that combines information extraction and generation techniques to produce summaries of reviews of individual songs. [sent-4, score-0.515]
3 We present an intrinsic evaluation of the extraction components, and of the informativeness of the summaries; and a user study of the impact of the song review summaries on users’ decision making processes. [sent-5, score-0.881]
4 Users were able to make quicker and more informed decisions when presented with the summary as compared to the full album review. [sent-6, score-0.634]
5 1 Introduction In recent years, the personal music collection of many individuals has significantly grown due to the availability of portable devices like MP3 players and of internet services. [sent-7, score-0.263]
6 Music listeners are now looking for techniques to help them manage their music collections and explore songs they may not even know they have (Clema, 2006). [sent-8, score-0.481]
7 Currently, most of those electronic devices follow a Universal Plug and Play (UPNP) protocol (UPN, 2008), and can be used in a simple network, on which the songs listened to can be monitored. [sent-9, score-0.276]
8 Most Product RSs include reviews from customers who bought or tried the product. [sent-16, score-0.349]
9 As the number of reviews available for each individual product in- creases, RSs may overwhelm the user if they make all those reviews available. [sent-17, score-0.746]
10 Additionally, in some reviews only few sentences actually describe the recommended product, hence, the interest in opinion mining and in summarizing those reviews. [sent-18, score-0.505]
11 Summarizing reviews becomes more complex: available data consists of album reviews, not individual song reviews (www . [sent-25, score-1.719]
12 Comments about a given song are fragmented all over an album review. [sent-30, score-1.079]
13 We devote Section 4 to our summarizer, that extracts comments on individual tracks from album reviews and produces a summary of those comments for each individual track recommended to the user. [sent-35, score-1.074]
14 We found that users make quicker and more informed decisions when presented with the song review summaries as opposed to the full album review. [sent-37, score-1.372]
15 Hence, summarizing customer reviews has mostly been studied as a combination of machine learning and NLP techniques (Hu and Liu, 2004; Gamon et al. [sent-43, score-0.42]
16 For example, (Hu and Liu, 2004) use associative mining techniques to identify features that frequently occur in reviews taken from www . [sent-45, score-0.475]
17 Most work on product reviews focuses on identifying sentences and polarity of opinion terms, not on generating a coherent summary from the extracted features, which is the main goal of our research. [sent-51, score-0.567]
18 , 2006), whose focus was on extracting domain specific ontologies in order to structure summarization of customer reviews. [sent-54, score-0.141]
19 Summarizing reviews on objects different from products, such as restaurants (Nguyen et al. [sent-55, score-0.346]
20 We are aware of only one piece of work that focuses on music reviews (Downie and Hu, 2006). [sent-58, score-0.541]
21 This study is mainly concerned with identifying descriptive patterns in positive or negative reviews but not on summarizing the reviews. [sent-59, score-0.395]
22 1 Summarizing song reviews is different As mentioned earlier, using album reviews for song summarization poses new challenges: a) Comments on features of a song are embedded and fragmented within the album reviews, as shown in Figure 1. [sent-61, score-3.401]
23 b) Each song needs to be identified each time it is referred to in the review. [sent-63, score-0.59]
24 Additionally, song titles need not be noun phrases and hence NP extraction algorithms miss many occurrences, as was shown by preliminary experiments we ran. [sent-67, score-0.642]
25 ), unlike in product reviews where manufacturer/designer are rarely mentioned. [sent-69, score-0.349]
26 Moving from window to window and from top to bottom: a) The top leftmost window shows different devices on which the user listens to songs. [sent-73, score-0.206]
27 We use the SimpleKMeans Clustering (Mitchell, 1997) to identify and rank the top twenty-five songs which belong to the same cluster and are closest to the given song. [sent-78, score-0.257]
28 Closeness between two songs in a cluster is measured as the number of attributes (album, artist etc) of the songs that match. [sent-79, score-0.513]
29 d) When the user clicks on More Info for one of the recommended songs, the pop-up, bottom window is displayed, which contains the summary of the reviews for the specific song. [sent-80, score-0.556]
30 Our framework can be applied to any domain where reviews of individual components need to be summarized from reviews of collections, such as reviews of different hotels and restaurants in a city. [sent-85, score-1.035]
31 It consists of 1350 album reviews across 27 albums (50 reviews per album). [sent-91, score-1.198]
32 This corpus was annotated for song titles and song features. [sent-95, score-1.135]
33 Feature annotation consists of marking a phrase as a feature and matching it with the song which the feature is attributed. [sent-96, score-0.634]
34 Note that we have no a priori inventory of features; what counts to Figure 3: Summarization Pipeline g p as features of songs emerged from the annotation, since annotators were asked to annotate for noun phrases which contain “any song related term or terms spoken in the context of a song”. [sent-97, score-0.795]
35 Ten randomly chosen reviews were doubly annotated for song titles and features. [sent-101, score-0.926]
36 We identify a string of words as the title of a song to be extracted from an album review if it (1) includes some or all the words in the title of a track of that album, and (2) this string occurs in the right context. [sent-109, score-1.25]
37 Constraint (2) is necessary because the string of words corresponding to the title may appear in the lyrics of the song or anywhere else in the review. [sent-110, score-0.617]
38 , 2007), our approach to song title extraction is based on n-grams. [sent-116, score-0.618]
39 Given the reviews for an album and the list of songs in that album, first, we build a lexicon of all the words in the song titles. [sent-118, score-1.608]
40 We also segment the reviews into sentences via sentence boundary detection. [sent-119, score-0.38]
41 All the n-grams remaining at this point are potential song titles. [sent-126, score-0.528]
42 This intersection always resulted in one single song ID, since song titles in each album differ by at least one content word. [sent-128, score-1.661]
43 Recall that the algorithm is run on reviews for each album separately. [sent-129, score-0.845]
44 2 Feature Extraction Once the song titles are identified in the album review, sentences with song titles are used as anchors to (1) identify segments of texts that talk about a specific song, and then (2) extract the feature(s) that the pertinent text segment discusses. [sent-131, score-1.892]
45 An album review is first divided into segments using the text tiling algorithm. [sent-138, score-0.686]
46 The segments that contain potential features of a song are identified using the following heuristics: Step 1: Include segi if it contains a song title. [sent-143, score-1.246]
47 1379 These segments are more likely to contain features of songs as they are composed of the sentences surrounding the song title. [sent-144, score-0.863]
48 Since we have no a priori inventory of features (the feature annotation will be used for evaluation, not for development), we use WordNet (Fellbaum, 1998) to identify feature terms: i. [sent-146, score-0.16]
49 Feature terms exclude the words “music”, “song”, the artist/band/album name as they are likely to occur across album reviews. [sent-149, score-0.526]
50 All feature terms in the final set of segments selected by the heuristics are taken to be features of the song described by that segment. [sent-150, score-0.652]
51 Removing parts of sentences that describe another feature, will have no effect on the summary as a whole as the portions that are removed will be present in the group of sentences that describe the other feature. [sent-157, score-0.161]
52 To identify portions of the sentence relevant to the single feature f, we use the Stanford Typed Dependency Parser (Klein and Manning, 2002; de Marnee and Manning, 2008). [sent-161, score-0.139]
53 By the end of this step, for each feature f of a certain song si, we have generated a set of f-sentences. [sent-182, score-0.581]
54 McCartney laid down the guitar lead for this track. [sent-189, score-0.141]
55 Identical lead guitar provide the rhythmic basis for this song. [sent-191, score-0.141]
56 The first sentence talks about the guitar solo, the second and the third about the lead guitar. [sent-192, score-0.205]
57 We 1380 Figure 6: Grouping sentences by sub-features identify subgroups via common n-grams between f-sentences, and make sure that only n-grams that are related to feature f are identified at this stage, as detailed in Figure 6. [sent-194, score-0.194]
58 Step 2 in Figure 6 will group b and c together since guitar lead properly contains the feature term guitar. [sent-196, score-0.194]
59 In Step 3, sentence a is sentence fx-si such that its LCN with all other sentences (b and c) contains only the feature term; hence, sentence a is left on its own. [sent-197, score-0.178]
60 As concerns polarity grouping, different reviews may express different opinions regarding a particular feature. [sent-199, score-0.394]
61 5 Selection and Ordering Finally, the generation of a coherent summary involves selection of the sentences to be included, and ordering them in a coherent fashion. [sent-213, score-0.188]
62 Our assumption is that among the original sentences, a sentence that talks about one feature only is likely to express a stronger opinion about that feature than a sentence in which other features are present. [sent-222, score-0.263]
63 We order the sentences by exploiting a music ontology (Giasson and Raimond, 2007). [sent-223, score-0.251]
64 The ordering of the domains is such that first we present the general features of the song (e. [sent-229, score-0.56]
65 5 Evaluation In this section we report three evaluations, two intrinsic and one extrinsic: evaluation of the song title and feature extraction steps; evaluation of the informativeness of summaries; and a user study to judge how summaries affect decision making. [sent-239, score-0.925]
66 1 Song Title and Feature Extraction The song title extraction and feature extraction algorithms (Sections 4. [sent-241, score-0.706]
67 2) were manually evaluated on 100 reviews randomly taken from the corpus (2 or 3 from each album). [sent-243, score-0.319]
68 The 100 reviews contained 1304 occurrences of song titles and 898 occurrences of song features, as previously annotated. [sent-245, score-1.514]
69 1294 occurrences of song titles were correctly identified; additionally, 123 spurious occurrences were also identified. [sent-246, score-0.667]
70 Of the 898 occurrences of song features, 853 were correctly identified by our feature extraction algorithm, with an additional 41 spurious occurrences. [sent-251, score-0.708]
71 Note that a feature (NP) is considered as correctly identified, if its head noun is annotated in a review for the song with correct ID. [sent-255, score-0.645]
72 We compared their algorithm to ours on 10 randomly chosen reviews from our corpus, for a total of about 500 sentences. [sent-257, score-0.319]
73 5% recall) is much lower than ours, and than their original results on product reviews (72% precision, and 80% recall). [sent-260, score-0.349]
74 2 Informativeness of the summaries To evaluate the information captured in the summary, we randomly selected 5 or 6 songs from 10 albums, and generated the corresponding 52 summaries, oneper song–this corresponds to atest set of about 500 album reviews (each album has about 50 reviews). [sent-262, score-1.74]
75 We approximate those goldstandard reference summaries with 2 or 3 critic reviews per album taken from www . [sent-265, score-1.146]
76 c First, we manually annotated both critic reviews and the automatically generated summaries for song titles and song features. [sent-272, score-1.675]
77 2% of the features identified in the critic reviews are also identified in the summaries (recall that a feature is considered as identified, if the head-noun of the NP is identified by both the critic review and the summary, and attributed to the same song). [sent-276, score-0.962]
78 It is not surprising that additional features may appear in the summaries: even if only one of the 50 album reviews talks about that feature, it is included in the summary. [sent-278, score-0.909]
79 In a second evaluation, we used our Feature Extraction algorithm to extract features from the critic reviews, for each song whose summary needs to be evaluated. [sent-280, score-0.718]
80 This is an indirect evaluation of that algorithm, in that it shows it is not affected by somewhat different data, since the critic reviews are more formally written. [sent-281, score-0.406]
81 375, or 95% of the features identified in the critic reviews are also identified in the summaries. [sent-282, score-0.562]
82 For each of the recommended songs, the baseline version provides only whole album reviews, the experimental version provides the automatically generated song feature summary, as shown in Figure 2. [sent-290, score-1.159]
83 The interface for the baseline version is similar, but the summary in the bottom window is replaced by the corresponding album review. [sent-291, score-0.635]
84 The presented re- view is the one among the 50 reviews for that album whose length is closest to the average length of album reviews in the corpus (478 words). [sent-292, score-1.69]
85 Each user was presented with 5 songs in succession, with 3 recommendations each (only the top 3 recommendations were presented among the available 25, see Section 3). [sent-293, score-0.364]
86 The first song was a test song for users to get acquainted with the system. [sent-296, score-1.139]
87 18 users interacted with the baseline version and 21 users with the experimental version (five additional subjects were run but their log data was not properly saved). [sent-299, score-0.188]
88 Our main measure is time on task, the total time taken to select the recommendations from song 2 to song 5 this excludes the time spent listening to the songs. [sent-301, score-1.095]
89 This is a positive result, because decreasing time to selection is important, given that music collections can include millions of songs. [sent-305, score-0.268]
90 However, time-on-task basically represents the time it takes users to peruse the review or summary, and the number of words in the summaries is significantly lower than the number of words in the reviews (p < 0. [sent-306, score-0.6]
91 Users in the baseline condition picked the first recommendation as often as the other two rec– ommendations combined; users in the experimental condition picked the second and third recommendations more often than the first, and the difference between the two conditions is significant (χ2 = 8. [sent-311, score-0.178]
92 If we examine behavior song by song, this holds true especially for song 3 (χ2 = 12. [sent-314, score-1.056]
93 We speculate that users in the experimental condition 1383 are more discriminatory in their choices, because important features of the recommended songs are evident in the summaries, but are buried in the album reviews. [sent-319, score-0.928]
94 For example, for Song 3, only one of the 20 sentences in the album review is about the first recommended song, and is not very positive. [sent-320, score-0.671]
95 Whereas these numbers in isolation are not too telling, they are at least suggestive that users did not find these summaries badly written. [sent-336, score-0.217]
96 6 Discussion and Conclusions Most summarization research on customer reviews focuses on obtaining features of the products, but not much work has been done on presenting them as a coherent summary. [sent-338, score-0.512]
97 In this paper, we described a system that uses information extraction and summarization techniques in order to generate summaries of individual songs from multiple album reviews. [sent-339, score-1.027]
98 Users made quicker decisions and their choice of recommendations was more varied when presented with song review summaries than with album reviews. [sent-343, score-1.328]
99 Our framework can be applied to any domain where reviews of individual components need to be summarized from reviews of collections, such as travel reviews that cover many cities in a country, or different restaurants in a city. [sent-344, score-1.035]
100 Summarization of multiple user reviews in the restaurant domain. [sent-432, score-0.37]
wordName wordTfidf (topN-words)
[('song', 0.528), ('album', 0.526), ('reviews', 0.319), ('songs', 0.235), ('music', 0.222), ('summaries', 0.134), ('guitar', 0.118), ('critic', 0.087), ('users', 0.083), ('www', 0.08), ('titles', 0.079), ('summary', 0.071), ('summarization', 0.07), ('review', 0.064), ('identified', 0.062), ('segi', 0.057), ('tiling', 0.057), ('recommendation', 0.056), ('title', 0.055), ('summarizing', 0.054), ('feature', 0.053), ('recommended', 0.052), ('user', 0.051), ('customer', 0.047), ('polarity', 0.045), ('coherent', 0.044), ('artist', 0.043), ('songrecommend', 0.043), ('subgroup', 0.043), ('tata', 0.043), ('upnp', 0.043), ('yag', 0.043), ('devices', 0.041), ('segments', 0.039), ('recommendations', 0.039), ('window', 0.038), ('quicker', 0.037), ('soubbotin', 0.037), ('informativeness', 0.037), ('com', 0.036), ('extraction', 0.035), ('ama', 0.034), ('rss', 0.034), ('albums', 0.034), ('lyrics', 0.034), ('silver', 0.034), ('features', 0.032), ('sentence', 0.032), ('hu', 0.032), ('intrinsic', 0.032), ('talks', 0.032), ('portions', 0.032), ('rs', 0.031), ('occurrences', 0.03), ('customers', 0.03), ('opinions', 0.03), ('product', 0.03), ('maxwell', 0.029), ('sentences', 0.029), ('opinion', 0.029), ('grouping', 0.029), ('bang', 0.028), ('downie', 0.028), ('giasson', 0.028), ('hammer', 0.028), ('harnly', 0.028), ('lcn', 0.028), ('marnee', 0.028), ('mccartney', 0.028), ('minnen', 0.028), ('octopus', 0.028), ('schedl', 0.028), ('subfeatures', 0.028), ('subgroups', 0.028), ('typed', 0.028), ('ai', 0.028), ('df', 0.027), ('individual', 0.027), ('restaurants', 0.027), ('grammaticality', 0.026), ('prototype', 0.026), ('comments', 0.026), ('clicks', 0.025), ('fragmented', 0.025), ('higashinaka', 0.025), ('mcroy', 0.025), ('regenerated', 0.025), ('swati', 0.025), ('recall', 0.025), ('collections', 0.024), ('domain', 0.024), ('extrinsic', 0.024), ('carenini', 0.023), ('lead', 0.023), ('template', 0.023), ('subjects', 0.022), ('positive', 0.022), ('step', 0.022), ('identify', 0.022), ('mining', 0.022)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000005 122 acl-2010-Generating Fine-Grained Reviews of Songs from Album Reviews
Author: Swati Tata ; Barbara Di Eugenio
Abstract: Music Recommendation Systems often recommend individual songs, as opposed to entire albums. The challenge is to generate reviews for each song, since only full album reviews are available on-line. We developed a summarizer that combines information extraction and generation techniques to produce summaries of reviews of individual songs. We present an intrinsic evaluation of the extraction components, and of the informativeness of the summaries; and a user study of the impact of the song review summaries on users’ decision making processes. Users were able to make quicker and more informed decisions when presented with the summary as compared to the full album review.
2 0.14699164 157 acl-2010-Last but Definitely Not Least: On the Role of the Last Sentence in Automatic Polarity-Classification
Author: Israela Becker ; Vered Aharonson
Abstract: Two psycholinguistic and psychophysical experiments show that in order to efficiently extract polarity of written texts such as customerreviews on the Internet, one should concentrate computational efforts on messages in the final position of the text.
3 0.12570563 188 acl-2010-Optimizing Informativeness and Readability for Sentiment Summarization
Author: Hitoshi Nishikawa ; Takaaki Hasegawa ; Yoshihiro Matsuo ; Genichiro Kikui
Abstract: We propose a novel algorithm for sentiment summarization that takes account of informativeness and readability, simultaneously. Our algorithm generates a summary by selecting and ordering sentences taken from multiple review texts according to two scores that represent the informativeness and readability of the sentence order. The informativeness score is defined by the number of sentiment expressions and the readability score is learned from the target corpus. We evaluate our method by summarizing reviews on restaurants. Our method outperforms an existing algorithm as indicated by its ROUGE score and human readability experiments.
4 0.11627862 38 acl-2010-Automatic Evaluation of Linguistic Quality in Multi-Document Summarization
Author: Emily Pitler ; Annie Louis ; Ani Nenkova
Abstract: To date, few attempts have been made to develop and validate methods for automatic evaluation of linguistic quality in text summarization. We present the first systematic assessment of several diverse classes of metrics designed to capture various aspects of well-written text. We train and test linguistic quality models on consecutive years of NIST evaluation data in order to show the generality of results. For grammaticality, the best results come from a set of syntactic features. Focus, coherence and referential clarity are best evaluated by a class of features measuring local coherence on the basis of cosine similarity between sentences, coreference informa- tion, and summarization specific features. Our best results are 90% accuracy for pairwise comparisons of competing systems over a test set of several inputs and 70% for ranking summaries of a specific input.
5 0.11047599 209 acl-2010-Sentiment Learning on Product Reviews via Sentiment Ontology Tree
Author: Wei Wei ; Jon Atle Gulla
Abstract: Existing works on sentiment analysis on product reviews suffer from the following limitations: (1) The knowledge of hierarchical relationships of products attributes is not fully utilized. (2) Reviews or sentences mentioning several attributes associated with complicated sentiments are not dealt with very well. In this paper, we propose a novel HL-SOT approach to labeling a product’s attributes and their associated sentiments in product reviews by a Hierarchical Learning (HL) process with a defined Sentiment Ontology Tree (SOT). The empirical analysis against a humanlabeled data set demonstrates promising and reasonable performance of the proposed HL-SOT approach. While this paper is mainly on sentiment analysis on reviews of one product, our proposed HLSOT approach is easily generalized to labeling a mix of reviews of more than one products.
6 0.10742395 77 acl-2010-Cross-Language Document Summarization Based on Machine Translation Quality Prediction
7 0.094240569 264 acl-2010-Wrapping up a Summary: From Representation to Generation
8 0.088330999 251 acl-2010-Using Anaphora Resolution to Improve Opinion Target Identification in Movie Reviews
9 0.084779963 208 acl-2010-Sentence and Expression Level Annotation of Opinions in User-Generated Discourse
10 0.083200842 14 acl-2010-A Risk Minimization Framework for Extractive Speech Summarization
11 0.081546031 124 acl-2010-Generating Image Descriptions Using Dependency Relational Patterns
12 0.080063105 106 acl-2010-Event-Based Hyperspace Analogue to Language for Query Expansion
13 0.076941907 80 acl-2010-Cross Lingual Adaptation: An Experiment on Sentiment Classifications
14 0.072533481 78 acl-2010-Cross-Language Text Classification Using Structural Correspondence Learning
15 0.071218044 8 acl-2010-A Hybrid Hierarchical Model for Multi-Document Summarization
16 0.070846252 125 acl-2010-Generating Templates of Entity Summaries with an Entity-Aspect Model and Pattern Mining
17 0.064378291 18 acl-2010-A Study of Information Retrieval Weighting Schemes for Sentiment Analysis
18 0.063413963 204 acl-2010-Recommendation in Internet Forums and Blogs
19 0.062609412 11 acl-2010-A New Approach to Improving Multilingual Summarization Using a Genetic Algorithm
20 0.062225588 39 acl-2010-Automatic Generation of Story Highlights
topicId topicWeight
[(0, -0.143), (1, 0.088), (2, -0.145), (3, 0.05), (4, -0.061), (5, -0.003), (6, -0.021), (7, -0.122), (8, -0.038), (9, 0.007), (10, 0.023), (11, -0.02), (12, -0.06), (13, -0.001), (14, -0.032), (15, 0.001), (16, 0.053), (17, -0.031), (18, 0.043), (19, 0.04), (20, -0.008), (21, -0.016), (22, 0.02), (23, 0.001), (24, 0.029), (25, 0.074), (26, 0.048), (27, 0.027), (28, -0.022), (29, 0.015), (30, 0.022), (31, 0.053), (32, -0.032), (33, 0.008), (34, 0.049), (35, 0.006), (36, 0.001), (37, 0.016), (38, 0.085), (39, 0.004), (40, -0.001), (41, 0.103), (42, -0.07), (43, -0.113), (44, -0.037), (45, -0.176), (46, 0.11), (47, -0.115), (48, 0.112), (49, 0.049)]
simIndex simValue paperId paperTitle
same-paper 1 0.935817 122 acl-2010-Generating Fine-Grained Reviews of Songs from Album Reviews
Author: Swati Tata ; Barbara Di Eugenio
Abstract: Music Recommendation Systems often recommend individual songs, as opposed to entire albums. The challenge is to generate reviews for each song, since only full album reviews are available on-line. We developed a summarizer that combines information extraction and generation techniques to produce summaries of reviews of individual songs. We present an intrinsic evaluation of the extraction components, and of the informativeness of the summaries; and a user study of the impact of the song review summaries on users’ decision making processes. Users were able to make quicker and more informed decisions when presented with the summary as compared to the full album review.
2 0.76406318 157 acl-2010-Last but Definitely Not Least: On the Role of the Last Sentence in Automatic Polarity-Classification
Author: Israela Becker ; Vered Aharonson
Abstract: Two psycholinguistic and psychophysical experiments show that in order to efficiently extract polarity of written texts such as customerreviews on the Internet, one should concentrate computational efforts on messages in the final position of the text.
3 0.60468423 63 acl-2010-Comparable Entity Mining from Comparative Questions
Author: Shasha Li ; Chin-Yew Lin ; Young-In Song ; Zhoujun Li
Abstract: Comparing one thing with another is a typical part of human decision making process. However, it is not always easy to know what to compare and what are the alternatives. To address this difficulty, we present a novel way to automatically mine comparable entities from comparative questions that users posted online. To ensure high precision and high recall, we develop a weakly-supervised bootstrapping method for comparative question identification and comparable entity extraction by leveraging a large online question archive. The experimental results show our method achieves F1measure of 82.5% in comparative question identification and 83.3% in comparable entity extraction. Both significantly outperform an existing state-of-the-art method. 1
4 0.52139074 38 acl-2010-Automatic Evaluation of Linguistic Quality in Multi-Document Summarization
Author: Emily Pitler ; Annie Louis ; Ani Nenkova
Abstract: To date, few attempts have been made to develop and validate methods for automatic evaluation of linguistic quality in text summarization. We present the first systematic assessment of several diverse classes of metrics designed to capture various aspects of well-written text. We train and test linguistic quality models on consecutive years of NIST evaluation data in order to show the generality of results. For grammaticality, the best results come from a set of syntactic features. Focus, coherence and referential clarity are best evaluated by a class of features measuring local coherence on the basis of cosine similarity between sentences, coreference informa- tion, and summarization specific features. Our best results are 90% accuracy for pairwise comparisons of competing systems over a test set of several inputs and 70% for ranking summaries of a specific input.
5 0.50852507 125 acl-2010-Generating Templates of Entity Summaries with an Entity-Aspect Model and Pattern Mining
Author: Peng Li ; Jing Jiang ; Yinglin Wang
Abstract: In this paper, we propose a novel approach to automatic generation of summary templates from given collections of summary articles. This kind of summary templates can be useful in various applications. We first develop an entity-aspect LDA model to simultaneously cluster both sentences and words into aspects. We then apply frequent subtree pattern mining on the dependency parse trees of the clustered and labeled sentences to discover sentence patterns that well represent the aspects. Key features of our method include automatic grouping of semantically related sentence patterns and automatic identification of template slots that need to be filled in. We apply our method on five Wikipedia entity categories and compare our method with two baseline methods. Both quantitative evaluation based on human judgment and qualitative comparison demonstrate the effectiveness and advantages of our method.
6 0.50319743 254 acl-2010-Using Speech to Reply to SMS Messages While Driving: An In-Car Simulator User Study
7 0.50259519 124 acl-2010-Generating Image Descriptions Using Dependency Relational Patterns
8 0.49271819 80 acl-2010-Cross Lingual Adaptation: An Experiment on Sentiment Classifications
9 0.48508304 188 acl-2010-Optimizing Informativeness and Readability for Sentiment Summarization
10 0.47875625 2 acl-2010-"Was It Good? It Was Provocative." Learning the Meaning of Scalar Adjectives
11 0.47780815 39 acl-2010-Automatic Generation of Story Highlights
12 0.47646037 78 acl-2010-Cross-Language Text Classification Using Structural Correspondence Learning
13 0.46714902 101 acl-2010-Entity-Based Local Coherence Modelling Using Topological Fields
14 0.43901724 136 acl-2010-How Many Words Is a Picture Worth? Automatic Caption Generation for News Images
15 0.43889827 264 acl-2010-Wrapping up a Summary: From Representation to Generation
16 0.43356448 8 acl-2010-A Hybrid Hierarchical Model for Multi-Document Summarization
17 0.40913501 204 acl-2010-Recommendation in Internet Forums and Blogs
18 0.40247968 209 acl-2010-Sentiment Learning on Product Reviews via Sentiment Ontology Tree
19 0.39849174 77 acl-2010-Cross-Language Document Summarization Based on Machine Translation Quality Prediction
20 0.37361073 18 acl-2010-A Study of Information Retrieval Weighting Schemes for Sentiment Analysis
topicId topicWeight
[(14, 0.011), (23, 0.283), (25, 0.055), (39, 0.011), (40, 0.014), (42, 0.032), (59, 0.074), (72, 0.056), (73, 0.054), (76, 0.013), (78, 0.022), (80, 0.011), (83, 0.113), (84, 0.038), (98, 0.1)]
simIndex simValue paperId paperTitle
1 0.78619099 63 acl-2010-Comparable Entity Mining from Comparative Questions
Author: Shasha Li ; Chin-Yew Lin ; Young-In Song ; Zhoujun Li
Abstract: Comparing one thing with another is a typical part of human decision making process. However, it is not always easy to know what to compare and what are the alternatives. To address this difficulty, we present a novel way to automatically mine comparable entities from comparative questions that users posted online. To ensure high precision and high recall, we develop a weakly-supervised bootstrapping method for comparative question identification and comparable entity extraction by leveraging a large online question archive. The experimental results show our method achieves F1measure of 82.5% in comparative question identification and 83.3% in comparable entity extraction. Both significantly outperform an existing state-of-the-art method. 1
same-paper 2 0.76769817 122 acl-2010-Generating Fine-Grained Reviews of Songs from Album Reviews
Author: Swati Tata ; Barbara Di Eugenio
Abstract: Music Recommendation Systems often recommend individual songs, as opposed to entire albums. The challenge is to generate reviews for each song, since only full album reviews are available on-line. We developed a summarizer that combines information extraction and generation techniques to produce summaries of reviews of individual songs. We present an intrinsic evaluation of the extraction components, and of the informativeness of the summaries; and a user study of the impact of the song review summaries on users’ decision making processes. Users were able to make quicker and more informed decisions when presented with the summary as compared to the full album review.
3 0.73291731 107 acl-2010-Exemplar-Based Models for Word Meaning in Context
Author: Katrin Erk ; Sebastian Pado
Abstract: This paper describes ongoing work on distributional models for word meaning in context. We abandon the usual one-vectorper-word paradigm in favor of an exemplar model that activates only relevant occurrences. On a paraphrasing task, we find that a simple exemplar model outperforms more complex state-of-the-art models.
4 0.54813457 209 acl-2010-Sentiment Learning on Product Reviews via Sentiment Ontology Tree
Author: Wei Wei ; Jon Atle Gulla
Abstract: Existing works on sentiment analysis on product reviews suffer from the following limitations: (1) The knowledge of hierarchical relationships of products attributes is not fully utilized. (2) Reviews or sentences mentioning several attributes associated with complicated sentiments are not dealt with very well. In this paper, we propose a novel HL-SOT approach to labeling a product’s attributes and their associated sentiments in product reviews by a Hierarchical Learning (HL) process with a defined Sentiment Ontology Tree (SOT). The empirical analysis against a humanlabeled data set demonstrates promising and reasonable performance of the proposed HL-SOT approach. While this paper is mainly on sentiment analysis on reviews of one product, our proposed HLSOT approach is easily generalized to labeling a mix of reviews of more than one products.
5 0.54781264 184 acl-2010-Open-Domain Semantic Role Labeling by Modeling Word Spans
Author: Fei Huang ; Alexander Yates
Abstract: Most supervised language processing systems show a significant drop-off in performance when they are tested on text that comes from a domain significantly different from the domain of the training data. Semantic role labeling techniques are typically trained on newswire text, and in tests their performance on fiction is as much as 19% worse than their performance on newswire text. We investigate techniques for building open-domain semantic role labeling systems that approach the ideal of a train-once, use-anywhere system. We leverage recently-developed techniques for learning representations of text using latent-variable language models, and extend these techniques to ones that provide the kinds of features that are useful for semantic role labeling. In experiments, our novel system reduces error by 16% relative to the previous state of the art on out-of-domain text.
6 0.5447548 251 acl-2010-Using Anaphora Resolution to Improve Opinion Target Identification in Movie Reviews
7 0.54232168 171 acl-2010-Metadata-Aware Measures for Answer Summarization in Community Question Answering
8 0.54107779 127 acl-2010-Global Learning of Focused Entailment Graphs
9 0.5405004 208 acl-2010-Sentence and Expression Level Annotation of Opinions in User-Generated Discourse
10 0.53942806 174 acl-2010-Modeling Semantic Relevance for Question-Answer Pairs in Web Social Communities
11 0.53912562 134 acl-2010-Hierarchical Sequential Learning for Extracting Opinions and Their Attributes
12 0.53909308 101 acl-2010-Entity-Based Local Coherence Modelling Using Topological Fields
13 0.53833306 113 acl-2010-Extraction and Approximation of Numerical Attributes from the Web
14 0.5364992 109 acl-2010-Experiments in Graph-Based Semi-Supervised Learning Methods for Class-Instance Acquisition
15 0.53576541 1 acl-2010-"Ask Not What Textual Entailment Can Do for You..."
16 0.53221422 18 acl-2010-A Study of Information Retrieval Weighting Schemes for Sentiment Analysis
17 0.53190494 39 acl-2010-Automatic Generation of Story Highlights
18 0.53184587 245 acl-2010-Understanding the Semantic Structure of Noun Phrase Queries
19 0.53096843 153 acl-2010-Joint Syntactic and Semantic Parsing of Chinese
20 0.5296101 230 acl-2010-The Manually Annotated Sub-Corpus: A Community Resource for and by the People