emnlp emnlp2011 emnlp2011-117 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Vahed Qazvinian ; Emily Rosengren ; Dragomir R. Radev ; Qiaozhu Mei
Abstract: A rumor is commonly defined as a statement whose true value is unverifiable. Rumors may spread misinformation (false information) or disinformation (deliberately false information) on a network of people. Identifying rumors is crucial in online social media where large amounts of information are easily spread across a large network by sources with unverified authority. In this paper, we address the problem of rumor detection in microblogs and explore the effectiveness of 3 categories of features: content-based, network-based, and microblog-specific memes for correctly identifying rumors. Moreover, we show how these features are also effective in identifying disinformers, users who endorse a rumor and further help it to spread. We perform our experiments on more than 10,000 manually annotated tweets collected from Twitter and show how our retrieval model achieves more than 0.95 in Mean Average Precision (MAP). Fi- nally, we believe that our dataset is the first large-scale dataset on rumor detection. It can open new dimensions in analyzing online misinformation and other aspects of microblog conversations.
Reference: text
sentIndex sentText sentNum sentScore
1 edu }@ Abstract A rumor is commonly defined as a statement whose true value is unverifiable. [sent-3, score-0.633]
2 Rumors may spread misinformation (false information) or disinformation (deliberately false information) on a network of people. [sent-4, score-0.349]
3 Identifying rumors is crucial in online social media where large amounts of information are easily spread across a large network by sources with unverified authority. [sent-5, score-0.551]
4 In this paper, we address the problem of rumor detection in microblogs and explore the effectiveness of 3 categories of features: content-based, network-based, and microblog-specific memes for correctly identifying rumors. [sent-6, score-0.806]
5 Moreover, we show how these features are also effective in identifying disinformers, users who endorse a rumor and further help it to spread. [sent-7, score-0.733]
6 We perform our experiments on more than 10,000 manually annotated tweets collected from Twitter and show how our retrieval model achieves more than 0. [sent-8, score-0.35]
7 Fi- nally, we believe that our dataset is the first large-scale dataset on rumor detection. [sent-10, score-0.683]
8 1 Introduction A rumor is an unverified and instrumentally relevant statement of information spread among people (DiFonzo and Bordia, 2007). [sent-12, score-0.784]
9 Social psychologists argue that rumors arise in contexts of ambiguity, when the meaning of a situation is not readily apparent, or potential threat, when people feel an acute need for security. [sent-13, score-0.352]
10 For instance rumors about ‘office renovation in a company’ is an example of an ambiguous context, and the rumor that ‘underarm deodorants cause breast cancer’ is an example of a context 1589 in which one’s well-being is at risk (DiFonzo et al. [sent-14, score-0.955]
11 The rapid growth of online social media has made it possible for rumors to spread more quickly. [sent-16, score-0.513]
12 Therefore, it is crucial to design systems that automatically detect misinformation and disinformation (the former often seen as simply false and the latter as deliberately false information). [sent-18, score-0.336]
13 Our definition of a rumor is established based on social psychology, where a rumor is defined as a statement whose truth-value is unverifiable or deliberately false. [sent-19, score-1.311]
14 In-depth rumor analysis such as determining the intent and impact behind the spread of a rumor is a very challenging task and is not possible without first retrieving the complete set of social conversations (e. [sent-20, score-1.395]
15 In our work, we take this first step to retrieve a complete set of tweets that discuss a specific rumor. [sent-23, score-0.343]
16 In the second problem, we try to identify tweets in which the rumor is endorsed (the posters show that they believe the rumor). [sent-26, score-0.932]
17 tc ho2d0s1 in A Nsasotucira tlio Lnan fogru Cagoem Ppruotcaetisosninagl, L pinag uesis 1ti5c8s9–159 , only recently begun to investigate how rumors are manifested and spread differently online. [sent-31, score-0.465]
18 Microblogging services, like Twitter, allow small pieces of information to spread quickly to large audiences, allowing rumors to be created and spread in new ways (Ratkiewicz et al. [sent-32, score-0.578]
19 Related research has used different methods to study the spread of memes and false information on the web. [sent-34, score-0.324]
20 use the evolution of quotes reproduced online to identify memes and track their spread overtime (Leskovec et al. [sent-36, score-0.251]
21 , 2010) created the “Truthy” system, identifying misleading political memes on Twitter using tweet features, including hashtags, links, and mentions. [sent-40, score-0.35]
22 Though our project builds on previous work, our work differs in its general focus on identifying rumors from a corpus of relevant phrases and our attempts to further discriminate between phrases that confirm, refute, question, and simply talk about rumors of interest. [sent-43, score-0.733]
23 They analyze the re-tweet network topology and find that the patterns of propagation in rumors differ from news because rumors tend to be questioned more than news by the Twitter community. [sent-47, score-0.704]
24 2 Sentiment Analysis The automated detection of rumors is similar to traditional NLP sentiment analysis tasks. [sent-49, score-0.429]
25 Though rumor classification is closely related to 1590 opinion mining and sentiment analysis, it presents a different class of problem because we are concerned not just with the opinion of the person posting a tweet, but with whether the statements they post appear controversial. [sent-58, score-0.793]
26 The automatic identification of rumors from a corpus is most closely related to the identification of memes done in (Leskovec et al. [sent-59, score-0.49]
27 Our work presents one of the first attempts at automatic rumor analysis. [sent-61, score-0.603]
28 Because posts are limited to 140 characters, tweets often contain information in an unusually compressed form and, as a result, grammar used may be unconventional. [sent-69, score-0.334]
29 The procedures we used for the collection and analysis of tweets are similar to those described in previous work. [sent-71, score-0.303]
30 However, our goal of developing computational methods to identify rumors being transmitted through tweets differentiates our project. [sent-72, score-0.655]
31 3 Problem Definition Assume we have a set of tweets that are about the same topic that has some controversial aspects. [sent-73, score-0.329]
32 Our objective in this work is two-fold: (1) Extract tweets that are about the controversial aspects of the story and spread misinformation (Rumor retrieval). [sent-74, score-0.58]
33 (2) Identify users who believe that misinformation versus users who refute or question the rumor (Belief NameRumorRegular Expression QueryStatus#tweets obamaIs Barack Obama muslim? [sent-75, score-0.944]
34 st aff & (mi che l le obama | first l ady | 1 st l ady ) partly true 299 palin Sarah Palin getting divorced? [sent-80, score-0.353]
35 pal in & divorce false 4423 Table 1: List of rumor examples and their corresponding queries used to collect data from Twitter classification). [sent-81, score-0.739]
36 The following two tweets are two instances of the tweets written about president Obama and the Muslim world. [sent-82, score-0.64]
37 The first tweet below is about president Obama and Muslim world, where the second tweet spread misinformation that president Obama is Muslim. [sent-83, score-0.685]
38 In the second task, we use the tweets that are flagged as rumorous, and identify users that endorse (believe) the rumor versus users who deny or question it. [sent-92, score-1.115]
39 The following three tweets are about the same story. [sent-93, score-0.303]
40 com) to Twitter and retrieve a large primitive set of tweets that is supposed to have a high recall. [sent-105, score-0.343]
41 This set however, contains a lot of false positives, tweets that match the regexp but are not about the rumor (e. [sent-106, score-1.022]
42 Moreover, a rumor is usually stated using various instances (e. [sent-109, score-0.603]
43 Our goal is then to design a learning framework that filters all such false positives and retrieves various instances of the same rumor Although our second task, belief classification, can be viewed as an opinion mining task, it is substantially different from opinion mining in nature. [sent-112, score-0.837]
44 4 Data As September 2010, Twitter reports that its users publish nearly 95 million tweets per day1 . [sent-118, score-0.379]
45 Our goal in this work was to collect and annotate a large dataset that includes all the tweets that are written about a rumor in a certain period of time. [sent-120, score-0.933]
46 To collect such a complete and self-contained dataset about a rumor, we used the Twitter search API, and retrieved all the tweets that matched a given regular expression. [sent-121, score-0.379]
47 To overcome the rate limit enforced by Twitter, we collected matching tweets once per hour, and remove any duplicates. [sent-123, score-0.303]
48 Each query represents a popular rumor that is listed as “false” or only “partly true” on About. [sent-126, score-0.631]
49 Table 1 lists the rumor examples that we used to collect our dataset along with their corresponding regular expression queries and the number of tweets collected. [sent-128, score-1.013]
50 1 Annotation We asked two annotators to go over all the tweets in the dataset and mark each tweet with a “1” if it is about any of the rumors from Table 1, and with a “0” otherwise. [sent-130, score-0.865]
51 This annotation scheme will be used in our first task to detect false positives, tweets that match the broad regular expressions and are retrieved, but are not about the rumor. [sent-131, score-0.425]
52 For instance, both of the following tweets match the regular expression for the pal in example, but only the second one is rumorous. [sent-132, score-0.383]
53 ly/iNxF” We also asked the annotators to mark each previously annotated rumorous tweet with “1 1” if the tweet poster endorses the rumor and with “12” if the user refutes the rumor, questions its credibility, or is neutral. [sent-136, score-1.171]
54 ly/15StNc” Palin to divorce Our annotation of more than 10,400 tweets shows that %35 of all the instances that matched the regular expressions are false positives, tweets that are not rumor-related but match the initial queries. [sent-139, score-0.791]
55 Moreover, among tweets that are about particular rumors, nearly %43 show the poster believe the rumor, demonstrating the importance of identifying misinformation and those who are misinformed. [sent-140, score-0.496]
56 Table 3 shows that annotators can reach a high agreement in both extracting rumors (κ = 0. [sent-150, score-0.352]
57 5 Approach In this section, we describe a general framework, which given a tweet, predicts (1) whether it is a rumor-related statement, and if so (2) whether the user believes the rumor or not. [sent-153, score-0.692]
58 We process the tweets as they appear in the user timeline, and do not perform any pre-processing. [sent-155, score-0.392]
59 The likelihood ratio expresses eh (o−w) many tgim deast more likely htoheo tweet t e xisunder the positive model than the negative model with respect to fi. [sent-161, score-0.322]
60 , 2010) and present the tweet with 2 different patterns: • Lexical patterns: All the words and segments iLne txhiec tweet are represented as they appear eanntds are tokenized using the space character. [sent-175, score-0.366]
61 Let’s suppose a user ui re-tweets a message t from the user uj (ui: “RT @uj t”). [sent-187, score-0.251]
62 Intuitively, t is more likely to be a rumor if (1) uj has a history of posting or re-tweeting rumors, or (2) ui has posted or retweeted rumors in the past. [sent-188, score-1.083]
63 The first feature is the log-likelihood ratio that ui is under a positive user model (USR1) and the second feature is the log-likelihood ratio that the tweet is re-tweeted from a user (uj) who is under a positive user model than a negative user model (USR2). [sent-193, score-0.817]
64 The distinction between the posting user and the re-tweeted user is important, since some times the users modify the re-tweeted message in a way that changes its meaning and intent. [sent-194, score-0.281]
65 The second user is re-tweeting the first user, but has added more content to the tweet and made it sound rumorous. [sent-196, score-0.31]
66 In our approach, we investigate whether hashtags used in rumor-related tweets are different from other tweets. [sent-209, score-0.392]
67 Moreover, we examine whether people who believe and spread rumors use hashtags that are different from those seen in tweets that deny or question a rumor. [sent-210, score-0.915]
68 Given a set of training tweets of positive and negative examples, we build two statistical models (θ+, θ−), each showing the usage probability distribution of various hashtags. [sent-211, score-0.376]
69 Twitter users share URLs in their tweets to refer to external sources or overcome the length limit forced by Twitter. [sent-217, score-0.379]
70 Intuitively, if a tweet is a positive instance, then it is likely to be similar to the content of URLs shared by other positive tweets. [sent-218, score-0.299]
71 Using the same reasoning, if a tweet is a negative instance, then it should be more similar to the web pages shared by other negative instances. [sent-219, score-0.251]
72 Given a set of training tweets, we fetch all the URLs in these tweets and build θ+ and θ− once for unigrams and once for bigrams. [sent-220, score-0.303]
73 Similar to previous features, we calculate the log-likelihood ratio of the content of each tweet with respect to θ+ and θ−for unigrams (URL1) and bigrams URL2). [sent-222, score-0.287]
74 In the first experiment we assess the effectiveness of the proposed method when employed in an Information Retrieval (IR) framework for rumor retrieval and in the second experiment we employ various features to detect users’ beliefs in rumors. [sent-245, score-0.65]
75 Each relevance set is an annotation of the entire 10,417 tweets, where each tweet is marked as relevant if it matches the regular expression query and is marked as a rumor-related tweet by the annotators. [sent-248, score-0.474]
76 For each query we use 5-fold cross-validation, and predict the relevance of tweets as a function of their features. [sent-250, score-0.331]
77 We use these predictions and rank all the tweets with respect to the query. [sent-251, score-0.303]
78 00 (since it will retrieve all the relevant documents), but will also retrieve false positives, tweets that match the regular expression but are not rumor-related. [sent-262, score-0.536]
79 Table 5 shows the Mean Average Precision (MAP) and Fβ=1 for each method in the rumor retrieval task. [sent-275, score-0.65]
80 This table shows that a method that employs training data to re-rank documents with respect to rumors makes significant improvements over the baselines and outperforms other strong retrieval systems. [sent-276, score-0.399]
81 Figure 1 shows the average precision and recall for our proposed optimization system when content-based (TXT1+TXT2+POS1+POS2), network-based (USR1+USR2), and twitter specific memes (TAG+URL1+URL2) are employed individually. [sent-280, score-0.359]
82 tweets do not share hashtags or are not written based on the contents of external URLs. [sent-283, score-0.392]
83 3 Domain Training Data As our last experiment with rumor retrieval we investigate how much new labeled data from an emergent rumor is required to effectively retrieve instances of that particular rumor. [sent-288, score-1.336]
84 To do this experiment, we use the obama story, which is a large dataset with a significant number of false positive instances. [sent-290, score-0.316]
85 We extract 400 randomly selected tweets from this dataset and keep them for testing. [sent-291, score-0.33]
86 966] Table 5: Mean Average Precision (MAP) and Fβ=1 of each method in the rumor retrieval task. [sent-338, score-0.65]
87 1597 vestigate whether this method, and in particular, the proposed features are useful in detecting users’ beliefs in a rumor that they post about. [sent-393, score-0.603]
88 Unlike retrieval, detecting whether a user endorses a rumor or refutes it may be possible using similar methods regardless of the rumor. [sent-394, score-0.742]
89 We perform this experiment by making a pool of all the tweets that are marked as “rumorous” in the annotation task. [sent-411, score-0.303]
90 Table 2 shows that there are 6,774 such tweets, from which 2,971 show belief and 3,803 tweets show that the user is doubtful, denies, or questions it. [sent-412, score-0.428]
91 Our contributions in this paper are two-fold: (1) We propose a general framework that employs statistical models and maximizes a linear function of log-likelihood ratios to retrieve rumorous tweets that match a more general query. [sent-416, score-0.432]
92 (2) We show the effectiveness of the proposed feature in capturing tweets that show user endorsement. [sent-417, score-0.392]
93 This will help us identify disinformers or users that spread false information in online social media. [sent-418, score-0.348]
94 Our work has resulted in a manually annotated dataset of 10,000 tweets from 5 different controversial topics. [sent-419, score-0.356]
95 To the knowledge of authors this is the first large-scale publicly available rumor dataset, and can open many new dimensions in studying the effects of misinformation or other aspects of information diffusion in online social media. [sent-420, score-0.816]
96 In this paper we effectively retrieve instances of rumors that are already identified and evaluated by an external source such as About. [sent-421, score-0.392]
97 Identifying new emergent rumors directly from the Twitter data is a more challenging task. [sent-423, score-0.395]
98 As our future work, we aim to build a system that employs our findings in this paper and the emergent patterns in the re-tweet network topology to identify whether a new trending topic is a rumor or not. [sent-424, score-0.646]
99 Wartime rumors of waste and special privilege: why some people 1598 believe them. [sent-431, score-0.378]
100 Detecting and tracking the spread of astroturf memes in microblog streams. [sent-539, score-0.276]
wordName wordTfidf (topN-words)
[('rumor', 0.603), ('rumors', 0.352), ('tweets', 0.303), ('twitter', 0.221), ('tweet', 0.183), ('obama', 0.177), ('memes', 0.138), ('misinformation', 0.138), ('muslim', 0.138), ('palin', 0.126), ('spread', 0.113), ('user', 0.089), ('hashtags', 0.089), ('ratkiewicz', 0.088), ('urls', 0.087), ('sentiment', 0.077), ('users', 0.076), ('false', 0.073), ('ratio', 0.066), ('difonzo', 0.063), ('divorce', 0.063), ('rumorous', 0.063), ('sarah', 0.054), ('allport', 0.05), ('regular', 0.049), ('social', 0.048), ('retrieval', 0.047), ('barack', 0.045), ('emergent', 0.043), ('regexp', 0.043), ('leskovec', 0.043), ('opinion', 0.043), ('pang', 0.043), ('url', 0.04), ('retrieve', 0.04), ('positive', 0.039), ('positives', 0.039), ('uj', 0.039), ('hassan', 0.039), ('content', 0.038), ('airfrance', 0.038), ('bifet', 0.038), ('bordia', 0.038), ('cel', 0.038), ('christ', 0.038), ('disinformers', 0.038), ('mendoza', 0.038), ('unverified', 0.038), ('vahed', 0.038), ('api', 0.037), ('microblogs', 0.036), ('belief', 0.036), ('gao', 0.035), ('negative', 0.034), ('ui', 0.034), ('president', 0.034), ('pak', 0.032), ('urban', 0.032), ('michelle', 0.032), ('deny', 0.032), ('expression', 0.031), ('posts', 0.031), ('statement', 0.03), ('credibility', 0.029), ('hussein', 0.029), ('hashtag', 0.029), ('identifying', 0.029), ('rt', 0.029), ('retrieving', 0.028), ('psychology', 0.028), ('kl', 0.028), ('posted', 0.028), ('query', 0.028), ('dataset', 0.027), ('posting', 0.027), ('loglikelihood', 0.027), ('diffusion', 0.027), ('deliberately', 0.027), ('believe', 0.026), ('ratios', 0.026), ('controversial', 0.026), ('ady', 0.025), ('ccoonntteenntt', 0.025), ('cellphone', 0.025), ('clarkson', 0.025), ('disinformation', 0.025), ('disputed', 0.025), ('doubtful', 0.025), ('endorse', 0.025), ('endorses', 0.025), ('ennals', 0.025), ('herman', 0.025), ('honeycutt', 0.025), ('legends', 0.025), ('lemur', 0.025), ('lphone', 0.025), ('microblog', 0.025), ('prashant', 0.025), ('refute', 0.025), ('refutes', 0.025)]
simIndex simValue paperId paperTitle
same-paper 1 0.99999905 117 emnlp-2011-Rumor has it: Identifying Misinformation in Microblogs
Author: Vahed Qazvinian ; Emily Rosengren ; Dragomir R. Radev ; Qiaozhu Mei
Abstract: A rumor is commonly defined as a statement whose true value is unverifiable. Rumors may spread misinformation (false information) or disinformation (deliberately false information) on a network of people. Identifying rumors is crucial in online social media where large amounts of information are easily spread across a large network by sources with unverified authority. In this paper, we address the problem of rumor detection in microblogs and explore the effectiveness of 3 categories of features: content-based, network-based, and microblog-specific memes for correctly identifying rumors. Moreover, we show how these features are also effective in identifying disinformers, users who endorse a rumor and further help it to spread. We perform our experiments on more than 10,000 manually annotated tweets collected from Twitter and show how our retrieval model achieves more than 0.95 in Mean Average Precision (MAP). Fi- nally, we believe that our dataset is the first large-scale dataset on rumor detection. It can open new dimensions in analyzing online misinformation and other aspects of microblog conversations.
2 0.29287112 41 emnlp-2011-Discriminating Gender on Twitter
Author: John D. Burger ; John Henderson ; George Kim ; Guido Zarrella
Abstract: Accurate prediction of demographic attributes from social media and other informal online content is valuable for marketing, personalization, and legal investigation. This paper describes the construction of a large, multilingual dataset labeled with gender, and investigates statistical models for determining the gender of uncharacterized Twitter users. We explore several different classifier types on this dataset. We show the degree to which classifier accuracy varies based on tweet volumes as well as when various kinds of profile metadata are included in the models. We also perform a large-scale human assessment using Amazon Mechanical Turk. Our methods significantly out-perform both baseline models and almost all humans on the same task.
3 0.27358553 89 emnlp-2011-Linguistic Redundancy in Twitter
Author: Fabio Massimo Zanzotto ; Marco Pennaccchiotti ; Kostas Tsioutsiouliklis
Abstract: In the last few years, the interest of the research community in micro-blogs and social media services, such as Twitter, is growing exponentially. Yet, so far not much attention has been paid on a key characteristic of microblogs: the high level of information redundancy. The aim of this paper is to systematically approach this problem by providing an operational definition of redundancy. We cast redundancy in the framework of Textual Entailment Recognition. We also provide quantitative evidence on the pervasiveness of redundancy in Twitter, and describe a dataset of redundancy-annotated tweets. Finally, we present a general purpose system for identifying redundant tweets. An extensive quantitative evaluation shows that our system successfully solves the redundancy detection task, improving over baseline systems with statistical significance.
4 0.25475091 71 emnlp-2011-Identifying and Following Expert Investors in Stock Microblogs
Author: Roy Bar-Haim ; Elad Dinur ; Ronen Feldman ; Moshe Fresko ; Guy Goldstein
Abstract: Information published in online stock investment message boards, and more recently in stock microblogs, is considered highly valuable by many investors. Previous work focused on aggregation of sentiment from all users. However, in this work we show that it is beneficial to distinguish expert users from non-experts. We propose a general framework for identifying expert investors, and use it as a basis for several models that predict stock rise from stock microblogging messages (stock tweets). In particular, we present two methods that combine expert identification and per-user unsupervised learning. These methods were shown to achieve relatively high precision in predicting stock rise, and significantly outperform our baseline. In addition, our work provides an in-depth analysis of the content and potential usefulness of stock tweets.
5 0.21391954 98 emnlp-2011-Named Entity Recognition in Tweets: An Experimental Study
Author: Alan Ritter ; Sam Clark ; Mausam ; Oren Etzioni
Abstract: People tweet more than 100 Million times daily, yielding a noisy, informal, but sometimes informative corpus of 140-character messages that mirrors the zeitgeist in an unprecedented manner. The performance of standard NLP tools is severely degraded on tweets. This paper addresses this issue by re-building the NLP pipeline beginning with part-of-speech tagging, through chunking, to named-entity recognition. Our novel T-NER system doubles F1 score compared with the Stanford NER system. T-NER leverages the redundancy inherent in tweets to achieve this performance, using LabeledLDA to exploit Freebase dictionaries as a source of distant supervision. LabeledLDA outperforms cotraining, increasing F1 by 25% over ten common entity types. Our NLP tools are available at: http : / / github .com/ aritt er /twitte r_nlp
6 0.14124914 33 emnlp-2011-Cooooooooooooooollllllllllllll!!!!!!!!!!!!!! Using Word Lengthening to Detect Sentiment in Microblogs
7 0.12480598 139 emnlp-2011-Twitter Catches The Flu: Detecting Influenza Epidemics using Twitter
8 0.099933393 28 emnlp-2011-Closing the Loop: Fast, Interactive Semi-Supervised Annotation With Queries on Features and Instances
9 0.077238619 133 emnlp-2011-The Imagination of Crowds: Conversational AAC Language Modeling using Crowdsourcing and Large Data Sources
10 0.07182239 38 emnlp-2011-Data-Driven Response Generation in Social Media
11 0.071803428 104 emnlp-2011-Personalized Recommendation of User Comments via Factor Models
12 0.06749481 126 emnlp-2011-Structural Opinion Mining for Graph-based Sentiment Representation
13 0.065309212 120 emnlp-2011-Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions
14 0.063831106 30 emnlp-2011-Compositional Matrix-Space Models for Sentiment Analysis
15 0.060104325 63 emnlp-2011-Harnessing WordNet Senses for Supervised Sentiment Classification
16 0.050630625 17 emnlp-2011-Active Learning with Amazon Mechanical Turk
17 0.045155037 24 emnlp-2011-Bootstrapping Semantic Parsers from Conversations
18 0.043046776 105 emnlp-2011-Predicting Thread Discourse Structure over Technical Web Forums
19 0.042902071 11 emnlp-2011-A Simple Word Trigger Method for Social Tag Suggestion
20 0.04207433 81 emnlp-2011-Learning General Connotation of Words using Graph-based Algorithms
topicId topicWeight
[(0, 0.154), (1, -0.328), (2, 0.403), (3, 0.049), (4, -0.161), (5, 0.019), (6, -0.029), (7, 0.005), (8, 0.049), (9, -0.031), (10, 0.004), (11, 0.037), (12, 0.054), (13, -0.056), (14, 0.018), (15, -0.015), (16, -0.049), (17, 0.039), (18, 0.026), (19, -0.006), (20, 0.045), (21, -0.05), (22, -0.004), (23, 0.068), (24, 0.064), (25, 0.003), (26, 0.034), (27, -0.049), (28, -0.035), (29, 0.041), (30, -0.044), (31, -0.036), (32, -0.042), (33, -0.022), (34, -0.039), (35, -0.03), (36, 0.027), (37, 0.072), (38, 0.03), (39, 0.024), (40, -0.01), (41, 0.077), (42, -0.008), (43, -0.008), (44, 0.027), (45, -0.002), (46, 0.018), (47, -0.01), (48, 0.022), (49, -0.014)]
simIndex simValue paperId paperTitle
same-paper 1 0.95544404 117 emnlp-2011-Rumor has it: Identifying Misinformation in Microblogs
Author: Vahed Qazvinian ; Emily Rosengren ; Dragomir R. Radev ; Qiaozhu Mei
Abstract: A rumor is commonly defined as a statement whose true value is unverifiable. Rumors may spread misinformation (false information) or disinformation (deliberately false information) on a network of people. Identifying rumors is crucial in online social media where large amounts of information are easily spread across a large network by sources with unverified authority. In this paper, we address the problem of rumor detection in microblogs and explore the effectiveness of 3 categories of features: content-based, network-based, and microblog-specific memes for correctly identifying rumors. Moreover, we show how these features are also effective in identifying disinformers, users who endorse a rumor and further help it to spread. We perform our experiments on more than 10,000 manually annotated tweets collected from Twitter and show how our retrieval model achieves more than 0.95 in Mean Average Precision (MAP). Fi- nally, we believe that our dataset is the first large-scale dataset on rumor detection. It can open new dimensions in analyzing online misinformation and other aspects of microblog conversations.
2 0.95279002 71 emnlp-2011-Identifying and Following Expert Investors in Stock Microblogs
Author: Roy Bar-Haim ; Elad Dinur ; Ronen Feldman ; Moshe Fresko ; Guy Goldstein
Abstract: Information published in online stock investment message boards, and more recently in stock microblogs, is considered highly valuable by many investors. Previous work focused on aggregation of sentiment from all users. However, in this work we show that it is beneficial to distinguish expert users from non-experts. We propose a general framework for identifying expert investors, and use it as a basis for several models that predict stock rise from stock microblogging messages (stock tweets). In particular, we present two methods that combine expert identification and per-user unsupervised learning. These methods were shown to achieve relatively high precision in predicting stock rise, and significantly outperform our baseline. In addition, our work provides an in-depth analysis of the content and potential usefulness of stock tweets.
3 0.86338854 89 emnlp-2011-Linguistic Redundancy in Twitter
Author: Fabio Massimo Zanzotto ; Marco Pennaccchiotti ; Kostas Tsioutsiouliklis
Abstract: In the last few years, the interest of the research community in micro-blogs and social media services, such as Twitter, is growing exponentially. Yet, so far not much attention has been paid on a key characteristic of microblogs: the high level of information redundancy. The aim of this paper is to systematically approach this problem by providing an operational definition of redundancy. We cast redundancy in the framework of Textual Entailment Recognition. We also provide quantitative evidence on the pervasiveness of redundancy in Twitter, and describe a dataset of redundancy-annotated tweets. Finally, we present a general purpose system for identifying redundant tweets. An extensive quantitative evaluation shows that our system successfully solves the redundancy detection task, improving over baseline systems with statistical significance.
4 0.84016567 41 emnlp-2011-Discriminating Gender on Twitter
Author: John D. Burger ; John Henderson ; George Kim ; Guido Zarrella
Abstract: Accurate prediction of demographic attributes from social media and other informal online content is valuable for marketing, personalization, and legal investigation. This paper describes the construction of a large, multilingual dataset labeled with gender, and investigates statistical models for determining the gender of uncharacterized Twitter users. We explore several different classifier types on this dataset. We show the degree to which classifier accuracy varies based on tweet volumes as well as when various kinds of profile metadata are included in the models. We also perform a large-scale human assessment using Amazon Mechanical Turk. Our methods significantly out-perform both baseline models and almost all humans on the same task.
5 0.839315 139 emnlp-2011-Twitter Catches The Flu: Detecting Influenza Epidemics using Twitter
Author: Eiji ARAMAKI ; Sachiko MASKAWA ; Mizuki MORITA
Abstract: Sachiko MASKAWA The University of Tokyo Tokyo, Japan s achi ko . mas kawa @ gma i . com l Mizuki MORITA National Institute of Biomedical Innovation Osaka, Japan mori ta . mi zuki @ gmai l com . posts more than 5.5 million messages (tweets) every day (reported by Twitter.com in March 201 1). With the recent rise in popularity and scale of social media, a growing need exists for systems that can extract useful information from huge amounts of data. We address the issue of detecting influenza epidemics. First, the proposed system extracts influenza related tweets using Twitter API. Then, only tweets that mention actual influenza patients are extracted by the support vector machine (SVM) based classifier. The experiment results demonstrate the feasibility of the proposed approach (0.89 correlation to the gold standard). Especially at the outbreak and early spread (early epidemic stage), the proposed method shows high correlation (0.97 correlation), which outperforms the state-of-the-art methods. This paper describes that Twitter texts reflect the real world, and that NLP techniques can be applied to extract only tweets that contain useful information. 1
6 0.48603588 33 emnlp-2011-Cooooooooooooooollllllllllllll!!!!!!!!!!!!!! Using Word Lengthening to Detect Sentiment in Microblogs
7 0.48121223 98 emnlp-2011-Named Entity Recognition in Tweets: An Experimental Study
8 0.32927859 104 emnlp-2011-Personalized Recommendation of User Comments via Factor Models
10 0.29300985 133 emnlp-2011-The Imagination of Crowds: Conversational AAC Language Modeling using Crowdsourcing and Large Data Sources
11 0.22977021 126 emnlp-2011-Structural Opinion Mining for Graph-based Sentiment Representation
12 0.22538319 120 emnlp-2011-Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions
13 0.2190828 38 emnlp-2011-Data-Driven Response Generation in Social Media
14 0.2028574 30 emnlp-2011-Compositional Matrix-Space Models for Sentiment Analysis
15 0.19699876 82 emnlp-2011-Learning Local Content Shift Detectors from Document-level Information
16 0.19204512 81 emnlp-2011-Learning General Connotation of Words using Graph-based Algorithms
18 0.18418419 11 emnlp-2011-A Simple Word Trigger Method for Social Tag Suggestion
19 0.18346025 105 emnlp-2011-Predicting Thread Discourse Structure over Technical Web Forums
20 0.16613251 24 emnlp-2011-Bootstrapping Semantic Parsers from Conversations
topicId topicWeight
[(15, 0.017), (23, 0.096), (36, 0.029), (37, 0.025), (45, 0.057), (52, 0.075), (53, 0.02), (54, 0.018), (57, 0.015), (61, 0.326), (62, 0.016), (64, 0.014), (66, 0.027), (69, 0.012), (79, 0.025), (82, 0.017), (87, 0.018), (96, 0.026), (97, 0.026), (98, 0.044)]
simIndex simValue paperId paperTitle
same-paper 1 0.70204574 117 emnlp-2011-Rumor has it: Identifying Misinformation in Microblogs
Author: Vahed Qazvinian ; Emily Rosengren ; Dragomir R. Radev ; Qiaozhu Mei
Abstract: A rumor is commonly defined as a statement whose true value is unverifiable. Rumors may spread misinformation (false information) or disinformation (deliberately false information) on a network of people. Identifying rumors is crucial in online social media where large amounts of information are easily spread across a large network by sources with unverified authority. In this paper, we address the problem of rumor detection in microblogs and explore the effectiveness of 3 categories of features: content-based, network-based, and microblog-specific memes for correctly identifying rumors. Moreover, we show how these features are also effective in identifying disinformers, users who endorse a rumor and further help it to spread. We perform our experiments on more than 10,000 manually annotated tweets collected from Twitter and show how our retrieval model achieves more than 0.95 in Mean Average Precision (MAP). Fi- nally, we believe that our dataset is the first large-scale dataset on rumor detection. It can open new dimensions in analyzing online misinformation and other aspects of microblog conversations.
2 0.67025572 39 emnlp-2011-Discovering Morphological Paradigms from Plain Text Using a Dirichlet Process Mixture Model
Author: Markus Dreyer ; Jason Eisner
Abstract: We present an inference algorithm that organizes observed words (tokens) into structured inflectional paradigms (types). It also naturally predicts the spelling of unobserved forms that are missing from these paradigms, and discovers inflectional principles (grammar) that generalize to wholly unobserved words. Our Bayesian generative model of the data explicitly represents tokens, types, inflections, paradigms, and locally conditioned string edits. It assumes that inflected word tokens are generated from an infinite mixture of inflectional paradigms (string tuples). Each paradigm is sampled all at once from a graphical model, whose potential functions are weighted finitestate transducers with language-specific parameters to be learned. These assumptions naturally lead to an elegant empirical Bayes inference procedure that exploits Monte Carlo EM, belief propagation, and dynamic programming. Given 50–100 seed paradigms, adding a 10million-word corpus reduces prediction error for morphological inflections by up to 10%.
3 0.41338366 56 emnlp-2011-Exploring Supervised LDA Models for Assigning Attributes to Adjective-Noun Phrases
Author: Matthias Hartung ; Anette Frank
Abstract: This paper introduces an attribute selection task as a way to characterize the inherent meaning of property-denoting adjectives in adjective-noun phrases, such as e.g. hot in hot summer denoting the attribute TEMPERATURE, rather than TASTE. We formulate this task in a vector space model that represents adjectives and nouns as vectors in a semantic space defined over possible attributes. The vectors incorporate latent semantic information obtained from two variants of LDA topic models. Our LDA models outperform previous approaches on a small set of 10 attributes with considerable gains on sparse representations, which highlights the strong smoothing power of LDA models. For the first time, we extend the attribute selection task to a new data set with more than 200 classes. We observe that large-scale attribute selection is a hard problem, but a subset of attributes performs robustly on the large scale as well. Again, the LDA models outperform the VSM baseline.
4 0.41120359 41 emnlp-2011-Discriminating Gender on Twitter
Author: John D. Burger ; John Henderson ; George Kim ; Guido Zarrella
Abstract: Accurate prediction of demographic attributes from social media and other informal online content is valuable for marketing, personalization, and legal investigation. This paper describes the construction of a large, multilingual dataset labeled with gender, and investigates statistical models for determining the gender of uncharacterized Twitter users. We explore several different classifier types on this dataset. We show the degree to which classifier accuracy varies based on tweet volumes as well as when various kinds of profile metadata are included in the models. We also perform a large-scale human assessment using Amazon Mechanical Turk. Our methods significantly out-perform both baseline models and almost all humans on the same task.
5 0.40016764 37 emnlp-2011-Cross-Cutting Models of Lexical Semantics
Author: Joseph Reisinger ; Raymond Mooney
Abstract: Context-dependent word similarity can be measured over multiple cross-cutting dimensions. For example, lung and breath are similar thematically, while authoritative and superficial occur in similar syntactic contexts, but share little semantic similarity. Both of these notions of similarity play a role in determining word meaning, and hence lexical semantic models must take them both into account. Towards this end, we develop a novel model, Multi-View Mixture (MVM), that represents words as multiple overlapping clusterings. MVM finds multiple data partitions based on different subsets of features, subject to the marginal constraint that feature subsets are distributed according to Latent Dirich- let Allocation. Intuitively, this constraint favors feature partitions that have coherent topical semantics. Furthermore, MVM uses soft feature assignment, hence the contribution of each data point to each clustering view is variable, isolating the impact of data only to views where they assign the most features. Through a series of experiments, we demonstrate the utility of MVM as an inductive bias for capturing relations between words that are intuitive to humans, outperforming related models such as Latent Dirichlet Allocation.
6 0.37504885 98 emnlp-2011-Named Entity Recognition in Tweets: An Experimental Study
7 0.36872235 139 emnlp-2011-Twitter Catches The Flu: Detecting Influenza Epidemics using Twitter
8 0.36765981 33 emnlp-2011-Cooooooooooooooollllllllllllll!!!!!!!!!!!!!! Using Word Lengthening to Detect Sentiment in Microblogs
9 0.36660087 108 emnlp-2011-Quasi-Synchronous Phrase Dependency Grammars for Machine Translation
10 0.36420804 30 emnlp-2011-Compositional Matrix-Space Models for Sentiment Analysis
11 0.36407518 71 emnlp-2011-Identifying and Following Expert Investors in Stock Microblogs
12 0.36258778 104 emnlp-2011-Personalized Recommendation of User Comments via Factor Models
13 0.36078274 89 emnlp-2011-Linguistic Redundancy in Twitter
14 0.3594963 1 emnlp-2011-A Bayesian Mixture Model for PoS Induction Using Multiple Features
15 0.35924867 120 emnlp-2011-Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions
16 0.35883117 28 emnlp-2011-Closing the Loop: Fast, Interactive Semi-Supervised Annotation With Queries on Features and Instances
17 0.35842675 35 emnlp-2011-Correcting Semantic Collocation Errors with L1-induced Paraphrases
18 0.35769123 68 emnlp-2011-Hypotheses Selection Criteria in a Reranking Framework for Spoken Language Understanding
19 0.35744724 6 emnlp-2011-A Generate and Rank Approach to Sentence Paraphrasing
20 0.35656098 23 emnlp-2011-Bootstrapped Named Entity Recognition for Product Attribute Extraction