acl acl2013 acl2013-169 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Oleg Rokhlenko ; Idan Szpektor
Abstract: We introduce the novel task of automatically generating questions that are relevant to a text but do not appear in it. One motivating example of its application is for increasing user engagement around news articles by suggesting relevant comparable questions, such as “is Beyonce a better singer than Madonna?”, for the user to answer. We present the first algorithm for the task, which consists of: (a) offline construction of a comparable question template database; (b) ranking of relevant templates to a given article; and (c) instantiation of templates only with entities in the article whose comparison under the template’s relation makes sense. We tested the suggestions generated by our algorithm via a Mechanical Turk experiment, which showed a significant improvement over the strongest baseline of more than 45% in all metrics.
Reference: text
sentIndex sentText sentNum sentScore
1 com Abstract We introduce the novel task of automatically generating questions that are relevant to a text but do not appear in it. [sent-3, score-0.597]
2 One motivating example of its application is for increasing user engagement around news articles by suggesting relevant comparable questions, such as “is Beyonce a better singer than Madonna? [sent-4, score-0.648]
3 In this paper we propose a new way to increase user engagement around news articles, namely suggesting questions for the user to answer, which are related to the viewed article. [sent-12, score-0.614]
4 One approach for generating comparable questions would be to employ traditional question generation, which syntactically transform assertions in a given text into questions (Mitkov et al. [sent-23, score-1.064]
5 Sadly, fun and engaging comparative questions are typically not found within the text of news articles. [sent-26, score-0.552]
6 A different approach would be to find concrete relevant questions within external collections of manually generated comparable questions. [sent-27, score-0.793]
7 However, it is highly unlikely that such sources will contain enough relevant questions for any news article due to typical sparseness issues as well as differences in interests between askers in CQA sites and news reporters. [sent-30, score-1.09]
8 To better address the motivating application above, we propose the novel task of automatically suggesting comparative questions that are relevant to a given input news article but do not appear in it. [sent-31, score-1.034]
9 To achieve broad coverage for our task, we present an algorithm that generates synthetic concrete questions from question templates, such as “Who is a better actor: #1 or #2? [sent-32, score-0.693]
10 An offline part constructs a database of comparative question templates that appear in a large question corpus. [sent-35, score-0.692]
11 c A2s0s1o3ci Aatsiosonc fioartio Cno fmorpu Ctoamtiopnuatalt Lioin gauli Lsitnicgsu,i psatgices 742–751, Figure 1: An example news article from OMG! [sent-38, score-0.38]
12 plates for the article by matching between the article content and typical template contexts. [sent-39, score-0.779]
13 The algorithm then instantiates each relevant template with two entities that appear in the article. [sent-40, score-0.678]
14 To test the performance of our algorithm, we conducted a Mechanical Turk experiment that assessed the quality of suggested questions for news articles on celebrities. [sent-45, score-0.534]
15 We compared our algorithm to a random baseline and to a partial version of our algorithm that includes a template relevance component but lacks filtering of candidate instantiations. [sent-46, score-0.591]
16 These results point at the importance of both picking relevant templates and smart instantiation selection to the quality of generated questions. [sent-48, score-0.541]
17 1 Motivation Given a news article, our algorithm generates a set of comparable questions for the article from question templates, e. [sent-52, score-1.152]
18 Though the template words typically do not appear in the article, they need to be relevant to it’s content, that is they should correspond to one of the main themes in the article or to one of the pub- Figure 2: A high-level overview of the comparable question generation algorithm. [sent-56, score-1.086]
19 ” is relevant to the article in Figure 1, while “who is faster #1 or #2 ? [sent-60, score-0.505]
20 Looking at the structure of comparable questions, we observed that a specific comparable relation, such as ‘better dad’ and ‘faster’, can usually be combined with named entities in several syntactic ways to construct a concrete question. [sent-63, score-0.708]
21 Using the above generic templates, ‘Jet Li’ and ‘Jackie Chan’ can be combined with the comparable relation ‘better fighter’ to generate “who is a better fighter: Jackie Chan or Jet Li? [sent-69, score-0.502]
22 Following, our algorithm separately maintains comparable relations and generic templates. [sent-72, score-0.498]
23 In this paper we constrain ourselves to generate comparable questions between entities that appear in the article. [sent-73, score-0.82]
24 Our algorithm thus needs to assess whether an instantiation is correct, that is whether the comparison between the two entities makes sense under the specific template. [sent-77, score-0.495]
25 The algorithm’s offline part constructs, from a large collection of questions, a database of comparable relations, together with their typical contexts. [sent-85, score-0.38]
26 It also extracts generic templates and the mapping to the relations that may instantiate them. [sent-86, score-0.445]
27 From this database, we learn: (a) a context profile per template for relevance matching; (b) a single entity model per template slot that identify valid instantiations; and (c) an entity pair model that detects pairs of entities that can be compared together under the template. [sent-87, score-0.977]
28 In the online part, these three models are applied to rank relevant templates for a given article and to generate only correct questions with respect to template instantiation. [sent-88, score-1.171]
29 3 Comparable Question Mining To suggest comparable questions our algorithm needs a database of question templates. [sent-90, score-0.849]
30 Specifically, in this study we utilize all questions submitted to Yahoo! [sent-92, score-0.423]
31 We next describe how comparable relations and generic comparable templates are extracted from this corpus. [sent-94, score-0.765]
32 1 Comparable Relation Extraction An important observation for the task of comparable relation extraction is that many relations are complex multiword expressions, and thus their automatic detection is not trivial. [sent-96, score-0.472]
33 Examples for such relations are marked in the questions “Who is the best rapper alive, Eminem or Jay-z? [sent-97, score-0.529]
34 As a pre-processing step for detecting comparable relations, our extraction algorithm identifies all the named entities of interest in our corpus, keeping only questions that contain at least two entities. [sent-103, score-0.889]
35 To train the CRF model, the authors manually tagged all comparable relation words in approximately 300 transformed questions in the filtered corpus. [sent-113, score-0.75]
36 The trained model was then applied to all other questions in the filtered corpus. [sent-122, score-0.399]
37 From this output we constructed a database consisting of all occurring relations; each relation is accompanied by its supporting questions, those questions in which the relation occurrences were found. [sent-124, score-0.795]
38 Yet, our filtering above of relations with low support left us with virtually 100% precision per relation and per occurrence. [sent-127, score-0.384]
39 2 Comparable Template Extraction Our second mining task is to extract generic comparable templates that appear in our corpus, as well as identifying which comparable relation can instantiate which generic template. [sent-140, score-1.039]
40 To this end, we replace each recognized relation sequence with a variable RE in the support questions annotated with #i variables. [sent-141, score-0.522]
41 While some questions contain many details besides the comparable generic template, others are simpler and contain only the generic template. [sent-146, score-0.781]
42 Finally, for each comparable relation we mark as applicable only generic templates that occur at least once in the supporting questions of this relation. [sent-150, score-1.011]
43 The algorithm starts with identifying the comparable relations in our database that are relevant to the article. [sent-157, score-0.637]
44 For each relevant relation, we then generate concrete questions by picking generic templates that are applicable for this relation and instantiating them with pairs of named entities appearing in the article. [sent-158, score-1.379]
45 Yet, as discussed before, only for some entity pairs the comparison under the specific relation makes sense, a quality which we refer to as instantiation correctness (see Section 2). [sent-159, score-0.663]
46 We next detail the two aspects of the online part: ranking relevant relations and correctly instantiating relations. [sent-161, score-0.398]
47 1 Ranking relevant relations To assess how relevant a given comparable relation is to an article, we model the relation’s typical context as a distribution over latent semantic topics. [sent-163, score-0.9]
48 To train an LDA model, we constructed for each comparable relation a pseudo-document consisting of all questions that contain this relation in our corpus (the supporting questions). [sent-166, score-0.911]
49 Then, a cosine similarity between this distribution and the context profile of each comparable relation in our database is computed and taken as the relevance score for this relation. [sent-172, score-0.61]
50 Finally, we rank all relations according to their relevance score and pick the top M as candidates for instantiation (M=3 in our experiment). [sent-173, score-0.524]
51 2 Correctly instantiating relations To generate useful questions from relevant comparable relations, we need to retain only correct instantiations of these relations. [sent-175, score-1.058]
52 For example, for the relation ‘is faster’, the single entity filter looks for athletes of all kinds, for whom this comparison is of interest to the reader. [sent-177, score-0.386]
53 The first is DBPedia3, which contains structured information on entries in Wikipedia, many of them are named entities that appear in news articles. [sent-186, score-0.411]
54 One way to utilize the CQA question corpus could be to extract co-occurring words with each target entity as relevant contexts. [sent-198, score-0.531]
55 For each named entity, we construct a histogram of the number of questions containing it that are assigned to each category. [sent-203, score-0.513]
56 2 Single entity filtering We view the task of single entity filtering as a classification task. [sent-209, score-0.52]
57 Positive examples are the entities that instantiate this relation in our CQA corpus. [sent-211, score-0.389]
58 As negative examples, we take named entities that were never seen instantiating the relation in the corpus, but still occurred in some questions. [sent-212, score-0.52]
59 We note that our named entity tagger could recognize more than 200,000 named entities, and most of them are negative for a given relation. [sent-213, score-0.383]
60 For each relation we select negative examples by sampling uniformly from its negative entity list, assuming that the probability of hitting false negatives is low for such a long list. [sent-214, score-0.389]
61 3 Entity pair filtering Similar to single entity filtering, we view the task of filtering entity pairs as a classification task, training a separate classifier for each relation. [sent-227, score-0.557]
62 Yet, the space of all the pairs that never instantiated the relation is huge, and the set of positive examples is relatively much smaller compared to the situation in the single entity classifier. [sent-229, score-0.415]
63 Therefore, we generate negative examples by sampling only from pairs of named entities that both pass the single entity filter for this relation. [sent-231, score-0.615]
64 For example, named entities that pass the single entity filtering for “be funny”, include Jay Leno, David Letterman (American TV hosts), Jim Carrey, and Steve Mar- two single entity feature vectors fa and fb tin (actors). [sent-239, score-0.725]
65 1 Experimental Settings To evaluate our algorithm’s performance, we designed a Mechanical Turk (MTurk) experiment in which human annotators assess the quality of the questions that our algorithm generates for a sample of news articles. [sent-242, score-0.565]
66 Test articles were selected by first randomly sampling 5,000 news article from those that were posted on OMG! [sent-245, score-0.458]
67 For each test article our algorithm obtained the top three relevant comparable relations, and for each relation selected the best instantiation (if exists). [sent-249, score-1.106]
68 The first random baseline chooses a rela- tion randomly out of all possible relations in the database and then instantiates it with a random pair of entities that appear in the article. [sent-251, score-0.47]
69 com/ 747 FR uaelneldavoalgmnocebritabsheamsleinlen Re3l52e749v% %anceCor574e73 c% %tnes Table 1: Relevance and correctness percentage by tested algorithm relation to the article based on our algorithm, but still instantiates it with a random pair. [sent-254, score-0.695]
70 In addition, it enabled us to measure the relative contribution of the instantiation mod- els on top of relevance model. [sent-257, score-0.438]
71 Each article was evaluated by 10 MTurk workers, which were asked to mark for each displayed question whether it is relevant and whether it is correct (see Section 2 for relevance and correctness definitions). [sent-258, score-0.916]
72 2 Results For each tested algorithm, we separately counted the percentage of annotations that marked each question as relevant and the percentage of annotations that marked each question as instantiated correctly, denoted relevance score and correctness score. [sent-262, score-0.873]
73 metric, since both the full algorithm and the relevance baseline use the same relevance component to rank relations by. [sent-276, score-0.557]
74 One explanation for this is that sometimes the instantiation filter eliminates all possible entity pairs for some relation that is incorrectly considered relevant by the algorithm. [sent-277, score-0.781]
75 To illustrate the differences between baselines and the full algorithm, Table 2 presents an example article together with the suggested generated questions by each algorithm. [sent-280, score-0.652]
76 The random baseline picked an irrelevant relation, and while the relevance baseline selected a relevant relation, “a better president”, it was instantiated incorrectly. [sent-281, score-0.458]
77 The full algorithm, on the other hand, both chose relevant relations for all three questions and instantiated them correctly. [sent-282, score-0.761]
78 Especially, the incorrectly instantiated relation in the relevance baseline is now correctly instantiated with plausible presidential candidates. [sent-283, score-0.544]
79 It is possible that for some articles not all three questions will be generated, due to instantiation filtering. [sent-289, score-0.663]
80 We found that for 85% of the articles all three questions were generated. [sent-290, score-0.442]
81 For the remaining 15% at least one question was always generated, and for 31 of them two questions were composed. [sent-291, score-0.507]
82 3 Error Analysis To better understand the performance of our algorithm, we looked at some low quality questions that were generated, either due to incorrect instantiation or due to irrelevance to the article. [sent-305, score-0.585]
83 Starting with relevance, one of the repeating mistakes was promoting relations that are related to a list of named entities in the article, but not to its main theme. [sent-306, score-0.381]
84 For example, the relation ‘who is a better actor’ was incorrectly ranked high for an article about Ricky Gervais claiming that he has been asked to host Globes again after he offended Angelina Jolie, Johnny Depp, Robert Downey Jr. [sent-307, score-0.446]
85 Yet, unlike other cases, here entity filtering does not help ignoring such errors, since the same entities that triggered the ranking of the relation are also valid instantiations for it. [sent-310, score-0.725]
86 Yet, she was Michael Jackson’s wife and they appear together in a lot of questions in our corpus. [sent-317, score-0.423]
87 To the best of our knowledge, there is no prior work on our task, which is to generate relevant synthetic questions whose content, except for the arguments, might not appear in the text. [sent-324, score-0.688]
88 (2010) comparative questions are identified as an intermediate step for the task of extracting compared entities, which are unknown in their setting. [sent-329, score-0.421]
89 We, on the other hand, detect the compared entities in a pre-processing step, and our target is the extraction of the comparable relations given known candidate entities. [sent-330, score-0.479]
90 Our algorithm ranks relevant templates based on the similarity between an article’s content and the typical context of each relation. [sent-331, score-0.435]
91 Prior work rank relevant concrete questions to a given input question, focusing on strong lexical similarities (Jeon et al. [sent-332, score-0.6]
92 7 Conclusions We introduced the novel task of automatically generating synthetic comparable questions that are relevant to a given news article but do not necessarily appear in it. [sent-345, score-1.222]
93 The offline part identifies comparable relations in a large collection of questions. [sent-347, score-0.381]
94 Its output is a database of comparable relations together with a context profile for each relation and models that detect correct instantiations of this relation, all learned from the question corpus. [sent-348, score-0.795]
95 In the online part, given a news article, the algorithm identifies relevant comparable relations based on the similarity between the article content and each relation’s context profile. [sent-349, score-0.94]
96 Then, relevant relations are instantiated only with pairs of named entities from the article whose comparison makes sense by applying the instantiation correctness models to candidate pairs. [sent-350, score-1.295]
97 These results show that our supervised filtering methods are successful in keeping only correct pairs, but they also serve as an additional filtering for relevant relations, on top of context matching. [sent-354, score-0.384]
98 In future work, we want to generate more diverse and intriguing questions by selecting relevant named entities for template instantiation that do not appear in the article. [sent-355, score-1.277]
99 Finding similar questions in collaborative question answering archives: toward bootstrapping-based equivalent pattern learning. [sent-407, score-0.507]
100 Finding similar questions in large question and answer archives. [sent-421, score-0.507]
wordName wordTfidf (topN-words)
[('questions', 0.364), ('article', 0.288), ('instantiation', 0.221), ('comparable', 0.193), ('relevance', 0.182), ('relevant', 0.174), ('entities', 0.165), ('template', 0.16), ('relation', 0.158), ('entity', 0.155), ('templates', 0.146), ('question', 0.143), ('correctness', 0.129), ('yahoo', 0.128), ('relations', 0.121), ('generic', 0.112), ('filtering', 0.105), ('instantiations', 0.103), ('instantiated', 0.102), ('named', 0.095), ('cqa', 0.092), ('news', 0.092), ('articles', 0.078), ('omg', 0.077), ('jindal', 0.077), ('jolie', 0.077), ('database', 0.077), ('crf', 0.075), ('filter', 0.073), ('algorithm', 0.072), ('angelina', 0.071), ('lda', 0.071), ('generation', 0.069), ('offline', 0.067), ('jet', 0.066), ('instantiate', 0.066), ('instantiating', 0.064), ('engagement', 0.064), ('concrete', 0.062), ('appear', 0.059), ('utilize', 0.059), ('fighter', 0.058), ('dad', 0.058), ('madonna', 0.058), ('sex', 0.058), ('comparative', 0.057), ('dbpedia', 0.055), ('histogram', 0.054), ('synthetic', 0.052), ('jackie', 0.05), ('pass', 0.05), ('instantiates', 0.048), ('user', 0.047), ('yet', 0.047), ('actors', 0.046), ('rus', 0.046), ('answers', 0.045), ('alive', 0.044), ('beckham', 0.044), ('beyonce', 0.044), ('carrey', 0.044), ('cornerback', 0.044), ('debbie', 0.044), ('globes', 0.044), ('leno', 0.044), ('letterman', 0.044), ('olney', 0.044), ('rapper', 0.044), ('ringo', 0.044), ('rowe', 0.044), ('schuth', 0.044), ('starr', 0.044), ('turk', 0.044), ('typical', 0.043), ('culotta', 0.043), ('faster', 0.043), ('heilman', 0.039), ('mitkov', 0.039), ('chan', 0.039), ('generate', 0.039), ('ranking', 0.039), ('colored', 0.039), ('haifa', 0.039), ('engaging', 0.039), ('funnier', 0.039), ('guitarist', 0.039), ('hosts', 0.039), ('pitt', 0.039), ('mechanical', 0.038), ('selectional', 0.038), ('supporting', 0.038), ('negative', 0.038), ('libsvm', 0.037), ('sites', 0.037), ('classifier', 0.037), ('assess', 0.037), ('stroudsburg', 0.036), ('topics', 0.036), ('enabled', 0.035), ('filtered', 0.035)]
simIndex simValue paperId paperTitle
same-paper 1 0.99999875 169 acl-2013-Generating Synthetic Comparable Questions for News Articles
Author: Oleg Rokhlenko ; Idan Szpektor
Abstract: We introduce the novel task of automatically generating questions that are relevant to a text but do not appear in it. One motivating example of its application is for increasing user engagement around news articles by suggesting relevant comparable questions, such as “is Beyonce a better singer than Madonna?”, for the user to answer. We present the first algorithm for the task, which consists of: (a) offline construction of a comparable question template database; (b) ranking of relevant templates to a given article; and (c) instantiation of templates only with entities in the article whose comparison under the template’s relation makes sense. We tested the suggestions generated by our algorithm via a Mechanical Turk experiment, which showed a significant improvement over the strongest baseline of more than 45% in all metrics.
2 0.22855891 272 acl-2013-Paraphrase-Driven Learning for Open Question Answering
Author: Anthony Fader ; Luke Zettlemoyer ; Oren Etzioni
Abstract: We study question answering as a machine learning problem, and induce a function that maps open-domain questions to queries over a database of web extractions. Given a large, community-authored, question-paraphrase corpus, we demonstrate that it is possible to learn a semantic lexicon and linear ranking function without manually annotating questions. Our approach automatically generalizes a seed lexicon and includes a scalable, parallelized perceptron parameter estimation scheme. Experiments show that our approach more than quadruples the recall of the seed lexicon, with only an 8% loss in precision.
Author: Guangyou Zhou ; Fang Liu ; Yang Liu ; Shizhu He ; Jun Zhao
Abstract: Community question answering (CQA) has become an increasingly popular research topic. In this paper, we focus on the problem of question retrieval. Question retrieval in CQA can automatically find the most relevant and recent questions that have been solved by other users. However, the word ambiguity and word mismatch problems bring about new challenges for question retrieval in CQA. State-of-the-art approaches address these issues by implicitly expanding the queried questions with additional words or phrases using monolingual translation models. While useful, the effectiveness of these models is highly dependent on the availability of quality parallel monolingual corpora (e.g., question-answer pairs) in the absence of which they are troubled by noise issue. In this work, we propose an alternative way to address the word ambiguity and word mismatch problems by taking advantage of potentially rich semantic information drawn from other languages. Our proposed method employs statistical machine translation to improve question retrieval and enriches the question representation with the translated words from other languages via matrix factorization. Experiments conducted on a real CQA data show that our proposed approach is promising.
4 0.16933776 291 acl-2013-Question Answering Using Enhanced Lexical Semantic Models
Author: Wen-tau Yih ; Ming-Wei Chang ; Christopher Meek ; Andrzej Pastusiak
Abstract: In this paper, we study the answer sentence selection problem for question answering. Unlike previous work, which primarily leverages syntactic analysis through dependency tree matching, we focus on improving the performance using models of lexical semantic resources. Experiments show that our systems can be consistently and significantly improved with rich lexical semantic information, regardless of the choice of learning algorithms. When evaluated on a benchmark dataset, the MAP and MRR scores are increased by 8 to 10 points, compared to one of our baseline systems using only surface-form matching. Moreover, our best system also outperforms pervious work that makes use of the dependency tree structure by a wide margin.
5 0.16911332 292 acl-2013-Question Classification Transfer
Author: Anne-Laure Ligozat
Abstract: Question answering systems have been developed for many languages, but most resources were created for English, which can be a problem when developing a system in another language such as French. In particular, for question classification, no labeled question corpus is available for French, so this paper studies the possibility to use existing English corpora and transfer a classification by translating the question and their labels. By translating the training corpus, we obtain results close to a monolingual setting.
6 0.1653229 218 acl-2013-Latent Semantic Tensor Indexing for Community-based Question Answering
7 0.16293277 290 acl-2013-Question Analysis for Polish Question Answering
8 0.15524088 352 acl-2013-Towards Accurate Distant Supervision for Relational Facts Extraction
9 0.13971888 376 acl-2013-Using Lexical Expansion to Learn Inference Rules from Sparse Data
10 0.13058431 71 acl-2013-Bootstrapping Entity Translation on Weakly Comparable Corpora
11 0.13033755 346 acl-2013-The Impact of Topic Bias on Quality Flaw Prediction in Wikipedia
12 0.12804186 159 acl-2013-Filling Knowledge Base Gaps for Distant Supervision of Relation Extraction
13 0.12611924 160 acl-2013-Fine-grained Semantic Typing of Emerging Entities
14 0.124332 233 acl-2013-Linking Tweets to News: A Framework to Enrich Short Text Data in Social Media
15 0.11740544 27 acl-2013-A Two Level Model for Context Sensitive Inference Rules
16 0.11636373 168 acl-2013-Generating Recommendation Dialogs by Extracting Information from User Reviews
17 0.11581391 74 acl-2013-Building Comparable Corpora Based on Bilingual LDA Model
18 0.11571974 129 acl-2013-Domain-Independent Abstract Generation for Focused Meeting Summarization
19 0.11545119 121 acl-2013-Discovering User Interactions in Ideological Discussions
20 0.11425245 179 acl-2013-HYENA-live: Fine-Grained Online Entity Type Classification from Natural-language Text
topicId topicWeight
[(0, 0.265), (1, 0.15), (2, 0.017), (3, -0.146), (4, 0.108), (5, 0.12), (6, 0.008), (7, -0.249), (8, 0.074), (9, -0.02), (10, 0.086), (11, -0.031), (12, 0.024), (13, 0.023), (14, 0.039), (15, -0.001), (16, 0.014), (17, -0.064), (18, -0.041), (19, 0.014), (20, -0.032), (21, -0.003), (22, 0.028), (23, 0.096), (24, 0.045), (25, 0.026), (26, 0.081), (27, -0.048), (28, -0.034), (29, 0.07), (30, 0.042), (31, -0.072), (32, 0.073), (33, -0.017), (34, -0.089), (35, 0.021), (36, 0.024), (37, -0.069), (38, -0.032), (39, -0.02), (40, -0.03), (41, 0.032), (42, 0.05), (43, -0.037), (44, -0.076), (45, 0.012), (46, 0.053), (47, 0.008), (48, -0.002), (49, 0.064)]
simIndex simValue paperId paperTitle
same-paper 1 0.97632796 169 acl-2013-Generating Synthetic Comparable Questions for News Articles
Author: Oleg Rokhlenko ; Idan Szpektor
Abstract: We introduce the novel task of automatically generating questions that are relevant to a text but do not appear in it. One motivating example of its application is for increasing user engagement around news articles by suggesting relevant comparable questions, such as “is Beyonce a better singer than Madonna?”, for the user to answer. We present the first algorithm for the task, which consists of: (a) offline construction of a comparable question template database; (b) ranking of relevant templates to a given article; and (c) instantiation of templates only with entities in the article whose comparison under the template’s relation makes sense. We tested the suggestions generated by our algorithm via a Mechanical Turk experiment, which showed a significant improvement over the strongest baseline of more than 45% in all metrics.
2 0.82471937 290 acl-2013-Question Analysis for Polish Question Answering
Author: Piotr Przybyla
Abstract: This study is devoted to the problem of question analysis for a Polish question answering system. The goal of the question analysis is to determine its general structure, type of an expected answer and create a search query for finding relevant documents in a textual knowledge base. The paper contains an overview of available solutions of these problems, description of their implementation and presents an evaluation based on a set of 1137 questions from a Polish quiz TV show. The results help to understand how an environment of a Slavonic language affects the performance of methods created for English.
3 0.80788577 272 acl-2013-Paraphrase-Driven Learning for Open Question Answering
Author: Anthony Fader ; Luke Zettlemoyer ; Oren Etzioni
Abstract: We study question answering as a machine learning problem, and induce a function that maps open-domain questions to queries over a database of web extractions. Given a large, community-authored, question-paraphrase corpus, we demonstrate that it is possible to learn a semantic lexicon and linear ranking function without manually annotating questions. Our approach automatically generalizes a seed lexicon and includes a scalable, parallelized perceptron parameter estimation scheme. Experiments show that our approach more than quadruples the recall of the seed lexicon, with only an 8% loss in precision.
4 0.75224316 266 acl-2013-PAL: A Chatterbot System for Answering Domain-specific Questions
Author: Yuanchao Liu ; Ming Liu ; Xiaolong Wang ; Limin Wang ; Jingjing Li
Abstract: In this paper, we propose PAL, a prototype chatterbot for answering non-obstructive psychological domain-specific questions. This system focuses on providing primary suggestions or helping people relieve pressure by extracting knowledge from online forums, based on which the chatterbot system is constructed. The strategies used by PAL, including semantic-extension-based question matching, solution management with personal information consideration, and XML-based knowledge pattern construction, are described and discussed. We also conduct a primary test for the feasibility of our system.
5 0.73974824 292 acl-2013-Question Classification Transfer
Author: Anne-Laure Ligozat
Abstract: Question answering systems have been developed for many languages, but most resources were created for English, which can be a problem when developing a system in another language such as French. In particular, for question classification, no labeled question corpus is available for French, so this paper studies the possibility to use existing English corpora and transfer a classification by translating the question and their labels. By translating the training corpus, we obtain results close to a monolingual setting.
6 0.7291286 218 acl-2013-Latent Semantic Tensor Indexing for Community-based Question Answering
7 0.6952129 291 acl-2013-Question Answering Using Enhanced Lexical Semantic Models
8 0.69434702 159 acl-2013-Filling Knowledge Base Gaps for Distant Supervision of Relation Extraction
10 0.68026417 60 acl-2013-Automatic Coupling of Answer Extraction and Information Retrieval
11 0.6554842 160 acl-2013-Fine-grained Semantic Typing of Emerging Entities
12 0.65193707 352 acl-2013-Towards Accurate Distant Supervision for Relational Facts Extraction
13 0.64338672 215 acl-2013-Large-scale Semantic Parsing via Schema Matching and Lexicon Extension
14 0.62838107 179 acl-2013-HYENA-live: Fine-Grained Online Entity Type Classification from Natural-language Text
15 0.59611088 346 acl-2013-The Impact of Topic Bias on Quality Flaw Prediction in Wikipedia
16 0.59103376 21 acl-2013-A Statistical NLG Framework for Aggregated Planning and Realization
17 0.55759686 356 acl-2013-Transfer Learning Based Cross-lingual Knowledge Extraction for Wikipedia
18 0.55497801 219 acl-2013-Learning Entity Representation for Entity Disambiguation
19 0.55158877 141 acl-2013-Evaluating a City Exploration Dialogue System with Integrated Question-Answering and Pedestrian Navigation
20 0.55106527 241 acl-2013-Minimum Bayes Risk based Answer Re-ranking for Question Answering
topicId topicWeight
[(0, 0.047), (6, 0.031), (11, 0.106), (15, 0.02), (24, 0.078), (26, 0.06), (28, 0.02), (35, 0.106), (42, 0.042), (48, 0.034), (70, 0.11), (88, 0.023), (90, 0.021), (91, 0.159), (95, 0.063)]
simIndex simValue paperId paperTitle
same-paper 1 0.88746238 169 acl-2013-Generating Synthetic Comparable Questions for News Articles
Author: Oleg Rokhlenko ; Idan Szpektor
Abstract: We introduce the novel task of automatically generating questions that are relevant to a text but do not appear in it. One motivating example of its application is for increasing user engagement around news articles by suggesting relevant comparable questions, such as “is Beyonce a better singer than Madonna?”, for the user to answer. We present the first algorithm for the task, which consists of: (a) offline construction of a comparable question template database; (b) ranking of relevant templates to a given article; and (c) instantiation of templates only with entities in the article whose comparison under the template’s relation makes sense. We tested the suggestions generated by our algorithm via a Mechanical Turk experiment, which showed a significant improvement over the strongest baseline of more than 45% in all metrics.
2 0.83971435 52 acl-2013-Annotating named entities in clinical text by combining pre-annotation and active learning
Author: Maria Skeppstedt
Abstract: For expanding a corpus of clinical text, annotated for named entities, a method that combines pre-tagging with a version of active learning is proposed. In order to facilitate annotation and to avoid bias, two alternative automatic pre-taggings are presented to the annotator, without revealing which of them is given a higher confidence by the pre-tagging system. The task of the annotator is to select the correct version among these two alternatives. To minimise the instances in which none of the presented pre-taggings is correct, the texts presented to the annotator are actively selected from a pool of unlabelled text, with the selection criterion that one of the presented pre-taggings should have a high probability of being correct, while still being useful for improving the result of an automatic classifier.
3 0.79237473 131 acl-2013-Dual Training and Dual Prediction for Polarity Classification
Author: Rui Xia ; Tao Wang ; Xuelei Hu ; Shoushan Li ; Chengqing Zong
Abstract: Bag-of-words (BOW) is now the most popular way to model text in machine learning based sentiment classification. However, the performance of such approach sometimes remains rather limited due to some fundamental deficiencies of the BOW model. In this paper, we focus on the polarity shift problem, and propose a novel approach, called dual training and dual prediction (DTDP), to address it. The basic idea of DTDP is to first generate artificial samples that are polarity-opposite to the original samples by polarity reversion, and then leverage both the original and opposite samples for (dual) training and (dual) prediction. Experimental results on four datasets demonstrate the effectiveness of the proposed approach for polarity classification. 1
4 0.77913398 272 acl-2013-Paraphrase-Driven Learning for Open Question Answering
Author: Anthony Fader ; Luke Zettlemoyer ; Oren Etzioni
Abstract: We study question answering as a machine learning problem, and induce a function that maps open-domain questions to queries over a database of web extractions. Given a large, community-authored, question-paraphrase corpus, we demonstrate that it is possible to learn a semantic lexicon and linear ranking function without manually annotating questions. Our approach automatically generalizes a seed lexicon and includes a scalable, parallelized perceptron parameter estimation scheme. Experiments show that our approach more than quadruples the recall of the seed lexicon, with only an 8% loss in precision.
5 0.77017897 85 acl-2013-Combining Intra- and Multi-sentential Rhetorical Parsing for Document-level Discourse Analysis
Author: Shafiq Joty ; Giuseppe Carenini ; Raymond Ng ; Yashar Mehdad
Abstract: We propose a novel approach for developing a two-stage document-level discourse parser. Our parser builds a discourse tree by applying an optimal parsing algorithm to probabilities inferred from two Conditional Random Fields: one for intrasentential parsing and the other for multisentential parsing. We present two approaches to combine these two stages of discourse parsing effectively. A set of empirical evaluations over two different datasets demonstrates that our discourse parser significantly outperforms the stateof-the-art, often by a wide margin.
6 0.76851535 58 acl-2013-Automated Collocation Suggestion for Japanese Second Language Learners
7 0.76617908 318 acl-2013-Sentiment Relevance
8 0.76556957 274 acl-2013-Parsing Graphs with Hyperedge Replacement Grammars
9 0.76551884 155 acl-2013-Fast and Accurate Shift-Reduce Constituent Parsing
10 0.75922483 224 acl-2013-Learning to Extract International Relations from Political Context
11 0.75916874 249 acl-2013-Models of Semantic Representation with Visual Attributes
12 0.75792181 212 acl-2013-Language-Independent Discriminative Parsing of Temporal Expressions
14 0.75729281 153 acl-2013-Extracting Events with Informal Temporal References in Personal Histories in Online Communities
15 0.75640148 168 acl-2013-Generating Recommendation Dialogs by Extracting Information from User Reviews
16 0.75509143 215 acl-2013-Large-scale Semantic Parsing via Schema Matching and Lexicon Extension
17 0.75319487 159 acl-2013-Filling Knowledge Base Gaps for Distant Supervision of Relation Extraction
18 0.75257933 80 acl-2013-Chinese Parsing Exploiting Characters
19 0.75241983 132 acl-2013-Easy-First POS Tagging and Dependency Parsing with Beam Search
20 0.75226939 339 acl-2013-Temporal Signals Help Label Temporal Relations