acl acl2013 acl2013-273 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Chenguang Wang ; Nan Duan ; Ming Zhou ; Ming Zhang
Abstract: Mismatch between queries and documents is a key issue for the web search task. In order to narrow down such mismatch, in this paper, we present an in-depth investigation on adapting a paraphrasing technique to web search from three aspects: a search-oriented paraphrasing model; an NDCG-based parameter optimization algorithm; an enhanced ranking model leveraging augmented features computed on paraphrases of original queries. Ex- periments performed on the large scale query-document data set show that, the search performance can be significantly improved, with +3.28% and +1.14% NDCG gains on dev and test sets respectively.
Reference: text
sentIndex sentText sentNum sentScore
1 com Abstract Mismatch between queries and documents is a key issue for the web search task. [sent-4, score-0.324]
2 Ex- periments performed on the large scale query-document data set show that, the search performance can be significantly improved, with +3. [sent-6, score-0.058]
3 Researchers have made great efforts to improve paraphrasing from different perspectives, such as paraphrase extraction (Zhao et al. [sent-10, score-0.862]
4 But as far as we know, none of previous work has explored the impact of using a well designed paraphrasing engine for web search ranking task specifically. [sent-14, score-0.812]
5 In web search, mismatches between queries and their relevant documents are usually caused by expressing the same meaning in different natural language ways. [sent-15, score-0.304]
6 The capability of paraphrasing is just right to alleviate such issues. [sent-19, score-0.465]
7 cn an in-depth study on adapting paraphrasing to web search. [sent-25, score-0.577]
8 First, we propose a search-oriented paraphrasing model, which includes specifically designed features for web queries that can enable a paraphrasing engine to learn preferences on different paraphrasing strategies. [sent-26, score-1.697]
9 Second, we optimize the parameters of the paraphrasing model according to the Normalized Discounted Cumulative Gain (NDCG) score, by leveraging the minimum error rate training (MERT) algorithm (Och, 2003). [sent-27, score-0.641]
10 Third, we propose an enhanced ranking model by using augmented features computed on paraphrases of original queries. [sent-28, score-0.547]
11 Many query reformulation approaches have been proposed to tackle the query-document mismatch issue, which can be generally summarized as query expansion and query substitution. [sent-29, score-0.972]
12 Query expansion (Baeza-Yates, 1992; Jing and Croft, 1994; Lavrenko and Croft, 2001 ; Cui et al. [sent-30, score-0.031]
13 , 2009) adds new terms extracted from different sources to the original query directly; while query substitution (Brill and Moore, 2000; Jones et al. [sent-34, score-0.598]
14 , 2008; Wang and Zhai, 2008; Dang and Croft, 2010) uses probabilistic models, such as graphical models, to predict the sequence of rewritten query words to form a new query. [sent-36, score-0.274]
15 Comparing to these works, our paraphrasing engine alters queries in a similar way to statistical machine translation, with systematic tuning and decoding components. [sent-37, score-0.667]
16 (2009) proposes an uni- fied paraphrasing framework that can be adapted to different applications using different usability models. [sent-39, score-0.465]
17 Our work can be seen as an extension along this line of research, by carrying out in-depth study on adapting paraphrasing to web search. [sent-40, score-0.577]
18 Experiments performed on the large scale data set show that, by leveraging additional matching features computed on query paraphrases, significant NDCG gains can be achieved on both dev 41 Proce dinSgosfi oa,f tB huel 5g1arsita, An Anu gauls Mt 4e-e9ti n2g01 o3f. [sent-41, score-0.485]
19 Given a monolingual corpus, Lin and Pantel (2001)’s method is used to extract paraphrases based on distributional hypothesis. [sent-50, score-0.126]
20 We use Miller (1995)’s approach to extract paraphrases from the synonym dictionary of WordNet. [sent-52, score-0.153]
21 Word alignments within each paraphrase pair are generated using GIZA++ (Och and Ney, 2000). [sent-53, score-0.424]
22 2 Search-Oriented Paraphrasing Model Similar to statistical machine translation (SMT), given an input query Q, our paraphrasing engine generates paraphrase candidates1 based on a linear model. [sent-55, score-1.205]
23 Qˆ = aQr0g∈Hm(Qax)P(Q0|Q) XM = aQr0g∈Hm(Qax)mX=1λmhm(Q,Q0) H(Q) is the hypothesis space containing all paraphrase sca thndei hdyatpeost oefs Q, hm ciso tthaein imngth a lfle aptaurraefunction with weight λm, Q0 denotes one candidate. [sent-56, score-0.446]
24 2Similar features have been demonstrated effective in (Jones et al. [sent-58, score-0.027]
25 But we use SMT-like model to generate query reformulations. [sent-60, score-0.301]
26 • • Word Addition feature hWADD (Q, Q0), wWhoircdh is defined as the number of words in the paraphrase candidate Q0 without being aligned to any word in the original query Q. [sent-61, score-0.775]
27 Word Deletion feature hWDEL(Q, Q0), wWhoircdh is defined as the number of words in the original query Q without being aligned to any word in the paraphrase candidate Q0. [sent-62, score-0.775]
28 • • • • • Word Overlap feature hWO (Q, Q0), which is dWeofirnded O as trhlaep pn fuematuberre o hf word pairs that align identical words between Q and Q0. [sent-63, score-0.027]
29 Word Alteration feature hWA(Q, Q0), which iWs odredfin Aeldte as othne neuatmubreer h of word pairs that align different words between Q and Q0. [sent-64, score-0.027]
30 Word Reorder feature hWR(Q, Q0), which is Wmoodredle Rde by a freealatutirvee distortion probability distribution, similar to the distortion model in (Koehn et al. [sent-65, score-0.108]
31 Length Difference feature hLD (Q, Q0), Lwehnicghth is defined as |Q0| − |Q| . [sent-67, score-0.027]
32 Edit Distance feature hED (Q, Q0), which is dEedfiitne Ddi as cthee f ecahtaurreact her-level edit distance between Q and Q0. [sent-68, score-0.027]
33 Besides, a set of traditional SMT features (Koehn et al. [sent-69, score-0.027]
34 , 2003) are also used in our paraphrasing model, including translation probability, lexical weight, word count, paraphrase rule count3, and language model feature. [sent-70, score-0.889]
35 3 NDCG-based Parameter Optimization We utilize minimum error rate training (MERT) (Och, 2003) to optimize feature weights of the paraphrasing model according to NDCG. [sent-72, score-0.642]
36 R is a ranking m Dod aesl4 t etha etn can rdaoncku mdoecnutm seetn. [sent-74, score-0.147]
37 Qi is th}e ith query and DLiabel ⊂ D is a subset of documents, in which Dthe relev⊂anc De i bse atw seuebns Qi fa dndo euamche dtso,c iunm wenhti cihs labeled by human annotators. [sent-77, score-0.274]
38 MERT is used to optimize feature weights of our linear-formed paraphrasing model. [sent-78, score-0.555]
39 For 3Paraphrase rule count is the number of rules that are used to generate paraphrase candidates. [sent-79, score-0.397]
40 , 2007) uses matching featuTrehse c roamnkpiuntged m boadseedl Ron (oLriiugin etal a lq. [sent-81, score-0.061]
41 42 each query Qi in {Qi}iS=1, we first generate Nbest paraphrase candidates and compute NpaDraCpGhr score fnodri eaatechs paraphrase based on documents ranked by the ranker R and labeled {Qji}jN=1, ddooccuummeennttss D rainLakbeedl. [sent-83, score-1.157]
42 When computing NDCG scores, these five levels are commonly mapped to the numerical scores 3 1, 15, 7, 3, 0 respectively. [sent-87, score-0.027]
43 4 Enhanced Ranking Model In web search, the key objective of the ranking model is to rank the retrieved documents based on their relevance to a given query. [sent-89, score-0.411]
44 Given a query Q and its retrieved document set D = {DQ}, for each DQ ∈ D, we use the following ranking mr eoadcehl t Do compute wtheei ru relevance, which is formulated as a weighted combination of matching features: XK R(Q,DQ) =XλkFk(Q,DQ) kX= X1 F = {F1, . [sent-90, score-0.529]
45 , FK} denotes a set of matching featFur =es t{hFat measure tdheen matching degrees hbientgw feeeanQ and DQ, Fk (Q, DQ) ∈ F is the kth matching feature, λk is its corresponding sfe tahteur ke weight. [sent-93, score-0.183]
46 Formally, given a query Q and its N-best paraphrase candidates {Q01 , . [sent-97, score-0.7]
47 , Q0N}, we enrich the original fcaeantduirdea vector F to {F, F1, . [sent-100, score-0.05]
48 , FN} fthore Q raingdin DQ, awtuhreere v aecllt ofrea Ftur toes { Fin, Fn have th}e same meanings as they are in F, however, their feature values are computed based on Q0n and DQ, instead of Q and DQ. [sent-103, score-0.058]
49 In this way, the paraphrase candidates act as hidden variables and expanded matching features between queries and documents, making our ranking model more tunable and flexible for web search. [sent-104, score-0.894]
50 7M queries from the log of a commercial search engine. [sent-111, score-0.22]
51 Word alignments of each paraphrase pair are trained by GIZA++. [sent-114, score-0.424]
52 The language model is trained based on a portion of queries, in which the frequency of each query is higher than a predefined threshold, 5. [sent-115, score-0.301]
53 The minimum length of paraphrase rule is 1, while the maximum length of paraphrase rule is 5. [sent-117, score-0.824]
54 We randomly select 2, 838 queries from the log of a commercial search engine, each of which attached with a set of documents that are annotated with relevance ratings described in Section 2. [sent-118, score-0.337]
55 We use the first 1, 419 queries together with their annotated documents as the development set to tune paraphrasing parameters (as we discussed in Section 2. [sent-120, score-0.658]
56 The ranking model is trained based on the development set. [sent-122, score-0.174]
57 NDCG is used as the evaluation metric of the web search task. [sent-123, score-0.131]
58 2 Baseline Systems The baselines of the paraphrasing and the ranking model are described as follows: The paraphrasing baseline is denoted as BLPara, which only uses traditional SMT features described at the end of Section 2. [sent-125, score-1.131]
59 Weights are optimized by MERT using BLEU (Papineni et al. [sent-127, score-0.028]
60 , 2007) is denoted as BL-Rank, which only uses matching features computed based on original queries and different meta-streams of web pages, including URL, page title, page body, meta-keywords, metadescription and anchor texts. [sent-132, score-0.408]
61 The feature functions we use include unigram/bigram/trigram BM25 and original/normalized Perfect-Match. [sent-133, score-0.027]
62 The ranking model is learned based on SV Mrank toolkit (Joachims, 2006) with default parameter setting. [sent-134, score-0.174]
63 To do so, we add these features into the paraphrasing model baseline, and denote it as BL-Para+SF, whose weights are optimized in the same way with BL-Para. [sent-137, score-0.573]
64 The ranking model baseline BL-Rank is used to rank the documents. [sent-138, score-0.174]
65 We then compare the NDCG@ 1 scores of the best documents retrieved using either original query, or query paraphrases generated by BL-Para and BLPara+SF respectively, and list comparison results in Table 1, where Cand@ 1denotes the best paraphrase candidate generated by each paraphrasing model. [sent-139, score-1.446]
66 From Table 1, we can see, even using the best query paraphrase, its corresponding NDCG score is still lower than the NDCG score of the original query. [sent-144, score-0.324]
67 This performance dropping makes sense, as changing user queries brings the risks of query drift. [sent-145, score-0.407]
68 When adding search-oriented features into the baseline, the performance changes little, as these two models are optimized based on BLEU score only, without considering characteristics of mismatches in search. [sent-146, score-0.093]
69 4 Impacts of Optimization Algorithm We then evaluate the impact of our NDCG-based optimization method. [sent-148, score-0.053]
70 We add the optimization algorithm described in Section 2. [sent-149, score-0.053]
71 Similar to the experiment in Table 1, we compare the NDCG@ 1 scores of the best documents retrieved using query paraphrases generated by BLPara+SF and BL-Para+SF+Opt respectively, with results shown in Table 2. [sent-152, score-0.507]
72 Table 2 indicates that, by leveraging NDCG as the error criterion for MERT, search-oriented features benefit more (+0. [sent-158, score-0.109]
73 53% NDCG) in selecting the best query paraphrase from the whole paraphrasing search space. [sent-159, score-1.194]
74 The quality of the top-1 paraphrase generated by BL-Para+SF+Opt is very close to the original query. [sent-163, score-0.447]
75 5 Impacts of Enhanced Ranking Model We last evaluate the effectiveness of the enhanced ranking model. [sent-165, score-0.24]
76 From Table 3, we can see that NDCG@k (k = 1, 5) scores of BL-Rank+Para outperforms BLRank on both dev and test sets. [sent-181, score-0.04]
77 Such end-to-end NDCG improvements come from the extra knowledge provided by the hidden paraphrases of original queries. [sent-184, score-0.21]
78 This narrows down the query-document mismatch issue to a certain extent. [sent-185, score-0.058]
79 4 Conclusion and Future Work In this paper, we present an in-depth study on using paraphrasing for web search, which pays close attention to various aspects of the application including choice of model and optimization technique. [sent-186, score-0.618]
80 In the future, we will compare and combine paraphrasing with other query reformulation techniques, e. [sent-187, score-0.8]
81 An improved error model for noisy channel spelling correction. [sent-205, score-0.057]
82 In Workshop on Tabulation in Parsing and Deduction, pages 133–137. [sent-210, score-0.034]
83 Mining term association patterns from search logs for effective query reformulation. [sent-294, score-0.332]
84 In Proceedings of the 1 ACM conference on Information and knowl7th edge management, Proceedings of CIKM, pages 479–488. [sent-295, score-0.034]
85 Improving pseudo-relevance feedback in web information retrieval using web page segmentation. [sent-304, score-0.196]
wordName wordTfidf (topN-words)
[('paraphrasing', 0.465), ('paraphrase', 0.397), ('ndcg', 0.382), ('query', 0.274), ('ranking', 0.147), ('dq', 0.133), ('queries', 0.133), ('sf', 0.126), ('paraphrases', 0.126), ('enhanced', 0.093), ('blpara', 0.087), ('impacts', 0.084), ('mert', 0.075), ('sigir', 0.074), ('web', 0.073), ('err', 0.071), ('engine', 0.069), ('qi', 0.068), ('matching', 0.061), ('reformulation', 0.061), ('documents', 0.06), ('mismatch', 0.058), ('search', 0.058), ('chappelier', 0.058), ('craswell', 0.058), ('dilabel', 0.058), ('dliabel', 0.058), ('dqi', 0.058), ('dqj', 0.058), ('npkar', 0.058), ('shiqi', 0.058), ('wwhoircdh', 0.058), ('relevance', 0.057), ('opt', 0.053), ('optimization', 0.053), ('zhao', 0.053), ('jones', 0.052), ('leveraging', 0.052), ('para', 0.051), ('elsas', 0.051), ('smucker', 0.051), ('qax', 0.051), ('original', 0.05), ('feedback', 0.05), ('hang', 0.049), ('hm', 0.049), ('bruce', 0.049), ('croft', 0.049), ('guo', 0.047), ('retrieved', 0.047), ('augmented', 0.046), ('cyk', 0.044), ('och', 0.044), ('yu', 0.043), ('xu', 0.042), ('bannard', 0.042), ('fk', 0.042), ('brill', 0.04), ('dev', 0.04), ('adapting', 0.039), ('lavrenko', 0.039), ('eecs', 0.038), ('mismatches', 0.038), ('optimize', 0.037), ('ming', 0.037), ('ting', 0.035), ('kk', 0.035), ('peking', 0.035), ('jaime', 0.035), ('microsoft', 0.035), ('smt', 0.035), ('wen', 0.034), ('extra', 0.034), ('www', 0.034), ('pages', 0.034), ('anchor', 0.033), ('cui', 0.032), ('computed', 0.031), ('expansion', 0.031), ('fn', 0.03), ('error', 0.03), ('minimum', 0.03), ('dang', 0.03), ('blog', 0.03), ('candidates', 0.029), ('commercial', 0.029), ('asia', 0.029), ('optimized', 0.028), ('feature', 0.027), ('franz', 0.027), ('alignments', 0.027), ('numerical', 0.027), ('model', 0.027), ('candidate', 0.027), ('synonym', 0.027), ('distortion', 0.027), ('jing', 0.027), ('josef', 0.027), ('features', 0.027), ('weights', 0.026)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000006 273 acl-2013-Paraphrasing Adaptation for Web Search Ranking
Author: Chenguang Wang ; Nan Duan ; Ming Zhou ; Ming Zhang
Abstract: Mismatch between queries and documents is a key issue for the web search task. In order to narrow down such mismatch, in this paper, we present an in-depth investigation on adapting a paraphrasing technique to web search from three aspects: a search-oriented paraphrasing model; an NDCG-based parameter optimization algorithm; an enhanced ranking model leveraging augmented features computed on paraphrases of original queries. Ex- periments performed on the large scale query-document data set show that, the search performance can be significantly improved, with +3.28% and +1.14% NDCG gains on dev and test sets respectively.
2 0.24590181 271 acl-2013-ParaQuery: Making Sense of Paraphrase Collections
Author: Lili Kotlerman ; Nitin Madnani ; Aoife Cahill
Abstract: Pivoting on bilingual parallel corpora is a popular approach for paraphrase acquisition. Although such pivoted paraphrase collections have been successfully used to improve the performance of several different NLP applications, it is still difficult to get an intrinsic estimate of the quality and coverage of the paraphrases contained in these collections. We present ParaQuery, a tool that helps a user interactively explore and characterize a given pivoted paraphrase collection, analyze its utility for a particular domain, and compare it to other popular lexical similarity resources all within a single interface.
3 0.19896819 272 acl-2013-Paraphrase-Driven Learning for Open Question Answering
Author: Anthony Fader ; Luke Zettlemoyer ; Oren Etzioni
Abstract: We study question answering as a machine learning problem, and induce a function that maps open-domain questions to queries over a database of web extractions. Given a large, community-authored, question-paraphrase corpus, we demonstrate that it is possible to learn a semantic lexicon and linear ranking function without manually annotating questions. Our approach automatically generalizes a seed lexicon and includes a scalable, parallelized perceptron parameter estimation scheme. Experiments show that our approach more than quadruples the recall of the seed lexicon, with only an 8% loss in precision.
4 0.14703283 55 acl-2013-Are Semantically Coherent Topic Models Useful for Ad Hoc Information Retrieval?
Author: Romain Deveaud ; Eric SanJuan ; Patrice Bellot
Abstract: The current topic modeling approaches for Information Retrieval do not allow to explicitly model query-oriented latent topics. More, the semantic coherence of the topics has never been considered in this field. We propose a model-based feedback approach that learns Latent Dirichlet Allocation topic models on the top-ranked pseudo-relevant feedback, and we measure the semantic coherence of those topics. We perform a first experimental evaluation using two major TREC test collections. Results show that retrieval perfor- mances tend to be better when using topics with higher semantic coherence.
5 0.12101363 304 acl-2013-SEMILAR: The Semantic Similarity Toolkit
Author: Vasile Rus ; Mihai Lintean ; Rajendra Banjade ; Nobal Niraula ; Dan Stefanescu
Abstract: We present in this paper SEMILAR, the SEMantic simILARity toolkit. SEMILAR implements a number of algorithms for assessing the semantic similarity between two texts. It is available as a Java library and as a Java standalone ap-plication offering GUI-based access to the implemented semantic similarity methods. Furthermore, it offers facilities for manual se-mantic similarity annotation by experts through its component SEMILAT (a SEMantic simILarity Annotation Tool). 1
7 0.11307842 183 acl-2013-ICARUS - An Extensible Graphical Search Tool for Dependency Treebanks
8 0.094122581 338 acl-2013-Task Alternation in Parallel Sentence Retrieval for Twitter Translation
9 0.093748048 290 acl-2013-Question Analysis for Polish Question Answering
10 0.086644441 231 acl-2013-Linggle: a Web-scale Linguistic Search Engine for Words in Context
11 0.08372511 174 acl-2013-Graph Propagation for Paraphrasing Out-of-Vocabulary Words in Statistical Machine Translation
12 0.08139468 291 acl-2013-Question Answering Using Enhanced Lexical Semantic Models
13 0.079110257 159 acl-2013-Filling Knowledge Base Gaps for Distant Supervision of Relation Extraction
14 0.078968391 300 acl-2013-Reducing Annotation Effort for Quality Estimation via Active Learning
15 0.077028535 60 acl-2013-Automatic Coupling of Answer Extraction and Information Retrieval
16 0.073885612 311 acl-2013-Semantic Neighborhoods as Hypergraphs
17 0.073725194 9 acl-2013-A Lightweight and High Performance Monolingual Word Aligner
18 0.072366633 329 acl-2013-Statistical Machine Translation Improves Question Retrieval in Community Question Answering via Matrix Factorization
19 0.069118291 230 acl-2013-Lightly Supervised Learning of Procedural Dialog Systems
20 0.06765651 18 acl-2013-A Sentence Compression Based Framework to Query-Focused Multi-Document Summarization
topicId topicWeight
[(0, 0.179), (1, -0.006), (2, 0.07), (3, -0.055), (4, 0.066), (5, 0.017), (6, 0.024), (7, -0.171), (8, 0.012), (9, 0.014), (10, 0.032), (11, 0.053), (12, 0.015), (13, 0.028), (14, 0.022), (15, 0.022), (16, 0.018), (17, 0.02), (18, 0.004), (19, 0.035), (20, -0.013), (21, 0.034), (22, -0.081), (23, 0.05), (24, 0.041), (25, -0.013), (26, -0.03), (27, 0.128), (28, -0.068), (29, 0.051), (30, -0.029), (31, 0.063), (32, -0.235), (33, -0.109), (34, -0.007), (35, -0.197), (36, 0.124), (37, -0.014), (38, 0.08), (39, 0.057), (40, -0.02), (41, -0.036), (42, 0.036), (43, 0.015), (44, 0.06), (45, -0.048), (46, 0.061), (47, -0.029), (48, -0.02), (49, 0.04)]
simIndex simValue paperId paperTitle
same-paper 1 0.93894517 273 acl-2013-Paraphrasing Adaptation for Web Search Ranking
Author: Chenguang Wang ; Nan Duan ; Ming Zhou ; Ming Zhang
Abstract: Mismatch between queries and documents is a key issue for the web search task. In order to narrow down such mismatch, in this paper, we present an in-depth investigation on adapting a paraphrasing technique to web search from three aspects: a search-oriented paraphrasing model; an NDCG-based parameter optimization algorithm; an enhanced ranking model leveraging augmented features computed on paraphrases of original queries. Ex- periments performed on the large scale query-document data set show that, the search performance can be significantly improved, with +3.28% and +1.14% NDCG gains on dev and test sets respectively.
2 0.78567713 271 acl-2013-ParaQuery: Making Sense of Paraphrase Collections
Author: Lili Kotlerman ; Nitin Madnani ; Aoife Cahill
Abstract: Pivoting on bilingual parallel corpora is a popular approach for paraphrase acquisition. Although such pivoted paraphrase collections have been successfully used to improve the performance of several different NLP applications, it is still difficult to get an intrinsic estimate of the quality and coverage of the paraphrases contained in these collections. We present ParaQuery, a tool that helps a user interactively explore and characterize a given pivoted paraphrase collection, analyze its utility for a particular domain, and compare it to other popular lexical similarity resources all within a single interface.
3 0.6586923 272 acl-2013-Paraphrase-Driven Learning for Open Question Answering
Author: Anthony Fader ; Luke Zettlemoyer ; Oren Etzioni
Abstract: We study question answering as a machine learning problem, and induce a function that maps open-domain questions to queries over a database of web extractions. Given a large, community-authored, question-paraphrase corpus, we demonstrate that it is possible to learn a semantic lexicon and linear ranking function without manually annotating questions. Our approach automatically generalizes a seed lexicon and includes a scalable, parallelized perceptron parameter estimation scheme. Experiments show that our approach more than quadruples the recall of the seed lexicon, with only an 8% loss in precision.
4 0.63746774 158 acl-2013-Feature-Based Selection of Dependency Paths in Ad Hoc Information Retrieval
Author: K. Tamsin Maxwell ; Jon Oberlander ; W. Bruce Croft
Abstract: Techniques that compare short text segments using dependency paths (or simply, paths) appear in a wide range of automated language processing applications including question answering (QA). However, few models in ad hoc information retrieval (IR) use paths for document ranking due to the prohibitive cost of parsing a retrieval collection. In this paper, we introduce a flexible notion of paths that describe chains of words on a dependency path. These chains, or catenae, are readily applied in standard IR models. Informative catenae are selected using supervised machine learning with linguistically informed features and compared to both non-linguistic terms and catenae selected heuristically with filters derived from work on paths. Automatically selected catenae of 1-2 words deliver significant performance gains on three TREC collections.
Author: Rohan Ramanath ; Monojit Choudhury ; Kalika Bali ; Rishiraj Saha Roy
Abstract: Query segmentation, like text chunking, is the first step towards query understanding. In this study, we explore the effectiveness of crowdsourcing for this task. Through carefully designed control experiments and Inter Annotator Agreement metrics for analysis of experimental data, we show that crowdsourcing may not be a suitable approach for query segmentation because the crowd seems to have a very strong bias towards dividing the query into roughly equal (often only two) parts. Similarly, in the case of hierarchical or nested segmentation, turkers have a strong preference towards balanced binary trees.
6 0.61302024 183 acl-2013-ICARUS - An Extensible Graphical Search Tool for Dependency Treebanks
7 0.56693417 231 acl-2013-Linggle: a Web-scale Linguistic Search Engine for Words in Context
8 0.55227435 290 acl-2013-Question Analysis for Polish Question Answering
9 0.48883778 55 acl-2013-Are Semantically Coherent Topic Models Useful for Ad Hoc Information Retrieval?
10 0.47582972 338 acl-2013-Task Alternation in Parallel Sentence Retrieval for Twitter Translation
11 0.46041206 159 acl-2013-Filling Knowledge Base Gaps for Distant Supervision of Relation Extraction
12 0.42947468 215 acl-2013-Large-scale Semantic Parsing via Schema Matching and Lexicon Extension
13 0.42410639 300 acl-2013-Reducing Annotation Effort for Quality Estimation via Active Learning
14 0.42230499 291 acl-2013-Question Answering Using Enhanced Lexical Semantic Models
15 0.41785678 140 acl-2013-Evaluating Text Segmentation using Boundary Edit Distance
16 0.41271517 285 acl-2013-Propminer: A Workflow for Interactive Information Extraction and Exploration using Dependency Trees
17 0.41138124 120 acl-2013-Dirt Cheap Web-Scale Parallel Text from the Common Crawl
18 0.40675032 60 acl-2013-Automatic Coupling of Answer Extraction and Information Retrieval
19 0.39834285 281 acl-2013-Post-Retrieval Clustering Using Third-Order Similarity Measures
20 0.39219889 225 acl-2013-Learning to Order Natural Language Texts
topicId topicWeight
[(0, 0.042), (4, 0.279), (6, 0.03), (11, 0.053), (15, 0.01), (24, 0.092), (26, 0.053), (28, 0.014), (35, 0.106), (42, 0.05), (48, 0.037), (70, 0.032), (88, 0.024), (90, 0.039), (95, 0.068)]
simIndex simValue paperId paperTitle
same-paper 1 0.78449887 273 acl-2013-Paraphrasing Adaptation for Web Search Ranking
Author: Chenguang Wang ; Nan Duan ; Ming Zhou ; Ming Zhang
Abstract: Mismatch between queries and documents is a key issue for the web search task. In order to narrow down such mismatch, in this paper, we present an in-depth investigation on adapting a paraphrasing technique to web search from three aspects: a search-oriented paraphrasing model; an NDCG-based parameter optimization algorithm; an enhanced ranking model leveraging augmented features computed on paraphrases of original queries. Ex- periments performed on the large scale query-document data set show that, the search performance can be significantly improved, with +3.28% and +1.14% NDCG gains on dev and test sets respectively.
2 0.76794648 315 acl-2013-Semi-Supervised Semantic Tagging of Conversational Understanding using Markov Topic Regression
Author: Asli Celikyilmaz ; Dilek Hakkani-Tur ; Gokhan Tur ; Ruhi Sarikaya
Abstract: Microsoft Research Microsoft Mountain View, CA, USA Redmond, WA, USA dilek @ ieee .org rus arika@mi cro s o ft . com gokhan .tur @ ieee .org performance (Tur and DeMori, 2011). This requires a tedious and time intensive data collection Finding concepts in natural language utterances is a challenging task, especially given the scarcity of labeled data for learning semantic ambiguity. Furthermore, data mismatch issues, which arise when the expected test (target) data does not exactly match the training data, aggravate this scarcity problem. To deal with these issues, we describe an efficient semisupervised learning (SSL) approach which has two components: (i) Markov Topic Regression is a new probabilistic model to cluster words into semantic tags (concepts). It can efficiently handle semantic ambiguity by extending standard topic models with two new features. First, it encodes word n-gram features from labeled source and unlabeled target data. Second, by going beyond a bag-of-words approach, it takes into account the inherent sequential nature of utterances to learn semantic classes based on context. (ii) Retrospective Learner is a new learning technique that adapts to the unlabeled target data. Our new SSL approach improves semantic tagging performance by 3% absolute over the baseline models, and also compares favorably on semi-supervised syntactic tagging.
3 0.731507 294 acl-2013-Re-embedding words
Author: Igor Labutov ; Hod Lipson
Abstract: We present a fast method for re-purposing existing semantic word vectors to improve performance in a supervised task. Recently, with an increase in computing resources, it became possible to learn rich word embeddings from massive amounts of unlabeled data. However, some methods take days or weeks to learn good embeddings, and some are notoriously difficult to train. We propose a method that takes as input an existing embedding, some labeled data, and produces an embedding in the same space, but with a better predictive performance in the supervised task. We show improvement on the task of sentiment classification with re- spect to several baselines, and observe that the approach is most useful when the training set is sufficiently small.
4 0.72746956 121 acl-2013-Discovering User Interactions in Ideological Discussions
Author: Arjun Mukherjee ; Bing Liu
Abstract: Online discussion forums are a popular platform for people to voice their opinions on any subject matter and to discuss or debate any issue of interest. In forums where users discuss social, political, or religious issues, there are often heated debates among users or participants. Existing research has studied mining of user stances or camps on certain issues, opposing perspectives, and contention points. In this paper, we focus on identifying the nature of interactions among user pairs. The central questions are: How does each pair of users interact with each other? Does the pair of users mostly agree or disagree? What is the lexicon that people often use to express agreement and disagreement? We present a topic model based approach to answer these questions. Since agreement and disagreement expressions are usually multiword phrases, we propose to employ a ranking method to identify highly relevant phrases prior to topic modeling. After modeling, we use the modeling results to classify the nature of interaction of each user pair. Our evaluation results using real-life discussion/debate posts demonstrate the effectiveness of the proposed techniques.
5 0.72690332 173 acl-2013-Graph-based Semi-Supervised Model for Joint Chinese Word Segmentation and Part-of-Speech Tagging
Author: Xiaodong Zeng ; Derek F. Wong ; Lidia S. Chao ; Isabel Trancoso
Abstract: This paper introduces a graph-based semisupervised joint model of Chinese word segmentation and part-of-speech tagging. The proposed approach is based on a graph-based label propagation technique. One constructs a nearest-neighbor similarity graph over all trigrams of labeled and unlabeled data for propagating syntactic information, i.e., label distributions. The derived label distributions are regarded as virtual evidences to regularize the learning of linear conditional random fields (CRFs) on unlabeled data. An inductive character-based joint model is obtained eventually. Empirical results on Chinese tree bank (CTB-7) and Microsoft Research corpora (MSR) reveal that the proposed model can yield better results than the supervised baselines and other competitive semi-supervised CRFs in this task.
6 0.66774917 287 acl-2013-Public Dialogue: Analysis of Tolerance in Online Discussions
7 0.63524079 82 acl-2013-Co-regularizing character-based and word-based models for semi-supervised Chinese word segmentation
8 0.62196636 309 acl-2013-Scaling Semi-supervised Naive Bayes with Feature Marginals
9 0.56874287 174 acl-2013-Graph Propagation for Paraphrasing Out-of-Vocabulary Words in Statistical Machine Translation
10 0.56323892 194 acl-2013-Improving Text Simplification Language Modeling Using Unsimplified Text Data
11 0.56247258 2 acl-2013-A Bayesian Model for Joint Unsupervised Induction of Sentiment, Aspect and Discourse Representations
12 0.56134766 341 acl-2013-Text Classification based on the Latent Topics of Important Sentences extracted by the PageRank Algorithm
13 0.55671906 377 acl-2013-Using Supervised Bigram-based ILP for Extractive Summarization
14 0.55342126 185 acl-2013-Identifying Bad Semantic Neighbors for Improving Distributional Thesauri
15 0.55338323 172 acl-2013-Graph-based Local Coherence Modeling
16 0.55271184 99 acl-2013-Crowd Prefers the Middle Path: A New IAA Metric for Crowdsourcing Reveals Turker Biases in Query Segmentation
17 0.55224425 342 acl-2013-Text Classification from Positive and Unlabeled Data using Misclassified Data Correction
18 0.54540217 318 acl-2013-Sentiment Relevance
19 0.54520702 231 acl-2013-Linggle: a Web-scale Linguistic Search Engine for Words in Context
20 0.54406798 272 acl-2013-Paraphrase-Driven Learning for Open Question Answering