emnlp emnlp2013 emnlp2013-123 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Gyorgy Szarvas ; Robert Busa-Fekete ; Eyke Hullermeier
Abstract: The problem to replace a word with a synonym that fits well in its sentential context is known as the lexical substitution task. In this paper, we tackle this task as a supervised ranking problem. Given a dataset of target words, their sentential contexts and the potential substitutions for the target words, the goal is to train a model that accurately ranks the candidate substitutions based on their contextual fitness. As a key contribution, we customize and evaluate several learning-to-rank models to the lexical substitution task, including classification-based and regression-based approaches. On two datasets widely used for lexical substitution, our best models signifi- cantly advance the state-of-the-art.
Reference: text
sentIndex sentText sentNum sentScore
1 Learning to rank lexical substitutions Gy¨ orgy Szarvas1 Amazon Inc. [sent-1, score-0.601]
2 com Abstract The problem to replace a word with a synonym that fits well in its sentential context is known as the lexical substitution task. [sent-3, score-0.667]
3 In this paper, we tackle this task as a supervised ranking problem. [sent-4, score-0.17]
4 Given a dataset of target words, their sentential contexts and the potential substitutions for the target words, the goal is to train a model that accurately ranks the candidate substitutions based on their contextual fitness. [sent-5, score-1.163]
5 As a key contribution, we customize and evaluate several learning-to-rank models to the lexical substitution task, including classification-based and regression-based approaches. [sent-6, score-0.59]
6 On two datasets widely used for lexical substitution, our best models signifi- cantly advance the state-of-the-art. [sent-7, score-0.157]
7 1 Introduction The task to generate lexical substitutions in context (McCarthy and Navigli, 2007), i. [sent-8, score-0.527]
8 to evaluate semantic models with regard to their accuracy in modeling word meaning in context (Erk and Pad o´, 2010). [sent-13, score-0.073]
9 Moreover, it provides a basis of NLP applications in many fields, including linguistic steganography (Topkara et al. [sent-14, score-0.059]
10 de lexical substitution does not rely on explicitly defined sense inventories (Dagan et al. [sent-28, score-0.531]
11 , 2006): the possible substitutions reflect all conceivable senses of the word, and the correct sense has to be ascertained to provide an accurate substitution. [sent-29, score-0.428]
12 While a few lexical sample datasets (McCarthy and Navigli, 2007; Biemann, 2012) with humanprovided substitutions exist and can be used to evaluate different lexical paraphrasing approaches, a practically useful system must also be able to rephrase unseen words, i. [sent-30, score-0.682]
13 The only supervised approach is limited to the combination of several knowledge-based lexical substitution models based on different underlying lexicons (Sinha and Mihalcea, A recent work by Szarvas scribes a tailor-made supervised 2009). [sent-34, score-0.619]
14 (2013) de- system based on delexicalized features that unlike earlier supervised approaches, and similar to unsupervised and knowledge-based methods proposed for this task is able to generalize to an open vocabulary. [sent-36, score-0.146]
15 Each candidate then constitutes a training (or test) – – 3Another notable example for supervised lexical substitution is Biemann (2012), but this is a lexical sample system applicable only to the target words of the training datasets. [sent-38, score-0.881]
16 The goal is then i) to predict how well a particular candidate fits in the original context, and ii) given these predictions for each of the can- didates, to correctly order the elements of the candidate set according to their contextual fitness. [sent-42, score-0.234]
17 That is, a model is successful ifit prioritizes plausible substitutions ahead ofless likely synonyms (given the context). [sent-43, score-0.428]
18 This model is able to generate paraphrases for target words not contained in the training material. [sent-44, score-0.171]
19 local n-gram frequencies in context) that are meaningfully comparable across the different target words and candidate substitutions they are computed from. [sent-47, score-0.637]
20 More importantly, their model also provides superior ranking results compared to state of the art unsupervised and knowledge based approaches and therefore it defines the current state of the art for open vocabulary lexical substitution. [sent-48, score-0.223]
21 (2013), we address lexical substitution as a supervised learning problem, and go beyond their approach from a methodological point of view. [sent-50, score-0.575]
22 Our experiments show that the performance on the lexical substitution task is strongly influenced by the way in which this task is formalized as a machine learning problem (i. [sent-51, score-0.531]
23 2 Related work Previous approaches to lexical substitution often seek to automatically generate a set of candidate substitutions for each target word first, and to rank the elements ofthis set ofcandidates afterward (Hassan et al. [sent-55, score-1.198]
24 Alternatively, the candidate set can be defined by all human-suggested substitutions for the given target word in all of its contexts; then, the focus is just on the ranking problem (Erk and Pad o´, 2010; Thater et al. [sent-59, score-0.724]
25 (2013) recently formalized the lexical substitution problem as a supervised learning task, using delexicalized fea- tures. [sent-64, score-0.677]
26 This non-lexical feature representation makes different target word/substitution pairs in different contexts4 directly comparable. [sent-65, score-0.109]
27 Thus, it becomes possible to learn an all-words system that is applicable to unseen words, using supervised methods, which provides superior ranking accuracy to unsupervised and knowledge based models. [sent-66, score-0.209]
28 We customize and experiment with several different learning-to-rank models, which are better tailored for this task. [sent-69, score-0.059]
29 The second dataset, TWSI (Biemann, 2012)6, consists of 24,647 sentences for a total of 1,012 target nouns, and lexical substitu4E. [sent-77, score-0.206]
30 , bright substituted with intelligent in “He was bright and independent andproud” and side forpart in “Find someone who can compose the biblical side ”. [sent-79, score-0.118]
31 de/data/ lexi cal-re s ource s /tws i -lexi cal -subst itut i / ons tions for each target word in context resulting from a crowdsourced annotation process. [sent-87, score-0.15]
32 For each sentence in each dataset, the annotators provided as many substitutions for the target word as they found appropriate in the context. [sent-88, score-0.535]
33 Each substitution is then labeled by the number of annotators who listed that word as a good lexical substitution. [sent-89, score-0.568]
34 The datasets are randomly split into 10 equal-sized folds on the target word level, such that all examples for a particular target word fall into either the training or the test set, but never both. [sent-92, score-0.278]
35 Each (sentence, target word, substitution) triplet represents an instance, and the feature values are computed from the sentence context, the target word and the substitution word. [sent-101, score-0.652]
36 The most important features describe the syntag- matic coherence of the substitution in context, measured as local n-gram frequencies obtained from web data. [sent-103, score-0.473]
37 The frequency for a 1-5gram context with the substitution word is computed and normalized with respect to either 1) the frequency of the original context (with the target word) or 2) the sum of frequencies observed for all possible substitutions. [sent-104, score-0.664]
38 A third feature computes similar frequencies for the substitution and the target word observed in the local context (as part of a conjunctive phrase). [sent-105, score-0.623]
39 Lastly a group of features capture shallow syntactic patterns of the target word and its local context in the form of 1) part of speech patterns (trigrams) in a sliding window around the target word using main POS categories, i. [sent-111, score-0.259]
40 only the first letter of the Penn Treebank codes, and 2) the detailed POS code of the candidate word assigned by a POS tagger. [sent-113, score-0.1]
41 Importantly, these delexicalized features are numerically comparable across the different target words and candidate substitutions they are computed from. [sent-116, score-0.7]
42 This property enables the models to generalize over the words in the datasets and thus enables a supervised, all-words lexical substitution system. [sent-117, score-0.623]
43 4 Learning-to-Rank methods Machine learning methods for ranking are traditionally classified into three categories. [sent-118, score-0.126]
44 In the pointwise approach, a model is trained that maps instances (in this case candidate substitutions in a context) to scores indicating their relevance or fitness; to this end, one typically applies standard regression techniques, which essentially look at individual instances in isolation (i. [sent-119, score-0.691]
45 To predict a ranking of a set of query instances, these are simply sorted by their predicted scores (Li et al. [sent-122, score-0.126]
46 The pairwise approach trains models that are able to compare pairs of instances. [sent-124, score-0.045]
47 Finally, in the listwise approach, tailor-made learning methods are used that directly optimize the ranking performance with respect to a global evaluation metric, i. [sent-127, score-0.228]
48 , a measure that evaluates the ranking of a complete set of query instances (Valizadegan et al. [sent-129, score-0.159]
49 1 MAXENT The ranking model proposed by Szarvas et al. [sent-136, score-0.126]
50 This is a pointwise approach based on a maximum entropy classifier, in which the ranking task is cast as a binary classification problem, namely to discriminate good (label > 0) from bad substitutions. [sent-138, score-0.182]
51 The actual label values for good substitutions were used for weighting the training examples. [sent-139, score-0.389]
52 The ranking is then produced by sorting the candidates in decreasing order according to this probability. [sent-144, score-0.161]
53 , 2013) is a pointwise method with listwise meta-learning step that exploits an ensemble of multi-class classifiers. [sent-147, score-0.158]
54 MH (Schapire and Singer, 1999) classifiers with several different weak learners (Busa- Fekete et al. [sent-150, score-0.082]
55 , 2011; K e´gl and Busa-Fekete, 2009) are trained to predict the level of relevance (quality) of a substitution (i. [sent-151, score-0.479]
56 , the number of annotators who proposed the candidate for that particular context). [sent-153, score-0.137]
57 Second, the classifiers are calibrated to obtain 7RankLib is available at http : / /people . [sent-154, score-0.102]
58 1929 an accurate posterior distribution; to this end, several calibration techniques, such as Platt scaling (Platt, 2000), are used to obtain a diverse pool of calibrated classifiers. [sent-160, score-0.102]
59 Note that this step takes advantage of the ordinal structure of the underlying scale of relevance levels, which is an important difference to MAXENT. [sent-161, score-0.138]
60 Third, the posteriors of these calibrated classifiers are additively combined, with the weight of each model being exponentially proportional to its GAP score (on the validation set). [sent-162, score-0.102]
61 This method has two hyperparameters: the number of boosting iterations T and the scaling factor in the exponential weighting scheme c. [sent-163, score-0.19]
62 The objective function is the rank loss (as opposed to ADABOOST, which optimizes the exponential loss). [sent-168, score-0.101]
63 In each boosting iteration, the weak classifier is chosen by maximizing the weighted rank loss. [sent-169, score-0.275]
64 For the weak learner, we used the decision stump described in (Freund et al. [sent-170, score-0.118]
65 , 2003), which is able to optimize the rank loss in an efficient way. [sent-171, score-0.069]
66 The only hyperparameter of RANKBOOST to be tuned is the number of iterations that we selected from the interval [1, 1000] . [sent-172, score-0.075]
67 4 RANKSVM RANKSVM (Joachims, 2006) is a pairwise method based on support vector machines, which formulates the ranking task as binary classification of pairs of instances. [sent-174, score-0.171]
68 001 and the regularization parameter was validated in the interval [10−6, 104] with a logarithmically increasing step size. [sent-177, score-0.042]
69 , 2010) is a listwise method based on the gradient boosted regression trees by Friedman (1999). [sent-180, score-0.229]
70 The ordinal labels are learned directly by the boosted regression trees whose parameters are tuned by using a gradientbased optimization method. [sent-181, score-0.184]
71 We tuned the number of boosting ERLDCMax ntmpaxEdkbniBSdatsoV eMstARTW4 35. [sent-183, score-0.158]
72 iterations in the interval [10, 1000] and the number of tree leaves in {8, 16, 32}. [sent-195, score-0.042]
73 (2013) the only exception is thep@1 score for EXPENS on the Semeval Lexical Substitution dataset and the candidate substitutions extracted from WordNet. [sent-198, score-0.489]
74 the LexSub dataset using substitution candidates taken from the gold standard (see Table 2). [sent-203, score-0.469]
75 The RANKBOOST is a boosted decision stump where, in each boosting iteration, the stump is found by maximizing the weighted exponential rank loss. [sent-207, score-0.455]
76 6 Conclusion and future work In this paper, we customized and applied some relatively novel algorithms from the field of learning-torank for ranking lexical substitutions in context. [sent-212, score-0.612]
77 In turn, we achieved significant improvements on the two prominent datasets for lexical substitution. [sent-213, score-0.189]
78 Our results indicate that an exploitation of the ordinal structure ofthe labels in the datasets can lead to considerable gains in terms of both ranking quality (GAP) and precision at 1 (p@ 1). [sent-214, score-0.279]
79 This observation is supported both for the theoretically simpler pointwise learning approach and for the most powerful listwise approach. [sent-215, score-0.158]
80 On the other hand, the pairwise methods that cannot naturally exploit this property, did not provide a consistent improvement over the baseline. [sent-216, score-0.045]
81 In the future, we plan to investigate this finding in the context of other, similar ranking problems in Natural Language Processing. [sent-217, score-0.167]
82 In *SEM 2012: The First Joint Conference on Lexical and Computational Semantics – Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012), pages 385–393, Montr ´eal, Canada. [sent-222, score-0.041]
83 Tune and mix: learning to rank using ensembles of calibrated multi-class classifiers. [sent-251, score-0.171]
84 Practical linguistic steganography using contextual synonym substitution and vertex colour coding. [sent-255, score-0.589]
85 In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1194–1203, Cambridge, MA. [sent-256, score-0.041]
86 In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, ACL-44, pages 449–456, Sydney, Australia. [sent-260, score-0.041]
87 In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1162–1 172, Cambridge, MA. [sent-264, score-0.041]
88 In Proceedings of the ACL 2010 Conference Short Papers, pages 92– 97, Uppsala, Sweden. [sent-268, score-0.041]
89 In Proceedings of 11th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL’11), pages 255–258, Ottawa, Canada. [sent-287, score-0.041]
90 FBK-irst: Lexical substitution task exploiting domain and syntagmatic coherence. [sent-293, score-0.434]
91 In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007), pages 145–148, Prague, Czech Republic. [sent-294, score-0.041]
92 UNT: SubFinder: Combining knowledge sources for automatic lexical substitution. [sent-297, score-0.097]
93 In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval2007), pages 410–413, Prague, Czech Republic. [sent-298, score-0.041]
94 In International Conference on Machine Learning, volume 26, pages 497–504, Montreal, Canada. [sent-309, score-0.041]
95 McRank: Learning to rank using multiple classification and gradient boosting. [sent-320, score-0.105]
96 In Advances in Neural Information Processing Systems, volume 19, pages 897–904. [sent-321, score-0.041]
97 MELB-MKB: Lexical substitution system based on relatives in context. [sent-325, score-0.434]
98 In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007), pages 237–240, Prague, Czech Republic. [sent-326, score-0.041]
99 In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007), pages 48–53, Prague, Czech Republic. [sent-330, score-0.041]
100 The hiding virtues of ambiguity: quantifiably resilient watermarking of natural language text through synonym substitutions. [sent-369, score-0.062]
wordName wordTfidf (topN-words)
[('substitution', 0.434), ('substitutions', 0.389), ('expens', 0.234), ('szarvas', 0.232), ('lambdamart', 0.178), ('boosting', 0.158), ('ranksvm', 0.127), ('rankboost', 0.127), ('ranking', 0.126), ('target', 0.109), ('calibrated', 0.102), ('listwise', 0.102), ('delexicalized', 0.102), ('candidate', 0.1), ('lexical', 0.097), ('ordinal', 0.093), ('synsets', 0.093), ('lexsub', 0.088), ('sinha', 0.088), ('topkara', 0.088), ('gl', 0.087), ('biemann', 0.082), ('thater', 0.078), ('freund', 0.078), ('mccarthy', 0.071), ('stump', 0.07), ('rank', 0.069), ('paraphrases', 0.062), ('synonym', 0.062), ('gap', 0.061), ('semeval', 0.061), ('datasets', 0.06), ('alfio', 0.059), ('bright', 0.059), ('customize', 0.059), ('eyke', 0.059), ('gipp', 0.059), ('giuliano', 0.059), ('ltet', 0.059), ('marburg', 0.059), ('mathemat', 0.059), ('multiboost', 0.059), ('steganography', 0.059), ('valizadegan', 0.059), ('navigli', 0.058), ('pointwise', 0.056), ('boosted', 0.056), ('schapire', 0.056), ('prague', 0.056), ('czech', 0.054), ('erk', 0.053), ('fourth', 0.053), ('evaluations', 0.052), ('benbouzid', 0.051), ('gliozzo', 0.051), ('martinez', 0.051), ('pad', 0.05), ('maxent', 0.05), ('weak', 0.048), ('orgy', 0.046), ('relevance', 0.045), ('pairwise', 0.045), ('supervised', 0.044), ('plagiarism', 0.043), ('distributional', 0.042), ('interval', 0.042), ('context', 0.041), ('pages', 0.041), ('gy', 0.041), ('hagen', 0.041), ('urstenau', 0.041), ('hungarian', 0.041), ('senses', 0.039), ('synonyms', 0.039), ('dagan', 0.039), ('ravi', 0.039), ('frequencies', 0.039), ('unseen', 0.039), ('annotators', 0.037), ('gradient', 0.036), ('ik', 0.036), ('manfred', 0.036), ('dinu', 0.036), ('candidates', 0.035), ('regression', 0.035), ('wordnet', 0.035), ('platt', 0.034), ('burges', 0.034), ('learners', 0.034), ('agirre', 0.034), ('contextual', 0.034), ('hassan', 0.033), ('hyperparameter', 0.033), ('sentential', 0.033), ('instances', 0.033), ('property', 0.032), ('exponential', 0.032), ('prominent', 0.032), ('semantic', 0.032), ('carlo', 0.031)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000007 123 emnlp-2013-Learning to Rank Lexical Substitutions
Author: Gyorgy Szarvas ; Robert Busa-Fekete ; Eyke Hullermeier
Abstract: The problem to replace a word with a synonym that fits well in its sentential context is known as the lexical substitution task. In this paper, we tackle this task as a supervised ranking problem. Given a dataset of target words, their sentential contexts and the potential substitutions for the target words, the goal is to train a model that accurately ranks the candidate substitutions based on their contextual fitness. As a key contribution, we customize and evaluate several learning-to-rank models to the lexical substitution task, including classification-based and regression-based approaches. On two datasets widely used for lexical substitution, our best models signifi- cantly advance the state-of-the-art.
2 0.097620621 39 emnlp-2013-Boosting Cross-Language Retrieval by Learning Bilingual Phrase Associations from Relevance Rankings
Author: Artem Sokokov ; Laura Jehl ; Felix Hieber ; Stefan Riezler
Abstract: We present an approach to learning bilingual n-gram correspondences from relevance rankings of English documents for Japanese queries. We show that directly optimizing cross-lingual rankings rivals and complements machine translation-based cross-language information retrieval (CLIR). We propose an efficient boosting algorithm that deals with very large cross-product spaces of word correspondences. We show in an experimental evaluation on patent prior art search that our approach, and in particular a consensus-based combination of boosting and translation-based approaches, yields substantial improvements in CLIR performance. Our training and test data are made publicly available.
3 0.094604865 28 emnlp-2013-Automated Essay Scoring by Maximizing Human-Machine Agreement
Author: Hongbo Chen ; Ben He
Abstract: Previous approaches for automated essay scoring (AES) learn a rating model by minimizing either the classification, regression, or pairwise classification loss, depending on the learning algorithm used. In this paper, we argue that the current AES systems can be further improved by taking into account the agreement between human and machine raters. To this end, we propose a rankbased approach that utilizes listwise learning to rank algorithms for learning a rating model, where the agreement between the human and machine raters is directly incorporated into the loss function. Various linguistic and statistical features are utilized to facilitate the learning algorithms. Experiments on the publicly available English essay dataset, Automated Student Assessment Prize (ASAP), show that our proposed approach outperforms the state-of-the-art algorithms, and achieves performance comparable to professional human raters, which suggests the effectiveness of our proposed method for automated essay scoring.
Author: Claire Gardent ; Lina M. Rojas Barahona
Abstract: This paper explores to what extent lemmatisation, lexical resources, distributional semantics and paraphrases can increase the accuracy of supervised models for dialogue management. The results suggest that each of these factors can help improve performance but that the impact will vary depending on their combination and on the evaluation mode.
5 0.073522165 96 emnlp-2013-Identifying Phrasal Verbs Using Many Bilingual Corpora
Author: Karl Pichotta ; John DeNero
Abstract: We address the problem of identifying multiword expressions in a language, focusing on English phrasal verbs. Our polyglot ranking approach integrates frequency statistics from translated corpora in 50 different languages. Our experimental evaluation demonstrates that combining statistical evidence from many parallel corpora using a novel ranking-oriented boosting algorithm produces a comprehensive set ofEnglish phrasal verbs, achieving performance comparable to a human-curated set.
6 0.070011213 64 emnlp-2013-Discriminative Improvements to Distributional Sentence Similarity
7 0.068717763 169 emnlp-2013-Semi-Supervised Representation Learning for Cross-Lingual Text Classification
8 0.067042284 60 emnlp-2013-Detecting Compositionality of Multi-Word Expressions using Nearest Neighbours in Vector Space Models
9 0.064764723 93 emnlp-2013-Harvesting Parallel News Streams to Generate Paraphrases of Event Relations
10 0.063648663 69 emnlp-2013-Efficient Collective Entity Linking with Stacking
11 0.0594921 138 emnlp-2013-Naive Bayes Word Sense Induction
12 0.057369109 9 emnlp-2013-A Log-Linear Model for Unsupervised Text Normalization
13 0.057099573 12 emnlp-2013-A Semantically Enhanced Approach to Determine Textual Similarity
14 0.056859318 165 emnlp-2013-Scaling to Large3 Data: An Efficient and Effective Method to Compute Distributional Thesauri
16 0.054958705 58 emnlp-2013-Dependency Language Models for Sentence Completion
17 0.054508045 134 emnlp-2013-Modeling and Learning Semantic Co-Compositionality through Prototype Projections and Neural Networks
18 0.05292172 181 emnlp-2013-The Effects of Syntactic Features in Automatic Prediction of Morphology
19 0.051881846 70 emnlp-2013-Efficient Higher-Order CRFs for Morphological Tagging
20 0.049877781 17 emnlp-2013-A Walk-Based Semantically Enriched Tree Kernel Over Distributed Word Representations
topicId topicWeight
[(0, -0.191), (1, 0.002), (2, -0.032), (3, -0.014), (4, -0.026), (5, 0.071), (6, 0.026), (7, 0.015), (8, -0.053), (9, -0.078), (10, 0.041), (11, 0.072), (12, -0.048), (13, 0.064), (14, 0.07), (15, -0.014), (16, -0.07), (17, 0.016), (18, 0.039), (19, 0.015), (20, 0.042), (21, -0.007), (22, -0.063), (23, 0.054), (24, 0.035), (25, 0.036), (26, -0.06), (27, 0.006), (28, -0.121), (29, -0.052), (30, 0.16), (31, 0.107), (32, 0.056), (33, -0.137), (34, 0.097), (35, 0.086), (36, -0.004), (37, -0.043), (38, -0.05), (39, -0.104), (40, 0.061), (41, -0.261), (42, 0.011), (43, 0.143), (44, -0.061), (45, 0.024), (46, 0.095), (47, 0.105), (48, -0.01), (49, -0.051)]
simIndex simValue paperId paperTitle
same-paper 1 0.91438293 123 emnlp-2013-Learning to Rank Lexical Substitutions
Author: Gyorgy Szarvas ; Robert Busa-Fekete ; Eyke Hullermeier
Abstract: The problem to replace a word with a synonym that fits well in its sentential context is known as the lexical substitution task. In this paper, we tackle this task as a supervised ranking problem. Given a dataset of target words, their sentential contexts and the potential substitutions for the target words, the goal is to train a model that accurately ranks the candidate substitutions based on their contextual fitness. As a key contribution, we customize and evaluate several learning-to-rank models to the lexical substitution task, including classification-based and regression-based approaches. On two datasets widely used for lexical substitution, our best models signifi- cantly advance the state-of-the-art.
2 0.71948731 28 emnlp-2013-Automated Essay Scoring by Maximizing Human-Machine Agreement
Author: Hongbo Chen ; Ben He
Abstract: Previous approaches for automated essay scoring (AES) learn a rating model by minimizing either the classification, regression, or pairwise classification loss, depending on the learning algorithm used. In this paper, we argue that the current AES systems can be further improved by taking into account the agreement between human and machine raters. To this end, we propose a rankbased approach that utilizes listwise learning to rank algorithms for learning a rating model, where the agreement between the human and machine raters is directly incorporated into the loss function. Various linguistic and statistical features are utilized to facilitate the learning algorithms. Experiments on the publicly available English essay dataset, Automated Student Assessment Prize (ASAP), show that our proposed approach outperforms the state-of-the-art algorithms, and achieves performance comparable to professional human raters, which suggests the effectiveness of our proposed method for automated essay scoring.
3 0.58903128 165 emnlp-2013-Scaling to Large3 Data: An Efficient and Effective Method to Compute Distributional Thesauri
Author: Martin Riedl ; Chris Biemann
Abstract: We introduce a new highly scalable approach for computing Distributional Thesauri (DTs). By employing pruning techniques and a distributed framework, we make the computation for very large corpora feasible on comparably small computational resources. We demonstrate this by releasing a DT for the whole vocabulary of Google Books syntactic n-grams. Evaluating against lexical resources using two measures, we show that our approach produces higher quality DTs than previous approaches, and is thus preferable in terms of speed and quality for large corpora.
Author: Claire Gardent ; Lina M. Rojas Barahona
Abstract: This paper explores to what extent lemmatisation, lexical resources, distributional semantics and paraphrases can increase the accuracy of supervised models for dialogue management. The results suggest that each of these factors can help improve performance but that the impact will vary depending on their combination and on the evaluation mode.
5 0.47473785 39 emnlp-2013-Boosting Cross-Language Retrieval by Learning Bilingual Phrase Associations from Relevance Rankings
Author: Artem Sokokov ; Laura Jehl ; Felix Hieber ; Stefan Riezler
Abstract: We present an approach to learning bilingual n-gram correspondences from relevance rankings of English documents for Japanese queries. We show that directly optimizing cross-lingual rankings rivals and complements machine translation-based cross-language information retrieval (CLIR). We propose an efficient boosting algorithm that deals with very large cross-product spaces of word correspondences. We show in an experimental evaluation on patent prior art search that our approach, and in particular a consensus-based combination of boosting and translation-based approaches, yields substantial improvements in CLIR performance. Our training and test data are made publicly available.
6 0.47462955 32 emnlp-2013-Automatic Idiom Identification in Wiktionary
7 0.46341315 60 emnlp-2013-Detecting Compositionality of Multi-Word Expressions using Nearest Neighbours in Vector Space Models
8 0.44946235 170 emnlp-2013-Sentiment Analysis: How to Derive Prior Polarities from SentiWordNet
9 0.44105896 138 emnlp-2013-Naive Bayes Word Sense Induction
10 0.43638083 64 emnlp-2013-Discriminative Improvements to Distributional Sentence Similarity
11 0.43150622 200 emnlp-2013-Well-Argued Recommendation: Adaptive Models Based on Words in Recommender Systems
12 0.42378938 79 emnlp-2013-Exploiting Multiple Sources for Open-Domain Hypernym Discovery
13 0.38812533 96 emnlp-2013-Identifying Phrasal Verbs Using Many Bilingual Corpora
14 0.38009793 93 emnlp-2013-Harvesting Parallel News Streams to Generate Paraphrases of Event Relations
15 0.3786456 199 emnlp-2013-Using Topic Modeling to Improve Prediction of Neuroticism and Depression in College Students
16 0.36322066 23 emnlp-2013-Animacy Detection with Voting Models
17 0.35903597 18 emnlp-2013-A temporal model of text periodicities using Gaussian Processes
18 0.34546998 196 emnlp-2013-Using Crowdsourcing to get Representations based on Regular Expressions
19 0.34452179 69 emnlp-2013-Efficient Collective Entity Linking with Stacking
20 0.34350947 195 emnlp-2013-Unsupervised Spectral Learning of WCFG as Low-rank Matrix Completion
topicId topicWeight
[(3, 0.032), (6, 0.012), (18, 0.039), (22, 0.059), (30, 0.071), (50, 0.014), (51, 0.181), (66, 0.027), (71, 0.411), (75, 0.042), (77, 0.012), (96, 0.03)]
simIndex simValue paperId paperTitle
1 0.99091083 161 emnlp-2013-Rule-Based Information Extraction is Dead! Long Live Rule-Based Information Extraction Systems!
Author: Laura Chiticariu ; Yunyao Li ; Frederick R. Reiss
Abstract: The rise of “Big Data” analytics over unstructured text has led to renewed interest in information extraction (IE). We surveyed the landscape ofIE technologies and identified a major disconnect between industry and academia: while rule-based IE dominates the commercial world, it is widely regarded as dead-end technology by the academia. We believe the disconnect stems from the way in which the two communities measure the benefits and costs of IE, as well as academia’s perception that rulebased IE is devoid of research challenges. We make a case for the importance of rule-based IE to industry practitioners. We then lay out a research agenda in advancing the state-of-theart in rule-based IE systems which we believe has the potential to bridge the gap between academic research and industry practice.
2 0.88697916 63 emnlp-2013-Discourse Level Explanatory Relation Extraction from Product Reviews Using First-Order Logic
Author: Qi Zhang ; Jin Qian ; Huan Chen ; Jihua Kang ; Xuanjing Huang
Abstract: Explanatory sentences are employed to clarify reasons, details, facts, and so on. High quality online product reviews usually include not only positive or negative opinions, but also a variety of explanations of why these opinions were given. These explanations can help readers get easily comprehensible information of the discussed products and aspects. Moreover, explanatory relations can also benefit sentiment analysis applications. In this work, we focus on the task of identifying subjective text segments and extracting their corresponding explanations from product reviews in discourse level. We propose a novel joint extraction method using firstorder logic to model rich linguistic features and long distance constraints. Experimental results demonstrate the effectiveness of the proposed method.
3 0.84511548 151 emnlp-2013-Paraphrasing 4 Microblog Normalization
Author: Wang Ling ; Chris Dyer ; Alan W Black ; Isabel Trancoso
Abstract: Compared to the edited genres that have played a central role in NLP research, microblog texts use a more informal register with nonstandard lexical items, abbreviations, and free orthographic variation. When confronted with such input, conventional text analysis tools often perform poorly. Normalization replacing orthographically or lexically idiosyncratic forms with more standard variants can improve performance. We propose a method for learning normalization rules from machine translations of a parallel corpus of microblog messages. To validate the utility of our approach, we evaluate extrinsically, showing that normalizing English tweets and then translating improves translation quality (compared to translating unnormalized text) using three standard web translation services as well as a phrase-based translation system trained — — on parallel microblog data.
same-paper 4 0.8057369 123 emnlp-2013-Learning to Rank Lexical Substitutions
Author: Gyorgy Szarvas ; Robert Busa-Fekete ; Eyke Hullermeier
Abstract: The problem to replace a word with a synonym that fits well in its sentential context is known as the lexical substitution task. In this paper, we tackle this task as a supervised ranking problem. Given a dataset of target words, their sentential contexts and the potential substitutions for the target words, the goal is to train a model that accurately ranks the candidate substitutions based on their contextual fitness. As a key contribution, we customize and evaluate several learning-to-rank models to the lexical substitution task, including classification-based and regression-based approaches. On two datasets widely used for lexical substitution, our best models signifi- cantly advance the state-of-the-art.
Author: Jinpeng Wang ; Wayne Xin Zhao ; Haitian Wei ; Hongfei Yan ; Xiaoming Li
Abstract: Hot trends are likely to bring new business opportunities. For example, “Air Pollution” might lead to a significant increase of the sales of related products, e.g., mouth mask. For ecommerce companies, it is very important to make rapid and correct response to these hot trends in order to improve product sales. In this paper, we take the initiative to study the task of how to identify trend related products. The major novelty of our work is that we automatically learn commercial intents revealed from microblogs. We carefully construct a data collection for this task and present quite a few insightful findings. In order to solve this problem, we further propose a graph based method, which jointly models relevance and associativity. We perform extensive experiments and the results showed that our methods are very effective.
6 0.60166919 47 emnlp-2013-Collective Opinion Target Extraction in Chinese Microblogs
8 0.56742197 144 emnlp-2013-Opinion Mining in Newspaper Articles by Entropy-Based Word Connections
9 0.56146371 202 emnlp-2013-Where Not to Eat? Improving Public Policy by Predicting Hygiene Inspections Using Online Reviews
10 0.55384117 77 emnlp-2013-Exploiting Domain Knowledge in Aspect Extraction
11 0.5505209 196 emnlp-2013-Using Crowdsourcing to get Representations based on Regular Expressions
12 0.54928994 143 emnlp-2013-Open Domain Targeted Sentiment
13 0.54615754 179 emnlp-2013-Summarizing Complex Events: a Cross-Modal Solution of Storylines Extraction and Reconstruction
14 0.544101 48 emnlp-2013-Collective Personal Profile Summarization with Social Networks
15 0.54211521 62 emnlp-2013-Detection of Product Comparisons - How Far Does an Out-of-the-Box Semantic Role Labeling System Take You?
16 0.53919429 18 emnlp-2013-A temporal model of text periodicities using Gaussian Processes
17 0.53813678 170 emnlp-2013-Sentiment Analysis: How to Derive Prior Polarities from SentiWordNet
18 0.53639454 94 emnlp-2013-Identifying Manipulated Offerings on Review Portals
19 0.53523469 9 emnlp-2013-A Log-Linear Model for Unsupervised Text Normalization