acl acl2013 acl2013-63 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Maria Yancheva ; Frank Rudzicz
Abstract: It is important that the testimony of children be admissible in court, especially given allegations of abuse. Unfortunately, children can be misled by interrogators or might offer false information, with dire consequences. In this work, we evaluate various parameterizations of five classifiers (including support vector machines, neural networks, and random forests) in deciphering truth from lies given transcripts of interviews with 198 victims of abuse between the ages of 4 and 7. These evaluations are performed using a novel set of syntactic features, including measures of complexity. Our results show that sentence length, the mean number of clauses per utterance, and the StajnerMitkov measure of complexity are highly informative syntactic features, that classification accuracy varies greatly by the age of the speaker, and that accuracy up to 91.7% can be achieved by support vector machines given a sufficient amount of data.
Reference: text
sentIndex sentText sentNum sentScore
1 Automatic detection of deception in child-produced speech using syntactic complexity features Maria Yancheva Division of Engineering Science, University of Toronto Toronto Ontario Canada mari a . [sent-1, score-0.872]
2 ca Abstract It is important that the testimony of children be admissible in court, especially given allegations of abuse. [sent-3, score-0.249]
3 Unfortunately, children can be misled by interrogators or might offer false information, with dire consequences. [sent-4, score-0.209]
4 In this work, we evaluate various parameterizations of five classifiers (including support vector machines, neural networks, and random forests) in deciphering truth from lies given transcripts of interviews with 198 victims of abuse between the ages of 4 and 7. [sent-5, score-0.334]
5 Our results show that sentence length, the mean number of clauses per utterance, and the StajnerMitkov measure of complexity are highly informative syntactic features, that classification accuracy varies greatly by the age of the speaker, and that accuracy up to 91. [sent-7, score-0.592]
6 1 Introduction The challenge of disambiguating between truth and deception is critical in determining the admissibility of court testimony. [sent-9, score-0.84]
7 Unfortunately, the testimony of maltreated children is often not admitted in court due to concerns about truthfulness since children can be instructed to deny transgressions or misled to elicit false accusations (Lyon and Dorado, 2008). [sent-10, score-0.7]
8 However, the child is often the only witness of the transgression (Undeutsch, 2008); automatically determining truthfulness in Frank Rudzicz Toronto Rehabilitation Institute; and Department of Computer Science, University of Toronto Toronto Ontario Canada frank@ cs . [sent-11, score-0.208]
9 2 Related Work Research in the detection of deception in adult speech has included analyses of verbal and nonverbal cues such as behavioral changes, facial expression, speech dysfluencies, and cognitive complexity (DePaulo et al. [sent-14, score-1.044]
10 Despite statistically significant predictors of deception such as shorter talking time, fewer semantic details, and less coherent statements, DePaulo et al. [sent-16, score-0.628]
11 Results suggested that 3-year-old children are capable of deception, and that non-verbal behaviors during deception include increases in ‘positive’ behaviors (e. [sent-26, score-0.761]
12 (1989) showed that humans are no more accurate in deciphering truth from deception in child speech than in adult speech, being only about 50% accurate. [sent-31, score-1.032]
13 (2003) inferred deception in transcribed, typed, and handwritten text by identifying features of linguistic style such as the use of personal pronouns 944 Proce dingSsof oifa, th Beu 5l1gsarti Aan,An u aglu Mste 4e-ti9n2g 0 o1f3 t. [sent-34, score-0.636]
14 These features were obtained with the Linguistic Inquiry and Word Count (LIWC) tool and used in a logistic regression classifier which achieved, on average, 61% accuracy on test data. [sent-39, score-0.134]
15 Feature analysis showed that deceptive stories were characterized by fewer self-references, more negative emotion words, and lower cognitive complexity, compared to non-deceptive language. [sent-40, score-0.271]
16 Another recent stylometric experiment in automatic identification of deception was performed by Mihalcea and Strapparava (2009). [sent-41, score-0.592]
17 The authors used a dataset of truthful and deceptive typed responses produced by adult subjects on three different topics, collected through the Amazon Mechanical Turk service. [sent-42, score-0.402]
18 The large variability of classifier performance based on the topic of deception suggests that performance is context-dependent. [sent-45, score-0.644]
19 A meta-analysis of features used in deception detection was performed by Hauch et al. [sent-57, score-0.684]
20 (2012) and revealed that verbal cues based on lexical categories extracted using the LIWC tool show statistically significant, though small, differences between truth- and lie-tellers. [sent-58, score-0.152]
21 In the speech community, analysis of deceptive speech has combined various acoustic, prosodic, and lexical features (Hirschberg et al. [sent-60, score-0.402]
22 (2006) combined two independent systems an acoustic Gaussian mixture model based on Mel cepstral features, and a prosodic — support vector machine based on features such as pitch, energy, and duration and achieved an accuracy of 64. [sent-63, score-0.176]
23 4% on a test subset of the ColumbiaSRI-Colorado (CSC) corpus of deceptive and nondeceptive speech (Hirschberg et al. [sent-64, score-0.325]
24 While previous studies have achieved some promising results in detecting deception with lexical, acoustic, and prosodic features, syntax remains relatively unexplored compared to LIWCbased features. [sent-66, score-0.645]
25 Syntactic complexity as a cue to deception is consistent with literature in social psychology which suggests that emotion suppression (e. [sent-67, score-0.713]
26 Additionally, the use of syntactic features is motivated by their successful use on adult-produced datasets for detecting deceptive or uncertain utterances (Feng et al. [sent-70, score-0.365]
27 Past work has focused on identifying deceptive speech produced by adults. [sent-75, score-0.291]
28 The problem of determining validity of child testimony in high-stakes child abuse court cases motivates the analysis of child-produced deceptive language. [sent-76, score-0.643]
29 Due to the rarity of real deceptive data, studies typically use arti945 ficially produced deceptive language which falls unambiguously in one of two classes: complete truth or complete deception (Newman et al. [sent-78, score-1.228]
30 Studies which make use of real high-stakes courtroom data containing partial truths, such as the Italian DECOUR corpus analyzed by Fornaciari and Poesio (2012), preprocess the dataset to eliminate any partially truthful utterances. [sent-80, score-0.126]
31 The present study evaluates the viability of a novel set of 17 syntactic features as markers of deception in five classifiers. [sent-82, score-0.683]
32 Moreover, to our knowledge, it is the first application of automatic deception detection to a real-life dataset of deceptive speech produced by maltreated children. [sent-83, score-1.017]
33 (2008), who conducted and transcribed a truth-induction experiment involving maltreated children awaiting court appearances in the Los Angeles County Dependency Court. [sent-87, score-0.315]
34 Subjects were children between the ages of4 and 7 (99 boys and 99 girls) who were interviewed regarding an unambiguous minor transgression involving play- ing with a toy. [sent-88, score-0.211]
35 During data collection, a confederate first engaged each child individually in one of four conditions: a) play, b) play and coach, c) no play, and d) no play and coach. [sent-90, score-0.415]
36 In the two play conditions, the confederate engaged the child in play with a toy house (in the no play conditions, they did not); in the two coach conditions, the confederate coached the child to lie (i. [sent-91, score-0.928]
37 , to deny playing if they played with the toy house, or to admit playing if they did not). [sent-93, score-0.308]
38 All children in theplay condition engaged in all seven actions, while children in the no play condition engaged in none. [sent-96, score-0.589]
39 An eighth feature is the lack of explicit denial of touching or playing with the toy house, which is considered to be truthful in the play condition, and deceptive in the no play condition (see the examples in the appendix). [sent-97, score-0.68]
40 A transcription is labeled as truth if at least half of these features are truthful (53. [sent-98, score-0.37]
41 Each child was subject to one of the four confederate conditions and one of the three induction conditions. [sent-104, score-0.225]
42 4 Methods Since the data consist of speech produced by 4- to 7-year-old children, the predictive features must depend on the level of syntactic competence of this age group. [sent-106, score-0.318]
43 The “continuity assumption” states that children have a complete system of abstract syntactic representation and have the same set of abstract functional categories accessible to adults (Pinker, 1984). [sent-107, score-0.254]
44 Recent experiments with syntactic priming showed that children’s representations of abstract passive constructions are well- developed as early as age 3 or 4, and young 946 children are generally able to form passive constructions with both action and non-action verbs (Thatcher et al. [sent-110, score-0.532]
45 These results suggest that measures of syntactic complexity that are typically used to evaluate adult language could be adapted to child speech, provided that the children are at least 3 or 4 years old. [sent-112, score-0.475]
46 Here, the complexity of speech is characterized by the length of utterances and by the frequency of dependent and coordinate clauses, with more complex speech consisting of longer utterances and a higher number of subordinate clauses. [sent-113, score-0.423]
47 We segmented the transcriptions into sentences, clauses and T-units, which are “minimally terminable units” consisting of a main clause and its dependent clauses (Hunt, 1965; O’Donnell et al. [sent-114, score-0.528]
48 Here, the frequency of dependent and coordinate clauses per constituent approximate clause-based measures of complexity. [sent-118, score-0.308]
49 We obtained FDG parses of the transcriptions using Connexor’s Machinese Syntax parser (Tapanainen and J ¨arvinen, 1997) and extracted the following 5 features: ARI Automated readability index. [sent-120, score-0.137]
50 predicators (Stajner aonsde The ratio of non-finite main in a passive construction (@– 1T-units include single clauses, two or more phrases in apposition, or clause fragments. [sent-130, score-0.192]
51 Generally, coordinate clauses are split into separate T-units, as are clauses interrupted by discourse boundary markers. [sent-131, score-0.346]
52 A onefactor ANOVA across the truth and lie groups showed three significant feature variations: average sentence length (ASL), sentence complexity (COM), and mean clauses per utterance (MCU). [sent-138, score-0.57]
53 As expected, the number of clauses (MCU) is dependent on sentence length (ASL) (r(344) = . [sent-140, score-0.189]
54 05), and the density of dependent clauses is independent of the density of coordinate phrases (r(344) = −. [sent-157, score-0.255]
55 The MLP includes a variable number of layers of hidden units, which 947 Figure 1: Independent and dependent feature pairs; data points are labeled as truth (blue) and lie (green). [sent-162, score-0.297]
56 1 Binary classification across all data The five classifiers were evaluated on the entire pooled data set with 10-fold cross validation. [sent-204, score-0.151]
57 2 Binary classification by age group Significant variation in syntactic complexity is expected across ages. [sent-217, score-0.323]
58 (2004) used data from only 35 children in their study of 5- and 6-year-olds. [sent-228, score-0.169]
59 In comparison with classification accuracy on pooled data, a paired t-test shows statistically significant improvement across all age groups using RF, t(3) = 10. [sent-230, score-0.371]
60 The best classifier at each age is shown in bold. [sent-254, score-0.212]
61 3 Binary classification by age group, on µ verbose transcriptions The length of speech, in number of words, varies widely (min = 1, max = 167, = 36. [sent-260, score-0.418]
62 To test the effect of verbosity, we segment the data by child age and select only the transcriptions with above-average word counts (i. [sent-263, score-0.436]
63 , ≥ 37 words), resulting in four groups: u1n2t transcriptions oofr 4-year-olds, g4 i8n o ffo u5-r year-olds, 39 of 6-year-olds, and 37 of7-year-olds. [sent-265, score-0.137]
64 3% of transcripts are labeled truth across age groups (using the same definition of truth as in section 3), with no substantial variation between ages; in the non-verbose case, 53. [sent-268, score-0.591]
65 The size of the training set for the youngest age category is low compared to the other age groups, which may reduce the reliability of the higher ac- curacy achieved in that group. [sent-273, score-0.32]
66 The other three age groups show a growing trend, which is consistent with expectations older children exhibit greater syntactic complexity in speech, allowing greater variability of feature values across truth and deception. [sent-274, score-0.693]
67 8% cross-validation accuracy in identifying deception in the speech of 7-year-old subjects. [sent-276, score-0.697]
68 8378† Table 5: Cross-validation accuracy of binary classification performed on transcriptions with above average word count (136 transcriptions), by age group. [sent-297, score-0.377]
69 4) Threshold variation To study the effect of the threshold between the truth and lie classes, we vary the value of the threshold, τ, from 1 to 8, requiring the admission of at least τ truthful details (out of 8 possible details) in order to label a transcription as truth. [sent-305, score-0.436]
70 The effect of τ on classification accuracy over the entire pooled dataset for each of the 5 classifiers is shown in Figure 2. [sent-306, score-0.189]
71 A one-factor ANOVA with τ as the independent variable with 8 levels, and cross-validation accuracy as the dependent variable, confirms that the effect ofthe threshold is statistically significant (F7,40 = 220. [sent-307, score-0.173]
72 We compare the performance of classifiers trained with our 17 syntactic features to those of classifiers trained with those LIWC-based features on the same data. [sent-315, score-0.273]
73 The set of 17 syntactic features proposed here result in significantly higher accuracies across classifiers and experiments (µ = 0. [sent-318, score-0.16]
74 6 Discussion and future work This paper evaluates automatic estimation oftruth- fulness in the utterances of children using a novel set of lexical-syntactic features across five types of classifiers. [sent-325, score-0.263]
75 , 2012; Fornaciari and Poesio, 2012), our results suggest that the set of syntactic features presented here perform significantly better than the LIWC feature set on our data, and across seven out of the eight experiments based on age groups and verbosity of transcriptions. [sent-330, score-0.306]
76 Statistical analyses showed that the average sentence length (ASL), the Stajner-Mitkov measure of sentence complexity (COM), and the mean number of clauses per utterance (MCU) are the features most predictive of truth and deception (see section 4. [sent-331, score-1.091]
77 This results in a set of only two features: sentence complexity (COM) and T-units per sentence (T/S). [sent-335, score-0.127]
78 This approach selects five features: verb phrases per Tunit (VP/T), passive sentences (PAS), coordinate phrases per clause (CP/C), sentence complexity (COM), and complex nominals per clause (CN/C). [sent-342, score-0.539]
79 Generalized cross-validation accuracy increases significantly given partitioned age groups, which suggests that the importance of features may be moderated by age. [sent-371, score-0.242]
80 A further incremental increase is achieved by considering only transcriptions above a minimum length. [sent-372, score-0.137]
81 (1967) examined syntactic complexity in the speech and writing of children aged 8 to 12, and found that speech complexity increases with age. [sent-374, score-0.498]
82 Future examination of the effect of age on feature saliency may yield more appropriate age-dependent features. [sent-376, score-0.16]
83 7% accuracy across age groups a significant improvement over the baselines of LR and NB, as well as over previous results. [sent-380, score-0.253]
84 Moreover, since the performance of human judges in identifying deception is not significantly better than chance — (Lewis et al. [sent-381, score-0.592]
85 Partially truthful transcriptions were scored using a gradient of 0 to 8 truthful details, and a threshold τ was used to perform binary classification. [sent-384, score-0.371]
86 Extreme values of τ lead to poor F-scores despite high accuracy, since the class distribution of transcriptions is very skewed towards either class. [sent-385, score-0.137]
87 Although acoustic data were not provided with this data set (Lyon and Dorado, 2008) (and, in practice, cannot be assured), future work should also examine the differences in the acoustics of children across truth conditions. [sent-390, score-0.398]
88 951 Appendix The following is an example of evasive deceptive speech from a 6-year-old after no truth induction (i. [sent-392, score-0.479]
89 Transcription excerpt labeled as truth by a threshold of τ = 1: 7-year-old child’s response (play, no coach condition), in which the child does not explicitly deny playing with the toy house, and admits to looking at it but does not confess to any of the other six actions: . [sent-405, score-0.653]
90 Transcription excerpt labeled as truth by a threshold of τ = 4: 7-year-old child’s response (play, no coach condition), in which the child does not explicitly deny playing, and admits to three actions: . [sent-414, score-0.507]
91 On the use of homogeneous sets of subjects in deceptive language analysis. [sent-447, score-0.264]
92 Combining prosodic lexical and cepstral systems for deceptive speech detection. [sent-451, score-0.344]
93 Linguistic cues to deception assessed by computer programs: A metaanalysis. [sent-456, score-0.662]
94 Truth induction in young maltreated children: the effects of oath-taking and reassurance on true and false disclosures. [sent-502, score-0.16]
95 Coaching, truth induction, and young maltreated children’s false allegations and false denials. [sent-510, score-0.388]
96 The lie detector: explorations in the automatic recognition of deceptive language. [sent-521, score-0.284]
97 A transformational analysis of oral and written grammatical structures in the language of children in grades three, five, and seven. [sent-536, score-0.169]
98 “I don’t know where he is not”: does deception research yet offer a basis for deception detectives? [sent-594, score-1.184]
99 Detecting deceit via analyses of verbal and nonverbal behavior in children and adults. [sent-598, score-0.249]
100 Hedge classification with syntactic dependency features based on an ensemble classifier. [sent-601, score-0.133]
wordName wordTfidf (topN-words)
[('deception', 0.592), ('deceptive', 0.224), ('truth', 0.188), ('children', 0.169), ('age', 0.16), ('clauses', 0.14), ('child', 0.139), ('transcriptions', 0.137), ('mlp', 0.137), ('liwc', 0.108), ('playing', 0.101), ('rf', 0.098), ('nb', 0.098), ('truthful', 0.092), ('confederate', 0.086), ('maltreated', 0.086), ('mcu', 0.086), ('verbose', 0.079), ('passive', 0.078), ('complexity', 0.074), ('newman', 0.072), ('play', 0.071), ('cues', 0.07), ('toronto', 0.069), ('classifiers', 0.069), ('coach', 0.069), ('depaulo', 0.069), ('fmainv', 0.069), ('richards', 0.069), ('truthfulness', 0.069), ('vartapetiance', 0.069), ('speech', 0.067), ('lyon', 0.066), ('coordinate', 0.066), ('asl', 0.063), ('clause', 0.062), ('svm', 0.062), ('deny', 0.061), ('court', 0.06), ('lie', 0.06), ('groups', 0.055), ('lr', 0.053), ('per', 0.053), ('anova', 0.053), ('prosodic', 0.053), ('classifier', 0.052), ('fornaciari', 0.052), ('gillam', 0.052), ('graciarena', 0.052), ('hauch', 0.052), ('mrmr', 0.052), ('predicators', 0.052), ('pspb', 0.052), ('stajner', 0.052), ('utterances', 0.05), ('threshold', 0.05), ('dependent', 0.049), ('engaged', 0.048), ('detection', 0.048), ('syntactic', 0.047), ('emotion', 0.047), ('adult', 0.046), ('transcription', 0.046), ('verbal', 0.046), ('ontario', 0.046), ('personality', 0.046), ('testimony', 0.046), ('toys', 0.046), ('hirschberg', 0.045), ('toy', 0.045), ('features', 0.044), ('house', 0.043), ('condition', 0.042), ('ages', 0.042), ('classification', 0.042), ('acoustic', 0.041), ('false', 0.04), ('pooled', 0.04), ('donnell', 0.04), ('subjects', 0.04), ('lewis', 0.038), ('accuracy', 0.038), ('adults', 0.038), ('pinker', 0.038), ('nominals', 0.038), ('statistically', 0.036), ('unlimited', 0.036), ('abuse', 0.035), ('allegations', 0.034), ('almela', 0.034), ('courtroom', 0.034), ('enos', 0.034), ('fdg', 0.034), ('interviewer', 0.034), ('nondeceptive', 0.034), ('nonverbal', 0.034), ('reassurance', 0.034), ('skillicorn', 0.034), ('tapanainen', 0.034), ('touching', 0.034)]
simIndex simValue paperId paperTitle
same-paper 1 0.9999994 63 acl-2013-Automatic detection of deception in child-produced speech using syntactic complexity features
Author: Maria Yancheva ; Frank Rudzicz
Abstract: It is important that the testimony of children be admissible in court, especially given allegations of abuse. Unfortunately, children can be misled by interrogators or might offer false information, with dire consequences. In this work, we evaluate various parameterizations of five classifiers (including support vector machines, neural networks, and random forests) in deciphering truth from lies given transcripts of interviews with 198 victims of abuse between the ages of 4 and 7. These evaluations are performed using a novel set of syntactic features, including measures of complexity. Our results show that sentence length, the mean number of clauses per utterance, and the StajnerMitkov measure of complexity are highly informative syntactic features, that classification accuracy varies greatly by the age of the speaker, and that accuracy up to 91.7% can be achieved by support vector machines given a sufficient amount of data.
2 0.16979226 350 acl-2013-TopicSpam: a Topic-Model based approach for spam detection
Author: Jiwei Li ; Claire Cardie ; Sujian Li
Abstract: Product reviews are now widely used by individuals and organizations for decision making (Litvin et al., 2008; Jansen, 2010). And because of the profits at stake, people have been known to try to game the system by writing fake reviews to promote target products. As a result, the task of deceptive review detection has been gaining increasing attention. In this paper, we propose a generative LDA-based topic modeling approach for fake review detection. Our model can aptly detect the subtle dif- ferences between deceptive reviews and truthful ones and achieves about 95% accuracy on review spam datasets, outperforming existing baselines by a large margin.
3 0.16150029 107 acl-2013-Deceptive Answer Prediction with User Preference Graph
Author: Fangtao Li ; Yang Gao ; Shuchang Zhou ; Xiance Si ; Decheng Dai
Abstract: In Community question answering (QA) sites, malicious users may provide deceptive answers to promote their products or services. It is important to identify and filter out these deceptive answers. In this paper, we first solve this problem with the traditional supervised learning methods. Two kinds of features, including textual and contextual features, are investigated for this task. We further propose to exploit the user relationships to identify the deceptive answers, based on the hypothesis that similar users will have similar behaviors to post deceptive or authentic answers. To measure the user similarity, we propose a new user preference graph based on the answer preference expressed by users, such as “helpful” voting and “best answer” selection. The user preference graph is incorporated into traditional supervised learning framework with the graph regularization technique. The experiment results demonstrate that the user preference graph can indeed help improve the performance of deceptive answer prediction.
4 0.08658573 379 acl-2013-Utterance-Level Multimodal Sentiment Analysis
Author: Veronica Perez-Rosas ; Rada Mihalcea ; Louis-Philippe Morency
Abstract: During real-life interactions, people are naturally gesturing and modulating their voice to emphasize specific points or to express their emotions. With the recent growth of social websites such as YouTube, Facebook, and Amazon, video reviews are emerging as a new source of multimodal and natural opinions that has been left almost untapped by automatic opinion analysis techniques. This paper presents a method for multimodal sentiment classification, which can identify the sentiment expressed in utterance-level visual datastreams. Using a new multimodal dataset consisting of sentiment annotated utterances extracted from video reviews, we show that multimodal sentiment analysis can be effectively performed, and that the joint use of visual, acoustic, and linguistic modalities can lead to error rate reductions of up to 10.5% as compared to the best performing individual modality.
5 0.079870515 253 acl-2013-Multilingual Affect Polarity and Valence Prediction in Metaphor-Rich Texts
Author: Zornitsa Kozareva
Abstract: Metaphor is an important way of conveying the affect of people, hence understanding how people use metaphors to convey affect is important for the communication between individuals and increases cohesion if the perceived affect of the concrete example is the same for the two individuals. Therefore, building computational models that can automatically identify the affect in metaphor-rich texts like “The team captain is a rock.”, “Time is money.”, “My lawyer is a shark.” is an important challenging problem, which has been of great interest to the research community. To solve this task, we have collected and manually annotated the affect of metaphor-rich texts for four languages. We present novel algorithms that integrate triggers for cognitive, affective, perceptual and social processes with stylistic and lexical information. By running evaluations on datasets in English, Spanish, Russian and Farsi, we show that the developed affect polarity and valence prediction technology of metaphor-rich texts is portable and works equally well for different languages.
6 0.067642704 282 acl-2013-Predicting and Eliciting Addressee's Emotion in Online Dialogue
7 0.056422208 184 acl-2013-Identification of Speakers in Novels
8 0.055886537 298 acl-2013-Recognizing Rare Social Phenomena in Conversation: Empowerment Detection in Support Group Chatrooms
10 0.052385006 351 acl-2013-Topic Modeling Based Classification of Clinical Reports
11 0.050406873 213 acl-2013-Language Acquisition and Probabilistic Models: keeping it simple
12 0.04952411 311 acl-2013-Semantic Neighborhoods as Hypergraphs
13 0.04797323 89 acl-2013-Computerized Analysis of a Verbal Fluency Test
14 0.047914036 78 acl-2013-Categorization of Turkish News Documents with Morphological Analysis
15 0.047372214 267 acl-2013-PARMA: A Predicate Argument Aligner
16 0.046443339 331 acl-2013-Stop-probability estimates computed on a large corpus improve Unsupervised Dependency Parsing
17 0.045788545 342 acl-2013-Text Classification from Positive and Unlabeled Data using Misclassified Data Correction
18 0.045301843 209 acl-2013-Joint Modeling of News Readerâ•Žs and Comment Writerâ•Žs Emotions
19 0.044631548 232 acl-2013-Linguistic Models for Analyzing and Detecting Biased Language
20 0.042946972 322 acl-2013-Simple, readable sub-sentences
topicId topicWeight
[(0, 0.141), (1, 0.05), (2, -0.023), (3, -0.018), (4, -0.028), (5, -0.027), (6, 0.063), (7, -0.051), (8, 0.001), (9, 0.042), (10, -0.012), (11, 0.028), (12, -0.037), (13, 0.026), (14, -0.073), (15, -0.048), (16, -0.043), (17, 0.079), (18, 0.023), (19, -0.017), (20, -0.004), (21, -0.055), (22, 0.07), (23, -0.057), (24, -0.022), (25, 0.064), (26, -0.04), (27, -0.078), (28, 0.009), (29, -0.002), (30, 0.017), (31, -0.073), (32, 0.062), (33, 0.07), (34, 0.065), (35, 0.068), (36, -0.057), (37, 0.036), (38, -0.043), (39, 0.016), (40, 0.085), (41, -0.012), (42, -0.109), (43, 0.184), (44, 0.051), (45, -0.105), (46, -0.057), (47, -0.046), (48, -0.056), (49, -0.008)]
simIndex simValue paperId paperTitle
same-paper 1 0.86563253 63 acl-2013-Automatic detection of deception in child-produced speech using syntactic complexity features
Author: Maria Yancheva ; Frank Rudzicz
Abstract: It is important that the testimony of children be admissible in court, especially given allegations of abuse. Unfortunately, children can be misled by interrogators or might offer false information, with dire consequences. In this work, we evaluate various parameterizations of five classifiers (including support vector machines, neural networks, and random forests) in deciphering truth from lies given transcripts of interviews with 198 victims of abuse between the ages of 4 and 7. These evaluations are performed using a novel set of syntactic features, including measures of complexity. Our results show that sentence length, the mean number of clauses per utterance, and the StajnerMitkov measure of complexity are highly informative syntactic features, that classification accuracy varies greatly by the age of the speaker, and that accuracy up to 91.7% can be achieved by support vector machines given a sufficient amount of data.
2 0.7477622 350 acl-2013-TopicSpam: a Topic-Model based approach for spam detection
Author: Jiwei Li ; Claire Cardie ; Sujian Li
Abstract: Product reviews are now widely used by individuals and organizations for decision making (Litvin et al., 2008; Jansen, 2010). And because of the profits at stake, people have been known to try to game the system by writing fake reviews to promote target products. As a result, the task of deceptive review detection has been gaining increasing attention. In this paper, we propose a generative LDA-based topic modeling approach for fake review detection. Our model can aptly detect the subtle dif- ferences between deceptive reviews and truthful ones and achieves about 95% accuracy on review spam datasets, outperforming existing baselines by a large margin.
3 0.60730469 107 acl-2013-Deceptive Answer Prediction with User Preference Graph
Author: Fangtao Li ; Yang Gao ; Shuchang Zhou ; Xiance Si ; Decheng Dai
Abstract: In Community question answering (QA) sites, malicious users may provide deceptive answers to promote their products or services. It is important to identify and filter out these deceptive answers. In this paper, we first solve this problem with the traditional supervised learning methods. Two kinds of features, including textual and contextual features, are investigated for this task. We further propose to exploit the user relationships to identify the deceptive answers, based on the hypothesis that similar users will have similar behaviors to post deceptive or authentic answers. To measure the user similarity, we propose a new user preference graph based on the answer preference expressed by users, such as “helpful” voting and “best answer” selection. The user preference graph is incorporated into traditional supervised learning framework with the graph regularization technique. The experiment results demonstrate that the user preference graph can indeed help improve the performance of deceptive answer prediction.
4 0.49735218 298 acl-2013-Recognizing Rare Social Phenomena in Conversation: Empowerment Detection in Support Group Chatrooms
Author: Elijah Mayfield ; David Adamson ; Carolyn Penstein Rose
Abstract: Automated annotation of social behavior in conversation is necessary for large-scale analysis of real-world conversational data. Important behavioral categories, though, are often sparse and often appear only in specific subsections of a conversation. This makes supervised machine learning difficult, through a combination of noisy features and unbalanced class distributions. We propose within-instance content selection, using cue features to selectively suppress sections of text and biasing the remaining representation towards minority classes. We show the effectiveness of this technique in automated annotation of empowerment language in online , support group chatrooms. Our technique is significantly more accurate than multiple baselines, especially when prioritizing high precision.
5 0.49715292 81 acl-2013-Co-Regression for Cross-Language Review Rating Prediction
Author: Xiaojun Wan
Abstract: The task of review rating prediction can be well addressed by using regression algorithms if there is a reliable training set of reviews with human ratings. In this paper, we aim to investigate a more challenging task of crosslanguage review rating prediction, which makes use of only rated reviews in a source language (e.g. English) to predict the rating scores of unrated reviews in a target language (e.g. German). We propose a new coregression algorithm to address this task by leveraging unlabeled reviews. Evaluation results on several datasets show that our proposed co-regression algorithm can consistently improve the prediction results. 1
6 0.43778336 184 acl-2013-Identification of Speakers in Novels
7 0.42668739 254 acl-2013-Multimodal DBN for Predicting High-Quality Answers in cQA portals
8 0.4265517 340 acl-2013-Text-Driven Toponym Resolution using Indirect Supervision
10 0.41121233 30 acl-2013-A computational approach to politeness with application to social factors
11 0.40633461 33 acl-2013-A user-centric model of voting intention from Social Media
12 0.39977357 346 acl-2013-The Impact of Topic Bias on Quality Flaw Prediction in Wikipedia
13 0.39388922 278 acl-2013-Patient Experience in Online Support Forums: Modeling Interpersonal Interactions and Medication Use
14 0.39229372 364 acl-2013-Typesetting for Improved Readability using Lexical and Syntactic Information
15 0.38956192 191 acl-2013-Improved Bayesian Logistic Supervised Topic Models with Data Augmentation
16 0.3828114 65 acl-2013-BRAINSUP: Brainstorming Support for Creative Sentence Generation
17 0.38245764 268 acl-2013-PATHS: A System for Accessing Cultural Heritage Collections
18 0.38196892 190 acl-2013-Implicatures and Nested Beliefs in Approximate Decentralized-POMDPs
19 0.37831751 161 acl-2013-Fluid Construction Grammar for Historical and Evolutionary Linguistics
20 0.37824041 90 acl-2013-Conditional Random Fields for Responsive Surface Realisation using Global Features
topicId topicWeight
[(0, 0.045), (6, 0.027), (11, 0.055), (15, 0.036), (24, 0.056), (26, 0.052), (28, 0.012), (29, 0.01), (35, 0.078), (42, 0.056), (48, 0.04), (60, 0.276), (70, 0.056), (82, 0.01), (88, 0.032), (90, 0.04), (95, 0.042)]
simIndex simValue paperId paperTitle
same-paper 1 0.75599545 63 acl-2013-Automatic detection of deception in child-produced speech using syntactic complexity features
Author: Maria Yancheva ; Frank Rudzicz
Abstract: It is important that the testimony of children be admissible in court, especially given allegations of abuse. Unfortunately, children can be misled by interrogators or might offer false information, with dire consequences. In this work, we evaluate various parameterizations of five classifiers (including support vector machines, neural networks, and random forests) in deciphering truth from lies given transcripts of interviews with 198 victims of abuse between the ages of 4 and 7. These evaluations are performed using a novel set of syntactic features, including measures of complexity. Our results show that sentence length, the mean number of clauses per utterance, and the StajnerMitkov measure of complexity are highly informative syntactic features, that classification accuracy varies greatly by the age of the speaker, and that accuracy up to 91.7% can be achieved by support vector machines given a sufficient amount of data.
2 0.62458122 267 acl-2013-PARMA: A Predicate Argument Aligner
Author: Travis Wolfe ; Benjamin Van Durme ; Mark Dredze ; Nicholas Andrews ; Charley Beller ; Chris Callison-Burch ; Jay DeYoung ; Justin Snyder ; Jonathan Weese ; Tan Xu ; Xuchen Yao
Abstract: We introduce PARMA, a system for crossdocument, semantic predicate and argument alignment. Our system combines a number of linguistic resources familiar to researchers in areas such as recognizing textual entailment and question answering, integrating them into a simple discriminative model. PARMA achieves state of the art results on an existing and a new dataset. We suggest that previous efforts have focussed on data that is biased and too easy, and we provide a more difficult dataset based on translation data with a low baseline which we beat by 17% F1.
3 0.60194904 5 acl-2013-A Decade of Automatic Content Evaluation of News Summaries: Reassessing the State of the Art
Author: Peter A. Rankel ; John M. Conroy ; Hoa Trang Dang ; Ani Nenkova
Abstract: How good are automatic content metrics for news summary evaluation? Here we provide a detailed answer to this question, with a particular focus on assessing the ability of automatic evaluations to identify statistically significant differences present in manual evaluation of content. Using four years of data from the Text Analysis Conference, we analyze the performance of eight ROUGE variants in terms of accuracy, precision and recall in finding significantly different systems. Our experiments show that some of the neglected variants of ROUGE, based on higher order n-grams and syntactic dependencies, are most accurate across the years; the commonly used ROUGE-1 scores find too many significant differences between systems which manual evaluation would deem comparable. We also test combinations ofROUGE variants and find that they considerably improve the accuracy of automatic prediction.
4 0.50996375 233 acl-2013-Linking Tweets to News: A Framework to Enrich Short Text Data in Social Media
Author: Weiwei Guo ; Hao Li ; Heng Ji ; Mona Diab
Abstract: Many current Natural Language Processing [NLP] techniques work well assuming a large context of text as input data. However they become ineffective when applied to short texts such as Twitter feeds. To overcome the issue, we want to find a related newswire document to a given tweet to provide contextual support for NLP tasks. This requires robust modeling and understanding of the semantics of short texts. The contribution of the paper is two-fold: 1. we introduce the Linking-Tweets-toNews task as well as a dataset of linked tweet-news pairs, which can benefit many NLP applications; 2. in contrast to previ- ous research which focuses on lexical features within the short texts (text-to-word information), we propose a graph based latent variable model that models the inter short text correlations (text-to-text information). This is motivated by the observation that a tweet usually only covers one aspect of an event. We show that using tweet specific feature (hashtag) and news specific feature (named entities) as well as temporal constraints, we are able to extract text-to-text correlations, and thus completes the semantic picture of a short text. Our experiments show significant improvement of our new model over baselines with three evaluation metrics in the new task.
5 0.50872797 318 acl-2013-Sentiment Relevance
Author: Christian Scheible ; Hinrich Schutze
Abstract: A number of different notions, including subjectivity, have been proposed for distinguishing parts of documents that convey sentiment from those that do not. We propose a new concept, sentiment relevance, to make this distinction and argue that it better reflects the requirements of sentiment analysis systems. We demonstrate experimentally that sentiment relevance and subjectivity are related, but different. Since no large amount of labeled training data for our new notion of sentiment relevance is available, we investigate two semi-supervised methods for creating sentiment relevance classifiers: a distant supervision approach that leverages structured information about the domain of the reviews; and transfer learning on feature representations based on lexical taxonomies that enables knowledge transfer. We show that both methods learn sentiment relevance classifiers that perform well.
6 0.50439686 2 acl-2013-A Bayesian Model for Joint Unsupervised Induction of Sentiment, Aspect and Discourse Representations
7 0.50304902 169 acl-2013-Generating Synthetic Comparable Questions for News Articles
8 0.50104392 82 acl-2013-Co-regularizing character-based and word-based models for semi-supervised Chinese word segmentation
9 0.50100821 225 acl-2013-Learning to Order Natural Language Texts
10 0.50063241 83 acl-2013-Collective Annotation of Linguistic Resources: Basic Principles and a Formal Model
11 0.50042176 224 acl-2013-Learning to Extract International Relations from Political Context
12 0.49922314 275 acl-2013-Parsing with Compositional Vector Grammars
13 0.49897334 85 acl-2013-Combining Intra- and Multi-sentential Rhetorical Parsing for Document-level Discourse Analysis
14 0.49833727 56 acl-2013-Argument Inference from Relevant Event Mentions in Chinese Argument Extraction
15 0.49795547 285 acl-2013-Propminer: A Workflow for Interactive Information Extraction and Exploration using Dependency Trees
16 0.49675933 172 acl-2013-Graph-based Local Coherence Modeling
17 0.49675122 250 acl-2013-Models of Translation Competitions
18 0.49674881 17 acl-2013-A Random Walk Approach to Selectional Preferences Based on Preference Ranking and Propagation
20 0.49607709 46 acl-2013-An Infinite Hierarchical Bayesian Model of Phrasal Translation