emnlp emnlp2012 emnlp2012-21 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Su-Youn Yoon ; Suma Bhat
Abstract: This study presents a novel method that measures English language learners’ syntactic competence towards improving automated speech scoring systems. In contrast to most previous studies which focus on the length of production units such as the mean length of clauses, we focused on capturing the differences in the distribution of morpho-syntactic features or grammatical expressions across proficiency. We estimated the syntactic competence through the use of corpus-based NLP techniques. Assuming that the range and so- phistication of grammatical expressions can be captured by the distribution of Part-ofSpeech (POS) tags, vector space models of POS tags were constructed. We use a large corpus of English learners’ responses that are classified into four proficiency levels by human raters. Our proposed feature measures the similarity of a given response with the most proficient group and is then estimates the learner’s syntactic competence level. Widely outperforming the state-of-the-art measures of syntactic complexity, our method attained a significant correlation with humanrated scores. The correlation between humanrated scores and features based on manual transcription was 0.43 and the same based on ASR-hypothesis was slightly lower, 0.42. An important advantage of our method is its robustness against speech recognition errors not to mention the simplicity of feature generation that captures a reasonable set of learnerspecific syntactic errors. 600 Measures Suma Bhat Beckman Institute, Urbana, IL 61801 . spbhat 2 @ i l l ino i edu s
Reference: text
sentIndex sentText sentNum sentScore
1 org Abstract This study presents a novel method that measures English language learners’ syntactic competence towards improving automated speech scoring systems. [sent-2, score-0.821]
2 In contrast to most previous studies which focus on the length of production units such as the mean length of clauses, we focused on capturing the differences in the distribution of morpho-syntactic features or grammatical expressions across proficiency. [sent-3, score-0.327]
3 We estimated the syntactic competence through the use of corpus-based NLP techniques. [sent-4, score-0.385]
4 Assuming that the range and so- phistication of grammatical expressions can be captured by the distribution of Part-ofSpeech (POS) tags, vector space models of POS tags were constructed. [sent-5, score-0.367]
5 We use a large corpus of English learners’ responses that are classified into four proficiency levels by human raters. [sent-6, score-0.67]
6 Our proposed feature measures the similarity of a given response with the most proficient group and is then estimates the learner’s syntactic competence level. [sent-7, score-0.825]
7 Widely outperforming the state-of-the-art measures of syntactic complexity, our method attained a significant correlation with humanrated scores. [sent-8, score-0.3]
8 An important advantage of our method is its robustness against speech recognition errors not to mention the simplicity of feature generation that captures a reasonable set of learnerspecific syntactic errors. [sent-12, score-0.331]
9 spbhat 2 @ i l l ino i edu s 1 Introduction This study provides a novel method that measures ESL (English as a second language) learners’ competence in grammar usage (syntactic competence). [sent-14, score-0.455]
10 Grammar usage is one of the dimensions of language ability that is assessed during non-native proficiency level testing in a foreign language. [sent-16, score-0.451]
11 Overall proficiency in the target language can be assessed by testing the abilities in various areas including fluency, pronunciation, and intonation; grammar and vocabulary; and discourse structure. [sent-17, score-0.453]
12 ” Features that measure syntactic complexity have been frequently studied in ESL literature and have been found to be highly correlated with students’ proficiency levels in writing. [sent-21, score-0.618]
13 Studies in automated speech scoring have focused on fluency (Cucchiarini et al. [sent-22, score-0.427]
14 More recently, Lu (2010), Chen and Yoon (201 1) and Chen and Zechner (201 1) have measured syntactic competence in speech scoring. [sent-30, score-0.515]
15 However, these studies found that these measures did not show satisfactory empirical performance in automatic speech scoring (Chen and Yoon, 2011; Chen and Zechner, 2011) when the features were calculated from the output of a speech recognition engine. [sent-33, score-0.55]
16 This study considers new features that measure syntactic complexity and is novel in two important ways. [sent-34, score-0.212]
17 First, in contrast to most features that infer syntactic complexity based upon the length of the unit, we directly measure students’ sophistication and range in grammar usage. [sent-35, score-0.337]
18 Second, instead of rating a student’s response using a scale based on native speech production, our experiments compare it with a similar body of learners’ speech. [sent-36, score-0.3]
19 Eliciting native speakers’ data and rating it for grammar usage (supervised approach) can be arbitrary, since there can be a very wide range of possible grammatical structures that native speakers utilize. [sent-37, score-0.378]
20 A large amount oflearners’ spoken responses were collected and classified into four groups according to their proficiency level. [sent-39, score-0.773]
21 We then sought to find how distinct the proficiency classes were based on the distribution of POS tags. [sent-40, score-0.441]
22 Given a student’s response, we calculated the similarity with a sample of responses for each score level based on the proportion and distribution of Part-of-Speech using NLP techniques. [sent-41, score-0.314]
23 (201 1) explored POS tag distribution to capture the differences in syntactic complexity between healthy subjects and subjects with mild cognitive impairment, 601 but no other research has used POS tag distribution in measuring syntactic complexity, to the best of au- thors’ knowledge. [sent-45, score-0.628]
24 An assessment of ESL learners’ syntactic competence should consider the structure of sentences as a whole - a task which may not be captured by the simplistic POS tag distribution. [sent-46, score-0.616]
25 However, studies of Lu (2010) and Chen and Zechner (201 1) showed that more complex syntactic features are unreliable in ASR-based scoring system. [sent-47, score-0.224]
26 Furthermore, we show that POS unigrams or bigrams indeed capture a reasonable portion of learners’ range and sophistication of grammar usage in our discussion in Section 7. [sent-48, score-0.22]
27 This paper will proceed as follows: we will review related work in Section 2 and present the method to calculate syntactic complexity in Section 3. [sent-49, score-0.212]
28 Finally, in Section 7, we discuss the levels of syntactic competence that are captured using our proposed measure. [sent-52, score-0.385]
29 2 Related Work Second Language Acquisition (SLA) researchers have developed many quantitative measures to es- timate the level of acquisition of syntactic competence. [sent-53, score-0.237]
30 The first group is related to the acquisition of specific morphosyntactic features or grammatical expressions. [sent-55, score-0.326]
31 Tests of negations or relative clauses - whether these expressions occurred in the test responses without errors fell into this group (hereafter, the expression-based group). [sent-56, score-0.521]
32 The second group is related to the length of the clause or the relationship between clauses and hence not tied to particular structures (hereafter, the length-based group). [sent-57, score-0.278]
33 Examples of the second group measures include the average length of clause unit and dependent clauses per sentence unit. [sent-58, score-0.364]
34 Ortega (2003) synthesized 25 research studies which employed syntactic measures on ESL writing and reported a significant relationship between the proposed features and writing proficiency. [sent-60, score-0.33]
35 He applied 14 syntactic measures to a large database of Chinese learners’ writing samples and found that syntactic measures were strong predictors of students’ writing proficiency. [sent-63, score-0.478]
36 Studies in the area of automated speech scoring have only recently begun to actively investigate the usefulness of syntactic measures for scoring spontaneous speech (Chen et al. [sent-64, score-0.834]
37 In addition to these conventional syntactic complexity features, Lu (2009) implemented an automated system that calculates the revised Developmental Level (D-Level) Scale (Covington et al. [sent-68, score-0.362]
38 They classified children’s grammatical acquisition into 7 different groups according to the presence of certain types of complex sentences. [sent-71, score-0.263]
39 In contrast to ESL writing, applying syntactic complexity features, both conventional length-based features and D-Level features, presents serious obstacles for speaking. [sent-76, score-0.212]
40 First, the length of the spoken responses are typically shorter than written responses. [sent-77, score-0.263]
41 Chen and Yoon (201 1) observed a marked decrease in correlation between syntactic measures and proficiency as response length decreased. [sent-79, score-0.835]
42 In addition, speech recognition errors only worsen the situation. [sent-80, score-0.223]
43 Chen and Zechner (201 1) showed that the significant correlation between syntactic measures and speech proficiency (correlation coefficient 602 = 0. [sent-81, score-0.796]
44 Errors in speech recognition seriously influenced the measures and decreased the performance. [sent-83, score-0.268]
45 Due to these problems, the existing syntactic measures do not seem reliable enough for being used in automated speech proficiency scoring. [sent-84, score-0.88]
46 In this study, we propose novel syntactic measures which are relatively robust against speech recognition errors and are reliable in short responses. [sent-85, score-0.417]
47 In contrast to recent studies focusing on length-based features, we focus on capturing differences in the distribution of morphosyntactic features or grammatical expressions across proficiency levels. [sent-86, score-0.745]
48 3 Method Many previous studies, that assess syntactic complexity based on the distribution of morphosyntactic features and grammatical expressions, limited their experiments to a few grammatical expressions. [sent-88, score-0.531]
49 We hypothesize that the level of acquired grammatical forms is signaled by the distribution of the POS tags, and the differences in grammatical proficiency result in differences in POS tag distribution. [sent-93, score-0.829]
50 Based on this assumption, we collected large amount of ESL learners’ spoken responses and classified them into four groups according to their proficiency levels. [sent-94, score-0.773]
51 The syntactic competence was estimated based on the similarity between the test responses and learners’ corpus. [sent-95, score-0.628]
52 A POS-based vector space model (VSM), in which the response belonging to separate proficiency levels were converted to vectors and the similarity between vectors were calculated using cosine similarity measure and tf-idf weighting, was em- ployed. [sent-96, score-0.651]
53 Such a score-category-based VSM has been used in automated essay scoring. [sent-97, score-0.237]
54 Proficient speakers use complicated grammatical expressions, while beginners use simple expressions and sentences with frequent grammatical errors. [sent-100, score-0.457]
55 POS tags (or sequences) capturing these expressions may be seen in corresponding proportions in each score group. [sent-101, score-0.216]
56 Temple (2000) pointed out that the proficient learners are characterized by increased automaticity in speech production. [sent-104, score-0.415]
57 We then compare the performance of our features with those in Lu (201 1), where the features are a collection of measures of syntactic complexity that have shown promising directions in previous studies. [sent-108, score-0.298]
58 Both were collections of responses from the AEST, a high-stakes test of English proficiency and had no overlaps. [sent-110, score-0.61]
59 The AEST assessment consists of 6 items in which speakers are prompted to provide responses lasting between 45 and 60 seconds per item. [sent-111, score-0.408]
60 Each response was rated by trained human raters using a 4-point scoring scale, where 1 indicates a low speaking proficiency and 4 indicates a high speaking proficiency. [sent-117, score-0.607]
61 2) were used in order to investigate the influence of speech recognition errors in the feature performance. [sent-132, score-0.223]
62 As a result, we generated three sets of POS units by this process: the original POS set without the compound unit (Base), the original set and an additional 50 compound units (Base+mi50), and the original set and an additional 110 units (Base+mi1 10). [sent-148, score-0.3]
63 Among these four values, the cos4, the similarity score to the responses in the score group 4, was selected as a feature with the following intuition. [sent-178, score-0.358]
64 cos4 measures the similarity of a given test response to the representative vector of score class 4; the larger the value, the closer it would be to score class 4. [sent-179, score-0.256]
65 1 Correlation Table 4 shows correlations between cosine similarity features and proficiency scores rated by experts. [sent-181, score-0.48]
66 By adding the mutual information-based compound units to the original POS tag sets, the performance of features improved in the unigram models. [sent-184, score-0.288]
67 Towards this, the clause boundaries of the ASR hypotheses, were automatically detected using the automated clause boundary detection method1 . [sent-194, score-0.334]
68 1The automated clause boundary detection method in this study was a Maximum Entropy Model based on word bigrams, POS tag bigrams, and pause features. [sent-195, score-0.361]
69 Finally, we calculated Pearson correlation coefficients between these features and human proficiency scores. [sent-197, score-0.508]
70 01 level As indicated in Table 5, the best performing feature was mean number of dependent clauses per clause (DCC) and the correlation r was 0. [sent-200, score-0.229]
71 For instance, the errors in the automated clause boundary detection may result in a serious drop in the performance. [sent-206, score-0.283]
72 With the spoken responses being particularly short (a typical response in the data set had 10 clauses on average), even one error in clause boundary detection can seriously affect the reliability of features. [sent-207, score-0.557]
73 Furthermore, this section will show that bigram POS sequences can yield significant information on the range and sophistication of grammar usage in the specific assessment context (spon- F-score of 0. [sent-209, score-0.282]
74 ESL speakers with high proficiency scores are expected to use more complicated grammatical expressions that result in a high proportion of POS tags related to these expressions in that score group. [sent-212, score-0.963]
75 The distribution of POS tags was analyzed in detail in order to investigate whether there were systematic distributional changes according to proficiency levels. [sent-213, score-0.567]
76 A1B4SIN37C3D3EC1C8ON4M8ix Table 6: Tag distribution and proficiency scores The ‘ABS’ class mostly consists of ‘WP’ and ‘WDT’ ; more than 50% of tags in this class are related to these two tags. [sent-219, score-0.567]
77 Therefore, the lack of these tags strongly support the hypothesis that the speakers in score group 1 showed incompetence in the use of relative clauses or their use in limited situations. [sent-222, score-0.404]
78 Finally, the relative clause group signals the presence of relative clauses. [sent-226, score-0.25]
79 The increased proportion of these tags reflects the use of more complicated tense forms and modal forms as well as more frequent use of relative clauses. [sent-227, score-0.243]
80 It supports the hypothesis that speakers with higher proficiency scores tend to use more complicated grammatical expressions. [sent-228, score-0.657]
81 The noun group is comprised of many noun or proper noun-related expressions, and their high proportions are consistent with the tendency that less proficient speakers use nouns more frequently. [sent-230, score-0.337]
82 The ‘UH’ tag is for interjection and filler words such as ‘uh’ and ‘um’, while the ‘GW’ tag is for word-fragments. [sent-233, score-0.238]
83 These two spontaneous speech phenomena are strongly related to fluency, and it signals problems in speech production. [sent-234, score-0.393]
84 Frequent occurrences of these two tags are evidence of frequent planning problems and their inclusion in the ‘DEC’ class suggests that instances of speech planning problems decrease with increased proficiency. [sent-235, score-0.294]
85 The non-compound tags are associated with the expressions that do not co-occur with strongly related words, and they tend to be related to errors. [sent-237, score-0.216]
86 For instance, the non-compound ‘MD’ tag signals that there is an expression that a modal verb is not followed by ‘VB’ (base form) and as seen in the ex607 amples, ‘the project may can change’ and ‘the others must can not be good’, they are related to grammatical errors. [sent-238, score-0.278]
87 ‘RBR’ is for comparative adverbs and ‘JJR’ is for comparative adjectives, and the combination of two tags is strongly related to double-marked errors such as ‘more easier’ . [sent-241, score-0.301]
88 In the intermediate stage in the acquisition of comparative form, learners tend to use the double-marked form. [sent-242, score-0.286]
89 The above analysis shows that the combination of original and compound POS tags correctly capture systematic changes in the grammatical expressions according to changes in proficiency levels. [sent-245, score-0.828]
90 The robust performance of our proposed measure to speech recognition errors may be better appreciated in the context of similar studies. [sent-246, score-0.223]
91 Compared with the state-of-the art measures of syntactic complexity proposed in Lu (201 1) our features achieve significantly better performance especially when generated from ASR hypotheses. [sent-247, score-0.298]
92 8 Conclusions In this paper, we presented features that measure syntactic competence for the automated speech scoring. [sent-249, score-0.665]
93 The features measured the range and sophistication of grammatical expressions based on POS tag distributions. [sent-250, score-0.403]
94 A corpus with a large number of learners’ responses was collected and classified into four groups according to proficiency levels. [sent-251, score-0.714]
95 The syntactic competence of the test response was estimated by identifying the most similar group from the learners’ corpus. [sent-252, score-0.631]
96 Furthermore, speech recognition errors only resulted in a minor performance drop. [sent-253, score-0.223]
97 The robustness against speech recognition errors is an important advantage of our method. [sent-254, score-0.223]
98 Computing and evaluating syntactic complexity features for automated scoring of spontaneous non-native speech. [sent-275, score-0.522]
99 Indicators of linguistic competence in the peer group conversational behavior ofmildly retarded adults. [sent-344, score-0.392]
100 Automatic scoring of non-native spontaneous speech in tests of spoken english. [sent-367, score-0.349]
wordName wordTfidf (topN-words)
[('proficiency', 0.406), ('competence', 0.277), ('responses', 0.204), ('aest', 0.201), ('zechner', 0.201), ('learners', 0.176), ('pos', 0.175), ('esl', 0.174), ('automated', 0.15), ('response', 0.131), ('speech', 0.13), ('tags', 0.126), ('tag', 0.119), ('grammatical', 0.116), ('group', 0.115), ('assessment', 0.112), ('syntactic', 0.108), ('yoon', 0.104), ('complexity', 0.104), ('lu', 0.103), ('clause', 0.092), ('speakers', 0.092), ('compound', 0.09), ('expressions', 0.09), ('spontaneous', 0.09), ('essay', 0.087), ('measures', 0.086), ('cucchiarini', 0.081), ('wdt', 0.081), ('sophistication', 0.078), ('fluency', 0.077), ('clauses', 0.071), ('scoring', 0.07), ('proficient', 0.069), ('vbn', 0.068), ('chen', 0.068), ('comparative', 0.067), ('correlation', 0.066), ('comprised', 0.061), ('franco', 0.06), ('neumeyer', 0.06), ('vsms', 0.06), ('witt', 0.06), ('xiaofei', 0.06), ('classified', 0.06), ('spoken', 0.059), ('pronunciation', 0.058), ('asr', 0.056), ('recognition', 0.052), ('chodorow', 0.052), ('tetreault', 0.052), ('covington', 0.052), ('morphosyntactic', 0.052), ('bigrams', 0.05), ('students', 0.049), ('rosenberg', 0.047), ('jjr', 0.047), ('dec', 0.047), ('vsm', 0.047), ('grammar', 0.047), ('vb', 0.047), ('studies', 0.046), ('writing', 0.045), ('usage', 0.045), ('groups', 0.044), ('rbr', 0.043), ('signals', 0.043), ('complicated', 0.043), ('acquisition', 0.043), ('errors', 0.041), ('uh', 0.041), ('abbeduto', 0.04), ('attali', 0.04), ('automaticity', 0.04), ('catia', 0.04), ('dcc', 0.04), ('helmer', 0.04), ('humanrated', 0.04), ('intonation', 0.04), ('oisnecreases', 0.04), ('ortega', 0.04), ('rubrics', 0.04), ('scoregroupj', 0.04), ('silke', 0.04), ('strik', 0.04), ('units', 0.04), ('mutual', 0.039), ('similarity', 0.039), ('native', 0.039), ('decrease', 0.038), ('forms', 0.037), ('ti', 0.037), ('vbg', 0.037), ('klaus', 0.037), ('calculated', 0.036), ('balanced', 0.035), ('correlations', 0.035), ('distribution', 0.035), ('vbd', 0.035), ('tf', 0.035)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000007 21 emnlp-2012-Assessment of ESL Learners' Syntactic Competence Based on Similarity Measures
Author: Su-Youn Yoon ; Suma Bhat
Abstract: This study presents a novel method that measures English language learners’ syntactic competence towards improving automated speech scoring systems. In contrast to most previous studies which focus on the length of production units such as the mean length of clauses, we focused on capturing the differences in the distribution of morpho-syntactic features or grammatical expressions across proficiency. We estimated the syntactic competence through the use of corpus-based NLP techniques. Assuming that the range and so- phistication of grammatical expressions can be captured by the distribution of Part-ofSpeech (POS) tags, vector space models of POS tags were constructed. We use a large corpus of English learners’ responses that are classified into four proficiency levels by human raters. Our proposed feature measures the similarity of a given response with the most proficient group and is then estimates the learner’s syntactic competence level. Widely outperforming the state-of-the-art measures of syntactic complexity, our method attained a significant correlation with humanrated scores. The correlation between humanrated scores and features based on manual transcription was 0.43 and the same based on ASR-hypothesis was slightly lower, 0.42. An important advantage of our method is its robustness against speech recognition errors not to mention the simplicity of feature generation that captures a reasonable set of learnerspecific syntactic errors. 600 Measures Suma Bhat Beckman Institute, Urbana, IL 61801 . spbhat 2 @ i l l ino i edu s
Author: Dan Garrette ; Jason Baldridge
Abstract: Past work on learning part-of-speech taggers from tag dictionaries and raw data has reported good results, but the assumptions made about those dictionaries are often unrealistic: due to historical precedents, they assume access to information about labels in the raw and test sets. Here, we demonstrate ways to learn hidden Markov model taggers from incomplete tag dictionaries. Taking the MINGREEDY algorithm (Ravi et al., 2010) as a starting point, we improve it with several intuitive heuristics. We also define a simple HMM emission initialization that takes advantage of the tag dictionary and raw data to capture both the openness of a given tag and its estimated prevalence in the raw data. Altogether, our augmentations produce improvements to per- formance over the original MIN-GREEDY algorithm for both English and Italian data.
3 0.11225528 114 emnlp-2012-Revisiting the Predictability of Language: Response Completion in Social Media
Author: Bo Pang ; Sujith Ravi
Abstract: The question “how predictable is English?” has long fascinated researchers. While prior work has focused on formal English typically used in news articles, we turn to texts generated by users in online settings that are more informal in nature. We are motivated by a novel application scenario: given the difficulty of typing on mobile devices, can we help reduce typing effort with message completion, especially in conversational settings? We propose a method for automatic response completion. Our approach models both the language used in responses and the specific context provided by the original message. Our experimental results on a large-scale dataset show that both components help reduce typing effort. We also perform an information-theoretic study in this setting and examine the entropy of user-generated content, especially in con- versational scenarios, to better understand predictability of user generated English.
4 0.11106859 70 emnlp-2012-Joint Chinese Word Segmentation, POS Tagging and Parsing
Author: Xian Qian ; Yang Liu
Abstract: In this paper, we propose a novel decoding algorithm for discriminative joint Chinese word segmentation, part-of-speech (POS) tagging, and parsing. Previous work often used a pipeline method Chinese word segmentation followed by POS tagging and parsing, which suffers from error propagation and is unable to leverage information in later modules for earlier components. In our approach, we train the three individual models separately during training, and incorporate them together in a unified framework during decoding. We extend the CYK parsing algorithm so that it can deal with word segmentation and POS tagging features. As far as we know, this is the first work on joint Chinese word segmentation, POS tagging and parsing. Our experimental results on Chinese Tree Bank 5 corpus show that our approach outperforms the state-of-the-art pipeline system. –
5 0.10055471 106 emnlp-2012-Part-of-Speech Tagging for Chinese-English Mixed Texts with Dynamic Features
Author: Jiayi Zhao ; Xipeng Qiu ; Shu Zhang ; Feng Ji ; Xuanjing Huang
Abstract: In modern Chinese articles or conversations, it is very popular to involve a few English words, especially in emails and Internet literature. Therefore, it becomes an important and challenging topic to analyze Chinese-English mixed texts. The underlying problem is how to tag part-of-speech (POS) for the English words involved. Due to the lack of specially annotated corpus, most of the English words are tagged as the oversimplified type, “foreign words”. In this paper, we present a method using dynamic features to tag POS of mixed texts. Experiments show that our method achieves higher performance than traditional sequence labeling methods. Meanwhile, our method also boosts the performance of POS tagging for pure Chinese texts.
6 0.089643531 81 emnlp-2012-Learning to Map into a Universal POS Tagset
7 0.070854627 12 emnlp-2012-A Transition-Based System for Joint Part-of-Speech Tagging and Labeled Non-Projective Dependency Parsing
8 0.06777551 2 emnlp-2012-A Beam-Search Decoder for Grammatical Error Correction
9 0.06765639 17 emnlp-2012-An "AI readability" Formula for French as a Foreign Language
10 0.066375233 64 emnlp-2012-Improved Parsing and POS Tagging Using Inter-Sentence Consistency Constraints
11 0.065392807 48 emnlp-2012-Exploring Adaptor Grammars for Native Language Identification
12 0.064903095 138 emnlp-2012-Wiki-ly Supervised Part-of-Speech Tagging
13 0.064612143 80 emnlp-2012-Learning Verb Inference Rules from Linguistically-Motivated Evidence
14 0.063222714 29 emnlp-2012-Concurrent Acquisition of Word Meaning and Lexical Categories
15 0.060378056 31 emnlp-2012-Cross-Lingual Language Modeling with Syntactic Reordering for Low-Resource Speech Recognition
16 0.057893936 24 emnlp-2012-Biased Representation Learning for Domain Adaptation
17 0.05721733 79 emnlp-2012-Learning Syntactic Categories Using Paradigmatic Representations of Word Context
18 0.055779286 95 emnlp-2012-N-gram-based Tense Models for Statistical Machine Translation
19 0.054459658 3 emnlp-2012-A Coherence Model Based on Syntactic Patterns
20 0.053416539 131 emnlp-2012-Unified Dependency Parsing of Chinese Morphological and Syntactic Structures
topicId topicWeight
[(0, 0.205), (1, -0.062), (2, 0.09), (3, 0.037), (4, 0.045), (5, 0.069), (6, 0.056), (7, -0.074), (8, 0.02), (9, -0.14), (10, 0.002), (11, -0.078), (12, -0.164), (13, -0.007), (14, -0.073), (15, -0.152), (16, -0.073), (17, -0.034), (18, -0.004), (19, 0.022), (20, 0.107), (21, -0.018), (22, 0.073), (23, -0.012), (24, -0.119), (25, 0.208), (26, 0.109), (27, -0.074), (28, 0.029), (29, -0.119), (30, -0.085), (31, -0.117), (32, 0.056), (33, 0.109), (34, -0.082), (35, 0.001), (36, -0.0), (37, -0.137), (38, 0.084), (39, 0.192), (40, -0.113), (41, 0.083), (42, 0.038), (43, 0.029), (44, 0.168), (45, -0.015), (46, -0.068), (47, 0.073), (48, -0.063), (49, 0.01)]
simIndex simValue paperId paperTitle
same-paper 1 0.95344603 21 emnlp-2012-Assessment of ESL Learners' Syntactic Competence Based on Similarity Measures
Author: Su-Youn Yoon ; Suma Bhat
Abstract: This study presents a novel method that measures English language learners’ syntactic competence towards improving automated speech scoring systems. In contrast to most previous studies which focus on the length of production units such as the mean length of clauses, we focused on capturing the differences in the distribution of morpho-syntactic features or grammatical expressions across proficiency. We estimated the syntactic competence through the use of corpus-based NLP techniques. Assuming that the range and so- phistication of grammatical expressions can be captured by the distribution of Part-ofSpeech (POS) tags, vector space models of POS tags were constructed. We use a large corpus of English learners’ responses that are classified into four proficiency levels by human raters. Our proposed feature measures the similarity of a given response with the most proficient group and is then estimates the learner’s syntactic competence level. Widely outperforming the state-of-the-art measures of syntactic complexity, our method attained a significant correlation with humanrated scores. The correlation between humanrated scores and features based on manual transcription was 0.43 and the same based on ASR-hypothesis was slightly lower, 0.42. An important advantage of our method is its robustness against speech recognition errors not to mention the simplicity of feature generation that captures a reasonable set of learnerspecific syntactic errors. 600 Measures Suma Bhat Beckman Institute, Urbana, IL 61801 . spbhat 2 @ i l l ino i edu s
2 0.59886622 17 emnlp-2012-An "AI readability" Formula for French as a Foreign Language
Author: Thomas Francois ; Cedrick Fairon
Abstract: This paper present a new readability formula for French as a foreign language (FFL), which relies on 46 textual features representative of the lexical, syntactic, and semantic levels as well as some of the specificities of the FFL context. We report comparisons between several techniques for feature selection and various learning algorithms. Our best model, based on support vector machines (SVM), significantly outperforms previous FFL formulas. We also found that semantic features behave poorly in our case, in contrast with some previous readability studies on English as a first language.
3 0.57666862 114 emnlp-2012-Revisiting the Predictability of Language: Response Completion in Social Media
Author: Bo Pang ; Sujith Ravi
Abstract: The question “how predictable is English?” has long fascinated researchers. While prior work has focused on formal English typically used in news articles, we turn to texts generated by users in online settings that are more informal in nature. We are motivated by a novel application scenario: given the difficulty of typing on mobile devices, can we help reduce typing effort with message completion, especially in conversational settings? We propose a method for automatic response completion. Our approach models both the language used in responses and the specific context provided by the original message. Our experimental results on a large-scale dataset show that both components help reduce typing effort. We also perform an information-theoretic study in this setting and examine the entropy of user-generated content, especially in con- versational scenarios, to better understand predictability of user generated English.
4 0.43765017 129 emnlp-2012-Type-Supervised Hidden Markov Models for Part-of-Speech Tagging with Incomplete Tag Dictionaries
Author: Dan Garrette ; Jason Baldridge
Abstract: Past work on learning part-of-speech taggers from tag dictionaries and raw data has reported good results, but the assumptions made about those dictionaries are often unrealistic: due to historical precedents, they assume access to information about labels in the raw and test sets. Here, we demonstrate ways to learn hidden Markov model taggers from incomplete tag dictionaries. Taking the MINGREEDY algorithm (Ravi et al., 2010) as a starting point, we improve it with several intuitive heuristics. We also define a simple HMM emission initialization that takes advantage of the tag dictionary and raw data to capture both the openness of a given tag and its estimated prevalence in the raw data. Altogether, our augmentations produce improvements to per- formance over the original MIN-GREEDY algorithm for both English and Italian data.
5 0.43618926 106 emnlp-2012-Part-of-Speech Tagging for Chinese-English Mixed Texts with Dynamic Features
Author: Jiayi Zhao ; Xipeng Qiu ; Shu Zhang ; Feng Ji ; Xuanjing Huang
Abstract: In modern Chinese articles or conversations, it is very popular to involve a few English words, especially in emails and Internet literature. Therefore, it becomes an important and challenging topic to analyze Chinese-English mixed texts. The underlying problem is how to tag part-of-speech (POS) for the English words involved. Due to the lack of specially annotated corpus, most of the English words are tagged as the oversimplified type, “foreign words”. In this paper, we present a method using dynamic features to tag POS of mixed texts. Experiments show that our method achieves higher performance than traditional sequence labeling methods. Meanwhile, our method also boosts the performance of POS tagging for pure Chinese texts.
6 0.42911145 122 emnlp-2012-Syntactic Surprisal Affects Spoken Word Duration in Conversational Contexts
7 0.42360991 48 emnlp-2012-Exploring Adaptor Grammars for Native Language Identification
8 0.36858922 138 emnlp-2012-Wiki-ly Supervised Part-of-Speech Tagging
9 0.35753909 81 emnlp-2012-Learning to Map into a Universal POS Tagset
10 0.3431831 70 emnlp-2012-Joint Chinese Word Segmentation, POS Tagging and Parsing
11 0.3211613 95 emnlp-2012-N-gram-based Tense Models for Statistical Machine Translation
12 0.31424114 83 emnlp-2012-Lexical Differences in Autobiographical Narratives from Schizophrenic Patients and Healthy Controls
13 0.29918346 64 emnlp-2012-Improved Parsing and POS Tagging Using Inter-Sentence Consistency Constraints
14 0.29023746 79 emnlp-2012-Learning Syntactic Categories Using Paradigmatic Representations of Word Context
15 0.28616843 45 emnlp-2012-Exploiting Chunk-level Features to Improve Phrase Chunking
16 0.27448374 24 emnlp-2012-Biased Representation Learning for Domain Adaptation
17 0.27184099 27 emnlp-2012-Characterizing Stylistic Elements in Syntactic Structure
18 0.27126274 125 emnlp-2012-Towards Efficient Named-Entity Rule Induction for Customizability
19 0.26432988 29 emnlp-2012-Concurrent Acquisition of Word Meaning and Lexical Categories
20 0.26046908 88 emnlp-2012-Minimal Dependency Length in Realization Ranking
topicId topicWeight
[(2, 0.01), (16, 0.024), (25, 0.011), (34, 0.032), (60, 0.054), (63, 0.047), (64, 0.028), (65, 0.014), (70, 0.011), (74, 0.626), (76, 0.026), (80, 0.011), (86, 0.015), (95, 0.016)]
simIndex simValue paperId paperTitle
same-paper 1 0.96923774 21 emnlp-2012-Assessment of ESL Learners' Syntactic Competence Based on Similarity Measures
Author: Su-Youn Yoon ; Suma Bhat
Abstract: This study presents a novel method that measures English language learners’ syntactic competence towards improving automated speech scoring systems. In contrast to most previous studies which focus on the length of production units such as the mean length of clauses, we focused on capturing the differences in the distribution of morpho-syntactic features or grammatical expressions across proficiency. We estimated the syntactic competence through the use of corpus-based NLP techniques. Assuming that the range and so- phistication of grammatical expressions can be captured by the distribution of Part-ofSpeech (POS) tags, vector space models of POS tags were constructed. We use a large corpus of English learners’ responses that are classified into four proficiency levels by human raters. Our proposed feature measures the similarity of a given response with the most proficient group and is then estimates the learner’s syntactic competence level. Widely outperforming the state-of-the-art measures of syntactic complexity, our method attained a significant correlation with humanrated scores. The correlation between humanrated scores and features based on manual transcription was 0.43 and the same based on ASR-hypothesis was slightly lower, 0.42. An important advantage of our method is its robustness against speech recognition errors not to mention the simplicity of feature generation that captures a reasonable set of learnerspecific syntactic errors. 600 Measures Suma Bhat Beckman Institute, Urbana, IL 61801 . spbhat 2 @ i l l ino i edu s
2 0.88234258 1 emnlp-2012-A Bayesian Model for Learning SCFGs with Discontiguous Rules
Author: Abby Levenberg ; Chris Dyer ; Phil Blunsom
Abstract: We describe a nonparametric model and corresponding inference algorithm for learning Synchronous Context Free Grammar derivations for parallel text. The model employs a Pitman-Yor Process prior which uses a novel base distribution over synchronous grammar rules. Through both synthetic grammar induction and statistical machine translation experiments, we show that our model learns complex translational correspondences— including discontiguous, many-to-many alignments—and produces competitive translation results. Further, inference is efficient and we present results on significantly larger corpora than prior work.
3 0.84033877 7 emnlp-2012-A Novel Discriminative Framework for Sentence-Level Discourse Analysis
Author: Shafiq Joty ; Giuseppe Carenini ; Raymond Ng
Abstract: We propose a complete probabilistic discriminative framework for performing sentencelevel discourse analysis. Our framework comprises a discourse segmenter, based on a binary classifier, and a discourse parser, which applies an optimal CKY-like parsing algorithm to probabilities inferred from a Dynamic Conditional Random Field. We show on two corpora that our approach outperforms the state-of-the-art, often by a wide margin.
4 0.63265562 130 emnlp-2012-Unambiguity Regularization for Unsupervised Learning of Probabilistic Grammars
Author: Kewei Tu ; Vasant Honavar
Abstract: We introduce a novel approach named unambiguity regularization for unsupervised learning of probabilistic natural language grammars. The approach is based on the observation that natural language is remarkably unambiguous in the sense that only a tiny portion of the large number of possible parses of a natural language sentence are syntactically valid. We incorporate an inductive bias into grammar learning in favor of grammars that lead to unambiguous parses on natural language sentences. The resulting family of algorithms includes the expectation-maximization algorithm (EM) and its variant, Viterbi EM, as well as a so-called softmax-EM algorithm. The softmax-EM algorithm can be implemented with a simple and computationally efficient extension to standard EM. In our experiments of unsupervised dependency grammar learn- ing, we show that unambiguity regularization is beneficial to learning, and in combination with annealing (of the regularization strength) and sparsity priors it leads to improvement over the current state of the art.
5 0.49958679 31 emnlp-2012-Cross-Lingual Language Modeling with Syntactic Reordering for Low-Resource Speech Recognition
Author: Ping Xu ; Pascale Fung
Abstract: This paper proposes cross-lingual language modeling for transcribing source resourcepoor languages and translating them into target resource-rich languages if necessary. Our focus is to improve the speech recognition performance of low-resource languages by leveraging the language model statistics from resource-rich languages. The most challenging work of cross-lingual language modeling is to solve the syntactic discrepancies between the source and target languages. We therefore propose syntactic reordering for cross-lingual language modeling, and present a first result that compares inversion transduction grammar (ITG) reordering constraints to IBM and local constraints in an integrated speech transcription and translation system. Evaluations on resource-poor Cantonese speech transcription and Cantonese to resource-rich Mandarin translation tasks show that our proposed approach improves the system performance significantly, up to 3.4% relative WER reduction in Cantonese transcription and 13.3% relative bilingual evaluation understudy (BLEU) score improvement in Mandarin transcription compared with the system without reordering.
6 0.48684403 88 emnlp-2012-Minimal Dependency Length in Realization Ranking
7 0.48501146 122 emnlp-2012-Syntactic Surprisal Affects Spoken Word Duration in Conversational Contexts
8 0.46829572 125 emnlp-2012-Towards Efficient Named-Entity Rule Induction for Customizability
9 0.46003041 136 emnlp-2012-Weakly Supervised Training of Semantic Parsers
10 0.45521957 109 emnlp-2012-Re-training Monolingual Parser Bilingually for Syntactic SMT
11 0.44737235 82 emnlp-2012-Left-to-Right Tree-to-String Decoding with Prediction
12 0.43790784 8 emnlp-2012-A Phrase-Discovering Topic Model Using Hierarchical Pitman-Yor Processes
13 0.43579477 51 emnlp-2012-Extracting Opinion Expressions with semi-Markov Conditional Random Fields
14 0.43049604 124 emnlp-2012-Three Dependency-and-Boundary Models for Grammar Induction
15 0.42249161 133 emnlp-2012-Unsupervised PCFG Induction for Grounded Language Learning with Highly Ambiguous Supervision
16 0.40806758 75 emnlp-2012-Large Scale Decipherment for Out-of-Domain Machine Translation
17 0.40588486 27 emnlp-2012-Characterizing Stylistic Elements in Syntactic Structure
18 0.39337415 74 emnlp-2012-Language Model Rest Costs and Space-Efficient Storage
19 0.38891271 14 emnlp-2012-A Weakly Supervised Model for Sentence-Level Semantic Orientation Analysis with Multiple Experts
20 0.38426921 70 emnlp-2012-Joint Chinese Word Segmentation, POS Tagging and Parsing