acl acl2010 acl2010-65 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Stephen Wu ; Asaf Bachrach ; Carlos Cardenas ; William Schuler
Abstract: Hierarchical HMM (HHMM) parsers make promising cognitive models: while they use a bounded model of working memory and pursue incremental hypotheses in parallel, they still achieve parsing accuracies competitive with chart-based techniques. This paper aims to validate that a right-corner HHMM parser is also able to produce complexity metrics, which quantify a reader’s incremental difficulty in understanding a sentence. Besides defining standard metrics in the HHMM framework, a new metric, embedding difference, is also proposed, which tests the hypothesis that HHMM store elements represents syntactic working memory. Results show that HHMM surprisal outperforms all other evaluated metrics in predicting reading times, and that embedding difference makes a significant, independent contribution.
Reference: text
sentIndex sentText sentNum sentScore
1 edu Abstract Hierarchical HMM (HHMM) parsers make promising cognitive models: while they use a bounded model of working memory and pursue incremental hypotheses in parallel, they still achieve parsing accuracies competitive with chart-based techniques. [sent-7, score-0.352]
2 This paper aims to validate that a right-corner HHMM parser is also able to produce complexity metrics, which quantify a reader’s incremental difficulty in understanding a sentence. [sent-8, score-0.228]
3 Besides defining standard metrics in the HHMM framework, a new metric, embedding difference, is also proposed, which tests the hypothesis that HHMM store elements represents syntactic working memory. [sent-9, score-0.437]
4 Results show that HHMM surprisal outperforms all other evaluated metrics in predicting reading times, and that embedding difference makes a significant, independent contribution. [sent-10, score-0.794]
5 1 Introduction Since the introduction of a parser-based calculation for surprisal by Hale (2001), statistical techniques have been become common as models of reading difficulty and linguistic complexity. [sent-11, score-0.526]
6 Many other complexity metrics have been suggested as mutually contributing to reading difficulty; for example, entropy reduction (Hale, 2006), bigram probabilities (McDonald and Shillcock, 2003), and split-syntactic/lexical versions of other metrics (Roark et al. [sent-14, score-0.568]
7 A parser-derived complexity metric such as surprisal can only be as good (empirically) as the model of language from which it derives (Frank, 2009). [sent-16, score-0.464]
8 Ideally, a psychologically-plausible lan- guage model would produce a surprisal that would correlate better with linguistic complexity. [sent-17, score-0.343]
9 However, it is difficult to quantify linguistic complexity and reading difficulty. [sent-19, score-0.229]
10 The two commonly-used empirical quantifications of reading difficulty are eye-tracking measurements and word-by-word reading times; this paper uses reading times to find the predictiveness of several parser-derived complexity metrics. [sent-20, score-0.644]
11 Three complexity metrics will be calculated in a Hierarchical Hidden Markov Model (HHMM) parser that recognizes trees in right-corner form (the left-right dual of left-corner form). [sent-24, score-0.283]
12 The purpose of this paper is to determine whether the language model defined by the HHMM parser can also predict reading times it would be strange if a psychologically plausible model did not also produce viable complexity metrics. [sent-34, score-0.375]
13 In the course of showing that the HHMM parser does, in fact, predict reading times, we will define surprisal and entropy reduction in the HHMM parser, and introduce a third metric called embedding difference. [sent-35, score-0.957]
14 Gibson (1998; 2000) hypothesized two types of syntactic processing costs: integration cost, in which incremental input is combined with existing structures; and memory cost, where unfinished syntactic constructions may incur some short-term memory usage. [sent-36, score-0.362]
15 HHMM surprisal and entropy reduction may be considered forms of integration cost. [sent-37, score-0.514]
16 On the other hand, embedding difference is designed to model the cost of storing centerembedded structures in working memory. [sent-40, score-0.267]
17 Chen, Gibson, and Wolf (2005) showed that sentences requiring more syntactic memory during sentence processing increased reading times, and it is widely understood that center-embedding incurs significant syntactic processing costs (Miller and Chomsky, 1963; Gibson, 1998). [sent-41, score-0.34]
18 Thus, we would expect for the usage of the center-embedding memory store in an HHMM parser to correlate with reading times (and therefore linguistic complexity). [sent-42, score-0.504]
19 The HHMM parser processes syntactic constructs using a bounded number of store states, defined to represent short-term memory elements; additional states are utilized whenever centerembedded syntactic structures are present. [sent-43, score-0.511]
20 This behavior is similar to the hypothesized size of a human short-term memory store (Cowan, 2001). [sent-46, score-0.235]
21 A positive result in predicting reading times will lend additional validity to the claim that the HHMM parser’s bounded memory corresponds to bounded memory in human sentence processing. [sent-47, score-0.541]
22 The methodology for evaluating the complexity metrics is described in Section 3, with actual results in Section 4. [sent-49, score-0.158]
23 2 Parsing Model This section describes an incremental parser in which surprisal and entropy reduction are sim- ple calculations (Section 2. [sent-51, score-0.64]
24 1 Surprisal and Entropy in HMMs Hidden Markov Models (HMMs) probabilistically connect sequences of observed states ot and hidden states qt at corresponding time steps t. [sent-61, score-0.413]
25 In parsing, observed states are words; hidden states can be a conglomerate state of linguistic information, here taken to be syntactic. [sent-62, score-0.212]
26 t have been observed at time t, regardless of which syntactic states q1. [sent-66, score-0.127]
27 τY= Y1 (1) (2) Here, probabilities arise from a Transition Model (ΘA) between hidden states and an Observation Model (ΘB) that generates an observed state from a hidden state. [sent-80, score-0.214]
28 –t1)) (3) This framing of prefix probability and surprisal in a time-series model is equivalent to Hale’s (2001 ; 2006), assuming that q1. [sent-86, score-0.343]
29 t and entropy reduction (Hale, 2003; Hale, 2006) at the tth word is then ER(ot) = max(0, Ht−1 − Ht) (5) Both of these metrics fall out naturally from the time-series representation of the language model. [sent-107, score-0.255]
30 The third complexity metric, embedding difference, will be discussed after additional background in Section 2. [sent-108, score-0.249]
31 In the implementation of an HMM, candidate states at a given time qt are kept in a trellis, with step-by-step backpointers to the highestprobability q1. [sent-110, score-0.261]
32 2 Also, the best qt are often kept in a beam Bt, discarding low-probability in a beam states. [sent-113, score-0.296]
33 , 2008b), complexity metrics in this paper are calculated on a beam rather than over all (unbounded) possible derivations Dt. [sent-121, score-0.238]
34 As such, qt is factored into sequences of depth-specific variables one for each of D levels in the HMM hierarchy. [sent-128, score-0.2]
35 ftDi (6) (7) Transition probabilities PΘA (qt | qt–1) over complex hidden states qt are calculated| iqn two phases: • • Reduce phase. [sent-136, score-0.324]
36 Note that only qt is present at the end of the probability calculation. [sent-145, score-0.2]
37 (b) considers the qtd store to be incremental syntactic information. [sent-155, score-0.33]
38 It is only dependent on the syntactic state at D (or the deepest active HHMM level). [sent-159, score-0.125]
39 3 Parsing right-corner trees In this HHMM formulation, states and dependencies are optimized for parsing right-corner trees (Schuler et al. [sent-164, score-0.146]
40 These can be used as a case study to see what kind of operations need to occur in an 3This is technically a pushdown automoton (PDA), where the store is limited to D elements. [sent-169, score-0.145]
41 There is one unique set of HHMM state values for each tree, so the operations can be seen on either the tree or the store elements. [sent-174, score-0.172]
42 New words are observed input, and the bottom occupied element (the “frontier” of the store) is the context; together, they determine what the store will look like at t+1. [sent-178, score-0.157]
43 Occupies a new store element at a given time step. [sent-180, score-0.157]
44 For example, at t = 1, a new store element is occupied which can interact with the observed word, “the. [sent-181, score-0.157]
45 Starts a new active constituent at an already-occupied store element; always follows an in-level reduction. [sent-187, score-0.24]
46 Transitions the store to a new state in the next time step at the same level, where the awaited constituent changes and the active constituent remains the same. [sent-190, score-0.369]
47 Vacates a store element on seeing a complete active constituent. [sent-193, score-0.221]
48 This occurs after t = 4; “off” completes the active (at depth 2) VBD constituent, and vacates store element 2. [sent-194, score-0.332]
49 This is accompanied with an in-level transition at depth 1, producing the store at t = 5. [sent-195, score-0.167]
50 It should be noted that with some probability, complet- ing the active constituent does not vacate the store element, and the in-level reduction case would have to be invoked. [sent-196, score-0.332]
51 At t = 3, another possible hypothesis would be to remain on store element 1 using an ILE instead of a CLE. [sent-198, score-0.157]
52 A shift variable qtd at depth d and time step t is a syntactic state that must represent the active and awaited constituents of right-corner form: qtd d=ef hgqAtd,gqWtdi (11) e. [sent-203, score-0.491]
53 ftd d=ef hkftd,gftdi (12) First, kfdt is a switching variable that differentiates between ILT, CLE/CLR, and ILE/ILR. [sent-208, score-0.294]
54 This switching is the most important aspect of ftd, so regardless of what gfdt is, we will use: • ftd ∈ F0 when kfdt = 0, (ILT/no-op) • ftd ∈ F1 when kfdt = 1, (CLE/CLR) • ftd ∈ FG when kfdt ∈ G. [sent-209, score-0.943]
55 (ILE/ILR) Then, gfdt is used to keep track of a completelyrecognized constituent whenever a reduction occurs (ILR or CLR). [sent-210, score-0.179]
56 Examining ΘF-ILR,d and ΘF-CLR,d, we see that the produced ftd variables are also used in the “if” statement. [sent-217, score-0.235]
57 These models can be thought of as picking out a ftd first, finding the matching case, then applying the probability models that matches. [sent-218, score-0.235]
58 5 Embedding difference in the HHMM It should be clear from Figure 1 that at any time step while parsing depth-bounded right-corner trees, the candidate hidden state qt will have a “frontier” depth d(qt). [sent-239, score-0.407]
59 At time t, the beam of possible hidden states qt stores the syntactic state (and a backpointer) along with its probability, P(o1. [sent-240, score-0.433]
60 The average embedding depth at a time step is then µEMB(o1. [sent-245, score-0.224]
61 t−1) (16) There is a strong computational correspondence between this definition of embedding difference and the previous definition of surprisal. [sent-261, score-0.212]
62 t) (3′) Both surprisal and embedding difference include summations over the elements of the beam, and are calculated as a difference between previous and current beam states. [sent-272, score-0.672]
63 For example, the difference in order of subtraction only assures that a positive correlation with reading times is ex- pected. [sent-274, score-0.24]
64 Therefore, the inclusion of the embedding depth, d(qt), is the only significant difference between the two metrics. [sent-277, score-0.212]
65 The result is a metric that, despite numerical correspondence to surprisal, models the HHMM’s hypotheses about memory cost. [sent-278, score-0.164]
66 3 Evaluation Surprisal, entropy reduction, and embedding difference from the HHMM parser were evaluated against a full array of factors (Table 1) on a corpus of word-by-word reading times using a linear mixed-effects model. [sent-279, score-0.599]
67 1194 The corpus of reading times for 23 native English speakers was collected on a set of four narratives (Bachrach et al. [sent-280, score-0.203]
68 ’s (2009) work on the same corpus, reading times above 1500 ms (for diverted attention) or below 150 ms (for button presses planned before the word appeared) were discarded. [sent-286, score-0.203]
69 Thus, one may expect reading times to differ for these two types of words. [sent-296, score-0.203]
70 We report factors as statisti- βˆ/SE(βˆ), cally significant contributors to reading time if the absolute value of the t-value is greater than 2. [sent-301, score-0.194]
71 Most notably, HHMM surprisal is seen here to be a standout predictive measure for reading times regardless of word class. [sent-318, score-0.617]
72 If the HHMM parser is a good psycholinguistic model, we would expect it to at least produce a viable surprisal metric, and Table 2 attests that this is indeed the case. [sent-319, score-0.461]
73 Considering the AIC on the full data, the worst model with surprisal 1195 CoefficienFtULLSDAtdT. [sent-321, score-0.343]
74 7 (AIC=-10589) outperformed the best model without it (AIC=-10478), indicating that the HHMM surprisal is well worth including in the model regardless of the presence of other significant factors. [sent-345, score-0.375]
75 HHMM entropy reduction predicts reading times on the full dataset and on closed-class words. [sent-346, score-0.374]
76 The HHMM’s average embedding difference is also significant except in the case of openclass words removing embedding difference on open-class data yields χ12 = 0. [sent-350, score-0.424]
77 Embedding difference and surprisal were relatively correlated compared to other predictors (see Table 3), which is expected because embedding difference is calculated like a weighted version of surprisal. [sent-354, score-0.659]
78 Thus, we can conclude that the average embedding depth component affects reading times i. [sent-356, score-0.427]
79 , the HHMM’s notion of working memory behaves as we would expect human working memory to behave. [sent-358, score-0.286]
80 The fact that HHMM surprisal outperforms even n-gram metrics points to the importance of including a notion of sentence structure. [sent-362, score-0.427]
81 ’s eye-tracking study (2008a): a richer language model predicts eye movements during reading better than an oversimplified one. [sent-365, score-0.155]
82 The comparison there is between phrase structure surprisal (based on Hale’s (2001) calculation from an Earley parser), and dependency grammar surprisal (based on Nivre’s (2007) dependency parser). [sent-366, score-0.686]
83 Frank (2009) similarly reports improvements in the reading-time predictiveness ofunlexicalized surprisal when using a language model that is more plausible than PCFGs. [sent-367, score-0.404]
84 Previous work with complexity metrics on this corpus (Roark et al. [sent-371, score-0.158]
85 In addition, their metrics are different from ours in that they are designed to tease apart lexical and syntactic contributions to reading difficulty. [sent-375, score-0.273]
86 Their notion of entropy, in particular, estimates Hale’s definition of entropy on whole derivations (2006) by isolating the predictive entropy; they then proceed to define separate lexical and syntactic predictive entropies. [sent-376, score-0.191]
87 Drawing more directly from Hale, our definition is a whole-derivation metric based on the conditional entropy of the words, given the root. [sent-377, score-0.126]
88 Another difference is that previous parsers have produced useful complexity metrics without main- taining arc-eager/arc-standard ambiguity. [sent-383, score-0.227]
89 Results show that including this ambiguity in the HHMM at least does not invalidate (and may in fact improve) surprisal or entropy reduction as readingtime predictors. [sent-384, score-0.514]
90 6 Conclusion The task at hand was to determine whether the HHMM could consistently be considered a plausible psycholinguistic model, producing viable complexity metrics while maintaining other characteristics such as bounded memory usage. [sent-385, score-0.411]
91 The linear mixed-effects models on reading times validate this claim. [sent-386, score-0.203]
92 The HHMM can straightforwardly produce highly-predictive, standard complexity metrics (surprisal and entropy reduction). [sent-387, score-0.237]
93 HHMM surprisal performs very well in predicting reading times regardless of word class. [sent-388, score-0.578]
94 Our formulation of entropy reduction is also significant except in open-class words. [sent-389, score-0.171]
95 The new metric, embedding difference, uses the average center-embedding depth of the HHMM to model syntactic-processing memory cost. [sent-390, score-0.341]
96 This metric can only be calculated on parsers with an explicit representation for short-term memory elements like the right-corner HHMM parser. [sent-391, score-0.228]
97 Results show that embedding difference does predict reading times except in open-class words, yielding a significant contribution independent of surprisal despite the fact that its definition is similar to that of surprisal. [sent-392, score-0.758]
98 Acknowledgments Thanks to Brian Roark for help on the reading times corpus, Tim Miller for the formulation of entropy reduction, Mark Holland for statistical insight, and the anonymous reviewers for their input. [sent-393, score-0.282]
99 Parsing costs as predictors of reading difficulty: An evaluation using the Potsdam Sentence Corpus. [sent-416, score-0.19]
100 Deriving lexical and syntactic expectation-based measures for psycholinguistic modeling via incremental top-down parsing. [sent-533, score-0.146]
wordName wordTfidf (topN-words)
[('hhmm', 0.647), ('surprisal', 0.343), ('ftd', 0.235), ('qt', 0.2), ('embedding', 0.175), ('reading', 0.155), ('schuler', 0.129), ('store', 0.118), ('qtd', 0.118), ('memory', 0.117), ('hale', 0.098), ('reduction', 0.092), ('roark', 0.086), ('metrics', 0.084), ('entropy', 0.079), ('aic', 0.077), ('complexity', 0.074), ('hmms', 0.067), ('parser', 0.066), ('active', 0.064), ('hidden', 0.063), ('states', 0.061), ('incremental', 0.06), ('emb', 0.059), ('kfdt', 0.059), ('ef', 0.059), ('constituent', 0.058), ('psycholinguistic', 0.052), ('bounded', 0.052), ('gibson', 0.05), ('depth', 0.049), ('beam', 0.048), ('times', 0.048), ('bachrach', 0.047), ('cardenas', 0.047), ('metric', 0.047), ('crocker', 0.044), ('awaited', 0.044), ('engineers', 0.044), ('dt', 0.043), ('predictive', 0.039), ('element', 0.039), ('factors', 0.039), ('miller', 0.039), ('demberg', 0.038), ('np', 0.038), ('vbd', 0.037), ('shift', 0.037), ('difference', 0.037), ('pre', 0.036), ('td', 0.036), ('asaf', 0.035), ('ile', 0.035), ('boston', 0.035), ('predictors', 0.035), ('ht', 0.035), ('syntactic', 0.034), ('cognitive', 0.034), ('completes', 0.033), ('reinhold', 0.033), ('hmm', 0.033), ('parsers', 0.032), ('calculated', 0.032), ('plausible', 0.032), ('regardless', 0.032), ('abney', 0.031), ('parsing', 0.031), ('ft', 0.031), ('centerembedded', 0.029), ('dahan', 0.029), ('gfdt', 0.029), ('ilr', 0.029), ('ilt', 0.029), ('minnesota', 0.029), ('predictiveness', 0.029), ('qxt', 0.029), ('vacates', 0.029), ('equations', 0.028), ('brants', 0.028), ('difficulty', 0.028), ('ot', 0.028), ('storage', 0.028), ('brian', 0.027), ('state', 0.027), ('operations', 0.027), ('carlos', 0.027), ('brain', 0.027), ('trees', 0.027), ('working', 0.026), ('reduce', 0.026), ('markov', 0.026), ('chomsky', 0.026), ('engbert', 0.026), ('pollatsek', 0.026), ('abdelrahman', 0.026), ('samir', 0.026), ('bates', 0.026), ('clr', 0.026), ('depths', 0.026), ('predictivity', 0.026)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000004 65 acl-2010-Complexity Metrics in an Incremental Right-Corner Parser
Author: Stephen Wu ; Asaf Bachrach ; Carlos Cardenas ; William Schuler
Abstract: Hierarchical HMM (HHMM) parsers make promising cognitive models: while they use a bounded model of working memory and pursue incremental hypotheses in parallel, they still achieve parsing accuracies competitive with chart-based techniques. This paper aims to validate that a right-corner HHMM parser is also able to produce complexity metrics, which quantify a reader’s incremental difficulty in understanding a sentence. Besides defining standard metrics in the HHMM framework, a new metric, embedding difference, is also proposed, which tests the hypothesis that HHMM store elements represents syntactic working memory. Results show that HHMM surprisal outperforms all other evaluated metrics in predicting reading times, and that embedding difference makes a significant, independent contribution.
2 0.27482259 220 acl-2010-Syntactic and Semantic Factors in Processing Difficulty: An Integrated Measure
Author: Jeff Mitchell ; Mirella Lapata ; Vera Demberg ; Frank Keller
Abstract: The analysis of reading times can provide insights into the processes that underlie language comprehension, with longer reading times indicating greater cognitive load. There is evidence that the language processor is highly predictive, such that prior context allows upcoming linguistic material to be anticipated. Previous work has investigated the contributions of semantic and syntactic contexts in isolation, essentially treating them as independent factors. In this paper we analyze reading times in terms of a single predictive measure which integrates a model of semantic composition with an incremental parser and a language model.
3 0.20154171 59 acl-2010-Cognitively Plausible Models of Human Language Processing
Author: Frank Keller
Abstract: We pose the development of cognitively plausible models of human language processing as a challenge for computational linguistics. Existing models can only deal with isolated phenomena (e.g., garden paths) on small, specifically selected data sets. The challenge is to build models that integrate multiple aspects of human language processing at the syntactic, semantic, and discourse level. Like human language processing, these models should be incremental, predictive, broad coverage, and robust to noise. This challenge can only be met if standardized data sets and evaluation measures are developed.
4 0.17058277 229 acl-2010-The Influence of Discourse on Syntax: A Psycholinguistic Model of Sentence Processing
Author: Amit Dubey
Abstract: Probabilistic models of sentence comprehension are increasingly relevant to questions concerning human language processing. However, such models are often limited to syntactic factors. This paper introduces a novel sentence processing model that consists of a parser augmented with a probabilistic logic-based model of coreference resolution, which allows us to simulate how context interacts with syntax in a reading task. Our simulations show that a Weakly Interactive cognitive architecture can explain data which had been provided as evidence for the Strongly Interactive hypothesis.
5 0.10535154 173 acl-2010-Modeling Norms of Turn-Taking in Multi-Party Conversation
Author: Kornel Laskowski
Abstract: Substantial research effort has been invested in recent decades into the computational study and automatic processing of multi-party conversation. While most aspects of conversational speech have benefited from a wide availability of analytic, computationally tractable techniques, only qualitative assessments are available for characterizing multi-party turn-taking. The current paper attempts to address this deficiency by first proposing a framework for computing turn-taking model perplexity, and then by evaluating several multi-participant modeling approaches. Experiments show that direct multi-participant models do not generalize to held out data, and likely never will, for practical reasons. In contrast, the Extended-Degree-of-Overlap model represents a suitable candidate for future work in this area, and is shown to successfully predict the distribution of speech in time and across participants in previously unseen conversations.
6 0.10072853 13 acl-2010-A Rational Model of Eye Movement Control in Reading
7 0.094666608 4 acl-2010-A Cognitive Cost Model of Annotations Based on Eye-Tracking Data
8 0.090315558 93 acl-2010-Dynamic Programming for Linear-Time Incremental Parsing
9 0.052302193 184 acl-2010-Open-Domain Semantic Role Labeling by Modeling Word Spans
10 0.047389906 169 acl-2010-Learning to Translate with Source and Target Syntax
11 0.04717372 24 acl-2010-Active Learning-Based Elicitation for Semi-Supervised Word Alignment
12 0.045175888 7 acl-2010-A Generalized-Zero-Preserving Method for Compact Encoding of Concept Lattices
13 0.044577245 99 acl-2010-Efficient Third-Order Dependency Parsers
14 0.044448536 216 acl-2010-Starting from Scratch in Semantic Role Labeling
15 0.044218071 114 acl-2010-Faster Parsing by Supertagger Adaptation
16 0.043149687 200 acl-2010-Profiting from Mark-Up: Hyper-Text Annotations for Guided Parsing
17 0.042114936 153 acl-2010-Joint Syntactic and Semantic Parsing of Chinese
18 0.04207734 211 acl-2010-Simple, Accurate Parsing with an All-Fragments Grammar
19 0.040570535 263 acl-2010-Word Representations: A Simple and General Method for Semi-Supervised Learning
20 0.04052569 130 acl-2010-Hard Constraints for Grammatical Function Labelling
topicId topicWeight
[(0, -0.151), (1, 0.018), (2, 0.033), (3, -0.078), (4, -0.053), (5, -0.048), (6, 0.01), (7, -0.033), (8, 0.102), (9, 0.029), (10, -0.105), (11, 0.058), (12, 0.172), (13, 0.207), (14, -0.165), (15, 0.203), (16, 0.029), (17, 0.011), (18, -0.118), (19, 0.067), (20, -0.135), (21, 0.054), (22, 0.091), (23, 0.071), (24, 0.035), (25, 0.024), (26, 0.043), (27, -0.143), (28, 0.039), (29, 0.035), (30, 0.013), (31, -0.006), (32, 0.0), (33, 0.018), (34, 0.08), (35, -0.007), (36, -0.053), (37, 0.054), (38, 0.106), (39, -0.064), (40, 0.012), (41, -0.06), (42, 0.013), (43, -0.006), (44, 0.084), (45, 0.072), (46, -0.028), (47, 0.075), (48, 0.029), (49, 0.05)]
simIndex simValue paperId paperTitle
same-paper 1 0.94009238 65 acl-2010-Complexity Metrics in an Incremental Right-Corner Parser
Author: Stephen Wu ; Asaf Bachrach ; Carlos Cardenas ; William Schuler
Abstract: Hierarchical HMM (HHMM) parsers make promising cognitive models: while they use a bounded model of working memory and pursue incremental hypotheses in parallel, they still achieve parsing accuracies competitive with chart-based techniques. This paper aims to validate that a right-corner HHMM parser is also able to produce complexity metrics, which quantify a reader’s incremental difficulty in understanding a sentence. Besides defining standard metrics in the HHMM framework, a new metric, embedding difference, is also proposed, which tests the hypothesis that HHMM store elements represents syntactic working memory. Results show that HHMM surprisal outperforms all other evaluated metrics in predicting reading times, and that embedding difference makes a significant, independent contribution.
2 0.82709843 59 acl-2010-Cognitively Plausible Models of Human Language Processing
Author: Frank Keller
Abstract: We pose the development of cognitively plausible models of human language processing as a challenge for computational linguistics. Existing models can only deal with isolated phenomena (e.g., garden paths) on small, specifically selected data sets. The challenge is to build models that integrate multiple aspects of human language processing at the syntactic, semantic, and discourse level. Like human language processing, these models should be incremental, predictive, broad coverage, and robust to noise. This challenge can only be met if standardized data sets and evaluation measures are developed.
3 0.77746189 220 acl-2010-Syntactic and Semantic Factors in Processing Difficulty: An Integrated Measure
Author: Jeff Mitchell ; Mirella Lapata ; Vera Demberg ; Frank Keller
Abstract: The analysis of reading times can provide insights into the processes that underlie language comprehension, with longer reading times indicating greater cognitive load. There is evidence that the language processor is highly predictive, such that prior context allows upcoming linguistic material to be anticipated. Previous work has investigated the contributions of semantic and syntactic contexts in isolation, essentially treating them as independent factors. In this paper we analyze reading times in terms of a single predictive measure which integrates a model of semantic composition with an incremental parser and a language model.
4 0.74763215 13 acl-2010-A Rational Model of Eye Movement Control in Reading
Author: Klinton Bicknell ; Roger Levy
Abstract: A number of results in the study of realtime sentence comprehension have been explained by computational models as resulting from the rational use of probabilistic linguistic information. Many times, these hypotheses have been tested in reading by linking predictions about relative word difficulty to word-aggregated eye tracking measures such as go-past time. In this paper, we extend these results by asking to what extent reading is well-modeled as rational behavior at a finer level of analysis, predicting not aggregate measures, but the duration and location of each fixation. We present a new rational model of eye movement control in reading, the central assumption of which is that eye move- ment decisions are made to obtain noisy visual information as the reader performs Bayesian inference on the identities of the words in the sentence. As a case study, we present two simulations demonstrating that the model gives a rational explanation for between-word regressions.
5 0.71325076 229 acl-2010-The Influence of Discourse on Syntax: A Psycholinguistic Model of Sentence Processing
Author: Amit Dubey
Abstract: Probabilistic models of sentence comprehension are increasingly relevant to questions concerning human language processing. However, such models are often limited to syntactic factors. This paper introduces a novel sentence processing model that consists of a parser augmented with a probabilistic logic-based model of coreference resolution, which allows us to simulate how context interacts with syntax in a reading task. Our simulations show that a Weakly Interactive cognitive architecture can explain data which had been provided as evidence for the Strongly Interactive hypothesis.
6 0.39118046 173 acl-2010-Modeling Norms of Turn-Taking in Multi-Party Conversation
7 0.38009909 93 acl-2010-Dynamic Programming for Linear-Time Incremental Parsing
8 0.37976635 7 acl-2010-A Generalized-Zero-Preserving Method for Compact Encoding of Concept Lattices
9 0.35993969 4 acl-2010-A Cognitive Cost Model of Annotations Based on Eye-Tracking Data
10 0.35035858 61 acl-2010-Combining Data and Mathematical Models of Language Change
11 0.30839282 157 acl-2010-Last but Definitely Not Least: On the Role of the Last Sentence in Automatic Polarity-Classification
12 0.30676341 232 acl-2010-The S-Space Package: An Open Source Package for Word Space Models
13 0.30133265 114 acl-2010-Faster Parsing by Supertagger Adaptation
14 0.29318991 66 acl-2010-Compositional Matrix-Space Models of Language
15 0.2892085 183 acl-2010-Online Generation of Locality Sensitive Hash Signatures
16 0.28389856 211 acl-2010-Simple, Accurate Parsing with an All-Fragments Grammar
17 0.27828547 74 acl-2010-Correcting Errors in Speech Recognition with Articulatory Dynamics
18 0.27579543 12 acl-2010-A Probabilistic Generative Model for an Intermediate Constituency-Dependency Representation
19 0.27447605 140 acl-2010-Identifying Non-Explicit Citing Sentences for Citation-Based Summarization.
20 0.2734749 102 acl-2010-Error Detection for Statistical Machine Translation Using Linguistic Features
topicId topicWeight
[(14, 0.028), (25, 0.052), (30, 0.19), (33, 0.017), (37, 0.044), (39, 0.034), (42, 0.017), (59, 0.083), (73, 0.037), (78, 0.038), (83, 0.091), (84, 0.129), (98, 0.105)]
simIndex simValue paperId paperTitle
1 0.89176643 176 acl-2010-Mood Patterns and Affective Lexicon Access in Weblogs
Author: Thin Nguyen
Abstract: The emergence of social media brings chances, but also challenges, to linguistic analysis. In this paper we investigate a novel problem of discovering patterns based on emotion and the association of moods and affective lexicon usage in blogosphere, a representative for social media. We propose the use ofnormative emotional scores for English words in combination with a psychological model of emotion measurement and a nonparametric clustering process for inferring meaningful emotion patterns automatically from data. Our results on a dataset consisting of more than 17 million mood-groundtruthed blogposts have shown interesting evidence of the emotion patterns automatically discovered that match well with the core- affect emotion model theorized by psychologists. We then present a method based on information theory to discover the association of moods and affective lexicon usage in the new media.
same-paper 2 0.81668067 65 acl-2010-Complexity Metrics in an Incremental Right-Corner Parser
Author: Stephen Wu ; Asaf Bachrach ; Carlos Cardenas ; William Schuler
Abstract: Hierarchical HMM (HHMM) parsers make promising cognitive models: while they use a bounded model of working memory and pursue incremental hypotheses in parallel, they still achieve parsing accuracies competitive with chart-based techniques. This paper aims to validate that a right-corner HHMM parser is also able to produce complexity metrics, which quantify a reader’s incremental difficulty in understanding a sentence. Besides defining standard metrics in the HHMM framework, a new metric, embedding difference, is also proposed, which tests the hypothesis that HHMM store elements represents syntactic working memory. Results show that HHMM surprisal outperforms all other evaluated metrics in predicting reading times, and that embedding difference makes a significant, independent contribution.
3 0.73641521 220 acl-2010-Syntactic and Semantic Factors in Processing Difficulty: An Integrated Measure
Author: Jeff Mitchell ; Mirella Lapata ; Vera Demberg ; Frank Keller
Abstract: The analysis of reading times can provide insights into the processes that underlie language comprehension, with longer reading times indicating greater cognitive load. There is evidence that the language processor is highly predictive, such that prior context allows upcoming linguistic material to be anticipated. Previous work has investigated the contributions of semantic and syntactic contexts in isolation, essentially treating them as independent factors. In this paper we analyze reading times in terms of a single predictive measure which integrates a model of semantic composition with an incremental parser and a language model.
4 0.73217636 216 acl-2010-Starting from Scratch in Semantic Role Labeling
Author: Michael Connor ; Yael Gertner ; Cynthia Fisher ; Dan Roth
Abstract: A fundamental step in sentence comprehension involves assigning semantic roles to sentence constituents. To accomplish this, the listener must parse the sentence, find constituents that are candidate arguments, and assign semantic roles to those constituents. Each step depends on prior lexical and syntactic knowledge. Where do children learning their first languages begin in solving this problem? In this paper we focus on the parsing and argumentidentification steps that precede Semantic Role Labeling (SRL) training. We combine a simplified SRL with an unsupervised HMM part of speech tagger, and experiment with psycholinguisticallymotivated ways to label clusters resulting from the HMM so that they can be used to parse input for the SRL system. The results show that proposed shallow representations of sentence structure are robust to reductions in parsing accuracy, and that the contribution of alternative representations of sentence structure to successful semantic role labeling varies with the integrity of the parsing and argumentidentification stages.
5 0.71796727 18 acl-2010-A Study of Information Retrieval Weighting Schemes for Sentiment Analysis
Author: Georgios Paltoglou ; Mike Thelwall
Abstract: Most sentiment analysis approaches use as baseline a support vector machines (SVM) classifier with binary unigram weights. In this paper, we explore whether more sophisticated feature weighting schemes from Information Retrieval can enhance classification accuracy. We show that variants of the classic tf.idf scheme adapted to sentiment analysis provide significant increases in accuracy, especially when using a sublinear function for term frequency weights and document frequency smoothing. The techniques are tested on a wide selection of data sets and produce the best accuracy to our knowledge.
6 0.71535945 59 acl-2010-Cognitively Plausible Models of Human Language Processing
7 0.70062411 136 acl-2010-How Many Words Is a Picture Worth? Automatic Caption Generation for News Images
8 0.69809902 103 acl-2010-Estimating Strictly Piecewise Distributions
9 0.69411302 158 acl-2010-Latent Variable Models of Selectional Preference
10 0.69164842 126 acl-2010-GernEdiT - The GermaNet Editing Tool
11 0.66029477 13 acl-2010-A Rational Model of Eye Movement Control in Reading
12 0.64221269 162 acl-2010-Learning Common Grammar from Multilingual Corpus
13 0.64178646 195 acl-2010-Phylogenetic Grammar Induction
14 0.63401604 39 acl-2010-Automatic Generation of Story Highlights
15 0.63287842 55 acl-2010-Bootstrapping Semantic Analyzers from Non-Contradictory Texts
16 0.63213772 184 acl-2010-Open-Domain Semantic Role Labeling by Modeling Word Spans
17 0.63159579 66 acl-2010-Compositional Matrix-Space Models of Language
18 0.63033617 202 acl-2010-Reading between the Lines: Learning to Map High-Level Instructions to Commands
19 0.63011527 211 acl-2010-Simple, Accurate Parsing with an All-Fragments Grammar
20 0.62865144 120 acl-2010-Fully Unsupervised Core-Adjunct Argument Classification