acl acl2010 acl2010-59 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Frank Keller
Abstract: We pose the development of cognitively plausible models of human language processing as a challenge for computational linguistics. Existing models can only deal with isolated phenomena (e.g., garden paths) on small, specifically selected data sets. The challenge is to build models that integrate multiple aspects of human language processing at the syntactic, semantic, and discourse level. Like human language processing, these models should be incremental, predictive, broad coverage, and robust to noise. This challenge can only be met if standardized data sets and evaluation measures are developed.
Reference: text
sentIndex sentText sentNum sentScore
1 uk Abstract We pose the development of cognitively plausible models of human language processing as a challenge for computational linguistics. [sent-4, score-0.644]
2 Existing models can only deal with isolated phenomena (e. [sent-5, score-0.165]
3 The challenge is to build models that integrate multiple aspects of human language processing at the syntactic, semantic, and discourse level. [sent-8, score-0.597]
4 Like human language processing, these models should be incremental, predictive, broad coverage, and robust to noise. [sent-9, score-0.251]
5 This challenge can only be met if standardized data sets and evaluation measures are developed. [sent-10, score-0.407]
6 1 Introduction In many respects, human language processing is the ultimate goldstandard for computational linguistics. [sent-11, score-0.18]
7 Given the impressive performance of humans as language processors, it seems natural to turn to psycholinguistics, the discipline that studies human language processing, as a source of information about the design of efficient language processing systems. [sent-14, score-0.18]
8 To test their theories, psycholinguists construct computational models of human language processing, but these models often fall short of the engineering standards that are generally accepted in the CL community (e. [sent-17, score-0.278]
9 , broad coverage, robustness, efficiency): typical psycholinguistic models only deal with isolated phenomena and fail to scale to realistic data sets. [sent-19, score-0.612]
10 In this paper, we propose a challenge that requires the combination of research efforts in computational linguistics and psycholinguistics: the development of cognitively plausible models of human language processing. [sent-21, score-0.559]
11 This task can be decomposed into a modeling challenge (building models that instantiate known properties of human language processing) and a data and evaluation challenge (accounting for experimental findings and evaluating against standardized data sets), which we will discuss in turn. [sent-22, score-0.902]
12 1 Challenge Key Properties The first part ofthe challenge is to develop a model that instantiates key properties of human language processing, as established by psycholinguistic experimentation (see Table 1 for an overview representative references). [sent-24, score-0.896]
13 1 and A striking property of the human language processor is its efficiency and robustness. [sent-25, score-0.305]
14 There is considerable experimental evi- 1Here an in the following, we will focus on sentence processing, which is often regarded as a central aspect of human language processing. [sent-27, score-0.191]
15 A more comprehensive answer to our modeling challenge should also include phonological and morphological processing, semantic inference, discourse processing, and other non-syntactic aspects of language processing. [sent-28, score-0.413]
16 Furthermore, established results regarding the interface between language processing and non-linguistic cognition (e. [sent-29, score-0.198]
17 The processor also achieves broad coverage: it can deal with a wide variety of syntactic constructions, and is not restricted by the domain, register, or modality of the input. [sent-38, score-0.312]
18 Readers and listeners experience differential processing difficulty during this integration process, depending on the properties of the new word and its relationship to the preceding context. [sent-41, score-0.223]
19 There is evidence that the processor instantiates a strict form of incrementality by building only fully connected trees. [sent-42, score-0.33]
20 Furthermore, the processor is able to make predictions about upcoming material on the basis of sentence prefixes. [sent-43, score-0.331]
21 Another key property of human language processing is the fact that it operates with limited memory, and that structures in memory are subject to decay and interference. [sent-48, score-0.408]
22 In particular, the processor is known to incur a distance-based memory cost: combining the head of a phrase with its syntactic dependents is more difficult the more dependents have to be integrated and the further away they are. [sent-49, score-0.402]
23 This integration process is also subject to interference from similar items that have to be held in memory at the same time. [sent-50, score-0.178]
24 2 Current Models The challenge is to develop a computational model that captures the key properties ofhuman language processing outlined in the previous section. [sent-52, score-0.503]
25 A number of relevant models have been developed, mostly based on probabilistic parsing techniques, but none of them instantiates all the key properties discussed above (Table 1gives an overview of model properties). [sent-53, score-0.298]
26 2 The earliest approaches were ranking-based models (Rank), which make psycholinguistic predictions based on the ranking of the syntactic analyses produced by a probabilistic parser. [sent-54, score-0.628]
27 Jurafsky (1996) assumes that processing difficulty is triggered if the correct analysis falls below a certain probability threshold (i. [sent-55, score-0.168]
28 Similarly, Crocker and Brants (2000) assume that processing difficulty ensures if the highest-ranked analysis changes from one word to the next. [sent-58, score-0.168]
29 Being based on probabilistic parsing techniques, ranking-based models generally achieve a broad coverage, but their efficiency and robustness has not been evaluated. [sent-60, score-0.346]
30 Also, they are not designed to capture syntactic prediction or memory effects (other than search with a narrow beam in Brants and Crocker 2000). [sent-61, score-0.336]
31 The ranking-based approach has been generalized by surprisal models (Surp), which predict processing difficulty based on the change in the probability distribution over possible analy- ses from one word to the next (Hale, 2001 ; Levy, 2008; Demberg and Keller, 2008a; Ferrara Boston et al. [sent-62, score-0.302]
32 These models have been successful in accounting for a range of experimental data, and they achieve broad coverage. [sent-65, score-0.263]
33 On the other hand, the efficiency and robustness of these models has largely not been evaluated, and memory costs are not modeled (again except for restrictions in beam size). [sent-68, score-0.469]
34 The prediction model (Pred) explicitly predicts syntactic structure for upcoming words (Demberg and Keller, 2008b, 2009), thus accounting for experimental results on predictive language processing. [sent-69, score-0.391]
35 , the set of assumptions that links model quantities to behavioral data (e. [sent-72, score-0.188]
36 (2008) erence Discourse coherence ref- Table 2: Semantic factors in human language processing mentality by building fully connected trees. [sent-81, score-0.18]
37 However, the current implementation of the prediction model is neither robust and efficient nor offers broad coverage. [sent-83, score-0.163]
38 Recently, a stack-based model (Stack) has been proposed that imposes explicit, cognitively motivated memory constraints on the parser, in effect limiting the stack size available to the parser (Schuler et al. [sent-84, score-0.297]
39 This delivers robustness, efficiency, and broad coverage, but does not model syntactic prediction. [sent-86, score-0.171]
40 Unlike the other models dis- cussed here, no psycholinguistic evaluation has been conducted on the stack-based model, so its cognitive plausibility is preliminary. [sent-87, score-0.634]
41 3 Beyond Parsing There is strong evidence that human language processing is driven by an interaction of syntactic, semantic, and discourse processes (see Table 2 for an overview and references). [sent-89, score-0.369]
42 Considerable experimental work has focused on the semantic properties of the verb of the sentence, and verb sense, selectional restrictions, and thematic roles have all been shown to interact with syntactic ambiguity resolution. [sent-90, score-0.274]
43 Another large body of research has elucidated the interaction of discourse processing and syntactic processing. [sent-91, score-0.304]
44 The most-well known effect is probably that of referential context: syntactic ambiguities can be resolved if a discourse context is provided that makes one of the syntactic alternatives more plausible. [sent-92, score-0.302]
45 For instance, in a context that provides two possible antecedents for a noun phrase, the processor will prefer attaching a PP or a relative clause such that it disambiguates between the two antecedents; garden paths are reduced or disappear. [sent-93, score-0.236]
46 Other results point to the importance of discourse coherence for sentence processing, an example being implicit causality. [sent-94, score-0.184]
47 The challenge facing researchers in computational and psycholinguistics therefore includes the development of language processing models that combine syntactic processing with semantic and discourse processing. [sent-95, score-0.749]
48 So far, this challenge is largely unmet: there are some examples of models that integrate semantic processes such as thematic role assignment into a parsing model (Narayanan and Jurafsky, 2002; Pad o´ et al. [sent-96, score-0.413]
49 However, other semantic factors are not accounted for by these models, and incorporating non-lexical aspects of semantics into models of sentence processing is a challenge for ongoing research. [sent-98, score-0.458]
50 Recently, Dubey (2010) has proposed an approach that combines a probabilistic parser with a model of co-reference and discourse inference based on probabilistic logic. [sent-99, score-0.246]
51 (2010), who combine a vector-space model of semantics (Landauer and Dumais, 1997) with a syntactic parser and show that this results in predictions of processing difficulty that can be validated against an eye-tracking corpus. [sent-102, score-0.358]
52 4 Acquisition and Crosslinguistics All models of human language processing discussed so far rely on supervised training data. [sent-104, score-0.248]
53 The challenge therefore is to develop a model of language acquisition that works with such small training sets, while also giving rise to a language processor that meets the key criteria in Table 1. [sent-106, score-0.458]
54 However, none of the existing unsupervised models has been evaluated against psycholinguistic data sets, and they are not designed to meet even basic psycholinguistic criteria such as incrementality. [sent-108, score-0.786]
55 A related modeling challenge is the development of processing models for languages other than English. [sent-109, score-0.43]
56 There is a growing body of experimental research investigating human language processing in other languages, but virtually all existing psycholinguistic models only work for English (the only exceptions we are aware of are Dubey et al. [sent-110, score-0.655]
57 Again, the CL community has made significant progress in crosslinguistic parsing, especially using dependency grammar (Haji ˇc, 2009), and psycholinguistic modeling could benefit from this in order to meet the challenge of developing crosslinguistically valid models of human language processing. [sent-113, score-0.799]
58 1 Test Sets The second key challenge that needs to be addressed in order to develop cognitively plausible models of human language processing concerns test data and model evaluation. [sent-115, score-0.748]
59 Here, the state of the art in psycholinguistic modeling lags significantly behind standards in the CL community. [sent-116, score-0.423]
60 The authors typically describe their performance on a small set of handpicked examples; no attempts are made to test on a range of items from the experimental literature and determine model fit directly against behavioral measures (e. [sent-118, score-0.284]
61 This makes it very hard to obtain a realistic estimate of how well the models achieve their aim of capturing human language processing behavior. [sent-121, score-0.248]
62 Two types of test data are required for psycholinguistic modeling. [sent-123, score-0.359]
63 This collection should contain the actual experimental materials (sentences or discourse fragments) used in the experiments, together with the behavioral measurements obtained (reading times, eye-movement records, rating judgments, etc. [sent-125, score-0.372]
64 , garden paths, syntactic complexity, memory effects, semantic and discourse factors. [sent-129, score-0.492]
65 Such a test set will enable the standardized evaluation of psycholinguistic models by comparing the model predictions (rankings, surprisal values, memory costs, etc. [sent-130, score-0.88]
66 ) against behavioral measures on a large set of items. [sent-131, score-0.236]
67 This way both the coverage of a model (how many phenomena can it account for) and its accuracy (how well does it fit the behavioral data) can be assessed. [sent-132, score-0.294]
68 The use of contextualized language data makes it possible to assess not only syntactic models, but also models that capture discourse effects. [sent-135, score-0.287]
69 These corpora need to be annotated with behavioral measures, e. [sent-136, score-0.188]
70 However, the usefulness of the psycholinguistic corpora in Table 3 is restricted by the absence of gold-standard linguistic annotation (though the French part of the Dundee corpus, which is syntactically annotated). [sent-145, score-0.359]
71 This makes it difficult to test the accuracy of the linguistic structures computed by a model, and restricts evaluation to behavioral predictions. [sent-146, score-0.188]
72 The challenge is therefore to collect a standardized test set of naturally occurring text or speech enriched not only with behavioral vari- ables, but also with syntactic and semantic annotation. [sent-147, score-0.63]
73 Such a data set could for example be constructed by eye-tracking section 23 of the Penn Treebank (which is also part of Propbank, and thus has both syntactic and thematic role annotation). [sent-148, score-0.171]
74 These challenges provide standardized task descriptions and data sets; participants can enter their cognitive models, which were then compared using a pre-defined evaluation metric. [sent-152, score-0.282]
75 3 – 3The ICCM 2009 challenge was the Dynamic Stock and Flows Task, for more information see http : / /www . [sent-153, score-0.213]
76 (2006) MIT Corpus English 3,534 23 Self-paced reading Bachrach (2008) Table 3: Test corpora that have been used for psycholinguistic modeling of sentence processing; note that the Potsdam Corpus consists of isolated sentences, rather than of continuous text 3. [sent-158, score-0.627]
77 2 Behavioral and Neural Data As outlined in the previous section, a number of authors have evaluated psycholinguistic models against eye-tracking or reading time corpora. [sent-159, score-0.579]
78 Part of the data and evaluation challenge is to extend this evaluation to neural data as provided by eventrelated potential (ERP) or brain imaging studies (e. [sent-160, score-0.342]
79 Neural data sets are considerably more complex than behavioral ones, and modeling them is an important new task that the community is only beginning to address. [sent-163, score-0.252]
80 4 This is a very promising direction, and the challenge is to extend this approach to the sentence and discourse level (see Bachrach 2008). [sent-167, score-0.397]
81 Again, it will again be necessary to develop standardized test sets of both experimental data and corpus data. [sent-168, score-0.248]
82 3 Evaluation Measures We also anticipate that the availability of new test data sets will facilitate the development of new evaluation measures that specifically test the validity of psycholinguistic models. [sent-170, score-0.407]
83 Established CL evaluation measures such as Parseval are of limited use, as they can only test the linguistic, but not the behavioral or neural predictions of a model. [sent-171, score-0.351]
84 So far, many authors have relied on qualitative evaluation: if a model predicts a difference in (for instance) reading time between two types of sentences where such a difference was also found experimentally, then that counts as a successful test. [sent-172, score-0.153]
85 In most cases, no quantitative evaluation is performed, as this would require modeling the reading times for individual item and individual participants. [sent-173, score-0.17]
86 An important open challenge is there to develop evaluation measures and associated statistical procedures that can deal with these problems. [sent-180, score-0.315]
87 4 Conclusions In this paper, we discussed the modeling and data/evaluation challenges involved in developing cognitively plausible models of human language processing. [sent-181, score-0.41]
88 Developing computational models is of scientific importance in so far as models are im- plemented theories: models of language processing allow us to test scientific hypothesis about the cognitive processes that underpin language processing. [sent-182, score-0.425]
89 This type of precise, formalized hypothesis testing is only possible if standardized data sets and uniform evaluation procedures are available, as outlined in the present paper. [sent-183, score-0.192]
90 Once computational models of human language processing are available, they can be used to predict the difficulty that humans experience when processing text or speech. [sent-186, score-0.416]
91 In machine translation, evaluating the fluency of system output is crucial, and a model that predicts processing difficulty could be used for this, or to guide the choice between alternative translations, and maybe even to inform human post-editing. [sent-194, score-0.31]
92 Data from eye-tracking corpora as evidence for theories of syntactic processing complexity. [sent-231, score-0.272]
93 A computational model of prediction in human parsing: Unifying locality and surprisal effects. [sent-240, score-0.284]
94 The influence of discourse on syntax: A psycholinguistic model of sentence processing. [sent-245, score-0.543]
95 Parsing costs as predictors of reading difficulty: An evaluation using the Potsdam Sentence Corpus. [sent-254, score-0.15]
96 Misinterpretations of garden-path sentences: Implications for models of sentence processing and reanalysis. [sent-258, score-0.201]
97 A Bayesian model predicts human parse preference and reading time in sentence processing. [sent-361, score-0.296]
98 A probabilistic model of semantic plausibility in sentence processing. [sent-368, score-0.174]
99 Compound effect of probabilistic disambiguation and memory retrievals on sentence processing: Evidence from an eyetracking corpus. [sent-372, score-0.281]
100 Deriving lexical and syntactic expectation-based measures for psycholinguistic modeling via incremental top-down parsing. [sent-397, score-0.554]
wordName wordTfidf (topN-words)
[('psycholinguistic', 0.359), ('challenge', 0.213), ('behavioral', 0.188), ('memory', 0.178), ('standardized', 0.146), ('processor', 0.141), ('cognitive', 0.136), ('discourse', 0.136), ('demberg', 0.133), ('pickering', 0.128), ('cognitively', 0.119), ('cognition', 0.113), ('frank', 0.111), ('crocker', 0.109), ('reading', 0.106), ('pynte', 0.102), ('keller', 0.097), ('human', 0.095), ('garden', 0.095), ('bachrach', 0.094), ('dubey', 0.094), ('thematic', 0.088), ('broad', 0.088), ('ferrara', 0.088), ('processing', 0.085), ('difficulty', 0.083), ('syntactic', 0.083), ('psycholinguistics', 0.079), ('upcoming', 0.079), ('vera', 0.079), ('vasishth', 0.077), ('imaging', 0.077), ('prediction', 0.075), ('cl', 0.072), ('plausibility', 0.071), ('instantiates', 0.07), ('efficiency', 0.069), ('models', 0.068), ('robustness', 0.066), ('altmann', 0.066), ('sturt', 0.066), ('hart', 0.066), ('incrementality', 0.066), ('reinhold', 0.066), ('shravan', 0.066), ('surprisal', 0.066), ('modeling', 0.064), ('plausible', 0.064), ('predictions', 0.063), ('kliegl', 0.062), ('patil', 0.062), ('potsdam', 0.062), ('accounting', 0.059), ('gibson', 0.059), ('coverage', 0.059), ('garnsey', 0.058), ('hedderik', 0.058), ('risley', 0.058), ('sanford', 0.058), ('stewart', 0.058), ('brants', 0.056), ('matthew', 0.056), ('boston', 0.055), ('properties', 0.055), ('probabilistic', 0.055), ('develop', 0.054), ('hale', 0.053), ('evidence', 0.053), ('neural', 0.052), ('roark', 0.051), ('amsterdam', 0.051), ('theories', 0.051), ('traxler', 0.051), ('erp', 0.051), ('dundee', 0.051), ('fmri', 0.051), ('grodner', 0.051), ('kamide', 0.051), ('rijn', 0.051), ('schuler', 0.051), ('staub', 0.051), ('umesh', 0.051), ('mitchell', 0.051), ('isolated', 0.05), ('key', 0.05), ('locality', 0.048), ('measures', 0.048), ('sentence', 0.048), ('experimental', 0.048), ('phenomena', 0.047), ('predicts', 0.047), ('niels', 0.047), ('psycholinguists', 0.047), ('outlined', 0.046), ('journal', 0.045), ('costs', 0.044), ('semantics', 0.044), ('kehler', 0.044), ('effortlessly', 0.044), ('largely', 0.044)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000008 59 acl-2010-Cognitively Plausible Models of Human Language Processing
Author: Frank Keller
Abstract: We pose the development of cognitively plausible models of human language processing as a challenge for computational linguistics. Existing models can only deal with isolated phenomena (e.g., garden paths) on small, specifically selected data sets. The challenge is to build models that integrate multiple aspects of human language processing at the syntactic, semantic, and discourse level. Like human language processing, these models should be incremental, predictive, broad coverage, and robust to noise. This challenge can only be met if standardized data sets and evaluation measures are developed.
2 0.34490615 220 acl-2010-Syntactic and Semantic Factors in Processing Difficulty: An Integrated Measure
Author: Jeff Mitchell ; Mirella Lapata ; Vera Demberg ; Frank Keller
Abstract: The analysis of reading times can provide insights into the processes that underlie language comprehension, with longer reading times indicating greater cognitive load. There is evidence that the language processor is highly predictive, such that prior context allows upcoming linguistic material to be anticipated. Previous work has investigated the contributions of semantic and syntactic contexts in isolation, essentially treating them as independent factors. In this paper we analyze reading times in terms of a single predictive measure which integrates a model of semantic composition with an incremental parser and a language model.
3 0.29390207 229 acl-2010-The Influence of Discourse on Syntax: A Psycholinguistic Model of Sentence Processing
Author: Amit Dubey
Abstract: Probabilistic models of sentence comprehension are increasingly relevant to questions concerning human language processing. However, such models are often limited to syntactic factors. This paper introduces a novel sentence processing model that consists of a parser augmented with a probabilistic logic-based model of coreference resolution, which allows us to simulate how context interacts with syntax in a reading task. Our simulations show that a Weakly Interactive cognitive architecture can explain data which had been provided as evidence for the Strongly Interactive hypothesis.
4 0.20154171 65 acl-2010-Complexity Metrics in an Incremental Right-Corner Parser
Author: Stephen Wu ; Asaf Bachrach ; Carlos Cardenas ; William Schuler
Abstract: Hierarchical HMM (HHMM) parsers make promising cognitive models: while they use a bounded model of working memory and pursue incremental hypotheses in parallel, they still achieve parsing accuracies competitive with chart-based techniques. This paper aims to validate that a right-corner HHMM parser is also able to produce complexity metrics, which quantify a reader’s incremental difficulty in understanding a sentence. Besides defining standard metrics in the HHMM framework, a new metric, embedding difference, is also proposed, which tests the hypothesis that HHMM store elements represents syntactic working memory. Results show that HHMM surprisal outperforms all other evaluated metrics in predicting reading times, and that embedding difference makes a significant, independent contribution.
5 0.14922057 4 acl-2010-A Cognitive Cost Model of Annotations Based on Eye-Tracking Data
Author: Katrin Tomanek ; Udo Hahn ; Steffen Lohmann ; Jurgen Ziegler
Abstract: We report on an experiment to track complex decision points in linguistic metadata annotation where the decision behavior of annotators is observed with an eyetracking device. As experimental conditions we investigate different forms of textual context and linguistic complexity classes relative to syntax and semantics. Our data renders evidence that annotation performance depends on the semantic and syntactic complexity of the decision points and, more interestingly, indicates that fullscale context is mostly negligible with – the exception of semantic high-complexity cases. We then induce from this observational data a cognitively grounded cost model of linguistic meta-data annotations and compare it with existing non-cognitive models. Our data reveals that the cognitively founded model explains annotation costs (expressed in annotation time) more adequately than non-cognitive ones.
6 0.12962863 13 acl-2010-A Rational Model of Eye Movement Control in Reading
7 0.11554974 157 acl-2010-Last but Definitely Not Least: On the Role of the Last Sentence in Automatic Polarity-Classification
8 0.11535048 158 acl-2010-Latent Variable Models of Selectional Preference
9 0.10657294 86 acl-2010-Discourse Structure: Theory, Practice and Use
10 0.096981578 155 acl-2010-Kernel Based Discourse Relation Recognition with Temporal Ordering Information
11 0.094851211 33 acl-2010-Assessing the Role of Discourse References in Entailment Inference
12 0.091566004 38 acl-2010-Automatic Evaluation of Linguistic Quality in Multi-Document Summarization
13 0.089872085 232 acl-2010-The S-Space Package: An Open Source Package for Word Space Models
14 0.072668321 85 acl-2010-Detecting Experiences from Weblogs
15 0.070536375 226 acl-2010-The Human Language Project: Building a Universal Corpus of the World's Languages
16 0.068165705 70 acl-2010-Contextualizing Semantic Representations Using Syntactically Enriched Vector Models
17 0.067897364 93 acl-2010-Dynamic Programming for Linear-Time Incremental Parsing
18 0.066804111 206 acl-2010-Semantic Parsing: The Task, the State of the Art and the Future
19 0.066421144 153 acl-2010-Joint Syntactic and Semantic Parsing of Chinese
20 0.062109914 216 acl-2010-Starting from Scratch in Semantic Role Labeling
topicId topicWeight
[(0, -0.221), (1, 0.097), (2, 0.029), (3, -0.128), (4, -0.04), (5, 0.016), (6, 0.0), (7, -0.047), (8, 0.077), (9, -0.009), (10, -0.073), (11, 0.088), (12, 0.218), (13, 0.316), (14, -0.193), (15, 0.304), (16, 0.041), (17, 0.034), (18, -0.047), (19, 0.014), (20, -0.131), (21, 0.064), (22, 0.076), (23, 0.082), (24, 0.019), (25, 0.066), (26, -0.007), (27, -0.101), (28, 0.001), (29, 0.059), (30, 0.027), (31, -0.009), (32, -0.004), (33, -0.039), (34, 0.09), (35, -0.048), (36, -0.021), (37, 0.072), (38, 0.059), (39, -0.004), (40, -0.015), (41, -0.03), (42, 0.012), (43, 0.006), (44, 0.05), (45, -0.034), (46, -0.02), (47, 0.028), (48, 0.014), (49, 0.036)]
simIndex simValue paperId paperTitle
same-paper 1 0.94317263 59 acl-2010-Cognitively Plausible Models of Human Language Processing
Author: Frank Keller
Abstract: We pose the development of cognitively plausible models of human language processing as a challenge for computational linguistics. Existing models can only deal with isolated phenomena (e.g., garden paths) on small, specifically selected data sets. The challenge is to build models that integrate multiple aspects of human language processing at the syntactic, semantic, and discourse level. Like human language processing, these models should be incremental, predictive, broad coverage, and robust to noise. This challenge can only be met if standardized data sets and evaluation measures are developed.
2 0.87482989 65 acl-2010-Complexity Metrics in an Incremental Right-Corner Parser
Author: Stephen Wu ; Asaf Bachrach ; Carlos Cardenas ; William Schuler
Abstract: Hierarchical HMM (HHMM) parsers make promising cognitive models: while they use a bounded model of working memory and pursue incremental hypotheses in parallel, they still achieve parsing accuracies competitive with chart-based techniques. This paper aims to validate that a right-corner HHMM parser is also able to produce complexity metrics, which quantify a reader’s incremental difficulty in understanding a sentence. Besides defining standard metrics in the HHMM framework, a new metric, embedding difference, is also proposed, which tests the hypothesis that HHMM store elements represents syntactic working memory. Results show that HHMM surprisal outperforms all other evaluated metrics in predicting reading times, and that embedding difference makes a significant, independent contribution.
3 0.85326564 229 acl-2010-The Influence of Discourse on Syntax: A Psycholinguistic Model of Sentence Processing
Author: Amit Dubey
Abstract: Probabilistic models of sentence comprehension are increasingly relevant to questions concerning human language processing. However, such models are often limited to syntactic factors. This paper introduces a novel sentence processing model that consists of a parser augmented with a probabilistic logic-based model of coreference resolution, which allows us to simulate how context interacts with syntax in a reading task. Our simulations show that a Weakly Interactive cognitive architecture can explain data which had been provided as evidence for the Strongly Interactive hypothesis.
4 0.82981318 220 acl-2010-Syntactic and Semantic Factors in Processing Difficulty: An Integrated Measure
Author: Jeff Mitchell ; Mirella Lapata ; Vera Demberg ; Frank Keller
Abstract: The analysis of reading times can provide insights into the processes that underlie language comprehension, with longer reading times indicating greater cognitive load. There is evidence that the language processor is highly predictive, such that prior context allows upcoming linguistic material to be anticipated. Previous work has investigated the contributions of semantic and syntactic contexts in isolation, essentially treating them as independent factors. In this paper we analyze reading times in terms of a single predictive measure which integrates a model of semantic composition with an incremental parser and a language model.
5 0.73010021 13 acl-2010-A Rational Model of Eye Movement Control in Reading
Author: Klinton Bicknell ; Roger Levy
Abstract: A number of results in the study of realtime sentence comprehension have been explained by computational models as resulting from the rational use of probabilistic linguistic information. Many times, these hypotheses have been tested in reading by linking predictions about relative word difficulty to word-aggregated eye tracking measures such as go-past time. In this paper, we extend these results by asking to what extent reading is well-modeled as rational behavior at a finer level of analysis, predicting not aggregate measures, but the duration and location of each fixation. We present a new rational model of eye movement control in reading, the central assumption of which is that eye move- ment decisions are made to obtain noisy visual information as the reader performs Bayesian inference on the identities of the words in the sentence. As a case study, we present two simulations demonstrating that the model gives a rational explanation for between-word regressions.
6 0.46259686 157 acl-2010-Last but Definitely Not Least: On the Role of the Last Sentence in Automatic Polarity-Classification
7 0.41736987 4 acl-2010-A Cognitive Cost Model of Annotations Based on Eye-Tracking Data
8 0.38818011 61 acl-2010-Combining Data and Mathematical Models of Language Change
9 0.37732175 232 acl-2010-The S-Space Package: An Open Source Package for Word Space Models
10 0.35947236 206 acl-2010-Semantic Parsing: The Task, the State of the Art and the Future
11 0.34447241 86 acl-2010-Discourse Structure: Theory, Practice and Use
12 0.33224556 173 acl-2010-Modeling Norms of Turn-Taking in Multi-Party Conversation
13 0.3258864 41 acl-2010-Automatic Selectional Preference Acquisition for Latin Verbs
14 0.31913784 158 acl-2010-Latent Variable Models of Selectional Preference
15 0.31757224 119 acl-2010-Fixed Length Word Suffix for Factored Statistical Machine Translation
16 0.31432435 93 acl-2010-Dynamic Programming for Linear-Time Incremental Parsing
17 0.31086874 101 acl-2010-Entity-Based Local Coherence Modelling Using Topological Fields
18 0.31016073 7 acl-2010-A Generalized-Zero-Preserving Method for Compact Encoding of Concept Lattices
19 0.31005213 66 acl-2010-Compositional Matrix-Space Models of Language
20 0.30830008 12 acl-2010-A Probabilistic Generative Model for an Intermediate Constituency-Dependency Representation
topicId topicWeight
[(14, 0.021), (25, 0.045), (37, 0.244), (39, 0.015), (42, 0.02), (44, 0.018), (59, 0.078), (73, 0.042), (76, 0.016), (78, 0.086), (80, 0.014), (83, 0.117), (84, 0.117), (98, 0.088)]
simIndex simValue paperId paperTitle
same-paper 1 0.80276287 59 acl-2010-Cognitively Plausible Models of Human Language Processing
Author: Frank Keller
Abstract: We pose the development of cognitively plausible models of human language processing as a challenge for computational linguistics. Existing models can only deal with isolated phenomena (e.g., garden paths) on small, specifically selected data sets. The challenge is to build models that integrate multiple aspects of human language processing at the syntactic, semantic, and discourse level. Like human language processing, these models should be incremental, predictive, broad coverage, and robust to noise. This challenge can only be met if standardized data sets and evaluation measures are developed.
2 0.68340653 98 acl-2010-Efficient Staggered Decoding for Sequence Labeling
Author: Nobuhiro Kaji ; Yasuhiro Fujiwara ; Naoki Yoshinaga ; Masaru Kitsuregawa
Abstract: The Viterbi algorithm is the conventional decoding algorithm most widely adopted for sequence labeling. Viterbi decoding is, however, prohibitively slow when the label set is large, because its time complexity is quadratic in the number of labels. This paper proposes an exact decoding algorithm that overcomes this problem. A novel property of our algorithm is that it efficiently reduces the labels to be decoded, while still allowing us to check the optimality of the solution. Experiments on three tasks (POS tagging, joint POS tagging and chunking, and supertagging) show that the new algorithm is several orders of magnitude faster than the basic Viterbi and a state-of-the-art algo- rithm, CARPEDIEM (Esposito and Radicioni, 2009).
3 0.6675176 220 acl-2010-Syntactic and Semantic Factors in Processing Difficulty: An Integrated Measure
Author: Jeff Mitchell ; Mirella Lapata ; Vera Demberg ; Frank Keller
Abstract: The analysis of reading times can provide insights into the processes that underlie language comprehension, with longer reading times indicating greater cognitive load. There is evidence that the language processor is highly predictive, such that prior context allows upcoming linguistic material to be anticipated. Previous work has investigated the contributions of semantic and syntactic contexts in isolation, essentially treating them as independent factors. In this paper we analyze reading times in terms of a single predictive measure which integrates a model of semantic composition with an incremental parser and a language model.
4 0.65899372 65 acl-2010-Complexity Metrics in an Incremental Right-Corner Parser
Author: Stephen Wu ; Asaf Bachrach ; Carlos Cardenas ; William Schuler
Abstract: Hierarchical HMM (HHMM) parsers make promising cognitive models: while they use a bounded model of working memory and pursue incremental hypotheses in parallel, they still achieve parsing accuracies competitive with chart-based techniques. This paper aims to validate that a right-corner HHMM parser is also able to produce complexity metrics, which quantify a reader’s incremental difficulty in understanding a sentence. Besides defining standard metrics in the HHMM framework, a new metric, embedding difference, is also proposed, which tests the hypothesis that HHMM store elements represents syntactic working memory. Results show that HHMM surprisal outperforms all other evaluated metrics in predicting reading times, and that embedding difference makes a significant, independent contribution.
5 0.64359784 216 acl-2010-Starting from Scratch in Semantic Role Labeling
Author: Michael Connor ; Yael Gertner ; Cynthia Fisher ; Dan Roth
Abstract: A fundamental step in sentence comprehension involves assigning semantic roles to sentence constituents. To accomplish this, the listener must parse the sentence, find constituents that are candidate arguments, and assign semantic roles to those constituents. Each step depends on prior lexical and syntactic knowledge. Where do children learning their first languages begin in solving this problem? In this paper we focus on the parsing and argumentidentification steps that precede Semantic Role Labeling (SRL) training. We combine a simplified SRL with an unsupervised HMM part of speech tagger, and experiment with psycholinguisticallymotivated ways to label clusters resulting from the HMM so that they can be used to parse input for the SRL system. The results show that proposed shallow representations of sentence structure are robust to reductions in parsing accuracy, and that the contribution of alternative representations of sentence structure to successful semantic role labeling varies with the integrity of the parsing and argumentidentification stages.
6 0.64289653 18 acl-2010-A Study of Information Retrieval Weighting Schemes for Sentiment Analysis
7 0.63717186 229 acl-2010-The Influence of Discourse on Syntax: A Psycholinguistic Model of Sentence Processing
8 0.61917889 158 acl-2010-Latent Variable Models of Selectional Preference
9 0.60590351 136 acl-2010-How Many Words Is a Picture Worth? Automatic Caption Generation for News Images
10 0.59870327 17 acl-2010-A Structured Model for Joint Learning of Argument Roles and Predicate Senses
11 0.59581059 70 acl-2010-Contextualizing Semantic Representations Using Syntactically Enriched Vector Models
12 0.5953126 13 acl-2010-A Rational Model of Eye Movement Control in Reading
13 0.5893206 153 acl-2010-Joint Syntactic and Semantic Parsing of Chinese
14 0.58764482 120 acl-2010-Fully Unsupervised Core-Adjunct Argument Classification
15 0.58740443 10 acl-2010-A Latent Dirichlet Allocation Method for Selectional Preferences
16 0.58533823 126 acl-2010-GernEdiT - The GermaNet Editing Tool
17 0.58374923 103 acl-2010-Estimating Strictly Piecewise Distributions
18 0.58330941 195 acl-2010-Phylogenetic Grammar Induction
19 0.57953429 130 acl-2010-Hard Constraints for Grammatical Function Labelling
20 0.5793342 184 acl-2010-Open-Domain Semantic Role Labeling by Modeling Word Spans