nips nips2000 nips2000-66 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Szabolcs KĂĄli, Peter Dayan
Abstract: In memory consolidation, declarative memories which initially require the hippocampus for their recall, ultimately become independent of it. Consolidation has been the focus of numerous experimental and qualitative modeling studies, but only little quantitative exploration. We present a consolidation model in which hierarchical connections in the cortex, that initially instantiate purely semantic information acquired through probabilistic unsupervised learning, come to instantiate episodic information as well. The hippocampus is responsible for helping complete partial input patterns before consolidation is complete, while also training the cortex to perform appropriate completion by itself.
Reference: text
sentIndex sentText sentNum sentScore
1 uk Abstract In memory consolidation, declarative memories which initially require the hippocampus for their recall, ultimately become independent of it. [sent-8, score-0.934]
2 Consolidation has been the focus of numerous experimental and qualitative modeling studies, but only little quantitative exploration. [sent-9, score-0.113]
3 We present a consolidation model in which hierarchical connections in the cortex, that initially instantiate purely semantic information acquired through probabilistic unsupervised learning, come to instantiate episodic information as well. [sent-10, score-1.331]
4 The hippocampus is responsible for helping complete partial input patterns before consolidation is complete, while also training the cortex to perform appropriate completion by itself. [sent-11, score-1.31]
5 1 Introduction The hippocampal formation and adjacent cortical areas have long been believed to be involved in the acquisition and retrieval of long-term memory for events and other declarative information. [sent-12, score-0.935]
6 Clinical studies in humans and animal experiments indicate that damage to these regions results in amnesia, whereby the ability to acquire new declarative memories is impaired and some of the memories acquired before the damage are lost [I]. [sent-13, score-1.291]
7 The observation that recent memories are more likely to be lost than old memories in these cases has generally been interpreted as evidence that the role of these medial temporal lobe structures in the storage and/orretrieval of declarative memories is only temporary. [sent-14, score-1.129]
8 However, other possible interpretations of the data have also been proposed [5]. [sent-17, score-0.033]
9 There have been several analyses of the computational issues underlying consolidation. [sent-18, score-0.074]
10 There is a general consensus that memory recall involves the reinstatement of cortical activation patterns which characterize the original episodes, based only on partial or noisy input. [sent-19, score-0.686]
11 Despite the popularity of the ideas outlined above, there have been surprisingly few attempts to construct quantitative models of memory consolidation. [sent-22, score-0.231]
12 Alvarez and Squire (1994) is the only model we could find that has actually been implemented and tested quantitatively. [sent-23, score-0.035]
13 In this paper, we consider consolidation using a model whose complexity brings to the fore consideration of computational issues that are invisible to simpler proposals. [sent-26, score-0.781]
14 In particular, it treats cortex as a hierarchical structure, with hierarchical codes for input patterns acquired through a process of unsupervised learning. [sent-27, score-0.606]
15 This allows us to study the relationship between coding for generic patterns, which forms a sort of semantic memory, and the coding for the specific patterns through consolidation. [sent-28, score-0.323]
16 It also allows us to consider consolidation as happening in hierarchical connections (in which the cortex abounds) as an alternative to consolidation only between disparate areas at the same level of the hierarchy. [sent-29, score-1.819]
17 The next section of the paper describes the model in detail and section 3 shows its performance. [sent-30, score-0.035]
18 2 The Model Figure la shows the architecture of the model, which involves three cortical areas (A, B, and C) that represent different aspects of the world. [sent-31, score-0.49]
19 We can understand consolidation as follows : across the whole spectrum of possible inputs, there is structure in the activity within each area; but there are no strong correlations between the activities in different areas (these are the generic patterns referred to above). [sent-32, score-1.152]
20 Thus, for instance, nothing in particular can be concluded about the pattern of activity in area C given just the activities in areas A and B. [sent-33, score-0.438]
21 However, for the specific patterns that form particular episodes, there are correlations in these activities. [sent-34, score-0.222]
22 As a result of this, it becomes possible to be much more definite about the pattern in C given activities in A and B that reinstate part of the episode. [sent-35, score-0.104]
23 Before consolidation, information about these correlations is stored in the hippocampus and related structures; after consolidation, the information is stored directly in the weights that construct cortical representations. [sent-36, score-0.738]
24 The model does not assume that there are any direct connections between the cortical areas. [sent-37, score-0.383]
25 Instead, as a closer match to the available anatomical data, we assume a hierarchy of cortical regions (in the present model having just two layers) below the hippocampus. [sent-38, score-0.543]
26 EIP is connected bidirectionally to all the cortical areas. [sent-40, score-0.314]
wordName wordTfidf (topN-words)
[('consolidation', 0.642), ('cortical', 0.278), ('hippocampus', 0.253), ('memories', 0.241), ('declarative', 0.197), ('areas', 0.145), ('patterns', 0.134), ('completion', 0.128), ('anatomical', 0.116), ('hierarchical', 0.113), ('memory', 0.105), ('neocortex', 0.099), ('szabolcs', 0.099), ('acquired', 0.09), ('instantiate', 0.085), ('disparate', 0.085), ('cortex', 0.079), ('episodes', 0.077), ('lost', 0.077), ('stored', 0.074), ('initially', 0.072), ('damage', 0.071), ('connections', 0.07), ('activities', 0.07), ('generic', 0.063), ('hippocampal', 0.063), ('regions', 0.063), ('recall', 0.061), ('correlations', 0.059), ('semantic', 0.058), ('formation', 0.058), ('quantitative', 0.058), ('humans', 0.055), ('qualitative', 0.055), ('hierarchy', 0.051), ('area', 0.048), ('happening', 0.043), ('clinical', 0.043), ('consolidated', 0.043), ('concluded', 0.043), ('efficacies', 0.043), ('episode', 0.043), ('episodic', 0.043), ('hc', 0.043), ('partial', 0.042), ('issues', 0.042), ('studies', 0.041), ('activity', 0.039), ('spartan', 0.039), ('acquire', 0.039), ('treats', 0.039), ('establishing', 0.039), ('decades', 0.039), ('double', 0.039), ('impaired', 0.039), ('intact', 0.039), ('rats', 0.039), ('unsupervised', 0.038), ('aspects', 0.037), ('connected', 0.036), ('consensus', 0.036), ('popularity', 0.036), ('whereby', 0.036), ('advocated', 0.036), ('medial', 0.036), ('model', 0.035), ('pattern', 0.034), ('london', 0.034), ('coding', 0.034), ('become', 0.034), ('hard', 0.034), ('structures', 0.034), ('brings', 0.033), ('reciprocal', 0.033), ('interpretations', 0.033), ('embodies', 0.033), ('supporting', 0.032), ('days', 0.032), ('outlined', 0.032), ('analyses', 0.032), ('ultimately', 0.032), ('serious', 0.032), ('complete', 0.032), ('role', 0.032), ('establish', 0.03), ('nothing', 0.03), ('adjacent', 0.03), ('dayan', 0.03), ('old', 0.03), ('animal', 0.03), ('eg', 0.03), ('believed', 0.03), ('involves', 0.03), ('call', 0.03), ('particular', 0.029), ('gradually', 0.029), ('retrieval', 0.029), ('shared', 0.029), ('limitations', 0.029), ('consideration', 0.029)]
simIndex simValue paperId paperTitle
same-paper 1 0.99999976 66 nips-2000-Hippocampally-Dependent Consolidation in a Hierarchical Model of Neocortex
Author: Szabolcs KĂĄli, Peter Dayan
Abstract: In memory consolidation, declarative memories which initially require the hippocampus for their recall, ultimately become independent of it. Consolidation has been the focus of numerous experimental and qualitative modeling studies, but only little quantitative exploration. We present a consolidation model in which hierarchical connections in the cortex, that initially instantiate purely semantic information acquired through probabilistic unsupervised learning, come to instantiate episodic information as well. The hippocampus is responsible for helping complete partial input patterns before consolidation is complete, while also training the cortex to perform appropriate completion by itself.
2 0.11117508 124 nips-2000-Spike-Timing-Dependent Learning for Oscillatory Networks
Author: Silvia Scarpetta, Zhaoping Li, John A. Hertz
Abstract: We apply to oscillatory networks a class of learning rules in which synaptic weights change proportional to pre- and post-synaptic activity, with a kernel A(r) measuring the effect for a postsynaptic spike a time r after the presynaptic one. The resulting synaptic matrices have an outer-product form in which the oscillating patterns are represented as complex vectors. In a simple model, the even part of A(r) enhances the resonant response to learned stimulus by reducing the effective damping, while the odd part determines the frequency of oscillation. We relate our model to the olfactory cortex and hippocampus and their presumed roles in forming associative memories and input representations. 1
3 0.086689278 87 nips-2000-Modelling Spatial Recall, Mental Imagery and Neglect
Author: Suzanna Becker, Neil Burgess
Abstract: We present a computational model of the neural mechanisms in the parietal and temporal lobes that support spatial navigation, recall of scenes and imagery of the products of recall. Long term representations are stored in the hippocampus, and are associated with local spatial and object-related features in the parahippocampal region. Viewer-centered representations are dynamically generated from long term memory in the parietal part of the model. The model thereby simulates recall and imagery of locations and objects in complex environments. After parietal damage, the model exhibits hemispatial neglect in mental imagery that rotates with the imagined perspective of the observer, as in the famous Milan Square experiment [1]. Our model makes novel predictions for the neural representations in the parahippocampal and parietal regions and for behavior in healthy volunteers and neuropsychological patients.
Author: Angelo Arleo, Fabrizio Smeraldi, Stéphane Hug, Wulfram Gerstner
Abstract: We model hippocampal place cells and head-direction cells by combining allothetic (visual) and idiothetic (proprioceptive) stimuli. Visual input, provided by a video camera on a miniature robot, is preprocessed by a set of Gabor filters on 31 nodes of a log-polar retinotopic graph. Unsupervised Hebbian learning is employed to incrementally build a population of localized overlapping place fields. Place cells serve as basis functions for reinforcement learning. Experimental results for goal-oriented navigation of a mobile robot are presented.
5 0.058111623 129 nips-2000-Temporally Dependent Plasticity: An Information Theoretic Account
Author: Gal Chechik, Naftali Tishby
Abstract: The paradigm of Hebbian learning has recently received a novel interpretation with the discovery of synaptic plasticity that depends on the relative timing of pre and post synaptic spikes. This paper derives a temporally dependent learning rule from the basic principle of mutual information maximization and studies its relation to the experimentally observed plasticity. We find that a supervised spike-dependent learning rule sharing similar structure with the experimentally observed plasticity increases mutual information to a stable near optimal level. Moreover, the analysis reveals how the temporal structure of time-dependent learning rules is determined by the temporal filter applied by neurons over their inputs. These results suggest experimental prediction as to the dependency of the learning rule on neuronal biophysical parameters 1
6 0.056671564 34 nips-2000-Competition and Arbors in Ocular Dominance
7 0.054099586 63 nips-2000-Hierarchical Memory-Based Reinforcement Learning
8 0.053980492 56 nips-2000-Foundations for a Circuit Complexity Theory of Sensory Processing
9 0.053268448 102 nips-2000-Position Variance, Recurrence and Perceptual Learning
10 0.050671995 147 nips-2000-Who Does What? A Novel Algorithm to Determine Function Localization
11 0.043838311 104 nips-2000-Processing of Time Series by Neural Circuits with Biologically Realistic Synaptic Dynamics
12 0.043649539 40 nips-2000-Dendritic Compartmentalization Could Underlie Competition and Attentional Biasing of Simultaneous Visual Stimuli
13 0.040855031 10 nips-2000-A Productive, Systematic Framework for the Representation of Visual Structure
14 0.038990147 108 nips-2000-Recognizing Hand-written Digits Using Hierarchical Products of Experts
15 0.037891276 19 nips-2000-Adaptive Object Representation with Hierarchically-Distributed Memory Sites
16 0.035655763 100 nips-2000-Permitted and Forbidden Sets in Symmetric Threshold-Linear Networks
17 0.032041728 55 nips-2000-Finding the Key to a Synapse
18 0.031732161 103 nips-2000-Probabilistic Semantic Video Indexing
19 0.03157657 26 nips-2000-Automated State Abstraction for Options using the U-Tree Algorithm
20 0.030319372 107 nips-2000-Rate-coded Restricted Boltzmann Machines for Face Recognition
topicId topicWeight
[(0, 0.107), (1, -0.089), (2, -0.062), (3, -0.03), (4, -0.023), (5, 0.028), (6, 0.048), (7, -0.066), (8, 0.078), (9, 0.019), (10, 0.039), (11, 0.011), (12, -0.011), (13, -0.058), (14, 0.117), (15, 0.039), (16, 0.036), (17, -0.035), (18, 0.165), (19, 0.088), (20, -0.086), (21, -0.006), (22, -0.015), (23, -0.031), (24, -0.199), (25, -0.008), (26, 0.159), (27, 0.036), (28, -0.039), (29, 0.083), (30, -0.147), (31, -0.029), (32, 0.097), (33, 0.051), (34, 0.136), (35, 0.111), (36, -0.058), (37, 0.131), (38, -0.111), (39, -0.077), (40, 0.048), (41, -0.016), (42, 0.074), (43, 0.307), (44, 0.067), (45, 0.063), (46, -0.027), (47, -0.101), (48, 0.197), (49, -0.151)]
simIndex simValue paperId paperTitle
same-paper 1 0.97814918 66 nips-2000-Hippocampally-Dependent Consolidation in a Hierarchical Model of Neocortex
Author: Szabolcs KĂĄli, Peter Dayan
Abstract: In memory consolidation, declarative memories which initially require the hippocampus for their recall, ultimately become independent of it. Consolidation has been the focus of numerous experimental and qualitative modeling studies, but only little quantitative exploration. We present a consolidation model in which hierarchical connections in the cortex, that initially instantiate purely semantic information acquired through probabilistic unsupervised learning, come to instantiate episodic information as well. The hippocampus is responsible for helping complete partial input patterns before consolidation is complete, while also training the cortex to perform appropriate completion by itself.
2 0.56873381 34 nips-2000-Competition and Arbors in Ocular Dominance
Author: Peter Dayan
Abstract: Hebbian and competitive Hebbian algorithms are almost ubiquitous in modeling pattern formation in cortical development. We analyse in theoretical detail a particular model (adapted from Piepenbrock & Obermayer, 1999) for the development of Id stripe-like patterns, which places competitive and interactive cortical influences, and free and restricted initial arborisation onto a common footing.
3 0.49691924 124 nips-2000-Spike-Timing-Dependent Learning for Oscillatory Networks
Author: Silvia Scarpetta, Zhaoping Li, John A. Hertz
Abstract: We apply to oscillatory networks a class of learning rules in which synaptic weights change proportional to pre- and post-synaptic activity, with a kernel A(r) measuring the effect for a postsynaptic spike a time r after the presynaptic one. The resulting synaptic matrices have an outer-product form in which the oscillating patterns are represented as complex vectors. In a simple model, the even part of A(r) enhances the resonant response to learned stimulus by reducing the effective damping, while the odd part determines the frequency of oscillation. We relate our model to the olfactory cortex and hippocampus and their presumed roles in forming associative memories and input representations. 1
4 0.41352874 87 nips-2000-Modelling Spatial Recall, Mental Imagery and Neglect
Author: Suzanna Becker, Neil Burgess
Abstract: We present a computational model of the neural mechanisms in the parietal and temporal lobes that support spatial navigation, recall of scenes and imagery of the products of recall. Long term representations are stored in the hippocampus, and are associated with local spatial and object-related features in the parahippocampal region. Viewer-centered representations are dynamically generated from long term memory in the parietal part of the model. The model thereby simulates recall and imagery of locations and objects in complex environments. After parietal damage, the model exhibits hemispatial neglect in mental imagery that rotates with the imagined perspective of the observer, as in the famous Milan Square experiment [1]. Our model makes novel predictions for the neural representations in the parahippocampal and parietal regions and for behavior in healthy volunteers and neuropsychological patients.
5 0.35017329 131 nips-2000-The Early Word Catches the Weights
Author: Mark A. Smith, Garrison W. Cottrell, Karen L. Anderson
Abstract: The strong correlation between the frequency of words and their naming latency has been well documented. However, as early as 1973, the Age of Acquisition (AoA) of a word was alleged to be the actual variable of interest, but these studies seem to have been ignored in most of the literature. Recently, there has been a resurgence of interest in AoA. While some studies have shown that frequency has no effect when AoA is controlled for, more recent studies have found independent contributions of frequency and AoA. Connectionist models have repeatedly shown strong effects of frequency, but little attention has been paid to whether they can also show AoA effects. Indeed, several researchers have explicitly claimed that they cannot show AoA effects. In this work, we explore these claims using a simple feed forward neural network. We find a significant contribution of AoA to naming latency, as well as conditions under which frequency provides an independent contribution. 1 Background Naming latency is the time between the presentation of a picture or written word and the beginning of the correct utterance of that word. It is undisputed that there are significant differences in the naming latency of many words, even when controlling word length, syllabic complexity, and other structural variants. The cause of differences in naming latency has been the subject of numerous studies. Earlier studies found that the frequency with which a word appears in spoken English is the best determinant of its naming latency (Oldfield & Wingfield, 1965). More recent psychological studies, however, show that the age at which a word is learned, or its Age of Acquisition (AoA), may be a better predictor of naming latency. Further, in many multiple regression analyses, frequency is not found to be significant when AoA is controlled for (Brown & Watson, 1987; Carroll & White, 1973; Morrison et al. 1992; Morrison & Ellis, 1995). These studies show that frequency and AoA are highly correlated (typically r =-.6) explaining the confound of older studies on frequency. However, still more recent studies question this finding and find that both AoA and frequency are significant and contribute independently to naming latency (Ellis & Morrison, 1998; Gerhand & Barry, 1998,1999). Much like their psychological counterparts, connectionist networks also show very strong frequency effects. However, the ability of a connectionist network to show AoA effects has been doubted (Gerhand & Barry, 1998; Morrison & Ellis, 1995). Most of these claims are based on the well known fact that connectionist networks exhibit
6 0.27969936 63 nips-2000-Hierarchical Memory-Based Reinforcement Learning
7 0.27406281 129 nips-2000-Temporally Dependent Plasticity: An Information Theoretic Account
8 0.26163304 56 nips-2000-Foundations for a Circuit Complexity Theory of Sensory Processing
9 0.22466895 29 nips-2000-Bayes Networks on Ice: Robotic Search for Antarctic Meteorites
11 0.20677105 138 nips-2000-The Use of Classifiers in Sequential Inference
12 0.18596117 105 nips-2000-Programmable Reinforcement Learning Agents
13 0.18509106 40 nips-2000-Dendritic Compartmentalization Could Underlie Competition and Attentional Biasing of Simultaneous Visual Stimuli
14 0.16143262 26 nips-2000-Automated State Abstraction for Options using the U-Tree Algorithm
15 0.15249091 19 nips-2000-Adaptive Object Representation with Hierarchically-Distributed Memory Sites
16 0.15077431 127 nips-2000-Structure Learning in Human Causal Induction
17 0.14823398 102 nips-2000-Position Variance, Recurrence and Perceptual Learning
18 0.14802913 136 nips-2000-The Missing Link - A Probabilistic Model of Document Content and Hypertext Connectivity
19 0.14560823 10 nips-2000-A Productive, Systematic Framework for the Representation of Visual Structure
20 0.14490953 55 nips-2000-Finding the Key to a Synapse
topicId topicWeight
[(10, 0.018), (17, 0.081), (33, 0.016), (38, 0.027), (55, 0.02), (62, 0.048), (65, 0.014), (67, 0.04), (76, 0.029), (79, 0.01), (81, 0.582), (90, 0.014)]
simIndex simValue paperId paperTitle
same-paper 1 0.96671963 66 nips-2000-Hippocampally-Dependent Consolidation in a Hierarchical Model of Neocortex
Author: Szabolcs KĂĄli, Peter Dayan
Abstract: In memory consolidation, declarative memories which initially require the hippocampus for their recall, ultimately become independent of it. Consolidation has been the focus of numerous experimental and qualitative modeling studies, but only little quantitative exploration. We present a consolidation model in which hierarchical connections in the cortex, that initially instantiate purely semantic information acquired through probabilistic unsupervised learning, come to instantiate episodic information as well. The hippocampus is responsible for helping complete partial input patterns before consolidation is complete, while also training the cortex to perform appropriate completion by itself.
2 0.92711377 141 nips-2000-Universality and Individuality in a Neural Code
Author: Elad Schneidman, Naama Brenner, Naftali Tishby, Robert R. de Ruyter van Steveninck, William Bialek
Abstract: The problem of neural coding is to understand how sequences of action potentials (spikes) are related to sensory stimuli, motor outputs, or (ultimately) thoughts and intentions. One clear question is whether the same coding rules are used by different neurons, or by corresponding neurons in different individuals. We present a quantitative formulation of this problem using ideas from information theory, and apply this approach to the analysis of experiments in the fly visual system. We find significant individual differences in the structure of the code, particularly in the way that temporal patterns of spikes are used to convey information beyond that available from variations in spike rate. On the other hand, all the flies in our ensemble exhibit a high coding efficiency, so that every spike carries the same amount of information in all the individuals. Thus the neural code has a quantifiable mixture of individuality and universality. 1
3 0.91874266 137 nips-2000-The Unscented Particle Filter
Author: Rudolph van der Merwe, Arnaud Doucet, Nando de Freitas, Eric A. Wan
Abstract: In this paper, we propose a new particle filter based on sequential importance sampling. The algorithm uses a bank of unscented filters to obtain the importance proposal distribution. This proposal has two very
4 0.85053265 103 nips-2000-Probabilistic Semantic Video Indexing
Author: Milind R. Naphade, Igor Kozintsev, Thomas S. Huang
Abstract: We propose a novel probabilistic framework for semantic video indexing. We define probabilistic multimedia objects (multijects) to map low-level media features to high-level semantic labels. A graphical network of such multijects (multinet) captures scene context by discovering intra-frame as well as inter-frame dependency relations between the concepts. The main contribution is a novel application of a factor graph framework to model this network. We model relations between semantic concepts in terms of their co-occurrence as well as the temporal dependencies between these concepts within video shots. Using the sum-product algorithm [1] for approximate or exact inference in these factor graph multinets, we attempt to correct errors made during isolated concept detection by forcing high-level constraints. This results in a significant improvement in the overall detection performance. 1
5 0.55914646 55 nips-2000-Finding the Key to a Synapse
Author: Thomas Natschläger, Wolfgang Maass
Abstract: Experimental data have shown that synapses are heterogeneous: different synapses respond with different sequences of amplitudes of postsynaptic responses to the same spike train. Neither the role of synaptic dynamics itself nor the role of the heterogeneity of synaptic dynamics for computations in neural circuits is well understood. We present in this article methods that make it feasible to compute for a given synapse with known synaptic parameters the spike train that is optimally fitted to the synapse, for example in the sense that it produces the largest sum of postsynaptic responses. To our surprise we find that most of these optimally fitted spike trains match common firing patterns of specific types of neurons that are discussed in the literature.
6 0.51783794 104 nips-2000-Processing of Time Series by Neural Circuits with Biologically Realistic Synaptic Dynamics
7 0.46632048 131 nips-2000-The Early Word Catches the Weights
8 0.45128086 146 nips-2000-What Can a Single Neuron Compute?
9 0.44132835 88 nips-2000-Multiple Timescales of Adaptation in a Neural Code
10 0.42996943 43 nips-2000-Dopamine Bonuses
11 0.38420472 80 nips-2000-Learning Switching Linear Models of Human Motion
12 0.37323156 129 nips-2000-Temporally Dependent Plasticity: An Information Theoretic Account
13 0.36709797 49 nips-2000-Explaining Away in Weight Space
14 0.36595657 124 nips-2000-Spike-Timing-Dependent Learning for Oscillatory Networks
15 0.35797748 71 nips-2000-Interactive Parts Model: An Application to Recognition of On-line Cursive Script
16 0.35746902 89 nips-2000-Natural Sound Statistics and Divisive Normalization in the Auditory System
17 0.34525922 90 nips-2000-New Approaches Towards Robust and Adaptive Speech Recognition
18 0.33949041 72 nips-2000-Keeping Flexible Active Contours on Track using Metropolis Updates
19 0.33837327 6 nips-2000-A Neural Probabilistic Language Model
20 0.3379761 30 nips-2000-Bayesian Video Shot Segmentation