nips nips2013 nips2013-15 knowledge-graph by maker-knowledge-mining

15 nips-2013-A memory frontier for complex synapses

Source: pdf

Author: Subhaneil Lahiri, Surya Ganguli

Abstract: An incredible gulf separates theoretical models of synapses, often described solely by a single scalar value denoting the size of a postsynaptic potential, from the immense complexity of molecular signaling pathways underlying real synapses. To understand the functional contribution of such molecular complexity to learning and memory, it is essential to expand our theoretical conception of a synapse from a single scalar to an entire dynamical system with many internal molecular functional states. Moreover, theoretical considerations alone demand such an expansion; network models with scalar synapses assuming ﬁnite numbers of distinguishable synaptic strengths have strikingly limited memory capacity. This raises the fundamental question, how does synaptic complexity give rise to memory? To address this, we develop new mathematical theorems elucidating the relationship between the structural organization and memory properties of complex synapses that are themselves molecular networks. Moreover, in proving such theorems, we uncover a framework, based on ﬁrst passage time theory, to impose an order on the internal states of complex synaptic models, thereby simplifying the relationship between synaptic structure and function. 1

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 A memory frontier for complex synapses Subhaneil Lahiri and Surya Ganguli Department of Applied Physics, Stanford University, Stanford CA sulahiri@stanford. [sent-1, score-0.737]

2 To understand the functional contribution of such molecular complexity to learning and memory, it is essential to expand our theoretical conception of a synapse from a single scalar to an entire dynamical system with many internal molecular functional states. [sent-4, score-0.549]

3 Moreover, theoretical considerations alone demand such an expansion; network models with scalar synapses assuming ﬁnite numbers of distinguishable synaptic strengths have strikingly limited memory capacity. [sent-5, score-1.255]

4 This raises the fundamental question, how does synaptic complexity give rise to memory? [sent-6, score-0.485]

5 To address this, we develop new mathematical theorems elucidating the relationship between the structural organization and memory properties of complex synapses that are themselves molecular networks. [sent-7, score-0.848]

6 Moreover, in proving such theorems, we uncover a framework, based on ﬁrst passage time theory, to impose an order on the internal states of complex synaptic models, thereby simplifying the relationship between synaptic structure and function. [sent-8, score-1.217]

7 1 Introduction It is widely thought that our very ability to remember the past over long time scales depends crucially on our ability to modify synapses in our brain in an experience dependent manner. [sent-9, score-0.401]

8 Classical models of synaptic plasticity model synaptic efﬁcacy as an analog scalar value, denoting the size of a postsynaptic potential injected into one neuron from another. [sent-10, score-1.191]

9 Theoretical work has shown that such models have a reasonable, extensive memory capacity, in which the number of long term associations that can be stored by a neuron is proportional its number of afferent synapses [1–3]. [sent-11, score-0.828]

10 However, recent experimental work has shown that many synapses are more digital than analog; they cannot robustly assume an inﬁnite continuum of analog values, but rather can only take on a ﬁnite number of distinguishable strengths, a number than can be as small as two [4–6] (though see [7]). [sent-12, score-0.426]

11 Intuitively, when synapses are digital, the storage of a new memory can ﬂip a population of synaptic switches, thereby rapidly erasing previous memories stored in the same synaptic population. [sent-14, score-1.908]

12 This result indicates that the dominant theoretical basis for the storage of long term memories in modiﬁable synaptic switches is ﬂawed. [sent-15, score-0.66]

13 Recent work [10–12] has suggested that a way out of this logarithmic catastrophe is to expand our theoretical conception of a synapse from a single scalar value to an entire stochastic dynamical system in its own right. [sent-16, score-0.274]

14 This conceptual expansion is further necessitated by the experimental reality that synapses contain within them immensely complex molecular signaling pathways, with many internal molecular functional states (e. [sent-17, score-0.772]

15 While externally, synaptic efﬁcacy could be digital, candidate patterns of electrical activity leading to potentiation or depression could yield transitions between these internal molecular states without necessarily inducing an associated change in 1 synaptic efﬁcacy. [sent-20, score-1.582]

16 This form of synaptic change, known as metaplasticity [15, 16], can allow the probability of synaptic potentiation or depression to acquire a rich dependence on the history of prior changes in efﬁcacy, thereby potentially improving memory capacity. [sent-21, score-1.648]

17 While these models can vastly outperform simple binary synaptic switches, these analyses leave open several deep and important questions. [sent-25, score-0.516]

18 For example, how does the structure of a synaptic dynamical system determine its memory performance? [sent-26, score-0.898]

19 What are the fundamental limits of memory performance over the space of all possible synaptic dynamical systems? [sent-27, score-0.93]

20 What is the structural organization of synaptic dynamical systems that achieve these limits? [sent-28, score-0.586]

21 In order to elucidate the functional contribution of this diverse molecular complexity to learning and memory, it is essential to move beyond the analysis of speciﬁc models and instead develop a general theory of learning and memory for complex synapses. [sent-30, score-0.512]

22 Moreover, such a general theory of complex synapses could aid in development of novel artiﬁcial memory storage devices. [sent-31, score-0.791]

23 Here we initiate such a general theory by proving upper bounds on the memory curve associated with any synaptic dynamical system, within the well established ideal observer framework of [10, 11, 18]. [sent-32, score-1.098]

24 Along the way we develop principles based on ﬁrst passage time theory to order the structure of synaptic dynamical systems and relate this structure to memory performance. [sent-33, score-0.975]

25 2 Overall framework: synaptic models and their memory curves In this section, we describe the class of models of synaptic plasticity that we are studying and how we quantify their memory performance. [sent-35, score-1.796]

26 We use a well established formalism for the study of learning and memory with complex synapses (see [10, 11, 18]). [sent-37, score-0.737]

27 In this approach, electrical patterns of activity corresponding to candidate potentiating and depressing plasticity events occur randomly and independently at all synapses at a Poisson rate r. [sent-38, score-0.596]

28 These events reﬂect possible synaptic changes due to either spontaneous network activity, or the storage of new memories. [sent-39, score-0.58]

29 We let f pot and f dep denote the fraction of these events that are candidate potentiating or depressing events respectively. [sent-40, score-0.466]

30 Furthermore, we assume our synaptic model has M internal molecular functional states, and that a candidate potentiating (depotentiating) event induces a stochastic transition in the internal state described by an M × M discrete time Markov transition matrix Mpot (Mdep ). [sent-41, score-0.93]

31 In this framework, the states of different synapses will be independent, and the entire synaptic population can be fully described by the probability distribution across these states, which we will indicate with the row-vector p(t). [sent-42, score-0.983]

32 Thus the i’th component of p(t) denotes the fraction of the synaptic population in state i. [sent-43, score-0.544]

33 Furthermore, each state i has its own synaptic weight, wi , which we take, in the worst case scenario, to be restricted to two values. [sent-44, score-0.518]

34 We also employ an “ideal observer” approach to the memory readout, where the synaptic weights are read directly. [sent-46, score-0.819]

35 We assume that any pattern of synaptic weights close to wideal is sufﬁcient to recall the memory. [sent-49, score-0.6]

36 However, the actual pattern of synaptic weights at some later time, t, will change to w(t) due to further modiﬁcations from the storage of subsequent memories. [sent-50, score-0.567]

37 (a) The cascade model of [10], showing transitions between states of high/low synaptic weight (red/blue circles) due to potentiation/depression (solid red/dashed blue arrows). [sent-53, score-0.637]

38 (c) The memory curves of these two models, showing the decay of the signal-to-noise ratio (to be deﬁned in §2) as subsequent memories are stored. [sent-55, score-0.503]

39 The extent to which the memory has been stored is described by a signal-to-noise ratio (SNR) [10, 11]: wideal · w(t) − wideal · w(∞) SNR(t) = . [sent-58, score-0.62]

40 There is a correction when potentiation and depression are imbalanced, but this will not affect the upper bounds that we will discuss below and will be ignored in the subsequent formulae. [sent-60, score-0.37]

41 A simple average memory curve can be derived as follows. [sent-61, score-0.441]

42 All of the preceding plasticity events, prior to t = 0, will put the population of synapses in its steady-state distribution, p∞ . [sent-62, score-0.474]

43 The memory we are tracking at t = 0 will change the internal state distribution to p∞ Mpot (or p∞ Mdep ) in those synapses that experience a candidate potentiation (or depression) event. [sent-63, score-1.035]

44 As the potentiating/depressing nature of the subsequent memories is independent of wideal , we can average over all sequences, resulting in the evolution of the probability distribution: dp(t) = rp(t)WF , where WF = f pot Mpot + f dep Mdep − I. [sent-64, score-0.514]

45 (2) dt Here WF is a continuous time transition matrix that models the process of forgetting the memory stored at time t = 0 due to random candidate potentiation/depression events occurring at each synapse due to the storage of subsequent memories. [sent-65, score-0.711]

46 This results in the following SNR √ F (3) SNR(t) = N 2f pot f dep p∞ Mpot − Mdep ertW w. [sent-67, score-0.277]

47 We will frequently refer to this function as the memory curve. [sent-69, score-0.334]

48 It can be thought of as the excess fraction of synapses (relative to equilibrium) that maintain their ideal synaptic strength at time t, as dictated by the stored memory at time t = 0. [sent-70, score-1.277]

49 Much of the previous work on these types of complex synaptic models has focused on understanding the memory curves of speciﬁc models, or choices of Mpot/dep . [sent-71, score-0.901]

50 We consider the entire space of these models and ﬁnd upper bounds on the memory capacity of any of them. [sent-77, score-0.464]

51 i f pot + f dep = 1, j i 3 wi = ±1, (4) The upper bounds on Mpot/dep and f pot/dep follow automatically from the other constraints. [sent-80, score-0.313]

52 ij The critical question is: what do these constraints imply about the space of achievable memory curves in (3)? [sent-81, score-0.464]

53 To answer this question, especially for limits on achievable memory at ﬁnite times, it will be useful to employ the eigenmode decomposition: WF = −qa ua va , va ub = δab , WF ua = −qa ua , va WF = −qa va . [sent-82, score-0.845]

54 This decomposition allows us to write the memory curve as a sum of exponentials, √ SNR(t) = N Ia e−rt/τa , (6) a where Ia = (2f pot f dep )p∞ (Mpot − Mdep )ua va w and τa = 1/qa . [sent-84, score-0.774]

55 3 Upper bounds on achievable memory capacity In the previous section, in (3) we have described an analytic expression for a memory curve as a function of the structure of a synaptic dynamical system, described by the pair of stochastic transition matrices Mpot/dep . [sent-87, score-1.469]

56 Since the performance measure for memory is an entire memory curve, and not just a single number, there is no universal scalar notion of optimal memory in the space of synaptic dynamical systems. [sent-88, score-1.622]

57 Instead there are tradeoffs between storing proximal and distal memories; often attempts to increase memory at late (early) times by changing Mpot/dep , incurs a performance loss in memory at early (late) times in speciﬁc models considered so far [10–12]. [sent-89, score-0.899]

58 Thus our end goal, achieved in §4, is to derive an envelope memory curve in the SNR-time plane, or a curve that forms an upper-bound on the entire memory curve of any model. [sent-90, score-1.141]

59 In order to achieve this goal, in this section, we must ﬁrst derive upper bounds, over the space of all possible synaptic models, on two different scalar functions of the memory curve: its initial SNR, and the area under the memory curve. [sent-91, score-1.327]

60 In the process of upper-bounding the area, we will develop an essential framework to organize the structure of synaptic dynamical systems based on ﬁrst passage time theory. [sent-92, score-0.641]

61 1 Bounding initial SNR We now give an upper bound on the initial SNR, √ SNR(0) = N 2f pot f dep p∞ Mpot − Mdep w, (7) over all possible models and also ﬁnd the class of models that saturate this bound. [sent-94, score-0.45]

62 (9) r This inequality becomes an equality if potentiation never decreases the synaptic weight and depression never increases it, which should be a property of any sensible model. [sent-97, score-0.791]

63 To maximize this ﬂux, potentiation from a weak state must be guaranteed to end in a strong state, and depression must do the reverse. [sent-98, score-0.359]

64 have the same memory curve) as a two state model with transition probabilities equal to 1, as shown in Figure 2(c). [sent-104, score-0.407]

65 (c) The equivalent two state model, with transition probabilities under potentiation and depression equal to one. [sent-108, score-0.379]

66 This two state model has the equilibrium distribution p∞ = (f dep , f pot ) and its ﬂux is given by Φ−+ = rf pot f dep . [sent-109, score-0.654]

67 This is maximized when f pot = f dep = 1 , leading to the upper bound: 2 √ SNR(0) ≤ N . [sent-110, score-0.313]

68 (10) We note that while this model has high initial SNR, it also has very fast memory decay – with a timescale τ ∼ 1 . [sent-111, score-0.407]

69 As the synapse is very plastic, the initial memory is encoded very easily, but r the subsequent memories also overwrite it rapidly. [sent-112, score-0.57]

70 This is one example of the tradeoff between optimizing memory at early versus late times. [sent-113, score-0.464]

71 In this order, which depends sensitively on the structure of Mpot/dep , states later (to the right in ﬁgures below) can be considered to be more potentiated than states earlier (to the left in ﬁgures below), despite the fact that they have the same synaptic efﬁcacy. [sent-126, score-0.688]

72 We will see that synaptic models that optimize various measures of memory have an exceedingly simple structure when, and only when, their states are arranged in this order. [sent-128, score-0.923]

73 3 Bounding area Now consider the area under the memory curve: ∞ A= dt SNR(t). [sent-143, score-0.494]

74 ηi > ηj ), pot dep we have proven that increasing Mij and decreasing Mij leads to an increase in area. [sent-150, score-0.297]

75 It is of the form shown in Figure 4(c),with potentiation moving one step to the right and depression moving one step to the left. [sent-153, score-0.306]

76 As these perturbations do not change the equilibrium distribution, this means that the area of any model is bounded by that of a linear chain with the same equilibrium distribution. [sent-157, score-0.294]

77 (16) A≤ r This analytical result is similar to a numerical result found in [18] under a slightly different information theoretic measure of memory performance. [sent-163, score-0.334]

78 6 The “sticky” end states result in very slow decay of memory, but they also make it difﬁcult to encode the memory in the ﬁrst place, since a small fraction of synapses are able to change synaptic efﬁcacy during the storage of a new memory. [sent-164, score-1.345]

79 Thus models that maximize area optimize memory at late times, at the expense of early times. [sent-165, score-0.575]

80 4 Memory curve envelope Now we will look at the implications of the upper bounds found in the previous section for the SNR at ﬁnite times. [sent-166, score-0.27]

81 As argued in (6), the memory curve can be written in the form √ SNR(t) = N Ia e−rt/τa . [sent-167, score-0.441]

82 (18) a a We are not claiming that these are a complete set of constraints: not every set {Ia , τa } that satisﬁes these inequalities will actually be achievable by a synaptic model. [sent-169, score-0.537]

83 This always results in a single nonzero Ia ; in essence, optimizing memory at a single time requires a single exponential. [sent-173, score-0.334]

84 The resulting optimal memory curve, along with the achieved memory at the chosen time, depends on t0 as follows: √ √ M −1 t0 ≤ =⇒ SNR(t) = N e−rt/(M −1) =⇒ SNR(t0 ) = N e−rt0 /(M −1) , r √ √ (19) M −1 N (M − 1)e−t/t0 N (M − 1) t0 ≥ =⇒ SNR(t) = =⇒ SNR(t0 ) = . [sent-174, score-0.668]

85 The function SNR(t0 ), the green curve in Figure 4(a), above forms a memory curve envelope with late-time power-law decay ∼ t−1 . [sent-177, score-0.7]

86 No synaptic model can have an 0 SNR that is greater than this at any time. [sent-178, score-0.485]

87 We can use this to ﬁnd an upper bound on the memory lifetime, τ ( ), by ﬁnding the point at which the envelope crosses : √ N (M − 1) , (20) τ( ) ≤ er where we assume N > ( e)2 . [sent-179, score-0.518]

88 Intriguingly, both the lifetime and memory envelope expand linearly with the number of internal states M , and increase as the square root of the number of synapses N . [sent-180, score-1.059]

89 At any time, can we ﬁnd a model whose memory curve touches the envelope? [sent-182, score-0.441]

90 However, we suspect that the area constraint is not the bottleneck for optimizing memory at times less than O( M ). [sent-186, score-0.439]

91 Nevertheless, we have proven rigorously that no model’s memory curve can ever exceed this envelope, and that it is at least tight for late times, longer than O( M ), r where models of the form in Figure 4(c)can come close to the envelope. [sent-188, score-0.595]

92 The red dashed curve shows the result of numerical optimization of synaptic models with random initialization. [sent-191, score-0.623]

93 In doing so, we have obtained several new mathematical results delineating the functional limits of memory achievable by synaptic complexity, and the structural characterization of synaptic dynamical systems that achieve these limits. [sent-197, score-1.496]

94 We have also derived memory constrained envelopes, by asking in the space of models that achieve a given SNR at a given time, what is the maximal SNR achievable at other times. [sent-201, score-0.417]

95 It would be interesting to systematically analyze the space of models that achieve good memory at multiple times, and understand their structural organization, and how they give rise to multiple exponentials, leading to power law memory decays. [sent-203, score-0.699]

96 Then given our theory, we could match this measured synaptic model to optimal models to understand for which timescales of memory, if any, biological synaptic dynamics may be tuned. [sent-205, score-1.001]

97 Collingridge, “A synaptic model of memory: long-term potentiation in the hippocampus,” Nature 361 (Jan, 1993) 31–39. [sent-235, score-0.649]

98 Wang, “Graded bidirectional synaptic plasticity is composed of switch-like unitary events,” Proc. [sent-261, score-0.559]

99 Abbott, “Limits on the memory storage capacity of bounded synapses,” Nat. [sent-295, score-0.426]

100 Bear, “Metaplasticity: the plasticity of synaptic plasticity,” Trends in Neurosciences 19 (1996) no. [sent-331, score-0.559]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('synaptic', 0.485), ('synapses', 0.374), ('snr', 0.336), ('memory', 0.334), ('mdep', 0.172), ('mpot', 0.172), ('potentiation', 0.164), ('dep', 0.152), ('depression', 0.142), ('envelope', 0.127), ('pot', 0.125), ('wideal', 0.115), ('ia', 0.11), ('curve', 0.107), ('memories', 0.094), ('late', 0.093), ('wf', 0.09), ('molecular', 0.089), ('area', 0.08), ('dynamical', 0.079), ('passage', 0.077), ('plasticity', 0.074), ('states', 0.073), ('internal', 0.068), ('equilibrium', 0.067), ('synapse', 0.062), ('ua', 0.058), ('potentiated', 0.057), ('stored', 0.056), ('va', 0.056), ('perturbations', 0.055), ('fusi', 0.055), ('storage', 0.054), ('achievable', 0.052), ('potentiating', 0.043), ('transitions', 0.041), ('events', 0.041), ('transition', 0.04), ('qa', 0.04), ('lifetime', 0.04), ('tij', 0.04), ('cacy', 0.039), ('metaplasticity', 0.038), ('cascade', 0.038), ('capacity', 0.038), ('early', 0.037), ('ux', 0.036), ('upper', 0.036), ('candidate', 0.035), ('ij', 0.034), ('neuron', 0.033), ('serial', 0.033), ('state', 0.033), ('limits', 0.032), ('abbott', 0.031), ('postsynaptic', 0.031), ('models', 0.031), ('scalar', 0.031), ('digital', 0.031), ('forgetting', 0.03), ('exceed', 0.03), ('functional', 0.029), ('complex', 0.029), ('observer', 0.029), ('catastrophe', 0.029), ('depressing', 0.029), ('eigenmode', 0.029), ('incredible', 0.029), ('kemeny', 0.029), ('lumpability', 0.029), ('nicoll', 0.029), ('ideal', 0.028), ('subsequent', 0.028), ('experience', 0.027), ('switches', 0.027), ('initial', 0.027), ('population', 0.026), ('conception', 0.025), ('graded', 0.025), ('overwrite', 0.025), ('entire', 0.025), ('times', 0.025), ('chain', 0.025), ('decay', 0.025), ('amit', 0.024), ('readout', 0.023), ('expand', 0.023), ('curves', 0.022), ('organization', 0.022), ('constraints', 0.022), ('bound', 0.021), ('analog', 0.021), ('signaling', 0.021), ('exponentials', 0.021), ('pathways', 0.021), ('saturates', 0.021), ('timescale', 0.021), ('topology', 0.021), ('weak', 0.02), ('increase', 0.02)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 1.0000004 15 nips-2013-A memory frontier for complex synapses

Author: Subhaneil Lahiri, Surya Ganguli

2 0.48322684 77 nips-2013-Correlations strike back (again): the case of associative memory retrieval

Author: Cristina Savin, Peter Dayan, Mate Lengyel

Abstract: It has long been recognised that statistical dependencies in neuronal activity need to be taken into account when decoding stimuli encoded in a neural population. Less studied, though equally pernicious, is the need to take account of dependencies between synaptic weights when decoding patterns previously encoded in an auto-associative memory. We show that activity-dependent learning generically produces such correlations, and failing to take them into account in the dynamics of memory retrieval leads to catastrophically poor recall. We derive optimal network dynamics for recall in the face of synaptic correlations caused by a range of synaptic plasticity rules. These dynamics involve well-studied circuit motifs, such as forms of feedback inhibition and experimentally observed dendritic nonlinearities. We therefore show how addressing the problem of synaptic correlations leads to a novel functional account of key biophysical features of the neural substrate. 1

3 0.20690006 246 nips-2013-Perfect Associative Learning with Spike-Timing-Dependent Plasticity

Author: Christian Albers, Maren Westkott, Klaus Pawelzik

Abstract: Recent extensions of the Perceptron as the Tempotron and the Chronotron suggest that this theoretical concept is highly relevant for understanding networks of spiking neurons in the brain. It is not known, however, how the computational power of the Perceptron might be accomplished by the plasticity mechanisms of real synapses. Here we prove that spike-timing-dependent plasticity having an anti-Hebbian form for excitatory synapses as well as a spike-timing-dependent plasticity of Hebbian shape for inhibitory synapses are sufﬁcient for realizing the original Perceptron Learning Rule if these respective plasticity mechanisms act in concert with the hyperpolarisation of the post-synaptic neurons. We also show that with these simple yet biologically realistic dynamics Tempotrons and Chronotrons are learned. The proposed mechanism enables incremental associative learning from a continuous stream of patterns and might therefore underly the acquisition of long term memories in cortex. Our results underline that learning processes in realistic networks of spiking neurons depend crucially on the interactions of synaptic plasticity mechanisms with the dynamics of participating neurons.

4 0.16776522 49 nips-2013-Bayesian Inference and Online Experimental Design for Mapping Neural Microcircuits

Author: Ben Shababo, Brooks Paige, Ari Pakman, Liam Paninski

Abstract: With the advent of modern stimulation techniques in neuroscience, the opportunity arises to map neuron to neuron connectivity. In this work, we develop a method for efﬁciently inferring posterior distributions over synaptic strengths in neural microcircuits. The input to our algorithm is data from experiments in which action potentials from putative presynaptic neurons can be evoked while a subthreshold recording is made from a single postsynaptic neuron. We present a realistic statistical model which accounts for the main sources of variability in this experiment and allows for signiﬁcant prior information about the connectivity and neuronal cell types to be incorporated if available. Due to the technical challenges and sparsity of these systems, it is important to focus experimental time stimulating the neurons whose synaptic strength is most ambiguous, therefore we also develop an online optimal design algorithm for choosing which neurons to stimulate at each trial. 1

5 0.14662914 210 nips-2013-Noise-Enhanced Associative Memories

Author: Amin Karbasi, Amir Hesam Salavati, Amin Shokrollahi, Lav R. Varshney

Abstract: Recent advances in associative memory design through structured pattern sets and graph-based inference algorithms allow reliable learning and recall of exponential numbers of patterns. Though these designs correct external errors in recall, they assume neurons compute noiselessly, in contrast to highly variable neurons in hippocampus and olfactory cortex. Here we consider associative memories with noisy internal computations and analytically characterize performance. As long as internal noise is less than a speciﬁed threshold, error probability in the recall phase can be made exceedingly small. More surprisingly, we show internal noise actually improves performance of the recall phase. Computational experiments lend additional support to our theoretical analysis. This work suggests a functional beneﬁt to noisy neurons in biological neuronal networks. 1

6 0.1192473 286 nips-2013-Robust learning of low-dimensional dynamics from large neural ensembles

7 0.09558022 267 nips-2013-Recurrent networks of coupled Winner-Take-All oscillators for solving constraint satisfaction problems

8 0.087641418 188 nips-2013-Memory Limited, Streaming PCA

9 0.078811996 304 nips-2013-Sparse nonnegative deconvolution for compressive calcium imaging: algorithms and phase transitions

10 0.074297838 254 nips-2013-Probabilistic Low-Rank Matrix Completion with Adaptive Spectral Regularization Algorithms

11 0.061768413 61 nips-2013-Capacity of strong attractor patterns to model behavioural and cognitive prototypes

12 0.046959702 121 nips-2013-Firing rate predictions in optimal balanced networks

13 0.046071399 231 nips-2013-Online Learning with Switching Costs and Other Adaptive Adversaries

14 0.045457661 155 nips-2013-Learning Hidden Markov Models from Non-sequence Data via Tensor Decomposition

15 0.043479901 141 nips-2013-Inferring neural population dynamics from multiple partial recordings of the same neural circuit

16 0.04144042 4 nips-2013-A Comparative Framework for Preconditioned Lasso Algorithms

17 0.038200926 146 nips-2013-Large Scale Distributed Sparse Precision Estimation

18 0.037889093 236 nips-2013-Optimal Neural Population Codes for High-dimensional Stimulus Variables

19 0.037645899 45 nips-2013-BIG & QUIC: Sparse Inverse Covariance Estimation for a Million Variables

20 0.036857586 334 nips-2013-Training and Analysing Deep Recurrent Neural Networks

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, 0.115), (1, 0.039), (2, -0.054), (3, -0.042), (4, -0.229), (5, -0.026), (6, -0.026), (7, -0.092), (8, 0.027), (9, 0.013), (10, 0.073), (11, 0.014), (12, 0.134), (13, 0.054), (14, -0.025), (15, -0.026), (16, -0.032), (17, 0.065), (18, 0.042), (19, -0.057), (20, 0.01), (21, -0.078), (22, 0.074), (23, 0.383), (24, -0.046), (25, 0.285), (26, 0.117), (27, -0.064), (28, -0.193), (29, -0.018), (30, 0.144), (31, 0.091), (32, 0.062), (33, -0.004), (34, 0.029), (35, 0.008), (36, 0.022), (37, 0.091), (38, -0.051), (39, 0.087), (40, 0.126), (41, -0.07), (42, -0.03), (43, -0.159), (44, -0.028), (45, 0.051), (46, 0.028), (47, 0.009), (48, -0.019), (49, -0.091)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.97051823 15 nips-2013-A memory frontier for complex synapses

Author: Subhaneil Lahiri, Surya Ganguli

2 0.90811479 77 nips-2013-Correlations strike back (again): the case of associative memory retrieval

Author: Cristina Savin, Peter Dayan, Mate Lengyel

3 0.73891765 246 nips-2013-Perfect Associative Learning with Spike-Timing-Dependent Plasticity

Author: Christian Albers, Maren Westkott, Klaus Pawelzik

4 0.63870919 61 nips-2013-Capacity of strong attractor patterns to model behavioural and cognitive prototypes

Author: Abbas Edalat

Abstract: We solve the mean ﬁeld equations for a stochastic Hopﬁeld network with temperature (noise) in the presence of strong, i.e., multiply stored, patterns, and use this solution to obtain the storage capacity of such a network. Our result provides for the ﬁrst time a rigorous solution of the mean ﬁled equations for the standard Hopﬁeld model and is in contrast to the mathematically unjustiﬁable replica technique that has been used hitherto for this derivation. We show that the critical temperature for stability of a strong pattern is equal to its degree or multiplicity, when the sum of the squares of degrees of the patterns is negligible compared to the network size. In the case of a single strong pattern, when the ratio of the number of all stored pattens and the network size is a positive constant, we obtain the distribution of the overlaps of the patterns with the mean ﬁeld and deduce that the storage capacity for retrieving a strong pattern exceeds that for retrieving a simple pattern by a multiplicative factor equal to the square of the degree of the strong pattern. This square law property provides justiﬁcation for using strong patterns to model attachment types and behavioural prototypes in psychology and psychotherapy. 1 Introduction: Multiply learned patterns in Hopﬁeld networks The Hopﬁeld network as a model of associative memory and unsupervised learning was introduced in [23] and has been intensively studied from a wide range of viewpoints in the past thirty years. However, properties of a strong pattern, as a pattern that has been multiply stored or learned in these networks, have only been examined very recently, a surprising delay given that repetition of an activity is the basis of learning by the Hebbian rule and long term potentiation. In particular, while the storage capacity of a Hopﬁeld network with certain correlated patterns has been tackled [13, 25], the storage capacity of a Hopﬁeld network in the presence of strong as well as random patterns has not been hitherto addressed. The notion of a strong pattern of a Hopﬁeld network has been proposed in [15] to model attachment types and behavioural prototypes in developmental psychology and psychotherapy. This suggestion has been motivated by reviewing the pioneering work of Bowlby [9] in attachment theory and highlighting how a number of academic biologists, psychiatrists, psychologists, sociologists and neuroscientists have consistently regarded Hopﬁeld-like artiﬁcial neural networks as suitable tools to model cognitive and behavioural constructs as patterns that are deeply and repeatedly learned by individuals [11, 22, 24, 30, 29, 10]. A number of mathematical properties of strong patterns in Hopﬁeld networks, which give rise to strong attractors, have been derived in [15]. These show in particular that strong attractors are strongly stable; a series of experiments have also been carried out which conﬁrm the mathematical 1 results and also indicate that a strong pattern stored in the network can be retrieved even in the presence of a large number of simple patterns, far exceeding the well-known maximum load parameter or storage capacity of the Hopﬁeld network with random patterns (αc ≈ 0.138). In this paper, we consider strong patterns in stochastic Hopﬁeld model with temperature, which accounts for various types of noise in the network. In these networks, the updating rule is probabilistic and depend on the temperature. Since analytical solution of such a system is not possible in general, one strives to obtain the average behaviour of the network when the input to each node, the so-called ﬁeld at the node, is replaced with its mean. This is the basis of mean ﬁeld theory for these networks. Due to the close connection between the Hopﬁeld network and the Ising model in ferromagnetism [1, 8], the mean ﬁeld approach for the Hopﬁeld network and its variations has been tackled using the replica method, starting with the pioneering work of Amit, Gutfreund and Sompolinsky [3, 2, 4, 19, 31, 1, 13]. Although this method has been widely used in the theory of spin glasses in statistical physics [26, 16] its mathematical justiﬁcation has proved to be elusive as we will discuss in the next section; see for example [20, page 264], [14, page 27], and [7, page 9]. In [17] and independently in [27], an alternative technique to the replica method for solving the mean ﬁeld equations has been proposed which is reproduced and characterised as heuristic in [20, section 2.5] since it relies on a number of assumptions that are not later justiﬁed and uses a number of mathematical steps that are not validated. Here, we use the basic idea of the above heuristic to develop a veriﬁable mathematical framework with provable results grounded on elements of probability theory, with which we assume the reader is familiar. This technique allows us to solve the mean ﬁeld equations for the Hopﬁeld network in the presence of strong patterns and use the results to study, ﬁrst, the stability of these patterns in the presence of temperature (noise) and, second, the storage capacity of the network with a single strong pattern at temperature zero. We show that the critical temperature for the stability of a strong pattern is equal to its degree (i.e., its multiplicity) when the ratio of the sum of the squares of degrees of the patterns to the network size tends to zero when the latter tends to inﬁnity. In the case that there is only one strong pattern present with its degree small compared to the number of patterns and the latter is a ﬁxed multiple of the number of nodes, we ﬁnd the distribution of the overlap of the mean ﬁeld and the patterns when the strong pattern is being retrieved. We use these distributions to prove that the storage capacity for retrieving a strong pattern exceeds that for a simple pattern by a multiplicative factor equal to the square of the degree of the strong attractor. This result matches the ﬁnding in [15] regarding the capacity of a network to recall strong patterns as mentioned above. Our results therefore show that strong patterns are robust and persistent in the network memory as attachment types and behavioural prototypes are in the human memory system. In this paper, we will several times use Lyapunov’s theorem in probability which provides a simple sufﬁcient condition to generalise the Central Limit theorem when we deal with independent but not necessarily identically distributed random variables. We require a general form of this theorem kn as follows. Let Yn = N i=1 Yni , for n ∈ I , be a triangular array of random variables such that for each n, the random variables Yni , for 1 ≤ i ≤ kn are independent with E(Yni ) = 0 2 2 and E(Yni ) = σni , where E(X) stands for the expected value of the random variable X. Let kn 2 2 sn = i=1 σni . We use the notation X ∼ Y when the two random variables X and Y have the same distribution (for large n if either or both of them depend on n). Theorem 1.1 (Lyapunov’s theorem [6, page 368]) If for some δ > 0, we have the condition: 1 E(|Yn |2+δ |) → 0 s2+δ n d d as n → ∞ then s1 Yn −→ N (0, 1) as n → ∞ where −→ denotes convergence in distribution, and we denote n by N (a, σ 2 ) the normal distribution with mean a and variance σ 2 . Thus, for large n we have Yn ∼ N (0, s2 ). n 2 2 Mean ﬁeld theory We consider a Hopﬁeld network with N neurons i = 1, . . . , N with values Si = ±1 and follow the notations in [20]. As in [15], we assume patterns can be multiply stored and the degree of a pattern is deﬁned as its multiplicity. The total number of patterns, counting their multiplicity, is denoted by p and we assume there are n patterns ξ 1 , . . . , ξ n with degrees d1 , . . . , dn ≥ 1 respectively and that n the remaining p − k=1 dk ≥ 0 patterns are simple, i.e., each has degree one. Note that by our assumptions there are precisely n p0 = p + n − dk k=1 distinct patterns, which we assume are independent and identically distributed with equal probability of taking value ±1 for each node. More generally, for any non-negative integer k ∈ I , we let N p0 dk . µ pk = µ=1 p µ µ 0 1 We use the generalized Hebbian rule for the synaptic couplings: wij = N µ=1 dµ ξi ξj for i = j with wii = 0 for 1 ≤ i, j ≤ N . As in the standard stochastic Hopﬁeld model [20], we use Glauber dynamics [18] for the stochastic updating rule with pseudo-temperature T > 0, which accounts for various types of noise in the network, and assume zero bias in the local ﬁeld. Putting β = 1/T (i.e., with the Boltzmann constant kB = 1) and letting fβ (h) = 1/(1 + exp(−2βh)), the stochastic updating rule at time t is given by: N Pr(Si (t + 1) = ±1) = fβ (±hi (t)), where hi (t) = wij Sj (t), (1) j=1 is the local ﬁeld at i at time t. The updating is implemented asynchronously in a random way. The energy of the network in the conﬁguration S = (Si )N is given by i=1 N 1 Si Sj wij . H(S) = − 2 i,j=1 For large N , this speciﬁes a complex system, with an underlying state space of dimension 2N , which in general cannot be solved exactly. However, mean ﬁeld theory has proved very useful in studying Hopﬁeld networks. The average updated value of Si (t + 1) in Equation (1) is Si (t + 1) = 1/(1 + e−2βhi (t) ) − 1/(1 + e2βhi (t) ) = tanh(βhi (t)), (2) where . . . denotes taking average with respect to the probability distribution in the updating rule in Equation (1). The stationary solution for the mean ﬁeld thus satisﬁes: Si = tanh(βhi ) , (3) The average overlap of pattern ξ µ with the mean ﬁeld at the nodes of the network is given by: mν = 1 N N ν ξi Si (4) i=1 The replica technique for solving the mean ﬁeld problem, used in the case p/N = α > 0 as N → ∞, seeks to obtain the average of the overlaps in Equation (4) by evaluating the partition function of the system, namely, Z = TrS exp(−βH(S)), where the trace TrS stands for taking sum over all possible conﬁgurations S = (Si )N . As it i=1 is generally the case in statistical physics, once the partition function of the system is obtained, 3 all required physical quantities can in principle be computed. However, in this case, the partition function is very difﬁcult to compute since it entails computing the average log Z of log Z, where . . . indicates averaging over the random distribution of the stored patterns ξ µ . To overcome this problem, the identity Zk − 1 log Z = lim k→0 k is used to reduce the problem to ﬁnding the average Z k of Z k , which is then computed for positive integer values of k. For such k, we have: Z k = TrS 1 TrS 2 . . . TrS k exp(−β(H(S 1 ) + H(S 1 ) + . . . + H(S k ))), where for each i = 1, . . . , k the super-scripted conﬁguration S i is a replica of the conﬁguration state. In computing the trace over each replica, various parameters are obtained and the replica symmetry condition assumes that these parameters are independent of the particular replica under consideration. Apart from this assumption, there are two basic mathematical problems with the technique which makes it unjustiﬁable [20, page 264]. Firstly, the positive integer k above is eventually treated as a real number near zero without any mathematical justiﬁcation. Secondly, the order of taking limits, in particular the order of taking the two limits k → 0 and N → ∞, are several times interchanged again without any mathematical justiﬁcation. Here, we develop a mathematically rigorous method for solving the mean ﬁeld problem, i.e., computing the average of the overlaps in Equation (4) in the case of p/N = α > 0 as N → ∞. Our method turns the basic idea of the heuristic presented in [17] and reproduced in [20] for solving the mean ﬁeld equation into a mathematically veriﬁable formalism, which for the standard Hopﬁeld network with random stored patterns gives the same result as the replica method, assuming replica symmetry. In the presence of strong patterns we obtain a set of new results as explained in the next two sections. The mean ﬁeld equation is obtained from Equation (3) by approximating the right hand side of N this equation by the value of tanh at the mean ﬁeld hi = j=1 wij Sj , ignoring the sum N j=1 wij (Sj − Sj ) for large N [17, page 32]: Si = tanh(β hi ) = tanh β N N j=1 p0 µ=1 µ µ dµ ξi ξj Sj . (5) Equation (5) gives the mean ﬁeld equation for the Hopﬁeld network with n possible strong patterns n ξ µ (1 ≤ µ ≤ n) and p − µ=1 dµ simple patterns ξ µ with n + 1 ≤ µ ≤ p0 . As in the standard Hopﬁeld model, where all patterns are simple, we have two cases to deal with. However, we now have to account for the presence of strong attractors and our two cases will be as follows: (i) In the p0 ﬁrst case we assume p2 := µ=1 d2 = o(N ), which includes the simpler case p2 N when p2 µ is ﬁxed and independent of N . (ii) In the second case we assume we have a single strong attractor with the load parameter p/N = α > 0. 3 Stability of strong patterns with noise: p2 = o(N ) The case of constant p and N → ∞ is usually referred to as α = 0 in the standard Hopﬁeld model. Here, we need to consider the sum of degrees of all stored patterns (and not just the number of patterns) compared to N . We solve the mean ﬁeld equation with T > 0 by using a method similar in spirit to [20, page 33] for the standard Hopﬁeld model, but in our case strong patterns induce a sequence of independent but non-identically distributed random variables in the crosstalk term, where the Central Limit Theorem cannot be used; we show however that Lyapunov’s theorem (Theorem (1.1) can be invoked. In retrieving pattern ξ 1 , we look for a solution of the mean ﬁled 1 equation of the form: Si = mξi , where m > 0 is a constant. Using Equation (5) and separating 1 the contribution of ξ in the argument of tanh, we obtain:  1 mξi = tanh    mβ  1 d1 ξi + N 4 µ µ 1 dµ ξi ξj ξj  . j=i,µ>1 (6) For each N , µ > 1 and j = i, let dµ µ µ 1 (7) ξ ξ ξ . N i j j 2 This gives (p0 − 1)(N − 1) independent random variables with E(YN µj ) = 0, E(YN µj ) = d2 /N 2 , µ 3 3 3 and E(|YN µj |) = dµ /N . We have: YN µj = s2 := N 2 E(YN µj ) = µ>1,j=i 1 N −1 d2 ∼ N 2 µ>1 µ N d2 . µ (8) µ>1 Thus, as N → ∞, we have: 1 s3 N 3 E(|YN µj |) ∼ √ µ>1,j=i µ>1 N( d3 µ µ>1 d2 )3/2 µ → 0. (9) as N → ∞ since for positive numbers dµ we always have µ>1 d3 < ( µ>1 d2 )3/2 . Thus the µ µ Lyapunov condition is satisﬁed for δ = 1. By Lyapunov’s theorem we deduce: 1 N µ µ 1 dµ ξi ξj ξj ∼ N d2 /N µ 0, (10) µ>1 µ>1,j=i Since we also have p2 = o(N ), it follows that we can ignore the second term, i.e., the crosstalk term, in the argument of tanh in Equation (6) as N → ∞; we thus obtain: m = tanh βd1 m. (11) To examine the ﬁxed points of the Equation (11), we let d = d1 for convenience and put x = βdm = dm/T , so that tanh x = T x/d; see Figure 1. It follows that Tc = d is the critical temperature. If T < d then there is a non-zero (non-trivial) solution for m, whereas for T > d we only have the trivial solution. For d = 1 our solution is that of the standard Hopﬁeld network as in [20, page 34]. (d < T) y>x y = x ( d = T) y = tanh x y

5 0.49669924 210 nips-2013-Noise-Enhanced Associative Memories

Author: Amin Karbasi, Amir Hesam Salavati, Amin Shokrollahi, Lav R. Varshney

6 0.42600882 267 nips-2013-Recurrent networks of coupled Winner-Take-All oscillators for solving constraint satisfaction problems

7 0.4155809 49 nips-2013-Bayesian Inference and Online Experimental Design for Mapping Neural Microcircuits

8 0.28075832 264 nips-2013-Reciprocally Coupled Local Estimators Implement Bayesian Information Integration Distributively

9 0.26759085 286 nips-2013-Robust learning of low-dimensional dynamics from large neural ensembles

10 0.25671932 86 nips-2013-Demixing odors - fast inference in olfaction

11 0.23048897 176 nips-2013-Linear decision rule as aspiration for simple decision heuristics

12 0.20927618 121 nips-2013-Firing rate predictions in optimal balanced networks

13 0.20157842 52 nips-2013-Bayesian inference as iterated random functions with applications to sequential inference in graphical models

14 0.1832182 141 nips-2013-Inferring neural population dynamics from multiple partial recordings of the same neural circuit

15 0.18190041 347 nips-2013-Variational Planning for Graph-based MDPs

16 0.1817068 343 nips-2013-Unsupervised Structure Learning of Stochastic And-Or Grammars

17 0.17960905 188 nips-2013-Memory Limited, Streaming PCA

18 0.17296776 37 nips-2013-Approximate Bayesian Image Interpretation using Generative Probabilistic Graphics Programs

19 0.17171988 342 nips-2013-Unsupervised Spectral Learning of Finite State Transducers

20 0.16994599 101 nips-2013-EDML for Learning Parameters in Directed and Undirected Graphical Models

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(0, 0.024), (16, 0.044), (33, 0.082), (34, 0.119), (38, 0.1), (41, 0.051), (49, 0.028), (56, 0.088), (70, 0.272), (85, 0.018), (89, 0.036), (93, 0.018)]

similar papers list:

simIndex simValue paperId paperTitle

1 0.92707419 267 nips-2013-Recurrent networks of coupled Winner-Take-All oscillators for solving constraint satisfaction problems

Author: Hesham Mostafa, Lorenz. K. Mueller, Giacomo Indiveri

Abstract: We present a recurrent neuronal network, modeled as a continuous-time dynamical system, that can solve constraint satisfaction problems. Discrete variables are represented by coupled Winner-Take-All (WTA) networks, and their values are encoded in localized patterns of oscillations that are learned by the recurrent weights in these networks. Constraints over the variables are encoded in the network connectivity. Although there are no sources of noise, the network can escape from local optima in its search for solutions that satisfy all constraints by modifying the effective network connectivity through oscillations. If there is no solution that satisﬁes all constraints, the network state changes in a seemingly random manner and its trajectory approximates a sampling procedure that selects a variable assignment with a probability that increases with the fraction of constraints satisﬁed by this assignment. External evidence, or input to the network, can force variables to speciﬁc values. When new inputs are applied, the network re-evaluates the entire set of variables in its search for states that satisfy the maximum number of constraints, while being consistent with the external input. Our results demonstrate that the proposed network architecture can perform a deterministic search for the optimal solution to problems with non-convex cost functions. The network is inspired by canonical microcircuit models of the cortex and suggests possible dynamical mechanisms to solve constraint satisfaction problems that can be present in biological networks, or implemented in neuromorphic electronic circuits. 1

2 0.87388569 16 nips-2013-A message-passing algorithm for multi-agent trajectory planning

Author: Jose Bento, Nate Derbinsky, Javier Alonso-Mora, Jonathan S. Yedidia

Abstract: We describe a novel approach for computing collision-free global trajectories for p agents with speciﬁed initial and ﬁnal conﬁgurations, based on an improved version of the alternating direction method of multipliers (ADMM). Compared with existing methods, our approach is naturally parallelizable and allows for incorporating different cost functionals with only minor adjustments. We apply our method to classical challenging instances and observe that its computational requirements scale well with p for several cost functionals. We also show that a specialization of our algorithm can be used for local motion planning by solving the problem of joint optimization in velocity space. 1

3 0.86494064 157 nips-2013-Learning Multi-level Sparse Representations

Author: Ferran Diego Andilla, Fred A. Hamprecht

Abstract: Bilinear approximation of a matrix is a powerful paradigm of unsupervised learning. In some applications, however, there is a natural hierarchy of concepts that ought to be reﬂected in the unsupervised analysis. For example, in the neurosciences image sequence considered here, there are the semantic concepts of pixel → neuron → assembly that should ﬁnd their counterpart in the unsupervised analysis. Driven by this concrete problem, we propose a decomposition of the matrix of observations into a product of more than two sparse matrices, with the rank decreasing from lower to higher levels. In contrast to prior work, we allow for both hierarchical and heterarchical relations of lower-level to higher-level concepts. In addition, we learn the nature of these relations rather than imposing them. Finally, we describe an optimization scheme that allows to optimize the decomposition over all levels jointly, rather than in a greedy level-by-level fashion. The proposed bilevel SHMF (sparse heterarchical matrix factorization) is the ﬁrst formalism that allows to simultaneously interpret a calcium imaging sequence in terms of the constituent neurons, their membership in assemblies, and the time courses of both neurons and assemblies. Experiments show that the proposed model fully recovers the structure from difﬁcult synthetic data designed to imitate the experimental data. More importantly, bilevel SHMF yields plausible interpretations of real-world Calcium imaging data. 1

4 0.85057318 84 nips-2013-Deep Neural Networks for Object Detection

Author: Christian Szegedy, Alexander Toshev, Dumitru Erhan

Abstract: Deep Neural Networks (DNNs) have recently shown outstanding performance on image classiﬁcation tasks [14]. In this paper we go one step further and address the problem of object detection using DNNs, that is not only classifying but also precisely localizing objects of various classes. We present a simple and yet powerful formulation of object detection as a regression problem to object bounding box masks. We deﬁne a multi-scale inference procedure which is able to produce high-resolution object detections at a low cost by a few network applications. State-of-the-art performance of the approach is shown on Pascal VOC. 1

5 0.84927744 90 nips-2013-Direct 0-1 Loss Minimization and Margin Maximization with Boosting

Author: Shaodan Zhai, Tian Xia, Ming Tan, Shaojun Wang

Abstract: We propose a boosting method, DirectBoost, a greedy coordinate descent algorithm that builds an ensemble classiﬁer of weak classiﬁers through directly minimizing empirical classiﬁcation error over labeled training examples; once the training classiﬁcation error is reduced to a local coordinatewise minimum, DirectBoost runs a greedy coordinate ascent algorithm that continuously adds weak classiﬁers to maximize any targeted arbitrarily deﬁned margins until reaching a local coordinatewise maximum of the margins in a certain sense. Experimental results on a collection of machine-learning benchmark datasets show that DirectBoost gives better results than AdaBoost, LogitBoost, LPBoost with column generation and BrownBoost, and is noise tolerant when it maximizes an n′ th order bottom sample margin. 1

same-paper 6 0.83633918 15 nips-2013-A memory frontier for complex synapses

7 0.81699681 56 nips-2013-Better Approximation and Faster Algorithm Using the Proximal Average

8 0.73295283 77 nips-2013-Correlations strike back (again): the case of associative memory retrieval

9 0.71457207 141 nips-2013-Inferring neural population dynamics from multiple partial recordings of the same neural circuit

10 0.69740736 121 nips-2013-Firing rate predictions in optimal balanced networks

11 0.66002434 162 nips-2013-Learning Trajectory Preferences for Manipulators via Iterative Improvement

12 0.64606726 264 nips-2013-Reciprocally Coupled Local Estimators Implement Bayesian Information Integration Distributively

13 0.64559829 136 nips-2013-Hierarchical Modular Optimization of Convolutional Networks Achieves Representations Similar to Macaque IT and Human Ventral Stream

14 0.64025712 22 nips-2013-Action is in the Eye of the Beholder: Eye-gaze Driven Model for Spatio-Temporal Action Localization

15 0.63401341 86 nips-2013-Demixing odors - fast inference in olfaction

16 0.626311 255 nips-2013-Probabilistic Movement Primitives

17 0.61881196 64 nips-2013-Compete to Compute

18 0.61841017 49 nips-2013-Bayesian Inference and Online Experimental Design for Mapping Neural Microcircuits

19 0.61391073 114 nips-2013-Extracting regions of interest from biological images with convolutional sparse block coding

20 0.61084175 236 nips-2013-Optimal Neural Population Codes for High-dimensional Stimulus Variables