nips nips2007 nips2007-205 knowledge-graph by maker-knowledge-mining

205 nips-2007-Theoretical Analysis of Learning with Reward-Modulated Spike-Timing-Dependent Plasticity

Source: pdf

Author: Dejan Pecevski, Wolfgang Maass, Robert A. Legenstein

Abstract: Reward-modulated spike-timing-dependent plasticity (STDP) has recently emerged as a candidate for a learning rule that could explain how local learning rules at single synapses support behaviorally relevant adaptive changes in complex networks of spiking neurons. However the potential and limitations of this learning rule could so far only be tested through computer simulations. This article provides tools for an analytic treatment of reward-modulated STDP, which allow us to predict under which conditions reward-modulated STDP will be able to achieve a desired learning effect. In particular, we can produce in this way a theoretical explanation and a computer model for a fundamental experimental ﬁnding on biofeedback in monkeys (reported in [1]).

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 at Abstract Reward-modulated spike-timing-dependent plasticity (STDP) has recently emerged as a candidate for a learning rule that could explain how local learning rules at single synapses support behaviorally relevant adaptive changes in complex networks of spiking neurons. [sent-3, score-0.269]

2 1 Introduction A major puzzle for understanding learning in biological organisms is the relationship between experimentally well-established learning rules for synapses (such as STDP) on the microscopic level and adaptive changes of the behavior of biological organisms on the macroscopic level. [sent-7, score-0.238]

3 It is well-known that the consolidation of changes of synaptic weights in response to pre- and postsynaptic neuronal activity requires the presence of such third signals [2]. [sent-9, score-0.362]

4 We will consider in this article only cases where the reward prediction error is equal to the current reward. [sent-11, score-0.19]

5 We will refer to d(t) simply as the reward signal. [sent-12, score-0.168]

6 Obviously such learning scheme (1) faces a large credit-assignment problem, since not only those synapses for which weight changes would increase the chances of future reward receive the top-down signal d(t), but billions of other synapses too. [sent-13, score-0.644]

7 The monkeys learnt quite reliably (on the time scale of 10’s of minutes) to change the ﬁring rate of this neuron in the currently rewarded direction 1. [sent-16, score-0.435]

8 We present in section 3 and 4 of this abstract a learning theory for (1), where the eligibility trace cij (t) results from standard forms of STDP, which is able to explain the success of the experiment in [1]. [sent-18, score-0.155]

9 In section 5 we leave this concrete learning experiment and investigate under what conditions neurons can learn through trial and error (via reward-modulated STDP) associations of speciﬁc ﬁring patterns to speciﬁc patterns of input spikes. [sent-21, score-0.204]

10 2 Models for neurons and synaptic plasticity (1) (2) (3) The spike train of a neuron i which ﬁres action potentials at times ti , ti , ti , . [sent-26, score-0.869]

11 (2) s In our simulations, fc (s) is a function of the form fc (s) = τse e− τe if s ≥ 0 and 0 otherwise, with time constant τe = 0. [sent-31, score-0.426]

12 The actual weight change is the product of the eligibility trace with the reward signal as deﬁned by equation (1). [sent-34, score-0.466]

13 We assume that weights are clipped at the lower boundary value 0 and an upper boundary wmax . [sent-35, score-0.267]

14 W (r) = post We use a linear Poisson neuron model whose output spike train Sj (t) is a realization of a Poisson process with the underlying instantaneous ﬁring rate Rj (t). [sent-36, score-0.765]

15 Since STDP according to [3] has been experimentally conﬁrmed only for excitatory synapses, we will consider plasticity only for excitatory connections and assume that wji ≥ 0 for all i and ǫ(s) ≥ 0 for all s. [sent-38, score-0.52]

16 Because the synaptic response is scaled by the synaptic weights, we can assume without loss of generality that ∞ the response kernel is normalized to 0 ds ǫ(s) = 1. [sent-39, score-0.332]

17 We see that the expected weight change depends on how the correlations between the pre- and postsynaptic neurons correlate with the reward signal. [sent-47, score-0.573]

18 Analogously to [5], we can drop the ensemble average on the left hand side and obtain: d wji (t) dt ∞ T = ∞ dr W (r) 0 ds fc (s) Dji (t, s, r) νji (t − s, r) 0 0 + T ∞ dr W (r) −∞ ds fc (s + r) Dji (t, s, r) νji (t − s, r) |r| T . [sent-49, score-1.993]

19 (7) In the following, we will always use the smooth time-averaged vector wji (t) T , but for brevity, we will drop the angular brackets. [sent-50, score-0.335]

20 If one assumes for simplicity that the impact of a pre-post spike pair on the eligibility trace is always triggered by the postsynaptic spike, one gets (see [6] for details): ∞ ∞ dwji (t) = ds fc (s) dr W (r) Dji (t, s, r) νji (t − s, r) T . [sent-51, score-1.334]

21 (8) dt 0 −∞ This assumption (which is common in STDP analysis) will introduce a small error for post-before pre spike pairs, since if a reward signal arrives at some time dr after the pairing, the weight update will be proportional to fc (dr ) instead of fc (dr + r). [sent-52, score-1.563]

22 Equation (8) shows that if the reward signal does not depend on pre- and postsynaptic spike statistics, the weight will change according to standard STDP scaled by a constant proportional to the mean reward. [sent-54, score-0.658]

23 The authors showed that it is possible to increase and decrease the ﬁring rate of a randomly chosen neuron by rewarding the monkey for its high (respectively low) ﬁring rates. [sent-56, score-0.321]

24 We assume in our model that a reward is delivered to all neurons in the simulated recurrent network with some delay dr every time a speciﬁc neuron k in the network produces an action potential ∞ d(t) = 0 post dr Sk (t − dr − r)ǫr (r). [sent-57, score-2.361]

25 (9) where ǫr (r) is the shape of the reward pulse corresponding to one postsynaptic spike of the rein∞ forced neuron. [sent-58, score-0.537]

26 We assume that the reward kernel ǫr has zero mass, i. [sent-59, score-0.193]

27 In ¯ our simulations, this reward kernel will have a positive bump in the ﬁrst few hundred milliseconds, and a long tailed negative bump afterwards. [sent-62, score-0.281]

28 With the linear Poisson neuron model (see Section 2), the correlation of the reward with pre-post spike pairs of the reinforced neuron is (see [6]) ∞ Dki (t, s, r) = wki dr′ ǫr (r′ )ǫ(s + r − dr − r′ ) + ǫr (s − dr ) ≈ ǫr (s − dr ). [sent-63, score-2.65]

29 (10) 0 The last approximation holds if the impact of a single input spike on the membrane potential is small. [sent-64, score-0.304]

30 The correlation of the reward with pre-post spike pairs of non-reinforced neurons is ∞ νkj (t − dr − r′ , s − dr − r′ ) + wki wji ǫ(s + r − dr − r′ )ǫ(r) . [sent-65, score-2.455]

31 νj (t − s) + wji ǫ(r) 0 (11) If the contribution of a single postsynaptic potential to the membrane potential is small, we can neglect the impact of the presynaptic spike and write Dji (t, s, r) = dr′ ǫr (r′ ) ∞ Dji (t, s, r) ≈ dr′ ǫr (r′ ) 0 νkj (t − dr − r′ , s − dr − r′ ) . [sent-66, score-1.864]

32 νj (t − s) (12) Hence, the reward-spike correlation of a non-reinforced neuron depends on the correlation of this neuron with the reinforced neuron. [sent-67, score-0.694]

33 The mean weight change for weights to the reinforced neuron is given by d wki (t) = dt ∞ ∞ ds fc (s + dr )ǫr (s) 0 dr W (r) νki (t − dr − s, r) −∞ T . [sent-68, score-2.47]

34 The mean weight change of neurons j = k is given by d wji (t) = dt ∞ ds fc (s) 0 ∞ ∞ dr′ ǫr (r′ ) dr W (r) −∞ 0 νkj (t − dr − r′ , s − dr − r′ ) νji (t − s, r) νj (t − s) T (14) If the output of neurons j and k are uncorrelated, this evaluates to approximately zero (see [6]). [sent-70, score-2.61]

35 Other neurons are trained by STDP with a learning rate proportional to their correlation with the reinforced neuron. [sent-73, score-0.375]

36 If a neuron is uncorrelated with the reinforced neuron, the learning rate is approximately zero. [sent-74, score-0.45]

37 1 Computer simulations In order to test the theoretical predictions for the experiment described in the previous section, we have performed a computer simulation with a generic neural microcircuit receiving a global reward signal. [sent-76, score-0.358]

38 This global reward signal increases its value every time a speciﬁc neuron (the reinforced neuron) in the circuit ﬁres. [sent-77, score-0.641]

39 The circuit consists of 1000 leaky integrate-and-ﬁre (LIF) neurons (80% excitatory and 20% inhibitory), which are interconnected by conductance based synapses. [sent-78, score-0.358]

40 The short term dynamics of synapses was modeled in accordance with experimental data (see [6]). [sent-79, score-0.171]

41 064 where the ee, ei, ie, ii indices designate the type of the presynaptic and postsynaptic neurons (excitatory or inhibitory). [sent-84, score-0.351]

42 To reproduce the synaptic background activity of neocortical neurons in vivo, an Ornstein-Uhlenbeck (OU) conductance noise process modeled according to ([7]) was injected in the neurons, which also elicited spontaneous ﬁring of the neurons in the circuit with an average rate of 4Hz. [sent-85, score-0.648]

43 The function fc (t) from equation (2) t had the form fc (t) = τte e− τe if t ≥ 0 and 0 otherwise, with time constant τe = 0. [sent-87, score-0.445]

44 The reward signal during the simulation was computed according to eq. [sent-89, score-0.231]

45 r + τr τr (15) The parameter values for ǫr (t) were chosen such as to produce a positive reward pulse with a peak ∞ delayed 0. [sent-91, score-0.19]

46 5s from the spike that caused it, and a long tailed negative bump so that 0 dt ǫr (t) = 0. [sent-92, score-0.323]

47 A) The ﬁring rate of the reinforced neuron (solid line) increases while the average ﬁring rate of 20 other randomly chosen neurons in the circuit (dashed line) remains unchanged. [sent-101, score-0.68]

48 B) Evolution of the average synaptic weight of excitatory synapses connecting to the reinforced neuron (solid line) and to other neurons (dashed line). [sent-102, score-0.995]

49 C) Spike trains of the reinforced neuron at the beginning and at the end of the simulation. [sent-103, score-0.434]

50 The learning rule (1) was applied to all synapses in the circuit which have excitatory presynaptic and postsynaptic neurons. [sent-105, score-0.505]

51 1 shows that the ﬁring rate and synaptic weights of the reinforced neuron increase within a few minutes of simulated biological time, while those of the other neurons remain largely unchanged. [sent-109, score-0.757]

52 Whereas a very low spontaneous ﬁring rate of 1 Hz was required in [3], this simulation shows that reinforcement learning is also feasible at rate levels which correspond to those reported in [1]. [sent-111, score-0.186]

53 The reward signal d(t) was given in dependence post on how well the output spike train Sj of the neuron j matched some rather arbitrary spike train S ∗ that was produced by some neuron that received the same n input spike trains as the trained neuron ∗ ∗ ∗ with arbitrary weights w∗ = (w1 , . [sent-113, score-2.041]

54 , wn )T , wi ∈ {0, wmax }, but in addition n′ − n further ∗ spike trains Sn+1 , . [sent-116, score-0.565]

55 This setup provides a generic reinforcement learning scenario, when a quite arbitrary (and not perfectly realizable) spike output is reinforced, but simultaneously the performance of the learner can be evaluated quite clearly according to how well its weights w1 , . [sent-120, score-0.347]

56 , wn match those of the target neuron for those n input spike trains which both of them receive. [sent-123, score-0.583]

57 The analysis also reveals which reward kernels κ are suitable for this learning setup. [sent-126, score-0.168]

58 Since weights are changing very slowly, we have wji (t − s − r) = wji (t). [sent-128, score-0.668]

59 In the following, we will drop the dependence of wji on t for brevity. [sent-129, score-0.335]

60 We assume that the eligibility function fc (dr ) ≈ fc (dr + r) if |r| is on a time scale of a PSP, the learning window, or the reward kernel, and that dr is large pre compared to these time scales. [sent-132, score-1.277]

61 ∗ We will now bound the expected weight change for synapses ji with wi = wmax and for synapses ∗ jk with wjk = 0. [sent-134, score-0.839]

62 In this way we can derive conditions for which the expected weight change for the former synapses is positive, and that for the latter type is negative. [sent-135, score-0.303]

63 First, we assume that the integral over the reward kernel is positive. [sent-136, score-0.193]

64 In this case, the weight change is negative for synapses i with pre post ¯ post ∗ ¯ wi = 0 if and only if νi > 0, and −νj W > wji Wǫ . [sent-137, score-1.185]

65 In the worst case, wji is wmax and νj post is small. [sent-138, score-0.748]

66 We have to guarantee some minimal output rate νmin such that even if wji = wmax , this ∗ inequality is fulﬁlled. [sent-139, score-0.592]

67 For synapses i with wi = wmax , we obtain two more conditions (see [6] for a derivation). [sent-141, score-0.5]

68 If these inequalities are fulﬁlled and input rates are positive, then the weight vector converges on average from any initial weight vector to w∗ . [sent-143, score-0.193]

69 ¯ ¯ −ν post W > wmax Wǫ (19) min ∞ dr W (r)ǫ(r)ǫκ (r) ≥ −∞ ∞ dr W (r)ǫκ (r) > post ¯ −νmax W ¯¯ −W κ ∞ dr ǫ(r)ǫκ (r) 0 post ν ∗ νmax −∞ wmax ¯ fc ν∗ post + + ν ∗ + νmax , fc (dr ) wmax (20) (21) post where νmax is the maximal output rate. [sent-144, score-3.739]

70 If this is the case, the ﬁrst condition (19) ensures that weights with w∗ = 0 are depressed while the third condition (21) ensures that weights with w∗ = wmax are potentiated. [sent-146, score-0.311]

71 Optimal reward kernels: From condition (21), we can deduce optimal reward kernels κ. [sent-147, score-0.336]

72 The ∞ kernel should be such that the integral −∞ dr W (r)ǫκ (r) is large, while the integral over κ is small (but positive). [sent-148, score-0.545]

73 Hence, reward is positive if the neuron spikes around the target spike or somewhat later, and negative if the neuron spikes much too early. [sent-152, score-1.041]

74 1 Computer simulations In the computer simulations we explored the learning rule in a more biologically realistic setting, where we used a leaky integrate-and-ﬁre (LIF) neuron with input synaptic connections coming from 1. [sent-154, score-0.564]

75 0 0 30 60 time [min] 90 120 0 1 2 time [sec] 3 4 Figure 2: Reinforcement learning of spike times. [sent-160, score-0.213]

76 The curves show the average of the synaptic weights ∗ that should converge to wi = 0 (dashed lines), and the average of the synaptic weights that should ∗ converge to wi = wmax (solid lines) with a different shading for each simulation run. [sent-162, score-0.805]

77 B) Comparison of the output of the trained neuron before (upper trace) and after learning (lower trace; the same input spike trains and the same noise inputs were used before and after training for 2 hours). [sent-163, score-0.595]

78 The second trace from above shows those spike times which are rewarded, the third trace shows the target spike train without the additional noise inputs. [sent-164, score-0.599]

79 1 2 3 4 5 6 Figure 3: Predicted average weight change (black bars) calculated from equation (18), and the estimated average weight change (gray bars) from simulations, presented for 6 different experiments with different parameter settings (see Table 1). [sent-175, score-0.285]

80 2 A) Weight change ∗ values for synapses with wi = wmax . [sent-176, score-0.518]

81 B) Weight change values ∗ for synapses with wi = 0. [sent-177, score-0.295]

82 The synapses were conductance based exhibiting short term facilitation and depression. [sent-180, score-0.212]

83 The trained neuron and the arbitrarily given neuron which produced the target spike train S ∗ (“target neuron”) both were connected to the same randomly chosen, 100 excitatory and 10 inhibitory neurons from the circuit. [sent-181, score-1.081]

84 The target neuron had 10 additional excitatory input connections (these weights were set to wmax ), not accessible to the trained neuron. [sent-182, score-0.725]

85 Only the synapses of the trained neuron connecting from excitatory neurons ∗ were set to be plastic. [sent-183, score-0.681]

86 The target neuron had a weight vector with wi = 0 for 0 ≤ i < 50 and ∗ wi = wmax for 50 ≤ i < 110. [sent-184, score-0.772]

87 The generic neural microcircuit from which the trained and the target neurons receive the input had 80% excitatory and 20% inhibitory neurons interconnected randomly with a probability of 0. [sent-185, score-0.602]

88 The neurons received background synaptic noise as modeled in [7], which caused spontaneous activity of the neurons with an average ﬁring rate of 6. [sent-187, score-0.534]

89 5s, and we used the same eligibility trace function fc (t) as in the simulations for the biofeedback experiment (see [6] for details). [sent-192, score-0.492]

90 In each of the 5 runs, the average synaptic weights of synapses with ∗ ∗ wi = wmax and wi = 0 approach their target values, as shown in Fig. [sent-196, score-0.801]

91 In order to test how 2 The values in the ﬁgure are calculated as ∆w = dw/dt tsim wmax /2 w(tsim )−w(0) wmax /2 for the simulations, and with ∆w = for the predicted value. [sent-198, score-0.484]

92 w(t) is the average weight over synapses with the same value of w∗ . [sent-199, score-0.266]

93 τǫ [ms] 1 10 2 7 3 20 4 7 5 10 6 25 A post κ wmax νmin [Hz] A+ 106 A− τ+ ,τ2 [ms] + 0. [sent-201, score-0.436]

94 closely the learning neuron reproduces the target spike train S ∗ after learning, we have performed additional simulations where the same spiking input SI is applied to the learning neuron before and after we conducted the learning experiment (results are reported in Fig. [sent-228, score-0.924]

95 The equations in section 5 deﬁne a parameter space for which the trained neuron can learn the target synapse pattern w∗ . [sent-230, score-0.391]

96 We have chosen 6 different parameter values encompassing cases with satisﬁed and non-satisﬁed constraints, and performed experiments where we compare the predicted average weight change from equation (18) with the actual average weight change produced by simulations. [sent-231, score-0.285]

97 reward functions) that would optimally support the speed of reward-modulated learning for the generic (but rather difﬁcult) learning tasks where a neuron is supposed to respond to input spikes with speciﬁc patterns of output spikes, and only spikes at the right times are rewarded. [sent-240, score-0.599]

98 Further work (see [6]) shows that reward-modulated STDP can in some cases replace supervised training of readout neurons from generic cortical microcircuit models. [sent-241, score-0.223]

99 Solving the distal reward problem through linkage of STDP and dopamine signaling. [sent-266, score-0.188]

100 Fluctuating synaptic conductances recreate in vivo-like activity in neocortical neurons. [sent-292, score-0.177]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('dr', 0.52), ('wji', 0.312), ('neuron', 0.257), ('stdp', 0.244), ('wmax', 0.223), ('post', 0.213), ('fc', 0.213), ('spike', 0.213), ('synapses', 0.171), ('reward', 0.168), ('neurons', 0.141), ('postsynaptic', 0.134), ('reinforced', 0.134), ('synaptic', 0.124), ('dji', 0.115), ('ring', 0.103), ('wi', 0.086), ('eligibility', 0.085), ('pre', 0.078), ('presynaptic', 0.076), ('ji', 0.076), ('weight', 0.074), ('excitatory', 0.073), ('biofeedback', 0.067), ('rewarded', 0.064), ('ds', 0.059), ('simulations', 0.057), ('dt', 0.053), ('sj', 0.052), ('lif', 0.051), ('microcircuit', 0.051), ('circuit', 0.051), ('trace', 0.051), ('spikes', 0.05), ('synapse', 0.049), ('target', 0.046), ('cji', 0.045), ('weights', 0.044), ('trains', 0.043), ('plasticity', 0.043), ('conductance', 0.041), ('reinforcement', 0.04), ('trained', 0.039), ('dwji', 0.038), ('fetz', 0.038), ('psp', 0.038), ('tsim', 0.038), ('wki', 0.038), ('rate', 0.038), ('spontaneous', 0.038), ('monkeys', 0.038), ('change', 0.038), ('ful', 0.038), ('simulation', 0.032), ('generic', 0.031), ('signal', 0.031), ('activity', 0.031), ('bump', 0.031), ('inhibitory', 0.03), ('poisson', 0.03), ('changes', 0.029), ('spiking', 0.026), ('interconnected', 0.026), ('leaky', 0.026), ('pecevski', 0.026), ('tailed', 0.026), ('monkey', 0.026), ('si', 0.026), ('kernel', 0.025), ('hz', 0.025), ('train', 0.025), ('membrane', 0.024), ('input', 0.024), ('correlation', 0.023), ('drop', 0.023), ('kj', 0.023), ('pairing', 0.022), ('neocortical', 0.022), ('neuromodulatory', 0.022), ('precentral', 0.022), ('pulse', 0.022), ('article', 0.022), ('potential', 0.022), ('ti', 0.022), ('impact', 0.021), ('lled', 0.021), ('average', 0.021), ('window', 0.021), ('uncorrelated', 0.021), ('dopamine', 0.02), ('graz', 0.02), ('legenstein', 0.02), ('conditions', 0.02), ('biological', 0.019), ('output', 0.019), ('equation', 0.019), ('milliseconds', 0.019), ('experiment', 0.019), ('connections', 0.019), ('correlations', 0.018)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.99999946 205 nips-2007-Theoretical Analysis of Learning with Reward-Modulated Spike-Timing-Dependent Plasticity

Author: Dejan Pecevski, Wolfgang Maass, Robert A. Legenstein

2 0.30970752 117 nips-2007-Learning to classify complex patterns using a VLSI network of spiking neurons

Author: Srinjoy Mitra, Giacomo Indiveri, Stefano Fusi

Abstract: We propose a compact, low power VLSI network of spiking neurons which can learn to classify complex patterns of mean ﬁring rates on–line and in real–time. The network of integrate-and-ﬁre neurons is connected by bistable synapses that can change their weight using a local spike–based plasticity mechanism. Learning is supervised by a teacher which provides an extra input to the output neurons during training. The synaptic weights are updated only if the current generated by the plastic synapses does not match the output desired by the teacher (as in the perceptron learning rule). We present experimental results that demonstrate how this VLSI network is able to robustly classify uncorrelated linearly separable spatial patterns of mean ﬁring rates.

3 0.24502751 177 nips-2007-Simplified Rules and Theoretical Analysis for Information Bottleneck Optimization and PCA with Spiking Neurons

Author: Lars Buesing, Wolfgang Maass

Abstract: We show that under suitable assumptions (primarily linearization) a simple and perspicuous online learning rule for Information Bottleneck optimization with spiking neurons can be derived. This rule performs on common benchmark tasks as well as a rather complex rule that has previously been proposed [1]. Furthermore, the transparency of this new learning rule makes a theoretical analysis of its convergence properties feasible. A variation of this learning rule (with sign changes) provides a theoretically founded method for performing Principal Component Analysis (PCA) with spiking neurons. By applying this rule to an ensemble of neurons, different principal components of the input can be extracted. In addition, it is possible to preferentially extract those principal components from incoming signals X that are related or are not related to some additional target signal YT . In a biological interpretation, this target signal YT (also called relevance variable) could represent proprioceptive feedback, input from other sensory modalities, or top-down signals. 1

4 0.2442871 14 nips-2007-A configurable analog VLSI neural network with spiking neurons and self-regulating plastic synapses

Author: Massimiliano Giulioni, Mario Pannunzi, Davide Badoni, Vittorio Dante, Paolo D. Giudice

Abstract: We summarize the implementation of an analog VLSI chip hosting a network of 32 integrate-and-ﬁre (IF) neurons with spike-frequency adaptation and 2,048 Hebbian plastic bistable spike-driven stochastic synapses endowed with a selfregulating mechanism which stops unnecessary synaptic changes. The synaptic matrix can be ﬂexibly conﬁgured and provides both recurrent and AER-based connectivity with external, AER compliant devices. We demonstrate the ability of the network to efﬁciently classify overlapping patterns, thanks to the self-regulating mechanism.

5 0.19878863 33 nips-2007-Bayesian Inference for Spiking Neuron Models with a Sparsity Prior

Author: Sebastian Gerwinn, Matthias Bethge, Jakob H. Macke, Matthias Seeger

Abstract: Generalized linear models are the most commonly used tools to describe the stimulus selectivity of sensory neurons. Here we present a Bayesian treatment of such models. Using the expectation propagation algorithm, we are able to approximate the full posterior distribution over all weights. In addition, we use a Laplacian prior to favor sparse solutions. Therefore, stimulus features that do not critically inﬂuence neural activity will be assigned zero weights and thus be effectively excluded by the model. This feature selection mechanism facilitates both the interpretation of the neuron model as well as its predictive abilities. The posterior distribution can be used to obtain conﬁdence intervals which makes it possible to assess the statistical signiﬁcance of the solution. In neural data analysis, the available amount of experimental measurements is often limited whereas the parameter space is large. In such a situation, both regularization by a sparsity prior and uncertainty estimates for the model parameters are essential. We apply our method to multi-electrode recordings of retinal ganglion cells and use our uncertainty estimate to test the statistical signiﬁcance of functional couplings between neurons. Furthermore we used the sparsity of the Laplace prior to select those ﬁlters from a spike-triggered covariance analysis that are most informative about the neural response. 1

6 0.16931045 104 nips-2007-Inferring Neural Firing Rates from Spike Trains Using Gaussian Processes

7 0.1633129 140 nips-2007-Neural characterization in partially observed populations of spiking neurons

8 0.15741165 36 nips-2007-Better than least squares: comparison of objective functions for estimating linear-nonlinear models

9 0.14041938 60 nips-2007-Contraction Properties of VLSI Cooperative Competitive Neural Networks of Spiking Neurons

10 0.12369066 17 nips-2007-A neural network implementing optimal state estimation based on dynamic spike train decoding

11 0.11951649 26 nips-2007-An online Hebbian learning rule that performs Independent Component Analysis

12 0.11430823 164 nips-2007-Receptive Fields without Spike-Triggering

13 0.093083411 32 nips-2007-Bayesian Co-Training

14 0.092875257 25 nips-2007-An in-silico Neural Model of Dynamic Routing through Neuronal Coherence

15 0.071988843 35 nips-2007-Bayesian binning beats approximate alternatives: estimating peri-stimulus time histograms

16 0.058035731 182 nips-2007-Sparse deep belief net model for visual area V2

17 0.049720805 168 nips-2007-Reinforcement Learning in Continuous Action Spaces through Sequential Monte Carlo Methods

18 0.044699885 101 nips-2007-How SVMs can estimate quantiles and the median

19 0.044385787 5 nips-2007-A Game-Theoretic Approach to Apprenticeship Learning

20 0.044345662 183 nips-2007-Spatial Latent Dirichlet Allocation

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, -0.172), (1, 0.109), (2, 0.405), (3, 0.091), (4, -0.018), (5, -0.108), (6, 0.191), (7, -0.075), (8, 0.106), (9, 0.15), (10, -0.089), (11, -0.068), (12, -0.039), (13, -0.049), (14, -0.077), (15, -0.005), (16, -0.011), (17, -0.025), (18, -0.113), (19, -0.096), (20, 0.003), (21, -0.009), (22, -0.008), (23, -0.081), (24, -0.011), (25, 0.092), (26, -0.05), (27, -0.029), (28, 0.007), (29, -0.024), (30, 0.055), (31, -0.008), (32, 0.02), (33, 0.021), (34, 0.007), (35, 0.025), (36, 0.003), (37, 0.033), (38, -0.06), (39, -0.049), (40, -0.032), (41, 0.085), (42, -0.02), (43, -0.024), (44, 0.031), (45, 0.087), (46, -0.049), (47, -0.048), (48, -0.023), (49, 0.011)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.97693801 205 nips-2007-Theoretical Analysis of Learning with Reward-Modulated Spike-Timing-Dependent Plasticity

Author: Dejan Pecevski, Wolfgang Maass, Robert A. Legenstein

2 0.894885 117 nips-2007-Learning to classify complex patterns using a VLSI network of spiking neurons

Author: Srinjoy Mitra, Giacomo Indiveri, Stefano Fusi

3 0.85345644 14 nips-2007-A configurable analog VLSI neural network with spiking neurons and self-regulating plastic synapses

Author: Massimiliano Giulioni, Mario Pannunzi, Davide Badoni, Vittorio Dante, Paolo D. Giudice

4 0.73030496 60 nips-2007-Contraction Properties of VLSI Cooperative Competitive Neural Networks of Spiking Neurons

Author: Emre Neftci, Elisabetta Chicca, Giacomo Indiveri, Jean-jeacques Slotine, Rodney J. Douglas

Abstract: A non–linear dynamic system is called contracting if initial conditions are forgotten exponentially fast, so that all trajectories converge to a single trajectory. We use contraction theory to derive an upper bound for the strength of recurrent connections that guarantees contraction for complex neural networks. Speciﬁcally, we apply this theory to a special class of recurrent networks, often called Cooperative Competitive Networks (CCNs), which are an abstract representation of the cooperative-competitive connectivity observed in cortex. This speciﬁc type of network is believed to play a major role in shaping cortical responses and selecting the relevant signal among distractors and noise. In this paper, we analyze contraction of combined CCNs of linear threshold units and verify the results of our analysis in a hybrid analog/digital VLSI CCN comprising spiking neurons and dynamic synapses. 1

5 0.68051946 177 nips-2007-Simplified Rules and Theoretical Analysis for Information Bottleneck Optimization and PCA with Spiking Neurons

Author: Lars Buesing, Wolfgang Maass

6 0.57864207 26 nips-2007-An online Hebbian learning rule that performs Independent Component Analysis

7 0.57625443 25 nips-2007-An in-silico Neural Model of Dynamic Routing through Neuronal Coherence

8 0.50166082 33 nips-2007-Bayesian Inference for Spiking Neuron Models with a Sparsity Prior

9 0.46113789 104 nips-2007-Inferring Neural Firing Rates from Spike Trains Using Gaussian Processes

10 0.43702385 140 nips-2007-Neural characterization in partially observed populations of spiking neurons

11 0.42538023 36 nips-2007-Better than least squares: comparison of objective functions for estimating linear-nonlinear models

12 0.36615545 164 nips-2007-Receptive Fields without Spike-Triggering

13 0.35535046 35 nips-2007-Bayesian binning beats approximate alternatives: estimating peri-stimulus time histograms

14 0.32647264 17 nips-2007-A neural network implementing optimal state estimation based on dynamic spike train decoding

15 0.26669863 188 nips-2007-Subspace-Based Face Recognition in Analog VLSI

16 0.24652629 191 nips-2007-Temporal Difference Updating without a Learning Rate

17 0.23491177 168 nips-2007-Reinforcement Learning in Continuous Action Spaces through Sequential Monte Carlo Methods

18 0.21279256 85 nips-2007-Experience-Guided Search: A Theory of Attentional Control

19 0.18868908 67 nips-2007-Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation

20 0.18549313 110 nips-2007-Learning Bounds for Domain Adaptation

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(5, 0.021), (13, 0.084), (16, 0.122), (19, 0.023), (21, 0.069), (31, 0.013), (34, 0.025), (35, 0.013), (36, 0.024), (47, 0.075), (58, 0.296), (83, 0.087), (87, 0.011), (90, 0.036)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.77313799 205 nips-2007-Theoretical Analysis of Learning with Reward-Modulated Spike-Timing-Dependent Plasticity

Author: Dejan Pecevski, Wolfgang Maass, Robert A. Legenstein

2 0.52806419 60 nips-2007-Contraction Properties of VLSI Cooperative Competitive Neural Networks of Spiking Neurons

Author: Emre Neftci, Elisabetta Chicca, Giacomo Indiveri, Jean-jeacques Slotine, Rodney J. Douglas

3 0.52769506 170 nips-2007-Robust Regression with Twinned Gaussian Processes

Author: Andrew Naish-guzman, Sean Holden

Abstract: We propose a Gaussian process (GP) framework for robust inference in which a GP prior on the mixing weights of a two-component noise model augments the standard process over latent function values. This approach is a generalization of the mixture likelihood used in traditional robust GP regression, and a specialization of the GP mixture models suggested by Tresp [1] and Rasmussen and Ghahramani [2]. The value of this restriction is in its tractable expectation propagation updates, which allow for faster inference and model selection, and better convergence than the standard mixture. An additional beneﬁt over the latter method lies in our ability to incorporate knowledge of the noise domain to inﬂuence predictions, and to recover with the predictive distribution information about the outlier distribution via the gating process. The model has asymptotic complexity equal to that of conventional robust methods, but yields more conﬁdent predictions on benchmark problems than classical heavy-tailed models and exhibits improved stability for data with clustered corruptions, for which they fail altogether. We show further how our approach can be used without adjustment for more smoothly heteroscedastic data, and suggest how it could be extended to more general noise models. We also address similarities with the work of Goldberg et al. [3].

4 0.52685678 206 nips-2007-Topmoumoute Online Natural Gradient Algorithm

Author: Nicolas L. Roux, Pierre-antoine Manzagol, Yoshua Bengio

Abstract: Guided by the goal of obtaining an optimization algorithm that is both fast and yields good generalization, we study the descent direction maximizing the decrease in generalization error or the probability of not increasing generalization error. The surprising result is that from both the Bayesian and frequentist perspectives this can yield the natural gradient direction. Although that direction can be very expensive to compute we develop an efﬁcient, general, online approximation to the natural gradient descent which is suited to large scale problems. We report experimental results showing much faster convergence in computation time and in number of iterations with TONGA (Topmoumoute Online natural Gradient Algorithm) than with stochastic gradient descent, even on very large datasets.

5 0.52249467 33 nips-2007-Bayesian Inference for Spiking Neuron Models with a Sparsity Prior

Author: Sebastian Gerwinn, Matthias Bethge, Jakob H. Macke, Matthias Seeger

6 0.52231973 140 nips-2007-Neural characterization in partially observed populations of spiking neurons

7 0.50808895 104 nips-2007-Inferring Neural Firing Rates from Spike Trains Using Gaussian Processes

8 0.50689274 164 nips-2007-Receptive Fields without Spike-Triggering

9 0.49307674 36 nips-2007-Better than least squares: comparison of objective functions for estimating linear-nonlinear models

10 0.49205554 177 nips-2007-Simplified Rules and Theoretical Analysis for Information Bottleneck Optimization and PCA with Spiking Neurons

11 0.49192747 117 nips-2007-Learning to classify complex patterns using a VLSI network of spiking neurons

12 0.49108487 195 nips-2007-The Generalized FITC Approximation

13 0.46965113 79 nips-2007-Efficient multiple hyperparameter learning for log-linear models

14 0.45817146 87 nips-2007-Fast Variational Inference for Large-scale Internet Diagnosis

15 0.45665088 24 nips-2007-An Analysis of Inference with the Universum

16 0.45264611 84 nips-2007-Expectation Maximization and Posterior Constraints

17 0.45164335 100 nips-2007-Hippocampal Contributions to Control: The Third Way

18 0.44981343 14 nips-2007-A configurable analog VLSI neural network with spiking neurons and self-regulating plastic synapses

19 0.44928685 102 nips-2007-Incremental Natural Actor-Critic Algorithms

20 0.44862181 158 nips-2007-Probabilistic Matrix Factorization