nips nips2012 nips2012-152 knowledge-graph by maker-knowledge-mining

152 nips-2012-Homeostatic plasticity in Bayesian spiking networks as Expectation Maximization with posterior constraints

Source: pdf

Author: Stefan Habenschuss, Johannes Bill, Bernhard Nessler

Abstract: Recent spiking network models of Bayesian inference and unsupervised learning frequently assume either inputs to arrive in a special format or employ complex computations in neuronal activation functions and synaptic plasticity rules. Here we show in a rigorous mathematical treatment how homeostatic processes, which have previously received little attention in this context, can overcome common theoretical limitations and facilitate the neural implementation and performance of existing models. In particular, we show that homeostatic plasticity can be understood as the enforcement of a ’balancing’ posterior constraint during probabilistic inference and learning with Expectation Maximization. We link homeostatic dynamics to the theory of variational inference, and show that nontrivial terms, which typically appear during probabilistic inference in a large class of models, drop out. We demonstrate the feasibility of our approach in a spiking WinnerTake-All architecture of Bayesian inference and learning. Finally, we sketch how the mathematical framework can be extended to richer recurrent network architectures. Altogether, our theory provides a novel perspective on the interplay of homeostatic processes and synaptic plasticity in cortical microcircuits, and points to an essential role of homeostasis during inference and learning in spiking networks. 1

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 Homeostatic plasticity in Bayesian spiking networks as Expectation Maximization with posterior constraints Stefan Habenschuss∗ , Johannes Bill∗ , Bernhard Nessler Institute for Theoretical Computer Science, Graz University of Technology {habenschuss,bill,nessler}@igi. [sent-1, score-0.71]

2 at Abstract Recent spiking network models of Bayesian inference and unsupervised learning frequently assume either inputs to arrive in a special format or employ complex computations in neuronal activation functions and synaptic plasticity rules. [sent-3, score-1.108]

3 Here we show in a rigorous mathematical treatment how homeostatic processes, which have previously received little attention in this context, can overcome common theoretical limitations and facilitate the neural implementation and performance of existing models. [sent-4, score-0.481]

4 In particular, we show that homeostatic plasticity can be understood as the enforcement of a ’balancing’ posterior constraint during probabilistic inference and learning with Expectation Maximization. [sent-5, score-0.933]

5 We link homeostatic dynamics to the theory of variational inference, and show that nontrivial terms, which typically appear during probabilistic inference in a large class of models, drop out. [sent-6, score-0.712]

6 We demonstrate the feasibility of our approach in a spiking WinnerTake-All architecture of Bayesian inference and learning. [sent-7, score-0.312]

7 Finally, we sketch how the mathematical framework can be extended to richer recurrent network architectures. [sent-8, score-0.237]

8 Altogether, our theory provides a novel perspective on the interplay of homeostatic processes and synaptic plasticity in cortical microcircuits, and points to an essential role of homeostasis during inference and learning in spiking networks. [sent-9, score-1.466]

9 A number of innovative models has been proposed recently which demonstrate that in principle, spiking networks can carry out quite complex probabilistic inference tasks [7, 8, 9, 10], and even learn to adapt to their inputs near optimally through various forms of plasticity [11, 12, 13, 14, 15]. [sent-12, score-0.748]

10 A theoretically related issue has been encountered by Deneve [7, 11], in which inference and learning is realized in a two-state Hidden Markov Model by a single spiking neuron. [sent-17, score-0.329]

11 Although synaptic learning rules are found to be locally computable, the learning update for intrinsic excitabilities remains intricate. [sent-18, score-0.609]

12 [13] have recently proposed a promising model for Bayes optimal sequence learning in spiking networks ∗ These authors contributed equally to this work. [sent-20, score-0.31]

13 1 in which a global reward signal, which is computed from the network state and synaptic weights, modulates otherwise purely local learning rules. [sent-21, score-0.35]

14 Also the recent innovative model for variational learning in recurrent spiking networks by Rezende et al. [sent-22, score-0.498]

15 With these goals in mind, we introduce here a novel theoretical perspective on homeostatic plasticity in Bayesian spiking networks that complements previous approaches by constraining statistical properties of the network response rather than the input distribution. [sent-25, score-1.352]

16 In particular we introduce ’balancing’ posterior constraints which can be implemented in a purely local manner by the spiking network through a simple rule that is strongly reminiscent of homeostatic intrinsic plasticity in cortex [18, 19]. [sent-26, score-1.478]

17 Importantly, it turns out that the emerging network dynamics eliminate a particular class of nontrivial computations that frequently arise in Bayesian spiking networks. [sent-27, score-0.521]

18 First we develop the mathematical framework for Expectation Maximization (EM) with homeostatic posterior constraints in an instructive Winner-Take-all network model of probabilistic inference and unsupervised learning. [sent-28, score-0.74]

19 Building upon the theoretical results of [20], we establish a rigorous link between homeostatic intrinsic plasticity and variational inference. [sent-29, score-1.049]

20 In a second step, we sketch how the framework can be extended to recurrent spiking networks; by introducing posterior constraints on the correlation structure, we recover local plasticity rules for recurrent synaptic weights. [sent-30, score-1.125]

21 As shown in [12], such a network N can implement probabilistic inference p(z|y, V ) through its spiking dynamics, and maximum likelihood learning through local synaptic learning rules (see Figure 1A). [sent-32, score-0.712]

22 Each WTA neuron k receives spiking inputs yi via synaptic weights Vki and responds with an instantaneous spiking probability which depends exponentially on its input potential uk in accordance with biological ﬁndings [21]. [sent-36, score-1.028]

23 The input is deﬁned as yi (t) = 1 if input neuron i emitted a spike within the last τ milliseconds, and 0 otherwise, corresponding to a rectangular post-synaptic potential (PSP) of length τ . [sent-38, score-0.228]

24 We deﬁne zk (t) = 1 at spike times t of neuron k and zk (t) = 0 otherwise. [sent-39, score-0.689]

25 Sketch of intrinsic homeostatic plasticity maintaining a certain target average activation. [sent-45, score-0.983]

26 Homeostatic plasticity induces average ﬁring rates (blue) close to target values (red). [sent-47, score-0.342]

27 In addition to the spiking input, each neuron’s potential uk features an intrinsic excitability −Ak +ˆk . [sent-53, score-0.541]

28 , time-varying Vki , it is unclear how biologically realistic neurons could communicate ongoing changes in synaptic weights from distal synaptic sites to the soma. [sent-57, score-0.667]

29 This critical issue was apparently identiﬁed in [12] and [15]; both papers circumvent the problem (in similar probabilistic models) by constraining the input y (and also the synaptic weights in [15]) in order to maintain constant and uniform values Ak across all WTA neurons. [sent-58, score-0.332]

30 Instead of assuming that the inputs y meet a normalization constraint, we constrain the network response during inference, by applying homeostatic dynamics to the intrinsic excitabilities. [sent-60, score-0.906]

31 In contrast, by applying homeostatic plasticity, the network response will be constrained to implement a variational posterior from a class of “homeostatic” distributions Q: the long-term average activation of each WTA neuron zk is constrained to an a priori deﬁned target value. [sent-69, score-1.185]

32 Notably, we will see that the resulting network response q ∗ describes an optimal variational E-Step in the sense that q ∗ (z|y) = arg minq∈Q KL (q(z|y) || p(z|y, V )). [sent-70, score-0.238]

33 Importantly, homeostatic plasticity fully regulates the intrinsic excitabilities, and as a side effect eliminates the non-local terms Ak in the E-step, 3 while synaptic plasticity of the weights Vki optimizes the underlying probabilistic model p(y, z|V ) in the M-step. [sent-71, score-1.565]

34 In summary, the network response implements q ∗ as the variational E-step, the M-Step can be performed via gradient ascent on (6) with respect to Vki . [sent-72, score-0.283]

35 Note that (8) is a spike-timing dependent plasticity rule (cf. [sent-75, score-0.357]

36 [12]) and is non-zero only at post-synaptic spike times t, for which zk (t) = 1. [sent-76, score-0.33]

37 The effect of the homeostatic intrinsic plasticity rule (7) is illustrated in Figure 1C: it aims to keep the long-term average activation of each WTA neuron k close to a certain target value mk . [sent-77, score-1.253]

38 More precisely, if rk is a neuron’s long-term average ﬁring rate, then homeostatic plasticity will ensure that rk /rnet ≈ mk . [sent-78, score-0.928]

39 The target activations mk ∈ (0, 1) can be chosen freely with the obvious constraint that k mk = 1. [sent-79, score-0.261]

40 Note that (7) is strongly reminiscent of homeostatic intrinsic plasticity in cortex [18, 19]. [sent-80, score-0.966]

41 We have implemented these dynamics in a computer simulation of a WTA spiking network N . [sent-81, score-0.452]

42 Figure 1D shows that, at the end of a 104 s learning period, homeostatic plasticity has indeed achieved that rk ≈ rnet · mk . [sent-83, score-0.968]

43 Figure 1F shows the output spiking behavior of the circuit before and after learning in response to a set of test patterns. [sent-86, score-0.306]

44 • The interplay of homeostatic and synaptic plasticity can be understood from the perspective of variational EM. [sent-92, score-1.161]

45 E-step: variational inference with homeostasis The variational distribution q(z|y) we consider for the model (1)-(3) is a 2N · K dimensional object. [sent-94, score-0.259]

46 In addition, we constrain q to be a “homeostatic” distribution q ∈ Q such that the average activation of each hidden variable (neuron) zk equals an a-priori speciﬁed mean activation mk under the input statistics p∗ (y). [sent-96, score-0.571]

47 Formally we deﬁne the constraint set, Q = {q : zk p∗ (y)q(z|y) = mk , for all k = 1 . [sent-98, score-0.405]

48 with (9) k 1 Without adaptation of intrinsic excitabilities, the network would start performing erroneous inference, learning would reinforce this erroneous behavior, and performance would quickly break down. [sent-102, score-0.323]

49 We have veriﬁed this in simulations for the present WTA model: Consistently across trials, a small subset of WTA neurons became dominantly active while most neurons remained silent. [sent-103, score-0.228]

50 Homeostatic posterior constraints in the WTA model: Under the variational distribution q, the average activation of each variable zk must equal mk . [sent-105, score-0.603]

51 For each set of synaptic weights V there exists a unique assignment of intrinsic excitabilities b, such that the constraints are fulﬁlled. [sent-107, score-0.635]

52 Theoretical decomposition of the intrinsic excitability bk into −Ak , ˆk and βk . [sent-109, score-0.327]

53 Durb ing variational EM the bk predominantly “track” the dynamically changing non-local terms −Ak (relative comparison between two WTA neurons from Figure 1). [sent-111, score-0.284]

54 We ﬁnd that the q ∗ which maximizes the objective function F during the E-step (and thus minimizes the KL-divergence to the posterior p(z|y, V )) ∗ ∗ has the convenient form q ∗ (z|y) ∝ p(z|y, V ) · exp( k βk zk ) with some βk . [sent-114, score-0.333]

55 Hence, it sufﬁces to consider distributions of the form, qβ (z|y) ∝ exp( Vki yi + ˆk − Ak + βk )) , b zk ( i k (10) =:bk for the maximization problem. [sent-115, score-0.328]

56 Note that any distribution of this form can be implemented by the spiking network N if the intrinsic excitabilities are set to bk = −Ak + ˆk + βk . [sent-117, score-0.851]

57 the variational parameter vector which maximizes the dual [20], βk mk − log Ψ(β) = k βk zk ) p(z|y, V ) exp( z p∗ (y) . [sent-120, score-0.506]

58 (11) k Due to concavity of the dual, a unique global maximizer β ∗ exists, and thus also the corresponding ∗ b optimal intrinsic excitabilities b∗ = −Ak +ˆk +βk are unique. [sent-121, score-0.359]

59 The theoretical relation between the intrinsic excitabilities bk , the original nontrivial term −Ak and the variational parameters βk is sketched in Figure 2C. [sent-123, score-0.601]

60 those intrinsic excitabilities which fulﬁll the homeostatic constraints, amounts to gradient ascent ∂β Ψ(β) on the dual, which leads to the following homeostatic learning rule for the intrinsic excitabilities, ∆bk ∝ ∂βk Ψ(β) = mk − zk p∗ (y)q(z|y) . [sent-127, score-1.936]

61 (12) Note that the intrinsic homeostatic plasticity rule (7) in the network corresponds to a sample-based stochastic version of this theoretically derived adaptation mechanism (12). [sent-128, score-1.142]

62 Hence, given enough time, homeostatic plasticity will automatically install near-optimal intrinsic excitabilities b ≈ b∗ and implement the correct variational distribution q ∗ up to stochastic ﬂuctuations in b due to the nonzero learning rate ηb . [sent-129, score-1.248]

63 The non-local terms Ak have entirely dropped out of the network dynamics, since the intrinsic excitabilities bk can be arbitrarily initialized, and are then fully regulated by the local homeostatic rule, which does not require knowledge of Ak . [sent-130, score-1.054]

64 As a side remark, note that although the variational parameters βk are not explicitly present in the implementation, they can be theoretically recovered from the network at any point, via 5 Figure 3: A. [sent-131, score-0.227]

65 2 Hence, a major effect of the local homeostatic plasticity rule during learning is to dynamically track and effectively implement the non-local terms −Ak . [sent-144, score-0.838]

66 This is shown in Figure 2D, in which the relative excitabilities of two WTA neurons bk − bj are plotted against the corresponding non-local Ak − Aj over the course of learning in the ﬁrst simulation (Figure 1). [sent-145, score-0.4]

67 M-step: interplay of synaptic and homeostatic intrinsic plasticity During the M-step, we aim to increase the EM lower bound F in (6) w. [sent-146, score-1.219]

68 Gradient ascent yields, ∂Vki F (V , q(z|y)) = ∂Vki log p(y, z|V ) p∗ (y)q(z|y) = zk · (yj − σ(Vki )) p∗ (y)q(z|y) (13) , (14) ∗ where q is the variational distribution determined during the E-step, i. [sent-150, score-0.384]

69 Note the formal correspondence of (14) with the network synaptic learning rule (8). [sent-153, score-0.382]

70 Indeed, if the network activity implements q ∗ , it can be shown easily that the expected update of synaptic weights due to the synaptic plasticity (8) is proportional to (14), and hence implements a stochastic version of the theoretical M-step (cf. [sent-154, score-0.98]

71 The following observations can be made: Due to the homeostatic constraint, each neuron responds on average to mk · T out of T presented inputs. [sent-160, score-0.679]

72 3 Homeostatic plasticity in recurrent spiking networks The neural model so far was essentially a feed-forward network, in which every postsynaptic spike can directly be interpreted as one sample of the instantaneous posterior distribution [12]. [sent-170, score-0.838]

73 We will now extend the concept of homeostatic processes as posterior constraints to the broader class of recurrent networks and sketch the utility of the developed framework beyond the regulation of intrinsic excitabilities. [sent-172, score-0.858]

74 Recently it was shown in [9, 10] that recurrent networks of stochastically spiking neurons can in principle carry out probabilistic inference through a sampling process. [sent-173, score-0.568]

75 However, [9] and [10] did not consider unsupervised learning on spiking input streams. [sent-175, score-0.318]

76 First, we deﬁne a Boltzmann distribution, ˆk zk + 1 ˆ p(z) = exp( b Wkj zk zj )/norm. [sent-177, score-0.623]

77 , (15) 2 k j=k ˆ ˆ with Wkj = Wjk as “prior” for the hidden variables z which will be represented by a recurrently ˆ connected network of K spiking neurons. [sent-178, score-0.426]

78 Secondly, we deﬁne a conditional distribution in the exponential-family form [23], Vki zk fi (y) − A(z, V )) , p(y|z, V ) = exp(f0 (y) + (16) k,i that speciﬁes the likelihood of observable inputs y, given a certain network state z. [sent-180, score-0.448]

79 We map this probabilistic model to the spiking network and deﬁne that for every k and every point in time t the variable zk (t) has the value 1, if the corresponding neuron has ﬁred within the time window (t − τ, t]. [sent-182, score-0.787]

80 are given by the decomposition of A(z, V ) along the binary combinations of z as, A(z, V ) = A0 (V ) + zk Ak (V ) + k 1 2 zk zj Akj (V ) + . [sent-197, score-0.623]

81 Inb stead we proceed as in the last section and specify a-priori desired properties of the average network response under the input distribution p∗ (y), ckj = zk zj p∗ (y)q(z|y) and 7 mk = z k p∗ (y)q(z|y) . [sent-201, score-0.697]

82 One obvious choice is closely re1 lated to the goal of maximizing the entropy of the output code by ﬁxing zk to K and zk zj 1 to zk zj = K 2 , thus enforcing second order correlations to be zero. [sent-203, score-0.963]

83 Another intuitive choice would be to set all zk zj very close to zero, which excludes that two neurons can be active simultaneously and thus recovers the function of a WTA. [sent-204, score-0.454]

84 With this in mind, we can formulate the goal of learning for the network in the context of EM with posterior constraints: we constrain the E-step such that the average posterior fulﬁlls the chosen targets, and adapt the forward weights V in the M-step according to (6). [sent-207, score-0.28]

85 Analogous to the ﬁrst-order case, the variational solution of the E-step under these constraints takes the form,   1 qβ,ω (z|y) ∝ p(z|y, V ) · exp  βk zk + ωkj zk zj  , (20) 2 k j=k with symmetric ωkl = ωlk as variational parameters. [sent-208, score-0.814]

86 A neural sampling network N with input weights Vki will sample from qβ,ω if the intrinsic excitabilities are set to bk = −Ak + ˆk + βk , and b ˆ the symmetric recurrent synaptic weights to Wkj = −Akj + Wkj + ωkj . [sent-209, score-0.979]

87 Analogous to the last section, this yields the intrinsic plasticity rule (12) for bk . [sent-211, score-0.604]

88 In addition, we obtain for the recurrent synapses Wkj , ∆Wkj ∝ ckj − zk zj p∗ (y)q(z|y) , (21) which translates to an anti-Hebbian spike-timing dependent plasticity rule in the network implementation. [sent-212, score-0.951]

89 4 Discussion Complex and non-local computations, which appear during probabilistic inference and learning, arguably constitute one of the cardinal challenges in the development of biologically realistic Bayesian spiking network models. [sent-221, score-0.517]

90 In this paper we have introduced homeostatic plasticity, which to the best of our knowledge had not been considered before in the context of EM in spiking networks, as a theoretically grounded approach to stabilize and facilitate learning in a large class of network models. [sent-222, score-0.903]

91 Indeed, our results challenge the hypothesis of [15] that feedforward inhibition is critical for correctly learning the structure of the data with biologically plausible plasticity rules. [sent-224, score-0.415]

92 More generally, it turns out that the enforcement of a balancing posterior constraint often simpliﬁes inference in recurrent spiking networks by eliminating nontrivial computations. [sent-225, score-0.566]

93 Our results suggest a crucial role of homeostatic plasticity in the Bayesian brain: to constrain activity patterns in cortex to assist the autonomous optimization of an internal model of the environment. [sent-226, score-0.863]

94 Neural dynamics as sampling: A model for stochastic computation in recurrent networks of spiking neurons. [sent-290, score-0.444]

95 Probabilistic inference in general graphical models through sampling in stochastic networks of spiking neurons. [sent-296, score-0.344]

96 STDP enables spiking neurons to detect hidden causes of their inputs. [sent-306, score-0.413]

97 Sequence learning with hidden units in spiking neural networks. [sent-315, score-0.299]

98 Feedforward inhibition and synaptic scaling–two sides of the same coin? [sent-332, score-0.255]

99 Plasticity in the intrinsic excitability of cortical pyramidal neurons. [sent-357, score-0.276]

100 Homeostatic plasticity and STDP: keeping a neurons cool in a ﬂuctuating world. [sent-362, score-0.439]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('homeostatic', 0.481), ('plasticity', 0.325), ('wta', 0.317), ('zk', 0.283), ('spiking', 0.278), ('vki', 0.266), ('synaptic', 0.223), ('excitabilities', 0.199), ('intrinsic', 0.16), ('ak', 0.16), ('network', 0.127), ('mk', 0.122), ('neurons', 0.114), ('wkj', 0.091), ('recurrent', 0.087), ('bk', 0.087), ('variational', 0.083), ('excitability', 0.08), ('neuron', 0.076), ('homeostasis', 0.059), ('zj', 0.057), ('nessler', 0.053), ('posterior', 0.05), ('dynamics', 0.047), ('em', 0.047), ('spike', 0.047), ('ki', 0.046), ('nontrivial', 0.044), ('akj', 0.041), ('activation', 0.04), ('input', 0.04), ('ckj', 0.04), ('rnet', 0.04), ('inputs', 0.038), ('ful', 0.036), ('biologically', 0.035), ('inference', 0.034), ('digits', 0.033), ('orban', 0.032), ('networks', 0.032), ('rule', 0.032), ('inhibition', 0.032), ('patterns', 0.032), ('interplay', 0.03), ('bayesian', 0.03), ('weights', 0.028), ('period', 0.028), ('sketched', 0.028), ('response', 0.028), ('rules', 0.027), ('implements', 0.027), ('keck', 0.027), ('rezende', 0.027), ('mnist', 0.026), ('constrain', 0.025), ('mind', 0.025), ('constraints', 0.025), ('yi', 0.025), ('computations', 0.025), ('ongoing', 0.024), ('fiser', 0.023), ('brea', 0.023), ('probabilistic', 0.023), ('uk', 0.023), ('feedforward', 0.023), ('sketch', 0.023), ('targets', 0.022), ('kl', 0.022), ('complements', 0.022), ('bill', 0.022), ('buesing', 0.022), ('neuromorphic', 0.022), ('hidden', 0.021), ('balancing', 0.021), ('spikes', 0.021), ('digit', 0.02), ('maximization', 0.02), ('lls', 0.02), ('afferent', 0.02), ('enforcement', 0.02), ('realistic', 0.02), ('pyramidal', 0.019), ('plos', 0.019), ('ring', 0.019), ('perspective', 0.019), ('instantaneous', 0.019), ('erroneous', 0.018), ('everyday', 0.018), ('kj', 0.018), ('innovative', 0.018), ('dual', 0.018), ('ascent', 0.018), ('specialized', 0.018), ('neuronal', 0.018), ('circumvent', 0.018), ('stdp', 0.018), ('cortical', 0.017), ('target', 0.017), ('berkes', 0.017), ('theoretically', 0.017)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.99999982 152 nips-2012-Homeostatic plasticity in Bayesian spiking networks as Expectation Maximization with posterior constraints

Author: Stefan Habenschuss, Johannes Bill, Bernhard Nessler

2 0.28262496 190 nips-2012-Learning optimal spike-based representations

Author: Ralph Bourdoukan, David Barrett, Sophie Deneve, Christian K. Machens

Abstract: How can neural networks learn to represent information optimally? We answer this question by deriving spiking dynamics and learning dynamics directly from a measure of network performance. We ﬁnd that a network of integrate-and-ﬁre neurons undergoing Hebbian plasticity can learn an optimal spike-based representation for a linear decoder. The learning rule acts to minimise the membrane potential magnitude, which can be interpreted as a representation error after learning. In this way, learning reduces the representation error and drives the network into a robust, balanced regime. The network becomes balanced because small representation errors correspond to small membrane potentials, which in turn results from a balance of excitation and inhibition. The representation is robust because neurons become self-correcting, only spiking if the representation error exceeds a threshold. Altogether, these results suggest that several observed features of cortical dynamics, such as excitatory-inhibitory balance, integrate-and-ﬁre dynamics and Hebbian plasticity, are signatures of a robust, optimal spike-based code. A central question in neuroscience is to understand how populations of neurons represent information and how they learn to do so. Usually, learning and information representation are treated as two different functions. From the outset, this separation seems like a good idea, as it reduces the problem into two smaller, more manageable chunks. Our approach, however, is to study these together. This allows us to treat learning and information representation as two sides of a single mechanism, operating at two different timescales. Experimental work has given us several clues about the regime in which real networks operate in the brain. Some of the most prominent observations are: (a) high trial-to-trial variability—a neuron responds differently to repeated, identical inputs [1, 2]; (b) asynchronous ﬁring at the network level—spike trains of different neurons are at most very weakly correlated [3, 4, 5]; (c) tight balance of excitation and inhibition—every excitatory input is met by an inhibitory input of equal or greater size [6, 7, 8] and (4) spike-timing-dependent plasticity (STDP)—the strength of synapses change as a function of presynaptic and postsynaptic spike times [9]. Previously, it has been shown that observations (a)–(c) can be understood as signatures of an optimal, spike-based code [10, 11]. The essential idea is to derive spiking dynamics from the assumption that neurons only ﬁre if their spike improves information representation. Information in a network may ∗ Authors contributed equally 1 originate from several possible sources: external sensory input, external neural network input, or alternatively, it may originate within the network itself as a memory, or as a computation. Whatever the source, this initial assumption leads directly to the conclusion that a network of integrate-and-ﬁre neurons can optimally represent a signal while exhibiting properties (a)–(c). A major problem with this framework is that network connectivity must be completely speciﬁed a priori, and requires the tuning of N 2 parameters, where N is the number of neurons in the network. Although this is feasible mathematically, it is unclear how a real network could tune itself into this optimal regime. In this work, we solve this problem using a simple synaptic learning rule. The key insight is that the plasticity rule can be derived from the same basic principle as the spiking rule in the earlier work—namely, that any change should improve information representation. Surprisingly, this can be achieved with a local, Hebbian learning rule, where synaptic plasticity is proportional to the product of presynaptic ﬁring rates with post-synaptic membrane potentials. Spiking and synaptic plasticity then work hand in hand towards the same goal: the spiking of a neuron decreases the representation error on a fast time scale, thereby giving rise to the actual population representation; synaptic plasticity decreases the representation error on a slower time scale, thereby improving or maintaining the population representation. For a large set of initial connectivities and spiking dynamics, neural networks are driven into a balanced regime, where excitation and inhibition cancel each other and where spike trains are asynchronous and irregular. Furthermore, the learning rule that we derive reproduces the main features of STDP (property (d) above). In this way, a network can learn to represent information optimally, with synaptic, neural and network dynamics consistent with those observed experimentally. 1 Derivation of the learning rule for a single neuron We begin by deriving a learning rule for a single neuron with an autapse (a self-connection) (Fig. 1A). Our approach is to derive synaptic dynamics for the autapse and spiking dynamics for the neuron such that the neuron learns to optimally represent a time-varying input signal. We will derive a learning rule for networks of neurons later, after we have developed the fundamental concepts for the single neuron case. Our ﬁrst step is to derive optimal spiking dynamics for the neuron, so that we have a target for our learning rule. We do this by making two simple assumptions [11]. First, we assume that the neuron can provide an estimate or read-out x(t) of a time-dependent signal x(t) by ﬁltering its spike train ˆ o(t) as follows: ˙ x(t) = −ˆ(t) + Γo(t), ˆ x (1) where Γ is a ﬁxed read-out weight, which we will refer to as the neuron’s “output kernel” and the spike train can be written as o(t) = i δ(t − ti ), where {ti } are the spike times. Next, we assume that the neuron only produces a spike if that spike improves the read-out, where we measure the read-out performance through a simple squared-error loss function: 2 L(t) = x(t) − x(t) . ˆ (2) With these two assumptions, we can now derive optimal spiking dynamics. First, we observe that if the neuron produces an additional spike at time t, the read-out increases by Γ, and the loss function becomes L(t|spike) = (x(t) − (x(t) + Γ))2 . This allows us to restate our spiking rule as follows: ˆ the neuron should only produce a spike if L(t|no spike) > L(t|spike), or (x(t) − x(t))2 > (x(t) − ˆ (x(t) + Γ))2 . Now, squaring both sides of this inequality, deﬁning V (t) ≡ Γ(x(t) − x(t)) and ˆ ˆ deﬁning T ≡ Γ2 /2 we ﬁnd that the neuron should only spike if: V (t) > T. (3) We interpret V (t) to be the membrane potential of the neuron, and we interpret T as the spike threshold. This interpretation allows us to understand the membrane potential functionally: the voltage is proportional to a prediction error—the difference between the read-out x(t) and the actual ˆ signal x(t). A spike is an error reduction mechanism—the neuron only spikes if the error exceeds the spike threshold. This is a greedy minimisation, in that the neuron ﬁres a spike whenever that action decreases L(t) without considering the future impact of that spike. Importantly, the neuron does not require direct access to the loss function L(t). 2 To determine the membrane potential dynamics, we take the derivative of the voltage, which gives ˙ ˙ us V = Γ(x − x). (Here, and in the following, we will drop the time index for notational brevity.) ˙ ˆ ˙ Now, using Eqn. (1) we obtain V = Γx − Γ(−x + Γo) = −Γ(x − x) + Γ(x + x) − Γ2 o, so that: ˙ ˆ ˆ ˙ ˙ V = −V + Γc − Γ2 o, (4) where c = x + x is the neural input. This corresponds exactly to the dynamics of a leaky integrate˙ and-ﬁre neuron with an inhibitory autapse1 of strength Γ2 , and a feedforward connection strength Γ. The dynamics and connectivity guarantee that a neuron spikes at just the right times to optimise the loss function (Fig. 1B). In addition, it is especially robust to noise of different forms, because of its error-correcting nature. If x is constant in time, the voltage will rise up to the threshold T at which point a spike is ﬁred, adding a delta function to the spike train o at time t, thereby producing a read-out x that is closer to x and causing an instantaneous drop in the voltage through the autapse, ˆ by an amount Γ2 = 2T , effectively resetting the voltage to V = −T . We now have a target for learning—we know the connection strength that a neuron must have at the end of learning if it is to represent information optimally, for a linear read-out. We can use this target to derive synaptic dynamics that can learn an optimal representation from experience. Speciﬁcally, we consider an integrate-and-ﬁre neuron with some arbitrary autapse strength ω. The dynamics of this neuron are given by ˙ V = −V + Γc − ωo. (5) This neuron will not produce the correct spike train for representing x through a linear read-out (Eqn. (1)) unless ω = Γ2 . Our goal is to derive a dynamical equation for the synapse ω so that the spike train becomes optimal. We do this by quantifying the loss that we are incurring by using the suboptimal strength, and then deriving a learning rule that minimises this loss with respect to ω. The loss function underlying the spiking dynamics determined by Eqn. (5) can be found by reversing the previous membrane potential analysis. First, we integrate the differential equation for V , assuming that ω changes on time scales much slower than the membrane potential. We obtain the following (formal) solution: V = Γx − ω¯, o (6) ˙ where o is determined by o = −¯ + o. The solution to this latter equation is o = h ∗ o, a convolution ¯ ¯ o ¯ of the spike train with the exponential kernel h(τ ) = θ(τ ) exp(−τ ). As such, it is analogous to the instantaneous ﬁring rate of the neuron. Now, using Eqn. (6), and rewriting the read-out as x = Γ¯, we obtain the loss incurred by the ˆ o sub-optimal neuron, L = (x − x)2 = ˆ 1 V 2 + 2(ω − Γ2 )¯ + (ω − Γ2 )2 o2 . o ¯ Γ2 (7) We observe that the last two terms of Eqn. (7) will vanish whenever ω = Γ2 , i.e., when the optimal reset has been found. We can therefore simplify the problem by deﬁning an alternative loss function, 1 2 V , (8) 2 which has the same minimum as the original loss (V = 0 or x = x, compare Eqn. (2)), but yields a ˆ simpler learning algorithm. We can now calculate how changes to ω affect LV : LV = ∂LV ∂V ∂o ¯ =V = −V o − V ω ¯ . (9) ∂ω ∂ω ∂ω We can ignore the last term in this equation (as we will show below). Finally, using simple gradient descent, we obtain a simple Hebbian-like synaptic plasticity rule: τω = − ˙ ∂LV = V o, ¯ ∂ω (10) where τ is the learning time constant. 1 This contribution of the autapse can also be interpreted as the reset of an integrate-and-ﬁre neuron. Later, when we generalise to networks of neurons, we shall employ this interpretation. 3 This synaptic learning rule is capable of learning the synaptic weight ω that minimises the difference between x and x (Fig. 1B). During learning, the synaptic weight changes in proportion to the postˆ synaptic voltage V and the pre-synaptic ﬁring rate o (Fig. 1C). As such, this is a Hebbian learning ¯ rule. Of course, in this single neuron case, the pre-synaptic neuron and post-synaptic neuron are the same neuron. The synaptic weight gradually approaches its optimal value Γ2 . However, it never completely stabilises, because learning never stops as long as neurons are spiking. Instead, the synapse oscillates closely about the optimal value (Fig. 1D). This is also a “greedy” learning rule, similar to the spiking rule, in that it seeks to minimise the error at each instant in time, without regard for the future impact of those changes. To demonstrate that the second term in Eqn. (5) can be neglected we note that the equations for V , o, and ω deﬁne a system ¯ of coupled differential equations that can be solved analytically by integrating between spikes. This results in a simple recurrence relation for changes in ω from the ith to the (i + 1)th spike, ωi+1 = ωi + ωi (ωi − 2T ) . τ (T − Γc − ωi ) (11) This iterative equation has a single stable ﬁxed point at ω = 2T = Γ2 , proving that the neuron’s autaptic weight or reset will approach the optimal solution. 2 Learning in a homogeneous network We now generalise our learning rule derivation to a network of N identical, homogeneously connected neurons. This generalisation is reasonably straightforward because many characteristics of the single neuron case are shared by a network of identical neurons. We will return to the more general case of heterogeneously connected neurons in the next section. We begin by deriving optimal spiking dynamics, as in the single neuron case. This provides a target for learning, which we can then use to derive synaptic dynamics. As before, we want our network to produce spikes that optimally represent a variable x for a linear read-out. We assume that the read-out x is provided by summing and ﬁltering the spike trains of all the neurons in the network: ˆ ˙ x = −ˆ + Γo, ˆ x (12) 2 where the row vector Γ = (Γ, . . . , Γ) contains the read-out weights of the neurons and the column vector o = (o1 , . . . , oN ) their spike trains. Here, we have used identical read-out weights for each neuron, because this indirectly leads to homogeneous connectivity, as we will demonstrate. Next, we assume that a neuron only spikes if that spike reduces a loss-function. This spiking rule is similar to the single neuron spiking rule except that this time there is some ambiguity about which neuron should spike to represent a signal. Indeed, there are many different spike patterns that provide exactly the same estimate x. For example, one neuron could ﬁre regularly at a high rate (exactly like ˆ our previous single neuron example) while all others are silent. To avoid this ﬁring rate ambiguity, we use a modiﬁed loss function, that selects amongst all equivalent solutions, those with the smallest neural ﬁring rates. We do this by adding a ‘metabolic cost’ term to our loss function, so that high ﬁring rates are penalised: ¯ L = (x − x)2 + µ o 2 , ˆ (13) where µ is a small positive constant that controls the cost-accuracy trade-off, akin to a regularisation parameter. Each neuron in the optimal network will seek to reduce this loss function by ﬁring a spike. Speciﬁcally, the ith neuron will spike whenever L(no spike in i) > L(spike in i). This leads to the following spiking rule for the ith neuron: Vi > Ti (14) where Vi ≡ Γ(x − x) − µoi and Ti ≡ Γ2 /2 + µ/2. We can naturally interpret Vi as the membrane ˆ potential of the ith neuron and Ti as the spiking threshold of that neuron. As before, we can now derive membrane potential dynamics: ˙ V = −V + ΓT c − (ΓT Γ + µI)o, 2 (15) The read-out weights must scale as Γ ∼ 1/N so that ﬁring rates are not unrealistically small in large networks. We can see this by calculating the average ﬁring rate N oi /N ≈ x/(ΓN ) ∼ O(N/N ) ∼ O(1). i=1 ¯ 4 where I is the identity matrix and ΓT Γ + µI is the network connectivity. We can interpret the selfconnection terms {Γ2 +µ} as voltage resets that decrease the voltage of any neuron that spikes. This optimal network is equivalent to a network of identical integrate-and-ﬁre neurons with homogeneous inhibitory connectivity. The network has some interesting dynamical properties. The voltages of all the neurons are largely synchronous, all increasing to the spiking threshold at about the same time3 (Fig. 1F). Nonetheless, neural spiking is asynchronous. The ﬁrst neuron to spike will reset itself by Γ2 + µ, and it will inhibit all the other neurons in the network by Γ2 . This mechanism prevents neurons from spik- x 3 The ﬁrst neuron to spike will be random if there is some membrane potential noise. V (A) (B) x x ˆ x 10 1 0.1 0 50 100 150 200 250 300 350 400 0 50 100 150 200 250 300 350 400 1 D 0.5 V V 0 ˆ x V ˆ x (C) 1 0 1 2 0 0.625 25 25.625 (D) start of learning 1 V 50 200.625 400 400.625 1 2.4 O 1.78 ω 1.77 25 neuron$ 0 1 2 !me$ 3 4 25 1 5 V 400.625 !me$ (F) 25 1 2.35 1.05 1.049 400 25.625 !me$ (E) neuron$ 100.625 200 end of learning 1.4 1.35 ω 100 !me$ 1 V 1 O 50.625 0 1 2 !me$ 3 4 5 V !me$ !me$ Figure 1: Learning in a single neuron and a homogeneous network. (A) A single neuron represents an input signal x by producing an output x. (B) During learning, the single neuron output x (solid red ˆ ˆ line, top panel) converges towards the input x (blue). Similarly, for a homogeneous network the output x (dashed red line, top panel) converges towards x. Connectivity also converges towards optimal ˆ connectivity in both the single neuron case (solid black line, middle panel) and the homogeneous net2 2 work case (dashed black line, middle panel), as quantiﬁed by D = maxi,j ( Ωij − Ωopt / Ωopt ) ij ij at each point in time. Consequently, the membrane potential reset (bottom panel) converges towards the optimal reset (green line, bottom panel). Spikes are indicated by blue vertical marks, and are produced when the membrane potential reaches threshold (bottom panel). Here, we have rescaled time, as indicated, for clarity. (C) Our learning rule dictates that the autapse ω in our single neuron (bottom panel) changes in proportion to the membrane potential (top panel) and the ﬁring rate (middle panel). (D) At the end of learning, the reset ω ﬂuctuates weakly about the optimal value. (E) For a homogeneous network, neurons spike regularly at the start of learning, as shown in this raster plot. Membrane potentials of different neurons are weakly correlated. (F) At the end of learning, spiking is very irregular and membrane potentials become more synchronous. 5 ing synchronously. The population as a whole acts similarly to the single neuron in our previous example. Each neuron ﬁres regularly, even if a different neuron ﬁres in every integration cycle. The design of this optimal network requires the tuning of N (N − 1) synaptic parameters. How can an arbitrary network of integrate-and-ﬁre neurons learn this optimum? As before, we address this question by using the optimal network as a target for learning. We start with an arbitrarily connected network of integrate-and-ﬁre neurons: ˙ V = −V + ΓT c − Ωo, (16) where Ω is a matrix of connectivity weights, which includes the resets of the individual neurons. Assuming that learning occurs on a slow time scale, we can rewrite this equation as V = ΓT x − Ω¯ . o (17) Now, repeating the arguments from the single neuron derivation, we modify the loss function to obtain an online learning rule. Speciﬁcally, we set LV = V 2 /2, and calculate the gradient: ∂LV = ∂Ωij Vk k ∂Vk =− ∂Ωij Vk δki oj − ¯ k Vk Ωkl kl ∂ ol ¯ . ∂Ωij (18) We can simplify this equation considerably by observing that the contribution of the second summation is largely averaged out under a wide variety of realistic conditions4 . Therefore, it can be neglected, and we obtain the following local learning rule: ∂LV ˙ = V i oj . ¯ τ Ωij = − ∂Ωij (19) This is a Hebbian plasticity rule, whereby connectivity changes in proportion to the presynaptic ﬁring rate oj and post-synaptic membrane potential Vi . We assume that the neural thresholds are set ¯ to a constant T and that the neural resets are set to their optimal values −T . In the previous section we demonstrated that these resets can be obtained by a Hebbian plasticity rule (Eqn. (10)). This learning rule minimises the difference between the read-out and the signal, by approaching the optimal recurrent connection strengths for the network (Fig. 1B). As in the single neuron case, learning does not stop, so the connection strengths ﬂuctuate close to their optimal value. During learning, network activity becomes progressively more asynchronous as it progresses towards optimal connectivity (Fig. 1E, F). 3 Learning in the general case Now that we have developed the fundamental concepts underlying our learning rule, we can derive a learning rule for the more general case of a network of N arbitrarily connected leaky integrateand-ﬁre neurons. Our goal is to understand how such networks can learn to optimally represent a ˙ J-dimensional signal x = (x1 , . . . , xJ ), using the read-out equation x = −x + Γo. We consider a network with the following membrane potential dynamics: ˙ V = −V + ΓT c − Ωo, (20) where c is a J-dimensional input. We assume that this input is related to the signal according to ˙ c = x + x. This assumption can be relaxed by treating the input as the control for an arbitrary linear dynamical system, in which case the signal represented by the network is the output of such a computation [11]. However, this further generalisation is beyond the scope of this work. As before, we need to identify the optimal recurrent connectivity so that we have a target for learning. Most generally, the optimal recurrent connectivity is Ωopt ≡ ΓT Γ + µI. The output kernels of the individual neurons, Γi , are given by the rows of Γ, and their spiking thresholds by Ti ≡ Γi 2 /2 + 4 From the deﬁnition of the membrane potential we can see that Vk ∼ O(1/N ) because Γ ∼ 1/N . Therefore, the size of the ﬁrst term in Eqn. (18) is k Vk δki oj = Vi oj ∼ O(1/N ). Therefore, the second term can ¯ ¯ be ignored if kl Vk Ωkl ∂ ol /∂Ωij ¯ O(1/N ). This happens if Ωkl O(1/N 2 ) as at the start of learning. It also happens towards the end of learning if the terms {Ωkl ∂ ol /∂Ωij } are weakly correlated with zero mean, ¯ or if the membrane potentials {Vi } are weakly correlated with zero mean. 6 µ/2. With these connections and thresholds, we ﬁnd that a network of integrate-and-ﬁre neurons ˆ ¯ will produce spike trains in such a way that the loss function L = x − x 2 + µ o 2 is minimised, ˆ where the read-out is given by x = Γ¯ . We can show this by prescribing a greedy5 spike rule: o a spike is ﬁred by neuron i whenever L(no spike in i) > L(spike in i) [11]. The resulting spike generation rule is Vi > Ti , (21) ˆ where Vi ≡ ΓT (x − x) − µ¯i is interpreted as the membrane potential. o i 5 Despite being greedy, this spiking rule can generate ﬁring rates that are practically identical to the optimal solutions: we checked this numerically in a large ensemble of networks with randomly chosen kernels. (A) x1 … x … 1 1 (B) xJJ x 10 L 10 T T 10 4 6 8 1 Viii V D ˆˆ ˆˆ x11 xJJ x x F 0.5 0 0.4 … … 0.2 0 0 2000 4000 !me (C) x V V 1 x 10 x 3 ˆ x 8 0 x 10 1 2 3 !me 4 5 4 0 1 4 0 1 8 V (F) Ρ(Δt) E-‐I input 0.4 ˆ x 0 3 0 1 x 10 1.3 0.95 x 10 ˆ x 4 V (E) 1 x 0 end of learning 50 neuron neuron 50 !me 2 0 ˆ x 0 0.5 ISI Δt 1 2 !me 4 5 4 1.5 1.32 3 2 0.1 Ρ(Δt) x E-‐I input (D) start of learning 0 2 !me 0 0 0.5 ISI Δt 1 Figure 2: Learning in a heterogeneous network. (A) A network of neurons represents an input ˆ signal x by producing an output x. (B) During learning, the loss L decreases (top panel). The difference between the connection strengths and the optimal strengths also decreases (middle panel), as 2 2 quantiﬁed by the mean difference (solid line), given by D = Ω − Ωopt / Ωopt and the maxi2 2 mum difference (dashed line), given by maxi,j ( Ωij − Ωopt / Ωopt ). The mean population ﬁring ij ij rate (solid line, bottom panel) also converges towards the optimal ﬁring rate (dashed line, bottom panel). (C, E) Before learning, a raster plot of population spiking shows that neurons produce bursts ˆ of spikes (upper panel). The network output x (red line, middle panel) fails to represent x (blue line, middle panel). The excitatory input (red, bottom left panel) and inhibitory input (green, bottom left panel) to a randomly selected neuron is not tightly balanced. Furthermore, a histogram of interspike intervals shows that spiking activity is not Poisson, as indicated by the red line that represents a best-ﬁt exponential distribution. (D, F) At the end of learning, spiking activity is irregular and ˆ Poisson-like, excitatory and inhibitory input is tightly balanced and x matches x. 7 How can we learn this optimal connection matrix? As before, we can derive a learning rule by minimising the cost function LV = V 2 /2. This leads to a Hebbian learning rule with the same form as before: ˙ τ Ωij = Vi oj . ¯ (22) Again, we assume that the neural resets are given by −Ti . Furthermore, in order for this learning rule to work, we must assume that the network input explores all possible directions in the J-dimensional input space (since the kernels Γi can point in any of these directions). The learning performance does not critically depend on how the input variable space is sampled as long as the exploration is extensive. In our simulations, we randomly sample the input c from a Gaussian white noise distribution at every time step for the entire duration of the learning. We ﬁnd that this learning rule decreases the loss function L, thereby approaching optimal network connectivity and producing optimal ﬁring rates for our linear decoder (Fig. 2B). In this example, we have chosen connectivity that is initially much too weak at the start of learning. Consequently, the initial network behaviour is similar to a collection of unconnected single neurons that ignore each other. Spike trains are not Poisson-like, ﬁring rates are excessively large, excitatory and inhibitory ˆ input is unbalanced and the decoded variable x is highly unreliable (Fig. 2C, E). As a result of learning, the network becomes tightly balanced and the spike trains become asynchronous, irregular and Poisson-like with much lower rates (Fig. 2D, F). However, despite this apparent variability, the population representation is extremely precise, only limited by the the metabolic cost and the discrete nature of a spike. This learnt representation is far more precise than a rate code with independent Poisson spike trains [11]. In particular, shufﬂing the spike trains in response to identical inputs drastically degrades this precision. 4 Conclusions and Discussion In population coding, large trial-to-trial spike train variability is usually interpreted as noise [2]. We show here that a deterministic network of leaky integrate-and-ﬁre neurons with a simple Hebbian plasticity rule can self-organise into a regime where information is represented far more precisely than in noisy rate codes, while appearing to have noisy Poisson-like spiking dynamics. Our learning rule (Eqn. (22)) has the basic properties of STDP. Speciﬁcally, a presynaptic spike occurring immediately before a post-synaptic spike will potentiate a synapse, because membrane potentials are positive immediately before a postsynaptic spike. Furthermore, a presynaptic spike occurring immediately after a post-synaptic spike will depress a synapse, because membrane potentials are always negative immediately after a postsynaptic spike. This is similar in spirit to the STDP rule proposed in [12], but different to classical STDP, which depends on post-synaptic spike times [9]. This learning rule can also be understood as a mechanism for generating a tight balance between excitatory and inhibitory input. We can see this by observing that membrane potentials after learning can be interpreted as representation errors (projected onto the read-out kernels). Therefore, learning acts to minimise the magnitude of membrane potentials. Excitatory and inhibitory input must be balanced if membrane potentials are small, so we can equate balance with optimal information representation. Previous work has shown that the balanced regime produces (quasi-)chaotic network dynamics, thereby accounting for much observed cortical spike train variability [13, 14, 4]. Moreover, the STDP rule has been known to produce a balanced regime [16, 17]. Additionally, recent theoretical studies have suggested that the balanced regime plays an integral role in network computation [15, 13]. In this work, we have connected these mechanisms and functions, to conclude that learning this balance is equivalent to the development of an optimal spike-based population code, and that this learning can be achieved using a simple Hebbian learning rule. Acknowledgements We are grateful for generous funding from the Emmy-Noether grant of the Deutsche Forschungsgemeinschaft (CKM) and the Chaire d’excellence of the Agence National de la Recherche (CKM, DB), as well as a James Mcdonnell Foundation Award (SD) and EU grants BACS FP6-IST-027140, BIND MECT-CT-20095-024831, and ERC FP7-PREDSPIKE (SD). 8 References [1] Tolhurst D, Movshon J, and Dean A (1982) The statistical reliability of signals in single neurons in cat and monkey visual cortex. Vision Res 23: 775–785. [2] Shadlen MN, Newsome WT (1998) The variable discharge of cortical neurons: implications for connectivity, computation, and information coding. J Neurosci 18(10): 3870–3896. [3] Zohary E, Newsome WT (1994) Correlated neuronal discharge rate and its implication for psychophysical performance. Nature 370: 140–143. [4] Renart A, de la Rocha J, Bartho P, Hollender L, Parga N, Reyes A, & Harris, KD (2010) The asynchronous state in cortical circuits. Science 327, 587–590. [5] Ecker AS, Berens P, Keliris GA, Bethge M, Logothetis NK, Tolias AS (2010) Decorrelated neuronal ﬁring in cortical microcircuits. Science 327: 584–587. [6] Okun M, Lampl I (2008) Instantaneous correlation of excitation and inhibition during ongoing and sensory-evoked activities. Nat Neurosci 11, 535–537. [7] Shu Y, Hasenstaub A, McCormick DA (2003) Turning on and off recurrent balanced cortical activity. Nature 423, 288–293. [8] Gentet LJ, Avermann M, Matyas F, Staiger JF, Petersen CCH (2010) Membrane potential dynamics of GABAergic neurons in the barrel cortex of behaving mice. Neuron 65: 422–435. [9] Caporale N, Dan Y (2008) Spike-timing-dependent plasticity: a Hebbian learning rule. Annu Rev Neurosci 31: 25–46. [10] Boerlin M, Deneve S (2011) Spike-based population coding and working memory. PLoS Comput Biol 7, e1001080. [11] Boerlin M, Machens CK, Deneve S (2012) Predictive coding of dynamic variables in balanced spiking networks. under review. [12] Clopath C, B¨ sing L, Vasilaki E, Gerstner W (2010) Connectivity reﬂects coding: a model of u voltage-based STDP with homeostasis. Nat Neurosci 13(3): 344–352. [13] van Vreeswijk C, Sompolinsky H (1998) Chaotic balanced state in a model of cortical circuits. Neural Comput 10(6): 1321–1371. [14] Brunel N (2000) Dynamics of sparsely connected networks of excitatory and inhibitory neurons. J Comput Neurosci 8, 183–208. [15] Vogels TP, Rajan K, Abbott LF (2005) Neural network dynamics. Annu Rev Neurosci 28: 357–376. [16] Vogels TP, Sprekeler H, Zenke F, Clopath C, Gerstner W. (2011) Inhibitory plasticity balances excitation and inhibition in sensory pathways and memory networks. Science 334(6062):1569– 73. [17] Song S, Miller KD, Abbott LF (2000) Competitive Hebbian learning through spike-timingdependent synaptic plasticity. Nat Neurosci 3(9): 919–926. 9

3 0.18155684 347 nips-2012-Towards a learning-theoretic analysis of spike-timing dependent plasticity

Author: David Balduzzi, Michel Besserve

Abstract: This paper suggests a learning-theoretic perspective on how synaptic plasticity beneﬁts global brain functioning. We introduce a model, the selectron, that (i) arises as the fast time constant limit of leaky integrate-and-ﬁre neurons equipped with spiking timing dependent plasticity (STDP) and (ii) is amenable to theoretical analysis. We show that the selectron encodes reward estimates into spikes and that an error bound on spikes is controlled by a spiking margin and the sum of synaptic weights. Moreover, the efﬁcacy of spikes (their usefulness to other reward maximizing selectrons) also depends on total synaptic strength. Finally, based on our analysis, we propose a regularized version of STDP, and show the regularization improves the robustness of neuronal learning when faced with multiple stimuli. 1

4 0.16934082 238 nips-2012-Neurally Plausible Reinforcement Learning of Working Memory Tasks

Author: Jaldert Rombouts, Pieter Roelfsema, Sander M. Bohte

Abstract: A key function of brains is undoubtedly the abstraction and maintenance of information from the environment for later use. Neurons in association cortex play an important role in this process: by learning these neurons become tuned to relevant features and represent the information that is required later as a persistent elevation of their activity [1]. It is however not well known how such neurons acquire these task-relevant working memories. Here we introduce a biologically plausible learning scheme grounded in Reinforcement Learning (RL) theory [2] that explains how neurons become selective for relevant information by trial and error learning. The model has memory units which learn useful internal state representations to solve working memory tasks by transforming partially observable Markov decision problems (POMDP) into MDPs. We propose that synaptic plasticity is guided by a combination of attentional feedback signals from the action selection stage to earlier processing levels and a globally released neuromodulatory signal. Feedback signals interact with feedforward signals to form synaptic tags at those connections that are responsible for the stimulus-response mapping. The neuromodulatory signal interacts with tagged synapses to determine the sign and strength of plasticity. The learning scheme is generic because it can train networks in different tasks, simply by varying inputs and rewards. It explains how neurons in association cortex learn to 1) temporarily store task-relevant information in non-linear stimulus-response mapping tasks [1, 3, 4] and 2) learn to optimally integrate probabilistic evidence for perceptual decision making [5, 6]. 1

5 0.13981609 362 nips-2012-Waveform Driven Plasticity in BiFeO3 Memristive Devices: Model and Implementation

Author: Christian Mayr, Paul Stärke, Johannes Partzsch, Love Cederstroem, Rene Schüffny, Yao Shuai, Nan Du, Heidemarie Schmidt

Abstract: Memristive devices have recently been proposed as efﬁcient implementations of plastic synapses in neuromorphic systems. The plasticity in these memristive devices, i.e. their resistance change, is deﬁned by the applied waveforms. This behavior resembles biological synapses, whose plasticity is also triggered by mechanisms that are determined by local waveforms. However, learning in memristive devices has so far been approached mostly on a pragmatic technological level. The focus seems to be on ﬁnding any waveform that achieves spike-timing-dependent plasticity (STDP), without regard to the biological veracity of said waveforms or to further important forms of plasticity. Bridging this gap, we make use of a plasticity model driven by neuron waveforms that explains a large number of experimental observations and adapt it to the characteristics of the recently introduced BiFeO3 memristive material. Based on this approach, we show STDP for the ﬁrst time for this material, with learning window replication superior to previous memristor-based STDP implementations. We also demonstrate in measurements that it is possible to overlay short and long term plasticity at a memristive device in the form of the well-known triplet plasticity. To the best of our knowledge, this is the ﬁrst implementations of triplet plasticity on any physical memristive device. 1

6 0.11714027 77 nips-2012-Complex Inference in Neural Circuits with Probabilistic Population Codes and Topic Models

7 0.11322547 195 nips-2012-Learning visual motion in recurrent neural networks

8 0.1073864 322 nips-2012-Spiking and saturating dendrites differentially expand single neuron computation capacity

9 0.083342746 239 nips-2012-Neuronal Spike Generation Mechanism as an Oversampling, Noise-shaping A-to-D converter

10 0.075301737 112 nips-2012-Efficient Spike-Coding with Multiplicative Adaptation in a Spike Response Model

11 0.073983796 138 nips-2012-Fully Bayesian inference for neural models with negative-binomial spiking

12 0.072757505 94 nips-2012-Delay Compensation with Dynamical Synapses

13 0.072708793 120 nips-2012-Exact and Stable Recovery of Sequences of Signals with Sparse Increments via Differential 1-Minimization

14 0.068658933 224 nips-2012-Multi-scale Hyper-time Hardware Emulation of Human Motor Nervous System Based on Spiking Neurons using FPGA

15 0.066652976 24 nips-2012-A mechanistic model of early sensory processing based on subtracting sparse representations

16 0.065391861 278 nips-2012-Probabilistic n-Choose-k Models for Classification and Ranking

17 0.063279644 354 nips-2012-Truly Nonparametric Online Variational Inference for Hierarchical Dirichlet Processes

18 0.061812852 158 nips-2012-ImageNet Classification with Deep Convolutional Neural Networks

19 0.059732649 91 nips-2012-Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images

20 0.05580391 114 nips-2012-Efficient coding provides a direct link between prior and likelihood in perceptual Bayesian inference

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, 0.131), (1, 0.029), (2, -0.092), (3, 0.14), (4, -0.085), (5, 0.279), (6, -0.003), (7, 0.041), (8, -0.005), (9, 0.121), (10, -0.01), (11, 0.03), (12, 0.08), (13, 0.016), (14, 0.01), (15, 0.036), (16, -0.027), (17, 0.022), (18, 0.011), (19, -0.013), (20, -0.054), (21, -0.086), (22, 0.023), (23, 0.054), (24, -0.033), (25, 0.018), (26, 0.042), (27, 0.026), (28, -0.062), (29, -0.006), (30, -0.014), (31, -0.053), (32, -0.016), (33, 0.068), (34, -0.047), (35, -0.002), (36, -0.055), (37, -0.036), (38, -0.144), (39, -0.043), (40, -0.006), (41, -0.036), (42, -0.102), (43, -0.033), (44, 0.064), (45, -0.07), (46, 0.13), (47, -0.018), (48, -0.036), (49, 0.131)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.94488436 152 nips-2012-Homeostatic plasticity in Bayesian spiking networks as Expectation Maximization with posterior constraints

Author: Stefan Habenschuss, Johannes Bill, Bernhard Nessler

2 0.8998462 362 nips-2012-Waveform Driven Plasticity in BiFeO3 Memristive Devices: Model and Implementation

Author: Christian Mayr, Paul Stärke, Johannes Partzsch, Love Cederstroem, Rene Schüffny, Yao Shuai, Nan Du, Heidemarie Schmidt

3 0.87136865 322 nips-2012-Spiking and saturating dendrites differentially expand single neuron computation capacity

Author: Romain Cazé, Mark Humphries, Boris S. Gutkin

Abstract: The integration of excitatory inputs in dendrites is non-linear: multiple excitatory inputs can produce a local depolarization departing from the arithmetic sum of each input’s response taken separately. If this depolarization is bigger than the arithmetic sum, the dendrite is spiking; if the depolarization is smaller, the dendrite is saturating. Decomposing a dendritic tree into independent dendritic spiking units greatly extends its computational capacity, as the neuron then maps onto a two layer neural network, enabling it to compute linearly non-separable Boolean functions (lnBFs). How can these lnBFs be implemented by dendritic architectures in practise? And can saturating dendrites equally expand computational capacity? To address these questions we use a binary neuron model and Boolean algebra. First, we conﬁrm that spiking dendrites enable a neuron to compute lnBFs using an architecture based on the disjunctive normal form (DNF). Second, we prove that saturating dendrites as well as spiking dendrites enable a neuron to compute lnBFs using an architecture based on the conjunctive normal form (CNF). Contrary to a DNF-based architecture, in a CNF-based architecture, dendritic unit tunings do not imply the neuron tuning, as has been observed experimentally. Third, we show that one cannot use a DNF-based architecture with saturating dendrites. Consequently, we show that an important family of lnBFs implemented with a CNF-architecture can require an exponential number of saturating dendritic units, whereas the same family implemented with either a DNF-architecture or a CNF-architecture always require a linear number of spiking dendritic units. This minimization could explain why a neuron spends energetic resources to make its dendrites spike. 1

4 0.83742702 190 nips-2012-Learning optimal spike-based representations

Author: Ralph Bourdoukan, David Barrett, Sophie Deneve, Christian K. Machens

5 0.80019122 347 nips-2012-Towards a learning-theoretic analysis of spike-timing dependent plasticity

Author: David Balduzzi, Michel Besserve

6 0.71033186 224 nips-2012-Multi-scale Hyper-time Hardware Emulation of Human Motor Nervous System Based on Spiking Neurons using FPGA

7 0.6847651 239 nips-2012-Neuronal Spike Generation Mechanism as an Oversampling, Noise-shaping A-to-D converter

8 0.63578278 238 nips-2012-Neurally Plausible Reinforcement Learning of Working Memory Tasks

9 0.55937421 77 nips-2012-Complex Inference in Neural Circuits with Probabilistic Population Codes and Topic Models

10 0.52495658 39 nips-2012-Analog readout for optical reservoir computers

11 0.44773486 24 nips-2012-A mechanistic model of early sensory processing based on subtracting sparse representations

12 0.43237469 195 nips-2012-Learning visual motion in recurrent neural networks

13 0.39570907 112 nips-2012-Efficient Spike-Coding with Multiplicative Adaptation in a Spike Response Model

14 0.3570146 79 nips-2012-Compressive neural representation of sparse, high-dimensional probabilities

15 0.34847444 91 nips-2012-Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images

16 0.33417675 354 nips-2012-Truly Nonparametric Online Variational Inference for Hierarchical Dirichlet Processes

17 0.32820213 94 nips-2012-Delay Compensation with Dynamical Synapses

18 0.32493436 177 nips-2012-Learning Invariant Representations of Molecules for Atomization Energy Prediction

19 0.31621182 158 nips-2012-ImageNet Classification with Deep Convolutional Neural Networks

20 0.30296978 73 nips-2012-Coding efficiency and detectability of rate fluctuations with non-Poisson neuronal firing

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(0, 0.032), (17, 0.017), (21, 0.07), (31, 0.273), (38, 0.144), (42, 0.03), (54, 0.014), (55, 0.034), (74, 0.025), (76, 0.095), (77, 0.023), (80, 0.079), (92, 0.037), (94, 0.012)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.76533973 152 nips-2012-Homeostatic plasticity in Bayesian spiking networks as Expectation Maximization with posterior constraints

Author: Stefan Habenschuss, Johannes Bill, Bernhard Nessler

2 0.73501325 119 nips-2012-Entropy Estimations Using Correlated Symmetric Stable Random Projections

Author: Ping Li, Cun-hui Zhang

Abstract: Methods for efﬁciently estimating Shannon entropy of data streams have important applications in learning, data mining, and network anomaly detections (e.g., the DDoS attacks). For nonnegative data streams, the method of Compressed Counting (CC) [11, 13] based on maximally-skewed stable random projections can provide accurate estimates of the Shannon entropy using small storage. However, CC is no longer applicable when entries of data streams can be below zero, which is a common scenario when comparing two streams. In this paper, we propose an algorithm for entropy estimation in general data streams which allow negative entries. In our method, the Shannon entropy is approximated by the ﬁnite difference of two correlated frequency moments estimated from correlated samples of symmetric stable random variables. Interestingly, the estimator for the moment we recommend for entropy estimation barely has bounded variance itself, whereas the common geometric mean estimator (which has bounded higher-order moments) is not sufﬁcient for entropy estimation. Our experiments conﬁrm that this method is able to well approximate the Shannon entropy using small storage.

3 0.68671978 259 nips-2012-Online Regret Bounds for Undiscounted Continuous Reinforcement Learning

Author: Ronald Ortner, Daniil Ryabko

Abstract: We derive sublinear regret bounds for undiscounted reinforcement learning in continuous state space. The proposed algorithm combines state aggregation with the use of upper conﬁdence bounds for implementing optimism in the face of uncertainty. Beside the existence of an optimal policy which satisﬁes the Poisson equation, the only assumptions made are H¨ lder continuity of rewards and transition o probabilities. 1

4 0.66718644 236 nips-2012-Near-Optimal MAP Inference for Determinantal Point Processes

Author: Jennifer Gillenwater, Alex Kulesza, Ben Taskar

Abstract: Determinantal point processes (DPPs) have recently been proposed as computationally efﬁcient probabilistic models of diverse sets for a variety of applications, including document summarization, image search, and pose estimation. Many DPP inference operations, including normalization and sampling, are tractable; however, ﬁnding the most likely conﬁguration (MAP), which is often required in practice for decoding, is NP-hard, so we must resort to approximate inference. This optimization problem, which also arises in experimental design and sensor placement, involves ﬁnding the largest principal minor of a positive semideﬁnite matrix. Because the objective is log-submodular, greedy algorithms have been used in the past with some empirical success; however, these methods only give approximation guarantees in the special case of monotone objectives, which correspond to a restricted class of DPPs. In this paper we propose a new algorithm for approximating the MAP problem based on continuous techniques for submodular function maximization. Our method involves a novel continuous relaxation of the log-probability function, which, in contrast to the multilinear extension used for general submodular functions, can be evaluated and differentiated exactly and efﬁciently. We obtain a practical algorithm with a 1/4-approximation guarantee for a more general class of non-monotone DPPs; our algorithm also extends to MAP inference under complex polytope constraints, making it possible to combine DPPs with Markov random ﬁelds, weighted matchings, and other models. We demonstrate that our approach outperforms standard and recent methods on both synthetic and real-world data. 1

5 0.60030133 64 nips-2012-Calibrated Elastic Regularization in Matrix Completion

Author: Tingni Sun, Cun-hui Zhang

Abstract: This paper concerns the problem of matrix completion, which is to estimate a matrix from observations in a small subset of indices. We propose a calibrated spectrum elastic net method with a sum of the nuclear and Frobenius penalties and develop an iterative algorithm to solve the convex minimization problem. The iterative algorithm alternates between imputing the missing entries in the incomplete matrix by the current guess and estimating the matrix by a scaled soft-thresholding singular value decomposition of the imputed matrix until the resulting matrix converges. A calibration step follows to correct the bias caused by the Frobenius penalty. Under proper coherence conditions and for suitable penalties levels, we prove that the proposed estimator achieves an error bound of nearly optimal order and in proportion to the noise level. This provides a uniﬁed analysis of the noisy and noiseless matrix completion problems. Simulation results are presented to compare our proposal with previous ones. 1

6 0.59954685 302 nips-2012-Scaling MPE Inference for Constrained Continuous Markov Random Fields with Consensus Optimization

7 0.59425807 300 nips-2012-Scalable nonconvex inexact proximal splitting

8 0.59222937 120 nips-2012-Exact and Stable Recovery of Sequences of Signals with Sparse Increments via Differential 1-Minimization

9 0.59190333 333 nips-2012-Synchronization can Control Regularization in Neural Systems via Correlated Noise Processes

10 0.59125984 113 nips-2012-Efficient and direct estimation of a neural subunit model for sensory coding

11 0.59086764 77 nips-2012-Complex Inference in Neural Circuits with Probabilistic Population Codes and Topic Models

12 0.58785951 114 nips-2012-Efficient coding provides a direct link between prior and likelihood in perceptual Bayesian inference

13 0.58777338 65 nips-2012-Cardinality Restricted Boltzmann Machines

14 0.58702666 105 nips-2012-Dynamic Pruning of Factor Graphs for Maximum Marginal Prediction

15 0.58434558 190 nips-2012-Learning optimal spike-based representations

16 0.58352429 83 nips-2012-Controlled Recognition Bounds for Visual Learning and Exploration

17 0.58187741 241 nips-2012-No-Regret Algorithms for Unconstrained Online Convex Optimization

18 0.58175617 18 nips-2012-A Simple and Practical Algorithm for Differentially Private Data Release

19 0.58154589 227 nips-2012-Multiclass Learning with Simplex Coding

20 0.58124048 216 nips-2012-Mirror Descent Meets Fixed Share (and feels no regret)