nips nips2012 nips2012-94 knowledge-graph by maker-knowledge-mining

94 nips-2012-Delay Compensation with Dynamical Synapses

Source: pdf

Author: Chi Fung, K. Wong, Si Wu

Abstract: Time delay is pervasive in neural information processing. To achieve real-time tracking, it is critical to compensate the transmission and processing delays in a neural system. In the present study we show that dynamical synapses with shortterm depression can enhance the mobility of a continuous attractor network to the extent that the system tracks time-varying stimuli in a timely manner. The state of the network can either track the instantaneous position of a moving stimulus perfectly (with zero-lag) or lead it with an effectively constant time, in agreement with experiments on the head-direction systems in rodents. The parameter regions for delayed, perfect and anticipative tracking correspond to network states that are static, ready-to-move and spontaneously moving, respectively, demonstrating the strong correlation between tracking performance and the intrinsic dynamics of the network. We also ﬁnd that when the speed of the stimulus coincides with the natural speed of the network state, the delay becomes effectively independent of the stimulus amplitude.

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 cn Abstract Time delay is pervasive in neural information processing. [sent-9, score-0.29]

2 In the present study we show that dynamical synapses with shortterm depression can enhance the mobility of a continuous attractor network to the extent that the system tracks time-varying stimuli in a timely manner. [sent-11, score-0.525]

3 The state of the network can either track the instantaneous position of a moving stimulus perfectly (with zero-lag) or lead it with an effectively constant time, in agreement with experiments on the head-direction systems in rodents. [sent-12, score-0.806]

4 The parameter regions for delayed, perfect and anticipative tracking correspond to network states that are static, ready-to-move and spontaneously moving, respectively, demonstrating the strong correlation between tracking performance and the intrinsic dynamics of the network. [sent-13, score-1.358]

5 We also ﬁnd that when the speed of the stimulus coincides with the natural speed of the network state, the delay becomes effectively independent of the stimulus amplitude. [sent-14, score-1.379]

6 1 Introduction Time delay is pervasive in neural information processing. [sent-15, score-0.29]

7 To achieve real-time tracking of fast moving objects, it is critical for a neural system to compensate for the delay; otherwise, the object position perceived by the neural system will lag behind the true object position considerably. [sent-21, score-1.006]

8 A natural way to compensate for delays is to predict the future position of the moving stimulus. [sent-22, score-0.409]

9 Experimental ﬁndings suggested that delay compensations are widely adopted in neural systems. [sent-23, score-0.29]

10 A remarkable example is the head-direction (HD) systems in rodents, which encode the head direction of a rodent in the horizontal plane relative to a static environment [14, 17]. [sent-24, score-0.36]

11 It was found that when the head of a rodent is moving continuously in space, the direction perceived by the HD neurons in the postsubicular cortex has nearly zero-lag with respect to the instantaneous position of the rodent head [18]. [sent-25, score-0.958]

12 More interestingly, in the anterior dorsal thalamic nucleus, the HD neurons perceive the future direction of the rodent head, leading the current position by a constant time [3]. [sent-26, score-0.391]

13 The similar anticipative behavior is also observed in the eyeposition neurons when animals make saccadic eye movement, the so-called saccadic remapping [16]. [sent-27, score-0.42]

14 In human psychophysical experiments, the classic ﬂash-lag effect also supports the notion of delay 1 (a) ext (b) 1 0. [sent-28, score-0.384]

15 6 ext I (x,t) u(x,t) -2 0 x-z(t) 2 0 0 0 50 t/τs 100 Figure 1: (a) Proﬁles of u (x, t) and I ext (x, t) in the absence of STD, where the center of mass of the stimulus is moving with constant velocity v = 0. [sent-34, score-0.903]

16 The underlying cause is that the visual system predicts the future position of the continuously moving object, but is unable do so for the unpredictable ﬂash. [sent-43, score-0.317]

17 In the case of self-motion, such as an animal rotating its head actively or performing saccadic eye movements, the motor command responsible for the motion can serve as a cue for delay compensation. [sent-45, score-0.577]

18 In the case of tracking an external moving stimulus, the neural system has to rely on the moving speed of the stimulus for prediction. [sent-48, score-1.283]

19 Asymmetric neural interactions have been proposed to drive the network states to catch up with changes in head directions [22] or positions [4]. [sent-49, score-0.409]

20 proposed a dynamical routing mechanism to compensate the transmission delay in the visual system, in which retinal neurons dynamically choose a pathway according to the speed of the stimulus, and transmit the signal directly to the future position in the cortex [13]. [sent-52, score-0.741]

21 When STD is sufﬁciently strong, the tracking state of the network can even overtake the moving stimulus, demonstrating its potential for generating predictions. [sent-59, score-0.668]

22 This property endows the neural system the capacity of tracking time-varying stimuli smoothly. [sent-62, score-0.466]

23 Consider a continuous stimulus x being encoded by a neural ensemble. [sent-63, score-0.396]

24 Neurons with preferred stimuli x produce the maximum response when an external stimulus is present at x. [sent-65, score-0.468]

25 For √ 2 k ≤ kc ≡ ρJ0 /(8 2πa), the network holds a continuous family of Gaussian-shaped stationary states when I ext (x, t) = 0. [sent-83, score-0.335]

26 (5) 4a2 where u is the rescaled variable u ≡ ρJ0 u, and u0 is the rescaled bump height. [sent-85, score-0.38]

27 Next, we consider the case that the network receives a moving input, [ ] (x − z0 (t))2 ext I (x, t) = A exp − , 4a2 (6) where A is the magnitude of the input and z0 the stimulus position. [sent-89, score-0.779]

28 Without loss of generality, we consider the stimulus position at time t = 0 to be z0 = 0, and the stimulus moves at a constant speed thereafter, i. [sent-90, score-0.856]

29 Let s ≡ z(t) − z0 (t) be the displacement between the network state and the stimulus position. [sent-93, score-0.643]

30 (7) τs 8a Note that s has the opposite sign of v, implying that the network state always trails behind the stimulus (see Fig. [sent-95, score-0.491]

31 This is due to the response delay of the network relative to the input. [sent-97, score-0.413]

32 3 3 Tracking in the Presence of STD The analysis of tracking in the presence of STD is more involved. [sent-98, score-0.315]

33 Motivated by the nearly Gaussianshaped proﬁle of the network states, we adopt a perturbation approach to solve the network dynamics [5]. [sent-99, score-0.393]

34 When STD is weak, the tracking state lags behind the stimulus. [sent-111, score-0.35]

35 When the ¯ STD strength increases to a critical value βperfect , s becomes effectively zero in a rather broad range of stimulus velocity, achieving perfect tracking. [sent-112, score-0.611]

36 When the STD strength is above the critical value, the tracking state leads the stimulus. [sent-113, score-0.41]

37 Hence delay compensation in a tracking task can be implemented at two different levels. [sent-114, score-0.662]

38 The ﬁrst one is perfect tracking, in which the tracking state has zero-lag with respect to the true stimulus position independent of the stimulus speed. [sent-115, score-1.227]

39 The second one is anticipative tracking, in which the tracking state leads by a constant time τant relative to the stimulus position, that is, the tracking state is at the position the stimulus will travel to at a later time τant . [sent-116, score-1.615]

40 To achieve a constant anticipation time, it requires the leading displacement to increase with the stimulus velocity proportionally, i. [sent-117, score-0.71]

41 Both forms of delay compensation have been observed in the head-direction systems of rodents, and may serve different functional purposes. [sent-120, score-0.347]

42 1 Prefect Tracking To analyze the parameter regime for perfect tracking, it is instructive to consider the 1st order perturbation of the network dynamics, i. [sent-122, score-0.326]

43 8 Figure 2: (a) The dependence of the displacement between the bump and the stimulus on the velocity ¯ ¯ ¯ of the moving stimulus for different values of β. [sent-150, score-1.291]

44 When vτd /a ≪ 1, the rescaled ¯ ¯ displacement s/a can be approximated by a power series expansion of the rescaled velocity vτd /a. [sent-169, score-0.465]

45 Since the displacement reverses sign when the velocity reverses, s/a is an odd function of vτd /a. [sent-170, score-0.308]

46 For perfect tracking in the low velocity limit, we have c1 = 0 and ﬁnd s C u0 τs ( vτd )3 ¯ =− , (18) ¯ a 2 A τd a where C is a parameter less than 1 (the detailed expression can be found in Supplementary Material). [sent-172, score-0.582]

47 For the network tracking a moving stimulus, the input magnitude cannot be too small. [sent-173, score-0.633]

48 Therefore, for tracking speeds up to vτd /a ∼ 1, the displacement ¯ ¯ s is very small and can be regarded as zero effectively (see Fig. [sent-175, score-0.546]

49 The velocity range in which the tracking is effectively perfect is rather broad, since it scales as (τd /τs )1/3 ≫ 1. [sent-177, score-0.63]

50 For reference, we also plot the boundary that separates the metastatic phase above it from the static phase below, as reported in the study of intrinsic properties of CANNs with STD in [6]. [sent-182, score-0.364]

51 In the static phase, the bump is stable at any position, whereas in the metastatic phase, the static bump starts to move spontanaeously once it is pushed. [sent-183, score-0.602]

52 8 1 k Figure 3: (a) The anticipatory time as a function of the speed of the stimulus. [sent-215, score-0.351]

53 (b) The contours of constant anticipatory ¯ ¯ time in the space of rescaled inhibition k and the rescaled STD strength β in the limit of very small stimulus speed. [sent-220, score-0.807]

54 2(b)) conﬁrms that βperfect does not change signiﬁcantly with A for ¯ This implies that the network with β = βperfect exhibits effectively perfect ¯ ¯ different values of k. [sent-226, score-0.328]

55 tracking performance because it is intrinsically in a ready-to-move state. [sent-227, score-0.315]

56 2 Anticipative Tracking We further explore the network dynamics when the STD strength is higher than that for achieving perfect tracking. [sent-229, score-0.394]

57 By solving the network dynamics with the perturbation expansion up to the 11th order, we obtain the relation between the displacement s and the stimulus speed v. [sent-230, score-0.864]

58 This implies that the network achieves a constant anticipatory time τant over a broad range of the stimulus speed. [sent-233, score-0.711]

59 To gain insights into how the anticipation time depends on the stimulus speed, we consider the regime of small displacements. [sent-234, score-0.439]

60 In this regime, the rescaled displacement s/a can be approximated by a power series expansion of the rescaled velocity vτd /a, leading to s/a = c1 (vτd /a) + c3 (vτd /a)3 . [sent-235, score-0.465]

61 (19) = − a τd a 3 vmax τd a Hence the anticipatory time is given by ( ) v2 τant (v) = τant (0) 1 − 2 . [sent-238, score-0.285]

62 3vmax (20) This shows that the anticipation time is effectively constant in a wide range of stimulus velocities, as shown in Fig. [sent-239, score-0.487]

63 The contours of anticipatory times for slowly moving stimuli are shown in Fig. [sent-243, score-0.468]

64 Hence the region of anticipative behavior effectively coincides with the metastatic phase, as indicated by the region above the phase line (dashed) in Fig. [sent-245, score-0.369]

65 In summary, there is a direct correspondence between delayed, perfect, and anticipative tracking on one hand, and the static, ready-to-move, and spontaneously moving beahviors on the other. [sent-247, score-0.769]

66 This demonstrates the strong correlation between the tracking performance and the intrinsic behaviors of the CANN. [sent-248, score-0.375]

67 rameter: k Natural Tracking For strong enough STD, a CANN holds spontaneously moving bump states. [sent-290, score-0.454]

68 The speed of the spontaneously moving bump is an intrinsic property of the network depending only on the network parameters. [sent-291, score-0.905]

69 We call this the natural speed of the network, denoted as vnatural . [sent-292, score-0.289]

70 An interesting issue is the tracking performance of the network when the stimulus is moving at its natural speed. [sent-293, score-1.007]

71 This point is referred to as the natural tracking point. [sent-298, score-0.365]

72 It has the important property ¯ that the lag is independent of the stimulus amplitude. [sent-299, score-0.373]

73 Hence the invariant point ¯ ¯ for natural tracking is given by (v, s) = (vnatural , −vnatural τs ) for all values of k and β. [sent-303, score-0.365]

74 ¯ We also consider natural tracking in the weak A limit. [sent-304, score-0.365]

75 Again we ﬁnd a conﬂuence point of the displacement curves at the natural speed, but the delay time (and in some cases the anticipation ¯ ¯ time) depends on the value of k. [sent-305, score-0.597]

76 3, the natural tracking point traces out an ¯ effectively linear curve in the space of v and s when β increases, with a slope equal to 0. [sent-307, score-0.443]

77 Since the delay time is ¯ limit, the natural tracking point is slowly different from the value of τs applicable in the strong A ¯ drifting from the weak to the strong A limit. [sent-312, score-0.612]

78 However, the magnitude of the natural time delay remains of the order of τs . [sent-313, score-0.297]

79 This is conﬁrmed by the analysis of the dynamical equations when the ¯ stimulus speed is vnatural + δv in the weak A limit. [sent-314, score-0.605]

80 4 Extension to other CANNs To investigate whether the delay compensation behavior and the prediction of the natural tracking point are general features of CANN models, we consider a network with Mexican-hat couplings. [sent-316, score-0.844]

81 8 Figure 5: (a) The dependence of anticipatory time on the stimulus speed in the Mexican-hat model. [sent-348, score-0.675]

82 5(a), the anticipatory times are effectively constant and similar in magnitude in the range of stimulus speed comparable to experimental settings. [sent-363, score-0.723]

83 5(b), the natural speed of the bump is zero for β less than a critical value. [sent-365, score-0.392]

84 5(c), the displacement s is plotted as a function of the stimulus speed v. [sent-368, score-0.603]

85 The invariance of the displacement at the natural speed, independent of the stimulus amplitude, also appears in the Mexican-hat model. [sent-369, score-0.526]

86 Furthermore, the displacement at the natural tracking point increases with the natural speed. [sent-371, score-0.567]

87 The latter induces translational instability of neural activities in a CANN and enhances the mobility of the network states in response to external inputs. [sent-374, score-0.347]

88 We found that for strong STD, the neural system can track moving stimuli with either zero-lag or a lead of a constant time. [sent-375, score-0.337]

89 The conditions for perfect and anticipative tracking hold for a wide range of stimulus speeds, making them applicable in practice. [sent-376, score-0.973]

90 By choosing biologically plausible parameters, our model successfully justiﬁes the experimentally observed delay compensation behaviors. [sent-377, score-0.347]

91 We also made an interesting prediction in the network dynamics, that is, when the speed of the stimulus coincides with the natural speed of the network state, the delay becomes effectively independent of the stimulus amplitude. [sent-378, score-1.511]

92 Compared with other delay compensation strategies relying on corollary discharge or dynamical routing, the mechanism we propose here is fully dependent on the intrinsic dynamics of the network, namely, the network automatically “adjusts” its tracking speed according to the input information. [sent-380, score-1.221]

93 There exists strong correlations between tracking performance and the intrinsic dynamics of the network. [sent-381, score-0.458]

94 The parameter regions for delayed, perfect and anticipative tracking correspond to network states being static, ready-to-move and spontaneously moving, respectively. [sent-382, score-0.9]

95 It has been suggested the anticipative response of HD neurons in anterior dorsal thalamus is due to the corollary discharge of motor neurons responsible for moving the head. [sent-383, score-0.85]

96 However, experimental studies revealed that when rats were moved passively (and hence no corollary discharge is available), either by hand or by a chart, the anticipative response of HD neurons still exists and has an even larger leading time [1]. [sent-384, score-0.372]

97 Anticipatory head direction signals in anterior thalamus: evidence for a thalamocortical circuit that integrates angular head motion to compute head direction. [sent-425, score-0.616]

98 Modeling attractor deformation in the rodent head direction system. [sent-476, score-0.368]

99 Path integration and cognitive mappping in a continuous attractor neural network model. [sent-529, score-0.286]

100 Comparisons of head direction cell activity in the postsubiculum and anterior thalamus of freely moving rats. [sent-558, score-0.579]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('stimulus', 0.324), ('tracking', 0.315), ('std', 0.285), ('delay', 0.247), ('anticipatory', 0.224), ('ant', 0.205), ('anticipative', 0.186), ('bump', 0.186), ('moving', 0.186), ('head', 0.162), ('displacement', 0.152), ('perfect', 0.148), ('ext', 0.137), ('network', 0.132), ('speed', 0.127), ('velocity', 0.119), ('anticipation', 0.115), ('vnatural', 0.112), ('compensation', 0.1), ('rescaled', 0.097), ('cann', 0.093), ('rodent', 0.093), ('neurons', 0.087), ('dynamics', 0.083), ('spontaneously', 0.082), ('attractor', 0.082), ('metastatic', 0.082), ('position', 0.081), ('static', 0.074), ('synaptic', 0.071), ('hd', 0.07), ('anterior', 0.066), ('neurotransmitters', 0.066), ('discharge', 0.065), ('ms', 0.063), ('thalamus', 0.061), ('vmax', 0.061), ('vt', 0.06), ('intrinsic', 0.06), ('stimuli', 0.058), ('saccadic', 0.057), ('canns', 0.056), ('rodents', 0.056), ('phase', 0.053), ('external', 0.052), ('mechanism', 0.05), ('natural', 0.05), ('system', 0.05), ('lag', 0.049), ('mobility', 0.049), ('effectively', 0.048), ('delays', 0.047), ('perturbation', 0.046), ('compensate', 0.045), ('motor', 0.045), ('fung', 0.043), ('neural', 0.043), ('dynamical', 0.042), ('synapses', 0.042), ('boundary', 0.042), ('uence', 0.042), ('depression', 0.041), ('hn', 0.041), ('dt', 0.041), ('wong', 0.039), ('neutrally', 0.037), ('nijhawan', 0.037), ('postsubiculum', 0.037), ('reverses', 0.037), ('taube', 0.037), ('states', 0.037), ('freely', 0.036), ('state', 0.035), ('delayed', 0.035), ('interactions', 0.035), ('limit', 0.034), ('separating', 0.034), ('kong', 0.034), ('perceived', 0.034), ('neuronal', 0.034), ('response', 0.034), ('eye', 0.033), ('angular', 0.033), ('dz', 0.033), ('command', 0.033), ('consumed', 0.033), ('dorsal', 0.033), ('routing', 0.033), ('curves', 0.033), ('hong', 0.032), ('strength', 0.031), ('speeds', 0.031), ('broad', 0.031), ('direction', 0.031), ('conjunctive', 0.03), ('symbols', 0.03), ('slope', 0.03), ('continuous', 0.029), ('cortex', 0.029), ('critical', 0.029)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.99999976 94 nips-2012-Delay Compensation with Dynamical Synapses

Author: Chi Fung, K. Wong, Si Wu

2 0.17662247 24 nips-2012-A mechanistic model of early sensory processing based on subtracting sparse representations

Author: Shaul Druckmann, Tao Hu, Dmitri B. Chklovskii

Abstract: Early stages of sensory systems face the challenge of compressing information from numerous receptors onto a much smaller number of projection neurons, a so called communication bottleneck. To make more efficient use of limited bandwidth, compression may be achieved using predictive coding, whereby predictable, or redundant, components of the stimulus are removed. In the case of the retina, Srinivasan et al. (1982) suggested that feedforward inhibitory connections subtracting a linear prediction generated from nearby receptors implement such compression, resulting in biphasic center-surround receptive fields. However, feedback inhibitory circuits are common in early sensory circuits and furthermore their dynamics may be nonlinear. Can such circuits implement predictive coding as well? Here, solving the transient dynamics of nonlinear reciprocal feedback circuits through analogy to a signal-processing algorithm called linearized Bregman iteration we show that nonlinear predictive coding can be implemented in an inhibitory feedback circuit. In response to a step stimulus, interneuron activity in time constructs progressively less sparse but more accurate representations of the stimulus, a temporally evolving prediction. This analysis provides a powerful theoretical framework to interpret and understand the dynamics of early sensory processing in a variety of physiological experiments and yields novel predictions regarding the relation between activity and stimulus statistics.

3 0.17035948 256 nips-2012-On the connections between saliency and tracking

Author: Vijay Mahadevan, Nuno Vasconcelos

Abstract: A model connecting visual tracking and saliency has recently been proposed. This model is based on the saliency hypothesis for tracking which postulates that tracking is achieved by the top-down tuning, based on target features, of discriminant center-surround saliency mechanisms over time. In this work, we identify three main predictions that must hold if the hypothesis were true: 1) tracking reliability should be larger for salient than for non-salient targets, 2) tracking reliability should have a dependence on the deﬁning variables of saliency, namely feature contrast and distractor heterogeneity, and must replicate the dependence of saliency on these variables, and 3) saliency and tracking can be implemented with common low level neural mechanisms. We conﬁrm that the ﬁrst two predictions hold by reporting results from a set of human behavior studies on the connection between saliency and tracking. We also show that the third prediction holds by constructing a common neurophysiologically plausible architecture that can computationally solve both saliency and tracking. This architecture is fully compliant with the standard physiological models of V1 and MT, and with what is known about attentional control in area LIP, while explaining the results of the human behavior experiments.

4 0.14462662 114 nips-2012-Efficient coding provides a direct link between prior and likelihood in perceptual Bayesian inference

Author: Xue-xin Wei, Alan Stocker

Abstract: A common challenge for Bayesian models of perception is the fact that the two fundamental Bayesian components, the prior distribution and the likelihood function, are formally unconstrained. Here we argue that a neural system that emulates Bayesian inference is naturally constrained by the way it represents sensory information in populations of neurons. More speciﬁcally, we show that an efﬁcient coding principle creates a direct link between prior and likelihood based on the underlying stimulus distribution. The resulting Bayesian estimates can show biases away from the peaks of the prior distribution, a behavior seemingly at odds with the traditional view of Bayesian estimation, yet one that has been reported in human perception. We demonstrate that our framework correctly accounts for the repulsive biases previously reported for the perception of visual orientation, and show that the predicted tuning characteristics of the model neurons match the reported orientation tuning properties of neurons in primary visual cortex. Our results suggest that efﬁcient coding is a promising hypothesis in constraining Bayesian models of perceptual inference. 1 Motivation Human perception is not perfect. Biases have been observed in a large number of perceptual tasks and modalities, of which the most salient ones constitute many well-known perceptual illusions. It has been suggested, however, that these biases do not reﬂect a failure of perception but rather an observer’s attempt to optimally combine the inherently noisy and ambiguous sensory information with appropriate prior knowledge about the world [13, 4, 14]. This hypothesis, which we will refer to as the Bayesian hypothesis, has indeed proven quite successful in providing a normative explanation of perception at a qualitative and, more recently, quantitative level (see e.g. [15]). A major challenge in forming models based on the Bayesian hypothesis is the correct selection of two main components: the prior distribution (belief) and the likelihood function. This has encouraged some to criticize the Bayesian hypothesis altogether, claiming that arbitrary choices for these components always allow for unjustiﬁed post-hoc explanations of the data [1]. We do not share this criticism, referring to a number of successful attempts to constrain prior beliefs and likelihood functions based on principled grounds. For example, prior beliefs have been deﬁned as the relative distribution of the sensory variable in the environment in cases where these statistics are relatively easy to measure (e.g. local visual orientations [16]), or where it can be assumed that subjects have learned them over the course of the experiment (e.g. time perception [17]). Other studies have constrained the likelihood function according to known noise characteristics of neurons that are crucially involved in the speciﬁc perceptual process (e.g motion tuned neurons in visual cor∗ http://www.sas.upenn.edu/ astocker/lab 1 world neural representation efficient encoding percept Bayesian decoding Figure 1: Encoding-decoding framework. A stimulus representing a sensory variable θ elicits a ﬁring rate response R = {r1 , r2 , ..., rN } in a population of N neurons. The perceptual task is to generate a ˆ good estimate θ(R) of the presented value of the sensory variable based on this population response. Our framework assumes that encoding is efﬁcient, and decoding is Bayesian based on the likelihood p(R|θ), the prior p(θ), and a squared-error loss function. tex [18]). However, we agree that ﬁnding appropriate constraints is generally difﬁcult and that prior beliefs and likelihood functions have been often selected on the basis of mathematical convenience. Here, we propose that the efﬁcient coding hypothesis [19] offers a joint constraint on the prior and likelihood function in neural implementations of Bayesian inference. Efﬁcient coding provides a normative description of how neurons encode sensory information, and suggests a direct link between measured perceptual discriminability, neural tuning characteristics, and environmental statistics [11]. We show how this link can be extended to a full Bayesian account of perception that includes perceptual biases. We validate our model framework against behavioral as well as neural data characterizing the perception of visual orientation. We demonstrate that we can account not only for the reported perceptual biases away from the cardinal orientations, but also for the speciﬁc response characteristics of orientation-tuned neurons in primary visual cortex. Our work is a novel proposal of how two important normative hypotheses in perception science, namely efﬁcient (en)coding and Bayesian decoding, might be linked. 2 Encoding-decoding framework We consider perception as an inference process that takes place along the simpliﬁed neural encodingdecoding cascade illustrated in Fig. 11 . 2.1 Efﬁcient encoding Efﬁcient encoding proposes that the tuning characteristics of a neural population are adapted to the prior distribution p(θ) of the sensory variable such that the population optimally represents the sensory variable [19]. Different deﬁnitions of “optimally” are possible, and may lead to different results. Here, we assume an efﬁcient representation that maximizes the mutual information between the sensory variable and the population response. With this deﬁnition and an upper limit on the total ﬁring activity, the square-root of the Fisher Information must be proportional to the prior distribution [12, 21]. In order to constrain the tuning curves of individual neurons in the population we also impose a homogeneity constraint, requiring that there exists a one-to-one mapping F (θ) that transforms the ˜ physical space with units θ to a homogeneous space with units θ = F (θ) in which the stimulus distribution becomes uniform. This deﬁnes the mapping as θ F (θ) = p(χ)dχ , (1) −∞ which is the cumulative of the prior distribution p(θ). We then assume a neural population with identical tuning curves that evenly tiles the stimulus range in this homogeneous space. The population provides an efﬁcient representation of the sensory variable θ according to the above constraints [11]. ˜ The tuning curves in the physical space are obtained by applying the inverse mapping F −1 (θ). Fig. 2 1 In the context of this paper, we consider ‘inferring’, ‘decoding’, and ‘estimating’ as synonymous. 2 stimulus distribution d samples # a Fisher information discriminability and average firing rates and b firing rate [ Hz] efficient encoding F likelihood function F -1 likelihood c symmetric asymmetric homogeneous space physical space Figure 2: Efﬁcient encoding constrains the likelihood function. a) Prior distribution p(θ) derived from stimulus statistics. b) Efﬁcient coding deﬁnes the shape of the tuning curves in the physical space by transforming a set of homogeneous neurons using a mapping F −1 that is the inverse of the cumulative of the prior p(θ) (see Eq. (1)). c) As a result, the likelihood shape is constrained by the prior distribution showing heavier tails on the side of lower prior density. d) Fisher information, discrimination threshold, and average ﬁring rates are all uniform in the homogeneous space. illustrates the applied efﬁcient encoding scheme, the mapping, and the concept of the homogeneous space for the example of a symmetric, exponentially decaying prior distribution p(θ). The key idea here is that by assuming efﬁcient encoding, the prior (i.e. the stimulus distribution in the world) directly constrains the likelihood function. In particular, the shape of the likelihood is determined by the cumulative distribution of the prior. As a result, the likelihood is generally asymmetric, as shown in Fig. 2, exhibiting heavier tails on the side of the prior with lower density. 2.2 Bayesian decoding Let us consider a population of N sensory neurons that efﬁciently represents a stimulus variable θ as described above. A stimulus θ0 elicits a speciﬁc population response that is characterized by the vector R = [r1 , r2 , ..., rN ] where ri is the spike-count of the ith neuron over a given time-window τ . Under the assumption that the variability in the individual ﬁring rates is governed by a Poisson process, we can write the likelihood function over θ as N p(R|θ) = (τ fi (θ))ri −τ fi (θ) e , ri ! i=1 (2) ˆ with fi (θ) describing the tuning curve of neuron i. We then deﬁne a Bayesian decoder θLSE as the estimator that minimizes the expected squared-error between the estimate and the true stimulus value, thus θp(R|θ)p(θ)dθ ˆ θLSE (R) = , (3) p(R|θ)p(θ)dθ where we use Bayes’ rule to appropriately combine the sensory evidence with the stimulus prior p(θ). 3 Bayesian estimates can be biased away from prior peaks Bayesian models of perception typically predict perceptual biases toward the peaks of the prior density, a characteristic often considered a hallmark of Bayesian inference. This originates from the 3 a b prior attraction prior prior attraction likelihood repulsion! likelihood c prior prior repulsive bias likelihood likelihood mean posterior mean posterior mean Figure 3: Bayesian estimates biased away from the prior. a) If the likelihood function is symmetric, then the estimate (posterior mean) is, on average, shifted away from the actual value of the sensory variable θ0 towards the prior peak. b) Efﬁcient encoding typically leads to an asymmetric likelihood function whose normalized mean is away from the peak of the prior (relative to θ0 ). The estimate is determined by a combination of prior attraction and shifted likelihood mean, and can exhibit an overall repulsive bias. c) If p(θ0 ) < 0 and the likelihood is relatively narrow, then (1/p(θ)2 ) > 0 (blue line) and the estimate is biased away from the prior peak (see Eq. (6)). common approach of choosing a parametric description of the likelihood function that is computationally convenient (e.g. Gaussian). As a consequence, likelihood functions are typically assumed to be symmetric (but see [23, 24]), leaving the bias of the Bayesian estimator to be mainly determined by the shape of the prior density, i.e. leading to biases toward the peak of the prior (Fig. 3a). In our model framework, the shape of the likelihood function is constrained by the stimulus prior via efﬁcient neural encoding, and is generally not symmetric for non-ﬂat priors. It has a heavier tail on the side with lower prior density (Fig. 3b). The intuition is that due to the efﬁcient allocation of neural resources, the side with smaller prior density will be encoded less accurately, leading to a broader likelihood function on that side. The likelihood asymmetry pulls the Bayes’ least-squares estimate away from the peak of the prior while at the same time the prior pulls it toward its peak. Thus, the resulting estimation bias is the combination of these two counter-acting forces - and both are determined by the prior! 3.1 General derivation of the estimation bias In the following, we will formally derive the mean estimation bias b(θ) of the proposed encodingdecoding framework. Speciﬁcally, we will study the conditions for which the bias is repulsive i.e. away from the peak of the prior density. ˆ We ﬁrst re-write the estimator θLSE (3) by replacing θ with the inverse of its mapping to the homo−1 ˜ geneous space, i.e., θ = F (θ). The motivation for this is that the likelihood in the homogeneous space is symmetric (Fig. 2). Given a value θ0 and the elicited population response R, we can write the estimator as ˜ ˜ ˜ ˜ θp(R|θ)p(θ)dθ F −1 (θ)p(R|F −1 (θ))p(F −1 (θ))dF −1 (θ) ˆ θLSE (R) = = . ˜ ˜ ˜ p(R|θ)p(θ)dθ p(R|F −1 (θ))p(F −1 (θ))dF −1 (θ) Calculating the derivative of the inverse function and noting that F is the cumulative of the prior density, we get 1 1 1 ˜ ˜ ˜ ˜ ˜ ˜ dθ = dθ. dF −1 (θ) = (F −1 (θ)) dθ = dθ = −1 (θ)) ˜ F (θ) p(θ) p(F ˆ Hence, we can simplify θLSE (R) as ˆ θLSE (R) = ˜ ˜ ˜ F −1 (θ)p(R|F −1 (θ))dθ . ˜ ˜ p(R|F −1 (θ))dθ With ˜ K(R, θ) = ˜ p(R|F −1 (θ)) ˜ ˜ p(R|F −1 (θ))dθ 4 we can further simplify the notation and get ˆ θLSE (R) = ˜ ˜ ˜ F −1 (θ)K(R, θ)dθ . (4) ˆ ˜ In order to get the expected value of the estimate, θLSE (θ), we marginalize (4) over the population response space S, ˆ ˜ ˜ ˜ ˜ θLSE (θ) = p(R)F −1 (θ)K(R, θ)dθdR S = F −1 ˜ (θ)( ˜ ˜ p(R)K(R, θ)dR)dθ = ˜ ˜ ˜ F −1 (θ)L(θ)dθ, S where we deﬁne ˜ L(θ) = ˜ p(R)K(R, θ)dR. S ˜ ˜ ˜ It follows that L(θ)dθ = 1. Due to the symmetry in this space, it can be shown that L(θ) is ˜0 . Intuitively, L(θ) can be thought as the normalized ˜ symmetric around the true stimulus value θ average likelihood in the homogeneous space. We can then compute the expected bias at θ0 as b(θ0 ) = ˜ ˜ ˜ ˜ F −1 (θ)L(θ)dθ − F −1 (θ0 ) (5) ˜ This is expression is general where F −1 (θ) is deﬁned as the inverse of the cumulative of an arbitrary ˜ prior density p(θ) (see Eq. (1)) and the dispersion of L(θ) is determined by the internal noise level. ˜ ˜ Assuming the prior density to be smooth, we expand F −1 in a neighborhood (θ0 − h, θ0 + h) that is larger than the support of the likelihood function. Using Taylor’s theorem with mean-value forms of the remainder, we get 1 ˜ ˜ ˜ ˜ ˜ ˜ ˜ ˜ F −1 (θ) = F −1 (θ0 ) + F −1 (θ0 ) (θ − θ0 ) + F −1 (θx ) (θ − θ0 )2 , 2 ˜ ˜ ˜ with θx lying between θ0 and θ. By applying this expression to (5), we ﬁnd ˜ θ0 +h b(θ0 ) = = 1 2 ˜ θ0 −h 1 −1 ˜ ˜ ˜ ˜ ˜ 1 F (θx )θ (θ − θ0 )2 L(θ)dθ = ˜ 2 2 ˜ θ0 +h −( ˜ θ0 −h p(θx )θ ˜ ˜ 2 ˜ ˜ 1 )(θ − θ0 ) L(θ)dθ = p(θx )3 4 ˜ θ0 +h 1 ˜ − θ0 )2 L(θ)dθ ˜ ˜ ˜ ( ) ˜(θ ˜ p(F −1 (θx )) θ ( 1 ˜ ˜ ˜ ˜ ) (θ − θ0 )2 L(θ)dθ. p(θx )2 θ ˜ θ0 −h ˜ θ0 +h ˜ θ0 −h In general, there is no simple rule to judge the sign of b(θ0 ). However, if the prior is monotonic ˜ ˜ on the interval F −1 ((θ0 − h, θ0 + h)), then the sign of ( p(θ1 )2 ) is always the same as the sign of x 1 1 ( p(θ0 )2 ) . Also, if the likelihood is sufﬁciently narrow we can approximate ( p(θ1 )2 ) by ( p(θ0 )2 ) , x and therefore approximate the bias as b(θ0 ) ≈ C( 1 ) , p(θ0 )2 (6) where C is a positive constant. The result is quite surprising because it states that as long as the prior is monotonic over the support of the likelihood function, the expected estimation bias is always away from the peaks of the prior! 3.2 Internal (neural) versus external (stimulus) noise The above derivation of estimation bias is based on the assumption that all uncertainty about the sensory variable is caused by neural response variability. This level of internal noise depends on the response magnitude, and thus can be modulated e.g. by changing stimulus contrast. This contrastcontrolled noise modulation is commonly exploited in perceptual studies (e.g. [18]). Internal noise will always lead to repulsive biases in our framework if the prior is monotonic. If internal noise is low, the likelihood is narrow and thus the bias is small. Increasing internal noise leads to increasingly 5 larger biases up to the point where the likelihood becomes wide enough such that monotonicity of the prior over the support of the likelihood is potentially violated. Stimulus noise is another way to modulate the noise level in perception (e.g. random-dot motion stimuli). Such external noise, however, has a different effect on the shape of the likelihood function as compared to internal noise. It modiﬁes the likelihood function (2) by convolving it with the noise kernel. External noise is frequently chosen as additive and symmetric (e.g. zero-mean Gaussian). It is straightforward to prove that such symmetric external noise does not lead to a change in the mean of the likelihood, and thus does not alter the repulsive effect induced by its asymmetry. However, by increasing the overall width of the likelihood, the attractive inﬂuence of the prior increases, resulting in an estimate that is closer to the prior peak than without external noise2 . 4 Perception of visual orientation We tested our framework by modelling the perception of visual orientation. Our choice was based on the fact that i) we have pretty good estimates of the prior distribution of local orientations in natural images, ii) tuning characteristics of orientation selective neurons in visual cortex are wellstudied (monkey/cat), and iii) biases in perceived stimulus orientation have been well characterized. We start by creating an efﬁcient neural population based on measured prior distributions of local visual orientation, and then compare the resulting tuning characteristics of the population and the predicted perceptual biases with reported data in the literature. 4.1 Efﬁcient neural model population for visual orientation Previous studies measured the statistics of the local orientation in large sets of natural images and consistently found that the orientation distribution is multimodal, peaking at the two cardinal orientations as shown in Fig. 4a [16, 20]. We assumed that the visual system’s prior belief over orientation p(θ) follows this distribution and approximate it formally as p(θ) ∝ 2 − | sin(θ)| (black line in Fig. 4b) . (7) Based on this prior distribution we deﬁned an efﬁcient neural representation for orientation. We assumed a population of model neurons (N = 30) with tuning curves that follow a von-Mises distribution in the homogeneous space on top of a constant spontaneous ﬁring rate (5 Hz). We then ˜ applied the inverse transformation F −1 (θ) to all these tuning curves to get the corresponding tuning curves in the physical space (Fig. 4b - red curves), where F (θ) is the cumulative of the prior (7). The concentration parameter for the von-Mises tuning curves was set to κ ≈ 1.6 in the homogeneous space in order to match the measured average tuning width (∼ 32 deg) of neurons in area V1 of the macaque [9]. 4.2 Predicted tuning characteristics of neurons in primary visual cortex The orientation tuning characteristics of our model population well match neurophysiological data of neurons in primary visual cortex (V1). Efﬁcient encoding predicts that the distribution of neurons’ preferred orientation follows the prior, with more neurons tuned to cardinal than oblique orientations by a factor of approximately 1.5. A similar ratio has been found for neurons in area V1 of monkey/cat [9, 10]. Also, the tuning widths of the model neurons vary between 25-42 deg depending on their preferred tuning (see Fig. 4c), matching the measured tuning width ratio of 0.6 between neurons tuned to the cardinal versus oblique orientations [9]. An important prediction of our model is that most of the tuning curves should be asymmetric. Such asymmetries have indeed been reported for the orientation tuning of neurons in area V1 [6, 7, 8]. We computed the asymmetry index for our model population as deﬁned in previous studies [6, 7], and plotted it as a function of the preferred tuning of each neuron (Fig. 4d). The overall asymmetry index in our model population is 1.24 ± 0.11, which approximately matches the measured values for neurons in area V1 of the cat (1.26 ± 0.06) [6]. It also predicts that neurons tuned to the cardinal and oblique orientations should show less symmetry than those tuned to orientations in between. Finally, 2 Note, that these predictions are likely to change if the external noise is not symmetric. 6 a b 25 firing rate(Hz) 0 orientation(deg) asymmetry vs. tuning width 1.0 2.0 90 2.0 e asymmetry 1.0 0 asymmetry index 50 30 width (deg) 10 90 preferred tuning(deg) -90 0 d 0 0 90 asymmetry index 0 orientation(deg) tuning width -90 0 0 probability 0 -90 c efficient representation 0.01 0.01 image statistics -90 0 90 preferred tuning(deg) 25 30 35 40 tuning width (deg) Figure 4: Tuning characteristics of model neurons. a) Distribution of local orientations in natural images, replotted from [16]. b) Prior used in the model (black) and predicted tuning curves according to efﬁcient coding (red). c) Tuning width as a function of preferred orientation. d) Tuning curves of cardinal and oblique neurons are more symmetric than those tuned to orientations in between. e) Both narrowly and broadly tuned neurons neurons show less asymmetry than neurons with tuning widths in between. neurons with tuning widths at the lower and upper end of the range are predicted to exhibit less asymmetry than those neurons whose widths lie in between these extremes (illustrated in Fig. 4e). These last two predictions have not been tested yet. 4.3 Predicted perceptual biases Our model framework also provides speciﬁc predictions for the expected perceptual biases. Humans show systematic biases in perceived orientation of visual stimuli such as e.g. arrays of Gabor patches (Fig. 5a,d). Two types of biases can be distinguished: First, perceived orientations show an absolute bias away from the cardinal orientations, thus away from the peaks of the orientation prior [2, 3]. We refer to these biases as absolute because they are typically measured by adjusting a noise-free reference until it matched the orientation of the test stimulus. Interestingly, these repulsive absolute biases are the larger the smaller the external stimulus noise is (see Fig. 5b). Second, the relative bias between the perceived overall orientations of a high-noise and a low-noise stimulus is toward the cardinal orientations as shown in Fig. 5c, and thus toward the peak of the prior distribution [3, 16]. The predicted perceptual biases of our model are shown Fig. 5e,f. We computed the likelihood function according to (2) and used the prior in (7). External noise was modeled by convolving the stimulus likelihood function with a Gaussian (different widths for different noise levels). The predictions well match both, the reported absolute bias away as well as the relative biases toward the cardinal orientations. Note, that our model framework correctly accounts for the fact that less external noise leads to larger absolute biases (see also discussion in section 3.2). 5 Discussion We have presented a modeling framework for perception that combines efﬁcient (en)coding and Bayesian decoding. Efﬁcient coding imposes constraints on the tuning characteristics of a population of neurons according to the stimulus distribution (prior). It thus establishes a direct link between prior and likelihood, and provides clear constraints on the latter for a Bayesian observer model of perception. We have shown that the resulting likelihoods are in general asymmetric, with 7 absolute bias (data) b c relative bias (data) -4 0 bias(deg) 4 a low-noise stimulus -90 e 90 absolute bias (model) low external noise high external noise 3 high-noise stimulus -90 f 0 90 relative bias (model) 0 bias(deg) d 0 attraction -3 repulsion -90 0 orientation (deg) 90 -90 0 orientation (deg) 90 Figure 5: Biases in perceived orientation: Human data vs. Model prediction. a,d) Low- and highnoise orientation stimuli of the type used in [3, 16]. b) Humans show absolute biases in perceived orientation that are away from the cardinal orientations. Data replotted from [2] (pink squares) and [3] (green (black) triangles: bias for low (high) external noise). c) Relative bias between stimuli with different external noise level (high minus low). Data replotted from [3] (blue triangles) and [16] (red circles). e,f) Model predictions for absolute and relative bias. heavier tails away from the prior peaks. We demonstrated that such asymmetric likelihoods can lead to the counter-intuitive prediction that a Bayesian estimator is biased away from the peaks of the prior distribution. Interestingly, such repulsive biases have been reported for human perception of visual orientation, yet a principled and consistent explanation of their existence has been missing so far. Here, we suggest that these counter-intuitive biases directly follow from the asymmetries in the likelihood function induced by efﬁcient neural encoding of the stimulus. The good match between our model predictions and the measured perceptual biases and orientation tuning characteristics of neurons in primary visual cortex provides further support of our framework. Previous work has suggested that there might be a link between stimulus statistics, neuronal tuning characteristics, and perceptual behavior based on efﬁcient coding principles, yet none of these studies has recognized the importance of the resulting likelihood asymmetries [16, 11]. We have demonstrated here that such asymmetries can be crucial in explaining perceptual data, even though the resulting estimates appear “anti-Bayesian” at ﬁrst sight (see also models of sensory adaptation [23]). Note, that we do not provide a neural implementation of the Bayesian inference step. However, we and others have proposed various neural decoding schemes that can approximate Bayes’ leastsquares estimation using efﬁcient coding [26, 25, 22]. It is also worth pointing out that our estimator is set to minimize total squared-error, and that other choices of the loss function (e.g. MAP estimator) could lead to different predictions. Our framework is general and should be directly applicable to other modalities. In particular, it might provide a new explanation for perceptual biases that are hard to reconcile with traditional Bayesian approaches [5]. Acknowledgments We thank M. Jogan and A. Tank for helpful comments on the manuscript. This work was partially supported by grant ONR N000141110744. 8 References [1] M. Jones, and B. C. Love. Bayesian fundamentalism or enlightenment? On the explanatory status and theoretical contributions of Bayesian models of cognition. Behavioral and Brain Sciences, 34, 169–231,2011. [2] D. P. Andrews. Perception of contours in the central fovea. Nature, 205:1218- 1220, 1965. [3] A. Tomassini, M. J.Morgam. and J. A. Solomon. Orientation uncertainty reduces perceived obliquity. Vision Res, 50, 541–547, 2010. [4] W. S. Geisler, D. Kersten. Illusions, perception and Bayes. Nature Neuroscience, 5(6):508- 510, 2002. [5] M. O. Ernst Perceptual learning: inverting the size-weight illusion. Current Biology, 19:R23- R25, 2009. [6] G. H. Henry, B. Dreher, P. O. Bishop. Orientation speciﬁcity of cells in cat striate cortex. J Neurophysiol, 37(6):1394-409,1974. [7] D. Rose, C. Blakemore An analysis of orientation selectivity in the cat’s visual cortex. Exp Brain Res., Apr 30;20(1):1-17, 1974. [8] N. V. Swindale. Orientation tuning curves: empirical description and estimation of parameters. Biol Cybern., 78(1):45-56, 1998. [9] R. L. De Valois, E. W. Yund, N. Hepler. The orientation and direction selectivity of cells in macaque visual cortex. Vision Res.,22, 531544,1982. [10] B. Li, M. R. Peterson, R. D. Freeman. The oblique effect: a neural basis in the visual cortex. J. Neurophysiol., 90, 204217, 2003. [11] D. Ganguli and E.P. Simoncelli. Implicit encoding of prior probabilities in optimal neural populations. In Adv. Neural Information Processing Systems NIPS 23, vol. 23:658–666, 2011. [12] M. D. McDonnell, N. G. Stocks. Maximally Informative Stimuli and Tuning Curves for Sigmoidal RateCoding Neurons and Populations. Phys Rev Lett., 101(5):058103, 2008. [13] H Helmholtz. Treatise on Physiological Optics (transl.). Thoemmes Press, Bristol, U.K., 2000. Original publication 1867. [14] Y. Weiss, E. Simoncelli, and E. Adelson. Motion illusions as optimal percept. Nature Neuroscience, 5(6):598–604, June 2002. [15] D.C. Knill and W. Richards, editors. Perception as Bayesian Inference. Cambridge University Press, 1996. [16] A R Girshick, M S Landy, and E P Simoncelli. Cardinal rules: visual orientation perception reﬂects knowledge of environmental statistics. Nat Neurosci, 14(7):926–932, Jul 2011. [17] M. Jazayeri and M.N. Shadlen. Temporal context calibrates interval timing. Nature Neuroscience, 13(8):914–916, 2010. [18] A.A. Stocker and E.P. Simoncelli. Noise characteristics and prior expectations in human visual speed perception. Nature Neuroscience, pages 578–585, April 2006. [19] H.B. Barlow. Possible principles underlying the transformation of sensory messages. In W.A. Rosenblith, editor, Sensory Communication, pages 217–234. MIT Press, Cambridge, MA, 1961. [20] D.M. Coppola, H.R. Purves, A.N. McCoy, and D. Purves The distribution of oriented contours in the real world. Proc Natl Acad Sci U S A., 95(7): 4002–4006, 1998. [21] N. Brunel and J.-P. Nadal. Mutual information, Fisher information and population coding. Neural Computation, 10, 7, 1731–1757, 1998. [22] X-X. Wei and A.A. Stocker. Bayesian inference with efﬁcient neural population codes. In Lecture Notes in Computer Science, Artiﬁcial Neural Networks and Machine Learning - ICANN 2012, Lausanne, Switzerland, volume 7552, pages 523–530, 2012. [23] A.A. Stocker and E.P. Simoncelli. Sensory adaptation within a Bayesian framework for perception. In Y. Weiss, B. Sch¨ lkopf, and J. Platt, editors, Advances in Neural Information Processing Systems 18, pages o 1291–1298. MIT Press, Cambridge, MA, 2006. Oral presentation. [24] D.C. Knill. Robust cue integration: A Bayesian model and evidence from cue-conﬂict studies with stereoscopic and ﬁgure cues to slant. Journal of Vision, 7(7):1–24, 2007. [25] Deep Ganguli. Efﬁcient coding and Bayesian inference with neural populations. PhD thesis, Center for Neural Science, New York University, New York, NY, September 2012. [26] B. Fischer. Bayesian estimates from heterogeneous population codes. In Proc. IEEE Intl. Joint Conf. on Neural Networks. IEEE, 2010. 9

5 0.13935928 195 nips-2012-Learning visual motion in recurrent neural networks

Author: Marius Pachitariu, Maneesh Sahani

Abstract: We present a dynamic nonlinear generative model for visual motion based on a latent representation of binary-gated Gaussian variables. Trained on sequences of images, the model learns to represent different movement directions in different variables. We use an online approximate inference scheme that can be mapped to the dynamics of networks of neurons. Probed with drifting grating stimuli and moving bars of light, neurons in the model show patterns of responses analogous to those of direction-selective simple cells in primary visual cortex. Most model neurons also show speed tuning and respond equally well to a range of motion directions and speeds aligned to the constraint line of their respective preferred speed. We show how these computations are enabled by a speciﬁc pattern of recurrent connections learned by the model. 1

6 0.12784673 56 nips-2012-Bayesian active learning with localized priors for fast receptive field characterization

7 0.11518808 262 nips-2012-Optimal Neural Tuning Curves for Arbitrary Stimulus Distributions: Discrimax, Infomax and Minimum $L p$ Loss

8 0.10210401 238 nips-2012-Neurally Plausible Reinforcement Learning of Working Memory Tasks

9 0.097641215 77 nips-2012-Complex Inference in Neural Circuits with Probabilistic Population Codes and Topic Models

10 0.089029796 190 nips-2012-Learning optimal spike-based representations

11 0.084064953 113 nips-2012-Efficient and direct estimation of a neural subunit model for sensory coding

12 0.081237964 48 nips-2012-Augmented-SVM: Automatic space partitioning for combining multiple non-linear dynamics

13 0.072757505 152 nips-2012-Homeostatic plasticity in Bayesian spiking networks as Expectation Maximization with posterior constraints

14 0.070865549 333 nips-2012-Synchronization can Control Regularization in Neural Systems via Correlated Noise Processes

15 0.065942377 112 nips-2012-Efficient Spike-Coding with Multiplicative Adaptation in a Spike Response Model

16 0.065465935 23 nips-2012-A lattice filter model of the visual pathway

17 0.06280461 153 nips-2012-How Prior Probability Influences Decision Making: A Unifying Probabilistic Model

18 0.060441464 140 nips-2012-Fusion with Diffusion for Robust Visual Tracking

19 0.057433311 347 nips-2012-Towards a learning-theoretic analysis of spike-timing dependent plasticity

20 0.057312842 157 nips-2012-Identification of Recurrent Patterns in the Activation of Brain Networks

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, 0.129), (1, 0.004), (2, -0.101), (3, 0.131), (4, -0.019), (5, 0.195), (6, 0.01), (7, 0.027), (8, -0.029), (9, 0.055), (10, -0.035), (11, -0.053), (12, 0.008), (13, -0.034), (14, 0.025), (15, 0.044), (16, -0.003), (17, 0.005), (18, -0.026), (19, 0.064), (20, 0.009), (21, 0.05), (22, 0.007), (23, 0.052), (24, 0.009), (25, -0.033), (26, -0.018), (27, 0.121), (28, 0.174), (29, -0.032), (30, -0.074), (31, 0.119), (32, -0.009), (33, 0.0), (34, 0.167), (35, -0.041), (36, 0.032), (37, -0.144), (38, 0.057), (39, 0.097), (40, 0.033), (41, 0.024), (42, 0.037), (43, 0.053), (44, -0.033), (45, -0.029), (46, -0.055), (47, 0.029), (48, -0.028), (49, -0.015)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.96824127 94 nips-2012-Delay Compensation with Dynamical Synapses

Author: Chi Fung, K. Wong, Si Wu

2 0.80817127 24 nips-2012-A mechanistic model of early sensory processing based on subtracting sparse representations

Author: Shaul Druckmann, Tao Hu, Dmitri B. Chklovskii

3 0.80292213 114 nips-2012-Efficient coding provides a direct link between prior and likelihood in perceptual Bayesian inference

Author: Xue-xin Wei, Alan Stocker

4 0.77619749 256 nips-2012-On the connections between saliency and tracking

Author: Vijay Mahadevan, Nuno Vasconcelos

5 0.65787584 262 nips-2012-Optimal Neural Tuning Curves for Arbitrary Stimulus Distributions: Discrimax, Infomax and Minimum $L p$ Loss

Author: Zhuo Wang, Alan Stocker, Daniel Lee

Abstract: In this work we study how the stimulus distribution inﬂuences the optimal coding of an individual neuron. Closed-form solutions to the optimal sigmoidal tuning curve are provided for a neuron obeying Poisson statistics under a given stimulus distribution. We consider a variety of optimality criteria, including maximizing discriminability, maximizing mutual information and minimizing estimation error under a general Lp norm. We generalize the Cramer-Rao lower bound and show how the Lp loss can be written as a functional of the Fisher Information in the asymptotic limit, by proving the moment convergence of certain functions of Poisson random variables. In this manner, we show how the optimal tuning curve depends upon the loss function, and the equivalence of maximizing mutual information with minimizing Lp loss in the limit as p goes to zero. 1

6 0.63556004 195 nips-2012-Learning visual motion in recurrent neural networks

7 0.57171702 113 nips-2012-Efficient and direct estimation of a neural subunit model for sensory coding

8 0.56019503 224 nips-2012-Multi-scale Hyper-time Hardware Emulation of Human Motor Nervous System Based on Spiking Neurons using FPGA

9 0.54040426 2 nips-2012-3D Social Saliency from Head-mounted Cameras

10 0.51000178 333 nips-2012-Synchronization can Control Regularization in Neural Systems via Correlated Noise Processes

11 0.50841874 56 nips-2012-Bayesian active learning with localized priors for fast receptive field characterization

12 0.50827104 238 nips-2012-Neurally Plausible Reinforcement Learning of Working Memory Tasks

13 0.45972422 23 nips-2012-A lattice filter model of the visual pathway

14 0.44171709 77 nips-2012-Complex Inference in Neural Circuits with Probabilistic Population Codes and Topic Models

15 0.41293794 46 nips-2012-Assessing Blinding in Clinical Trials

16 0.40672266 153 nips-2012-How Prior Probability Influences Decision Making: A Unifying Probabilistic Model

17 0.37095934 157 nips-2012-Identification of Recurrent Patterns in the Activation of Brain Networks

18 0.36314723 190 nips-2012-Learning optimal spike-based representations

19 0.35064709 140 nips-2012-Fusion with Diffusion for Robust Visual Tracking

20 0.34418106 152 nips-2012-Homeostatic plasticity in Bayesian spiking networks as Expectation Maximization with posterior constraints

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(0, 0.028), (11, 0.015), (17, 0.023), (21, 0.075), (22, 0.011), (33, 0.312), (38, 0.131), (39, 0.015), (42, 0.023), (54, 0.019), (55, 0.027), (74, 0.045), (76, 0.078), (80, 0.064), (92, 0.05)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.77563453 94 nips-2012-Delay Compensation with Dynamical Synapses

Author: Chi Fung, K. Wong, Si Wu

2 0.51346761 113 nips-2012-Efficient and direct estimation of a neural subunit model for sensory coding

Author: Brett Vintch, Andrew Zaharia, J Movshon, Hhmi) Hhmi), Eero P. Simoncelli

Abstract: Many visual and auditory neurons have response properties that are well explained by pooling the rectiﬁed responses of a set of spatially shifted linear ﬁlters. These ﬁlters cannot be estimated using spike-triggered averaging (STA). Subspace methods such as spike-triggered covariance (STC) can recover multiple ﬁlters, but require substantial amounts of data, and recover an orthogonal basis for the subspace in which the ﬁlters reside rather than the ﬁlters themselves. Here, we assume a linear-nonlinear–linear-nonlinear (LN-LN) cascade model in which the ﬁrst linear stage is a set of shifted (‘convolutional’) copies of a common ﬁlter, and the ﬁrst nonlinear stage consists of rectifying scalar nonlinearities that are identical for all ﬁlter outputs. We refer to these initial LN elements as the ‘subunits’ of the receptive ﬁeld. The second linear stage then computes a weighted sum of the responses of the rectiﬁed subunits. We present a method for directly ﬁtting this model to spike data, and apply it to both simulated and real neuronal data from primate V1. The subunit model signiﬁcantly outperforms STA and STC in terms of cross-validated accuracy and efﬁciency. 1

3 0.51161653 302 nips-2012-Scaling MPE Inference for Constrained Continuous Markov Random Fields with Consensus Optimization

Author: Stephen Bach, Matthias Broecheler, Lise Getoor, Dianne O'leary

Abstract: Probabilistic graphical models are powerful tools for analyzing constrained, continuous domains. However, ﬁnding most-probable explanations (MPEs) in these models can be computationally expensive. In this paper, we improve the scalability of MPE inference in a class of graphical models with piecewise-linear and piecewise-quadratic dependencies and linear constraints over continuous domains. We derive algorithms based on a consensus-optimization framework and demonstrate their superior performance over state of the art. We show empirically that in a large-scale voter-preference modeling problem our algorithms scale linearly in the number of dependencies and constraints. 1

4 0.51034087 300 nips-2012-Scalable nonconvex inexact proximal splitting

Author: Suvrit Sra

Abstract: We study a class of large-scale, nonsmooth, and nonconvex optimization problems. In particular, we focus on nonconvex problems with composite objectives. This class includes the extensively studied class of convex composite objective problems as a subclass. To solve composite nonconvex problems we introduce a powerful new framework based on asymptotically nonvanishing errors, avoiding the common stronger assumption of vanishing errors. Within our new framework we derive both batch and incremental proximal splitting algorithms. To our knowledge, our work is ﬁrst to develop and analyze incremental nonconvex proximalsplitting algorithms, even if we were to disregard the ability to handle nonvanishing errors. We illustrate one instance of our general framework by showing an application to large-scale nonsmooth matrix factorization. 1

5 0.50784856 114 nips-2012-Efficient coding provides a direct link between prior and likelihood in perceptual Bayesian inference

Author: Xue-xin Wei, Alan Stocker

6 0.50713968 195 nips-2012-Learning visual motion in recurrent neural networks

7 0.50450182 105 nips-2012-Dynamic Pruning of Factor Graphs for Maximum Marginal Prediction

8 0.50450021 333 nips-2012-Synchronization can Control Regularization in Neural Systems via Correlated Noise Processes

9 0.50147098 18 nips-2012-A Simple and Practical Algorithm for Differentially Private Data Release

10 0.50044799 83 nips-2012-Controlled Recognition Bounds for Visual Learning and Exploration

11 0.4989402 190 nips-2012-Learning optimal spike-based representations

12 0.49878389 120 nips-2012-Exact and Stable Recovery of Sequences of Signals with Sparse Increments via Differential 1-Minimization

13 0.4962543 112 nips-2012-Efficient Spike-Coding with Multiplicative Adaptation in a Spike Response Model

14 0.49582711 23 nips-2012-A lattice filter model of the visual pathway

15 0.49477169 216 nips-2012-Mirror Descent Meets Fixed Share (and feels no regret)

16 0.49343273 178 nips-2012-Learning Label Trees for Probabilistic Modelling of Implicit Feedback

17 0.49293271 241 nips-2012-No-Regret Algorithms for Unconstrained Online Convex Optimization

18 0.49283341 238 nips-2012-Neurally Plausible Reinforcement Learning of Working Memory Tasks

19 0.49177873 24 nips-2012-A mechanistic model of early sensory processing based on subtracting sparse representations

20 0.49145174 65 nips-2012-Cardinality Restricted Boltzmann Machines