nips nips2002 nips2002-171 knowledge-graph by maker-knowledge-mining

171 nips-2002-Reconstructing Stimulus-Driven Neural Networks from Spike Times

Source: pdf

Author: Duane Q. Nykamp

Abstract: We present a method to distinguish direct connections between two neurons from common input originating from other, unmeasured neurons. The distinction is computed from the spike times of the two neurons in response to a white noise stimulus. Although the method is based on a highly idealized linear-nonlinear approximation of neural response, we demonstrate via simulation that the approach can work with a more realistic, integrate-and-ﬁre neuron model. We propose that the approach exempliﬁed by this analysis may yield viable tools for reconstructing stimulus-driven neural networks from data gathered in neurophysiology experiments.

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 edu Abstract We present a method to distinguish direct connections between two neurons from common input originating from other, unmeasured neurons. [sent-4, score-0.992]

2 The distinction is computed from the spike times of the two neurons in response to a white noise stimulus. [sent-5, score-0.951]

3 Although the method is based on a highly idealized linear-nonlinear approximation of neural response, we demonstrate via simulation that the approach can work with a more realistic, integrate-and-ﬁre neuron model. [sent-6, score-0.509]

4 1 Introduction The pattern of connectivity between neurons in the brain is fundamental to understanding the function the brain’s neural networks. [sent-8, score-0.464]

5 Unfortunately, the complexity of higher organisms makes obtaining combined functional and connectivity data extraordinarily difﬁcult. [sent-10, score-0.282]

6 The most common tool for recording in vivo the activity of neurons in higher organisms is the extracellular electrode. [sent-11, score-0.489]

7 In such an experiment, the states of the measured neurons remain hidden. [sent-13, score-0.387]

8 The ability to infer connectivity patterns from spike times alone would greatly expand the attainable connectivity data and provide the opportunity to better address the link between function and connectivity. [sent-14, score-0.697]

9 Attempts to infer connectivity from spike time data have focused on second-order statistics of the spike times of two simultaneously recorded neurons. [sent-15, score-0.821]

10 However, the JPSTH and correlogram cannot distinguish correlations induced by connections between the two measured neurons (direct connection correlations) from correlations induced by common connections from a third, unmeasured neuron (common input correlations). [sent-17, score-2.286]

11 Inferences from the JPSTH or correlogram about the connections between the two measured neurons are ambiguous. [sent-18, score-0.536]

12 Analysis tools such as partial coherence [4] can distinguish between a direct connection and common input when one can also measure neurons inducing the common input effects. [sent-19, score-1.323]

13 The distinction of present approach is that all other neurons are unmeasured. [sent-20, score-0.395]

14 We demonstrate that, by characterizing how each neuron responds to the stimulus, one may be able to distinguish between direct connection and common input correlations. [sent-21, score-1.163]

15 In that case, one could determine if a connection existed between two neurons simply by measuring their spike times in response to a stimulus. [sent-22, score-1.034]

16 Since the properties of the neurons would be determined by the same measurements, such an analysis would be the basis for inferring links between connectivity and function. [sent-23, score-0.464]

17 2 The model To make the subtle distinction between direct connection correlations and common input correlations, one needs to exploit an explicit model. [sent-24, score-0.962]

18 Let Rp = 1 if neuron p spiked at the discrete time point i and be zero otherwise. [sent-30, score-0.443]

19 The linear-nonlinear model of a single neuron can be completely reconstructed from measured spike times in response to white noise [5]. [sent-34, score-1.051]

20 We will demonstrate that the network of linear-nonlinear neurons can be similarly analyzed to determine the connectivity between two measured neurons. [sent-35, score-0.65]

21 3 Analysis of model Let neurons 1 and 2 be the only two measured neurons. [sent-36, score-0.387]

22 The spike times of all other neurons will remain unmeasured. [sent-37, score-0.661]

23 Given further simplifying assumptions detailed below, we can ¯j ¯j isolate the connectivity terms between neurons 1 and 2 (W12 and W21 ). [sent-38, score-0.464]

24 We will outline a method to determined these connectivity terms from a few statistics of the two measured spikes trains and the white noise stimulus. [sent-39, score-0.5]

25 Common input correlations are second order in the Wqp because common input requires two connections. [sent-43, score-0.532]

26 Since our analysis must include common input to ¯j ¯k the measured neurons, we retain terms containing Wp1 Wq2 with p, q > 2. [sent-44, score-0.334]

27 The second assumption is that the unmeasured neurons do not respond to essentially identical stimulus features as the measured neurons (1 & 2) or each other. [sent-45, score-0.951]

28 We quantify similarity ¯k to stimulus features by the inner product between linear kernels, cos θpq = hi−k · hi . [sent-46, score-0.372]

29 We p q ¯ ¯ ¯ require each cos θ to be small so that we can ignore terms of the form W cos θ. [sent-47, score-0.222]

30 We al¯k ¯ low one exception and retain W cos θ21 terms so that no assumption is made on the two measured neurons. [sent-48, score-0.221]

31 2 Outline of method The ﬁrst step in analyzing the network response is to ignore the fact that the neurons are embedded in a neural network and analyze the spike trains of neurons 1 and 2 as though each were an isolated linear-nonlinear system. [sent-51, score-1.122]

32 [5], one can determine the effective linear-nonlinear parameters from the average ﬁring rates i i i i (E{R1 } and E{R2 })1 and the stimulus-spike correlations (E{XR1 } and E{XR2 }). [sent-53, score-0.318]

33 These effective linear-nonlinear parameters clearly will not match the parameters for neurons 1 and 2 in the complete system (Eq. [sent-54, score-0.337]

34 The network connections alter the mean rates and stimulus-spike correlations of neurons 1 and 2, changing the linear-nonlinear parameters reconstructed from these measurements. [sent-56, score-0.727]

35 The connectivity between neurons 1 and 2 can then be determined from the correlation i i−k between their spikes (E{R1 R2 } measured at different positive and negative delays k and i i−k the correlation of their spike pairs with the stimulus (E{XR1 R2 }) as follows. [sent-59, score-1.368]

36 For each delay k, we i i−k i i−k obtain three equations: one from E{R1 R2 }, one from the projection of E{XR1 R2 } i−k i i i−k onto E{XR1 }, and one from the projection of E{XR1 R2 } onto E{XR2 }. [sent-61, score-0.402]

37 ¯ The factor W k is the direct connection between neurons 1 and 2 (the direction of the con¯ nection depends on the sign of the delay k). [sent-67, score-1.044]

38 The factor U k is the common input to neuron 2 and neuron 1 (k times steps delayed) from all other neurons in the network. [sent-68, score-1.378]

39 i i i To analyze spike train data, we approximate the statistics E{R1 }, E{R2 }, E{XR1 }, i i i−k i i−k E{XR2 }, E{R1 R2 }, and E{XR1 R2 } by averages over an experiment. [sent-72, score-0.314]

40 4 Demonstration We demonstrate the ability of the measures W and U to distinguish direct connection correlations from common input correlations with three example simulations. [sent-75, score-1.26]

41 In the ﬁrst two examples, we simulated a network of three coupled linear-nonlinear neurons (Eqs. [sent-76, score-0.399]

42 In the third example, we simulated a pair of integrate-and-ﬁre neurons driven by the stimulus in a manner similar to the linear-nonlinear neurons. [sent-78, score-0.509]

43 In each example, we measured only the spike times of neuron 1 and neuron 2. [sent-79, score-1.208]

44 Since the white noise stimulus does not repeat, one cannot calculate a JPSTH or shufﬂecorrected correlogram. [sent-80, score-0.274]

45 Instead, for comparison we calculated the covariance between the i−k i i−k i spike times, C k = R1 R2 − R1 R2 , and a stimulus independent correlation meai i−k k sure introduced in Ref. [sent-81, score-0.515]

46 The quantity ν21 is the expected value of R1 R2 if neurons 1 and 2 were independent linear-nonlinear systems responding to the same stimulus. [sent-83, score-0.31]

47 We used spatio-temporal linear kernels of the form hp (j, t) = te − τt h e− |j|2 40 sin((j1 cos φp + j2 sin φp )fp + kp ) (5) for t > 0 (hp = 0 otherwise), where j = (j1 , j2 ) denotes a discrete space point. [sent-84, score-0.23]

48 For the linear-nonlinear simulations, we sampled this function on a 20 × 20 × 20 grid in space and time, normalizing the resulting vector to obtain the unit vector hi . [sent-85, score-0.22]

49 The only geometry of ¯k the kernels that appears in the equations is their inner products cos θpq = hi−k · hi . [sent-87, score-0.316]

50 Neuron 2 had an excitatory connection onto neuron 1 with a delay of 5–6 units of time (a positive ¯5 ¯6 delay for our sign convention): W21 = W21 = 0. [sent-89, score-1.534]

51 Neuron 3 had one excitatory connection onto neuron 1 and second excitatory connection onto neuron 2 that was delayed by ¯1 ¯2 ¯8 ¯9 6–8 units of time (a negative delay): W31 = W31 = W32 = W32 = 1. [sent-91, score-1.783]

52 In this way, the spike times from neuron 1 and 2 had positive correlations due to both a direct connection and common input. [sent-93, score-1.572]

53 1 shows the results after simulating for 600,000 units of time, obtaining 16,000–22,000 spikes per neuron. [sent-95, score-0.199]

54 The covariance C has peaks at both positive and negative delays, corresponding to the direct connection and common input, respectively, as well as a small peak around zero due to the shared stimulus (see Ref. [sent-96, score-0.825]

55 The measure S eliminates the stimulus-induced correlation, but still cannot distinguish the direct connection from the common input. [sent-98, score-0.71]

56 W contains a peak only at the positive delay corresponding to the direct connection from neuron 2 to neuron 1; U contains a peak only at the negative delay corresponding to the common input from the (unmeasured) third neuron. [sent-100, score-2.245]

57 On the order of 20,000 spikes were needed to get clean results even in this idealized simulation, a long experiment given the typically low ﬁring rates in response to white noise stimuli. [sent-102, score-0.356]

58 Theoretically, the method should handle inhibitory connections just as well as excitatory −3 a C 4 x 10 2 0 −30 −20 −10 0 Delay 10 20 30 −20 −10 0 Delay 10 20 30 −20 −10 0 Delay 10 20 30 −20 −10 0 Delay 10 20 30 −3 b 3 x 10 S 2 1 0 −30 W c 1 0. [sent-103, score-0.248]

59 5 0 −30 Figure 1: Results from the spike times of two neurons in a simulation of three linearnonlinear neurons. [sent-105, score-0.719]

60 Delay is in units of time and is the spike time of neuron 1 minus the spike time of neuron 2. [sent-106, score-1.487]

61 The correlations at a positive delay are due to a direct connection, while those a negative delay are due to common input. [sent-107, score-1.2]

62 (a) The covariance C between the spike times of neuron 1 and neuron 2 reﬂects both connections. [sent-108, score-1.131]

63 The third peak around zero delay, due to similarity in the kernels hi and hi , is induced by the common stimulus. [sent-109, score-0.568]

64 (b) 1 2 The correlation measure S removes the correlation induced by the common stimulus, but cannot distinguish between the two connectivity induced correlations. [sent-110, score-0.69]

65 (c–d) The measures W and U do distinguish the connectivity induced correlations. [sent-111, score-0.34]

66 W reﬂects only the direct ¯ connection (c); U reﬂects only the common input (d). [sent-112, score-0.654]

67 To test the inhibitory case, we modiﬁed the connections so that neuron 1 ¯5 ¯6 received an inhibitory connection from neuron 2 (W21 = W21 = −0. [sent-125, score-1.344]

68 Neuron 2 con¯2 received an inhibitory connection from neuron 3 (W ¯8 ¯9 tinued to receive an excitatory connection from neuron 3 (W32 = W32 = 1. [sent-128, score-1.565]

69 The low ﬁring rates of neurons, however, makes inhibition more difﬁcult to detect via correlations [3]. [sent-130, score-0.287]

70 2 shows the results after simulating for 1,200,000 units of time, obtaining 130,000–140,000 spikes per neuron. [sent-134, score-0.199]

71 With this extraordinarily large number of spikes, W and U successfully distinguish the direct connection correlations from the common input correlations. [sent-135, score-1.018]

72 To test the robustness of the method to deviations from the linear-nonlinear model, we simulated a system of two integrate-and-ﬁre neurons whose input was a threshold-linear function of the stimulus. [sent-136, score-0.439]

73 The neurons received common input from a threshold-linear unit, −3 a x 10 C 5 0 −5 −30 −20 −10 0 Delay 10 20 30 −4 −30 −20 −10 0 Delay 10 20 30 0 −0. [sent-137, score-0.606]

74 1, except that the connections from both neuron 2 and neuron 3 onto neuron 1 were made inhibitory. [sent-142, score-1.308]

75 W reﬂects only the direct connection correlations, and U reﬂects only the common input correlations. [sent-146, score-0.654]

76 and neuron 1 received a direct connection from neuron 2 (see Fig. [sent-157, score-1.282]

77 (5) on a 20 × 20 × 30 grid in space and time, using a 2 ms grid in time, and normalized the resulting vector to obtain the unit vector h i . [sent-160, score-0.221]

78 p A two millisecond sample rate of discrete white noise is unrealistic in many experiments, so we departed further from the assumptions of the derivation and let the stimulus be white √ noise sampled at 10 ms. [sent-161, score-0.472]

79 We let the stimulus standard deviation be σ = 1/ 5 so that it had the same power as discrete white noise sampled at 2 ms with σ = 1. [sent-162, score-0.456]

80 4 shows that the method still effectively distinguishes direct connection correlations from common input correlations. [sent-165, score-0.877]

81 The separation isn’t perfect as W becomes negative where the common input correlation is positive and U becomes negative where the direct input correlation is positive. [sent-166, score-0.775]

82 To determine whether a combination of positive W and negative U, for example, indicates positive direct connection correlation or negative common input correlation, one simply needs to look to see if S is positive or negative. [sent-167, score-0.982]

83 The noise is due to the conditioning of the (non-square) matrix in the j T1 h1 j j 1 Tsp,1 j 2 Tsp,1 T3 h3 X T2 j h2 Figure 3: Diagram of two integrate-and-ﬁre neurons (circles) receiving threshold-linear input from the stimulus. [sent-171, score-0.442]

84 The neurons received common input from threshold-linear unit 3, and neuron 1 received a direct connection from neuron 2. [sent-172, score-1.914]

85 The evolution of the voltage dV of neuron p in response to input gp (t) was given by τm dtp + Vp + gp (t)(Vp − Es ) = 0. [sent-173, score-0.781]

86 When Vp (t) reached 1, a spike was recorded, and the voltage was reset to 0 and held ext int there for an absolute refractory period of length τref . [sent-174, score-0.409]

87 We let gp (t) = gp (t) + gp (t), j ext j where the external input was gp (t) = 0. [sent-175, score-0.579]

88 The internal input g2 (t) to neuron 2 was set to zero, and the internal input to neuron 1 was set to reﬂect an excitatory connection from neuron 2, j j int g1 (t) = 0. [sent-179, score-1.761]

89 05 j G(t − Tsp,2 − δ21 ), where the Tsp,2 are the spike times of neuron 2. [sent-180, score-0.741]

90 One can use a less noisy measure such as S to ﬁnd signiﬁcant stimulus-independent correlations and determine their magnitudes. [sent-187, score-0.259]

91 Then, assuming one can rule out causes like covariation in latency or excitability [7], one simply needs to determine if the correlations were caused by a direct connection or by common input. [sent-188, score-0.828]

92 Clearly, extensions beyond the presented model will be necessary since the linear-nonlinear model can adequately describe the behavior of only a small subset of neurons in primary sensory areas. [sent-191, score-0.31]

93 Though limited in scope and model-dependent, we have demonstrated what appears to be the ﬁrst example of a deﬁnitive dissociation between direct connection and common input correlations from spike time data. [sent-193, score-1.193]

94 At least in the case of excitatory connections, this distinction can be made with a realistic, albeit large, amount of data. [sent-194, score-0.181]

95 2 −150 Figure 4: Results from the simulation of two integrate-and-ﬁre neurons, where neuron 2 had an excitatory connection onto neuron 1 with a delay δ21 = 50 ms. [sent-202, score-1.564]

96 Both neurons received common input, but the common input to neuron 2 was delayed (δ 1 = 0 ms, δ2 = 60 ms). [sent-203, score-1.181]

97 W and U successfully distinguish the direct connection correlations from the common input correlations, but also negatively reﬂect each other. [sent-207, score-0.968]

98 On the signiﬁcance of correlations among neuronal spike trains. [sent-250, score-0.552]

99 The Fourier approach to the identiﬁcation of functional coupling between neuronal spike trains. [sent-264, score-0.355]

100 A spike correlation measure that eliminates stimulus effects in response to white noise. [sent-280, score-0.723]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('neuron', 0.39), ('neurons', 0.31), ('delay', 0.304), ('spike', 0.287), ('connection', 0.277), ('correlations', 0.223), ('connectivity', 0.154), ('direct', 0.153), ('common', 0.139), ('hi', 0.132), ('stimulus', 0.129), ('ms', 0.129), ('unmeasured', 0.125), ('wqp', 0.125), ('cos', 0.111), ('gp', 0.111), ('jpsth', 0.1), ('correlation', 0.099), ('white', 0.098), ('excitatory', 0.096), ('distinguish', 0.091), ('connections', 0.089), ('spikes', 0.086), ('distinction', 0.085), ('input', 0.085), ('measured', 0.077), ('nykamp', 0.075), ('received', 0.072), ('re', 0.066), ('times', 0.064), ('tp', 0.064), ('inhibitory', 0.063), ('correlogram', 0.06), ('response', 0.06), ('reconstructing', 0.059), ('simulation', 0.058), ('induced', 0.054), ('ckj', 0.05), ('ext', 0.05), ('extraordinarily', 0.05), ('gerstein', 0.05), ('delays', 0.05), ('eliminates', 0.05), ('peak', 0.05), ('onto', 0.049), ('vp', 0.048), ('int', 0.048), ('noise', 0.047), ('ects', 0.047), ('ring', 0.047), ('units', 0.046), ('delayed', 0.046), ('network', 0.045), ('tools', 0.044), ('simulated', 0.044), ('shuf', 0.044), ('ref', 0.044), ('rq', 0.043), ('neuronal', 0.042), ('measures', 0.041), ('unknowns', 0.04), ('organisms', 0.04), ('erf', 0.04), ('positive', 0.039), ('negative', 0.038), ('greatly', 0.038), ('equations', 0.038), ('trains', 0.038), ('obtaining', 0.038), ('aertsen', 0.037), ('determine', 0.036), ('kernels', 0.035), ('panels', 0.035), ('idealized', 0.033), ('pq', 0.033), ('inferences', 0.033), ('retain', 0.033), ('grid', 0.033), ('inhibition', 0.032), ('viable', 0.032), ('rates', 0.032), ('sin', 0.031), ('sampled', 0.029), ('hp', 0.029), ('simulating', 0.029), ('ect', 0.029), ('time', 0.029), ('demonstrate', 0.028), ('reconstructed', 0.028), ('effective', 0.027), ('analyze', 0.027), ('unit', 0.026), ('coupling', 0.026), ('theoretically', 0.026), ('third', 0.026), ('increased', 0.026), ('rp', 0.025), ('nonlinearity', 0.025), ('discrete', 0.024), ('voltage', 0.024)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 1.0000001 171 nips-2002-Reconstructing Stimulus-Driven Neural Networks from Spike Times

Author: Duane Q. Nykamp

2 0.33641508 76 nips-2002-Dynamical Constraints on Computing with Spike Timing in the Cortex

Author: Arunava Banerjee, Alexandre Pouget

Abstract: If the cortex uses spike timing to compute, the timing of the spikes must be robust to perturbations. Based on a recent framework that provides a simple criterion to determine whether a spike sequence produced by a generic network is sensitive to initial conditions, and numerical simulations of a variety of network architectures, we argue within the limits set by our model of the neuron, that it is unlikely that precise sequences of spike timings are used for computation under conditions typically found in the cortex.

3 0.25223055 43 nips-2002-Binary Coding in Auditory Cortex

Author: Michael R. Deweese, Anthony M. Zador

Abstract: Cortical neurons have been reported to use both rate and temporal codes. Here we describe a novel mode in which each neuron generates exactly 0 or 1 action potentials, but not more, in response to a stimulus. We used cell-attached recording, which ensured single-unit isolation, to record responses in rat auditory cortex to brief tone pips. Surprisingly, the majority of neurons exhibited binary behavior with few multi-spike responses; several dramatic examples consisted of exactly one spike on 100% of trials, with no trial-to-trial variability in spike count. Many neurons were tuned to stimulus frequency. Since individual trials yielded at most one spike for most neurons, the information about stimulus frequency was encoded in the population, and would not have been accessible to later stages of processing that only had access to the activity of a single unit. These binary units allow a more efficient population code than is possible with conventional rate coding units, and are consistent with a model of cortical processing in which synchronous packets of spikes propagate stably from one neuronal population to the next. 1 Binary coding in auditory cortex We recorded responses of neurons in the auditory cortex of anesthetized rats to pure-tone pips of different frequencies [1, 2]. Each pip was presented repeatedly, allowing us to assess the variability of the neural response to multiple presentations of each stimulus. We first recorded multi-unit activity with conventional tungsten electrodes (Fig. 1a). The number of spikes in response to each pip fluctuated markedly from one trial to the next (Fig. 1e), as though governed by a random mechanism such as that generating the ticks of a Geiger counter. Highly variable responses such as these, which are at least as variable as a Poisson process, are the norm in the cortex [3-7], and have contributed to the widely held view that cortical spike trains are so noisy that only the average firing rate can be used to encode stimuli. Because we were recording the activity of an unknown number of neurons, we could not be sure whether the strong trial-to-trial fluctuations reflected the underlying variability of the single units. We therefore used an alternative technique, cell- a b Single-unit recording method 5mV Multi-unit 1sec Raw cellattached voltage 10 kHz c Single-unit . . . . .. .. ... . . .... . ... . Identified spikes Threshold e 28 kHz d Single-unit 80 120 160 200 Time (msec) N = 29 tones 3 2 1 Poisson N = 11 tones ry 40 4 na bi 38 kHz 0 Response variance/mean (spikes/trial) High-pass filtered 0 0 1 2 3 Mean response (spikes/trial) Figure 1: Multi-unit spiking activity was highly variable, but single units obeyed binomial statistics. a Multi-unit spike rasters from a conventional tungsten electrode recording showed high trial-to-trial variability in response to ten repetitions of the same 50 msec pure tone stimulus (bottom). Darker hash marks indicate spike times within the response period, which were used in the variability analysis. b Spikes recorded in cell-attached mode were easily identified from the raw voltage trace (top) by applying a high-pass filter (bottom) and thresholding (dark gray line). Spike times (black squares) were assigned to the peaks of suprathreshold segments. c Spike rasters from a cell-attached recording of single-unit responses to 25 repetitions of the same tone consisted of exactly one well-timed spike per trial (latency standard deviation = 1.0 msec), unlike the multi-unit responses (Fig. 1a). Under the Poisson assumption, this would have been highly unlikely (P ~ 10 -11). d The same neuron as in Fig. 1c responds with lower probability to repeated presentations of a different tone, but there are still no multi-spike responses. e We quantified response variability for each tone by dividing the variance in spike count by the mean spike count across all trials for that tone. Response variability for multi-unit tungsten recording (open triangles) was high for each of the 29 tones (out of 32) that elicited at least one spike on one trial. All but one point lie above one (horizontal gray line), which is the value produced by a Poisson process with any constant or time varying event rate. Single unit responses recorded in cell-attached mode were far less variable (filled circles). Ninety one percent (10/11) of the tones that elicited at least one spike from this neuron produced no multi-spike responses in 25 trials; the corresponding points fall on the diagonal line between (0,1) and (1,0), which provides a strict lower bound on the variability for any response set with a mean between 0 and 1. No point lies above one. attached recording with a patch pipette [8, 9], in order to ensure single unit isolation (Fig. 1b). This recording mode minimizes both of the main sources of error in spike detection: failure to detect a spike in the unit under observation (false negatives), and contamination by spikes from nearby neurons (false positives). It also differs from conventional extracellular recording methods in its selection bias: With cell- attached recording neurons are selected solely on the basis of the experimenter’s ability to form a seal, rather than on the basis of neuronal activity and responsiveness to stimuli as in conventional methods. Surprisingly, single unit responses were far more orderly than suggested by the multi-unit recordings; responses typically consisted of either 0 or 1 spikes per trial, and not more (Fig. 1c-e). In the most dramatic examples, each presentation of the same tone pip elicited exactly one spike (Fig. 1c). In most cases, however, some presentations failed to elicit a spike (Fig. 1d). Although low-variability responses have recently been observed in the cortex [10, 11] and elsewhere [12, 13], the binary behavior described here has not previously been reported for cortical neurons. a 1.4 N = 3055 response sets b 1.2 1 Poisson 28 kHz - 100 msec 0.8 0.6 0.4 0.2 0 0 ry na bi Response variance/mean (spikes/trial) The majority of the neurons (59%) in our study for which statistical significance could be assessed (at the p<0.001 significance level; see Fig. 2, caption) showed noisy binary behavior—“binary” because neurons produced either 0 or 1 spikes, and “noisy” because some stimuli elicited both single spikes and failures. In a substantial fraction of neurons, however, the responses showed more variability. We found no correlation between neuronal variability and cortical layer (inferred from the depth of the recording electrode), cortical area (inside vs. outside of area A1) or depth of anesthesia. Moreover, the binary mode of spiking was not due to the brevity (25 msec) of the stimuli; responses that were binary for short tones were comparably binary when longer (100 msec) tones were used (Fig. 2b). Not assessable Not significant Significant (p<0.001) 0.2 0.4 0.6 0.8 1 1.2 Mean response (spikes/trial) 28 kHz - 25 msec 1.4 0 40 80 120 160 Time (msec) 200 Figure 2: Half of the neuronal population exhibited binary firing behavior. a Of the 3055 sets of responses to 25 msec tones, 2588 (gray points) could not be assessed for significance at the p<0.001 level, 225 (open circles) were not significantly binary, and 242 were significantly binary (black points; see Identification methods for group statistics below). All points were jittered slightly so that overlying points could be seen in the figure. 2165 response sets contained no multi-spike responses; the corresponding points fell on the line from [0,1] to [1,0]. b The binary nature of single unit responses was insensitive to tone duration, even for frequencies that elicited the largest responses. Twenty additional spike rasters from the same neuron (and tone frequency) as in Fig. 1c contain no multi-spike responses whether in response to 100 msec tones (above) or 25 msec tones (below). Across the population, binary responses were as prevalent for 100 msec tones as for 25 msec tones (see Identification methods for group statistics). In many neurons, binary responses showed high temporal precision, with latencies sometimes exhibiting standard deviations as low as 1 msec (Fig. 3; see also Fig. 1c), comparable to previous observations in the auditory cortex [14], and only slightly more precise than in monkey visual area MT [5]. High temporal precision was positively correlated with high response probability (Fig. 3). a b N = (44 cells)x(32 tones) 14 N = 32 tones 12 30 Jitter (msec) Jitter (msec) 40 10 8 6 20 10 4 2 0 0 0 0.2 0.4 0.6 0.8 Mean response (spikes/trial) 1 0 0.4 0.8 1.2 1.6 Mean response (spikes/trial) 2 Figure 3: Trial-to-trial variability in latency of response to repeated presentations of the same tone decreased with increasing response probability. a Scatter plot of standard deviation of latency vs. mean response for 25 presentations each of 32 tones for a different neuron as in Figs. 1 and 2 (gray line is best linear fit). Rasters from 25 repeated presentations of a low response tone (upper left inset, which corresponds to left-most data point) display much more variable latencies than rasters from a high response tone (lower right inset; corresponds to right-most data point). b The negative correlation between latency variability and response size was present on average across the population of 44 neurons described in Identification methods for group statistics (linear fit, gray). The low trial-to-trial variability ruled out the possibility that the firing statistics could be accounted for by a simple rate-modulated Poisson process (Fig. 4a1,a2). In other systems, low variability has sometimes been modeled as a Poisson process followed by a post-spike refractory period [10, 12]. In our system, however, the range in latencies of evoked binary responses was often much greater than the refractory period, which could not have been longer than the 2 msec inter-spike intervals observed during epochs of spontaneous spiking, indicating that binary spiking did not result from any intrinsic property of the spike generating mechanism (Fig. 4a3). Moreover, a single stimulus-evoked spike could suppress subsequent spikes for as long as hundreds of milliseconds (e.g. Figs. 1d,4d), supporting the idea that binary spiking arises through a circuit-level, rather than a single-neuron, mechanism. Indeed, the fact that this suppression is observed even in the cortex of awake animals [15] suggests that binary spiking is not a special property of the anesthetized state. It seems surprising that binary spiking in the cortex has not previously been remarked upon. In the auditory cortex the explanation may be in part technical: Because firing rates in the auditory cortex tend to be low, multi-unit recording is often used to maximize the total amount of data collected. Moreover, our use of cell-attached recording minimizes the usual bias toward responsive or active neurons. Such explanations are not, however, likely to account for the failure to observe binary spiking in the visual cortex, where spike count statistics have been scrutinized more closely [3-7]. One possibility is that this reflects a fundamental difference between the auditory and visual systems. An alternative interpretation— a1 b Response probability 100 spikes/s 2 kHz Poisson simulation c 100 200 300 400 Time (msec) 500 20 Ratio of pool sizes a2 0 16 12 8 4 0 a3 Poisson with refractory period 0 40 80 120 160 200 Time (msec) d Response probability PSTH 0.2 0.4 0.6 0.8 1 Mean spike count per neuron 1 0.8 N = 32 tones 0.6 0.4 0.2 0 2.0 3.8 7.1 13.2 24.9 46.7 Tone frequency (kHz) Figure 4: a The lack of multi-spike responses elicited by the neuron shown in Fig. 3a were not due to an absolute refractory period since the range of latencies for many tones, like that shown here, was much greater than any reasonable estimate for the neuron’s refractory period. (a1) Experimentally recorded responses. (a2) Using the smoothed post stimulus time histogram (PSTH; bottom) from the set of responses in Fig. 4a, we generated rasters under the assumption of Poisson firing. In this representative example, four double-spike responses (arrows at left) were produced in 25 trials. (a3) We then generated rasters assuming that the neuron fired according to a Poisson process subject to a hard refractory period of 2 msec. Even with a refractory period, this representative example includes one triple- and three double-spike responses. The minimum interspike-interval during spontaneous firing events was less than two msec for five of our neurons, so 2 msec is a conservative upper bound for the refractory period. b. Spontaneous activity is reduced following high-probability responses. The PSTH (top; 0.25 msec bins) of the combined responses from the 25% (8/32) of tones that elicited the largest responses from the same neuron as in Figs. 3a and 4a illustrates a preclusion of spontaneous and evoked activity for over 200 msec following stimulation. The PSTHs from progressively less responsive groups of tones show progressively less preclusion following stimulation. c Fewer noisy binary neurons need to be pooled to achieve the same “signal-to-noise ratio” (SNR; see ref. [24]) as a collection of Poisson neurons. The ratio of the number of Poisson to binary neurons required to achieve the same SNR is plotted against the mean number of spikes elicited per neuron following stimulation; here we have defined the SNR to be the ratio of the mean spike count to the standard deviation of the spike count. d Spike probability tuning curve for the same neuron as in Figs. 1c-e and 2b fit to a Gaussian in tone frequency. and one that we favor—is that the difference rests not in the sensory modality, but instead in the difference between the stimuli used. In this view, the binary responses may not be limited to the auditory cortex; neurons in visual and other sensory cortices might exhibit similar responses to the appropriate stimuli. For example, the tone pips we used might be the auditory analog of a brief flash of light, rather than the oriented moving edges or gratings usually used to probe the primary visual cortex. Conversely, auditory stimuli analogous to edges or gratings [16, 17] may be more likely to elicit conventional, rate-modulated Poisson responses in the auditory cortex. Indeed, there may be a continuum between binary and Poisson modes. Thus, even in conventional rate-modulated responses, the first spike is often privileged in that it carries most of the information in the spike train [5, 14, 18]. The first spike may be particularly important as a means of rapidly signaling stimulus transients. Binary responses suggest a mode that complements conventional rate coding. In the simplest rate-coding model, a stimulus parameter (such as the frequency of a tone) governs only the rate at which a neuron generates spikes, but not the detailed positions of the spikes; the actual spike train itself is an instantiation of a random process (such as a Poisson process). By contrast, in the binomial model, the stimulus parameter (frequency) is encoded as the probability of firing (Fig. 4d). Binary coding has implications for cortical computation. In the rate coding model, stimulus encoding is “ergodic”: a stimulus parameter can be read out either by observing the activity of one neuron for a long time, or a population for a short time. By contrast, in the binary model the stimulus value can be decoded only by observing a neuronal population, so that there is no benefit to integrating over long time periods (cf. ref. [19]). One advantage of binary encoding is that it allows the population to signal quickly; the most compact message a neuron can send is one spike [20]. Binary coding is also more efficient in the context of population coding, as quantified by the signal-to-noise ratio (Fig. 4c). The precise organization of both spike number and time we have observed suggests that cortical activity consists, at least under some conditions, of packets of spikes synchronized across populations of neurons. Theoretical work [21-23] has shown how such packets can propagate stably from one population to the next, but only if neurons within each population fire at most one spike per packet; otherwise, the number of spikes per packet—and hence the width of each packet—grows at each propagation step. Interestingly, one prediction of stable propagation models is that spike probability should be related to timing precision, a prediction born out by our observations (Fig. 3). The role of these packets in computation remains an open question. 2 Identification methods for group statistics We recorded responses to 32 different 25 msec tones from each of 175 neurons from the auditory cortices of 16 Sprague-Dawley rats; each tone was repeated between 5 and 75 times (mean = 19). Thus our ensemble consisted of 32x175=5600 response sets, with between 5 and 75 samples in each set. Of these, 3055 response sets contained at least one spike on at least on trial. For each response set, we tested the hypothesis that the observed variability was significantly lower than expected from the null hypothesis of a Poisson process. The ability to assess significance depended on two parameters: the sample size (5-75) and the firing probability. Intuitively, the dependence on firing probability arises because at low firing rates most responses produce only trials with 0 or 1 spikes under both the Poisson and binary models; only at high firing rates do the two models make different predictions, since in that case the Poisson model includes many trials with 2 or even 3 spikes while the binary model generates only solitary spikes (see Fig. 4a1,a2). Using a stringent significance criterion of p<0.001, 467 response sets had a sufficient number of repeats to assess significance, given the observed firing probability. Of these, half (242/467=52%) were significantly less variable than expected by chance, five hundred-fold higher than the 467/1000=0.467 response sets expected, based on the 0.001 significance criterion, to yield a binary response set. Seventy-two neurons had at least one response set for which significance could be assessed, and of these, 49 neurons (49/72=68%) had at least one significantly sub-Poisson response set. Of this population of 49 neurons, five achieved low variability through repeatable bursty behavior (e.g., every spike count was either 0 or 3, but not 1 or 2) and were excluded from further analysis. The remaining 44 neurons formed the basis for the group statistics analyses shown in Figs. 2a and 3b. Nine of these neurons were subjected to an additional protocol consisting of at least 10 presentations each of 100 msec tones and 25 msec tones of all 32 frequencies. Of the 100 msec stimulation response sets, 44 were found to be significantly sub-Poisson at the p<0.05 level, in good agreement with the 43 found to be significant among the responses to 25 msec tones. 3 Bibliography 1. Kilgard, M.P. and M.M. Merzenich, Cortical map reorganization enabled by nucleus basalis activity. Science, 1998. 279(5357): p. 1714-8. 2. Sally, S.L. and J.B. Kelly, Organization of auditory cortex in the albino rat: sound frequency. J Neurophysiol, 1988. 59(5): p. 1627-38. 3. Softky, W.R. and C. Koch, The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs. J Neurosci, 1993. 13(1): p. 334-50. 4. Stevens, C.F. and A.M. Zador, Input synchrony and the irregular firing of cortical neurons. Nat Neurosci, 1998. 1(3): p. 210-7. 5. Buracas, G.T., A.M. Zador, M.R. DeWeese, and T.D. Albright, Efficient discrimination of temporal patterns by motion-sensitive neurons in primate visual cortex. Neuron, 1998. 20(5): p. 959-69. 6. Shadlen, M.N. and W.T. Newsome, The variable discharge of cortical neurons: implications for connectivity, computation, and information coding. J Neurosci, 1998. 18(10): p. 3870-96. 7. Tolhurst, D.J., J.A. Movshon, and A.F. Dean, The statistical reliability of signals in single neurons in cat and monkey visual cortex. Vision Res, 1983. 23(8): p. 775-85. 8. Otmakhov, N., A.M. Shirke, and R. Malinow, Measuring the impact of probabilistic transmission on neuronal output. Neuron, 1993. 10(6): p. 1101-11. 9. Friedrich, R.W. and G. Laurent, Dynamic optimization of odor representations by slow temporal patterning of mitral cell activity. Science, 2001. 291(5505): p. 889-94. 10. Kara, P., P. Reinagel, and R.C. Reid, Low response variability in simultaneously recorded retinal, thalamic, and cortical neurons. Neuron, 2000. 27(3): p. 635-46. 11. Gur, M., A. Beylin, and D.M. Snodderly, Response variability of neurons in primary visual cortex (V1) of alert monkeys. J Neurosci, 1997. 17(8): p. 2914-20. 12. Berry, M.J., D.K. Warland, and M. Meister, The structure and precision of retinal spike trains. Proc Natl Acad Sci U S A, 1997. 94(10): p. 5411-6. 13. de Ruyter van Steveninck, R.R., G.D. Lewen, S.P. Strong, R. Koberle, and W. Bialek, Reproducibility and variability in neural spike trains. Science, 1997. 275(5307): p. 1805-8. 14. Heil, P., Auditory cortical onset responses revisited. I. First-spike timing. J Neurophysiol, 1997. 77(5): p. 2616-41. 15. Lu, T., L. Liang, and X. Wang, Temporal and rate representations of timevarying signals in the auditory cortex of awake primates. Nat Neurosci, 2001. 4(11): p. 1131-8. 16. Kowalski, N., D.A. Depireux, and S.A. Shamma, Analysis of dynamic spectra in ferret primary auditory cortex. I. Characteristics of single-unit responses to moving ripple spectra. J Neurophysiol, 1996. 76(5): p. 350323. 17. deCharms, R.C., D.T. Blake, and M.M. Merzenich, Optimizing sound features for cortical neurons. Science, 1998. 280(5368): p. 1439-43. 18. Panzeri, S., R.S. Petersen, S.R. Schultz, M. Lebedev, and M.E. Diamond, The role of spike timing in the coding of stimulus location in rat somatosensory cortex. Neuron, 2001. 29(3): p. 769-77. 19. Britten, K.H., M.N. Shadlen, W.T. Newsome, and J.A. Movshon, The analysis of visual motion: a comparison of neuronal and psychophysical performance. J Neurosci, 1992. 12(12): p. 4745-65. 20. Delorme, A. and S.J. Thorpe, Face identification using one spike per neuron: resistance to image degradations. Neural Netw, 2001. 14(6-7): p. 795-803. 21. Diesmann, M., M.O. Gewaltig, and A. Aertsen, Stable propagation of synchronous spiking in cortical neural networks. Nature, 1999. 402(6761): p. 529-33. 22. Marsalek, P., C. Koch, and J. Maunsell, On the relationship between synaptic input and spike output jitter in individual neurons. Proc Natl Acad Sci U S A, 1997. 94(2): p. 735-40. 23. Kistler, W.M. and W. Gerstner, Stable propagation of activity pulses in populations of spiking neurons. Neural Comp., 2002. 14: p. 987-997. 24. Zohary, E., M.N. Shadlen, and W.T. Newsome, Correlated neuronal discharge rate and its implications for psychophysical performance. Nature, 1994. 370(6485): p. 140-3. 25. Abbott, L.F. and P. Dayan, The effect of correlated variability on the accuracy of a population code. Neural Comput, 1999. 11(1): p. 91-101.

4 0.24436805 11 nips-2002-A Model for Real-Time Computation in Generic Neural Microcircuits

Author: Wolfgang Maass, Thomas Natschläger, Henry Markram

Abstract: A key challenge for neural modeling is to explain how a continuous stream of multi-modal input from a rapidly changing environment can be processed by stereotypical recurrent circuits of integrate-and-ﬁre neurons in real-time. We propose a new computational model that is based on principles of high dimensional dynamical systems in combination with statistical learning theory. It can be implemented on generic evolved or found recurrent circuitry.

5 0.21657993 184 nips-2002-Spectro-Temporal Receptive Fields of Subthreshold Responses in Auditory Cortex

Author: Christian K. Machens, Michael Wehr, Anthony M. Zador

Abstract: How do cortical neurons represent the acoustic environment? This question is often addressed by probing with simple stimuli such as clicks or tone pips. Such stimuli have the advantage of yielding easily interpreted answers, but have the disadvantage that they may fail to uncover complex or higher-order neuronal response properties. Here we adopt an alternative approach, probing neuronal responses with complex acoustic stimuli, including animal vocalizations and music. We have used in vivo whole cell methods in the rat auditory cortex to record subthreshold membrane potential ﬂuctuations elicited by these stimuli. Whole cell recording reveals the total synaptic input to a neuron from all the other neurons in the circuit, instead of just its output—a sparse binary spike train—as in conventional single unit physiological recordings. Whole cell recording thus provides a much richer source of information about the neuron’s response. Many neurons responded robustly and reliably to the complex stimuli in our ensemble. Here we analyze the linear component—the spectrotemporal receptive ﬁeld (STRF)—of the transformation from the sound (as represented by its time-varying spectrogram) to the neuron’s membrane potential. We ﬁnd that the STRF has a rich dynamical structure, including excitatory regions positioned in general accord with the prediction of the simple tuning curve. We also ﬁnd that in many cases, much of the neuron’s response, although deterministically related to the stimulus, cannot be predicted by the linear component, indicating the presence of as-yet-uncharacterized nonlinear response properties.

6 0.21654247 141 nips-2002-Maximally Informative Dimensions: Analyzing Neural Responses to Natural Signals

7 0.1967552 116 nips-2002-Interpreting Neural Response Variability as Monte Carlo Sampling of the Posterior

8 0.1911412 148 nips-2002-Morton-Style Factorial Coding of Color in Primary Visual Cortex

9 0.168193 102 nips-2002-Hidden Markov Model of Cortical Synaptic Plasticity: Derivation of the Learning Rule

10 0.16788018 5 nips-2002-A Digital Antennal Lobe for Pattern Equalization: Analysis and Design

11 0.16452587 50 nips-2002-Circuit Model of Short-Term Synaptic Dynamics

12 0.15597104 28 nips-2002-An Information Theoretic Approach to the Functional Classification of Neurons

13 0.13980378 129 nips-2002-Learning in Spiking Neural Assemblies

14 0.13642691 187 nips-2002-Spikernels: Embedding Spiking Neurons in Inner-Product Spaces

15 0.1325663 180 nips-2002-Selectivity and Metaplasticity in a Unified Calcium-Dependent Model

16 0.13237052 103 nips-2002-How Linear are Auditory Cortical Responses?

17 0.1304998 44 nips-2002-Binary Tuning is Optimal for Neural Rate Coding with High Temporal Resolution

18 0.12828897 154 nips-2002-Neuromorphic Bisable VLSI Synapses with Spike-Timing-Dependent Plasticity

19 0.12606822 186 nips-2002-Spike Timing-Dependent Plasticity in the Address Domain

20 0.12585424 199 nips-2002-Timing and Partial Observability in the Dopamine System

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, -0.249), (1, 0.416), (2, 0.095), (3, -0.16), (4, 0.022), (5, -0.003), (6, 0.022), (7, 0.003), (8, -0.071), (9, 0.02), (10, -0.039), (11, -0.033), (12, 0.001), (13, -0.051), (14, -0.062), (15, -0.03), (16, -0.098), (17, 0.086), (18, -0.084), (19, 0.1), (20, -0.099), (21, 0.077), (22, -0.019), (23, 0.02), (24, 0.054), (25, -0.015), (26, -0.057), (27, 0.075), (28, 0.011), (29, 0.121), (30, 0.036), (31, -0.018), (32, -0.041), (33, 0.066), (34, 0.014), (35, 0.115), (36, -0.065), (37, 0.02), (38, -0.08), (39, -0.012), (40, 0.022), (41, -0.031), (42, 0.007), (43, 0.072), (44, -0.065), (45, -0.007), (46, -0.067), (47, -0.057), (48, 0.096), (49, -0.004)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.98953867 171 nips-2002-Reconstructing Stimulus-Driven Neural Networks from Spike Times

Author: Duane Q. Nykamp

2 0.91469026 76 nips-2002-Dynamical Constraints on Computing with Spike Timing in the Cortex

Author: Arunava Banerjee, Alexandre Pouget

3 0.83575451 43 nips-2002-Binary Coding in Auditory Cortex

Author: Michael R. Deweese, Anthony M. Zador

4 0.81162417 11 nips-2002-A Model for Real-Time Computation in Generic Neural Microcircuits

Author: Wolfgang Maass, Thomas Natschläger, Henry Markram

5 0.65981632 5 nips-2002-A Digital Antennal Lobe for Pattern Equalization: Analysis and Design

Author: Alex Holub, Gilles Laurent, Pietro Perona

Abstract: Re-mapping patterns in order to equalize their distribution may greatly simplify both the structure and the training of classifiers. Here, the properties of one such map obtained by running a few steps of discrete-time dynamical system are explored. The system is called 'Digital Antennal Lobe' (DAL) because it is inspired by recent studies of the antennallobe, a structure in the olfactory system of the grasshopper. The pattern-spreading properties of the DAL as well as its average behavior as a function of its (few) design parameters are analyzed by extending previous results of Van Vreeswijk and Sompolinsky. Furthermore, a technique for adapting the parameters of the initial design in order to obtain opportune noise-rejection behavior is suggested. Our results are demonstrated with a number of simulations. 1

6 0.62463582 141 nips-2002-Maximally Informative Dimensions: Analyzing Neural Responses to Natural Signals

7 0.5896309 160 nips-2002-Optoelectronic Implementation of a FitzHugh-Nagumo Neural Model

8 0.58721781 50 nips-2002-Circuit Model of Short-Term Synaptic Dynamics

9 0.58499199 44 nips-2002-Binary Tuning is Optimal for Neural Rate Coding with High Temporal Resolution

10 0.57893538 184 nips-2002-Spectro-Temporal Receptive Fields of Subthreshold Responses in Auditory Cortex

11 0.55090356 102 nips-2002-Hidden Markov Model of Cortical Synaptic Plasticity: Derivation of the Learning Rule

12 0.54624325 180 nips-2002-Selectivity and Metaplasticity in a Unified Calcium-Dependent Model

13 0.5268265 148 nips-2002-Morton-Style Factorial Coding of Color in Primary Visual Cortex

14 0.51372081 116 nips-2002-Interpreting Neural Response Variability as Monte Carlo Sampling of the Posterior

15 0.50454497 60 nips-2002-Convergence Properties of Some Spike-Triggered Analysis Techniques

16 0.46308634 12 nips-2002-A Neural Edge-Detection Model for Enhanced Auditory Sensitivity in Modulated Noise

17 0.44485441 154 nips-2002-Neuromorphic Bisable VLSI Synapses with Spike-Timing-Dependent Plasticity

18 0.44321117 71 nips-2002-Dopamine Induced Bistability Enhances Signal Processing in Spiny Neurons

19 0.44223323 186 nips-2002-Spike Timing-Dependent Plasticity in the Address Domain

20 0.42828485 136 nips-2002-Linear Combinations of Optic Flow Vectors for Estimating Self-Motion - a Real-World Test of a Neural Model

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(23, 0.027), (42, 0.04), (54, 0.082), (55, 0.031), (57, 0.011), (64, 0.011), (67, 0.011), (68, 0.535), (74, 0.041), (92, 0.02), (98, 0.106)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.95867449 171 nips-2002-Reconstructing Stimulus-Driven Neural Networks from Spike Times

Author: Duane Q. Nykamp

2 0.79871809 73 nips-2002-Dynamic Bayesian Networks with Deterministic Latent Tables

Author: David Barber

Abstract: The application of latent/hidden variable Dynamic Bayesian Networks is constrained by the complexity of marginalising over latent variables. For this reason either small latent dimensions or Gaussian latent conditional tables linearly dependent on past states are typically considered in order that inference is tractable. We suggest an alternative approach in which the latent variables are modelled using deterministic conditional probability tables. This specialisation has the advantage of tractable inference even for highly complex non-linear/non-Gaussian visible conditional probability tables. This approach enables the consideration of highly complex latent dynamics whilst retaining the beneﬁts of a tractable probabilistic model. 1

3 0.74547225 62 nips-2002-Coulomb Classifiers: Generalizing Support Vector Machines via an Analogy to Electrostatic Systems

Author: Sepp Hochreiter, Michael C. Mozer, Klaus Obermayer

Abstract: We introduce a family of classiﬁers based on a physical analogy to an electrostatic system of charged conductors. The family, called Coulomb classiﬁers, includes the two best-known support-vector machines (SVMs), the ν–SVM and the C–SVM. In the electrostatics analogy, a training example corresponds to a charged conductor at a given location in space, the classiﬁcation function corresponds to the electrostatic potential function, and the training objective function corresponds to the Coulomb energy. The electrostatic framework provides not only a novel interpretation of existing algorithms and their interrelationships, but it suggests a variety of new methods for SVMs including kernels that bridge the gap between polynomial and radial-basis functions, objective functions that do not require positive-deﬁnite kernels, regularization techniques that allow for the construction of an optimal classiﬁer in Minkowski space. Based on the framework, we propose novel SVMs and perform simulation studies to show that they are comparable or superior to standard SVMs. The experiments include classiﬁcation tasks on data which are represented in terms of their pairwise proximities, where a Coulomb Classiﬁer outperformed standard SVMs. 1

4 0.67724574 76 nips-2002-Dynamical Constraints on Computing with Spike Timing in the Cortex

Author: Arunava Banerjee, Alexandre Pouget

5 0.67247283 49 nips-2002-Charting a Manifold

Author: Matthew Brand

Abstract: We construct a nonlinear mapping from a high-dimensional sample space to a low-dimensional vector space, effectively recovering a Cartesian coordinate system for the manifold from which the data is sampled. The mapping preserves local geometric relations in the manifold and is pseudo-invertible. We show how to estimate the intrinsic dimensionality of the manifold from samples, decompose the sample data into locally linear low-dimensional patches, merge these patches into a single lowdimensional coordinate system, and compute forward and reverse mappings between the sample and coordinate spaces. The objective functions are convex and their solutions are given in closed form. 1 Nonlinear dimensionality reduction (NLDR) by charting Charting is the problem of assigning a low-dimensional coordinate system to data points in a high-dimensional sample space. It is presumed that the data lies on or near a lowdimensional manifold embedded in the sample space, and that there exists a 1-to-1 smooth nonlinear transform between the manifold and a low-dimensional vector space. The datamodeler’s goal is to estimate smooth continuous mappings between the sample and coordinate spaces. Often this analysis will shed light on the intrinsic variables of the datagenerating phenomenon, for example, revealing perceptual or conﬁguration spaces. Our goal is to ﬁnd a mapping—expressed as a kernel-based mixture of linear projections— that minimizes information loss about the density and relative locations of sample points. This constraint is expressed in a posterior that combines a standard gaussian mixture model (GMM) likelihood function with a prior that penalizes uncertainty due to inconsistent projections in the mixture. Section 3 develops a special case where this posterior is unimodal and maximizable in closed form, yielding a GMM whose covariances reveal a patchwork of overlapping locally linear subspaces that cover the manifold. Section 4 shows that for this (or any) GMM and a choice of reduced dimension d, there is a unique, closed-form solution for a minimally distorting merger of the subspaces into a d-dimensional coordinate space, as well as an reverse mapping deﬁning the surface of the manifold in the sample space. The intrinsic dimensionality d of the data manifold can be estimated from the growth process of point-to-point distances. In analogy to differential geometry, we call the subspaces “charts” and their merger the “connection.” Section 5 considers example problems where these methods are used to untie knots, unroll and untwist sheets, and visualize video data. 1.1 Background Topology-neutral NLDR algorithms can be divided into those that compute mappings, and those that directly compute low-dimensional embeddings. The ﬁ has its roots in mapeld ping algorithms: DeMers and Cottrell [3] proposed using auto-encoding neural networks with a hidden layer “ bottleneck,” effectively casting dimensionality reduction as a compression problem. Hastie deﬁ principal curves [5] as nonparametric 1 D curves that pass ned through the center of “ nearby” data points. A rich literature has grown up around properly regularizing this approach and extending it to surfaces. Smola and colleagues [10] analyzed the NLDR problem in the broader framework of regularized quantization methods. More recent advances aim for embeddings: Gomes and Mojsilovic [4] treat manifold completion as an anisotropic diffusion problem, iteratively expanding points until they connect to their neighbors. The I SO M AP algorithm [12] represents remote distances as sums of a trusted set of distances between immediate neighbors, then uses multidimensional scaling to compute a low-dimensional embedding that minimally distorts all distances. The locally linear embedding algorithm (LLE) [9] represents each point as a weighted combination of a trusted set of nearest neighbors, then computes a minimally distorting low-dimensional barycentric embedding. They have complementary strengths: I SO M AP handles holes well but can fail if the data hull is nonconvex [12]; and vice versa for LLE [9]. Both offer embeddings without mappings. It has been noted that trusted-set methods are vulnerable to noise because they consider the subset of point-to-point relationships that has the lowest signal-to-noise ratio; small changes to the trusted set can induce large changes in the set of constraints on the embedding, making solutions unstable [1]. In a return to mapping, Roweis and colleagues [8] proposed global coordination— learning a mixture of locally linear projections from sample to coordinate space. They constructed a posterior that penalizes distortions in the mapping, and gave a expectation-maximization (EM) training rule. Innovative use of variational methods highlighted the difﬁ culty of even hill-climbing their multimodal posterior. Like [2, 7, 6, 8], the method we develop below is a decomposition of the manifold into locally linear neighborhoods. It bears closest relation to global coordination [8], although by a different construction of the problem, we avoid hill-climbing a spiky posterior and instead develop a closed-form solution. 2 Estimating locally linear scale and intrinsic dimensionality . We begin with matrix of sample points Y = [y1 , · · · , yN ], yn ∈ RD populating a Ddimensional sample space, and a conjecture that these points are samples from a manifold M of intrinsic dimensionality d < D. We seek a mapping onto a vector space . G(Y) → X = [x1 , · · · , xN ], xn ∈ Rd and 1-to-1 reverse mapping G−1 (X) → Y such that local relations between nearby points are preserved (this will be formalized below). The map G should be non-catastrophic, that is, without folds: Parallel lines on the manifold in RD should map to continuous smooth non-intersecting curves in Rd . This guarantees that linear operations on X such as interpolation will have reasonable analogues on Y. Smoothness means that at some scale r the mapping from a neighborhood on M to Rd is effectively linear. Consider a ball of radius r centered on a data point and containing n(r) data points. The count n(r) grows as rd , but only at the locally linear scale; the grow rate is inﬂated by isotropic noise at smaller scales and by embedding curvature at larger scales. . To estimate r, we look at how the r-ball grows as points are added to it, tracking c(r) = d d log n(r) log r. At noise scales, c(r) ≈ 1/D < 1/d, because noise has distributed points in all directions with equal probability. At the scale at which curvature becomes signiﬁ cant, c(r) < 1/d, because the manifold is no longer perpendicular to the surface of the ball, so the ball does not have to grow as fast to accommodate new points. At the locally linear scale, the process peaks at c(r) = 1/d, because points are distributed only in the directions of the manifold’s local tangent space. The maximum of c(r) therefore gives an estimate of both the scale and the local dimensionality of the manifold (see ﬁ gure 1), provided that the ball hasn’t expanded to a manifold boundary— boundaries have lower dimension than Scale behavior of a 1D manifold in 2-space Point−count growth process on a 2D manifold in 3−space 1 10 radial growth process 1D hypothesis 2D hypothesis 3D hypothesis radius (log scale) samples noise scale locally linear scale curvature scale 0 10 2 1 10 2 10 #points (log scale) 3 10 Figure 1: Point growth processes. L EFT: At the locally linear scale, the number of points in an r-ball grows as rd ; at noise and curvature scales it grows faster. R IGHT: Using the point-count growth process to ﬁ the intrinsic dimensionality of a 2D manifold nonlinearly nd embedded in 3-space (see ﬁ gure 2). Lines of slope 1/3 , 1/2 , and 1 are ﬁ tted to sections of the log r/ log nr curve. For neighborhoods of radius r ≈ 1 with roughly n ≈ 10 points, the slope peaks at 1/2 indicating a dimensionality of d = 2. Below that, the data appears 3 D because it is dominated by noise (except for n ≤ D points); above, the data appears >2 D because of manifold curvature. As the r-ball expands to cover the entire data-set the dimensionality appears to drop to 1 as the process begins to track the 1D edges of the 2D sheet. the manifold. For low-dimensional manifolds such as sheets, the boundary submanifolds (edges and corners) are very small relative to the full manifold, so the boundary effect is typically limited to a small rise in c(r) as r approaches the scale of the entire data set. In practice, our code simply expands an r-ball at every point and looks for the ﬁ peak in rst c(r), averaged over many nearby r-balls. One can estimate d and r globally or per-point. 3 Charting the data In the charting step we ﬁ a soft partitioning of the data into locally linear low-dimensional nd neighborhoods, as a prelude to computing the connection that gives the global lowdimensional embedding. To minimize information loss in the connection, we require that the data points project into a subspace associated with each neighborhood with (1) minimal loss of local variance and (2) maximal agreement of the projections of nearby points into nearby neighborhoods. Criterion (1) is served by maximizing the likelihood function of a Gaussian mixture model (GMM) density ﬁ tted to the data: . p(yi |µ, Σ) = ∑ j p(yi |µ j , Σ j ) p j = ∑ j N (yi ; µ j , Σ j ) p j . (1) Each gaussian component deﬁ a local neighborhood centered around µ j with axes denes ﬁ ned by the eigenvectors of Σ j . The amount of data variance along each axis is indicated by the eigenvalues of Σ j ; if the data manifold is locally linear in the vicinity of the µ j , all but the d dominant eigenvalues will be near-zero, implying that the associated eigenvectors constitute the optimal variance-preserving local coordinate system. To some degree likelihood maximization will naturally realize this property: It requires that the GMM components shrink in volume to ﬁ the data as tightly as possible, which is best achieved by t positioning the components so that they “ pancake” onto locally ﬂat collections of datapoints. However, this state of affairs is easily violated by degenerate (zero-variance) GMM components or components ﬁ tted to overly small enough locales where the data density off the manifold is comparable to density on the manifold (e.g., at the noise scale). Consequently a prior is needed. Criterion (2) implies that neighboring partitions should have dominant axes that span similar subspaces, since disagreement (large subspace angles) would lead to inconsistent projections of a point and therefore uncertainty about its location in a low-dimensional coordinate space. The principal insight is that criterion (2) is exactly the cost of coding the location of a point in one neighborhood when it is generated by another neighborhood— the cross-entropy between the gaussian models deﬁ ning the two neighborhoods: D(N1 N2 ) = = dy N (y; µ1 ,Σ1 ) log N (y; µ1 ,Σ1 ) N (y; µ2 ,Σ2 ) (log |Σ−1 Σ2 | + trace(Σ−1 Σ1 ) + (µ2 −µ1 ) Σ−1 (µ2 −µ1 ) − D)/2. (2) 1 2 2 Roughly speaking, the terms in (2) measure differences in size, orientation, and position, respectively, of two coordinate frames located at the means µ1 , µ2 with axes speciﬁ by ed the eigenvectors of Σ1 , Σ2 . All three terms decline to zero as the overlap between the two frames is maximized. To maximize consistency between adjacent neighborhoods, we form . the prior p(µ, Σ) = exp[− ∑i= j mi (µ j )D(Ni N j )], where mi (µ j ) is a measure of co-locality. Unlike global coordination [8], we are not asking that the dominant axes in neighboring charts are aligned— only that they span nearly the same subspace. This is a much easier objective to satisfy, and it contains a useful special case where the posterior p(µ, Σ|Y) ∝ ∑i p(yi |µ, Σ)p(µ, Σ) is unimodal and can be maximized in closed form: Let us associate a gaussian neighborhood with each data-point, setting µi = yi ; take all neighborhoods to be a priori equally probable, setting pi = 1/N; and let the co-locality measure be determined from some local kernel. For example, in this paper we use mi (µ j ) ∝ N (µ j ; µi , σ2 ), with the scale parameter σ specifying the expected size of a neighborhood on the manifold in sample space. A reasonable choice is σ = r/2, so that 2erf(2) > 99.5% of the density of mi (µ j ) is contained in the area around yi where the manifold is expected to be locally linear. With uniform pi and µi , mi (µ j ) and ﬁ xed, the MAP estimates of the GMM covariances are Σi = ∑ mi (µ j ) (y j − µi )(y j − µi ) + (µ j − µi )(µ j − µi ) + Σ j j ∑ mi (µ j ) (3) . j Note that each covariance Σi is dependent on all other Σ j . The MAP estimators for all covariances can be arranged into a set of fully constrained linear equations and solved exactly for their mutually optimal values. This key step brings nonlocal information about the manifold’s shape into the local description of each neighborhood, ensuring that adjoining neighborhoods have similar covariances and small angles between their respective subspaces. Even if a local subset of data points are dense in a direction perpendicular to the manifold, the prior encourages the local chart to orient parallel to the manifold as part of a globally optimal solution, protecting against a pathology noted in [8]. Equation (3) is easily adapted to give a reduced number of charts and/or charts centered on local centroids. 4 Connecting the charts We now build a connection for set of charts speciﬁ as an arbitrary nondegenerate GMM. A ed GMM gives a soft partitioning of the dataset into neighborhoods of mean µk and covariance Σk . The optimal variance-preserving low-dimensional coordinate system for each neighborhood derives from its weighted principal component analysis, which is exactly speciﬁ ed by the eigenvectors of its covariance matrix: Eigendecompose Vk Λk Vk ← Σk with eigen. values in descending order on the diagonal of Λk and let Wk = [Id , 0]Vk be the operator . th projecting points into the k local chart, such that local chart coordinate uki = Wk (yi − µk ) . and Uk = [uk1 , · · · , ukN ] holds the local coordinates of all points. Our goal is to sew together all charts into a globally consistent low-dimensional coordinate system. For each chart there will be a low-dimensional afﬁ transform Gk ∈ R(d+1)×d ne that projects Uk into the global coordinate space. Summing over all charts, the weighted average of the projections of point yi into the low-dimensional vector space is W j (y − µ j ) 1 . x|y = ∑ G j j p j|y (y) . xi |yi = ∑ G j ⇒ u ji 1 j p j|y (yi ), (4) where pk|y (y) ∝ pk N (y; µk , Σk ), ∑k pk|y (y) = 1 is the probability that chart k generates point y. As pointed out in [8], if a point has nonzero probabilities in two charts, then there should be afﬁ transforms of those two charts that map the point to the same place in a ne global coordinate space. We set this up as a weighted least-squares problem: . G = [G1 , · · · , GK ] = arg min uki 1 ∑ pk|y (yi )p j|y (yi ) Gk Gk ,G j i −Gj u ji 1 2 . (5) F Equation (5) generates a homogeneous set of equations that determines a solution up to an afﬁ transform of G. There are two solution methods. First, let us temporarily anchor one ne neighborhood at the origin to ﬁ this indeterminacy. This adds the constraint G1 = [I, 0] . x . To solve, deﬁ indicator matrix Fk = [0, · · · , 0, I, 0, · · · , 0] with the identity mane . trix occupying the kth block, such that Gk = GFk . Let the diagonal of Pk = diag([pk|y (y1 ), · · · , pk|y (yN )]) record the per-point posteriors of chart k. The squared error of the connection is then a sum of of all patch-to-anchor and patch-to-patch inconsistencies: . E =∑ (GUk − k U1 0 2 )Pk P1 F + ∑ (GU j − GUk )P j Pk j=k 2 F ; . Uk = Fk Uk 1 . (6) Setting dE /dG = 0 and solving to minimize convex E gives −1 G = ∑ Uk P2 k k ∑ j=k P2 j Uk − ∑ ∑ Uk P2 P2 k 1 Uk P2 P2 U j k j k j=k U1 0 . (7) We now remove the dependence on a reference neighborhood G1 by rewriting equation 5, G = arg min ∑ j=k (GU j − GUk )P j Pk G 2 F = GQ 2 F = trace(GQQ G ) , (8) . where Q = ∑ j=k U j − Uk P j Pk . If we require that GG = I to prevent degenerate solutions, then equation (8) is solved (up to rotation in coordinate space) by setting G to the eigenvectors associated with the smallest eigenvalues of QQ . The eigenvectors can be computed efﬁ ciently without explicitly forming QQ ; other numerical efﬁ ciencies obtain by zeroing any vanishingly small probabilities in each Pk , yielding a sparse eigenproblem. A more interesting strategy is to numerically condition the problem by calculating the trailing eigenvectors of QQ + 1. It can be shown that this maximizes the posterior 2 p(G|Q) ∝ p(Q|G)p(G) ∝ e− GQ F e− G1 , where the prior p(G) favors a mapping G whose unit-norm rows are also zero-mean. This maximizes variance in each row of G and thereby spreads the projected points broadly and evenly over coordinate space. The solutions for MAP charts (equation (5)) and connection (equation (8)) can be applied to any well-ﬁ tted mixture of gaussians/factors1 /PCAs density model; thus large eigenproblems can be avoided by connecting just a small number of charts that cover the data. 1 We thank reviewers for calling our attention to Teh & Roweis ([11]— in this volume), which shows how to connect a set of given local dimensionality reducers in a generalized eigenvalue problem that is related to equation (8). LLE, n=5 charting (projection onto coordinate space) charting best Isomap LLE, n=6 LLE, n=7 LLE, n=8 random subset of local charts XYZ view LLE, n=9 LLE, n=10 XZ view data (linked) embedding, XY view XY view original data reconstruction (back−projected coordinate grid) best LLE (regularized) Figure 2: The twisted curl problem. L EFT: Comparison of charting, I SO M AP, & LLE. 400 points are randomly sampled from the manifold with noise. Charting is the only method that recovers the original space without catastrophes (folding), albeit with some shear. R IGHT: The manifold is regularly sampled (with noise) to illustrate the forward and backward projections. Samples are shown linked into lines to help visualize the manifold structure. Coordinate axes of a random selection of charts are shown as bold lines. Connecting subsets of charts such as this will also give good mappings. The upper right quadrant shows various LLE results. At bottom we show the charting solution and the reconstructed (back-projected) manifold, which smooths out the noise. Once the connection is solved, equation (4) gives the forward projection of any point y down into coordinate space. There are several numerically distinct candidates for the backprojection: posterior mean, mode, or exact inverse. In general, there may not be a unique posterior mode and the exact inverse is not solvable in closed form (this is also true of [8]). Note that chart-wise projection deﬁ a complementary density in coordinate space nes px|k (x) = N (x; Gk 0 1 , Gk [Id , 0]Λk [Id , 0] 0 0 0 Gk ). (9) Let p(y|x, k), used to map x into subspace k on the surface of the manifold, be a Dirac delta function whose mean is a linear function of x. Then the posterior mean back-projection is obtained by integrating out uncertainty over which chart generates x: y|x = ∑ pk|x (x) k µk + Wk Gk I 0 + x − Gk 0 1 , (10) where (·)+ denotes pseudo-inverse. In general, a back-projecting map should not reconstruct the original points. Instead, equation (10) generates a surface that passes through the weighted average of the µi of all the neighborhoods in which yi has nonzero probability, much like a principal curve passes through the center of each local group of points. 5 Experiments Synthetic examples: 400 2 D points were randomly sampled from a 2 D square and embedded in 3 D via a curl and twist, then contaminated with gaussian noise. Even if noiselessly sampled, this manifold cannot be “ unrolled” without distortion. In addition, the outer curl is sampled much less densely than the inner curl. With an order of magnitude fewer points, higher noise levels, no possibility of an isometric mapping, and uneven sampling, this is arguably a much more challenging problem than the “ swiss roll” and “ s-curve” problems featured in [12, 9, 8, 1]. Figure 2LEFT contrasts the (unique) output of charting and the best outputs obtained from I SO M AP and LLE (considering all neighborhood sizes between 2 and 20 points). I SO M AP and LLE show catastrophic folding; we had to change LLE’s b. data, yz view c. local charts d. 2D embedding e. 1D embedding 1D ordinate a. data, xy view true manifold arc length Figure 3: Untying a trefoil knot ( ) by charting. 900 noisy samples from a 3 D-embedded 1 D manifold are shown as connected dots in front (a) and side (b) views. A subset of charts is shown in (c). Solving for the 2 D connection gives the “ unknot” in (d). After removing some points to cut the knot, charting gives a 1 D embedding which we plot against true manifold arc length in (e); monotonicity (modulo noise) indicates correctness. Three principal degrees of freedom recovered from raw jittered images pose scale expression images synthesized via backprojection of straight lines in coordinate space Figure 4: Modeling the manifold of facial images from raw video. Each row contains images synthesized by back-projecting an axis-parallel straight line in coordinate space onto the manifold in image space. Blurry images correspond to points on the manifold whose neighborhoods contain few if any nearby data points. regularization in order to coax out nondegenerate (>1 D) solutions. Although charting is not designed for isometry, after afﬁ transform the forward-projected points disagree with ne the original points with an RMS error of only 1.0429, lower than the best LLE (3.1423) or best I SO M AP (1.1424, not shown). Figure 2RIGHT shows the same problem where points are sampled regularly from a grid, with noise added before and after embedding. Figure 3 shows a similar treatment of a 1 D line that was threaded into a 3 D trefoil knot, contaminated with gaussian noise, and then “ untied” via charting. Video: We obtained a 1965-frame video sequence (courtesy S. Roweis and B. Frey) of 20 × 28-pixel images in which B.F. strikes a variety of poses and expressions. The video is heavily contaminated with synthetic camera jitters. We used raw images, though image processing could have removed this and other uninteresting sources of variation. We took a 500-frame subsequence and left-right mirrored it to obtain 1000 points in 20 × 28 = 560D image space. The point-growth process peaked just above d = 3 dimensions. We solved for 25 charts, each centered on a random point, and a 3D connection. The recovered degrees of freedom— recognizable as pose, scale, and expression— are visualized in ﬁ gure 4. original data stereographic map to 3D fishbowl charting Figure 5: Flattening a ﬁ shbowl. From the left: Original 2000×2D points; their stereographic mapping to a 3D ﬁ shbowl; its 2D embedding recovered using 500 charts; and the stereographic map. Fewer charts lead to isometric mappings that fold the bowl (not shown). Conformality: Some manifolds can be ﬂattened conformally (preserving local angles) but not isometrically. Figure 5 shows that if the data is ﬁ nely charted, the connection behaves more conformally than isometrically. This problem was suggested by J. Tenenbaum. 6 Discussion Charting breaks kernel-based NLDR into two subproblems: (1) Finding a set of datacovering locally linear neighborhoods (“ charts” ) such that adjoining neighborhoods span maximally similar subspaces, and (2) computing a minimal-distortion merger (“ connection” ) of all charts. The solution to (1) is optimal w.r.t. the estimated scale of local linearity r; the solution to (2) is optimal w.r.t. the solution to (1) and the desired dimensionality d. Both problems have Bayesian settings. By ofﬂoading the nonlinearity onto the kernels, we obtain least-squares problems and closed form solutions. This scheme is also attractive because large eigenproblems can be avoided by using a reduced set of charts. The dependence on r, like trusted-set methods, is a potential source of solution instability. In practice the point-growth estimate seems fairly robust to data perturbations (to be expected if the data density changes slowly over a manifold of integral Hausdorff dimension), while the use of a soft neighborhood partitioning appears to make charting solutions reasonably stable to variations in r. Eigenvalue stability analyses may prove useful here. Ultimately, we would prefer to integrate r out. In contrast, use of d appears to be a virtue: Unlike other eigenvector-based methods, the best d-dimensional embedding is not merely a linear projection of the best d + 1-dimensional embedding; a unique distortion is found for each value of d that maximizes the information content of its embedding. Why does charting performs well on datasets where the signal-to-noise ratio confounds recent state-of-the-art methods? Two reasons may be adduced: (1) Nonlocal information is used to construct both the system of local charts and their global connection. (2) The mapping only preserves the component of local point-to-point distances that project onto the manifold; relationships perpendicular to the manifold are discarded. Thus charting uses global shape information to suppress noise in the constraints that determine the mapping. Acknowledgments Thanks to J. Buhmann, S. Makar, S. Roweis, J. Tenenbaum, and anonymous reviewers for insightful comments and suggested “ challenge” problems. References [1] M. Balasubramanian and E. L. Schwartz. The IsoMap algorithm and topological stability. Science, 295(5552):7, January 2002. [2] C. Bregler and S. Omohundro. Nonlinear image interpolation using manifold learning. In NIPS–7, 1995. [3] D. DeMers and G. Cottrell. Nonlinear dimensionality reduction. In NIPS–5, 1993. [4] J. Gomes and A. Mojsilovic. A variational approach to recovering a manifold from sample points. In ECCV, 2002. [5] T. Hastie and W. Stuetzle. Principal curves. J. Am. Statistical Assoc, 84(406):502–516, 1989. [6] G. Hinton, P. Dayan, and M. Revow. Modeling the manifolds of handwritten digits. IEEE Trans. Neural Networks, 8, 1997. [7] N. Kambhatla and T. Leen. Dimensionality reduction by local principal component analysis. Neural Computation, 9, 1997. [8] S. Roweis, L. Saul, and G. Hinton. Global coordination of linear models. In NIPS–13, 2002. [9] S. T. Roweis and L. K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290:2323–2326, December 22 2000. [10] A. Smola, S. Mika, B. Schölkopf, and R. Williamson. Regularized principal manifolds. Machine Learning, 1999. [11] Y. W. Teh and S. T. Roweis. Automatic alignment of hidden representations. In NIPS–15, 2003. [12] J. B. Tenenbaum, V. de Silva, and J. C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290:2319–2323, December 22 2000.

6 0.57321233 5 nips-2002-A Digital Antennal Lobe for Pattern Equalization: Analysis and Design

7 0.57003641 51 nips-2002-Classifying Patterns of Visual Motion - a Neuromorphic Approach

8 0.50707597 11 nips-2002-A Model for Real-Time Computation in Generic Neural Microcircuits

9 0.49599993 102 nips-2002-Hidden Markov Model of Cortical Synaptic Plasticity: Derivation of the Learning Rule

10 0.48727593 43 nips-2002-Binary Coding in Auditory Cortex

11 0.47773662 50 nips-2002-Circuit Model of Short-Term Synaptic Dynamics

12 0.46201119 186 nips-2002-Spike Timing-Dependent Plasticity in the Address Domain

13 0.45568386 160 nips-2002-Optoelectronic Implementation of a FitzHugh-Nagumo Neural Model

14 0.45461833 148 nips-2002-Morton-Style Factorial Coding of Color in Primary Visual Cortex

15 0.45258456 128 nips-2002-Learning a Forward Model of a Reflex

16 0.44765222 199 nips-2002-Timing and Partial Observability in the Dopamine System

17 0.4434076 184 nips-2002-Spectro-Temporal Receptive Fields of Subthreshold Responses in Auditory Cortex

18 0.44261006 164 nips-2002-Prediction of Protein Topologies Using Generalized IOHMMs and RNNs

19 0.43946376 123 nips-2002-Learning Attractor Landscapes for Learning Motor Primitives

20 0.43715757 180 nips-2002-Selectivity and Metaplasticity in a Unified Calcium-Dependent Model