nips nips2003 nips2003-10 knowledge-graph by maker-knowledge-mining

10 nips-2003-A Low-Power Analog VLSI Visual Collision Detector

Source: pdf

Author: Reid R. Harrison

Abstract: We have designed and tested a single-chip analog VLSI sensor that detects imminent collisions by measuring radially expansive optic flow. The design of the chip is based on a model proposed to explain leg-extension behavior in flies during landing approaches. A new elementary motion detector (EMD) circuit was developed to measure optic flow. This EMD circuit models the bandpass nature of large monopolar cells (LMCs) immediately postsynaptic to photoreceptors in the fly visual system. A 16 × 16 array of 2-D motion detectors was fabricated on a 2.24 mm × 2.24 mm die in a standard 0.5-µm CMOS process. The chip consumes 140 µW of power from a 5 V supply. With the addition of wide-angle optics, the sensor is able to detect collisions around 500 ms before impact in complex, real-world scenes. 1

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 edu Abstract We have designed and tested a single-chip analog VLSI sensor that detects imminent collisions by measuring radially expansive optic flow. [sent-4, score-0.324]

2 The design of the chip is based on a model proposed to explain leg-extension behavior in flies during landing approaches. [sent-5, score-0.387]

3 A new elementary motion detector (EMD) circuit was developed to measure optic flow. [sent-6, score-0.592]

4 This EMD circuit models the bandpass nature of large monopolar cells (LMCs) immediately postsynaptic to photoreceptors in the fly visual system. [sent-7, score-0.676]

5 A 16 × 16 array of 2-D motion detectors was fabricated on a 2. [sent-8, score-0.334]

6 The chip consumes 140 µW of power from a 5 V supply. [sent-12, score-0.185]

7 With the addition of wide-angle optics, the sensor is able to detect collisions around 500 ms before impact in complex, real-world scenes. [sent-13, score-0.138]

8 1 Introduction Many animals – from flies to humans – are capable of visually detecting imminent collisions caused either by a rapidly approaching object or self-motion towards an obstacle. [sent-14, score-0.181]

9 Borst and Bahde have shown that flies use visual information to time the extension of their legs on landing approaches [3]. [sent-16, score-0.199]

10 While several models have been proposed to explain collision detection, the model proposed in [3] is particularly amenable to hardware implementation. [sent-17, score-0.259]

11 1, employs a radially-oriented array of motion detectors centered in the direction of flight. [sent-19, score-0.368]

12 As the animal approaches a static object, an expansive optic flow field is produced on the retina. [sent-20, score-0.254]

13 A wide angle field of view is useful since optic flow in the direction of flight will be zero. [sent-21, score-0.196]

14 The response of this radial array of motion detectors is summed and then passed through a leaky integrator (a lowpass filter). [sent-22, score-0.653]

15 If this response exceeds a fixed threshold, an imminent collision is detected and the animal can take evasive action or prepare for a landing. [sent-23, score-0.331]

16 This expansive optic flow model has recently been used to explain landing and collision avoidance responses in the fruit fly [4]. [sent-24, score-0.694]

17 In this work, we present a single-chip analog VLSI sensor developed to implement this model. [sent-26, score-0.091]

18 Field of View radially-oriented D elementary motion detectors (EMDs) spatial summation leaky integrator comparator τ = RC threshold collision detect Figure 1: Diagram of collision detection algorithm. [sent-27, score-0.878]

19 2 E l emen tary Mo ti on Dete ctors Our collision detection algorithm uses an array of radially-oriented elementary motion detectors (EMDs) to sense image expansion. [sent-28, score-0.638]

20 We use an enhanced version of the familiar delay-and-correlate or “Reichardt” EMD first proposed by Hassenstein and Reichardt in the 1950s to explain the optomotor response of beetles [7]. [sent-30, score-0.096]

21 2 shows a diagram of the EMD used in our collision sensor. [sent-32, score-0.218]

22 Since we are interested in motion, it is also advantageous to amplify transient signals. [sent-35, score-0.073]

23 Suppressing dc illumination and enhancing ac components of photoreceptor signals is a common theme in many biological visual systems. [sent-36, score-0.268]

24 In flies, large monopolar cells (LMCs) directly postsynaptic to photoreceptors exhibit transient biphasic impulse responses approximately 40-200 ms in duration [8], [9]. [sent-37, score-0.319]

25 In the frequency domain, this can be seen as a bandpass filtering operation that attenuates dc signals while amplifying signals in the 2-40 Hz range [9], [10]. [sent-38, score-0.365]

26 In the lateral geniculate nucleus of cats, “lagged” and “non-lagged” cells exhibit transient biphasic impulse responses 200-300 ms in duration and act as bandpass filters amplifying signals in the 1-10 Hz range [11]. [sent-39, score-0.553]

27 This filtering has recently been explained in terms of temporal decorrelation, and can be seen as way of removing redundant information from the photoreceptor signal before further processing [9], [12]. [sent-40, score-0.071]

28 After this “transient enhancement”, or temporal decorrelation, the signals are delayed using the phase lag of a lowpass filter. [sent-41, score-0.142]

29 While not a true time delay, the lowpass filter matches data from animal experiments and makes the Reichardt EMD equivalent to the oriented spatiotemporal energy filter proposed by Adelson and Bergen [13]. [sent-42, score-0.239]

30 Before correlating the adjacent delayed and non-delayed signals, we apply a saturating static nonlinearity to each channel. [sent-43, score-0.123]

31 In fly tangential neurons, motion responses show a quadratic dependence only at very low contrasts, then quickly become largely independent of image contrast for contrasts above 30%. [sent-45, score-0.35]

32 Egelhaaf and Borst proposed the presence of this nonlinearity in the biological EMD to explain this contrast independence [14]. [sent-46, score-0.107]

33 3 vphoto-R vphoto-L LMC LMC temporal decorrelation (Large Monopolar Cells) vLMC-R vLMC-L delay Fig. [sent-49, score-0.136]

34 This reliability is improved by the addition of LMC bandpass filters and saturating nonlinearities. [sent-53, score-0.197]

35 Experiments using earlier versions of silicon EMDs have demonstrated the ability of delay-and-correlate motion detectors to work at very low signal-to-noise ratios [16]. [sent-54, score-0.245]

36 3 shows a schematic of the photoreceptor and LMC bandpass filter. [sent-58, score-0.211]

37 A 35 µm × 35 µm well-substrate photodiode with diode-connected pMOS load converts the diode photocurrent into a voltage vphoto that is a logarithmic function of light intensity. [sent-59, score-0.119]

38 A pMOS source follower biased by ISF = 700 pA buffers this signal so that the input capacitance of the LMC circuit does not load the photoreceptor. [sent-60, score-0.256]

39 The LMC bandpass filter consists of two operational transconductance amplifiers (OTAs) and three capacitors. [sent-61, score-0.233]

40 The OTAs in the circuit are implemented with pMOS differential pairs using diode-connected transistors for source degeneration for extended linear range (see inset, Fig. [sent-62, score-0.406]

41 τ 1 = βτ 0 Q= β (K + N ) (4) (5) The output signal v LMC is centered around VREF, a dc voltage which was set to 1. [sent-66, score-0.189]

42 We sized the capacitors in our circuit to give A = 20 and K = 5 (with C = 70 fF). [sent-68, score-0.256]

43 The transconductance of the lower OTA was set by adjusting its bias current IB: gm = I κ ⋅ B (κ + 1) 2U T (6) where κ is the weak inversion slope (typically between 0. [sent-69, score-0.121]

44 9) and UT is the thermal voltage kT/q (approximately 26 mV at room temperature). [sent-71, score-0.061]

45 As we see from (1), the LMC circuit acts as an ac-coupled bandpass filter centered at f1 = 1/2πτ1, with a quality factor Q set to 2. [sent-73, score-0.513]

46 The circuit also has a zero at βf 1, but since β = 25 in our circuit, the zero takes effect outside that passband and thus has little practical effect on the filter. [sent-75, score-0.256]

47 This LMC circuit represents a significant improvement over a previous silicon EMD design, which used only a first-order highpass filter to block dc illumination [16]. [sent-77, score-0.431]

48 The LMC circuit presented here allows the designer to adjust the center frequency and Q factor to selectively amplify frequencies present in moving images. [sent-78, score-0.256]

49 The LMC circuits from each photoreceptor pass their signals to the the delay-andcorrelate circuit shown in Fig. [sent-79, score-0.442]

50 The delay is implemented as a first-order lowpass filter. [sent-81, score-0.152]

51 The OTAs in this circuit used two diode-connected transistors in series for extended linear range. [sent-82, score-0.35]

52 The time constant of this filter is given by τ LPF = C LPF g m − LPF (7) vLMC-L vLMC-R vdelay-L gm-LPF CLPF CLPF Imult Imult VREF VW vdelay-R gm-LPF VREF VREF VW VW iout-L- VREF VW iout-Riout-R+ iout-L+ iout+ iout- Figure 4: Schematic of delay-and-correlate circuit. [sent-83, score-0.081]

53 We used CLPF = 700 fF and set τLPF to around 25 ms, which is in the range of biological motion detectors. [sent-86, score-0.171]

54 As the input signals grow larger, the tanh nonlinearity dominates and the circuit acts more like a digital exclusive-or gate. [sent-91, score-0.5]

55 We use this inherent circuit nonlinearity as the desired saturating nonlinearity in our EMD model (see Fig. [sent-92, score-0.445]

56 The previous LMC circuit provides sufficient gain to ensure that we are usually operating well outside the linear range of the multipliers. [sent-94, score-0.256]

57 Traditional CMOS Gilbert multipliers require that the dc level of the upper differential input be shifted relative to the dc level of the lower differential input. [sent-95, score-0.337]

58 This is required to keep the transistors in saturation. [sent-96, score-0.094]

59 To avoid the cost in chip area, power consumption, and mismatch associated with level shifters, we introduce a novel circuit modification that allows both the upper and lower differential inputs to operate at the same dc level. [sent-97, score-0.591]

60 We lower the well potential of the lower pMOS transistors from VDD to a dc voltage VW (see Fig. [sent-98, score-0.249]

61 This lowered well voltage causes the sources of these transistors to operate at a lower potential, which keeps the upper transistors in saturation. [sent-100, score-0.249]

62 (Care must be taken not to make VW too low, as parasitic source-well-substrate pnp transistors can be activated. [sent-103, score-0.094]

63 Ultra-wide-angle optics gave the chip a field of view ranging from ±52° to ±74°. [sent-105, score-0.317]

64 The signals from the left and right correlators are easily subtracted by summing their currents appropriately. [sent-107, score-0.123]

65 Similarly, current summation on two global wires is used to sum the motion signals over the entire EMD array. [sent-108, score-0.275]

66 4 Experimental Results We fabricated a 16 × 16 EMD array in a 0. [sent-109, score-0.089]

67 24 mm die contained a 17 × 17 array of “pixels,” each measuring 100 µm × 100 µm. [sent-113, score-0.134]

68 Each pixel contained a photoreceptor, LMC circuit, lowpass “delay” filter, and four correlators. [sent-114, score-0.113]

69 These correlators were used to implement two independent EMDs: a vertical motion detector connected to the pixel below and a horizontal motion detector connected to the pixel to the right. [sent-115, score-0.566]

70 The output signals from a subset of the EMDs representing radial outward motion were connected to two global wires, giving a differential current signal that was taken off chip on two pins. [sent-116, score-0.516]

71 5 shows the EMDs that were summed to produce the global radial motion signal. [sent-118, score-0.247]

72 The center 4 × 4 pixels were ignored, as motion near the center of the field of view is typically very small in collision situations. [sent-120, score-0.471]

73 We used custombuilt ultra-wide-angle optics to give the chip a field of view ranging from ±52° at the sides to ±74° at the corners. [sent-121, score-0.317]

74 Simulations revealed that a field of view of around ±60° was necessary for reasonable performance using this algorithm [6]. [sent-122, score-0.082]

75 Before testing the array, we characterized an individual LMC circuit configured to have a voltage input vphoto provided from off chip using a function generator. [sent-123, score-0.56]

76 4 Hz, 100 mVpp square wave and observed the LMC circuit output. [sent-125, score-0.256]

77 6a, the LMC circuit exhibits a transient oscillatory step response similar to its biological counterpart. [sent-127, score-0.384]

78 Using a spectrum analyzer, we measured the transfer function of the circuit (see Fig. [sent-128, score-0.256]

79 The LMC circuit acts as a bandpass filter centered at 19 Hz, with a measured Q of 2. [sent-130, score-0.513]

80 Most of this was consumed by peripheral biasing circuits; the 17 × 17 pixel array used only 5. [sent-136, score-0.125]

81 To test the complete collision detection chip, we implemented the leaky integrator (τleak = 50 ms) and comparator from Fig. [sent-138, score-0.368]

82 In future implementations, these circuits could be built on chip using little power. [sent-140, score-0.235]

83 We tested the chip by mounting it on a small motorized vehicle facing forward with the lens centered 11 cm above the floor. [sent-141, score-0.253]

84 7 shows the output from the leaky integrator as the chip moves across the floor and collides with the center of a 38 cm × 38 cm trash can in our lab. [sent-144, score-0.403]

85 The peak response of the chip occurs approximately 500 ms before contact, which corresponds to a distance of 14 cm. [sent-145, score-0.297]

86 After this point, the edges of the can move beyond the chip’s field of view, and the response decays rapidly. [sent-147, score-0.137]

87 The rebound in response observed in the last 100 ms may be due to the chip seeing the expanding shadow cast by its own lens on the side of the can just before contact. [sent-148, score-0.297]

88 While more complex models positing the measurement of true image velocity and object size have been used to explain this peculiar time course [1], we observe that a simple model integrating the output of a radial EMD array gives qualitatively similar responses. [sent-150, score-0.211]

89 We have demonstrated that this model of collision detection can be implemented in a small, low-power, single-chip sensor. [sent-151, score-0.257]

90 Further testing of the chip on mobile platforms should better characterize its performance. [sent-152, score-0.185]

91 Figure 7: Measured output of collision detection chip. [sent-161, score-0.257]

92 Frost, “Computation of different optical variables of looming objects in pigeon nucleus rotundus neurons,” Nature Neurosci. [sent-165, score-0.09]

93 Bahde, “Visual information processing in the fly’s landing system,” J. [sent-169, score-0.084]

94 Dickinson, “Collision-avoidance and landing responses are mediated by separate pathways in the fruit fly, Drosophila melanogaster,” J. [sent-177, score-0.128]

95 Harrison, “An algorithm for visual collision detection in real-world scenes,” submitted to NIPS 2003. [sent-190, score-0.295]

96 Laughlin, “Matching coding, circuits, cells, and molecules to signals – general principles of retinal design in the fly’s eye,” Progress in Ret. [sent-198, score-0.065]

97 van Hateren, “Theoretical predictions of spatiotemporal receptive fields of fly LMCs, and experimental validation,” J. [sent-202, score-0.135]

98 Humphrey, “Spatial and temporal response properties of lagged and nonlagged cells in cat lateral geniculate nucleus,” J. [sent-214, score-0.236]

99 Atick, “Temporal decorrelation: a theory of lagged and nonlagged responses in the lateral geniculate nucleus,” Network 6:159-178, 1995. [sent-221, score-0.181]

100 Koch, “A robust analog VLSI Reichardt motion sensor,” Analog Integrated Circuits and Signal Processing 24:213-229, 2000. [sent-252, score-0.227]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('lmc', 0.483), ('emd', 0.348), ('circuit', 0.256), ('collision', 0.218), ('chip', 0.185), ('emds', 0.174), ('motion', 0.171), ('fly', 0.135), ('reichardt', 0.135), ('vref', 0.116), ('bandpass', 0.102), ('vw', 0.1), ('ota', 0.097), ('transistors', 0.094), ('dc', 0.094), ('array', 0.089), ('landing', 0.084), ('field', 0.082), ('filter', 0.081), ('borst', 0.077), ('flies', 0.077), ('harrison', 0.077), ('lowpass', 0.077), ('delay', 0.075), ('detectors', 0.074), ('tanh', 0.073), ('transient', 0.073), ('photoreceptor', 0.071), ('optic', 0.071), ('gm', 0.071), ('cmos', 0.068), ('lpf', 0.067), ('nonlinearity', 0.066), ('signals', 0.065), ('voltage', 0.061), ('gilbert', 0.061), ('decorrelation', 0.061), ('pmos', 0.061), ('ref', 0.061), ('vlsi', 0.059), ('clpf', 0.058), ('correlators', 0.058), ('expansive', 0.058), ('imminent', 0.058), ('lagged', 0.058), ('lmcs', 0.058), ('monopolar', 0.058), ('otas', 0.058), ('vphoto', 0.058), ('ms', 0.057), ('integrator', 0.057), ('saturating', 0.057), ('analog', 0.056), ('differential', 0.056), ('response', 0.055), ('leaky', 0.054), ('nucleus', 0.051), ('optics', 0.05), ('transconductance', 0.05), ('hz', 0.05), ('circuits', 0.05), ('detector', 0.047), ('elementary', 0.047), ('collisions', 0.046), ('mm', 0.045), ('cells', 0.044), ('responses', 0.044), ('flow', 0.043), ('ib', 0.043), ('photoreceptors', 0.043), ('velocity', 0.042), ('explain', 0.041), ('geniculate', 0.04), ('acts', 0.04), ('detection', 0.039), ('radial', 0.039), ('amplifying', 0.039), ('bahde', 0.039), ('egelhaaf', 0.039), ('hassenstein', 0.039), ('hateren', 0.039), ('imult', 0.039), ('ioutl', 0.039), ('ioutr', 0.039), ('isf', 0.039), ('lake', 0.039), ('laughlin', 0.039), ('nonlagged', 0.039), ('pigeon', 0.039), ('trash', 0.039), ('wires', 0.039), ('filters', 0.038), ('schematic', 0.038), ('visual', 0.038), ('multipliers', 0.037), ('summed', 0.037), ('pixel', 0.036), ('sensor', 0.035), ('cm', 0.034), ('centered', 0.034)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 1.0000002 10 nips-2003-A Low-Power Analog VLSI Visual Collision Detector

Author: Reid R. Harrison

2 0.18889233 93 nips-2003-Information Dynamics and Emergent Computation in Recurrent Circuits of Spiking Neurons

Author: Thomas Natschläger, Wolfgang Maass

Abstract: We employ an efﬁcient method using Bayesian and linear classiﬁers for analyzing the dynamics of information in high-dimensional states of generic cortical microcircuit models. It is shown that such recurrent circuits of spiking neurons have an inherent capability to carry out rapid computations on complex spike patterns, merging information contained in the order of spike arrival with previously acquired context information. 1

3 0.18798892 18 nips-2003-A Summating, Exponentially-Decaying CMOS Synapse for Spiking Neural Systems

Author: Rock Z. Shi, Timothy K. Horiuchi

Abstract: Synapses are a critical element of biologically-realistic, spike-based neural computation, serving the role of communication, computation, and modiﬁcation. Many different circuit implementations of synapse function exist with different computational goals in mind. In this paper we describe a new CMOS synapse design that separately controls quiescent leak current, synaptic gain, and time-constant of decay. This circuit implements part of a commonly-used kinetic model of synaptic conductance. We show a theoretical analysis and experimental data for prototypes fabricated in a commercially-available 1.5µm CMOS process. 1

4 0.13137867 11 nips-2003-A Mixed-Signal VLSI for Real-Time Generation of Edge-Based Image Vectors

Author: Masakazu Yagi, Hideo Yamasaki, Tadashi Shibata

Abstract: A mixed-signal image filtering VLSI has been developed aiming at real-time generation of edge-based image vectors for robust image recognition. A four-stage asynchronous median detection architecture based on analog digital mixed-signal circuits has been introduced to determine the threshold value of edge detection, the key processing parameter in vector generation. As a result, a fully seamless pipeline processing from threshold detection to edge feature map generation has been established. A prototype chip was designed in a 0.35-µm double-polysilicon three-metal-layer CMOS technology and the concept was verified by the fabricated chip. The chip generates a 64-dimension feature vector from a 64x64-pixel gray scale image every 80µsec. This is about 104 times faster than the software computation, making a real-time image recognition system feasible. 1 In tro du c ti o n The development of human-like image recognition systems is a key issue in information technology. However, a number of algorithms developed for robust image recognition so far [1]-[3] are mostly implemented as software systems running on general-purpose computers. Since the algorithms are generally complex and include a lot of floating point operations, they are computationally too expensive to build real-time systems. Development of hardware-friendly algorithms and their direct VLSI implementation would be a promising solution for real-time response systems. Being inspired by the biological principle that edge information is firstly detected in the visual cortex, we have developed an edge-based image representation algorithm compatible to hardware processing. In this algorithm, multiple-direction edges extracted from an original gray scale image is utilized to form a feature vector. Since the spatial distribution of principal edges is represented by a vector, it was named Projected Principal-Edge Distribution (PPED) [4],[5], or formerly called Principal Axis Projection (PAP) [6],[7]. (The algorithm is explained later.) Since the PPED vectors very well represent the human perception of similarity among images, robust image recognition systems have been developed using PPED vectors in conjunction with the analog soft pattern classifier [4],[8], the digital VQ (Vector Quantization) processor [9], and support vector machines [10] . The robust nature of PPED representation is demonstrated in Fig. 1, where the system was applied to cephalometric landmark identification (identifying specific anatomical landmarks on medical radiographs) as an example, one of the most important clinical practices of expert dentists in orthodontics [6],[7]. Typical X-ray images to be experienced by apprentice doctors were converted to PPED vectors and utilized as templates for vector matching. The system performance has been proven for 250 head film samples regarding the fundamental 26 landmarks [11]. Important to note is the successful detection of the landmark on the soft tissue boundary (the tip of the lower lip) shown in Fig. 1(c). Landmarks on soft tissues are very difficult to detect as compared to landmarks on hard tissues (solid bones) because only faint images are captured on radiographs. The successful detection is due to the median algorithm that determines the threshold value for edge detection. Sella Nasion Orbitale By our system (a) By expert dentists Landmark on soft tissue (b) (c) Fig. 1: Image recognition using PPED vectors: (a,b) cephalometric landmark identification; (c) successful landmark detection on soft tissue. We have adopted the median value of spatial variance of luminance within the filtering kernel (5x5 pixels), which allows us to extract all essential features in a delicate gray scale image. However, the problem is the high computational cost in determining the median value. It takes about 0.6 sec to generate one PPED vector from a 64x64-pixel image (a standard image size for recognition in our system) on a SUN workstation, making real time processing unrealistic. About 90% of the computation time is for edge detection from an input image, in which most of the time is spent for median detection. Then the purpose of this work is to develop a new architecture median-filter VLSI subsystem for real-time PPED-vector generation. Special attention has been paid to realize a fully seamless pipeline processing from threshold detection to edge feature map generation by employing the four-stage asynchronous median detection architecture. 2 P r o je c t e d P r i n c i pa l E dg e Dis tribution (PPED ) Projected Principal Edge Distribution (PPED) algorithm [5],[6] is briefly explained using Fig. 2(a). A 5x5-pixel block taken from a 64x64-pixel target image is subjected to edge detection filtering in four principal directions, i.e. horizontal, vertical, and ±45-degree directions. In the figure, horizontal edge filtering is shown as an example. (The filtering kernels used for edge detection are given in Fig. 2(b).) In order to determine the threshold value for edge detection, all the absolute-value differences between two neighboring pixels are calculated in both vertical and horizontal directions and the median value is taken as the threshold. By scanning the 5x5-pixel filtering kernels in the target image, four 64x64 edge-flag maps are generated, which are called feature maps. In the horizontal feature map, for example, edge flags in every four rows are accumulated and spatial distribution of edge flags are represented by a histogram having 16 elements. Similar procedures are applied to other three directions to form respective histograms each having 16 elements. Finally, a 64-dimension vector is formed by series-connecting the four histograms in the order of horizontal, +45-degree, vertical, and –45-degree. Edge Detection 64x64 Feature Map (64x64) (Horizontal) 0 0 0 0 0 1 1 1 1 1 0 0 0 0 0 -1-1-1-1-1 0 0 0 0 0 (Horizontal) Threshold || Median Scan (16 elements) Edge Filter PPED Vector (Horizontal Section) 0 0 0 0 0 1 1 1 1 1 0 0 0 0 0 -1 -1 -1 -1 -1 0 0 0 0 0 0 0 0 1 0 0 1 1 0 -1 0 1 0 -1 0 1 0 -1 -1 0 0 -1 0 0 0 Horizontal +45-degree 0 0 0 0 0 Threshold Detection Absolute value difference between neiboring pels. 1 1 1 1 1 0 -1 0 -1 0 -1 0 -1 0 -1 0 0 0 0 0 0 -1 0 0 0 1 0 -1 -1 0 0 1 0 -1 0 0 1 1 0 -1 0 0 0 1 0 Vertical (a) -45-degree (b) Fig. 2: PPED algorithm (a) and filtering kernels for edge detection (b). 3 Sy stem Orga ni za ti o n The system organization of the feature map generation VLSI is illustrated in Fig. 3. The system receives one column of data (8-b x 5 pixels) at each clock and stores the data in the last column of the 5x6 image buffer. The image buffer shifts all the stored data to the right at every clock. Before the edge filtering circuit (EFC) starts detecting four direction edges with respect to the center pixel in the 5x5 block, the threshold value calculated from all the pixel data in the 5x5 block must be ready in time for the processing. In order to keep the coherence of the threshold detection and the edge filtering processing, the two last-in data locating at column 5 and 6 are given to median filter circuit (MFC) in advance via absolute value circuit (AVC). AVC calculates all luminance differences between two neighboring pixels in columns 5 and 6. In this manner, a fully seamless pipeline processing from threshold detection to edge feature map generation has been established. The key requirement here is that MFC must determine the median value of the 40 luminance difference data from the 5x5-pixel block fast enough to carry out the seamless pipeline processing. For this purpose, a four-stage asynchronous median detection architecture has been developed which is explained in the following. Edge Filtering Circuit (EFC) 6 5 4 3 2 1 Edge flags H +45 V Image buffer 8-b x 5 pixels (One column) Absolute Value Circuit (AVC) Threshold value Median Filter Circuit (MFC) -45 Feature maps Fig. 3: System organization of feature map generation VLSI. The well-known binary search algorithm was adopted for fast execution of median detection. The median search processing for five 4-b data is illustrated in Fig. 4 for the purpose of explanation. In the beginning, majority voting is carried out for the MSB’s of all data. Namely, the number of 1’s is compared with the number of 0’s and the majority group wins. The majority group flag (“0” in this example) is stored as the MSB of the median value. In addition, the loser group is withdrawn in the following voting by changing all remaining bits to the loser MSB (“1” in this example). By repeating the processing, the median value is finally stored in the median value register. Elapse of time Median Register : 0 1 X X 0 1 1 0 0 0 1 1 1 0 0 1 1 1 0 0 0 0 0 0 0 0 0 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 0 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 MVC0 MVC1 MVC2 MVC3 MVC0 MVC1 MVC2 MVC3 MVC0 MVC1 MVC2 MVC3 MVC0 MVC1 MVC2 MVC3 Majority Flag : 0 0 X X X Majority Voting Circuit (MVC) Fig. 4: Hardware algorithm for median detection by binary search. How the median value is detected from all the 40 8-b data (20 horizontal luminance difference data and 20 vertical luminance difference data) is illustrated in Fig. 5. All the data are stored in the array of median detection units (MDU’s). At each clock, the array receives four vertical luminance difference data and five horizontal luminance difference data calculated from the data in column 5 and 6 in Fig. 3. The entire data are shifted downward at each clock. The median search is carried out for the upper four bits and the lower four bits separately in order to enhance the throughput by pipelining. For this purpose, the chip is equipped with eight majority voting circuits (MVC 0~7). The upper four bits from all the data are processed by MVC 4~7 in a single clock cycle to yield the median value. In the next clock cycle, the loser information is transferred to the lower four bits within each MDU and MVC0~3 carry out the median search for the lower four bits from all the data in the array. Vertical Luminance Difference AVC AVC AVC AVC Horizontal Luminance Difference AVC AVC AVC AVC AVC Shift Shift Median Detection Unit (MDU) x (40 Units) Lower 4bit Upper 4bit MVC0 MVC2 MVC1 MVC3 MVC4 MVC5 MVC6 MVC7 MVCs for upper 4bit MVCs for lower 4bit Fig. 5: Median detection architecture for all 40 luminance difference data. The majority voting circuit (MVC) is shown in Fig. 6. Output connected CMOS inverters are employed as preamplifiers for majority detection which was first proposed in Ref. [12]. In the present implementation, however, two preamps receiving input data and inverted input data are connected to a 2-stage differential amplifier. Although this doubles the area penalty, the instability in the threshold for majority detection due to process and temperature variations has been remarkably improved as compared to the single inverter thresholding in Ref. [12]. The MVC in Fig. 6 has 41 input terminals although 40 bits of data are inputted to the circuit at one time. Bit “0” is always given to the terminal IN40 to yield “0” as the majority when there is a tie in the majority voting. PREAMP IN0 PREAMP 2W/L IN0 2W/L OUT W/L ENBL W/L W/L IN1 IN1 2W/L 2W/L W/L ENBL IN40 W/L W/L IN40 Fig. 6: Majority voting circuit (MVC). The edge filtering circuit (EFC) in Fig. 3 is composed as a four-stage pipeline of regular CMOS digital logic. In the first two stages, four-direction edge gradients are computed, and in the succeeding two stages, the detection of the largest gradient and the thresholding is carried out to generate four edge flags. 4 E x p e r i m e n t a l R es u l t s The feature map generation VLSI was fabricated in a 0.35-µm double-poly three-metal-layer CMOS technology. A photomicrograph of the proof-of-concept chip is shown in Fig. 7. The measured waveforms of the MVC at operating frequencies of 10MHz and 90MHz are demonstrated in Fig. 8. The input condition is in the worst case. Namely, 21 “1” bits and 20 “0” bits were fed to the inputs. The observed computation time is about 12 nsec which is larger than the simulation result of 2.5 nsec. This was caused by the capacitance loading due to the probing of the test circuit. In the real circuit without external probing, we confirmed the average computation time of 4~5 nsec. Edge-detection Filtering Circuit Processing Technology 0.35µm CMOS 2-Poly 3-Metal Median Filter Control Unit Chip Size 4.5mm x 4.5mm MVC Majority Voting Circuit X8 Supply Voltage 3.3 V Operation Frequengy 50MHz Vector Generator Fig. 7: Photomicrograph and specification of the fabricated proof-of-concept chip. 1V/div 5ns/div MVC_Output 1V/div 8ns/div MVC_OUT IN IN 1 Majority Voting operation (a) Majority Voting operation (b) Fig. 8: Measured waveforms of majority voting circuit (MVC) at operation frequencies of 10MHz (a) and 90 MHz (b) for the worst-case input data. The feature maps generated by the chip at the operation frequency of 25 MHz are demonstrated in Fig. 9. The power dissipation was 224 mW. The difference between the flag bits detected by the chip and those obtained by computer simulation are also shown in the figure. The number of error flags was from 80 to 120 out of 16,384 flags, only a 0.6% of the total. The occurrence of such error bits is anticipated since we employed analog circuits for median detection. However, such error does not cause any serious problems in the PPED algorithm as demonstrated in Figs. 10 and 11. The template matching results with the top five PPED vector candidates in Sella identification are demonstrated in Fig. 11, where Manhattan distance was adopted as the dissimilarity measure. The error in the feature map generation processing yields a constant bias to the dissimilarity and does not affect the result of the maximum likelihood search. Generated Feature maps Difference as compared to computer simulation Sella Horizontal Plus 45-degrees Vertical Minus 45-degrees Fig. 9: Feature maps for Sella pattern generated by the chip. Generated PPED vector by the chip Sella Difference as compared to computer simulation Dissimilarity (by Manhattan Distance) Fig. 10: PPED vector for Sella pattern generated by the chip. The difference in the vector components between the PPED vector generated by the chip and that obtained by computer simulation is also shown. 1200 Measured Data 1000 800 Computer Simulation 600 400 200 0 1st (Correct) 2nd 3rd 4th 5th Candidates in Sella recognition Fig. 11: Comparison of template matching results. 5 Conclusion A mixed-signal median filter VLSI circuit for PPED vector generation is presented. A four-stage asynchronous median detection architecture based on analog digital mixed-signal circuits has been introduced. As a result, a fully seamless pipeline processing from threshold detection to edge feature map generation has been established. A prototype chip was designed in a 0.35-µm CMOS technology and the fab- ricated chip generates an edge based image vector every 80 µsec, which is about 10 4 times faster than the software computation. Acknowledgments The VLSI chip in this study was fabricated in the chip fabrication program of VLSI Design and Education Center (VDEC), the University of Tokyo with the collaboration by Rohm Corporation and Toppan Printing Corporation. The work is partially supported by the Ministry of Education, Science, Sports, and Culture under Grant-in-Aid for Scientific Research (No. 14205043) and by JST in the program of CREST. References [1] C. Liu and Harry Wechsler, “Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition”, IEEE Transactions on Image Processing, Vol. 11, No.4, Apr. 2002. [2] C. Yen-ting, C. Kuo-sheng, and L. Ja-kuang, “Improving cephalogram analysis through feature subimage extraction”, IEEE Engineering in Medicine and Biology Magazine, Vol. 18, No. 1, 1999, pp. 25-31. [3] H. Potlapalli and R. C. Luo, “Fractal-based classification of natural textures”, IEEE Transactions on Industrial Electronics, Vol. 45, No. 1, Feb. 1998. [4] T. Yamasaki and T. Shibata, “Analog Soft-Pattern-Matching Classifier Using Floating-Gate MOS Technology,” Advances in Neural Information Processing Systems 14, Vol. II, pp. 1131-1138. [5] Masakazu Yagi, Tadashi Shibata, “An Image Representation Algorithm Compatible to Neural-Associative-Processor-Based Hardware Recognition Systems,” IEEE Trans. Neural Networks, Vol. 14, No. 5, pp. 1144-1161, September (2003). [6] M. Yagi, M. Adachi, and T. Shibata,

5 0.12892383 129 nips-2003-Minimising Contrastive Divergence in Noisy, Mixed-mode VLSI Neurons

Author: Hsin Chen, Patrice Fleury, Alan F. Murray

Abstract: This paper presents VLSI circuits with continuous-valued probabilistic behaviour realized by injecting noise into each computing unit(neuron). Interconnecting the noisy neurons forms a Continuous Restricted Boltzmann Machine (CRBM), which has shown promising performance in modelling and classifying noisy biomedical data. The Minimising-Contrastive-Divergence learning algorithm for CRBM is also implemented in mixed-mode VLSI, to adapt the noisy neurons’ parameters on-chip. 1

6 0.1129192 7 nips-2003-A Functional Architecture for Motion Pattern Processing in MSTd

7 0.11130781 183 nips-2003-Synchrony Detection by Analogue VLSI Neurons with Bimodal STDP Synapses

8 0.10602792 61 nips-2003-Entrainment of Silicon Central Pattern Generators for Legged Locomotory Control

9 0.10470028 16 nips-2003-A Recurrent Model of Orientation Maps with Simple and Complex Cells

10 0.10419355 28 nips-2003-Application of SVMs for Colour Classification and Collision Detection with AIBO Robots

11 0.092692673 140 nips-2003-Nonlinear Processing in LGN Neurons

12 0.08794374 13 nips-2003-A Neuromorphic Multi-chip Model of a Disparity Selective Complex Cell

13 0.075097866 37 nips-2003-Automatic Annotation of Everyday Movements

14 0.059254669 192 nips-2003-Using the Forest to See the Trees: A Graphical Model Relating Features, Objects, and Scenes

15 0.054753091 67 nips-2003-Eye Micro-movements Improve Stimulus Detection Beyond the Nyquist Limit in the Peripheral Retina

16 0.05454661 45 nips-2003-Circuit Optimization Predicts Dynamic Networks for Chemosensory Orientation in Nematode C. elegans

17 0.053300593 15 nips-2003-A Probabilistic Model of Auditory Space Representation in the Barn Owl

18 0.052158203 119 nips-2003-Local Phase Coherence and the Perception of Blur

19 0.051438645 106 nips-2003-Learning Non-Rigid 3D Shape from 2D Motion

20 0.048164975 133 nips-2003-Mutual Boosting for Contextual Inference

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, -0.127), (1, 0.071), (2, 0.311), (3, 0.034), (4, -0.018), (5, -0.085), (6, -0.086), (7, 0.016), (8, 0.0), (9, -0.028), (10, -0.073), (11, -0.007), (12, -0.093), (13, 0.077), (14, -0.062), (15, -0.076), (16, -0.018), (17, -0.03), (18, -0.045), (19, -0.095), (20, 0.135), (21, -0.135), (22, 0.183), (23, -0.046), (24, 0.124), (25, -0.079), (26, 0.05), (27, 0.045), (28, 0.148), (29, 0.006), (30, -0.052), (31, 0.091), (32, -0.115), (33, 0.18), (34, 0.113), (35, -0.005), (36, -0.063), (37, -0.043), (38, -0.012), (39, -0.054), (40, 0.045), (41, -0.028), (42, -0.111), (43, 0.094), (44, 0.145), (45, -0.024), (46, 0.011), (47, 0.007), (48, -0.085), (49, 0.043)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.97523105 10 nips-2003-A Low-Power Analog VLSI Visual Collision Detector

Author: Reid R. Harrison

2 0.66669989 11 nips-2003-A Mixed-Signal VLSI for Real-Time Generation of Edge-Based Image Vectors

Author: Masakazu Yagi, Hideo Yamasaki, Tadashi Shibata

3 0.59771764 18 nips-2003-A Summating, Exponentially-Decaying CMOS Synapse for Spiking Neural Systems

Author: Rock Z. Shi, Timothy K. Horiuchi

4 0.57904649 129 nips-2003-Minimising Contrastive Divergence in Noisy, Mixed-mode VLSI Neurons

Author: Hsin Chen, Patrice Fleury, Alan F. Murray

5 0.5685575 93 nips-2003-Information Dynamics and Emergent Computation in Recurrent Circuits of Spiking Neurons

Author: Thomas Natschläger, Wolfgang Maass

6 0.44727787 16 nips-2003-A Recurrent Model of Orientation Maps with Simple and Complex Cells

7 0.44393665 13 nips-2003-A Neuromorphic Multi-chip Model of a Disparity Selective Complex Cell

8 0.42107549 7 nips-2003-A Functional Architecture for Motion Pattern Processing in MSTd

9 0.3693046 61 nips-2003-Entrainment of Silicon Central Pattern Generators for Legged Locomotory Control

10 0.29347649 28 nips-2003-Application of SVMs for Colour Classification and Collision Detection with AIBO Robots

11 0.28886905 37 nips-2003-Automatic Annotation of Everyday Movements

12 0.28178969 67 nips-2003-Eye Micro-movements Improve Stimulus Detection Beyond the Nyquist Limit in the Peripheral Retina

13 0.27874538 69 nips-2003-Factorization with Uncertainty and Missing Data: Exploiting Temporal Coherence

14 0.25279239 183 nips-2003-Synchrony Detection by Analogue VLSI Neurons with Bimodal STDP Synapses

15 0.24754529 140 nips-2003-Nonlinear Processing in LGN Neurons

16 0.24192737 184 nips-2003-The Diffusion-Limited Biochemical Signal-Relay Channel

17 0.23139305 43 nips-2003-Bounded Invariance and the Formation of Place Fields

18 0.22103728 119 nips-2003-Local Phase Coherence and the Perception of Blur

19 0.21457249 15 nips-2003-A Probabilistic Model of Auditory Space Representation in the Barn Owl

20 0.20033979 182 nips-2003-Subject-Independent Magnetoencephalographic Source Localization by a Multilayer Perceptron

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(0, 0.021), (11, 0.05), (30, 0.028), (35, 0.027), (53, 0.075), (56, 0.383), (59, 0.068), (63, 0.031), (71, 0.032), (76, 0.017), (82, 0.018), (85, 0.069), (91, 0.075)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.85195971 10 nips-2003-A Low-Power Analog VLSI Visual Collision Detector

Author: Reid R. Harrison

2 0.52101308 66 nips-2003-Extreme Components Analysis

Author: Max Welling, Christopher Williams, Felix V. Agakov

Abstract: Principal components analysis (PCA) is one of the most widely used techniques in machine learning and data mining. Minor components analysis (MCA) is less well known, but can also play an important role in the presence of constraints on the data distribution. In this paper we present a probabilistic model for “extreme components analysis” (XCA) which at the maximum likelihood solution extracts an optimal combination of principal and minor components. For a given number of components, the log-likelihood of the XCA model is guaranteed to be larger or equal than that of the probabilistic models for PCA and MCA. We describe an efﬁcient algorithm to solve for the globally optimal solution. For log-convex spectra we prove that the solution consists of principal components only, while for log-concave spectra the solution consists of minor components. In general, the solution admits a combination of both. In experiments we explore the properties of XCA on some synthetic and real-world datasets.

3 0.35247838 18 nips-2003-A Summating, Exponentially-Decaying CMOS Synapse for Spiking Neural Systems

Author: Rock Z. Shi, Timothy K. Horiuchi

4 0.3406522 185 nips-2003-The Doubly Balanced Network of Spiking Neurons: A Memory Model with High Capacity

Author: Yuval Aviel, David Horn, Moshe Abeles

Abstract: A balanced network leads to contradictory constraints on memory models, as exemplified in previous work on accommodation of synfire chains. Here we show that these constraints can be overcome by introducing a 'shadow' inhibitory pattern for each excitatory pattern of the model. This is interpreted as a doublebalance principle, whereby there exists both global balance between average excitatory and inhibitory currents and local balance between the currents carrying coherent activity at any given time frame. This principle can be applied to networks with Hebbian cell assemblies, leading to a high capacity of the associative memory. The number of possible patterns is limited by a combinatorial constraint that turns out to be P=0.06N within the specific model that we employ. This limit is reached by the Hebbian cell assembly network. To the best of our knowledge this is the first time that such high memory capacities are demonstrated in the asynchronous state of models of spiking neurons. 1 In trod u ction Numerous studies analyze the different phases of unstructured networks of spiking neurons [1, 2]. These networks with random connectivity possess a phase of asynchronous activity, the asynchronous state (AS), which is the most interesting one from the biological perspective, since it is similar to physiological data. Unstructured networks, however, do not hold information in their connectivity matrix, and therefore do not store memories. Binary networks with ordered connectivity matrices, or structured networks, and their ability to store and retrieve memories, have been extensively studied in the past [3-8]. Applicability of these results to biologically plausible neuronal models is questionable. In particular, models of spiking neurons are known to have modes of synchronous global oscillations. Avoiding such modes, and staying in an AS, is a major constraint on networks of spiking neurons that is absent in most binary neural networks. As we will show below, it is this constraint that imposes a limit on capacity in our model. Existing associative memory models of spiking neurons have not strived for maximal pattern capacity [3, 4, 8]. Here, using an integrate-and-fire model, we embed structured synaptic connections in an otherwise unstructured network and study the capacity limit of the system. The system is therefore macroscopically unstructured, but microscopically structured. The unstructured network model is based on Brunel's [1] balanced network of integrate-and-fire neurons. In his model, the network possesses different phases, one of which is the AS. We replace his unstructured excitatory connectivity by a semistructured one, including a super-position of either synfire chains or Hebbian cell assemblies. The existence of a stable AS is a fundamental prerequisite of the system. There are two reasons for that: First, physiological measurements of cortical tissues reveal an irregular neuronal activity and an asynchronous population activity. These findings match the properties of the AS. Second, in term of information content, the entropy of the system is the highest when firing probability is uniformly distributed, as in an AS. In general, embedding one or two patterns will not destabilize the AS. Increasing the number of embedded patterns, however, will eventually destabilize the AS, leading to global oscillations. In previous work [9], we have demonstrated that the cause of AS instability is correlations between neurons that result from the presence of structure in the network. The patterns, be it Hebbian cell assemblies (HCA) or pools occurring in synfire chains (SFC), have an important characteristic: neurons that are members of the same pattern (or pool) share a large portion of their inputs. This common input correlates neuronal activities both when a pattern is activated and when both neurons are influenced by random activity. If too many patterns are embedded in the network, too many neurons become correlated due to common inputs, leading to globally synchronized deviations from mean activity. A qualitative understanding of this state of affairs is provided by a simple model of a threshold linear pair of neurons that receive n excitatory common, and correlated, inputs, and K-n excitatory, as well as K inhibitory, non-common uncorrelated inputs. Thinking of these neurons as belonging to a pattern or a pool within a network, we can obtain an interesting self-consistent result by assuming the correlation of the pair of neurons to be also the correlation in their common correlated input (as is likely to be the case in a network loaded with HCA or SFC). We find then [9] that there exists a critical pattern size, n c , below which correlations decay but above which correlations are amplified. Furthermore, the following scaling was found to exist (1) nc = rc K . Implications of this model for the whole network are that: (i) rc is independent of N, the size of the network, (ii) below nc the AS is stable, and (iii) above nc the AS is unstable. Using extensive computer simulations we were able [9] to validate all these predictions. In addition, keeping n nmin, by the requirement that n excitatory post-synaptic potentials (PSPs), on average, drive a neuron across its threshold. Since N>K and typically N>>K, together with Eq. (1) it follows that N >> (n min / rc ) . Hence rc and nmin set the lower bound of the network's size, 2 above which it is possible to embed a reasonable number of patterns in the network without losing the AS. In this paper we propose a solution that enables small n min and large r values, which in turn enables embedding a large number of patterns in much smaller networks. This is made possible by the doubly-balanced construction to be outlined below. 2 The double-balance principle Counteracting the excitatory correlations with inhibitory ones is the principle that will allow us to solve the problem. Since we deal with balanced networks, in which the mean excitatory input is balanced by an inhibitory one, we note that this principle imposes a second type of balancing condition, hence we refer to it as the double- balance principle. In the following, we apply this principle by introducing synaptic connections between any excitatory pattern and its randomly chosen inhibitory pattern. These inhibitory patterns, which we call shadow patterns, are activated after the excitatory patterns fire, but have no special in-pattern connectivity or structured projections onto other patterns. The premise is that correlations evolved in the excitatory patterns will elicit correlated inhibitory activity, thus balancing the network's average correlation level. The size of the shadow pattern has to be small enough, so that the global network activity will not be quenched, yet large enough, so that the excitatory correlation will be counteracted. A balanced network that is embedded with patterns and their shadow patterns will be referred to as a doubly balanced network (DBN), to be contrasted with the singly balanced network (SBN) where shadow patterns are absent. 3 3.1 Application of the double balance principle. The Network We model neuronal activity with the Integrate and Fire [10] model. All neurons have the same parameters: τ = 10ms , τ ref = 2.5ms , C=250pF. PSPs are modeled by a delta function with fixed delay. The number of synapses on a neuron is fixed and set to KE excitatory synapses from the local network, KE excitatory synapses from external sources and KI inhibitory synapses from the local network. See Aviel et al [9] for details. All synapses of each group will be given fixed values. It is allowed for one pre-synaptic neuron to make more than one connection to one postsynaptic neuron. The network possesses NE excitatory neurons and N I ≡ γN E inhibitory neurons. Connectivity is sparse, ε = 0.1 ). K E N E = K I N I = ε , (we use A Poisson process with rate vext=10Hz models the external source. If a neuron of population y innervates a neuron of population x its synaptic strength J xy is defined as J xE ≡ J 0 K E , J xI ≡ − gJ 0 with J0=10, and g=5. Note that J xI = − g γ KI J xE , hence g γ controls the balance between the two populations. Within an HCA pattern the neurons have high connection probability with one another. Here it is achieved by requiring L of the synapses of a neuron in the excitatory pattern to originate from within the pattern. Similarly, a neuron in the inhibitory shadow pattern dedicates L of its synapses to the associated excitatory pattern. In a SFC, each neuron in an excitatory pool is fed by L neurons from the previous pool. This forms a feed forward connectivity. In addition, when shadow pools are present, each neuron in a shadow pool is fed by L neurons from its associated excitatory pool. In both cases L = C L K E , with CL=2.5. The size of the excitatory patterns (i.e. the number of neurons participating in a pattern) or pools, nE, is also chosen to be proportional to K E (see Aviel et al. 2003 [9]), nE ≡ Cn K E , where Cn varies. This is a suitable choice, because of the behavior of the critical nc of Eq. (1), and is needed for the meaningful memory activity (of the HCA or SFC) to overcome synaptic noise. ~ The size of a shadow pattern is defined as nI ≡ d nE . This leads to the factor d, representing the relative strength of inhibitory and excitatory currents, due to a pattern or pool, affecting a neuron that is connected to both: d≡ (2) Thus it fixes nI = d − J xI nI J xE nE = gJ 0 K E d gd . = J0 K I γ ( )n . In the simulations reported below d varied between 1 γ g E and 3. Wiring the network is done in two stages, first all excitatory patterns are wired, and then random connections are added, complying with the fixed number of synapses. A volley of w spikes, normally distributed over time with width of 1ms, is used to ignite a memory pattern. In the case of SFC, the first pool is ignited, and under the right conditions the volley propagates along the chain without fading away and without destabilizing the AS. 3.2 Results First we show that the AS remains stable when embedding HCAs in a small DBN, whereas global oscillations take place if embedding is done without shadow pools. Figure 1 displays clearly the sustained activity of an HCA in the DBN. The same principle also enables embedding of SFCs in a small network. This is to be contrasted with the conclusions drawn in Aviel et al [9], where it was shown that otherwise very large networks are necessary to reach this goal. Figure 1: HCAs are embedded in a balanced network without (left) and with (right) shadow patterns. P=300 HCAs of size nE=194 excitatory neurons were embedded in a network of NE=15,000 excitatory neurons. The eleventh pattern is externally ignited at time t=100ms. A raster plot of 200ms is displayed. Without shadow patterns the network exhibits global oscillations, but with shadow patterns the network exhibits only minute oscillations, enabling the activity of the ignited pattern to be sustained. The size of the shadow patterns is set according to Eq. (2) with d=1. Neurons that participate in more than one HCA may appear more than once on the raster plot, whose y-axis is ordered according to HCAs, and represents every second neuron in each pattern. Figure 2: SFCs embedded in a balanced network without (left) and with (right) shadow patterns. The first pool is externally ignited at time t=100ms. d=0.5. The rest of the parameters are as in Figure 1. Here again, without shadow pools, the network exhibits global oscillations, but with shadow pools it has only minute oscillation, enabling a stable propagation of the synfire wave. 3.3 Maximum Capacity In this section we show that, within our DBN, it is the fixed number of synapses (rather than dynamical constraints) that dictates the maximal number of patterns or pools P that may be loaded onto the network. Let us start by noting that a neuron of population x (E or I) can participate in at most m ≡ K E L  patterns, hence N x m sets an upper bound on the number of neurons that participate in all patterns: P n x P ≤ m ⋅ N x . Next, defining α x ≡ , we find that Nx αx ≤ (3) m nx =  K E CL K E    nx To leading order in NE this turns into  K E CL K E   N = C C D −1 N − O αxNx =  n L x E E D C K (4) x n E ( ) ( NE ) (g γ ) if x=I, or 1 for x=E. where Dx ≡ d Thus we conclude that synaptic combinatorial considerations lead to a maximal number of patterns P. If DI<1, including the case DI=0 of the SBN, the excitatory neurons determine the limit to be P = (C n C L ) N E . If, as is the case in our DBN, −1 DI>1, then ( γα I < α E P = C n C L DI ) −1 and the inhibitory neurons set the maximum value to NE . For example, setting Cn=3.5, CL=2.4, g=3 and d=3, in Eq. (4), we get P=0.06NE. In Figure 3 we use these parameters. The capacity of a DBN is compared to that of an SBN for different network sizes. The maximal load is defined by the presence of global oscillation strong enough to prohibit sustained activity of patterns. The DBN reaches the combinatorial limit, whereas the SBN does not increase with N and obviously does not reach its combinatorial limit. 1400 1200 Pmax 1000 DBN SBN DBN Upper Limit SBN Upper Limit 800 600 400 200 0 0 5000 10000 NE 15000 Figure 3: A balanced network maximally loaded with HCAs. Left: A raster plot of a maximally loaded DBN. P=408, NE=6,000. At time t=450ms, the seventh pattern is ignited for a duration of 10ms, leading to termination of another pattern's activity (upper stripe) and to sustained activity of the ignited pattern (lower stripe). Right: P(NE) as inferred from simulations of a SBN (

5 0.33886197 169 nips-2003-Sample Propagation

Author: Mark A. Paskin

Abstract: Rao–Blackwellization is an approximation technique for probabilistic inference that ﬂexibly combines exact inference with sampling. It is useful in models where conditioning on some of the variables leaves a simpler inference problem that can be solved tractably. This paper presents Sample Propagation, an efﬁcient implementation of Rao–Blackwellized approximate inference for a large class of models. Sample Propagation tightly integrates sampling with message passing in a junction tree, and is named for its simple, appealing structure: it walks the clusters of a junction tree, sampling some of the current cluster’s variables and then passing a message to one of its neighbors. We discuss the application of Sample Propagation to conditional Gaussian inference problems such as switching linear dynamical systems. 1

6 0.32889152 139 nips-2003-Nonlinear Filtering of Electron Micrographs by Means of Support Vector Regression

7 0.32772642 20 nips-2003-All learning is Local: Multi-agent Learning in Global Reward Games

8 0.32757252 93 nips-2003-Information Dynamics and Emergent Computation in Recurrent Circuits of Spiking Neurons

9 0.32535386 101 nips-2003-Large Margin Classifiers: Convex Loss, Low Noise, and Convergence Rates

10 0.32291558 125 nips-2003-Maximum Likelihood Estimation of a Stochastic Integrate-and-Fire Neural Model

11 0.32222995 183 nips-2003-Synchrony Detection by Analogue VLSI Neurons with Bimodal STDP Synapses

12 0.3214795 57 nips-2003-Dynamical Modeling with Kernels for Nonlinear Time Series Prediction

13 0.32098711 54 nips-2003-Discriminative Fields for Modeling Spatial Dependencies in Natural Images

14 0.3205817 173 nips-2003-Semi-supervised Protein Classification Using Cluster Kernels

15 0.32004085 177 nips-2003-Simplicial Mixtures of Markov Chains: Distributed Modelling of Dynamic User Profiles

16 0.31984204 113 nips-2003-Learning with Local and Global Consistency

17 0.31855941 28 nips-2003-Application of SVMs for Colour Classification and Collision Detection with AIBO Robots

18 0.31780353 109 nips-2003-Learning a Rare Event Detection Cascade by Direct Feature Selection

19 0.31757966 4 nips-2003-A Biologically Plausible Algorithm for Reinforcement-shaped Representational Learning

20 0.31744659 3 nips-2003-AUC Optimization vs. Error Rate Minimization