nips nips2001 nips2001-19 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Lance R. Williams, John W. Zweck
Abstract: We describe a neural network which enhances and completes salient closed contours. Our work is different from all previous work in three important ways. First, like the input provided to V1 by LGN, the input to our computation is isotropic. That is, the input is composed of spots not edges. Second, our network computes a well defined function of the input based on a distribution of closed contours characterized by a random process. Third, even though our computation is implemented in a discrete network, its output is invariant to continuous rotations and translations of the input pattern.
Reference: text
sentIndex sentText sentNum sentScore
1 of Maryland Baltimore County Baltimore, MD 21250 Abstract We describe a neural network which enhances and completes salient closed contours. [sent-7, score-0.458]
2 First, like the input provided to V1 by LGN, the input to our computation is isotropic. [sent-9, score-0.308]
3 That is, the input is composed of spots not edges. [sent-10, score-0.38]
4 Second, our network computes a well defined function of the input based on a distribution of closed contours characterized by a random process. [sent-11, score-0.434]
5 Third, even though our computation is implemented in a discrete network, its output is invariant to continuous rotations and translations of the input pattern. [sent-12, score-0.754]
6 1 Introduction There is a long history of research on neural networks inspired by the structure of visual cortex whose functions have been described as contour completion, saliency enhancement, orientation sharpening, or segmentation[6, 7, 8, 9, 12]. [sent-13, score-0.481]
7 A similiar network has been proposed as a model of visual hallucinations[1]. [sent-14, score-0.154]
8 In this paper, we describe a neural network which enhances and completes salient closed contours. [sent-15, score-0.458]
9 First, like the input provided to V1 by LGN, the input to our computation is isotropic. [sent-17, score-0.308]
10 That is, the input is composed of spots not edges. [sent-18, score-0.38]
11 Second, our network computes a well defined function of the input based on a distribution of closed contours characterized by a random process. [sent-19, score-0.434]
12 Third, even though our computation is implemented in a discrete network, its output is invariant to continuous rotations and translations of the input pattern. [sent-20, score-0.754]
13 There are two important properties which a computation must possess if it is to be invariant to rotations and translations, i. [sent-21, score-0.324]
14 First, the input, the output, and all intermediate representations must be Euclidean invariant. [sent-24, score-0.044]
15 Second, all transformations of these representations must also be Euclidean invariant. [sent-25, score-0.074]
16 The models described in [6, 7, 8, 9, 12] are not Euclidean invariant, first and foremost, because their input representations are not Euclidean invariant. [sent-26, score-0.116]
17 That is, not all rotations and translations of the input can be represented equally well. [sent-27, score-0.406]
18 This problem is often skirted by researchers by choosing input patterns which match particular choices of sampling rate and phase. [sent-28, score-0.116]
19 For example, Li [7] used only six samples in orientation (including ) and Heitger and von der Heydt[5] only twelve (including , and ). [sent-29, score-0.23]
20 Li’s first test pattern was a dashed line of orientation, , while Heitger and von der Heydt used a Kanizsa Triangle with sides of , , and ¡ ¢ ¡ £ ¤ ¡ © ¡ ¥ § ¨¦ ¡ ¥ ¤ ¡ £ ¡ ¥ ¡ ¥ orientation. [sent-30, score-0.146]
21 There is no reason to believe that the experimental results they showed would be similiar if the input patterns were rotated by as little as . [sent-31, score-0.286]
22 ¡ § ¨¦ 2 A continuum formulation of the saliency problem The following section reviews the continuum formulation of the contour completion and saliency problem as described in Williams and Thornber[11]. [sent-33, score-0.998]
23 1 Shape distribution Mumford[3] observed that the probability distribution of object boundary shapes could be modeled by a Fokker-Planck equation of the following form: 3 4¢ ¦ (1) B &9 E"4D5 2 § ¨ 0 ¡ 0 ¢ 0 ¡ 1) 7 '(&¥¡ %$" ¥¡ ©§¥¡ #! [sent-35, score-0.078]
24 ¨ ¨ ¦ ¤ ¢ £¡ ¢ £¡ ¢ £¡ B C¤ 9 7 5 686¢ where is the probability that a particle is located at position, , and is moving in direction, , at time, . [sent-36, score-0.044]
25 This partial differential equation can be viewed as a set of independent advection equations in and (the first and second terms) coupled in the dimension by the diffusion equation (the third term). [sent-37, score-0.218]
26 The advection equations translate probability mass in direction, , with unit speed, while the diffusion term models the Brownian motion in direction, with diffusion parameter, . [sent-38, score-0.323]
27 The combined effect of these three terms is that particles tend to travel in straight lines, but over time they drift to the left or right by an amount proportional to . [sent-39, score-0.131]
28 Finally, the effect of the fourth term is that particles decay over time, with a half-life given by the decay constant, . [sent-40, score-0.181]
29 2 The propagators B V XW¤ T 7S P H U9 RQI¤ F @ A T 9 7 G85 The Green’s function, , gives the probability that a particle observed at position, , and direction, , at time, , will later be observed at position, , and direction, , at time, . [sent-42, score-0.119]
30 1 The short-time propagator: (2) (3) S 7X5 B 685 9 7 g V f¦ e B T 7S ¥I9 dP 6X5 H c 9 7 gives the probability that and are from the boundary of a single object but are really the same edge. [sent-46, score-0.078]
31 The cut-off function is characterized by three parameters, , , and . [sent-48, score-0.046]
32 The parameter, , is the scale of the edge detection process. [sent-50, score-0.043]
33 d d 1 We assume that the probability that two edges are the same depends only on the distance between them, and that for particles travelling at unit speed. [sent-51, score-0.154]
34 3 Eigenfunctions B C5 3 The integral linear operator, , combines three sources of information: 1) the probability that two edges belong to the same object; 2) the probability that the two edges are distinct; and 3) the probability that the two edges exist. [sent-53, score-0.225]
35 4 Stochastic completion field B T 7S ¥I9 85 The magnitude of the stochastic completion field, , equals the probability that a closed contour satisfying a subset of the constraints exists at . [sent-56, score-0.837]
36 §¢ in this way is to remove the contribution, , The purpose of writing of closed contours at scales smaller than which would otherwise dominate the completion field. [sent-63, score-0.435]
37 Given the above expression for the completion field, it is clear that the key problem is computing the eigenfunction, , of with largest positive real eigenvalue. [sent-64, score-0.363]
38 To accomplish this, we can use the well known power method (see [4]). [sent-65, score-0.08]
39 In this case, the power method involves repeated application of the linear operator, , to the function, , followed by normalization: B3 y5 B3 y5 3 (13) B C5 3 B T 7S bU9 85 ! [sent-66, score-0.08]
40 ¦ B3 y5 ¦ In the limit, as gets very large, converges to the eigenfunction of , with largest positive real eigenvalue. [sent-70, score-0.187]
41 We observe that the above computation can be considered a continuous state, discrete time, recurrent neural network. [sent-71, score-0.192]
42 3 A discrete implementation of the continuum formulation The continuous functions comprising the state of the computation are represented as weighted sums of a finite set of shiftable-twistable basis functions. [sent-72, score-0.593]
43 The computation we describe is biologically plausible in the sense that all transformations of state are effected by linear transformations (or other vector parallel operations) on the coefficient vectors. [sent-74, score-0.224]
44 1 Shiftable-twistable bases The input and output of the above computation are functions defined on the continuous space, , of positions in the plane, , and directions in the circle, . [sent-76, score-0.433]
45 For such computations, the important symmetry is determined by those transformations, , of , which perform a shift in by , followed by a twist in through an angle, . [sent-77, score-0.239]
46 A twist through an angle, , consists of two parts: (1) a rotation, , of and (2) a translation in , both by . [sent-78, score-0.183]
47 Correspondingly, we define a shiftable-twistable basis 2 of functions on to be a set of functions on with the property that whenever a function, , is in their span, then so is , for every choice of in . [sent-82, score-0.299]
48 As such, the notion of a shiftable-twistable basis on generalizes that of a shiftablesteerable basis on [2, 10]. [sent-83, score-0.358]
49 H £ &0 ¡ H B V 9 V 85 7 H 7 B 685 9 7 ¡ &0 H © © ¦ ¨ ¥ 5 6 £ ¡ 0 B 685 6 9 7 £ ¡ 0 B 9 7 CB G85 £ ¡ 870 0 Shiftable-twistable bases can be constructed as follows. [sent-84, score-0.046]
50 Let be a function on which is periodic (with period ) in both spatial variables, . [sent-85, score-0.098]
51 In analogy with the , we say that is shiftable-twistable on definition of a shiftable-steerable function on if there are integers, and , and interpolation functions, , such that for each , the shift-twist of by is a linear combination of a finite number of basic shift-twists of by amounts , i. [sent-86, score-0.094]
52 ¦ F H £¡¢0 GB V 9 Q R¦ B 9 7 CB G85 £ A¡ 7 V X5 £ 0 © © ¦¨ PI¥ 5 9 Here is the basic shift amount and is the basic twist amount. [sent-92, score-0.375]
53 B Y `H 9 ¨ H 5 ¦ C XU 7H 7 C c ¦ & BU WV@ d ¦ d B Y H 9 ¨ H b a c & The Gaussian-Fourier basis is the product of a shiftable-steerable basis of Gaussians in and a Fourier series basis in . [sent-94, score-0.592]
54 For the experiments in this paper, the standard deviation of , equals the basic shift amount, . [sent-95, score-0.218]
55 We the Gaussian basis function, regard as a periodic function of period, , which is chosen to be much larger than , so that and its derivatives are essentially zero. [sent-96, score-0.227]
56 2 We use this terminology even though the basis functions need not be linearly independent. [sent-98, score-0.239]
57 2 Power method update formula can be represented in the Gaussian-Fourier basis as (17) '¦ ¦ B G85 9 7 s ¦ F 9 s ¦'F ¦ s ¦ F Q (! [sent-100, score-0.302]
58 , the basic step in the power method) can be implemented as a discrete linear transform in a Gaussian-Fourier shiftable-twistable basis: (! [sent-111, score-0.229]
59 3 The propagation operator P (19) ¢ In practice, we do not explicitly represent the matrix, . [sent-114, score-0.141]
60 Instead we compute the necessary matrix-vector product using the advection-diffusion-decay operator in the Gaussian-Fourier shiftable-twistable basis, , described in detail in Zweck and Williams[13]: D '¦F d s ¦ F ¦ ! [sent-115, score-0.196]
61 ¡ ¤¢ £ ¦ (V and where: (22) , is a discrete convolution: ' In the shiftable-twistable basis, the advection operator, (23) s f 1) 5 ' %$ 20¨ 4(&©B #"w f #! [sent-128, score-0.177]
62 ¨ where (25) In the continuum, the bias operator effects a multiplication of the function, , by the input bias function, . [sent-137, score-0.425]
63 Our aim is to identify an equivalent linear operator in the shiftabletwistable basis. [sent-138, score-0.141]
64 Suppose that both and are represented in a Gaussian basis, . [sent-139, score-0.051]
65 Their product is: B 7 bX5 ¦F d B 7 ¥X5 ¡ ¦ ¦ ¦F ! [sent-140, score-0.055]
66 d ¦ ¡ ¦ Q 9b87 5 ¦ F d 1¦ ¦ F Q ¦F ¦ ¦F B 8 B 3 b85 B 7 ¦ ¡ B 7 B 7 185 b85 ¦ ¡ Now, the product of two Gaussian basis functions, and ance which cannot be represented in the Gaussian basis, (26) , is a Gaussian of smaller vari. [sent-143, score-0.285]
67 Because is a linear combination of the products of pairs of Gaussian basis functions, it cannot be represented in the Gaussian basis either. [sent-144, score-0.409]
68 However, we observe that the convolution of and , where , can be represented in the a Gaussian, Gaussian basis. [sent-145, score-0.101]
69 D where are the interpolation functions, equals , and shift amount, to express in the Gaussian basis. [sent-148, score-0.228]
70 The standard deviation of the Gaussian was set equal to the shift amount, . [sent-150, score-0.126]
71 For illustration purposes, all functions were rendered at a resolution of . [sent-151, score-0.06]
72 The diffusion parameter, , equaled , and the decay constant, , equaled . [sent-152, score-0.352]
73 The time step, , used to solve the Fokker-Planck equation in the basis equaled . [sent-153, score-0.277]
74 § ¥ § ¥¦ 2 ¦ ¦ 3 B 3 ¦ @ ¦ 3 ) ¦ ¤ B 7 b85 d BU WV@ ¦ ¤ R§ d ¡ Y¦ ¡ § ¨¦ § 3 ¤ R§ ¦ U d In the first experiment, the input bias function, , consisted of twenty randomly positioned spots and twenty spots on the boundary of an avocado. [sent-155, score-1.084]
75 The stochastic completion field computed using 32 iterations of the power method is shown in Fig. [sent-161, score-0.384]
76 In the second experiment, the input bias function from the first experiment was rotated by and translated by half the distance between the centers of adjacent basis functions, . [sent-163, score-0.522]
77 The stochastic completion field is identical (up to rotation and translation) to the one computed in the first experiment. [sent-166, score-0.426]
78 The estimate of the largest positive real eigenvalue, , as a function of , the power method iteration is shown in Fig. [sent-170, score-0.201]
79 5 £ ¡ ¨ 5 Conclusion We described a neural network which enhances and completes salient closed contours. [sent-173, score-0.458]
80 Even though the computation is implemented in a discrete network, its output is invariant under continuous rotations and translations of the input pattern. [sent-174, score-0.754]
81 B 7 ¥X5 ¡ Figure 1: Left: The input bias function, . [sent-182, score-0.2]
82 Twenty randomly positioned spots were added to twenty spots on the boundary of an avocado. [sent-183, score-0.773]
83 Right: The stochastic completion field, , computed using basis functions. [sent-187, score-0.483]
84 § £ ¡ § § ¡ £ ¦ ¡ £ ¦ T i bU9 85 B T 7S £e Figure 2: Left: The input bias function from Fig. [sent-188, score-0.2]
85 1, rotated by and translated by half the distance between the centers of adjacent basis functions, . [sent-189, score-0.322]
86 Right: The stochastic completion field, is identical (up to rotation and translation) to the one shown in Fig. [sent-190, score-0.426]
87 02 0 5 10 15 20 25 30 35 & 0 ¨ Figure 3: The estimate of the largest positive real eigenvalue, , as a function of , the power method iteration. [sent-201, score-0.201]
88 Both the final value and all intermediate values are identical in the rotated and non-rotated cases. [sent-202, score-0.183]
wordName wordTfidf (topN-words)
[('spots', 0.264), ('completion', 0.242), ('saliency', 0.197), ('basis', 0.179), ('zweck', 0.151), ('operator', 0.141), ('contour', 0.14), ('williams', 0.129), ('eld', 0.128), ('shift', 0.126), ('rotations', 0.125), ('invariant', 0.123), ('euclidean', 0.12), ('salient', 0.119), ('dp', 0.119), ('input', 0.116), ('translations', 0.114), ('advection', 0.113), ('heitger', 0.113), ('heydt', 0.113), ('thornber', 0.113), ('twist', 0.113), ('twenty', 0.111), ('continuum', 0.111), ('diffusion', 0.105), ('closed', 0.101), ('equaled', 0.098), ('rotated', 0.095), ('contours', 0.092), ('bu', 0.09), ('enhances', 0.09), ('integers', 0.09), ('bias', 0.084), ('orientation', 0.084), ('eigenfunctions', 0.083), ('wv', 0.083), ('cb', 0.08), ('power', 0.08), ('network', 0.079), ('baltimore', 0.079), ('particles', 0.079), ('boundary', 0.078), ('rotation', 0.078), ('computation', 0.076), ('cdvq', 0.075), ('propagator', 0.075), ('propagators', 0.075), ('similiar', 0.075), ('edges', 0.075), ('von', 0.075), ('transformations', 0.074), ('formula', 0.072), ('der', 0.071), ('translation', 0.07), ('completes', 0.069), ('largest', 0.066), ('eigenfunction', 0.066), ('illusory', 0.066), ('discrete', 0.064), ('vision', 0.063), ('stochastic', 0.062), ('functions', 0.06), ('rp', 0.06), ('adelson', 0.06), ('lgn', 0.06), ('yv', 0.06), ('direction', 0.058), ('gaussian', 0.058), ('ee', 0.057), ('positioned', 0.056), ('sd', 0.056), ('real', 0.055), ('product', 0.055), ('mumford', 0.052), ('interpolation', 0.052), ('amount', 0.052), ('continuous', 0.052), ('decay', 0.051), ('represented', 0.051), ('li', 0.05), ('equals', 0.05), ('period', 0.05), ('convolution', 0.05), ('periodic', 0.048), ('translated', 0.048), ('xv', 0.048), ('characterized', 0.046), ('eigenvalue', 0.046), ('valued', 0.046), ('bases', 0.046), ('md', 0.046), ('identical', 0.044), ('fourier', 0.044), ('intermediate', 0.044), ('particle', 0.044), ('implemented', 0.043), ('detection', 0.043), ('basic', 0.042), ('positions', 0.042), ('output', 0.041)]
simIndex simValue paperId paperTitle
same-paper 1 0.99999946 19 nips-2001-A Rotation and Translation Invariant Discrete Saliency Network
Author: Lance R. Williams, John W. Zweck
Abstract: We describe a neural network which enhances and completes salient closed contours. Our work is different from all previous work in three important ways. First, like the input provided to V1 by LGN, the input to our computation is isotropic. That is, the input is composed of spots not edges. Second, our network computes a well defined function of the input based on a distribution of closed contours characterized by a random process. Third, even though our computation is implemented in a discrete network, its output is invariant to continuous rotations and translations of the input pattern.
2 0.097595379 75 nips-2001-Fast, Large-Scale Transformation-Invariant Clustering
Author: Brendan J. Frey, Nebojsa Jojic
Abstract: In previous work on “transformed mixtures of Gaussians” and “transformed hidden Markov models”, we showed how the EM algorithm in a discrete latent variable model can be used to jointly normalize data (e.g., center images, pitch-normalize spectrograms) and learn a mixture model of the normalized data. The only input to the algorithm is the data, a list of possible transformations, and the number of clusters to find. The main criticism of this work was that the exhaustive computation of the posterior probabilities over transformations would make scaling up to large feature vectors and large sets of transformations intractable. Here, we describe how a tremendous speed-up is acheived through the use of a variational technique for decoupling transformations, and a fast Fourier transform method for computing posterior probabilities. For N ×N images, learning C clusters under N rotations, N scales, N x-translations and N y-translations takes only (C + 2 log N )N 2 scalar operations per iteration. In contrast, the original algorithm takes CN 6 operations to account for these transformations. We give results on learning a 4-component mixture model from a video sequence with frames of size 320 ×240. The model accounts for 360 rotations and 76,800 translations. Each iteration of EM takes only 10 seconds per frame in MATLAB, which is over 5 million times faster than the original algorithm. 1
3 0.095412821 65 nips-2001-Effective Size of Receptive Fields of Inferior Temporal Visual Cortex Neurons in Natural Scenes
Author: Thomas P. Trappenberg, Edmund T. Rolls, Simon M. Stringer
Abstract: Inferior temporal cortex (IT) neurons have large receptive fields when a single effective object stimulus is shown against a blank background, but have much smaller receptive fields when the object is placed in a natural scene. Thus, translation invariant object recognition is reduced in natural scenes, and this may help object selection. We describe a model which accounts for this by competition within an attractor in which the neurons are tuned to different objects in the scene, and the fovea has a higher cortical magnification factor than the peripheral visual field. Furthermore, we show that top-down object bias can increase the receptive field size, facilitating object search in complex visual scenes, and providing a model of object-based attention. The model leads to the prediction that introduction of a second object into a scene with blank background will reduce the receptive field size to values that depend on the closeness of the second object to the target stimulus. We suggest that mechanisms of this type enable the output of IT to be primarily about one object, so that the areas that receive from IT can select the object as a potential target for action.
4 0.092351988 111 nips-2001-Learning Lateral Interactions for Feature Binding and Sensory Segmentation
Author: Heiko Wersing
Abstract: We present a new approach to the supervised learning of lateral interactions for the competitive layer model (CLM) dynamic feature binding architecture. The method is based on consistency conditions, which were recently shown to characterize the attractor states of this linear threshold recurrent network. For a given set of training examples the learning problem is formulated as a convex quadratic optimization problem in the lateral interaction weights. An efficient dimension reduction of the learning problem can be achieved by using a linear superposition of basis interactions. We show the successful application of the method to a medical image segmentation problem of fluorescence microscope cell images.
5 0.089159146 54 nips-2001-Contextual Modulation of Target Saliency
Author: Antonio Torralba
Abstract: The most popular algorithms for object detection require the use of exhaustive spatial and scale search procedures. In such approaches, an object is defined by means of local features. fu this paper we show that including contextual information in object detection procedures provides an efficient way of cutting down the need for exhaustive search. We present results with real images showing that the proposed scheme is able to accurately predict likely object classes, locations and sizes. 1
6 0.08464665 52 nips-2001-Computing Time Lower Bounds for Recurrent Sigmoidal Neural Networks
7 0.080349118 73 nips-2001-Eye movements and the maturation of cortical orientation selectivity
8 0.07890448 78 nips-2001-Fragment Completion in Humans and Machines
9 0.078608103 28 nips-2001-Adaptive Nearest Neighbor Classification Using Support Vector Machines
10 0.077036366 164 nips-2001-Sampling Techniques for Kernel Methods
11 0.073259726 153 nips-2001-Product Analysis: Learning to Model Observations as Products of Hidden Variables
12 0.071555302 163 nips-2001-Risk Sensitive Particle Filters
13 0.071490034 82 nips-2001-Generating velocity tuning by asymmetric recurrent connections
14 0.069122337 116 nips-2001-Linking Motor Learning to Function Approximation: Learning in an Unlearnable Force Field
15 0.066404827 129 nips-2001-Multiplicative Updates for Classification by Mixture Models
16 0.066366956 46 nips-2001-Categorization by Learning and Combining Object Parts
17 0.065743268 183 nips-2001-The Infinite Hidden Markov Model
18 0.065619498 38 nips-2001-Asymptotic Universality for Learning Curves of Support Vector Machines
19 0.064026207 80 nips-2001-Generalizable Relational Binding from Coarse-coded Distributed Representations
20 0.060544934 21 nips-2001-A Variational Approach to Learning Curves
topicId topicWeight
[(0, -0.213), (1, -0.076), (2, -0.031), (3, -0.037), (4, -0.053), (5, -0.005), (6, -0.108), (7, 0.055), (8, 0.055), (9, 0.01), (10, -0.024), (11, 0.056), (12, 0.11), (13, -0.046), (14, 0.045), (15, 0.09), (16, 0.02), (17, -0.052), (18, 0.018), (19, -0.004), (20, 0.032), (21, 0.031), (22, 0.104), (23, -0.005), (24, 0.011), (25, 0.046), (26, 0.006), (27, -0.017), (28, -0.057), (29, 0.129), (30, -0.028), (31, -0.011), (32, -0.017), (33, 0.097), (34, -0.093), (35, 0.051), (36, -0.015), (37, 0.0), (38, -0.122), (39, -0.096), (40, -0.078), (41, -0.13), (42, 0.056), (43, -0.099), (44, -0.058), (45, -0.067), (46, 0.065), (47, -0.096), (48, -0.072), (49, 0.019)]
simIndex simValue paperId paperTitle
same-paper 1 0.95405942 19 nips-2001-A Rotation and Translation Invariant Discrete Saliency Network
Author: Lance R. Williams, John W. Zweck
Abstract: We describe a neural network which enhances and completes salient closed contours. Our work is different from all previous work in three important ways. First, like the input provided to V1 by LGN, the input to our computation is isotropic. That is, the input is composed of spots not edges. Second, our network computes a well defined function of the input based on a distribution of closed contours characterized by a random process. Third, even though our computation is implemented in a discrete network, its output is invariant to continuous rotations and translations of the input pattern.
2 0.48366764 111 nips-2001-Learning Lateral Interactions for Feature Binding and Sensory Segmentation
Author: Heiko Wersing
Abstract: We present a new approach to the supervised learning of lateral interactions for the competitive layer model (CLM) dynamic feature binding architecture. The method is based on consistency conditions, which were recently shown to characterize the attractor states of this linear threshold recurrent network. For a given set of training examples the learning problem is formulated as a convex quadratic optimization problem in the lateral interaction weights. An efficient dimension reduction of the learning problem can be achieved by using a linear superposition of basis interactions. We show the successful application of the method to a medical image segmentation problem of fluorescence microscope cell images.
3 0.47489211 52 nips-2001-Computing Time Lower Bounds for Recurrent Sigmoidal Neural Networks
Author: M. Schmitt
Abstract: Recurrent neural networks of analog units are computers for realvalued functions. We study the time complexity of real computation in general recurrent neural networks. These have sigmoidal, linear, and product units of unlimited order as nodes and no restrictions on the weights. For networks operating in discrete time, we exhibit a family of functions with arbitrarily high complexity, and we derive almost tight bounds on the time required to compute these functions. Thus, evidence is given of the computational limitations that time-bounded analog recurrent neural networks are subject to. 1
4 0.46557581 151 nips-2001-Probabilistic principles in unsupervised learning of visual structure: human data and a model
Author: Shimon Edelman, Benjamin P. Hiles, Hwajin Yang, Nathan Intrator
Abstract: To find out how the representations of structured visual objects depend on the co-occurrence statistics of their constituents, we exposed subjects to a set of composite images with tight control exerted over (1) the conditional probabilities of the constituent fragments, and (2) the value of Barlow’s criterion of “suspicious coincidence” (the ratio of joint probability to the product of marginals). We then compared the part verification response times for various probe/target combinations before and after the exposure. For composite probes, the speedup was much larger for targets that contained pairs of fragments perfectly predictive of each other, compared to those that did not. This effect was modulated by the significance of their co-occurrence as estimated by Barlow’s criterion. For lone-fragment probes, the speedup in all conditions was generally lower than for composites. These results shed light on the brain’s strategies for unsupervised acquisition of structural information in vision. 1 Motivation How does the human visual system decide for which objects it should maintain distinct and persistent internal representations of the kind typically postulated by theories of object recognition? Consider, for example, the image shown in Figure 1, left. This image can be represented as a monolithic hieroglyph, a pair of Chinese characters (which we shall refer to as and ), a set of strokes, or, trivially, as a collection of pixels. Note that the second option is only available to a system previously exposed to various combinations of Chinese characters. Indeed, a principled decision whether to represent this image as , or otherwise can only be made on the basis of prior exposure to related images. £ ¡ £¦ ¡ £ ¥¨§¢ ¥¤¢ ¢ According to Barlow’s [1] insight, one useful principle is tallying suspicious coincidences: two candidate fragments and should be combined into a composite object if the probability of their joint appearance is much higher than , which is the probability expected in the case of their statistical independence. This criterion may be compared to the Minimum Description Length (MDL) principle, which has been previously discussed in the context of object representation [2, 3]. In a simplified form [4], MDL calls for representing explicitly as a whole if , just as the principle of suspicious coincidences does. £ ©¢ £ ¢ ¥¤¥ £¦ ¢ ¥ £ ¢ £¦ ¢ ¥¤¥! ¨§¥ £ ¢ £ ©¢ £¦ £ ¨§¢¥ ¡ ¢ While the Barlow/MDL criterion certainly indicates a suspicious coincidence, there are additional probabilistic considerations that may be used and . One example is the possiin setting the degree of association between ble perfect predictability of from and vice versa, as measured by . If , then and are perfectly predictive of each other and should really be coded by a single symbol, whereas the MDL criterion may suggest merely that some association between the representation of and that of be established. In comparison, if and are not perfectly predictive of each other ( ), there is a case to be made in favor of coding them separately to allow for a maximally expressive representation, whereas MDL may actually suggest a high degree of association ). In this study we investigated whether the human (if visual system uses a criterion based on alongside MDL while learning (in an unsupervised manner) to represent composite objects. £ £ £ ¢ ¥ ¥ © §¥ ¡ ¢ ¨¦¤
5 0.45601597 176 nips-2001-Stochastic Mixed-Signal VLSI Architecture for High-Dimensional Kernel Machines
Author: Roman Genov, Gert Cauwenberghs
Abstract: A mixed-signal paradigm is presented for high-resolution parallel innerproduct computation in very high dimensions, suitable for efficient implementation of kernels in image processing. At the core of the externally digital architecture is a high-density, low-power analog array performing binary-binary partial matrix-vector multiplication. Full digital resolution is maintained even with low-resolution analog-to-digital conversion, owing to random statistics in the analog summation of binary products. A random modulation scheme produces near-Bernoulli statistics even for highly correlated inputs. The approach is validated with real image data, and with experimental results from a CID/DRAM analog array prototype in 0.5 m CMOS. ¢
6 0.45161188 108 nips-2001-Learning Body Pose via Specialized Maps
7 0.45009765 82 nips-2001-Generating velocity tuning by asymmetric recurrent connections
8 0.449101 26 nips-2001-Active Portfolio-Management based on Error Correction Neural Networks
9 0.44891629 153 nips-2001-Product Analysis: Learning to Model Observations as Products of Hidden Variables
10 0.4263868 158 nips-2001-Receptive field structure of flow detectors for heading perception
11 0.42135969 75 nips-2001-Fast, Large-Scale Transformation-Invariant Clustering
12 0.40756878 65 nips-2001-Effective Size of Receptive Fields of Inferior Temporal Visual Cortex Neurons in Natural Scenes
13 0.40555793 76 nips-2001-Fast Parameter Estimation Using Green's Functions
14 0.395423 73 nips-2001-Eye movements and the maturation of cortical orientation selectivity
15 0.39220634 28 nips-2001-Adaptive Nearest Neighbor Classification Using Support Vector Machines
16 0.38426846 79 nips-2001-Gaussian Process Regression with Mismatched Models
17 0.38371262 84 nips-2001-Global Coordination of Local Linear Models
18 0.38286868 103 nips-2001-Kernel Feature Spaces and Nonlinear Blind Souce Separation
19 0.38147151 21 nips-2001-A Variational Approach to Learning Curves
20 0.37635919 106 nips-2001-Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering
topicId topicWeight
[(14, 0.025), (17, 0.031), (19, 0.028), (27, 0.114), (30, 0.084), (38, 0.488), (59, 0.023), (72, 0.032), (79, 0.026), (83, 0.023), (91, 0.068)]
simIndex simValue paperId paperTitle
1 0.94688302 80 nips-2001-Generalizable Relational Binding from Coarse-coded Distributed Representations
Author: Randall C. O'Reilly, R. S. Busby
Abstract: We present a model of binding of relationship information in a spatial domain (e.g., square above triangle) that uses low-order coarse-coded conjunctive representations instead of more popular temporal synchrony mechanisms. Supporters of temporal synchrony argue that conjunctive representations lack both efficiency (i.e., combinatorial numbers of units are required) and systematicity (i.e., the resulting representations are overly specific and thus do not support generalization to novel exemplars). To counter these claims, we show that our model: a) uses far fewer hidden units than the number of conjunctions represented, by using coarse-coded, distributed representations where each unit has a broad tuning curve through high-dimensional conjunction space, and b) is capable of considerable generalization to novel inputs.
same-paper 2 0.88910842 19 nips-2001-A Rotation and Translation Invariant Discrete Saliency Network
Author: Lance R. Williams, John W. Zweck
Abstract: We describe a neural network which enhances and completes salient closed contours. Our work is different from all previous work in three important ways. First, like the input provided to V1 by LGN, the input to our computation is isotropic. That is, the input is composed of spots not edges. Second, our network computes a well defined function of the input based on a distribution of closed contours characterized by a random process. Third, even though our computation is implemented in a discrete network, its output is invariant to continuous rotations and translations of the input pattern.
3 0.84066427 72 nips-2001-Exact differential equation population dynamics for integrate-and-fire neurons
Author: Julian Eggert, Berthold Bäuml
Abstract: Mesoscopical, mathematical descriptions of dynamics of populations of spiking neurons are getting increasingly important for the understanding of large-scale processes in the brain using simulations. In our previous work, integral equation formulations for population dynamics have been derived for a special type of spiking neurons. For Integrate- and- Fire type neurons , these formulations were only approximately correct. Here, we derive a mathematically compact, exact population dynamics formulation for Integrate- and- Fire type neurons. It can be shown quantitatively in simulations that the numerical correspondence with microscopically modeled neuronal populations is excellent. 1 Introduction and motivation The goal of the population dynamics approach is to model the time course of the collective activity of entire populations of functionally and dynamically similar neurons in a compact way, using a higher descriptionallevel than that of single neurons and spikes. The usual observable at the level of neuronal populations is the populationaveraged instantaneous firing rate A(t), with A(t)6.t being the number of neurons in the population that release a spike in an interval [t, t+6.t). Population dynamics are formulated in such a way, that they match quantitatively the time course of a given A(t), either gained experimentally or by microscopical, detailed simulation. At least three main reasons can be formulated which underline the importance of the population dynamics approach for computational neuroscience. First, it enables the simulation of extensive networks involving a massive number of neurons and connections, which is typically the case when dealing with biologically realistic functional models that go beyond the single neuron level. Second, it increases the analytical understanding of large-scale neuronal dynamics , opening the way towards better control and predictive capabilities when dealing with large networks. Third, it enables a systematic embedding of the numerous neuronal models operating at different descriptional scales into a generalized theoretic framework, explaining the relationships, dependencies and derivations of the respective models. Early efforts on population dynamics approaches date back as early as 1972, to the work of Wilson and Cowan [8] and Knight [4], which laid the basis for all current population-averaged graded-response models (see e.g. [6] for modeling work using these models). More recently, population-based approaches for spiking neurons were developed, mainly by Gerstner [3, 2] and Knight [5]. In our own previous work [1], we have developed a theoretical framework which enables to systematize and simulate a wide range of models for population-based dynamics. It was shown that the equations of the framework produce results that agree quantitatively well with detailed simulations using spiking neurons, so that they can be used for realistic simulations involving networks with large numbers of spiking neurons. Nevertheless, for neuronal populations composed of Integrate-and-Fire (I&F;) neurons, this framework was only correct in an approximation. In this paper, we derive the exact population dynamics formulation for I&F; neurons. This is achieved by reducing the I&F; population dynamics to a point process and by taking advantage of the particular properties of I&F; neurons. 2 2.1 Background: Integrate-and-Fire dynamics Differential form We start with the standard Integrate- and- Fire (I&F;) model in form of the wellknown differential equation [7] (1) which describes the dynamics of the membrane potential Vi of a neuron i that is modeled as a single compartment with RC circuit characteristics. The membrane relaxation time is in this case T = RC with R being the membrane resistance and C the membrane capacitance. The resting potential v R est is the stationary potential that is approached in the no-input case. The input arriving from other neurons is described in form of a current ji. In addition to eq. (1), which describes the integrate part of the I&F; model, the neuronal dynamics are completed by a nonlinear step. Every time the membrane potential Vi reaches a fixed threshold () from below, Vi is lowered by a fixed amount Ll > 0, and from the new value of the membrane potential integration according to eq. (1) starts again. if Vi(t) = () (from below) . (2) At the same time, it is said that the release of a spike occurred (i.e., the neuron fired), and the time ti = t of this singular event is stored. Here ti indicates the time of the most recent spike. Storing all the last firing times , we gain the sequence of spikes {t{} (spike ordering index j, neuronal index i). 2.2 Integral form Now we look at the single neuron in a neuronal compound. We assume that the input current contribution ji from presynaptic spiking neurons can be described using the presynaptic spike times tf, a response-function ~ and a connection weight W¡ . ',J ji(t) = Wi ,j ~(t - tf) (3) l: l: j f Integrating the I&F; equation (1) beginning at the last spiking time tT, which determines the initial condition by Vi(ti) = vi(ti - 0) - 6., where vi(ti - 0) is the membrane potential just before the neuron spikes, we get 1 Vi(t) = v Rest + fj(t - t:) + l: Wi ,j l: a(t - t:; t - tf) , j - Vi(t:)) e- S / T (4) f with the refractory function fj(s) = - (v Rest (5) and the alpha-function r ds
4 0.53267121 12 nips-2001-A Model of the Phonological Loop: Generalization and Binding
Author: Randall C. O'Reilly, R. Soto
Abstract: We present a neural network model that shows how the prefrontal cortex, interacting with the basal ganglia, can maintain a sequence of phonological information in activation-based working memory (i.e., the phonological loop). The primary function of this phonological loop may be to transiently encode arbitrary bindings of information necessary for tasks - the combinatorial expressive power of language enables very flexible binding of essentially arbitrary pieces of information. Our model takes advantage of the closed-class nature of phonemes, which allows different neural representations of all possible phonemes at each sequential position to be encoded. To make this work, we suggest that the basal ganglia provide a region-specific update signal that allocates phonemes to the appropriate sequential coding slot. To demonstrate that flexible, arbitrary binding of novel sequences can be supported by this mechanism, we show that the model can generalize to novel sequences after moderate amounts of training. 1
5 0.51961529 65 nips-2001-Effective Size of Receptive Fields of Inferior Temporal Visual Cortex Neurons in Natural Scenes
Author: Thomas P. Trappenberg, Edmund T. Rolls, Simon M. Stringer
Abstract: Inferior temporal cortex (IT) neurons have large receptive fields when a single effective object stimulus is shown against a blank background, but have much smaller receptive fields when the object is placed in a natural scene. Thus, translation invariant object recognition is reduced in natural scenes, and this may help object selection. We describe a model which accounts for this by competition within an attractor in which the neurons are tuned to different objects in the scene, and the fovea has a higher cortical magnification factor than the peripheral visual field. Furthermore, we show that top-down object bias can increase the receptive field size, facilitating object search in complex visual scenes, and providing a model of object-based attention. The model leads to the prediction that introduction of a second object into a scene with blank background will reduce the receptive field size to values that depend on the closeness of the second object to the target stimulus. We suggest that mechanisms of this type enable the output of IT to be primarily about one object, so that the areas that receive from IT can select the object as a potential target for action.
6 0.51721597 27 nips-2001-Activity Driven Adaptive Stochastic Resonance
7 0.48184797 29 nips-2001-Adaptive Sparseness Using Jeffreys Prior
8 0.47351813 57 nips-2001-Correlation Codes in Neuronal Populations
9 0.46458191 54 nips-2001-Contextual Modulation of Target Saliency
10 0.4381336 110 nips-2001-Learning Hierarchical Structures with Linear Relational Embedding
11 0.4363991 46 nips-2001-Categorization by Learning and Combining Object Parts
12 0.43444046 52 nips-2001-Computing Time Lower Bounds for Recurrent Sigmoidal Neural Networks
13 0.42779079 182 nips-2001-The Fidelity of Local Ordinal Encoding
14 0.41552082 131 nips-2001-Neural Implementation of Bayesian Inference in Population Codes
15 0.40834004 75 nips-2001-Fast, Large-Scale Transformation-Invariant Clustering
16 0.4069947 37 nips-2001-Associative memory in realistic neuronal networks
17 0.40665382 197 nips-2001-Why Neuronal Dynamics Should Control Synaptic Learning Rules
19 0.40546566 161 nips-2001-Reinforcement Learning with Long Short-Term Memory
20 0.4035643 141 nips-2001-Orientation-Selective aVLSI Spiking Neurons