nips nips2008 nips2008-232 knowledge-graph by maker-knowledge-mining

232 nips-2008-The Conjoint Effect of Divisive Normalization and Orientation Selectivity on Redundancy Reduction

Source: pdf

Author: Fabian H. Sinz, Matthias Bethge

Abstract: Bandpass ﬁltering, orientation selectivity, and contrast gain control are prominent features of sensory coding at the level of V1 simple cells. While the effect of bandpass ﬁltering and orientation selectivity can be assessed within a linear model, contrast gain control is an inherently nonlinear computation. Here we employ the class of Lp elliptically contoured distributions to investigate the extent to which the two features—orientation selectivity and contrast gain control—are suited to model the statistics of natural images. Within this framework we ﬁnd that contrast gain control can play a signiﬁcant role for the removal of redundancies in natural images. Orientation selectivity, in contrast, has only a very limited potential for redundancy reduction. 1

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 de Abstract Bandpass ﬁltering, orientation selectivity, and contrast gain control are prominent features of sensory coding at the level of V1 simple cells. [sent-5, score-0.703]

2 While the effect of bandpass ﬁltering and orientation selectivity can be assessed within a linear model, contrast gain control is an inherently nonlinear computation. [sent-6, score-0.856]

3 Here we employ the class of Lp elliptically contoured distributions to investigate the extent to which the two features—orientation selectivity and contrast gain control—are suited to model the statistics of natural images. [sent-7, score-0.684]

4 Within this framework we ﬁnd that contrast gain control can play a signiﬁcant role for the removal of redundancies in natural images. [sent-8, score-0.562]

5 Orientation selectivity, in contrast, has only a very limited potential for redundancy reduction. [sent-9, score-0.299]

6 Motivated by information theoretic principles, Attneave and Barlow suggested that one important purpose of this adaptation in sensory coding is to model and reduce the redundancies [4; 3] by transforming the signal into a statistically independent representation. [sent-12, score-0.164]

7 The problem of redundancy reduction can be split into two parts: (i) ﬁnding a good statistical model of the natural signals and (ii) a way to map them into a factorial representation. [sent-13, score-0.674]

8 The second part offers a way to link neural response properties to computational principles, since neural representations of natural signals must be advantageous in terms of redundancy reduction if the hypothesis were true. [sent-15, score-0.488]

9 Both aspects have been extensively studied for natural images [2; 5; 8; 19; 20; 21; 24]. [sent-16, score-0.144]

10 In particular, it has been shown that applying Independent Component Analysis (ICA) to natural images consistently and robustly yields ﬁlters that are localized, oriented and show bandpass characteristics [19; 5]. [sent-17, score-0.214]

11 Since those features are also ascribed to the receptive ﬁelds of neurons in the primary visual cortex (V1), it has been suggested that the receptive ﬁelds of V1 neurons are shaped to form a minimally redundant representation of natural images [5; 19]. [sent-18, score-0.274]

12 From a redundancy reduction point of view, ICA offers a small but signiﬁcant advantage over other linear representations [6]. [sent-19, score-0.426]

13 In terms of density estimation, however, it is a poor model for natural images since already a simple non-factorial spherically symmetric model yields a much better ﬁt to the data [10]. [sent-20, score-0.596]

14 Recently, Lyu and Simoncelli proposed a method that converts any spherically symmetric distribution into a (factorial) Gaussian (or Normal distribution) by using a non-linear transformation of the 1 norm of the image patches [17]. [sent-21, score-0.594]

15 This yields a non-linear redundancy reduction mechanism, which exploits the superiority of the spherically symmetric model over ICA. [sent-22, score-0.828]

16 Interestingly, the non-linearity of this Radial Gaussianization method closely resembles another feature of the early visual system, known as contrast gain control [13] or divisive normalization [20]. [sent-23, score-0.573]

17 However, since spherically symmetric models are invariant under orthogonal transformations, they are agnostic to the particular choice of basis in the whitened space. [sent-24, score-0.554]

18 Combining the observations from the two models of natural images, we can draw two conclusions: On the one hand, ICA is not a good model for natural images, because a simple spherically symmetric model yields a much better ﬁt [10]. [sent-26, score-0.571]

19 On the other hand, the spherically symmetric model in Radial Gaussianization cannot capture that ICA ﬁlters do yield a higher redundancy reduction than other linear transformations. [sent-27, score-0.828]

20 This leaves us with the questions whether we can understand the emergence of oriented ﬁlters in a more general redundancy reduction framework, which also includes a mechanism for contrast gain control. [sent-28, score-0.8]

21 In this work we address this question by using the more general class of Lp -spherically symmetric models [23; 12; 15]. [sent-29, score-0.196]

22 These models are quite similar to spherically symmetric models, but do depend on the particular shape of the linear ﬁlters. [sent-30, score-0.401]

23 Just like spherically symmetric models can be nonlinearly transformed into isotropic Gaussians, Lp -spherically symmetric models can be mapped into a unique class of factorial distributions, called p-generalized Normal distributions [11]. [sent-31, score-0.84]

24 Thus, we are able to quantify the inﬂuence of orientation selective ﬁlters and contrast gain control on the redundancy reduction of natural images in a joint model. [sent-32, score-1.224]

25 1 Models and Methods Decorrelation and Filters All probabilistic models in this paper are deﬁned on whitened natural images. [sent-34, score-0.146]

26 , xm of image patches, then C − 2 1 constitutes the symmetric whitening transform. [sent-38, score-0.316]

27 V C 2 yield the linear ﬁlters that are applied to the raw image patches before feeding them in the probabilistic models described below. [sent-40, score-0.169]

28 y = C − 2 x contains the coefﬁcients in the symmetric whitening basis. [sent-44, score-0.277]

29 From a biological perspective, this case is interesting as the ﬁlters resemble receptive ﬁelds of retinal ganglion cells with center-surround properties. [sent-45, score-0.193]

30 For natural image patches, ICA is known to yield orientation selective ﬁlters in resemblance to V1 simple cells. [sent-47, score-0.307]

31 While other orientation selective bases are possible, the ﬁlters deﬁned by VICA correspond to the optimal choice for redundancy reduction under the restriction to linear models. [sent-48, score-0.684]

32 2 Lp -spherically Symmetric Distributions The contour lines of spherically symmetric distributions have constant Euclidean norm. [sent-54, score-0.457]

33 Similarly, the contour lines of Lp -spherically symmetric distributions have constant p-norm1 ||y||p := 1 Note that ||y||p is only a norm in the strict sense if p ≥ 1. [sent-55, score-0.252]

34 For p = 2 the distribution is not invariant under arbitrary orthogonal transformations, which means that the choice of the basis V can make a difference in the likelihood of the data. [sent-59, score-0.137]

35 The Gaussian is the only L2 -spherically symmetric distribution with independent marginals. [sent-62, score-0.198]

36 A multivariate random variable Y is called Lp -spherically symmetric distributed if it can be written as a product Y = RU , where U is uniformly distributed on Sn−1 (1) and R is a univariate nonp negative random variable with an arbitrary distribution [23; 12]. [sent-68, score-0.231]

37 This immediately p suggests two ways of constructing an Lp -spherically symmetric distribution. [sent-74, score-0.174]

38 p (1) Analogous to the Gaussian being the only factorial spherically symmetric distribution [1], this distribution is the only Lp -spherically symmetric distribution with independent marginals [22]. [sent-77, score-0.817]

39 In our experiments, we will use the p-generalized Normal to model linear marginal independence by ﬁtting it to the coefﬁcients of the various bases in whitened space. [sent-79, score-0.165]

40 Since this distribution is sensitive to the particular ﬁlter shapes for p = 2, we can assess how well the distribution of the linearly transformed image patches is matched by a factorial model. [sent-80, score-0.49]

41 An alternative way of constructing an Lp -spherically symmetric distribution is to specify the radial distribution r . [sent-81, score-0.396]

42 In particular, for a ﬁxed p, any Lp -spherically symmetric distribution can be transformed into a factorial one by the transform z = g(y) · y = −1 (F2 ◦ F1 )(||y||p ) y. [sent-85, score-0.416]

43 ||y||p This transform closely resembles contrast gain control models for primary visual cortex [13; 20], 1 which use a different gain function having the form g (y) = c+r with r = ||y||2 [17]. [sent-86, score-0.782]

44 ˜ 2 We will use the distribution of equation (2) to describe the joint model consisting of a linear ﬁltering step followed by a contrast gain control mechanism. [sent-87, score-0.495]

45 Once, the linear ﬁlter responses in whitened space are ﬁtted with this distribution, we non-linearly transform it into a the factorial p-generalized −1 Normal by the transformation g(y) · y = (FgN ◦ FRMixLogN )(||y||p )/||y||p · y. [sent-88, score-0.294]

46 Finally, note that because a Lp -spherically symmetric distribution is speciﬁed by its univariate radial distribution, ﬁtting it to data boils down to estimating the univariate density for R, which can be done efﬁciently and robustly. [sent-89, score-0.465]

47 From each image circa 5000 patches of size 15×15 pixels were drawn at random locations for training (circa 40000 patches in total) as well as circa 6250 patches per image for testing (circa 50000 patches in total). [sent-93, score-0.668]

48 Before computing the linear ﬁlters, the DC component was projected out with an orthogonal transformation using a QR decomposition. [sent-96, score-0.137]

49 Afterwards, the data was rescaled in order to make whitening a volume conserving transformation (a transformation with determinant one) since those transformations leave the entropy unchanged. [sent-97, score-0.22]

50 2 Evaluation Measure In all our experiments, we used the Average Log Loss (ALL) to assess the quality of the ﬁt and m 1 1 the redundancy reduction achieved. [sent-99, score-0.456]

51 The difference between the ALLs of two models equals the reduction in multi-information (see Extra Material) and can therefore be used to quantify the amount of redundancy reduction. [sent-103, score-0.47]

52 3 Experiments We ﬁtted the Lp -spherically symmetric distributions from equations (1) and (2) to the image patches in the bases HAD, SYM, and ICA by a maximum likelihood ﬁt on the radial component. [sent-105, score-0.597]

53 For the mixture of Log-Normal distributions, we used EM for a mixture of Gaussians on the logarithm of the p-norm of the image patches. [sent-106, score-0.115]

54 Left: ALL for the bases HAD, SYM and ICA under the p-generalized Normal (HAD, SYM, ICA) and the factorial Lp -spherically symmetric model with the radial component modeled by a mixture of Log-Normal distributions (cHAD, cSYM, cICA). [sent-114, score-0.719]

55 In order to compare the redundancy reduction of the different transforms with respect to the pixel basis (PIX), we computed a non-parametric estimate of the marginal entropies of the patches before the DC component was projected out [6]. [sent-116, score-0.703]

56 The upper curve bundle represents the factorial p-generalized Normal model, the lower bundle the nonfactorial model with the radial component modeled by a mixture of Log-Normal distributions with ﬁve mixtures. [sent-120, score-0.545]

57 The ALL for the factorial models always exceeds the ALL for the non-factorial models. [sent-121, score-0.185]

58 As mentioned in the introduction, the p-generalized Normal is the only factorial Lp -spherically symmetric distribution [22]. [sent-124, score-0.361]

59 From the left plot in Figure 2, we can assess the inﬂuence of the different ﬁlter shapes and contrast gain control on the redundancy reduction of natural images. [sent-126, score-1.038]

60 We used the best ALL of the HAD basis under the p-generalized Normal as a baseline for a whitening transformation without contrast gain control (HAD). [sent-127, score-0.661]

61 Analogously, we used the best ALL of the HAD basis under the non-factorial model as a baseline for a pure contrast gain control model (cHAD). [sent-128, score-0.56]

62 Because the ﬁlters of SYM and ICA resemble receptive ﬁeld properties of retinal ganglion cells and V1 simple cells, respectively, we can assess their possible inﬂuence on the redundancy reduction with and without contrast gain control. [sent-130, score-0.968]

63 The factorial model corresponds to the case without contrast gain control (SYM and ICA). [sent-131, score-0.634]

64 Since we have shown that the non-factorial model can be transformed into a factorial one by a p-norm based divisive normalization operation, these scores correspond to the cases with contrast gain control (cSYM and cICA). [sent-132, score-0.735]

65 As already reported in other works, plain orientation selectivity adds only very little to the redundancy reduction achieved by decorrelation and is less effective than the baseline contrast gain control model [10; 6; 17]. [sent-134, score-1.317]

66 If both orientation selectivity and contrast gain control are combined (cICA) it is possible to achieve about 9% extra redundancy reduction in addition to baseline whitening 5 Absolute Difference [Bits/Comp. [sent-135, score-1.338]

67 The column on the right shows the relative difference with respect to the largest reduction achieved by ICA with non-factorial model. [sent-163, score-0.149]

68 Figure 3: The curve in the upper right corner depicts the trans−1 formation ||z||p = (FgN ◦ FRMixLogN )(||y||p ) of the radial component in the ICA basis for gray scale images. [sent-164, score-0.287]

69 The resulting radial distribution over ||z||p corresponds to the radial distribution of the p-generalized Normal. [sent-165, score-0.396]

70 The inset shows the gain function F (||y|| ) g(||y||p ) = RMixLogN p p in log||y|| log coordinates. [sent-166, score-0.258]

71 The scale parameter of the p-generalized normal was chosen such that the marginal had unit variance. [sent-167, score-0.125]

72 By setting the other models in relation to the best joint model (cICA:= 100%), we are able to tell apart the relative contributions of bandpass ﬁltering (HAD= 91%), particular ﬁlter shapes (SYM= 93%, ICA= 94%), contrast gain control (cHAD= 98. [sent-169, score-0.635]

73 6%) as well as combined models (cSYM= 99%, cICA := 100%) to redundancy reduction (see Table 1). [sent-170, score-0.448]

74 Thus, orientation selectivity (ICA) contributes less to the overall redundancy reduction than any model with contrast gain control (cHAD, cSYM, cICA). [sent-171, score-1.235]

75 Additionally, the relative difference between the joint model (cICA) and plain contrast gain control (cHAD) is only about 1. [sent-172, score-0.52]

76 The difference in redundancy reduction between center-surround ﬁlters and orientation selective ﬁlters becomes even smaller in combination with contrast gain control (1. [sent-176, score-1.102]

77 (F −1 ◦FRMixLogN )(||y||p ) When examining the gain functions g(||y||p ) = gN resulting from the transforma||y||p c tion of the radial components, we ﬁnd that they approximately exhibit the form g(||y||p ) = ||y||κ . [sent-183, score-0.407]

78 p The inset in Figure 3 shows the gain control function g(||y||p ) in a log-log plot. [sent-184, score-0.359]

79 While standard contrast gain control models assume p = 2 and κ = 2, we ﬁnd that κ between 0. [sent-185, score-0.47]

80 In addition, existing contrast gain models assume the form g(||y||2 ) = σ+||y||2 , 2 while we ﬁnd that σ must be approximately zero. [sent-190, score-0.369]

81 In the results above, the ICA ﬁlters always achieve the lowest ALL under both p-spherically symmetric models. [sent-191, score-0.174]

82 For examining whether these ﬁlters really represent the best choice, we also optimized the ﬁlter shapes under the model of equation (2) via maximum likelihood estimation on the orthogonal group in whitened space [9; 18]. [sent-192, score-0.205]

83 Figure 4 shows the ﬁlter shapes for ICA and the ones obtained from the optimization, where we used either the ICA solution or a random orthogonal matrix as starting point. [sent-193, score-0.12]

84 The ALL also changed just 6 Figure 4: Filters optimized for ICA (left) and for the p-spherically symmetric model with radial mixture of Log-Normal distributions starting from the ICA solution (middle) and from a random basis (right). [sent-195, score-0.502]

85 Thus, the ICA ﬁlters are a stable and optimal solution under the model with contrast gain control, too. [sent-212, score-0.37]

86 4 Summary In this report, we studied the conjoint effect of contrast gain control and orientation selectivity on redundancy reduction for natural images. [sent-213, score-1.313]

87 In particular, we showed how the Lp -spherically distribution can be used to tune a nonlinearity of contrast gain control to remove higher-order redundancies in natural images. [sent-214, score-0.611]

88 The idea of using an Lp -spherically symmetric model for natural images has already been brought up by Hyv¨ rinen and K¨ ster in the context of Independent Subspace Analysis [15]. [sent-215, score-0.404]

89 However, they a o do not use the Lp -distribution for contrast gain control, but apply a global contrast gain control ﬁlter on the images before ﬁtting their model. [sent-216, score-0.877]

90 They also use a less ﬂexible Lp -distribution since their goal is to ﬁt an ISA model to natural images and not to carry out a quantitative comparison as we did. [sent-217, score-0.167]

91 In our work, we ﬁnd that the gain control function turns out to follow a power law, which parallels the classical model of contrast gain control. [sent-218, score-0.704]

92 In addition, we ﬁnd that edge ﬁlters also emerge in the non-linear model which includes contrast gain control. [sent-219, score-0.37]

93 The relevance of orientation selectivity for redundancy reduction, however, is further reduced. [sent-220, score-0.637]

94 In the linear framework (possibly endowed with a point-wise nonlinearity for each neuron) the contribution of orientation selectivity to redundancy reduction has been shown to be smaller than 5% relative to whitening (i. [sent-221, score-0.892]

95 Here, we found that the contribution of orientation selectivity is even smaller than two percent relative to whitening plus gain control. [sent-224, score-0.674]

96 Thus, this quantitative model comparison provides further evidence that orientation selectivity is not critical for redundancy reduction, while contrast gain control may play a more important role. [sent-225, score-1.108]

97 Factorial coding of natural images: How effective are linear model in removing higher-order dependencies? [sent-259, score-0.123]

98 Simple cell coding of natural images in V1: How much use is orientation selectivity? [sent-304, score-0.374]

99 Nonlinear extraction of ’independent components’ of elliptically symmetric densities using radial Gaussianization. [sent-348, score-0.378]

100 Independent component ﬁlters of natural images compared with simple cells in primary visual cortex. [sent-400, score-0.251]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('ica', 0.38), ('redundancy', 0.299), ('sym', 0.242), ('cica', 0.237), ('gain', 0.233), ('spherically', 0.205), ('lp', 0.196), ('lters', 0.195), ('csym', 0.178), ('symmetric', 0.174), ('radial', 0.174), ('selectivity', 0.172), ('orientation', 0.166), ('factorial', 0.163), ('chad', 0.158), ('pix', 0.158), ('reduction', 0.127), ('contrast', 0.114), ('patches', 0.108), ('whitening', 0.103), ('control', 0.101), ('normal', 0.097), ('images', 0.082), ('circa', 0.079), ('shapes', 0.072), ('bandpass', 0.07), ('lter', 0.064), ('rinen', 0.063), ('natural', 0.062), ('whitened', 0.062), ('frmixlogn', 0.059), ('hyv', 0.056), ('redundancies', 0.052), ('bases', 0.052), ('sensory', 0.051), ('receptive', 0.05), ('distributions', 0.05), ('orthogonal', 0.048), ('component', 0.045), ('sinz', 0.044), ('transformation', 0.044), ('basis', 0.043), ('selective', 0.04), ('divisive', 0.04), ('conjoint', 0.039), ('fgn', 0.039), ('rmixlogn', 0.039), ('vica', 0.039), ('wachtler', 0.039), ('image', 0.039), ('coding', 0.038), ('mixture', 0.038), ('ltering', 0.037), ('hadamard', 0.037), ('gerwinn', 0.035), ('hyperspectral', 0.035), ('gaussianization', 0.035), ('univariate', 0.033), ('cells', 0.032), ('decorrelation', 0.032), ('filters', 0.032), ('normalization', 0.031), ('transformed', 0.03), ('assess', 0.03), ('dc', 0.03), ('ganglion', 0.03), ('elliptically', 0.03), ('lyu', 0.03), ('entropies', 0.03), ('visual', 0.03), ('transformations', 0.029), ('marginals', 0.029), ('marginal', 0.028), ('biological', 0.028), ('lines', 0.028), ('cients', 0.027), ('density', 0.027), ('retinal', 0.027), ('emergence', 0.027), ('plain', 0.027), ('cell', 0.026), ('resemble', 0.026), ('bundle', 0.026), ('gn', 0.026), ('simoncelli', 0.026), ('transform', 0.025), ('gray', 0.025), ('inset', 0.025), ('nonlinearity', 0.025), ('german', 0.025), ('distribution', 0.024), ('resembles', 0.024), ('pixel', 0.023), ('baseline', 0.023), ('spherical', 0.023), ('mpi', 0.023), ('model', 0.023), ('models', 0.022), ('difference', 0.022), ('coef', 0.022)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 1.0000004 232 nips-2008-The Conjoint Effect of Divisive Normalization and Orientation Selectivity on Redundancy Reduction

Author: Fabian H. Sinz, Matthias Bethge

2 0.35205173 192 nips-2008-Reducing statistical dependencies in natural signals using radial Gaussianization

Author: Siwei Lyu, Eero P. Simoncelli

Abstract: We consider the problem of transforming a signal to a representation in which the components are statistically independent. When the signal is generated as a linear transformation of independent Gaussian or non-Gaussian sources, the solution may be computed using a linear transformation (PCA or ICA, respectively). Here, we consider a complementary case, in which the source is non-Gaussian but elliptically symmetric. Such a source cannot be decomposed into independent components using a linear transform, but we show that a simple nonlinear transformation, which we call radial Gaussianization (RG), is able to remove all dependencies. We apply this methodology to natural signals, demonstrating that the joint distributions of nearby bandpass ﬁlter responses, for both sounds and images, are closer to being elliptically symmetric than linearly transformed factorial sources. Consistent with this, we demonstrate that the reduction in dependency achieved by applying RG to either pairs or blocks of bandpass ﬁlter responses is signiﬁcantly greater than that achieved by PCA or ICA.

3 0.14917809 102 nips-2008-ICA based on a Smooth Estimation of the Differential Entropy

Author: Lev Faivishevsky, Jacob Goldberger

Abstract: In this paper we introduce the MeanNN approach for estimation of main information theoretic measures such as differential entropy, mutual information and divergence. As opposed to other nonparametric approaches the MeanNN results in smooth differentiable functions of the data samples with clear geometrical interpretation. Then we apply the proposed estimators to the ICA problem and obtain a smooth expression for the mutual information that can be analytically optimized by gradient descent methods. The improved performance of the proposed ICA algorithm is demonstrated on several test examples in comparison with state-ofthe-art techniques. 1

4 0.12449401 234 nips-2008-The Infinite Factorial Hidden Markov Model

Author: Jurgen V. Gael, Yee W. Teh, Zoubin Ghahramani

Abstract: We introduce a new probability distribution over a potentially inﬁnite number of binary Markov chains which we call the Markov Indian buffet process. This process extends the IBP to allow temporal dependencies in the hidden variables. We use this stochastic process to build a nonparametric extension of the factorial hidden Markov model. After constructing an inference scheme which combines slice sampling and dynamic programming we demonstrate how the inﬁnite factorial hidden Markov model can be used for blind source separation. 1

5 0.092544451 118 nips-2008-Learning Transformational Invariants from Natural Movies

Author: Charles Cadieu, Bruno A. Olshausen

Abstract: We describe a hierarchical, probabilistic model that learns to extract complex motion from movies of the natural environment. The model consists of two hidden layers: the ﬁrst layer produces a sparse representation of the image that is expressed in terms of local amplitude and phase variables. The second layer learns the higher-order structure among the time-varying phase variables. After training on natural movies, the top layer units discover the structure of phase-shifts within the ﬁrst layer. We show that the top layer units encode transformational invariants: they are selective for the speed and direction of a moving pattern, but are invariant to its spatial structure (orientation/spatial-frequency). The diversity of units in both the intermediate and top layers of the model provides a set of testable predictions for representations that might be found in V1 and MT. In addition, the model demonstrates how feedback from higher levels can inﬂuence representations at lower levels as a by-product of inference in a graphical model. 1

6 0.085057706 58 nips-2008-Dependence of Orientation Tuning on Recurrent Excitation and Inhibition in a Network Model of V1

7 0.059402753 137 nips-2008-Modeling Short-term Noise Dependence of Spike Counts in Macaque Prefrontal Cortex

8 0.058902349 49 nips-2008-Clusters and Coarse Partitions in LP Relaxations

9 0.057893947 119 nips-2008-Learning a discriminative hidden part model for human action recognition

10 0.057242617 148 nips-2008-Natural Image Denoising with Convolutional Networks

11 0.054832082 112 nips-2008-Kernel Measures of Independence for non-iid Data

12 0.053762287 77 nips-2008-Evaluating probabilities under high-dimensional latent variable models

13 0.053449143 74 nips-2008-Estimating the Location and Orientation of Complex, Correlated Neural Activity using MEG

14 0.053384099 180 nips-2008-Playing Pinball with non-invasive BCI

15 0.052968092 62 nips-2008-Differentiable Sparse Coding

16 0.0525213 66 nips-2008-Dynamic visual attention: searching for coding length increments

17 0.049690492 75 nips-2008-Estimating vector fields using sparse basis field expansions

18 0.048443411 95 nips-2008-Grouping Contours Via a Related Image

19 0.048118904 145 nips-2008-Multi-stage Convex Relaxation for Learning with Sparse Regularization

20 0.047758266 56 nips-2008-Deep Learning with Kernel Regularization for Visual Recognition

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, -0.162), (1, -0.028), (2, 0.106), (3, 0.029), (4, 0.023), (5, 0.022), (6, -0.064), (7, 0.001), (8, 0.104), (9, 0.019), (10, -0.011), (11, -0.026), (12, 0.067), (13, 0.116), (14, -0.187), (15, -0.214), (16, 0.198), (17, 0.213), (18, -0.021), (19, 0.146), (20, -0.103), (21, 0.126), (22, -0.016), (23, 0.091), (24, -0.044), (25, -0.184), (26, 0.003), (27, 0.046), (28, -0.159), (29, -0.097), (30, 0.053), (31, 0.167), (32, -0.046), (33, 0.032), (34, 0.023), (35, -0.166), (36, -0.054), (37, 0.02), (38, 0.039), (39, 0.194), (40, -0.01), (41, 0.036), (42, -0.048), (43, -0.01), (44, 0.139), (45, 0.04), (46, 0.086), (47, -0.007), (48, 0.087), (49, 0.11)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.96958417 232 nips-2008-The Conjoint Effect of Divisive Normalization and Orientation Selectivity on Redundancy Reduction

Author: Fabian H. Sinz, Matthias Bethge

2 0.90856194 192 nips-2008-Reducing statistical dependencies in natural signals using radial Gaussianization

Author: Siwei Lyu, Eero P. Simoncelli

3 0.45307416 234 nips-2008-The Infinite Factorial Hidden Markov Model

Author: Jurgen V. Gael, Yee W. Teh, Zoubin Ghahramani

4 0.4454838 102 nips-2008-ICA based on a Smooth Estimation of the Differential Entropy

Author: Lev Faivishevsky, Jacob Goldberger

5 0.30462852 66 nips-2008-Dynamic visual attention: searching for coding length increments

Author: Xiaodi Hou, Liqing Zhang

Abstract: A visual attention system should respond placidly when common stimuli are presented, while at the same time keep alert to anomalous visual inputs. In this paper, a dynamic visual attention model based on the rarity of features is proposed. We introduce the Incremental Coding Length (ICL) to measure the perspective entropy gain of each feature. The objective of our model is to maximize the entropy of the sampled visual features. In order to optimize energy consumption, the limit amount of energy of the system is re-distributed amongst features according to their Incremental Coding Length. By selecting features with large coding length increments, the computational system can achieve attention selectivity in both static and dynamic scenes. We demonstrate that the proposed model achieves superior accuracy in comparison to mainstream approaches in static saliency map generation. Moreover, we also show that our model captures several less-reported dynamic visual search behaviors, such as attentional swing and inhibition of return. 1

6 0.30194435 118 nips-2008-Learning Transformational Invariants from Natural Movies

7 0.28340334 61 nips-2008-Diffeomorphic Dimensionality Reduction

8 0.27912265 126 nips-2008-Localized Sliced Inverse Regression

9 0.27585405 227 nips-2008-Supervised Exponential Family Principal Component Analysis via Convex Optimization

10 0.25759715 156 nips-2008-Nonparametric sparse hierarchical models describe V1 fMRI responses to natural images

11 0.25522423 45 nips-2008-Characterizing neural dependencies with copula models

12 0.25321415 180 nips-2008-Playing Pinball with non-invasive BCI

13 0.23932168 148 nips-2008-Natural Image Denoising with Convolutional Networks

14 0.23741609 38 nips-2008-Bio-inspired Real Time Sensory Map Realignment in a Robotic Barn Owl

15 0.23736733 58 nips-2008-Dependence of Orientation Tuning on Recurrent Excitation and Inhibition in a Network Model of V1

16 0.23323418 119 nips-2008-Learning a discriminative hidden part model for human action recognition

17 0.22814317 95 nips-2008-Grouping Contours Via a Related Image

18 0.22347248 173 nips-2008-Optimization on a Budget: A Reinforcement Learning Approach

19 0.21824694 30 nips-2008-Bayesian Experimental Design of Magnetic Resonance Imaging Sequences

20 0.21744931 75 nips-2008-Estimating vector fields using sparse basis field expansions

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(6, 0.064), (7, 0.085), (12, 0.038), (15, 0.015), (28, 0.13), (37, 0.287), (57, 0.109), (59, 0.021), (63, 0.017), (71, 0.021), (77, 0.039), (78, 0.023), (83, 0.044), (88, 0.013)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.79157913 232 nips-2008-The Conjoint Effect of Divisive Normalization and Orientation Selectivity on Redundancy Reduction

Author: Fabian H. Sinz, Matthias Bethge

2 0.72538596 182 nips-2008-Posterior Consistency of the Silverman g-prior in Bayesian Model Choice

Author: Zhihua Zhang, Michael I. Jordan, Dit-Yan Yeung

Abstract: Kernel supervised learning methods can be uniﬁed by utilizing the tools from regularization theory. The duality between regularization and prior leads to interpreting regularization methods in terms of maximum a posteriori estimation and has motivated Bayesian interpretations of kernel methods. In this paper we pursue a Bayesian interpretation of sparsity in the kernel setting by making use of a mixture of a point-mass distribution and prior that we refer to as “Silverman’s g-prior.” We provide a theoretical analysis of the posterior consistency of a Bayesian model choice procedure based on this prior. We also establish the asymptotic relationship between this procedure and the Bayesian information criterion. 1

3 0.62058663 192 nips-2008-Reducing statistical dependencies in natural signals using radial Gaussianization

Author: Siwei Lyu, Eero P. Simoncelli

4 0.57860363 27 nips-2008-Artificial Olfactory Brain for Mixture Identification

Author: Mehmet K. Muezzinoglu, Alexander Vergara, Ramon Huerta, Thomas Nowotny, Nikolai Rulkov, Henry Abarbanel, Allen Selverston, Mikhail Rabinovich

Abstract: The odor transduction process has a large time constant and is susceptible to various types of noise. Therefore, the olfactory code at the sensor/receptor level is in general a slow and highly variable indicator of the input odor in both natural and artiﬁcial situations. Insects overcome this problem by using a neuronal device in their Antennal Lobe (AL), which transforms the identity code of olfactory receptors to a spatio-temporal code. This transformation improves the decision of the Mushroom Bodies (MBs), the subsequent classiﬁer, in both speed and accuracy. Here we propose a rate model based on two intrinsic mechanisms in the insect AL, namely integration and inhibition. Then we present a MB classiﬁer model that resembles the sparse and random structure of insect MB. A local Hebbian learning procedure governs the plasticity in the model. These formulations not only help to understand the signal conditioning and classiﬁcation methods of insect olfactory systems, but also can be leveraged in synthetic problems. Among them, we consider here the discrimination of odor mixtures from pure odors. We show on a set of records from metal-oxide gas sensors that the cascade of these two new models facilitates fast and accurate discrimination of even highly imbalanced mixtures from pure odors. 1

5 0.57096153 66 nips-2008-Dynamic visual attention: searching for coding length increments

Author: Xiaodi Hou, Liqing Zhang

6 0.57039863 208 nips-2008-Shared Segmentation of Natural Scenes Using Dependent Pitman-Yor Processes

7 0.56772095 62 nips-2008-Differentiable Sparse Coding

8 0.56625456 200 nips-2008-Robust Kernel Principal Component Analysis

9 0.56592029 100 nips-2008-How memory biases affect information transmission: A rational analysis of serial reproduction

10 0.56394637 116 nips-2008-Learning Hybrid Models for Image Annotation with Partially Labeled Data

11 0.56129622 118 nips-2008-Learning Transformational Invariants from Natural Movies

12 0.56040287 197 nips-2008-Relative Performance Guarantees for Approximate Inference in Latent Dirichlet Allocation

13 0.55942047 63 nips-2008-Dimensionality Reduction for Data in Multiple Feature Representations

14 0.55601609 71 nips-2008-Efficient Sampling for Gaussian Process Inference using Control Variables

15 0.55573964 79 nips-2008-Exploring Large Feature Spaces with Hierarchical Multiple Kernel Learning

16 0.5556035 75 nips-2008-Estimating vector fields using sparse basis field expansions

17 0.55380952 176 nips-2008-Partially Observed Maximum Entropy Discrimination Markov Networks

18 0.55376238 42 nips-2008-Cascaded Classification Models: Combining Models for Holistic Scene Understanding

19 0.55327499 226 nips-2008-Supervised Dictionary Learning

20 0.55319691 221 nips-2008-Stochastic Relational Models for Large-scale Dyadic Data using MCMC