nips nips2009 nips2009-111 nips2009-111-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Matthias Bethge, Eero P. Simoncelli, Fabian H. Sinz
Abstract: We introduce a new family of distributions, called Lp -nested symmetric distributions, whose densities are expressed in terms of a hierarchical cascade of Lp norms. This class generalizes the family of spherically and Lp -spherically symmetric distributions which have recently been successfully used for natural image modeling. Similar to those distributions it allows for a nonlinear mechanism to reduce the dependencies between its variables. With suitable choices of the parameters and norms, this family includes the Independent Subspace Analysis (ISA) model as a special case, which has been proposed as a means of deriving filters that mimic complex cells found in mammalian primary visual cortex. Lp -nested distributions are relatively easy to estimate and allow us to explore the variety of models between ISA and the Lp -spherically symmetric models. By fitting the generalized Lp -nested model to 8 × 8 image patches, we show that the subspaces obtained from ISA are in fact more dependent than the individual filter coefficients within a subspace. When first applying contrast gain control as preprocessing, however, there are no dependencies left that could be exploited by ISA. This suggests that complex cell modeling can only be useful for redundancy reduction in larger image patches. 1
[1] F. Attneave. Informational aspects of visual perception. Psychological Review, 61:183–193, 1954.
[2] R. Baddeley. Searching for filters with “interesting” output distributions: an uninteresting direction to explore? Network: Computation in Neural Systems, 7(2):409–421, 1996.
[3] H. B. Barlow. Sensory mechanisms, the reduction of redundancy, and intelligence. 1959.
[4] Anthony J. Bell and Terrence J. Sejnowski. An Information-Maximization approach to blind separation and blind deconvolution. Neural Computation, 7(6):1129–1159, November 1995.
[5] Matthias Bethge. Factorial coding of natural images: how effective are linear models in removing higherorder dependencies? Journal of the Optical Society of America A, 23(6):1253–1268, June 2006.
[6] Jan Eichhorn, Fabian Sinz, and Matthias Bethge. Natural image coding in v1: How much use is orientation selectivity? PLoS Comput Biol, 5(4):e1000336, April 2009.
[7] Carmen Fernandez, Jacek Osiewalski, and Mark F. J. Steel. Modeling and inference with ν-spherical distributions. Journal of the American Statistical Association, 90(432):1331–1340, Dec 1995.
[8] Irwin R. Goodman and Samuel Kotz. Multivariate θ-generalized normal distributions. Journal of Multivariate Analysis, 3(2):204–219, Jun 1973.
[9] A. K. Gupta and D. Song. lp -norm spherical distribution. Journal of Statistical Planning and Inference, 60:241–260, 1997.
[10] A. Hyvarinen and U. Koster. Complex cell pooling and the statistics of natural images. Network: Computation in Neural Systems, 18(2):81–100, 2007.
[11] S Lyu and E P Simoncelli. Nonlinear extraction of ’independent components’ of natural images using radial Gaussianization. Neural Computation, 21(6):1485–1519, June 2009.
[12] S Lyu and E P Simoncelli. Reducing statistical dependencies in natural signals using radial Gaussianization. In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, editors, Adv. Neural Information Processing Systems 21, volume 21, pages 1009–1016, Cambridge, MA, May 2009. MIT Press.
[13] Siwei Lyu and E.P. Simoncelli. Modeling multiscale subbands of photographic images with fields of gaussian scale mixtures. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 31(4):693– 706, 2009.
[14] J. H. Manton. Optimization algorithms exploiting unitary constraints. IEEE Transactions on Signal Processing, 50:635 – 650, 2002.
[15] J. A. Nelder and R. Mead. A simplex method for function minimization. The Computer Journal, 7(4):308– 313, Jan 1965.
[16] Bruno A. Olshausen and David J. Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381(6583):607–609, June 1996.
[17] Liam Paninski. Estimation of entropy and mutual information. Neural Computation, 15(6):1191–1253, Jun 2003.
[18] S. Roth and M.J. Black. Fields of experts: a framework for learning image priors. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, volume 2, pages 860–867 vol. 2, 2005.
[19] David W. Scott. On optimal and data-based histograms. Biometrika, 66(3):605–610, Dec 1979.
[20] E.P. Simoncelli. Statistical models for images: compression, restoration and synthesis. In Signals, Systems & Computers, 1997. Conference Record of the Thirty-First Asilomar Conference on, volume 1, pages 673–678 vol.1, 1997.
[21] F. Sinz and M. Bethge. The conjoint effect of divisive normalization and orientation selectivity on redundancy reduction. In Neural Information Processing Systems 2008, 2009.
[22] F. H. Sinz, S. Gerwinn, and M. Bethge. Characterization of the p-generalized normal distribution. Journal of Multivariate Analysis, 100(5):817–820, 05 2009.
[23] M. J. Wainwright and E. P. Simoncelli. Scale mixtures of gaussians and the statistics of natural images. In Advances in neural information processing systems, volume 12, pages 855–861, 2000.
[24] Christoph Zetzsche, Gerhard Krieger, and Bernhard Wegmann. The atoms of vision: Cartesian or polar? Journal of the Optical Society of America A, 16(7):1554–1565, Jul 1999. 9