nips nips2007 nips2007-145 nips2007-145-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Pietro Berkes, Richard Turner, Maneesh Sahani
Abstract: Computational models of visual cortex, and in particular those based on sparse coding, have enjoyed much recent attention. Despite this currency, the question of how sparse or how over-complete a sparse representation should be, has gone without principled answer. Here, we use Bayesian model-selection methods to address these questions for a sparse-coding model based on a Student-t prior. Having validated our methods on toy data, we find that natural images are indeed best modelled by extremely sparse distributions; although for the Student-t prior, the associated optimal basis size is only modestly over-complete. 1
[1] B.A. Olshausen and D.J. Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381(6583):607–609, 1996.
[2] B.A. Olshausen and D.J. Field. Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research, 37:3311–3325, 1997.
[3] Y.W Teh, M. Welling, S. Osindero, and G.E. Hinton. Energy-based models for sparse overcomplete representations. Journal of Machine Learning Research, 4:1235–1260, 2003.
[4] A.J. Bell and T.J. Sejnowski. The ‘independent components’ of natural scenes are edge filters. Vision Research, 37(23):3327–3338, 1997.
[5] S. Osindero, M. Welling, and G.E. Hinton. Topographic product models applied to natural scene statistics. Neural Computation, 18:381–344, 2006.
[6] D.J.C. McKay. Bayesian interpolation. Neural Comput, 4(3):415–447, 1992.
[7] C.M. Bishop. Variational principal components. In ICANN 1999 Proceedings, pages 509–514, 1999.
[8] M.J. Beal. Variational Algorithms for Approximate Bayesian Inference. PhD thesis, Gatsby Computational Neuroscience Unit, University College London, 2003.
[9] R.M. Neal. Annealed importance sampling. Statistics and Computing, 11:125–139, 2001.
[10] J.H. van Hateren and A. van der Schaaf. Independent component filters of natural images compared with simple cells in primary visual cortex. Proc. R. Soc. Lond. B, 265:359–366, 1998. 8