jmlr jmlr2006 jmlr2006-47 jmlr2006-47-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Michael W. Spratling
Abstract: In order to perform object recognition it is necessary to learn representations of the underlying components of images. Such components correspond to objects, object-parts, or features. Nonnegative matrix factorisation is a generative model that has been specifically proposed for finding such meaningful representations of image data, through the use of non-negativity constraints on the factors. This article reports on an empirical investigation of the performance of non-negative matrix factorisation algorithms. It is found that such algorithms need to impose additional constraints on the sparseness of the factors in order to successfully deal with occlusion. However, these constraints can themselves result in these algorithms failing to identify image components under certain conditions. In contrast, a recognition model (a competitive learning neural network algorithm) reliably and accurately learns representations of elementary image features without such constraints. Keywords: non-negative matrix factorisation, competitive learning, dendritic inhibition, object recognition
S. C. Ahalt, A. K. Krishnamurthy, P. Chen, and D. E. Melton. Competitive learning algorithms for vector quantization. Neural Networks, 3:277–90, 1990. H. B. Barlow. Conditions for versatile learning, Helmholtz’s unconscious inference, and the task of perception. Vision Research, 30:1561–71, 1990. H. B. Barlow. The neuron doctrine in perception. In M. S. Gazzaniga, editor, The Cognitive Neurosciences, chapter 26. MIT Press, Cambridge, MA, 1995. D. Charles and C. Fyfe. Discovering independent sources with an adapted PCA neural network. In D. W. Pearson, editor, Proceedings of the 2nd International ICSC Symposium on Soft Computing (SOCO97). NAISO Academic Press, 1997. D. Charles and C. Fyfe. Modelling multiple cause structure using rectification constraints. Network: Computation in Neural Systems, 9(2):167–82, 1998. G. M. Davies, J. W. Shepherd, and H. D. Ellis. Similarity effects in face recognition. American Journal of Psychology, 92:507–23, 1979. P. Dayan and R. S. Zemel. Competition and multiple cause models. Neural Computation, 7:565–79, 1995. T. Feng, S. Z. Li, H.-Y. Shum, and H. Zhang. Local non-negative matrix factorization as a visual representation. In Proceedings of the 2nd International Conference on Development and Learning (ICDL02), pages 178–86, 2002. P. F¨ ldi´ k. Adaptive network for optimal linear feature extraction. In Proceedings of the IEEE/INNS o a International Joint Conference on Neural Networks, volume 1, pages 401–5, New York, NY, 1989. IEEE Press. 813 S PRATLING P. F¨ ldi´ k. Forming sparse representations by local anti-Hebbian learning. Biological Cybernetics, o a 64:165–70, 1990. P. F¨ ldi´ k and M. P. Young. Sparse coding in the primate cortex. In M. A. Arbib, editor, The o a Handbook of Brain Theory and Neural Networks, pages 895–8. MIT Press, Cambridge, MA, 1995. B. J. Frey, P. Dayan, and G. E. Hinton. A simple algorithm that discovers efficient perceptual codes. In M. Jenkin and L. R. Harris, editors, Computational and Psychophysical Mechanisms of Visual Coding. Cambridge University Press, Cambridge, UK, 1997. C. Fyfe. Independence seeking negative feedback networks. In D. W. Pearson, editor, Proceedings of the 2nd International ICSC Symposium on Soft Computing (SOCO97). NAISO Academic Press, 1997a. C. Fyfe. A neural net for PCA and beyond. Neural Processing Letters, 6(1-2):33–41, 1997b. X. Ge and S. Iwata. Learning the parts of objects by auto-association. Neural Networks, 15(2): 285–95, 2002. G. Harpur and R. Prager. Development of low entropy coding in a recurrent network. Network: Computation in Neural Systems, 7(2):277–84, 1996. G. E. Hinton, P. Dayan, B. J. Frey, and R. M. Neal. The wake-sleep algorithm for unsupervised neural networks. Science, 268(5214):1158–61, 1995. G. E. Hinton and Z. Ghahramani. Generative models for discovering sparse distributed representations. Philosophical Transactions of the Royal Society of London. Series B, 352(1358):1177–90, 1997. S. Hochreiter and J. Schmidhuber. Feature extraction through LOCOCODE. Neural Computation, 11:679–714, 1999. P. O. Hoyer. Non-negative sparse coding. In Neural Networks for Signal Processing XII: Proceedings of the IEEE Workshop on Neural Networks for Signal Processing, pages 557–65, 2002. P. O. Hoyer. Non-negative matrix factorization with sparseness constraints. Journal of Machine Learning Research, 5:1457–69, 2004. C. Jutten and J. Herault. Blind separation of sources, part I: an adaptive algorithm based on neuromimetic architecture. Signal Processing, 24:1–10, 1991. T. Kohonen. Self-Organizing Maps. Springer-Verlag, Berlin, 1997. D. D. Lee and H. S. Seung. Learning the parts of objects by non-negative matrix factorization. Nature, 401:788–91, 1999. D. D. Lee and H. S. Seung. Algorithms for non-negative matrix factorization. In T. K. Leen, T. G. Dietterich, and V. Tresp, editors, Advances in Neural Information Processing Systems 13, Cambridge, MA, 2001. MIT Press. 814 L EARNING I MAGE C OMPONENTS FOR O BJECT R ECOGNITION S. Z. Li, X. Hou, H. Zhang, and Q. Cheng. Learning spatially localized, parts-based representations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR01), volume 1, pages 207–12, 2001. W. Liu and N. Zheng. Non-negative matrix factorization based methods for object recognition. Pattern Recognition Letters, 25(8):893–7, 2004. W. Liu, N. Zheng, and X. Lu. Non-negative matrix factorization for visual coding. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP03), volume 3, pages 293–6, 2003. J. L¨ cke and C. von der Malsburg. Rapid processing and unsupervised learning in a model of the u cortical macrocolumn. Neural Computation, 16(3):501–33, 2004. M. Meila and M. I. Jordan. Learning with mixtures of trees. Journal of Machine Learning Research, 1:1–48, 2000. E. Oja. Principal components, minor components, and linear neural networks. Neural Networks, 5: 927–35, 1992. B. A. Olshausen and D. J. Field. Sparse coding of sensory inputs. Current Opinion in Neurobiology, 14:481–7, 2004. R. C. O’Reilly. Generalization in interactive networks: The benefits of inhibitory competition and Hebbian learning. Neural Computation, 13(6):1199–1242, 2001. T. J. Palmeri and I. Gauthier. Visual object understanding. Nature Reviews Neuroscience, 5(4): 291–303, 2004. M. D. Plumbley. Adaptive lateral inhibition for non-negative ICA. In Proceedings of the international Conference on Independent Component Analysis and Blind Signal Separation (ICA2001), pages 516–21, 2001. E. Saund. A multiple cause mixture model for unsupervised learning. Neural Computation, 7(1): 51–71, 1995. P. Sinha and T. Poggio. I think I know that face...,. Nature, 384(6608):404, 1996. M. W. Spratling and M. H. Johnson. Pre-integration lateral inhibition enhances unsupervised learning. Neural Computation, 14(9):2157–79, 2002. M. W. Spratling and M. H. Johnson. Exploring the functional significance of dendritic inhibition in cortical pyramidal cells. Neurocomputing, 52-54:389–95, 2003. M. W. Spratling and M. H. Johnson. Neural coding strategies and mechanisms of competition. Cognitive Systems Research, 5(2):93–117, 2004. M. W. Spratling and M. H. Johnson. A feedback model of perceptual learning and categorisation. Visual Cognition, 13(2):129–65, 2006. 815