nips nips2013 nips2013-214 nips2013-214-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Siwei Lyu, Xin Wang
Abstract: Nonnegative matrix factorization (NMF) is a popular data analysis method, the objective of which is to approximate a matrix with all nonnegative components into the product of two nonnegative matrices. In this work, we describe a new simple and efficient algorithm for multi-factor nonnegative matrix factorization (mfNMF) problem that generalizes the original NMF problem to more than two factors. Furthermore, we extend the mfNMF algorithm to incorporate a regularizer based on the Dirichlet distribution to encourage the sparsity of the components of the obtained factors. Our sparse mfNMF algorithm affords a closed form and an intuitive interpretation, and is more efficient in comparison with previous works that use fix point iterations. We demonstrate the effectiveness and efficiency of our algorithms on both synthetic and real data sets. 1
[1] Daniel D. Lee and H. Sebastian Seung. Algorithms for nonnegative matrix factorization. In Advances in Neural Information Processing Systems (NIPS 13), 2001. 1, 2
[2] A. Cichocki, R. Zdunek, A.H. Phan, and S. Amari. Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. Wiley, 2009. 1
[3] Seungjin Choi Jong-Hoon Ahn and Jong-Hoon Oh. A multiplicative up-propagation algorithm. In ICML, 2004. 2
[4] Nicolas Gillis and Fran cois Glineur. A multilevel approach for nonnegative matrix factorization. Journal of Computational and Applied Mathematics, 236 (7):1708–1723, 2012. 2
[5] A Cichocki and R Zdunek. Multilayer nonnegative matrix factorisation. Electronics Letters, 42(16):947– 948, 2006. 2, 7
[6] Patrik O. Hoyer and Peter Dayan. Non-negative matrix factorization with sparseness constraints. Journal of Machine Learning Research, 5:1457–1469, 2004. 1, 2
[7] Bhiksha Raj Madhusudana Shashanka and Paris Smaragdis. Sparse overcomplete latent variable decomposition of counts data. In NIPS, 2007. 1, 2
[8] Martin Larsson and Johan Ugander. A concave regularization technique for sparse mixture models. In NIPS, 2011. 1, 2, 5, 8
[9] Suvrit Sra and Inderjit S Dhillon. Nonnegative matrix approximation: Algorithms and applications. Computer Science Department, University of Texas at Austin, 2006. 2, 7, 8
[10] Jussi Kujala. Sparse topic modeling with concave-convex procedure: EMish algorithm for latent dirichlet allocation. In Technical Report, 2004. 2
[11] Jagannadan Varadarajan, R´ mi Emonet, and Jean-Marc Odobez. A sequential topic model for mining e recurrent activities from long term video logs. International Journal of Computer Vision, 103(1):100– 126, 2013. 2, 6
[12] S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2005. 4
[13] P. Gahinet, P. Apkarian, and M. Chilali. Affine parameter-dependent Lyapunov functions and real parametric uncertainty. IEEE Transactions on Automatic Control, 41(3):436–442, 1996. 4
[14] Wray Buntine and Aleks Jakulin. Discrete component analysis. In Subspace, Latent Structure and Feature Selection Techniques. Springer-Verlag, 2006. 5
[15] N. Hurley and Scott Rickard. Comparing measures of sparsity. Information Theory, IEEE Transactions on, 55(10):4723–4741, 2009. 6
[16] Misha Denil and Nando de Freitas. Recklessly approximate sparse coding. CoRR, abs/1208.0959, 2012. 6
[17] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998. 7
[18] Andrzej Cichocki, Rafal Zdunek, and Shun-ichi Amari. Csiszar’s divergences for non-negative matrix factorization: Family of new algorithms. In Independent Component Analysis and Blind Signal Separation, pages 32–39. Springer, 2006. 8
[19] C´ dric F´ votte, Nancy Bertin, and Jean-Louis Durrieu. Nonnegative matrix factorization with the itakurae e saito divergence: With application to music analysis. Neural Computation, 21(3):793–830, 2009. 8
[20] Andrzej Cichocki, Rafal Zdunek, Seungjin Choi, Robert Plemmons, and Shun-Ichi Amari. Non-negative tensor factorization using alpha and beta divergences. In Acoustics, Speech and Signal Processing, 2007. ICASSP 2007. IEEE International Conference on, volume 3, pages III–1393. IEEE, 2007. 8
[21] V. Chv´ tal. Linear Programming. W. H. Freeman and Company, New York, 1983. a 9