nips nips2013 nips2013-203 nips2013-203-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Mark Rogers, Lei Li, Stuart Russell
Abstract: Data in the sciences frequently occur as sequences of multidimensional arrays called tensors. How can hidden, evolving trends in such data be extracted while preserving the tensor structure? The model that is traditionally used is the linear dynamical system (LDS) with Gaussian noise, which treats the latent state and observation at each time slice as a vector. We present the multilinear dynamical system (MLDS) for modeling tensor time series and an expectation–maximization (EM) algorithm to estimate the parameters. The MLDS models each tensor observation in the time series as the multilinear projection of the corresponding member of a sequence of latent tensors. The latent tensors are again evolving with respect to a multilinear projection. Compared to the LDS with an equal number of parameters, the MLDS achieves higher prediction accuracy and marginal likelihood for both artificial and real datasets. 1
[1] Jan R. Magnus and Heinz Neudecker. Matrix Differential Calculus with Applications in Statistics and Econometrics. Wiley, revised edition, 1999.
[2] Vin De Silva and Lek-Heng Lim. Tensor rank and the ill-posedness of the best low-rank approximation problem. SIAM Journal on Matrix Analysis and Applications, 30(3):1084–1127, 2008.
[3] Tamara G. Kolda. Tensor decompositions and applications. SIAM Review, 51(3):455–500, 2009.
[4] Peter J. Basser and Sinisa Pajevic. A normal distribution for tensor-valued random variables: applications to diffusion tensor MRI. IEEE Transactions on Medical Imaging, 22(7):785–794, 2003.
[5] Arthur P. Dempster, Nan M. Laird, and Donald B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological), 39(1):1–38, 1977.
[6] Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer, 1st edition, 2006.
[7] NOAA/Pacific Marine Environmental Laboratory. Tropical Atmosphere Ocean Project. http://www. pmel.noaa.gov/tao/data_deliv/deliv.html. Accessed: May 23, 2013.
[8] Zoubin Ghahramani and Geoffrey E. Hinton. Parameter estimation for linear dynamical systems. Technical Report CRG-TR-96-2, University of Toronto Department of Computer Science, 1996.
[9] Jimeng Sun, Dacheng Tao, and Christos Faloutsos. Beyond streams and graphs: dynamic tensor analysis. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 374–383. ACM, 2006.
[10] Liang Xiong, Xi Chen, Tzu-Kuo Huang, Jeff Schneider, and Jaime G. Carbonell. Temporal collaborative filtering with Bayesian probabilistic tensor factorization. In Proceedings of SIAM Data Mining, 2010.
[11] Haipin Lu, Konstantinos N. Plataniotis, and Anastasios N. Venetsanopoulos. MPCA: Multilinear principal components analysis of tensor objects. IEEE Transactions on Neural Networks, 19(1), 2008.
[12] Tamara Kolda and Jimeng Sun. Scalable tensor decompositions for multi-aspect data mining. In Eighth IEEE International Conference on Data Mining. IEEE, 2008.
[13] Dacheng Tao, Mingli Song, Xuelong Li, Jialie Shen, Jimeng Sun, Xindong Wu, Christos Faloutsos, and Stephen J. Maybank. Bayesian tensor approach for 3-D face modeling. IEEE Transactions on Circuits and Systems for Video Technology, 18(10):1397–1410, 2008.
[14] Yehuda Koren, Robert Bell, and Chris Volinsky. Matrix factorization techniques for recommender systems. Computer, 42(8):30–37, 2009.
[15] Ruslan Salakhutdinov and Andriy Mnih. Probabilistic matrix factorization. In Advances in Neural Information Processing Systems, volume 20, pages 1257–1264, 2008.
[16] Ruslan Salakhutdinov and Andriy Mnih. Bayesian probabilistic matrix factorization using Markov chain Monte Carlo. In Proceedings of the 25th International Conference on Machine Learning. ACM, 2008.
[17] Cyril Goutte and Massih-Reza Amini. Probabilistic tensor factorization and model selection. In Tensors, Kernels, and Machine Learning (TKLM 2010), pages 1–4, 2010.
[18] Christian F. Beckmann and Stephen M. Smith. Tensorial extensions of independent component analysis for multisubject FMRI analysis. Neuroimage, 25(1):294–311, 2005.
[19] Y. Kenan Yilmaz, A. Taylan Cemgil, and Umut Simsekli. Generalized coupled tensor factorization. In Neural Information Processing Systems. MIT Press, 2011. 9