nips nips2006 nips2006-200 nips2006-200-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Ali Rahimi, Ben Recht
Abstract: We derive a cost functional for estimating the relationship between highdimensional observations and the low-dimensional process that generated them with no input-output examples. Limiting our search to invertible observation functions confers numerous benefits, including a compact representation and no suboptimal local minima. Our approximation algorithms for optimizing this cost functional are fast and give diagnostic bounds on the quality of their solution. Our method can be viewed as a manifold learning algorithm that utilizes a prior on the low-dimensional manifold coordinates. The benefits of taking advantage of such priors in manifold learning and searching for the inverse observation functions in system identification are demonstrated empirically by learning to track moving targets from raw measurements in a sensor network setting and in an RFID tracking experiment. 1
[1] P.H. Algoet and T.M. Cover. A sandwich proof of the Shannon-McMillan-Breiman theorem. The Annals of Probability, 16:899–909, 1988.
[2] Aharon Ben-Tal and Arkadi Nemirovski. Non-euclidean restricted memory level method for large-scale convex optimization. Mathematical Programming, 102:407–456, 2005.
[3] Z. Ghahramani and S. Roweis. Learning nonlinear dynamical systems using an EM algorithm. In Advances in Neural Information Processing Systems (NIPS), 1998.
[4] G. Golub and C.F. Van Loan. Matrix Computations. The Johns Hopkins University Press, 1989.
[5] V. Guilleman and A. Pollack. Differential Topology. Prentice Hall, Englewood Cliffs, New Jersey, 1974.
[6] O. Jenkins and M. Mataric. A spatio-temporal extension to isomap nonlinear dimension reduction. In International Conference on Machine Learning (ICML), 2004.
[7] F. Jones. Advanced Calculus. http://www.owlnet.rice.edu/˜fjones, unpublished.
[8] A. Juditsky, H. Hjalmarsson, A. Benveniste, B. Delyon, L. Ljung, J. Sj¨ berg, and Q. Zhang. Nonlinear o black-box models in system identification: Mathematical foundations. Automatica, 31(12):1725–1750, 1995.
[9] N. D. Lawrence. Gaussian process latent variable models for visualisation of high dimensional data. In Advances in Neural Information Processing Systems (NIPS), 2004.
[10] J. Patten, H. Ishii, J. Hines, and G. Pangaro. Sensetable: A wireless object tracking platform for tangible user interfaces. In CHI, 2001.
[11] N. Patwari and A. O. Hero. Manifold learning algorithms for localization in wireless sensor networks. In International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2004.
[12] A. Rahimi. Learning to Transform Time Series with a Few Examples. PhD thesis, Massachusetts Institute of Technology, Computer Science and AI Lab, Cambridge, Massachusetts, USA, 2005.
[13] A. Rahimi, B. Recht, and T. Darrell. Learning appearance manifolds from video. In Computer Vision and Pattern Recognition (CVPR), 2005.
[14] I. K. Rana. An Introduction to Measure Theory and Integration. AMA, second edition, 2002.
[15] B. Sch¨ lkopf, A. Smola, and K-R. M¨ ller. Nonlinear component analysis as a kernel eigenvalue problem. o u Neural Computation, 10:1299–1319, 1998.
[16] A. Smola, S. Mika, B. Schoelkopf, and R. C. Williamson. Regularized principal manifolds. Journal of Machine Learning, 1:179–209, 2001.
[17] M. Pontil T. Evgeniou and T. Poggio. Regularization networks and support vector machines. Advances in Computational Mathematics, 2000.
[18] J. B. Tenenbaum, V. de Silva, and J. C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500):2319–2323, 2000.
[19] H. Valpola and J. Karhunen. An unsupervised ensemble learning method for nonlinear dynamic statespace models. Neural Computation, 14(11):2647–2692, 2002.
[20] Lieven Vandenberghe, Stephen Boyd, and Shao-Po Wu. Determinant maximization with linear matrix inequality constraints. SIAM Journal on Matrix Analysis and Applications, 19(2):499–533, 1998.