nips nips2001 nips2001-103 nips2001-103-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Stefan Harmeling, Andreas Ziehe, Motoaki Kawanabe, Klaus-Robert Müller
Abstract: In kernel based learning the data is mapped to a kernel feature space of a dimension that corresponds to the number of training data points. In practice, however, the data forms a smaller submanifold in feature space, a fact that has been used e.g. by reduced set techniques for SVMs. We propose a new mathematical construction that permits to adapt to the intrinsic dimension and to find an orthonormal basis of this submanifold. In doing so, computations get much simpler and more important our theoretical framework allows to derive elegant kernelized blind source separation (BSS) algorithms for arbitrary invertible nonlinear mixings. Experiments demonstrate the good performance and high computational efficiency of our kTDSEP algorithm for the problem of nonlinear BSS.
[1] A. Belouchrani, K. Abed Meraim, J.-F. Cardoso, and E. Moulines. A blind source separation technique based on second order statistics. IEEE Trans. on Signal Processing, 45(2):434–444, 1997.
[2] G. Burel. Blind separation of sources: a nonlinear neural algorithm. 5(6):937–947, 1992. Neural Networks,
[3] C.J.C. Burges. A tutorial on support vector machines for pattern recognition. Knowledge Discovery and Data Mining, 2(2):121–167, 1998.
[4] J.-F. Cardoso. Blind signal separation: statistical principles. 9(10):2009–2025, 1998. Proceedings of the IEEE,
[5] J.-F. Cardoso and A. Souloumiac. Jacobi angles for simultaneous diagonalization. SIAM J.Mat.Anal.Appl., 17(1):161 ff., 1996.
[6] N. Cristianini and J. Shawe-Taylor. An Introduction to Support Vector Machines. Cambridge University Press, Cambridge, UK, 2000.
[7] C. Fyfe and P. L. Lai. ICA using kernel canonical correlation analysis. In Proc. Int. Workshop on Independent Component Analysis and Blind Signal Separation (ICA2000), pages 279–284, Helsinki, Finland, 2000.
[8] A. Hyvarinen, J. Karhunen, and E. Oja. Independent Component Analysis. Wiley, 2001.
[9] T.-W. Lee, B.U. Koehler, and R. Orglmeister. Blind source separation of nonlinear mixing models. In Neural Networks for Signal Processing VII, pages 406–415. IEEE Press, 1997.
[10] J. K. Lin, D. G. Grier, and J. D. Cowan. Faithful representation of separable distributions. Neural Computation, 9(6):1305–1320, 1997.
[11] G. Marques and L. Almeida. Separation of nonlinear mixtures using pattern repulsion. In Proc. Int. Workshop on Independent Component Analysis and Signal Separation (ICA’99), pages 277– 282, Aussois, France, 1999.
[12] K.-R. Müller, S. Mika, G. Rätsch, K. Tsuda, and B. Schölkopf. An introduction to kernel-based learning algorithms. IEEE Transactions on Neural Networks, 12(2):181–201, 2001.
[13] P. Pajunen, A. Hyvärinen, and J. Karhunen. Nonlinear blind source separation by selforganizing maps. In Proc. Int. Conf. on Neural Information Processing, pages 1207–1210, Hong Kong, 1996.
[14] P. Pajunen and J. Karhunen. A maximum likelihood approach to nonlinear blind source separation. In Proceedings of the 1997 Int. Conf. on Artificial Neural Networks (ICANN’97), pages 541–546, Lausanne, Switzerland, 1997.
[15] B. Schölkopf, S. Mika, C.J.C. Burges, P. Knirsch, K.-R. Müller, G. Rätsch, and A.J. Smola. Input space vs. feature space in kernel-based methods. IEEE Transactions on Neural Networks, 10(5):1000–1017, September 1999.
[16] B. Schölkopf, A.J. Smola, and K.-R. Müller. Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10:1299–1319, 1998.
[17] A. Taleb and C. Jutten. Source separation in post-nonlinear mixtures. IEEE Trans. on Signal Processing, 47(10):2807–2820, 1999.
[18] H. Valpola, X. Giannakopoulos, A. Honkela, and J. Karhunen. Nonlinear independent component analysis using ensemble learning: Experiments and discussion. In Proc. Int. Workshop on Independent Component Analysis and Blind Signal Separation (ICA2000), pages 351–356, Helsinki, Finland, 2000.
[19] V.N. Vapnik. The nature of statistical learning theory. Springer Verlag, New York, 1995.
[20] H. H. Yang, S.-I. Amari, and A. Cichocki. Information-theoretic approach to blind separation of sources in non-linear mixture. Signal Processing, 64(3):291–300, 1998.
[21] A. Ziehe and K.-R. Müller. TDSEP—an efficient algorithm for blind separation using time structure. In Proc. Int. Conf. on Artificial Neural Networks (ICANN’98), pages 675–680, Skövde, Sweden, 1998.