jmlr jmlr2007 jmlr2007-87 jmlr2007-87-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: ZoltĂĄn SzabĂł, BarnabĂĄs PĂłczos, AndrĂĄs LĹ‘rincz
Abstract: We introduce the blind subspace deconvolution (BSSD) problem, which is the extension of both the blind source deconvolution (BSD) and the independent subspace analysis (ISA) tasks. We examine the case of the undercomplete BSSD (uBSSD). Applying temporal concatenation we reduce this problem to ISA. The associated ‘high dimensional’ ISA problem can be handled by a recent technique called joint f-decorrelation (JFD). Similar decorrelation methods have been used previously for kernel independent component analysis (kernel-ICA). More precisely, the kernel canonical correlation (KCCA) technique is a member of this family, and, as is shown in this paper, the kernel generalized variance (KGV) method can also be seen as a decorrelation method in the feature space. These kernel based algorithms will be adapted to the ISA task. In the numerical examples, we (i) examine how efÄ?Ĺš ciently the emerging higher dimensional ISA tasks can be tackled, and (ii) explore the working and advantages of the derived kernel-ISA methods. Keywords: undercomplete blind subspace deconvolution, independent subspace analysis, joint decorrelation, kernel methods
Karim Abed-Meraim and Adel Belouchrani. Algorithms for joint block diagonalization. In Proceedings of European Signal Processing Conference (EUSIPCO 2004), pages 209–212, 2004. Shotaro Akaho, Yasuhiko Kiuchi, and Shinji Umeyama. MICA: Multimodal independent component analysis. In Proceedings of International Joint Conference on Neural Networks (IJCNN ’99), pages 927–932, 1999. Ian F. Akyildiz, WellJan Su, Yogesh Sankarasubramaniam, and Erdal Cayirci. Wireless sensor networks: a survey. Computer Networks, 38(4):393–422, 2002. Shun-ichi Amari, Andrzej Cichocki, and Howard H. Yang. A new learning algorithm for blind signal separation. Advances in Neural Information Processing Systems, 8:757–763, 1996. Shoko Araki, Shoji Makino, Ryo Mukai, Tsuyoki Nishikawa, and Hiroshi Saruwatari. Fundamental limitation of frequency domain blind source separation for convolved mixture of speech. IEEE Transactions on Speech and Audio Processing, 11(2):109–116, 2003. Nachman Aronszajn. Theory of reproducing kernels. Transactions of the American Mathematical Society, 68:337–404, 1950. Francis R. Bach and Michael I. Jordan. Kernel independent component analysis. Journal of Machine Learning Research, 3:1–48, 2002. Francis R. Bach and Michael I. Jordan. Beyond independent components: Trees and clusters. Journal of Machine Learning Research, 4:1205–1233, 2003. Jean-Francois Cardoso. Multidimensional independent component analysis. In Proceedings of ¸ International Conference on Acoustics, Speech, and Signal Processing (ICASSP ’98), volume 4, pages 1941–1944, Seattle, WA, USA, 1998. Andrzej Cichocki and Shun-ichi Amari. Adaptive Blind Signal and Image Processing. John Wiley & Sons, 2002. Pierre Comon. Independent component analysis, a new concept? Signal Processing, 36:287–314, 1994. Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. John Wiley and Sons, New York, USA, 1991. Fani Deligianni, Benny Lo, and Guang-Zhong Yang. Source recovery for body sensor network. In Proceedings of International Workshop on Wearable and Implantable Body Sensor Networks 2006 (BSN 2006), pages 199–202, 2006. Scott C. Douglas, Hiroshi Sawada, and Shoji Makino. Natural gradient multichannel blind deconvolution and speech separation using causal FIR Ä?Ĺš lters. IEEE Transactions on Speech and Audio Processing, 13(1):92–104, 2005. Mads Dyrholm, Scott Makeig, and Lars Kai Hansen. Model selection for convolutive ICA with an application to spatio-temporal analysis of EEG. Neural Computation, apr 2007. 1092 U NDERCOMPLETE B LIND S UBSPACE D ECONVOLUTION Alan Edelman, Tomas Arias, and Steven T. Smith. The geometry of algorithms with orthogonality constraints. SIAM Journal on Matrix Analysis and Applications, 20(2):303–353, 1998. Kai-Tai Fang, Samuel Kotz, and Kai Wang Ng. Symmetric Multivariate and Related Distributions. Chapman and Hall, 1990. C´ dric F´ votte and Christian Doncarli. A uniÄ?Ĺš ed presentation of blind source separation for convoe e lutive mixtures using block-diagonalization. In Proceedings of Independent Component Analysis and Blind Signal Separation (ICA 2003), pages 349–354, Nara, Japan, 2003. Kenji Fukumizu, Francis R. Bach, and Arthur Gretton. Statistical consistency of kernel canonical correlation analysis. Journal of Machine Learning Research, 8:361–383, 2007. Gary H. Glover. Deconvolution of impulse response in event-related BOLD fMRI. NeuroImage, 9: 416–429, 1999. ¨ Arthur Gretton, Alexander Smola, Olivier Bousquet, and Bernhard Sch olkopf. Kernel methods for measuring independence. Journal of Machine Learning Research, 6:2075–2129, 2005. John B. Hedgepeth, Vincent F. Gallucci, F. O’Sullivan, and Richard E. Thorne. An expectation maximization and smoothing approach for indirect acoustic estimation of Ä?Ĺš sh size and density. ICES Journal of Marine Science, 56(1):36–50, 1999. Aapo Hyv¨ rinen and Patrik O. Hoyer. Emergence of phase and shift invariant features by decompoa sition of natural images into independent feature subspaces. Neural Computation, 12:1705–1720, 2000. Aapo Hyv¨ rinen, Juha Karhunen, and Erkki Oja. Independent Component Analysis. John Wiley & a Sons, 2001. Aapo Hyv¨ rinen and Urs K¨ ster. FastISA: A fast Ä?Ĺš xed-point algorithm for independent subspace a o analysis. In Proceedings of European Symposium on ArtiÄ?Ĺš cial Neural Networks (ESANN 2006), Bruges, Belgium, 2006. Aapo Hyv¨ rinen and Erkki Oja. A fast Ä?Ĺš xed-point algorithm for independent component analysis. a Neural Computation, 9(7):1483–1492, 1997. Tzyy-Ping Jung, Scott Makeig, Te-Won Lee, Martin J. McKeown, Glen Brown, Anthony J. Bell, and Terrence J. Sejnowski. Independent component analysis of biomedical signals. In Proceedings of International Workshop on Independent Component Analysis and Signal Separation (ICA 2000), pages 633–644, Helsinki, 2000. Christian Jutten and Jeanny Herault. Blind separation of sources: An adaptive algorithm based on neuromimetic architecture. Signal Processing, 24:1–10, 1991. Hakan KarslĂ„Ä…. Further improvement of temporal resolution of seismic data by autoregressive (AR) spectral extrapolation. Journal of Applied Geophysics, 59:324–336, 2006. Tuvia Kotzer, Nir Cohen, and Joseph Shamir. Generalized projection algorithms with applications to optics and signal restoration. Optics Communications, 156(1):77–91, 1998. 1093 ´ ´ Ă‹? S ZAB O , P OCZOS AND L ORINCZ Ross A. Lippert. Nonlinear Eigenvalue Problems. PhD thesis, Massachusetts Institute of Technology, 1998. Adam MacDonald and Stephen Cain. Derivation and application of an anisoplanatic optical transfer function for blind deconvolution of laser radar imagery. Unconventional Imaging, 5896:9–20, 2005. Nikolaos Mitianoudis and Michael E. Davies. Audio source separation of convolutive mixtures. IEEE Transactions on Speech and Audio Processing, 11(5):489–497, 2003. Yasunori Nishimori, Shotaro Akaho, and Mark D. Plumbley. Riemannian optimization method on the Ä?Ĺš‚ag manifold for independent subspace analysis. In Proceedings of Independent Component Analysis and Blind Signal Separation (ICA 2006), volume 3889 of LNCS, pages 295–302. Springer, 2006. ¨ Guido Nolte, Frank C. Meinecke, Andreas Ziehe, and Klaus-Robert M uller. Identifying interactions in mixed and noisy complex systems. Physical Review E, 73(051913), 2006. Michael S. Pedersen, Jan Larsen, Ulrik Kjems, and Lucas C. Parra. A survey of convolutive blind source separation methods. In Springer Handbook of Speech (to appear). Springer Press, sep 2007. URL http://www2.imm.dtu.dk/pubdb/p.php?4924. Mark D. Plumbley. Lie group methods for optimization with orthogonality constraints. In Proceedings of Independent Component Analysis and Blind Signal Separation (ICA 2004), volume 3195 of LNCS, pages 1245–1252. Springer, 2004. Barnab´ s P´ czos and Andr´ s LĂ‹? rincz. Independent subspace analysis using geodesic spanning trees. a o a o In Proceedings of International Conference on Machine Learning (ICML 2005), pages 673–680, Bonn, Germany, 2005a. Barnab´ s P´ czos and Andr´ s LĂ‹? rincz. Independent subspace analysis using k-nearest neighborhood a o a o distances. ArtiÄ?Ĺš cial Neural Networks: Formal Models and their Applications - ICANN 2005, pt 2, Proceedings, 3697:163–168, 2005b. Barnab´ s P´ czos and Andr´ s LĂ‹? rincz. Non-combinatorial estimation of independent autoregressive a o a o sources. Neurocomputing Letters, 69:2416–2419, 2006. Nicolas Quinquis, Isao Yamada, and Kohichi Sakaniwa. EfÄ?Ĺš cient dual Cayley parametrization technique for ICA with orthogonality constraints. In Proceedings of ICA Research Network International Workshop (ICARN 2006), pages 123–126, Liverpool, U.K., 2006. Ravikiran Rajagopal and Lee C. Potter. Multivariate MIMO FIR inverses. IEEE Transactions on Image Processing, 12:458 – 465, 2003. Michael J. Roan, Mark R. Gramann, Josh G. Erling, and Leon H. Sibul. Blind deconvolution applied to acoustical systems identiÄ?Ĺš cation with supporting experimental results. The Journal of the Acoustical Society of America, 114(4):1988–1996, 2003. Bernhard Sch¨ lkopf, Christopher J. C. Burges, and Alexander J. Smola. Advances in Kernel Metho ods - Support Vector Learning. MIT Press, Cambridge, MA, 1999. 1094 U NDERCOMPLETE B LIND S UBSPACE D ECONVOLUTION Harald St¨ gbauer, Alexander Kraskov, Sergey A. Astakhov, and Peter Grassberger. Least dependent o component analysis based on mutual information. Physical Review E - Statistical, Nonlinear, and Soft Matter Physics, 70(066123), 2004. Zolt´ n Szab´ and Andr´ s LĂ‹? rincz. Real and complex independent subspace analysis by generalized a o a o variance. In Proceedings of ICA Research Network International Workshop (ICARN 2006), pages 85–88, Liverpool, U.K., 2006. http://arxiv.org/abs/math.ST/0610438. Zolt´ n Szab´ , Barnab´ s P´ czos, and Andr´ s LĂ‹? rincz. Cross-entropy optimization for independent a o a o a o process analysis. In Proceedings of Independent Component Analysis and Blind Signal Separation (ICA 2006), volume 3889 of LNCS, pages 909–916. Springer, 2006a. Zolt´ n Szab´ , Barnab´ s P´ czos, and Andr´ s LĂ‹? rincz. Separation theorem for K-independent suba o a o a o ¨ o space analysis with sufÄ?Ĺš cient conditions. Technical report, E otv¨ s Lor´ nd University, Budapest, a 2006b. http://arxiv.org/abs/math.ST/0608100. Seiji Takano. The inequalities of Fisher information and entropy power for dependent variables. In Symposium on Probability Theory and Mathematical Statistics, 1995. Fabian J. Theis. Uniqueness of complex and multidimensional independent component analysis. Signal Processing, 84(5):951–956, 2004. Fabian J. Theis. Blind signal separation into groups of dependent signals using joint block diagonalization. In Proceedings of International Society for Computer Aided Surgery (ISCAS 2005), pages 5878–5881, Kobe, Japan, 2005a. Fabian J. Theis. Multidimensional independent component analysis using characteristic functions. In Proceedings of European Signal Processing Conference (EUSIPCO 2005), 2005b. Fabian J. Theis. Towards a general independent subspace analysis. In Proceedings of Neural Information Processing Systems (NIPS 2006), 2006. Roland Vollgraf and Klaus Obermayer. Multi-dimensional ICA to separate correlated sources. In Proceedings of Neural Information Processing Systems (NIPS 2001), volume 14, pages 993– 1000, 2001. Cabir Vural and William A. Sethares. Blind image deconvolution via dispersion minimization. Digital Signal Processing, 16:137–148, 2006. Grace Wahba. Support vector machines, reproducing kernel hilbert spaces, and randomized GACV. In Advances in Kernel Methods, pages 69–88. MIT Press, 1999. 1095