jmlr jmlr2011 jmlr2011-10 jmlr2011-10-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Lars Omlor, Martin A. Giese
Abstract: Blind source separation problems emerge in many applications, where signals can be modeled as superpositions of multiple sources. Many popular applications of blind source separation are based on linear instantaneous mixture models. If specific invariance properties are known about the sources, for example, translation or rotation invariance, the simple linear model can be extended by inclusion of the corresponding transformations. When the sources are invariant against translations (spatial displacements or time shifts) the resulting model is called an anechoic mixing model. We present a new algorithmic framework for the solution of anechoic problems in arbitrary dimensions. This framework is derived from stochastic time-frequency analysis in general, and the marginal properties of the Wigner-Ville spectrum in particular. The method reduces the general anechoic problem to a set of anechoic problems with non-negativity constraints and a phase retrieval problem. The first type of subproblem can be solved by existing algorithms, for example by an appropriate modification of non-negative matrix factorization (NMF). The second subproblem is solved by established phase retrieval methods. We discuss and compare implementations of this new algorithmic framework for several example problems with synthetic and real-world data, including music streams, natural 2D images, human motion trajectories and two-dimensional shapes. Keywords: blind source separation, anechoic mixtures, time-frequency transformations, linear canonical transform, Wigner-Ville spectrum
I. Ahmad and M. T. Ibrahim. Image classification and retrieval using correlation. In CRV ’06: Proceedings of the The 3rd Canadian Conference on Computer and Robot Vision, page 60, Washington, DC, USA, 2006. IEEE Computer Society. F. Aires, A. Chedin, and J.-P. Nadal. Independent component analysis of multivariate time series: Application to the tropical sst variability. Journal of Geophysical Research, 105:1743717455, 2000. M.S. Akbar and L.J. Douglas. Optimal kernels for nonstationary spectral estimation. IEEE Transactions on Signal Processing, 43(2):478–491, 1995. T. Alieva and M. J. Bastiaans. Properties of the linear canonical integral transformation. Journal of the Optical Society America A, 24(11):3658–3665, 2007. 1141 O MLOR AND G IESE J. B. Anthony and T. J. Sejnowski. An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7(6):1129–1159, 1995. S. Arberet, R. Gribonval, and F. Bimbot. A robust method to count and locate audio sources in a stereophonic linear anechoic mixture. In IEEE International Conference on Acoustics, Speech and Signal Processing. ICASSP 2007., volume 3, pages III–745 –III–748, 15-20 2007. L. Bar, N. Sochen, and N. Kiryati. Image deblurring in the presence of salt-and-pepper noise. Scale Space and PDE Methods in Computer Vision, pages 107–118, 2005. R.G. Baraniuk. Wigner-ville spectrum estimation via wavelet soft-thresholding. In Time-Frequency and Time-Scale Analysis, 1994., Proceedings of the IEEE-SP International Symposium on, pages 452 –455, October 1994. doi: 10.1109/TFSA.1994.467316. A. Barliya, L. Omlor, M.A. Giese, and T. Flash. An analytical formulation of the law of intersegmental coordination during human locomotion. Experimental Brain Research, 193(3):371–385, 2009. M. J. Bastiaans and K. B. Wolf. Phase reconstruction from intensity measurements in linear systems. Journal of the Optical Society America A, 20(6):1046–1049, 2003. E. Be’ery and A. Yeredor. Blind separation of superimposed shifted images using parameterized joint diagonalization. IEEE Transactions on Image Processing, 17(3):340–353, 2008. R. E. Bellman. Dynamic Programming. Princeton University Press, Princeton, NJ., 1957. A. Belouchrani and M.G. Amin. Blind source separation based on time-frequency signal representations. IEEE Transactions on Signal Processing, 46(11):2888–2897, 1998. S. Bjorklund and L. Ljung. A review of time-delay estimation techniques. In Decision and Control, 2003. Proceedings. 42nd IEEE Conference on, volume 3, pages 2502 – 2507, 2003. doi: 10. 1109/CDC.2003.1272997. P. Bofill. Underdetermined blind separation of delayed sound sources in the frequency domain. Neurocomputing, 55:627–641, 2003. S. Bounkong, B. Toch, D. Saad, and D. Lowe. Ica for watermarking digital images. Journal of Machine Learning Research, 4(7-8):1471–1498, 2004. A. Bultheel and H. Mart´nez-Sulbaran. Recent developments in the theory of the fractional Fourier ı transforms and linear canonical transforms. Bulletin of the Belgian Mathematical Society- Simon Stevin, 13:971–1005, 2006. J.F. Cardoso. Blind singal separation: statistical principles. Proceedings of the IEEE, 9(10):2009– 2025, 1998. G. Carter. Time delay estimation for passive sonar signal processing. IEEE Transactions on Acoustics, Speech and Signal Processing, 29(3):463–470, 1981. T. Chau. A review of analytical techniques for gait data. part 1: fuzzy, statistical and fractal methods. Gait & Posture, 13(1):49–66, 2001. 1142 A NECHOIC B LIND S OURCE S EPARATION U SING W IGNER M ARGINALS J. Chen and X.Z. Wang. A new approach to near-infrared spectral data analysis using independent component analysis. Journal of Chemical Information and Computer Sciences, 41(4):992–1001, 2001. J. Chen, J. Benesty, and Y. Huang. Time delay estimation in room acoustic environments: an overview. EURASIP Journal of Applied Signal Processing, pages 170–170, 2006. Z. Chen and A. Cichocki. Nonnegative matrix factorization with temporal smoothness and/or spatial decorrelation constraints. In Laboratory for Advanced Brain Signal Processing, RIKEN, Tech. Rep, 2005. E. C. Cherry. Some experiments on the recognition of speech, with one and with two ears. Journal of Acoustical Society of America, 25(5):975–979, 1953. S. Choi, A. Cichocki, H.H. Park, and S.Y. Lee. Blind signal separation and independent component analysis: A review. Neural Information Processing – Letters and Reviews, 5:1–57, 2005. A. Cichocki and S. Amari. Adaptive blind signal and image processing. John Wiley, Chichester, 2002. A. Cichocki, H. Lee, Y.D. Kim, and S. Choi. Non-negative matrix factorization with alphadivergence. Pattern Recognition Letters, 29(9):1433–1440, 2008. L. Cohen. Time-frequency distributions-a review. Proceedings of the IEEE, 77(7):941–981, 1989. P. Comon and C. Jutten. Handbook of Blind Source Separation, Independent Component Analysis and Applications. Academic Press, 2010. W. X. Cong, N. X. Chen, and B. Y. Gu. Recursive algorithm for phase retrieval in the fractional fourier transform domain. Applied Optics, 37(29):6906–6910, 1998. A. d’Avella and E. Bizzi. Shared and specific muscle synergies in natural motor behaviors. Proc Natl Acad Sci U S A, 102(8):3076–81, 2005. A. d’Avella, L. Fernandez, A. Portone, and F. Lacquaniti. Modulation of Phasic and Tonic Muscle Synergies With Reaching Direction and Speed. J Neurophysiol, 100(3):1433–1454, 2008. D.B. David B. Kirk and W. W. Hwu. Programming Massively Parallel Processors: A Hands-on Approach. Morgan Kaufman Publishers, San Fransisco, CA, 2010. C. Ding, T. Li, W. Peng, and H. Park. Orthogonal nonnegative matrix tri-factorizations for clustering. In Proceedings of the Twelfth ACM SIGKDD international conference on knowledge discovery and data mining, pages 126–135, 2006. B.A. Draper, K. Baek, M.S. Bartlett, and J.R. Beveridge. Recognizing faces with PCA and ICA. Computer Vision and Image Understanding, 91(1-2):115–137, 2003. W.X. Du, C.H. Thurber, and D. Eberhart-Phillips. Earthquake relocation using cross-correlation time delay estimates verified with the bispectrum method. Bulletin of the Seismological Society of America, 94(3):856–866, 2004. 1143 O MLOR AND G IESE B. Emile and P. Comon. Estimation of time delays between unknown colored signals. Signal Processing, 69(1):93–100, 1998. T. Flash and B. Hochner. Motor primitives in vertebrates and invertebrates. Current Opinion in Neurobiology, 15 (6):660–666, 2005. P. Georgiev, F. Theis, and A. Cichocki. Sparse component analysis and blind source separation of underdetermined mixtures. IEEE Transactions on Neural Networks, 16(4):992 –996, 2005. R. W. Gerchberg and W. O. Saxton. A practical algorithm for the determination of phase from image and diffraction plane pictures. Optik, 35:237–246, 1972. J. Gluckman. Higher order whitening of natural images. Computer Vision and Pattern Recognition, IEEE Computer Society Conference on, 2:354–360, 2005. R. A. Harshman, S. Hong, and M. E. Lundy. Shifted factor analysis–part i: Models and properties. Journal of Chemometrics, 17:363–378, 2003. J.J. Healy, P. O’Grady, and J.T. Sheridan. Simulating paraxial optical systems using the linear canonical transform: properties, issues and applications. Optics and Photonics for Information Processing II, 7072(1):70720E, 2008. F. Hlawatsch and W. Kozek. The Wigner distribution of a linear signal space. IEEE Transactions on Signal Processing, 41(3):1248–1258, 1993. P. Hoyer and E. Oja. Image denoising by sparse code shrinkage. In Intelligent Signal Processing. IEEE Press, 2000. W. Hu and L.M. Collins. Classification of closely spaced subsurface objects using electromagnetic induction data and blind source separation algorithms. Radio Science, 39:1–13, 2004. W. Ilg, G. H. Bakir, J. Mezger, and M.A. Giese. On the representation, learning and transfer of spatio-temporal movement characteristics. International Journal of Humanoid Robotics, 1:613– 636, 2004. A. J. E. M. Janssen. Application of the Wigner Distribution to Harmonic Analysis of Generalized Stochastic Processes. Mathematical Centre Tracts 114, Amsterdam, 1979. O. C. Jenkins and M. J. Mataric. Deriving action and behavior primitives from human motion data. Intelligent Robots and System, 2002. IEEE/RSJ International Conference on, 3:2551–2556, 2002. S. Karako-Eilon, A. Yeredor, and D. Mendlovic. Blind source separation based on the fractional fourier transform. Proc. 4th Int. Symp. on Independent Component Analysis, 46(11):615–620, 2003. D. D. Lee and H. S. Seung. Learning the parts of objects by non-negative matrix factorization. Nature, 401(6755):788–791, 1999. C.-T. Leung and W.-C. Siu. A general contrast function based blind source separation method for convolutively mixed independent sources. Signal Processing, 87(1):107–123, 2007. 1144 A NECHOIC B LIND S OURCE S EPARATION U SING W IGNER M ARGINALS Y. Li, A. Cichocki, and S. Amari. Analysis of sparse representation and blind source separation. Neural Computation, 16(6):1193–1234, 2004. Y. Lin and D.D. Lee. Bayesian regularization and nonnegative deconvolution for time delay estimation. In in Advances in Neural Information Processing Systems 17, pages 809–816. MIT Press, 2005. H. Ling and D.W. Jacobs. Shape classification using the inner-distance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(2):286–299, 2007. W. Martin. Time-frequency analysis of random signals. In Proc. IEEE ICASSP, pages 1325–1328, Paris, 1982. W. Martin and P. Flandrin. Wigner-Ville spectral analysis of nonstationary processes. IEEE Transactions on Acoustics, Speech, and. Signal Processing, 33(6):1461–1470, 1985. K. Matsuoka. Minimal distortion principle for blind source separation. In SICE 2002. Proceedings of the 41st SICE Annual Conference, volume 4, pages 2138 – 2143, 2002. doi: 10.1109/SICE. 2002.1195729. G. Matz and F. Hlawatsch. Wigner distributions (nearly) everywhere: time-frequency analysis of signals, systems, random processes, signal spaces, and frames. Signal Processing, 83(7):1355– 1378, 2003. W. Mecklenbruker and F. Hlawatsch. The Wigner Distribution. Elsevier Science, 1997. N. Mohanty, T. Rath, A. Lee, and R. Manmatha. Learning shapes for image classification and retrieval. In W.-K. Leow, M. Lew, T. S. Chua, W.-Y. Ma, L. Chaisorn, and E. Bakker, editors, Image and Video Retrieval, volume 3568 of Lecture Notes in Computer Science, pages 589–598. Springer Berlin / Heidelberg, 2005. M. Mørup, K. H. Madsen, and L. K. Hansen. Shifted independent component analysis. In M. E. Davies, C. J. James, S. A. Abdallah, and M. D. Plumbley, editors, ICA2007, pages 89–96. Springer, Berlin, 2007. M. Mørup, L.K. Hansen, S.M. Arnfred, L. Lim, and K.H. Madsen. Shift invariant multilinear decomposition of neuroimaging data. NeuroImage, 42(4):1439–50, 2008. A. Mukovskiy, A. Park, L. Omlor, and M.A. Giese. Self-organization of character behavior by mixing of learned movement primitives. In Proceedings of the 13th Fall Workshop on Vision, Modeling, and Visualization (VMV), Konstanz, Germany, 2008. C. Namgook and C.-C. Jay Kuo. Underdetermined audio source separation from anechoic mixtures with long time delay. In Proceedings of the 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP ’09, pages 1557–1560, Washington, DC, USA, 2009. IEEE Computer Society. D. Nuzillard and A. Bijaoui. Blind source separation and analysis of multispectral astronomical images. Astronomy and Astrophysics Suppl. Series, 147:129–138, 2000. 1145 O MLOR AND G IESE L. Omlor and M. A. Giese. Learning of translation-invariant independent components: Multivariate anechoic mixtures. In M. E. Davies, C. J. James, S. A. Abdallah, and M. D. Plumbley, editors, ICA 2007, pages 762–769. Springer, Berlin, 2007a. L. Omlor and M. A. Giese. Blind source separation for over-determined delayed mixtures. In B. Sch¨ lkopf, J. Platt, and T. Hoffman, editors, Advances in Neural Information Processing Syso tems 19, pages 1049–1056. MIT Press, Cambridge, MA, 2007b. S. J. Osher and R. P. Fedkiw. Level Set Methods and Dynamic Implicit Surfaces. Springer, 2002. H. M. Ozaktas, Z. Zalevsky, and M. A. Kutay. The Fractional Fourier Transform with Applications in Optics and Signal Processing. John Wiley & Sons, 2001. P. O. Ogrady, B. A. Pearlmutter, and S. T. Rickard. Survey of sparse and non-sparse methods in source separation. International Journal of Imaging Systems and Technology, 15:2005, 2005. A. Papoulis, P.S. Unnikrishna, and S. Unnikrishna. Probability, Random Variables, and Stochastic Processes. McGraw Hill Higher Education, 2001. A. Park, A. Mukovskiy, L. Omlor, and M.A. Giese. Self organized character animation based on learned synergies from full-body motion capture data. In International Conference on Cognitive Systems (CogSys), Karlsruhe, Germany, 2008. M. S. Pedersen, J. Larsen, U. Kjems, and L. C. Parra. A survey of convolutive blind source separation methods. In Springer Handbook of Speech Processing, pages 1065–1094. Springer Verlag, Berlin, 2007. J.C. Platt and F. Faggin. Networks for the separation of sources that are superimposed and delayed. Advances in Neural Information Processing Systems, 4:730–737, 1992. M.D. Plumbley and E. Oja. A ”nonnegative pca” algorithm for independent component analysis. IEEE Transactions on Neural Networks, 15(1):66–76, 2004. L. R. Rabiner and B. Gold. Theory and Application of Digital Signal P’rocessing. Englewood Cliffs, NJ : Prentice-Hall, 1975. K. Raja Rajeswari and D.E. Rani. Time-delay estimation using mle approach for wide-band radar systems. Fourth International Conference on Signal Processing,Proceedings,ICSP ’98., 2:1493– 1496, 1998. S. T. Roweis and L. K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 5500(290):2268–2269, 2000. A. Safonova, J. K. Hodgins, and N. S. Pollard. Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces. ACM Transactions on Graphics, 23(3):514–521, 2004. T. B. Sebastian, P. N. Klein, and B. B. Kimia. Recognition of shapes by editing shock graphs. In Proceedings of ICCV, pages 550 – 571, 2001. 1146 A NECHOIC B LIND S OURCE S EPARATION U SING W IGNER M ARGINALS K. Seki, M. Narusawa, and P. Smaragdis. Blind separation of convolved mixtures in the frequency domain - signal processing. Neurocomputing, 22(14):21–34, 1998. K. K. Sharmaa and S. D. Joshi. Signal separation using linear canonical and fractional fourier transforms. Optics Communications, 265:454–460, 2006. P. Smaragdis, B. Raj, and M. V. S. Shashanka. Supervised and semi-supervised separation of sounds from single-channel mixtures. In M. E Davies, C. J. James, S. A. Abdallah, and M. D. Plumbley, editors, ICA, volume 4666 of Lecture Notes in Computer Science, pages 414–421. Springer, 2007. A. Swindelhurst. Time delay and spatial signature estimation using known asynchronous signals. IEEE Transactions on Signal Processing, ASSP-33,no. 6:1461–1470, 1998. T. Takeuchi, M. Sako, and S. Yoshida. Multipath delay estimation for indoor wireless communication. Vehicular Technology Conference, 1990 IEEE 40th, pages 401–406, 1990. K. Torkkola. Blind separation of convolved sources based on information maximization. In In IEEE Workshop on Neural Networks for Signal Processing, pages 423–432, 1996a. K. Torkkola. Blind separation of delayed sources based on information maximization. IEEE International Conference on Acoustics, Speech, and Signal Processing, 6:3509–3512, 1996b. M.C. Tresch, V.C. Cheung, and A. d’Avella. Matrix factorization algorithms for the identification of muscle synergies: evaluation on simulated and experimental data sets. Journal of Neurophysiology, 95:2199–2212, 2006. R. Veltkamp and M. Hagedoorn. State-of-the-art in shape matching. Technical Report UU-CS1999-27, Utrecht University, the Netherlands, 1999. J. Ville. Th´ orie et applications de la notion de signal analytique. Cˆ bles et Trans., 2:61–74, 1948. e a R. Vollgraf, M. Stetter, and K. Obermayer. Convolutive decorrelation procedures for blind source separation. In In Proceedings of the 2. International Workshop on, pages 515–520, 2000. N. Wiener. Extrapolation, Interpolation, and Smoothing of Stationary Time Series. MIT Press, Cambridge, MA, 1964. E. Wigner. On the quantum correction for thermodynamic equilibrium. Physical Review, 40(5): 749–759, 1932. J. Xiao and P. Flandrin. Multitaper time-frequency reassignment for nonstationary spectrum estimation and chirp enhancement. IEEE Transactions on Signal Processing, 55(6):2851–2860, 2007. S. Yang and Z. Yi. Nonnegative matrix factorization for independent component analysis. In International Conference on Communications, Circuits and Systems., pages 769–771, 2007. A. Yeredor. Time-delay estimation in mixtures. Acoustics, Speech, and Signal Processing, 5:237– 40, 2003. 1147 O MLOR AND G IESE O. Yilmaz and S. Rickard. Blind separation of speech mixtures via time-frequency masking. IEEE Transactions on Signal Processing, 52(7):1830–1847, 2004. Z. Zalevsky, D. Mendlovic, and R. G. Dorsch. Gerchberg-Saxton algorithm applied in the fractional Fourier or the Fresnel domain. Optics Letters, 21:842–844, 1996. D. S. Zhang and G. Lu. Review of shape. representation and description techniques. Pattern Recognition, 37(1):1–19, 2004. 1148