jmlr jmlr2006 jmlr2006-57 jmlr2006-57-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Rasmus Kongsgaard Olsson, Lars Kai Hansen
Abstract: We apply a type of generative modelling to the problem of blind source separation in which prior knowledge about the latent source signals, such as time-varying auto-correlation and quasiperiodicity, are incorporated into a linear state-space model. In simulations, we show that in terms of signal-to-error ratio, the sources are inferred more accurately as a result of the inclusion of strong prior knowledge. We explore different schemes of maximum-likelihood optimization for the purpose of learning the model parameters. The Expectation Maximization algorithm, which is often considered the standard optimization method in this context, results in slow convergence when the noise variance is small. In such scenarios, quasi-Newton optimization yields substantial improvements in a range of signal to noise ratios. We analyze the performance of the methods on convolutive mixtures of speech signals. Keywords: blind source separation, state-space model, independent component analysis, convolutive model, EM, speech modelling
J. Allen and D. A. Berkley. Image method for efficiently simulating small-room acoustics. Journal of the Acoustical Society of America, 65:943–950, 1979. J. Anem¨ ller and B. Kollmeier. Amplitude modulation decorrelation for convolutive blind source u separation. In Proc. ICA 2000, pages 215–220, 2000. A J. Bell and T. J. Sejnowski. An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7(6):1129–1159, 1995. O. Bermond and J.-F. Cardoso. Approximate likelihood for noisy mixtures. In Proc. ICA, pages 325–330, 1999. M. Brandstein. On the use of explicit speech modeling in microphone array applications. In Proc. ICASSP, pages 3613–3616, 1998. J.-F. Cardoso, H. Snoussi, J. Delabrouille, and G. Patanchon. Blind separation of noisy Gaussian stationary sources. Application to cosmic microwave background imaging. In Proc. EUSIPCO, pages 561–564, 2002. J. F. Cardoso and A. Souloumiac. Blind beamforming for non-gaussian signals. IEE Proceedings F, 140(6):362–370, 1993. P. Comon. Independent component analysis, a new concept? Signal processing, 36(3):287–314, 1994. 2600 L INEAR S TATE -S PACE M ODELS FOR B LIND S OURCE S EPARATION A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of Royal Statistics Society, Series B, 39:1–38, 1977. M. Dyrholm and L. K. Hansen. CICAAR: Convolutive ICA with an auto-regressive inverse model. In Proc. ICA 2004, pages 594–601, 2004. P. A. Højen-Sørensen, Ole Winther, and Lars Kai Hansen. Mean-field approaches to independent component analysis. Neural Computation, 14(4):889–918, 2002. A. Hyvarinen, J. Karhunen, and E. Oja. Independent Component Analysis. John Wiley & Sons, Inc, 2001. R. E. Kalman and R. S. Bucy. New results in linear filtering and prediction theory. Journal of Basic Engineering ASME Transactions, 83:95–107, 1960. T.W. Lee, A. J. Bell, and R. H. Lambert. Blind separation of delayed and convolved sources. In Advances of Neural Information Processing Systems, volume 9, page 758, 1997. R.J. McAulay and T.F. Quateri. Speech analysis/synthesis based on a sinusoidal representation. IEEE Trans. on Acoustics, Speech and Signal Processing, 34(4):744–754, 1986. M. McKeown, L.K. Hansen, and T.J. Sejnowski. Independent component analysis for fmri: What is signal and what is noise? Current Opinion in Neurobiology, 13(5):620–629, 2003. E. Moulines, J. Cardoso, and E. Cassiat. Maximum likelihood for blind separation and deconvolution of noisy signals using mixture models. In Proc. ICASSP, volume 5, pages 3617–3620, 1997. H. B. Nielsen. UCMINF - an algorithm for unconstrained nonlinear optimization. Technical Report IMM-Rep-2000-19, Technical University of Denmark, 2000. R. K. Olsson and L. K. Hansen. A harmonic excitation state-space approach to blind separation of speech. In Advances in Neural Information Processing Systems, volume 17, pages 993–1000, 2005. R. K. Olsson and L. K. Hansen. Blind separation of more sources than sensors in convolutive mixtures. In International Conference on Acoustics, Speech and Signal Processing, 2006. R. K. Olsson, K. B. Petersen, and T. Lehn-Schiøler. State-space models - from the EM algorithm to a gradient approach. Neural Computation - to appear, 2006. L. Parra and C. Spence. Convolutive blind separation of non-stationary sources. IEEE Transactions, Speech and Audio Processing, pages 320–7, 5 2000. B. A. Pearlmutter and L. C. Parra. A context-sensitive generalization of ICA. In In Advances in Neural Information Processing Systems 9, pages 613–619, 1997. H. E. Rauch, F. Tung, and C. T. Striebel. Maximum likelihood estimates of linear dynamic systems. AIAA Journal, 3(8):1445–1450, 1965. 2601 O LSSON AND H ANSEN H. Robbins and S. Monro. A stochastic approximation method. Annals of Mathematical Statistics, 22(3):400–407, 1951. S. Roweis and Z. Ghahramani. A unifying review of linear Gaussian models. Neural Computation, 11:305–345, 1999. R. Salakhutdinov and S. Roweis. Adaptive overrelaxed bound optimization methods. In International Conference on Machine Learning, pages 664–671, 2003. R. Salakhutdinov, S. T. Roweis, and Z. Ghahramani. Optimization with EM and ExpectationConjugate-Gradient. In International Conference on Machine Learning, volume 20, pages 672– 679, 2003. R. Shumway and D. Stoffer. An approach to time series smoothing and forecasting using the EM algorithm. J. Time Series Anal., 3(4):253–264, 1982. E. Weinstein, M. Feder, and A. V. Oppenheim. Multi-channel signal separation by decorrelation. IEEE Transactions on Speech and Audio Processing, 1(4), 1993. D. Yellin and E. Weinstein. Multichannel signal separation: Methods and analysis. IEEE Transactions on Signal Processing, 44(1):106–118, 1996. 2602