jmlr jmlr2011 jmlr2011-61 jmlr2011-61-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Lu Ren, Lan Du, Lawrence Carin, David Dunson
Abstract: A logistic stick-breaking process (LSBP) is proposed for non-parametric clustering of general spatially- or temporally-dependent data, imposing the belief that proximate data are more likely to be clustered together. The sticks in the LSBP are realized via multiple logistic regression functions, with shrinkage priors employed to favor contiguous and spatially localized segments. The LSBP is also extended for the simultaneous processing of multiple data sets, yielding a hierarchical logistic stick-breaking process (H-LSBP). The model parameters (atoms) within the H-LSBP are shared across the multiple learning tasks. Efficient variational Bayesian inference is derived, and comparisons are made to related techniques in the literature. Experimental analysis is performed for audio waveforms and images, and it is demonstrated that for segmentation applications the LSBP yields generally homogeneous segments with sharp boundaries. Keywords: Bayesian, nonparametric, dependent, hierarchical models, segmentation
T. Ahonen and M. Pietik¨ inen. Image discription using joint distribution of filter bank responses. a Pattern Recognition Letters, 30:368–376, 2009. Q. An, C. Wang, I. Shterev, E. Wang, L. Carin, and D. B. Dunson. Hierarchical kernel Stickbreaking process for multi-task image analysis. In Proceedings of the International Conference of Machine Learning, 2008. 236 L OGISTIC S TICK -B REAKING P ROCESS C. E. Antoniak. Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. Ann. Statist., 2(6):1152–1174, 1974. S. Banerjee, B. P. Carlin, and A. E. Gelfand. Hierarchical Modeling and Analysis for Spatial Data. Chapman and Hall/CRC, 1st edition, 2003. K. Barnard, P. Duygulu, D. Forsyth, N. D. Freitas, D. M. Blei, and M. I. Jordan. Matching words and pictures. Journal of Machine Learning Research, 3:1107–1135, 2003. M. J. Beal. Variational Algorithms for Approximate Bayesian Inference. PhD thesis, The Gatsby Computational Neuroscience Unit, University College London, 2003. M. J. Beal, Z. Ghahramani, and C. E. Rasmussen. The infinite hidden Markov model. In Proceedings of the Advances in Neural Information Processing Systems 14, 2002. M. Ben, M. Betser, F. Bimbot, and G. Gravier. Speaker diarization using bottom-up clustering based on a parameter-derived distance between adapted GMMs. In Proceedings of the International Conference on Spoken Language Processing, 2004. C. M. Bishop and M. Svens´ n. Bayesian hierarchical mixture of experts. In Proceedings of the e Ninetheenth Conference of Uncertainty in Artificial Intelligence, 2003. C. M. Bishop and M. E. Tipping. Variational relevance vector machines. In Proceedings of the sixth Conference of Uncertainty in Artificial Intelligence, 2000. D. Blackwell and J. B. MacQueen. Ferguson distributions via Polya urn schemes. Ann. Statist., 1 (2):353–355, 1973. D. M. Blei, A. Y. Ng, M. I. Jordan, and J. Lafferty. Latent Dirichlet allocation. Journal of Machine Learning Research, 3:993–1022, 2003. L. Cao and F. Li. Spatially coherent latent topic model for concurrent object segmentation and classification. In Proceedings of ICCV, 2007. R. Caruana. Multitask learning. Kluwer Academic Publishers, 1997. Y. Chung and D. B. Dunson. Nonparametric Bayes conditional distribution modeling with variable selection. Journal of the American Statistical Association, 2009. T. Cour, F. Benezit, and J. Shi. Spectral segmentation with multiscale graph decomposition. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, 2005. J. A. Duan, M. Guindani, and A. E. Gelfand. Biometrika, 94(4):809–825, 2007. Generalized spatial Dirichlet process models. D. B. Dunson and J. Park. Kernel Stick-breaking processes. Biometrika, 95:307–323, 2007. M. D. Escobar and M. West. Bayesian density estimation and inference using mixtures. Journal of the American Statistical Association, 90(430):577–588, 1995. 237 R EN , D U , C ARIN AND D UNSON P. F. Felzenszwalb and D. P. Huttenlocher. Efficient graph-based image segmentation. International Journal of Computer Vision, 59:167–181, 2004. E. B. Fox, E. B. Sudderth, M. I. Jordan, and A. S. Willsky. The sticky HDP-HMM for systems with state persistence. In Proceedings of the International Conference on Machine Learning, 2008. T. Ganchev, N. Fakotakis, and G. Kokkinakis. Comparative evaluation of various MFCC implementations on the speaker verification task. In Proceedings of the International Conference on Speech and Computer, 2005. A. Gersho and R. M. Gray. Vector Quantization and Signal Compression. Springer, 1991. W. R. Gilks, S. Richardson, and D. Spiegelhalter. Markov Chain Monte Carlo in Practice. Chapman and Hall/CRC, 1st edition, 1998. R. Gomes, M. Welling, and P. Perona. Memory bounded inference in topic models. In Proceedings of the International Conference of Machine Learning, 2008. H. Ishwaran and L. F. James. Gibbs sampling methods for Stick-breaking priors. Journal of the American Statistical Association, 96(453):161–173, 2001. H. Ishwaran and J.S. Rao. Spike and slab variable selection: Frequentist and Bayesian strategies. Annals of Statistics, 2005. M. Meil˘ . Comparing clusterings by the variation of information. In Proceedings of the Sixteenth a Annual Conference ofn Computational Learning Theory, 2003. G. Mori. Guiding model search using segmentation. In Proceedings of the International Conference on Computer Vision, 2005. X. Nguyen and A. E. Gelfand. The Dirichlet labeling process for functional data analysis. Technical Report T.R. 08-37, Dept. of Statistical Science, Duke University, 2008. P. Orbanz and J. M. Buhmann. Nonparametric Bayesian image segmentation. International Journal of Computer Vision, 77:25–45, 2008. C. Pantofaru, C. Schmid, and M. Hebert. Object recognition by integrating multiple image segmentations. In ECCV, 2008. J. Park and D. B. Dunson. Bayesian generalized product partition model. Statistica Sinica, 2009. S. Petrone, M. Guindani, and A. E. Gelfand. Hybrid Dirichlet mixture models for functional data. Journal Royal Statistical Society, Ser. B, 2009. C. E. Rasmussen. The infinite Gaussian mixture model. In Proceedings of the Advances in Neural Information Processing Systems 12, 2000. X. Ren and J. Malik. Learning a classification model for segmentation. In Proceedings of the International Conference on Computer Vision, 2003. J. Sethuraman. A constructive definition of Dirichlet priors. Statistica Sinica, 2:639–650, 1994. 238 L OGISTIC S TICK -B REAKING P ROCESS G. Sfikas, C. Nikou, and N. P.Galatsanos. Robust image segmentation with mixtures of student’s t-distributions. In Proceedings of the IEEE International conference on Image Processing, 2007. G. Sfikas, C. Nikou, and N. P.Galatsanos. Edge preserving spatially varying mixtures for image segmentaiton. In Proceedings of the International conference on Computer Vision and Pattern recognition, 2008. J. Shi and J. Malik. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22:888–905, 2000. E. B. Sudderth and M. I. Jordan. Shared segmentation of natural scenes using dependent Pitman-Yor processes. In Proceedings of the Neural Information Processing Systems, 2008. Y. W. Teh, M. I. Jordan, M. J. Beal, and D. M. Blei. Hierarchical Dirichlet processes. Journal of the American Statistical Association, 101:1566–1582, 2005. M. E. Tipping. Sparse Bayesian learning and the relevance vector machine. Journal of Machine Learning Research, 1:211–244, 2001. S. Tranter and D. Reynolds. An overview of automatic speaker diarisation systems. IEEE Trans. on Audio, Speech, and Language Processing, 14:1557–1565, 2006. R. Unnikrishnan, C. Pantofaru, and M. Hebert. Toward objective evaluation of image segmentation algorithms. IEEE Trans. on Pattern Analysis and Machine Intelligence, 29(6):929–944, 2007. M. Varma and A. Zisserman. Classifying images of materials: Achieving viewpoint and illumination independence. In Proceedings of the 7th European Conference on Computer Vision (ECCV), 2002. X. Wang and E. Grimson. Spatial latent Dirichlet allocation. In Proceedings of the Neural Information Processing Systems, 2007. G. C. G. Wei and M. A. Tanner. A Monte Carlo implementation of the EM algorithm and the poor mans data augmentation algorithms. Journal of the American Statistical Association, 85(411): 699–704, 1990. J. V. D. Weijer and C. Schmid. Coloring local feature extraction. In Proceedings of the 9th European Conference on Computer Vision, 2006. J. Winn and C. M. Bishop. Variational message passing. J. Machine Learning Research, 6:661–694, 2005. R. Zabih and V. Kolmogorov. Spatially coherent clustering using Graph Cuts. In Proceedings of CVPR’04, 2004. 239