nips nips2006 nips2006-118 nips2006-118-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Chi-hoon Lee, Shaojun Wang, Feng Jiao, Dale Schuurmans, Russell Greiner
Abstract: We present a novel, semi-supervised approach to training discriminative random fields (DRFs) that efficiently exploits labeled and unlabeled training data to achieve improved accuracy in a variety of image processing tasks. We formulate DRF training as a form of MAP estimation that combines conditional loglikelihood on labeled data, given a data-dependent prior, with a conditional entropy regularizer defined on unlabeled data. Although the training objective is no longer concave, we develop an efficient local optimization procedure that produces classifiers that are more accurate than ones based on standard supervised DRF training. We then apply our semi-supervised approach to train DRFs to segment both synthetic and real data sets, and demonstrate significant improvements over supervised DRFs in each case.
[1] Y. Altun, D. McAllester, and M. Belkin. Maximum margin semi-supervised learning for structured variables. In NIPS 18. 2006.
[2] J. Besag. On the statistical analysis of dirty pictures. Journal of Royal Statistical Society. Series B, 48:3:259–302, 1986.
[3] A. Blum and T. Mitchell. Combining labeled and unlabeled data with co-training. In COLT, 1998.
[4] Yuri Boykov, Olga Veksler, and Ramin Zabih. Fast approximate energy minimization via graph cuts. In ICCV (1), pages 377–384, 1999.
[5] G. Celeux and G. Govaert. A classification EM algorithm for clustering and two stochastic versions. Comput. Stat. Data Anal., 14(3):315–332, 1992.
[6] A. Corduneanu and T. Jaakkola. Data dependent regularization. In O. Chapelle, B. Schoelkopf, and A. Zien, editors, Semi-Supervised Learning. MIT Press, 2006.
[7] C. Garcia and J.A. Moreno. Kernel based method for segmentation and modeling of magnetic resonance images. LNCS, 3315:636–645, Oct 2004.
[8] Y. Grandvalet and Y. Bengio. Semi-supervised learning by entropy minimization. In NIPS 17, 2004.
[9] F. Jiao, S. Wang, C. Lee, R. Greiner, and D Schuurmans. Semi-supervised conditional random fields for improved sequence segmentation and labeling. In COLING/ACL, 2006.
[10] S. Kumar and M. Hebert. Discriminative fields for modeling spatial dependencies in natural images. In NIPS 16, 2003.
[11] S. Kumar and M. Hebert. Discriminative random fields: A discriminative framework for contextual interaction in classification. In CVPR, 2003.
[12] J. Lafferty, F. Pereira, and A. McCallum. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In ICML, 2001.
[13] C. Lee, R. Greiner, and O. Za¨ane. Efficient spatial classification using decoupled conditional ı random fields. In 10th European Conference on Principles and Practice of Knowledge Discovery in Databases, pages 272–283, 2006.
[14] K. Nigam, A. McCallum, S. Thrun, and T. Mitchell. Text classification from labeled and unlabeled documents using EM. Machine Learning, 39(2/3):103–134, 2000.
[15] A. Quattoni, M. Collins, and T. Darrell. Conditional random fields for object recognition. In NIPS 17, 2004.
[16] S. Roberts, R. Everson, and I. Rezek. Maximum certainty data partitioning, 2000.
[17] A. Torralba, K. Murphy, and W. Freeman. Contextual models for object detection using boosted random fields. In NIPS 17, 2004.
[18] V. Vapnik. Statistical Learning Theory. John-Wiley, 1998.
[19] S.V.N. Vishwanathan, N. Schraudolph, M. Schmidt, and K. Murphy. Accelerated training of conditional random fields with stochastic gradient methods. In ICML, 2006.
[20] J. Yedidia, W. Freeman, and Y. Weiss. Generalized belief propagation. In NIPS 13, pages 689–695, 2000.
[21] J. Zhang, K. Ma, M.H. Er, and V. Chong. Tumor segmentation from magnetic resonance imaging by learning via one-class support vector machine. Intl. Workshop on Advanced Image Technology, 2004.
[22] D. Zhou, O. Bousquet, T. Navin Lal, J. Weston, and B. Sch¨ lkopf. Learning with local and o global consistency. In NIPS 16, 2004.
[23] D. Zhou, J. Huang, and B. Sch¨ lkopf. Learning from labeled and unlabeled data on a directed o graph. In ICML, 2005.
[24] X. Zhu, Z. Ghahramani, and J. Lafferty. Semi-supervised learning using gaussian fields and harmonic functions. In ICML, 2003.