nips nips2007 nips2007-166 nips2007-166-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Ke Chen, Shihai Wang
Abstract: Semi-supervised inductive learning concerns how to learn a decision rule from a data set containing both labeled and unlabeled data. Several boosting algorithms have been extended to semi-supervised learning with various strategies. To our knowledge, however, none of them takes local smoothness constraints among data into account during ensemble learning. In this paper, we introduce a local smoothness regularizer to semi-supervised boosting algorithms based on the universal optimization framework of margin cost functionals. Our regularizer is applicable to existing semi-supervised boosting algorithms to improve their generalization and speed up their training. Comparative results on synthetic, benchmark and real world tasks demonstrate the effectiveness of our local smoothness regularizer. We discuss relevant issues and relate our regularizer to previous work. 1
[1] Chapelle, O., Sch¨ lkopf, B., & Zien, A. (2006) Semi-Supervised Learning. Cambridge, MA: MIT Press. o
[2] Zhu, X. (2006) Semi-supervised learning literature survey. Computer Science TR-1530, University of Wisconsin - Madison, U.S.A.
[3] Bousquet, O., Chapelle, O., & Hein, M. (2004) Measure based regularization. In Advances in Neural Information Processing Systems 16. Cambridge, MA: MIT Press.
[4] Belkin, M., Niyogi, P., & Sindhwani, V. (2004) Manifold regularization: a geometric framework for learning from examples. Technical Report, University of Michigan, U.S.A.
[5] Szummer, M., & Jaakkola, T. (2003) Information regularization with partially labeled data. In Advances in Neural Information Processing Systems 15. Cambridge, MA: MIT Press.
[6] Grandvalet, Y., & Begio, Y. (2005) Semi-supervised learning by entropy minimization. In Advances in Neural Information Processing Systems 17. Cambridge, MA: MIT Press.
[7] Zhu, X., & Lafferty, J. (2005) Harmonic mixtures: combining mixture models and graph-based methods for inductive and scalable semi-supervised learning. In Proc. Int. Conf. Machine Learning, pp. 1052-1059.
[8] Zhou, D., Bousquet, O., Lal, T., Weston, J., & Sch¨ lkopf, B. (2004) Learning with local and global consistency. In Advances in Neural Information Processing Systems 16. Cambridge, MA: MIT Press.
[9] Mason, L., Bartlett, P., Baxter, J., & Frean, M. (2000) Functional gradient techniques for combining hypotheses. In Advances in Large Margin Classifiers. Cambridge, MA: MIT Press.
[10] d’Alch´ -Buc, F., Grandvalet, Y., & Ambroise, C. (2002) Semi-supervised MarginBoost. In Advances in e Neural Information Processing Systems 14. Cambridge, MA: MIT Press.
[11] Bennett, K., Demiriz, A., & Maclin, R. (2002) Expoliting unlabeled data in ensemble methods. In Proc. ACM Int. Conf. Knowledge Discovery and Data Mining, pp. 289-296.
[12] Collins, M., & Singer, Y. (1999) Unsupervised models for the named entity classification. In Proc. SIGDAT Conf. Empirical Methods in Natural Language Processing and Very Large Corpora.
[13] Leskes, B. (2005) The value of agreement, a new boosting algorithm. In Proc. Int. Conf. Algorithmic Learning Theory (LNAI 3559), pp. 95-110, Berlin: Springer-Verlag.
[14] G¨ nther, E., & Pfeiffer, K.P. (2005) Multiclass boosting for weak classifiers. Journal of Machine Learning u Research 6:189-210.
[15] Nigam, K., McCallum, A., Thrum, S., & Mitchell, T. (2000) Using EM to classify text from labeled and unlabeled documents. Machine Learning 39:103-134.
[16] Blake, C., Keogh, E., & Merz, C.J. (1998) UCI repository of machine learning databases. University of California, Irvine. [on-line] http://www.ics.uci.edu/ mlearn/MLRepository.html
[17] The JAFFE Database. [Online] http://www.kasrl.org/jaffe.html
[18] Fasel, B. (2002) Robust face analysis using convolutional neural networks. In Proc. Int. Conf. Pattern Recognition, vol. 2, pp. 40-43.
[19] K´ gl, B., & Wang, L. (2004) Boosting on manifolds: adaptive regularization of base classifier. In Advances e in Neural Information Processing Systems 16. Cambridge, MA: MIT Press.