nips nips2001 nips2001-167 nips2001-167-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Florence D'alché-buc, Yves Grandvalet, Christophe Ambroise
Abstract: In many discrimination problems a large amount of data is available but only a few of them are labeled. This provides a strong motivation to improve or develop methods for semi-supervised learning. In this paper, boosting is generalized to this task within the optimization framework of MarginBoost . We extend the margin definition to unlabeled data and develop the gradient descent algorithm that corresponds to the resulting margin cost function. This meta-learning scheme can be applied to any base classifier able to benefit from unlabeled data. We propose here to apply it to mixture models trained with an Expectation-Maximization algorithm. Promising results are presented on benchmarks with different rates of labeled data. 1
[1] C. Ambroise and G. Govaert. EM algorithm for partially known labels. In IFCS 2000, july 2000.
[2] J.-P. Aubin. L 'analyse non lineaire et ses applications d l'economie. Masson , 1984.
[3] K P. Bennett and A. Demiriz. Semi-supervised support vector machines. In D. Cohn, M. Kearns, and S. Solla, editors, Advances in Neural Information Processing Systems, pages 368-374. MIT Press, 1999.
[4] C.M. Bishop and M.E. Tipping. A hierarchical latent variable model for data vizualization. IEEE PAMI, 20:281- 293, 1998.
[5] A. Blum and Tom Mitchell. Combining labeled and unlabeled data with co-training. In Proceedings of th e 1998 Conference on Computational Learning Th eory, July 1998.
[6] L. Breiman. Prediction games and arcing algorithms. Technical Report 504, Statistics Department , University of California at Berkeley, 1997.
[7] Y . Freund and R. E. Schapire. Experiments with a new boosting algorithm. In Machin e Learning: Proceedings of th e Thirteenth International Conference, pages 148- 156. Morgan Kauffman, 1996.
[8] J. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: a statistical view of boosting. The Annals of Statistics, 28(2):337- 407, 2000.
[9] Y. Grandvalet, F. d'Alche Buc, and C. Ambroise. Boosting mixture models for semisupervised learning. In ICANN 2001 , august 200l.
[10] L. Mason , J. Baxter, P. L. Bartlett, and M. Frean. Functional gradient techniques for combining hypotheses. In Advances in Large Margin Classifiers. MIT, 2000.
[11] G.J. McLachlan and T. Krishnan. Th e EM algorithm and extensions. Wiley, 1997.
[12] K Nigam, A. K McCallum, S. Thrun, and T. Mitchell. Text classification from labeled and unlabeled documents using EM. Machine learning, 39(2/3):135- 167, 2000.
[13] G. Riitsch, T. Onoda, and K-R. Muller. Soft margins for AdaBoost. Technical report, Department of Computer Science, Royal Holloway, London , 1998.
[14] G. Riitsch, T. Onoda, and K-R. Muller. Soft margins for AdaBoost. Machine Learning, 42(3):287- 320, 200l.
[15] R. E. Schapire, Y. Freund, P. Bartlett, and W. S. Lee. Boosting the margin: A new explanation for the effectiveness of voting methods. Th e Annals of Statistics, 26(5):1651- 1686, 1998.
[16] Matthias Seeger. Learning with data,www.citeseer.nj.nec.com/seegerOllearning.html. labeled and unlabeled