nips nips2006 nips2006-104 nips2006-104-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Ivor W. Tsang, James T. Kwok
Abstract: Semi-supervised learning is more powerful than supervised learning by using both labeled and unlabeled data. In particular, the manifold regularization framework, together with kernel methods, leads to the Laplacian SVM (LapSVM) that has demonstrated state-of-the-art performance. However, the LapSVM solution typically involves kernel expansions of all the labeled and unlabeled examples, and is slow on testing. Moreover, existing semi-supervised learning methods, including the LapSVM, can only handle a small number of unlabeled examples. In this paper, we integrate manifold regularization with the core vector machine, which has been used for large-scale supervised and unsupervised learning. By using a sparsified manifold regularizer and formulating as a center-constrained minimum enclosing ball problem, the proposed method produces sparse solutions with low time and space complexities. Experimental results show that it is much faster than the LapSVM, and can handle a million unlabeled examples on a standard PC; while the LapSVM can only handle several thousand patterns. 1
[1] M. Belkin, P. Niyogi, and V. Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, 7:2399–2434, 2006.
[2] O. Chapelle, B. Sch¨ lkopf, and A. Zien. Semi-Supervised Learning. MIT Press, Cambridge, MA, USA, o 2006.
[3] O. Delalleau, Y. Bengio, and N. L. Roux. Efficient non-parametric function induction in semi-supervised learning. In Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics, Barbados, January 2005.
[4] G. Desaulniers, J. Desrosiers, and M.M. Solomon. Column Generation. Springer, 2005.
[5] J. Garcke and M. Griebel. Semi-supervised learning with sparse grids. In Proceedings of the ICML Workshop on Learning with Partially Classified Training Data, Bonn, Germany, August 2005.
[6] T. G¨ rtner, Q.V. Le, S. Burton, A. Smola, and S.V.N. Vishwanathan. Large-scale multiclass transduction. a In Y. Weiss, B. Sch¨ lkopf, and J. Platt, editors, Advances in Neural Information Processing Systems 18. o MIT Press, Cambridge, MA, 2006.
[7] F. Girosi. An equivalence between sparse approximation and support vector machines. Neural Computation, 10(6):1455–1480, 1998.
[8] Y.-J. Lee and O.L. Mangasarian. RSVM: Reduced support vector machines. In Proceeding of the First SIAM International Conference on Data Mining, 2001.
[9] V. Sindhwani, M. Belkin, and P. Niyogi. The geometric basis of semi-supervised learning. In Semisupervised Learning. MIT Press, 2005.
[10] V. Sindhwani, P. Niyogi, and M. Belkin. Beyond the point cloud: from transductive to semi-supervised learning. In Proceedings of the Twenty-Second International Conference on Machine Learning, pages 825–832, Bonn, Germany, August 2005.
[11] R. Tibshirani. Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society: Series B, 58:267–288, 1996.
[12] I. W. Tsang, J. T. Kwok, and P.-M. Cheung. Core vector machines: Fast SVM training on very large data sets. Journal of Machine Learning Research, 6:363–392, 2005.
[13] I. W. Tsang, J. T. Kwok, and K. T. Lai. Core vector regression for very large regression problems. In Proceedings of the Twenty-Second International Conference on Machine Learning, pages 913–920, Bonn, Germany, August 2005.
[14] X. Zhu. Semi-supervised learning literature survey. Technical Report 1530, Department of Computer Sciences, University of Wisconsin - Madison, 2005.
[15] X. Zhu and J. Lafferty. Harmonic mixtures: Combining mixture models and graph-based methods. In Proceedings of the Twenty-Second International Conference on Machine Learning, Bonn, Germany, August 2005.