nips nips2007 nips2007-69 nips2007-69-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Yuhong Guo, Dale Schuurmans
Abstract: Active learning sequentially selects unlabeled instances to label with the goal of reducing the effort needed to learn a good classifier. Most previous studies in active learning have focused on selecting one unlabeled instance to label at a time while retraining in each iteration. Recently a few batch mode active learning approaches have been proposed that select a set of most informative unlabeled instances in each iteration under the guidance of heuristic scores. In this paper, we propose a discriminative batch mode active learning approach that formulates the instance selection task as a continuous optimization problem over auxiliary instance selection variables. The optimization is formulated to maximize the discriminative classification performance of the target classifier, while also taking the unlabeled data into account. Although the objective is not convex, we can manipulate a quasi-Newton method to obtain a good local solution. Our empirical studies on UCI datasets show that the proposed active learning is more effective than current state-of-the art batch mode active learning algorithms. 1
[1] K. Bennett and E. Parrado-Hernandez. The interplay of optimization and machine learning research. Journal of Machine Learning Research, 7, 2006.
[2] K. Brinker. Incorporating diversity in active learning with support vector machines. In Proceedings of the 20th International Conference on Machine learning, 2003.
[3] C. Campbell, N. Cristianini, and A. Smola. Query learning with large margin classifiers. In Proceedings of the 17th International Conference on Machine Learning, 2000.
[4] D. Cohn, Z. Ghahramani, and M. Jordan. Active learning with statistical models. Journal of Artificial Intelligence Research, 4, 1996.
[5] Y. Freund, H. S. Seung, E. Shamir, and N. Tishby. Selective sampling using the query by committee algorithm. Machine Learning, 28, 1997.
[6] Y. Grandvalet and Y. Bengio. Semi-supervised learning by entropy minimization. In Advances in Neural Information Processing Systems, 2005.
[7] Y. Guo and R. Greiner. Optimistic active learning using mutual information. In Proceedings of the International Joint Conference on Artificial Intelligence, 2007.
[8] S. Hoi, R. Jin, and M. Lyu. Large-scale text categorization by batch mode active learning. In Proceedings of the International World Wide Web Conference, 2006.
[9] S. Hoi, R. Jin, J. Zhu, and M. Lyu. Batch mode active learning and its application to medical image classification. In Proceedings of the 23rd International Conference on Machine Learning, 2006.
[10] D. Lewis and W. Gale. A sequential algorithm for training text classifiers. In Proceedings of the International ACM-SIGIR Conference on Research and Development in Information Retrieval, 1994.
[11] A. McCallum and K. Nigam. Employing EM in pool-based active learning for text classification. In Proceedings of the 15th International Conference on Machine Learning, 1998.
[12] T. Minka. A comparison of numerical optimizers for logistic regression. Technical report, 2003. http://research.microsoft.com/ minka/papers/logreg/.
[13] I. Muslea, S. Minton, and C. Knoblock. Active + semi-supervised learning = robust multi-view learning. In Proceedings of the 19th International Conference on Machine Learning, 2002.
[14] H. Nguyen and A. Smeulders. Active learning using pre-clustering. In Proceedings of the 21st International Conference on Machine Learning, 2004.
[15] J. Nocedal and S.J. Wright. Numerical Optimization. Springer, New York, 1999.
[16] N. Roy and A. McCallum. Toward optimal active learning through sampling estimation of error reduction. In Proceedings of the 18th International Conference on Machine Learning, 2001.
[17] G. Schohn and D. Cohn. Less is more: Active learning with support vector machines. In Proceedings of the 17th International Conference on Machine Learning, 2000.
[18] S. Tong and D. Koller. Support vector machine active learning with applications to text classification. In Proceedings of the 17th International Conference on Machine Learning, 2000.
[19] Z. Xu, K. Yu, V. Tresp, X. Xu, and J. Wang. Representative sampling for text classification using support vector machines. In Proceedings of the 25th European Conference on Information Retrieval Research, 2003.
[20] C. Zhang and T. Chen. An active learning framework for content-based information retrieval. IEEE Trans on Multimedia, 4:260–258, 2002.
[21] J. Zhu and T. Hastie. Kernel logistic regression and the import vector machine. Journal of Computational and Graphical Statistics, 14, 2005.
[22] X. Zhu, J. Lafferty, and Z. Ghahramani. Combining active learning and semi-supervised learning using gaussian fields and harmonic functions. In ICML Workshop on The Continuum from Labeled to Unlabeled Data in Machine Learning and Data Mining, 2003.