nips nips2008 nips2008-56 nips2008-56-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Kai Yu, Wei Xu, Yihong Gong
Abstract: In this paper we aim to train deep neural networks for rapid visual recognition. The task is highly challenging, largely due to the lack of a meaningful regularizer on the functions realized by the networks. We propose a novel regularization method that takes advantage of kernel methods, where an oracle kernel function represents prior knowledge about the recognition task of interest. We derive an efficient algorithm using stochastic gradient descent, and demonstrate encouraging results on a wide range of recognition tasks, in terms of both accuracy and speed. 1
[1] A. Ahmed, K. Yu, W. Xu, Y. Gong, and E. P. Xing. Training hierarchical feed-forward visual recognition models using transfer learning from pseudo tasks. European Conference on Computer Vision, 2008.
[2] R. K. Ando and T. Zhang. A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research, 2005.
[3] S. Baluja and H. Rowley. Boosting sex identification performance. Journal of Computer Vision, 2007.
[4] Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle. Greedy layer-wise training of deep networks. Neural Information Processing Systems, 2007.
[5] A. Bosch, A. Zisserman, and X. Mun˜ z. Image classification using ROIs and multiple kernel learning. o 2008. submitted to International Journal of Computer Vison.
[6] O. Chapelle, J. Weston, and B. Sch¨ lkopf. Cluster kernels for semi-supervised learning. Neural Informao tion Processing Systems, 2003.
[7] L. Fei-Fei, R. Fergus, and P. Perona. Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categories. CVPR Workshop, 2004.
[8] J. Goldberger, S. Roweis, G. Hinton, and R. Salakhutdinov. Neighbourhood components analysis. Neural Information Processing Systems, 2005.
[9] G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313(5786):504 – 507, July 2006.
[10] S. Lazebnik, C. Schmid, and J. Ponce. Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. IEEE Conference on Computer Vision and Pattern Recognition, 2006.
[11] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
[12] J. Mutch and D. G. Lowe. Multiclass object recognition with sparse, localized features. IEEE Conference on Computer Vision and Pattern Recognition, 2006.
[13] P. J. Philips, P. J. Flynn, T. Scruggs, K. W. Bower, and W. Worek. Preliminary face recognition grand challenge results. IEEE Conference on Automatic Face and Gesture Recgonition, 2006.
[14] R. Raina, A. Battle, H. Lee, B. Packer, and A. Y. Ng. Self-taught learning: Transfer learning from unlabeled data. International Conference on Machine Learning, 2007.
[15] M. Ranzato, F.-J. Huang, Y.-L. Boureau, and Y. LeCun. Unsupervised learning of invariant feature hierarchies with applications to object recognition. IEEE Conference on Computer Vision and Pattern Recognition, 2007.
[16] M. Ranzato and M. Szummer. Semi-supervised learning of compact document representations with deep networks. International Conferenece on Machine Learning, 2008.
[17] T. Serre, L. Wolf, and T. Poggio. Object recognition with features inspired by visual cortex. IEEE Conference on Computer Vision and Pattern Recognition, 2005.
[18] J. Weston, F. Ratle, and R. Collobert. Deep learning via semi-supervised embedding. International Conference on Machine Learning, 2008.
[19] C. Williams and M. Seeger. Using the Nystr¨ m method to speed up kernel machines. Neural Information o Processing Systems, 2001. 8