nips nips2012 nips2012-145 nips2012-145-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Samory Kpotufe, Abdeslam Boularias
Abstract: In regression problems over Rd , the unknown function f often varies more in some coordinates than in others. We show that weighting each coordinate i with the estimated norm of the ith derivative of f is an efficient way to significantly improve the performance of distance-based regressors, e.g. kernel and k-NN regressors. We propose a simple estimator of these derivative norms and prove its consistency. Moreover, the proposed estimator is efficiently learned online. 1
[1] Kilian Q. Weinberger and Gerald Tesauro. Metric learning for kernel regression. Journal of Machine Learning Research - Proceedings Track, 2:612–619, 2007.
[2] Bo Xiao, Xiaokang Yang, Yi Xu, and Hongyuan Zha. Learning distance metric for regression by semidefinite programming with application to human age estimation. In Proceedings of the 17th ACM international conference on Multimedia, pages 451–460, 2009.
[3] Shai Shalev-shwartz, Yoram Singer, and Andrew Y. Ng. Online and batch learning of pseudometrics. In ICML, pages 743–750. ACM Press, 2004.
[4] Jason V. Davis, Brian Kulis, Prateek Jain, Suvrit Sra, and Inderjit S. Dhillon. Informationtheoretic metric learning. In ICML, pages 209–216, 2007.
[5] W. H¨ rdle and T. Gasser. On robust kernel estimation of derivatives of regression functions. a Scandinavian journal of statistics, pages 233–240, 1985.
[6] J. Lafferty and L. Wasserman. Rodeo: Sparse nonparametric regression in high dimensions. Arxiv preprint math/0506342, 2005.
[7] L. Rosasco, S. Villa, S. Mosci, M. Santoro, and A. Verri. Nonparametric sparsity and regularization. http://arxiv.org/abs/1208.2572, 2012.
[8] L. Gyorfi, M. Kohler, A. Krzyzak, and H. Walk. A Distribution Free Theory of Nonparametric Regression. Springer, New York, NY, 2002.
[9] S. Kpotufe. k-NN Regression Adapts to Local Intrinsic Dimension. NIPS, 2011.
[10] Duy Nguyen-Tuong, Matthias W. Seeger, and Jan Peters. Model learning with local gaussian process regression. Advanced Robotics, 23(15):2015–2034, 2009.
[11] Duy Nguyen-Tuong and Jan Peters. Incremental online sparsification for model learning in real-time robot control. Neurocomputing, 74(11):1859–1867, 2011.
[12] A. Frank and A. Asuncion. UCI machine learning repository. http://archive.ics. uci.edu/ml. University of California, Irvine, School of Information and Computer Sciences, 2012.
[13] Luis Torgo. Regression datasets. http://www.liaad.up.pt/˜ltorgo. University of Porto, Department of Computer Science, 2012.
[14] A. Beygelzimer, S. Kakade, and J. Langford. Cover trees for nearest neighbors. ICML, 2006. 9