nips nips2009 nips2009-67 nips2009-67-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Yi-hao Kao, Benjamin V. Roy, Xiang Yan
Abstract: When used to guide decisions, linear regression analysis typically involves estimation of regression coefficients via ordinary least squares and their subsequent use to make decisions. When there are multiple response variables and features do not perfectly capture their relationships, it is beneficial to account for the decision objective when computing regression coefficients. Empirical optimization does so but sacrifices performance when features are well-chosen or training data are insufficient. We propose directed regression, an efficient algorithm that combines merits of ordinary least squares and empirical optimization. We demonstrate through a computational study that directed regression can generate significant performance gains over either alternative. We also develop a theory that motivates the algorithm. 1
[1] J.-Y. Audibert. Aggregated estimators and empirical complexity for least square regression. Annales de l’Institut Henri Poincare Probability and Statistics, 40(6):685–736, 2004.
[2] P. L. Bartlett and S. Mendelson. Empirical minimization. Probability Theory and Related Fields, 135(3):311–334, 2006.
[3] D. Bertsimas and A. Thiele. Robust and data-driven optimization: Modern decision-making under uncertainty. In Tutorials on Operations Research. INFORMS, 2006.
[4] O. Besbes, R. Philips, and A. Zeevi. Testing the validity of a demand model: An operations perspective. 2007.
[5] F. Bunea, A. B. Tsybakov, and M. H. Wegkamp. Aggregation for Gaussian regression. The Annals of Statistics, 35(4):1674–1697, 2007.
[6] D. Haussler. Decision theoretic generalizations of the PAC model for neural net and other learning applications. Information and Computation, 100:78–150, 1992.
[7] K. Kim and N. Timm. Univariate and Multivariate General Linear Models: Theory and Applications with SAS. Chapman & Hall/CRC, 2006.
[8] K. E. Muller and P. W. Stewart. Linear Model Theory: Univariate, Multivariate, and Mixed Models. Wiley, 2006.
[9] R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, 1998.
[10] R. Tibshirani. Regression shrinkage and selection via the lasso. Journal of Royal Statistical Society, 1996. 9