nips nips2006 nips2006-41 nips2006-41-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Hugh A. Chipman, Edward I. George, Robert E. Mcculloch
Abstract: We develop a Bayesian “sum-of-trees” model, named BART, where each tree is constrained by a prior to be a weak learner. Fitting and inference are accomplished via an iterative backfitting MCMC algorithm. This model is motivated by ensemble methods in general, and boosting algorithms in particular. Like boosting, each weak learner (i.e., each weak tree) contributes a small amount to the overall model. However, our procedure is defined by a statistical model: a prior and a likelihood, while boosting is defined by an algorithm. This model-based approach enables a full and accurate assessment of uncertainty in model predictions, while remaining highly competitive in terms of predictive accuracy. 1
Abreveya, J. & McCulloch, R. (2006), Reversal of fortune: a statistical analysis of penalty calls in the national hockey league, Technical report, Purdue University. Breiman, L. (2001), ‘Random forests’, Machine Learning 45, 5–32. Chipman, H. A., George, E. I. & McCulloch, R. E. (1998), ‘Bayesian CART model search (C/R: p948-960)’, Journal of the American Statistical Association 93, 935–948. Chipman, H. A., George, E. I. & McCulloch, R. E. (2006), BART: Bayesian additive regression trees, Technical report, University of Chicago. Cook, R. D. (1977), ‘Detection of influential observations in linear regression’, Technometrics 19(1), 15–18. Efron, B., Hastie, T., Johnstone, I. & Tibshirani, R. (2004), ‘Least angle regression’, Annals of Statistics 32, 407–499. Freund, Y. & Schapire, R. E. (1997), ‘A decision-theoretic generalization of on-line learning and an application to boosting’, Journal of Computer and System Sciences 55, 119–139. Friedman, J. H. (2001), ‘Greedy function approximation: A gradient boosting machine’, The Annals of Statistics 29, 1189–1232. Green, P. J. (1995), ‘Reversible jump MCMC computation and Bayesian model determination’, Biometrika 82, 711–732. Hastie, T. & Tibshirani, R. (2000), ‘Bayesian backfitting (with comments and a rejoinder by the authors’, Statistical Science 15(3), 196–223. Hill, J. L. & McCulloch, R. E. (2006), Bayesian nonparametric modeling for causal inference, Technical report, Columbia University. Kim, H., Loh, W.-Y., Shih, Y.-S. & Chaudhuri, P. (2007), ‘Visualizable and interpretable regression models with good prediction power’, IEEE Transactions: Special Issue on Data Mining and Web Mining. In press. Meek, C., Thiesson, B. & Heckerman, D. (2002), Staged mixture modelling and boosting, Technical Report MS-TR-2002-45, Microsoft Research. Wu, Y., Tjelmeland, H. & West, M. (2007), ‘Bayesian CART: Prior specification and posterior simulation’, Journal of Computational and Graphical Statistics. In press.