nips nips2002 nips2002-46 nips2002-46-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Saharon Rosset, Eran Segal
Abstract: Several authors have suggested viewing boosting as a gradient descent search for a good fit in function space. We apply gradient-based boosting methodology to the unsupervised learning problem of density estimation. We show convergence properties of the algorithm and prove that a strength of weak learnability property applies to this problem as well. We illustrate the potential of this approach through experiments with boosting Bayesian networks to learn density models.
[1] Kegg: Kyoto encyclopedia of genes and genomes. In http://www.genome.ad.jp/kegg.
[2] C. M. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, Oxford, U.K., 1995.
[3] L. Breiman, J.H. Friedman, R. Olshen, and C. Stone. Classification and Regression Trees. Wardsworth International Group, 1984.
[4] P. Cheeseman and J. Stutz. Bayesian Classification (AutoClass): Theory and Results. AAAI Press, 1995.
[5] R. O. Duda and P. E. Hart. Pattern Classification and Scene Analysis. John Wiley & Sons, New York, 1973.
[6] Y. Freund and R.E. Scahpire. A decision theoretic generalization of on-line learning and an application to boosting. In the 2nd Eurpoean Conference on Computational Learning Theory, 1995.
[7] J.H. Friedman. Greedy function approximation: A gradient boosting machine. Annals of Statistics, Vol. 29 No. 5, 2001.
[8] J.H. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: a statistical view of boosting. Annals of Statistics, Vol. 28 pp. 337-407, 2000.
[9] N. Friedman and D. Koller. Being bayesian about network structure: A bayesian approach to structure discovery in bayesian networks. Machine Learning Journal, 2002.
[10] A.P. Gasch, P.T. Spellman, C.M. Kao, O.Carmel-Harel, M.B. Eisen, G.Storz, D.Botstein, and P.O. Brown. Genomic expression program in the response of yeast cells to environmental changes. Mol. Bio. Cell, 11:4241–4257, 2000.
[11] D. Heckerman. A tutorial on learning with Bayesian networks. In M. I. Jordan, editor, Learning in Graphical Models. MIT Press, Cambridge, MA, 1998.
[12] T. R. Hughes et al. Functional discovery via a compendium of expression profiles. Cell, 102(1):109–26, 2000.
[13] L. Mason, J. Baxter, P. Bartlett, and P. Frean. Boosting algorithms as gradient descent in function space. In Proc. NIPS, number 12, pages 512–518, 1999.
[14] M. Meila and T. Jaakkola. Tractable bayesian learning of tree belief networks. Technical Report CMU-RI-TR-00-15, Robotics institute, Carnegie Mellon University, 2000.
[15] D. Pe’er, A. Regev, G. Elidan, and N. Friedman. Inferring subnetworks from perturbed expression profiles. In ISMB’01, 2001.
[16] J.R. Quinlan. C4.5 - Programs for Machine Learning. Morgan-Kaufmann, 1993.
[17] S. Rosset, J. Zhu, and T. Hastie. Boosting as a regularized path to a margin maximizer. Submitted to NIPS 2002.
[18] R.E. Scahpire, Y. Freund, P. Bartlett, and W.S. Lee. Boosting the margin: a new explanation for the effectiveness of voting methods. Annals of Statistics, Vol. 26 No. 5, 1998.
[19] P. T. Spellman et al. Comprehensive identification of cell cycle-regulated genes of the yeast saccharomyces cerevisiae by microarray hybridization. Mol. Biol. Cell, 9(12):3273–97, 1998.
[20] B. Thiesson, C. Meek, and D. Heckerman. Learning mixtures of dag models. Technical Report MSR-TR-98-12, Microsoft Research, 1997.
[21] R.S. Zemel and T. Pitassi. A gradient-based boosting algorithm for regression problems. In Proc. NIPS, 2001.