nips nips2006 nips2006-100 nips2006-100-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Yevgeny Seldin, Noam Slonim, Naftali Tishby
Abstract: We present a general model-independent approach to the analysis of data in cases when these data do not appear in the form of co-occurrence of two variables X, Y , but rather as a sample of values of an unknown (stochastic) function Z(X, Y ). For example, in gene expression data, the expression level Z is a function of gene X and condition Y ; or in movie ratings data the rating Z is a function of viewer X and movie Y . The approach represents a consistent extension of the Information Bottleneck method that has previously relied on the availability of co-occurrence statistics. By altering the relevance variable we eliminate the need in the sample of joint distribution of all input variables. This new formulation also enables simple MDL-like model complexity control and prediction of missing values of Z. The approach is analyzed and shown to be on a par with the best known clustering algorithms for a wide range of domains. For the prediction of missing values (collaborative filtering) it improves the currently best known results. 1
[1] Noan Slonim and Naftali Tishby. Document clustering using word clusters via the information bottleneck method. In Proceedings of 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2000.
[2] Noam Slonim. The Information Bottleneck: Theory and Applications. PhD thesis, The Hebrew University of Jerusalem, 2002.
[3] Sara C. Madeira and Arlindo L. Oliveira. Biclustering algorithms for biological data analysis: A survey. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 1(1):24–45, January 2004.
[4] Naftali Tishby, Fernando Pereira, and William Bialek. The information bottleneck method. In Allerton Conference on Communication, Control and Computation, volume 37, pages 368–379. 1999.
[5] Janne Sinkkonen and Samuel Kaski. Clustering based on conditional distributions in an auxiliary space. Neural Computation, 14(1):217–239, 2002.
[6] David Gondek and Thomas Hofmann. Non-redundant data clustering. In 4th IEEE International Conference on Data Mining, 2004.
[7] Susanne Still, William Bialek, and L´ on Bottou. Geometric clustering using the information bottleneck e method. In Advances in Neural Information Processing Systems 16.
[8] Noam Slonim, Nir Friedman, and Naftali Tishby. Multivariate information bottleneck. Neural Computation, 18, 2006.
[9] Naftali Tishby and Noam Slonim. Data clustering by markovian relaxation and the information bottleneck method. In NIPS, 2000.
[10] Noam Slonim, Gurinder Singh Atwal, Gasper Tracik, and William Bialek. Information-based clustering. In Proceedings of the National Academy of Science (PNAS), volume 102, pages 18297–1830, Dec. 2005.
[11] J. Herlocker, J. Konstan, L. Terveen, and J. Riedl. Evaluating collaborative filtering recommender systems. In ACM Transactions on Information Systems, volume 22(1), pages 5–53, January 2004.
[12] Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. Wiley Series in Telecommunications. John Wiley & Sons, New York, NY, 1991.
[13] Noam Slonim, Nir Friedman, and Naftali Tishby. Unsupervised document classification using sequential information maximization. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2002.
[14] A. Barron, J. Rissanen, and B. Yu. The minimum description length principle in coding and modeling. IEEE Trans. Info. Theory, 44:2743–2760, 1998.
[15] J. Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Mateo, CA: Morgan Kaufman Publishers, 1988.
[16] Gasch A.P., Spellman P.T., Kao C.M., Carmel-Harel O., Eisen M.B., Storz G., Botstein D., and Brown P.O. Genomic expression programs in the response of yeast cells to environmental changes. Molecular Biology. Cell, 11(12):4241–57, December 2000.
[17] A.P. Gasch. The environmental stress response: a common yeast response to environmental stresses. Topics in Current Genetics (series editor S. Hohmann), 1:11–70, 2002.
[18] Segal E., Shapira M., Regev A., Pe’er D., Botstein D., Koller D., and Friedman N. Module networks: identifying regulatory modules and their condition-specific regulators from gene expression data. Natural Genetics, 34(2):166–76, 2003.
[19] M. Ashburner, C. A. Ball, J. A. Blake, D. Botstein, H. Butler, J. M. Cherry, A. P. Davis, K. Dolinski, S. S. Dwight, J. T. Eppig, M. A. Harris, D. P. Hill, L. Issel-Tarver, A. Kasarskis, S. Lewis, J. C. Matese, J. E. Richardson, M. Ringwald, G. M. Rubin, and G. Sherlock. Gene ontology: tool for the unification of biology. Nature Genetics, 25:25–29, May 2000.
[20] Thomas Hoffman. Latent semantic models for collaborative filtering. In ACM Transactions on Information Systems, volume 22, January 2004.
[21] Gal Chechik, Amir Globerson, Naftali Tishby, and Yair Weiss. Gaussian information bottleneck. Journal of Machine Learning Research, 6:165–188, January 2005.