nips nips2012 nips2012-26 nips2012-26-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Konstantina Palla, Zoubin Ghahramani, David A. Knowles
Abstract: Factor analysis models effectively summarise the covariance structure of high dimensional data, but the solutions are typically hard to interpret. This motivates attempting to find a disjoint partition, i.e. a simple clustering, of observed variables into highly correlated subsets. We introduce a Bayesian non-parametric approach to this problem, and demonstrate advantages over heuristic methods proposed to date. Our Dirichlet process variable clustering (DPVC) model can discover blockdiagonal covariance structures in data. We evaluate our method on both synthetic and gene expression analysis problems. 1
Alon, U., Barkai, N., Notterman, D., Gish, K., Ybarra, S., Mack, D., and Levine, A. (1999). Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proceedings of the National Academy of Sciences, 96(12):6745. Basak, S., Balaban, A., Grunwald, G., and Gute, B. (2000a). Topological indices: their nature and mutual relatedness. Journal of chemical information and computer sciences, 40(4):891–898. Basak, S., Grunwald, G., Gute, B., Balasubramanian, K., and Opitz, D. (2000b). Use of statistical and neural net approaches in predicting toxicity of chemicals. Journal of Chemical Information and Computer Sciences, 40(4):885–890. Carvalho, C. M., Chang, J., Lucas, J. E., Nevins, J. R., Wang, Q., and West, M. (2008). Highdimensional sparse factor modeling: Applications in gene expression genomics. Journal of the American Statistical Association, 103(484):1438–1456. D’haeseleer, P. et al. (2005). How does gene expression clustering work? Nature biotechnology, 23(12):1499–1502. Duda, R. O., Hart, P. E., and Stork, D. G. (2001). Pattern Classification. Wiley-Interscience, 2nd edition. Eisen, M., Spellman, P., Brown, P., Botstein, D., Sherlock, G., Zhang, M., Iyer, V., Anders, K., Botstein, D., Futcher, B., et al. (1998). Gene expression: Clustering. Proc Natl Acad Sci US A, 95(25):14863–8. Fevotte, C. and Godsill, S. J. (2006). A Bayesian approach for blind separation of sparse sources. Audio, Speech, and Language Processing, IEEE Transactions on, 14(6):2174–2188. Fokoue, E. (2004). Stochastic determination of the intrinsic structure in Bayesian factor analysis. Technical report, Statistical and Applied Mathematical Sciences Institute. Gasch, A., Spellman, P., Kao, C., Carmel-Harel, O., Eisen, M., Storz, G., Botstein, D., and Brown, P. (2000). Genomic expression programs in the response of yeast cells to environmental changes. Science’s STKE, 11(12):4241. Geweke, J. (2004). Getting it right. Journal of the American Statistical Association, 99(467):799– 804. Hotelling, H. (1933). Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 24:417–441. Kaufman, G. M. and Press, S. J. (1973). Bayesian factor analysis. Technical Report 662-73, Sloan School of Management, University of Chicago. Knowles, D. A. and Ghahramani, Z. (2007). Infinite sparse factor analysis and infinite independent components analysis. In 7th International Conference on Independent Component Analysis and Signal Separation, volume 4666, pages 381–388. Springer. Knowles, D. A. and Ghahramani, Z. (2011). Nonparametric Bayesian sparse factor models with application to gene expression modeling. The Annals of Applied Statistics, 5(2B):1534–1552. Minka, T. P. (2001). Expectation propagation for approximate Bayesian inference. In Conference on Uncertainty in Artificial Intelligence (UAI), volume 17. Minka, T. P., Winn, J. M., Guiver, J. P., and Knowles, D. A. (2010). Infer.NET 2.4. Neal, R. M. (2000). Markov chain sampling methods for Dirichlet process mixture models. Journal of computational and graphical statistics, 9(2):249–265. Neal, R. M. (2003). Slice sampling. The Annals of Statistics, 31(3):705–741. Niu, D., Dy, J., and Ghahramani, Z. (2012). A nonparametric bayesian model for multiple clustering with overlapping feature views. Journal of Machine Learning Research, 22:814–822. 8 Pearson, K. (1901). On lines and planes of closest fit to systems of points in space. Philosophical Magazine Series 6, 2:559–572. Pitman, J. (2002). Combinatorial stochastic processes. Technical report, Department of Statistics, University of California at Berkeley. Rai, P. and Daum´ III, H. (2008). The infinite hierarchical factor regression model. In Advances in e Neural Information Processing Systems (NIPS). Rowe, D. B. and Press, S. J. (1998). Gibbs sampling and hill climbing in Bayesian factor analysis. Technical Report 255, Department of Statistics, University of California Riverside. Roweis, S. (1998). EM algorithms for PCA and SPCA. In Advances in Neural Information Processing Systems (NIPS), pages 626–632. MIT Press. Sanche, R. and Lonergan, K. (2006). Variable reduction for predictive modeling with clustering. In Casualty Actuarial Society Forum, pages 89–100. Shafto, P., Kemp, C., Mansinghka, V., Gordon, M., and Tenenbaum, J. (2006). Learning crosscutting systems of categories. In Proceedings of the 28th annual conference of the Cognitive Science Society, pages 2146–2151. Silva, R., Scheines, R., Glymour, C., and Spirtes, P. (2006). Learning the structure of linear latent variable models. The Journal of Machine Learning Research, 7:191–246. Tipping, M. E. and Bishop, C. M. (1999). Probabilistic principal component analysis. Journal of the Royal Statistical Society. Series B (Statistical Methodology), 61(3):611–622. Vigneau, E. and Qannari, E. (2003). Clustering of variables around latent components. Communications in Statistics-Simulation and Computation, 32(4):1131–1150. West, M., Chang, J., Lucas, J., Nevins, J. R., Wang, Q., and Carvalho, C. (2007). High-dimensional sparse factor modelling: Applications in gene expression genomics. Technical report, ISDS, Duke University. Winn, J. and Bishop, C. M. (2006). Variational message passing. Journal of Machine Learning Research, 6(1):661. Young, G. (1941). Maximum likelihood estimation and factor analysis. Psychometrika, 6(1):49–53. 9