nips nips2012 nips2012-211 nips2012-211-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Melanie Rey, Volker Roth
Abstract: We present a reformulation of the information bottleneck (IB) problem in terms of copula, using the equivalence between mutual information and negative copula entropy. Focusing on the Gaussian copula we extend the analytical IB solution available for the multivariate Gaussian case to distributions with a Gaussian dependence structure but arbitrary marginal densities, also called meta-Gaussian distributions. This opens new possibles applications of IB to continuous data and provides a solution more robust to outliers. 1
[1] N. Tishby, F.C. Pereira, and W. Bialek. The information bottleneck method. The 37th annual Allerton Conference on Communication, Control, and Computing, (29-30):368–377, 1999.
[2] O. Shamir, S. Sabato, and N. Tishby. Learning and generalization with the information bottleneck. Theor. Comput. Sci., 411(29-30):2696–2711, 2010.
[3] G. Chechik, A. Globerson, N. Tishby, and Y. Weiss. Information bottleneck for Gaussian variables. Journal of Machine Learning Research, 6:165–188, 2005.
[4] A. Globerson and N. Tishby. On the optimality of the Gaussian information bottleneck curve. Hebrew University Technical Report, 2004.
[5] R.M. Hecht, E. Noor, and N. Tishby. Speaker recognition by Gaussian information bottleneck. INTERSPEECH, pages 1567–1570, 2009.
[6] J. Ma and Z. Sun. Mutual information is copula entropy. arXiv:0808.0845v1, 2008. `
[7] A. Sklar. Fonctions de r´ partition a n dimensions et leurs marges. Publications de l’Institut de Statistique e de l’Universit´ de Paris, 8:229–231, 1959. e
[8] A. J. McNeil, R. Frey, and P. Embrechts. Quantitative Risk Management. Princeton Series in Finance. Princeton University Press, 2005.
[9] G. Elidan. Copula bayesian networks. Proceedings of the Neural Information Processing Systems (NIPS), 2010.
[10] C. Genest, K. Ghoudhi, and L.P. Rivet. A semiparametric estimation procedure of dependence parameters in multivariate families of distributions. Biometrika, 82(3):543–552, 1995.
[11] H. Tsukahara. Semiparametric estimation in copula models. 33(3):357–375, 2005. The Canadian Journal of Statistics,
[12] Peter D. Hoff. Extending the rank likelihood for semiparametric copula estimation. Annals of Applied Statistics, 1(1):273, 2007.
[13] K. Boudt, J. Cornelissen, and C. Croux. The gaussian rank correlation estimator: Robustness properties. Statistics and Computing, 22:471–483, 2012.
[14] D. P´ l, B. P´ czos, and C. Szepesv´ ri. Estimation of R´ nyi entropy and mutual information based on a o a e generalized nearest-neighbor graphs. Proceedings of the Neural Information Processing Systems (NIPS), 2010. 9