nips nips2007 nips2007-7 nips2007-7-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Arthur Gretton, Kenji Fukumizu, Choon H. Teo, Le Song, Bernhard Schölkopf, Alex J. Smola
Abstract: Although kernel measures of independence have been widely applied in machine learning (notably in kernel ICA), there is as yet no method to determine whether they have detected statistically significant dependence. We provide a novel test of the independence hypothesis for one particular kernel independence measure, the Hilbert-Schmidt independence criterion (HSIC). The resulting test costs O(m2 ), where m is the sample size. We demonstrate that this test outperforms established contingency table and functional correlation-based tests, and that this advantage is greater for multivariate data. Finally, we show the HSIC test also applies to text (and to structured data more generally), for which no other independence test presently exists.
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22]
[23]
[24]
[25] F. Bach and M. Jordan. Tree-dependent component analysis. In UAI 18, 2002. F. R. Bach and M. I. Jordan. Kernel independent component analysis. J. Mach. Learn. Res., 3:1–48, 2002. I. Calvino. If on a winter’s night a traveler. Harvest Books, Florida, 1982. J. Dauxois and G. M. Nkiet. Nonlinear canonical analysis and independence tests. Ann. Statist., 26(4):1254–1278, 1998. L. Devroye, L. Gy¨ rfi, and G. Lugosi. A Probabilistic Theory of Pattern Recognition. Number 31 in o Applications of mathematics. Springer, New York, 1996. Andrey Feuerverger. A consistent test for bivariate dependence. International Statistical Review, 61(3):419–433, 1993. K. Fukumizu, F. R. Bach, and M. I. Jordan. Dimensionality reduction for supervised learning with reproducing kernel Hilbert spaces. Journal of Machine Learning Research, 5:73–99, 2004. A. Gretton, K. Borgwardt, M. Rasch, B. Sch¨ lkopf, and A. Smola. A kernel method for the two-sampleo problem. In NIPS 19, pages 513–520, Cambridge, MA, 2007. MIT Press. A. Gretton, O. Bousquet, A.J. Smola, and B. Sch¨ lkopf. Measuring statistical dependence with Hilberto Schmidt norms. In ALT, pages 63–77, 2005. A. Gretton, K. Fukumizu, C.-H. Teo, L. Song, B. Sch¨ lkopf, and A. Smola. A kernel statistical test of o independence. Technical Report 168, MPI for Biological Cybernetics, 2008. A. Gretton, R. Herbrich, A. Smola, O. Bousquet, and B. Sch¨ lkopf. Kernel methods for measuring o independence. J. Mach. Learn. Res., 6:2075–2129, 2005. N. L. Johnson, S. Kotz, and N. Balakrishnan. Continuous Univariate Distributions. Volume 1 (Second Edition). John Wiley and Sons, 1994. A. Kankainen. Consistent Testing of Total Independence Based on the Empirical Characteristic Function. PhD thesis, University of Jyv¨ skyl¨ , 1995. a a Juha Karvanen. A resampling test for the total independence of stationary time series: Application to the performance evaluation of ica algorithms. Neural Processing Letters, 22(3):311 – 324, 2005. C.-J. Ku and T. Fine. Testing for stochastic independence: application to blind source separation. IEEE Transactions on Signal Processing, 53(5):1815–1826, 2005. C. Leslie, E. Eskin, and W. S. Noble. The spectrum kernel: A string kernel for SVM protein classification. In Pacific Symposium on Biocomputing, pages 564–575, 2002. T. Read and N. Cressie. Goodness-Of-Fit Statistics for Discrete Multivariate Analysis. Springer-Verlag, New York, 1988. A. R´ nyi. On measures of dependence. Acta Math. Acad. Sci. Hungar., 10:441–451, 1959. e M. Rosenblatt. A quadratic measure of deviation of two-dimensional density estimates and a test of independence. The Annals of Statistics, 3(1):1–14, 1975. B. Sch¨ lkopf, K. Tsuda, and J.-P. Vert. Kernel Methods in Computational Biology. MIT Press, 2004. o R. Serfling. Approximation Theorems of Mathematical Statistics. Wiley, New York, 1980. L. Song, A. Smola, A. Gretton, K. Borgwardt, and J. Bedo. Supervised feature selection via dependence estimation. In Proc. Intl. Conf. Machine Learning, pages 823–830. Omnipress, 2007. I. Steinwart. The influence of the kernel on the consistency of support vector machines. Journal of Machine Learning Research, 2, 2002. C. H. Teo and S. V. N. Vishwanathan. Fast and space efficient string kernels using suffix arrays. In ICML, pages 929–936, 2006. F.J. Theis. Towards a general independent subspace analysis. In NIPS 19, 2007. 8