nips nips2003 nips2003-82 nips2003-82-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Susanne Still, William Bialek, Léon Bottou
Abstract: We argue that K–means and deterministic annealing algorithms for geometric clustering can be derived from the more general Information Bottleneck approach. If we cluster the identities of data points to preserve information about their location, the set of optimal solutions is massively degenerate. But if we treat the equations that define the optimal solution as an iterative algorithm, then a set of “smooth” initial conditions selects solutions with the desired geometrical properties. In addition to conceptual unification, we argue that this approach can be more efficient and robust than classic algorithms. 1
[1] W. Bialek and C. G. Callan and S. P. Strong, Phys. Rev. Lett. 77 (1996) 4693-4697, http://arxiv.org/abs/cond-mat/9607180 ´
[2] W. Bialek in Physics of bio-molecules and cells; Ecole d’ete de physique th´ orique Les Houches e Session LXXV Eds.: H. Flyvbjerg, F. J¨ licher, P. Ormos and F. David (2001) Springer-Verlag, u pp.485–577, http://arxiv.org/abs/physics/0205030
[3] M. Blatt, S. Wiseman and E. Domany, Phys. Rev. Lett. 76 (1996) 3251-3254, http://arxiv.org/abs/cond-mat/9702072
[4] C. Fraley and A. Raftery, J. Am. Stat. Assoc. 97 (2002) 611-631.
[5] A. D. Gordon, Classification, (1999) Chapmann and Hall/CRC Press, London.
[6] P. Hall and E. J. Hannan, Biometrika 75, 4 (1988) 705-714.
[7] D. Horn and A. Gottlieb, Phys. Rev. Lett. 88 (2002) 018702, extended version: http://arxiv.org/abs/physics/0107063
[8] J. MacQueen in Proc. 5th Berkeley Symp. Math. Statistics and Probability Eds.: L.M.L Cam and J. Neyman (1967) University of California Press, pp. 281-297 (Vol. I)
[9] K. Rose, E. Gurewitz and G. C. Fox, Phys. Rev. Lett. 65 (1990) 945; and: K. Rose, Proceedings of the IEEE 86, 11 (1998) pp. 2210-2239.
[10] C. E. Shannon, Bell System Tech. J. 27, (1948). pp. 379-423, 623-656. See also: C. Shannon and W. Weaver, The Mathematical Theory of Communication (1963) University of Illinois Press
[11] P. Smyth, Statistics and Computing 10, 1 (2000) 63-72.
[12] S. Still and W. Bialek (2003, submitted), available at http://arxiv.org/abs/physics/0303011
[13] N. Tishby, F. Pereira and W. Bialek in Proc. 37th Annual Allerton Conf. Eds.: B. Hajek and R. S. Sreenivas (1999) University of Illinois, http://arxiv.org/abs/physics/0004057