nips nips2002 nips2002-114 nips2002-114-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Martin Szummer, Tommi S. Jaakkola
Abstract: Classification with partially labeled data requires using a large number of unlabeled examples (or an estimated marginal P (x)), to further constrain the conditional P (y|x) beyond a few available labeled examples. We formulate a regularization approach to linking the marginal and the conditional in a general way. The regularization penalty measures the information that is implied about the labels over covering regions. No parametric assumptions are required and the approach remains tractable even for continuous marginal densities P (x). We develop algorithms for solving the regularization problem for finite covers, establish a limiting differential equation, and exemplify the behavior of the new regularization approach in simple cases.
[1] Tommi Jaakkola, Marina Meila, and Tony Jebara. Maximum entropy discrimination. Technical Report AITR-1668, Mass. Inst. of Technology AI lab, 1999. http://www.ai.mit.edu/.
[2] Naftali Tishby and Noam Slonim. Data clustering by markovian relaxation and the information bottleneck method. In Advances in Neural Information Processing Systems (NIPS), volume 13, pages 640–646. MIT Press, 2001.
[3] Stephen Roberts, C. Holmes, and D. Denison. Minimum-entropy data partitioning using reversible jump Markov chain Monte Carlo. IEEE Trans. Pattern Analysis and Mach. Intell. (PAMI), 23(8):909–914, 2001.
[4] Matthias Seeger. Input-dependent regularization of conditional density models. Unpublished. http://www.dai.ed.ac.uk/homes/seeger/, 2001.
[5] Thomas Cover and Joy Thomas. Elements of Information Theory. Wiley, 1991.
[6] Robert Weinstock. Calculus of Variations. Dover, 1974.