nips nips2001 nips2001-194 nips2001-194-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Thomas L. Griffiths, Joshua B. Tenenbaum
Abstract: Estimating the parameters of sparse multinomial distributions is an important component of many statistical learning tasks. Recent approaches have used uncertainty over the vocabulary of symbols in a multinomial distribution as a means of accounting for sparsity. We present a Bayesian approach that allows weak prior knowledge, in the form of a small set of approximate candidate vocabularies, to be used to dramatically improve the resulting estimates. We demonstrate these improvements in applications to text compression and estimating distributions over words in newsgroup data. 1
[1] T. C. Bell, J. G. Cleary, and 1. H. Witten. Text compression. Prentice Hall, 1990.
[2] G. E. P. Box and G. C. Tiao. Bayesian Inference in Statistical Analysis. AddisonWesley, 1973.
[3] S. F. Chen and J. Goodman. An empirical study of smoothing techniques for language modeling. Technical Report TR-10-98, Center for Research in Computing Technology, Harvard University, 1998.
[4] N. Friedman and Y. Singer. Efficient Bayesian parameter estimation in large discrete domains. In Neural Information Processing Systems, 1998.
[5] H. Jeffreys. An invariant form for the prior probability in estimation problems. Proceedings of the Royal Society A, 186:453-461, 1946.
[6] P.-S. Laplace. Philosophical Essay on Probabilities. Springer-Verlag, 1995. Originally published 1825.
[7] G. Lidstone. Note on the general case of the Bayes-Laplace formula for inductive or a posteriori probabilities. Transactions of the Faculty of Actuaries, 8:182-192, 1920.
[8] K. Nigam, A. K. Mccallum, S. Thrun, and T. Mitchell. Text classification fro'in labeled and unlabeled documents using EM. Machine Learning, 39:103-134, 2000.
[9] W. Perks. Some observations on inverse probability, including a new indifference rule. Journal of the Institute of Actuaries, 73:285-312, 1947.
[10] E. S. Ristad. A natural law ·of succession. Technical Report CS-TR-895-95, Department of Computer Science, Princeton University, 1995.