nips nips2001 nips2001-61 nips2001-61-reference knowledge-graph by maker-knowledge-mining

61 nips-2001-Distribution of Mutual Information


Source: pdf

Author: Marcus Hutter

Abstract: The mutual information of two random variables z and J with joint probabilities {7rij} is commonly used in learning Bayesian nets as well as in many other fields. The chances 7rij are usually estimated by the empirical sampling frequency nij In leading to a point estimate J(nij In) for the mutual information. To answer questions like


reference text

[AS74] M. Abramowitz and 1. A. Stegun, editors. Handbook of mathematical functions. Dover publications, inc., 1974. [Bra99] M. Brand. Structure learning in conditional probability models via an entropic prior and parameter extinction. N eural Computation, 11(5):1155- 1182, 1999. [Bun96] W. Buntine. A guide to the literature on learning probabilistic networks from data. IEEE Transactions on Knowledge and Data Engineering, 8:195- 210, 1996. [CT91] T. M. Cover and J. A. Thomas. Elements of Information Theory. Wiley Series in Telecommunications. John Wiley & Sons, New York, NY, USA, 1991. [GCSR95] A. Gelman, J. B. Carlin, H. S. Stern, and D. B. Rubin. Bayesian Data Analysis. Chapman, 1995. Learnig in [Hec98] D. Heckerman. A tutorial on learning with Bayesian networks. Graphical Models, pages 301-354, 1998. [KJ96] G. D. Kleiter and R. Jirousek. Learning Bayesian networks under the control of mutual information. Proceedings of the 6th International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems (IPMU-1996), pages 985- 990, 1996. [Kle99] G. D. Kleiter. The posterior probability of Bayes nets with strong dependences. Soft Computing, 3:162- 173, 1999. [PFTV92] W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling. Numerical R ecipes in C: Th e Art of Scientific Computing. Cambridge University Press, Cambridge, second edition, 1992. [SooOO] E. S. Soofi. Principal information theoretic approaches. Journal of the American Statistical Association, 95:1349- 1353, 2000. [WW93] D. R. Wolf and D. H. Wolpert. Estimating functions of distributions from A finite set of samples, part 2: Bayes estimators for mutual information, chisquared, covariance and other statistics. Technical Report LANL-LA-UR-93833, Los Alamos National Laboratory, 1993. Also Santa Fe Insitute report SFI-TR-93-07-047.