nips nips2005 nips2005-2 nips2005-2-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Manfred K. Warmuth
Abstract: The classical Bayes rule computes the posterior model probability from the prior probability and the data likelihood. We generalize this rule to the case when the prior is a density matrix (symmetric positive definite and trace one) and the data likelihood a covariance matrix. The classical Bayes rule is retained as the special case when the matrices are diagonal. In the classical setting, the calculation of the probability of the data is an expected likelihood, where the expectation is over the prior distribution. In the generalized setting, this is replaced by an expected variance calculation where the variance is computed along the eigenvectors of the prior density matrix and the expectation is over the eigenvalues of the density matrix (which form a probability vector). The variances along any direction is determined by the covariance matrix. Curiously enough this expected variance calculation is a quantum measurement where the co-variance matrix specifies the instrument and the prior density matrix the mixture state of the particle. We motivate both the classical and the generalized Bayes rule with a minimum relative entropy principle, where the Kullbach-Leibler version gives the classical Bayes rule and Umegaki’s quantum relative entropy the new Bayes rule for density matrices. 1
[Bha97] R. Bhatia. Matrix Analysis. Springer, Berlin, 1997. [CA99] N. J. Cerf and C. Adam. Quantum extension of conditional probability. Physical Review A, 60(2):893–897, August 1999. [KW97] J. Kivinen and M. K. Warmuth. Additive versus exponentiated gradient updates for linear prediction. Information and Computation, 132(1):1– 64, January 1997. [KW99] J. Kivinen and M. K. Warmuth. Averaging expert predictions. In Computational Learning Theory: 4th European Conference (EuroCOLT ’99), pages 153–167, Berlin, March 1999. Springer. [NC00] M.A. Nielsen and I.L. Chuang. Quantum Computation and Quantum Information. Cambridge University Press, 2000. [SBC01] R. Schack, T. A. Brun, and C. M. Caves. Quantum Bayes rule. Physical Review A, 64(014305), 2001. [Sim79] Barry Simon. Functional Integration and Quantum Physics. Academic Press, New York, 1979. [SWRL03] R. Singh, M. K. Warmuth, B. Raj, and P. Lamere. Classificaton with free energy at raised temperatures. In Proc. of EUROSPEECH 2003, pages 1773–1776, September 2003. [TRW05] K. Tsuda, G. R¨tsch, and M. K. Warmuth. Matrix exponentiated graa dient updates for on-line learning and Bregman projections. Journal of Machine Learning Research, 6:995–1018, June 2005.