nips nips2013 nips2013-88 nips2013-88-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Liming Wang, David Carlson, Miguel Rodrigues, David Wilcox, Robert Calderbank, Lawrence Carin
Abstract: We consider design of linear projection measurements for a vector Poisson signal model. The projections are performed on the vector Poisson rate, X ∈ Rn , and the + observed data are a vector of counts, Y ∈ Zm . The projection matrix is designed + by maximizing mutual information between Y and X, I(Y ; X). When there is a latent class label C ∈ {1, . . . , L} associated with X, we consider the mutual information with respect to Y and C, I(Y ; C). New analytic expressions for the gradient of I(Y ; X) and I(Y ; C) are presented, with gradient performed with respect to the measurement matrix. Connections are made to the more widely studied Gaussian measurement model. Example results are presented for compressive topic modeling of a document corpora (word counting), and hyperspectral compressive sensing for chemical classification (photon counting). 1
[1] R. Atar and T. Weissman. Mutual information, relative entropy, and estimation in the Poisson channel. IEEE Transactions on Information Theory, 58(3):1302–1318, March 2012.
[2] A. Banerjee, S. Merugu, I.S. Dhillon, and J. Ghosh. Clustering with bregman divergences. JMLR, 2005.
[3] M.W Berry, M. Browne, A.N. Langville, V.P. Pauca, and R. J. Plemmons. Algorithms and applications for approximate nonnegative matrix factorization. Computational Statistics & Data Analysis, 2007.
[4] D.M. Blei, A.Y. Ng, and M.I. Jordan. Latent Dirichlet allocation. JMLR, 2003.
[5] L.M. Bregman. The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR computational mathematics and mathematical physics, 1967.
[6] E. Cand` s, J. Romberg, and T. Tao. Robust uncertainty principles: Exact signal reconstruction from e highly incomplete frequency information. IEEE Trans. on Inform. Theory, 2006.
[7] W.R. Carson, M. Chen, M.R.D. Rodrigues, R. Calderbank, and L. Carin. Communications-inspired projection design with application to compressive sensing. SIAM J. Imaging Sciences, 2013.
[8] M. Chen, W. Carson, M. Rodrigues, R. Calderbank, and L. Carin. Communications inspired linear discriminant analysis. In ICML, 2012.
[9] G.B. Folland. Real Analysis: Modern Techniques and Their Applications. Wiley New York, 1999.
[10] D. Guo. Information and estimation over binomial and negative binomial models. arXiv preprint arXiv:1207.7144, 2012.
[11] D. Guo, S. Shamai, and S. Verd´ . Mutual information and minimum mean-square error in Gaussian u channels. IEEE Transactions on Information Theory, 51(4):1261–1282, April 2005.
[12] D. Guo, S. Shamai, and S. Verd´ . Mutual information and conditional mean estimation in Poisson chanu nels. IEEE Transactions on Information Theory, 54(5):1837–1849, May 2008.
[13] S.M. Haas and J.H. Shapiro. Capacity of wireless optical communications. IEEE Journal on Selected Areas in Communications, 21(8):1346–1357, Aug. 2003.
[14] M. Hellman and J. Raviv. Probability of error, equivocation, and the Chernoff bound. IEEE Transactions on Information Theory, 1970.
[15] A. Lapidoth and S. Shamai. The poisson multiple-access channel. IEEE Transactions on Information Theory, 44(2):488–501, Feb. 1998.
[16] R.S. Liptser and A.N. Shiryaev. Statistics of Random Processes: II. Applications, volume 2. Springer, 2000.
[17] D.P. Palomar and S. Verd´ . Gradient of mutual information in linear vector Gaussian channels. IEEE u Transactions on Information Theory, 52(1):141–154, Jan. 2006.
[18] D.P. Palomar and S. Verd´ . Representation of mutual information via input estimates. IEEE Transactions u on Information Theory, 53(2):453–470, Feb. 2007.
[19] S. Prasad. Certain relations between mutual information and fidelity of statistical estimation. http://arxiv.org/pdf/1010.1508v1.pdf, 2012.
[20] M. Raginsky, R.M. Willett, Z.T. Harmany, and R.F. Marcia. Compressed sensing performance bounds under poisson noise. IEEE Trans. Signal Processing, 2010.
[21] M. Seeger, H. Nickisch, R. Pohmann, and B. Schoelkopf. Optimization of k-space trajectories for compressed sensing by bayesian experimental design. Magnetic Resonance in Medicine, 2010.
[22] C.G. Taborda and F. Perez-Cruz. Mutual information and relative entropy over the binomial and negative binomial channels. In IEEE International Symposium on Information Theory Proceedings (ISIT), pages 696–700. IEEE, 2012.
[23] S. Verd´ . Mismatched estimation and relative entropy. IEEE Transactions on Information Theory, u 56(8):3712–3720, Aug. 2010.
[24] T. Weissman. The relationship between causal and noncausal mismatched estimation in continuous-time awgn channels. IEEE Transactions on Information Theory, 2010.
[25] D.S. Wilcox, G.T. Buzzard, B.J. Lucier, P. Wang, and D. Ben-Amotz. Photon level chemical classification using digital compressive detection. Analytica Chimica Acta, 2012.
[26] M. Zhou, L. Hannah, D. Dunson, and L. Carin. Beta-negative binomial process and Poisson factor analysis. AISTATS, 2012. 9