nips nips2013 nips2013-236 nips2013-236-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Zhuo Wang, Alan Stocker, Daniel Lee
Abstract: In many neural systems, information about stimulus variables is often represented in a distributed manner by means of a population code. It is generally assumed that the responses of the neural population are tuned to the stimulus statistics, and most prior work has investigated the optimal tuning characteristics of one or a small number of stimulus variables. In this work, we investigate the optimal tuning for diffeomorphic representations of high-dimensional stimuli. We analytically derive the solution that minimizes the L2 reconstruction loss. We compared our solution with other well-known criteria such as maximal mutual information. Our solution suggests that the optimal weights do not necessarily decorrelate the inputs, and the optimal nonlinearity differs from the conventional equalization solution. Results illustrating these optimal representations are shown for some input distributions that may be relevant for understanding the coding of perceptual pathways. 1
[1] K Kang, RM Shapley, and H Sompolinsky. Information tuning of populations of neurons in primary visual cortex. Journal of neuroscience, 24(15):3726–3735, 2004.
[2] AP Georgopoulos, AB Schwartz, and RE Kettner. Adaptation of the motion-sensitive neuron h1 is generated locally and governed by contrast frequency. Science, 233:1416–1419, 1986.
[3] FE Theunissen and JP Miller. Representation of sensory information in the cricket cercal sensory system. II. information theoretic calculation of system accuracy and optimal tuningcurve widths of four primary interneurons. J Neurophysiol, 66(5):1690–1703, November 1991.
[4] DC Fitzpatrick, R Batra, TR Stanford, and S Kuwada. A neuronal population code for sound localization. Nature, 388:871–874, 1997. 8
[5] NS Harper and D McAlpine. Optimal neural population coding of an auditory spatial cue. Nature, 430:682–686, 2004.
[6] N Brenner, W Bialek, and R de Ruyter van Steveninck. Adaptive rescaling maximizes information transmission. Neuron, 26:695–702, 2000.
[7] Tvd Twer and DIA MacLeod. Optimal nonlinear codes for the perception of natural colours. Network: Computation in Neural Systems, 12(3):395–407, 2001.
[8] I Dean, NS Harper, and D McAlpine. Neural population coding of sound level adapts to stimulus statistics. Nature neuroscience, 8:1684–1689, 2005.
[9] Y Ozuysal and SA Baccus. Linking the computational structure of variance adaptation to biophysical mechanisms. Neuron, 73:1002–1015, 2012.
[10] SB Laughlin. A simple coding procedure enhances a neurons information capacity. Z. Naturforschung, 36c(3):910–912, 1981.
[11] J-P Nadal and N Parga. Non linear neurons in the low noise limit: A factorial code maximizes information transfer, 1994.
[12] M Bethge, D Rotermund, and K Pawelzik. Optimal short-term population coding: when Fisher information fails. Neural Computation, 14:2317–2351, 2002.
[13] M Bethge, D Rotermund, and K Pawelzik. Optimal neural rate coding leads to bimodal firing rate distributions. Netw. Comput. Neural Syst., 14:303–319, 2003.
[14] MD McDonnell and NG Stocks. Maximally informative stimuli and tuning curves for sigmoidal rate-coding neurons and populations. Phys. Rev. Lett., 101:058103, 2008.
[15] Z Wang, A Stocker, and DD Lee. Optimal neural tuning curves for arbitrary stimulus distributions: Discrimax, infomax and minimum lp loss. Adv. Neural Information Processing Systems, 25:2177–2185, 2012.
[16] N Brunel and J-P Nadal. Mutual information, fisher information and population coding. Neural Computation, 10(7):1731–1757, 1998.
[17] K Zhang and TJ Sejnowski. Neuronal tuning: To sharpen or broaden? Neural Computation, 11:75–84, 1999.
[18] A Pouget, S Deneve, J-C Ducom, and PE Latham. Narrow versus wide tuning curves: Whats best for a population code? Neural Computation, 11:85–90, 1999.
[19] H Sompolinsky and H Yoon. The effect of correlations on the fisher information of population codes. Advances in Neural Information Processing Systems, 11, 1999.
[20] AP Nikitin, NG Stocks, RP Morse, and MD McDonnell. Neural population coding is optimized by discrete tuning curves. Phys. Rev. Lett., 103:138101, 2009.
[21] D Ganguli and EP Simoncelli. Implicit encoding of prior probabilities in optimal neural populations. Adv. Neural Information Processing Systems, 23:658–666, 2010.
[22] S Yaeli and R Meir. Error-based analysis of optimal tuning functions explains phenomena observed in sensory neurons. Front Comput Neurosci, 4, 2010.
[23] E Doi and MS Lewicki. Characterization of minimum error linear coding with sensory and neural noise. Neural Computation, 23, 2011.
[24] AJ Bell and TJ Sejnowski. An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7:1129–1159, 1995.
[25] DJ Field BA Olshausen. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381:607–609, 1996.
[26] A Hyvarinen and E Oja. Independent component analysis: Algorithms and applications. Neural Networks, 13:411–430, 2000.
[27] P Berens, A Ecker, S Gerwinn, AS Tolias, and M Bethge. Reassessing optimal neural population codes with neurometric functions. Proceedings of the National Academy of Sciences, 11:4423–4428, 2011.
[28] TM Cover and J Thomas. Elements of Information Theory. Wiley, 1991.
[29] EL Lehmann and G Casella. Theory of point estimation. New York: Springer-Verlag., 1999.
[30] GH Hardy, JE Littlewood, and G Polya. Inequalities, 2nd ed. Cambridge University Press, 1988. 9