nips nips2008 nips2008-206 nips2008-206-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Angela J. Yu, Jonathan D. Cohen
Abstract: In a variety of behavioral tasks, subjects exhibit an automatic and apparently suboptimal sequential effect: they respond more rapidly and accurately to a stimulus if it reinforces a local pattern in stimulus history, such as a string of repetitions or alternations, compared to when it violates such a pattern. This is often the case even if the local trends arise by chance in the context of a randomized design, such that stimulus history has no real predictive power. In this work, we use a normative Bayesian framework to examine the hypothesis that such idiosyncrasies may reflect the inadvertent engagement of mechanisms critical for adapting to a changing environment. We show that prior belief in non-stationarity can induce experimentally observed sequential effects in an otherwise Bayes-optimal algorithm. The Bayesian algorithm is shown to be well approximated by linear-exponential filtering of past observations, a feature also apparent in the behavioral data. We derive an explicit relationship between the parameters and computations of the exact Bayesian algorithm and those of the approximate linear-exponential filter. Since the latter is equivalent to a leaky-integration process, a commonly used model of neuronal dynamics underlying perceptual decision-making and trial-to-trial dependencies, our model provides a principled account of why such dynamics are useful. We also show that parameter-tuning of the leaky-integration process is possible, using stochastic gradient descent based only on the noisy binary inputs. This is a proof of concept that not only can neurons implement near-optimal prediction based on standard neuronal dynamics, but that they can also learn to tune the processing parameters without explicitly representing probabilities. 1
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22]
[23]
[24]
[25]
[26] Skinner, B F (1948). J. Exp. Psychol. 38: 168-72. Ecott, C L & Critchfield, T S (2004). J. App. Beh. Analysis 37: 249-65. Laming, D R J (1968). Information Theory of of Choice-Reaction Times, Academic Press, London. Soetens, E, Boer, L C, & Hueting, J E (1985). JEP: HPP 11: 598-616. Cho, R, et al (2002). Cognitive, Affective, & Behavioral Neurosci. 2: 283-99. Sugrue, L P, Corrado, G S, & Newsome, W T (2004). Science 304: 1782-7. Smith, P L & Ratcliff, R. Trends Neurosci. 27: 161-8. Gold, J I & Shadlen, M N (2002). Neuron 36: 299-308. Wald, A & Wolfowitz, J (1948). Ann. Math. Statisti. 19: 326-39. Bogacz, et al (2006). Psychological Review 113: 700-65. Cook, E P & Maunsell, J H R (2002). Nat. Neurosci. 5: 985-94. Grice, G R (1972). Perception & Psychophysics 12: 103-7. McClelland, J L. Attention & Performance XIV: 655-88. MIT Press. Smith, P L (1995). Psychol. Rev. 10: 567-93. Yu, A J (2007). Adv. in Neur. Info. Proc. Systems 19: 1545-52. Dayan, P & Yu, A J (2003). IETE J. Research 49: 171-81. Kim, C & Myung, I J (1995). 17th Ann. Meeting. of Cog. Sci. Soc.: 472-7. Mozer, M C, Colagrosso, M D, & Huber, D E (2002). Adv. in Neur. Info. Proc. Systems 14: 51-57. Mozer, M C, Kinoshita, S, & Shettel, M (2007). Integrated Models of Cog. Sys.: 180-93. Simen, P, Cohen, J D, & Holmes, P (2006). Neur. Netw. 19: 1013-26. Sutton, R S & Barto, A G (1998). Reinforcement Learning: An Introduction, MIT Press. Behrens, T E J, Woolrich, M W, Walton, M E, & Rushworth, M F S (2007). Nat. Neurosci. 10: 1214-21. Kording, K P, Tenenbaum, J B, & Shadmehr, R (2007). Nat. Neurosci. 10: 779-86. Huettel, S A, Mack, P B, & McCarthy, G (2002). Nat. Neurosci. 5: 485-90. Hertwig, R & Ortmann, A (2001). Behavioral & Brain Sciences 24: 383-403. Bialek, W (2005). Preprint q-bio.NC/0508044, Princeton University. 8