nips nips2000 nips2000-43 nips2000-43-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Sham Kakade, Peter Dayan
Abstract: Substantial data support a temporal difference (TO) model of dopamine (OA) neuron activity in which the cells provide a global error signal for reinforcement learning. However, in certain circumstances, OA activity seems anomalous under the TO model, responding to non-rewarding stimuli. We address these anomalies by suggesting that OA cells multiplex information about reward bonuses, including Sutton's exploration bonuses and Ng et al's non-distorting shaping bonuses. We interpret this additional role for OA in terms of the unconditional attentional and psychomotor effects of dopamine, having the computational role of guiding exploration. 1
[1] Bertsekas, DP & Tsitsitklis, IN (1996). Neuro-dynamic Programming. Cambridge, MA: Athena Scientific.
[2] Cohen, JD, Braver, TS & O'Reilly, RC (1998). In AC Roberts, TW Robbins, editors, The Prefrontal Cortex: Executive and Cognitive Functions. Oxford: OUP.
[3] Dayan, P, & Sejnowski, TJ (1996) . Machine Learning, 25: 5-22.
[4] Horvitz, Je, Stewart, T, & Jacobs, B, (1997). Brain Research, 759:251-258.
[5] Ikemoto, S, & Panksepp, J, (1999). Brain Research Reviews, 31:6-41.
[6] Montague, PR, Dayan, P, & Sejnowski, TJ, (1996). Journal of Neuroscience, 16:1936-1947.
[7] Ng, AY, Harada, D, and Russell, S, (1999) . Proceedings of the Sixteenth International Conference on Machine Learning.
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15] Redgrave, P, Prescott, T, & Gurney, K (1999). Trends in Neurosciences, 22: 146-151. Schultz, W, (1992). Seminars in the Neurosciences, 4: 129-138. Schultz, W, (1998). Journal ofNeurophysiologJ 80: 1-27. J, Schultz, W, Apicella, P, & Ljungberg, T, (1993) . Journal of Neuroscience, 13: 900-913. Schultz, W, Dayan, P, and Montague, PR, (1997). Science, 275: 1593-1599. Schultz, W, & Romo, R, (1990). Journal of Neuroscience, 63: 607-624. Sutton, RS, (1990). Machine Learning: Proceedings of the Seventh International Conference, 216-224. Sutton, RS & Barto, AG (1998). Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press.