nips nips2003 nips2003-130 nips2003-130-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Aaron C. Courville, Geoffrey J. Gordon, David S. Touretzky, Nathaniel D. Daw
Abstract: We develop a framework based on Bayesian model averaging to explain how animals cope with uncertainty about contingencies in classical conditioning experiments. Traditional accounts of conditioning fit parameters within a fixed generative model of reinforcer delivery; uncertainty over the model structure is not considered. We apply the theory to explain the puzzling relationship between second-order conditioning and conditioned inhibition, two similar conditioning regimes that nonetheless result in strongly divergent behavioral outcomes. According to the theory, second-order conditioning results when limited experience leads animals to prefer a simpler world model that produces spurious correlations; conditioned inhibition results when a more complex model is justified by additional experience. 1
[1] A. C. Courville and D. S. Touretzky. Modeling temporal structure in classical conditioning. In Advances in Neural Information Processing Systems 14, pages 3–10, Cambridge, MA, 2002. MIT Press.
[2] P. J. Green. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika, 82:711–732, 1995.
[3] Y. Iba. Extended ensemble Monte Carlo. International Journal of Modern Physics C, 12(5):623–656, 2001.
[4] S. Kakade and P. Dayan. Acquisition and extinction in autoshaping. Psychological Review, 109:533–544, 2002.
[5] D. J. C. MacKay. Bayesian model comparison and backprop nets. In Advances in Neural Information Processing Systems 4, Cambridge, MA, 1991. MIT Press.
[6] J. M. Pearce. Similarity and discrimination: A selective review and a connectionist model. Psychological Review, 101:587–607, 1994.
[7] R. A. Rescorla and A. R. Wagner. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black and W. F. Prokasy, editors, Classical Conditioning II. Appleton-Century-Crofts, 1972.
[8] R. S. Sutton and A. G. Barto. Time-derivative models of Pavlovian reinforcement. In M. Gabriel and J. Moore, editors, Learning and Computational Neuroscience: Foundations of Adaptive Networks, chapter 12, pages 497–537. MIT Press, 1990.
[9] J. Tenenbaum and T. Griffiths. Structure learning in human causal induction. In Advances in Neural Information Processing Systems 13, pages 59–65, Cambridge, MA, 2001. MIT Press.
[10] D. A. Williams and J. B. Overmier. Some types of conditioned inhibitors carry collateral excitatory associations. Learning and Motivation, 19:345–368, 1988.
[11] H. Yin, R. C. Barnet, and R. R. Miller. Second-order conditioning and Pavlovian conditioned inhibition: Operational similarities and differences. Journal of Experimental Psychology: Animal Behavior Processes, 20(4):419–428, 1994.