nips nips2001 nips2001-123 nips2001-123-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Aaron C. Courville, David S. Touretzky
Abstract: The Temporal Coding Hypothesis of Miller and colleagues [7] suggests that animals integrate related temporal patterns of stimuli into single memory representations. We formalize this concept using quasi-Bayes estimation to update the parameters of a constrained hidden Markov model. This approach allows us to account for some surprising temporal effects in the second order conditioning experiments of Miller et al. [1 , 2, 3], which other models are unable to explain. 1
[1] R. C. Barnet, H. M. Arnold, and R. R. Miller. Simultaneous conditioning demonstrated in second-order conditioning: Evidence for similar associative structure in forward and simultaneous conditioning. Learning and Motivation, 22:253- 268, 1991.
[2] R. P. Cole, R. C. Barnet, and R. R . Miller. Temporal encoding in trace conditioning. Animal Learning and Behavior, 23(2) :144- 153, 1995 .
[3] R. P. Cole and R. R. Miller. Conditioned excitation and conditioned inhibition acquired through backward conditioning. Learning and Motivation , 30:129- 156, 1999.
[4] P. Dayan. Improving generalization for temporal difference learning: the successor representation. Neural Computation, 5:613- 624, 1993.
[5] Q. Huo and C.-H. Lee. On-line adaptive learning of the continuous density hidden Markov model based on approximate recursive Bayes estimate. IEEE Transactions on Speech and Audio Processing, 5(2):161- 172, 1997.
[6] V . Krishnamurthy and J . B. Moore. On-line estimation of hidden Markov model parameters based on the Kullback-Leibler information measure. IEEE Transactions on Signal Processing, 41(8):2557- 2573, 1993.
[7] L. D. Matzel , F. P. Held, and R. R. Miller. Information and the expression of simultaneous and backward associations: Implications for contiguity theory. Learning and Motivation, 19:317- 344, 1988.
[8] R. R. Miller and R . C. Barnet. The role of time in elementary associations. Current Directions in Psychological Sci ence, 2(4):106- 111 , 1993.
[9] 1. P. Pavlov. Conditioned Reflexes. Oxford University Press, 1927.
[10] L. R. Rabiner. A tutorial on hidden Markov models and selected applications speech recognition. Proceedings of th e IEEE, 77(2) :257- 285, 1989. III
[11] R. A. Rescorla and A. R. Wagner. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement . In A. H. Black and W. F. Prokasy, editors, Classical Conditioning II. Appleton-Century-Crofts, 1972.
[12] A. F . M. Smith and U. E . Makov . A quasi-Bayes sequential procedure for mixtures. Journal of th e Royal Statistical Soci ety, 40(1):106- 112, 1978.
[13] R. E. Suri and W. Schultz. Temporal difference model reproduces anticipatory neural activity. N eural Computation, 13(4):841- 862, 200l.
[14] R. S. Sutton and A. G. Barto. Time-derivative models of Pavlovian reinforcement. In M. Gabriel and J. Moore, editors, Learning and Computational N euroscience: Foundations of Adaptive N etworks, chapter 12 , pages 497- 537. MIT Press, 1990.
[15] R. S. Sutton and B. Pinette. The learning of world models by connectionist networks. In L. Erlbaum, editor, Proceedings of the seventh annual conference of the cognitive science society, pages 54- 64, Irvine, California, August 1985.