nips nips2012 nips2012-288 nips2012-288-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Nisheeth Srivastava, Paul R. Schrater
Abstract: Statistical decision theory axiomatically assumes that the relative desirability of different options that humans perceive is well described by assigning them optionspecific scalar utility functions. However, this assumption is refuted by observed human behavior, including studies wherein preferences have been shown to change systematically simply through variation in the set of choice options presented. In this paper, we show that interpreting desirability as a relative comparison between available options at any particular decision instance results in a rational theory of value-inference that explains heretofore intractable violations of rational choice behavior in human subjects. Complementarily, we also characterize the conditions under which a rational agent selecting optimal options indicated by dynamic value inference in our framework will behave identically to one whose preferences are encoded using a static ordinal utility function. 1
[1] A.G. Barto and R.S. Sutton. Reinforcement Learning: an introduction. Univesity of Cambridge Press, 1998.
[2] J. R. Busemeyer, R. Barkan, S. Mehta, and A. Chaturvedi. Context effects and models of preferential choice: implications for consumer behavior. Marketing Theory, 7(1):39–58, 2007.
[3] J.R. Busemeyer and J.T. Townsend. Decision field theory: A dynamic cognition approach to decision making. Psychological Review, 100:432–459, 1993.
[4] K. Canini, M. Shashkov, and T. Griffiths. Modeling transfer learning in human categorization with the hierarchical dirichlet process. In ICML, pages 151–158, 2010.
[5] U. Chajewska, D. Koller, and D. Ormoneit. Learning an agent’s utility function by observing behavior. In ICML, pages 35–42, 2001.
[6] N. Chater. Rational and mechanistic perspectives on reinforcement learning. Cognition, 113(3):350 – 364, 2009. Reinforcement learning and higher cognition.
[7] N. Daw and M. Frank. Reinforcement learning and higher level cognition: Introduction to special issue. Cognition, 113(3):259 – 261, 2009. Reinforcement learning and higher cognition.
[8] L. Gabora and D. Aerts. Contextualizing concepts using a mathematical generalization of the quantum formalism. Joural of Experimental and Theoretical Artificial Intelligence, 14(4):327– 358, 2002.
[9] D. Hensher, J. Rose, and W. Greene. Applied Choice Analysis: A Primer. Cambridge University Press, 2005.
[10] A. Jern, C. Lucas, and C. Kemp. Evaluating the inverse decision-making approach to preference learning. In NIPS, pages 2276–2284, 2011.
[11] D. Kahneman. Perception, action and utility: the tangled skein. In M. Rabinovich, K. Friston, and P. Varona, editors, Principles of Brain Dynamics: Global State Interactions. MIT Pres, 2012.
[12] D. Kreps. A Course in Microeconomic Theory, pages 17–69. Princeton University Press, 1990.
[13] W. Leong and D. Hensher. Embedding decision heuristics in discrete choice models: A review. Transport Reviews, 32(3):313–331, 2012.
[14] C.G. Lucas, T. Griffiths, F. Xu, and C. Fawcett. A rational model of preference learning and choice prediction by children. In NIPS, pages 985–992, 2008.
[15] R. D. Luce and H. Raiffa. Games and Decisions: Introduction and Critical Survey. Wiley, New York, 1957.
[16] J.v. Neumann and O. Morgenstern. Theory of Games and Economic Behavior. Princeton University Press, 1953.
[17] A. Y. Ng and S. J. Russell. Algorithms for inverse reinforcement learning. In Proceedings of the Seventeenth International Conference on Machine Learning, ICML ’00, pages 663–670, 2000.
[18] M. Rabin. Psychology and economics. Journal of Economic Literature, 36(1):pp. 11–46, 1998.
[19] S.J. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. MIT Press, 1998.
[20] L. Shi and T. Griffiths. Neural Implementation of Hierarchical Bayesian Inference by Importance Sampling. In Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, and A. Culotta, editors, Advances in Neural Information Processing Systems 22, pages 1669–1677. 2009.
[21] A. Tversky and I. Simonson. Context-dependent preferences. Management Science, 39(10):pp. 1179–1189, 1993.
[22] I. Vlaev, N. Chater, N. Stewart, and G. Brown. Does the brain calculate value? Trends in Cognitive Sciences, 15(11):546 – 554, 2011.
[23] I. Vlaev, B. Seymour, R.J. Dolan, and N. Chater. The price of pain and the value of suffering. Psychological Science, 20(3):309–317, 2009. 9