nips nips2009 nips2009-25 nips2009-25-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Daniel Cavagnaro, Jay Myung, Mark A. Pitt
Abstract: In cognitive science, empirical data collected from participants are the arbiters in model selection. Model discrimination thus depends on designing maximally informative experiments. It has been shown that adaptive design optimization (ADO) allows one to discriminate models as efficiently as possible in simulation experiments. In this paper we use ADO in a series of experiments with people to discriminate the Power, Exponential, and Hyperbolic models of memory retention, which has been a long-standing problem in cognitive science, providing an ideal setting in which to test the application of ADO for addressing questions about human cognition. Using an optimality criterion based on mutual information, ADO is able to find designs that are maximally likely to increase our certainty about the true model upon observation of the experiment outcomes. Results demonstrate the usefulness of ADO and also reveal some challenges in its implementation. 1
[1] D. J. Navarro, M. A. Pitt, and I. J. Myung. Assessing the distinguishability of models and the informativeness of data. Cognitive Psychology, 49:47–84, 2004.
[2] D. C. Rubin and A. E. Wenzel. One hundred years of forgetting: A quantitative description of retention. Psychological Review, 103(4):734–760, 1996.
[3] J. T. Wixted and E. B. Ebbesen. On the form of forgetting. Psychological Science, 2(6):409– 415, 1991.
[4] D. R. Cavagnaro, J. I Myung, M. A. Pitt, and Y. Tang. Better data with fewer participants and trials: improving experiment efficiency with adaptive design optimization. In N. A. Taatgen and H. Van Rijn, editors, Proceedings of the 31st Annual Conference of the Cognitive Science Society, pages 93–98. Cognitive Science Society, 2009.
[5] D. R. Cavagnaro, J. I. Myung, M. A. Pitt, and J. V. Kujala. Adaptive design optimization: A mutual information based approach to model discrimination in cognitive science. Neural Computation, 2009. In press.
[6] J. V. Kujala and T. J. Lukka. Bayesian adaptive estimation: The next dimension. Journal of Mathematical Psychology, 50(4):369–389, 2006.
[7] J. Lewi, R. Butera, and L. Paninski. Sequential optimal design of neurophysiology experiments. Neural Computation, 21:619–687, 2009.
[8] K. Chaloner and I. Verdinelli. Bayesian experimental design: A review. Statistical Science, 10(3):273–304, 1995.
[9] P. Gr¨ nwald. A tutorial introduction to the minimum description length principle. In u P. Gr¨ nwald, I. J. Myung, and M. A. Pitt, editors, Advances in Minimum Description Length: u Theory and Applications. The M.I.T. Press, 2005.
[10] J. I. Myung and M. A. Pitt. Optimal experimental design for model discrimination. Psychological Review, in press.
[11] A. Heavens, T. Kitching, and L. Verde. On model selection forecasting, dark energy and modified gravity. Monthly Notices of the Royal Astronomical Society, 380(3):1029–1035, 2007.
[12] T. M. Cover and J. A. Thomas. Elements of Information Theory. John Wiley & Sons, Inc., 1991.
[13] P. M¨ ller, B. Sanso, and M. De Iorio. Optimal bayesian design by inhomogeneous markov u chain simulation. Journal of the American Statistical Association, 99(467):788–798, 2004.
[14] W. R. Gilks, S. Richardson, and D. Spiegelhalter. Markov Chain Monte Carlo in Practice. Chapman & Hall, 1996.
[15] S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. Optimization by simulated annealing. Science, 220:671–680, 1983.
[16] B. Amzal, F. Y. Bois, E. Parent, and C. P. Robert. Bayesian-optimal design via interacting particle systems. Journal of the American Statistical Association, 101(474):773–785, 2006.
[17] A. Gelman, J. B. Carlin, H. S. Stern, and D. B. Rubin. Bayesian Data Analysis. Chapman & Hall, 2004.
[18] J. T. Wixted and E. B. Ebbesen. Genuine power curves in forgetting: A quantitative analysis of individual subject forgetting functions. Memory & cognition, 25(5):731–739, 1997.
[19] J. A. Brown. Some tests of the decay theory of immediate memory. Quarterly Journal of Experimental Psychology, 10:12–21, 1958.
[20] L. R. Peterson and M. J. Peterson. Short-term retention of individual verbal items. Journal of Experimental Psychology, 58:193–198, 1959.
[21] S. D. Guikema. Formulating informative, data-based priors for failure probability estimation in reliability analysis. Reliability Engineering & System Safety, 92:490–502, 2007.
[22] M. D. Lee. A bayesian analysis of retention functions. Journal of Mathematical Psychology, 48:310–321, 2004.
[23] H. P. Carlin and T. A. Louis. Bayes and empirical Bayes methods for data analysis, 2nd ed. Chapman & Hall, 2000. 9