andrew_gelman_stats andrew_gelman_stats-2012 andrew_gelman_stats-2012-1529 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Psychology researcher Alison Gopnik discusses the idea that some of the systematic problems with human reasoning can be explained by systematic flaws in the statistical models we implicitly use. I really like this idea and I’ll return to it in a bit. But first I need to discuss a minor (but, I think, ultimately crucial) disagreement I have with how Gopnik describes Bayesian inference. She writes: The Bayesian idea is simple, but it turns out to be very powerful. It’s so powerful, in fact, that computer scientists are using it to design intelligent learning machines, and more and more psychologists think that it might explain human intelligence. Bayesian inference is a way to use statistical data to evaluate hypotheses and make predictions. These might be scientific hypotheses and predictions or everyday ones. So far, so good. Next comes the problem (as I see it). Gopnik writes: Here’s a simple bit of Bayesian election thinking. In early September, the polls suddenly im
sentIndex sentText sentNum sentScore
1 Psychology researcher Alison Gopnik discusses the idea that some of the systematic problems with human reasoning can be explained by systematic flaws in the statistical models we implicitly use. [sent-1, score-0.563]
2 It’s so powerful, in fact, that computer scientists are using it to design intelligent learning machines, and more and more psychologists think that it might explain human intelligence. [sent-5, score-0.327]
3 Bayesian inference is a way to use statistical data to evaluate hypotheses and make predictions. [sent-6, score-0.325]
4 These might be scientific hypotheses and predictions or everyday ones. [sent-7, score-0.339]
5 Combining your prior beliefs about the hypotheses and the likelihood of the data can help you . [sent-19, score-0.368]
6 In this case, the inspiring convention idea is both likely to begin with and likely to have led to the change in the polls, so it wins out over the other two. [sent-22, score-0.551]
7 As noted, this point is minor–I have no problem with Gopnik’s summary that one of the hypotheses “wins out over the other two. [sent-28, score-0.263]
8 (This is probably a good place for me to plug my article with Kari Lock from a couple years ago on Bayesian combination of state polls and election forecasts, where we use continuous weighting. [sent-30, score-0.412]
9 ) Blame the discrete models, not the priors One way this seemingly minor point can matter is when we follow Gopnik’s suggestion that Bayesian inference “might explain human intelligence. [sent-31, score-0.662]
10 But discrete thinking does not describe how much of the biological social world works. [sent-33, score-0.338]
11 If we, as humans, take these continuous phenomena and try to model them discretely, we will trip up, in predictable ways–even if we use (discrete) Bayesian methods. [sent-35, score-0.276]
12 To put it another way: what if Josh Tenenbaum and his colleagues (not mentioned in Gopnik’s article but you can search for them here on the blog) are right that our brains use some sort of approximate discrete Bayesian reasoning to make decisions and perform inferences about the world? [sent-36, score-0.523]
13 ” She’s referring to this experiment done in her lab: “We gave 4-year-olds and adults evidence about a toy that worked in an unusual way. [sent-39, score-0.332]
14 The 4-year-olds were actually more likely to figure out the toy than the adults were. [sent-41, score-0.417]
15 ” In that example, Gopnik might well be correct: it seems reasonable to suspect that a kid will have a better prior than an adult on how a toy works. [sent-42, score-0.351]
16 More generally, though, I think we should avoid the temptation to think that, when a Bayesian inference goes wrong, it has to be a problem with the prior. [sent-43, score-0.325]
17 In many cases, the model matters (for example, in our discussion above about natural-seeming but flawed discrete models). [sent-45, score-0.258]
18 If, as I think is the case, our brains like discrete models (perhaps they can be more quickly coded and computed) but the world is continuous and varying, this suggests interesting systematic ways that our brains might be misunderstanding the world in everyday reasoning. [sent-47, score-1.27]
19 (Conversely, if discrete models really do have major computational advantages, maybe statisticians like myself should be giving them a second look. [sent-48, score-0.358]
20 This post had been titled, “I notice a (slightly) garbled version of Bayesian inference, which provokes some thoughts on the applicability of Bayesian models of human reasoning. [sent-51, score-0.367]
wordName wordTfidf (topN-words)
[('gopnik', 0.563), ('discrete', 0.258), ('toy', 0.202), ('hypotheses', 0.19), ('bayesian', 0.19), ('brains', 0.151), ('continuous', 0.134), ('adults', 0.13), ('polls', 0.111), ('minor', 0.11), ('varying', 0.11), ('garbled', 0.109), ('systematic', 0.109), ('human', 0.107), ('models', 0.1), ('convention', 0.098), ('slightly', 0.096), ('predictable', 0.091), ('prior', 0.09), ('everyday', 0.09), ('likelihood', 0.088), ('wins', 0.087), ('likely', 0.085), ('inference', 0.084), ('world', 0.08), ('idea', 0.075), ('problem', 0.073), ('discretely', 0.066), ('led', 0.066), ('reasoning', 0.063), ('alison', 0.063), ('rationally', 0.06), ('election', 0.06), ('might', 0.059), ('kari', 0.058), ('tenenbaum', 0.058), ('think', 0.058), ('plug', 0.056), ('inspiring', 0.055), ('manipulated', 0.053), ('lock', 0.053), ('explain', 0.053), ('correct', 0.052), ('temptation', 0.052), ('applicability', 0.051), ('pollsters', 0.051), ('use', 0.051), ('intelligent', 0.05), ('matter', 0.05), ('overly', 0.05)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999988 1529 andrew gelman stats-2012-10-11-Bayesian brains?
Introduction: Psychology researcher Alison Gopnik discusses the idea that some of the systematic problems with human reasoning can be explained by systematic flaws in the statistical models we implicitly use. I really like this idea and I’ll return to it in a bit. But first I need to discuss a minor (but, I think, ultimately crucial) disagreement I have with how Gopnik describes Bayesian inference. She writes: The Bayesian idea is simple, but it turns out to be very powerful. It’s so powerful, in fact, that computer scientists are using it to design intelligent learning machines, and more and more psychologists think that it might explain human intelligence. Bayesian inference is a way to use statistical data to evaluate hypotheses and make predictions. These might be scientific hypotheses and predictions or everyday ones. So far, so good. Next comes the problem (as I see it). Gopnik writes: Here’s a simple bit of Bayesian election thinking. In early September, the polls suddenly im
2 0.35101178 617 andrew gelman stats-2011-03-17-“Why Preschool Shouldn’t Be Like School”?
Introduction: Under the heading, “Why Preschool Shouldn’t Be Like School,” cognitive psychologist Alison Gopnik describes research showing that four-year-olds learn better if they’re encouraged to discover and show to others, rather than if they’re taught what to do. This makes sense, but it’s not clear to me why this wouldn’t apply to older kids and adults. It’s a commonplace in teaching at all levels that students learn by doing and by demonstrating what they can do. Even when a student is doing nothing but improvising from a template, we generally believe the student will learn better by explaining what’s going on, by having a mental model of the process to go along with the proverbial 10,000 hours or practice. The challenge is in the implementation, how to get students interested, motivated, and focused enough to put the effort into learning. So why the headline above? Why does Gopnik’s research support the idea that preschool should be different from school? I’m not trying to disagree
3 0.24077415 634 andrew gelman stats-2011-03-29-A.I. is Whatever We Can’t Yet Automate
Introduction: A common aphorism among artificial intelligence practitioners is that A.I. is whatever machines can’t currently do. Adam Gopnik, writing for the New Yorker , has a review called Get Smart in the most recent issue (4 April 2011). Ostensibly, the piece is a review of new books, one by Joshua Foer, Moonwalking with Einstein: The Art and Science of Remembering Everything , and one by Stephen Baker Final Jeopardy: Man vs. Machine and the Quest to Know Everything (which would explain Baker’s spate of Jeopardy!-related blog posts ). But like many such pieces in highbrow magazines, the book reviews are just a cover for staking out a philosophical position. Gopnik does a typically New Yorker job in explaining the title of this blog post. Gopnik describes his mother as “a logician, linguist, and early Fortran speaker” and goes on to add that she worked on an early machine translation project in Canada. I’m guessing she’s the Myrna Gopnik behind this 1968 COLING paper (LE
4 0.19805454 1469 andrew gelman stats-2012-08-25-Ways of knowing
Introduction: In this discussion from last month, computer science student and Judea Pearl collaborator Elias Barenboim expressed an attitude that hierarchical Bayesian methods might be fine in practice but that they lack theory, that Bayesians can’t succeed in toy problems. I posted a P.S. there which might not have been noticed so I will put it here: I now realize that there is some disagreement about what constitutes a “guarantee.” In one of his comments, Barenboim writes, “the assurance we have that the result must hold as long as the assumptions in the model are correct should be regarded as a guarantee.” In that sense, yes, we have guarantees! It is fundamental to Bayesian inference that the result must hold if the assumptions in the model are correct. We have lots of that in Bayesian Data Analysis (particularly in the first four chapters but implicitly elsewhere as well), and this is also covered in the classic books by Lindley, Jaynes, and others. This sort of guarantee is indeed p
5 0.19799316 781 andrew gelman stats-2011-06-28-The holes in my philosophy of Bayesian data analysis
Introduction: I’ve been writing a lot about my philosophy of Bayesian statistics and how it fits into Popper’s ideas about falsification and Kuhn’s ideas about scientific revolutions. Here’s my long, somewhat technical paper with Cosma Shalizi. Here’s our shorter overview for the volume on the philosophy of social science. Here’s my latest try (for an online symposium), focusing on the key issues. I’m pretty happy with my approach–the familiar idea that Bayesian data analysis iterates the three steps of model building, inference, and model checking–but it does have some unresolved (maybe unresolvable) problems. Here are a couple mentioned in the third of the above links. Consider a simple model with independent data y_1, y_2, .., y_10 ~ N(θ,σ^2), with a prior distribution θ ~ N(0,10^2) and σ known and taking on some value of approximately 10. Inference about μ is straightforward, as is model checking, whether based on graphs or numerical summaries such as the sample variance and skewn
6 0.17580295 1228 andrew gelman stats-2012-03-25-Continuous variables in Bayesian networks
7 0.16645314 2003 andrew gelman stats-2013-08-30-Stan Project: Continuous Relaxations for Discrete MRFs
8 0.16599371 114 andrew gelman stats-2010-06-28-More on Bayesian deduction-induction
9 0.16308229 811 andrew gelman stats-2011-07-20-Kind of Bayesian
10 0.15973055 1695 andrew gelman stats-2013-01-28-Economists argue about Bayes
11 0.14745997 1554 andrew gelman stats-2012-10-31-It not necessary that Bayesian methods conform to the likelihood principle
12 0.1424417 1510 andrew gelman stats-2012-09-25-Incoherence of Bayesian data analysis
13 0.14002772 2263 andrew gelman stats-2014-03-24-Empirical implications of Empirical Implications of Theoretical Models
14 0.13405317 1247 andrew gelman stats-2012-04-05-More philosophy of Bayes
15 0.13364737 1431 andrew gelman stats-2012-07-27-Overfitting
16 0.13335288 1205 andrew gelman stats-2012-03-09-Coming to agreement on philosophy of statistics
17 0.13334803 217 andrew gelman stats-2010-08-19-The “either-or” fallacy of believing in discrete models: an example of folk statistics
18 0.13289793 244 andrew gelman stats-2010-08-30-Useful models, model checking, and external validation: a mini-discussion
19 0.13255657 1779 andrew gelman stats-2013-03-27-“Two Dogmas of Strong Objective Bayesianism”
20 0.13183151 1575 andrew gelman stats-2012-11-12-Thinking like a statistician (continuously) rather than like a civilian (discretely)
topicId topicWeight
[(0, 0.231), (1, 0.129), (2, -0.021), (3, 0.063), (4, -0.125), (5, -0.01), (6, -0.024), (7, 0.047), (8, 0.036), (9, -0.011), (10, -0.009), (11, 0.016), (12, -0.022), (13, -0.034), (14, 0.018), (15, 0.036), (16, 0.038), (17, -0.005), (18, -0.03), (19, 0.043), (20, -0.064), (21, 0.023), (22, -0.069), (23, -0.001), (24, -0.017), (25, -0.007), (26, 0.026), (27, -0.021), (28, -0.019), (29, 0.004), (30, -0.006), (31, -0.003), (32, -0.003), (33, -0.034), (34, 0.024), (35, -0.01), (36, -0.007), (37, -0.039), (38, 0.014), (39, -0.02), (40, -0.036), (41, 0.031), (42, 0.015), (43, 0.013), (44, -0.044), (45, 0.008), (46, 0.039), (47, 0.017), (48, 0.035), (49, 0.032)]
simIndex simValue blogId blogTitle
same-blog 1 0.97651911 1529 andrew gelman stats-2012-10-11-Bayesian brains?
Introduction: Psychology researcher Alison Gopnik discusses the idea that some of the systematic problems with human reasoning can be explained by systematic flaws in the statistical models we implicitly use. I really like this idea and I’ll return to it in a bit. But first I need to discuss a minor (but, I think, ultimately crucial) disagreement I have with how Gopnik describes Bayesian inference. She writes: The Bayesian idea is simple, but it turns out to be very powerful. It’s so powerful, in fact, that computer scientists are using it to design intelligent learning machines, and more and more psychologists think that it might explain human intelligence. Bayesian inference is a way to use statistical data to evaluate hypotheses and make predictions. These might be scientific hypotheses and predictions or everyday ones. So far, so good. Next comes the problem (as I see it). Gopnik writes: Here’s a simple bit of Bayesian election thinking. In early September, the polls suddenly im
2 0.85884774 1182 andrew gelman stats-2012-02-24-Untangling the Jeffreys-Lindley paradox
Introduction: Ryan Ickert writes: I was wondering if you’d seen this post , by a particle physicist with some degree of influence. Dr. Dorigo works at CERN and Fermilab. The penultimate paragraph is: From the above expression, the Frequentist researcher concludes that the tracker is indeed biased, and rejects the null hypothesis H0, since there is a less-than-2% probability (P’<α) that a result as the one observed could arise by chance! A Frequentist thus draws, strongly, the opposite conclusion than a Bayesian from the same set of data. How to solve the riddle? He goes on to not solve the riddle. Perhaps you can? Surely with the large sample size they have (n=10^6), the precision on the frequentist p-value is pretty good, is it not? My reply: The first comment on the site (by Anonymous [who, just to be clear, is not me; I have no idea who wrote that comment], 22 Feb 2012, 21:27pm) pretty much nails it: In setting up the Bayesian model, Dorigo assumed a silly distribution on th
3 0.85168415 1280 andrew gelman stats-2012-04-24-Non-Bayesian analysis of Bayesian agents?
Introduction: Econometrician and statistician Dale Poirier writes: 24 years ago (1988, Journal of Economics Perspectives) I [Poirier] noted cognitive dissonance among some economists who treat the agents in their theoretical framework as Bayesians, but then analyze the data (even in the same paper!) as a frequentist. Recently, I have found similar cases in cognitive science. I suspect other disciplines exhibit such behavior. Do you know of any examples in political science? My reply: I don’t know of any such examples in political science. Game theoretic models are popular in poli sci, but I haven’t seen much in the way of models of Bayesian decision making. Here are two references (not in political science) that might be helpful. 1. I have argued that the utility model (popular in economics and political science as a way of providing “microfoundations” for analyses of aggregate behavior) is actually more of a bit of folk-psychology that should not be taken seriously. To me, it is si
4 0.84994584 1469 andrew gelman stats-2012-08-25-Ways of knowing
Introduction: In this discussion from last month, computer science student and Judea Pearl collaborator Elias Barenboim expressed an attitude that hierarchical Bayesian methods might be fine in practice but that they lack theory, that Bayesians can’t succeed in toy problems. I posted a P.S. there which might not have been noticed so I will put it here: I now realize that there is some disagreement about what constitutes a “guarantee.” In one of his comments, Barenboim writes, “the assurance we have that the result must hold as long as the assumptions in the model are correct should be regarded as a guarantee.” In that sense, yes, we have guarantees! It is fundamental to Bayesian inference that the result must hold if the assumptions in the model are correct. We have lots of that in Bayesian Data Analysis (particularly in the first four chapters but implicitly elsewhere as well), and this is also covered in the classic books by Lindley, Jaynes, and others. This sort of guarantee is indeed p
5 0.84071898 1157 andrew gelman stats-2012-02-07-Philosophy of Bayesian statistics: my reactions to Hendry
Introduction: Continuing with my discussion here and here of the articles in the special issue of the journal Rationality, Markets and Morals on the philosophy of Bayesian statistics: David Hendry, “Empirical Economic Model Discovery and Theory Evaluation”: Hendry presents a wide-ranging overview of scientific learning, with an interesting comparison of physical with social sciences. (For some reason, he discusses many physical sciences but restricts his social-science examples to economics and psychology.) The only part of Hendry’s long and interesting article that I will discuss, however, is the part where he decides to take a gratuitous swing at Bayes. I don’t know why he did this, but maybe it’s part of some fraternity initiation thing, like TP-ing the dean’s house on Halloween. Here’s the story. Hendry writes: ‘Prior distributions’ widely used in Bayesian analyses, whether subjective or ‘objective’, cannot be formed in such a setting either, absent a falsely assumed crys
6 0.83983791 114 andrew gelman stats-2010-06-28-More on Bayesian deduction-induction
7 0.83934033 1779 andrew gelman stats-2013-03-27-“Two Dogmas of Strong Objective Bayesianism”
8 0.82729363 244 andrew gelman stats-2010-08-30-Useful models, model checking, and external validation: a mini-discussion
9 0.82203132 291 andrew gelman stats-2010-09-22-Philosophy of Bayes and non-Bayes: A dialogue with Deborah Mayo
10 0.82086319 811 andrew gelman stats-2011-07-20-Kind of Bayesian
11 0.8116287 1205 andrew gelman stats-2012-03-09-Coming to agreement on philosophy of statistics
12 0.80972511 117 andrew gelman stats-2010-06-29-Ya don’t know Bayes, Jack
13 0.80812782 2254 andrew gelman stats-2014-03-18-Those wacky anti-Bayesians used to be intimidating, but now they’re just pathetic
14 0.80547553 792 andrew gelman stats-2011-07-08-The virtues of incoherence?
15 0.7967658 2368 andrew gelman stats-2014-06-11-Bayes in the research conversation
17 0.79105783 1719 andrew gelman stats-2013-02-11-Why waste time philosophizing?
18 0.79031909 1438 andrew gelman stats-2012-07-31-What is a Bayesian?
19 0.78855401 1208 andrew gelman stats-2012-03-11-Gelman on Hennig on Gelman on Bayes
20 0.78701419 1510 andrew gelman stats-2012-09-25-Incoherence of Bayesian data analysis
topicId topicWeight
[(2, 0.011), (15, 0.037), (16, 0.102), (21, 0.024), (24, 0.145), (44, 0.054), (55, 0.02), (63, 0.015), (65, 0.032), (86, 0.053), (87, 0.021), (95, 0.017), (96, 0.016), (99, 0.322)]
simIndex simValue blogId blogTitle
same-blog 1 0.98257136 1529 andrew gelman stats-2012-10-11-Bayesian brains?
Introduction: Psychology researcher Alison Gopnik discusses the idea that some of the systematic problems with human reasoning can be explained by systematic flaws in the statistical models we implicitly use. I really like this idea and I’ll return to it in a bit. But first I need to discuss a minor (but, I think, ultimately crucial) disagreement I have with how Gopnik describes Bayesian inference. She writes: The Bayesian idea is simple, but it turns out to be very powerful. It’s so powerful, in fact, that computer scientists are using it to design intelligent learning machines, and more and more psychologists think that it might explain human intelligence. Bayesian inference is a way to use statistical data to evaluate hypotheses and make predictions. These might be scientific hypotheses and predictions or everyday ones. So far, so good. Next comes the problem (as I see it). Gopnik writes: Here’s a simple bit of Bayesian election thinking. In early September, the polls suddenly im
Introduction: I’ve recently started a regular column on ethics, appearing every three months in Chance magazine . My first column, “Open Data and Open Methods,” is here , and my second column, “Statisticians: When we teach, we don’t practice what we preach” (coauthored with Eric Loken) will be appearing in the next issue. Statistical ethics is a wide-open topic, and I’d be very interested in everyone’s thoughts, questions, and stories. I’d like to get beyond generic questions such as, Is it right to do a randomized trial when you think the treatment is probably better than the control?, and I’d also like to avoid the really easy questions such as, Is it ethical to copy Wikipedia entries and then sell the resulting publication for $2800 a year? [Note to people who are sick of hearing about this particular story: I'll consider stopping my blogging on it, the moment that the people involved consider apologizing for their behavior.] Please insert your thoughts, questions, stories, links, et
3 0.97699392 788 andrew gelman stats-2011-07-06-Early stopping and penalized likelihood
Introduction: Maximum likelihood gives the beat fit to the training data but in general overfits, yielding overly-noisy parameter estimates that don’t perform so well when predicting new data. A popular solution to this overfitting problem takes advantage of the iterative nature of most maximum likelihood algorithms by stopping early. In general, an iterative optimization algorithm goes from a starting point to the maximum of some objective function. If the starting point has some good properties, then early stopping can work well, keeping some of the virtues of the starting point while respecting the data. This trick can be performed the other way, too, starting with the data and then processing it to move it toward a model. That’s how the iterative proportional fitting algorithm of Deming and Stephan (1940) works to fit multivariate categorical data to known margins. In any case, the trick is to stop at the right point–not so soon that you’re ignoring the data but not so late that you en
4 0.97344911 865 andrew gelman stats-2011-08-22-Blogging is “destroying the business model for quality”?
Introduction: Journalist Jonathan Rauch writes that the internet is Sturgeon squared: This is the blogosphere. I’m not getting paid to be here. I’m here to get incredibly famous (in my case, even more incredibly famous) so that I can get paid somewhere else. . . . The average quality of newspapers and (published) novels is far, far better than the average quality of blog posts (and–ugh!–comments). This is because people pay for newspapers and novels. What distinguishes newspapers and novels is how much does not get published in them, because people won’t pay for it. Payment is a filter, and a pretty good one. Imperfect, of course. But pointing out the defects of the old model is merely changing the subject if the new model is worse. . . . Yes, the new model is bringing a lot of new content into being. But most of it is bad. And it’s displacing a lot of better content, by destroying the business model for quality. Even in the information economy, there’s no free lunch. . . . Yes, there’s g
5 0.97342312 2161 andrew gelman stats-2014-01-07-My recent debugging experience
Introduction: OK, so this sort of thing happens sometimes. I was working on a new idea (still working on it; if it ultimately works out—or if it doesn’t—I’ll let you know) and as part of it I was fitting little models in Stan, in a loop. I thought it would make sense to start with linear regression with normal priors and known data variance, because then the exact solution is Gaussian and I can also work with the problem analytically. So I programmed up the algorithm and, no surprise, it didn’t work. I went through my R code, put in print statements here and there, and cleared out bug after bug until at least it stopped crashing. But the algorithm still wasn’t doing what it was supposed to do. So I decided to do something simpler, and just check that the Stan linear regression gave the same answer as the analytic posterior distribution: I ran Stan for tons of iterations, then computed the sample mean and variance of the simulations. It was an example with two coefficients—I’d originally cho
6 0.97296089 2137 andrew gelman stats-2013-12-17-Replication backlash
7 0.97233415 2177 andrew gelman stats-2014-01-19-“The British amateur who debunked the mathematics of happiness”
9 0.97175866 324 andrew gelman stats-2010-10-07-Contest for developing an R package recommendation system
10 0.97174752 1910 andrew gelman stats-2013-06-22-Struggles over the criticism of the “cannabis users and IQ change” paper
11 0.97120148 1878 andrew gelman stats-2013-05-31-How to fix the tabloids? Toward replicable social science research
12 0.97102422 617 andrew gelman stats-2011-03-17-“Why Preschool Shouldn’t Be Like School”?
13 0.97037494 1435 andrew gelman stats-2012-07-30-Retracted articles and unethical behavior in economics journals?
14 0.97026545 2227 andrew gelman stats-2014-02-27-“What Can we Learn from the Many Labs Replication Project?”
15 0.97006989 1266 andrew gelman stats-2012-04-16-Another day, another plagiarist
16 0.96995246 2263 andrew gelman stats-2014-03-24-Empirical implications of Empirical Implications of Theoretical Models
17 0.9693464 1422 andrew gelman stats-2012-07-20-Likelihood thresholds and decisions
18 0.96922523 110 andrew gelman stats-2010-06-26-Philosophy and the practice of Bayesian statistics
19 0.96913671 1760 andrew gelman stats-2013-03-12-Misunderstanding the p-value
20 0.96878296 2223 andrew gelman stats-2014-02-24-“Edlin’s rule” for routinely scaling down published estimates