andrew_gelman_stats andrew_gelman_stats-2011 andrew_gelman_stats-2011-1004 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: In the context of a debate between economists Brad DeLong and Tyler Cowen on the “IS-LM model” [no, I don't know what it is, either!], Kaiser writes : Since a model is an abstraction, a simplification of reality, no model is above critique. I [Kaiser] consider the following types of critique not deserving: 1) The critique that the modeler makes an assumption 2) The critique that the modeler makes an assumption for mathematical convenience 3) The critique that the model omits some feature 4) The critique that the model doesn’t fit one’s intuition 5) The critique that the model fails to make a specific prediction Above all, a serious critique must include an alternative model that is provably better than the one it criticises. It is not enough to show that the alternative solves the problems being pointed out; the alternative must do so while preserving the useful aspects of the model being criticized. I have mixed feelings about Kaiser’s rules. On one hand, I agree wit
sentIndex sentText sentNum sentScore
1 In the context of a debate between economists Brad DeLong and Tyler Cowen on the “IS-LM model” [no, I don't know what it is, either! [sent-1, score-0.06]
2 ], Kaiser writes : Since a model is an abstraction, a simplification of reality, no model is above critique. [sent-2, score-0.762]
3 It is not enough to show that the alternative solves the problems being pointed out; the alternative must do so while preserving the useful aspects of the model being criticized. [sent-4, score-1.243]
4 On one hand, I agree with his point that a model is a practical tool and that an imperfection is no reason to abandon a useful model. [sent-6, score-0.689]
5 On the other hand, I think that much can be learned from rejection of a model, even without reference to any alternative. [sent-7, score-0.081]
6 Let me put it this way: That a model makes assumptions, even that a model makes wrong assumptions, is not news. [sent-8, score-0.976]
7 If “wrong” is enough to kill, then all our models are dead on arrival anyway. [sent-9, score-0.183]
8 But it’s good to understand the ways in which a model disagrees with the data at hand, or with other aspects of reality. [sent-10, score-0.541]
9 As Kuhn and Lakatos knew, highlighting, isolating, and exploring anomalies are crucial steps in moving toward improvement—even if no alternative model is currently in the picture. [sent-11, score-0.865]
wordName wordTfidf (topN-words)
[('critique', 0.558), ('model', 0.329), ('alternative', 0.219), ('modeler', 0.196), ('kaiser', 0.192), ('hand', 0.127), ('makes', 0.122), ('assumption', 0.12), ('imperfection', 0.119), ('abstraction', 0.119), ('isolating', 0.119), ('provably', 0.119), ('aspects', 0.116), ('arrival', 0.112), ('deserving', 0.112), ('omits', 0.108), ('assumptions', 0.104), ('anomalies', 0.104), ('solves', 0.104), ('abandon', 0.104), ('simplification', 0.104), ('delong', 0.101), ('preserving', 0.098), ('highlighting', 0.098), ('disagrees', 0.096), ('kuhn', 0.094), ('lakatos', 0.089), ('fails', 0.086), ('brad', 0.084), ('must', 0.082), ('rejection', 0.081), ('convenience', 0.08), ('kill', 0.079), ('exploring', 0.077), ('useful', 0.076), ('feelings', 0.075), ('wrong', 0.074), ('crucial', 0.072), ('dead', 0.071), ('improvement', 0.069), ('intuition', 0.069), ('mixed', 0.068), ('reality', 0.064), ('steps', 0.064), ('feature', 0.064), ('types', 0.063), ('cowen', 0.062), ('tool', 0.061), ('tyler', 0.061), ('debate', 0.06)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 1004 andrew gelman stats-2011-11-11-Kaiser Fung on how not to critique models
Introduction: In the context of a debate between economists Brad DeLong and Tyler Cowen on the “IS-LM model” [no, I don't know what it is, either!], Kaiser writes : Since a model is an abstraction, a simplification of reality, no model is above critique. I [Kaiser] consider the following types of critique not deserving: 1) The critique that the modeler makes an assumption 2) The critique that the modeler makes an assumption for mathematical convenience 3) The critique that the model omits some feature 4) The critique that the model doesn’t fit one’s intuition 5) The critique that the model fails to make a specific prediction Above all, a serious critique must include an alternative model that is provably better than the one it criticises. It is not enough to show that the alternative solves the problems being pointed out; the alternative must do so while preserving the useful aspects of the model being criticized. I have mixed feelings about Kaiser’s rules. On one hand, I agree wit
Introduction: In response to my remarks on his online book, Think Bayes, Allen Downey wrote: I [Downey] have a question about one of your comments: My [Gelman's] main criticism with both books is that they talk a lot about inference but not so much about model building or model checking (recall the three steps of Bayesian data analysis). I think it’s ok for an introductory book to focus on inference, which of course is central to the data-analytic process—but I’d like them to at least mention that Bayesian ideas arise in model building and model checking as well. This sounds like something I agree with, and one of the things I tried to do in the book is to put modeling decisions front and center. But the word “modeling” is used in lots of ways, so I want to see if we are talking about the same thing. For example, in many chapters, I start with a simple model of the scenario, do some analysis, then check whether the model is good enough, and iterate. Here’s the discussion of modeling
3 0.14520721 781 andrew gelman stats-2011-06-28-The holes in my philosophy of Bayesian data analysis
Introduction: I’ve been writing a lot about my philosophy of Bayesian statistics and how it fits into Popper’s ideas about falsification and Kuhn’s ideas about scientific revolutions. Here’s my long, somewhat technical paper with Cosma Shalizi. Here’s our shorter overview for the volume on the philosophy of social science. Here’s my latest try (for an online symposium), focusing on the key issues. I’m pretty happy with my approach–the familiar idea that Bayesian data analysis iterates the three steps of model building, inference, and model checking–but it does have some unresolved (maybe unresolvable) problems. Here are a couple mentioned in the third of the above links. Consider a simple model with independent data y_1, y_2, .., y_10 ~ N(θ,σ^2), with a prior distribution θ ~ N(0,10^2) and σ known and taking on some value of approximately 10. Inference about μ is straightforward, as is model checking, whether based on graphs or numerical summaries such as the sample variance and skewn
4 0.14019462 1817 andrew gelman stats-2013-04-21-More on Bayesian model selection in high-dimensional settings
Introduction: David Rossell writes: A friend pointed out that you were having an interesting philosophical discussion on my paper with Val Johnson [on Bayesian model selection in high-dimensional settings]. I agree with the view that in almost all practical situations the true model is not in the set under consideration. Still, asking a model choice procedure to be able to pick up the correct model when it is in the set under consideration seems a minimal requirement (though perhaps not sufficient). In other words, if a procedure is unable to pick the data-generating model even when it is one of the models under consideration, I don’t have high hopes for it working well in more realistic scenarios either. Most results in the history in statistics seem to have been obtained under an assumed model, e.g. why even do MLE or penalized-likelihood if we don’t trust the model. While unrealistic, these results were useful to help understand important basic principles. In our case Val and I are defe
5 0.1381516 754 andrew gelman stats-2011-06-09-Difficulties with Bayesian model averaging
Introduction: In response to this article by Cosma Shalizi and myself on the philosophy of Bayesian statistics, David Hogg writes: I [Hogg] agree–even in physics and astronomy–that the models are not “True” in the God-like sense of being absolute reality (that is, I am not a realist); and I have argued (a philosophically very naive paper, but hey, I was new to all this) that for pretty fundamental reasons we could never arrive at the True (with a capital “T”) model of the Universe. The goal of inference is to find the “best” model, where “best” might have something to do with prediction, or explanation, or message length, or (horror!) our utility. Needless to say, most of my physics friends *are* realists, even in the face of “effective theories” as Newtonian mechanics is an effective theory of GR and GR is an effective theory of “quantum gravity” (this plays to your point, because if you think any theory is possibly an effective theory, how could you ever find Truth?). I also liked the i
6 0.13189295 1392 andrew gelman stats-2012-06-26-Occam
7 0.13035506 244 andrew gelman stats-2010-08-30-Useful models, model checking, and external validation: a mini-discussion
8 0.12918283 1999 andrew gelman stats-2013-08-27-Bayesian model averaging or fitting a larger model
9 0.12908533 780 andrew gelman stats-2011-06-27-Bridges between deterministic and probabilistic models for binary data
10 0.12438065 2133 andrew gelman stats-2013-12-13-Flexibility is good
12 0.11917171 320 andrew gelman stats-2010-10-05-Does posterior predictive model checking fit with the operational subjective approach?
13 0.11602131 1431 andrew gelman stats-2012-07-27-Overfitting
14 0.11406988 1527 andrew gelman stats-2012-10-10-Another reason why you can get good inferences from a bad model
15 0.11212133 2186 andrew gelman stats-2014-01-26-Infoviz on top of stat graphic on top of spreadsheet
16 0.11080108 388 andrew gelman stats-2010-11-01-The placebo effect in pharma
17 0.10633472 2007 andrew gelman stats-2013-09-03-Popper and Jaynes
18 0.10469859 1141 andrew gelman stats-2012-01-28-Using predator-prey models on the Canadian lynx series
19 0.10415211 24 andrew gelman stats-2010-05-09-Special journal issue on statistical methods for the social sciences
20 0.10171846 1247 andrew gelman stats-2012-04-05-More philosophy of Bayes
topicId topicWeight
[(0, 0.145), (1, 0.116), (2, 0.005), (3, 0.049), (4, 0.001), (5, -0.019), (6, -0.009), (7, -0.001), (8, 0.149), (9, 0.047), (10, -0.015), (11, 0.062), (12, -0.085), (13, -0.007), (14, -0.135), (15, -0.013), (16, 0.006), (17, 0.065), (18, -0.009), (19, 0.005), (20, 0.015), (21, -0.05), (22, -0.054), (23, -0.102), (24, -0.046), (25, 0.032), (26, -0.007), (27, 0.001), (28, -0.004), (29, 0.027), (30, -0.081), (31, -0.082), (32, 0.002), (33, 0.073), (34, 0.008), (35, -0.015), (36, -0.004), (37, -0.045), (38, 0.007), (39, -0.059), (40, -0.004), (41, 0.019), (42, 0.004), (43, 0.031), (44, 0.022), (45, 0.008), (46, -0.002), (47, -0.039), (48, -0.0), (49, 0.03)]
simIndex simValue blogId blogTitle
same-blog 1 0.96394962 1004 andrew gelman stats-2011-11-11-Kaiser Fung on how not to critique models
Introduction: In the context of a debate between economists Brad DeLong and Tyler Cowen on the “IS-LM model” [no, I don't know what it is, either!], Kaiser writes : Since a model is an abstraction, a simplification of reality, no model is above critique. I [Kaiser] consider the following types of critique not deserving: 1) The critique that the modeler makes an assumption 2) The critique that the modeler makes an assumption for mathematical convenience 3) The critique that the model omits some feature 4) The critique that the model doesn’t fit one’s intuition 5) The critique that the model fails to make a specific prediction Above all, a serious critique must include an alternative model that is provably better than the one it criticises. It is not enough to show that the alternative solves the problems being pointed out; the alternative must do so while preserving the useful aspects of the model being criticized. I have mixed feelings about Kaiser’s rules. On one hand, I agree wit
2 0.8406707 2133 andrew gelman stats-2013-12-13-Flexibility is good
Introduction: If I made a separate post for each interesting blog discussion, we’d get overwhelmed. That’s why I often leave detailed responses in the comments section, even though I’m pretty sure that most readers don’t look in the comments at all. Sometimes, though, I think it’s good to bring such discussions to light. Here’s a recent example. Michael wrote : Poor predictive performance usually indicates that the model isn’t sufficiently flexible to explain the data, and my understanding of the proper Bayesian strategy is to feed that back into your original model and try again until you achieve better performance. Corey replied : It was my impression that — in ML at least — poor predictive performance is more often due to the model being too flexible and fitting noise. And Rahul agreed : Good point. A very flexible model will describe your training data perfectly and then go bonkers when unleashed on wild data. But I wrote : Overfitting comes from a model being flex
3 0.81210166 780 andrew gelman stats-2011-06-27-Bridges between deterministic and probabilistic models for binary data
Introduction: For the analysis of binary data, various deterministic models have been proposed, which are generally simpler to fit and easier to understand than probabilistic models. We claim that corresponding to any deterministic model is an implicit stochastic model in which the deterministic model fits imperfectly, with errors occurring at random. In the context of binary data, we consider a model in which the probability of error depends on the model prediction. We show how to fit this model using a stochastic modification of deterministic optimization schemes. The advantages of fitting the stochastic model explicitly (rather than implicitly, by simply fitting a deterministic model and accepting the occurrence of errors) include quantification of uncertainty in the deterministic model’s parameter estimates, better estimation of the true model error rate, and the ability to check the fit of the model nontrivially. We illustrate this with a simple theoretical example of item response data and w
4 0.81154066 1817 andrew gelman stats-2013-04-21-More on Bayesian model selection in high-dimensional settings
Introduction: David Rossell writes: A friend pointed out that you were having an interesting philosophical discussion on my paper with Val Johnson [on Bayesian model selection in high-dimensional settings]. I agree with the view that in almost all practical situations the true model is not in the set under consideration. Still, asking a model choice procedure to be able to pick up the correct model when it is in the set under consideration seems a minimal requirement (though perhaps not sufficient). In other words, if a procedure is unable to pick the data-generating model even when it is one of the models under consideration, I don’t have high hopes for it working well in more realistic scenarios either. Most results in the history in statistics seem to have been obtained under an assumed model, e.g. why even do MLE or penalized-likelihood if we don’t trust the model. While unrealistic, these results were useful to help understand important basic principles. In our case Val and I are defe
Introduction: In response to my remarks on his online book, Think Bayes, Allen Downey wrote: I [Downey] have a question about one of your comments: My [Gelman's] main criticism with both books is that they talk a lot about inference but not so much about model building or model checking (recall the three steps of Bayesian data analysis). I think it’s ok for an introductory book to focus on inference, which of course is central to the data-analytic process—but I’d like them to at least mention that Bayesian ideas arise in model building and model checking as well. This sounds like something I agree with, and one of the things I tried to do in the book is to put modeling decisions front and center. But the word “modeling” is used in lots of ways, so I want to see if we are talking about the same thing. For example, in many chapters, I start with a simple model of the scenario, do some analysis, then check whether the model is good enough, and iterate. Here’s the discussion of modeling
6 0.80744088 448 andrew gelman stats-2010-12-03-This is a footnote in one of my papers
7 0.79254282 1141 andrew gelman stats-2012-01-28-Using predator-prey models on the Canadian lynx series
8 0.78337938 24 andrew gelman stats-2010-05-09-Special journal issue on statistical methods for the social sciences
9 0.78044152 754 andrew gelman stats-2011-06-09-Difficulties with Bayesian model averaging
10 0.7793591 554 andrew gelman stats-2011-02-04-An addition to the model-makers’ oath
11 0.77576435 2007 andrew gelman stats-2013-09-03-Popper and Jaynes
12 0.77307332 1392 andrew gelman stats-2012-06-26-Occam
13 0.77099359 614 andrew gelman stats-2011-03-15-Induction within a model, deductive inference for model evaluation
14 0.76145768 1162 andrew gelman stats-2012-02-11-Adding an error model to a deterministic model
15 0.75983524 1999 andrew gelman stats-2013-08-27-Bayesian model averaging or fitting a larger model
16 0.75714666 1521 andrew gelman stats-2012-10-04-Columbo does posterior predictive checks
17 0.75269008 1431 andrew gelman stats-2012-07-27-Overfitting
18 0.75009692 823 andrew gelman stats-2011-07-26-Including interactions or not
19 0.74353456 217 andrew gelman stats-2010-08-19-The “either-or” fallacy of believing in discrete models: an example of folk statistics
20 0.74222344 328 andrew gelman stats-2010-10-08-Displaying a fitted multilevel model
topicId topicWeight
[(5, 0.022), (7, 0.014), (9, 0.014), (16, 0.048), (24, 0.241), (33, 0.013), (34, 0.024), (36, 0.011), (72, 0.033), (76, 0.011), (83, 0.013), (86, 0.058), (92, 0.147), (94, 0.014), (95, 0.011), (99, 0.212)]
simIndex simValue blogId blogTitle
same-blog 1 0.94045615 1004 andrew gelman stats-2011-11-11-Kaiser Fung on how not to critique models
Introduction: In the context of a debate between economists Brad DeLong and Tyler Cowen on the “IS-LM model” [no, I don't know what it is, either!], Kaiser writes : Since a model is an abstraction, a simplification of reality, no model is above critique. I [Kaiser] consider the following types of critique not deserving: 1) The critique that the modeler makes an assumption 2) The critique that the modeler makes an assumption for mathematical convenience 3) The critique that the model omits some feature 4) The critique that the model doesn’t fit one’s intuition 5) The critique that the model fails to make a specific prediction Above all, a serious critique must include an alternative model that is provably better than the one it criticises. It is not enough to show that the alternative solves the problems being pointed out; the alternative must do so while preserving the useful aspects of the model being criticized. I have mixed feelings about Kaiser’s rules. On one hand, I agree wit
2 0.90955591 1563 andrew gelman stats-2012-11-05-Someone is wrong on the internet, part 2
Introduction: My coblogger John Sides feeds a troll. It’s a tough call. Yesterday I gave my reasoning for ignoring these provocateurs, but in this case the troll in question is writing for a major newspaper so it makes sense for John to go to the trouble of shooting him down. Even though I suspect the columnist was trolling for no better reason than . . . he had a deadline and nothing to say so he thought he’d wade into a controversy. On the plus side, as a statistician I’m happy that statistics is considered important enough that it’s worth trolling! When they start attacking like this, they must feel a bit on the defensive. . . .
Introduction: Last year we discussed an important challenge in causal inference: The standard advice (given in many books, including ours) for causal inference is to control for relevant pre-treatment variables as much as possible. But, as Judea Pearl has pointed out, instruments (as in “instrumental variables”) are pre-treatment variables that we would not want to “control for” in a matching or regression sense. At first, this seems like a minor modification, with the new recommendation being to apply instrumental variables estimation using all pre-treatment instruments, and to control for all other pre-treatment variables. But that can’t really work as general advice. What about weak instruments or covariates that have some instrumental aspects? I asked Paul Rosenbaum for his thoughts on the matter, and he wrote the following: In section 18.2 of Design of Observational Studies (DOS), I [Rosenbaum] discuss “seemingly innocuous confounding” defined to be a covariate that predicts a su
4 0.900576 2231 andrew gelman stats-2014-03-03-Running into a Stan Reference by Accident
Introduction: We were talking about parallelizing MCMC and I came up with what I thought was a neat idea for parallelizing MCMC (sample with fractional prior, average samples on a per-draw basis). But then I realized this approach could get the right posterior mean or right posterior variance, but not both, depending on how the prior was divided (for a beta-binomial example). Then Aki told me it had already been done in a more general form in a paper of Scott et al., Bayes and Big Data , which was then used as the baseline in: Willie Neiswanger, Chong Wang, and Eric Xing. 2013. Asymptotically Exact, Embarrassingly Parallel MCMC . arXiv 1311.4780. It’s a neat paper, which Xi’an already blogged about months ago. But what really struck me was the following quote: We use Stan, an automated Hamiltonian Monte Carlo (HMC) software package, to perform sampling for both the true posterior (for groundtruth and comparison methods) and for the subposteriors on each machine. One advantage o
5 0.89687192 1785 andrew gelman stats-2013-04-02-So much artistic talent
Introduction: I saw this excellent art show the other day, and it reminded me how much artistic talent is out there. I really have no idea whassup with those all-black canvases and the other stuff you see at modern art museums, given that there’s so much interesting new stuff being created every year. I see a big difference between art made by people who feel they have something they want to say, compared to art being made by people who feel they are supposed to make art because they’re artists. And there’s also the internal logic of art responding to other art, as Tom Wolfe discussed in The Painted Word.
6 0.89541078 953 andrew gelman stats-2011-10-11-Steve Jobs’s cancer and science-based medicine
7 0.8951146 491 andrew gelman stats-2010-12-29-Don’t try this at home
8 0.89441121 1368 andrew gelman stats-2012-06-06-Question 27 of my final exam for Design and Analysis of Sample Surveys
9 0.89204466 779 andrew gelman stats-2011-06-25-Avoiding boundary estimates using a prior distribution as regularization
10 0.89158118 1455 andrew gelman stats-2012-08-12-Probabilistic screening to get an approximate self-weighted sample
11 0.89094824 2017 andrew gelman stats-2013-09-11-“Informative g-Priors for Logistic Regression”
12 0.89061791 1376 andrew gelman stats-2012-06-12-Simple graph WIN: the example of birthday frequencies
13 0.88921368 846 andrew gelman stats-2011-08-09-Default priors update?
15 0.88751346 197 andrew gelman stats-2010-08-10-The last great essayist?
16 0.88703567 1999 andrew gelman stats-2013-08-27-Bayesian model averaging or fitting a larger model
18 0.8862536 1367 andrew gelman stats-2012-06-05-Question 26 of my final exam for Design and Analysis of Sample Surveys
19 0.88601935 1062 andrew gelman stats-2011-12-16-Mr. Pearson, meet Mr. Mandelbrot: Detecting Novel Associations in Large Data Sets
20 0.88456947 278 andrew gelman stats-2010-09-15-Advice that might make sense for individuals but is negative-sum overall