andrew_gelman_stats andrew_gelman_stats-2010 andrew_gelman_stats-2010-72 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: With the completion of the last edition of Jose Bernardo’s Valencia (Spain) conference on Bayesian statistics–I didn’t attend, but many of my friends were there–I thought I’d share my strongest memory of the Valencia conference that I attended in 1991. I contributed a poster and a discussion, both on the topic of inference from iterative simulation, but what I remember most vividly, and what bothered me the most, was how little interest there was in checking model fit. Not only had people mostly not checked the fit of their models to data, and not only did they seem uninterested in such checks, even worse was that many of these Bayesians felt that it was basically illegal to check model fit. I don’t want to get too down on Bayesians for this. Lots of non-Bayesian statisticians go around not checking their models too. With Bayes, though, model checking seems particularly important because Bayesians rely on their models so strongly, not just as a way of getting point estimates bu
sentIndex sentText sentNum sentScore
1 With the completion of the last edition of Jose Bernardo’s Valencia (Spain) conference on Bayesian statistics–I didn’t attend, but many of my friends were there–I thought I’d share my strongest memory of the Valencia conference that I attended in 1991. [sent-1, score-1.335]
2 I contributed a poster and a discussion, both on the topic of inference from iterative simulation, but what I remember most vividly, and what bothered me the most, was how little interest there was in checking model fit. [sent-2, score-1.025]
3 Not only had people mostly not checked the fit of their models to data, and not only did they seem uninterested in such checks, even worse was that many of these Bayesians felt that it was basically illegal to check model fit. [sent-3, score-1.012]
4 Lots of non-Bayesian statisticians go around not checking their models too. [sent-5, score-0.42]
5 With Bayes, though, model checking seems particularly important because Bayesians rely on their models so strongly, not just as a way of getting point estimates but to get full probability distributions. [sent-6, score-0.543]
6 I remember feeling very frustrated and disillusioned at that 1991 conference, to see all these people who seemed to have no interest in going back to first principles and thinking about what they were doing. [sent-7, score-0.755]
7 After that, most people are just stuck in their ways. [sent-9, score-0.157]
8 The above were just my reactions, and I’m sure that others since then have had similar reactions to my own mistakes. [sent-13, score-0.184]
wordName wordTfidf (topN-words)
[('valencia', 0.362), ('conference', 0.296), ('bayesians', 0.278), ('checking', 0.229), ('reactions', 0.184), ('jose', 0.165), ('bernardo', 0.155), ('disillusioned', 0.155), ('vividly', 0.149), ('spain', 0.143), ('uninterested', 0.143), ('completion', 0.136), ('remember', 0.136), ('poster', 0.127), ('strongest', 0.127), ('attend', 0.125), ('models', 0.124), ('attended', 0.119), ('frustrated', 0.119), ('illegal', 0.116), ('iterative', 0.116), ('contributed', 0.114), ('interest', 0.113), ('virtue', 0.109), ('memory', 0.108), ('checks', 0.103), ('grad', 0.099), ('edition', 0.097), ('model', 0.097), ('checked', 0.097), ('simulation', 0.094), ('claiming', 0.094), ('rely', 0.093), ('bothered', 0.093), ('stuck', 0.089), ('mistakes', 0.086), ('principles', 0.084), ('friends', 0.081), ('feeling', 0.08), ('felt', 0.077), ('basically', 0.077), ('mostly', 0.075), ('share', 0.075), ('strongly', 0.073), ('bayes', 0.073), ('worse', 0.072), ('special', 0.071), ('people', 0.068), ('statisticians', 0.067), ('check', 0.066)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 72 andrew gelman stats-2010-06-07-Valencia: Summer of 1991
Introduction: With the completion of the last edition of Jose Bernardo’s Valencia (Spain) conference on Bayesian statistics–I didn’t attend, but many of my friends were there–I thought I’d share my strongest memory of the Valencia conference that I attended in 1991. I contributed a poster and a discussion, both on the topic of inference from iterative simulation, but what I remember most vividly, and what bothered me the most, was how little interest there was in checking model fit. Not only had people mostly not checked the fit of their models to data, and not only did they seem uninterested in such checks, even worse was that many of these Bayesians felt that it was basically illegal to check model fit. I don’t want to get too down on Bayesians for this. Lots of non-Bayesian statisticians go around not checking their models too. With Bayes, though, model checking seems particularly important because Bayesians rely on their models so strongly, not just as a way of getting point estimates bu
2 0.15269503 1719 andrew gelman stats-2013-02-11-Why waste time philosophizing?
Introduction: I’ll answer the above question after first sharing some background and history on the the philosophy of Bayesian statistics, which appeared at the end of our rejoinder to the discussion to which I linked the other day: When we were beginning our statistical educations, the word ‘Bayesian’ conveyed membership in an obscure cult. Statisticians who were outside the charmed circle could ignore the Bayesian subfield, while Bayesians themselves tended to be either apologetic or brazenly defiant. These two extremes manifested themselves in ever more elaborate proposals for non-informative priors, on the one hand, and declarations of the purity of subjective probability, on the other. Much has changed in the past 30 years. ‘Bayesian’ is now often used in casual scientific parlance as a synonym for ‘rational’, the anti-Bayesians have mostly disappeared, and non-Bayesian statisticians feel the need to keep up with developments in Bayesian modelling and computation. Bayesians themselves
3 0.14833015 110 andrew gelman stats-2010-06-26-Philosophy and the practice of Bayesian statistics
Introduction: Here’s an article that I believe is flat-out entertaining to read. It’s about philosophy, so it’s supposed to be entertaining, in any case. Here’s the abstract: A substantial school in the philosophy of science identifies Bayesian inference with inductive inference and even rationality as such, and seems to be strengthened by the rise and practical success of Bayesian statistics. We argue that the most successful forms of Bayesian statistics do not actually support that particular philosophy but rather accord much better with sophisticated forms of hypothetico-deductivism. We examine the actual role played by prior distributions in Bayesian models, and the crucial aspects of model checking and model revision, which fall outside the scope of Bayesian confirmation theory. We draw on the literature on the consistency of Bayesian updating and also on our experience of applied work in social science. Clarity about these matters should benefit not just philosophy of science, but
4 0.13685355 1469 andrew gelman stats-2012-08-25-Ways of knowing
Introduction: In this discussion from last month, computer science student and Judea Pearl collaborator Elias Barenboim expressed an attitude that hierarchical Bayesian methods might be fine in practice but that they lack theory, that Bayesians can’t succeed in toy problems. I posted a P.S. there which might not have been noticed so I will put it here: I now realize that there is some disagreement about what constitutes a “guarantee.” In one of his comments, Barenboim writes, “the assurance we have that the result must hold as long as the assumptions in the model are correct should be regarded as a guarantee.” In that sense, yes, we have guarantees! It is fundamental to Bayesian inference that the result must hold if the assumptions in the model are correct. We have lots of that in Bayesian Data Analysis (particularly in the first four chapters but implicitly elsewhere as well), and this is also covered in the classic books by Lindley, Jaynes, and others. This sort of guarantee is indeed p
5 0.13279223 1560 andrew gelman stats-2012-11-03-Statistical methods that work in some settings but not others
Introduction: David Hogg pointed me to this post by Larry Wasserman: 1. The Horwitz-Thompson estimator satisfies the following condition: for every , where — the parameter space — is the set of all functions . (There are practical improvements to the Horwitz-Thompson estimator that we discussed in our earlier posts but we won’t revisit those here.) 2. A Bayes estimator requires a prior for . In general, if is not a function of then (1) will not hold. . . . 3. If you let be a function if , (1) still, in general, does not hold. 4. If you make a function if in just the right way, then (1) will hold. . . . There is nothing wrong with doing this, but in our opinion this is not in the spirit of Bayesian inference. . . . 7. This example is only meant to show that Bayesian estimators do not necessarily have good frequentist properties. This should not be surprising. There is no reason why we should in general expect a Bayesian method to have a frequentist property
6 0.13051027 2254 andrew gelman stats-2014-03-18-Those wacky anti-Bayesians used to be intimidating, but now they’re just pathetic
9 0.124929 291 andrew gelman stats-2010-09-22-Philosophy of Bayes and non-Bayes: A dialogue with Deborah Mayo
10 0.12490034 788 andrew gelman stats-2011-07-06-Early stopping and penalized likelihood
11 0.1242364 754 andrew gelman stats-2011-06-09-Difficulties with Bayesian model averaging
12 0.123535 1695 andrew gelman stats-2013-01-28-Economists argue about Bayes
13 0.11917462 117 andrew gelman stats-2010-06-29-Ya don’t know Bayes, Jack
14 0.11830396 1911 andrew gelman stats-2013-06-23-AI Stats conference on Stan etc.
15 0.11824013 24 andrew gelman stats-2010-05-09-Special journal issue on statistical methods for the social sciences
16 0.11548857 1482 andrew gelman stats-2012-09-04-Model checking and model understanding in machine learning
17 0.11330587 1438 andrew gelman stats-2012-07-31-What is a Bayesian?
18 0.11256739 453 andrew gelman stats-2010-12-07-Biostatistics via Pragmatic and Perceptive Bayes.
19 0.11101837 781 andrew gelman stats-2011-06-28-The holes in my philosophy of Bayesian data analysis
20 0.10650849 244 andrew gelman stats-2010-08-30-Useful models, model checking, and external validation: a mini-discussion
topicId topicWeight
[(0, 0.165), (1, 0.072), (2, -0.065), (3, 0.069), (4, -0.038), (5, 0.039), (6, -0.016), (7, 0.026), (8, 0.074), (9, -0.016), (10, 0.018), (11, 0.005), (12, -0.069), (13, -0.0), (14, -0.014), (15, -0.003), (16, 0.051), (17, -0.02), (18, -0.013), (19, 0.031), (20, -0.009), (21, -0.025), (22, -0.011), (23, -0.045), (24, -0.018), (25, -0.02), (26, -0.056), (27, -0.02), (28, -0.018), (29, -0.007), (30, -0.03), (31, 0.022), (32, 0.011), (33, -0.018), (34, 0.004), (35, -0.009), (36, -0.007), (37, 0.014), (38, -0.005), (39, 0.013), (40, 0.021), (41, 0.013), (42, 0.005), (43, 0.057), (44, 0.007), (45, -0.035), (46, -0.016), (47, -0.016), (48, -0.017), (49, -0.013)]
simIndex simValue blogId blogTitle
same-blog 1 0.97594994 72 andrew gelman stats-2010-06-07-Valencia: Summer of 1991
Introduction: With the completion of the last edition of Jose Bernardo’s Valencia (Spain) conference on Bayesian statistics–I didn’t attend, but many of my friends were there–I thought I’d share my strongest memory of the Valencia conference that I attended in 1991. I contributed a poster and a discussion, both on the topic of inference from iterative simulation, but what I remember most vividly, and what bothered me the most, was how little interest there was in checking model fit. Not only had people mostly not checked the fit of their models to data, and not only did they seem uninterested in such checks, even worse was that many of these Bayesians felt that it was basically illegal to check model fit. I don’t want to get too down on Bayesians for this. Lots of non-Bayesian statisticians go around not checking their models too. With Bayes, though, model checking seems particularly important because Bayesians rely on their models so strongly, not just as a way of getting point estimates bu
2 0.84088898 614 andrew gelman stats-2011-03-15-Induction within a model, deductive inference for model evaluation
Introduction: Jonathan Livengood writes: I have a couple of questions on your paper with Cosma Shalizi on “Philosophy and the practice of Bayesian statistics.” First, you distinguish between inductive approaches and hypothetico-deductive approaches to inference and locate statistical practice (at least, the practice of model building and checking) on the hypothetico-deductive side. Do you think that there are any interesting elements of statistical practice that are properly inductive? For example, suppose someone is playing around with a system that more or less resembles a toy model, like drawing balls from an urn or some such, and where the person has some well-defined priors. The person makes a number of draws from the urn and applies Bayes theorem to get a posterior. On your view, is that person making an induction? If so, how much space is there in statistical practice for genuine inductions like this? Second, I agree with you that one ought to distinguish induction from other kind
Introduction: David Rohde writes: I have been thinking a lot lately about your Bayesian model checking approach. This is in part because I have been working on exploratory data analysis and wishing to avoid controversy and mathematical statistics we omitted model checking from our discussion. This is something that the refereeing process picked us up on and we ultimately added a critical discussion of null-hypothesis testing to our paper . The exploratory technique we discussed was essentially a 2D histogram approach, but we used Polya models as a formal model for the histogram. We are currently working on a new paper, and we are thinking through how or if we should do “confirmatory analysis” or model checking in the paper. What I find most admirable about your statistical work is that you clearly use the Bayesian approach to do useful applied statistical analysis. My own attempts at applied Bayesian analysis makes me greatly admire your applied successes. On the other hand it may be t
4 0.82948911 1041 andrew gelman stats-2011-12-04-David MacKay and Occam’s Razor
Introduction: In my comments on David MacKay’s 2003 book on Bayesian inference, I wrote that I hate all the Occam-factor stuff that MacKay talks about, and I linked to this quote from Radford Neal: Sometimes a simple model will outperform a more complex model . . . Nevertheless, I believe that deliberately limiting the complexity of the model is not fruitful when the problem is evidently complex. Instead, if a simple model is found that outperforms some particular complex model, the appropriate response is to define a different complex model that captures whatever aspect of the problem led to the simple model performing well. MacKay replied as follows: When you said you disagree with me on Occam factors I think what you meant was that you agree with me on them. I’ve read your post on the topic and completely agreed with you (and Radford) that we should be using models the size of a house, models that we believe in, and that anyone who thinks it is a good idea to bias the model toward
5 0.82485598 244 andrew gelman stats-2010-08-30-Useful models, model checking, and external validation: a mini-discussion
Introduction: I sent a copy of my paper (coauthored with Cosma Shalizi) on Philosophy and the practice of Bayesian statistics in the social sciences to Richard Berk , who wrote: I read your paper this morning. I think we are pretty much on the same page about all models being wrong. I like very much the way you handle this in the paper. Yes, Newton’s work is wrong, but surely useful. I also like your twist on Bayesian methods. Makes good sense to me. Perhaps most important, your paper raises some difficult issues I have been trying to think more carefully about. 1. If the goal of a model is to be useful, surely we need to explore that “useful” means. At the very least, usefulness will depend on use. So a model that is useful for forecasting may or may not be useful for causal inference. 2. Usefulness will be a matter of degree. So that for each use we will need one or more metrics to represent how useful the model is. In what looks at first to be simple example, if the use is forecasting,
6 0.8208949 1856 andrew gelman stats-2013-05-14-GPstuff: Bayesian Modeling with Gaussian Processes
7 0.81551868 193 andrew gelman stats-2010-08-09-Besag
8 0.81068468 110 andrew gelman stats-2010-06-26-Philosophy and the practice of Bayesian statistics
9 0.81022668 811 andrew gelman stats-2011-07-20-Kind of Bayesian
10 0.80930382 496 andrew gelman stats-2011-01-01-Tukey’s philosophy
11 0.80701542 1392 andrew gelman stats-2012-06-26-Occam
13 0.7982772 1719 andrew gelman stats-2013-02-11-Why waste time philosophizing?
14 0.79776919 1521 andrew gelman stats-2012-10-04-Columbo does posterior predictive checks
15 0.79497677 964 andrew gelman stats-2011-10-19-An interweaving-transformation strategy for boosting MCMC efficiency
16 0.79260612 1510 andrew gelman stats-2012-09-25-Incoherence of Bayesian data analysis
17 0.78544104 754 andrew gelman stats-2011-06-09-Difficulties with Bayesian model averaging
18 0.78203321 781 andrew gelman stats-2011-06-28-The holes in my philosophy of Bayesian data analysis
19 0.77811635 575 andrew gelman stats-2011-02-15-What are the trickiest models to fit?
20 0.77540618 1205 andrew gelman stats-2012-03-09-Coming to agreement on philosophy of statistics
topicId topicWeight
[(3, 0.014), (14, 0.016), (16, 0.074), (21, 0.011), (24, 0.056), (32, 0.013), (51, 0.017), (61, 0.173), (66, 0.015), (69, 0.012), (70, 0.015), (82, 0.025), (86, 0.049), (90, 0.014), (95, 0.011), (99, 0.392)]
simIndex simValue blogId blogTitle
1 0.98382461 9 andrew gelman stats-2010-04-28-But it all goes to pay for gas, car insurance, and tolls on the turnpike
Introduction: As a New Yorker I think I’m obliged to pass on the occasional Jersey joke (most recently, this one , which annoyingly continues to attract spam comments). I’ll let the above title be my comment on this entry from Tyler Cowen entitled, “Which Americans are ‘best off’?”: If you consult human development indices the answer is Asians living in New Jersey. The standard is: The index factors in life expectancy at birth, educational degree attainment among adults 25-years or older, school enrollment for people at least three years old and median annual gross personal earnings. More generally, these sorts of rankings and ndexes seem to be cheap ways of grabbing headlines. This has always irritated me but really maybe I should go with the flow and invent a few of these indexes myself.
2 0.96496183 21 andrew gelman stats-2010-05-07-Environmentally induced cancer “grossly underestimated”? Doubtful.
Introduction: The (U.S.) “President’s Cancer Panel” has released its 2008-2009 annual report, which includes a cover letter that says “the true burden of environmentally induced cancer has been grossly underestimated.” The report itself discusses exposures to various types of industrial chemicals, some of which are known carcinogens, in some detail, but gives nearly no data or analysis to suggest that these exposures are contributing to significant numbers of cancers. In fact, there is pretty good evidence that they are not. The plot above shows age-adjusted cancer mortality for men, by cancer type, in the U.S. The plot below shows the same for women. In both cases, the cancers with the highest mortality rates are shown, but not all cancers (e.g. brain cancer is not shown). For what it’s worth, I’m not sure how trustworthy the rates are from the 1930s — it seems possible that reporting, autopsies, or both, were less careful during the Great Depression — so I suggest focusing on the r
3 0.96262443 1714 andrew gelman stats-2013-02-09-Partial least squares path analysis
Introduction: Wayne Folta writes: I [Folta] was looking for R packages to address a project I’m working on and stumbled onto a package called ‘plspm’. It seems to be a nice package, but the thing I wanted to pass on is the PDF that Gaston Sanchez, its author, wrote that describes PLS Path Analysis in general and shows how to use plspm in particular. It’s like a 200-page R vignette that’s really informative and fun to read. I’d recommend it to you and your readers: even if you don’t want to delve into PLS and plspm deeply, the first seven pages and the Appendix A provide a great read about a grad student, PLS Path Analysis, and the history of the field. It’s written at a more popular level than you might like. For example, he says at one point: “A moderating effect is the fancy term that some authors use to say that there is a nosy variable M influencing the effect between an independent variable X and a dependent variable Y.” You would obviously never write anything like that [yup --- AG]
4 0.95751309 2349 andrew gelman stats-2014-05-26-WAIC and cross-validation in Stan!
Introduction: Aki and I write : The Watanabe-Akaike information criterion (WAIC) and cross-validation are methods for estimating pointwise out-of-sample prediction accuracy from a fitted Bayesian model. WAIC is based on the series expansion of leave-one-out cross-validation (LOO), and asymptotically they are equal. With finite data, WAIC and cross-validation address different predictive questions and thus it is useful to be able to compute both. WAIC and an importance-sampling approximated LOO can be estimated directly using the log-likelihood evaluated at the posterior simulations of the parameter values. We show how to compute WAIC, IS-LOO, K-fold cross-validation, and related diagnostic quantities in the Bayesian inference package Stan as called from R. This is important, I think. One reason the deviance information criterion (DIC) has been so popular is its implementation in Bugs. We think WAIC and cross-validation make more sense than DIC, especially from a Bayesian perspective in whic
5 0.95303285 1028 andrew gelman stats-2011-11-26-Tenure lets you handle students who cheat
Introduction: The other day, a friend of mine who is an untenured professor (not in statistics or political science) was telling me about a class where many of the students seemed to be resubmitting papers that they had already written for previous classes. (The supposition was based on internal evidence of the topics of the submitted papers.) It would be possible to check this and then kick the cheating students out of the program—but why do it? It would be a lot of work, also some of the students who are caught might complain, then word would get around that my friend is a troublemaker. And nobody likes a troublemaker. Once my friend has tenure it would be possible to do the right thing. But . . . here’s the hitch: most college instructors do not have tenure, and one result, I suspect, is a decline in ethical standards. This is something I hadn’t thought of in our earlier discussion of job security for teachers: tenure gives you the freedom to kick out cheating students.
same-blog 6 0.9510991 72 andrew gelman stats-2010-06-07-Valencia: Summer of 1991
7 0.94934583 714 andrew gelman stats-2011-05-16-NYT Labs releases Openpaths, a utility for saving your iphone data
8 0.94699407 1558 andrew gelman stats-2012-11-02-Not so fast on levees and seawalls for NY harbor?
9 0.94294262 2070 andrew gelman stats-2013-10-20-The institution of tenure
10 0.94242269 2134 andrew gelman stats-2013-12-14-Oswald evidence
11 0.93979514 561 andrew gelman stats-2011-02-06-Poverty, educational performance – and can be done about it
12 0.93976831 1975 andrew gelman stats-2013-08-09-Understanding predictive information criteria for Bayesian models
13 0.93782258 1370 andrew gelman stats-2012-06-07-Duncan Watts and the Titanic
14 0.93603611 827 andrew gelman stats-2011-07-28-Amusing case of self-defeating science writing
15 0.93323779 776 andrew gelman stats-2011-06-22-Deviance, DIC, AIC, cross-validation, etc
16 0.93287247 16 andrew gelman stats-2010-05-04-Burgess on Kipling
17 0.93173975 767 andrew gelman stats-2011-06-15-Error in an attribution of an error
19 0.92369503 2033 andrew gelman stats-2013-09-23-More on Bayesian methods and multilevel modeling
20 0.91717708 604 andrew gelman stats-2011-03-08-More on the missing conservative psychology researchers