andrew_gelman_stats andrew_gelman_stats-2011 andrew_gelman_stats-2011-1092 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: A couple days ago we discussed some remarks by Tony O’Hagan and Jim Berger on weakly informative priors. Jim followed up on Deborah Mayo’s blog with this: Objective Bayesian priors are often improper (i.e., have infinite total mass), but this is not a problem when they are developed correctly. But not every improper prior is satisfactory. For instance, the constant prior is known to be unsatisfactory in many situations. The ‘solution’ pseudo-Bayesians often use is to choose a constant prior over a large but bounded set (a ‘weakly informative’ prior), saying it is now proper and so all is well. This is not true; if the constant prior on the whole parameter space is bad, so will be the constant prior over the bounded set. The problem is, in part, that some people confuse proper priors with subjective priors and, having learned that true subjective priors are fine, incorrectly presume that weakly informative proper priors are fine. I have a few reactions to this: 1. I agree
sentIndex sentText sentNum sentScore
1 A couple days ago we discussed some remarks by Tony O’Hagan and Jim Berger on weakly informative priors. [sent-1, score-0.61]
2 Jim followed up on Deborah Mayo’s blog with this: Objective Bayesian priors are often improper (i. [sent-2, score-0.608]
3 For instance, the constant prior is known to be unsatisfactory in many situations. [sent-6, score-0.587]
4 The ‘solution’ pseudo-Bayesians often use is to choose a constant prior over a large but bounded set (a ‘weakly informative’ prior), saying it is now proper and so all is well. [sent-7, score-1.014]
5 This is not true; if the constant prior on the whole parameter space is bad, so will be the constant prior over the bounded set. [sent-8, score-1.395]
6 The problem is, in part, that some people confuse proper priors with subjective priors and, having learned that true subjective priors are fine, incorrectly presume that weakly informative proper priors are fine. [sent-9, score-3.197]
7 I agree with Berger that improper priors can sometimes work OK. [sent-11, score-0.536]
8 I’m not sure who Berger’s “pseudo-Bayesians” are, but I agree that it’s not a good idea to simply use a flat but bounded prior distribution. [sent-14, score-0.664]
9 I don’t know if this helps, but when I do a Google Scholar search on weakly informative priors, the top two hits are my own papers, where we indeed get more reasonable and stable estimates than would be obtained using priors that are traditionally considered noninformative. [sent-18, score-1.059]
10 Berger writes that some people have “learned that true subjective priors are fine. [sent-20, score-0.713]
11 Or, more to the point, in any case, how do you know that a subjective prior is actually “true”? [sent-23, score-0.575]
12 I often do find it helpful or even necessary to include prior information in an analysis, but it would be too much to ask me to supply anything like a “true subjective prior. [sent-24, score-0.775]
13 ” The idea of weakly informative priors is to get some (but not all) of the benefit of the prior information while mitigating some (but not all) of the risk of including information that’s not really there . [sent-25, score-1.69]
14 We took real prior information from the scientific literature and we showed how it could be used in a Bayesian or non-Bayesian fashion (the latter using retrospective design analysis to show that, in the problem at hand, any plausible signal would be overwhelmed by noise). [sent-28, score-0.587]
15 Berger considers weakly informative priors to be a particularly bad choice involving flat distributions over a constrained space. [sent-36, score-1.145]
16 I define weakly informative priors as containing real prior information, just less than one might have for any particular problem. [sent-37, score-1.425]
17 Since I actually use these priors in my own applied work, I think I should get to decide the name! [sent-38, score-0.478]
18 Beyond this, I think there’s a statistical point that Berger is missing, which is that there’s something between attempts at noninformativeness (on one hand) and a fully informative prior (on the other). [sent-40, score-0.69]
19 To me, that’s the key idea of weak prior information: as O’Hagan writes, it can often be “difficult to formulate a genuine prior distribution carefully” (or, as I might but it, difficult to set up a full probability model). [sent-41, score-1.033]
20 But at that point we don’t need to retreat to noninformativity; we can take a halfway point and set up a weakly informative prior that includes some, but not all, the substantive information that is available. [sent-42, score-1.296]
wordName wordTfidf (topN-words)
[('priors', 0.405), ('prior', 0.368), ('weakly', 0.337), ('berger', 0.283), ('informative', 0.273), ('subjective', 0.207), ('constant', 0.165), ('bounded', 0.157), ('hagan', 0.153), ('proper', 0.133), ('improper', 0.131), ('information', 0.128), ('jim', 0.106), ('space', 0.102), ('true', 0.101), ('distribution', 0.075), ('use', 0.073), ('beauty', 0.072), ('often', 0.072), ('parameter', 0.07), ('term', 0.07), ('purposes', 0.069), ('flat', 0.066), ('constraints', 0.065), ('bad', 0.064), ('objective', 0.062), ('weak', 0.059), ('sex', 0.055), ('unridiculous', 0.054), ('noninformativity', 0.054), ('unsatisfactory', 0.054), ('model', 0.053), ('mitigating', 0.051), ('learned', 0.05), ('point', 0.049), ('weakliem', 0.049), ('problem', 0.047), ('set', 0.046), ('presume', 0.046), ('halfway', 0.046), ('purity', 0.046), ('genuine', 0.045), ('considered', 0.044), ('overwhelmed', 0.044), ('tony', 0.044), ('confuse', 0.043), ('spaces', 0.043), ('unconstrained', 0.043), ('particular', 0.042), ('bayesian', 0.042)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 1092 andrew gelman stats-2011-12-29-More by Berger and me on weakly informative priors
Introduction: A couple days ago we discussed some remarks by Tony O’Hagan and Jim Berger on weakly informative priors. Jim followed up on Deborah Mayo’s blog with this: Objective Bayesian priors are often improper (i.e., have infinite total mass), but this is not a problem when they are developed correctly. But not every improper prior is satisfactory. For instance, the constant prior is known to be unsatisfactory in many situations. The ‘solution’ pseudo-Bayesians often use is to choose a constant prior over a large but bounded set (a ‘weakly informative’ prior), saying it is now proper and so all is well. This is not true; if the constant prior on the whole parameter space is bad, so will be the constant prior over the bounded set. The problem is, in part, that some people confuse proper priors with subjective priors and, having learned that true subjective priors are fine, incorrectly presume that weakly informative proper priors are fine. I have a few reactions to this: 1. I agree
Introduction: Deborah Mayo sent me this quote from Jim Berger: Too often I see people pretending to be subjectivists, and then using “weakly informative” priors that the objective Bayesian community knows are terrible and will give ridiculous answers; subjectivism is then being used as a shield to hide ignorance. . . . In my own more provocative moments, I claim that the only true subjectivists are the objective Bayesians, because they refuse to use subjectivism as a shield against criticism of sloppy pseudo-Bayesian practice. This caught my attention because I’ve become more and more convinced that weakly informative priors are the right way to go in many different situations. I don’t think Berger was talking about me , though, as the above quote came from a publication in 2006, at which time I’d only started writing about weakly informative priors. Going back to Berger’s article , I see that his “weakly informative priors” remark was aimed at this article by Anthony O’Hagan, who w
3 0.46216294 1155 andrew gelman stats-2012-02-05-What is a prior distribution?
Introduction: Some recent blog discussion revealed some confusion that I’ll try to resolve here. I wrote that I’m not a big fan of subjective priors. Various commenters had difficulty with this point, and I think the issue was most clearly stated by Bill Jeff re erys, who wrote : It seems to me that your prior has to reflect your subjective information before you look at the data. How can it not? But this does not mean that the (subjective) prior that you choose is irrefutable; Surely a prior that reflects prior information just does not have to be inconsistent with that information. But that still leaves a range of priors that are consistent with it, the sort of priors that one would use in a sensitivity analysis, for example. I think I see what Bill is getting at. A prior represents your subjective belief, or some approximation to your subjective belief, even if it’s not perfect. That sounds reasonable but I don’t think it works. Or, at least, it often doesn’t work. Let’s start
4 0.34949902 2109 andrew gelman stats-2013-11-21-Hidden dangers of noninformative priors
Introduction: Following up on Christian’s post [link fixed] on the topic, I’d like to offer a few thoughts of my own. In BDA, we express the idea that a noninformative prior is a placeholder: you can use the noninformative prior to get the analysis started, then if your posterior distribution is less informative than you would like, or if it does not make sense, you can go back and add prior information. Same thing for the data model (the “likelihood”), for that matter: it often makes sense to start with something simple and conventional and then go from there. So, in that sense, noninformative priors are no big deal, they’re just a way to get started. Just don’t take them too seriously. Traditionally in statistics we’ve worked with the paradigm of a single highly informative dataset with only weak external information. But if the data are sparse and prior information is strong, we have to think differently. And, when you increase the dimensionality of a problem, both these things hap
5 0.34855571 846 andrew gelman stats-2011-08-09-Default priors update?
Introduction: Ryan King writes: I was wondering if you have a brief comment on the state of the art for objective priors for hierarchical generalized linear models (generalized linear mixed models). I have been working off the papers in Bayesian Analysis (2006) 1, Number 3 (Browne and Draper, Kass and Natarajan, Gelman). There seems to have been continuous work for matching priors in linear mixed models, but GLMMs less so because of the lack of an analytic marginal likelihood for the variance components. There are a number of additional suggestions in the literature since 2006, but little robust practical guidance. I’m interested in both mean parameters and the variance components. I’m almost always concerned with logistic random effect models. I’m fascinated by the matching-priors idea of higher-order asymptotic improvements to maximum likelihood, and need to make some kind of defensible default recommendation. Given the massive scale of the datasets (genetics …), extensive sensitivity a
6 0.34804156 1946 andrew gelman stats-2013-07-19-Prior distributions on derived quantities rather than on parameters themselves
7 0.34265903 1941 andrew gelman stats-2013-07-16-Priors
8 0.30675375 1486 andrew gelman stats-2012-09-07-Prior distributions for regression coefficients
9 0.30394846 1858 andrew gelman stats-2013-05-15-Reputations changeable, situations tolerable
10 0.28803426 1454 andrew gelman stats-2012-08-11-Weakly informative priors for Bayesian nonparametric models?
11 0.27965978 1695 andrew gelman stats-2013-01-28-Economists argue about Bayes
12 0.27730545 2017 andrew gelman stats-2013-09-11-“Informative g-Priors for Logistic Regression”
13 0.27273247 1474 andrew gelman stats-2012-08-29-More on scaled-inverse Wishart and prior independence
14 0.26729867 1149 andrew gelman stats-2012-02-01-Philosophy of Bayesian statistics: my reactions to Cox and Mayo
15 0.26372305 801 andrew gelman stats-2011-07-13-On the half-Cauchy prior for a global scale parameter
16 0.25552526 779 andrew gelman stats-2011-06-25-Avoiding boundary estimates using a prior distribution as regularization
17 0.25313762 1046 andrew gelman stats-2011-12-07-Neutral noninformative and informative conjugate beta and gamma prior distributions
18 0.24099196 2200 andrew gelman stats-2014-02-05-Prior distribution for a predicted probability
19 0.22855923 1465 andrew gelman stats-2012-08-21-D. Buggin
20 0.22744067 468 andrew gelman stats-2010-12-15-Weakly informative priors and imprecise probabilities
topicId topicWeight
[(0, 0.23), (1, 0.28), (2, -0.001), (3, 0.115), (4, -0.111), (5, -0.151), (6, 0.245), (7, 0.066), (8, -0.325), (9, 0.131), (10, 0.03), (11, 0.016), (12, 0.123), (13, 0.08), (14, -0.022), (15, -0.011), (16, 0.006), (17, 0.015), (18, 0.034), (19, 0.036), (20, -0.102), (21, -0.072), (22, -0.068), (23, 0.03), (24, -0.031), (25, 0.024), (26, 0.076), (27, -0.05), (28, -0.059), (29, 0.048), (30, 0.014), (31, -0.078), (32, 0.014), (33, -0.001), (34, -0.029), (35, 0.053), (36, 0.026), (37, 0.027), (38, 0.048), (39, 0.014), (40, -0.008), (41, -0.05), (42, 0.09), (43, 0.027), (44, 0.015), (45, 0.022), (46, -0.069), (47, -0.038), (48, -0.034), (49, 0.047)]
simIndex simValue blogId blogTitle
same-blog 1 0.98335469 1092 andrew gelman stats-2011-12-29-More by Berger and me on weakly informative priors
Introduction: A couple days ago we discussed some remarks by Tony O’Hagan and Jim Berger on weakly informative priors. Jim followed up on Deborah Mayo’s blog with this: Objective Bayesian priors are often improper (i.e., have infinite total mass), but this is not a problem when they are developed correctly. But not every improper prior is satisfactory. For instance, the constant prior is known to be unsatisfactory in many situations. The ‘solution’ pseudo-Bayesians often use is to choose a constant prior over a large but bounded set (a ‘weakly informative’ prior), saying it is now proper and so all is well. This is not true; if the constant prior on the whole parameter space is bad, so will be the constant prior over the bounded set. The problem is, in part, that some people confuse proper priors with subjective priors and, having learned that true subjective priors are fine, incorrectly presume that weakly informative proper priors are fine. I have a few reactions to this: 1. I agree
Introduction: Deborah Mayo sent me this quote from Jim Berger: Too often I see people pretending to be subjectivists, and then using “weakly informative” priors that the objective Bayesian community knows are terrible and will give ridiculous answers; subjectivism is then being used as a shield to hide ignorance. . . . In my own more provocative moments, I claim that the only true subjectivists are the objective Bayesians, because they refuse to use subjectivism as a shield against criticism of sloppy pseudo-Bayesian practice. This caught my attention because I’ve become more and more convinced that weakly informative priors are the right way to go in many different situations. I don’t think Berger was talking about me , though, as the above quote came from a publication in 2006, at which time I’d only started writing about weakly informative priors. Going back to Berger’s article , I see that his “weakly informative priors” remark was aimed at this article by Anthony O’Hagan, who w
3 0.93165565 1858 andrew gelman stats-2013-05-15-Reputations changeable, situations tolerable
Introduction: David Kessler, Peter Hoff, and David Dunson write : Marginally specified priors for nonparametric Bayesian estimation Prior specification for nonparametric Bayesian inference involves the difficult task of quantifying prior knowledge about a parameter of high, often infinite, dimension. Realistically, a statistician is unlikely to have informed opinions about all aspects of such a parameter, but may have real information about functionals of the parameter, such the population mean or variance. This article proposes a new framework for nonparametric Bayes inference in which the prior distribution for a possibly infinite-dimensional parameter is decomposed into two parts: an informative prior on a finite set of functionals, and a nonparametric conditional prior for the parameter given the functionals. Such priors can be easily constructed from standard nonparametric prior distributions in common use, and inherit the large support of the standard priors upon which they are based. Ad
4 0.9229238 1155 andrew gelman stats-2012-02-05-What is a prior distribution?
Introduction: Some recent blog discussion revealed some confusion that I’ll try to resolve here. I wrote that I’m not a big fan of subjective priors. Various commenters had difficulty with this point, and I think the issue was most clearly stated by Bill Jeff re erys, who wrote : It seems to me that your prior has to reflect your subjective information before you look at the data. How can it not? But this does not mean that the (subjective) prior that you choose is irrefutable; Surely a prior that reflects prior information just does not have to be inconsistent with that information. But that still leaves a range of priors that are consistent with it, the sort of priors that one would use in a sensitivity analysis, for example. I think I see what Bill is getting at. A prior represents your subjective belief, or some approximation to your subjective belief, even if it’s not perfect. That sounds reasonable but I don’t think it works. Or, at least, it often doesn’t work. Let’s start
Introduction: Jouni Kerman did a cool bit of research justifying the Beta (1/3, 1/3) prior as noninformative for binomial data, and the Gamma (1/3, 0) prior for Poisson data. You probably thought that nothing new could be said about noninformative priors in such basic problems, but you were wrong! Here’s the story : The conjugate binomial and Poisson models are commonly used for estimating proportions or rates. However, it is not well known that the conventional noninformative conjugate priors tend to shrink the posterior quantiles toward the boundary or toward the middle of the parameter space, making them thus appear excessively informative. The shrinkage is always largest when the number of observed events is small. This behavior persists for all sample sizes and exposures. The effect of the prior is therefore most conspicuous and potentially controversial when analyzing rare events. As alternative default conjugate priors, I [Jouni] introduce Beta(1/3, 1/3) and Gamma(1/3, 0), which I cal
6 0.89432591 468 andrew gelman stats-2010-12-15-Weakly informative priors and imprecise probabilities
7 0.87016505 2138 andrew gelman stats-2013-12-18-In Memoriam Dennis Lindley
8 0.86300272 801 andrew gelman stats-2011-07-13-On the half-Cauchy prior for a global scale parameter
9 0.86223793 1474 andrew gelman stats-2012-08-29-More on scaled-inverse Wishart and prior independence
10 0.8589918 1946 andrew gelman stats-2013-07-19-Prior distributions on derived quantities rather than on parameters themselves
11 0.8497417 2109 andrew gelman stats-2013-11-21-Hidden dangers of noninformative priors
12 0.84763098 846 andrew gelman stats-2011-08-09-Default priors update?
13 0.8409825 1454 andrew gelman stats-2012-08-11-Weakly informative priors for Bayesian nonparametric models?
14 0.83688229 2017 andrew gelman stats-2013-09-11-“Informative g-Priors for Logistic Regression”
15 0.78537697 1486 andrew gelman stats-2012-09-07-Prior distributions for regression coefficients
16 0.78431892 1941 andrew gelman stats-2013-07-16-Priors
17 0.77441114 669 andrew gelman stats-2011-04-19-The mysterious Gamma (1.4, 0.4)
18 0.7728492 1465 andrew gelman stats-2012-08-21-D. Buggin
19 0.76805562 779 andrew gelman stats-2011-06-25-Avoiding boundary estimates using a prior distribution as regularization
20 0.7597872 1130 andrew gelman stats-2012-01-20-Prior beliefs about locations of decision boundaries
topicId topicWeight
[(15, 0.012), (16, 0.054), (21, 0.013), (24, 0.528), (47, 0.011), (66, 0.01), (84, 0.017), (86, 0.012), (95, 0.017), (96, 0.01), (99, 0.191)]
simIndex simValue blogId blogTitle
Introduction: Jouni Kerman did a cool bit of research justifying the Beta (1/3, 1/3) prior as noninformative for binomial data, and the Gamma (1/3, 0) prior for Poisson data. You probably thought that nothing new could be said about noninformative priors in such basic problems, but you were wrong! Here’s the story : The conjugate binomial and Poisson models are commonly used for estimating proportions or rates. However, it is not well known that the conventional noninformative conjugate priors tend to shrink the posterior quantiles toward the boundary or toward the middle of the parameter space, making them thus appear excessively informative. The shrinkage is always largest when the number of observed events is small. This behavior persists for all sample sizes and exposures. The effect of the prior is therefore most conspicuous and potentially controversial when analyzing rare events. As alternative default conjugate priors, I [Jouni] introduce Beta(1/3, 1/3) and Gamma(1/3, 0), which I cal
2 0.99309063 240 andrew gelman stats-2010-08-29-ARM solutions
Introduction: People sometimes email asking if a solution set is available for the exercises in ARM. The answer, unfortunately, is no. Many years ago, I wrote up 50 solutions for BDA and it was a lot of work–really, it was like writing a small book in itself. The trouble is that, once I started writing them up, I wanted to do it right, to set a good example. That’s a lot more effort than simply scrawling down some quick answers.
3 0.99257857 643 andrew gelman stats-2011-04-02-So-called Bayesian hypothesis testing is just as bad as regular hypothesis testing
Introduction: Steve Ziliak points me to this article by the always-excellent Carl Bialik, slamming hypothesis tests. I only wish Carl had talked with me before so hastily posting, though! I would’ve argued with some of the things in the article. In particular, he writes: Reese and Brad Carlin . . . suggest that Bayesian statistics are a better alternative, because they tackle the probability that the hypothesis is true head-on, and incorporate prior knowledge about the variables involved. Brad Carlin does great work in theory, methods, and applications, and I like the bit about the prior knowledge (although I might prefer the more general phrase “additional information”), but I hate that quote! My quick response is that the hypothesis of zero effect is almost never true! The problem with the significance testing framework–Bayesian or otherwise–is in the obsession with the possibility of an exact zero effect. The real concern is not with zero, it’s with claiming a positive effect whe
4 0.99045873 545 andrew gelman stats-2011-01-30-New innovations in spam
Introduction: I received the following (unsolicited) email today: Hello Andrew, I’m interested in whether you are accepting guest article submissions for your site Statistical Modeling, Causal Inference, and Social Science? I’m the owner of the recently created nonprofit site OnlineEngineeringDegree.org and am interested in writing / submitting an article for your consideration to be published on your site. Is that something you’d be willing to consider, and if so, what specs in terms of topics or length requirements would you be looking for? Thanks you for your time, and if you have any questions or are interested, I’d appreciate you letting me know. Sincerely, Samantha Rhodes Huh? P.S. My vote for most obnoxious spam remains this one , which does its best to dilute whatever remains of the reputation of Wolfram Research. Or maybe that particular bit of spam was written by a particularly awesome cellular automaton that Wolfram discovered? I guess in the world of big-time software
5 0.98898041 1437 andrew gelman stats-2012-07-31-Paying survey respondents
Introduction: I agree with Casey Mulligan that participants in government surveys should be paid, and I think it should be part of the code of ethics for commercial pollsters to compensate their respondents also. As Mulligan points out, if a survey is worth doing, it should be worth compensating the participants for their time and effort. P.S. Just to clarify, I do not recommend that Census surveys be made voluntary, I just think that respondents (who can be required to participate) should be paid a small amount. P.P.S. More rant here .
6 0.98628432 471 andrew gelman stats-2010-12-17-Attractive models (and data) wanted for statistical art show.
7 0.98196471 38 andrew gelman stats-2010-05-18-Breastfeeding, infant hyperbilirubinemia, statistical graphics, and modern medicine
9 0.97576439 938 andrew gelman stats-2011-10-03-Comparing prediction errors
10 0.97536319 241 andrew gelman stats-2010-08-29-Ethics and statistics in development research
same-blog 11 0.97515053 1092 andrew gelman stats-2011-12-29-More by Berger and me on weakly informative priors
12 0.97413021 1479 andrew gelman stats-2012-09-01-Mothers and Moms
13 0.96844113 1787 andrew gelman stats-2013-04-04-Wanna be the next Tyler Cowen? It’s not as easy as you might think!
14 0.96751559 482 andrew gelman stats-2010-12-23-Capitalism as a form of voluntarism
15 0.96498591 2229 andrew gelman stats-2014-02-28-God-leaf-tree
17 0.95630062 743 andrew gelman stats-2011-06-03-An argument that can’t possibly make sense
18 0.95439655 59 andrew gelman stats-2010-05-30-Extended Binary Format Support for Mac OS X
20 0.94621813 373 andrew gelman stats-2010-10-27-It’s better than being forwarded the latest works of you-know-who