andrew_gelman_stats andrew_gelman_stats-2010 andrew_gelman_stats-2010-468 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Giorgio Corani writes: Your work on weakly informative priors is close to some research I [Corani] did (together with Prof. Zaffalon) in the last years using the so-called imprecise probabilities. The idea is to work with a set of priors (containing even very different priors); to update them via Bayes’ rule and then compute a set of posteriors. The set of priors is convex and the priors are Dirichlet (thus, conjugate to the likelihood); this allows to compute the set of posteriors exactly and efficiently. I [Corani] have used this approach for classification, extending naive Bayes and TAN to imprecise probabilities. Classifiers based on imprecise probabilities return more classes when they find that the most probable class is prior-dependent, i.e., if picking different priors in the convex set leads to identify different classes as the most probable one. Instead of returning a single (unreliable) prior-dependent class, credal classifiers in this case preserve reliability by
sentIndex sentText sentNum sentScore
1 Giorgio Corani writes: Your work on weakly informative priors is close to some research I [Corani] did (together with Prof. [sent-1, score-0.488]
2 Zaffalon) in the last years using the so-called imprecise probabilities. [sent-2, score-0.449]
3 The idea is to work with a set of priors (containing even very different priors); to update them via Bayes’ rule and then compute a set of posteriors. [sent-3, score-1.039]
4 The set of priors is convex and the priors are Dirichlet (thus, conjugate to the likelihood); this allows to compute the set of posteriors exactly and efficiently. [sent-4, score-1.621]
5 I [Corani] have used this approach for classification, extending naive Bayes and TAN to imprecise probabilities. [sent-5, score-0.67]
6 Classifiers based on imprecise probabilities return more classes when they find that the most probable class is prior-dependent, i. [sent-6, score-0.94]
7 , if picking different priors in the convex set leads to identify different classes as the most probable one. [sent-8, score-1.33]
8 Instead of returning a single (unreliable) prior-dependent class, credal classifiers in this case preserve reliability by issuing a set-valued classification. [sent-9, score-0.672]
9 By experiments, we have consistently found that Bayesian classifiers are unreliable on the instances which are classified in an indeterminate way (but reliably) by our classifiers. [sent-10, score-0.834]
10 It’s not an approach I’ve ever thought about much (nor do I really have the time to think about it now, unfortunately), but I thought I’d post the link to some papers for those of you who might be interested. [sent-12, score-0.184]
11 Imprecise priors have been proposed as a method for encoding weak prior information, so perhaps there is something important there. [sent-13, score-0.587]
wordName wordTfidf (topN-words)
[('imprecise', 0.449), ('corani', 0.37), ('priors', 0.367), ('classifiers', 0.304), ('convex', 0.196), ('unreliable', 0.177), ('probable', 0.174), ('set', 0.159), ('compute', 0.125), ('classes', 0.121), ('issuing', 0.112), ('giorgio', 0.112), ('indeterminate', 0.112), ('tan', 0.101), ('bayes', 0.099), ('encoding', 0.098), ('classified', 0.095), ('conjugate', 0.095), ('preserve', 0.092), ('dirichlet', 0.092), ('posteriors', 0.092), ('reliably', 0.09), ('returning', 0.087), ('class', 0.083), ('containing', 0.083), ('extending', 0.081), ('classification', 0.08), ('reliability', 0.077), ('instances', 0.074), ('consistently', 0.072), ('approach', 0.072), ('weakly', 0.07), ('naive', 0.068), ('different', 0.067), ('picking', 0.066), ('potentially', 0.064), ('weak', 0.061), ('proposed', 0.061), ('update', 0.061), ('allows', 0.061), ('return', 0.057), ('leads', 0.057), ('identify', 0.056), ('probabilities', 0.056), ('thought', 0.056), ('rule', 0.052), ('informative', 0.051), ('unfortunately', 0.05), ('experiments', 0.05), ('via', 0.049)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 468 andrew gelman stats-2010-12-15-Weakly informative priors and imprecise probabilities
Introduction: Giorgio Corani writes: Your work on weakly informative priors is close to some research I [Corani] did (together with Prof. Zaffalon) in the last years using the so-called imprecise probabilities. The idea is to work with a set of priors (containing even very different priors); to update them via Bayes’ rule and then compute a set of posteriors. The set of priors is convex and the priors are Dirichlet (thus, conjugate to the likelihood); this allows to compute the set of posteriors exactly and efficiently. I [Corani] have used this approach for classification, extending naive Bayes and TAN to imprecise probabilities. Classifiers based on imprecise probabilities return more classes when they find that the most probable class is prior-dependent, i.e., if picking different priors in the convex set leads to identify different classes as the most probable one. Instead of returning a single (unreliable) prior-dependent class, credal classifiers in this case preserve reliability by
2 0.22744067 1092 andrew gelman stats-2011-12-29-More by Berger and me on weakly informative priors
Introduction: A couple days ago we discussed some remarks by Tony O’Hagan and Jim Berger on weakly informative priors. Jim followed up on Deborah Mayo’s blog with this: Objective Bayesian priors are often improper (i.e., have infinite total mass), but this is not a problem when they are developed correctly. But not every improper prior is satisfactory. For instance, the constant prior is known to be unsatisfactory in many situations. The ‘solution’ pseudo-Bayesians often use is to choose a constant prior over a large but bounded set (a ‘weakly informative’ prior), saying it is now proper and so all is well. This is not true; if the constant prior on the whole parameter space is bad, so will be the constant prior over the bounded set. The problem is, in part, that some people confuse proper priors with subjective priors and, having learned that true subjective priors are fine, incorrectly presume that weakly informative proper priors are fine. I have a few reactions to this: 1. I agree
Introduction: Deborah Mayo sent me this quote from Jim Berger: Too often I see people pretending to be subjectivists, and then using “weakly informative” priors that the objective Bayesian community knows are terrible and will give ridiculous answers; subjectivism is then being used as a shield to hide ignorance. . . . In my own more provocative moments, I claim that the only true subjectivists are the objective Bayesians, because they refuse to use subjectivism as a shield against criticism of sloppy pseudo-Bayesian practice. This caught my attention because I’ve become more and more convinced that weakly informative priors are the right way to go in many different situations. I don’t think Berger was talking about me , though, as the above quote came from a publication in 2006, at which time I’d only started writing about weakly informative priors. Going back to Berger’s article , I see that his “weakly informative priors” remark was aimed at this article by Anthony O’Hagan, who w
4 0.1657767 1877 andrew gelman stats-2013-05-30-Infill asymptotics and sprawl asymptotics
Introduction: Anirban Bhattacharya, Debdeep Pati, Natesh Pillai, and David Dunson write : Penalized regression methods, such as L1 regularization, are routinely used in high-dimensional applications, and there is a rich literature on optimality properties under sparsity assumptions. In the Bayesian paradigm, sparsity is routinely induced through two-component mixture priors having a probability mass at zero, but such priors encounter daunting computational problems in high dimensions. This has motivated an amazing variety of continuous shrinkage priors, which can be expressed as global-local scale mixtures of Gaussians, facilitating computation. In sharp contrast to the corresponding frequentist literature, very little is known about the properties of such priors. Focusing on a broad class of shrinkage priors, we provide precise results on prior and posterior concentration. Interestingly, we demonstrate that most commonly used shrinkage priors, including the Bayesian Lasso, are suboptimal in hig
5 0.16130751 846 andrew gelman stats-2011-08-09-Default priors update?
Introduction: Ryan King writes: I was wondering if you have a brief comment on the state of the art for objective priors for hierarchical generalized linear models (generalized linear mixed models). I have been working off the papers in Bayesian Analysis (2006) 1, Number 3 (Browne and Draper, Kass and Natarajan, Gelman). There seems to have been continuous work for matching priors in linear mixed models, but GLMMs less so because of the lack of an analytic marginal likelihood for the variance components. There are a number of additional suggestions in the literature since 2006, but little robust practical guidance. I’m interested in both mean parameters and the variance components. I’m almost always concerned with logistic random effect models. I’m fascinated by the matching-priors idea of higher-order asymptotic improvements to maximum likelihood, and need to make some kind of defensible default recommendation. Given the massive scale of the datasets (genetics …), extensive sensitivity a
7 0.15825027 1946 andrew gelman stats-2013-07-19-Prior distributions on derived quantities rather than on parameters themselves
8 0.15004501 1858 andrew gelman stats-2013-05-15-Reputations changeable, situations tolerable
9 0.14232399 801 andrew gelman stats-2011-07-13-On the half-Cauchy prior for a global scale parameter
10 0.14076276 1486 andrew gelman stats-2012-09-07-Prior distributions for regression coefficients
11 0.12629497 1209 andrew gelman stats-2012-03-12-As a Bayesian I want scientists to report their data non-Bayesianly
12 0.12498356 1454 andrew gelman stats-2012-08-11-Weakly informative priors for Bayesian nonparametric models?
13 0.11891926 2129 andrew gelman stats-2013-12-10-Cross-validation and Bayesian estimation of tuning parameters
14 0.11247364 1825 andrew gelman stats-2013-04-25-It’s binless! A program for computing normalizing functions
15 0.10995802 2109 andrew gelman stats-2013-11-21-Hidden dangers of noninformative priors
16 0.10874852 850 andrew gelman stats-2011-08-11-Understanding how estimates change when you move to a multilevel model
17 0.10759702 1465 andrew gelman stats-2012-08-21-D. Buggin
18 0.10496546 1674 andrew gelman stats-2013-01-15-Prior Selection for Vector Autoregressions
19 0.10483045 1999 andrew gelman stats-2013-08-27-Bayesian model averaging or fitting a larger model
20 0.10408042 1155 andrew gelman stats-2012-02-05-What is a prior distribution?
topicId topicWeight
[(0, 0.105), (1, 0.103), (2, -0.016), (3, 0.04), (4, -0.037), (5, -0.031), (6, 0.094), (7, 0.028), (8, -0.138), (9, 0.042), (10, 0.029), (11, 0.019), (12, 0.056), (13, 0.026), (14, 0.019), (15, -0.022), (16, 0.006), (17, 0.013), (18, -0.01), (19, 0.017), (20, -0.038), (21, -0.014), (22, -0.022), (23, 0.023), (24, -0.007), (25, 0.003), (26, 0.048), (27, 0.004), (28, 0.018), (29, 0.019), (30, 0.013), (31, -0.042), (32, 0.021), (33, -0.011), (34, -0.015), (35, -0.017), (36, -0.001), (37, 0.011), (38, 0.032), (39, 0.021), (40, -0.016), (41, -0.004), (42, 0.067), (43, 0.008), (44, -0.008), (45, -0.001), (46, -0.049), (47, -0.0), (48, -0.02), (49, 0.022)]
simIndex simValue blogId blogTitle
same-blog 1 0.97290778 468 andrew gelman stats-2010-12-15-Weakly informative priors and imprecise probabilities
Introduction: Giorgio Corani writes: Your work on weakly informative priors is close to some research I [Corani] did (together with Prof. Zaffalon) in the last years using the so-called imprecise probabilities. The idea is to work with a set of priors (containing even very different priors); to update them via Bayes’ rule and then compute a set of posteriors. The set of priors is convex and the priors are Dirichlet (thus, conjugate to the likelihood); this allows to compute the set of posteriors exactly and efficiently. I [Corani] have used this approach for classification, extending naive Bayes and TAN to imprecise probabilities. Classifiers based on imprecise probabilities return more classes when they find that the most probable class is prior-dependent, i.e., if picking different priors in the convex set leads to identify different classes as the most probable one. Instead of returning a single (unreliable) prior-dependent class, credal classifiers in this case preserve reliability by
2 0.89828116 1092 andrew gelman stats-2011-12-29-More by Berger and me on weakly informative priors
Introduction: A couple days ago we discussed some remarks by Tony O’Hagan and Jim Berger on weakly informative priors. Jim followed up on Deborah Mayo’s blog with this: Objective Bayesian priors are often improper (i.e., have infinite total mass), but this is not a problem when they are developed correctly. But not every improper prior is satisfactory. For instance, the constant prior is known to be unsatisfactory in many situations. The ‘solution’ pseudo-Bayesians often use is to choose a constant prior over a large but bounded set (a ‘weakly informative’ prior), saying it is now proper and so all is well. This is not true; if the constant prior on the whole parameter space is bad, so will be the constant prior over the bounded set. The problem is, in part, that some people confuse proper priors with subjective priors and, having learned that true subjective priors are fine, incorrectly presume that weakly informative proper priors are fine. I have a few reactions to this: 1. I agree
Introduction: Deborah Mayo sent me this quote from Jim Berger: Too often I see people pretending to be subjectivists, and then using “weakly informative” priors that the objective Bayesian community knows are terrible and will give ridiculous answers; subjectivism is then being used as a shield to hide ignorance. . . . In my own more provocative moments, I claim that the only true subjectivists are the objective Bayesians, because they refuse to use subjectivism as a shield against criticism of sloppy pseudo-Bayesian practice. This caught my attention because I’ve become more and more convinced that weakly informative priors are the right way to go in many different situations. I don’t think Berger was talking about me , though, as the above quote came from a publication in 2006, at which time I’d only started writing about weakly informative priors. Going back to Berger’s article , I see that his “weakly informative priors” remark was aimed at this article by Anthony O’Hagan, who w
4 0.87728167 1858 andrew gelman stats-2013-05-15-Reputations changeable, situations tolerable
Introduction: David Kessler, Peter Hoff, and David Dunson write : Marginally specified priors for nonparametric Bayesian estimation Prior specification for nonparametric Bayesian inference involves the difficult task of quantifying prior knowledge about a parameter of high, often infinite, dimension. Realistically, a statistician is unlikely to have informed opinions about all aspects of such a parameter, but may have real information about functionals of the parameter, such the population mean or variance. This article proposes a new framework for nonparametric Bayes inference in which the prior distribution for a possibly infinite-dimensional parameter is decomposed into two parts: an informative prior on a finite set of functionals, and a nonparametric conditional prior for the parameter given the functionals. Such priors can be easily constructed from standard nonparametric prior distributions in common use, and inherit the large support of the standard priors upon which they are based. Ad
5 0.87484586 801 andrew gelman stats-2011-07-13-On the half-Cauchy prior for a global scale parameter
Introduction: Nick Polson and James Scott write : We generalize the half-Cauchy prior for a global scale parameter to the wider class of hypergeometric inverted-beta priors. We derive expressions for posterior moments and marginal densities when these priors are used for a top-level normal variance in a Bayesian hierarchical model. Finally, we prove a result that characterizes the frequentist risk of the Bayes estimators under all priors in the class. These arguments provide an alternative, classical justification for the use of the half-Cauchy prior in Bayesian hierarchical models, complementing the arguments in Gelman (2006). This makes me happy, of course. It’s great to be validated. The only think I didn’t catch is how they set the scale parameter for the half-Cauchy prior. In my 2006 paper I frame it as a weakly informative prior and recommend that the scale be set based on actual prior knowledge. But Polson and Scott are talking about a default choice. I used to think that such a
6 0.83537215 846 andrew gelman stats-2011-08-09-Default priors update?
7 0.8290121 1155 andrew gelman stats-2012-02-05-What is a prior distribution?
9 0.81420648 1946 andrew gelman stats-2013-07-19-Prior distributions on derived quantities rather than on parameters themselves
10 0.81116247 1454 andrew gelman stats-2012-08-11-Weakly informative priors for Bayesian nonparametric models?
11 0.77420169 1474 andrew gelman stats-2012-08-29-More on scaled-inverse Wishart and prior independence
12 0.76655024 2017 andrew gelman stats-2013-09-11-“Informative g-Priors for Logistic Regression”
13 0.75665492 2109 andrew gelman stats-2013-11-21-Hidden dangers of noninformative priors
14 0.74986982 2138 andrew gelman stats-2013-12-18-In Memoriam Dennis Lindley
15 0.72318202 1486 andrew gelman stats-2012-09-07-Prior distributions for regression coefficients
16 0.71833158 1465 andrew gelman stats-2012-08-21-D. Buggin
17 0.71693093 779 andrew gelman stats-2011-06-25-Avoiding boundary estimates using a prior distribution as regularization
18 0.70924622 63 andrew gelman stats-2010-06-02-The problem of overestimation of group-level variance parameters
19 0.69697559 669 andrew gelman stats-2011-04-19-The mysterious Gamma (1.4, 0.4)
20 0.68836939 1877 andrew gelman stats-2013-05-30-Infill asymptotics and sprawl asymptotics
topicId topicWeight
[(15, 0.028), (16, 0.061), (19, 0.274), (23, 0.011), (24, 0.266), (25, 0.013), (53, 0.016), (79, 0.057), (82, 0.013), (84, 0.022), (86, 0.036), (99, 0.088)]
simIndex simValue blogId blogTitle
same-blog 1 0.87333769 468 andrew gelman stats-2010-12-15-Weakly informative priors and imprecise probabilities
Introduction: Giorgio Corani writes: Your work on weakly informative priors is close to some research I [Corani] did (together with Prof. Zaffalon) in the last years using the so-called imprecise probabilities. The idea is to work with a set of priors (containing even very different priors); to update them via Bayes’ rule and then compute a set of posteriors. The set of priors is convex and the priors are Dirichlet (thus, conjugate to the likelihood); this allows to compute the set of posteriors exactly and efficiently. I [Corani] have used this approach for classification, extending naive Bayes and TAN to imprecise probabilities. Classifiers based on imprecise probabilities return more classes when they find that the most probable class is prior-dependent, i.e., if picking different priors in the convex set leads to identify different classes as the most probable one. Instead of returning a single (unreliable) prior-dependent class, credal classifiers in this case preserve reliability by
2 0.70607185 2024 andrew gelman stats-2013-09-15-Swiss Jonah Lehrer update
Introduction: Nassim Taleb adds this link to the Dobelli story . I’m confused. I thought Swiss dudes were supposed to plagiarize their own stuff, not rip off other people’s. Whassup with that?
Introduction: Jouni Kerman did a cool bit of research justifying the Beta (1/3, 1/3) prior as noninformative for binomial data, and the Gamma (1/3, 0) prior for Poisson data. You probably thought that nothing new could be said about noninformative priors in such basic problems, but you were wrong! Here’s the story : The conjugate binomial and Poisson models are commonly used for estimating proportions or rates. However, it is not well known that the conventional noninformative conjugate priors tend to shrink the posterior quantiles toward the boundary or toward the middle of the parameter space, making them thus appear excessively informative. The shrinkage is always largest when the number of observed events is small. This behavior persists for all sample sizes and exposures. The effect of the prior is therefore most conspicuous and potentially controversial when analyzing rare events. As alternative default conjugate priors, I [Jouni] introduce Beta(1/3, 1/3) and Gamma(1/3, 0), which I cal
4 0.69224501 643 andrew gelman stats-2011-04-02-So-called Bayesian hypothesis testing is just as bad as regular hypothesis testing
Introduction: Steve Ziliak points me to this article by the always-excellent Carl Bialik, slamming hypothesis tests. I only wish Carl had talked with me before so hastily posting, though! I would’ve argued with some of the things in the article. In particular, he writes: Reese and Brad Carlin . . . suggest that Bayesian statistics are a better alternative, because they tackle the probability that the hypothesis is true head-on, and incorporate prior knowledge about the variables involved. Brad Carlin does great work in theory, methods, and applications, and I like the bit about the prior knowledge (although I might prefer the more general phrase “additional information”), but I hate that quote! My quick response is that the hypothesis of zero effect is almost never true! The problem with the significance testing framework–Bayesian or otherwise–is in the obsession with the possibility of an exact zero effect. The real concern is not with zero, it’s with claiming a positive effect whe
5 0.69171917 545 andrew gelman stats-2011-01-30-New innovations in spam
Introduction: I received the following (unsolicited) email today: Hello Andrew, I’m interested in whether you are accepting guest article submissions for your site Statistical Modeling, Causal Inference, and Social Science? I’m the owner of the recently created nonprofit site OnlineEngineeringDegree.org and am interested in writing / submitting an article for your consideration to be published on your site. Is that something you’d be willing to consider, and if so, what specs in terms of topics or length requirements would you be looking for? Thanks you for your time, and if you have any questions or are interested, I’d appreciate you letting me know. Sincerely, Samantha Rhodes Huh? P.S. My vote for most obnoxious spam remains this one , which does its best to dilute whatever remains of the reputation of Wolfram Research. Or maybe that particular bit of spam was written by a particularly awesome cellular automaton that Wolfram discovered? I guess in the world of big-time software
6 0.69105744 240 andrew gelman stats-2010-08-29-ARM solutions
7 0.69038159 1437 andrew gelman stats-2012-07-31-Paying survey respondents
8 0.68901759 471 andrew gelman stats-2010-12-17-Attractive models (and data) wanted for statistical art show.
9 0.68774718 1978 andrew gelman stats-2013-08-12-Fixing the race, ethnicity, and national origin questions on the U.S. Census
10 0.68205649 38 andrew gelman stats-2010-05-18-Breastfeeding, infant hyperbilirubinemia, statistical graphics, and modern medicine
11 0.68159324 1787 andrew gelman stats-2013-04-04-Wanna be the next Tyler Cowen? It’s not as easy as you might think!
12 0.68116802 482 andrew gelman stats-2010-12-23-Capitalism as a form of voluntarism
13 0.68080211 938 andrew gelman stats-2011-10-03-Comparing prediction errors
14 0.68013513 1092 andrew gelman stats-2011-12-29-More by Berger and me on weakly informative priors
15 0.67923677 241 andrew gelman stats-2010-08-29-Ethics and statistics in development research
16 0.67920953 1479 andrew gelman stats-2012-09-01-Mothers and Moms
17 0.67642385 2229 andrew gelman stats-2014-02-28-God-leaf-tree
19 0.66999447 59 andrew gelman stats-2010-05-30-Extended Binary Format Support for Mac OS X
20 0.66787261 743 andrew gelman stats-2011-06-03-An argument that can’t possibly make sense