andrew_gelman_stats andrew_gelman_stats-2013 andrew_gelman_stats-2013-1858 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: David Kessler, Peter Hoff, and David Dunson write : Marginally specified priors for nonparametric Bayesian estimation Prior specification for nonparametric Bayesian inference involves the difficult task of quantifying prior knowledge about a parameter of high, often infinite, dimension. Realistically, a statistician is unlikely to have informed opinions about all aspects of such a parameter, but may have real information about functionals of the parameter, such the population mean or variance. This article proposes a new framework for nonparametric Bayes inference in which the prior distribution for a possibly infinite-dimensional parameter is decomposed into two parts: an informative prior on a finite set of functionals, and a nonparametric conditional prior for the parameter given the functionals. Such priors can be easily constructed from standard nonparametric prior distributions in common use, and inherit the large support of the standard priors upon which they are based. Ad
sentIndex sentText sentNum sentScore
1 David Kessler, Peter Hoff, and David Dunson write : Marginally specified priors for nonparametric Bayesian estimation Prior specification for nonparametric Bayesian inference involves the difficult task of quantifying prior knowledge about a parameter of high, often infinite, dimension. [sent-1, score-2.388]
2 Realistically, a statistician is unlikely to have informed opinions about all aspects of such a parameter, but may have real information about functionals of the parameter, such the population mean or variance. [sent-2, score-0.541]
3 Such priors can be easily constructed from standard nonparametric prior distributions in common use, and inherit the large support of the standard priors upon which they are based. [sent-4, score-1.716]
4 Additionally, posterior approximations under these informative priors can generally be made via minor adjustments to existing Markov chain approximation algorithms for standard nonparametric prior distributions. [sent-5, score-1.805]
5 We illustrate the use of such priors in the context of multivariate density estimation using Dirichlet process mixture models, and in the modeling of high-dimensional sparse contingency tables. [sent-6, score-0.886]
wordName wordTfidf (topN-words)
[('nonparametric', 0.47), ('prior', 0.292), ('priors', 0.287), ('functionals', 0.27), ('parameter', 0.266), ('hoff', 0.232), ('dunson', 0.202), ('decomposed', 0.123), ('marginally', 0.123), ('standard', 0.116), ('estimation', 0.115), ('informative', 0.112), ('contingency', 0.111), ('realistically', 0.104), ('dirichlet', 0.101), ('quantifying', 0.101), ('additionally', 0.097), ('proposes', 0.093), ('infinite', 0.092), ('david', 0.09), ('specification', 0.088), ('adjustments', 0.088), ('approximations', 0.087), ('constructed', 0.085), ('markov', 0.085), ('sparse', 0.085), ('specified', 0.079), ('approximation', 0.078), ('inference', 0.078), ('finite', 0.075), ('unlikely', 0.075), ('density', 0.074), ('algorithms', 0.073), ('task', 0.073), ('multivariate', 0.073), ('chain', 0.072), ('mixture', 0.071), ('informed', 0.07), ('illustrate', 0.07), ('involves', 0.069), ('peter', 0.069), ('minor', 0.068), ('opinions', 0.066), ('bayesian', 0.064), ('upon', 0.063), ('framework', 0.063), ('parts', 0.062), ('existing', 0.062), ('aspects', 0.06), ('possibly', 0.059)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 1858 andrew gelman stats-2013-05-15-Reputations changeable, situations tolerable
Introduction: David Kessler, Peter Hoff, and David Dunson write : Marginally specified priors for nonparametric Bayesian estimation Prior specification for nonparametric Bayesian inference involves the difficult task of quantifying prior knowledge about a parameter of high, often infinite, dimension. Realistically, a statistician is unlikely to have informed opinions about all aspects of such a parameter, but may have real information about functionals of the parameter, such the population mean or variance. This article proposes a new framework for nonparametric Bayes inference in which the prior distribution for a possibly infinite-dimensional parameter is decomposed into two parts: an informative prior on a finite set of functionals, and a nonparametric conditional prior for the parameter given the functionals. Such priors can be easily constructed from standard nonparametric prior distributions in common use, and inherit the large support of the standard priors upon which they are based. Ad
2 0.30394846 1092 andrew gelman stats-2011-12-29-More by Berger and me on weakly informative priors
Introduction: A couple days ago we discussed some remarks by Tony O’Hagan and Jim Berger on weakly informative priors. Jim followed up on Deborah Mayo’s blog with this: Objective Bayesian priors are often improper (i.e., have infinite total mass), but this is not a problem when they are developed correctly. But not every improper prior is satisfactory. For instance, the constant prior is known to be unsatisfactory in many situations. The ‘solution’ pseudo-Bayesians often use is to choose a constant prior over a large but bounded set (a ‘weakly informative’ prior), saying it is now proper and so all is well. This is not true; if the constant prior on the whole parameter space is bad, so will be the constant prior over the bounded set. The problem is, in part, that some people confuse proper priors with subjective priors and, having learned that true subjective priors are fine, incorrectly presume that weakly informative proper priors are fine. I have a few reactions to this: 1. I agree
3 0.27386504 1941 andrew gelman stats-2013-07-16-Priors
Introduction: Nick Firoozye writes: While I am absolutely sympathetic to the Bayesian agenda I am often troubled by the requirement of having priors. We must have priors on the parameter of an infinite number of model we have never seen before and I find this troubling. There is a similarly troubling problem in economics of utility theory. Utility is on consumables. To be complete a consumer must assign utility to all sorts of things they never would have encountered. More recent versions of utility theory instead make consumption goods a portfolio of attributes. Cadillacs are x many units of luxury y of transport etc etc. And we can automatically have personal utilities to all these attributes. I don’t ever see parameters. Some model have few and some have hundreds. Instead, I see data. So I don’t know how to have an opinion on parameters themselves. Rather I think it far more natural to have opinions on the behavior of models. The prior predictive density is a good and sensible notion. Also
4 0.26485059 1454 andrew gelman stats-2012-08-11-Weakly informative priors for Bayesian nonparametric models?
Introduction: Nathaniel Egwu writes: I am a PhD student working on machine learning using artificial neural networks . . . Do you have some recent publications related to how one can construct priors depending on the type of input data available for training? I intend to construct a prior distribution for a given trade-off parameter of my non model obtained through training a neural network. At this stage, my argument is due to the fact that Bayesian nonparameteric estimation offers some insight on how to proceed on this problem. As I’ve been writing here for awhile, I’ve been interested in weakly informative priors. But I have little experience with nonparametric models. Perhaps Aki Vehtari or David Dunson or some other expert on these models can discuss how to set them up with weakly informative priors? This sounds like it could be important to me.
5 0.24320394 1155 andrew gelman stats-2012-02-05-What is a prior distribution?
Introduction: Some recent blog discussion revealed some confusion that I’ll try to resolve here. I wrote that I’m not a big fan of subjective priors. Various commenters had difficulty with this point, and I think the issue was most clearly stated by Bill Jeff re erys, who wrote : It seems to me that your prior has to reflect your subjective information before you look at the data. How can it not? But this does not mean that the (subjective) prior that you choose is irrefutable; Surely a prior that reflects prior information just does not have to be inconsistent with that information. But that still leaves a range of priors that are consistent with it, the sort of priors that one would use in a sensitivity analysis, for example. I think I see what Bill is getting at. A prior represents your subjective belief, or some approximation to your subjective belief, even if it’s not perfect. That sounds reasonable but I don’t think it works. Or, at least, it often doesn’t work. Let’s start
6 0.24275199 1946 andrew gelman stats-2013-07-19-Prior distributions on derived quantities rather than on parameters themselves
7 0.23603745 2109 andrew gelman stats-2013-11-21-Hidden dangers of noninformative priors
8 0.2166445 1474 andrew gelman stats-2012-08-29-More on scaled-inverse Wishart and prior independence
10 0.19732815 801 andrew gelman stats-2011-07-13-On the half-Cauchy prior for a global scale parameter
11 0.19514894 1046 andrew gelman stats-2011-12-07-Neutral noninformative and informative conjugate beta and gamma prior distributions
12 0.18957281 1757 andrew gelman stats-2013-03-11-My problem with the Lindley paradox
13 0.18724757 2200 andrew gelman stats-2014-02-05-Prior distribution for a predicted probability
14 0.18377107 2129 andrew gelman stats-2013-12-10-Cross-validation and Bayesian estimation of tuning parameters
15 0.18366732 1792 andrew gelman stats-2013-04-07-X on JLP
16 0.18341286 1695 andrew gelman stats-2013-01-28-Economists argue about Bayes
17 0.18325794 903 andrew gelman stats-2011-09-13-Duke postdoctoral fellowships in nonparametric Bayes & high-dimensional data
18 0.18244223 1486 andrew gelman stats-2012-09-07-Prior distributions for regression coefficients
19 0.18160331 846 andrew gelman stats-2011-08-09-Default priors update?
20 0.17906103 1877 andrew gelman stats-2013-05-30-Infill asymptotics and sprawl asymptotics
topicId topicWeight
[(0, 0.146), (1, 0.254), (2, 0.005), (3, 0.074), (4, -0.085), (5, -0.059), (6, 0.158), (7, 0.034), (8, -0.241), (9, 0.083), (10, 0.01), (11, -0.019), (12, 0.081), (13, 0.055), (14, -0.007), (15, -0.023), (16, -0.016), (17, 0.005), (18, 0.02), (19, 0.001), (20, -0.046), (21, -0.061), (22, 0.003), (23, 0.035), (24, 0.02), (25, 0.028), (26, 0.022), (27, -0.004), (28, -0.007), (29, 0.042), (30, -0.007), (31, -0.05), (32, -0.013), (33, -0.023), (34, -0.025), (35, 0.002), (36, 0.009), (37, -0.009), (38, 0.03), (39, 0.032), (40, -0.034), (41, 0.009), (42, 0.003), (43, 0.038), (44, -0.002), (45, 0.009), (46, -0.007), (47, -0.026), (48, -0.028), (49, -0.008)]
simIndex simValue blogId blogTitle
same-blog 1 0.99404407 1858 andrew gelman stats-2013-05-15-Reputations changeable, situations tolerable
Introduction: David Kessler, Peter Hoff, and David Dunson write : Marginally specified priors for nonparametric Bayesian estimation Prior specification for nonparametric Bayesian inference involves the difficult task of quantifying prior knowledge about a parameter of high, often infinite, dimension. Realistically, a statistician is unlikely to have informed opinions about all aspects of such a parameter, but may have real information about functionals of the parameter, such the population mean or variance. This article proposes a new framework for nonparametric Bayes inference in which the prior distribution for a possibly infinite-dimensional parameter is decomposed into two parts: an informative prior on a finite set of functionals, and a nonparametric conditional prior for the parameter given the functionals. Such priors can be easily constructed from standard nonparametric prior distributions in common use, and inherit the large support of the standard priors upon which they are based. Ad
Introduction: Jouni Kerman did a cool bit of research justifying the Beta (1/3, 1/3) prior as noninformative for binomial data, and the Gamma (1/3, 0) prior for Poisson data. You probably thought that nothing new could be said about noninformative priors in such basic problems, but you were wrong! Here’s the story : The conjugate binomial and Poisson models are commonly used for estimating proportions or rates. However, it is not well known that the conventional noninformative conjugate priors tend to shrink the posterior quantiles toward the boundary or toward the middle of the parameter space, making them thus appear excessively informative. The shrinkage is always largest when the number of observed events is small. This behavior persists for all sample sizes and exposures. The effect of the prior is therefore most conspicuous and potentially controversial when analyzing rare events. As alternative default conjugate priors, I [Jouni] introduce Beta(1/3, 1/3) and Gamma(1/3, 0), which I cal
3 0.91274166 1092 andrew gelman stats-2011-12-29-More by Berger and me on weakly informative priors
Introduction: A couple days ago we discussed some remarks by Tony O’Hagan and Jim Berger on weakly informative priors. Jim followed up on Deborah Mayo’s blog with this: Objective Bayesian priors are often improper (i.e., have infinite total mass), but this is not a problem when they are developed correctly. But not every improper prior is satisfactory. For instance, the constant prior is known to be unsatisfactory in many situations. The ‘solution’ pseudo-Bayesians often use is to choose a constant prior over a large but bounded set (a ‘weakly informative’ prior), saying it is now proper and so all is well. This is not true; if the constant prior on the whole parameter space is bad, so will be the constant prior over the bounded set. The problem is, in part, that some people confuse proper priors with subjective priors and, having learned that true subjective priors are fine, incorrectly presume that weakly informative proper priors are fine. I have a few reactions to this: 1. I agree
4 0.89971948 801 andrew gelman stats-2011-07-13-On the half-Cauchy prior for a global scale parameter
Introduction: Nick Polson and James Scott write : We generalize the half-Cauchy prior for a global scale parameter to the wider class of hypergeometric inverted-beta priors. We derive expressions for posterior moments and marginal densities when these priors are used for a top-level normal variance in a Bayesian hierarchical model. Finally, we prove a result that characterizes the frequentist risk of the Bayes estimators under all priors in the class. These arguments provide an alternative, classical justification for the use of the half-Cauchy prior in Bayesian hierarchical models, complementing the arguments in Gelman (2006). This makes me happy, of course. It’s great to be validated. The only think I didn’t catch is how they set the scale parameter for the half-Cauchy prior. In my 2006 paper I frame it as a weakly informative prior and recommend that the scale be set based on actual prior knowledge. But Polson and Scott are talking about a default choice. I used to think that such a
5 0.88658774 846 andrew gelman stats-2011-08-09-Default priors update?
Introduction: Ryan King writes: I was wondering if you have a brief comment on the state of the art for objective priors for hierarchical generalized linear models (generalized linear mixed models). I have been working off the papers in Bayesian Analysis (2006) 1, Number 3 (Browne and Draper, Kass and Natarajan, Gelman). There seems to have been continuous work for matching priors in linear mixed models, but GLMMs less so because of the lack of an analytic marginal likelihood for the variance components. There are a number of additional suggestions in the literature since 2006, but little robust practical guidance. I’m interested in both mean parameters and the variance components. I’m almost always concerned with logistic random effect models. I’m fascinated by the matching-priors idea of higher-order asymptotic improvements to maximum likelihood, and need to make some kind of defensible default recommendation. Given the massive scale of the datasets (genetics …), extensive sensitivity a
6 0.88468128 1474 andrew gelman stats-2012-08-29-More on scaled-inverse Wishart and prior independence
7 0.86771262 1155 andrew gelman stats-2012-02-05-What is a prior distribution?
8 0.86161488 779 andrew gelman stats-2011-06-25-Avoiding boundary estimates using a prior distribution as regularization
10 0.8376143 1454 andrew gelman stats-2012-08-11-Weakly informative priors for Bayesian nonparametric models?
11 0.83714288 468 andrew gelman stats-2010-12-15-Weakly informative priors and imprecise probabilities
12 0.82827669 1946 andrew gelman stats-2013-07-19-Prior distributions on derived quantities rather than on parameters themselves
13 0.81664878 442 andrew gelman stats-2010-12-01-bayesglm in Stata?
14 0.81539202 2109 andrew gelman stats-2013-11-21-Hidden dangers of noninformative priors
15 0.8095879 669 andrew gelman stats-2011-04-19-The mysterious Gamma (1.4, 0.4)
16 0.80757082 2017 andrew gelman stats-2013-09-11-“Informative g-Priors for Logistic Regression”
17 0.80256742 1941 andrew gelman stats-2013-07-16-Priors
18 0.80031794 1130 andrew gelman stats-2012-01-20-Prior beliefs about locations of decision boundaries
19 0.79724127 63 andrew gelman stats-2010-06-02-The problem of overestimation of group-level variance parameters
20 0.79520714 1465 andrew gelman stats-2012-08-21-D. Buggin
topicId topicWeight
[(12, 0.119), (16, 0.054), (20, 0.012), (21, 0.018), (24, 0.202), (42, 0.01), (53, 0.069), (62, 0.012), (71, 0.026), (75, 0.02), (77, 0.022), (84, 0.032), (86, 0.06), (89, 0.01), (99, 0.22)]
simIndex simValue blogId blogTitle
same-blog 1 0.95993936 1858 andrew gelman stats-2013-05-15-Reputations changeable, situations tolerable
Introduction: David Kessler, Peter Hoff, and David Dunson write : Marginally specified priors for nonparametric Bayesian estimation Prior specification for nonparametric Bayesian inference involves the difficult task of quantifying prior knowledge about a parameter of high, often infinite, dimension. Realistically, a statistician is unlikely to have informed opinions about all aspects of such a parameter, but may have real information about functionals of the parameter, such the population mean or variance. This article proposes a new framework for nonparametric Bayes inference in which the prior distribution for a possibly infinite-dimensional parameter is decomposed into two parts: an informative prior on a finite set of functionals, and a nonparametric conditional prior for the parameter given the functionals. Such priors can be easily constructed from standard nonparametric prior distributions in common use, and inherit the large support of the standard priors upon which they are based. Ad
2 0.91430938 547 andrew gelman stats-2011-01-31-Using sample size in the prior distribution
Introduction: Mike McLaughlin writes: Consider the Seeds example in vol. 1 of the BUGS examples. There, a binomial likelihood has a p parameter constructed, via logit, from two covariates. What I am wondering is: Would it be legitimate, in a binomial + logit problem like this, to allow binomial p[i] to be a function of the corresponding n[i] or would that amount to using the data in the prior? In other words, in the context of the Seeds example, is r[] the only data or is n[] data as well and therefore not permissible in a prior formulation? I [McLaughlin] currently have a model with a common beta prior for all p[i] but would like to mitigate this commonality (a kind of James-Stein effect) when there are lots of observations for some i. But this seems to feed the data back into the prior. Does it really? It also occurs to me [McLaughlin] that, perhaps, a binomial likelihood is not the one to use here (not flexible enough). My reply: Strictly speaking, “n” is data, and so what you wa
3 0.91163576 211 andrew gelman stats-2010-08-17-Deducer update
Introduction: A year ago we blogged about Ian Fellows’s R Gui called Deducer (oops, my bad, I meant to link to this ). Fellows sends in this update: Since version 0.1, I [Fellows] have added: 1. A nice plug-in interface, so that people can extend Deducer’s capability without leaving the comfort of R. (see: http://www.deducer.org/pmwiki/pmwiki.php?n=Main.Development ) 2. Several new dialogs. 3. A one-step installer for windows. 4. A plug-in package (DeducerExtras) which extends the scope of analyses covered. 5. A plotting GUI that can create anything from simple histograms to complex custom graphics. Deducer is designed to be a free easy to use alternative to proprietary data analysis software such as SPSS, JMP, and Minitab. It has a menu system to do common data manipulation and analysis tasks, and an excel-like spreadsheet in which to view and edit data frames. The goal of the project is two fold. Provide an intuitive interface so that non-technical users can learn and p
4 0.90961748 1871 andrew gelman stats-2013-05-27-Annals of spam
Introduction: I received the following email, subject line “Want to Buy Text Link from andrewgelman.com”: Dear, I am Mary Taylor. I have started a link building campaign for my growing websites. For this, I need your cooperation. The campaign is quite diverse and large scale and if you take some time to understand it – it will benefit us. First I want to clarify that I do not want “blogroll” ”footer” or any other type of “site wide links”. Secondly I want links from inner pages of site – with good page rank of course. Third links should be within text so that Google may not mark them as spam – not for you and not for me. Hence this link building will cause almost no harm to your site or me. Because content links are fine with Google. Now I should come to the requirements. I will accept links from Page Rank 3 to as high as you have got. Also kindly note that I can buy 1 to 50 links from one site – so you should understand the scale of the project. If you have multiple sites with co
5 0.90332997 2149 andrew gelman stats-2013-12-26-Statistical evidence for revised standards
Introduction: In response to the discussion of X and me of his recent paper , Val Johnson writes: I would like to thank Andrew for forwarding his comments on uniformly most powerful Bayesian tests (UMPBTs) to me and his invitation to respond to them. I think he (and also Christian Robert) raise a number of interesting points concerning this new class of Bayesian tests, but I think that they may have confounded several issues that might more usefully be examined separately. The first issue involves the choice of the Bayesian evidence threshold, gamma, used in rejecting a null hypothesis in favor of an alternative hypothesis. Andrew objects to the higher values of gamma proposed in my recent PNAS article on grounds that too many important scientific effects would be missed if thresholds of 25-50 were routinely used. These evidence thresholds correspond roughly to p-values of 0.005; Andrew suggests that evidence thresholds around 5 should continue to be used (gamma=5 corresponds approximate
6 0.89661872 846 andrew gelman stats-2011-08-09-Default priors update?
7 0.89624405 752 andrew gelman stats-2011-06-08-Traffic Prediction
9 0.89525443 899 andrew gelman stats-2011-09-10-The statistical significance filter
10 0.89498818 687 andrew gelman stats-2011-04-29-Zero is zero
11 0.89470142 1438 andrew gelman stats-2012-07-31-What is a Bayesian?
12 0.89458251 1367 andrew gelman stats-2012-06-05-Question 26 of my final exam for Design and Analysis of Sample Surveys
13 0.89454693 953 andrew gelman stats-2011-10-11-Steve Jobs’s cancer and science-based medicine
14 0.89371896 779 andrew gelman stats-2011-06-25-Avoiding boundary estimates using a prior distribution as regularization
15 0.89247334 1240 andrew gelman stats-2012-04-02-Blogads update
16 0.89216769 1474 andrew gelman stats-2012-08-29-More on scaled-inverse Wishart and prior independence
17 0.89209437 2312 andrew gelman stats-2014-04-29-Ken Rice presents a unifying approach to statistical inference and hypothesis testing
18 0.89120281 1062 andrew gelman stats-2011-12-16-Mr. Pearson, meet Mr. Mandelbrot: Detecting Novel Associations in Large Data Sets
19 0.89071697 1155 andrew gelman stats-2012-02-05-What is a prior distribution?
20 0.88971233 494 andrew gelman stats-2010-12-31-Type S error rates for classical and Bayesian single and multiple comparison procedures