andrew_gelman_stats andrew_gelman_stats-2012 andrew_gelman_stats-2012-1518 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Following a recent email exchange regarding path sampling and thermodynamic integration (sadly, I’ve gotten rusty and haven’t thought seriously about these challenges for many years), a correspondent referred to the marginal distribution of the data under a model as “the evidence.” I hate that expression! As we discuss in chapter 6 of BDA, for continuous-parametered models, this quantity can be completely sensitive to aspects of the prior that have essentially no impact on the posterior. In the examples I’ve seen, this marginal probability is not “evidence” in any useful sense of the term. When I told this to my correspondent, he replied, I actually don’t find “the evidence” too bothersome. I don’t have BDA at home where I’m working from at the moment, so I’ll read up on chapter 6 later, but I assume you refer to the problem of the marginal likelihood being strongly sensitive to the prior in a way that the posterior typically isn’t, thereby diminishing the value of the margi
sentIndex sentText sentNum sentScore
1 Following a recent email exchange regarding path sampling and thermodynamic integration (sadly, I’ve gotten rusty and haven’t thought seriously about these challenges for many years), a correspondent referred to the marginal distribution of the data under a model as “the evidence. [sent-1, score-1.613]
2 As we discuss in chapter 6 of BDA, for continuous-parametered models, this quantity can be completely sensitive to aspects of the prior that have essentially no impact on the posterior. [sent-3, score-0.822]
3 In the examples I’ve seen, this marginal probability is not “evidence” in any useful sense of the term. [sent-4, score-0.376]
4 If so, I understand, but I think you might be fighting a losing battle as “the evidence” is seemingly now popular in the stats literature as well as the physics … I replied that I’ll fight that battle forever. [sent-7, score-1.432]
5 I really really hate the use of linguistically-loaded terms such as “bias,” “evidence,” “empirical Bayes,” etc. [sent-8, score-0.178]
wordName wordTfidf (topN-words)
[('marginal', 0.376), ('battle', 0.243), ('correspondent', 0.225), ('evidence', 0.219), ('bda', 0.214), ('sensitive', 0.212), ('hate', 0.178), ('thermodynamic', 0.162), ('replied', 0.16), ('likelihood', 0.151), ('chapter', 0.145), ('diminishing', 0.141), ('sadly', 0.138), ('thereby', 0.126), ('fighting', 0.126), ('statistic', 0.121), ('losing', 0.117), ('prior', 0.117), ('integration', 0.112), ('fight', 0.111), ('gotten', 0.109), ('quantity', 0.107), ('expression', 0.107), ('referred', 0.106), ('seemingly', 0.104), ('path', 0.101), ('exchange', 0.1), ('stats', 0.099), ('refer', 0.098), ('challenges', 0.096), ('moment', 0.092), ('motivated', 0.086), ('physics', 0.085), ('aspects', 0.084), ('home', 0.083), ('impact', 0.082), ('seriously', 0.081), ('empirical', 0.079), ('ll', 0.078), ('selection', 0.078), ('bias', 0.077), ('strongly', 0.077), ('choose', 0.076), ('bayes', 0.076), ('essentially', 0.075), ('sampling', 0.074), ('etc', 0.073), ('popular', 0.072), ('well', 0.072), ('email', 0.071)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 1518 andrew gelman stats-2012-10-02-Fighting a losing battle
Introduction: Following a recent email exchange regarding path sampling and thermodynamic integration (sadly, I’ve gotten rusty and haven’t thought seriously about these challenges for many years), a correspondent referred to the marginal distribution of the data under a model as “the evidence.” I hate that expression! As we discuss in chapter 6 of BDA, for continuous-parametered models, this quantity can be completely sensitive to aspects of the prior that have essentially no impact on the posterior. In the examples I’ve seen, this marginal probability is not “evidence” in any useful sense of the term. When I told this to my correspondent, he replied, I actually don’t find “the evidence” too bothersome. I don’t have BDA at home where I’m working from at the moment, so I’ll read up on chapter 6 later, but I assume you refer to the problem of the marginal likelihood being strongly sensitive to the prior in a way that the posterior typically isn’t, thereby diminishing the value of the margi
2 0.2501134 1983 andrew gelman stats-2013-08-15-More on AIC, WAIC, etc
Introduction: Following up on our discussion from the other day, Angelika van der Linde sends along this paper from 2012 (link to journal here ). And Aki pulls out this great quote from Geisser and Eddy (1979): This discussion makes clear that in the nested case this method, as Akaike’s, is not consistent; i.e., even if $M_k$ is true, it will be rejected with probability $\alpha$ as $N\to\infty$. This point is also made by Schwarz (1978). However, from the point of view of prediction, this is of no great consequence. For large numbers of observations, a prediction based on the falsely assumed $M_k$, will not differ appreciably from one based on the true $M_k$. For example, if we assert that two normal populations have different means when in fact they have the same mean, then the use of the group mean as opposed to the grand mean for predicting a future observation results in predictors which are asymptotically equivalent and whose predictive variances are $\sigma^2[1 + (1/2n)]$ and $\si
3 0.2123778 779 andrew gelman stats-2011-06-25-Avoiding boundary estimates using a prior distribution as regularization
Introduction: For awhile I’ve been fitting most of my multilevel models using lmer/glmer, which gives point estimates of the group-level variance parameters (maximum marginal likelihood estimate for lmer and an approximation for glmer). I’m usually satisfied with this–sure, point estimation understates the uncertainty in model fitting, but that’s typically the least of our worries. Sometimes, though, lmer/glmer estimates group-level variances at 0 or estimates group-level correlation parameters at +/- 1. Typically, when this happens, it’s not that we’re so sure the variance is close to zero or that the correlation is close to 1 or -1; rather, the marginal likelihood does not provide a lot of information about these parameters of the group-level error distribution. I don’t want point estimates on the boundary. I don’t want to say that the unexplained variance in some dimension is exactly zero. One way to handle this problem is full Bayes: slap a prior on sigma, do your Gibbs and Metropolis
4 0.17180885 1474 andrew gelman stats-2012-08-29-More on scaled-inverse Wishart and prior independence
Introduction: I’ve had a couple of email conversations in the past couple days on dependence in multivariate prior distributions. Modeling the degrees of freedom and scale parameters in the t distribution First, in our Stan group we’ve been discussing the choice of priors for the degrees-of-freedom parameter in the t distribution. I wrote that also there’s the question of parameterization. It does not necessarily make sense to have independent priors on the df and scale parameters. In some sense, the meaning of the scale parameter changes with the df. Prior dependence between correlation and scale parameters in the scaled inverse-Wishart model The second case of parameterization in prior distribution arose from an email I received from Chris Chatham pointing me to this exploration by Matt Simpson of the scaled inverse-Wishart prior distribution for hierarchical covariance matrices. Simpson writes: A popular prior for Σ is the inverse-Wishart distribution [ not the same as the
5 0.15761912 754 andrew gelman stats-2011-06-09-Difficulties with Bayesian model averaging
Introduction: In response to this article by Cosma Shalizi and myself on the philosophy of Bayesian statistics, David Hogg writes: I [Hogg] agree–even in physics and astronomy–that the models are not “True” in the God-like sense of being absolute reality (that is, I am not a realist); and I have argued (a philosophically very naive paper, but hey, I was new to all this) that for pretty fundamental reasons we could never arrive at the True (with a capital “T”) model of the Universe. The goal of inference is to find the “best” model, where “best” might have something to do with prediction, or explanation, or message length, or (horror!) our utility. Needless to say, most of my physics friends *are* realists, even in the face of “effective theories” as Newtonian mechanics is an effective theory of GR and GR is an effective theory of “quantum gravity” (this plays to your point, because if you think any theory is possibly an effective theory, how could you ever find Truth?). I also liked the i
7 0.13274772 2200 andrew gelman stats-2014-02-05-Prior distribution for a predicted probability
8 0.13173419 1247 andrew gelman stats-2012-04-05-More philosophy of Bayes
9 0.12490258 2109 andrew gelman stats-2013-11-21-Hidden dangers of noninformative priors
10 0.12281869 341 andrew gelman stats-2010-10-14-Confusion about continuous probability densities
12 0.11544949 612 andrew gelman stats-2011-03-14-Uh-oh
13 0.11511656 2267 andrew gelman stats-2014-03-26-Is a steal really worth 9 points?
15 0.11300824 1941 andrew gelman stats-2013-07-16-Priors
16 0.10955085 2017 andrew gelman stats-2013-09-11-“Informative g-Priors for Logistic Regression”
17 0.10919419 338 andrew gelman stats-2010-10-12-Update on Mankiw’s work incentives
18 0.10895004 398 andrew gelman stats-2010-11-06-Quote of the day
20 0.10521974 991 andrew gelman stats-2011-11-04-Insecure researchers aren’t sharing their data
topicId topicWeight
[(0, 0.173), (1, 0.109), (2, -0.004), (3, 0.037), (4, -0.048), (5, -0.033), (6, 0.094), (7, 0.003), (8, -0.025), (9, -0.012), (10, -0.032), (11, 0.01), (12, 0.0), (13, 0.012), (14, -0.048), (15, -0.038), (16, 0.005), (17, 0.001), (18, -0.005), (19, -0.001), (20, 0.076), (21, -0.049), (22, 0.005), (23, 0.035), (24, -0.023), (25, 0.027), (26, 0.019), (27, 0.01), (28, 0.064), (29, -0.006), (30, -0.053), (31, 0.029), (32, -0.013), (33, 0.039), (34, -0.022), (35, 0.017), (36, 0.048), (37, -0.035), (38, 0.001), (39, 0.0), (40, -0.002), (41, -0.011), (42, 0.024), (43, -0.06), (44, 0.021), (45, -0.01), (46, -0.025), (47, 0.058), (48, 0.027), (49, -0.035)]
simIndex simValue blogId blogTitle
same-blog 1 0.96060389 1518 andrew gelman stats-2012-10-02-Fighting a losing battle
Introduction: Following a recent email exchange regarding path sampling and thermodynamic integration (sadly, I’ve gotten rusty and haven’t thought seriously about these challenges for many years), a correspondent referred to the marginal distribution of the data under a model as “the evidence.” I hate that expression! As we discuss in chapter 6 of BDA, for continuous-parametered models, this quantity can be completely sensitive to aspects of the prior that have essentially no impact on the posterior. In the examples I’ve seen, this marginal probability is not “evidence” in any useful sense of the term. When I told this to my correspondent, he replied, I actually don’t find “the evidence” too bothersome. I don’t have BDA at home where I’m working from at the moment, so I’ll read up on chapter 6 later, but I assume you refer to the problem of the marginal likelihood being strongly sensitive to the prior in a way that the posterior typically isn’t, thereby diminishing the value of the margi
2 0.80034709 1983 andrew gelman stats-2013-08-15-More on AIC, WAIC, etc
Introduction: Following up on our discussion from the other day, Angelika van der Linde sends along this paper from 2012 (link to journal here ). And Aki pulls out this great quote from Geisser and Eddy (1979): This discussion makes clear that in the nested case this method, as Akaike’s, is not consistent; i.e., even if $M_k$ is true, it will be rejected with probability $\alpha$ as $N\to\infty$. This point is also made by Schwarz (1978). However, from the point of view of prediction, this is of no great consequence. For large numbers of observations, a prediction based on the falsely assumed $M_k$, will not differ appreciably from one based on the true $M_k$. For example, if we assert that two normal populations have different means when in fact they have the same mean, then the use of the group mean as opposed to the grand mean for predicting a future observation results in predictors which are asymptotically equivalent and whose predictive variances are $\sigma^2[1 + (1/2n)]$ and $\si
3 0.75130028 2029 andrew gelman stats-2013-09-18-Understanding posterior p-values
Introduction: David Kaplan writes: I came across your paper “Understanding Posterior Predictive P-values”, and I have a question regarding your statement “If a posterior predictive p-value is 0.4, say, that means that, if we believe the model, we think there is a 40% chance that tomorrow’s value of T(y_rep) will exceed today’s T(y).” This is perfectly understandable to me and represents the idea of calibration. However, I am unsure how this relates to statements about fit. If T is the LR chi-square or Pearson chi-square, then your statement that there is a 40% chance that tomorrows value exceeds today’s value indicates bad fit, I think. Yet, some literature indicates that high p-values suggest good fit. Could you clarify this? My reply: I think that “fit” depends on the question being asked. In this case, I’d say the model fits for this particular purpose, even though it might not fit for other purposes. And here’s the abstract of the paper: Posterior predictive p-values do not i
4 0.73677182 1713 andrew gelman stats-2013-02-08-P-values and statistical practice
Introduction: From my new article in the journal Epidemiology: Sander Greenland and Charles Poole accept that P values are here to stay but recognize that some of their most common interpretations have problems. The casual view of the P value as posterior probability of the truth of the null hypothesis is false and not even close to valid under any reasonable model, yet this misunderstanding persists even in high-stakes settings (as discussed, for example, by Greenland in 2011). The formal view of the P value as a probability conditional on the null is mathematically correct but typically irrelevant to research goals (hence, the popularity of alternative—if wrong—interpretations). A Bayesian interpretation based on a spike-and-slab model makes little sense in applied contexts in epidemiology, political science, and other fields in which true effects are typically nonzero and bounded (thus violating both the “spike” and the “slab” parts of the model). I find Greenland and Poole’s perspective t
5 0.71678537 2200 andrew gelman stats-2014-02-05-Prior distribution for a predicted probability
Introduction: I received the following email: I have an interesting thought on a prior for a logistic regression, and would love your input on how to make it “work.” Some of my research, two published papers, are on mathematical models of **. Along those lines, I’m interested in developing more models for **. . . . Empirical studies show that the public is rather smart and that the wisdom-of-the-crowd is fairly accurate. So, my thought would be to tread the public’s probability of the event as a prior, and then see how adding data, through a model, would change or perturb our inferred probability of **. (Similarly, I could envision using previously published epidemiological research as a prior probability of a disease, and then seeing how the addition of new testing protocols would update that belief.) However, everything I learned about hierarchical Bayesian models has a prior as a distribution on the coefficients. I don’t know how to start with a prior point estimate for the probabili
6 0.71193975 1474 andrew gelman stats-2012-08-29-More on scaled-inverse Wishart and prior independence
7 0.7050581 2208 andrew gelman stats-2014-02-12-How to think about “identifiability” in Bayesian inference?
8 0.68465114 996 andrew gelman stats-2011-11-07-Chi-square FAIL when many cells have small expected values
9 0.68112993 1208 andrew gelman stats-2012-03-11-Gelman on Hennig on Gelman on Bayes
10 0.6776005 669 andrew gelman stats-2011-04-19-The mysterious Gamma (1.4, 0.4)
11 0.67540497 1247 andrew gelman stats-2012-04-05-More philosophy of Bayes
12 0.66911316 1946 andrew gelman stats-2013-07-19-Prior distributions on derived quantities rather than on parameters themselves
13 0.66877407 1941 andrew gelman stats-2013-07-16-Priors
14 0.66098809 519 andrew gelman stats-2011-01-16-Update on the generalized method of moments
16 0.65016162 2109 andrew gelman stats-2013-11-21-Hidden dangers of noninformative priors
17 0.6458376 779 andrew gelman stats-2011-06-25-Avoiding boundary estimates using a prior distribution as regularization
18 0.6451813 1723 andrew gelman stats-2013-02-15-Wacky priors can work well?
19 0.64357203 398 andrew gelman stats-2010-11-06-Quote of the day
20 0.64085156 341 andrew gelman stats-2010-10-14-Confusion about continuous probability densities
topicId topicWeight
[(16, 0.078), (21, 0.062), (24, 0.167), (27, 0.064), (48, 0.074), (63, 0.022), (66, 0.012), (86, 0.089), (97, 0.04), (99, 0.293)]
simIndex simValue blogId blogTitle
same-blog 1 0.97486353 1518 andrew gelman stats-2012-10-02-Fighting a losing battle
Introduction: Following a recent email exchange regarding path sampling and thermodynamic integration (sadly, I’ve gotten rusty and haven’t thought seriously about these challenges for many years), a correspondent referred to the marginal distribution of the data under a model as “the evidence.” I hate that expression! As we discuss in chapter 6 of BDA, for continuous-parametered models, this quantity can be completely sensitive to aspects of the prior that have essentially no impact on the posterior. In the examples I’ve seen, this marginal probability is not “evidence” in any useful sense of the term. When I told this to my correspondent, he replied, I actually don’t find “the evidence” too bothersome. I don’t have BDA at home where I’m working from at the moment, so I’ll read up on chapter 6 later, but I assume you refer to the problem of the marginal likelihood being strongly sensitive to the prior in a way that the posterior typically isn’t, thereby diminishing the value of the margi
2 0.95827413 2118 andrew gelman stats-2013-11-30-???
Introduction: I received the following unsolicited email, subject line Technology and Engineering Research: Dear Editor We have done research in some of the cutting edge technology and engineering field and would like to if you will be able to write about it in your news section. Our Primarily research focus on building high performance systems that are helping in social networks, web, finding disease, cancer and sports using BIG DATA . Hope to hear from you some time soon. Thanks, ***, PhD Chartered Scientist IBM Corportation ***@us.ibm.com 916 *** **** I thought IBM was a professional operation—don’t they have their own public relations department?
3 0.94741279 1238 andrew gelman stats-2012-03-31-Dispute about ethics of data sharing
Introduction: Several months ago, Sam Behseta, the new editor of Chance magazine, asked me if I’d like to have a column. I said yes, I’d like to write on ethics and statistics. My first column was called “Open Data and Open Methods” and I discussed the ethical obligation to share data and make our computations transparent wherever possible. In my column, I recounted a story from a bit over 20 years ago when I noticed a problem in a published analysis (involving electromagnetic fields and calcium flow in chicken brains) and contacted the researcher in charge of the study, who would not share his data with me. Two of the people from that research team—biologist Carl Blackman and statistician Dennis House—saw my Chance column and felt that I had misrepresented the situation and had criticized them unfairly. Blackman and House expressed their concerns in letters to the editor which were just published, along with my reply, in the latest issue of Chance . Seeing as I posted my article here, I
4 0.94467318 212 andrew gelman stats-2010-08-17-Futures contracts, Granger causality, and my preference for estimation to testing
Introduction: José Iparraguirre writes: There’s a letter in the latest issue of The Economist (July 31st) signed by Sir Richard Branson (Virgin), Michael Masters (Masters Capital Management) and David Frenk (Better Markets) about an “>OECD report on speculation and the prices of commodities, which includes the following: “The report uses a Granger causality test to measure the relationship between the level of commodities futures contracts held by swap dealers, and the prices of those commodities. Granger tests, however, are of dubious applicability to extremely volatile variables like commodities prices.” The report says: Granger causality is a standard statistical technique for determining whether one time series is useful in forecasting another. It is important to bear in mind that the term causality is used in a statistical sense, and not in a philosophical one of structural causation. More precisely a variable A is said to Granger cause B if knowing the time paths of B and A toge
5 0.94409704 1278 andrew gelman stats-2012-04-23-“Any old map will do” meets “God is in every leaf of every tree”
Introduction: As a statistician I am particularly worried about the rhetorical power of anecdotes (even though I use them in my own reasoning; see discussion below). But much can be learned from a true anecdote. The rough edges—the places where the anecdote doesn’t fit your thesis—these are where you learn. We have recently had a discussion ( here and here ) of Karl Weick, a prominent scholar of business management who plagiarized a story and then went on to draw different lessons from the pilfered anecdote in several different publications published over many years. Setting aside an issues of plagiarism and rulebreaking, I argue that, by hiding the source of the story and changing its form, Weick and his management-science audience are losing their ability to get anything out of it beyond empty confirmation. A full discussion follows. 1. The lost Hungarian soldiers Thomas Basbøll (who has the unusual (to me) job of “writing consultant” at the Copenhagen Business School) has been
6 0.94265175 2281 andrew gelman stats-2014-04-04-The Notorious N.H.S.T. presents: Mo P-values Mo Problems
7 0.94177413 781 andrew gelman stats-2011-06-28-The holes in my philosophy of Bayesian data analysis
8 0.94105846 1266 andrew gelman stats-2012-04-16-Another day, another plagiarist
9 0.94066274 1910 andrew gelman stats-2013-06-22-Struggles over the criticism of the “cannabis users and IQ change” paper
10 0.9401018 777 andrew gelman stats-2011-06-23-Combining survey data obtained using different modes of sampling
12 0.93896359 897 andrew gelman stats-2011-09-09-The difference between significant and not significant…
13 0.93728578 464 andrew gelman stats-2010-12-12-Finite-population standard deviation in a hierarchical model
14 0.93688571 2201 andrew gelman stats-2014-02-06-Bootstrap averaging: Examples where it works and where it doesn’t work
15 0.93682837 2058 andrew gelman stats-2013-10-11-Gladwell and Chabris, David and Goliath, and science writing as stone soup
16 0.93681753 2184 andrew gelman stats-2014-01-24-Parables vs. stories
17 0.93658757 804 andrew gelman stats-2011-07-15-Static sensitivity analysis
18 0.9364233 1971 andrew gelman stats-2013-08-07-I doubt they cheated
19 0.93578178 116 andrew gelman stats-2010-06-29-How to grab power in a democracy – in 5 easy non-violent steps