andrew_gelman_stats andrew_gelman_stats-2012 andrew_gelman_stats-2012-1422 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: David Hogg points me to this discussion: Martin Strasbourg and I [Hogg] discussed his project to detect new satellites of M31 in the PAndAS survey. He can construct a likelihood ratio (possibly even a marginalized likelihood ratio) at every position in the M31 imaging, between the best-fit satellite-plus-background model and the best nothing-plus-background model. He can make a two-dimensional map of these likelihood ratios and show a the histogram of them. Looking at this histogram, which has a tail to very large ratios, he asked me, where should I put my cut? That is, at what likelihood ratio does a candidate deserve follow-up? Here’s my unsatisfying answer: To a statistician, the distribution of likelihood ratios is interesting and valuable to study. To an astronomer, it is uninteresting. You don’t want to know the distribution of likelihoods, you want to find satellites . . . I wrote that I think this makes sense and that it would actualy be an interesting and useful rese
sentIndex sentText sentNum sentScore
1 David Hogg points me to this discussion: Martin Strasbourg and I [Hogg] discussed his project to detect new satellites of M31 in the PAndAS survey. [sent-1, score-0.516]
2 He can construct a likelihood ratio (possibly even a marginalized likelihood ratio) at every position in the M31 imaging, between the best-fit satellite-plus-background model and the best nothing-plus-background model. [sent-2, score-1.238]
3 He can make a two-dimensional map of these likelihood ratios and show a the histogram of them. [sent-3, score-0.914]
4 Looking at this histogram, which has a tail to very large ratios, he asked me, where should I put my cut? [sent-4, score-0.181]
5 That is, at what likelihood ratio does a candidate deserve follow-up? [sent-5, score-0.769]
6 Here’s my unsatisfying answer: To a statistician, the distribution of likelihood ratios is interesting and valuable to study. [sent-6, score-0.999]
7 You don’t want to know the distribution of likelihoods, you want to find satellites . [sent-8, score-0.396]
8 I wrote that I think this makes sense and that it would actualy be an interesting and useful research project to formalize this as a decision problem. [sent-11, score-0.318]
9 I’ve seen this sort of question arise in genetics (where should the p-value threshold be when you’re selecting N out of a million genes) but it’s frustrating because the cost-benefit calculations always seem implicit. [sent-12, score-0.618]
10 In one—just one—of my papers we put an explicit utility model: http://arxiv. [sent-15, score-0.351]
11 2233 The utility model is on page 17 and we use it explicitly on page 18 and on. [sent-17, score-0.564]
12 com ) that has to *decide* whether to return results to the user or not, given probabilistic information about a submitted image. [sent-21, score-0.336]
wordName wordTfidf (topN-words)
[('likelihood', 0.323), ('hogg', 0.315), ('satellites', 0.294), ('ratios', 0.283), ('ratio', 0.264), ('histogram', 0.227), ('utility', 0.177), ('marginalized', 0.147), ('astronomer', 0.139), ('unsatisfying', 0.133), ('http', 0.132), ('imaging', 0.128), ('project', 0.125), ('formalize', 0.118), ('page', 0.11), ('selecting', 0.108), ('likelihoods', 0.108), ('deserve', 0.108), ('tail', 0.107), ('genes', 0.107), ('distribution', 0.102), ('explicit', 0.1), ('genetics', 0.098), ('detect', 0.097), ('construct', 0.094), ('martin', 0.094), ('probabilistic', 0.092), ('frustrating', 0.092), ('threshold', 0.09), ('inside', 0.088), ('model', 0.087), ('submitted', 0.086), ('calculations', 0.084), ('user', 0.084), ('valuable', 0.083), ('cut', 0.081), ('map', 0.081), ('explicitly', 0.08), ('interesting', 0.075), ('arise', 0.075), ('decide', 0.075), ('return', 0.074), ('candidate', 0.074), ('put', 0.074), ('possibly', 0.071), ('million', 0.071), ('web', 0.07), ('cool', 0.07), ('replied', 0.069), ('running', 0.065)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 1422 andrew gelman stats-2012-07-20-Likelihood thresholds and decisions
Introduction: David Hogg points me to this discussion: Martin Strasbourg and I [Hogg] discussed his project to detect new satellites of M31 in the PAndAS survey. He can construct a likelihood ratio (possibly even a marginalized likelihood ratio) at every position in the M31 imaging, between the best-fit satellite-plus-background model and the best nothing-plus-background model. He can make a two-dimensional map of these likelihood ratios and show a the histogram of them. Looking at this histogram, which has a tail to very large ratios, he asked me, where should I put my cut? That is, at what likelihood ratio does a candidate deserve follow-up? Here’s my unsatisfying answer: To a statistician, the distribution of likelihood ratios is interesting and valuable to study. To an astronomer, it is uninteresting. You don’t want to know the distribution of likelihoods, you want to find satellites . . . I wrote that I think this makes sense and that it would actualy be an interesting and useful rese
2 0.18816295 248 andrew gelman stats-2010-09-01-Ratios where the numerator and denominator both change signs
Introduction: A couple years ago, I used a question by Benjamin Kay as an excuse to write that it’s usually a bad idea to study a ratio whose denominator has uncertain sign. As I wrote then: Similar problems arise with marginal cost-benefit ratios, LD50 in logistic regression (see chapter 3 of Bayesian Data Analysis for an example), instrumental variables, and the Fieller-Creasy problem in theoretical statistics. . . . In general, the story is that the ratio completely changes in interpretation when the denominator changes sign. More recently, Kay sent in a related question: I [Kay] wondered if you have any advice on handling ratios when the signs change as a result of a parameter. I have three functions, one C * x^a, another D * x^a, and a third f(x,a) in my paper such that: C * x^a, < f(x,a) < D * x^a C,D and a all have the same signs. We can divide through by C * x^a but the results depend on the sign of C either 1< f(x,a) / C * x^a < D * x^a / C * x^a, or 1 / f(x,a
Introduction: Ratio estimates are common in statistics. In survey sampling, the ratio estimate is when you use y/x to estimate Y/X (using the notation in which x,y are totals of sample measurements and X,Y are population totals). In textbook sampling examples, the denominator X will be an all-positive variable, something that is easy to measure and is, ideally, close to proportional to Y. For example, X is last year’s sales and Y is this year’s sales, or X is the number of people in a cluster and Y is some count. Ratio estimation doesn’t work so well if X can be either positive or negative. More generally we can consider any estimate of a ratio, with no need for a survey sampling context. The problem with estimating Y/X is that the very interpretation of Y/X can change completely if the sign of X changes. Everything is ok for a point estimate: you get X.hat and Y.hat, you can take the ratio Y.hat/X.hat, no problem. But the inference falls apart if you have enough uncertainty in X.hat th
4 0.15732108 398 andrew gelman stats-2010-11-06-Quote of the day
Introduction: “A statistical model is usually taken to be summarized by a likelihood, or a likelihood and a prior distribution, but we go an extra step by noting that the parameters of a model are typically batched, and we take this batching as an essential part of the model.”
5 0.14917059 1510 andrew gelman stats-2012-09-25-Incoherence of Bayesian data analysis
Introduction: Hogg writes: At the end this article you wonder about consistency. Have you ever considered the possibility that utility might resolve some of the problems? I have no idea if it would—I am not advocating that position—I just get some kind of intuition from phrases like “Judgment is required to decide…”. Perhaps there is a coherent and objective description of what is—or could be—done under a coherent “utility” model (like a utility that could be objectively agreed upon and computed). Utilities are usually subjective—true—but priors are usually subjective too. My reply: I’m happy to think about utility, for some particular problem or class of problems going to the effort of assigning costs and benefits to different outcomes. I agree that a utility analysis, even if (necessarily) imperfect, can usefully focus discussion. For example, if a statistical method for selecting variables is justified on the basis of cost, I like the idea of attempting to quantify the costs of ga
6 0.13960972 754 andrew gelman stats-2011-06-09-Difficulties with Bayesian model averaging
7 0.1283406 1825 andrew gelman stats-2013-04-25-It’s binless! A program for computing normalizing functions
8 0.12717615 1200 andrew gelman stats-2012-03-06-Some economists are skeptical about microfoundations
9 0.12661049 1086 andrew gelman stats-2011-12-27-The most dangerous jobs in America
10 0.12066182 2364 andrew gelman stats-2014-06-08-Regression and causality and variable ordering
11 0.11719774 1401 andrew gelman stats-2012-06-30-David Hogg on statistics
12 0.11151357 1560 andrew gelman stats-2012-11-03-Statistical methods that work in some settings but not others
15 0.09753371 922 andrew gelman stats-2011-09-24-Economists don’t think like accountants—but maybe they should
16 0.096745074 547 andrew gelman stats-2011-01-31-Using sample size in the prior distribution
17 0.090395376 1074 andrew gelman stats-2011-12-20-Reading a research paper != agreeing with its claims
18 0.088849053 1554 andrew gelman stats-2012-10-31-It not necessary that Bayesian methods conform to the likelihood principle
19 0.087301172 247 andrew gelman stats-2010-09-01-How does Bayes do it?
20 0.087121882 1680 andrew gelman stats-2013-01-18-“If scientists wrote horoscopes, this is what yours would say”
topicId topicWeight
[(0, 0.146), (1, 0.057), (2, 0.003), (3, 0.022), (4, 0.005), (5, -0.011), (6, 0.021), (7, -0.02), (8, -0.017), (9, -0.018), (10, -0.012), (11, -0.017), (12, -0.003), (13, -0.021), (14, -0.064), (15, 0.011), (16, 0.058), (17, 0.008), (18, -0.012), (19, -0.012), (20, 0.043), (21, -0.017), (22, 0.016), (23, -0.033), (24, -0.003), (25, 0.026), (26, 0.038), (27, -0.008), (28, 0.04), (29, 0.013), (30, -0.072), (31, -0.002), (32, 0.016), (33, 0.007), (34, 0.023), (35, -0.012), (36, -0.002), (37, -0.051), (38, -0.028), (39, 0.018), (40, 0.063), (41, 0.012), (42, -0.016), (43, 0.025), (44, 0.052), (45, -0.007), (46, 0.047), (47, -0.03), (48, -0.011), (49, 0.06)]
simIndex simValue blogId blogTitle
same-blog 1 0.95667517 1422 andrew gelman stats-2012-07-20-Likelihood thresholds and decisions
Introduction: David Hogg points me to this discussion: Martin Strasbourg and I [Hogg] discussed his project to detect new satellites of M31 in the PAndAS survey. He can construct a likelihood ratio (possibly even a marginalized likelihood ratio) at every position in the M31 imaging, between the best-fit satellite-plus-background model and the best nothing-plus-background model. He can make a two-dimensional map of these likelihood ratios and show a the histogram of them. Looking at this histogram, which has a tail to very large ratios, he asked me, where should I put my cut? That is, at what likelihood ratio does a candidate deserve follow-up? Here’s my unsatisfying answer: To a statistician, the distribution of likelihood ratios is interesting and valuable to study. To an astronomer, it is uninteresting. You don’t want to know the distribution of likelihoods, you want to find satellites . . . I wrote that I think this makes sense and that it would actualy be an interesting and useful rese
2 0.70877022 788 andrew gelman stats-2011-07-06-Early stopping and penalized likelihood
Introduction: Maximum likelihood gives the beat fit to the training data but in general overfits, yielding overly-noisy parameter estimates that don’t perform so well when predicting new data. A popular solution to this overfitting problem takes advantage of the iterative nature of most maximum likelihood algorithms by stopping early. In general, an iterative optimization algorithm goes from a starting point to the maximum of some objective function. If the starting point has some good properties, then early stopping can work well, keeping some of the virtues of the starting point while respecting the data. This trick can be performed the other way, too, starting with the data and then processing it to move it toward a model. That’s how the iterative proportional fitting algorithm of Deming and Stephan (1940) works to fit multivariate categorical data to known margins. In any case, the trick is to stop at the right point–not so soon that you’re ignoring the data but not so late that you en
3 0.70243436 729 andrew gelman stats-2011-05-24-Deviance as a difference
Introduction: Peng Yu writes: On page 180 of BDA2, deviance is defined as D(y,\theta)=-2log p(y|\theta). However, according to GLM 2/e by McCullagh and Nelder, deviance is the different of the log-likelihood of the full model and the base model (times 2) (see the equation on the wiki webpage). The english word ‘deviance’ implies the difference from a standard (in this case, the base model). I’m wondering what the rationale for your definition of deviance, which consists of only 1 term rather than 2 terms. My reply: Deviance is typically computed as a relative quantity; that is, people look at the difference in deviance. So the two definitions are equivalent.
Introduction: Jean Richardson writes: Do you know what might lead to a large negative cross-correlation (-0.95) between deviance and one of the model parameters? Here’s the (brief) background: I [Richardson] have written a Bayesian hierarchical site occupancy model for presence of disease on individual amphibians. The response variable is therefore binary (disease present/absent) and the probability of disease being present in an individual (psi) depends on various covariates (species of amphibian, location sampled, etc.) paramaterized using a logit link function. Replicates are individuals sampled (tested for presence of disease) together. The possibility of imperfect detection is included as p = (prob. disease detected given disease is present). Posterior distributions were estimated using WinBUGS via R2WinBUGS. Simulated data from the model fit the real data very well and posterior distribution densities seem robust to any changes in the model (different priors, etc.) All autocor
5 0.66980296 234 andrew gelman stats-2010-08-25-Modeling constrained parameters
Introduction: Mike McLaughlin writes: In general, is there any way to do MCMC with a fixed constraint? E.g., suppose I measure the three internal angles of a triangle with errors ~dnorm(0, tau) where tau might be different for the three measurements. This would be an easy BUGS/WinBUGS/JAGS exercise but suppose, in addition, I wanted to include prior information to the effect that the three angles had to total 180 degrees exactly. Is this feasible? Could you point me to any BUGS model in which a constraint of this type is implemented? Note: Even in my own (non-hierarchical) code which tends to be component-wise, random-walk Metropolis with tuned Laplacian proposals, I cannot see how I could incorporate such a constraint. My reply: See page 508 of Bayesian Data Analysis (2nd edition). We have an example of such a model there (from this paper with Bois and Jiang).
6 0.66829395 2311 andrew gelman stats-2014-04-29-Bayesian Uncertainty Quantification for Differential Equations!
7 0.66566199 547 andrew gelman stats-2011-01-31-Using sample size in the prior distribution
8 0.65996659 2342 andrew gelman stats-2014-05-21-Models with constraints
9 0.65952677 2208 andrew gelman stats-2014-02-12-How to think about “identifiability” in Bayesian inference?
10 0.65560752 776 andrew gelman stats-2011-06-22-Deviance, DIC, AIC, cross-validation, etc
11 0.65450352 1460 andrew gelman stats-2012-08-16-“Real data can be a pain”
12 0.65437245 1527 andrew gelman stats-2012-10-10-Another reason why you can get good inferences from a bad model
13 0.6442132 2176 andrew gelman stats-2014-01-19-Transformations for non-normal data
14 0.63593829 1189 andrew gelman stats-2012-02-28-Those darn physicists
15 0.6344505 1983 andrew gelman stats-2013-08-15-More on AIC, WAIC, etc
16 0.63426149 1955 andrew gelman stats-2013-07-25-Bayes-respecting experimental design and other things
18 0.63181776 1401 andrew gelman stats-2012-06-30-David Hogg on statistics
19 0.62610126 1510 andrew gelman stats-2012-09-25-Incoherence of Bayesian data analysis
20 0.62375683 398 andrew gelman stats-2010-11-06-Quote of the day
topicId topicWeight
[(2, 0.01), (9, 0.017), (15, 0.043), (16, 0.115), (17, 0.093), (24, 0.182), (42, 0.013), (53, 0.018), (55, 0.034), (65, 0.022), (86, 0.019), (94, 0.021), (95, 0.072), (97, 0.016), (99, 0.22)]
simIndex simValue blogId blogTitle
same-blog 1 0.94439507 1422 andrew gelman stats-2012-07-20-Likelihood thresholds and decisions
Introduction: David Hogg points me to this discussion: Martin Strasbourg and I [Hogg] discussed his project to detect new satellites of M31 in the PAndAS survey. He can construct a likelihood ratio (possibly even a marginalized likelihood ratio) at every position in the M31 imaging, between the best-fit satellite-plus-background model and the best nothing-plus-background model. He can make a two-dimensional map of these likelihood ratios and show a the histogram of them. Looking at this histogram, which has a tail to very large ratios, he asked me, where should I put my cut? That is, at what likelihood ratio does a candidate deserve follow-up? Here’s my unsatisfying answer: To a statistician, the distribution of likelihood ratios is interesting and valuable to study. To an astronomer, it is uninteresting. You don’t want to know the distribution of likelihoods, you want to find satellites . . . I wrote that I think this makes sense and that it would actualy be an interesting and useful rese
2 0.91686893 1557 andrew gelman stats-2012-11-01-‘Researcher Degrees of Freedom’
Introduction: False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant [I]t is unacceptably easy to publish “statistically significant” evidence consistent with any hypothesis. The culprit is a construct we refer to as researcher degrees of freedom. In the course of collecting and analyzing data, researchers have many decisions to make: Should more data be collected? Should some observations be excluded? Which conditions should be combined and which ones compared? Which control variables should be considered? Should specific measures be combined or transformed or both? It is rare, and sometimes impractical, for researchers to make all these decisions beforehand. Rather, it is common (and accepted practice) for researchers to explore various analytic alternatives, to search for a combination that yields “statistical significance,” and to then report only what “worked.” The problem, of course, is that the likelihood of at leas
3 0.9139061 2324 andrew gelman stats-2014-05-07-Once more on nonparametric measures of mutual information
Introduction: Ben Murell writes: Our reply to Kinney and Atwal has come out (http://www.pnas.org/content/early/2014/04/29/1403623111.full.pdf) along with their response (http://www.pnas.org/content/early/2014/04/29/1404661111.full.pdf). I feel like they somewhat missed the point. If you’re still interested in this line of discussion, feel free to post, and maybe the Murrells and Kinney can bash it out in your comments! Background: Too many MC’s not enough MIC’s, or What principles should govern attempts to summarize bivariate associations in large multivariate datasets? Heller, Heller, and Gorfine on univariate and multivariate information measures Kinney and Atwal on the maximal information coefficient Mr. Pearson, meet Mr. Mandelbrot: Detecting Novel Associations in Large Data Sets Gorfine, Heller, Heller, Simon, and Tibshirani don’t like MIC The fun thing is that all these people are sending me their papers, and I’m enough of an outsider in this field that each of the
4 0.91192383 1230 andrew gelman stats-2012-03-26-Further thoughts on nonparametric correlation measures
Introduction: Malka Gorfine, Ruth Heller, and Yair Heller write a comment on the paper of Reshef et al. that we discussed a few months ago. Just to remind you what’s going on here, here’s my quick summary from December: Reshef et al. propose a new nonlinear R-squared-like measure. Unlike R-squared, this new method depends on a tuning parameter that controls the level of discretization, in a “How long is the coast of Britain” sort of way. The dependence on scale is inevitable for such a general method. Just consider: if you sample 1000 points from the unit bivariate normal distribution, (x,y) ~ N(0,I), you’ll be able to fit them perfectly by a 999-degree polynomial fit to the data. So the scale of the fit matters. The clever idea of the paper is that, instead of going for an absolute measure (which, as we’ve seen, will be scale-dependent), they focus on the problem of summarizing the grid of pairwise dependences in a large set of variables. As they put it: “Imagine a data set with hundreds
Introduction: Mike Spagat sent me an email with the above heading, referring to this paper by Leontine Alkema and Jin Rou New, which begins: National estimates of the under-5 mortality rate (U5MR) are used to track progress in reducing child mortality and to evaluate countries’ performance related to United Nations Millennium Development Goal 4, which calls for a reduction in the U5MR by two-thirds between 1990 and 2015. However, for the great majority of developing countries without well-functioning vital registration systems, estimating levels and trends in child mortality is challenging, not only because of limited data availability but also because of issues with data quality. Global U5MR estimates are often constructed without accounting for potential biases in data series, which may lead to inaccurate point estimates and/or credible intervals. We describe a Bayesian penalized B-spline regression model for assessing levels and trends in the U5MR for all countries in the world, whereby bi
6 0.91015381 266 andrew gelman stats-2010-09-09-The future of R
7 0.90835214 301 andrew gelman stats-2010-09-28-Correlation, prediction, variation, etc.
8 0.90709907 639 andrew gelman stats-2011-03-31-Bayes: radical, liberal, or conservative?
9 0.90667683 799 andrew gelman stats-2011-07-13-Hypothesis testing with multiple imputations
10 0.90251052 1206 andrew gelman stats-2012-03-10-95% intervals that I don’t believe, because they’re from a flat prior I don’t believe
11 0.90245199 2179 andrew gelman stats-2014-01-20-The AAA Tranche of Subprime Science
12 0.90231055 447 andrew gelman stats-2010-12-03-Reinventing the wheel, only more so.
13 0.90203559 2288 andrew gelman stats-2014-04-10-Small multiples of lineplots > maps (ok, not always, but yes in this case)
14 0.90055549 1240 andrew gelman stats-2012-04-02-Blogads update
15 0.9002077 599 andrew gelman stats-2011-03-03-Two interesting posts elsewhere on graphics
16 0.8995592 783 andrew gelman stats-2011-06-30-Don’t stop being a statistician once the analysis is done
17 0.89909273 42 andrew gelman stats-2010-05-19-Updated solutions to Bayesian Data Analysis homeworks
18 0.89854747 1093 andrew gelman stats-2011-12-30-Strings Attached: Untangling the Ethics of Incentives
19 0.89787853 898 andrew gelman stats-2011-09-10-Fourteen magic words: an update
20 0.89782804 2121 andrew gelman stats-2013-12-02-Should personal genetic testing be regulated? Battle of the blogroll