andrew_gelman_stats andrew_gelman_stats-2012 andrew_gelman_stats-2012-1422 knowledge-graph by maker-knowledge-mining

1422 andrew gelman stats-2012-07-20-Likelihood thresholds and decisions


meta infos for this blog

Source: html

Introduction: David Hogg points me to this discussion: Martin Strasbourg and I [Hogg] discussed his project to detect new satellites of M31 in the PAndAS survey. He can construct a likelihood ratio (possibly even a marginalized likelihood ratio) at every position in the M31 imaging, between the best-fit satellite-plus-background model and the best nothing-plus-background model. He can make a two-dimensional map of these likelihood ratios and show a the histogram of them. Looking at this histogram, which has a tail to very large ratios, he asked me, where should I put my cut? That is, at what likelihood ratio does a candidate deserve follow-up? Here’s my unsatisfying answer: To a statistician, the distribution of likelihood ratios is interesting and valuable to study. To an astronomer, it is uninteresting. You don’t want to know the distribution of likelihoods, you want to find satellites . . . I wrote that I think this makes sense and that it would actualy be an interesting and useful rese


Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 David Hogg points me to this discussion: Martin Strasbourg and I [Hogg] discussed his project to detect new satellites of M31 in the PAndAS survey. [sent-1, score-0.516]

2 He can construct a likelihood ratio (possibly even a marginalized likelihood ratio) at every position in the M31 imaging, between the best-fit satellite-plus-background model and the best nothing-plus-background model. [sent-2, score-1.238]

3 He can make a two-dimensional map of these likelihood ratios and show a the histogram of them. [sent-3, score-0.914]

4 Looking at this histogram, which has a tail to very large ratios, he asked me, where should I put my cut? [sent-4, score-0.181]

5 That is, at what likelihood ratio does a candidate deserve follow-up? [sent-5, score-0.769]

6 Here’s my unsatisfying answer: To a statistician, the distribution of likelihood ratios is interesting and valuable to study. [sent-6, score-0.999]

7 You don’t want to know the distribution of likelihoods, you want to find satellites . [sent-8, score-0.396]

8 I wrote that I think this makes sense and that it would actualy be an interesting and useful research project to formalize this as a decision problem. [sent-11, score-0.318]

9 I’ve seen this sort of question arise in genetics (where should the p-value threshold be when you’re selecting N out of a million genes) but it’s frustrating because the cost-benefit calculations always seem implicit. [sent-12, score-0.618]

10 In one—just one—of my papers we put an explicit utility model: http://arxiv. [sent-15, score-0.351]

11 2233 The utility model is on page 17 and we use it explicitly on page 18 and on. [sent-17, score-0.564]

12 com ) that has to *decide* whether to return results to the user or not, given probabilistic information about a submitted image. [sent-21, score-0.336]


similar blogs computed by tfidf model

tfidf for this blog:

wordName wordTfidf (topN-words)

[('likelihood', 0.323), ('hogg', 0.315), ('satellites', 0.294), ('ratios', 0.283), ('ratio', 0.264), ('histogram', 0.227), ('utility', 0.177), ('marginalized', 0.147), ('astronomer', 0.139), ('unsatisfying', 0.133), ('http', 0.132), ('imaging', 0.128), ('project', 0.125), ('formalize', 0.118), ('page', 0.11), ('selecting', 0.108), ('likelihoods', 0.108), ('deserve', 0.108), ('tail', 0.107), ('genes', 0.107), ('distribution', 0.102), ('explicit', 0.1), ('genetics', 0.098), ('detect', 0.097), ('construct', 0.094), ('martin', 0.094), ('probabilistic', 0.092), ('frustrating', 0.092), ('threshold', 0.09), ('inside', 0.088), ('model', 0.087), ('submitted', 0.086), ('calculations', 0.084), ('user', 0.084), ('valuable', 0.083), ('cut', 0.081), ('map', 0.081), ('explicitly', 0.08), ('interesting', 0.075), ('arise', 0.075), ('decide', 0.075), ('return', 0.074), ('candidate', 0.074), ('put', 0.074), ('possibly', 0.071), ('million', 0.071), ('web', 0.07), ('cool', 0.07), ('replied', 0.069), ('running', 0.065)]

similar blogs list:

simIndex simValue blogId blogTitle

same-blog 1 1.0 1422 andrew gelman stats-2012-07-20-Likelihood thresholds and decisions

Introduction: David Hogg points me to this discussion: Martin Strasbourg and I [Hogg] discussed his project to detect new satellites of M31 in the PAndAS survey. He can construct a likelihood ratio (possibly even a marginalized likelihood ratio) at every position in the M31 imaging, between the best-fit satellite-plus-background model and the best nothing-plus-background model. He can make a two-dimensional map of these likelihood ratios and show a the histogram of them. Looking at this histogram, which has a tail to very large ratios, he asked me, where should I put my cut? That is, at what likelihood ratio does a candidate deserve follow-up? Here’s my unsatisfying answer: To a statistician, the distribution of likelihood ratios is interesting and valuable to study. To an astronomer, it is uninteresting. You don’t want to know the distribution of likelihoods, you want to find satellites . . . I wrote that I think this makes sense and that it would actualy be an interesting and useful rese

2 0.18816295 248 andrew gelman stats-2010-09-01-Ratios where the numerator and denominator both change signs

Introduction: A couple years ago, I used a question by Benjamin Kay as an excuse to write that it’s usually a bad idea to study a ratio whose denominator has uncertain sign. As I wrote then: Similar problems arise with marginal cost-benefit ratios, LD50 in logistic regression (see chapter 3 of Bayesian Data Analysis for an example), instrumental variables, and the Fieller-Creasy problem in theoretical statistics. . . . In general, the story is that the ratio completely changes in interpretation when the denominator changes sign. More recently, Kay sent in a related question: I [Kay] wondered if you have any advice on handling ratios when the signs change as a result of a parameter. I have three functions, one C * x^a, another D * x^a, and a third f(x,a) in my paper such that: C * x^a, < f(x,a) < D * x^a C,D and a all have the same signs. We can divide through by C * x^a but the results depend on the sign of C either 1< f(x,a) / C * x^a < D * x^a / C * x^a, or 1 / f(x,a

3 0.17084049 775 andrew gelman stats-2011-06-21-Fundamental difficulty of inference for a ratio when the denominator could be positive or negative

Introduction: Ratio estimates are common in statistics. In survey sampling, the ratio estimate is when you use y/x to estimate Y/X (using the notation in which x,y are totals of sample measurements and X,Y are population totals). In textbook sampling examples, the denominator X will be an all-positive variable, something that is easy to measure and is, ideally, close to proportional to Y. For example, X is last year’s sales and Y is this year’s sales, or X is the number of people in a cluster and Y is some count. Ratio estimation doesn’t work so well if X can be either positive or negative. More generally we can consider any estimate of a ratio, with no need for a survey sampling context. The problem with estimating Y/X is that the very interpretation of Y/X can change completely if the sign of X changes. Everything is ok for a point estimate: you get X.hat and Y.hat, you can take the ratio Y.hat/X.hat, no problem. But the inference falls apart if you have enough uncertainty in X.hat th

4 0.15732108 398 andrew gelman stats-2010-11-06-Quote of the day

Introduction: “A statistical model is usually taken to be summarized by a likelihood, or a likelihood and a prior distribution, but we go an extra step by noting that the parameters of a model are typically batched, and we take this batching as an essential part of the model.”

5 0.14917059 1510 andrew gelman stats-2012-09-25-Incoherence of Bayesian data analysis

Introduction: Hogg writes: At the end this article you wonder about consistency. Have you ever considered the possibility that utility might resolve some of the problems? I have no idea if it would—I am not advocating that position—I just get some kind of intuition from phrases like “Judgment is required to decide…”. Perhaps there is a coherent and objective description of what is—or could be—done under a coherent “utility” model (like a utility that could be objectively agreed upon and computed). Utilities are usually subjective—true—but priors are usually subjective too. My reply: I’m happy to think about utility, for some particular problem or class of problems going to the effort of assigning costs and benefits to different outcomes. I agree that a utility analysis, even if (necessarily) imperfect, can usefully focus discussion. For example, if a statistical method for selecting variables is justified on the basis of cost, I like the idea of attempting to quantify the costs of ga

6 0.13960972 754 andrew gelman stats-2011-06-09-Difficulties with Bayesian model averaging

7 0.1283406 1825 andrew gelman stats-2013-04-25-It’s binless! A program for computing normalizing functions

8 0.12717615 1200 andrew gelman stats-2012-03-06-Some economists are skeptical about microfoundations

9 0.12661049 1086 andrew gelman stats-2011-12-27-The most dangerous jobs in America

10 0.12066182 2364 andrew gelman stats-2014-06-08-Regression and causality and variable ordering

11 0.11719774 1401 andrew gelman stats-2012-06-30-David Hogg on statistics

12 0.11151357 1560 andrew gelman stats-2012-11-03-Statistical methods that work in some settings but not others

13 0.11054666 774 andrew gelman stats-2011-06-20-The pervasive twoishness of statistics; in particular, the “sampling distribution” and the “likelihood” are two different models, and that’s a good thing

14 0.10540088 1972 andrew gelman stats-2013-08-07-When you’re planning on fitting a model, build up to it by fitting simpler models first. Then, once you have a model you like, check the hell out of it

15 0.09753371 922 andrew gelman stats-2011-09-24-Economists don’t think like accountants—but maybe they should

16 0.096745074 547 andrew gelman stats-2011-01-31-Using sample size in the prior distribution

17 0.090395376 1074 andrew gelman stats-2011-12-20-Reading a research paper != agreeing with its claims

18 0.088849053 1554 andrew gelman stats-2012-10-31-It not necessary that Bayesian methods conform to the likelihood principle

19 0.087301172 247 andrew gelman stats-2010-09-01-How does Bayes do it?

20 0.087121882 1680 andrew gelman stats-2013-01-18-“If scientists wrote horoscopes, this is what yours would say”


similar blogs computed by lsi model

lsi for this blog:

topicId topicWeight

[(0, 0.146), (1, 0.057), (2, 0.003), (3, 0.022), (4, 0.005), (5, -0.011), (6, 0.021), (7, -0.02), (8, -0.017), (9, -0.018), (10, -0.012), (11, -0.017), (12, -0.003), (13, -0.021), (14, -0.064), (15, 0.011), (16, 0.058), (17, 0.008), (18, -0.012), (19, -0.012), (20, 0.043), (21, -0.017), (22, 0.016), (23, -0.033), (24, -0.003), (25, 0.026), (26, 0.038), (27, -0.008), (28, 0.04), (29, 0.013), (30, -0.072), (31, -0.002), (32, 0.016), (33, 0.007), (34, 0.023), (35, -0.012), (36, -0.002), (37, -0.051), (38, -0.028), (39, 0.018), (40, 0.063), (41, 0.012), (42, -0.016), (43, 0.025), (44, 0.052), (45, -0.007), (46, 0.047), (47, -0.03), (48, -0.011), (49, 0.06)]

similar blogs list:

simIndex simValue blogId blogTitle

same-blog 1 0.95667517 1422 andrew gelman stats-2012-07-20-Likelihood thresholds and decisions

Introduction: David Hogg points me to this discussion: Martin Strasbourg and I [Hogg] discussed his project to detect new satellites of M31 in the PAndAS survey. He can construct a likelihood ratio (possibly even a marginalized likelihood ratio) at every position in the M31 imaging, between the best-fit satellite-plus-background model and the best nothing-plus-background model. He can make a two-dimensional map of these likelihood ratios and show a the histogram of them. Looking at this histogram, which has a tail to very large ratios, he asked me, where should I put my cut? That is, at what likelihood ratio does a candidate deserve follow-up? Here’s my unsatisfying answer: To a statistician, the distribution of likelihood ratios is interesting and valuable to study. To an astronomer, it is uninteresting. You don’t want to know the distribution of likelihoods, you want to find satellites . . . I wrote that I think this makes sense and that it would actualy be an interesting and useful rese

2 0.70877022 788 andrew gelman stats-2011-07-06-Early stopping and penalized likelihood

Introduction: Maximum likelihood gives the beat fit to the training data but in general overfits, yielding overly-noisy parameter estimates that don’t perform so well when predicting new data. A popular solution to this overfitting problem takes advantage of the iterative nature of most maximum likelihood algorithms by stopping early. In general, an iterative optimization algorithm goes from a starting point to the maximum of some objective function. If the starting point has some good properties, then early stopping can work well, keeping some of the virtues of the starting point while respecting the data. This trick can be performed the other way, too, starting with the data and then processing it to move it toward a model. That’s how the iterative proportional fitting algorithm of Deming and Stephan (1940) works to fit multivariate categorical data to known margins. In any case, the trick is to stop at the right point–not so soon that you’re ignoring the data but not so late that you en

3 0.70243436 729 andrew gelman stats-2011-05-24-Deviance as a difference

Introduction: Peng Yu writes: On page 180 of BDA2, deviance is defined as D(y,\theta)=-2log p(y|\theta). However, according to GLM 2/e by McCullagh and Nelder, deviance is the different of the log-likelihood of the full model and the base model (times 2) (see the equation on the wiki webpage). The english word ‘deviance’ implies the difference from a standard (in this case, the base model). I’m wondering what the rationale for your definition of deviance, which consists of only 1 term rather than 2 terms. My reply: Deviance is typically computed as a relative quantity; that is, people look at the difference in deviance. So the two definitions are equivalent.

4 0.6782189 1221 andrew gelman stats-2012-03-19-Whassup with deviance having a high posterior correlation with a parameter in the model?

Introduction: Jean Richardson writes: Do you know what might lead to a large negative cross-correlation (-0.95) between deviance and one of the model parameters? Here’s the (brief) background: I [Richardson] have written a Bayesian hierarchical site occupancy model for presence of disease on individual amphibians. The response variable is therefore binary (disease present/absent) and the probability of disease being present in an individual (psi) depends on various covariates (species of amphibian, location sampled, etc.) paramaterized using a logit link function. Replicates are individuals sampled (tested for presence of disease) together. The possibility of imperfect detection is included as p = (prob. disease detected given disease is present). Posterior distributions were estimated using WinBUGS via R2WinBUGS. Simulated data from the model fit the real data very well and posterior distribution densities seem robust to any changes in the model (different priors, etc.) All autocor

5 0.66980296 234 andrew gelman stats-2010-08-25-Modeling constrained parameters

Introduction: Mike McLaughlin writes: In general, is there any way to do MCMC with a fixed constraint? E.g., suppose I measure the three internal angles of a triangle with errors ~dnorm(0, tau) where tau might be different for the three measurements. This would be an easy BUGS/WinBUGS/JAGS exercise but suppose, in addition, I wanted to include prior information to the effect that the three angles had to total 180 degrees exactly. Is this feasible? Could you point me to any BUGS model in which a constraint of this type is implemented? Note: Even in my own (non-hierarchical) code which tends to be component-wise, random-walk Metropolis with tuned Laplacian proposals, I cannot see how I could incorporate such a constraint. My reply: See page 508 of Bayesian Data Analysis (2nd edition). We have an example of such a model there (from this paper with Bois and Jiang).

6 0.66829395 2311 andrew gelman stats-2014-04-29-Bayesian Uncertainty Quantification for Differential Equations!

7 0.66566199 547 andrew gelman stats-2011-01-31-Using sample size in the prior distribution

8 0.65996659 2342 andrew gelman stats-2014-05-21-Models with constraints

9 0.65952677 2208 andrew gelman stats-2014-02-12-How to think about “identifiability” in Bayesian inference?

10 0.65560752 776 andrew gelman stats-2011-06-22-Deviance, DIC, AIC, cross-validation, etc

11 0.65450352 1460 andrew gelman stats-2012-08-16-“Real data can be a pain”

12 0.65437245 1527 andrew gelman stats-2012-10-10-Another reason why you can get good inferences from a bad model

13 0.6442132 2176 andrew gelman stats-2014-01-19-Transformations for non-normal data

14 0.63593829 1189 andrew gelman stats-2012-02-28-Those darn physicists

15 0.6344505 1983 andrew gelman stats-2013-08-15-More on AIC, WAIC, etc

16 0.63426149 1955 andrew gelman stats-2013-07-25-Bayes-respecting experimental design and other things

17 0.6323697 1875 andrew gelman stats-2013-05-28-Simplify until your fake-data check works, then add complications until you can figure out where the problem is coming from

18 0.63181776 1401 andrew gelman stats-2012-06-30-David Hogg on statistics

19 0.62610126 1510 andrew gelman stats-2012-09-25-Incoherence of Bayesian data analysis

20 0.62375683 398 andrew gelman stats-2010-11-06-Quote of the day


similar blogs computed by lda model

lda for this blog:

topicId topicWeight

[(2, 0.01), (9, 0.017), (15, 0.043), (16, 0.115), (17, 0.093), (24, 0.182), (42, 0.013), (53, 0.018), (55, 0.034), (65, 0.022), (86, 0.019), (94, 0.021), (95, 0.072), (97, 0.016), (99, 0.22)]

similar blogs list:

simIndex simValue blogId blogTitle

same-blog 1 0.94439507 1422 andrew gelman stats-2012-07-20-Likelihood thresholds and decisions

Introduction: David Hogg points me to this discussion: Martin Strasbourg and I [Hogg] discussed his project to detect new satellites of M31 in the PAndAS survey. He can construct a likelihood ratio (possibly even a marginalized likelihood ratio) at every position in the M31 imaging, between the best-fit satellite-plus-background model and the best nothing-plus-background model. He can make a two-dimensional map of these likelihood ratios and show a the histogram of them. Looking at this histogram, which has a tail to very large ratios, he asked me, where should I put my cut? That is, at what likelihood ratio does a candidate deserve follow-up? Here’s my unsatisfying answer: To a statistician, the distribution of likelihood ratios is interesting and valuable to study. To an astronomer, it is uninteresting. You don’t want to know the distribution of likelihoods, you want to find satellites . . . I wrote that I think this makes sense and that it would actualy be an interesting and useful rese

2 0.91686893 1557 andrew gelman stats-2012-11-01-‘Researcher Degrees of Freedom’

Introduction: False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant [I]t is unacceptably easy to publish “statistically significant” evidence consistent with any hypothesis. The culprit is a construct we refer to as researcher degrees of freedom. In the course of collecting and analyzing data, researchers have many decisions to make: Should more data be collected? Should some observations be excluded? Which conditions should be combined and which ones compared? Which control variables should be considered? Should specific measures be combined or transformed or both? It is rare, and sometimes impractical, for researchers to make all these decisions beforehand. Rather, it is common (and accepted practice) for researchers to explore various analytic alternatives, to search for a combination that yields “statistical significance,” and to then report only what “worked.” The problem, of course, is that the likelihood of at leas

3 0.9139061 2324 andrew gelman stats-2014-05-07-Once more on nonparametric measures of mutual information

Introduction: Ben Murell writes: Our reply to Kinney and Atwal has come out (http://www.pnas.org/content/early/2014/04/29/1403623111.full.pdf) along with their response (http://www.pnas.org/content/early/2014/04/29/1404661111.full.pdf). I feel like they somewhat missed the point. If you’re still interested in this line of discussion, feel free to post, and maybe the Murrells and Kinney can bash it out in your comments! Background: Too many MC’s not enough MIC’s, or What principles should govern attempts to summarize bivariate associations in large multivariate datasets? Heller, Heller, and Gorfine on univariate and multivariate information measures Kinney and Atwal on the maximal information coefficient Mr. Pearson, meet Mr. Mandelbrot: Detecting Novel Associations in Large Data Sets Gorfine, Heller, Heller, Simon, and Tibshirani don’t like MIC The fun thing is that all these people are sending me their papers, and I’m enough of an outsider in this field that each of the

4 0.91192383 1230 andrew gelman stats-2012-03-26-Further thoughts on nonparametric correlation measures

Introduction: Malka Gorfine, Ruth Heller, and Yair Heller write a comment on the paper of Reshef et al. that we discussed a few months ago. Just to remind you what’s going on here, here’s my quick summary from December: Reshef et al. propose a new nonlinear R-squared-like measure. Unlike R-squared, this new method depends on a tuning parameter that controls the level of discretization, in a “How long is the coast of Britain” sort of way. The dependence on scale is inevitable for such a general method. Just consider: if you sample 1000 points from the unit bivariate normal distribution, (x,y) ~ N(0,I), you’ll be able to fit them perfectly by a 999-degree polynomial fit to the data. So the scale of the fit matters. The clever idea of the paper is that, instead of going for an absolute measure (which, as we’ve seen, will be scale-dependent), they focus on the problem of summarizing the grid of pairwise dependences in a large set of variables. As they put it: “Imagine a data set with hundreds

5 0.91052359 2135 andrew gelman stats-2013-12-15-The UN Plot to Force Bayesianism on Unsuspecting Americans (penalized B-Spline edition)

Introduction: Mike Spagat sent me an email with the above heading, referring to this paper by Leontine Alkema and Jin Rou New, which begins: National estimates of the under-5 mortality rate (U5MR) are used to track progress in reducing child mortality and to evaluate countries’ performance related to United Nations Millennium Development Goal 4, which calls for a reduction in the U5MR by two-thirds between 1990 and 2015. However, for the great majority of developing countries without well-functioning vital registration systems, estimating levels and trends in child mortality is challenging, not only because of limited data availability but also because of issues with data quality. Global U5MR estimates are often constructed without accounting for potential biases in data series, which may lead to inaccurate point estimates and/or credible intervals. We describe a Bayesian penalized B-spline regression model for assessing levels and trends in the U5MR for all countries in the world, whereby bi

6 0.91015381 266 andrew gelman stats-2010-09-09-The future of R

7 0.90835214 301 andrew gelman stats-2010-09-28-Correlation, prediction, variation, etc.

8 0.90709907 639 andrew gelman stats-2011-03-31-Bayes: radical, liberal, or conservative?

9 0.90667683 799 andrew gelman stats-2011-07-13-Hypothesis testing with multiple imputations

10 0.90251052 1206 andrew gelman stats-2012-03-10-95% intervals that I don’t believe, because they’re from a flat prior I don’t believe

11 0.90245199 2179 andrew gelman stats-2014-01-20-The AAA Tranche of Subprime Science

12 0.90231055 447 andrew gelman stats-2010-12-03-Reinventing the wheel, only more so.

13 0.90203559 2288 andrew gelman stats-2014-04-10-Small multiples of lineplots > maps (ok, not always, but yes in this case)

14 0.90055549 1240 andrew gelman stats-2012-04-02-Blogads update

15 0.9002077 599 andrew gelman stats-2011-03-03-Two interesting posts elsewhere on graphics

16 0.8995592 783 andrew gelman stats-2011-06-30-Don’t stop being a statistician once the analysis is done

17 0.89909273 42 andrew gelman stats-2010-05-19-Updated solutions to Bayesian Data Analysis homeworks

18 0.89854747 1093 andrew gelman stats-2011-12-30-Strings Attached: Untangling the Ethics of Incentives

19 0.89787853 898 andrew gelman stats-2011-09-10-Fourteen magic words: an update

20 0.89782804 2121 andrew gelman stats-2013-12-02-Should personal genetic testing be regulated? Battle of the blogroll