andrew_gelman_stats andrew_gelman_stats-2010 andrew_gelman_stats-2010-248 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: A couple years ago, I used a question by Benjamin Kay as an excuse to write that it’s usually a bad idea to study a ratio whose denominator has uncertain sign. As I wrote then: Similar problems arise with marginal cost-benefit ratios, LD50 in logistic regression (see chapter 3 of Bayesian Data Analysis for an example), instrumental variables, and the Fieller-Creasy problem in theoretical statistics. . . . In general, the story is that the ratio completely changes in interpretation when the denominator changes sign. More recently, Kay sent in a related question: I [Kay] wondered if you have any advice on handling ratios when the signs change as a result of a parameter. I have three functions, one C * x^a, another D * x^a, and a third f(x,a) in my paper such that: C * x^a, < f(x,a) < D * x^a C,D and a all have the same signs. We can divide through by C * x^a but the results depend on the sign of C either 1< f(x,a) / C * x^a < D * x^a / C * x^a, or 1 / f(x,a
sentIndex sentText sentNum sentScore
1 A couple years ago, I used a question by Benjamin Kay as an excuse to write that it’s usually a bad idea to study a ratio whose denominator has uncertain sign. [sent-1, score-0.749]
2 As I wrote then: Similar problems arise with marginal cost-benefit ratios, LD50 in logistic regression (see chapter 3 of Bayesian Data Analysis for an example), instrumental variables, and the Fieller-Creasy problem in theoretical statistics. [sent-2, score-0.379]
3 In general, the story is that the ratio completely changes in interpretation when the denominator changes sign. [sent-6, score-0.927]
4 More recently, Kay sent in a related question: I [Kay] wondered if you have any advice on handling ratios when the signs change as a result of a parameter. [sent-7, score-0.725]
5 I have three functions, one C * x^a, another D * x^a, and a third f(x,a) in my paper such that: C * x^a, < f(x,a) < D * x^a C,D and a all have the same signs. [sent-8, score-0.065]
6 We can divide through by C * x^a but the results depend on the sign of C either 1< f(x,a) / C * x^a < D * x^a / C * x^a, or 1 / f(x,a) / C * x^a > D * x^a / C * x^a, That is, when the sign on a changes, the inequalities flip. [sent-9, score-0.575]
7 I want to say something about the ratio C/D being close to one so that I can say something about how tight the bounds are on f(x,a) / C * x^a. [sent-10, score-0.715]
8 So being close (say within 5%) has a confusing presentation. [sent-11, score-0.194]
9 I cannot be the first person to have to deal with this, so I wondered if you had any suggestions? [sent-16, score-0.18]
10 My reply: I have to admit I can’t understand much of your notation but I think I get the general picture. [sent-17, score-0.239]
11 In other settings what I’ve done is to try to reformulate the problem. [sent-18, score-0.197]
12 For example, instead of looking at C/D, look at C – D, or perhaps delta*(C-D), where delta is a positive quantity set to a reasonable value (of the order of magnitude of |C+D|). [sent-19, score-0.529]
13 My feeling is that if you carefully express these things as decision problems, ultimately it’s differences rather than ratios that really matter. [sent-20, score-0.552]
14 We use ratios because they are conveniently scale-free, but really it shouldn’t be hard to scale a difference in a reasonable way. [sent-21, score-0.561]
15 The small amount of effort placed into scaling can pay off big-time in clean and direct interpretation. [sent-22, score-0.325]
wordName wordTfidf (topN-words)
[('kay', 0.409), ('ratios', 0.351), ('delta', 0.273), ('ratio', 0.245), ('denominator', 0.19), ('changes', 0.181), ('wondered', 0.18), ('sign', 0.143), ('interpretation', 0.13), ('reformulate', 0.129), ('inequalities', 0.119), ('conveniently', 0.115), ('tight', 0.107), ('benjamin', 0.107), ('close', 0.105), ('bounds', 0.102), ('notation', 0.102), ('scaling', 0.1), ('signs', 0.099), ('handling', 0.095), ('uncertain', 0.095), ('reasonable', 0.095), ('excuse', 0.092), ('confusing', 0.089), ('instrumental', 0.089), ('placed', 0.089), ('divide', 0.089), ('quantity', 0.085), ('depend', 0.081), ('say', 0.078), ('magnitude', 0.076), ('functions', 0.075), ('problems', 0.075), ('marginal', 0.075), ('clean', 0.074), ('express', 0.072), ('logistic', 0.07), ('general', 0.07), ('arise', 0.07), ('settings', 0.068), ('suggestions', 0.068), ('admit', 0.067), ('feeling', 0.066), ('third', 0.065), ('question', 0.065), ('shouldn', 0.063), ('carefully', 0.063), ('mostly', 0.062), ('amount', 0.062), ('whose', 0.062)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999988 248 andrew gelman stats-2010-09-01-Ratios where the numerator and denominator both change signs
Introduction: A couple years ago, I used a question by Benjamin Kay as an excuse to write that it’s usually a bad idea to study a ratio whose denominator has uncertain sign. As I wrote then: Similar problems arise with marginal cost-benefit ratios, LD50 in logistic regression (see chapter 3 of Bayesian Data Analysis for an example), instrumental variables, and the Fieller-Creasy problem in theoretical statistics. . . . In general, the story is that the ratio completely changes in interpretation when the denominator changes sign. More recently, Kay sent in a related question: I [Kay] wondered if you have any advice on handling ratios when the signs change as a result of a parameter. I have three functions, one C * x^a, another D * x^a, and a third f(x,a) in my paper such that: C * x^a, < f(x,a) < D * x^a C,D and a all have the same signs. We can divide through by C * x^a but the results depend on the sign of C either 1< f(x,a) / C * x^a < D * x^a / C * x^a, or 1 / f(x,a
Introduction: Ratio estimates are common in statistics. In survey sampling, the ratio estimate is when you use y/x to estimate Y/X (using the notation in which x,y are totals of sample measurements and X,Y are population totals). In textbook sampling examples, the denominator X will be an all-positive variable, something that is easy to measure and is, ideally, close to proportional to Y. For example, X is last year’s sales and Y is this year’s sales, or X is the number of people in a cluster and Y is some count. Ratio estimation doesn’t work so well if X can be either positive or negative. More generally we can consider any estimate of a ratio, with no need for a survey sampling context. The problem with estimating Y/X is that the very interpretation of Y/X can change completely if the sign of X changes. Everything is ok for a point estimate: you get X.hat and Y.hat, you can take the ratio Y.hat/X.hat, no problem. But the inference falls apart if you have enough uncertainty in X.hat th
3 0.18816295 1422 andrew gelman stats-2012-07-20-Likelihood thresholds and decisions
Introduction: David Hogg points me to this discussion: Martin Strasbourg and I [Hogg] discussed his project to detect new satellites of M31 in the PAndAS survey. He can construct a likelihood ratio (possibly even a marginalized likelihood ratio) at every position in the M31 imaging, between the best-fit satellite-plus-background model and the best nothing-plus-background model. He can make a two-dimensional map of these likelihood ratios and show a the histogram of them. Looking at this histogram, which has a tail to very large ratios, he asked me, where should I put my cut? That is, at what likelihood ratio does a candidate deserve follow-up? Here’s my unsatisfying answer: To a statistician, the distribution of likelihood ratios is interesting and valuable to study. To an astronomer, it is uninteresting. You don’t want to know the distribution of likelihoods, you want to find satellites . . . I wrote that I think this makes sense and that it would actualy be an interesting and useful rese
4 0.14908341 1086 andrew gelman stats-2011-12-27-The most dangerous jobs in America
Introduction: Robin Hanson writes: On the criteria of potential to help people avoid death, this would seem to be among the most important news I’ve ever heard. [In his recent Ph.D. thesis , Ken Lee finds that] death rates depend on job details more than on race, gender, marriage status, rural vs. urban, education, and income combined ! Now for the details. The US Department of Labor has described each of 807 occupations with over 200 detailed features on how jobs are done, skills required, etc.. Lee looked at seven domains of such features, each containing 16 to 57 features, and for each domain Lee did a factor analysis of those features to find the top 2-4 factors. This gave Lee a total of 22 domain factors. Lee also found four overall factors to describe his total set of 225 job and 9 demographic features. (These four factors explain 32%, 15%, 7%, and 4% of total variance.) Lee then tried to use these 26 job factors, along with his other standard predictors (age, race, gender, m
5 0.12915497 2210 andrew gelman stats-2014-02-13-Stopping rules and Bayesian analysis
Introduction: I happened to receive two questions about stopping rules on the same day. First, from Tom Cunningham: I’ve been arguing with my colleagues about whether the stopping rule is relevant (a presenter disclosed that he went out to collect more data because the first experiment didn’t get significant results) — and I believe you have some qualifications to the Bayesian irrelevance argument but I don’t properly understand them. Then, from Benjamin Kay: I have a question that may be of interest for your blog. I was reading about the early history of AIDS and learned that the the trial of AZT was ended early because it was so effective : The trial reported in the New England Journal of medicine, had produced a dramatic result. Before the planned 24 week duration of the study, after a mean period of participation of about 120 days, nineteen participants receiving placebo had died while there was only a single death among those receiving AZT. This appeared to be a momentous break
6 0.12864 549 andrew gelman stats-2011-02-01-“Roughly 90% of the increase in . . .” Hey, wait a minute!
7 0.098240778 1074 andrew gelman stats-2011-12-20-Reading a research paper != agreeing with its claims
8 0.093568772 1941 andrew gelman stats-2013-07-16-Priors
9 0.092507683 1503 andrew gelman stats-2012-09-19-“Poor Smokers in New York State Spend 25% of Income on Cigarettes, Study Finds”
10 0.090362847 368 andrew gelman stats-2010-10-25-Is instrumental variables analysis particularly susceptible to Type M errors?
11 0.086573079 263 andrew gelman stats-2010-09-08-The China Study: fact or fallacy?
12 0.085954621 2368 andrew gelman stats-2014-06-11-Bayes in the research conversation
14 0.082093894 1695 andrew gelman stats-2013-01-28-Economists argue about Bayes
15 0.080969609 1870 andrew gelman stats-2013-05-26-How to understand coefficients that reverse sign when you start controlling for things?
16 0.080447175 888 andrew gelman stats-2011-09-03-A psychology researcher asks: Is Anova dead?
17 0.079766452 2018 andrew gelman stats-2013-09-12-Do you ever have that I-just-fit-a-model feeling?
18 0.079182275 1301 andrew gelman stats-2012-05-05-Related to z-statistics
19 0.077627487 1560 andrew gelman stats-2012-11-03-Statistical methods that work in some settings but not others
20 0.077016018 1038 andrew gelman stats-2011-12-02-Donate Your Data to Science!
topicId topicWeight
[(0, 0.17), (1, 0.029), (2, 0.027), (3, -0.032), (4, 0.012), (5, -0.036), (6, 0.056), (7, 0.004), (8, 0.031), (9, -0.037), (10, -0.008), (11, -0.018), (12, 0.026), (13, -0.001), (14, 0.021), (15, 0.018), (16, 0.018), (17, 0.02), (18, -0.006), (19, 0.017), (20, 0.011), (21, 0.018), (22, 0.037), (23, 0.013), (24, 0.024), (25, 0.037), (26, 0.024), (27, -0.056), (28, -0.004), (29, -0.005), (30, 0.004), (31, 0.016), (32, -0.007), (33, -0.007), (34, 0.01), (35, -0.018), (36, 0.022), (37, 0.029), (38, -0.024), (39, 0.004), (40, 0.024), (41, 0.02), (42, -0.027), (43, -0.002), (44, 0.019), (45, 0.018), (46, 0.018), (47, 0.011), (48, 0.014), (49, 0.026)]
simIndex simValue blogId blogTitle
same-blog 1 0.96099389 248 andrew gelman stats-2010-09-01-Ratios where the numerator and denominator both change signs
Introduction: A couple years ago, I used a question by Benjamin Kay as an excuse to write that it’s usually a bad idea to study a ratio whose denominator has uncertain sign. As I wrote then: Similar problems arise with marginal cost-benefit ratios, LD50 in logistic regression (see chapter 3 of Bayesian Data Analysis for an example), instrumental variables, and the Fieller-Creasy problem in theoretical statistics. . . . In general, the story is that the ratio completely changes in interpretation when the denominator changes sign. More recently, Kay sent in a related question: I [Kay] wondered if you have any advice on handling ratios when the signs change as a result of a parameter. I have three functions, one C * x^a, another D * x^a, and a third f(x,a) in my paper such that: C * x^a, < f(x,a) < D * x^a C,D and a all have the same signs. We can divide through by C * x^a but the results depend on the sign of C either 1< f(x,a) / C * x^a < D * x^a / C * x^a, or 1 / f(x,a
Introduction: Ratio estimates are common in statistics. In survey sampling, the ratio estimate is when you use y/x to estimate Y/X (using the notation in which x,y are totals of sample measurements and X,Y are population totals). In textbook sampling examples, the denominator X will be an all-positive variable, something that is easy to measure and is, ideally, close to proportional to Y. For example, X is last year’s sales and Y is this year’s sales, or X is the number of people in a cluster and Y is some count. Ratio estimation doesn’t work so well if X can be either positive or negative. More generally we can consider any estimate of a ratio, with no need for a survey sampling context. The problem with estimating Y/X is that the very interpretation of Y/X can change completely if the sign of X changes. Everything is ok for a point estimate: you get X.hat and Y.hat, you can take the ratio Y.hat/X.hat, no problem. But the inference falls apart if you have enough uncertainty in X.hat th
3 0.80251813 368 andrew gelman stats-2010-10-25-Is instrumental variables analysis particularly susceptible to Type M errors?
Introduction: Hendrik Juerges writes: I am an applied econometrician. The reason I am writing is that I am pondering a question for some time now and I am curious whether you have any views on it. One problem the practitioner of instrumental variables estimation faces is large standard errors even with very large samples. Part of the problem is of course that one estimates a ratio. Anyhow, more often than not, I and many other researchers I know end up with large point estimates and standard errors when trying IV on a problem. Sometimes some of us are lucky and get a statistically significant result. Those estimates that make it beyond the 2 standard error threshold are often ridiculously large (one famous example in my line of research being Lleras-Muney’s estimates of the 10% effect of one year of schooling on mortality). The standard defense here is that IV estimates the complier-specific causal effect (which is mathematically correct). But still, I find many of the IV results (including my
Introduction: I received the following note from someone who’d like to remain anonymous: I read your post on ethics and statistics, and the comments therein, with much interest. I did notice, however, that most of the dialogue was about ethical behavior of scientists. Herein I’d like to suggest a different take, one that focuses on the statistical methods of scientists. For example, fitting a line to a scatter plot of data using OLS [linear regression] gives more weight to outliers. If each data point represents a person we are weighting people differently. And surely the ethical implications are different if we use a least absolute deviation estimator. Recently I reviewed a paper where the authors claimed one advantage of non-parametric rank-based tests is their robustness to outliers. Again, maybe that outlier is the 10th person who dies from an otherwise beneficial medicine. Should we ignore him in assessing the effect of the medicine? I guess this gets me partly into loss f
5 0.78924578 2204 andrew gelman stats-2014-02-09-Keli Liu and Xiao-Li Meng on Simpson’s paradox
Introduction: XL sent me this paper , “A Fruitful Resolution to Simpson’s Paradox via Multi-Resolution Inference.” I told Keli and Xiao-Li that I wasn’t sure I fully understood the paper—as usual, XL is subtle and sophisticated, also I only get about half of his jokes—but I sent along these thoughts: 1. I do not think counterfactuals or potential outcomes are necessary for Simpson’s paradox. I say this because one can set up Simpson’s paradox with variables that cannot be manipulated, or for which manipulations are not directly of interest. 2. Simpson’s paradox is part of a more general issue that regression coefs change if you add more predictors, the flipping of sign is not really necessary. Here’s an example that I use in my teaching that illustrates both points: I can run a regression predicting income from sex and height. I find that the coef of sex is $10,000 (i.e., comparing a man and woman of the same height, on average the man will make $10,000 more) and the coefficient of h
6 0.78515345 770 andrew gelman stats-2011-06-15-Still more Mr. P in public health
8 0.76059395 708 andrew gelman stats-2011-05-12-Improvement of 5 MPG: how many more auto deaths?
9 0.75833845 706 andrew gelman stats-2011-05-11-The happiness gene: My bottom line (for now)
10 0.75420785 2223 andrew gelman stats-2014-02-24-“Edlin’s rule” for routinely scaling down published estimates
11 0.7541455 566 andrew gelman stats-2011-02-09-The boxer, the wrestler, and the coin flip, again
13 0.74720699 1769 andrew gelman stats-2013-03-18-Tibshirani announces new research result: A significance test for the lasso
17 0.73444194 518 andrew gelman stats-2011-01-15-Regression discontinuity designs: looking for the keys under the lamppost?
18 0.73401797 1793 andrew gelman stats-2013-04-08-The Supreme Court meets the fallacy of the one-sided bet
19 0.7334305 549 andrew gelman stats-2011-02-01-“Roughly 90% of the increase in . . .” Hey, wait a minute!
20 0.73331374 2210 andrew gelman stats-2014-02-13-Stopping rules and Bayesian analysis
topicId topicWeight
[(15, 0.046), (16, 0.059), (21, 0.035), (24, 0.155), (50, 0.012), (53, 0.169), (72, 0.02), (76, 0.026), (84, 0.012), (86, 0.02), (95, 0.054), (97, 0.025), (99, 0.268)]
simIndex simValue blogId blogTitle
1 0.96381581 1856 andrew gelman stats-2013-05-14-GPstuff: Bayesian Modeling with Gaussian Processes
Introduction: I think it’s part of my duty as a blogger to intersperse, along with the steady flow of jokes, rants, and literary criticism, some material that will actually be useful to you. So here goes. Jarno Vanhatalo, Jaakko Riihimäki, Jouni Hartikainen, Pasi Jylänki, Ville Tolvanen, and Aki Vehtari write : The GPstuff toolbox is a versatile collection of Gaussian process models and computational tools required for Bayesian inference. The tools include, among others, various inference methods, sparse approximations and model assessment methods. We can actually now fit Gaussian processes in Stan . But for big problems (or even moderately-sized problems), full Bayes can be slow. GPstuff uses EP, which is faster. At some point we’d like to implement EP in Stan. (Right now we’re working with Dave Blei to implement VB.) GPstuff really works. I saw Aki use it to fit a nonparametric version of the Bangladesh well-switching example in ARM. He was sitting in his office and just whip
2 0.96304142 1905 andrew gelman stats-2013-06-18-There are no fat sprinters
Introduction: This post is by Phil. A little over three years ago I wrote a post about exercise and weight loss in which I described losing a fair amount of weight due to (I believe) an exercise regime, with no effort to change my diet; this contradicted the prediction of studies that had recently been released. The comment thread on that post is quite interesting: a lot of people had had similar experiences — losing weight, or keeping it off, with an exercise program that includes very short periods of exercise at maximal intensity — while other people expressed some skepticism about my claims. Some commenters said that I risked injury; others said it was too early to judge anything because my weight loss might not last. The people who predicted injury were right: running the curve during a 200m sprint a month or two after that post, I strained my Achilles tendon. Nothing really serious, but it did keep me off the track for a couple of months, and rather than go back to sprinting I switched t
3 0.96150053 1677 andrew gelman stats-2013-01-16-Greenland is one tough town
Introduction: Americans (including me) don’t know much about other countries. Jeff Lax sent me to this blog post by Myrddin pointing out that Belgium has a higher murder rate than the rest of Western Europe. I have no particular take on this, but it’s a good reminder that other countries differ from each other. Here in the U.S., we tend to think all western European countries are the same, all eastern European countries are the same, etc. In reality, Sweden is not Finland . P.S. According to the Wiki , Greenland is one tough town. I guess there’s nothing much to do out there but watch satellite TV, chew the blubber, and kill people.
4 0.95959628 1468 andrew gelman stats-2012-08-24-Multilevel modeling and instrumental variables
Introduction: Terence Teo writes: I was wondering if multilevel models can be used as an alternative to 2SLS or IV models to deal with (i) endogeneity and (ii) selection problems. More concretely, I am trying to assess the impact of investment treaties on foreign investment. Aside from the fact that foreign investment is correlated over time, it may be the case that countries that already receive sufficient amounts of foreign investment need not sign treaties, and countries that sign treaties are those that need foreign investment in the first place. Countries thus “select” into treatment; treaty signing is non-random. As such, I argue that to properly estimate the impact of treaties on investment, we must model the determinants of treaty signing. I [Teo] am currently modeling this as two separate models: (1) regress predictors on likelihood of treaty signing, (2) regress treaty (with interactions, etc) on investment (I’ve thought of using propensity score matching for this part of the model)
5 0.95329332 991 andrew gelman stats-2011-11-04-Insecure researchers aren’t sharing their data
Introduction: Jelte Wicherts writes: I thought you might be interested in reading this paper that is to appear this week in PLoS ONE. In it we [Wicherts, Marjan Bakker, and Dylan Molenaar] show that the willingness to share data from published psychological research is associated both with “the strength of the evidence” (against H0) and the prevalence of errors in the reporting of p-values. The issue of data archiving will likely be put on the agenda of granting bodies and the APA/APS because of what Diederik Stapel did . I hate hate hate hate hate when people don’t share their data. In fact, that’s the subject of my very first column on ethics for Chance magazine. I have a story from 22 years ago, when I contacted some scientists and showed them how I could reanalyze their data more efficiently (based on a preliminary analysis of their published summary statistics). They seemed to feel threatened by the suggestion and refused to send me their raw data. (It was an animal experiment
6 0.9530614 1047 andrew gelman stats-2011-12-08-I Am Too Absolutely Heteroskedastic for This Probit Model
7 0.95160383 1589 andrew gelman stats-2012-11-25-Life as a blogger: the emails just get weirder and weirder
10 0.9492346 495 andrew gelman stats-2010-12-31-“Threshold earners” and economic inequality
11 0.94154227 46 andrew gelman stats-2010-05-21-Careers, one-hit wonders, and an offer of a free book
12 0.93580389 1902 andrew gelman stats-2013-06-17-Job opening at new “big data” consulting firm!
13 0.9355492 547 andrew gelman stats-2011-01-31-Using sample size in the prior distribution
14 0.9354015 733 andrew gelman stats-2011-05-27-Another silly graph
15 0.93508345 1802 andrew gelman stats-2013-04-14-Detecting predictability in complex ecosystems
16 0.93303859 446 andrew gelman stats-2010-12-03-Is 0.05 too strict as a p-value threshold?
17 0.93209004 2022 andrew gelman stats-2013-09-13-You heard it here first: Intense exercise can suppress appetite
same-blog 18 0.9303546 248 andrew gelman stats-2010-09-01-Ratios where the numerator and denominator both change signs
19 0.92792439 880 andrew gelman stats-2011-08-30-Annals of spam
20 0.92303932 354 andrew gelman stats-2010-10-19-There’s only one Amtrak