andrew_gelman_stats andrew_gelman_stats-2014 andrew_gelman_stats-2014-2220 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: I received a few emails today on bloggable topics. Rather than expanding each response into a full post, I thought I’d just handle them all quickly. 1. Steve Roth asks what I think of this graph : I replied: Interseting but perhaps misleading, as of course any estimate of elasticity of -20 or +5 or whatever is just crap, and so the real question is what is happening in the more reasonable range. 2. One of my General Social Studies colleagues pointed me to this report , writing “FYI – some interesting new results, using the linked GSS-NDI data. I trust this study will raise some heated discussions.” Another colleague wrote, “I’m rather skeptical of the result but at least they spelled GSS right.” The topic is a paper, “Anti-Gay Prejudice and All-Cause Mortality Among Heterosexuals in the United States,” published by Mark Hatzenbuehler, Anna Bellatorre, and Peter Muennig. My reaction: Yes, it seems ludicrous to me. Especially this: The researchers wante
sentIndex sentText sentNum sentScore
1 Steve Roth asks what I think of this graph : I replied: Interseting but perhaps misleading, as of course any estimate of elasticity of -20 or +5 or whatever is just crap, and so the real question is what is happening in the more reasonable range. [sent-4, score-0.069]
2 I trust this study will raise some heated discussions. [sent-7, score-0.181]
3 ” Another colleague wrote, “I’m rather skeptical of the result but at least they spelled GSS right. [sent-8, score-0.143]
4 ” The topic is a paper, “Anti-Gay Prejudice and All-Cause Mortality Among Heterosexuals in the United States,” published by Mark Hatzenbuehler, Anna Bellatorre, and Peter Muennig. [sent-9, score-0.066]
5 I love the way, in a study of “who had died by the end of the study period,” the factor “age” is just listed as one among many control variables. [sent-12, score-0.317]
6 I can’t be sure but it looks like they just included age as a linear factor. [sent-14, score-0.136]
7 Given the huge variation of antigay prejudice by age and cohort, I can’t take this sort of analysis at all seriously. [sent-15, score-0.32]
8 Somebody else wrote: My colleague sent me this ridiculous article but this might not pass the smell test and it might be silly for you to write about an obviously crap paper. [sent-18, score-0.249]
9 But it is published in a peer reviewed journal, and mentioned in a reputable newspaper, plus it is right along the lines of what you’ve been writing about recently. [sent-19, score-0.066]
10 The news article is entitled, “Physicists are more intelligent than social scientists, paper says,” and continues: “The difference is statistically significant only for physics and political science. [sent-21, score-0.182]
11 But the paper’s co-author, Edward Dutton, adjunct professor (docent) in anthropology at the University of Oulu in Finland, said that the smaller differences between other subjects ‘went the same way’ . [sent-22, score-0.157]
12 Also this weird bit, “‘Intelligence and religious and political differences among members of the US academic elite’, published in the Interdisciplinary Journal of Research on Religion, draws primarily on a 1967 study of 148 male academics at the University of Cambridge. [sent-29, score-0.368]
13 In any case, sure, I agree that physicists are more intelligent than social scientists, on average. [sent-34, score-0.284]
14 But the part I really loved was when the author of the study told me he didn’t trust his statistics until they were peer-reviewed! [sent-37, score-0.181]
15 Partner Robert Gerst, will be presenting, What Matters: How statistical significance demolishes productivity, competitiveness, and effective public policy, at the 20th Annual International Deming Research Seminar . [sent-41, score-0.242]
16 This year marks the twentieth anniversary of The Bell Curve, a publishing phenomena claiming to provide scientific evidence of significant differences in intelligence among human races. [sent-44, score-0.295]
17 Gerst details the statistical confidence trick of The Bell Curve that so successfully duped the public, leading media and news outlets . [sent-48, score-0.165]
18 “Executives and HR departments, for example, use the same junk-science as The Bell Curve, to measure employee engagement and improve productivity, and then wonder why productivity and engagement drop,” said Gerst. [sent-52, score-0.63]
19 “Billions have been spent on, Value Added Assessments in education, Six Sigma programs in business, performance evaluation and accountability reporting in government. [sent-53, score-0.197]
20 They’re playing the same statistical con as The Bell Curve and getting the same quality of results. [sent-54, score-0.069]
wordName wordTfidf (topN-words)
[('gerst', 0.251), ('bell', 0.206), ('curve', 0.184), ('prejudice', 0.184), ('productivity', 0.174), ('engagement', 0.171), ('dutton', 0.167), ('niggle', 0.167), ('age', 0.136), ('accountability', 0.126), ('intelligent', 0.116), ('employee', 0.114), ('study', 0.106), ('among', 0.105), ('physicists', 0.102), ('crap', 0.1), ('intelligence', 0.099), ('public', 0.098), ('education', 0.097), ('trick', 0.092), ('differences', 0.091), ('research', 0.09), ('colleague', 0.077), ('huh', 0.077), ('demolishes', 0.076), ('fyi', 0.076), ('competitiveness', 0.076), ('misusing', 0.076), ('trust', 0.075), ('journal', 0.073), ('confidence', 0.073), ('including', 0.073), ('operational', 0.072), ('smell', 0.072), ('anna', 0.072), ('reporting', 0.071), ('con', 0.069), ('dr', 0.069), ('deming', 0.069), ('ludicrous', 0.069), ('hr', 0.069), ('corrupting', 0.069), ('elasticity', 0.069), ('significance', 0.068), ('religiosity', 0.066), ('spelled', 0.066), ('confounded', 0.066), ('anthropology', 0.066), ('published', 0.066), ('social', 0.066)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999988 2220 andrew gelman stats-2014-02-22-Quickies
Introduction: I received a few emails today on bloggable topics. Rather than expanding each response into a full post, I thought I’d just handle them all quickly. 1. Steve Roth asks what I think of this graph : I replied: Interseting but perhaps misleading, as of course any estimate of elasticity of -20 or +5 or whatever is just crap, and so the real question is what is happening in the more reasonable range. 2. One of my General Social Studies colleagues pointed me to this report , writing “FYI – some interesting new results, using the linked GSS-NDI data. I trust this study will raise some heated discussions.” Another colleague wrote, “I’m rather skeptical of the result but at least they spelled GSS right.” The topic is a paper, “Anti-Gay Prejudice and All-Cause Mortality Among Heterosexuals in the United States,” published by Mark Hatzenbuehler, Anna Bellatorre, and Peter Muennig. My reaction: Yes, it seems ludicrous to me. Especially this: The researchers wante
2 0.12578215 2301 andrew gelman stats-2014-04-22-Ticket to Baaaaarf
Introduction: A link from the comments here took me to the wonderfully named Barfblog and a report by Don Schaffner on some reporting. First, the background: A university in England issued a press release saying that “Food picked up just a few seconds after being dropped is less likely to contain bacteria than if it is left for longer periods of time . . . The findings suggest there may be some scientific basis to the ‘5 second rule’ – the urban myth about it being fine to eat food that has only had contact with the floor for five seconds or less. Although people have long followed the 5 second rule, until now it was unclear whether it actually helped.” According to the press release, the study was “undertaken by final year Biology students” and led by a professor of microbiology. The press release hit the big time, hitting NPR, Slate, Forbes, the Daily News, etc etc. Some typical headlines: “5-second rule backed up by science” — Atlanta Journal Constitution “Eating food off the floo
3 0.12552229 970 andrew gelman stats-2011-10-24-Bell Labs
Introduction: Sining Chen told me they’re hiring in the statistics group at Bell Labs . I’ll do my bit for economic stimulus by announcing this job (see below). I love Bell Labs. I worked there for three summers, in a physics lab in 1985-86 under the supervision of Loren Pfeiffer, and by myself in the statistics group in 1990. I learned a lot working for Loren. He was a really smart and driven guy. His lab was a small set of rooms—in Bell Labs, everything’s in a small room, as they value the positive externality of close physical proximity of different labs, which you get by making each lab compact—and it was Loren, his assistant (a guy named Ken West who kept everything running in the lab), and three summer students: me, Gowton Achaibar, and a girl whose name I’ve forgotten. Gowtan and I had a lot of fun chatting in the lab. One day I made a silly comment about Gowton’s accent—he was from Guyana and pronounced “three” as “tree”—and then I apologized and said: Hey, here I am making fun o
4 0.12400755 486 andrew gelman stats-2010-12-26-Age and happiness: The pattern isn’t as clear as you might think
Introduction: A couple people pointed me to this recent news article which discusses “why, beyond middle age, people get happier as they get older.” Here’s the story: When people start out on adult life, they are, on average, pretty cheerful. Things go downhill from youth to middle age until they reach a nadir commonly known as the mid-life crisis. So far, so familiar. The surprising part happens after that. Although as people move towards old age they lose things they treasure–vitality, mental sharpness and looks–they also gain what people spend their lives pursuing: happiness. This curious finding has emerged from a new branch of economics that seeks a more satisfactory measure than money of human well-being. Conventional economics uses money as a proxy for utility–the dismal way in which the discipline talks about happiness. But some economists, unconvinced that there is a direct relationship between money and well-being, have decided to go to the nub of the matter and measure happiness i
5 0.1180828 1435 andrew gelman stats-2012-07-30-Retracted articles and unethical behavior in economics journals?
Introduction: Stan Liebowitz writes: Have you ever heard of an article being retracted in economics? I know you have only been doing this for a few years but I suspect that the answer is that none or very few are retracted. No economist would ever deceive another. There is virtually no interest in detecting cheating. And what good would that do if there is no form of punishment? I say this because I think I have found a case in one of our top journals but the editor allowed the authors of the original article to write an anonymous referee report defending themselves and used this report to reject my comment even though an independent referee recommended publication. My reply: I wonder how this sort of thing will change in the future as journals become less important. My impression is that, on one side, researchers are increasingly citing NBER reports, Arxiv preprints, and the like; while, from the other direction, journals such as Science and Nature are developing the reputations of being “t
7 0.11391481 2223 andrew gelman stats-2014-02-24-“Edlin’s rule” for routinely scaling down published estimates
8 0.11235424 1881 andrew gelman stats-2013-06-03-Boot
10 0.10796792 2137 andrew gelman stats-2013-12-17-Replication backlash
11 0.10598058 2114 andrew gelman stats-2013-11-26-“Please make fun of this claim”
12 0.10556027 2048 andrew gelman stats-2013-10-03-A comment on a post at the Monkey Cage
13 0.10539216 1139 andrew gelman stats-2012-01-26-Suggested resolution of the Bem paradox
14 0.10408491 1928 andrew gelman stats-2013-07-06-How to think about papers published in low-grade journals?
16 0.10356971 2245 andrew gelman stats-2014-03-12-More on publishing in journals
17 0.10297848 1053 andrew gelman stats-2011-12-11-This one is so dumb it makes me want to barf
18 0.10250062 1860 andrew gelman stats-2013-05-17-How can statisticians help psychologists do their research better?
19 0.10226606 709 andrew gelman stats-2011-05-13-D. Kahneman serves up a wacky counterfactual
20 0.10119242 1670 andrew gelman stats-2013-01-13-More Bell Labs happy talk
topicId topicWeight
[(0, 0.241), (1, -0.087), (2, 0.007), (3, -0.139), (4, -0.017), (5, 0.003), (6, -0.041), (7, -0.024), (8, -0.061), (9, 0.039), (10, -0.004), (11, -0.018), (12, -0.001), (13, 0.034), (14, 0.007), (15, 0.01), (16, 0.049), (17, 0.003), (18, 0.006), (19, -0.039), (20, 0.03), (21, 0.019), (22, -0.042), (23, -0.036), (24, 0.028), (25, -0.029), (26, -0.015), (27, -0.055), (28, 0.006), (29, -0.015), (30, -0.049), (31, 0.001), (32, -0.005), (33, -0.003), (34, 0.011), (35, 0.025), (36, 0.006), (37, 0.04), (38, 0.007), (39, 0.019), (40, 0.029), (41, -0.001), (42, 0.052), (43, 0.036), (44, 0.034), (45, -0.044), (46, 0.005), (47, 0.052), (48, -0.035), (49, -0.035)]
simIndex simValue blogId blogTitle
same-blog 1 0.97536349 2220 andrew gelman stats-2014-02-22-Quickies
Introduction: I received a few emails today on bloggable topics. Rather than expanding each response into a full post, I thought I’d just handle them all quickly. 1. Steve Roth asks what I think of this graph : I replied: Interseting but perhaps misleading, as of course any estimate of elasticity of -20 or +5 or whatever is just crap, and so the real question is what is happening in the more reasonable range. 2. One of my General Social Studies colleagues pointed me to this report , writing “FYI – some interesting new results, using the linked GSS-NDI data. I trust this study will raise some heated discussions.” Another colleague wrote, “I’m rather skeptical of the result but at least they spelled GSS right.” The topic is a paper, “Anti-Gay Prejudice and All-Cause Mortality Among Heterosexuals in the United States,” published by Mark Hatzenbuehler, Anna Bellatorre, and Peter Muennig. My reaction: Yes, it seems ludicrous to me. Especially this: The researchers wante
2 0.86112589 2301 andrew gelman stats-2014-04-22-Ticket to Baaaaarf
Introduction: A link from the comments here took me to the wonderfully named Barfblog and a report by Don Schaffner on some reporting. First, the background: A university in England issued a press release saying that “Food picked up just a few seconds after being dropped is less likely to contain bacteria than if it is left for longer periods of time . . . The findings suggest there may be some scientific basis to the ‘5 second rule’ – the urban myth about it being fine to eat food that has only had contact with the floor for five seconds or less. Although people have long followed the 5 second rule, until now it was unclear whether it actually helped.” According to the press release, the study was “undertaken by final year Biology students” and led by a professor of microbiology. The press release hit the big time, hitting NPR, Slate, Forbes, the Daily News, etc etc. Some typical headlines: “5-second rule backed up by science” — Atlanta Journal Constitution “Eating food off the floo
3 0.83021814 1128 andrew gelman stats-2012-01-19-Sharon Begley: Worse than Stephen Jay Gould?
Introduction: Commenter Tggp links to a criticism of science journalist Sharon Begley by science journalist Matthew Hutson. I learned of this dispute after reporting that Begley had received the American Statistical Association’s Excellence in Statistical Reporting Award, a completely undeserved honor, if Hutson is to believed. The two journalists have somewhat similar profiles: Begley was science editor at Newsweek (she’s now at Reuters) and author of “Train Your Mind, Change Your Brain: How a New Science Reveals Our Extraordinary Potential to Transform Ourselves,” and Hutson was news editor at Psychology Today and wrote the similarly self-helpy-titled, “The 7 Laws of Magical Thinking: How Irrational Beliefs Keep Us Happy, Healthy, and Sane.” Hutson writes : Psychological Science recently published a fascinating new study on jealousy. I was interested to read Newsweek’s 1300-word article covering the research by their science editor, Sharon Begley. But part-way through the article, I
4 0.82955486 1160 andrew gelman stats-2012-02-09-Familial Linkage between Neuropsychiatric Disorders and Intellectual Interests
Introduction: When I spoke at Princeton last year, I talked with neuroscientist Sam Wang, who told me about a project he did surveying incoming Princeton freshmen about mental illness in their families. He and his coauthor Benjamin Campbell found some interesting results, which they just published : A link between intellect and temperament has long been the subject of speculation. . . . Studies of the artistically inclined report linkage with familial depression, while among eminent and creative scientists, a lower incidence of affective disorders is found. In the case of developmental disorders, a heightened prevalence of autism spectrum disorders (ASDs) has been found in the families of mathematicians, physicists, and engineers. . . . We surveyed the incoming class of 2014 at Princeton University about their intended academic major, familial incidence of neuropsychiatric disorders, and demographic variables. . . . Consistent with prior findings, we noticed a relation between intended academ
Introduction: Aaron Carroll shoots down a politically-loaded claim about cancer survival. Lots of useful background from science reporter Sharon Begley: With the United States spending more on healthcare than any other country — $2.5 trillion, or just over $8,000 per capita, in 2009 — the question has long been, is it worth it? At least for spending on cancer, a controversial new study answers with an emphatic “yes.” . . . Experts shown an advance copy of the paper by Reuters argued that the tricky statistics of cancer outcomes tripped up the authors. “This study is pure folly,” said biostatistician Dr. Don Berry of MD Anderson Cancer Center in Houston. “It’s completely misguided and it’s dangerous. Not only are the authors’ analyses flawed but their conclusions are also wrong.” Ouch. Arguably the study shouldn’t be getting any coverage at all, but given that it’s in the news, it’s good to see it get shot down. I wonder if the authors will respond to Don Berry and say they’re sorr
6 0.81093538 1671 andrew gelman stats-2013-01-13-Preregistration of Studies and Mock Reports
8 0.79731375 1053 andrew gelman stats-2011-12-11-This one is so dumb it makes me want to barf
9 0.79443455 1766 andrew gelman stats-2013-03-16-“Nightshifts Linked to Increased Risk for Ovarian Cancer”
10 0.7928732 2114 andrew gelman stats-2013-11-26-“Please make fun of this claim”
12 0.77747554 2179 andrew gelman stats-2014-01-20-The AAA Tranche of Subprime Science
14 0.77707195 908 andrew gelman stats-2011-09-14-Type M errors in the lab
16 0.76916361 1860 andrew gelman stats-2013-05-17-How can statisticians help psychologists do their research better?
17 0.76290864 1878 andrew gelman stats-2013-05-31-How to fix the tabloids? Toward replicable social science research
18 0.75876075 2361 andrew gelman stats-2014-06-06-Hurricanes vs. Himmicanes
19 0.75637364 382 andrew gelman stats-2010-10-30-“Presidential Election Outcomes Directly Influence Suicide Rates”
topicId topicWeight
[(3, 0.017), (13, 0.028), (15, 0.041), (16, 0.061), (24, 0.183), (34, 0.036), (35, 0.011), (36, 0.019), (43, 0.019), (51, 0.013), (53, 0.031), (54, 0.016), (56, 0.021), (68, 0.031), (80, 0.01), (82, 0.019), (86, 0.027), (95, 0.032), (98, 0.016), (99, 0.295)]
simIndex simValue blogId blogTitle
same-blog 1 0.97715539 2220 andrew gelman stats-2014-02-22-Quickies
Introduction: I received a few emails today on bloggable topics. Rather than expanding each response into a full post, I thought I’d just handle them all quickly. 1. Steve Roth asks what I think of this graph : I replied: Interseting but perhaps misleading, as of course any estimate of elasticity of -20 or +5 or whatever is just crap, and so the real question is what is happening in the more reasonable range. 2. One of my General Social Studies colleagues pointed me to this report , writing “FYI – some interesting new results, using the linked GSS-NDI data. I trust this study will raise some heated discussions.” Another colleague wrote, “I’m rather skeptical of the result but at least they spelled GSS right.” The topic is a paper, “Anti-Gay Prejudice and All-Cause Mortality Among Heterosexuals in the United States,” published by Mark Hatzenbuehler, Anna Bellatorre, and Peter Muennig. My reaction: Yes, it seems ludicrous to me. Especially this: The researchers wante
2 0.97468227 2140 andrew gelman stats-2013-12-19-Revised evidence for statistical standards
Introduction: X and I heard about this much-publicized recent paper by Val Johnson, who suggests changing the default level of statistical significance from z=2 to z=3 (or, as he puts it, going from p=.05 to p=.005 or .001). Val argues that you need to go out to 3 standard errors to get a Bayes factor of 25 or 50 in favor of the alternative hypothesis. I don’t really buy this, first because Val’s model is a weird (to me) mixture of two point masses, which he creates in order to make a minimax argument, and second because I don’t see why you need a Bayes factor of 25 to 50 in order to make a claim. I’d think that a factor of 5:1, say, provides strong information already—if you really believe those odds. The real issue, as I see it, is that we’re getting Bayes factors and posterior probabilities we don’t believe, because we’re assuming flat priors that don’t really make sense. This is a topic that’s come up over and over in recent months on this blog, for example in this discussion of why I d
3 0.97262812 1162 andrew gelman stats-2012-02-11-Adding an error model to a deterministic model
Introduction: Daniel Lakeland asks , “Where do likelihoods come from?” He describes a class of problems where you have a deterministic dynamic model that you want to fit to data. The data won’t fit perfectly so, if you want to do Bayesian inference, you need to introduce an error model. This looks a little bit different from the usual way that models are presented in statistics textbooks, where the focus is typically on the random error process, not on the deterministic part of the model. A focus on the error process makes sense in some applications that have inherent randomness or variation (for example, genetics, psychology, and survey sampling) but not so much in the physical sciences, where the deterministic model can be complicated and is typically the essence of the study. Often in these sorts of studies, the staring point (and sometimes the ending point) is what the physicists call “nonlinear least squares” or what we would call normally-distributed errors. That’s what we did for our
Introduction: As regular readers of this blog are aware, a few months ago Val Johnson published an article, “Revised standards for statistical evidence,” making a Bayesian argument that researchers and journals should use a p=0.005 publication threshold rather than the usual p=0.05. Christian Robert and I were unconvinced by Val’s reasoning and wrote a response , “Revised evidence for statistical standards,” in which we wrote: Johnson’s minimax prior is not intended to correspond to any distribution of effect sizes; rather, it represents a worst case scenario under some mathematical assumptions. Minimax and tradeoffs do well together, and it is hard for us to see how any worst case procedure can supply much guidance on how to balance between two different losses. . . . We would argue that the appropriate significance level depends on the scenario and that what worked well for agricultural experiments in the 1920s might not be so appropriate for many applications in modern biosciences . . .
5 0.97001016 899 andrew gelman stats-2011-09-10-The statistical significance filter
Introduction: I’ve talked about this a bit but it’s never had its own blog entry (until now). Statistically significant findings tend to overestimate the magnitude of effects. This holds in general (because E(|x|) > |E(x)|) but even more so if you restrict to statistically significant results. Here’s an example. Suppose a true effect of theta is unbiasedly estimated by y ~ N (theta, 1). Further suppose that we will only consider statistically significant results, that is, cases in which |y| > 2. The estimate “|y| conditional on |y|>2″ is clearly an overestimate of |theta|. First off, if |theta|<2, the estimate |y| conditional on statistical significance is not only too high in expectation, it's always too high. This is a problem, given that |theta| is in reality probably is less than 2. (The low-hangning fruit have already been picked, remember?) But even if |theta|>2, the estimate |y| conditional on statistical significance will still be too high in expectation. For a discussion o
6 0.96935678 2149 andrew gelman stats-2013-12-26-Statistical evidence for revised standards
8 0.96858799 2055 andrew gelman stats-2013-10-08-A Bayesian approach for peer-review panels? and a speculation about Bruno Frey
9 0.96716207 936 andrew gelman stats-2011-10-02-Covariate Adjustment in RCT - Model Overfitting in Multilevel Regression
10 0.96663249 1117 andrew gelman stats-2012-01-13-What are the important issues in ethics and statistics? I’m looking for your input!
11 0.96629882 2080 andrew gelman stats-2013-10-28-Writing for free
13 0.96582717 247 andrew gelman stats-2010-09-01-How does Bayes do it?
14 0.96430707 2201 andrew gelman stats-2014-02-06-Bootstrap averaging: Examples where it works and where it doesn’t work
15 0.96430087 447 andrew gelman stats-2010-12-03-Reinventing the wheel, only more so.
16 0.96398574 2109 andrew gelman stats-2013-11-21-Hidden dangers of noninformative priors
17 0.96394372 1240 andrew gelman stats-2012-04-02-Blogads update
18 0.96391577 466 andrew gelman stats-2010-12-13-“The truth wears off: Is there something wrong with the scientific method?”
19 0.96391094 687 andrew gelman stats-2011-04-29-Zero is zero
20 0.96383631 970 andrew gelman stats-2011-10-24-Bell Labs