andrew_gelman_stats andrew_gelman_stats-2014 andrew_gelman_stats-2014-2177 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Andrew Anthony tells the excellent story of how Nick Brown, Alan Sokal, and Harris Friedman shot down some particularly silly work in psychology. (“According to the graph, it all came down to a specific ratio of positive emotions to negative emotions. If your ratio was greater than 2.9013 positive emotions to 1 negative emotion you were flourishing in life. If your ratio was less than that number you were languishing.” And, yes, the work they were shooting down really is that bad.) If you want to see what the fuss is about, just google “2.9013.” Here’s an example (from 2012) of an uncritical reporting of the claim, here’s another one from 2010, here’s one from 2011 . . . well, you get the idea. And here’s a quick summary posted by Rolf Zwaan after Brown et al. came out with their paper. I know Sokal and Brown and so this story was not news to me. I didn’t post anything about it on this blog because it seemed like it was getting enough coverage elsewhere. I think Ni
sentIndex sentText sentNum sentScore
1 (“According to the graph, it all came down to a specific ratio of positive emotions to negative emotions. [sent-2, score-0.233]
2 9013 positive emotions to 1 negative emotion you were flourishing in life. [sent-4, score-0.184]
3 I think Nick Brown did a good job tracking this one down, and I don’t envy them the effort it took for them to publish their criticism in the American Psychologist. [sent-17, score-0.367]
4 And of course I did not attempt to publish a letter in Psychological Science for each of their flawed papers (that would be a lot of letters! [sent-19, score-0.174]
5 ), nor did I bother writing a letter to PNAS regarding that horrible, horrible cubic polynomial fit leading to the implausible claim that a particular sort of air pollution is causing 500 million Chinese people to lose an average of five years of life. [sent-20, score-0.498]
6 I was bothered by a headline But that’s not why I wrote this post. [sent-23, score-0.179]
7 The reason is that I was bothered by the subheadline, “The astonishing story of Nick Brown, the British man who began a part-time psychology course in his 50s – and ended up taking on America’s academic establishment. [sent-24, score-0.4]
8 ” I have no doubt that the author of the debunked work is indeed an establishment figure in American academia; the news article (see link at top of this post) makes that clear enough. [sent-26, score-0.437]
9 But I’m also pretty sure that (a) most of America’s academic establishment had never heard of this work, and (b) many of the establishment who had heard of it, thought it was silly. [sent-27, score-0.806]
10 After all, I’m part of America’s academic establishment too! [sent-28, score-0.452]
11 So, while I liked Andrew Anthony’s reporting on this one, I thought the subheadline was misleading. [sent-29, score-0.192]
12 I think they found a bad paper and then put in a big pile of effort to not just refute it, but to get their refutation published in a visible place. [sent-32, score-0.231]
13 It’s not so much that they needed to fight the establishment; it’s more that they had to swim through the molasses which is the conventional attitude that presumes that a scientific claim is correct, just because it has been published in a respected journal. [sent-33, score-0.331]
14 Truthteller struggles to get people to accept the discomfort of publicly reassessing a published claim. [sent-39, score-0.56]
15 Story #1 does happen sometime (see here , for example), and of course people who break the rules have a notorious pattern of not admitting it even after getting caught, but this more recent case seems more like an example of story #2. [sent-40, score-0.199]
16 I doubt that a lot of researchers in psychology actually believed that “2. [sent-41, score-0.156]
17 Similarly, I doubt many researchers actually believe the claim about the air pollution causing 5 years of life being lost in half of China, but I think it would be difficult for me to publish my criticism of the claim in as high-visibility a place as where the original article appeared. [sent-43, score-0.797]
18 At one level, I can understand this—the original article requires substantive research, while the correction can often be done just using methodological criticism alone—but the result is that lots of stuff that shouldn’t really be believed sits around in the literature. [sent-46, score-0.279]
19 That is a story worth telling, and that’s why I wanted to emphasize that the tale of Nick Brown, while it does feature some colorful characters (including the inventor of the notorious 2. [sent-47, score-0.199]
20 9013), really fits into the category of: “Truthteller struggles to get people to accept the discomfort of publicly reassessing a published claim. [sent-48, score-0.56]
wordName wordTfidf (topN-words)
[('brown', 0.365), ('establishment', 0.354), ('nick', 0.212), ('sokal', 0.174), ('criticism', 0.136), ('story', 0.135), ('subheadline', 0.135), ('truthteller', 0.135), ('effort', 0.125), ('reassessing', 0.123), ('emotions', 0.123), ('fuss', 0.111), ('ratio', 0.11), ('claim', 0.107), ('published', 0.106), ('publish', 0.106), ('bothered', 0.104), ('discomfort', 0.104), ('pollution', 0.104), ('academic', 0.098), ('shooting', 0.093), ('struggles', 0.09), ('america', 0.089), ('friedman', 0.089), ('anthony', 0.086), ('doubt', 0.083), ('causing', 0.082), ('publicly', 0.078), ('headline', 0.075), ('believed', 0.073), ('air', 0.072), ('methodological', 0.07), ('letter', 0.068), ('eventually', 0.067), ('american', 0.067), ('horrible', 0.065), ('notorious', 0.064), ('man', 0.063), ('emotion', 0.061), ('weeded', 0.061), ('attitude', 0.06), ('accept', 0.059), ('confront', 0.058), ('presumes', 0.058), ('skeptic', 0.058), ('disappear', 0.058), ('heroic', 0.058), ('bissell', 0.058), ('mina', 0.058), ('reporting', 0.057)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999976 2177 andrew gelman stats-2014-01-19-“The British amateur who debunked the mathematics of happiness”
Introduction: Andrew Anthony tells the excellent story of how Nick Brown, Alan Sokal, and Harris Friedman shot down some particularly silly work in psychology. (“According to the graph, it all came down to a specific ratio of positive emotions to negative emotions. If your ratio was greater than 2.9013 positive emotions to 1 negative emotion you were flourishing in life. If your ratio was less than that number you were languishing.” And, yes, the work they were shooting down really is that bad.) If you want to see what the fuss is about, just google “2.9013.” Here’s an example (from 2012) of an uncritical reporting of the claim, here’s another one from 2010, here’s one from 2011 . . . well, you get the idea. And here’s a quick summary posted by Rolf Zwaan after Brown et al. came out with their paper. I know Sokal and Brown and so this story was not news to me. I didn’t post anything about it on this blog because it seemed like it was getting enough coverage elsewhere. I think Ni
2 0.19495292 967 andrew gelman stats-2011-10-20-Picking on Gregg Easterbrook
Introduction: I don’t want to make a habit of this, but . . . I was curious what Easterbrook would write as a follow-up to his recent Huntsman puff, and here’s what he came up with: Tired of cookie-cutter political contests between hauntingly similar candidates? Then you’re going to like the upcoming race for one of the Senate seats in the late Ted Kennedy’s haunting grounds. Elizabeth Warren, best known for creating and fighting for the Consumer Financial Protection Bureau, is hoping to challenge Republican incumbent Scott Brown. They’re both qualified, but they couldn’t be more different — personally or politically. Um, no. 1. It seems a bit of a stretch to say the two candidates “couldn’t be more different personally.” Brown is a 52-year-old married white lawyer with two children. Warren is a 62-year-old married white lawyer with three children. According to Wikipedia, they both had middle-class backgrounds, Brown in Massachusetts and Warren in Oklahoma, and they suffered some per
3 0.17856659 928 andrew gelman stats-2011-09-27-Hey, look over here! Another rant!
Introduction: Bigshot establishment dude Peter Orszag thinks bigshot establishment dudes don’t have enough power. (Also politically related but not a rant: Joe McCarthy Versus Powerman and the Debt-Ceiling Destroyers, Part One. )
Introduction: The other day we discussed that paper on ovulation and voting (you may recall that the authors reported a scattered bunch of comparisons, significance tests, and p-values, and I recommended that they would’ve done better to simply report complete summaries of their data, so that readers could see the comparisons of interest in full context), and I was thinking a bit more about why I was so bothered that it was published in Psychological Science, which I’d thought of as a serious research journal. My concern isn’t just that that the paper is bad—after all, lots of bad papers get published—but rather that it had nothing really going for it, except that it was headline bait. It was a survey done on Mechanical Turk, that’s it. No clever design, no clever questions, no care in dealing with nonresponse problems, no innovative data analysis, no nothing. The paper had nothing to offer, except that it had no obvious flaws. Psychology is a huge field full of brilliant researchers.
Introduction: Jeff Leek points to a post by Alex Holcombe, who disputes the idea that science is self-correcting. Holcombe writes [scroll down to get to his part]: The pace of scientific production has quickened, and self-correction has suffered. Findings that might correct old results are considered less interesting than results from more original research questions. Potential corrections are also more contested. As the competition for space in prestigious journals has become increasingly frenzied, doing and publishing studies that would confirm the rapidly accumulating new discoveries, or would correct them, became a losing proposition. Holcombe picks up on some points that we’ve discussed a lot here in the past year. Here’s Holcombe: In certain subfields, almost all new work appears in only a very few journals, all associated with a single professional society. There is then no way around the senior gatekeepers, who may then suppress corrections with impunity. . . . The bias agai
6 0.14968373 2353 andrew gelman stats-2014-05-30-I posted this as a comment on a sociology blog
7 0.14453727 2245 andrew gelman stats-2014-03-12-More on publishing in journals
10 0.14075091 1139 andrew gelman stats-2012-01-26-Suggested resolution of the Bem paradox
11 0.140471 2137 andrew gelman stats-2013-12-17-Replication backlash
12 0.13184869 1435 andrew gelman stats-2012-07-30-Retracted articles and unethical behavior in economics journals?
13 0.1315379 400 andrew gelman stats-2010-11-08-Poli sci plagiarism update, and a note about the benefits of not caring
14 0.13138454 1928 andrew gelman stats-2013-07-06-How to think about papers published in low-grade journals?
15 0.1291385 1727 andrew gelman stats-2013-02-19-Beef with data
16 0.12777913 2244 andrew gelman stats-2014-03-11-What if I were to stop publishing in journals?
17 0.12689263 1690 andrew gelman stats-2013-01-23-When are complicated models helpful in psychology research and when are they overkill?
18 0.12067336 1878 andrew gelman stats-2013-05-31-How to fix the tabloids? Toward replicable social science research
19 0.11935019 44 andrew gelman stats-2010-05-20-Boris was right
20 0.11918664 1844 andrew gelman stats-2013-05-06-Against optimism about social science
topicId topicWeight
[(0, 0.215), (1, -0.112), (2, -0.053), (3, -0.062), (4, -0.064), (5, -0.075), (6, 0.062), (7, -0.064), (8, -0.001), (9, -0.008), (10, 0.052), (11, 0.012), (12, -0.059), (13, 0.009), (14, 0.002), (15, -0.013), (16, 0.006), (17, 0.014), (18, 0.029), (19, -0.024), (20, -0.022), (21, 0.005), (22, -0.006), (23, -0.034), (24, 0.004), (25, 0.014), (26, -0.046), (27, 0.003), (28, -0.014), (29, 0.002), (30, 0.014), (31, -0.001), (32, 0.005), (33, -0.003), (34, -0.018), (35, 0.002), (36, -0.013), (37, 0.005), (38, -0.017), (39, -0.014), (40, 0.006), (41, 0.008), (42, 0.008), (43, 0.015), (44, 0.005), (45, 0.033), (46, 0.002), (47, 0.011), (48, -0.014), (49, -0.011)]
simIndex simValue blogId blogTitle
same-blog 1 0.9752664 2177 andrew gelman stats-2014-01-19-“The British amateur who debunked the mathematics of happiness”
Introduction: Andrew Anthony tells the excellent story of how Nick Brown, Alan Sokal, and Harris Friedman shot down some particularly silly work in psychology. (“According to the graph, it all came down to a specific ratio of positive emotions to negative emotions. If your ratio was greater than 2.9013 positive emotions to 1 negative emotion you were flourishing in life. If your ratio was less than that number you were languishing.” And, yes, the work they were shooting down really is that bad.) If you want to see what the fuss is about, just google “2.9013.” Here’s an example (from 2012) of an uncritical reporting of the claim, here’s another one from 2010, here’s one from 2011 . . . well, you get the idea. And here’s a quick summary posted by Rolf Zwaan after Brown et al. came out with their paper. I know Sokal and Brown and so this story was not news to me. I didn’t post anything about it on this blog because it seemed like it was getting enough coverage elsewhere. I think Ni
2 0.89624697 1435 andrew gelman stats-2012-07-30-Retracted articles and unethical behavior in economics journals?
Introduction: Stan Liebowitz writes: Have you ever heard of an article being retracted in economics? I know you have only been doing this for a few years but I suspect that the answer is that none or very few are retracted. No economist would ever deceive another. There is virtually no interest in detecting cheating. And what good would that do if there is no form of punishment? I say this because I think I have found a case in one of our top journals but the editor allowed the authors of the original article to write an anonymous referee report defending themselves and used this report to reject my comment even though an independent referee recommended publication. My reply: I wonder how this sort of thing will change in the future as journals become less important. My impression is that, on one side, researchers are increasingly citing NBER reports, Arxiv preprints, and the like; while, from the other direction, journals such as Science and Nature are developing the reputations of being “t
Introduction: Jeff Leek points to a post by Alex Holcombe, who disputes the idea that science is self-correcting. Holcombe writes [scroll down to get to his part]: The pace of scientific production has quickened, and self-correction has suffered. Findings that might correct old results are considered less interesting than results from more original research questions. Potential corrections are also more contested. As the competition for space in prestigious journals has become increasingly frenzied, doing and publishing studies that would confirm the rapidly accumulating new discoveries, or would correct them, became a losing proposition. Holcombe picks up on some points that we’ve discussed a lot here in the past year. Here’s Holcombe: In certain subfields, almost all new work appears in only a very few journals, all associated with a single professional society. There is then no way around the senior gatekeepers, who may then suppress corrections with impunity. . . . The bias agai
4 0.87944728 2233 andrew gelman stats-2014-03-04-Literal vs. rhetorical
Introduction: Thomas Basbøll pointed me to a discussion on the orgtheory blog in which Jerry Davis, the editor of a journal of business management argued that it is difficult for academic researchers to communicate with the public because “the public prefers Cheetos to a healthy salad” and when serious papers are discussed on the internet, “everyone is a methodologist.” The discussion heated up when an actual methodologist, Steve Morgan, joined in to argue that the salad in question was not so healthy and that the much-derided internet commenters made some valuable points. The final twist was that one of the orgtheory bloggers deleted a comment and then closed the thread entirely when the discussion got too conflictual. In a few days I’ll return to the meta-topic of the discussion, but right now I want to focus on one thing Davis wrote, a particular statement that illustrates to me the gap between the rhetorical and the literal, the way in which a statement can sound good but make no sense. He
5 0.87728077 1901 andrew gelman stats-2013-06-16-Evilicious: Why We Evolved a Taste for Being Bad
Introduction: The other day, a friend told me that when he saw me blogging on Noam Chomsky, he was surprised not to see any mention of disgraced primatologist Marc Hauser. I was like, whaaaaaa? I had no idea these two had any connection. In fact, though, they wrote papers together. This made me wonder what Chomsky thought of Hauser’s data scandal. I googled *marc hauser noam chomsky* and the first item that came up was this, from July 2011, reported by Tom Bartlett: I [Bartlett] asked Chomsky for his comment on the Hauser resignation and he e-mailed the following: Mark Hauser is a fine scientist with an outstanding record of accomplishment. His resignation is a serious loss for Harvard, and given the nature of the attack on him, for science generally. Chomsky is a mentor of Hauser so I can’t fault Chomsky for defending the guy. But why couldn’t he have stuck with something more general, something like, “I respect and admire Mark Hauser and am not aware of any improprieties in his w
6 0.87544543 1139 andrew gelman stats-2012-01-26-Suggested resolution of the Bem paradox
7 0.87536126 2269 andrew gelman stats-2014-03-27-Beyond the Valley of the Trolls
8 0.85970104 2353 andrew gelman stats-2014-05-30-I posted this as a comment on a sociology blog
9 0.85815436 834 andrew gelman stats-2011-08-01-I owe it all to the haters
10 0.85378969 675 andrew gelman stats-2011-04-22-Arrow’s other theorem
11 0.84621596 883 andrew gelman stats-2011-09-01-Arrow’s theorem update
12 0.84564215 2245 andrew gelman stats-2014-03-12-More on publishing in journals
13 0.84260029 2137 andrew gelman stats-2013-12-17-Replication backlash
15 0.83879179 902 andrew gelman stats-2011-09-12-The importance of style in academic writing
18 0.83208543 2006 andrew gelman stats-2013-09-03-Evaluating evidence from published research
19 0.83187175 1844 andrew gelman stats-2013-05-06-Against optimism about social science
20 0.83138525 1683 andrew gelman stats-2013-01-19-“Confirmation, on the other hand, is not sexy”
topicId topicWeight
[(2, 0.014), (15, 0.091), (16, 0.116), (18, 0.015), (21, 0.031), (22, 0.012), (24, 0.148), (27, 0.082), (48, 0.01), (55, 0.02), (63, 0.029), (84, 0.012), (86, 0.037), (93, 0.02), (95, 0.028), (99, 0.237)]
simIndex simValue blogId blogTitle
same-blog 1 0.9525485 2177 andrew gelman stats-2014-01-19-“The British amateur who debunked the mathematics of happiness”
Introduction: Andrew Anthony tells the excellent story of how Nick Brown, Alan Sokal, and Harris Friedman shot down some particularly silly work in psychology. (“According to the graph, it all came down to a specific ratio of positive emotions to negative emotions. If your ratio was greater than 2.9013 positive emotions to 1 negative emotion you were flourishing in life. If your ratio was less than that number you were languishing.” And, yes, the work they were shooting down really is that bad.) If you want to see what the fuss is about, just google “2.9013.” Here’s an example (from 2012) of an uncritical reporting of the claim, here’s another one from 2010, here’s one from 2011 . . . well, you get the idea. And here’s a quick summary posted by Rolf Zwaan after Brown et al. came out with their paper. I know Sokal and Brown and so this story was not news to me. I didn’t post anything about it on this blog because it seemed like it was getting enough coverage elsewhere. I think Ni
2 0.94577992 2277 andrew gelman stats-2014-03-31-The most-cited statistics papers ever
Introduction: Robert Grant has a list . I’ll just give the ones with more than 10,000 Google Scholar cites: Cox (1972) Regression and life tables: 35,512 citations. Dempster, Laird, Rubin (1977) Maximum likelihood from incomplete data via the EM algorithm: 34,988 Bland & Altman (1986) Statistical methods for assessing agreement between two methods of clinical measurement: 27,181 Geman & Geman (1984) Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images: 15,106 We can find some more via searching Google scholar for familiar names and topics; thus: Metropolis et al. (1953) Equation of state calculations by fast computing machines: 26,000 Benjamini and Hochberg (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing: 21,000 White (1980) A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity: 18,000 Heckman (1977) Sample selection bias as a specification error:
3 0.94189608 120 andrew gelman stats-2010-06-30-You can’t put Pandora back in the box
Introduction: Rajiv Sethi writes : I suspect that within a decade, blogs will be a cornerstone of research in economics. Many original and creative contributions to the discipline will first be communicated to the profession (and the world at large) in the form of blog posts, since the medium allows for material of arbitrary length, depth and complexity. Ideas first expressed in this form will make their way (with suitable attribution) into reading lists, doctoral dissertations and more conventionally refereed academic publications. And blogs will come to play a central role in the process of recruitment, promotion and reward at major research universities. This genie is not going back into its bottle. And he thinks this is a good thing: In fact, the refereeing process for blog posts is in some respects more rigorous than that for journal articles. Reports are numerous, non-anonymous, public, rapidly and efficiently produced, and collaboratively constructed. It is not obvious to me [Sethi]
4 0.93072844 1238 andrew gelman stats-2012-03-31-Dispute about ethics of data sharing
Introduction: Several months ago, Sam Behseta, the new editor of Chance magazine, asked me if I’d like to have a column. I said yes, I’d like to write on ethics and statistics. My first column was called “Open Data and Open Methods” and I discussed the ethical obligation to share data and make our computations transparent wherever possible. In my column, I recounted a story from a bit over 20 years ago when I noticed a problem in a published analysis (involving electromagnetic fields and calcium flow in chicken brains) and contacted the researcher in charge of the study, who would not share his data with me. Two of the people from that research team—biologist Carl Blackman and statistician Dennis House—saw my Chance column and felt that I had misrepresented the situation and had criticized them unfairly. Blackman and House expressed their concerns in letters to the editor which were just published, along with my reply, in the latest issue of Chance . Seeing as I posted my article here, I
5 0.92851037 2179 andrew gelman stats-2014-01-20-The AAA Tranche of Subprime Science
Introduction: In our new ethics column for Chance , Eric Loken and I write about our current favorite topic: One of our ongoing themes when discussing scientific ethics is the central role of statistics in recognizing and communicating uncer- tainty. Unfortunately, statistics—and the scientific process more generally—often seems to be used more as a way of laundering uncertainty, processing data until researchers and consumers of research can feel safe acting as if various scientific hypotheses are unquestionably true. . . . We have in mind an analogy with the notorious AAA-class bonds created during the mid-2000s that led to the subprime mortgage crisis. Lower-quality mortgages—that is, mortgages with high probability of default and, thus, high uncertainty—were packaged and transformed into financial instruments that were (in retrospect, falsely) characterized as low risk. There was a tremendous interest in these securities, not just among the most unscrupulous market manipulators, but in a
6 0.92696321 804 andrew gelman stats-2011-07-15-Static sensitivity analysis
7 0.92518127 2353 andrew gelman stats-2014-05-30-I posted this as a comment on a sociology blog
8 0.92424893 481 andrew gelman stats-2010-12-22-The Jumpstart financial literacy survey and the different purposes of tests
9 0.92186028 343 andrew gelman stats-2010-10-15-?
10 0.92162424 1959 andrew gelman stats-2013-07-28-50 shades of gray: A research story
11 0.92003334 902 andrew gelman stats-2011-09-12-The importance of style in academic writing
12 0.91983867 586 andrew gelman stats-2011-02-23-A statistical version of Arrow’s paradox
13 0.91925061 2244 andrew gelman stats-2014-03-11-What if I were to stop publishing in journals?
15 0.91861564 1774 andrew gelman stats-2013-03-22-Likelihood Ratio ≠ 1 Journal
16 0.91761732 134 andrew gelman stats-2010-07-08-“What do you think about curved lines connecting discrete data-points?”
17 0.91751611 2248 andrew gelman stats-2014-03-15-Problematic interpretations of confidence intervals
19 0.91690826 2227 andrew gelman stats-2014-02-27-“What Can we Learn from the Many Labs Replication Project?”
20 0.91593242 2004 andrew gelman stats-2013-09-01-Post-publication peer review: How it (sometimes) really works