andrew_gelman_stats andrew_gelman_stats-2013 andrew_gelman_stats-2013-2014 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Alison George interviews Elizabeth Loftus. It’s good stuff, important stuff, and it relates to my view as a statistician that uncertainty and variation are important. Uncertainty is relevant here because there are things that people remember that never happened. Variation is important because different people remember different things. Loftus’s work also seems relevant to the problems with pseudoscience that we’ve been discussing recently on the blog, studies where researchers follow the forms of scientific reasoning and publish in scientific journals, but what they are publishing is essentially unreplicable noise. Perhaps there’s some connection between all those people Loftus has interviewed, who remember events that never happened to them, and people such as Daryl Bem, who think they’ve computed rigorous p-values even though their analyses seem so clearly to be contingent on the data. It’s almost like a false memory, that scientists convince themselves that the analysis the
sentIndex sentText sentNum sentScore
1 It’s good stuff, important stuff, and it relates to my view as a statistician that uncertainty and variation are important. [sent-2, score-0.708]
2 Uncertainty is relevant here because there are things that people remember that never happened. [sent-3, score-0.633]
3 Variation is important because different people remember different things. [sent-4, score-0.586]
4 Loftus’s work also seems relevant to the problems with pseudoscience that we’ve been discussing recently on the blog, studies where researchers follow the forms of scientific reasoning and publish in scientific journals, but what they are publishing is essentially unreplicable noise. [sent-5, score-1.441]
5 Perhaps there’s some connection between all those people Loftus has interviewed, who remember events that never happened to them, and people such as Daryl Bem, who think they’ve computed rigorous p-values even though their analyses seem so clearly to be contingent on the data. [sent-6, score-1.5]
6 It’s almost like a false memory, that scientists convince themselves that the analysis they happened to do after the data arrived, was exactly the analysis they’d planned to do ahead of time. [sent-7, score-0.765]
7 Unfortunately there never seems to be a pre-data record of such a plan, but the memory is there. [sent-8, score-0.582]
wordName wordTfidf (topN-words)
[('loftus', 0.418), ('memory', 0.249), ('remember', 0.235), ('pseudoscience', 0.19), ('alison', 0.179), ('variation', 0.171), ('uncertainty', 0.167), ('happened', 0.156), ('never', 0.155), ('arrived', 0.15), ('unreplicable', 0.15), ('contingent', 0.147), ('stuff', 0.147), ('elizabeth', 0.144), ('relevant', 0.139), ('convince', 0.134), ('daryl', 0.131), ('interviewed', 0.128), ('scientific', 0.127), ('interviews', 0.127), ('planned', 0.125), ('computed', 0.123), ('relates', 0.122), ('bem', 0.121), ('rigorous', 0.12), ('forms', 0.109), ('people', 0.104), ('events', 0.101), ('record', 0.1), ('ahead', 0.098), ('connection', 0.097), ('important', 0.097), ('discussing', 0.096), ('george', 0.096), ('publishing', 0.093), ('plan', 0.091), ('reasoning', 0.091), ('false', 0.088), ('journals', 0.086), ('unfortunately', 0.085), ('essentially', 0.084), ('analyses', 0.082), ('publish', 0.082), ('analysis', 0.082), ('seems', 0.078), ('view', 0.076), ('clearly', 0.076), ('different', 0.075), ('statistician', 0.075), ('follow', 0.075)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 2014 andrew gelman stats-2013-09-09-False memories and statistical analysis
Introduction: Alison George interviews Elizabeth Loftus. It’s good stuff, important stuff, and it relates to my view as a statistician that uncertainty and variation are important. Uncertainty is relevant here because there are things that people remember that never happened. Variation is important because different people remember different things. Loftus’s work also seems relevant to the problems with pseudoscience that we’ve been discussing recently on the blog, studies where researchers follow the forms of scientific reasoning and publish in scientific journals, but what they are publishing is essentially unreplicable noise. Perhaps there’s some connection between all those people Loftus has interviewed, who remember events that never happened to them, and people such as Daryl Bem, who think they’ve computed rigorous p-values even though their analyses seem so clearly to be contingent on the data. It’s almost like a false memory, that scientists convince themselves that the analysis the
2 0.14713715 6 andrew gelman stats-2010-04-27-Jelte Wicherts lays down the stats on IQ
Introduction: Good stuff.
3 0.13737984 434 andrew gelman stats-2010-11-28-When Small Numbers Lead to Big Errors
Introduction: My column in Scientific American . Check out the comments. I have to remember never ever to write about guns.
4 0.12236651 1998 andrew gelman stats-2013-08-25-A new Bem theory
Introduction: The other day I was talking with someone who knows Daryl Bem a bit, and he was sharing his thoughts on that notorious ESP paper that was published in a leading journal in the field but then was mocked, shot down, and was repeatedly replicated with no success. My friend said that overall the Bem paper had positive effects in forcing psychologists to think more carefully about what sorts of research results should or should not be published in top journals, the role of replications, and other things. I expressed agreement and shared my thought that, at some level, I don’t think Bem himself fully believes his ESP effects are real. Why do I say this? Because he seemed oddly content to publish results that were not quite conclusive. He ran a bunch of experiments, looked at the data, and computed some post-hoc p-values in the .01 to .05 range. If he really were confident that the phenomenon was real (that is, that the results would apply to new data), then he could’ve easily run the
5 0.11977392 1139 andrew gelman stats-2012-01-26-Suggested resolution of the Bem paradox
Introduction: There has been an increasing discussion about the proliferation of flawed research in psychology and medicine, with some landmark events being John Ioannides’s article , “Why most published research findings are false” (according to Google Scholar, cited 973 times since its appearance in 2005), the scandals of Marc Hauser and Diederik Stapel, two leading psychology professors who resigned after disclosures of scientific misconduct, and Daryl Bem’s dubious recent paper on ESP, published to much fanfare in Journal of Personality and Social Psychology, one of the top journals in the field. Alongside all this are the plagiarism scandals, which are uninteresting from a scientific context but are relevant in that, in many cases, neither the institutions housing the plagiarists nor the editors and publishers of the plagiarized material seem to care. Perhaps these universities and publishers are more worried about bad publicity (and maybe lawsuits, given that many of the plagiarism cas
6 0.099943057 1122 andrew gelman stats-2012-01-16-“Groundbreaking or Definitive? Journals Need to Pick One”
7 0.091234267 1844 andrew gelman stats-2013-05-06-Against optimism about social science
8 0.090329498 2179 andrew gelman stats-2014-01-20-The AAA Tranche of Subprime Science
9 0.090142876 762 andrew gelman stats-2011-06-13-How should journals handle replication studies?
10 0.08910197 51 andrew gelman stats-2010-05-26-If statistics is so significantly great, why don’t statisticians use statistics?
11 0.088091753 1435 andrew gelman stats-2012-07-30-Retracted articles and unethical behavior in economics journals?
12 0.086043283 734 andrew gelman stats-2011-05-28-Funniest comment ever
13 0.086011365 2013 andrew gelman stats-2013-09-08-What we need here is some peer review for statistical graphics
14 0.084966846 2245 andrew gelman stats-2014-03-12-More on publishing in journals
15 0.084622025 1878 andrew gelman stats-2013-05-31-How to fix the tabloids? Toward replicable social science research
17 0.078298382 72 andrew gelman stats-2010-06-07-Valencia: Summer of 1991
18 0.074781477 1666 andrew gelman stats-2013-01-10-They’d rather be rigorous than right
20 0.073523253 371 andrew gelman stats-2010-10-26-Musical chairs in econ journals
topicId topicWeight
[(0, 0.138), (1, -0.042), (2, -0.049), (3, -0.064), (4, -0.049), (5, -0.039), (6, 0.009), (7, -0.018), (8, 0.025), (9, -0.009), (10, 0.011), (11, 0.024), (12, -0.025), (13, -0.028), (14, -0.02), (15, -0.021), (16, -0.011), (17, -0.001), (18, 0.007), (19, -0.001), (20, 0.003), (21, 0.003), (22, -0.007), (23, 0.021), (24, -0.042), (25, -0.008), (26, -0.029), (27, 0.015), (28, 0.008), (29, 0.003), (30, 0.038), (31, 0.034), (32, 0.02), (33, -0.029), (34, 0.007), (35, 0.0), (36, 0.046), (37, 0.019), (38, -0.008), (39, -0.019), (40, 0.001), (41, -0.017), (42, 0.002), (43, 0.006), (44, 0.07), (45, 0.018), (46, -0.025), (47, 0.011), (48, 0.038), (49, -0.034)]
simIndex simValue blogId blogTitle
same-blog 1 0.95396101 2014 andrew gelman stats-2013-09-09-False memories and statistical analysis
Introduction: Alison George interviews Elizabeth Loftus. It’s good stuff, important stuff, and it relates to my view as a statistician that uncertainty and variation are important. Uncertainty is relevant here because there are things that people remember that never happened. Variation is important because different people remember different things. Loftus’s work also seems relevant to the problems with pseudoscience that we’ve been discussing recently on the blog, studies where researchers follow the forms of scientific reasoning and publish in scientific journals, but what they are publishing is essentially unreplicable noise. Perhaps there’s some connection between all those people Loftus has interviewed, who remember events that never happened to them, and people such as Daryl Bem, who think they’ve computed rigorous p-values even though their analyses seem so clearly to be contingent on the data. It’s almost like a false memory, that scientists convince themselves that the analysis the
2 0.78039515 2179 andrew gelman stats-2014-01-20-The AAA Tranche of Subprime Science
Introduction: In our new ethics column for Chance , Eric Loken and I write about our current favorite topic: One of our ongoing themes when discussing scientific ethics is the central role of statistics in recognizing and communicating uncer- tainty. Unfortunately, statistics—and the scientific process more generally—often seems to be used more as a way of laundering uncertainty, processing data until researchers and consumers of research can feel safe acting as if various scientific hypotheses are unquestionably true. . . . We have in mind an analogy with the notorious AAA-class bonds created during the mid-2000s that led to the subprime mortgage crisis. Lower-quality mortgages—that is, mortgages with high probability of default and, thus, high uncertainty—were packaged and transformed into financial instruments that were (in retrospect, falsely) characterized as low risk. There was a tremendous interest in these securities, not just among the most unscrupulous market manipulators, but in a
3 0.76614219 2006 andrew gelman stats-2013-09-03-Evaluating evidence from published research
Introduction: Following up on my entry the other day on post-publication peer review, Dan Kahan writes: You give me credit, I think, for merely participating in what I think is a systemic effect in the practice of empirical inquiry that conduces to quality control & hence the advance of knowledge by such means (likely the title conveys that!). I’d say: (a) by far the greatest weakness in the “publication regime” in social sciences today is the systematic disregard for basic principles of valid causal inference, a deficiency either in comprehension or craft that is at the root of scholars’ resort to (and journals’ tolerance for) invalid samples, the employment of designs that don’t generate observations more consistent with a hypothesis than with myriad rival ones, and the resort to deficient statistical modes of analysis that treat detection of “statististically significant difference” rather than “practical corroboration of practical meaningful effect” as the goal of such analysis (especial
Introduction: I had a brief email exchange with Jeff Leek regarding our recent discussions of replication, criticism, and the self-correcting process of science. Jeff writes: (1) I can see the problem with serious, evidence-based criticisms not being published in the same journal (and linked to) studies that are shown to be incorrect. I have been mostly seeing these sorts of things show up in blogs. But I’m not sure that is a bad thing. I think people read blogs more than they read the literature. I wonder if this means that blogs will eventually be a sort of “shadow literature”? (2) I think there is a ton of bad literature out there, just like there is a ton of bad stuff on Google. If we focus too much on the bad stuff we will be paralyzed. I still manage to find good papers despite all the bad papers. (3) I think one positive solution to this problem is to incentivize/publish referee reports and give people credit for a good referee report just like they get credit for a good paper. T
Introduction: Jeff Leek points to a post by Alex Holcombe, who disputes the idea that science is self-correcting. Holcombe writes [scroll down to get to his part]: The pace of scientific production has quickened, and self-correction has suffered. Findings that might correct old results are considered less interesting than results from more original research questions. Potential corrections are also more contested. As the competition for space in prestigious journals has become increasingly frenzied, doing and publishing studies that would confirm the rapidly accumulating new discoveries, or would correct them, became a losing proposition. Holcombe picks up on some points that we’ve discussed a lot here in the past year. Here’s Holcombe: In certain subfields, almost all new work appears in only a very few journals, all associated with a single professional society. There is then no way around the senior gatekeepers, who may then suppress corrections with impunity. . . . The bias agai
6 0.74872118 1139 andrew gelman stats-2012-01-26-Suggested resolution of the Bem paradox
7 0.74622047 1760 andrew gelman stats-2013-03-12-Misunderstanding the p-value
8 0.74584097 2177 andrew gelman stats-2014-01-19-“The British amateur who debunked the mathematics of happiness”
9 0.73646533 1683 andrew gelman stats-2013-01-19-“Confirmation, on the other hand, is not sexy”
10 0.73075008 2137 andrew gelman stats-2013-12-17-Replication backlash
11 0.72892046 1844 andrew gelman stats-2013-05-06-Against optimism about social science
13 0.7150172 1448 andrew gelman stats-2012-08-07-Scientific fraud, double standards and institutions protecting themselves
14 0.71493345 1435 andrew gelman stats-2012-07-30-Retracted articles and unethical behavior in economics journals?
15 0.71489102 675 andrew gelman stats-2011-04-22-Arrow’s other theorem
16 0.71074164 1861 andrew gelman stats-2013-05-17-Where do theories come from?
18 0.69965029 360 andrew gelman stats-2010-10-21-Forensic bioinformatics, or, Don’t believe everything you read in the (scientific) papers
19 0.69686991 1281 andrew gelman stats-2012-04-25-Dyson’s baffling love of crackpots
20 0.69419652 1272 andrew gelman stats-2012-04-20-More proposals to reform the peer-review system
topicId topicWeight
[(2, 0.068), (10, 0.07), (15, 0.173), (16, 0.059), (19, 0.018), (24, 0.105), (32, 0.043), (44, 0.023), (55, 0.016), (86, 0.012), (99, 0.304)]
simIndex simValue blogId blogTitle
Introduction: After noticing these remarks on expensive textbooks and this comment on the company that bribes professors to use their books, Preston McAfee pointed me to this update (complete with a picture of some guy who keeps threatening to sue him but never gets around to it). The story McAfee tells is sad but also hilarious. Especially the part about “smuck.” It all looks like one more symptom of the imploding market for books. Prices for intro stat and econ books go up and up (even mediocre textbooks routinely cost $150), and the publishers put more and more effort into promotion. McAfee adds: I [McAfee] hope a publisher sues me about posting the articles I wrote. Even a takedown notice would be fun. I would be pretty happy to start posting about that, especially when some of them are charging $30 per article. Ted Bergstrom and I used state Freedom of Information acts to extract the journal price deals at state university libraries. We have about 35 of them so far. Like te
same-blog 2 0.95422286 2014 andrew gelman stats-2013-09-09-False memories and statistical analysis
Introduction: Alison George interviews Elizabeth Loftus. It’s good stuff, important stuff, and it relates to my view as a statistician that uncertainty and variation are important. Uncertainty is relevant here because there are things that people remember that never happened. Variation is important because different people remember different things. Loftus’s work also seems relevant to the problems with pseudoscience that we’ve been discussing recently on the blog, studies where researchers follow the forms of scientific reasoning and publish in scientific journals, but what they are publishing is essentially unreplicable noise. Perhaps there’s some connection between all those people Loftus has interviewed, who remember events that never happened to them, and people such as Daryl Bem, who think they’ve computed rigorous p-values even though their analyses seem so clearly to be contingent on the data. It’s almost like a false memory, that scientists convince themselves that the analysis the
3 0.9535265 133 andrew gelman stats-2010-07-08-Gratuitous use of “Bayesian Statistics,” a branding issue?
Introduction: I’m on an island in Maine for a few weeks (big shout out for North Haven!) This morning I picked up a copy of “Working Waterfront,” a newspaper that focuses on issues of coastal fishing communities. I came across an article about modeling “fish” populations — actually lobsters, I guess they’re considered “fish” for regulatory purposes. When I read it, I thought “wow, this article is really well-written, not dumbed down like articles in most newspapers.” I think it’s great that a small coastal newspaper carries reporting like this. (The online version has a few things that I don’t recall in the print version, too, so it’s even better). But in addition to being struck by finding such a good article in a small newspaper, I was struck by this: According to [University of Maine scientist Yong] Chen, there are four main areas where his model improved on the prior version. “We included the inshore trawl data from Maine and other state surveys, in addition to federal survey data; we h
4 0.95046443 1541 andrew gelman stats-2012-10-19-Statistical discrimination again
Introduction: Mark Johnstone writes: I’ve recently been investigating a new European Court of Justice ruling on insurance calculations (on behalf of MoneySuperMarket) and I found something related to statistics that caught my attention. . . . The ruling (which comes into effect in December 2012) states that insurers in Europe can no longer provide different premiums based on gender. Despite the fact that women are statistically safer drivers, unless it’s biologically proven there is a causal relationship between being female and being a safer driver, this is now seen as an act of discrimination (more on this from the Wall Street Journal). However, where do you stop with this? What about age? What about other factors? And what does this mean for the application of statistics in general? Is it inherently unjust in this context? One proposal has been to fit ‘black boxes’ into cars so more individual data can be collected, as opposed to relying heavily on aggregates. For fans of data and s
5 0.95003188 834 andrew gelman stats-2011-08-01-I owe it all to the haters
Introduction: Sometimes when I submit an article to a journal it is accepted right away or with minor alterations. But many of my favorite articles were rejected or had to go through an exhausting series of revisions. For example, this influential article had a very hostile referee and we had to seriously push the journal editor to accept it. This one was rejected by one or two journals before finally appearing with discussion. This paper was rejected by the American Political Science Review with no chance of revision and we had to publish it in the British Journal of Political Science, which was a bit odd given that the article was 100% about American politics. And when I submitted this instant classic (actually at the invitation of the editor), the referees found it to be trivial, and the editor did me the favor of publishing it but only by officially labeling it as a discussion of another article that appeared in the same issue. Some of my most influential papers were accepted right
6 0.94851893 1908 andrew gelman stats-2013-06-21-Interpreting interactions in discrete-data regression
7 0.94751507 1624 andrew gelman stats-2012-12-15-New prize on causality in statstistics education
8 0.94490409 1794 andrew gelman stats-2013-04-09-My talks in DC and Baltimore this week
9 0.94411069 945 andrew gelman stats-2011-10-06-W’man < W’pedia, again
10 0.94214249 1833 andrew gelman stats-2013-04-30-“Tragedy of the science-communication commons”
11 0.93692917 1998 andrew gelman stats-2013-08-25-A new Bem theory
12 0.93491924 1081 andrew gelman stats-2011-12-24-Statistical ethics violation
13 0.9336217 1779 andrew gelman stats-2013-03-27-“Two Dogmas of Strong Objective Bayesianism”
14 0.93309677 908 andrew gelman stats-2011-09-14-Type M errors in the lab
16 0.92889023 2278 andrew gelman stats-2014-04-01-Association for Psychological Science announces a new journal
17 0.92773569 1393 andrew gelman stats-2012-06-26-The reverse-journal-submission system
18 0.92767787 1954 andrew gelman stats-2013-07-24-Too Good To Be True: The Scientific Mass Production of Spurious Statistical Significance
19 0.92714119 1122 andrew gelman stats-2012-01-16-“Groundbreaking or Definitive? Journals Need to Pick One”
20 0.92652011 2215 andrew gelman stats-2014-02-17-The Washington Post reprints university press releases without editing them