andrew_gelman_stats andrew_gelman_stats-2010 andrew_gelman_stats-2010-302 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Somebody I know sent me a link to this news article by Martin Robbins describing a potential scientific breakthrough. I express some skepticism but in a vague enough way that, in the unlikely event that the research claim turns out to be correct, there’s no paper trail showing that I was wrong. I have some comments on the graphs–the tables are horrible, no need to even discuss them!–and I’d prefer if the authors of the paper could display their data and model on a single graph. I realize that their results reached a standard level of statistical significance, but it’s hard for me to interpret their claims until I see their estimates on some sort of direct real-world scale. In any case, though, I’m sure these researchers are working hard, and I wish them the best of luck in their future efforts to replicate their findings. I’m sure they’ll have no problem replicating, whether or not their claims are actually true. That’s the way science works: Once you know what you’re looking
sentIndex sentText sentNum sentScore
1 Somebody I know sent me a link to this news article by Martin Robbins describing a potential scientific breakthrough. [sent-1, score-0.707]
2 I express some skepticism but in a vague enough way that, in the unlikely event that the research claim turns out to be correct, there’s no paper trail showing that I was wrong. [sent-2, score-1.439]
3 I have some comments on the graphs–the tables are horrible, no need to even discuss them! [sent-3, score-0.331]
4 –and I’d prefer if the authors of the paper could display their data and model on a single graph. [sent-4, score-0.546]
5 I realize that their results reached a standard level of statistical significance, but it’s hard for me to interpret their claims until I see their estimates on some sort of direct real-world scale. [sent-5, score-1.148]
6 In any case, though, I’m sure these researchers are working hard, and I wish them the best of luck in their future efforts to replicate their findings. [sent-6, score-0.87]
7 I’m sure they’ll have no problem replicating, whether or not their claims are actually true. [sent-7, score-0.321]
8 That’s the way science works: Once you know what you’re looking for, you’ll find it! [sent-8, score-0.17]
wordName wordTfidf (topN-words)
[('robbins', 0.264), ('trail', 0.213), ('replicating', 0.204), ('claims', 0.196), ('luck', 0.187), ('vague', 0.172), ('martin', 0.169), ('reached', 0.167), ('replicate', 0.161), ('unlikely', 0.161), ('hard', 0.152), ('skepticism', 0.152), ('wish', 0.146), ('describing', 0.146), ('tables', 0.146), ('event', 0.143), ('efforts', 0.142), ('horrible', 0.141), ('express', 0.139), ('interpret', 0.137), ('turns', 0.136), ('display', 0.134), ('somebody', 0.13), ('sure', 0.125), ('realize', 0.124), ('showing', 0.121), ('ll', 0.12), ('significance', 0.117), ('direct', 0.112), ('paper', 0.11), ('future', 0.109), ('prefer', 0.107), ('works', 0.107), ('potential', 0.106), ('correct', 0.104), ('sent', 0.103), ('discuss', 0.101), ('authors', 0.098), ('graphs', 0.097), ('single', 0.097), ('news', 0.094), ('claim', 0.092), ('estimates', 0.091), ('scientific', 0.088), ('level', 0.086), ('looking', 0.085), ('link', 0.085), ('know', 0.085), ('comments', 0.084), ('standard', 0.083)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 302 andrew gelman stats-2010-09-28-This is a link to a news article about a scientific paper
Introduction: Somebody I know sent me a link to this news article by Martin Robbins describing a potential scientific breakthrough. I express some skepticism but in a vague enough way that, in the unlikely event that the research claim turns out to be correct, there’s no paper trail showing that I was wrong. I have some comments on the graphs–the tables are horrible, no need to even discuss them!–and I’d prefer if the authors of the paper could display their data and model on a single graph. I realize that their results reached a standard level of statistical significance, but it’s hard for me to interpret their claims until I see their estimates on some sort of direct real-world scale. In any case, though, I’m sure these researchers are working hard, and I wish them the best of luck in their future efforts to replicate their findings. I’m sure they’ll have no problem replicating, whether or not their claims are actually true. That’s the way science works: Once you know what you’re looking
2 0.12734945 1139 andrew gelman stats-2012-01-26-Suggested resolution of the Bem paradox
Introduction: There has been an increasing discussion about the proliferation of flawed research in psychology and medicine, with some landmark events being John Ioannides’s article , “Why most published research findings are false” (according to Google Scholar, cited 973 times since its appearance in 2005), the scandals of Marc Hauser and Diederik Stapel, two leading psychology professors who resigned after disclosures of scientific misconduct, and Daryl Bem’s dubious recent paper on ESP, published to much fanfare in Journal of Personality and Social Psychology, one of the top journals in the field. Alongside all this are the plagiarism scandals, which are uninteresting from a scientific context but are relevant in that, in many cases, neither the institutions housing the plagiarists nor the editors and publishers of the plagiarized material seem to care. Perhaps these universities and publishers are more worried about bad publicity (and maybe lawsuits, given that many of the plagiarism cas
3 0.12449743 2172 andrew gelman stats-2014-01-14-Advice on writing research articles
Introduction: From a few years ago : General advice Both the papers sent to me appear to have strong research results. Now that the research has been done, I’d recommend rewriting both articles from scratch, using the following template: 1. Start with the conclusions. Write a couple pages on what you’ve found and what you recommend. In writing these conclusions, you should also be writing some of the introduction, in that you’ll need to give enough background so that general readers can understand what you’re talking about and why they should care. But you want to start with the conclusions, because that will determine what sort of background information you’ll need to give. 2. Now step back. What is the principal evidence for your conclusions? Make some graphs and pull out some key numbers that represent your research findings which back up your claims. 3. Back one more step, now. What are the methods and data you used to obtain your research findings. 4. Now go back and write the l
4 0.12123359 937 andrew gelman stats-2011-10-02-That advice not to work so hard
Introduction: We often hear that at the end of life, people often wish they hadn’t worked so hard. (I’m assuming this is coming from executive types who have the option of working less, not people who had to work hard just to put food on the table.) I don’t understand this. Work is ok, but in almost any moment I much prefer relaxing to working. Nonetheless I often wish I were working harder or had worked harder. I don’t feel that I work too much. So I don’t know what to think. Am I just unusual? Or maybe I already don’t work so hard, so there’s nothing for me to regret? Or—and this is the scary option—maybe right now I wish I were working harder, but in twenty years I’ll regret that I spent so much time working? Here’s one thing. I like almost all the research papers I’ve written, but the vast majority (including some of my favorites) have had very few citations and, I assume, very little impact. So maybe I worked too hard on some of them?
5 0.12012996 2137 andrew gelman stats-2013-12-17-Replication backlash
Introduction: Raghuveer Parthasarathy pointed me to an article in Nature by Mina Bissell, who writes , “The push to replicate findings could shelve promising research and unfairly damage the reputations of careful, meticulous scientists.” I can see where she’s coming from: if you work hard day after day in the lab, it’s gotta be a bit frustrating to find all your work questioned, for the frauds of the Dr. Anil Pottis and Diederik Stapels to be treated as a reason for everyone else’s work to be considered guilty until proven innocent. That said, I pretty much disagree with Bissell’s article, and really the best thing I can say about it is that I think it’s a good sign that the push for replication is so strong that now there’s a backlash against it. Traditionally, leading scientists have been able to simply ignore the push for replication. If they are feeling that the replication movement is strong enough that they need to fight it, that to me is good news. I’ll explain a bit in the conte
6 0.11214879 1338 andrew gelman stats-2012-05-23-Advice on writing research articles
7 0.10876296 700 andrew gelman stats-2011-05-06-Suspicious pattern of too-strong replications of medical research
8 0.10843408 372 andrew gelman stats-2010-10-27-A use for tables (really)
9 0.10602954 1176 andrew gelman stats-2012-02-19-Standardized writing styles and standardized graphing styles
11 0.1002249 2269 andrew gelman stats-2014-03-27-Beyond the Valley of the Trolls
12 0.098409414 1878 andrew gelman stats-2013-05-31-How to fix the tabloids? Toward replicable social science research
15 0.097118638 1883 andrew gelman stats-2013-06-04-Interrogating p-values
16 0.097018495 1327 andrew gelman stats-2012-05-18-Comments on “A Bayesian approach to complex clinical diagnoses: a case-study in child abuse”
17 0.096483625 855 andrew gelman stats-2011-08-16-Infovis and statgraphics update update
19 0.093528152 1511 andrew gelman stats-2012-09-26-What do statistical p-values mean when the sample = the population?
20 0.092447571 1403 andrew gelman stats-2012-07-02-Moving beyond hopeless graphics
topicId topicWeight
[(0, 0.203), (1, -0.025), (2, -0.043), (3, -0.065), (4, 0.034), (5, -0.104), (6, -0.037), (7, -0.038), (8, -0.002), (9, -0.012), (10, 0.033), (11, 0.008), (12, 0.006), (13, -0.009), (14, -0.019), (15, 0.021), (16, -0.01), (17, 0.006), (18, -0.054), (19, 0.003), (20, -0.005), (21, -0.023), (22, -0.007), (23, -0.026), (24, -0.045), (25, -0.003), (26, 0.039), (27, 0.007), (28, 0.006), (29, 0.018), (30, 0.023), (31, 0.004), (32, 0.013), (33, -0.008), (34, 0.046), (35, 0.024), (36, -0.014), (37, 0.008), (38, 0.007), (39, -0.03), (40, 0.052), (41, 0.016), (42, -0.009), (43, 0.011), (44, 0.017), (45, -0.024), (46, -0.05), (47, -0.004), (48, 0.007), (49, 0.05)]
simIndex simValue blogId blogTitle
same-blog 1 0.97885954 302 andrew gelman stats-2010-09-28-This is a link to a news article about a scientific paper
Introduction: Somebody I know sent me a link to this news article by Martin Robbins describing a potential scientific breakthrough. I express some skepticism but in a vague enough way that, in the unlikely event that the research claim turns out to be correct, there’s no paper trail showing that I was wrong. I have some comments on the graphs–the tables are horrible, no need to even discuss them!–and I’d prefer if the authors of the paper could display their data and model on a single graph. I realize that their results reached a standard level of statistical significance, but it’s hard for me to interpret their claims until I see their estimates on some sort of direct real-world scale. In any case, though, I’m sure these researchers are working hard, and I wish them the best of luck in their future efforts to replicate their findings. I’m sure they’ll have no problem replicating, whether or not their claims are actually true. That’s the way science works: Once you know what you’re looking
Introduction: Jessica Tracy and Alec Beall, authors of that paper that claimed that women at peak fertility were more likely to wear red or pink shirts (see further discussion here and here ), and then a later paper that claimed that this happens in some weather but not others, just informed me that they have posted a note in disagreement with an paper by Eric Loken and myself. Our paper is unpublished, but I do have the megaphone of this blog, and Tracy and Beall do not, so I think it’s only fair to link to their note right away. I’ll quote from their note (but if you’re interested, please follow the link and read the whole thing ) and then give some background and my own reaction. Tracy and Beall write: Although Gelman and Loken are using our work as an example of a broader problem that pervades the field–a problem we generally agree about–we are concerned that readers will take their speculations about our methods and analyses as factual claims about our scientific integrity. Fu
3 0.79842508 933 andrew gelman stats-2011-09-30-More bad news: The (mis)reporting of statistical results in psychology journals
Introduction: Another entry in the growing literature on systematic flaws in the scientific research literature. This time the bad tidings come from Marjan Bakker and Jelte Wicherts, who write : Around 18% of statistical results in the psychological literature are incorrectly reported. Inconsistencies were more common in low-impact journals than in high-impact journals. Moreover, around 15% of the articles contained at least one statistical conclusion that proved, upon recalculation, to be incorrect; that is, recalculation rendered the previously significant result insignificant, or vice versa. These errors were often in line with researchers’ expectations. Their research also had a qualitative component: To obtain a better understanding of the origins of the errors made in the reporting of statistics, we contacted the authors of the articles with errors in the second study and asked them to send us the raw data. Regrettably, only 24% of the authors shared their data, despite our request
4 0.79764509 2172 andrew gelman stats-2014-01-14-Advice on writing research articles
Introduction: From a few years ago : General advice Both the papers sent to me appear to have strong research results. Now that the research has been done, I’d recommend rewriting both articles from scratch, using the following template: 1. Start with the conclusions. Write a couple pages on what you’ve found and what you recommend. In writing these conclusions, you should also be writing some of the introduction, in that you’ll need to give enough background so that general readers can understand what you’re talking about and why they should care. But you want to start with the conclusions, because that will determine what sort of background information you’ll need to give. 2. Now step back. What is the principal evidence for your conclusions? Make some graphs and pull out some key numbers that represent your research findings which back up your claims. 3. Back one more step, now. What are the methods and data you used to obtain your research findings. 4. Now go back and write the l
5 0.79544669 1413 andrew gelman stats-2012-07-11-News flash: Probability and statistics are hard to understand
Introduction: Two people pointed me to an article by Emre Soyer and Robin Hogarth that was linked to by Felix Salmon. Here are my reactions: 1. Soyer and Hogarth’s paper seems very strong to me, and Salmon’s presentation is an impressive condensation of it. I’d say good job on the science and the reporting. 2. I don’t see the point of focusing on economists. This seems just like a gimmick to me. But, then again, I’m not an economist. So of course I’d be more interested in a similar paper studying political scientists or statisticians. This should be easy enough for someone to do, of course. 3. To elaborate on this last point: I’m not surprised that people, even expert practitioners, screw up with statistics. Kahneman and Tversky found this with psychology researchers back in the 1970s. I’m not knocking the current paper by Soyer and Hogarth but I don’t see it as surprising. Perhaps the focus on economists is what allowed it to get all this attention. If you want people to re
6 0.78986382 1412 andrew gelman stats-2012-07-10-More questions on the contagion of obesity, height, etc.
7 0.78400022 1338 andrew gelman stats-2012-05-23-Advice on writing research articles
10 0.77700806 1096 andrew gelman stats-2012-01-02-Graphical communication for legal scholarship
11 0.77261001 1171 andrew gelman stats-2012-02-16-“False-positive psychology”
12 0.76107389 1844 andrew gelman stats-2013-05-06-Against optimism about social science
13 0.75922292 2241 andrew gelman stats-2014-03-10-Preregistration: what’s in it for you?
14 0.75499529 2137 andrew gelman stats-2013-12-17-Replication backlash
15 0.75332963 783 andrew gelman stats-2011-06-30-Don’t stop being a statistician once the analysis is done
16 0.75243521 2236 andrew gelman stats-2014-03-07-Selection bias in the reporting of shaky research
17 0.7451635 1889 andrew gelman stats-2013-06-08-Using trends in R-squared to measure progress in criminology??
19 0.74431312 2269 andrew gelman stats-2014-03-27-Beyond the Valley of the Trolls
topicId topicWeight
[(16, 0.105), (21, 0.058), (24, 0.126), (55, 0.013), (86, 0.036), (96, 0.197), (99, 0.361)]
simIndex simValue blogId blogTitle
1 0.98214269 1731 andrew gelman stats-2013-02-21-If a lottery is encouraging addictive gambling, don’t expand it!
Introduction: This story from Vivian Yee seems just horrible to me. First the background: Pronto Lotto’s real business takes place in the carpeted, hushed area where its most devoted customers watch video screens from a scattering of tall silver tables, hour after hour, day after day. The players — mostly men, about a dozen at any given time — come on their lunch breaks or after work to study the screens, which are programmed with the Quick Draw lottery game, and flash a new set of winning numbers every four minutes. They have helped make Pronto Lotto the top Quick Draw vendor in the state, selling $3.3 million worth of tickets last year, more than $1 million more than the second busiest location, a World Books shop in Penn Station. Some stay for just a few minutes. Others play for the length of a workday, repeatedly traversing the few yards between their seats and the cash register as they hand the next wager to a clerk with a dollar bill or two, and return to wait. “It’s like my job, 24
2 0.98028308 1306 andrew gelman stats-2012-05-07-Lists of Note and Letters of Note
Introduction: These (from Shaun Usher) are surprisingly good, especially since he appears to come up with new lists and letters pretty regularly. I suppose a lot of them get sent in from readers, but still. Here’s my favorite recent item, a letter sent to the Seattle Bureau of Prohibition in 1931: Dear Sir: My husband is in the habit of buying a quart of wiskey every other day from a Chinese bootlegger named Chin Waugh living at 317-16th near Alder street. We need this money for household expenses. Will you please have his place raided? He keeps a supply planted in the garden and a smaller quantity under the back steps for quick delivery. If you make the raid at 9:30 any morning you will be sure to get the goods and Chin also as he leaves the house at 10 o’clock and may clean up before he goes. Thanking you in advance, I remain yours truly, Mrs. Hillyer
Introduction: Paul Nee sends in this amusing item: MELA Sciences claimed success in a clinical trial of its experimental skin cancer detection device only by altering the statistical method used to analyze the data in violation of an agreement with U.S. regulators, charges an independent healthcare analyst in a report issued last week. . . The BER report, however, relies on its own analysis to suggest that MELA struck out with FDA because the agency’s medical device reviewers discovered the MELAFind pivotal study failed to reach statistical significance despite the company’s claims to the contrary. And now here’s where it gets interesting: MELA claims that a phase III study of MELAFind met its primary endpoint by detecting accurately 112 of 114 eligible melanomas for a “sensitivity” rate of 98%. The lower confidence bound of the sensitivity analysis was 95.1%, which met the FDA’s standard for statistical significance in the study spelled out in a binding agreement with MELA, the compa
4 0.95856547 327 andrew gelman stats-2010-10-07-There are never 70 distinct parameters
Introduction: Sam Seaver writes: I’m a graduate student in computational biology, and I’m relatively new to advanced statistics, and am trying to teach myself how best to approach a problem I have. My dataset is a small sparse matrix of 150 cases and 70 predictors, it is sparse as in many zeros, not many ‘NA’s. Each case is a nutrient that is fed into an in silico organism, and its response is whether or not it stimulates growth, and each predictor is one of 70 different pathways that the nutrient may or may not belong to. Because all of the nutrients do not belong to all of the pathways, there are thus many zeros in my matrix. My goal is to be able to use the pathways themselves to predict whether or not a nutrient could stimulate growth, thus I wanted to compute regression coefficients for each pathway, with which I could apply to other nutrients for other species. There are quite a few singularities in the dataset (summary(glm) reports that 14 coefficients are not defined because of sin
5 0.95736265 1023 andrew gelman stats-2011-11-22-Going Beyond the Book: Towards Critical Reading in Statistics Teaching
Introduction: My article with the above title is appearing in the journal Teaching Statistics. Here’s the introduction: We can improve our teaching of statistical examples from books by collecting further data, reading cited articles and performing further data analysis. This should not come as a surprise, but what might be new is the realization of how close to the surface these research opportunities are: even influential and celebrated books can have examples where more can be learned with a small amount of additional effort. We discuss three examples that have arisen in our own teaching: an introductory textbook that motivated us to think more carefully about categorical and continuous variables; a book for the lay reader that misreported a study of menstruation and accidents; and a monograph on the foundations of probability that over interpreted statistically insignificant fluctuations in sex ratios. And here’s the conclusion: Individually, these examples are of little importance.
6 0.94952679 319 andrew gelman stats-2010-10-04-“Who owns Congress”
same-blog 7 0.94764829 302 andrew gelman stats-2010-09-28-This is a link to a news article about a scientific paper
8 0.94159806 205 andrew gelman stats-2010-08-13-Arnold Zellner
9 0.93839067 787 andrew gelman stats-2011-07-05-Different goals, different looks: Infovis and the Chris Rock effect
10 0.93791711 99 andrew gelman stats-2010-06-19-Paired comparisons
11 0.92929351 934 andrew gelman stats-2011-09-30-Nooooooooooooooooooo!
12 0.92872798 405 andrew gelman stats-2010-11-10-Estimation from an out-of-date census
13 0.92675459 1887 andrew gelman stats-2013-06-07-“Happy Money: The Science of Smarter Spending”
14 0.92205584 690 andrew gelman stats-2011-05-01-Peter Huber’s reflections on data analysis
15 0.92108554 1338 andrew gelman stats-2012-05-23-Advice on writing research articles
16 0.91624331 2065 andrew gelman stats-2013-10-17-Cool dynamic demographic maps provide beautiful illustration of Chris Rock effect
17 0.91443658 2296 andrew gelman stats-2014-04-19-Index or indicator variables
18 0.91125822 1405 andrew gelman stats-2012-07-04-“Titanic Thompson: The Man Who Would Bet on Everything”
19 0.91117364 1642 andrew gelman stats-2012-12-28-New book by Stef van Buuren on missing-data imputation looks really good!
20 0.90713501 814 andrew gelman stats-2011-07-21-The powerful consumer?