andrew_gelman_stats andrew_gelman_stats-2013 andrew_gelman_stats-2013-2091 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Jeremy Fox writes: You’ve probably seen this [by Matthew Hankins]. . . . Everyone else on Twitter already has. It’s a graph of the frequency with which the phrase “marginally significant” occurs in association with different P values. Apparently it’s real data, from a Google Scholar search, though I haven’t tried to replicate the search myself. My reply: I admire the effort that went into the data collection and the excellent display (following Bill Cleveland etc., I’d prefer a landscape rather than portrait orientation of the graph, also I’d prefer a gritty histogram rather than a smooth density, and I don’t like the y-axis going below zero, nor do I like the box around the graph, also there’s that weird R default where the axis labels are so far from the actual axes, I don’t know whassup with that . . . but these are all minor, minor issues, certainly I’ve done much worse myself many times even in published articles; see the presentation here for lots of examples), an
sentIndex sentText sentNum sentScore
1 It’s a graph of the frequency with which the phrase “marginally significant” occurs in association with different P values. [sent-6, score-0.443]
2 Apparently it’s real data, from a Google Scholar search, though I haven’t tried to replicate the search myself. [sent-7, score-0.208]
3 My reply: I admire the effort that went into the data collection and the excellent display (following Bill Cleveland etc. [sent-8, score-0.166]
4 but these are all minor, minor issues, certainly I’ve done much worse myself many times even in published articles; see the presentation here for lots of examples), and I used to love this sort of thing, but in my grumpy middle age I’m sort of tired of it. [sent-12, score-0.461]
5 In the old days I’d see such a graph (for example, various histograms of published z-scores showing peaks around 2. [sent-17, score-0.634]
6 95) and think about how researchers can select what results to publish and how they can play around a bit and go from p=0. [sent-19, score-0.203]
7 But after what’s come out in the past few years (most notably, the article by Simmons, Nelson, and Simonsohn; see here for a popular summary of the concerns or here for a bunch of my recent papers on the topic), now I feel the problem is much more serious. [sent-22, score-0.073]
8 A serious researcher can easily get statistical significance when nothing is going on at all (or, of course, when something is going on but where the population comparison of interest is of indeterminate magnitude and direction). [sent-23, score-0.545]
9 And this can happen without the researcher even trying, just from doing an analysis that seems reasonable for the data at hand. [sent-24, score-0.118]
10 It’s worth looking at—again, I applaud what Hankins did—but what it’s focusing on is the least of our problems. [sent-29, score-0.183]
wordName wordTfidf (topN-words)
[('hankins', 0.455), ('graph', 0.191), ('minor', 0.152), ('indeterminate', 0.138), ('gritty', 0.138), ('marginally', 0.138), ('grumpy', 0.13), ('portrait', 0.13), ('peaks', 0.125), ('search', 0.124), ('histograms', 0.12), ('whassup', 0.12), ('researcher', 0.118), ('around', 0.118), ('beside', 0.117), ('landscape', 0.117), ('prefer', 0.112), ('axes', 0.111), ('histogram', 0.107), ('going', 0.106), ('simmons', 0.103), ('applaud', 0.103), ('nelson', 0.102), ('cleveland', 0.1), ('orientation', 0.1), ('jeremy', 0.1), ('tired', 0.099), ('smooth', 0.098), ('twitter', 0.094), ('fox', 0.093), ('labels', 0.092), ('simonsohn', 0.091), ('notably', 0.09), ('admire', 0.09), ('occurs', 0.09), ('axis', 0.088), ('scholar', 0.087), ('frequency', 0.087), ('matthew', 0.086), ('select', 0.085), ('replicate', 0.084), ('density', 0.083), ('box', 0.082), ('weird', 0.082), ('published', 0.08), ('focusing', 0.08), ('magnitude', 0.077), ('collection', 0.076), ('phrase', 0.075), ('concerns', 0.073)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 2091 andrew gelman stats-2013-11-06-“Marginally significant”
Introduction: Jeremy Fox writes: You’ve probably seen this [by Matthew Hankins]. . . . Everyone else on Twitter already has. It’s a graph of the frequency with which the phrase “marginally significant” occurs in association with different P values. Apparently it’s real data, from a Google Scholar search, though I haven’t tried to replicate the search myself. My reply: I admire the effort that went into the data collection and the excellent display (following Bill Cleveland etc., I’d prefer a landscape rather than portrait orientation of the graph, also I’d prefer a gritty histogram rather than a smooth density, and I don’t like the y-axis going below zero, nor do I like the box around the graph, also there’s that weird R default where the axis labels are so far from the actual axes, I don’t know whassup with that . . . but these are all minor, minor issues, certainly I’ve done much worse myself many times even in published articles; see the presentation here for lots of examples), an
2 0.13604519 1171 andrew gelman stats-2012-02-16-“False-positive psychology”
Introduction: Everybody’s talkin bout this paper by Joseph Simmons, Leif Nelson and Uri Simonsohn, who write : Despite empirical psychologists’ nominal endorsement of a low rate of false-positive findings (≤ .05), flexibility in data collection, analysis, and reporting dramatically increases actual false-positive rates. In many cases, a researcher is more likely to falsely find evidence that an effect exists than to correctly find evidence that it does not. We [Simmons, Nelson, and Simonsohn] present computer simulations and a pair of actual experiments that demonstrate how unacceptably easy it is to accumulate (and report) statistically significant evidence for a false hypothesis. Second, we suggest a simple, low-cost, and straightforwardly effective disclosure-based solution to this problem. The solution involves six concrete requirements for authors and four guidelines for reviewers, all of which impose a minimal burden on the publication process. Whatever you think about these recommend
Introduction: I continue to struggle to convey my thoughts on statistical graphics so I’ll try another approach, this time giving my own story. For newcomers to this discussion: the background is that Antony Unwin and I wrote an article on the different goals embodied in information visualization and statistical graphics, but I have difficulty communicating on this point with the infovis people. Maybe if I tell my own story, and then they tell their stories, this will point a way forward to a more constructive discussion. So here goes. I majored in physics in college and I worked in a couple of research labs during the summer. Physicists graph everything. I did most of my plotting on graph paper–this continued through my second year of grad school–and became expert at putting points at 1/5, 2/5, 3/5, and 4/5 between the x and y grid lines. In grad school in statistics, I continued my physics habits and graphed everything I could. I did notice, though, that the faculty and the other
4 0.12999268 61 andrew gelman stats-2010-05-31-A data visualization manifesto
Introduction: Details matter (at least, they do for me), but we don’t yet have a systematic way of going back and forth between the structure of a graph, its details, and the underlying questions that motivate our visualizations. (Cleveland, Wilkinson, and others have written a bit on how to formalize these connections, and I’ve thought about it too, but we have a ways to go.) I was thinking about this difficulty after reading an article on graphics by some computer scientists that was well-written but to me lacked a feeling for the linkages between substantive/statistical goals and graphical details. I have problems with these issues too, and my point here is not to criticize but to move the discussion forward. When thinking about visualization, how important are the details? Aleks pointed me to this article by Jeffrey Heer, Michael Bostock, and Vadim Ogievetsky, “A Tour through the Visualization Zoo: A survey of powerful visualization techniques, from the obvious to the obscure.” Th
5 0.11888768 2266 andrew gelman stats-2014-03-25-A statistical graphics course and statistical graphics advice
Introduction: Dean Eckles writes: Some of my coworkers at Facebook and I have worked with Udacity to create an online course on exploratory data analysis, including using data visualizations in R as part of EDA. The course has now launched at https://www.udacity.com/course/ud651 so anyone can take it for free. And Kaiser Fung has reviewed it . So definitely feel free to promote it! Criticism is also welcome (we are still fine-tuning things and adding more notes throughout). I wrote some more comments about the course here , including highlighting the interviews with my great coworkers. I didn’t have a chance to look at the course so instead I responded with some generic comments about eda and visualization (in no particular order): - Think of a graph as a comparison. All graphs are comparison (indeed, all statistical analyses are comparisons). If you already have the graph in mind, think of what comparisons it’s enabling. Or if you haven’t settled on the graph yet, think of what
6 0.11557622 488 andrew gelman stats-2010-12-27-Graph of the year
8 0.10932489 2154 andrew gelman stats-2013-12-30-Bill Gates’s favorite graph of the year
9 0.10335617 486 andrew gelman stats-2010-12-26-Age and happiness: The pattern isn’t as clear as you might think
10 0.10206389 671 andrew gelman stats-2011-04-20-One more time-use graph
11 0.10071275 1439 andrew gelman stats-2012-08-01-A book with a bunch of simple graphs
12 0.099228099 2245 andrew gelman stats-2014-03-12-More on publishing in journals
13 0.099192299 1376 andrew gelman stats-2012-06-12-Simple graph WIN: the example of birthday frequencies
14 0.098592624 855 andrew gelman stats-2011-08-16-Infovis and statgraphics update update
15 0.097190663 2065 andrew gelman stats-2013-10-17-Cool dynamic demographic maps provide beautiful illustration of Chris Rock effect
17 0.093391821 502 andrew gelman stats-2011-01-04-Cash in, cash out graph
18 0.093161136 1104 andrew gelman stats-2012-01-07-A compelling reason to go to London, Ontario??
19 0.092446737 899 andrew gelman stats-2011-09-10-The statistical significance filter
20 0.089319289 2092 andrew gelman stats-2013-11-07-Data visualizations gone beautifully wrong
topicId topicWeight
[(0, 0.201), (1, -0.052), (2, -0.029), (3, -0.014), (4, 0.077), (5, -0.139), (6, -0.031), (7, 0.018), (8, -0.009), (9, -0.018), (10, 0.03), (11, 0.001), (12, -0.008), (13, -0.011), (14, 0.041), (15, -0.007), (16, 0.016), (17, 0.003), (18, 0.018), (19, -0.017), (20, 0.013), (21, 0.026), (22, -0.027), (23, -0.027), (24, -0.019), (25, 0.003), (26, 0.013), (27, -0.021), (28, -0.003), (29, -0.023), (30, 0.038), (31, -0.006), (32, -0.022), (33, -0.047), (34, -0.05), (35, 0.044), (36, -0.027), (37, -0.059), (38, -0.027), (39, 0.023), (40, 0.024), (41, 0.022), (42, 0.015), (43, 0.054), (44, -0.021), (45, 0.008), (46, 0.034), (47, 0.001), (48, -0.007), (49, -0.039)]
simIndex simValue blogId blogTitle
same-blog 1 0.97985893 2091 andrew gelman stats-2013-11-06-“Marginally significant”
Introduction: Jeremy Fox writes: You’ve probably seen this [by Matthew Hankins]. . . . Everyone else on Twitter already has. It’s a graph of the frequency with which the phrase “marginally significant” occurs in association with different P values. Apparently it’s real data, from a Google Scholar search, though I haven’t tried to replicate the search myself. My reply: I admire the effort that went into the data collection and the excellent display (following Bill Cleveland etc., I’d prefer a landscape rather than portrait orientation of the graph, also I’d prefer a gritty histogram rather than a smooth density, and I don’t like the y-axis going below zero, nor do I like the box around the graph, also there’s that weird R default where the axis labels are so far from the actual axes, I don’t know whassup with that . . . but these are all minor, minor issues, certainly I’ve done much worse myself many times even in published articles; see the presentation here for lots of examples), an
2 0.87558079 1253 andrew gelman stats-2012-04-08-Technology speedup graph
Introduction: Dan Kahan sends along this awesome graph (click on the image to see the whole thing): and writes: I [Kahan] saw it at http://www.theatlantic.com/technology/archive/2012/04/the-100-year-march-of-technology-in-1-graph/255573/ , which misidentified the source (not “visual economics”; visualizingeconomics .com , which attributes it to Nicholas Felton , who apparently condensed this version , which I worry could cause a stroke). But it did have a good write-up that (I’m glad) caught my attention. It made me [Kahan] start to wonder about what sorts of qualities of a technology will influence its dissemination & also about the availability of benchmarks for proliferation of various sorts of things (e.g, fads & trends, health-promoting behaviors, knowledge of a scientific discovery) that one could use to gauge how meaningful the apparent increase in rates of proliferation of these technologies has been over time. That in turn made me wonder whether — indeed, suspect th
Introduction: To understand the above title, see here . Masanao writes: This report claims that eating meat increases the risk of cancer. I’m sure you can’t read the page but you probably can understand the graphs. Different bars represent subdivision in the amount of the particular type of meat one consumes. And each chunk is different types of meat. Left is for male right is for female. They claim that the difference is significant, but they are clearly not!! I’m for not eating much meat but this is just way too much… Here’s the graph: I don’t know what to think. If you look carefully you can find one or two statistically significant differences but overall the pattern doesn’t look so compelling. I don’t know what the top and bottom rows are, though. Overall, the pattern in the top row looks like it could represent a real trend, while the graphs on the bottom row look like noise. This could be a good example for our multiple comparisons paper. If the researchers won’t
4 0.85734963 502 andrew gelman stats-2011-01-04-Cash in, cash out graph
Introduction: David Afshartous writes: I thought this graph [from Ed Easterling] might be good for your blog. The 71 outlined squares show the main story, and the regions of the graph present the information nicely. Looks like the bins for the color coding are not of equal size and of course the end bins are unbounded. Might be interesting to graph the distribution of the actual data for the 71 outlined squares. In addition, I assume that each period begins on Jan 1 so data size could be naturally increased by looking at intervals that start on June 1 as well (where the limit of this process would be to have it at the granularity of one day; while it most likely wouldn’t make much difference, I’ve seen some graphs before where 1 year returns can be quite sensitive to starting date, etc). I agree that (a) the graph could be improved in small ways–in particular, adding half-year data seems like a great idea–and (b) it’s a wonderful, wonderful graph as is. And the NYT graphics people ad
Introduction: John Kastellec points me to this blog by Ezra Klein criticizing the following graph from a recent Republican Party report: Klein (following Alexander Hart ) slams the graph for not going all the way to zero on the y-axis, thus making the projected change seem bigger than it really is. I agree with Klein and Hart that, if you’re gonna do a bar chart, you want the bars to go down to 0. On the other hand, a projected change from 19% to 23% is actually pretty big, and I don’t see the point of using a graphical display that hides it. The solution: Ditch the bar graph entirely and replace it by a lineplot , in particular, a time series with year-by-year data. The time series would have several advantages: 1. Data are placed in context. You’d see every year, instead of discrete averages, and you’d get to see the changes in the context of year-to-year variation. 2. With the time series, you can use whatever y-axis works with the data. No need to go to zero. P.S. I l
6 0.82786351 2154 andrew gelman stats-2013-12-30-Bill Gates’s favorite graph of the year
7 0.82557833 1376 andrew gelman stats-2012-06-12-Simple graph WIN: the example of birthday frequencies
9 0.8159793 2266 andrew gelman stats-2014-03-25-A statistical graphics course and statistical graphics advice
10 0.81379163 1684 andrew gelman stats-2013-01-20-Ugly ugly ugly
11 0.81111115 61 andrew gelman stats-2010-05-31-A data visualization manifesto
12 0.80956352 1011 andrew gelman stats-2011-11-15-World record running times vs. distance
13 0.8085928 1104 andrew gelman stats-2012-01-07-A compelling reason to go to London, Ontario??
14 0.80397618 488 andrew gelman stats-2010-12-27-Graph of the year
15 0.80339861 2246 andrew gelman stats-2014-03-13-An Economist’s Guide to Visualizing Data
16 0.80265236 671 andrew gelman stats-2011-04-20-One more time-use graph
17 0.79919285 829 andrew gelman stats-2011-07-29-Infovis vs. statgraphics: A clear example of their different goals
18 0.79546076 1439 andrew gelman stats-2012-08-01-A book with a bunch of simple graphs
20 0.79272705 915 andrew gelman stats-2011-09-17-(Worst) graph of the year
topicId topicWeight
[(0, 0.026), (5, 0.034), (12, 0.029), (15, 0.044), (16, 0.147), (17, 0.012), (19, 0.013), (21, 0.039), (24, 0.166), (31, 0.011), (41, 0.019), (44, 0.036), (68, 0.011), (92, 0.012), (93, 0.017), (99, 0.301)]
simIndex simValue blogId blogTitle
1 0.9765619 586 andrew gelman stats-2011-02-23-A statistical version of Arrow’s paradox
Introduction: Unfortunately, when we deal with scientists, statisticians are often put in a setting reminiscent of Arrow’s paradox, where we are asked to provide estimates that are informative and unbiased and confidence statements that are correct conditional on the data and also on the underlying true parameter. [It's not generally possible for an estimate to do all these things at the same time -- ed.] Larry Wasserman feels that scientists are truly frequentist, and Don Rubin has told me how he feels that scientists interpret all statistical estimates Bayesianly. I have no doubt that both Larry and Don are correct. Voters want lower taxes and more services, and scientists want both Bayesian and frequency coverage; as the saying goes, everybody wants to go to heaven but nobody wants to die.
2 0.96949387 503 andrew gelman stats-2011-01-04-Clarity on my email policy
Introduction: I never read email before 4. That doesn’t mean I never send email before 4.
3 0.96655256 2227 andrew gelman stats-2014-02-27-“What Can we Learn from the Many Labs Replication Project?”
Introduction: Aki points us to this discussion from Rolf Zwaan: The first massive replication project in psychology has just reached completion (several others are to follow). . . . What can we learn from the ManyLabs project? The results here show the effect sizes for the replication efforts (in green and grey) as well as the original studies (in blue). The 99% confidence intervals are for the meta-analysis of the effect size (the green dots); the studies are ordered by effect size. Let’s first consider what we canNOT learn from these data. Of the 13 replication attempts (when the first four are taken together), 11 succeeded and 2 did not (in fact, at some point ManyLabs suggests that a third one, Imagined Contact also doesn’t really replicate). We cannot learn from this that the vast majority of psychological findings will replicate . . . But even if we had an accurate estimate of the percentage of findings that replicate, how useful would that be? Rather than trying to arrive at a mo
4 0.96578962 2179 andrew gelman stats-2014-01-20-The AAA Tranche of Subprime Science
Introduction: In our new ethics column for Chance , Eric Loken and I write about our current favorite topic: One of our ongoing themes when discussing scientific ethics is the central role of statistics in recognizing and communicating uncer- tainty. Unfortunately, statistics—and the scientific process more generally—often seems to be used more as a way of laundering uncertainty, processing data until researchers and consumers of research can feel safe acting as if various scientific hypotheses are unquestionably true. . . . We have in mind an analogy with the notorious AAA-class bonds created during the mid-2000s that led to the subprime mortgage crisis. Lower-quality mortgages—that is, mortgages with high probability of default and, thus, high uncertainty—were packaged and transformed into financial instruments that were (in retrospect, falsely) characterized as low risk. There was a tremendous interest in these securities, not just among the most unscrupulous market manipulators, but in a
5 0.96565205 411 andrew gelman stats-2010-11-13-Ethical concerns in medical trials
Introduction: I just read this article on the treatment of medical volunteers, written by doctor and bioethicist Carl Ellliott. As a statistician who has done a small amount of consulting for pharmaceutical companies, I have a slightly different perspective. As a doctor, Elliott focuses on individual patients, whereas, as a statistician, I’ve been trained to focus on the goal of accurately estimate treatment effects. I’ll go through Elliott’s article and give my reactions. Elliott: In Miami, investigative reporters for Bloomberg Markets magazine discovered that a contract research organisation called SFBC International was testing drugs on undocumented immigrants in a rundown motel; since that report, the motel has been demolished for fire and safety violations. . . . SFBC had recently been named one of the best small businesses in America by Forbes magazine. The Holiday Inn testing facility was the largest in North America, and had been operating for nearly ten years before inspecto
6 0.96423811 2137 andrew gelman stats-2013-12-17-Replication backlash
7 0.96417135 1755 andrew gelman stats-2013-03-09-Plaig
8 0.96328098 434 andrew gelman stats-2010-11-28-When Small Numbers Lead to Big Errors
9 0.96307635 447 andrew gelman stats-2010-12-03-Reinventing the wheel, only more so.
10 0.96294045 481 andrew gelman stats-2010-12-22-The Jumpstart financial literacy survey and the different purposes of tests
11 0.96264315 1712 andrew gelman stats-2013-02-07-Philosophy and the practice of Bayesian statistics (with all the discussions!)
same-blog 12 0.9618395 2091 andrew gelman stats-2013-11-06-“Marginally significant”
13 0.96135277 120 andrew gelman stats-2010-06-30-You can’t put Pandora back in the box
14 0.96043408 1878 andrew gelman stats-2013-05-31-How to fix the tabloids? Toward replicable social science research
15 0.95952219 2248 andrew gelman stats-2014-03-15-Problematic interpretations of confidence intervals
16 0.95858973 1871 andrew gelman stats-2013-05-27-Annals of spam
17 0.95857197 1016 andrew gelman stats-2011-11-17-I got 99 comparisons but multiplicity ain’t one
18 0.95781589 807 andrew gelman stats-2011-07-17-Macro causality
19 0.95744836 2353 andrew gelman stats-2014-05-30-I posted this as a comment on a sociology blog