andrew_gelman_stats andrew_gelman_stats-2010 andrew_gelman_stats-2010-276 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Here’s a good one if you want to tell your students about question wording bias. It’s fun because the data are all on the web–the research is something that students could do on their own–if they know what to look for. Another win for Google. Here’s the story. I found the following graph on the front page of the American Enterprise Institute, a well-known D.C. think tank: My first thought was that they should replace this graph by a time series, which would show so much more information. I did a web search and, indeed, looking at a broad range of poll questions over time gives us a much richer perspective on public opinion about Afghanistan than is revealed in the above graph. I did a quick google search (“polling report afghanistan”) and found this . The quick summary is that roughly 40% of Americans favor the Afghan war (down from about 50% from 2006 through early 2009). The Polling Report page also features the Quninipiac poll featured in the above graph; here it r
sentIndex sentText sentNum sentScore
1 It’s fun because the data are all on the web–the research is something that students could do on their own–if they know what to look for. [sent-2, score-0.142]
2 I found the following graph on the front page of the American Enterprise Institute, a well-known D. [sent-5, score-0.243]
3 think tank: My first thought was that they should replace this graph by a time series, which would show so much more information. [sent-7, score-0.274]
4 I did a web search and, indeed, looking at a broad range of poll questions over time gives us a much richer perspective on public opinion about Afghanistan than is revealed in the above graph. [sent-8, score-0.695]
5 I did a quick google search (“polling report afghanistan”) and found this . [sent-9, score-0.237]
6 The quick summary is that roughly 40% of Americans favor the Afghan war (down from about 50% from 2006 through early 2009). [sent-10, score-0.184]
7 The Polling Report page also features the Quninipiac poll featured in the above graph; here it reports that, as of July 2010, 48% think the U. [sent-11, score-0.366]
8 is “doing the right thing” by fighting the war in Afghanistan and 43% think the U. [sent-13, score-0.3]
9 ” This phrasing seems to elicit more support–I guess people don’t want to think that the U. [sent-16, score-0.204]
10 OK, so we have 40% support, or maybe 48% support . [sent-19, score-0.199]
11 how did the AEI get the 58% support highlighted on its graph? [sent-22, score-0.179]
12 Searching the Polling Report page for the word “worthwhile,” I find this from the Quinnipiac poll: “Do you think eliminating the threat from terrorists operating from Afghanistan is a worthwhile goal for American troops to fight and possibly die for or not? [sent-23, score-1.307]
13 ” “Is worthwhile”: 59% “Is not worthwhile”: 34% But is the AEI correct to identify “a worthwhile goal” with the statement that “fighting in Afghanistan is worthwhile”? [sent-24, score-0.36]
14 Consider the next question from that Q poll: “Do you think the United States will be successful in eliminating the threat from terrorists operating from Afghanistan or not? [sent-27, score-0.794]
15 ” “Will be successful”: 35% “Will not be successful”: 55% It looks to me that, out of the 60% of the people who think that eliminating the terrorist threat in Afghanistan is a worthy goal, about 20% (that is, one-third of the 60%) don’t see the war as a good way to do that. [sent-28, score-0.66]
16 I think the AEI graph is misleading–although maybe not intentionally so. [sent-29, score-0.365]
17 This could be a good example in your political science or statistics classes of the peril of pulling out just one number without looking at the broader range of survey questions. [sent-30, score-0.337]
18 I’m just trying to report public opinion accurately. [sent-35, score-0.274]
19 Which, as can be seen here, is much easier to do if we take advantage of the wealth of poll information directly available on the web. [sent-36, score-0.265]
20 I re-checked at the AEI site and the graph shown above is no longer there. [sent-40, score-0.158]
wordName wordTfidf (topN-words)
[('afghanistan', 0.465), ('worthwhile', 0.36), ('aei', 0.335), ('poll', 0.215), ('eliminating', 0.188), ('threat', 0.16), ('graph', 0.158), ('afghan', 0.153), ('terrorists', 0.153), ('polling', 0.149), ('war', 0.122), ('goal', 0.12), ('successful', 0.118), ('support', 0.115), ('fighting', 0.112), ('operating', 0.109), ('report', 0.107), ('opinion', 0.093), ('page', 0.085), ('maybe', 0.084), ('american', 0.083), ('classes', 0.082), ('students', 0.076), ('public', 0.074), ('web', 0.073), ('hayward', 0.072), ('elicit', 0.072), ('range', 0.071), ('terrorist', 0.069), ('peril', 0.069), ('search', 0.068), ('troops', 0.066), ('phrasing', 0.066), ('think', 0.066), ('fun', 0.066), ('highlighted', 0.064), ('tank', 0.064), ('quick', 0.062), ('political', 0.062), ('wording', 0.061), ('merits', 0.057), ('commenting', 0.057), ('intentionally', 0.057), ('enterprise', 0.056), ('worthy', 0.055), ('pulling', 0.053), ('practitioners', 0.053), ('richer', 0.051), ('wealth', 0.05), ('time', 0.05)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 276 andrew gelman stats-2010-09-14-Don’t look at just one poll number–unless you really know what you’re doing!
Introduction: Here’s a good one if you want to tell your students about question wording bias. It’s fun because the data are all on the web–the research is something that students could do on their own–if they know what to look for. Another win for Google. Here’s the story. I found the following graph on the front page of the American Enterprise Institute, a well-known D.C. think tank: My first thought was that they should replace this graph by a time series, which would show so much more information. I did a web search and, indeed, looking at a broad range of poll questions over time gives us a much richer perspective on public opinion about Afghanistan than is revealed in the above graph. I did a quick google search (“polling report afghanistan”) and found this . The quick summary is that roughly 40% of Americans favor the Afghan war (down from about 50% from 2006 through early 2009). The Polling Report page also features the Quninipiac poll featured in the above graph; here it r
Introduction: Steven Hayward at the American Enterprise Institute wrote an article , sure to attract the attention of people such as myself, entitled, “The irrelevance of modern political science,” in which he discusses some silly-sounding papers presented at the recent American Political Science Association and then moves to a larger critique of quantitative political science: I [Hayward] have often taken a random article from the American Political Science Review, which resembles a mathematical journal on most of its pages, and asked students if they can envision this method providing the mathematical formula that will deliver peace in the Middle East. Even the dullest students usually grasp the point without difficulty. At the sister blog, John Sides discusses and dismisses Hayward’s arguments, point on that, among other things, political science might very well be useful even if it doesn’t deliver peace in the Middle East. After all, the U.S. Army didn’t deliver peace in the Midd
Introduction: I continue to struggle to convey my thoughts on statistical graphics so I’ll try another approach, this time giving my own story. For newcomers to this discussion: the background is that Antony Unwin and I wrote an article on the different goals embodied in information visualization and statistical graphics, but I have difficulty communicating on this point with the infovis people. Maybe if I tell my own story, and then they tell their stories, this will point a way forward to a more constructive discussion. So here goes. I majored in physics in college and I worked in a couple of research labs during the summer. Physicists graph everything. I did most of my plotting on graph paper–this continued through my second year of grad school–and became expert at putting points at 1/5, 2/5, 3/5, and 4/5 between the x and y grid lines. In grad school in statistics, I continued my physics habits and graphed everything I could. I did notice, though, that the faculty and the other
Introduction: Everybody knows how you can lie with statistics by manipulating numbers, making inappropriate comparisons, misleading graphs, etc. But, as I like to remind students, the simplest way to lie with statistics is to just lie! You see this all the time, advocates who make up numbers or present numbers with such little justification that they might as well be made up (as in this purported survey of the “super-rich”). Here I’m not talking about the innumeracy of a Samantha Power or a David Runciman, or Michael Barone-style confusion or Gregg Easterbrook-style cluelessness or even Tucker Carlson-style asininity . No, I’m talking about flat-out lying by a professional who has the numbers and deliberately chooses to misrepresent them. The culprit is pollster Doug Schoen, and the catch was made by Jay Livingston. Schoen wrote the following based on a survey he took of Occupy Wall Street participants: On Oct. 10 and 11, Arielle Alter Confino, a senior researcher at my polli
5 0.099746637 1787 andrew gelman stats-2013-04-04-Wanna be the next Tyler Cowen? It’s not as easy as you might think!
Introduction: Someone told me he ran into someone who said his goal was to be Tyler Cowen. OK, fine, it’s a worthy goal, but I don’t think it’s so easy .
6 0.098239966 100 andrew gelman stats-2010-06-19-Unsurprisingly, people are more worried about the economy and jobs than about deficits
7 0.097500756 985 andrew gelman stats-2011-11-01-Doug Schoen has 2 poll reports
8 0.097480074 2255 andrew gelman stats-2014-03-19-How Americans vote
9 0.093599662 913 andrew gelman stats-2011-09-16-Groundhog day in August?
11 0.08805263 2266 andrew gelman stats-2014-03-25-A statistical graphics course and statistical graphics advice
12 0.085240208 1929 andrew gelman stats-2013-07-07-Stereotype threat!
13 0.083017662 962 andrew gelman stats-2011-10-17-Death!
14 0.081706688 130 andrew gelman stats-2010-07-07-A False Consensus about Public Opinion on Torture
15 0.080475941 2221 andrew gelman stats-2014-02-23-Postdoc with Huffpost Pollster to do Bayesian poll tracking
17 0.076310866 1570 andrew gelman stats-2012-11-08-Poll aggregation and election forecasting
18 0.075746126 1517 andrew gelman stats-2012-10-01-“On Inspiring Students and Being Human”
19 0.07571061 531 andrew gelman stats-2011-01-22-Third-party Dream Ticket
20 0.072829261 364 andrew gelman stats-2010-10-22-Politics is not a random walk: Momentum and mean reversion in polling
topicId topicWeight
[(0, 0.145), (1, -0.077), (2, 0.037), (3, 0.036), (4, 0.044), (5, 0.001), (6, -0.045), (7, 0.044), (8, -0.043), (9, -0.022), (10, 0.032), (11, -0.023), (12, -0.014), (13, 0.037), (14, 0.004), (15, -0.005), (16, 0.023), (17, 0.01), (18, -0.006), (19, -0.005), (20, 0.008), (21, 0.012), (22, -0.032), (23, -0.019), (24, 0.022), (25, -0.014), (26, 0.049), (27, -0.025), (28, -0.044), (29, 0.023), (30, -0.031), (31, -0.022), (32, -0.016), (33, -0.016), (34, -0.062), (35, -0.007), (36, -0.018), (37, -0.067), (38, 0.03), (39, 0.008), (40, 0.029), (41, 0.004), (42, 0.018), (43, 0.004), (44, -0.023), (45, 0.025), (46, 0.034), (47, -0.018), (48, 0.006), (49, -0.017)]
simIndex simValue blogId blogTitle
same-blog 1 0.955127 276 andrew gelman stats-2010-09-14-Don’t look at just one poll number–unless you really know what you’re doing!
Introduction: Here’s a good one if you want to tell your students about question wording bias. It’s fun because the data are all on the web–the research is something that students could do on their own–if they know what to look for. Another win for Google. Here’s the story. I found the following graph on the front page of the American Enterprise Institute, a well-known D.C. think tank: My first thought was that they should replace this graph by a time series, which would show so much more information. I did a web search and, indeed, looking at a broad range of poll questions over time gives us a much richer perspective on public opinion about Afghanistan than is revealed in the above graph. I did a quick google search (“polling report afghanistan”) and found this . The quick summary is that roughly 40% of Americans favor the Afghan war (down from about 50% from 2006 through early 2009). The Polling Report page also features the Quninipiac poll featured in the above graph; here it r
2 0.76216918 2167 andrew gelman stats-2014-01-10-Do you believe that “humans and other living things have evolved over time”?
Introduction: The other day on the sister blog we discussed a recent Pew Research survey that seemed to show that Republicans are becoming more partisan about evolution (or, as Paul Krugman put it, “So what happened after 2009 that might be driving Republican views? . . . Republicans are being driven to identify in all ways with their tribe — and the tribal belief system is dominated by anti-science fundamentalists”). We presented some discussion and evidence from Dan Kahan suggesting that the evidence for such a change was not so clear at all. Kahan drew his conclusions from a more detailed analysis of the much-discussed Pew data, along with a comparison to a recent Gallup poll. Also following up on this is sociologist David Wealiem, who pulls some more data into the discussion: Although the Pew report mentions only the 2009 survey, the question has been asked a number of times since 2005. Here are the results—the numbers represent the percent saying “evolved” minus the percent sayin
3 0.72698057 1669 andrew gelman stats-2013-01-12-The power of the puzzlegraph
Introduction: The Organisation for Economic Co-operation and Development reports that the following project from Krisztina Szucs and Mate Cziner has won their visualization challenge, “launched in September 2012 to solicit visualisations based on the OECD’s data-rich Education at a Glance report”: (The graph is interactive. Click on the above image and click again to see the full version.) From the press release: Entries from around the world focused on data related to the economic costs and return on investment in education . . . [The winning entry] takes a detailed look at public vs. private and men vs. women for selected countries . . . The judges were particularly impressed by the angled slope format of the visualisation, which encourages comparison between the upper-secondary and tertiary benefits of education. Szucs and Cziner were also lauded for their striking visual design, which draws users into exploring their piece [emphasis added]. I used boldface to highlight a p
Introduction: John Kastellec points me to this blog by Ezra Klein criticizing the following graph from a recent Republican Party report: Klein (following Alexander Hart ) slams the graph for not going all the way to zero on the y-axis, thus making the projected change seem bigger than it really is. I agree with Klein and Hart that, if you’re gonna do a bar chart, you want the bars to go down to 0. On the other hand, a projected change from 19% to 23% is actually pretty big, and I don’t see the point of using a graphical display that hides it. The solution: Ditch the bar graph entirely and replace it by a lineplot , in particular, a time series with year-by-year data. The time series would have several advantages: 1. Data are placed in context. You’d see every year, instead of discrete averages, and you’d get to see the changes in the context of year-to-year variation. 2. With the time series, you can use whatever y-axis works with the data. No need to go to zero. P.S. I l
5 0.71539754 1258 andrew gelman stats-2012-04-10-Why display 6 years instead of 30?
Introduction: I continue to be the go-to guy for bad graphs. Today (i.e., 22 Feb), I received an email from Gary Rosin: I [Rosin] thought you might be interested in this graph showing the decline in median prices of homes since 1997. It exaggerates the proportions by using $150,000 as the floor, rather than zero. Indeed. Here’s the graph: A line plot, rather than a bar plot, would be appropriate here. Also, it’s weird that the headline says “10 years” but the graph has only 6 years. Why not give some perspective and show, say, 30 years?
6 0.70096058 609 andrew gelman stats-2011-03-13-Coauthorship norms
9 0.68561375 502 andrew gelman stats-2011-01-04-Cash in, cash out graph
10 0.6760996 1357 andrew gelman stats-2012-06-01-Halloween-Valentine’s update
12 0.67357022 2154 andrew gelman stats-2013-12-30-Bill Gates’s favorite graph of the year
13 0.67294014 1253 andrew gelman stats-2012-04-08-Technology speedup graph
14 0.67126697 113 andrew gelman stats-2010-06-28-Advocacy in the form of a “deliberative forum”
15 0.67100346 2181 andrew gelman stats-2014-01-21-The Commissar for Traffic presents the latest Five-Year Plan
16 0.66852266 915 andrew gelman stats-2011-09-17-(Worst) graph of the year
17 0.66317582 670 andrew gelman stats-2011-04-20-Attractive but hard-to-read graph could be made much much better
18 0.66221189 2203 andrew gelman stats-2014-02-08-“Guys who do more housework get less sex”
19 0.66131741 1061 andrew gelman stats-2011-12-16-CrossValidated: A place to post your statistics questions
20 0.65836316 443 andrew gelman stats-2010-12-02-Automating my graphics advice
topicId topicWeight
[(2, 0.018), (14, 0.014), (15, 0.015), (16, 0.051), (21, 0.02), (24, 0.106), (45, 0.013), (47, 0.016), (63, 0.011), (66, 0.012), (77, 0.022), (86, 0.278), (97, 0.02), (99, 0.262)]
simIndex simValue blogId blogTitle
1 0.98503983 1427 andrew gelman stats-2012-07-24-More from the sister blog
Introduction: Anthropologist Bruce Mannheim reports that a recent well-publicized study on the genetics of native Americans, which used genetic analysis to find “at least three streams of Asian gene flow,” is in fact a confirmation of a long-known fact. Mannheim writes: This three-way distinction was known linguistically since the 1920s (for example, Sapir 1921). Basically, it’s a division among the Eskimo-Aleut languages, which straddle the Bering Straits even today, the Athabaskan languages (which were discovered to be related to a small Siberian language family only within the last few years, not by Greenberg as Wade suggested), and everything else. This is not to say that the results from genetics are unimportant, but it’s good to see how it fits with other aspects of our understanding.
2 0.98410678 1530 andrew gelman stats-2012-10-11-Migrating your blog from Movable Type to WordPress
Introduction: Cord Blomquist, who did a great job moving us from horrible Movable Type to nice nice WordPress, writes: I [Cord] wanted to share a little news with you related to the original work we did for you last year. When ReadyMadeWeb converted your Movable Type blog to WordPress, we got a lot of other requestes for the same service, so we started thinking about a bigger market for such a product. After a bit of research, we started work on automating the data conversion, writing rules, and exceptions to the rules, on how Movable Type and TypePad data could be translated to WordPress. After many months of work, we’re getting ready to announce TP2WP.com , a service that converts Movable Type and TypePad export files to WordPress import files, so anyone who wants to migrate to WordPress can do so easily and without losing permalinks, comments, images, or other files. By automating our service, we’ve been able to drop the price to just $99. I recommend it (and, no, Cord is not paying m
3 0.97745323 873 andrew gelman stats-2011-08-26-Luck or knowledge?
Introduction: Joan Ginther has won the Texas lottery four times. First, she won $5.4 million, then a decade later, she won $2million, then two years later $3million and in the summer of 2010, she hit a $10million jackpot. The odds of this has been calculated at one in eighteen septillion and luck like this could only come once every quadrillion years. According to Forbes, the residents of Bishop, Texas, seem to believe God was behind it all. The Texas Lottery Commission told Mr Rich that Ms Ginther must have been ‘born under a lucky star’, and that they don’t suspect foul play. Harper’s reporter Nathanial Rich recently wrote an article about Ms Ginther, which calls the the validity of her ‘luck’ into question. First, he points out, Ms Ginther is a former math professor with a PhD from Stanford University specialising in statistics. More at Daily Mail. [Edited Saturday] In comments, C Ryan King points to the original article at Harper’s and Bill Jefferys to Wired .
4 0.96980298 253 andrew gelman stats-2010-09-03-Gladwell vs Pinker
Introduction: I just happened to notice this from last year. Eric Loken writes : Steven Pinker reviewed Malcolm Gladwell’s latest book and criticized him rather harshly for several shortcomings. Gladwell appears to have made things worse for himself in a letter to the editor of the NYT by defending a manifestly weak claim from one of his essays – the claim that NFL quarterback performance is unrelated to the order they were drafted out of college. The reason w [Loken and his colleagues] are implicated is that Pinker identified an earlier blog post of ours as one of three sources he used to challenge Gladwell (yay us!). But Gladwell either misrepresented or misunderstood our post in his response, and admonishes Pinker by saying “we should agree that our differences owe less to what can be found in the scientific literature than they do to what can be found on Google.” Well, here’s what you can find on Google. Follow this link to request the data for NFL quarterbacks drafted between 1980 and
5 0.96422428 1718 andrew gelman stats-2013-02-11-Toward a framework for automatic model building
Introduction: Patrick Caldon writes: I saw your recent blog post where you discussed in passing an iterative-chain-of models approach to AI. I essentially built such a thing for my PhD thesis – not in a Bayesian context, but in a logic programming context – and proved it had a few properties and showed how you could solve some toy problems. The important bit of my framework was that at various points you also go and get more data in the process – in a statistical context this might be seen as building a little univariate model on a subset of the data, then iteratively extending into a better model with more data and more independent variables – a generalized forward stepwise regression if you like. It wrapped a proper computational framework around E.M. Gold’s identification/learning in the limit based on a logic my advisor (Eric Martin) had invented. What’s not written up in the thesis is a few months of failed struggle trying to shoehorn some simple statistical inference into this
6 0.96270335 76 andrew gelman stats-2010-06-09-Both R and Stata
7 0.9624176 904 andrew gelman stats-2011-09-13-My wikipedia edit
8 0.94672614 558 andrew gelman stats-2011-02-05-Fattening of the world and good use of the alpha channel
9 0.93815386 1547 andrew gelman stats-2012-10-25-College football, voting, and the law of large numbers
12 0.92487121 436 andrew gelman stats-2010-11-29-Quality control problems at the New York Times
13 0.91758674 759 andrew gelman stats-2011-06-11-“2 level logit with 2 REs & large sample. computational nightmare – please help”
14 0.91636342 2219 andrew gelman stats-2014-02-21-The world’s most popular languages that the Mac documentation hasn’t been translated into
same-blog 15 0.91412014 276 andrew gelman stats-2010-09-14-Don’t look at just one poll number–unless you really know what you’re doing!
16 0.90739751 305 andrew gelman stats-2010-09-29-Decision science vs. social psychology
17 0.90522587 2082 andrew gelman stats-2013-10-30-Berri Gladwell Loken football update
18 0.88743943 1278 andrew gelman stats-2012-04-23-“Any old map will do” meets “God is in every leaf of every tree”
19 0.8858664 1586 andrew gelman stats-2012-11-21-Readings for a two-week segment on Bayesian modeling?
20 0.88480973 1971 andrew gelman stats-2013-08-07-I doubt they cheated