andrew_gelman_stats andrew_gelman_stats-2010 andrew_gelman_stats-2010-131 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Jeff the Productivity Sapper points me to this insulting open letter to Nate Silver written by pollster John Zogby. I’ll go through bits of Zogby’s note line by line. (Conflict of interest warning: I have collaborated with Nate and I blog on his site). Zogby writes: Here is some advice from someone [Zogby] who has been where you [Silver] are today. Sorry, John. (I can call you that, right? Since you’re calling Nate “Nate”?). Yes, you were once the hot pollster. But, no, you were never where Nate is today. Don’t kid yourself. Zogby writes: You [Nate] are hot right now – using an aggregate of other people’s work, you got 49 of 50 states right in 2008. Yes, Nate used other people’s work. That’s what’s called “making use of available data.” Or, to use a more technical term employed in statistics, it’s called “not being an idiot.” Only in the wacky world of polling are you supposed to draw inferences about the U.S.A. using only a single survey organization. I do
sentIndex sentText sentNum sentScore
1 Zogby writes: You [Nate] are hot right now – using an aggregate of other people’s work, you got 49 of 50 states right in 2008. [sent-12, score-0.24]
2 ” Only in the wacky world of polling are you supposed to draw inferences about the U. [sent-16, score-0.105]
3 Until then, how about some division of labor, where analysts such as Nate make use of polling data and pollsters such as John respect that their polls will be used in all sorts of interesting ways once their data go out the door. [sent-23, score-0.759]
4 Zogby writes: Hey, I have been right within a few tenths of a percent – but you are a probabilities guy and even a 95% confidence level and a margin of sampling error are not enough for some. [sent-24, score-0.523]
5 A basic understanding of sampling and nonsampling error will tell you that being “right within a few tenths of a percent” is luck, not skill. [sent-25, score-0.332]
6 A few tenths of a percent on a poll with a 3% margin of error . [sent-27, score-0.48]
7 sure, that’ll happen sometime, but perhaps it’s a good idea for you as a pollster to explain to people how randomness works. [sent-30, score-0.187]
8 Even if Nate conducted his own polls–heck, maybe he’s doing that right now, I have no idea—he’d be a complete an utter idiot to make forecasts from them and not use others’ polls as well. [sent-35, score-0.468]
9 Zogby writes: We pollsters are data-based problem-solvers. [sent-37, score-0.259]
10 This involves lots of people skills, a passion to get it right and do right by people who trust us. [sent-39, score-0.334]
11 Your [Silver's] ratings come with and generate a lot of vitriol. [sent-41, score-0.097]
12 You wouldn’t want us to treat all polls equally, would you? [sent-46, score-0.288]
13 To mix a high quality poll such as yours with some crappy robopoll or some discredited partisan hack job? [sent-47, score-0.178]
14 Sure, Nate could rate the pollsters and keep his rating a secret, but I think it actually will “make our world a better place” (as you put it) for Nate to be completely open about his procedures and release his poll ratings publicly, where they can be shared, challenged, and improved upon. [sent-49, score-0.534]
15 ” Releasing poll ratings is a way to increase transparency, no? [sent-51, score-0.232]
16 You should conduct some polls and learn that the rest of us good pollsters survey people, not statistics. [sent-53, score-0.751]
17 Haven’t you ever heard about the division of labor ? [sent-54, score-0.117]
18 In all seriousness, though, I think pollsters are extremely important in forming our understandings about politics, and I agree that there are all sorts of skills that good pollsters have. [sent-65, score-0.702]
19 There’s no way I could ever conduct a good poll myself, and I rely in so much of my research on the expertise of pollsters in the field. [sent-66, score-0.557]
20 There’s no reason for John to be so defensive: Nate Silver, Andy Gelman, Larry Bartels and all the rest of us rely crucially on the efforts of public opinion professionals from Gallup to Zogby to gather the information we use in our analyses. [sent-67, score-0.235]
wordName wordTfidf (topN-words)
[('zogby', 0.525), ('nate', 0.514), ('pollsters', 0.259), ('polls', 0.24), ('silver', 0.155), ('tenths', 0.143), ('poll', 0.135), ('ratings', 0.097), ('percent', 0.087), ('right', 0.087), ('pollster', 0.081), ('john', 0.068), ('hot', 0.066), ('margin', 0.065), ('conduct', 0.062), ('polling', 0.062), ('weights', 0.06), ('people', 0.059), ('division', 0.059), ('labor', 0.058), ('rely', 0.054), ('skills', 0.052), ('writes', 0.05), ('error', 0.05), ('survey', 0.05), ('within', 0.05), ('us', 0.048), ('nonsampling', 0.048), ('utter', 0.048), ('good', 0.047), ('make', 0.047), ('use', 0.046), ('used', 0.046), ('timely', 0.045), ('forecaster', 0.045), ('releasing', 0.045), ('rest', 0.045), ('lack', 0.044), ('world', 0.043), ('understandings', 0.043), ('election', 0.043), ('quality', 0.043), ('re', 0.043), ('agree', 0.042), ('hey', 0.042), ('heck', 0.042), ('crucially', 0.042), ('passion', 0.042), ('projections', 0.042), ('sampling', 0.041)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000002 131 andrew gelman stats-2010-07-07-A note to John
Introduction: Jeff the Productivity Sapper points me to this insulting open letter to Nate Silver written by pollster John Zogby. I’ll go through bits of Zogby’s note line by line. (Conflict of interest warning: I have collaborated with Nate and I blog on his site). Zogby writes: Here is some advice from someone [Zogby] who has been where you [Silver] are today. Sorry, John. (I can call you that, right? Since you’re calling Nate “Nate”?). Yes, you were once the hot pollster. But, no, you were never where Nate is today. Don’t kid yourself. Zogby writes: You [Nate] are hot right now – using an aggregate of other people’s work, you got 49 of 50 states right in 2008. Yes, Nate used other people’s work. That’s what’s called “making use of available data.” Or, to use a more technical term employed in statistics, it’s called “not being an idiot.” Only in the wacky world of polling are you supposed to draw inferences about the U.S.A. using only a single survey organization. I do
2 0.3797968 364 andrew gelman stats-2010-10-22-Politics is not a random walk: Momentum and mean reversion in polling
Introduction: Nate Silver and Justin Wolfers are having a friendly blog-dispute about momentum in political polling. Nate and Justin each make good points but are also missing parts of the picture. These questions relate to my own research so I thought I’d discuss them here. There ain’t no mo’ Nate led off the discussion by writing that pundits are always talking about “momentum” in the polls: Turn on the news or read through much of the analysis put out by some of our friends, and you’re likely to hear a lot of talk about “momentum”: the term is used about 60 times per day by major media outlets in conjunction with articles about polling. When people say a particular candidate has momentum, what they are implying is that present trends are likely to perpetuate themselves into the future. Say, for instance, that a candidate trailed by 10 points in a poll three weeks ago — and now a new poll comes out showing the candidate down by just 5 points. It will frequently be said that this
3 0.30316135 1634 andrew gelman stats-2012-12-21-Two reviews of Nate Silver’s new book, from Kaiser Fung and Cathy O’Neil
Introduction: People keep asking me what I think of Nate’s book, and I keep replying that, as a blogger, I’m spoiled. I’m so used to getting books for free that I wouldn’t go out and buy a book just for the purpose of reviewing it. (That reminds me that I should post reviews of some of those books I’ve received in the mail over the past few months.) I have, however, encountered a couple of reviews of The Signal and the Noise so I thought I’d pass them on to you. Both these reviews are by statisticians / data scientists who work here in NYC in the non-academic “real world” so in that sense they are perhaps better situated than me to review the book (also, they have not collaborated with Nate so they have no conflict of interest). Kaiser Fung gives a positive review : It is in the subtitle—“why so many predictions fail – but some don’t”—that one learns the core philosophy of Silver: he is most concerned with the honest evaluation of the performance of predictive models. The failure to look
4 0.29144272 391 andrew gelman stats-2010-11-03-Some thoughts on election forecasting
Introduction: I’ve written a lot on polls and elections (“a poll is a snapshot, not a forecast,” etc., or see here for a more technical paper with Kari Lock) but had a few things to add in light of Sam Wang’s recent efforts . As a biologist with a physics degree, Wang brings an outsider’s perspective to political forecasting, which can be a good thing. (I’m a bit of an outsider to political science myself, as is my sometime collaborator Nate Silver, who’s done a lot of good work in the past few years.) But there are two places where Wang misses the point, I think. He refers to his method as a “transparent, low-assumption calculation” and compares it favorably to “fancy modeling” and “assumption-laden models.” Assumptions are a bad thing, right? Well, no, I don’t think so. Bad assumptions are a bad thing. Good assumptions are just fine. Similarly for fancy modeling. I don’t see why a model should get credit for not including a factor that might be important. Let me clarify. I
5 0.24744396 270 andrew gelman stats-2010-09-12-Comparison of forecasts for the 2010 congressional elections
Introduction: Yesterday at the sister blog , Nate Silver forecast that the Republicans have a two-thirds chance of regaining the House of Representatives in the upcoming election, with an expected gain of 45 House seats. Last month, Bafumi, Erikson, and Wlezien released their forecast that gives the Republicans an 80% chance of takeover and an expected gain of 50 seats. As all the above writers emphasize, these forecasts are full of uncertainty, so I treat the two predictions–a 45-seat swing or a 50-seat swing–as essentially identical at the national level. And, as regular readers know, as far back as a year ago , the generic Congressional ballot (those questions of the form, “Which party do you plan to vote for in November?”) was also pointing to big Republican gains. As Bafumi et al. point out, early generic polls are strongly predictive of the election outcome, but they need to be interpreted carefully. The polls move in a generally predictable manner during the year leading up to an
6 0.23336154 300 andrew gelman stats-2010-09-28-A calibrated Cook gives Dems the edge in Nov, sez Sandy
8 0.22754245 1570 andrew gelman stats-2012-11-08-Poll aggregation and election forecasting
9 0.18137936 987 andrew gelman stats-2011-11-02-How Khan Academy is using Machine Learning to Assess Student Mastery
10 0.17103738 43 andrew gelman stats-2010-05-19-What do Tuesday’s elections tell us about November?
11 0.17049021 1610 andrew gelman stats-2012-12-06-Yes, checking calibration of probability forecasts is part of Bayesian statistics
12 0.16046505 1512 andrew gelman stats-2012-09-27-A Non-random Walk Down Campaign Street
13 0.149289 210 andrew gelman stats-2010-08-16-What I learned from those tough 538 commenters
14 0.13320819 2343 andrew gelman stats-2014-05-22-Big Data needs Big Model
15 0.12556732 1695 andrew gelman stats-2013-01-28-Economists argue about Bayes
16 0.12173035 87 andrew gelman stats-2010-06-15-Statistical analysis and visualization of the drug war in Mexico
17 0.12097146 120 andrew gelman stats-2010-06-30-You can’t put Pandora back in the box
18 0.10691824 474 andrew gelman stats-2010-12-18-The kind of frustration we could all use more of
19 0.10670092 1832 andrew gelman stats-2013-04-29-The blogroll
20 0.10334364 85 andrew gelman stats-2010-06-14-Prior distribution for design effects
topicId topicWeight
[(0, 0.165), (1, -0.048), (2, 0.07), (3, 0.062), (4, -0.026), (5, 0.022), (6, -0.049), (7, -0.015), (8, 0.01), (9, -0.061), (10, 0.082), (11, -0.033), (12, 0.033), (13, -0.046), (14, -0.132), (15, 0.022), (16, -0.017), (17, 0.036), (18, 0.047), (19, -0.007), (20, -0.035), (21, 0.096), (22, -0.051), (23, 0.098), (24, 0.001), (25, 0.017), (26, 0.02), (27, 0.011), (28, -0.027), (29, 0.115), (30, -0.035), (31, 0.036), (32, 0.014), (33, -0.06), (34, 0.016), (35, 0.108), (36, 0.051), (37, -0.002), (38, 0.007), (39, -0.021), (40, -0.01), (41, 0.091), (42, -0.024), (43, -0.049), (44, -0.105), (45, 0.059), (46, -0.039), (47, -0.035), (48, 0.061), (49, 0.055)]
simIndex simValue blogId blogTitle
same-blog 1 0.93238676 131 andrew gelman stats-2010-07-07-A note to John
Introduction: Jeff the Productivity Sapper points me to this insulting open letter to Nate Silver written by pollster John Zogby. I’ll go through bits of Zogby’s note line by line. (Conflict of interest warning: I have collaborated with Nate and I blog on his site). Zogby writes: Here is some advice from someone [Zogby] who has been where you [Silver] are today. Sorry, John. (I can call you that, right? Since you’re calling Nate “Nate”?). Yes, you were once the hot pollster. But, no, you were never where Nate is today. Don’t kid yourself. Zogby writes: You [Nate] are hot right now – using an aggregate of other people’s work, you got 49 of 50 states right in 2008. Yes, Nate used other people’s work. That’s what’s called “making use of available data.” Or, to use a more technical term employed in statistics, it’s called “not being an idiot.” Only in the wacky world of polling are you supposed to draw inferences about the U.S.A. using only a single survey organization. I do
2 0.8980962 364 andrew gelman stats-2010-10-22-Politics is not a random walk: Momentum and mean reversion in polling
Introduction: Nate Silver and Justin Wolfers are having a friendly blog-dispute about momentum in political polling. Nate and Justin each make good points but are also missing parts of the picture. These questions relate to my own research so I thought I’d discuss them here. There ain’t no mo’ Nate led off the discussion by writing that pundits are always talking about “momentum” in the polls: Turn on the news or read through much of the analysis put out by some of our friends, and you’re likely to hear a lot of talk about “momentum”: the term is used about 60 times per day by major media outlets in conjunction with articles about polling. When people say a particular candidate has momentum, what they are implying is that present trends are likely to perpetuate themselves into the future. Say, for instance, that a candidate trailed by 10 points in a poll three weeks ago — and now a new poll comes out showing the candidate down by just 5 points. It will frequently be said that this
3 0.87852979 300 andrew gelman stats-2010-09-28-A calibrated Cook gives Dems the edge in Nov, sez Sandy
Introduction: Sandy Gordon sends along this fun little paper forecasting the 2010 midterm election using expert predictions (the Cook and Rothenberg Political Reports). Gordon’s gimmick is that he uses past performance to calibrate the reports’ judgments based on “solid,” “likely,” “leaning,” and “toss-up” categories, and then he uses the calibrated versions of the current predictions to make his forecast. As I wrote a few weeks ago in response to Nate’s forecasts, I think the right way to go, if you really want to forecast the election outcome, is to use national information to predict the national swing and then do regional, state, and district-level adjustments using whatever local information is available. I don’t see the point of using only the expert forecasts and no other data. Still, Gordon is bringing new information (his calibrations) to the table, so I wanted to share it with you. Ultimately I like the throw-in-everything approach that Nate uses (although I think Nate’s descr
4 0.81178069 1570 andrew gelman stats-2012-11-08-Poll aggregation and election forecasting
Introduction: At the sister blog, Henry writes about poll averaging and election forecasts. Henry writes that “These models need to crunch lots of polls, at the state and national level, if they’re going to provide good predictions.” Actually, you can get reasonable predictions from national-level forecasting models plus previous state-level election results, then when the election comes closer you can use national and state polls as needed. See my paper with Kari Lock, Bayesian combination of state polls and election forecasts . (That said, the method in that paper is fairly complicated, much more so than simply taking weighted averages of state polls, if such abundant data happen to be available. And I’m sure our approach would need to be altered if it were used for real-time forecasts.) Having a steady supply of polls of varying quality from various sources allows poll aggregators to produce news every day (in the sense of pushing their estimates around) but it doesn’t help much with a
5 0.81143785 270 andrew gelman stats-2010-09-12-Comparison of forecasts for the 2010 congressional elections
Introduction: Yesterday at the sister blog , Nate Silver forecast that the Republicans have a two-thirds chance of regaining the House of Representatives in the upcoming election, with an expected gain of 45 House seats. Last month, Bafumi, Erikson, and Wlezien released their forecast that gives the Republicans an 80% chance of takeover and an expected gain of 50 seats. As all the above writers emphasize, these forecasts are full of uncertainty, so I treat the two predictions–a 45-seat swing or a 50-seat swing–as essentially identical at the national level. And, as regular readers know, as far back as a year ago , the generic Congressional ballot (those questions of the form, “Which party do you plan to vote for in November?”) was also pointing to big Republican gains. As Bafumi et al. point out, early generic polls are strongly predictive of the election outcome, but they need to be interpreted carefully. The polls move in a generally predictable manner during the year leading up to an
6 0.80769575 2343 andrew gelman stats-2014-05-22-Big Data needs Big Model
7 0.7521944 43 andrew gelman stats-2010-05-19-What do Tuesday’s elections tell us about November?
8 0.73611557 391 andrew gelman stats-2010-11-03-Some thoughts on election forecasting
9 0.73413396 1512 andrew gelman stats-2012-09-27-A Non-random Walk Down Campaign Street
10 0.67763907 508 andrew gelman stats-2011-01-08-More evidence of growing nationalization of congressional elections
11 0.66029501 478 andrew gelman stats-2010-12-20-More on why “all politics is local” is an outdated slogan
12 0.64959437 2181 andrew gelman stats-2014-01-21-The Commissar for Traffic presents the latest Five-Year Plan
14 0.62171721 87 andrew gelman stats-2010-06-15-Statistical analysis and visualization of the drug war in Mexico
15 0.61619318 292 andrew gelman stats-2010-09-23-Doug Hibbs on the fundamentals in 2010
16 0.61593997 210 andrew gelman stats-2010-08-16-What I learned from those tough 538 commenters
17 0.60367054 934 andrew gelman stats-2011-09-30-Nooooooooooooooooooo!
18 0.59620559 1544 andrew gelman stats-2012-10-22-Is it meaningful to talk about a probability of “65.7%” that Obama will win the election?
19 0.58813858 1634 andrew gelman stats-2012-12-21-Two reviews of Nate Silver’s new book, from Kaiser Fung and Cathy O’Neil
20 0.58526492 987 andrew gelman stats-2011-11-02-How Khan Academy is using Machine Learning to Assess Student Mastery
topicId topicWeight
[(2, 0.018), (5, 0.106), (13, 0.015), (16, 0.058), (21, 0.031), (24, 0.12), (27, 0.013), (50, 0.012), (52, 0.011), (53, 0.017), (86, 0.061), (89, 0.026), (90, 0.077), (99, 0.239)]
simIndex simValue blogId blogTitle
same-blog 1 0.94969171 131 andrew gelman stats-2010-07-07-A note to John
Introduction: Jeff the Productivity Sapper points me to this insulting open letter to Nate Silver written by pollster John Zogby. I’ll go through bits of Zogby’s note line by line. (Conflict of interest warning: I have collaborated with Nate and I blog on his site). Zogby writes: Here is some advice from someone [Zogby] who has been where you [Silver] are today. Sorry, John. (I can call you that, right? Since you’re calling Nate “Nate”?). Yes, you were once the hot pollster. But, no, you were never where Nate is today. Don’t kid yourself. Zogby writes: You [Nate] are hot right now – using an aggregate of other people’s work, you got 49 of 50 states right in 2008. Yes, Nate used other people’s work. That’s what’s called “making use of available data.” Or, to use a more technical term employed in statistics, it’s called “not being an idiot.” Only in the wacky world of polling are you supposed to draw inferences about the U.S.A. using only a single survey organization. I do
Introduction: Jonathan Robinson writes: I’m a survey researcher who mostly does political work, but I also have a strong interest in economics. I have a question about this graph you commonly see in the economics literature. It is of a concept called the Beveridge Curve [recently in the newspaper here ]. It is one of the more interesting concepts in labor economics, relating the vacancy rate in jobs to the unemployment rate. A good primer is here . However, despite being one of the more interesting concepts in economics, the way it is displayed visually is nothing short of atrocious: These graphs are nothing short of unreadable and pretty much the standard (Brad Delong has linked to this graph above and it can appear like this in publication as well). I’ve only really seen one representation of the curve that is more clear than this and it is at this link : Do you have any ideas of any way of making these graphs more readable? I like the second Cleveland Fed graph, but I ha
Introduction: Last month we reported on some claims of irregularities in the recent Russian elections. Just as a reminder, here are a couple graphs: Yesterday someone pointed me to two online articles: Mathematical proof of fraud in Russian elections unsound and US elections are as ‘non-normal’ as Russian elections . I know nothing about Russian elections and will defer to the author and his commenters on the details. That said, I don’t find the arguments to be at all persuasive. The protesters show drastic differences between the patterns of votes of Putin’s party and the others, and the linked articles seem a bit too eager to debunk. I wouldn’t necessarily blog on this but I was unhappy to see this material on the website of Significance, which is an official publication of the American Statistical Association and the Royal Statistical Society. The quality control at this site seems low. I clicked through the links and found this : Barring the revelation of a hoax
4 0.93126655 513 andrew gelman stats-2011-01-12-“Tied for Warmest Year On Record”
Introduction: The National Climatic Data Center has tentatively announced that 2010 is, get this, “tied” for warmest on record. Presumably they mean it’s tied to the precision that they quote (1.12 F above the 20th-century average). The uncertainty in the measurements, as well as some fuzziness about exactly what is being measured (how much of the atmosphere, and the oceans) makes these global-average things really suspect. For instance, if there’s more oceanic turnover one year, that can warm the deep ocean but cool the shallow ocean and atmosphere, so even though the heat content of the atmosphere-ocean system goes up, some of these “global-average” estimates can go down. The reverse can happen too. And of course there are various sources of natural variability that are not, these days, what most people are most interested in. So everybody who knows about the climate professes to hate the emphasis on climate records. And yet, they’re irresistible. I’m sure we’ll see the usual clamor of som
5 0.93016839 87 andrew gelman stats-2010-06-15-Statistical analysis and visualization of the drug war in Mexico
Introduction: Christian points me to this interesting (but sad) analysis by Diego Valle with an impressive series of graphs. There are a few things I’d change (notably the R default settings which result in ridiculously over-indexed y-axes, as well as axes for homicide rates which should (but do not) go town to zero (and sometimes, bizarrely, go negative), and a lack of coherent ordering of the 32 states (including D.F.), I’m no expert on Mexico (despite having coauthored a paper on Mexican politics) so I’ll leave it to others to evaluate the substantive claims in Valle’s blog. Just looking at what he’s done, though, it seems impressive to me. To put it another way, it’s like something Nate Silver might do.
7 0.92315477 1606 andrew gelman stats-2012-12-05-The Grinch Comes Back
8 0.91745174 364 andrew gelman stats-2010-10-22-Politics is not a random walk: Momentum and mean reversion in polling
9 0.91621256 15 andrew gelman stats-2010-05-03-Public Opinion on Health Care Reform
10 0.91458565 1512 andrew gelman stats-2012-09-27-A Non-random Walk Down Campaign Street
11 0.91449976 1052 andrew gelman stats-2011-12-11-Rational Turbulence
12 0.91327757 665 andrew gelman stats-2011-04-17-Yes, your wish shall be granted (in 25 years)
14 0.91289151 1250 andrew gelman stats-2012-04-07-Hangman tips
15 0.91252971 391 andrew gelman stats-2010-11-03-Some thoughts on election forecasting
16 0.91021281 10 andrew gelman stats-2010-04-29-Alternatives to regression for social science predictions
17 0.90740967 1417 andrew gelman stats-2012-07-15-Some decision analysis problems are pretty easy, no?
19 0.9053573 1286 andrew gelman stats-2012-04-28-Agreement Groups in US Senate and Dynamic Clustering
20 0.9033525 530 andrew gelman stats-2011-01-22-MS-Bayes?