andrew_gelman_stats andrew_gelman_stats-2010 andrew_gelman_stats-2010-300 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Sandy Gordon sends along this fun little paper forecasting the 2010 midterm election using expert predictions (the Cook and Rothenberg Political Reports). Gordon’s gimmick is that he uses past performance to calibrate the reports’ judgments based on “solid,” “likely,” “leaning,” and “toss-up” categories, and then he uses the calibrated versions of the current predictions to make his forecast. As I wrote a few weeks ago in response to Nate’s forecasts, I think the right way to go, if you really want to forecast the election outcome, is to use national information to predict the national swing and then do regional, state, and district-level adjustments using whatever local information is available. I don’t see the point of using only the expert forecasts and no other data. Still, Gordon is bringing new information (his calibrations) to the table, so I wanted to share it with you. Ultimately I like the throw-in-everything approach that Nate uses (although I think Nate’s descr
sentIndex sentText sentNum sentScore
1 Sandy Gordon sends along this fun little paper forecasting the 2010 midterm election using expert predictions (the Cook and Rothenberg Political Reports). [sent-1, score-0.93]
2 Gordon’s gimmick is that he uses past performance to calibrate the reports’ judgments based on “solid,” “likely,” “leaning,” and “toss-up” categories, and then he uses the calibrated versions of the current predictions to make his forecast. [sent-2, score-1.23]
3 I don’t see the point of using only the expert forecasts and no other data. [sent-4, score-0.413]
4 Still, Gordon is bringing new information (his calibrations) to the table, so I wanted to share it with you. [sent-5, score-0.376]
5 Ultimately I like the throw-in-everything approach that Nate uses (although I think Nate’s description of his own method could be a bit confusing in that it downplays the national-swing estimate which is so crucial to having it all work) . [sent-6, score-0.466]
6 Maybe Nate can throw Gordon’s information in too. [sent-7, score-0.231]
7 Note to John Barnard : Yes, I know, this is more politics stuff. [sent-8, score-0.065]
8 But the forecasting principles apply more generally, I think. [sent-9, score-0.312]
wordName wordTfidf (topN-words)
[('gordon', 0.514), ('nate', 0.373), ('uses', 0.21), ('forecasts', 0.181), ('forecasting', 0.172), ('information', 0.155), ('sandy', 0.148), ('predictions', 0.143), ('expert', 0.14), ('calibrations', 0.139), ('election', 0.132), ('barnard', 0.121), ('gimmick', 0.121), ('leaning', 0.121), ('reports', 0.121), ('national', 0.119), ('calibrated', 0.119), ('midterm', 0.11), ('calibrate', 0.107), ('judgments', 0.106), ('adjustments', 0.106), ('regional', 0.102), ('confusing', 0.097), ('bringing', 0.097), ('swing', 0.095), ('cook', 0.094), ('using', 0.092), ('versions', 0.092), ('crucial', 0.088), ('forecast', 0.087), ('solid', 0.087), ('categories', 0.085), ('sends', 0.077), ('throw', 0.076), ('weeks', 0.076), ('principles', 0.075), ('table', 0.074), ('description', 0.071), ('local', 0.07), ('outcome', 0.069), ('predict', 0.069), ('performance', 0.067), ('share', 0.067), ('ultimately', 0.066), ('apply', 0.065), ('politics', 0.065), ('fun', 0.064), ('note', 0.061), ('wanted', 0.057), ('current', 0.055)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 300 andrew gelman stats-2010-09-28-A calibrated Cook gives Dems the edge in Nov, sez Sandy
Introduction: Sandy Gordon sends along this fun little paper forecasting the 2010 midterm election using expert predictions (the Cook and Rothenberg Political Reports). Gordon’s gimmick is that he uses past performance to calibrate the reports’ judgments based on “solid,” “likely,” “leaning,” and “toss-up” categories, and then he uses the calibrated versions of the current predictions to make his forecast. As I wrote a few weeks ago in response to Nate’s forecasts, I think the right way to go, if you really want to forecast the election outcome, is to use national information to predict the national swing and then do regional, state, and district-level adjustments using whatever local information is available. I don’t see the point of using only the expert forecasts and no other data. Still, Gordon is bringing new information (his calibrations) to the table, so I wanted to share it with you. Ultimately I like the throw-in-everything approach that Nate uses (although I think Nate’s descr
2 0.31786469 270 andrew gelman stats-2010-09-12-Comparison of forecasts for the 2010 congressional elections
Introduction: Yesterday at the sister blog , Nate Silver forecast that the Republicans have a two-thirds chance of regaining the House of Representatives in the upcoming election, with an expected gain of 45 House seats. Last month, Bafumi, Erikson, and Wlezien released their forecast that gives the Republicans an 80% chance of takeover and an expected gain of 50 seats. As all the above writers emphasize, these forecasts are full of uncertainty, so I treat the two predictions–a 45-seat swing or a 50-seat swing–as essentially identical at the national level. And, as regular readers know, as far back as a year ago , the generic Congressional ballot (those questions of the form, “Which party do you plan to vote for in November?”) was also pointing to big Republican gains. As Bafumi et al. point out, early generic polls are strongly predictive of the election outcome, but they need to be interpreted carefully. The polls move in a generally predictable manner during the year leading up to an
3 0.29276741 391 andrew gelman stats-2010-11-03-Some thoughts on election forecasting
Introduction: I’ve written a lot on polls and elections (“a poll is a snapshot, not a forecast,” etc., or see here for a more technical paper with Kari Lock) but had a few things to add in light of Sam Wang’s recent efforts . As a biologist with a physics degree, Wang brings an outsider’s perspective to political forecasting, which can be a good thing. (I’m a bit of an outsider to political science myself, as is my sometime collaborator Nate Silver, who’s done a lot of good work in the past few years.) But there are two places where Wang misses the point, I think. He refers to his method as a “transparent, low-assumption calculation” and compares it favorably to “fancy modeling” and “assumption-laden models.” Assumptions are a bad thing, right? Well, no, I don’t think so. Bad assumptions are a bad thing. Good assumptions are just fine. Similarly for fancy modeling. I don’t see why a model should get credit for not including a factor that might be important. Let me clarify. I
4 0.23336154 131 andrew gelman stats-2010-07-07-A note to John
Introduction: Jeff the Productivity Sapper points me to this insulting open letter to Nate Silver written by pollster John Zogby. I’ll go through bits of Zogby’s note line by line. (Conflict of interest warning: I have collaborated with Nate and I blog on his site). Zogby writes: Here is some advice from someone [Zogby] who has been where you [Silver] are today. Sorry, John. (I can call you that, right? Since you’re calling Nate “Nate”?). Yes, you were once the hot pollster. But, no, you were never where Nate is today. Don’t kid yourself. Zogby writes: You [Nate] are hot right now – using an aggregate of other people’s work, you got 49 of 50 states right in 2008. Yes, Nate used other people’s work. That’s what’s called “making use of available data.” Or, to use a more technical term employed in statistics, it’s called “not being an idiot.” Only in the wacky world of polling are you supposed to draw inferences about the U.S.A. using only a single survey organization. I do
Introduction: The other day we had a fun little discussion in the comments section of the sister blog about the appropriateness of stating forecast probabilities to the nearest tenth of a percentage point. It started when Josh Tucker posted this graph from Nate Silver : My first reaction was: this looks pretty but it’s hyper-precise. I’m a big fan of Nate’s work, but all those little wiggles on the graph can’t really mean anything. And what could it possibly mean to compute this probability to that level of precision? In the comments, people came at me from two directions. From one side, Jeffrey Friedman expressed a hard core attitude that it’s meaningless to give a probability forecast of a unique event: What could it possibly mean, period, given that this election will never be repeated? . . . I know there’s a vast literature on this, but I’m still curious, as a non-statistician, what it could mean for there to be a meaningful 65% probability (as opposed to a non-quantifiab
6 0.20802191 364 andrew gelman stats-2010-10-22-Politics is not a random walk: Momentum and mean reversion in polling
7 0.16382554 1634 andrew gelman stats-2012-12-21-Two reviews of Nate Silver’s new book, from Kaiser Fung and Cathy O’Neil
8 0.15663342 1570 andrew gelman stats-2012-11-08-Poll aggregation and election forecasting
9 0.1467718 43 andrew gelman stats-2010-05-19-What do Tuesday’s elections tell us about November?
10 0.13534302 1610 andrew gelman stats-2012-12-06-Yes, checking calibration of probability forecasts is part of Bayesian statistics
11 0.12811102 292 andrew gelman stats-2010-09-23-Doug Hibbs on the fundamentals in 2010
12 0.12614968 142 andrew gelman stats-2010-07-12-God, Guns, and Gaydar: The Laws of Probability Push You to Overestimate Small Groups
13 0.11897452 2343 andrew gelman stats-2014-05-22-Big Data needs Big Model
14 0.11693159 237 andrew gelman stats-2010-08-27-Bafumi-Erikson-Wlezien predict a 50-seat loss for Democrats in November
15 0.11634918 987 andrew gelman stats-2011-11-02-How Khan Academy is using Machine Learning to Assess Student Mastery
16 0.10540854 30 andrew gelman stats-2010-05-13-Trips to Cleveland
17 0.10246968 924 andrew gelman stats-2011-09-24-“Income can’t be used to predict political opinion”
18 0.098846287 563 andrew gelman stats-2011-02-07-Evaluating predictions of political events
19 0.095664926 1214 andrew gelman stats-2012-03-15-Of forecasts and graph theory and characterizing a statistical method by the information it uses
20 0.089670993 1395 andrew gelman stats-2012-06-27-Cross-validation (What is it good for?)
topicId topicWeight
[(0, 0.129), (1, -0.007), (2, 0.089), (3, 0.071), (4, -0.029), (5, 0.005), (6, -0.056), (7, -0.038), (8, -0.01), (9, -0.022), (10, 0.097), (11, 0.004), (12, 0.037), (13, -0.065), (14, -0.106), (15, 0.034), (16, -0.006), (17, 0.028), (18, 0.026), (19, -0.017), (20, -0.037), (21, 0.098), (22, -0.014), (23, 0.077), (24, 0.034), (25, 0.021), (26, 0.042), (27, -0.006), (28, -0.007), (29, 0.081), (30, 0.0), (31, 0.044), (32, 0.014), (33, -0.109), (34, 0.041), (35, 0.056), (36, 0.056), (37, -0.021), (38, 0.045), (39, -0.033), (40, -0.054), (41, 0.046), (42, 0.02), (43, -0.046), (44, -0.097), (45, 0.04), (46, -0.036), (47, -0.045), (48, 0.071), (49, 0.081)]
simIndex simValue blogId blogTitle
same-blog 1 0.96251279 300 andrew gelman stats-2010-09-28-A calibrated Cook gives Dems the edge in Nov, sez Sandy
Introduction: Sandy Gordon sends along this fun little paper forecasting the 2010 midterm election using expert predictions (the Cook and Rothenberg Political Reports). Gordon’s gimmick is that he uses past performance to calibrate the reports’ judgments based on “solid,” “likely,” “leaning,” and “toss-up” categories, and then he uses the calibrated versions of the current predictions to make his forecast. As I wrote a few weeks ago in response to Nate’s forecasts, I think the right way to go, if you really want to forecast the election outcome, is to use national information to predict the national swing and then do regional, state, and district-level adjustments using whatever local information is available. I don’t see the point of using only the expert forecasts and no other data. Still, Gordon is bringing new information (his calibrations) to the table, so I wanted to share it with you. Ultimately I like the throw-in-everything approach that Nate uses (although I think Nate’s descr
2 0.8825472 270 andrew gelman stats-2010-09-12-Comparison of forecasts for the 2010 congressional elections
Introduction: Yesterday at the sister blog , Nate Silver forecast that the Republicans have a two-thirds chance of regaining the House of Representatives in the upcoming election, with an expected gain of 45 House seats. Last month, Bafumi, Erikson, and Wlezien released their forecast that gives the Republicans an 80% chance of takeover and an expected gain of 50 seats. As all the above writers emphasize, these forecasts are full of uncertainty, so I treat the two predictions–a 45-seat swing or a 50-seat swing–as essentially identical at the national level. And, as regular readers know, as far back as a year ago , the generic Congressional ballot (those questions of the form, “Which party do you plan to vote for in November?”) was also pointing to big Republican gains. As Bafumi et al. point out, early generic polls are strongly predictive of the election outcome, but they need to be interpreted carefully. The polls move in a generally predictable manner during the year leading up to an
3 0.83938354 364 andrew gelman stats-2010-10-22-Politics is not a random walk: Momentum and mean reversion in polling
Introduction: Nate Silver and Justin Wolfers are having a friendly blog-dispute about momentum in political polling. Nate and Justin each make good points but are also missing parts of the picture. These questions relate to my own research so I thought I’d discuss them here. There ain’t no mo’ Nate led off the discussion by writing that pundits are always talking about “momentum” in the polls: Turn on the news or read through much of the analysis put out by some of our friends, and you’re likely to hear a lot of talk about “momentum”: the term is used about 60 times per day by major media outlets in conjunction with articles about polling. When people say a particular candidate has momentum, what they are implying is that present trends are likely to perpetuate themselves into the future. Say, for instance, that a candidate trailed by 10 points in a poll three weeks ago — and now a new poll comes out showing the candidate down by just 5 points. It will frequently be said that this
4 0.79988605 1570 andrew gelman stats-2012-11-08-Poll aggregation and election forecasting
Introduction: At the sister blog, Henry writes about poll averaging and election forecasts. Henry writes that “These models need to crunch lots of polls, at the state and national level, if they’re going to provide good predictions.” Actually, you can get reasonable predictions from national-level forecasting models plus previous state-level election results, then when the election comes closer you can use national and state polls as needed. See my paper with Kari Lock, Bayesian combination of state polls and election forecasts . (That said, the method in that paper is fairly complicated, much more so than simply taking weighted averages of state polls, if such abundant data happen to be available. And I’m sure our approach would need to be altered if it were used for real-time forecasts.) Having a steady supply of polls of varying quality from various sources allows poll aggregators to produce news every day (in the sense of pushing their estimates around) but it doesn’t help much with a
5 0.79513532 131 andrew gelman stats-2010-07-07-A note to John
Introduction: Jeff the Productivity Sapper points me to this insulting open letter to Nate Silver written by pollster John Zogby. I’ll go through bits of Zogby’s note line by line. (Conflict of interest warning: I have collaborated with Nate and I blog on his site). Zogby writes: Here is some advice from someone [Zogby] who has been where you [Silver] are today. Sorry, John. (I can call you that, right? Since you’re calling Nate “Nate”?). Yes, you were once the hot pollster. But, no, you were never where Nate is today. Don’t kid yourself. Zogby writes: You [Nate] are hot right now – using an aggregate of other people’s work, you got 49 of 50 states right in 2008. Yes, Nate used other people’s work. That’s what’s called “making use of available data.” Or, to use a more technical term employed in statistics, it’s called “not being an idiot.” Only in the wacky world of polling are you supposed to draw inferences about the U.S.A. using only a single survey organization. I do
6 0.77736032 43 andrew gelman stats-2010-05-19-What do Tuesday’s elections tell us about November?
7 0.75059289 2343 andrew gelman stats-2014-05-22-Big Data needs Big Model
8 0.74277139 1512 andrew gelman stats-2012-09-27-A Non-random Walk Down Campaign Street
9 0.73381293 391 andrew gelman stats-2010-11-03-Some thoughts on election forecasting
10 0.67670983 292 andrew gelman stats-2010-09-23-Doug Hibbs on the fundamentals in 2010
12 0.62335116 210 andrew gelman stats-2010-08-16-What I learned from those tough 538 commenters
13 0.62016392 934 andrew gelman stats-2011-09-30-Nooooooooooooooooooo!
15 0.61795741 1000 andrew gelman stats-2011-11-10-Forecasting 2012: How much does ideology matter?
16 0.61587805 508 andrew gelman stats-2011-01-08-More evidence of growing nationalization of congressional elections
17 0.61583543 2181 andrew gelman stats-2014-01-21-The Commissar for Traffic presents the latest Five-Year Plan
18 0.61521852 478 andrew gelman stats-2010-12-20-More on why “all politics is local” is an outdated slogan
19 0.61164856 237 andrew gelman stats-2010-08-27-Bafumi-Erikson-Wlezien predict a 50-seat loss for Democrats in November
20 0.58277756 1544 andrew gelman stats-2012-10-22-Is it meaningful to talk about a probability of “65.7%” that Obama will win the election?
topicId topicWeight
[(5, 0.063), (10, 0.017), (16, 0.037), (21, 0.019), (24, 0.109), (29, 0.013), (34, 0.064), (55, 0.013), (76, 0.245), (84, 0.01), (86, 0.031), (90, 0.016), (97, 0.016), (99, 0.24)]
simIndex simValue blogId blogTitle
same-blog 1 0.92013478 300 andrew gelman stats-2010-09-28-A calibrated Cook gives Dems the edge in Nov, sez Sandy
Introduction: Sandy Gordon sends along this fun little paper forecasting the 2010 midterm election using expert predictions (the Cook and Rothenberg Political Reports). Gordon’s gimmick is that he uses past performance to calibrate the reports’ judgments based on “solid,” “likely,” “leaning,” and “toss-up” categories, and then he uses the calibrated versions of the current predictions to make his forecast. As I wrote a few weeks ago in response to Nate’s forecasts, I think the right way to go, if you really want to forecast the election outcome, is to use national information to predict the national swing and then do regional, state, and district-level adjustments using whatever local information is available. I don’t see the point of using only the expert forecasts and no other data. Still, Gordon is bringing new information (his calibrations) to the table, so I wanted to share it with you. Ultimately I like the throw-in-everything approach that Nate uses (although I think Nate’s descr
2 0.89050388 337 andrew gelman stats-2010-10-12-Election symposium at Columbia Journalism School
Introduction: Here’s the announcement: ELECTIONS 2010 How sustainable is the current Republican coalition? Why has support for the Democratic Party taken a nosedive? Has there been a fundamental change in political conflict since Obama took office? How might policy (health care, TARP, the stimulus) influence election outcomes? Join J-School Prof. TOM EDSALL and a bevy of some of the quickest political minds around for an examination of the upcoming November elections: * BEN SMITH & MAGGIE HABERMAN of Politico * CHARLES BLOW of the New York Times * MELINDA HENNEBERGER of Politics Daily * SAM STEIN of the Huffington Post * J-School Prof. TODD GITLIN and * Columbia University political scientists ANDREW GELMAN & BOB ERIKSON. This lecture is sponsored in part by the Sevellon Brown Lecture Fund. TUESDAY, OCTOBER 12 * 7 P.M. Journalism School Lecture Hall
Introduction: Sandeep Baliga writes : [In a recent study , Gilles Duranton and Matthew Turner write:] For interstate highways in metropolitan areas we [Duranton and Turner] find that VKT (vehicle kilometers traveled) increases one for one with interstate highways, confirming the fundamental law of highway congestion.’ Provision of public transit also simply leads to the people taking public transport being replaced by drivers on the road. Therefore: These findings suggest that both road capacity expansions and extensions to public transit are not appropriate policies with which to combat traffic congestion. This leaves congestion pricing as the main candidate tool to curb traffic congestion. To which I reply: Sure, if your goal is to curb traffic congestion . But what sort of goal is that? Thinking like a microeconomist, my policy goal is to increase people’s utility. Sure, traffic congestion is annoying, but there must be some advantages to driving on that crowded road or pe
4 0.88187677 1551 andrew gelman stats-2012-10-28-A convenience sample and selected treatments
Introduction: Charlie Saunders writes: A study has recently been published in the New England Journal of Medicine (NEJM) which uses survival analysis to examine long-acting reversible contraception (e.g. intrauterine devices [IUDs]) vs. short-term commonly prescribed methods of contraception (e.g. oral contraceptive pills) on unintended pregnancies. The authors use a convenience sample of over 7,000 women. I am not well versed-enough in sampling theory to determine the appropriateness of this but it would seem that the use of a non-probability sampling would be a significant drawback. If you could give me your opinion on this, I would appreciate it. The NEJM is one of the top medical journals in the country. Could this type of sampling method coupled with this method of analysis be published in a journal like JASA? My reply: There are two concerns, first that it is a convenience sample and thus not representative of the population, and second that the treatments are chosen rather tha
5 0.84937012 1351 andrew gelman stats-2012-05-29-A Ph.D. thesis is not really a marathon
Introduction: Thomas Basbøll writes : A blog called The Thesis Whisperer was recently pointed out to me. I [Basbøll] haven’t looked at it closely, but I’ll be reading it regularly for a while before I recommend it. I’m sure it’s a good place to go to discover that you’re not alone, especially when you’re struggling with your dissertation. One post caught my eye immediately. It suggested that writing a thesis is not a sprint, it’s a marathon. As a metaphorical adjustment to a particular attitude about writing, it’s probably going to help some people. But if we think it through, it’s not really a very good analogy. No one is really a “sprinter”; and writing a dissertation is nothing like running a marathon. . . . Here’s Ben’s explication of the analogy at the Thesis Whisperer, which seems initially plausible. …writing a dissertation is a lot like running a marathon. They are both endurance events, they last a long time and they require a consistent and carefully calculated amount of effor
6 0.8422336 1850 andrew gelman stats-2013-05-10-The recursion of pop-econ
7 0.8418901 1835 andrew gelman stats-2013-05-02-7 ways to separate errors from statistics
9 0.83555245 283 andrew gelman stats-2010-09-17-Vote Buying: Evidence from a List Experiment in Lebanon
10 0.81788146 51 andrew gelman stats-2010-05-26-If statistics is so significantly great, why don’t statisticians use statistics?
11 0.81307518 368 andrew gelman stats-2010-10-25-Is instrumental variables analysis particularly susceptible to Type M errors?
12 0.81253105 1084 andrew gelman stats-2011-12-26-Tweeting the Hits?
13 0.80838692 257 andrew gelman stats-2010-09-04-Question about standard range for social science correlations
14 0.80339473 928 andrew gelman stats-2011-09-27-Hey, look over here! Another rant!
15 0.79518116 1818 andrew gelman stats-2013-04-22-Goal: Rules for Turing chess
16 0.79297078 1600 andrew gelman stats-2012-12-01-$241,364.83 – $13,000 = $228,364.83
17 0.79029757 32 andrew gelman stats-2010-05-14-Causal inference in economics
18 0.78824246 2246 andrew gelman stats-2014-03-13-An Economist’s Guide to Visualizing Data
19 0.77820706 1105 andrew gelman stats-2012-01-08-Econ debate about prices at a fancy restaurant
20 0.77225423 922 andrew gelman stats-2011-09-24-Economists don’t think like accountants—but maybe they should