andrew_gelman_stats andrew_gelman_stats-2012 andrew_gelman_stats-2012-1544 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: The other day we had a fun little discussion in the comments section of the sister blog about the appropriateness of stating forecast probabilities to the nearest tenth of a percentage point. It started when Josh Tucker posted this graph from Nate Silver : My first reaction was: this looks pretty but it’s hyper-precise. I’m a big fan of Nate’s work, but all those little wiggles on the graph can’t really mean anything. And what could it possibly mean to compute this probability to that level of precision? In the comments, people came at me from two directions. From one side, Jeffrey Friedman expressed a hard core attitude that it’s meaningless to give a probability forecast of a unique event: What could it possibly mean, period, given that this election will never be repeated? . . . I know there’s a vast literature on this, but I’m still curious, as a non-statistician, what it could mean for there to be a meaningful 65% probability (as opposed to a non-quantifiab
sentIndex sentText sentNum sentScore
1 The other day we had a fun little discussion in the comments section of the sister blog about the appropriateness of stating forecast probabilities to the nearest tenth of a percentage point. [sent-1, score-0.757]
2 And what could it possibly mean to compute this probability to that level of precision? [sent-4, score-0.328]
3 From one side, Jeffrey Friedman expressed a hard core attitude that it’s meaningless to give a probability forecast of a unique event: What could it possibly mean, period, given that this election will never be repeated? [sent-6, score-0.729]
4 Similarly, I might describe someone as being 5 feet 8 inches tall, or even 5 feet 8 1/2 inches tall, but it would be silly to call him 5 feet 8. [sent-34, score-0.725]
5 One reason is that intrade prices are quoted to 3 digit precision — viz. [sent-38, score-0.399]
6 Nate’s page doesn’t give a standard error, but let’s suppose that his forecast for Obama’s popular vote share is a normal distribution with mean 50. [sent-68, score-0.909]
7 Then the probability Obama will win the popular vote is pnorm((50. [sent-75, score-0.684]
8 A big lead in the probability (65%-35%) corresponds to a liny lead in the vote (50. [sent-82, score-0.756]
9 Now suppose that our popular vote forecast is off by one-tenth of a percentage point. [sent-85, score-0.871]
10 Given all our uncertainties, it would seem pretty ridiculous to claim we could forecast to that precision anyway, right? [sent-86, score-0.401]
11 If we bump Obama’s predicted 2-party vote share up to 50. [sent-87, score-0.439]
12 If we ratchet Obama’s expected vote share down to 50. [sent-92, score-0.555]
13 1% in Obama’s expected vote share corresponds to a change of 2. [sent-98, score-0.771]
14 1 percentage points, then his expected percentage of the two-party vote must be qnorm(0. [sent-105, score-0.749]
15 I can’t see that it can possibly make sense to imagine an election forecast with that level of precision. [sent-112, score-0.41]
16 Even multiplying everything by ten—specifying win probabilities to the nearest percentage point—corresponds to specifying expected vote shares to within 0. [sent-113, score-1.15]
17 Really, I think it would be just fine to specify win probabilities to the nearest 10%, which will register shifts of 0. [sent-115, score-0.437]
18 On the other hand, people want news, hence the pressure to report essentially meaningless statistics such as a change in probability from 65. [sent-125, score-0.343]
19 To see this in another way, imagine that your forecast uncertainty about the election is summarized by 1000 simulations of the election outcome, that is, a 1000 x 51 matrix of simulated vote shares by state. [sent-133, score-0.923]
20 What shift in the vote would carry just 10 out of 1000 simulations over the bar? [sent-137, score-0.452]
wordName wordTfidf (topN-words)
[('nate', 0.368), ('vote', 0.297), ('obama', 0.282), ('forecast', 0.265), ('probability', 0.199), ('probabilities', 0.18), ('intrade', 0.179), ('percentage', 0.168), ('inches', 0.151), ('nearest', 0.144), ('share', 0.142), ('feet', 0.141), ('corresponds', 0.138), ('precision', 0.136), ('pnorm', 0.134), ('expected', 0.116), ('win', 0.113), ('winning', 0.103), ('tall', 0.099), ('wins', 0.088), ('digit', 0.084), ('election', 0.08), ('shift', 0.078), ('change', 0.078), ('simulations', 0.077), ('popular', 0.075), ('event', 0.073), ('balls', 0.072), ('specifying', 0.067), ('suppose', 0.066), ('oil', 0.066), ('meaningless', 0.066), ('possibly', 0.065), ('shares', 0.065), ('friedman', 0.065), ('mean', 0.064), ('outcome', 0.063), ('lead', 0.061), ('uncertainty', 0.059), ('point', 0.057), ('national', 0.054), ('given', 0.054), ('identical', 0.054), ('randomly', 0.051), ('drawn', 0.051), ('means', 0.048), ('market', 0.048), ('events', 0.047), ('price', 0.047), ('elections', 0.046)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 1544 andrew gelman stats-2012-10-22-Is it meaningful to talk about a probability of “65.7%” that Obama will win the election?
Introduction: The other day we had a fun little discussion in the comments section of the sister blog about the appropriateness of stating forecast probabilities to the nearest tenth of a percentage point. It started when Josh Tucker posted this graph from Nate Silver : My first reaction was: this looks pretty but it’s hyper-precise. I’m a big fan of Nate’s work, but all those little wiggles on the graph can’t really mean anything. And what could it possibly mean to compute this probability to that level of precision? In the comments, people came at me from two directions. From one side, Jeffrey Friedman expressed a hard core attitude that it’s meaningless to give a probability forecast of a unique event: What could it possibly mean, period, given that this election will never be repeated? . . . I know there’s a vast literature on this, but I’m still curious, as a non-statistician, what it could mean for there to be a meaningful 65% probability (as opposed to a non-quantifiab
Introduction: I recently wrote about the difficulty people have with probabilities, in this case the probability that Obama wins the election. If the probability is reported as 70%, people think Obama is going to win. Actually, though, it just means that Obama is predicted to get about 50.8% of the two-party vote, with an uncertainty of something like 2 percentage points. So, as I wrote, the election really is too close to call in the sense that the predicted vote margin is less than its uncertainty. But . . . when people see a number such as 70%, they tend to attribute too much certainty to it. Especially when the estimated probability has increased from, say 60%. How to get the point across? Commenter HS had what seems like a good suggestion: Say that Obama will win, but there is 25% chance (or whatever) that this prediction is wrong? Same point, just slightly different framing, but somehow, this seems far less incendiary. I like that. Somehow a stated probability of 75% sounds a
3 0.31079116 270 andrew gelman stats-2010-09-12-Comparison of forecasts for the 2010 congressional elections
Introduction: Yesterday at the sister blog , Nate Silver forecast that the Republicans have a two-thirds chance of regaining the House of Representatives in the upcoming election, with an expected gain of 45 House seats. Last month, Bafumi, Erikson, and Wlezien released their forecast that gives the Republicans an 80% chance of takeover and an expected gain of 50 seats. As all the above writers emphasize, these forecasts are full of uncertainty, so I treat the two predictions–a 45-seat swing or a 50-seat swing–as essentially identical at the national level. And, as regular readers know, as far back as a year ago , the generic Congressional ballot (those questions of the form, “Which party do you plan to vote for in November?”) was also pointing to big Republican gains. As Bafumi et al. point out, early generic polls are strongly predictive of the election outcome, but they need to be interpreted carefully. The polls move in a generally predictable manner during the year leading up to an
4 0.29014686 391 andrew gelman stats-2010-11-03-Some thoughts on election forecasting
Introduction: I’ve written a lot on polls and elections (“a poll is a snapshot, not a forecast,” etc., or see here for a more technical paper with Kari Lock) but had a few things to add in light of Sam Wang’s recent efforts . As a biologist with a physics degree, Wang brings an outsider’s perspective to political forecasting, which can be a good thing. (I’m a bit of an outsider to political science myself, as is my sometime collaborator Nate Silver, who’s done a lot of good work in the past few years.) But there are two places where Wang misses the point, I think. He refers to his method as a “transparent, low-assumption calculation” and compares it favorably to “fancy modeling” and “assumption-laden models.” Assumptions are a bad thing, right? Well, no, I don’t think so. Bad assumptions are a bad thing. Good assumptions are just fine. Similarly for fancy modeling. I don’t see why a model should get credit for not including a factor that might be important. Let me clarify. I
5 0.25352505 1574 andrew gelman stats-2012-11-12-How to Lie With Statistics example number 12,498,122
Introduction: This post is by Phil Price. Bill Kristol notes that “Four presidents in the last century have won more than 51 percent of the vote twice: Roosevelt, Eisenhower, Reagan and Obama”. I’m not sure why Kristol, a conservative, is promoting the idea that Obama has a mandate, but that’s up to him. I’m more interested in the remarkable bit of cherry-picking that led to this “only four presidents” statistic. There was one way in which Obama’s victory was large: he won the electoral college 332-206. That’s a thrashing. But if you want to claim that Obama has a “popular mandate” — which people seem to interpret as an overwhelming preference of The People such that the opposition is morally obligated to give way — you can’t make that argument based on the electoral college, you have to look at the popular vote. That presents you with a challenge for the 2012 election, since Obama’s 2.7-point margin in the popular vote was the 12th-smallest out of the 57 elections we’ve had. There’s a nice sor
6 0.23547797 364 andrew gelman stats-2010-10-22-Politics is not a random walk: Momentum and mean reversion in polling
7 0.23007865 131 andrew gelman stats-2010-07-07-A note to John
8 0.20874043 300 andrew gelman stats-2010-09-28-A calibrated Cook gives Dems the edge in Nov, sez Sandy
9 0.20319004 2255 andrew gelman stats-2014-03-19-How Americans vote
10 0.18618114 1027 andrew gelman stats-2011-11-25-Note to student journalists: Google is your friend
11 0.17736702 292 andrew gelman stats-2010-09-23-Doug Hibbs on the fundamentals in 2010
12 0.17726415 1569 andrew gelman stats-2012-11-08-30-30-40 Nation
13 0.17429008 1634 andrew gelman stats-2012-12-21-Two reviews of Nate Silver’s new book, from Kaiser Fung and Cathy O’Neil
14 0.16895102 1075 andrew gelman stats-2011-12-20-This guy has a regular column at Reuters
15 0.16613114 389 andrew gelman stats-2010-11-01-Why it can be rational to vote
16 0.16613114 1565 andrew gelman stats-2012-11-06-Why it can be rational to vote
17 0.16600144 934 andrew gelman stats-2011-09-30-Nooooooooooooooooooo!
18 0.15949091 237 andrew gelman stats-2010-08-27-Bafumi-Erikson-Wlezien predict a 50-seat loss for Democrats in November
19 0.15604043 84 andrew gelman stats-2010-06-14-Is it 1930?
topicId topicWeight
[(0, 0.205), (1, -0.047), (2, 0.192), (3, 0.142), (4, -0.101), (5, -0.023), (6, -0.035), (7, -0.033), (8, -0.015), (9, -0.145), (10, 0.074), (11, 0.037), (12, 0.057), (13, -0.16), (14, -0.164), (15, 0.003), (16, -0.02), (17, 0.021), (18, 0.033), (19, 0.011), (20, -0.006), (21, 0.161), (22, 0.035), (23, 0.072), (24, 0.011), (25, 0.05), (26, 0.058), (27, 0.024), (28, -0.059), (29, -0.03), (30, -0.056), (31, 0.081), (32, -0.047), (33, -0.017), (34, -0.002), (35, 0.032), (36, 0.085), (37, -0.05), (38, -0.069), (39, -0.066), (40, -0.055), (41, -0.05), (42, 0.034), (43, -0.062), (44, -0.038), (45, 0.034), (46, -0.01), (47, 0.07), (48, -0.021), (49, -0.044)]
simIndex simValue blogId blogTitle
same-blog 1 0.96859759 1544 andrew gelman stats-2012-10-22-Is it meaningful to talk about a probability of “65.7%” that Obama will win the election?
Introduction: The other day we had a fun little discussion in the comments section of the sister blog about the appropriateness of stating forecast probabilities to the nearest tenth of a percentage point. It started when Josh Tucker posted this graph from Nate Silver : My first reaction was: this looks pretty but it’s hyper-precise. I’m a big fan of Nate’s work, but all those little wiggles on the graph can’t really mean anything. And what could it possibly mean to compute this probability to that level of precision? In the comments, people came at me from two directions. From one side, Jeffrey Friedman expressed a hard core attitude that it’s meaningless to give a probability forecast of a unique event: What could it possibly mean, period, given that this election will never be repeated? . . . I know there’s a vast literature on this, but I’m still curious, as a non-statistician, what it could mean for there to be a meaningful 65% probability (as opposed to a non-quantifiab
2 0.88406527 934 andrew gelman stats-2011-09-30-Nooooooooooooooooooo!
Introduction: Michael Axelrod writes: Quantitative historian Allan Lichtman claims to have discovered 13 predictors that determine who will win the popular vote in presidential elections. He predicts Obama will win in 2012. Writing in his New York Times column, “538,” Nate Silver attempted a critique Lichtman’s prediction. Soon afterward Lichtman wrote a rejoinder. Evidently Lichtman has correctly and publicly predicted the popular vote winners in the last 7 presidential elections. I think he predicted Gore would win in 2000. He got the popular vote winner right, but not electoral college vote winner. Lichtman presents his methods in his early 1980s book, “The Keys to the White House.” Lichtman consulted with Volodia Keilis-Borok, and used a kernel discriminant analysis approach on election results from 1860-1980 as the training set. I think there is some argument as to scoring because Lichtman claims more than 7 successes. I guess he divided the data into a training and validation sets and w
Introduction: I recently wrote about the difficulty people have with probabilities, in this case the probability that Obama wins the election. If the probability is reported as 70%, people think Obama is going to win. Actually, though, it just means that Obama is predicted to get about 50.8% of the two-party vote, with an uncertainty of something like 2 percentage points. So, as I wrote, the election really is too close to call in the sense that the predicted vote margin is less than its uncertainty. But . . . when people see a number such as 70%, they tend to attribute too much certainty to it. Especially when the estimated probability has increased from, say 60%. How to get the point across? Commenter HS had what seems like a good suggestion: Say that Obama will win, but there is 25% chance (or whatever) that this prediction is wrong? Same point, just slightly different framing, but somehow, this seems far less incendiary. I like that. Somehow a stated probability of 75% sounds a
4 0.80598509 1574 andrew gelman stats-2012-11-12-How to Lie With Statistics example number 12,498,122
Introduction: This post is by Phil Price. Bill Kristol notes that “Four presidents in the last century have won more than 51 percent of the vote twice: Roosevelt, Eisenhower, Reagan and Obama”. I’m not sure why Kristol, a conservative, is promoting the idea that Obama has a mandate, but that’s up to him. I’m more interested in the remarkable bit of cherry-picking that led to this “only four presidents” statistic. There was one way in which Obama’s victory was large: he won the electoral college 332-206. That’s a thrashing. But if you want to claim that Obama has a “popular mandate” — which people seem to interpret as an overwhelming preference of The People such that the opposition is morally obligated to give way — you can’t make that argument based on the electoral college, you have to look at the popular vote. That presents you with a challenge for the 2012 election, since Obama’s 2.7-point margin in the popular vote was the 12th-smallest out of the 57 elections we’ve had. There’s a nice sor
5 0.80192095 1556 andrew gelman stats-2012-11-01-Recently in the sister blogs: special pre-election edition!
Introduction: 1. I can simultaneously (a) accept that Obama has a 72 percent chance of winning and (b) say the election is too close to call 2. Michael’s a Republican, Susan’s a Democrat 3. The narcissism of the narcissism of small differences 4. Obamanomics: A Counter-counterhistory 5. Not a gaffe 6. Categories influence predictions about individual consistency
6 0.79788315 1540 andrew gelman stats-2012-10-18-“Intrade to the 57th power”
7 0.7883532 292 andrew gelman stats-2010-09-23-Doug Hibbs on the fundamentals in 2010
8 0.7371071 270 andrew gelman stats-2010-09-12-Comparison of forecasts for the 2010 congressional elections
9 0.72526842 1075 andrew gelman stats-2011-12-20-This guy has a regular column at Reuters
10 0.70761913 1000 andrew gelman stats-2011-11-10-Forecasting 2012: How much does ideology matter?
11 0.70430744 1512 andrew gelman stats-2012-09-27-A Non-random Walk Down Campaign Street
12 0.68842417 1027 andrew gelman stats-2011-11-25-Note to student journalists: Google is your friend
13 0.68434447 237 andrew gelman stats-2010-08-27-Bafumi-Erikson-Wlezien predict a 50-seat loss for Democrats in November
14 0.67938805 391 andrew gelman stats-2010-11-03-Some thoughts on election forecasting
15 0.67393202 389 andrew gelman stats-2010-11-01-Why it can be rational to vote
16 0.67393202 1565 andrew gelman stats-2012-11-06-Why it can be rational to vote
17 0.67266405 210 andrew gelman stats-2010-08-16-What I learned from those tough 538 commenters
18 0.67251384 364 andrew gelman stats-2010-10-22-Politics is not a random walk: Momentum and mean reversion in polling
19 0.66881853 123 andrew gelman stats-2010-07-01-Truth in headlines
20 0.65199471 300 andrew gelman stats-2010-09-28-A calibrated Cook gives Dems the edge in Nov, sez Sandy
topicId topicWeight
[(2, 0.012), (5, 0.042), (9, 0.036), (15, 0.02), (16, 0.075), (21, 0.021), (24, 0.12), (27, 0.013), (44, 0.014), (66, 0.095), (76, 0.01), (81, 0.03), (86, 0.042), (93, 0.031), (95, 0.01), (98, 0.026), (99, 0.275)]
simIndex simValue blogId blogTitle
1 0.95998049 1322 andrew gelman stats-2012-05-15-Question 5 of my final exam for Design and Analysis of Sample Surveys
Introduction: 5. Which of the following better describes changes in public opinion on most issues? (Choose only one.) (a) Dynamic stability: On any given issue, average opinion remains stable but liberals and conservatives move back and forth in opposite directions (the “accordion model”) (b) Uniform swing: Average opinion on an issue can move but the liberals and conservatives don’t move much relative to each other (the disribution of opinions is a “solid block of wood”) (c) Compensating tradeoffs: When considering multiple survey questions on the same general topic, average opinion can move sharply to the left or right on individual questions while the average over all the questions remains stable (the “rubber band model”) Solution to question 4 From yesterday : 4. Researchers have found that survey respondents overreport church attendance. Thus, naive estimates from surveys overstate the percentage of Americans who attend church regularly. Does this have a large impact on estimate
2 0.95871532 536 andrew gelman stats-2011-01-24-Trends in partisanship by state
Introduction: Matthew Yglesias discusses how West Virginia used to be a Democratic state but is now solidly Republican. I thought it would be helpful to expand this to look at trends since 1948 (rather than just 1988) and all 50 states (rather than just one). This would represent a bit of work, except that I already did it a couple years ago, so here it is (right-click on the image to see the whole thing): I cheated a bit to get reasonable-looking groupings, for example putting Indiana in the Border South rather than Midwest, and putting Alaska in Mountain West and Hawaii in West Coast. Also, it would help to distinguish states by color (to be able to disentangle New Jersey and Delaware, for example) but we didn’t do this because the book is mostly black and white. In any case, the picture makes it clear that there have been strong regional trends all over during the past sixty years. P.S. My graph comes from Red State Blue State so no 2008 data, but 2008 was pretty much a shift
3 0.95815206 204 andrew gelman stats-2010-08-12-Sloppily-written slam on moderately celebrated writers is amusing nonetheless
Introduction: Via J. Robert Lennon , I discovered this amusing blog by Anis Shivani on “The 15 Most Overrated Contemporary American Writers.” Lennon found it so annoying that he refused to even link to it, but I actually enjoyed Shivani’s bit of performance art. The literary criticism I see is so focused on individual books that it’s refreshing to see someone take on an entire author’s career in a single paragraph. I agree with Lennon that Shivani’s blog doesn’t have much content –it’s full of terms such as “vacuity” and “pap,” compared to which “trendy” and “fashionable” are precision instruments–but Shivani covers a lot of ground and it’s fun to see this all in one place. My main complaint with Shivani, beyond his sloppy writing (but, hey, it’s just a blog; I’m sure he saves the good stuff for his paid gigs) is his implicit assumption that everyone should agree with him. I’m as big a Kazin fan as anyone, but I still think he completely undervalued Marquand . The other thing I noticed
same-blog 4 0.95772135 1544 andrew gelman stats-2012-10-22-Is it meaningful to talk about a probability of “65.7%” that Obama will win the election?
Introduction: The other day we had a fun little discussion in the comments section of the sister blog about the appropriateness of stating forecast probabilities to the nearest tenth of a percentage point. It started when Josh Tucker posted this graph from Nate Silver : My first reaction was: this looks pretty but it’s hyper-precise. I’m a big fan of Nate’s work, but all those little wiggles on the graph can’t really mean anything. And what could it possibly mean to compute this probability to that level of precision? In the comments, people came at me from two directions. From one side, Jeffrey Friedman expressed a hard core attitude that it’s meaningless to give a probability forecast of a unique event: What could it possibly mean, period, given that this election will never be repeated? . . . I know there’s a vast literature on this, but I’m still curious, as a non-statistician, what it could mean for there to be a meaningful 65% probability (as opposed to a non-quantifiab
5 0.95541584 1200 andrew gelman stats-2012-03-06-Some economists are skeptical about microfoundations
Introduction: A few months ago, I wrote : Economists seem to rely heavily on a sort of folk psychology, a relic of the 1920s-1950s in which people calculate utilities (or act as if they are doing so) in order to make decisions. A central tenet of economics is that inference or policy recommendation be derived from first principles from this folk-psychology model. This just seems silly to me, as if astronomers justified all their calculations with an underlying appeal to Aristotle’s mechanics. Or maybe the better analogy is the Stalinist era in which everything had to be connected to Marxist principles (followed, perhaps, by an equationful explanation of how the world can be interpreted as if Marxism were valid). Mark Thoma and Paul Krugman seem to agree with me on this one (as does my Barnard colleague Rajiv Sethi ). They don’t go so far as to identify utility etc as folk psychology, but maybe that will come next. P.S. Perhaps this will clarify: In a typical economics research pap
6 0.955221 1271 andrew gelman stats-2012-04-20-Education could use some systematic evaluation
7 0.9539603 1010 andrew gelman stats-2011-11-14-“Free energy” and economic resources
8 0.95202106 1192 andrew gelman stats-2012-03-02-These people totally don’t know what Chance magazine is all about
9 0.95028144 680 andrew gelman stats-2011-04-26-My talk at Berkeley on Wednesday
10 0.94550323 1323 andrew gelman stats-2012-05-16-Question 6 of my final exam for Design and Analysis of Sample Surveys
11 0.94334298 922 andrew gelman stats-2011-09-24-Economists don’t think like accountants—but maybe they should
12 0.94085443 814 andrew gelman stats-2011-07-21-The powerful consumer?
13 0.93768346 1439 andrew gelman stats-2012-08-01-A book with a bunch of simple graphs
15 0.93657207 797 andrew gelman stats-2011-07-11-How do we evaluate a new and wacky claim?
16 0.93577504 674 andrew gelman stats-2011-04-21-Handbook of Markov Chain Monte Carlo
17 0.93526936 2341 andrew gelman stats-2014-05-20-plus ça change, plus c’est la même chose
18 0.9352023 982 andrew gelman stats-2011-10-30-“There’s at least as much as an 80 percent chance . . .”
20 0.93272388 1914 andrew gelman stats-2013-06-25-Is there too much coauthorship in economics (and science more generally)? Or too little?