andrew_gelman_stats andrew_gelman_stats-2014 andrew_gelman_stats-2014-2181 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: What do Paul Samuelson and the U.S. Department of Transportation have in common? Phil Price points us to this news article by Clark Williams-Derry: As the State Smart Transportation Initiative at the University of Wisconsin points out, the US Department of Transportation has been making the virtually identical vehicle travel forecasts for well over a decade. All of those forecasts project rapid and incessant growth in vehicle travel for as far as the eye can see. Meanwhile, actual traffic volumes have flattened out, and may actually be falling. Each of the rising colored lines represents a forecast from a different year. The black line represents actual traffic trends on US roads—which never rose as quickly as the forecasters had predicted, and actually started a modest decline in 2007. I’d like to see a label on the y-axis, and I’d recommend labeling the x-axis at 5-year intervals rather than every year, but the point seems pretty clear. Williams-Derry continues:
sentIndex sentText sentNum sentScore
1 Phil Price points us to this news article by Clark Williams-Derry: As the State Smart Transportation Initiative at the University of Wisconsin points out, the US Department of Transportation has been making the virtually identical vehicle travel forecasts for well over a decade. [sent-4, score-0.855]
2 All of those forecasts project rapid and incessant growth in vehicle travel for as far as the eye can see. [sent-5, score-0.508]
3 Meanwhile, actual traffic volumes have flattened out, and may actually be falling. [sent-6, score-0.399]
4 Each of the rising colored lines represents a forecast from a different year. [sent-7, score-0.298]
5 The black line represents actual traffic trends on US roads—which never rose as quickly as the forecasters had predicted, and actually started a modest decline in 2007. [sent-8, score-0.488]
6 These forecasts are a “roll-up” of forecasts made by state DOTs. [sent-11, score-0.587]
7 The US agency just collects the forecasts and reports them to the public: garbage in, garbage out. [sent-12, score-0.616]
8 But in a way, that’s even more sobering than if the fault were localized in USDOT, since it provides clear and compelling evidence that the nation’s entire transportation forecasting apparatus is completely broken. [sent-13, score-0.54]
9 In the aggregate, all of those hard working forecasters in all of those state DOTs are just making up numbers. [sent-14, score-0.306]
10 In subsequent editions, Samuelson provided no acknowledgment of his past failure to predict and little commentary beyond remarks about ‘bad weather’ in the Soviet Union. [sent-17, score-0.35]
11 If you’ve had bad weather in the past, maybe the possibility of future bad weather should be incorporated into the forecast, no? [sent-19, score-0.732]
12 As with the Commissar for Traffic, the problem was the linear model, the assumption that what has changed in the past will continue to change in the same way in the future. [sent-21, score-0.161]
13 What’s unforgivable, both in the case of Samuelson in the 1960s and the Commissars today, is not in making the mistake (surely, a linear extrapolation is quite reasonable in many settings) but in not recognizing the problem, even after the forecast has been completely destroyed by the data. [sent-22, score-0.505]
wordName wordTfidf (topN-words)
[('samuelson', 0.43), ('transportation', 0.278), ('forecasts', 0.252), ('weather', 0.235), ('traffic', 0.187), ('usdot', 0.181), ('forecasters', 0.148), ('forecast', 0.146), ('us', 0.144), ('vehicle', 0.143), ('garbage', 0.143), ('editions', 0.143), ('tabarrok', 0.125), ('subsequent', 0.116), ('travel', 0.113), ('fault', 0.102), ('bad', 0.097), ('past', 0.088), ('represents', 0.087), ('state', 0.083), ('acknowledgment', 0.082), ('localized', 0.082), ('sympathies', 0.082), ('overwhelm', 0.082), ('commissar', 0.082), ('communists', 0.082), ('paul', 0.079), ('department', 0.078), ('sobering', 0.078), ('flattened', 0.078), ('collects', 0.078), ('mistake', 0.075), ('making', 0.075), ('socialist', 0.074), ('leftist', 0.074), ('soviet', 0.074), ('linear', 0.073), ('destroyed', 0.07), ('future', 0.068), ('volumes', 0.068), ('commentators', 0.068), ('extrapolation', 0.066), ('catching', 0.066), ('actual', 0.066), ('rising', 0.065), ('legendary', 0.065), ('clark', 0.065), ('points', 0.064), ('commentary', 0.064), ('pushed', 0.064)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 2181 andrew gelman stats-2014-01-21-The Commissar for Traffic presents the latest Five-Year Plan
Introduction: What do Paul Samuelson and the U.S. Department of Transportation have in common? Phil Price points us to this news article by Clark Williams-Derry: As the State Smart Transportation Initiative at the University of Wisconsin points out, the US Department of Transportation has been making the virtually identical vehicle travel forecasts for well over a decade. All of those forecasts project rapid and incessant growth in vehicle travel for as far as the eye can see. Meanwhile, actual traffic volumes have flattened out, and may actually be falling. Each of the rising colored lines represents a forecast from a different year. The black line represents actual traffic trends on US roads—which never rose as quickly as the forecasters had predicted, and actually started a modest decline in 2007. I’d like to see a label on the y-axis, and I’d recommend labeling the x-axis at 5-year intervals rather than every year, but the point seems pretty clear. Williams-Derry continues:
2 0.13141613 2107 andrew gelman stats-2013-11-20-NYT (non)-retraction watch
Introduction: Mark Palko is irritated by the Times’s refusal to retract a recounting of a hoax regarding Dickens and Dostoevsky. All I can say is, the Times refuses to retract mistakes of fact that are far more current than that! See here for two examples that particularly annoyed me, to the extent that I contacted various people at the Times but ran into refusals to retract. I guess a daily newspaper publishes so much material that they can’t be expected to run a retraction every time they publish something false, even when such things are brought to their attention. Speaking of corrections, I wonder if later editions of the Samuelson economics textbook discussed their notorious graph predicting Soviet economic performance. The easiest thing would be just to remove the graph, but I think it would be a better economics lesson to discuss the error! Similarly, I think the NYT would do well to run an article on their Dickens-Dostoevsky mistake, along with a column by Arthur Brooks on how
3 0.12645371 391 andrew gelman stats-2010-11-03-Some thoughts on election forecasting
Introduction: I’ve written a lot on polls and elections (“a poll is a snapshot, not a forecast,” etc., or see here for a more technical paper with Kari Lock) but had a few things to add in light of Sam Wang’s recent efforts . As a biologist with a physics degree, Wang brings an outsider’s perspective to political forecasting, which can be a good thing. (I’m a bit of an outsider to political science myself, as is my sometime collaborator Nate Silver, who’s done a lot of good work in the past few years.) But there are two places where Wang misses the point, I think. He refers to his method as a “transparent, low-assumption calculation” and compares it favorably to “fancy modeling” and “assumption-laden models.” Assumptions are a bad thing, right? Well, no, I don’t think so. Bad assumptions are a bad thing. Good assumptions are just fine. Similarly for fancy modeling. I don’t see why a model should get credit for not including a factor that might be important. Let me clarify. I
4 0.12562007 270 andrew gelman stats-2010-09-12-Comparison of forecasts for the 2010 congressional elections
Introduction: Yesterday at the sister blog , Nate Silver forecast that the Republicans have a two-thirds chance of regaining the House of Representatives in the upcoming election, with an expected gain of 45 House seats. Last month, Bafumi, Erikson, and Wlezien released their forecast that gives the Republicans an 80% chance of takeover and an expected gain of 50 seats. As all the above writers emphasize, these forecasts are full of uncertainty, so I treat the two predictions–a 45-seat swing or a 50-seat swing–as essentially identical at the national level. And, as regular readers know, as far back as a year ago , the generic Congressional ballot (those questions of the form, “Which party do you plan to vote for in November?”) was also pointing to big Republican gains. As Bafumi et al. point out, early generic polls are strongly predictive of the election outcome, but they need to be interpreted carefully. The polls move in a generally predictable manner during the year leading up to an
Introduction: Sandeep Baliga writes : [In a recent study , Gilles Duranton and Matthew Turner write:] For interstate highways in metropolitan areas we [Duranton and Turner] find that VKT (vehicle kilometers traveled) increases one for one with interstate highways, confirming the fundamental law of highway congestion.’ Provision of public transit also simply leads to the people taking public transport being replaced by drivers on the road. Therefore: These findings suggest that both road capacity expansions and extensions to public transit are not appropriate policies with which to combat traffic congestion. This leaves congestion pricing as the main candidate tool to curb traffic congestion. To which I reply: Sure, if your goal is to curb traffic congestion . But what sort of goal is that? Thinking like a microeconomist, my policy goal is to increase people’s utility. Sure, traffic congestion is annoying, but there must be some advantages to driving on that crowded road or pe
6 0.10271557 1946 andrew gelman stats-2013-07-19-Prior distributions on derived quantities rather than on parameters themselves
7 0.093809567 2343 andrew gelman stats-2014-05-22-Big Data needs Big Model
8 0.09328863 1302 andrew gelman stats-2012-05-06-Fun with google autocomplete
9 0.093244284 649 andrew gelman stats-2011-04-05-Internal and external forecasting
10 0.09191744 2337 andrew gelman stats-2014-05-18-Never back down: The culture of poverty and the culture of journalism
11 0.088031575 300 andrew gelman stats-2010-09-28-A calibrated Cook gives Dems the edge in Nov, sez Sandy
12 0.087079868 1501 andrew gelman stats-2012-09-18-More studies on the economic effects of climate change
13 0.084385753 1981 andrew gelman stats-2013-08-14-The robust beauty of improper linear models in decision making
14 0.082587495 1147 andrew gelman stats-2012-01-30-Statistical Murder
15 0.077631108 1634 andrew gelman stats-2012-12-21-Two reviews of Nate Silver’s new book, from Kaiser Fung and Cathy O’Neil
16 0.077281237 2308 andrew gelman stats-2014-04-27-White stripes and dead armadillos
17 0.077028111 431 andrew gelman stats-2010-11-26-One fun thing about physicists . . .
18 0.075322084 752 andrew gelman stats-2011-06-08-Traffic Prediction
19 0.075303808 114 andrew gelman stats-2010-06-28-More on Bayesian deduction-induction
20 0.074860275 1544 andrew gelman stats-2012-10-22-Is it meaningful to talk about a probability of “65.7%” that Obama will win the election?
topicId topicWeight
[(0, 0.122), (1, -0.04), (2, 0.029), (3, 0.046), (4, 0.009), (5, -0.029), (6, -0.024), (7, 0.005), (8, -0.01), (9, 0.012), (10, 0.005), (11, 0.012), (12, -0.002), (13, -0.02), (14, -0.041), (15, 0.029), (16, 0.022), (17, 0.008), (18, 0.007), (19, -0.03), (20, -0.003), (21, 0.046), (22, -0.036), (23, 0.038), (24, 0.042), (25, -0.013), (26, 0.014), (27, -0.006), (28, 0.001), (29, 0.043), (30, -0.013), (31, -0.006), (32, -0.041), (33, -0.057), (34, -0.029), (35, 0.007), (36, 0.033), (37, -0.021), (38, 0.019), (39, -0.028), (40, -0.028), (41, 0.017), (42, 0.0), (43, -0.05), (44, -0.036), (45, 0.019), (46, -0.019), (47, -0.016), (48, 0.008), (49, -0.007)]
simIndex simValue blogId blogTitle
same-blog 1 0.94941491 2181 andrew gelman stats-2014-01-21-The Commissar for Traffic presents the latest Five-Year Plan
Introduction: What do Paul Samuelson and the U.S. Department of Transportation have in common? Phil Price points us to this news article by Clark Williams-Derry: As the State Smart Transportation Initiative at the University of Wisconsin points out, the US Department of Transportation has been making the virtually identical vehicle travel forecasts for well over a decade. All of those forecasts project rapid and incessant growth in vehicle travel for as far as the eye can see. Meanwhile, actual traffic volumes have flattened out, and may actually be falling. Each of the rising colored lines represents a forecast from a different year. The black line represents actual traffic trends on US roads—which never rose as quickly as the forecasters had predicted, and actually started a modest decline in 2007. I’d like to see a label on the y-axis, and I’d recommend labeling the x-axis at 5-year intervals rather than every year, but the point seems pretty clear. Williams-Derry continues:
2 0.77369785 300 andrew gelman stats-2010-09-28-A calibrated Cook gives Dems the edge in Nov, sez Sandy
Introduction: Sandy Gordon sends along this fun little paper forecasting the 2010 midterm election using expert predictions (the Cook and Rothenberg Political Reports). Gordon’s gimmick is that he uses past performance to calibrate the reports’ judgments based on “solid,” “likely,” “leaning,” and “toss-up” categories, and then he uses the calibrated versions of the current predictions to make his forecast. As I wrote a few weeks ago in response to Nate’s forecasts, I think the right way to go, if you really want to forecast the election outcome, is to use national information to predict the national swing and then do regional, state, and district-level adjustments using whatever local information is available. I don’t see the point of using only the expert forecasts and no other data. Still, Gordon is bringing new information (his calibrations) to the table, so I wanted to share it with you. Ultimately I like the throw-in-everything approach that Nate uses (although I think Nate’s descr
3 0.75419611 364 andrew gelman stats-2010-10-22-Politics is not a random walk: Momentum and mean reversion in polling
Introduction: Nate Silver and Justin Wolfers are having a friendly blog-dispute about momentum in political polling. Nate and Justin each make good points but are also missing parts of the picture. These questions relate to my own research so I thought I’d discuss them here. There ain’t no mo’ Nate led off the discussion by writing that pundits are always talking about “momentum” in the polls: Turn on the news or read through much of the analysis put out by some of our friends, and you’re likely to hear a lot of talk about “momentum”: the term is used about 60 times per day by major media outlets in conjunction with articles about polling. When people say a particular candidate has momentum, what they are implying is that present trends are likely to perpetuate themselves into the future. Say, for instance, that a candidate trailed by 10 points in a poll three weeks ago — and now a new poll comes out showing the candidate down by just 5 points. It will frequently be said that this
Introduction: My friend Seth, whom I know from Berkeley (we taught a course together on left-handedness), has a blog on topics ranging from thoughtful discussions of scientific evidence, to experiences with his unconventional weight-loss scheme, offbeat self-experimentation, and advocacy of fringe scientific theories, leavened with occasional dollops of cynicism and political extremism . I agree with Seth on some things but not others. ( Here’s Seth’s reason for not attempting a clinical trial of his diet.) Recently I was disturbed (but, I’m sorry to say, not surprised) to see Seth post the following: Predictions of climate models versus reality . I [Seth] have only seen careful prediction-vs-reality comparisons made by AGW [anthropogenic global warming] skeptics. Those who believe humans are dangerously warming the planet appear to be silent on this subject. In response, Phil commented : Funny, on the day you [Seth] made your post saying that you haven’t seen comparis
5 0.72631919 131 andrew gelman stats-2010-07-07-A note to John
Introduction: Jeff the Productivity Sapper points me to this insulting open letter to Nate Silver written by pollster John Zogby. I’ll go through bits of Zogby’s note line by line. (Conflict of interest warning: I have collaborated with Nate and I blog on his site). Zogby writes: Here is some advice from someone [Zogby] who has been where you [Silver] are today. Sorry, John. (I can call you that, right? Since you’re calling Nate “Nate”?). Yes, you were once the hot pollster. But, no, you were never where Nate is today. Don’t kid yourself. Zogby writes: You [Nate] are hot right now – using an aggregate of other people’s work, you got 49 of 50 states right in 2008. Yes, Nate used other people’s work. That’s what’s called “making use of available data.” Or, to use a more technical term employed in statistics, it’s called “not being an idiot.” Only in the wacky world of polling are you supposed to draw inferences about the U.S.A. using only a single survey organization. I do
6 0.70918959 1201 andrew gelman stats-2012-03-07-Inference = data + model
8 0.70246983 180 andrew gelman stats-2010-08-03-Climate Change News
10 0.69890404 1570 andrew gelman stats-2012-11-08-Poll aggregation and election forecasting
11 0.69816923 508 andrew gelman stats-2011-01-08-More evidence of growing nationalization of congressional elections
12 0.68920451 358 andrew gelman stats-2010-10-20-When Kerry Met Sally: Politics and Perceptions in the Demand for Movies
13 0.68303269 391 andrew gelman stats-2010-11-03-Some thoughts on election forecasting
14 0.66674721 513 andrew gelman stats-2011-01-12-“Tied for Warmest Year On Record”
15 0.66632611 87 andrew gelman stats-2010-06-15-Statistical analysis and visualization of the drug war in Mexico
16 0.66404182 2076 andrew gelman stats-2013-10-24-Chasing the noise: W. Edwards Deming would be spinning in his grave
17 0.65932274 270 andrew gelman stats-2010-09-12-Comparison of forecasts for the 2010 congressional elections
18 0.64232719 1501 andrew gelman stats-2012-09-18-More studies on the economic effects of climate change
19 0.63439506 563 andrew gelman stats-2011-02-07-Evaluating predictions of political events
20 0.63392299 607 andrew gelman stats-2011-03-11-Rajiv Sethi on the interpretation of prediction market data
topicId topicWeight
[(14, 0.015), (16, 0.049), (18, 0.097), (20, 0.046), (21, 0.059), (24, 0.096), (27, 0.017), (28, 0.01), (34, 0.045), (51, 0.027), (55, 0.025), (63, 0.037), (66, 0.011), (71, 0.013), (75, 0.014), (76, 0.033), (77, 0.054), (81, 0.013), (85, 0.014), (99, 0.201)]
simIndex simValue blogId blogTitle
same-blog 1 0.9196856 2181 andrew gelman stats-2014-01-21-The Commissar for Traffic presents the latest Five-Year Plan
Introduction: What do Paul Samuelson and the U.S. Department of Transportation have in common? Phil Price points us to this news article by Clark Williams-Derry: As the State Smart Transportation Initiative at the University of Wisconsin points out, the US Department of Transportation has been making the virtually identical vehicle travel forecasts for well over a decade. All of those forecasts project rapid and incessant growth in vehicle travel for as far as the eye can see. Meanwhile, actual traffic volumes have flattened out, and may actually be falling. Each of the rising colored lines represents a forecast from a different year. The black line represents actual traffic trends on US roads—which never rose as quickly as the forecasters had predicted, and actually started a modest decline in 2007. I’d like to see a label on the y-axis, and I’d recommend labeling the x-axis at 5-year intervals rather than every year, but the point seems pretty clear. Williams-Derry continues:
2 0.88845098 969 andrew gelman stats-2011-10-22-Researching the cost-effectiveness of political lobbying organisations
Introduction: Sally Murray from Giving What We Can writes: We are an organisation that assesses different charitable (/fundable) interventions, to estimate which are the most cost-effective (measured in terms of the improvement of life for people in developing countries gained for every dollar invested). Our research guides and encourages greater donations to the most cost-effective charities we thus identify, and our members have so far pledged a total of $14m to these causes, with many hundreds more relying on our advice in a less formal way. I am specifically researching the cost-effectiveness of political lobbying organisations. We are initially focusing on organisations that lobby for ‘big win’ outcomes such as increased funding of the most cost-effective NTD treatments/ vaccine research, changes to global trade rules (potentially) and more obscure lobbies such as “Keep Antibiotics Working”. We’ve a great deal of respect for your work and the superbly rational way you go about it, and
3 0.88679343 698 andrew gelman stats-2011-05-05-Shocking but not surprising
Introduction: Much-honored playwright Tony Kushner was set to receive one more honor–a degree from John Jay College–but it was suddenly taken away from him on an 11-1 vote of the trustees of the City University of New York. This was the first rejection of an honorary degree nomination since 1961. The news article focuses on one trustee, Jeffrey Wiesenfeld, an investment adviser and onetime political aide, who opposed Kushner’s honorary degree, but to me the relevant point is that the committee as a whole voted 11-1 to ding him. Kusnher said, “I’m sickened,” he added, “that this is happening in New York City. Shocked, really.” I can see why he’s shocked, but perhaps it’s not so surprising that it’s happening in NYC. Recall the famous incident from 1940 in which Bertrand Russell was invited and then uninvited to teach at City College. The problem that time was Russell’s views on free love (as they called it back then). There seems to be a long tradition of city college officials being will
4 0.88312656 588 andrew gelman stats-2011-02-24-In case you were wondering, here’s the price of milk
Introduction: Seeing Sarah Palin’s recent witticism : It’s no wonder Michelle Obama is telling everybody you need to breast feed your babies … the price of milk is so high! I was reminded of Dan Quayle’s quip during the 1988 campaign: The governor of Massachusetts, he lost his top naval adviser last week. His rubber ducky drowned in the bathtub. And this got me wondering: how often do legitimate political figures–not talk show hosts, but actual politicians–communicate via schoolyard-style taunts? I’m not talking here about dry wit of the Bob Dole or Morris Udall variety, or political gamesmanship such as Ronald Reagan’s “make my day,” or flat-out partisanship like Alan Grayson’s “the Republicans want you to die quickly” or James Watt’s line about “liberals and Americans.” It’s gotta be an actual joke. There must be some other examples of Palin/Quayle style humor, but I’m not sure where to look. I keep thinking of Veronica Geng’s hilarious story which includes the line: In
5 0.88180339 1292 andrew gelman stats-2012-05-01-Colorless green facts asserted resolutely
Introduction: Thomas Basbøll [yes, I've learned how to smoothly do this using alt-o] gives some writing advice : What gives a text presence is our commitment to asserting facts. We have to face the possibility that we may be wrong about them resolutely, and we do this by writing about them as though we are right. This and an earlier remark by Basbøll are closely related in my mind to predictive model checking and to Bayesian statistics : we make strong assumptions and then engage the data and the assumptions in a dialogue: assumptions + data -> inference, and we can then compare the inference to the data which can reveal problems with our model (or problems with the data, but that’s really problems with the model too, in this case problems with the model for the data). I like the idea that a condition for a story to be useful is that we put some belief into it. (One doesn’t put belief into a joke.) And also the converse, that thnking hard about a story and believing it can be the pre
6 0.88144928 718 andrew gelman stats-2011-05-18-Should kids be able to bring their own lunches to school?
7 0.87729245 1967 andrew gelman stats-2013-08-04-What are the key assumptions of linear regression?
8 0.87400424 1319 andrew gelman stats-2012-05-14-I hate to get all Gerd Gigerenzer on you here, but . . .
10 0.86631382 1382 andrew gelman stats-2012-06-17-How to make a good fig?
11 0.86631328 1691 andrew gelman stats-2013-01-25-Extreem p-values!
12 0.86626667 1204 andrew gelman stats-2012-03-08-The politics of economic and statistical models
13 0.8629086 2046 andrew gelman stats-2013-10-01-I’ll say it again
15 0.85847223 252 andrew gelman stats-2010-09-02-R needs a good function to make line plots
16 0.85676765 621 andrew gelman stats-2011-03-20-Maybe a great idea in theory, didn’t work so well in practice
17 0.85539877 659 andrew gelman stats-2011-04-13-Jim Campbell argues that Larry Bartels’s “Unequal Democracy” findings are not robust
18 0.85490847 2239 andrew gelman stats-2014-03-09-Reviewing the peer review process?
19 0.85384911 2186 andrew gelman stats-2014-01-26-Infoviz on top of stat graphic on top of spreadsheet
20 0.85332179 114 andrew gelman stats-2010-06-28-More on Bayesian deduction-induction