andrew_gelman_stats andrew_gelman_stats-2012 andrew_gelman_stats-2012-1634 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: People keep asking me what I think of Nate’s book, and I keep replying that, as a blogger, I’m spoiled. I’m so used to getting books for free that I wouldn’t go out and buy a book just for the purpose of reviewing it. (That reminds me that I should post reviews of some of those books I’ve received in the mail over the past few months.) I have, however, encountered a couple of reviews of The Signal and the Noise so I thought I’d pass them on to you. Both these reviews are by statisticians / data scientists who work here in NYC in the non-academic “real world” so in that sense they are perhaps better situated than me to review the book (also, they have not collaborated with Nate so they have no conflict of interest). Kaiser Fung gives a positive review : It is in the subtitle—“why so many predictions fail – but some don’t”—that one learns the core philosophy of Silver: he is most concerned with the honest evaluation of the performance of predictive models. The failure to look
sentIndex sentText sentNum sentScore
1 I’m so used to getting books for free that I wouldn’t go out and buy a book just for the purpose of reviewing it. [sent-2, score-0.206]
2 Both these reviews are by statisticians / data scientists who work here in NYC in the non-academic “real world” so in that sense they are perhaps better situated than me to review the book (also, they have not collaborated with Nate so they have no conflict of interest). [sent-5, score-0.341]
3 Science reporters and authors keep bombarding us with stories of success in data mining, when in fact most statistical models in the social sciences have high rates of error. [sent-8, score-0.235]
4 For example, one of the subheads in the chapter about a baseball player performance forecasting system he developed prior to entering the world of political polls reads: “PECOTA Versus Scouts: Scouts Win. [sent-14, score-0.283]
5 After reading Silver’s book, you should be thinking critically about how predictions are evaluated (and in some cases, how they may be impossible to verify). [sent-18, score-0.203]
6 Cathy O’Neil is more critical : As a modeler myself, I am extremely concerned about how models affect the public, so the book’s success is wonderful news. [sent-31, score-0.278]
7 It would be reasonable for Silver to tell us about his baseball models . [sent-42, score-0.243]
8 What is not reasonable, however, is for Silver to claim to understand how the financial crisis was a result of a few inaccurate models, and how medical research need only switch from being frequentist to being Bayesian to become more accurate. [sent-49, score-0.287]
9 But each example is actually a success (for the insiders) if you look at a slightly larger picture and understand the incentives inside the system. [sent-62, score-0.198]
10 We didn’t have a financial crisis because of a bad model or a few bad models. [sent-66, score-0.308]
11 We had bad models because of a corrupt and criminally fraudulent financial system. [sent-67, score-0.308]
12 That’s an important distinction, because we could fix a few bad models with a few good mathematicians, but we can’t fix the entire system so easily. [sent-68, score-0.351]
13 ) This assumption generally holds in his experience: poker, baseball, and polling are all arenas in which one’s incentive is to be as accurate as possible. [sent-75, score-0.193]
14 [But] the flaws in these medical models will be hard to combat, because they advance the interests of the insiders: competition among academic researchers to publish and get tenure is fierce, and there are enormous financial incentives for pharmaceutical companies. [sent-84, score-0.456]
15 The bad models are a consequence of misaligned incentives. [sent-90, score-0.192]
16 He spends very little time on the question of how people act inside larger systems, where a given modeler might be more interested in keeping their job or getting a big bonus than in making their model as accurate as possible. [sent-96, score-0.21]
17 This comports with my impression that Nate is a hard worker and an excellent analyst who can get right to the point of whatever he is studying. [sent-120, score-0.279]
18 Kaiser’s review is positive because he’s treating The Signal and the Noise as a pop-statistics book along the lines of Freakonomics, and he (Kaiser) is happy to see Nate’s openness and questioning spirit, allied with solid practical recommendations. [sent-126, score-0.274]
19 Nate considers non-statistical issues in some small cases (for example, when writing about the varying incentives of different groups of weather forecasters) but is, according to Cathy, too accepting of face-value motivations when discussing finance, medicine, and politics. [sent-132, score-0.29]
20 Even while we recognize that people often have strong motivations to make inaccurate predictions when money and reputations are on the line. [sent-134, score-0.196]
wordName wordTfidf (topN-words)
[('silver', 0.524), ('nate', 0.307), ('cathy', 0.176), ('book', 0.158), ('kaiser', 0.13), ('reviews', 0.123), ('baseball', 0.122), ('models', 0.121), ('financial', 0.116), ('analyst', 0.114), ('poker', 0.106), ('modeler', 0.1), ('worker', 0.095), ('incentives', 0.093), ('weather', 0.087), ('polling', 0.079), ('predictions', 0.078), ('insiders', 0.076), ('scouts', 0.076), ('critically', 0.076), ('forecasters', 0.073), ('bad', 0.071), ('hard', 0.07), ('inaccurate', 0.065), ('heroes', 0.065), ('accurate', 0.062), ('system', 0.061), ('review', 0.06), ('chess', 0.058), ('success', 0.057), ('finance', 0.057), ('considers', 0.057), ('keep', 0.057), ('practical', 0.056), ('medical', 0.056), ('ratings', 0.055), ('motivations', 0.053), ('bayesian', 0.052), ('incentive', 0.052), ('chapter', 0.051), ('stock', 0.051), ('crisis', 0.05), ('fix', 0.049), ('science', 0.049), ('world', 0.049), ('thinking', 0.049), ('errors', 0.049), ('inside', 0.048), ('signal', 0.048), ('books', 0.048)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000007 1634 andrew gelman stats-2012-12-21-Two reviews of Nate Silver’s new book, from Kaiser Fung and Cathy O’Neil
Introduction: People keep asking me what I think of Nate’s book, and I keep replying that, as a blogger, I’m spoiled. I’m so used to getting books for free that I wouldn’t go out and buy a book just for the purpose of reviewing it. (That reminds me that I should post reviews of some of those books I’ve received in the mail over the past few months.) I have, however, encountered a couple of reviews of The Signal and the Noise so I thought I’d pass them on to you. Both these reviews are by statisticians / data scientists who work here in NYC in the non-academic “real world” so in that sense they are perhaps better situated than me to review the book (also, they have not collaborated with Nate so they have no conflict of interest). Kaiser Fung gives a positive review : It is in the subtitle—“why so many predictions fail – but some don’t”—that one learns the core philosophy of Silver: he is most concerned with the honest evaluation of the performance of predictive models. The failure to look
2 0.30316135 131 andrew gelman stats-2010-07-07-A note to John
Introduction: Jeff the Productivity Sapper points me to this insulting open letter to Nate Silver written by pollster John Zogby. I’ll go through bits of Zogby’s note line by line. (Conflict of interest warning: I have collaborated with Nate and I blog on his site). Zogby writes: Here is some advice from someone [Zogby] who has been where you [Silver] are today. Sorry, John. (I can call you that, right? Since you’re calling Nate “Nate”?). Yes, you were once the hot pollster. But, no, you were never where Nate is today. Don’t kid yourself. Zogby writes: You [Nate] are hot right now – using an aggregate of other people’s work, you got 49 of 50 states right in 2008. Yes, Nate used other people’s work. That’s what’s called “making use of available data.” Or, to use a more technical term employed in statistics, it’s called “not being an idiot.” Only in the wacky world of polling are you supposed to draw inferences about the U.S.A. using only a single survey organization. I do
3 0.22076593 364 andrew gelman stats-2010-10-22-Politics is not a random walk: Momentum and mean reversion in polling
Introduction: Nate Silver and Justin Wolfers are having a friendly blog-dispute about momentum in political polling. Nate and Justin each make good points but are also missing parts of the picture. These questions relate to my own research so I thought I’d discuss them here. There ain’t no mo’ Nate led off the discussion by writing that pundits are always talking about “momentum” in the polls: Turn on the news or read through much of the analysis put out by some of our friends, and you’re likely to hear a lot of talk about “momentum”: the term is used about 60 times per day by major media outlets in conjunction with articles about polling. When people say a particular candidate has momentum, what they are implying is that present trends are likely to perpetuate themselves into the future. Say, for instance, that a candidate trailed by 10 points in a poll three weeks ago — and now a new poll comes out showing the candidate down by just 5 points. It will frequently be said that this
4 0.21231055 1610 andrew gelman stats-2012-12-06-Yes, checking calibration of probability forecasts is part of Bayesian statistics
Introduction: Yes, checking calibration of probability forecasts is part of Bayesian statistics. At the end of this post are three figures from Chapter 1 of Bayesian Data Analysis illustrating empirical evaluation of forecasts. But first the background. Why am I bringing this up now? It’s because of something Larry Wasserman wrote the other day : One of the striking facts about [baseball/political forecaster Nate Silver's recent] book is the emphasis the Silver places on frequency calibration. . . . Have no doubt about it: Nate Silver is a frequentist. For example, he says: One of the most important tests of a forecast — I would argue that it is the single most important one — is called calibration. Out of all the times you said there was a 40 percent chance of rain, how often did rain actually occur? If over the long run, it really did rain about 40 percent of the time, that means your forecasts were well calibrated. I had some discussion with Larry in the comments section of h
5 0.20698065 987 andrew gelman stats-2011-11-02-How Khan Academy is using Machine Learning to Assess Student Mastery
Introduction: This is sooooo cool. The actual statistical methods they are using are pretty crude, but that’s fine. What’s important is their focus on the important goal. It’s sort of like Bill James or Nate Silver: if you’re using good information, and you’re focused on good questions, then the fancy statistics can come later (or from others). In most educational efforts I know of (including my own), very little is done to target assessments to improvements for individual students. I really like what they’re doing here and it reminds me how I want to figure out how to do something similar in my own teaching and course materials.
6 0.19565985 391 andrew gelman stats-2010-11-03-Some thoughts on election forecasting
8 0.16865382 1695 andrew gelman stats-2013-01-28-Economists argue about Bayes
9 0.16382554 300 andrew gelman stats-2010-09-28-A calibrated Cook gives Dems the edge in Nov, sez Sandy
10 0.15551263 270 andrew gelman stats-2010-09-12-Comparison of forecasts for the 2010 congressional elections
11 0.14314815 665 andrew gelman stats-2011-04-17-Yes, your wish shall be granted (in 25 years)
13 0.13427559 87 andrew gelman stats-2010-06-15-Statistical analysis and visualization of the drug war in Mexico
14 0.13284923 120 andrew gelman stats-2010-06-30-You can’t put Pandora back in the box
15 0.12947291 1832 andrew gelman stats-2013-04-29-The blogroll
16 0.12269197 1625 andrew gelman stats-2012-12-15-“I coach the jumpers here at Boise State . . .”
17 0.12221625 1512 andrew gelman stats-2012-09-27-A Non-random Walk Down Campaign Street
18 0.11605486 8 andrew gelman stats-2010-04-28-Advice to help the rich get richer
19 0.11578655 2106 andrew gelman stats-2013-11-19-More on “data science” and “statistics”
20 0.11526429 2255 andrew gelman stats-2014-03-19-How Americans vote
topicId topicWeight
[(0, 0.263), (1, -0.021), (2, -0.043), (3, 0.061), (4, -0.05), (5, 0.005), (6, -0.025), (7, 0.006), (8, 0.082), (9, 0.037), (10, 0.02), (11, -0.029), (12, 0.001), (13, -0.065), (14, -0.054), (15, -0.012), (16, -0.045), (17, 0.104), (18, 0.1), (19, -0.042), (20, -0.028), (21, 0.095), (22, -0.043), (23, 0.069), (24, 0.022), (25, 0.019), (26, 0.077), (27, 0.016), (28, 0.018), (29, 0.038), (30, -0.117), (31, -0.033), (32, 0.006), (33, -0.003), (34, 0.069), (35, 0.09), (36, 0.018), (37, -0.034), (38, 0.041), (39, -0.073), (40, -0.038), (41, 0.079), (42, -0.053), (43, -0.003), (44, -0.066), (45, 0.006), (46, 0.003), (47, 0.016), (48, 0.041), (49, 0.013)]
simIndex simValue blogId blogTitle
same-blog 1 0.95439708 1634 andrew gelman stats-2012-12-21-Two reviews of Nate Silver’s new book, from Kaiser Fung and Cathy O’Neil
Introduction: People keep asking me what I think of Nate’s book, and I keep replying that, as a blogger, I’m spoiled. I’m so used to getting books for free that I wouldn’t go out and buy a book just for the purpose of reviewing it. (That reminds me that I should post reviews of some of those books I’ve received in the mail over the past few months.) I have, however, encountered a couple of reviews of The Signal and the Noise so I thought I’d pass them on to you. Both these reviews are by statisticians / data scientists who work here in NYC in the non-academic “real world” so in that sense they are perhaps better situated than me to review the book (also, they have not collaborated with Nate so they have no conflict of interest). Kaiser Fung gives a positive review : It is in the subtitle—“why so many predictions fail – but some don’t”—that one learns the core philosophy of Silver: he is most concerned with the honest evaluation of the performance of predictive models. The failure to look
2 0.73990363 1783 andrew gelman stats-2013-03-31-He’s getting ready to write a book
Introduction: Eric Novik does some open-source planning : My co-author, Jacki Buros, and I [Novik] have just signed a contract with Apress to write a book tentatively entitled “Predictive Analytics with R”, which will cover programming best practices, data munging, data exploration, and single and multi-level models with case studies in social media, healthcare, politics, marketing, and the stock market. Why does the world need another R book? We think there is a shortage of books that deal with the complete and programmer centric analysis of real, dirty, and sometimes unstructured data. Our target audience are people who have some familiarity with statistics, but do not have much experience with programming. . . . The book is projected to be about 300 pages across 8 chapters. This is my first experience with writing a book and everything I heard about the process tells me that this is going to be a long and arduous endeavor lasting anywhere from 6 to 8 months. Novik emailed me and wrot
3 0.73229784 1927 andrew gelman stats-2013-07-05-“Numbersense: How to use big data to your advantage”
Introduction: Business statistician Kaiser Fung just came out with another book, this one full of stories about how organizations use data: 1. Why do law school deans send each other junk mail? 2. Can a new statistic make us less fat? 3. How can sellouts ruin a business? 4. Will personalizing deals save Groupon? 5. Why do marketers send you mixed messages? 6. Are they new jobs if no one can apply? 7. How much did you pay for the eggs? 8. Are you a better coach or manager? Unlike most books of this sort, there’s no hero: These are not stories about a fabulous businessman who made millions of dollars by following his dream and taking the customer seriously, nor are they Gladwellian sagas of brilliant scientists, nor are they auto-Gladwellian tales of the Ariely variety. In some ways, the stories in Fung’s book have the form of opened-up business reporting, in which you get to see the statistical models underlying various assumptions and conclusions. In that sense, this b
4 0.72645772 364 andrew gelman stats-2010-10-22-Politics is not a random walk: Momentum and mean reversion in polling
Introduction: Nate Silver and Justin Wolfers are having a friendly blog-dispute about momentum in political polling. Nate and Justin each make good points but are also missing parts of the picture. These questions relate to my own research so I thought I’d discuss them here. There ain’t no mo’ Nate led off the discussion by writing that pundits are always talking about “momentum” in the polls: Turn on the news or read through much of the analysis put out by some of our friends, and you’re likely to hear a lot of talk about “momentum”: the term is used about 60 times per day by major media outlets in conjunction with articles about polling. When people say a particular candidate has momentum, what they are implying is that present trends are likely to perpetuate themselves into the future. Say, for instance, that a candidate trailed by 10 points in a poll three weeks ago — and now a new poll comes out showing the candidate down by just 5 points. It will frequently be said that this
5 0.7248317 11 andrew gelman stats-2010-04-29-Auto-Gladwell, or Can fractals be used to predict human history?
Introduction: I just reviewed the book Bursts, by Albert-László Barabási, for Physics Today. But I had a lot more to say that couldn’t fit into the magazine’s 800-word limit. Here I’ll reproduce what I sent to Physics Today, followed by my additional thoughts. The back cover of Bursts book promises “a revolutionary new theory showing how we can predict human behavior.” I wasn’t fully convinced on that score, but the book does offer a well-written and thought-provoking window into author Albert-László Barabási’s research in power laws and network theory. Power laws–the mathematical pattern that little things are common and large things are rare–have been observed in many different domains, including incomes (as noted by economist Vilfredo Pareto in the nineteenth century), word frequencies (as noted by linguist George Zipf), city sizes, earthquakes, and virtually anything else that can be measured. In the mid-twentieth century, the mathematician Benoit Mandelbrot devoted an influential caree
6 0.72423035 2297 andrew gelman stats-2014-04-20-Fooled by randomness
7 0.7216301 131 andrew gelman stats-2010-07-07-A note to John
8 0.66889858 1625 andrew gelman stats-2012-12-15-“I coach the jumpers here at Boise State . . .”
9 0.66523838 2189 andrew gelman stats-2014-01-28-History is too important to be left to the history professors
10 0.66470832 1570 andrew gelman stats-2012-11-08-Poll aggregation and election forecasting
11 0.66330063 102 andrew gelman stats-2010-06-21-Why modern art is all in the mind
12 0.66243517 203 andrew gelman stats-2010-08-12-John McPhee, the Anti-Malcolm
13 0.65983903 893 andrew gelman stats-2011-09-06-Julian Symons on Frances Newman
14 0.64774048 16 andrew gelman stats-2010-05-04-Burgess on Kipling
15 0.64657891 391 andrew gelman stats-2010-11-03-Some thoughts on election forecasting
16 0.64467752 285 andrew gelman stats-2010-09-18-Fiction is not for tirades? Tell that to Saul Bellow!
17 0.64399642 2181 andrew gelman stats-2014-01-21-The Commissar for Traffic presents the latest Five-Year Plan
18 0.64212888 2327 andrew gelman stats-2014-05-09-Nicholas Wade and the paradox of racism
19 0.64106238 1303 andrew gelman stats-2012-05-06-I’m skeptical about this skeptical article about left-handedness
20 0.64007807 2251 andrew gelman stats-2014-03-17-In the best alternative histories, the real world is what’s ultimately real
topicId topicWeight
[(5, 0.07), (9, 0.023), (14, 0.046), (15, 0.047), (16, 0.076), (21, 0.026), (22, 0.01), (24, 0.117), (27, 0.012), (34, 0.019), (42, 0.019), (65, 0.017), (72, 0.017), (86, 0.032), (99, 0.27)]
simIndex simValue blogId blogTitle
same-blog 1 0.97113842 1634 andrew gelman stats-2012-12-21-Two reviews of Nate Silver’s new book, from Kaiser Fung and Cathy O’Neil
Introduction: People keep asking me what I think of Nate’s book, and I keep replying that, as a blogger, I’m spoiled. I’m so used to getting books for free that I wouldn’t go out and buy a book just for the purpose of reviewing it. (That reminds me that I should post reviews of some of those books I’ve received in the mail over the past few months.) I have, however, encountered a couple of reviews of The Signal and the Noise so I thought I’d pass them on to you. Both these reviews are by statisticians / data scientists who work here in NYC in the non-academic “real world” so in that sense they are perhaps better situated than me to review the book (also, they have not collaborated with Nate so they have no conflict of interest). Kaiser Fung gives a positive review : It is in the subtitle—“why so many predictions fail – but some don’t”—that one learns the core philosophy of Silver: he is most concerned with the honest evaluation of the performance of predictive models. The failure to look
Introduction: Jonathan Robinson writes: I’m a survey researcher who mostly does political work, but I also have a strong interest in economics. I have a question about this graph you commonly see in the economics literature. It is of a concept called the Beveridge Curve [recently in the newspaper here ]. It is one of the more interesting concepts in labor economics, relating the vacancy rate in jobs to the unemployment rate. A good primer is here . However, despite being one of the more interesting concepts in economics, the way it is displayed visually is nothing short of atrocious: These graphs are nothing short of unreadable and pretty much the standard (Brad Delong has linked to this graph above and it can appear like this in publication as well). I’ve only really seen one representation of the curve that is more clear than this and it is at this link : Do you have any ideas of any way of making these graphs more readable? I like the second Cleveland Fed graph, but I ha
Introduction: Economist Stan Liebowitz has a longstanding interest in the difficulties of flagging published research errors. Recently he wrote on the related topic of dishonest authorship: While not about direct research fraud, I thought you might be interested in this paper . It discusses the manner in which credit is given for economics articles, and I suspect it applies to many other areas as well. One of the conclusions is that the lack of complete proration per author will lead to excessive coauthorship, reducing overall research output by inducing the use of larger than efficient-sized teams. Under these circumstances, false authorship can be a response to the warped reward system and false authorship might improve research efficiency since it might keep actual research teams (as opposed to nominal teams) from being too large to produce research efficiently. One of the questions I rhetorically ask in the paper is whether anyone has ever been ‘punished’ for having their name included on
4 0.96038854 391 andrew gelman stats-2010-11-03-Some thoughts on election forecasting
Introduction: I’ve written a lot on polls and elections (“a poll is a snapshot, not a forecast,” etc., or see here for a more technical paper with Kari Lock) but had a few things to add in light of Sam Wang’s recent efforts . As a biologist with a physics degree, Wang brings an outsider’s perspective to political forecasting, which can be a good thing. (I’m a bit of an outsider to political science myself, as is my sometime collaborator Nate Silver, who’s done a lot of good work in the past few years.) But there are two places where Wang misses the point, I think. He refers to his method as a “transparent, low-assumption calculation” and compares it favorably to “fancy modeling” and “assumption-laden models.” Assumptions are a bad thing, right? Well, no, I don’t think so. Bad assumptions are a bad thing. Good assumptions are just fine. Similarly for fancy modeling. I don’t see why a model should get credit for not including a factor that might be important. Let me clarify. I
5 0.95863974 120 andrew gelman stats-2010-06-30-You can’t put Pandora back in the box
Introduction: Rajiv Sethi writes : I suspect that within a decade, blogs will be a cornerstone of research in economics. Many original and creative contributions to the discipline will first be communicated to the profession (and the world at large) in the form of blog posts, since the medium allows for material of arbitrary length, depth and complexity. Ideas first expressed in this form will make their way (with suitable attribution) into reading lists, doctoral dissertations and more conventionally refereed academic publications. And blogs will come to play a central role in the process of recruitment, promotion and reward at major research universities. This genie is not going back into its bottle. And he thinks this is a good thing: In fact, the refereeing process for blog posts is in some respects more rigorous than that for journal articles. Reports are numerous, non-anonymous, public, rapidly and efficiently produced, and collaboratively constructed. It is not obvious to me [Sethi]
6 0.95795017 431 andrew gelman stats-2010-11-26-One fun thing about physicists . . .
7 0.95646822 1610 andrew gelman stats-2012-12-06-Yes, checking calibration of probability forecasts is part of Bayesian statistics
8 0.95501816 131 andrew gelman stats-2010-07-07-A note to John
9 0.95463574 167 andrew gelman stats-2010-07-27-Why don’t more medical discoveries become cures?
10 0.95452189 210 andrew gelman stats-2010-08-16-What I learned from those tough 538 commenters
11 0.95345974 2137 andrew gelman stats-2013-12-17-Replication backlash
12 0.95344514 1774 andrew gelman stats-2013-03-22-Likelihood Ratio ≠ 1 Journal
14 0.95319325 1435 andrew gelman stats-2012-07-30-Retracted articles and unethical behavior in economics journals?
15 0.95217526 2227 andrew gelman stats-2014-02-27-“What Can we Learn from the Many Labs Replication Project?”
16 0.95203376 1713 andrew gelman stats-2013-02-08-P-values and statistical practice
17 0.95149577 61 andrew gelman stats-2010-05-31-A data visualization manifesto
18 0.95105672 245 andrew gelman stats-2010-08-31-Predicting marathon times
19 0.95068061 1696 andrew gelman stats-2013-01-29-The latest in economics exceptionalism
20 0.95045733 2177 andrew gelman stats-2014-01-19-“The British amateur who debunked the mathematics of happiness”