andrew_gelman_stats andrew_gelman_stats-2014 andrew_gelman_stats-2014-2297 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: From 2006 : Naseem Taleb ‘s publisher sent me a copy of “Fooled by randomness: the hidden role of chance in life and the markets” to review. It’s an important topic, and the book is written in a charming style—I’ll try to respond in kind, with some miscellaneous comments. On the cover of the book is a blurb, “Named by Fortune one of the smartest books of all time.” But Taleb instructs us on page 161-162 to ignore book reviews because of selection bias (the mediocre reviews don’t make it to the book cover). Books vs. articles I prefer writing books to writing journal articles because books are written for the reader (and also, in the case of textbooks, for the teacher), whereas articles are written for referees. Taleb definitely seems to be writing to the reader, not the referee. There is risk in book-writing, since in some ways referees are the ideal audience of experts, but I enjoy the freedom in book-writing of being able to say what I really think. Variation and rando
sentIndex sentText sentNum sentScore
1 It’s an important topic, and the book is written in a charming style—I’ll try to respond in kind, with some miscellaneous comments. [sent-2, score-0.189]
2 On the cover of the book is a blurb, “Named by Fortune one of the smartest books of all time. [sent-3, score-0.282]
3 ” But Taleb instructs us on page 161-162 to ignore book reviews because of selection bias (the mediocre reviews don’t make it to the book cover). [sent-4, score-0.414]
4 articles I prefer writing books to writing journal articles because books are written for the reader (and also, in the case of textbooks, for the teacher), whereas articles are written for referees. [sent-6, score-0.647]
5 Variation and randomness Taleb’s general points—about variation, randomness, and selection bias—will be familiar with statisticians and also to readers of social scientists and biologists such as Niall Ferguson, A. [sent-9, score-0.169]
6 He writes, “As an empiricist (actually a skeptical empiricist) I despise the moralizers beyond anything on this planet . [sent-16, score-0.393]
7 ) Turing Test On page 72, Taleb writes about the Turing test: “A computer can be said to be intelligent if it can (on aveage) fool a human into mistaking it for another human. [sent-27, score-0.145]
8 If you think I can be fooled easily, don’t use me as a judge, either. [sent-33, score-0.169]
9 People have written books about this, “The Keys to the White House,” etc. [sent-37, score-0.198]
10 Anyway, the past 50 years have seen four Presidential elections that have been, essentially (from any forecasting standpoint), ties: 1960, 1968, 1976, 2000. [sent-38, score-0.358]
11 Any forecasting method should get no credit for forecasting the winner in any of these elections, and no blame for getting it wrong. [sent-39, score-0.268]
12 Also in the past 50 years, there have been four Presidential elections that were landslides: 1956, 1964, 1972, 1984. [sent-40, score-0.088]
13 ) Any forecasting method better get these right, otherwise it’s not to be taken seriously at all. [sent-42, score-0.134]
14 Lotteries I once talked with someone who wanted to write a book called Winners, interviewing a bunch of lottery winners. [sent-46, score-0.418]
15 Actually Bruce Sacerdote and others have done statistical studies of lottery winners, using the lottery win as a randomly assigned treatment. [sent-47, score-0.32]
16 But my response was to write a book called Losers, interviewing a bunch of randomly-selected lottery players, almost all of which, of course, would be net losers . [sent-48, score-0.509]
17 Setting aside possible fradulence, I think they have a motivation to be certain, because we’re unlikely to follow their advice if they qualify it. [sent-56, score-0.094]
18 In the other direction, academics like me perhaps have a motivation to overstate uncertainty, to avoid the potential loss in reputation from saying something stupid. [sent-57, score-0.094]
19 But in practice, people seem to understate our uncertainty most of the time. [sent-58, score-0.133]
20 (And, yes, I’ve seen horrible statistical consulting in the private sector as well. [sent-70, score-0.137]
wordName wordTfidf (topN-words)
[('taleb', 0.48), ('randomness', 0.169), ('fooled', 0.169), ('despise', 0.163), ('lottery', 0.16), ('forecasting', 0.134), ('empiricist', 0.126), ('books', 0.114), ('book', 0.105), ('planet', 0.104), ('swan', 0.1), ('yahoo', 0.097), ('turing', 0.095), ('motivation', 0.094), ('interviewing', 0.092), ('losers', 0.091), ('mandelbrot', 0.089), ('vision', 0.089), ('elections', 0.088), ('experts', 0.084), ('written', 0.084), ('collective', 0.083), ('dynamics', 0.083), ('finance', 0.081), ('fool', 0.081), ('winners', 0.079), ('interviewed', 0.078), ('variation', 0.077), ('seminar', 0.077), ('consulting', 0.076), ('uncertainty', 0.076), ('years', 0.075), ('sampling', 0.075), ('forecasts', 0.07), ('reviews', 0.07), ('page', 0.064), ('articles', 0.064), ('test', 0.064), ('presidential', 0.064), ('cover', 0.063), ('black', 0.062), ('called', 0.061), ('seen', 0.061), ('reader', 0.059), ('maxim', 0.057), ('sacerdote', 0.057), ('hayek', 0.057), ('understate', 0.057), ('pitfall', 0.057), ('hyperbole', 0.057)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 2297 andrew gelman stats-2014-04-20-Fooled by randomness
Introduction: From 2006 : Naseem Taleb ‘s publisher sent me a copy of “Fooled by randomness: the hidden role of chance in life and the markets” to review. It’s an important topic, and the book is written in a charming style—I’ll try to respond in kind, with some miscellaneous comments. On the cover of the book is a blurb, “Named by Fortune one of the smartest books of all time.” But Taleb instructs us on page 161-162 to ignore book reviews because of selection bias (the mediocre reviews don’t make it to the book cover). Books vs. articles I prefer writing books to writing journal articles because books are written for the reader (and also, in the case of textbooks, for the teacher), whereas articles are written for referees. Taleb definitely seems to be writing to the reader, not the referee. There is risk in book-writing, since in some ways referees are the ideal audience of experts, but I enjoy the freedom in book-writing of being able to say what I really think. Variation and rando
2 0.35732469 392 andrew gelman stats-2010-11-03-Taleb + 3.5 years
Introduction: I recently had the occasion to reread my review of The Black Swan, from April 2007. It was fun reading my review (and also this pre-review ; “nothing useful escapes from a blackbody,” indeed). It was like a greatest hits of all my pet ideas that I’ve never published. Looking back, I realize that Taleb really was right about a lot of things. Now that the financial crisis has happened, we tend to forget that the experts who Taleb bashes were not always reasonable at all. Here’s what I wrote in my review, three and a half years ago: On page 19, Taleb refers to the usual investment strategy (which I suppose I actually use myself) as “picking pennies in front of a steamroller.” That’s a cute phrase; did he come up with it? I’m also reminded of the famous Martingale betting system. Several years ago in a university library I came across a charming book by Maxim (of gun fame) where he went through chapter after chapter demolishing the Martingale system. (For those who don’t kno
3 0.31380039 278 andrew gelman stats-2010-09-15-Advice that might make sense for individuals but is negative-sum overall
Introduction: There’s a lot of free advice out there. As I wrote a couple years ago, it’s usually presented as advice to individuals, but it’s also interesting to consider the possible total effects if the advice is taken. For example, Nassim Taleb has a webpage that includes a bunch of one-line bits of advice (scroll to item 132 on the linked page). Here’s his final piece of advice: If you dislike someone, leave him alone or eliminate him; don’t attack him verbally. I’m a big Taleb fan (search this blog to see), but this seems like classic negative-sum advice. I can see how it can be a good individual strategy to keep your mouth shut, bide your time, and then sandbag your enemies. But it can’t be good if lots of people are doing this. Verbal attacks are great, as long as there’s a chance to respond. I’ve been in environments where people follow Taleb’s advice, saying nothing and occasionally trying to “eliminate” people, and it’s not pretty. I much prefer for people to be open
4 0.17704812 719 andrew gelman stats-2011-05-19-Everything is Obvious (once you know the answer)
Introduction: Duncan Watts gave his new book the above title, reflecting his irritation with those annoying people who, upon hearing of the latest social science research, reply with: Duh-I-knew-that. (I don’t know how to say Duh in Australian; maybe someone can translate that for me?) I, like Duncan, am easily irritated, and I looked forward to reading the book. I enjoyed it a lot, even though it has only one graph, and that graph has a problem with its y-axis. (OK, the book also has two diagrams and a graph of fake data, but that doesn’t count.) Before going on, let me say that I agree wholeheartedly with Duncan’s central point: social science research findings are often surprising, but the best results cause us to rethink our world in such a way that they seem completely obvious, in retrospect. (Don Rubin used to tell us that there’s no such thing as a “paradox”: once you fully understand a phenomenon, it should not seem paradoxical any more. When learning science, we sometimes speak
5 0.14649287 2024 andrew gelman stats-2013-09-15-Swiss Jonah Lehrer update
Introduction: Nassim Taleb adds this link to the Dobelli story . I’m confused. I thought Swiss dudes were supposed to plagiarize their own stuff, not rip off other people’s. Whassup with that?
6 0.1185123 2255 andrew gelman stats-2014-03-19-How Americans vote
7 0.11393527 2245 andrew gelman stats-2014-03-12-More on publishing in journals
8 0.11308144 1453 andrew gelman stats-2012-08-10-Quotes from me!
9 0.11124749 1628 andrew gelman stats-2012-12-17-Statistics in a world where nothing is random
10 0.10731962 2287 andrew gelman stats-2014-04-09-Advice: positive-sum, zero-sum, or negative-sum
12 0.10682616 8 andrew gelman stats-2010-04-28-Advice to help the rich get richer
13 0.10571658 978 andrew gelman stats-2011-10-28-Cool job opening with brilliant researchers at Yahoo
14 0.10535824 1605 andrew gelman stats-2012-12-04-Write This Book
15 0.10359599 391 andrew gelman stats-2010-11-03-Some thoughts on election forecasting
16 0.10347099 499 andrew gelman stats-2011-01-03-5 books
17 0.10314979 1625 andrew gelman stats-2012-12-15-“I coach the jumpers here at Boise State . . .”
19 0.10212797 1634 andrew gelman stats-2012-12-21-Two reviews of Nate Silver’s new book, from Kaiser Fung and Cathy O’Neil
20 0.099804185 2021 andrew gelman stats-2013-09-13-Swiss Jonah Lehrer
topicId topicWeight
[(0, 0.233), (1, -0.078), (2, -0.035), (3, 0.05), (4, -0.009), (5, 0.046), (6, 0.031), (7, -0.001), (8, 0.07), (9, -0.001), (10, 0.021), (11, -0.043), (12, 0.022), (13, -0.027), (14, 0.007), (15, -0.054), (16, 0.0), (17, -0.004), (18, 0.066), (19, -0.049), (20, -0.054), (21, 0.047), (22, 0.027), (23, 0.078), (24, -0.002), (25, 0.023), (26, 0.026), (27, 0.039), (28, 0.003), (29, 0.022), (30, -0.063), (31, 0.009), (32, -0.013), (33, -0.018), (34, 0.011), (35, -0.004), (36, 0.026), (37, -0.008), (38, 0.007), (39, -0.025), (40, -0.004), (41, 0.002), (42, -0.017), (43, 0.026), (44, -0.021), (45, -0.044), (46, 0.001), (47, -0.028), (48, -0.026), (49, 0.005)]
simIndex simValue blogId blogTitle
same-blog 1 0.96726209 2297 andrew gelman stats-2014-04-20-Fooled by randomness
Introduction: From 2006 : Naseem Taleb ‘s publisher sent me a copy of “Fooled by randomness: the hidden role of chance in life and the markets” to review. It’s an important topic, and the book is written in a charming style—I’ll try to respond in kind, with some miscellaneous comments. On the cover of the book is a blurb, “Named by Fortune one of the smartest books of all time.” But Taleb instructs us on page 161-162 to ignore book reviews because of selection bias (the mediocre reviews don’t make it to the book cover). Books vs. articles I prefer writing books to writing journal articles because books are written for the reader (and also, in the case of textbooks, for the teacher), whereas articles are written for referees. Taleb definitely seems to be writing to the reader, not the referee. There is risk in book-writing, since in some ways referees are the ideal audience of experts, but I enjoy the freedom in book-writing of being able to say what I really think. Variation and rando
2 0.89388037 392 andrew gelman stats-2010-11-03-Taleb + 3.5 years
Introduction: I recently had the occasion to reread my review of The Black Swan, from April 2007. It was fun reading my review (and also this pre-review ; “nothing useful escapes from a blackbody,” indeed). It was like a greatest hits of all my pet ideas that I’ve never published. Looking back, I realize that Taleb really was right about a lot of things. Now that the financial crisis has happened, we tend to forget that the experts who Taleb bashes were not always reasonable at all. Here’s what I wrote in my review, three and a half years ago: On page 19, Taleb refers to the usual investment strategy (which I suppose I actually use myself) as “picking pennies in front of a steamroller.” That’s a cute phrase; did he come up with it? I’m also reminded of the famous Martingale betting system. Several years ago in a university library I came across a charming book by Maxim (of gun fame) where he went through chapter after chapter demolishing the Martingale system. (For those who don’t kno
3 0.83856189 4 andrew gelman stats-2010-04-26-Prolefeed
Introduction: From Anthony Burgess’s review of “The Batsford Companion to Popular Literature,” by Victor Neuberg: Arthur J. Burks (1898-1974) was no gentleman. During the 1930s, when he would sometimes have nearly two million words in current publication, he aimed at producing 18,000 words a day. Editors would call me up and ask me to do a novelette by the next afternoon, and I would, but it nearly killed me. . . . I once appeared on the covers of eleven magazines the same month, and then almost killed myself for years trying to make it twelve. I never did. [Masanao: I think you know where I'm heading with that story.] Ursula Bloom, born 1985 and still with us [this was written sometime between 1978 and 1985], is clearly no lady. Writing also under the pseudonyms of Lozania Prole (there’s an honest name for you), Sheila Burnes and Mary Essex, she has produced 486 boooks, beginning with Tiger at the age of seven. . . . Was Richard Horatio Edgar Wallace (1875-1932) a gentleman? .
4 0.8281827 203 andrew gelman stats-2010-08-12-John McPhee, the Anti-Malcolm
Introduction: This blog is threatening to turn into Statistical Modeling, Causal Inference, Social Science, and Literature Criticism, but I’m just going to go with the conversational flow, so here’s another post about an essayist. I’m not a big fan of Janet Malcolm’s essays — and I don’t mean I don’t like her attitude or her pro-murderer attitude, I mean I don’t like them all that much as writing. They’re fine, I read them, they don’t bore me, but I certainly don’t think she’s “our” best essayist. But that’s not a debate I want to have right now, and if I did I’m quite sure most of you wouldn’t want to read it anyway. So instead, I’ll just say something about John McPhee. As all right-thinking people agree, in McPhee’s long career he has written two kinds of books: good, short books, and bad, long books. (He has also written many New Yorker essays, and perhaps other essays for other magazines too; most of these are good, although I haven’t seen any really good recent work from him, and so
5 0.82508177 11 andrew gelman stats-2010-04-29-Auto-Gladwell, or Can fractals be used to predict human history?
Introduction: I just reviewed the book Bursts, by Albert-László Barabási, for Physics Today. But I had a lot more to say that couldn’t fit into the magazine’s 800-word limit. Here I’ll reproduce what I sent to Physics Today, followed by my additional thoughts. The back cover of Bursts book promises “a revolutionary new theory showing how we can predict human behavior.” I wasn’t fully convinced on that score, but the book does offer a well-written and thought-provoking window into author Albert-László Barabási’s research in power laws and network theory. Power laws–the mathematical pattern that little things are common and large things are rare–have been observed in many different domains, including incomes (as noted by economist Vilfredo Pareto in the nineteenth century), word frequencies (as noted by linguist George Zipf), city sizes, earthquakes, and virtually anything else that can be measured. In the mid-twentieth century, the mathematician Benoit Mandelbrot devoted an influential caree
6 0.81865579 719 andrew gelman stats-2011-05-19-Everything is Obvious (once you know the answer)
7 0.81039989 2189 andrew gelman stats-2014-01-28-History is too important to be left to the history professors
8 0.80951482 1616 andrew gelman stats-2012-12-10-John McAfee is a Heinlein hero
9 0.80235732 2168 andrew gelman stats-2014-01-12-Things that I like that almost nobody else is interested in
10 0.79734999 1634 andrew gelman stats-2012-12-21-Two reviews of Nate Silver’s new book, from Kaiser Fung and Cathy O’Neil
11 0.7933408 285 andrew gelman stats-2010-09-18-Fiction is not for tirades? Tell that to Saul Bellow!
12 0.78766495 1641 andrew gelman stats-2012-12-27-The Möbius strip, or, marketing that is impervious to criticism
13 0.78717011 115 andrew gelman stats-2010-06-28-Whassup with those crappy thrillers?
15 0.78557056 499 andrew gelman stats-2011-01-03-5 books
16 0.7833184 432 andrew gelman stats-2010-11-27-Neumann update
17 0.78188819 1453 andrew gelman stats-2012-08-10-Quotes from me!
18 0.78128672 258 andrew gelman stats-2010-09-05-A review of a review of a review of a decade
19 0.78080839 57 andrew gelman stats-2010-05-29-Roth and Amsterdam
20 0.77981931 886 andrew gelman stats-2011-09-02-The new Helen DeWitt novel
topicId topicWeight
[(2, 0.039), (15, 0.053), (16, 0.06), (21, 0.029), (24, 0.161), (29, 0.024), (42, 0.015), (43, 0.015), (49, 0.021), (62, 0.011), (63, 0.015), (66, 0.011), (70, 0.014), (77, 0.06), (82, 0.025), (86, 0.036), (89, 0.054), (99, 0.265)]
simIndex simValue blogId blogTitle
same-blog 1 0.97332722 2297 andrew gelman stats-2014-04-20-Fooled by randomness
Introduction: From 2006 : Naseem Taleb ‘s publisher sent me a copy of “Fooled by randomness: the hidden role of chance in life and the markets” to review. It’s an important topic, and the book is written in a charming style—I’ll try to respond in kind, with some miscellaneous comments. On the cover of the book is a blurb, “Named by Fortune one of the smartest books of all time.” But Taleb instructs us on page 161-162 to ignore book reviews because of selection bias (the mediocre reviews don’t make it to the book cover). Books vs. articles I prefer writing books to writing journal articles because books are written for the reader (and also, in the case of textbooks, for the teacher), whereas articles are written for referees. Taleb definitely seems to be writing to the reader, not the referee. There is risk in book-writing, since in some ways referees are the ideal audience of experts, but I enjoy the freedom in book-writing of being able to say what I really think. Variation and rando
2 0.96406209 562 andrew gelman stats-2011-02-06-Statistician cracks Toronto lottery
Introduction: Christian points me to this amusing story by Jonah Lehrer about Mohan Srivastava, (perhaps the same person as R. Mohan Srivastava, coauthor of a book called Applied Geostatistics) who discovered a flaw in a scratch-off game in which he could figure out which tickets were likely to win based on partial information visible on the ticket. It appears that scratch-off lotteries elsewhere have similar flaws in their design. The obvious question is, why doesn’t the lottery create the patterns on the tickets (including which “teaser” numbers to reveal) completely at random? It shouldn’t be hard to design this so that zero information is supplied from the outside. in which case Srivastava’s trick would be impossible. So why not put down the numbers randomly? Lehrer quotes Srivastava as saying: The tickets are clearly mass-produced, which means there must be some computer program that lays down the numbers. Of course, it would be really nice if the computer could just spit out random
Introduction: Jed Dougherty points me to this opinion piece by Jacqueline Stevens, a professor of art at Northwestern University, who writes: Artists are defensive these days because in May the House passed an amendment to a bill eliminating the National Endowment for the Arts. Colleagues, especially those who have received N.E.A. grants, will loathe me for saying this, but just this once I’m sympathetic with the anti-intellectual Republicans behind this amendment. Why? The bill incited a national conversation about a subject that has troubled me for decades: the government — disproportionately — supports art that I do not like. Actually, just about nobody likes modern art. All those soup cans—what’s that all about? The stuff they have in museums nowadays, my 4-year-old could do better than that. Two-thirds of so-called modern artists are drunk and two-thirds are frauds. And, no, I didn’t get my math wrong—there’s just a lot of overlap among these categories! It’s an open secret in my
4 0.95912433 401 andrew gelman stats-2010-11-08-Silly old chi-square!
Introduction: Brian Mulford writes: I [Mulford] ran across this blog post and found myself questioning the relevance of the test used. I’d think Chi-Square would be inappropriate for trying to measure significance of choice in the manner presented here; irrespective of the cute hamster. Since this is a common test for marketers and website developers – I’d be interested in which techniques you might suggest? For tests of this nature, I typically measure a variety of variables (image placement, size, type, page speed, “page feel” as expressed in a factor, etc) and use LOGIT, Cluster and possibly a simple Bayesian model to determine which variables were most significant (chosen). Pearson Chi-squared may be used to express relationships between variables and outcome but I’ve typically not used it to simply judge a 0/1 choice as statistically significant or not. My reply: I like the decision-theoretic way that the blogger (Jason Cohen, according to the webpage) starts: If you wait too
5 0.95692509 1247 andrew gelman stats-2012-04-05-More philosophy of Bayes
Introduction: Konrad Scheffler writes: I was interested by your paper “Induction and deduction in Bayesian data analysis” and was wondering if you would entertain a few questions: – Under the banner of objective Bayesianism, I would posit something like this as a description of Bayesian inference: “Objective Bayesian probability is not a degree of belief (which would necessarily be subjective) but a measure of the plausibility of a hypothesis, conditional on a formally specified information state. One way of specifying a formal information state is to specify a model, which involves specifying both a prior distribution (typically for a set of unobserved variables) and a likelihood function (typically for a set of observed variables, conditioned on the values of the unobserved variables). Bayesian inference involves calculating the objective degree of plausibility of a hypothesis (typically the truth value of the hypothesis is a function of the variables mentioned above) given such a
6 0.95212698 1171 andrew gelman stats-2012-02-16-“False-positive psychology”
7 0.95168775 2089 andrew gelman stats-2013-11-04-Shlemiel the Software Developer and Unknown Unknowns
8 0.95167553 1162 andrew gelman stats-2012-02-11-Adding an error model to a deterministic model
9 0.95150375 1702 andrew gelman stats-2013-02-01-Don’t let your standard errors drive your research agenda
10 0.95074755 1980 andrew gelman stats-2013-08-13-Test scores and grades predict job performance (but maybe not at Google)
11 0.95010579 1438 andrew gelman stats-2012-07-31-What is a Bayesian?
12 0.95003819 2299 andrew gelman stats-2014-04-21-Stan Model of the Week: Hierarchical Modeling of Supernovas
13 0.94907534 1604 andrew gelman stats-2012-12-04-An epithet I can live with
15 0.94767272 2244 andrew gelman stats-2014-03-11-What if I were to stop publishing in journals?
16 0.94703674 231 andrew gelman stats-2010-08-24-Yet another Bayesian job opportunity
17 0.94692457 902 andrew gelman stats-2011-09-12-The importance of style in academic writing
19 0.94619727 207 andrew gelman stats-2010-08-14-Pourquoi Google search est devenu plus raisonnable?
20 0.94571686 1196 andrew gelman stats-2012-03-04-Piss-poor monocausal social science