andrew_gelman_stats andrew_gelman_stats-2013 andrew_gelman_stats-2013-1724 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: A moviegoing colleague writes: I just watched the movie Zero Dark Thirty about the hunt for Osama Bin Laden. What struck me about it was: (1) Bayes theorem underlies the whole movie; (2) CIA top brass do not know Bayes theorem (at least as portrayed in the movie). Obviously one does not need to know physics to play billiards, but it helps with the reasoning. Essentially, at some point the key CIA agent locates what she strongly believes is OBL’s hidding place in Pakistan. Then it takes the White House some 150 days to make the decision to attack the compound. Why so long? And why, even on the eve of the operation, were senior brass only some 60% OBL was there? Fear of false positives is the answer. After all, the compound could belong to a drug lord, or some other terrorist. Here is the math: There are two possibilities, according to movie: OBL is in a compound (C) in a city or he is in the mountains in tribal regions. Say P(OBL in C) = 0.5. A diagnosis is made on
sentIndex sentText sentNum sentScore
1 A moviegoing colleague writes: I just watched the movie Zero Dark Thirty about the hunt for Osama Bin Laden. [sent-1, score-0.295]
2 What struck me about it was: (1) Bayes theorem underlies the whole movie; (2) CIA top brass do not know Bayes theorem (at least as portrayed in the movie). [sent-2, score-0.805]
3 Obviously one does not need to know physics to play billiards, but it helps with the reasoning. [sent-3, score-0.061]
4 Essentially, at some point the key CIA agent locates what she strongly believes is OBL’s hidding place in Pakistan. [sent-4, score-0.1]
5 And why, even on the eve of the operation, were senior brass only some 60% OBL was there? [sent-7, score-0.543]
6 After all, the compound could belong to a drug lord, or some other terrorist. [sent-9, score-0.383]
7 Here is the math: There are two possibilities, according to movie: OBL is in a compound (C) in a city or he is in the mountains in tribal regions. [sent-10, score-0.376]
8 A diagnosis is made on the compound on the basis of X criteria (e. [sent-13, score-0.394]
9 high walls, nobody enters or leaves except courier, etc. [sent-15, score-0.06]
10 Say the “compound test” is positive (+) when all criteria are met and negative otherwise. [sent-18, score-0.078]
11 The CIA agent is very sure this is OBL’s compound. [sent-19, score-0.1]
12 This would seem like a “slam dunk” but what if say P(+|OBL not in C)=0. [sent-22, score-0.073]
13 Then, by Bayes Theorem, P(OB in C|+) is only 60%, which is what the senior brass thinks on the eve of the operation. [sent-24, score-0.543]
14 One can play around with the parameters, but I draw three observations: First, I don’t know if this was the right assessment or not but what I gather from the discussion — as portrayed in the movie — is that top brass did not focus explicitly on debating the arguments to the Bayes formula. [sent-25, score-0.833]
15 The evidence was not structured, but casually spoken in hunches and guesses. [sent-26, score-0.177]
16 Second, it seems the debacle on WMD in Iraq has made the CIA more worried of false positives! [sent-27, score-0.148]
17 Third, given what the film portrays about the compound, I think it belonging to a drug lord was pretty minimal, so I would say P(+|OBL not in C)=0. [sent-28, score-0.434]
18 This suggests the White House was extremely risk averse: it seemed very likely they had OBL and not somebody else yet they sat on it for half a year (at substantial risk to homeland)! [sent-31, score-0.12]
19 (Alternatively, the fear of false positives was getting in the way of the top brass judgement (e. [sent-35, score-0.743]
20 maybe they thought it was 90% but did not want to say it)). [sent-37, score-0.073]
wordName wordTfidf (topN-words)
[('obl', 0.63), ('brass', 0.35), ('compound', 0.256), ('cia', 0.24), ('movie', 0.187), ('positives', 0.154), ('ob', 0.14), ('eve', 0.115), ('bayes', 0.112), ('lord', 0.111), ('theorem', 0.108), ('portrayed', 0.108), ('agent', 0.1), ('false', 0.088), ('fear', 0.08), ('criteria', 0.078), ('senior', 0.078), ('say', 0.073), ('drug', 0.071), ('top', 0.071), ('house', 0.064), ('belonging', 0.064), ('hunches', 0.064), ('wmd', 0.064), ('portrays', 0.064), ('casually', 0.064), ('white', 0.062), ('play', 0.061), ('mountains', 0.06), ('debacle', 0.06), ('enters', 0.06), ('averse', 0.06), ('underlies', 0.06), ('hunt', 0.06), ('diagnosis', 0.06), ('osama', 0.06), ('tribal', 0.06), ('risk', 0.06), ('homeland', 0.058), ('belong', 0.056), ('debating', 0.056), ('walls', 0.054), ('bin', 0.052), ('film', 0.051), ('spoken', 0.049), ('alternatively', 0.049), ('operation', 0.048), ('watched', 0.048), ('thirty', 0.047), ('iraq', 0.046)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 1724 andrew gelman stats-2013-02-16-Zero Dark Thirty and Bayes’ theorem
Introduction: A moviegoing colleague writes: I just watched the movie Zero Dark Thirty about the hunt for Osama Bin Laden. What struck me about it was: (1) Bayes theorem underlies the whole movie; (2) CIA top brass do not know Bayes theorem (at least as portrayed in the movie). Obviously one does not need to know physics to play billiards, but it helps with the reasoning. Essentially, at some point the key CIA agent locates what she strongly believes is OBL’s hidding place in Pakistan. Then it takes the White House some 150 days to make the decision to attack the compound. Why so long? And why, even on the eve of the operation, were senior brass only some 60% OBL was there? Fear of false positives is the answer. After all, the compound could belong to a drug lord, or some other terrorist. Here is the math: There are two possibilities, according to movie: OBL is in a compound (C) in a city or he is in the mountains in tribal regions. Say P(OBL in C) = 0.5. A diagnosis is made on
2 0.091424808 3 andrew gelman stats-2010-04-26-Bayes in the news…in a somewhat frustrating way
Introduction: I’m not sure how the New York Times defines a blog versus an article, so perhaps this post should be called “Bayes in the blogs.” Whatever. A recent NY Times article/blog post discusses a classic Bayes’ Theorem application — probability that the patient has cancer, given a “positive” mammogram — and purports to give a solution that is easy for students to understand because it doesn’t require Bayes’ Theorem, which is of course complicated and confusing. You can see my comment (#17) here.
3 0.063459419 1158 andrew gelman stats-2012-02-07-The more likely it is to be X, the more likely it is to be Not X?
Introduction: This post is by Phil Price. A paper by Wood, Douglas, and Sutton looks at “Beliefs in Contradictory Conspiracy Theories.” Unfortunately the subjects were 140 undergraduate psychology students, so one wonders how general the results are. I found this sort of arresting: In Study 1 (n=137), the more participants believed that Princess Diana faked her own death, the more they believed she was murdered. In Study 2 (n=102), the more participants believed that Osama Bin Laden was already dead when U.S. Special Forces raided his compound in Pakistan, the more they believed he is still alive. As the article says, “conspiracy advocates’ distrust of official narratives may be so strong that many alternative theories are simultaneously endorsed in spite of any contradictions between them.” But I think the authors overstate things when they say “One would think that there ought to be a negative correlation between beliefs in contradictory accounts of events — the more one believes in
4 0.062722437 1973 andrew gelman stats-2013-08-08-For chrissake, just make up an analysis already! We have a lab here to run, y’know?
Introduction: Ben Hyde sends along this : Stuck in the middle of the supplemental data, reporting the total workup for their compounds, was this gem: Emma, please insert NMR data here! where are they? and for this compound, just make up an elemental analysis . . . I’m reminded of our recent discussions of coauthorship, where I argued that I see real advantages to having multiple people taking responsibility for the result. Jay Verkuilen responded: “On the flipside of collaboration . . . is diffusion of responsibility, where everybody thinks someone else ‘has that problem’ and thus things don’t get solved.” That’s what seems to have happened (hilariously) here.
5 0.06257049 428 andrew gelman stats-2010-11-24-Flawed visualization of U.S. voting maybe has some good features
Introduction: Aleks points me to this attractive visualization by David Sparks of U.S. voting. On the plus side, the pictures and associated movie (showing an oddly horizontally-stretched-out United States) are pretty and seem to have gotten a bit of attention–the maps have received 31 comments, which is more than we get on almost all our blog entries here. On the minus side, the movie is misleading. In many years it shows the whole U.S. as a single color, even when candidates from both parties won some votes. The text has errors too, for example the false claim that the South favored a Democratic candidate in 1980. The southern states that Jimmy Carter carried in 1980 were Georgia and . . . that’s it. But, as Aleks says, once this tool is out there, maybe people can use it to do better. It’s in that spirit that I’m linking. Ya gotta start somewhere. Also, this is a good example of a general principle: When you make a graph, look at it carefully to see if it makes sense!
6 0.061828401 1970 andrew gelman stats-2013-08-06-New words of 1917
7 0.058767181 2369 andrew gelman stats-2014-06-11-“I can’t drive home now. Not just yet. First I need to go to Utrecht.”
8 0.051988356 220 andrew gelman stats-2010-08-20-Why I blog?
9 0.050783902 2093 andrew gelman stats-2013-11-07-I’m negative on the expression “false positives”
10 0.048781626 1945 andrew gelman stats-2013-07-18-“How big is your chance of dying in an ordinary play?”
11 0.047027081 358 andrew gelman stats-2010-10-20-When Kerry Met Sally: Politics and Perceptions in the Demand for Movies
12 0.046354562 38 andrew gelman stats-2010-05-18-Breastfeeding, infant hyperbilirubinemia, statistical graphics, and modern medicine
13 0.045872428 1883 andrew gelman stats-2013-06-04-Interrogating p-values
14 0.045607273 1289 andrew gelman stats-2012-04-29-We go to war with the data we have, not the data we want
15 0.045278545 1171 andrew gelman stats-2012-02-16-“False-positive psychology”
16 0.04449553 2197 andrew gelman stats-2014-02-04-Peabody here.
17 0.044134524 158 andrew gelman stats-2010-07-22-Tenants and landlords
18 0.043849502 1575 andrew gelman stats-2012-11-12-Thinking like a statistician (continuously) rather than like a civilian (discretely)
19 0.043385945 2155 andrew gelman stats-2013-12-31-No on Yes-No decisions
20 0.041321941 949 andrew gelman stats-2011-10-10-Grrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
topicId topicWeight
[(0, 0.07), (1, -0.019), (2, 0.005), (3, -0.002), (4, -0.015), (5, -0.01), (6, 0.031), (7, 0.006), (8, 0.01), (9, -0.028), (10, -0.03), (11, 0.016), (12, -0.002), (13, -0.016), (14, 0.003), (15, 0.004), (16, 0.022), (17, 0.0), (18, 0.003), (19, -0.011), (20, -0.016), (21, 0.015), (22, -0.005), (23, 0.005), (24, -0.0), (25, -0.012), (26, -0.019), (27, 0.022), (28, 0.007), (29, 0.008), (30, 0.011), (31, -0.003), (32, 0.014), (33, -0.01), (34, 0.008), (35, -0.037), (36, 0.002), (37, 0.012), (38, -0.005), (39, 0.008), (40, -0.037), (41, 0.002), (42, -0.003), (43, 0.005), (44, -0.01), (45, 0.005), (46, 0.011), (47, 0.009), (48, -0.007), (49, -0.014)]
simIndex simValue blogId blogTitle
same-blog 1 0.93795115 1724 andrew gelman stats-2013-02-16-Zero Dark Thirty and Bayes’ theorem
Introduction: A moviegoing colleague writes: I just watched the movie Zero Dark Thirty about the hunt for Osama Bin Laden. What struck me about it was: (1) Bayes theorem underlies the whole movie; (2) CIA top brass do not know Bayes theorem (at least as portrayed in the movie). Obviously one does not need to know physics to play billiards, but it helps with the reasoning. Essentially, at some point the key CIA agent locates what she strongly believes is OBL’s hidding place in Pakistan. Then it takes the White House some 150 days to make the decision to attack the compound. Why so long? And why, even on the eve of the operation, were senior brass only some 60% OBL was there? Fear of false positives is the answer. After all, the compound could belong to a drug lord, or some other terrorist. Here is the math: There are two possibilities, according to movie: OBL is in a compound (C) in a city or he is in the mountains in tribal regions. Say P(OBL in C) = 0.5. A diagnosis is made on
2 0.72697777 489 andrew gelman stats-2010-12-28-Brow inflation
Introduction: In an article headlined, “Hollywood moves away from middlebrow,” Brooks Barnes writes : As Hollywood plowed into 2010, there was plenty of clinging to the tried and true: humdrum remakes like “The Wolfman” and “The A-Team”; star vehicles like “Killers” with Ashton Kutcher and “The Tourist” with Angelina Jolie and Johnny Depp; and shoddy sequels like “Sex and the City 2.” All arrived at theaters with marketing thunder intended to fill multiplexes on opening weekend, no matter the quality of the film. . . . But the audience pushed back. One by one, these expensive yet middle-of-the-road pictures delivered disappointing results or flat-out flopped. Meanwhile, gambles on original concepts paid off. “Inception,” a complicated thriller about dream invaders, racked up more than $825 million in global ticket sales; “The Social Network” has so far delivered $192 million, a stellar result for a highbrow drama. . . . the message that the year sent about quality and originality is real enoug
3 0.72329372 1187 andrew gelman stats-2012-02-27-“Apple confronts the law of large numbers” . . . huh?
Introduction: I was reading this news article by famed business reporter James Stewart: Measured by market capitalization, Apple is the world’s biggest public company. . . . Sales for the quarter that ended Dec. 31 . . . totaled $46.33 billion, up 73 percent from the year before. Earnings more than doubled. . . . Here is the rub: Apple is so big, it’s running up against the law of large numbers. Huh? At this point I sat up, curious. Stewart continued: Also known as the golden theorem, with a proof attributed to the 17th-century Swiss mathematician Jacob Bernoulli, the law states that a variable will revert to a mean over a large sample of results. In the case of the largest companies, it suggests that high earnings growth and a rapid rise in share price will slow as those companies grow ever larger. If Apple’s share price grew even 20 percent a year for the next decade, which is far below its current blistering pace, its $500 billion market capitalization would be more than $3 tri
4 0.71949685 1058 andrew gelman stats-2011-12-14-Higgs bozos: Rosencrantz and Guildenstern are spinning in their graves
Introduction: David Hogg sends in this bizarre bit of news reporting by Robert Evans: Until now, in the four decades since it was first posited, no one has convincingly claimed to have glimpsed the Higgs Boson, let alone proved that it actually exists. At an eagerly awaited briefing on Tuesday at the CERN research centre near Geneva, two independent teams of “Higgs Hunters” – a term they themselves hate – were widely expected to suggest they were fairly confident they had spotted it. But not confident enough, in the physics world of ultra-precision where certainty has to be measured at nothing less than 100 percent, to announce “a discovery.” In the jargon, this level is described as 5 sigma . . . So far, so good. But then comes this doozy: As one scientist explained, that level of accuracy would equate to the 17th-century discoverer of gravity, Isaac Newton, sitting under his apple tree and a million apples one after another falling on his head without one missing. Huh? A free
5 0.71811658 731 andrew gelman stats-2011-05-26-Lottery probability update
Introduction: It was reported last year that the national lottery of Israel featured the exact same 6 numbers (out of 45) twice in the same month, and statistics professor Isaac Meilijson of Tel Aviv University was quoted as saying that “the incident of six numbers repeating themselves within a month is an event of once in 10,000 years.” I shouldn’t mock when it comes to mathematics–after all, I proved a false theorem once! (Or, to be precise, my collaborator and I published a false claim which we thought we’d proved, thus we thought was a theorem.) So let me retract the mockery and move, first to the mathematics and then to the statistics. First, how many possibilities are there in pick 6 out of 45? It’s (45*44*43*42*41*40)/6! = 8,145,060. Let’s call this number N. Second, what’s the probability that the same numbers repeat in a single calendar month? I’ve been told that the Israeli lottery has 2 draws per week, That’s 104/12=8.67 draws per month. Or maybe they skip some holiday
7 0.71463704 17 andrew gelman stats-2010-05-05-Taking philosophical arguments literally
8 0.71173161 2270 andrew gelman stats-2014-03-28-Creating a Lenin-style democracy
9 0.70823961 68 andrew gelman stats-2010-06-03-…pretty soon you’re talking real money.
10 0.6989187 1789 andrew gelman stats-2013-04-05-Elites have alcohol problems too!
11 0.6946032 2119 andrew gelman stats-2013-12-01-Separated by a common blah blah blah
12 0.68492007 1245 andrew gelman stats-2012-04-03-Redundancy and efficiency: In praise of Penn Station
13 0.68409926 2053 andrew gelman stats-2013-10-06-Ideas that spread fast and slow
14 0.68003523 158 andrew gelman stats-2010-07-22-Tenants and landlords
15 0.6799897 1646 andrew gelman stats-2013-01-01-Back when fifty years was a long time ago
16 0.67988992 1621 andrew gelman stats-2012-12-13-Puzzles of criminal justice
17 0.67644161 2341 andrew gelman stats-2014-05-20-plus ça change, plus c’est la même chose
18 0.6758725 284 andrew gelman stats-2010-09-18-Continuing efforts to justify false “death panels” claim
19 0.67079997 2352 andrew gelman stats-2014-05-29-When you believe in things that you don’t understand
20 0.66829675 1342 andrew gelman stats-2012-05-24-The Used TV Price is Too Damn High
topicId topicWeight
[(2, 0.05), (5, 0.031), (14, 0.271), (16, 0.072), (24, 0.104), (27, 0.014), (56, 0.015), (61, 0.011), (63, 0.015), (77, 0.021), (86, 0.021), (95, 0.012), (99, 0.155)]
simIndex simValue blogId blogTitle
same-blog 1 0.86958015 1724 andrew gelman stats-2013-02-16-Zero Dark Thirty and Bayes’ theorem
Introduction: A moviegoing colleague writes: I just watched the movie Zero Dark Thirty about the hunt for Osama Bin Laden. What struck me about it was: (1) Bayes theorem underlies the whole movie; (2) CIA top brass do not know Bayes theorem (at least as portrayed in the movie). Obviously one does not need to know physics to play billiards, but it helps with the reasoning. Essentially, at some point the key CIA agent locates what she strongly believes is OBL’s hidding place in Pakistan. Then it takes the White House some 150 days to make the decision to attack the compound. Why so long? And why, even on the eve of the operation, were senior brass only some 60% OBL was there? Fear of false positives is the answer. After all, the compound could belong to a drug lord, or some other terrorist. Here is the math: There are two possibilities, according to movie: OBL is in a compound (C) in a city or he is in the mountains in tribal regions. Say P(OBL in C) = 0.5. A diagnosis is made on
2 0.86850369 2344 andrew gelman stats-2014-05-23-The gremlins did it? Iffy statistics drive strong policy recommendations
Introduction: Recently in the sister blog . Yet another chapter in the continuing saga, Don’t Trust Polynomials. P.S. More here .
3 0.84489936 1770 andrew gelman stats-2013-03-19-Retraction watch
Introduction: Here (from the Annals of Applied Statistics ). “Thus, arguably, all of Section 3 is wrong until proven otherwise.” As with retractions in general, it makes me wonder about the rest of this guy’s work. Dr. Anil Potti would be pooping in his pants spinning in his retirement .
4 0.82969403 755 andrew gelman stats-2011-06-09-Recently in the award-winning sister blog
Introduction: In case you haven’t been following: - Top ten excuses for plagiarism - Why I won’t be sad to see Anthony Weiner retire - U.S. voter participation has not fallen steadily over the past few decades - Scott Adams had an interesting idea
5 0.80835962 1809 andrew gelman stats-2013-04-17-NUTS discussed on Xi’an’s Og
Introduction: Xi’an’s Og (aka Christian Robert’s blog) is featuring a very nice presentation of NUTS by Marco Banterle, with discussion and some suggestions. I’m not even sure how they found Michael Betancourt’s paper on geometric NUTS — I don’t see it on the arXiv yet, or I’d provide a link.
6 0.78610134 130 andrew gelman stats-2010-07-07-A False Consensus about Public Opinion on Torture
7 0.77159739 1051 andrew gelman stats-2011-12-10-Towards a Theory of Trust in Networks of Humans and Computers
8 0.76702368 824 andrew gelman stats-2011-07-26-Milo and Milo
9 0.74422646 1471 andrew gelman stats-2012-08-27-Why do we never see a full decision analysis for a clinical trial?
11 0.72814429 1696 andrew gelman stats-2013-01-29-The latest in economics exceptionalism
12 0.70540255 245 andrew gelman stats-2010-08-31-Predicting marathon times
13 0.63387203 1303 andrew gelman stats-2012-05-06-I’m skeptical about this skeptical article about left-handedness
14 0.62732321 252 andrew gelman stats-2010-09-02-R needs a good function to make line plots
15 0.62676054 2237 andrew gelman stats-2014-03-08-Disagreeing to disagree
16 0.62522864 1236 andrew gelman stats-2012-03-29-Resolution of Diederik Stapel case
18 0.61079943 2117 andrew gelman stats-2013-11-29-The gradual transition to replicable science
19 0.60705149 2059 andrew gelman stats-2013-10-12-Visualization, “big data”, and EDA
20 0.60345006 2346 andrew gelman stats-2014-05-24-Buzzfeed, Porn, Kansas…That Can’t Be Good