andrew_gelman_stats andrew_gelman_stats-2011 andrew_gelman_stats-2011-661 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Coming back from Chicago we flew right over Manhattan. Very impressive as always, to see all those buildings so densely packed. But think of how impressive it must have seemed in 1950! The world had a lot less of everything back in 1950 (well, we had more oil in the ground, but that’s about it), so Manhattan must have just seemed amazing. I can see how American leaders of that period could’ve been pretty smug. Our #1 city was leading the world by so much, it was decades ahead of its time, still impressive even now after 60 years of decay.
sentIndex sentText sentNum sentScore
1 Coming back from Chicago we flew right over Manhattan. [sent-1, score-0.481]
2 Very impressive as always, to see all those buildings so densely packed. [sent-2, score-1.014]
3 But think of how impressive it must have seemed in 1950! [sent-3, score-0.93]
4 The world had a lot less of everything back in 1950 (well, we had more oil in the ground, but that’s about it), so Manhattan must have just seemed amazing. [sent-4, score-1.214]
5 I can see how American leaders of that period could’ve been pretty smug. [sent-5, score-0.473]
6 Our #1 city was leading the world by so much, it was decades ahead of its time, still impressive even now after 60 years of decay. [sent-6, score-1.397]
wordName wordTfidf (topN-words)
[('impressive', 0.474), ('decay', 0.273), ('flew', 0.262), ('densely', 0.253), ('seemed', 0.227), ('oil', 0.213), ('manhattan', 0.21), ('buildings', 0.21), ('must', 0.198), ('ground', 0.185), ('leaders', 0.178), ('world', 0.175), ('chicago', 0.174), ('city', 0.155), ('back', 0.153), ('ahead', 0.149), ('period', 0.147), ('decades', 0.137), ('leading', 0.134), ('everything', 0.109), ('coming', 0.108), ('american', 0.105), ('see', 0.077), ('less', 0.075), ('always', 0.075), ('pretty', 0.071), ('still', 0.066), ('right', 0.066), ('lot', 0.064), ('years', 0.063), ('well', 0.061), ('ve', 0.049), ('time', 0.047), ('much', 0.044), ('even', 0.044), ('could', 0.042), ('think', 0.031)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 661 andrew gelman stats-2011-04-14-NYC 1950
Introduction: Coming back from Chicago we flew right over Manhattan. Very impressive as always, to see all those buildings so densely packed. But think of how impressive it must have seemed in 1950! The world had a lot less of everything back in 1950 (well, we had more oil in the ground, but that’s about it), so Manhattan must have just seemed amazing. I can see how American leaders of that period could’ve been pretty smug. Our #1 city was leading the world by so much, it was decades ahead of its time, still impressive even now after 60 years of decay.
2 0.11716852 875 andrew gelman stats-2011-08-28-Better than Dennis the dentist or Laura the lawyer
Introduction: Kieran Healy points to Robin Mahfood, the CEO of the charity Food for the Poor. This really is pretty impressive: you see a lot of good first-name or last-name matches but not so many where the entire name forms a coherent and relevant phrase.
3 0.11657184 87 andrew gelman stats-2010-06-15-Statistical analysis and visualization of the drug war in Mexico
Introduction: Christian points me to this interesting (but sad) analysis by Diego Valle with an impressive series of graphs. There are a few things I’d change (notably the R default settings which result in ridiculously over-indexed y-axes, as well as axes for homicide rates which should (but do not) go town to zero (and sometimes, bizarrely, go negative), and a lack of coherent ordering of the 32 states (including D.F.), I’m no expert on Mexico (despite having coauthored a paper on Mexican politics) so I’ll leave it to others to evaluate the substantive claims in Valle’s blog. Just looking at what he’s done, though, it seems impressive to me. To put it another way, it’s like something Nate Silver might do.
4 0.10174805 2086 andrew gelman stats-2013-11-03-How best to compare effects measured in two different time periods?
Introduction: I received the following email from someone who wishes to remain anonymous: My colleague and I are trying to understand the best way to approach a problem involving measuring a group of individuals’ abilities across time, and are hoping you can offer some guidance. We are trying to analyze the combined effect of two distinct groups of people (A and B, with no overlap between A and B) who collaborate to produce a binary outcome, using a mixed logistic regression along the lines of the following. Outcome ~ (1 | A) + (1 | B) + Other variables What we’re interested in testing was whether the observed A random effects in period 1 are predictive of the A random effects in the following period 2. Our idea being create two models, each using a different period’s worth of data, to create two sets of A coefficients, then observe the relationship between the two. If the A’s have a persistent ability across periods, the coefficients should be correlated or show a linear-ish relationshi
5 0.091859765 1653 andrew gelman stats-2013-01-04-Census dotmap
Introduction: Andrew Vande Moere points to this impressive interactive map from Brandon Martin-Anderson showing the locations of all the residents of the United States and Canada. It says, “The map has 341,817,095 dots – one for each person.” Not quite . . . I was hoping to zoom into my building (approximately 10 people live on our floor, I say approximately because two of the apartments are split between two floors and I’m not sure how they would assign the residents), but unfortunately our entire block is just a solid mass of black. Also, they put a few dots in the park and in the river by accident (presumably because the borders of the census blocks were specified only approximately). But, hey, no algorithm is perfect. It’s hard to know what to do about this. The idea of mapping every person is cool, but you’ll always run into trouble displaying densely populated areas. Smaller dots might work, but then that might depend on the screen being used for display.
6 0.091027856 140 andrew gelman stats-2010-07-10-SeeThroughNY
7 0.090151496 263 andrew gelman stats-2010-09-08-The China Study: fact or fallacy?
8 0.081792921 1383 andrew gelman stats-2012-06-18-Hierarchical modeling as a framework for extrapolation
9 0.081679307 2255 andrew gelman stats-2014-03-19-How Americans vote
10 0.079851426 280 andrew gelman stats-2010-09-16-Meet Hipmunk, a really cool flight-finder that doesn’t actually work
11 0.076568067 90 andrew gelman stats-2010-06-16-Oil spill and corn production
12 0.076269455 2251 andrew gelman stats-2014-03-17-In the best alternative histories, the real world is what’s ultimately real
13 0.072274841 1397 andrew gelman stats-2012-06-27-Stand Your Ground laws and homicides
14 0.069896907 1289 andrew gelman stats-2012-04-29-We go to war with the data we have, not the data we want
15 0.06984511 654 andrew gelman stats-2011-04-09-There’s no evidence that voters choose presidential candidates based on their looks
16 0.064709291 906 andrew gelman stats-2011-09-14-Another day, another stats postdoc
17 0.062919095 2245 andrew gelman stats-2014-03-12-More on publishing in journals
18 0.062525712 970 andrew gelman stats-2011-10-24-Bell Labs
19 0.061743382 2347 andrew gelman stats-2014-05-25-Why I decided not to be a physicist
20 0.058890544 1807 andrew gelman stats-2013-04-17-Data problems, coding errors…what can be done?
topicId topicWeight
[(0, 0.094), (1, -0.055), (2, 0.001), (3, 0.037), (4, -0.002), (5, -0.006), (6, 0.033), (7, 0.004), (8, 0.03), (9, 0.012), (10, -0.013), (11, -0.007), (12, -0.005), (13, 0.0), (14, -0.005), (15, -0.006), (16, 0.017), (17, 0.005), (18, 0.018), (19, -0.008), (20, -0.026), (21, 0.002), (22, -0.027), (23, 0.008), (24, 0.006), (25, -0.008), (26, -0.026), (27, 0.016), (28, 0.014), (29, 0.031), (30, 0.028), (31, -0.007), (32, -0.005), (33, -0.011), (34, -0.007), (35, -0.027), (36, -0.04), (37, 0.015), (38, -0.021), (39, -0.01), (40, -0.027), (41, 0.018), (42, -0.021), (43, -0.018), (44, -0.009), (45, -0.006), (46, 0.009), (47, 0.007), (48, -0.006), (49, 0.031)]
simIndex simValue blogId blogTitle
same-blog 1 0.94251543 661 andrew gelman stats-2011-04-14-NYC 1950
Introduction: Coming back from Chicago we flew right over Manhattan. Very impressive as always, to see all those buildings so densely packed. But think of how impressive it must have seemed in 1950! The world had a lot less of everything back in 1950 (well, we had more oil in the ground, but that’s about it), so Manhattan must have just seemed amazing. I can see how American leaders of that period could’ve been pretty smug. Our #1 city was leading the world by so much, it was decades ahead of its time, still impressive even now after 60 years of decay.
2 0.80788189 1370 andrew gelman stats-2012-06-07-Duncan Watts and the Titanic
Introduction: Daniel Mendelsohn recently asked , “Why do we love the Titanic?”, seeking to understand how it has happened that: It may not be true that ‘the three most written-about subjects of all time are Jesus, the Civil War, and the Titanic,’ as one historian has put it, but it’s not much of an exaggeration. . . . The inexhaustible interest suggests that the Titanic’s story taps a vein much deeper than the morbid fascination that has attached to other disasters. The explosion of the Hindenburg, for instance, and even the torpedoing, just three years after the Titanic sank, of the Lusitania, another great liner whose passenger list boasted the rich and the famous, were calamities that shocked the world but have failed to generate an obsessive preoccupation. . . . If the Titanic has gripped our imagination so forcefully for the past century, it must be because of something bigger than any fact of social or political or cultural history. To get to the bottom of why we can’t forget it, yo
3 0.8049652 1831 andrew gelman stats-2013-04-29-The Great Race
Introduction: This post is by Phil. Last summer my wife and I took a 3.5-month vacation that included a wide range of activities. When I got back, people would ask “what were the highlights or your trip?”, and I was somewhat at a loss: we had done so many things that were so different, many of which seemed really great…how could I pick? Someone said, wisely, that in six months or a year I’d be able to answer the question because some memories would be more vivid than others. They were right, and I was recently thinking back on our vacation and putting together a list of highlights — enjoyable in itself, but also worth doing to help plan future vacations. One of the things we did was go to four evenings of track and field events at the London Olympics. After we got back, people would ask what we had seen at the Olympics. I would say “We saw Usain Bolt run the 200m, we saw the women’s 4x100m relay and the men’s 4×400, we saw the last events of the decathlon…lots of great stuff. But my favorite was
4 0.78382593 335 andrew gelman stats-2010-10-11-How to think about Lou Dobbs
Introduction: I was unsurprised to read that Lou Dobbs, the former CNN host who crusaded against illegal immigrants, had actually hired a bunch of them himself to maintain his large house and his horse farm. (OK, I have to admit I was surprised by the part about the horse farm.) But I think most of the reactions to this story missed the point. Isabel Macdonald’s article that broke the story was entitled, “Lou Dobbs, American Hypocrite,” and most of the discussion went from there, with some commenters piling on Dobbs and others defending him by saying that Dobbs hired his laborers through contractors and may not have known they were in the country illegally. To me, though, the key issue is slightly different. And Macdonald’s story is relevant whether or not Dobbs knew he was hiring illegals. My point is not that Dobbs is a bad guy, or a hypocrite, or whatever. My point is that, in his setting, it would take an extraordinary effort to not hire illegal immigrants to take care of his house
5 0.7812537 1621 andrew gelman stats-2012-12-13-Puzzles of criminal justice
Introduction: Four recent news stories about crime and punishment made me realize, yet again, how little I understand all this. 1. “HSBC to Pay $1.92 Billion to Settle Charges of Money Laundering” : State and federal authorities decided against indicting HSBC in a money-laundering case over concerns that criminal charges could jeopardize one of the world’s largest banks and ultimately destabilize the global financial system. Instead, HSBC announced on Tuesday that it had agreed to a record $1.92 billion settlement with authorities. . . . I don’t understand this idea of punishing the institution. I have the same problem when the NCAA punishes a college football program. These are individual people breaking the law (or the rules), right? So why not punish them directly? Giving 40 lashes to a bunch of HSBC executives and garnisheeing their salaries for life, say, that wouldn’t destabilize the global financial system would it? From the article: “A money-laundering indictment, or a guilt
6 0.77483004 1874 andrew gelman stats-2013-05-28-Nostalgia
7 0.77374232 431 andrew gelman stats-2010-11-26-One fun thing about physicists . . .
8 0.76764905 641 andrew gelman stats-2011-04-01-So many topics, so little time
9 0.76737952 1646 andrew gelman stats-2013-01-01-Back when fifty years was a long time ago
10 0.76668382 2300 andrew gelman stats-2014-04-21-Ticket to Baaaath
11 0.76325262 158 andrew gelman stats-2010-07-22-Tenants and landlords
12 0.7630899 1245 andrew gelman stats-2012-04-03-Redundancy and efficiency: In praise of Penn Station
13 0.75948882 489 andrew gelman stats-2010-12-28-Brow inflation
14 0.7589615 17 andrew gelman stats-2010-05-05-Taking philosophical arguments literally
15 0.75654334 487 andrew gelman stats-2010-12-27-Alfred Kahn
17 0.75606769 633 andrew gelman stats-2011-03-28-“The New Tyranny: Carbon Monoxide Detectors?”
18 0.75113869 867 andrew gelman stats-2011-08-23-The economics of the mac? A paradox of competition
19 0.74894792 970 andrew gelman stats-2011-10-24-Bell Labs
20 0.74766725 2197 andrew gelman stats-2014-02-04-Peabody here.
topicId topicWeight
[(15, 0.026), (16, 0.069), (20, 0.197), (24, 0.034), (45, 0.024), (80, 0.061), (93, 0.032), (99, 0.406)]
simIndex simValue blogId blogTitle
1 0.96161753 480 andrew gelman stats-2010-12-21-Instead of “confidence interval,” let’s say “uncertainty interval”
Introduction: I’ve become increasingly uncomfortable with the term “confidence interval,” for several reasons: - The well-known difficulties in interpretation (officially the confidence statement can be interpreted only on average, but people typically implicitly give the Bayesian interpretation to each case), - The ambiguity between confidence intervals and predictive intervals. (See the footnote in BDA where we discuss the difference between “inference” and “prediction” in the classical framework.) - The awkwardness of explaining that confidence intervals are big in noisy situations where you have less confidence, and confidence intervals are small when you have more confidence. So here’s my proposal. Let’s use the term “uncertainty interval” instead. The uncertainty interval tells you how much uncertainty you have. That works pretty well, I think. P.S. As of this writing, “confidence interval” outGoogles “uncertainty interval” by the huge margin of 9.5 million to 54000. So we
2 0.95351106 910 andrew gelman stats-2011-09-15-Google Refine
Introduction: Tools worth knowing about: Google Refine is a power tool for working with messy data, cleaning it up, transforming it from one format into another, extending it with web services, and linking it to databases like Freebase. A recent discussion on the Polmeth list about the ANES Cumulative File is a setting where I think Refine might help (admittedly 49760×951 is bigger than I’d really like to deal with in the browser with js… but on a subset yes). [I might write this example up later.] Go watch the screencast videos for Refine. Data-entry problems are rampant in stuff we all use — leading or trailing spaces; mixed decimal-indicators; different units or transformations used in the same column; mixed lettercase leading to false duplicates; that’s only the beginning. Refine certainly would help find duplicates, and it counts things for you too. Just counting rows is too much for researchers sometimes (see yesterday’s post )! Refine 2.0 adds some data-collection tools for
same-blog 3 0.94257486 661 andrew gelman stats-2011-04-14-NYC 1950
Introduction: Coming back from Chicago we flew right over Manhattan. Very impressive as always, to see all those buildings so densely packed. But think of how impressive it must have seemed in 1950! The world had a lot less of everything back in 1950 (well, we had more oil in the ground, but that’s about it), so Manhattan must have just seemed amazing. I can see how American leaders of that period could’ve been pretty smug. Our #1 city was leading the world by so much, it was decades ahead of its time, still impressive even now after 60 years of decay.
4 0.94036555 1937 andrew gelman stats-2013-07-13-Meritocracy rerun
Introduction: I’ve said it here so often, this time I put it on the sister blog. . . .
Introduction: Gur Huberman points to an article on the financial crisis by Bethany McLean, who writes : lthough our understanding of what instigated the 2008 global financial crisis remains at best incomplete, there are a few widely agreed upon contributing factors. One of them is a 2004 rule change by the U.S. Securities and Exchange Commission that allowed investment banks to load up on leverage. This disastrous decision has been cited by a host of prominent economists, including Princeton professor and former Federal Reserve Vice-Chairman Alan Blinder and Nobel laureate Joseph Stiglitz. It has even been immortalized in Hollywood, figuring into the dark financial narrative that propelled the Academy Award-winning film Inside Job. . . . Here’s just one problem with this story line: It’s not true. Nor is it hard to prove that. Look at the historical leverage of the big five investment banks — Bear Stearns, Lehman Brothers, Merrill Lynch, Goldman Sachs and Morgan Stanley. The Government Accou
6 0.92695946 831 andrew gelman stats-2011-07-30-A Wikipedia riddle!
7 0.92253727 479 andrew gelman stats-2010-12-20-WWJD? U can find out!
8 0.9076488 592 andrew gelman stats-2011-02-26-“Do you need ideal conditions to do great work?”
9 0.90634263 974 andrew gelman stats-2011-10-26-NYC jobs in applied statistics, psychometrics, and causal inference!
10 0.90594751 1652 andrew gelman stats-2013-01-03-“The Case for Inductive Theory Building”
11 0.90475994 254 andrew gelman stats-2010-09-04-Bayesian inference viewed as a computational approximation to classical calculations
12 0.90395278 1629 andrew gelman stats-2012-12-18-It happened in Connecticut
13 0.89973432 194 andrew gelman stats-2010-08-09-Data Visualization
14 0.89510667 270 andrew gelman stats-2010-09-12-Comparison of forecasts for the 2010 congressional elections
15 0.89395332 1912 andrew gelman stats-2013-06-24-Bayesian quality control?
16 0.89116812 461 andrew gelman stats-2010-12-09-“‘Why work?’”
17 0.89028221 306 andrew gelman stats-2010-09-29-Statistics and the end of time
18 0.8888486 1642 andrew gelman stats-2012-12-28-New book by Stef van Buuren on missing-data imputation looks really good!
19 0.88855648 740 andrew gelman stats-2011-06-01-The “cushy life” of a University of Illinois sociology professor
20 0.88742113 1270 andrew gelman stats-2012-04-19-Demystifying Blup