andrew_gelman_stats andrew_gelman_stats-2010 andrew_gelman_stats-2010-135 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: The recent discussion of pollsters reminded me of a story from a couple years ago that perhaps is still relevant . . . I was looking up the governors’ popularity numbers on the web, and came across this page from Rasmussen Reports which shows Sarah Palin as the 3rd-most-popular governor. But then I looked more carefully. Janet Napolitano of Arizona was viewed as Excellent by 28% of respondents, Good by 27%, Fair by 26%, and Poor by 27%. That adds up to 108%! What’s going on? I’d think they would have a computer program to pipe the survey results directly into the spreadsheet. But I guess not, someone must be typing in these numbers one at a time. Another possibility is that they are altering their numbers by hand, and someone made a mistake with the Napolitano numbers, adding a few percent in one place and forgetting to subtract elsewhere. Or maybe there’s another explanation? P.S. Here are some thoughts from Mark Blumenthal P.P.S. I checked the Rasmussen link toda
sentIndex sentText sentNum sentScore
1 The recent discussion of pollsters reminded me of a story from a couple years ago that perhaps is still relevant . [sent-1, score-0.284]
2 I was looking up the governors’ popularity numbers on the web, and came across this page from Rasmussen Reports which shows Sarah Palin as the 3rd-most-popular governor. [sent-4, score-0.468]
3 Janet Napolitano of Arizona was viewed as Excellent by 28% of respondents, Good by 27%, Fair by 26%, and Poor by 27%. [sent-6, score-0.104]
4 I’d think they would have a computer program to pipe the survey results directly into the spreadsheet. [sent-9, score-0.288]
5 But I guess not, someone must be typing in these numbers one at a time. [sent-10, score-0.614]
6 Another possibility is that they are altering their numbers by hand, and someone made a mistake with the Napolitano numbers, adding a few percent in one place and forgetting to subtract elsewhere. [sent-11, score-1.097]
7 Here are some thoughts from Mark Blumenthal P. [sent-15, score-0.063]
8 I checked the Rasmussen link today (9 July 2010) and the Napolitano numbers still add up to 108%. [sent-18, score-0.582]
9 So I guess nobody at Ramussen noticed my blog that I posted earlier on the topic! [sent-19, score-0.327]
10 In case you were wondering: No, I can’t see how you could possibly get to 108% from rounding error. [sent-24, score-0.202]
wordName wordTfidf (topN-words)
[('napolitano', 0.541), ('rasmussen', 0.309), ('numbers', 0.284), ('blumenthal', 0.155), ('governors', 0.143), ('palin', 0.139), ('pipe', 0.139), ('subtract', 0.132), ('altering', 0.132), ('arizona', 0.132), ('typing', 0.132), ('forgetting', 0.129), ('sarah', 0.129), ('pollsters', 0.127), ('rounding', 0.123), ('janet', 0.117), ('popularity', 0.113), ('july', 0.106), ('guess', 0.105), ('viewed', 0.104), ('checked', 0.096), ('adds', 0.096), ('someone', 0.093), ('possibility', 0.092), ('explanation', 0.089), ('respondents', 0.086), ('adding', 0.085), ('reminded', 0.082), ('possibly', 0.079), ('web', 0.079), ('another', 0.078), ('computer', 0.078), ('wondering', 0.077), ('poor', 0.077), ('noticed', 0.077), ('still', 0.075), ('mistake', 0.075), ('excellent', 0.075), ('percent', 0.075), ('nobody', 0.073), ('fair', 0.073), ('posted', 0.072), ('program', 0.071), ('shows', 0.071), ('mark', 0.07), ('looked', 0.067), ('reports', 0.067), ('today', 0.064), ('thoughts', 0.063), ('add', 0.063)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 135 andrew gelman stats-2010-07-09-Rasmussen sez: “108% of Respondents Say . . .”
Introduction: The recent discussion of pollsters reminded me of a story from a couple years ago that perhaps is still relevant . . . I was looking up the governors’ popularity numbers on the web, and came across this page from Rasmussen Reports which shows Sarah Palin as the 3rd-most-popular governor. But then I looked more carefully. Janet Napolitano of Arizona was viewed as Excellent by 28% of respondents, Good by 27%, Fair by 26%, and Poor by 27%. That adds up to 108%! What’s going on? I’d think they would have a computer program to pipe the survey results directly into the spreadsheet. But I guess not, someone must be typing in these numbers one at a time. Another possibility is that they are altering their numbers by hand, and someone made a mistake with the Napolitano numbers, adding a few percent in one place and forgetting to subtract elsewhere. Or maybe there’s another explanation? P.S. Here are some thoughts from Mark Blumenthal P.P.S. I checked the Rasmussen link toda
2 0.14676949 159 andrew gelman stats-2010-07-23-Popular governor, small state
Introduction: A couple years ago, upon the selection of Sarah Palin as vice-presidential nominee, I made some graphs of the popularity of governors of different-sized states: As I wrote at the time : It seems to be easier to maintain high approval in a small state. What’s going on? Some theories: in a large state, there will be more ambitious politicians on the other side, eager to knock off the incumbent governor; small states often have part-time legislatures and thus the governor is involved in less political conflict; small states (notably Alaska) tend to get more funds per capita from the federal government, and it’s easier to be popular when you can disburse more funds; large states tend to be more heterogeneous and so it’s harder to keep all the voters happy. I was curious how things have been going more recently, and Hanfei made an updated graph using data from this archive . Here’s the story: There’s lots of variation–clearly there are many other factors than state popu
3 0.11172337 1399 andrew gelman stats-2012-06-28-Life imitates blog
Introduction: I just noticed this from a couple years ago!
4 0.091682151 939 andrew gelman stats-2011-10-03-DBQQ rounding for labeling charts and communicating tolerances
Introduction: This is a mini research note, not deserving of a paper, but perhaps useful to others. It reinvents what has already appeared on this blog. Let’s say we have a line chart with numbers between 152.134 and 210.823, with the mean of 183.463. How should we label the chart with about 3 tics? Perhaps 152.132, 181.4785 and 210.823? Don’t do it! Objective is to fit about 3-7 tics at the optimal level of rounding. I use the following sequence: decimal rounding : fitting integer power and single-digit decimal i , rounding to i * 10^ power (example: 100 200 300) binary having power , fitting single-digit decimal i and binary b , rounding to 2* i /(1+ b ) * 10^ power (150 200 250) (optional) quaternary having power , fitting single-digit decimal i and quaternary q (0,1,2,3) round to 4* i /(1+ q ) * 10^ power (150 175 200) quinary having power , fitting single-digit decimal i and quinary f (0,1,2,3,4) round to 5* i /(1+ f ) * 10^ power (160 180 200)
Introduction: I was thinking more about David Brooks’s anti-data column from yesterday, and I realized what is really bothering me. Brooks expresses skepticism about numbers, about the limitations of raw data, about the importance of human thinking. Fine, I agree with all of this, to some extent. But then Brooks turns around uses numbers and unquestioningly and uncritically (OK, not completely uncritically; see P.S. below). In a notorious recent case, Brooks wrote, in the context of college admissions: You’re going to want to argue with Unz’s article all the way along, especially for its narrow, math-test-driven view of merit. But it’s potentially ground-shifting. Unz’s other big point is that Jews are vastly overrepresented at elite universities and that Jewish achievement has collapsed. In the 1970s, for example, 40 percent of top scorers in the Math Olympiad had Jewish names. Now 2.5 percent do. But these numbers are incorrect, as I learned from a professor of oncology at the Univ
7 0.084435649 909 andrew gelman stats-2011-09-15-7 steps to successful infographics
8 0.081588998 187 andrew gelman stats-2010-08-05-Update on state size and governors’ popularity
9 0.079933479 797 andrew gelman stats-2011-07-11-How do we evaluate a new and wacky claim?
10 0.076603793 1083 andrew gelman stats-2011-12-26-The quals and the quants
11 0.075867549 93 andrew gelman stats-2010-06-17-My proposal for making college admissions fairer
12 0.07268329 588 andrew gelman stats-2011-02-24-In case you were wondering, here’s the price of milk
13 0.071735598 1768 andrew gelman stats-2013-03-18-Mertz’s reply to Unz’s response to Mertz’s comments on Unz’s article
14 0.070435338 1743 andrew gelman stats-2013-02-28-Different modes of discourse
16 0.06721767 872 andrew gelman stats-2011-08-26-Blog on applied probability modeling
18 0.06673792 953 andrew gelman stats-2011-10-11-Steve Jobs’s cancer and science-based medicine
19 0.066538729 677 andrew gelman stats-2011-04-24-My NOAA story
20 0.065506138 284 andrew gelman stats-2010-09-18-Continuing efforts to justify false “death panels” claim
topicId topicWeight
[(0, 0.115), (1, -0.06), (2, 0.019), (3, 0.018), (4, 0.025), (5, -0.004), (6, 0.032), (7, 0.004), (8, 0.015), (9, -0.04), (10, 0.014), (11, -0.026), (12, -0.014), (13, 0.033), (14, -0.011), (15, 0.091), (16, -0.01), (17, 0.009), (18, 0.017), (19, 0.007), (20, -0.015), (21, 0.02), (22, -0.001), (23, -0.013), (24, -0.008), (25, 0.028), (26, -0.023), (27, 0.038), (28, -0.025), (29, -0.007), (30, 0.007), (31, 0.031), (32, 0.027), (33, -0.028), (34, -0.023), (35, -0.001), (36, -0.006), (37, -0.042), (38, 0.016), (39, 0.024), (40, -0.003), (41, 0.018), (42, 0.034), (43, 0.001), (44, -0.011), (45, 0.033), (46, -0.039), (47, -0.01), (48, 0.004), (49, -0.001)]
simIndex simValue blogId blogTitle
same-blog 1 0.97090894 135 andrew gelman stats-2010-07-09-Rasmussen sez: “108% of Respondents Say . . .”
Introduction: The recent discussion of pollsters reminded me of a story from a couple years ago that perhaps is still relevant . . . I was looking up the governors’ popularity numbers on the web, and came across this page from Rasmussen Reports which shows Sarah Palin as the 3rd-most-popular governor. But then I looked more carefully. Janet Napolitano of Arizona was viewed as Excellent by 28% of respondents, Good by 27%, Fair by 26%, and Poor by 27%. That adds up to 108%! What’s going on? I’d think they would have a computer program to pipe the survey results directly into the spreadsheet. But I guess not, someone must be typing in these numbers one at a time. Another possibility is that they are altering their numbers by hand, and someone made a mistake with the Napolitano numbers, adding a few percent in one place and forgetting to subtract elsewhere. Or maybe there’s another explanation? P.S. Here are some thoughts from Mark Blumenthal P.P.S. I checked the Rasmussen link toda
2 0.75195879 1115 andrew gelman stats-2012-01-12-Where are the larger-than-life athletes?
Introduction: Jonathan Cantor points to this poll estimating rifle-armed QB Tim Tebow as America’s favorite pro athlete: In an ESPN survey of 1,502 Americans age 12 or older, three percent identified Tebow as their favorite professional athlete. Tebow finished in front of Kobe Bryant (2 percent), Aaron Rodgers (1.9 percent), Peyton Manning (1.8 percent), and Tom Brady (1.5 percent). Amusing. What this survey says to me is that there are no super-popular athletes who are active in America today. Which actually sounds about right. No Tiger Woods, no Magic Johnson, Muhammed Ali, John Elway, Pete Rose, Billie Jean King, etc etc. Tebow is an amusing choice, people might as well pick him now while he’s still on top. As a sports celeb, he’s like Bill Lee or the Refrigerator: colorful and a solid pro athlete, but no superstar. When you think about all the colorful superstar athletes of times gone by, it’s perhaps surprising that there’s nobody out there right now to play the role. I supp
3 0.72623324 69 andrew gelman stats-2010-06-04-A Wikipedia whitewash
Introduction: After hearing a few times about the divorce predictions of researchers John Gottman and James Murray (work that was featured in Blink with a claim that they could predict with 83 percent accuracy whether a couple would be divorced–after meeting with them for 15 minutes) and feeling some skepticism , I decided to do the Lord’s work and amend Gottman’s wikipedia entry, which had a paragraph saying: Gottman found his methodology predicts with 90% accuracy which newlywed couples will remain married and which will divorce four to six years later. It is also 81% percent accurate in predicting which marriages will survive after seven to nine years. I added the following: Gottman’s claim of 81% or 90% accuracy is misleading, however, because the accuracy is measured only after fitting a model to his data. There is no evidence that he can predict the outcome of a marriage with high accuracy in advance. As Laurie Abraham writes, “For the 1998 study, which focused on videotapes of 57
4 0.71484774 1025 andrew gelman stats-2011-11-24-Always check your evidence
Introduction: Logical reasoning typically takes the following form: 1. I know that A is true. 2. I know that A implies B. 3. Therefore, I can conclude that B is true. I, like Lewis Carroll, have problems with this process sometimes, but it’s pretty standard. There is also a statistical version in which the above statements are replaced by averages (“A usually happens,” etc.). But in all these stories, the argument can fall down if you get the facts wrong. Perhaps that’s one reason that statisticians can be obsessed with detail. For example, David Brooks wrote the following, in a column called “Living with Mistakes”: The historian Leslie Hannah identified the ten largest American companies in 1912. None of those companies ranked in the top 100 companies by 1990. Huh? Could that really be? I googled “ten largest american companies 1912″ and found this , from Leslie Hannah: No big deal: two still in the top 10 rather than zero in the top 100, but Brooks’s general
5 0.69670409 1187 andrew gelman stats-2012-02-27-“Apple confronts the law of large numbers” . . . huh?
Introduction: I was reading this news article by famed business reporter James Stewart: Measured by market capitalization, Apple is the world’s biggest public company. . . . Sales for the quarter that ended Dec. 31 . . . totaled $46.33 billion, up 73 percent from the year before. Earnings more than doubled. . . . Here is the rub: Apple is so big, it’s running up against the law of large numbers. Huh? At this point I sat up, curious. Stewart continued: Also known as the golden theorem, with a proof attributed to the 17th-century Swiss mathematician Jacob Bernoulli, the law states that a variable will revert to a mean over a large sample of results. In the case of the largest companies, it suggests that high earnings growth and a rapid rise in share price will slow as those companies grow ever larger. If Apple’s share price grew even 20 percent a year for the next decade, which is far below its current blistering pace, its $500 billion market capitalization would be more than $3 tri
8 0.67621434 141 andrew gelman stats-2010-07-12-Dispute over counts of child deaths in Iraq due to sanctions
9 0.67581844 204 andrew gelman stats-2010-08-12-Sloppily-written slam on moderately celebrated writers is amusing nonetheless
10 0.65799451 1240 andrew gelman stats-2012-04-02-Blogads update
11 0.65704358 1730 andrew gelman stats-2013-02-20-Unz on Unz
12 0.65616488 142 andrew gelman stats-2010-07-12-God, Guns, and Gaydar: The Laws of Probability Push You to Overestimate Small Groups
13 0.65552002 129 andrew gelman stats-2010-07-05-Unrelated to all else
14 0.64622265 2166 andrew gelman stats-2014-01-10-3 years out of date on the whole Dennis the dentist thing!
15 0.63727701 731 andrew gelman stats-2011-05-26-Lottery probability update
16 0.63630241 1033 andrew gelman stats-2011-11-28-Greece to head statistician: Tell the truth, go to jail
17 0.63561976 2280 andrew gelman stats-2014-04-03-As the boldest experiment in journalism history, you admit you made a mistake
18 0.63394117 1177 andrew gelman stats-2012-02-20-Joshua Clover update
19 0.63243914 108 andrew gelman stats-2010-06-24-Sometimes the raw numbers are better than a percentage
20 0.6323446 730 andrew gelman stats-2011-05-25-Rechecking the census
topicId topicWeight
[(16, 0.138), (21, 0.02), (24, 0.192), (27, 0.018), (34, 0.183), (43, 0.022), (46, 0.012), (48, 0.014), (55, 0.017), (77, 0.036), (79, 0.019), (86, 0.011), (99, 0.202)]
simIndex simValue blogId blogTitle
same-blog 1 0.9242087 135 andrew gelman stats-2010-07-09-Rasmussen sez: “108% of Respondents Say . . .”
Introduction: The recent discussion of pollsters reminded me of a story from a couple years ago that perhaps is still relevant . . . I was looking up the governors’ popularity numbers on the web, and came across this page from Rasmussen Reports which shows Sarah Palin as the 3rd-most-popular governor. But then I looked more carefully. Janet Napolitano of Arizona was viewed as Excellent by 28% of respondents, Good by 27%, Fair by 26%, and Poor by 27%. That adds up to 108%! What’s going on? I’d think they would have a computer program to pipe the survey results directly into the spreadsheet. But I guess not, someone must be typing in these numbers one at a time. Another possibility is that they are altering their numbers by hand, and someone made a mistake with the Napolitano numbers, adding a few percent in one place and forgetting to subtract elsewhere. Or maybe there’s another explanation? P.S. Here are some thoughts from Mark Blumenthal P.P.S. I checked the Rasmussen link toda
2 0.91130674 929 andrew gelman stats-2011-09-27-Visual diagnostics for discrete-data regressions
Introduction: Jeff asked me what I thought of this recent AJPS article by Brian Greenhill, Michael Ward, and Audrey Sacks, “The Separation Plot: A New Visual Method for Evaluating the Fit of Binary Models.” It’s similar to a graph of observed vs. predicted values, but using color rather than the y-axis to display the observed values. It seems like it could be useful, also could be applied more generally to discrete-data regressions with more than two categories. When it comes to checking the model fit, I recommend binned residual plots, as discussed in this 2000 article with Yuri Goegebeur, Francis Tuerlinckx, and Iven Van Mechelen.
3 0.90633202 1911 andrew gelman stats-2013-06-23-AI Stats conference on Stan etc.
Introduction: Jaakko Peltonen writes: The Seventeenth International Conference on Artificial Intelligence and Statistics (http://www.aistats.org) will be next April in Reykjavik, Iceland. AISTATS is an interdisciplinary conference at the intersection of computer science, artificial intelligence, machine learning, statistics, and related areas. ============================================================================== AISTATS 2014 Call for Papers Seventeenth International Conference on Artificial Intelligence and Statistics April 22 – 25, 2014, Reykjavik, Iceland http://www.aistats.org Colocated with a MLSS Machine Learning Summer School ============================================================================== AISTATS is an interdisciplinary gathering of researchers at the intersection of computer science, artificial intelligence, machine learning, statistics, and related areas. Since its inception in 1985, the primary goal of AISTATS has been to broaden research in the
4 0.90209115 1144 andrew gelman stats-2012-01-29-How many parameters are in a multilevel model?
Introduction: Stephen Collins writes: I’m reading your Multilevel modeling book and am trying to apply it to my work. I’m concerned with how to estimate a random intercept model if there are hundreds/thousands of levels. In the Gibbs sampling, am I sampling a parameter for each level? Or, just the hyper-parameters? In other words, say I had 500 zipcode intercepts modeled as ~ N(m,s). Would my posterior be two dimensional, sampling for “m” and “s,” or would it have 502 dimensions? My reply: Indeed you will have hundreds or thousands of parameters—or, in classical terms, hundreds or thousands of predictive quantities. But that’s ok. Even if none of those predictions is precise, you’re learning about the model. See page 526 of the book for more discussion of the number of parameters in a multilevel model.
5 0.89995694 1734 andrew gelman stats-2013-02-23-Life in the C-suite: A graph that is both ugly and bad, and an unrelated story
Introduction: Jemes Keirstead sends along this infographic : He hates it: First we’ve got an hourglass metaphor wrecked by the fact that “now” (i.e. the pinch point in the glass) is actually 3-5 years in the future and the past sand includes “up to three years” in the future. Then there are the percentages which are appear to represent a vertical distance, not volume of sand or width of the hourglass. Add to that a strange color scheme in which green goes from dark to light to dark again. I know January’s not even finished yet, but surely a competitor for worst infographic of 2013? Keirstead doesn’t even comment on what I see as the worst aspect of the graph, which is that the “3-5 years” band is the narrowest on the graph, but expressed as a per-year rate it is actually the highest of all the percentages. The hourglass visualization does the astounding feat of taking the period where the executives expect the highest rate of change and presenting it as a minimum in the graph.
6 0.88748372 1501 andrew gelman stats-2012-09-18-More studies on the economic effects of climate change
7 0.87843227 936 andrew gelman stats-2011-10-02-Covariate Adjustment in RCT - Model Overfitting in Multilevel Regression
8 0.87471914 2095 andrew gelman stats-2013-11-09-Typo in Ghitza and Gelman MRP paper
9 0.8734448 292 andrew gelman stats-2010-09-23-Doug Hibbs on the fundamentals in 2010
11 0.85224974 2288 andrew gelman stats-2014-04-10-Small multiples of lineplots > maps (ok, not always, but yes in this case)
12 0.84826005 1723 andrew gelman stats-2013-02-15-Wacky priors can work well?
14 0.84549421 2186 andrew gelman stats-2014-01-26-Infoviz on top of stat graphic on top of spreadsheet
15 0.84291005 2203 andrew gelman stats-2014-02-08-“Guys who do more housework get less sex”
16 0.84105277 1842 andrew gelman stats-2013-05-05-Cleaning up science
17 0.83945239 956 andrew gelman stats-2011-10-13-Hey, you! Don’t take that class!
18 0.83581161 1293 andrew gelman stats-2012-05-01-Huff the Magic Dragon
19 0.83510125 799 andrew gelman stats-2011-07-13-Hypothesis testing with multiple imputations
20 0.83364379 388 andrew gelman stats-2010-11-01-The placebo effect in pharma