andrew_gelman_stats andrew_gelman_stats-2013 andrew_gelman_stats-2013-1653 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Andrew Vande Moere points to this impressive interactive map from Brandon Martin-Anderson showing the locations of all the residents of the United States and Canada. It says, “The map has 341,817,095 dots – one for each person.” Not quite . . . I was hoping to zoom into my building (approximately 10 people live on our floor, I say approximately because two of the apartments are split between two floors and I’m not sure how they would assign the residents), but unfortunately our entire block is just a solid mass of black. Also, they put a few dots in the park and in the river by accident (presumably because the borders of the census blocks were specified only approximately). But, hey, no algorithm is perfect. It’s hard to know what to do about this. The idea of mapping every person is cool, but you’ll always run into trouble displaying densely populated areas. Smaller dots might work, but then that might depend on the screen being used for display.
sentIndex sentText sentNum sentScore
1 Andrew Vande Moere points to this impressive interactive map from Brandon Martin-Anderson showing the locations of all the residents of the United States and Canada. [sent-1, score-0.917]
2 It says, “The map has 341,817,095 dots – one for each person. [sent-2, score-0.568]
3 I was hoping to zoom into my building (approximately 10 people live on our floor, I say approximately because two of the apartments are split between two floors and I’m not sure how they would assign the residents), but unfortunately our entire block is just a solid mass of black. [sent-6, score-1.981]
4 Also, they put a few dots in the park and in the river by accident (presumably because the borders of the census blocks were specified only approximately). [sent-7, score-1.307]
5 The idea of mapping every person is cool, but you’ll always run into trouble displaying densely populated areas. [sent-10, score-0.703]
6 Smaller dots might work, but then that might depend on the screen being used for display. [sent-11, score-0.729]
wordName wordTfidf (topN-words)
[('dots', 0.366), ('approximately', 0.303), ('residents', 0.268), ('map', 0.202), ('zoom', 0.184), ('brandon', 0.184), ('floors', 0.174), ('apartments', 0.166), ('densely', 0.161), ('river', 0.161), ('borders', 0.156), ('populated', 0.14), ('locations', 0.138), ('accident', 0.138), ('blocks', 0.134), ('floor', 0.134), ('interactive', 0.124), ('mapping', 0.123), ('block', 0.122), ('screen', 0.122), ('displaying', 0.121), ('park', 0.121), ('split', 0.12), ('specified', 0.119), ('assign', 0.117), ('census', 0.112), ('solid', 0.109), ('depend', 0.109), ('hoping', 0.103), ('mass', 0.103), ('impressive', 0.101), ('presumably', 0.101), ('algorithm', 0.097), ('smaller', 0.094), ('display', 0.093), ('united', 0.091), ('building', 0.09), ('cool', 0.088), ('trouble', 0.088), ('live', 0.086), ('showing', 0.084), ('entire', 0.084), ('unfortunately', 0.082), ('hey', 0.081), ('andrew', 0.08), ('states', 0.073), ('run', 0.07), ('two', 0.069), ('says', 0.069), ('might', 0.066)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 1653 andrew gelman stats-2013-01-04-Census dotmap
Introduction: Andrew Vande Moere points to this impressive interactive map from Brandon Martin-Anderson showing the locations of all the residents of the United States and Canada. It says, “The map has 341,817,095 dots – one for each person.” Not quite . . . I was hoping to zoom into my building (approximately 10 people live on our floor, I say approximately because two of the apartments are split between two floors and I’m not sure how they would assign the residents), but unfortunately our entire block is just a solid mass of black. Also, they put a few dots in the park and in the river by accident (presumably because the borders of the census blocks were specified only approximately). But, hey, no algorithm is perfect. It’s hard to know what to do about this. The idea of mapping every person is cool, but you’ll always run into trouble displaying densely populated areas. Smaller dots might work, but then that might depend on the screen being used for display.
2 0.28140602 469 andrew gelman stats-2010-12-16-2500 people living in a park in Chicago?
Introduction: Frank Hansen writes: Columbus Park is on Chicago’s west side, in the Austin neighborhood. The park is a big green area which includes a golf course. Here is the google satellite view. Here is the nytimes page. Go to Chicago, and zoom over to the census tract 2521, which is just north of the horizontal gray line (Eisenhower Expressway, aka I290) and just east of Oak Park. The park is labeled on the nytimes map. The census data have around 50 dots (they say 50 people per dot) in the park which has no residential buildings. Congressional district is Danny Davis, IL7. Here’s a map of the district. So, how do we explain the map showing ~50 dots worth of people living in the park. What’s up with the algorithm to place the dots? I dunno. I leave this one to you, the readers.
3 0.15465826 1810 andrew gelman stats-2013-04-17-Subway series
Introduction: Abby points us to a spare but cool visualization . I don’t like the curvy connect-the-dots line, but my main suggested improvement would be a closer link to the map . Showing median income on census tracts along subway lines is cool, but ultimately it’s a clever gimmick that pulls me in and makes me curious about what the map looks like. (And, thanks to google, the map was easy to find.)
4 0.14815839 730 andrew gelman stats-2011-05-25-Rechecking the census
Introduction: Sam Roberts writes : The Census Bureau [reported] that though New York City’s population reached a record high of 8,175,133 in 2010, the gain of 2 percent, or 166,855 people, since 2000 fell about 200,000 short of what the bureau itself had estimated. Public officials were incredulous that a city that lures tens of thousands of immigrants each year and where a forest of new buildings has sprouted could really have recorded such a puny increase. How, they wondered, could Queens have grown by only one-tenth of 1 percent since 2000? How, even with a surge in foreclosures, could the number of vacant apartments have soared by nearly 60 percent in Queens and by 66 percent in Brooklyn? That does seem a bit suspicious. So the newspaper did its own survey: Now, a house-to-house New York Times survey of three representative square blocks where the Census Bureau said vacancies had increased and the population had declined since 2000 suggests that the city’s outrage is somewhat ju
Introduction: Kaiser Fung shares this graph from Ritchie King: Kaiser writes: What they did right: - Did not put the data on a map - Ordered the countries by the most recent data point rather than alphabetically - Scale labels are found only on outer edge of the chart area, rather than one set per panel - Only used three labels for the 11 years on the plot - Did not overdo the vertical scale either The nicest feature was the XL scale applied only to South Korea. This destroys the small-multiples principle but draws attention to the top left corner, where the designer wants our eyes to go. I would have used smaller fonts throughout. I agree with all of Kaiser’s comments. I could even add a few more, like using light gray for the backgrounds and a bright blue for the lines, spacing the graphs well, using full country names rather than three-letter abbreviations. There are so many standard mistakes that go into default data displays that it is refreshing to see a simple graph done
7 0.10160232 1504 andrew gelman stats-2012-09-20-Could someone please lock this guy and Niall Ferguson in a room together?
8 0.091859765 661 andrew gelman stats-2011-04-14-NYC 1950
9 0.090996996 558 andrew gelman stats-2011-02-05-Fattening of the world and good use of the alpha channel
10 0.089259803 636 andrew gelman stats-2011-03-29-The Conservative States of America
12 0.082279317 2065 andrew gelman stats-2013-10-17-Cool dynamic demographic maps provide beautiful illustration of Chris Rock effect
13 0.081687614 678 andrew gelman stats-2011-04-25-Democrats do better among the most and least educated groups
15 0.07694491 1472 andrew gelman stats-2012-08-28-Migrating from dot to underscore
16 0.073944129 787 andrew gelman stats-2011-07-05-Different goals, different looks: Infovis and the Chris Rock effect
17 0.073643908 492 andrew gelman stats-2010-12-30-That puzzle-solving feeling
18 0.069885969 1006 andrew gelman stats-2011-11-12-Val’s Number Scroll: Helping kids visualize math
19 0.069655754 624 andrew gelman stats-2011-03-22-A question about the economic benefits of universities
20 0.067924134 1014 andrew gelman stats-2011-11-16-Visualizations of NYPD stop-and-frisk data
topicId topicWeight
[(0, 0.086), (1, -0.028), (2, 0.021), (3, 0.034), (4, 0.047), (5, -0.034), (6, -0.016), (7, -0.003), (8, -0.013), (9, -0.015), (10, -0.01), (11, -0.027), (12, -0.003), (13, 0.012), (14, -0.004), (15, 0.02), (16, 0.034), (17, -0.007), (18, 0.01), (19, -0.016), (20, -0.021), (21, -0.012), (22, -0.017), (23, 0.014), (24, 0.015), (25, -0.023), (26, -0.028), (27, 0.052), (28, 0.011), (29, 0.011), (30, 0.061), (31, -0.032), (32, -0.003), (33, 0.029), (34, 0.012), (35, 0.017), (36, -0.019), (37, -0.026), (38, -0.024), (39, -0.006), (40, -0.045), (41, -0.01), (42, 0.0), (43, -0.036), (44, 0.025), (45, 0.023), (46, -0.04), (47, -0.002), (48, 0.007), (49, 0.008)]
simIndex simValue blogId blogTitle
same-blog 1 0.9351095 1653 andrew gelman stats-2013-01-04-Census dotmap
Introduction: Andrew Vande Moere points to this impressive interactive map from Brandon Martin-Anderson showing the locations of all the residents of the United States and Canada. It says, “The map has 341,817,095 dots – one for each person.” Not quite . . . I was hoping to zoom into my building (approximately 10 people live on our floor, I say approximately because two of the apartments are split between two floors and I’m not sure how they would assign the residents), but unfortunately our entire block is just a solid mass of black. Also, they put a few dots in the park and in the river by accident (presumably because the borders of the census blocks were specified only approximately). But, hey, no algorithm is perfect. It’s hard to know what to do about this. The idea of mapping every person is cool, but you’ll always run into trouble displaying densely populated areas. Smaller dots might work, but then that might depend on the screen being used for display.
2 0.80624497 469 andrew gelman stats-2010-12-16-2500 people living in a park in Chicago?
Introduction: Frank Hansen writes: Columbus Park is on Chicago’s west side, in the Austin neighborhood. The park is a big green area which includes a golf course. Here is the google satellite view. Here is the nytimes page. Go to Chicago, and zoom over to the census tract 2521, which is just north of the horizontal gray line (Eisenhower Expressway, aka I290) and just east of Oak Park. The park is labeled on the nytimes map. The census data have around 50 dots (they say 50 people per dot) in the park which has no residential buildings. Congressional district is Danny Davis, IL7. Here’s a map of the district. So, how do we explain the map showing ~50 dots worth of people living in the park. What’s up with the algorithm to place the dots? I dunno. I leave this one to you, the readers.
Introduction: Aleks points me to this article showing some pretty maps by Eric Fisher showing where people of different ethnicity live within several metro areas within the U.S. The idea is simple but effective; in the words of Cliff Kuang: Fisher used a straight forward method borrowed from Rankin: Using U.S. Census data from 2000, he created a map where one dot equals 25 people. The dots are then color-coded based on race: White is pink; Black is blue; Hispanic is orange, and Asian is green. The results for various cities are fascinating: Just like every city is different, every city is integrated (or segregated) in different ways. New York is shown below. No, San Francisco is not “very, very white” But I worry that these maps are difficult for non-experts to read. For example, Kuang writes the following:: San Francisco proper is very, very white. This is an understandable mistake coming from someone who, I assume, has never lived in the Bay Area. But what’s amazing i
4 0.72049254 2065 andrew gelman stats-2013-10-17-Cool dynamic demographic maps provide beautiful illustration of Chris Rock effect
Introduction: Robert Gonzalez reports on some beautiful graphs from John Nelson. Here’s Nelson: The sexes start out homogenous, go super segregated in the teen years, segregate for business in the twenty-somethings, and re-couple for co-habitation years. Then the lights fade into faint pockets of pink. I [Nelson] am using simple tract-level population/gender counts from the US Census Bureau. Because their tract boundaries extend into the water and vacant area, I used NYC’s Bytes of the Big Apple zoning shapes to clip the census tracts to residentially zoned areas -giving me a more realistic (and more recognizable) definition of populated areas. The census breaks out their population counts by gender for five-year age spans ranging from teeny tiny infants through esteemed 85+ year-olds. And here’s Gonzalez: Between ages 0 and 14, the entire map is more or less an evenly mixed purple landscape; newborns, children and adolescents, after all, can’t really choose where the
5 0.68607384 536 andrew gelman stats-2011-01-24-Trends in partisanship by state
Introduction: Matthew Yglesias discusses how West Virginia used to be a Democratic state but is now solidly Republican. I thought it would be helpful to expand this to look at trends since 1948 (rather than just 1988) and all 50 states (rather than just one). This would represent a bit of work, except that I already did it a couple years ago, so here it is (right-click on the image to see the whole thing): I cheated a bit to get reasonable-looking groupings, for example putting Indiana in the Border South rather than Midwest, and putting Alaska in Mountain West and Hawaii in West Coast. Also, it would help to distinguish states by color (to be able to disentangle New Jersey and Delaware, for example) but we didn’t do this because the book is mostly black and white. In any case, the picture makes it clear that there have been strong regional trends all over during the past sixty years. P.S. My graph comes from Red State Blue State so no 2008 data, but 2008 was pretty much a shift
6 0.64243728 2333 andrew gelman stats-2014-05-13-Personally, I’d rather go with Teragram
7 0.63310879 1810 andrew gelman stats-2013-04-17-Subway series
8 0.61953062 787 andrew gelman stats-2011-07-05-Different goals, different looks: Infovis and the Chris Rock effect
10 0.61572474 1124 andrew gelman stats-2012-01-17-How to map geographically-detailed survey responses?
11 0.61387455 597 andrew gelman stats-2011-03-02-RStudio – new cross-platform IDE for R
12 0.61333966 182 andrew gelman stats-2010-08-03-Nebraska never looked so appealing: anatomy of a zombie attack. Oops, I mean a recession.
13 0.61327082 19 andrew gelman stats-2010-05-06-OK, so this is how I ended up working with three different guys named Matt
14 0.60790461 624 andrew gelman stats-2011-03-22-A question about the economic benefits of universities
15 0.60695171 2038 andrew gelman stats-2013-09-25-Great graphs of names
16 0.60171229 2184 andrew gelman stats-2014-01-24-Parables vs. stories
17 0.59692675 730 andrew gelman stats-2011-05-25-Rechecking the census
18 0.59508586 488 andrew gelman stats-2010-12-27-Graph of the year
19 0.59432733 428 andrew gelman stats-2010-11-24-Flawed visualization of U.S. voting maybe has some good features
20 0.5941695 2308 andrew gelman stats-2014-04-27-White stripes and dead armadillos
topicId topicWeight
[(15, 0.015), (16, 0.1), (21, 0.016), (24, 0.192), (27, 0.016), (35, 0.016), (43, 0.018), (45, 0.052), (48, 0.019), (64, 0.22), (86, 0.042), (95, 0.033), (99, 0.154)]
simIndex simValue blogId blogTitle
same-blog 1 0.91911221 1653 andrew gelman stats-2013-01-04-Census dotmap
Introduction: Andrew Vande Moere points to this impressive interactive map from Brandon Martin-Anderson showing the locations of all the residents of the United States and Canada. It says, “The map has 341,817,095 dots – one for each person.” Not quite . . . I was hoping to zoom into my building (approximately 10 people live on our floor, I say approximately because two of the apartments are split between two floors and I’m not sure how they would assign the residents), but unfortunately our entire block is just a solid mass of black. Also, they put a few dots in the park and in the river by accident (presumably because the borders of the census blocks were specified only approximately). But, hey, no algorithm is perfect. It’s hard to know what to do about this. The idea of mapping every person is cool, but you’ll always run into trouble displaying densely populated areas. Smaller dots might work, but then that might depend on the screen being used for display.
2 0.85524118 1521 andrew gelman stats-2012-10-04-Columbo does posterior predictive checks
Introduction: I’m already on record as saying that Ronald Reagan was a statistician so I think this is ok too . . . Here’s what Columbo does. He hears the killer’s story and he takes it very seriously (it’s murder, and Columbo never jokes about murder), examines all its implications, and finds where it doesn’t fit the data. Then Columbo carefully examines the discrepancies, tries some model expansion, and eventually concludes that he’s proved there’s a problem. OK, now you’re saying: Yeah, yeah, sure, but how does that differ from any other fictional detective? The difference, I think, is that the tradition is for the detective to find clues and use these to come up with hypotheses, or to trap the killer via internal contradictions in his or her statement. I see Columbo is different—and more in keeping with chapter 6 of Bayesian Data Analysis—in that he is taking the killer’s story seriously and exploring all its implications. That’s the essence of predictive model checking: you t
3 0.80456531 118 andrew gelman stats-2010-06-30-Question & Answer Communities
Introduction: StackOverflow has been a popular community where software developers would help one another. Recently they raised some VC funding , and to make profits they are selling job postings and expanding the model to other areas. Metaoptimize LLC has started a similar website, using the open-source OSQA framework for such as statistics and machine learning. Here’s a description: You and other data geeks can ask and answer questions on machine learning, natural language processing, artificial intelligence, text analysis, information retrieval, search, data mining, statistical modeling, and data visualization. Here you can ask and answer questions, comment and vote for the questions of others and their answers. Both questions and answers can be revised and improved. Questions can be tagged with the relevant keywords to simplify future access and organize the accumulated material. If you work very hard on your questions and answers, you will receive badges like “Guru”, “Studen
4 0.78230727 985 andrew gelman stats-2011-11-01-Doug Schoen has 2 poll reports
Introduction: According to Chris Wilson , there are two versions of the report of the Occupy Wall Street poll from so-called hack pollster Doug Schoen. Here’s the report that Azi Paybarah says that Schoen sent to him, and here’s the final question from the poll: And here’s what’s on Schoen’s own website: Very similar, except for that last phrase, “no matter what the cost.” I have no idea which was actually asked to the survey participants, but it’s a reminder of the difficulties of public opinion research—sometimes you don’t even know what question was asked! I’m not implying anything sinister on Schoen’s part, it’s just interesting to see these two documents floating around. P.S. More here from Kaiser Fung on fundamental flaws with Schoen’s poll.
5 0.77163267 1637 andrew gelman stats-2012-12-24-Textbook for data visualization?
Introduction: Dave Choi writes: I’m building a course called “Exploring and visualizing data,” for Heinz college in Carnegie Mellon (public policy and information systems). Do you know any books that might be good for such a course? I’m hoping to get non-statisticians to appreciate the statistician’s point of view on this subject. I immediately thought of Bill Cleveland’s 1985 classic, The Elements of Graphing Data, but I wasn’t sure of what comes next. There are a lot of books on how to make graphics in R, but I’m not quite sure that’s the point. And I’m loath to recommend Tufte since it would be kinda scary if a student were to take all of his ideas too seriously. Any suggestions?
6 0.76727831 1109 andrew gelman stats-2012-01-09-Google correlate links statistics with minorities
7 0.75068796 595 andrew gelman stats-2011-02-28-What Zombies see in Scatterplots
8 0.7480951 1058 andrew gelman stats-2011-12-14-Higgs bozos: Rosencrantz and Guildenstern are spinning in their graves
9 0.74288654 1584 andrew gelman stats-2012-11-19-Tradeoffs in information graphics
12 0.73686731 599 andrew gelman stats-2011-03-03-Two interesting posts elsewhere on graphics
13 0.7361573 1293 andrew gelman stats-2012-05-01-Huff the Magic Dragon
14 0.73183167 1258 andrew gelman stats-2012-04-10-Why display 6 years instead of 30?
15 0.72889805 11 andrew gelman stats-2010-04-29-Auto-Gladwell, or Can fractals be used to predict human history?
17 0.72671932 953 andrew gelman stats-2011-10-11-Steve Jobs’s cancer and science-based medicine
18 0.72604918 846 andrew gelman stats-2011-08-09-Default priors update?
19 0.72594786 1019 andrew gelman stats-2011-11-19-Validation of Software for Bayesian Models Using Posterior Quantiles
20 0.72541386 1080 andrew gelman stats-2011-12-24-Latest in blog advertising