andrew_gelman_stats andrew_gelman_stats-2012 andrew_gelman_stats-2012-1498 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: I saw this graph posted by Tyler Cowen: and my first thought was that the bar plot should be replaced by a line plot: Six lines, one for each income category, with each line being a time series of these changes. With a line plot, you can more easily see each time series (these are hard to see in the barplot because you have to follow each color and jump from decade to decade) and also compare the patterns for each category. The line plot pretty much dominates the bar plot. At least that was the theory. Now here’s what actually happened. I downloaded the data as Excel files, saved them as csv, then read them into R. In all, it took close to an hour to get the data set up in the format that was needed to make the graphs. At this point it was pretty easy to make the line plot. But the result was disappointing: The six lines are hard to untangle (sure, a better color scheme might help, but it wouldn’t really solve the problem) and the graph as a whole is much l
sentIndex sentText sentNum sentScore
1 I saw this graph posted by Tyler Cowen: and my first thought was that the bar plot should be replaced by a line plot: Six lines, one for each income category, with each line being a time series of these changes. [sent-1, score-0.848]
2 With a line plot, you can more easily see each time series (these are hard to see in the barplot because you have to follow each color and jump from decade to decade) and also compare the patterns for each category. [sent-2, score-0.071]
3 The line plot pretty much dominates the bar plot. [sent-3, score-0.365]
4 At this point it was pretty easy to make the line plot. [sent-8, score-0.071]
5 But the result was disappointing: The six lines are hard to untangle (sure, a better color scheme might help, but it wouldn’t really solve the problem) and the graph as a whole is much less clear than the original bar plot. [sent-9, score-0.356]
6 Then I thought of plotting the incomes over time (all these income values are inflation-adjusted, of course): I like this one a lot. [sent-12, score-0.331]
7 In particular, it shows that the drop from 2000-2010 is really a drop since 2007. [sent-13, score-0.189]
8 Still, even the time-trends graph is not quite a replacement for the original bar plot which shows so much drama. [sent-16, score-0.426]
9 I think my recommended solution is to give the bar plot for the initial impression and then follow up immediately with the time-trends graph, which shows the big picture much more clearly. [sent-17, score-0.345]
10 The data are in the location indicated by the caption of the first graph above. [sent-20, score-0.081]
11 Here’s my (ugly) R code to make the graphs: n_years <- 64 # Save F02AR_2010 as csv file income_share <- read. [sent-21, score-0.111]
12 =year_income_mean)==0) year <- year_income_share else stop() income <- (income_share[,2:7]/100)*income_mean[,6] income[,1:5] <- income[,1:5]/. [sent-30, score-0.428]
13 pdf", height=6, width=8) y <- avg_changes x_labels <- rownames (y) line_labels <- c("Lowest fifth", "Second fifth", "Third fifth", "Fourth fifth", "Highest fifth", "Top 5 percent") n_x <- nrow (y) n_lines <- ncol (y) par (mar=c(3,4,1,1), mgp=c(2,. [sent-33, score-0.584]
14 01) plot (c(1,n_x), range(y), xlab="", ylab="Avg annual change", xaxt="n", yaxt="n", bty="l", type="n") y_ticks <- seq (-2,4,2) axis (2, y_ticks/100, paste (y_ticks, "%", sep="")) par (mgp=c(1,. [sent-35, score-0.86]
15 pdf", height=4, width=5) y <- avg_changes x_labels <- rownames (y) line_labels <- c("Lowest fifth", "Second fifth", "Third fifth", "Fourth fifth", "Highest fifth", "Top 5 percent") n_x <- nrow (y) n_lines <- ncol (y) par (mar=c(3,4,1,1), mgp=c(2,. [sent-38, score-0.584]
16 01, mfrow=c(3,2)) for (i in 1:n_lines){ plot (c(1,n_x), c(-. [sent-40, score-0.169]
17 05), xlab="", ylab="Avg annual change", xaxt="n", yaxt="n", bty="l", yaxs="i", type="n") y_ticks <- seq (-2,4,2) axis (2, y_ticks, paste (y_ticks, "%", sep="")) par (mgp=c(2,1. [sent-42, score-0.691]
18 5,0)) axis (1, 1:n_x, x_labels) lines (1:n_x, y[,i]) mtext (line_labels[i]) } mtext ("Average annual change in mean family income, 1950-2010,\nby quintile and for the top 5 percent", 3, -1, outer=TRUE) dev. [sent-43, score-0.99]
19 pdf", height=6, width=8) y <- income x_labels <- year line_labels <- c("Lowest fifth", "Second fifth", "Third fifth", "Fourth fifth", "Highest fifth", "Top 5 percent") n_x <- nrow (y) n_lines <- ncol (y) par (mar=c(3,4,1,1), mgp=c(2,. [sent-45, score-0.934]
20 88, line_labels[i]) } mtext ("Trends in mean family income, 1947-2010,\nby quintile and for the top 5 percent", 3, -1) dev. [sent-48, score-0.44]
wordName wordTfidf (topN-words)
[('fifth', 0.497), ('income', 0.331), ('nrow', 0.215), ('mgp', 0.185), ('seq', 0.185), ('plot', 0.169), ('par', 0.162), ('mtext', 0.157), ('axis', 0.15), ('nby', 0.129), ('ncol', 0.129), ('bar', 0.125), ('quintile', 0.118), ('xaxt', 0.118), ('yaxt', 0.118), ('tck', 0.111), ('col', 0.111), ('ylab', 0.111), ('csv', 0.111), ('percent', 0.107), ('bty', 0.106), ('xlab', 0.103), ('avg', 0.099), ('annual', 0.099), ('year', 0.097), ('paste', 0.095), ('decades', 0.093), ('lines', 0.089), ('width', 0.088), ('top', 0.088), ('sep', 0.086), ('substr', 0.086), ('fourth', 0.084), ('mar', 0.083), ('graph', 0.081), ('lowest', 0.081), ('pdf', 0.079), ('colors', 0.079), ('rownames', 0.078), ('family', 0.077), ('height', 0.075), ('line', 0.071), ('drop', 0.069), ('highest', 0.065), ('six', 0.061), ('third', 0.056), ('skip', 0.056), ('range', 0.055), ('change', 0.055), ('shows', 0.051)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 1498 andrew gelman stats-2012-09-16-Choices in graphing parallel time series
Introduction: I saw this graph posted by Tyler Cowen: and my first thought was that the bar plot should be replaced by a line plot: Six lines, one for each income category, with each line being a time series of these changes. With a line plot, you can more easily see each time series (these are hard to see in the barplot because you have to follow each color and jump from decade to decade) and also compare the patterns for each category. The line plot pretty much dominates the bar plot. At least that was the theory. Now here’s what actually happened. I downloaded the data as Excel files, saved them as csv, then read them into R. In all, it took close to an hour to get the data set up in the format that was needed to make the graphs. At this point it was pretty easy to make the line plot. But the result was disappointing: The six lines are hard to untangle (sure, a better color scheme might help, but it wouldn’t really solve the problem) and the graph as a whole is much l
2 0.36166871 672 andrew gelman stats-2011-04-20-The R code for those time-use graphs
Introduction: By popular demand, here’s my R script for the time-use graphs : # The data a1 <- c(4.2,3.2,11.1,1.3,2.2,2.0) a2 <- c(3.9,3.2,10.0,0.8,3.1,3.1) a3 <- c(6.3,2.5,9.8,0.9,2.2,2.4) a4 <- c(4.4,3.1,9.8,0.8,3.3,2.7) a5 <- c(4.8,3.0,9.9,0.7,3.3,2.4) a6 <- c(4.0,3.4,10.5,0.7,3.3,2.1) a <- rbind(a1,a2,a3,a4,a5,a6) avg <- colMeans (a) avg.array <- t (array (avg, rev(dim(a)))) diff <- a - avg.array country.name <- c("France", "Germany", "Japan", "Britain", "USA", "Turkey") # The line plots par (mfrow=c(2,3), mar=c(4,4,2,.5), mgp=c(2,.7,0), tck=-.02, oma=c(3,0,4,0), bg="gray96", fg="gray30") for (i in 1:6){ plot (c(1,6), c(-1,1.7), xlab="", ylab="", xaxt="n", yaxt="n", bty="l", type="n") lines (1:6, diff[i,], col="blue") points (1:6, diff[i,], pch=19, col="black") if (i>3){ axis (1, c(1,3,5), c ("Work,\nstudy", "Eat,\nsleep", "Leisure"), mgp=c(2,1.5,0), tck=0, cex.axis=1.2) axis (1, c(2,4,6), c ("Unpaid\nwork", "Personal\nCare", "Other"), mgp=c(2,1.5,0),
3 0.25728807 379 andrew gelman stats-2010-10-29-Could someone please set this as the new R default in base graphics?
Introduction: par (mar=c(3,3,2,1), mgp=c(2,.7,0), tck=-.01) Thank you.
4 0.15134293 687 andrew gelman stats-2011-04-29-Zero is zero
Introduction: Nathan Roseberry writes: I thought I had read on your blog that bar charts should always include zero on the scale, but a search of your blog (or google) didn’t return what I was looking for. Is it considered a best practice to always include zero on the axis for bar charts? Has this been written in a book? My reply: The idea is that the area of the bar represents “how many” or “how much.” The bar has to go down to 0 for that to work. You don’t have to have your y-axis go to zero, but if you want the axis to go anywhere else, don’t use a bar graph, use a line graph. Usually line graphs are better anyway. I’m sure this is all in a book somewhere.
5 0.1457963 1145 andrew gelman stats-2012-01-30-A tax on inequality, or a tax to keep inequality at the current level?
Introduction: My sometime coauthor Aaron Edlin cowrote (with Ian Ayres) an op-ed recommending a clever approach to taxing the rich. In their article they employ a charming bit of economics jargon, using the word “earn” to mean “how much money you make.” They “propose an automatic extra tax on the income of the top 1 percent of earners.” I assume their tax would apply to unearned income as well, but they (or their editor at the Times) are just so used to describing income as “earnings” that they just threw that in. Funny. Also, there’s a part of the article that doesn’t make sense to me. Ayres and Edlin first describe the level of inequality: In 1980 the average 1-percenter made 12.5 times the median income, but in 2006 (the latest year for which data is available) the average income of our richest 1 percent was a whopping 36 times greater than that of the median household. Then they lay out their solution: Enough is enough. . . . we propose an automatic extra tax on the income
6 0.14330381 495 andrew gelman stats-2010-12-31-“Threshold earners” and economic inequality
7 0.12102897 1334 andrew gelman stats-2012-05-21-Question 11 of my final exam for Design and Analysis of Sample Surveys
8 0.118363 1587 andrew gelman stats-2012-11-21-Red state blue state, or, states and counties are not persons
9 0.11040447 709 andrew gelman stats-2011-05-13-D. Kahneman serves up a wacky counterfactual
10 0.10194758 1821 andrew gelman stats-2013-04-24-My talk midtown this Friday noon (and at Columbia Monday afternoon)
11 0.097932279 1090 andrew gelman stats-2011-12-28-“. . . extending for dozens of pages”
12 0.09599416 1258 andrew gelman stats-2012-04-10-Why display 6 years instead of 30?
16 0.088153459 1767 andrew gelman stats-2013-03-17-The disappearing or non-disappearing middle class
17 0.088080928 673 andrew gelman stats-2011-04-20-Upper-income people still don’t realize they’re upper-income
19 0.085879035 1470 andrew gelman stats-2012-08-26-Graphs showing regression uncertainty: the code!
20 0.083213605 829 andrew gelman stats-2011-07-29-Infovis vs. statgraphics: A clear example of their different goals
topicId topicWeight
[(0, 0.095), (1, -0.049), (2, 0.056), (3, 0.048), (4, 0.083), (5, -0.083), (6, -0.021), (7, 0.03), (8, -0.024), (9, -0.0), (10, -0.023), (11, -0.011), (12, -0.037), (13, 0.039), (14, 0.037), (15, 0.003), (16, 0.062), (17, 0.02), (18, -0.029), (19, -0.018), (20, 0.104), (21, 0.045), (22, -0.009), (23, -0.025), (24, 0.035), (25, -0.013), (26, -0.01), (27, -0.047), (28, -0.018), (29, 0.013), (30, 0.038), (31, -0.029), (32, -0.055), (33, -0.026), (34, -0.029), (35, -0.044), (36, 0.026), (37, -0.017), (38, -0.035), (39, 0.009), (40, -0.021), (41, 0.039), (42, 0.012), (43, 0.058), (44, -0.057), (45, 0.003), (46, -0.023), (47, 0.034), (48, -0.011), (49, -0.024)]
simIndex simValue blogId blogTitle
same-blog 1 0.97526377 1498 andrew gelman stats-2012-09-16-Choices in graphing parallel time series
Introduction: I saw this graph posted by Tyler Cowen: and my first thought was that the bar plot should be replaced by a line plot: Six lines, one for each income category, with each line being a time series of these changes. With a line plot, you can more easily see each time series (these are hard to see in the barplot because you have to follow each color and jump from decade to decade) and also compare the patterns for each category. The line plot pretty much dominates the bar plot. At least that was the theory. Now here’s what actually happened. I downloaded the data as Excel files, saved them as csv, then read them into R. In all, it took close to an hour to get the data set up in the format that was needed to make the graphs. At this point it was pretty easy to make the line plot. But the result was disappointing: The six lines are hard to untangle (sure, a better color scheme might help, but it wouldn’t really solve the problem) and the graph as a whole is much l
2 0.84926945 672 andrew gelman stats-2011-04-20-The R code for those time-use graphs
Introduction: By popular demand, here’s my R script for the time-use graphs : # The data a1 <- c(4.2,3.2,11.1,1.3,2.2,2.0) a2 <- c(3.9,3.2,10.0,0.8,3.1,3.1) a3 <- c(6.3,2.5,9.8,0.9,2.2,2.4) a4 <- c(4.4,3.1,9.8,0.8,3.3,2.7) a5 <- c(4.8,3.0,9.9,0.7,3.3,2.4) a6 <- c(4.0,3.4,10.5,0.7,3.3,2.1) a <- rbind(a1,a2,a3,a4,a5,a6) avg <- colMeans (a) avg.array <- t (array (avg, rev(dim(a)))) diff <- a - avg.array country.name <- c("France", "Germany", "Japan", "Britain", "USA", "Turkey") # The line plots par (mfrow=c(2,3), mar=c(4,4,2,.5), mgp=c(2,.7,0), tck=-.02, oma=c(3,0,4,0), bg="gray96", fg="gray30") for (i in 1:6){ plot (c(1,6), c(-1,1.7), xlab="", ylab="", xaxt="n", yaxt="n", bty="l", type="n") lines (1:6, diff[i,], col="blue") points (1:6, diff[i,], pch=19, col="black") if (i>3){ axis (1, c(1,3,5), c ("Work,\nstudy", "Eat,\nsleep", "Leisure"), mgp=c(2,1.5,0), tck=0, cex.axis=1.2) axis (1, c(2,4,6), c ("Unpaid\nwork", "Personal\nCare", "Other"), mgp=c(2,1.5,0),
3 0.75429821 1258 andrew gelman stats-2012-04-10-Why display 6 years instead of 30?
Introduction: I continue to be the go-to guy for bad graphs. Today (i.e., 22 Feb), I received an email from Gary Rosin: I [Rosin] thought you might be interested in this graph showing the decline in median prices of homes since 1997. It exaggerates the proportions by using $150,000 as the floor, rather than zero. Indeed. Here’s the graph: A line plot, rather than a bar plot, would be appropriate here. Also, it’s weird that the headline says “10 years” but the graph has only 6 years. Why not give some perspective and show, say, 30 years?
Introduction: Kaiser Fung discusses the following graph that is captioned, “A study of 54 nations–ranked below–found that those with more progressive tax rates had happier citizens, on average.” As Kaiser writes, “from a purely graphical perspective, the chart is well executed . . . they have 54 points, and the chart still doesn’t look too crammed . . .” But he also points out that the graph’s implicit claims (that tax rates can explain happiness or cause more happiness) are not supported. Kaiser and I are not being picky-picky-picky here. Taken literally, the graph title says nothing about causation, but I think the phrasing implies it. Also, from a purely descriptive perspective, the graph is somewhat at war with its caption. The caption announces a relationship, but in the graph, the x and y variables have only a very weak correlation. The caption says that happiness and progressive tax rates go together, but the graph uses the U.S. as a baseline, and when you move from the U.S
5 0.73877841 915 andrew gelman stats-2011-09-17-(Worst) graph of the year
Introduction: This (forwarded to me from Jeff, from a powerpoint by Willam Gawthrop) wins not on form but on content: Really this graph should stand alone but it’s so wonderful that I can’t resist pointing out a few things: - The gap between 610 and 622 A.D. seems to be about the same as the previous 600 years, and only a little less than the 1400 years before that. - “Pious and devout” Jews are portrayed as having steadily increased in nonviolence up to the present day. Been to Israel lately? - I assume the line labeled “Bible” is referring to Christians? I’m sort of amazed to see pious and devout Christians listed as being maximally violent at the beginning. Huh? I thought Christ was supposed to be a nonviolent, mellow dude. The line starts at 3 B.C., implying that baby Jesus was at the extreme of violence. Gong forward, we can learn from the graph that pious and devout Christians in 1492 or 1618, say, were much more peaceful than Jesus and his crew. - Most amusingly g
6 0.71704954 1011 andrew gelman stats-2011-11-15-World record running times vs. distance
7 0.71347082 670 andrew gelman stats-2011-04-20-Attractive but hard-to-read graph could be made much much better
8 0.70008725 671 andrew gelman stats-2011-04-20-One more time-use graph
9 0.69934106 2146 andrew gelman stats-2013-12-24-NYT version of birthday graph
10 0.68699038 2154 andrew gelman stats-2013-12-30-Bill Gates’s favorite graph of the year
11 0.67276585 1439 andrew gelman stats-2012-08-01-A book with a bunch of simple graphs
12 0.66703039 829 andrew gelman stats-2011-07-29-Infovis vs. statgraphics: A clear example of their different goals
13 0.66682506 488 andrew gelman stats-2010-12-27-Graph of the year
15 0.65935248 1684 andrew gelman stats-2013-01-20-Ugly ugly ugly
16 0.65376157 502 andrew gelman stats-2011-01-04-Cash in, cash out graph
17 0.65351397 2308 andrew gelman stats-2014-04-27-White stripes and dead armadillos
20 0.64284384 1478 andrew gelman stats-2012-08-31-Watercolor regression
topicId topicWeight
[(15, 0.02), (16, 0.059), (21, 0.139), (24, 0.081), (26, 0.012), (27, 0.015), (34, 0.016), (38, 0.166), (45, 0.039), (53, 0.027), (59, 0.02), (73, 0.059), (76, 0.04), (95, 0.038), (99, 0.14)]
simIndex simValue blogId blogTitle
same-blog 1 0.88832808 1498 andrew gelman stats-2012-09-16-Choices in graphing parallel time series
Introduction: I saw this graph posted by Tyler Cowen: and my first thought was that the bar plot should be replaced by a line plot: Six lines, one for each income category, with each line being a time series of these changes. With a line plot, you can more easily see each time series (these are hard to see in the barplot because you have to follow each color and jump from decade to decade) and also compare the patterns for each category. The line plot pretty much dominates the bar plot. At least that was the theory. Now here’s what actually happened. I downloaded the data as Excel files, saved them as csv, then read them into R. In all, it took close to an hour to get the data set up in the format that was needed to make the graphs. At this point it was pretty easy to make the line plot. But the result was disappointing: The six lines are hard to untangle (sure, a better color scheme might help, but it wouldn’t really solve the problem) and the graph as a whole is much l
2 0.80025941 1874 andrew gelman stats-2013-05-28-Nostalgia
Introduction: Saw Argo the other day, was impressed by the way it was filmed in such a 70s style, sorta like that movie The Limey or an episode of the Rockford Files. I also felt nostalgia for that relatively nonviolent era. All those hostages and nobody was killed. It’s a good thing the Ayatollah didn’t have some fundamentalist Shiite equivalent of John Yoo telling him to waterboard everybody. At the time we were all so angry and upset about the hostage-taking, but from the perspective of our suicide-bomber era, that whole hostage episode seems so comfortingly mild.
3 0.76161915 672 andrew gelman stats-2011-04-20-The R code for those time-use graphs
Introduction: By popular demand, here’s my R script for the time-use graphs : # The data a1 <- c(4.2,3.2,11.1,1.3,2.2,2.0) a2 <- c(3.9,3.2,10.0,0.8,3.1,3.1) a3 <- c(6.3,2.5,9.8,0.9,2.2,2.4) a4 <- c(4.4,3.1,9.8,0.8,3.3,2.7) a5 <- c(4.8,3.0,9.9,0.7,3.3,2.4) a6 <- c(4.0,3.4,10.5,0.7,3.3,2.1) a <- rbind(a1,a2,a3,a4,a5,a6) avg <- colMeans (a) avg.array <- t (array (avg, rev(dim(a)))) diff <- a - avg.array country.name <- c("France", "Germany", "Japan", "Britain", "USA", "Turkey") # The line plots par (mfrow=c(2,3), mar=c(4,4,2,.5), mgp=c(2,.7,0), tck=-.02, oma=c(3,0,4,0), bg="gray96", fg="gray30") for (i in 1:6){ plot (c(1,6), c(-1,1.7), xlab="", ylab="", xaxt="n", yaxt="n", bty="l", type="n") lines (1:6, diff[i,], col="blue") points (1:6, diff[i,], pch=19, col="black") if (i>3){ axis (1, c(1,3,5), c ("Work,\nstudy", "Eat,\nsleep", "Leisure"), mgp=c(2,1.5,0), tck=0, cex.axis=1.2) axis (1, c(2,4,6), c ("Unpaid\nwork", "Personal\nCare", "Other"), mgp=c(2,1.5,0),
4 0.75412083 894 andrew gelman stats-2011-09-07-Hipmunk FAIL: Graphics without content is not enough
Introduction: I love a good GUI but not if it doesn’t give me the information I need. I again tried Hipmunk and it again failed (this time for a trip to Baltimore where it gave only a useless subset of the available Amtrak trains). I don’t know anything about the internet biz. What I’m guessing is that they set up this cool website that is pretty much functional, with the goal of selling it for a few million dollars to Travelocity or Expedia or Kayak. What I’m wondering is, why haven’t they made the deal already? Hipmunk’s GUI is great. The site is useless because it’s missing so many flights, but if you put it in an actual travel site such as Expedia, it would be great. It’s enough to make me want to hit someone with an i-phone . . .
5 0.73430133 1073 andrew gelman stats-2011-12-20-Not quite getting the point
Introduction: I gave this talk the other day and afterwards, a white guy came up to me and said he thought it was no coincidence that the researcher who made the mistake was “Oriental.” He then went on for about 5 minutes explaining his theory. I couldn’t keep myself from laughing—I had to start coughing into a napkin to hide it.
7 0.72713685 151 andrew gelman stats-2010-07-16-Wanted: Probability distributions for rank orderings
8 0.72571695 393 andrew gelman stats-2010-11-04-Estimating the effect of A on B, and also the effect of B on A
9 0.72504544 1275 andrew gelman stats-2012-04-22-Please stop me before I barf again
11 0.71758711 432 andrew gelman stats-2010-11-27-Neumann update
12 0.71726143 1401 andrew gelman stats-2012-06-30-David Hogg on statistics
13 0.71475571 514 andrew gelman stats-2011-01-13-News coverage of statistical issues…how did I do?
14 0.71068966 527 andrew gelman stats-2011-01-20-Cars vs. trucks
15 0.70700097 62 andrew gelman stats-2010-06-01-Two Postdoc Positions Available on Bayesian Hierarchical Modeling
17 0.7049107 1675 andrew gelman stats-2013-01-15-“10 Things You Need to Know About Causal Effects”
19 0.69794035 1857 andrew gelman stats-2013-05-15-Does quantum uncertainty have a place in everyday applied statistics?
20 0.69284678 659 andrew gelman stats-2011-04-13-Jim Campbell argues that Larry Bartels’s “Unequal Democracy” findings are not robust