andrew_gelman_stats andrew_gelman_stats-2010 andrew_gelman_stats-2010-372 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: After our recent discussion of semigraphic displays, Jay Ulfelder sent along a semigraphic table from his recent book. He notes, “When countries are the units of analysis, it’s nice that you can use three-letter codes, so all the proper names have the same visual weight.” Ultimately I think that graphs win over tables for display. However in our work we spend a lot of time looking at raw data, often simply to understand what data we have. This use of tables has, I think, been forgotten in the statistical graphics literature. So I’d like to refocus the eternal tables vs. graphs discussion. If the goal is to present information, comparisons, relationships, models, data, etc etc, graphs win. Forget about tables. But . . . when you’re looking at your data, it can often help to see the raw numbers. Once you’re looking at numbers, it makes sense to organize them. Even a displayed matrix in R is a form of table, after all. And once you’re making a table, it can be sensible to
sentIndex sentText sentNum sentScore
1 After our recent discussion of semigraphic displays, Jay Ulfelder sent along a semigraphic table from his recent book. [sent-1, score-1.569]
2 He notes, “When countries are the units of analysis, it’s nice that you can use three-letter codes, so all the proper names have the same visual weight. [sent-2, score-0.648]
3 ” Ultimately I think that graphs win over tables for display. [sent-3, score-0.728]
4 However in our work we spend a lot of time looking at raw data, often simply to understand what data we have. [sent-4, score-0.745]
5 This use of tables has, I think, been forgotten in the statistical graphics literature. [sent-5, score-0.656]
6 If the goal is to present information, comparisons, relationships, models, data, etc etc, graphs win. [sent-8, score-0.484]
7 when you’re looking at your data, it can often help to see the raw numbers. [sent-13, score-0.514]
8 Once you’re looking at numbers, it makes sense to organize them. [sent-14, score-0.309]
9 Even a displayed matrix in R is a form of table, after all. [sent-15, score-0.293]
10 And once you’re making a table, it can be sensible to set it up as a semigraphic display. [sent-16, score-0.712]
11 So if there is room for tables in statistics, that’s where they go, I think. [sent-17, score-0.483]
wordName wordTfidf (topN-words)
[('semigraphic', 0.498), ('tables', 0.388), ('table', 0.264), ('graphs', 0.194), ('raw', 0.194), ('looking', 0.17), ('eternal', 0.159), ('ulfelder', 0.159), ('etc', 0.149), ('codes', 0.149), ('organize', 0.139), ('displayed', 0.123), ('forgotten', 0.119), ('units', 0.116), ('displays', 0.114), ('sensible', 0.111), ('relationships', 0.111), ('jay', 0.11), ('proper', 0.108), ('matrix', 0.104), ('forget', 0.098), ('visual', 0.096), ('data', 0.096), ('room', 0.095), ('re', 0.094), ('often', 0.093), ('notes', 0.092), ('names', 0.091), ('recent', 0.091), ('win', 0.089), ('countries', 0.087), ('nice', 0.082), ('graphics', 0.081), ('ultimately', 0.079), ('spend', 0.077), ('comparisons', 0.074), ('present', 0.072), ('goal', 0.069), ('sent', 0.069), ('use', 0.068), ('form', 0.066), ('simply', 0.063), ('numbers', 0.061), ('however', 0.059), ('along', 0.058), ('help', 0.057), ('think', 0.057), ('making', 0.053), ('understand', 0.052), ('set', 0.05)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 372 andrew gelman stats-2010-10-27-A use for tables (really)
Introduction: After our recent discussion of semigraphic displays, Jay Ulfelder sent along a semigraphic table from his recent book. He notes, “When countries are the units of analysis, it’s nice that you can use three-letter codes, so all the proper names have the same visual weight.” Ultimately I think that graphs win over tables for display. However in our work we spend a lot of time looking at raw data, often simply to understand what data we have. This use of tables has, I think, been forgotten in the statistical graphics literature. So I’d like to refocus the eternal tables vs. graphs discussion. If the goal is to present information, comparisons, relationships, models, data, etc etc, graphs win. Forget about tables. But . . . when you’re looking at your data, it can often help to see the raw numbers. Once you’re looking at numbers, it makes sense to organize them. Even a displayed matrix in R is a form of table, after all. And once you’re making a table, it can be sensible to
2 0.1999681 296 andrew gelman stats-2010-09-26-A simple semigraphic display
Introduction: John Tukey wrote about semigraphic displays. I think his most famous effort in that area–the stem-and-leaf plot–is just horrible. But the general idea of viewing tables as graphs is good, and it’s been a success at least since the early 1900s, when Ramanujan famously intuited the behavior of the partition number by seeing a table of numbers and implicitly reading it as a graph on the logarithmic scale. To return to the present, Steve Roth sent me a link to these table/graphs that he made: Europe vs. US: Who’s Winning? and State Taxes and Prosperity, Revisited . He writes: I [Roth] find the layout with the red/black gives a simultaneous numeric and graphical representation of the situation, and condenses a lot of immediately apprehensible info into a small space. It also helps me avoid at least one axis of cherry-picking (periods), which I am as prone to as all humans are. Any thoughts welcome. In particular, do you think the average and count aggregates at the bot
3 0.15858734 1403 andrew gelman stats-2012-07-02-Moving beyond hopeless graphics
Introduction: I was at a talk awhile ago where the speaker presented tables with 4, 5, 6, even 8 significant digits even though, as is usual, only the first or second digit of each number conveyed any useful information. A graph would be better, but even if you’re too lazy to make a plot, a bit of rounding would seem to be required. I mentioned this to a colleague, who responded: I don’t know how to stop this practice. Logic doesn’t work. Maybe ridicule? Best hope is the departure from field who do it. (Theories don’t die, but the people who follow those theories retire.) Another possibility, I think, is helpful software defaults. If we can get to the people who write the software, maybe we could have some impact. Once the software is written, however, it’s probably too late. I’m not far from the center of the R universe, but I don’t know if I’ll ever succeed in my goals of increasing the default number of histogram bars or reducing the default number of decimal places in regression
4 0.15106978 855 andrew gelman stats-2011-08-16-Infovis and statgraphics update update
Introduction: To continue our discussion from last week , consider three positions regarding the display of information: (a) The traditional tabular approach. This is how most statisticians, econometricians, political scientists, sociologists, etc., seem to operate. They understand the appeal of a pretty graph, and they’re willing to plot some data as part of an exploratory data analysis, but they see their serious research as leading to numerical estimates, p-values, tables of numbers. These people might use a graph to illustrate their points but they don’t see them as necessary in their research. (b) Statistical graphics as performed by Howard Wainer, Bill Cleveland, Dianne Cook, etc. They–we–see graphics as central to the process of statistical modeling and data analysis and are interested in graphs (static and dynamic) that display every data point as transparently as possible. (c) Information visualization or infographics, as performed by graphics designers and statisticians who are
Introduction: I continue to struggle to convey my thoughts on statistical graphics so I’ll try another approach, this time giving my own story. For newcomers to this discussion: the background is that Antony Unwin and I wrote an article on the different goals embodied in information visualization and statistical graphics, but I have difficulty communicating on this point with the infovis people. Maybe if I tell my own story, and then they tell their stories, this will point a way forward to a more constructive discussion. So here goes. I majored in physics in college and I worked in a couple of research labs during the summer. Physicists graph everything. I did most of my plotting on graph paper–this continued through my second year of grad school–and became expert at putting points at 1/5, 2/5, 3/5, and 4/5 between the x and y grid lines. In grad school in statistics, I continued my physics habits and graphed everything I could. I did notice, though, that the faculty and the other
6 0.13362393 1775 andrew gelman stats-2013-03-23-In which I disagree with John Maynard Keynes
7 0.13257973 1275 andrew gelman stats-2012-04-22-Please stop me before I barf again
8 0.13173099 736 andrew gelman stats-2011-05-29-Response to “Why Tables Are Really Much Better Than Graphs”
10 0.12627938 1078 andrew gelman stats-2011-12-22-Tables as graphs: The Ramanujan principle
11 0.12446038 2266 andrew gelman stats-2014-03-25-A statistical graphics course and statistical graphics advice
12 0.12014011 2172 andrew gelman stats-2014-01-14-Advice on writing research articles
14 0.11795542 1413 andrew gelman stats-2012-07-11-News flash: Probability and statistics are hard to understand
15 0.1139527 319 andrew gelman stats-2010-10-04-“Who owns Congress”
16 0.10997504 2279 andrew gelman stats-2014-04-02-Am I too negative?
17 0.10843408 302 andrew gelman stats-2010-09-28-This is a link to a news article about a scientific paper
18 0.10454371 1439 andrew gelman stats-2012-08-01-A book with a bunch of simple graphs
20 0.099212497 1176 andrew gelman stats-2012-02-19-Standardized writing styles and standardized graphing styles
topicId topicWeight
[(0, 0.139), (1, -0.001), (2, -0.032), (3, 0.053), (4, 0.126), (5, -0.128), (6, -0.098), (7, 0.045), (8, -0.04), (9, 0.012), (10, 0.021), (11, 0.008), (12, -0.033), (13, -0.007), (14, 0.005), (15, -0.004), (16, -0.003), (17, -0.02), (18, 0.008), (19, 0.008), (20, 0.015), (21, 0.022), (22, 0.0), (23, 0.031), (24, -0.035), (25, -0.013), (26, 0.037), (27, 0.013), (28, 0.009), (29, 0.024), (30, 0.008), (31, 0.025), (32, -0.017), (33, 0.021), (34, 0.009), (35, 0.025), (36, 0.015), (37, 0.009), (38, 0.004), (39, -0.014), (40, 0.05), (41, -0.008), (42, -0.032), (43, 0.013), (44, -0.012), (45, -0.029), (46, -0.057), (47, 0.005), (48, 0.025), (49, -0.001)]
simIndex simValue blogId blogTitle
same-blog 1 0.95139843 372 andrew gelman stats-2010-10-27-A use for tables (really)
Introduction: After our recent discussion of semigraphic displays, Jay Ulfelder sent along a semigraphic table from his recent book. He notes, “When countries are the units of analysis, it’s nice that you can use three-letter codes, so all the proper names have the same visual weight.” Ultimately I think that graphs win over tables for display. However in our work we spend a lot of time looking at raw data, often simply to understand what data we have. This use of tables has, I think, been forgotten in the statistical graphics literature. So I’d like to refocus the eternal tables vs. graphs discussion. If the goal is to present information, comparisons, relationships, models, data, etc etc, graphs win. Forget about tables. But . . . when you’re looking at your data, it can often help to see the raw numbers. Once you’re looking at numbers, it makes sense to organize them. Even a displayed matrix in R is a form of table, after all. And once you’re making a table, it can be sensible to
2 0.87106824 855 andrew gelman stats-2011-08-16-Infovis and statgraphics update update
Introduction: To continue our discussion from last week , consider three positions regarding the display of information: (a) The traditional tabular approach. This is how most statisticians, econometricians, political scientists, sociologists, etc., seem to operate. They understand the appeal of a pretty graph, and they’re willing to plot some data as part of an exploratory data analysis, but they see their serious research as leading to numerical estimates, p-values, tables of numbers. These people might use a graph to illustrate their points but they don’t see them as necessary in their research. (b) Statistical graphics as performed by Howard Wainer, Bill Cleveland, Dianne Cook, etc. They–we–see graphics as central to the process of statistical modeling and data analysis and are interested in graphs (static and dynamic) that display every data point as transparently as possible. (c) Information visualization or infographics, as performed by graphics designers and statisticians who are
Introduction: I continue to struggle to convey my thoughts on statistical graphics so I’ll try another approach, this time giving my own story. For newcomers to this discussion: the background is that Antony Unwin and I wrote an article on the different goals embodied in information visualization and statistical graphics, but I have difficulty communicating on this point with the infovis people. Maybe if I tell my own story, and then they tell their stories, this will point a way forward to a more constructive discussion. So here goes. I majored in physics in college and I worked in a couple of research labs during the summer. Physicists graph everything. I did most of my plotting on graph paper–this continued through my second year of grad school–and became expert at putting points at 1/5, 2/5, 3/5, and 4/5 between the x and y grid lines. In grad school in statistics, I continued my physics habits and graphed everything I could. I did notice, though, that the faculty and the other
4 0.86231887 2266 andrew gelman stats-2014-03-25-A statistical graphics course and statistical graphics advice
Introduction: Dean Eckles writes: Some of my coworkers at Facebook and I have worked with Udacity to create an online course on exploratory data analysis, including using data visualizations in R as part of EDA. The course has now launched at https://www.udacity.com/course/ud651 so anyone can take it for free. And Kaiser Fung has reviewed it . So definitely feel free to promote it! Criticism is also welcome (we are still fine-tuning things and adding more notes throughout). I wrote some more comments about the course here , including highlighting the interviews with my great coworkers. I didn’t have a chance to look at the course so instead I responded with some generic comments about eda and visualization (in no particular order): - Think of a graph as a comparison. All graphs are comparison (indeed, all statistical analyses are comparisons). If you already have the graph in mind, think of what comparisons it’s enabling. Or if you haven’t settled on the graph yet, think of what
5 0.84811807 1606 andrew gelman stats-2012-12-05-The Grinch Comes Back
Introduction: Wayne Folta writes: In keeping with your interest in graphs, this might interest or inspire you, if you haven’t seen it already, which features 20 scientific graphs that Wired likes, ranging from drawn illustrations to trajectory plots. My reaction: I looked at the first 10. I liked 1, 3, and 5, I didn’t like 2, 7, 8, 9, and 10. I have neutral feelings about 4 and 6. I won’t explain all these feelings, but, just for example, from my perspective, image 9 fails as a statistical graphic (although it might be fine as an infovis) by trying to cram to much into a single image. I don’t think it works to have all the colors on the single wheels; instead I’d prefer some sort of grid of images. Also, I don’t see the point of the circular display. That makes no sense at all; it’s a misleading feature. That said, the graphs I dislike can still be fine for their purpose. A graph in a journal such as Science or Nature is meant to grab the eye of a busy reader (or to go viral on
6 0.83809453 37 andrew gelman stats-2010-05-17-Is chartjunk really “more useful” than plain graphs? I don’t think so.
7 0.83318901 319 andrew gelman stats-2010-10-04-“Who owns Congress”
8 0.82524574 1584 andrew gelman stats-2012-11-19-Tradeoffs in information graphics
9 0.82228959 736 andrew gelman stats-2011-05-29-Response to “Why Tables Are Really Much Better Than Graphs”
10 0.81814539 1896 andrew gelman stats-2013-06-13-Against the myth of the heroic visualization
11 0.81518227 61 andrew gelman stats-2010-05-31-A data visualization manifesto
13 0.81138849 1684 andrew gelman stats-2013-01-20-Ugly ugly ugly
14 0.81052202 252 andrew gelman stats-2010-09-02-R needs a good function to make line plots
15 0.80923945 1439 andrew gelman stats-2012-08-01-A book with a bunch of simple graphs
16 0.80628705 1775 andrew gelman stats-2013-03-23-In which I disagree with John Maynard Keynes
17 0.80094343 1604 andrew gelman stats-2012-12-04-An epithet I can live with
18 0.79666328 1275 andrew gelman stats-2012-04-22-Please stop me before I barf again
19 0.79583031 488 andrew gelman stats-2010-12-27-Graph of the year
20 0.77858758 829 andrew gelman stats-2011-07-29-Infovis vs. statgraphics: A clear example of their different goals
topicId topicWeight
[(4, 0.033), (9, 0.017), (12, 0.232), (16, 0.022), (17, 0.024), (21, 0.017), (24, 0.15), (76, 0.037), (88, 0.02), (89, 0.043), (99, 0.276)]
simIndex simValue blogId blogTitle
1 0.96553665 677 andrew gelman stats-2011-04-24-My NOAA story
Introduction: I recently learned we have some readers at the National Oceanic and Atmospheric Administration so I thought I’d share an old story. About 35 years ago my brother worked briefly as a clerk at NOAA in their D.C. (or maybe it was D.C.-area) office. His job was to enter the weather numbers that came in. He had a boss who was very orderly. At one point there was a hurricane that wiped out some weather station in the Caribbean, and his boss told him to put in the numbers anyway. My brother protested that they didn’t have the data, to which his boss replied: “I know what the numbers are.” Nowadays we call this sort of thing “imputation” and we like it. But not in the raw data! I bet nowadays they have an NA code.
2 0.93818307 211 andrew gelman stats-2010-08-17-Deducer update
Introduction: A year ago we blogged about Ian Fellows’s R Gui called Deducer (oops, my bad, I meant to link to this ). Fellows sends in this update: Since version 0.1, I [Fellows] have added: 1. A nice plug-in interface, so that people can extend Deducer’s capability without leaving the comfort of R. (see: http://www.deducer.org/pmwiki/pmwiki.php?n=Main.Development ) 2. Several new dialogs. 3. A one-step installer for windows. 4. A plug-in package (DeducerExtras) which extends the scope of analyses covered. 5. A plotting GUI that can create anything from simple histograms to complex custom graphics. Deducer is designed to be a free easy to use alternative to proprietary data analysis software such as SPSS, JMP, and Minitab. It has a menu system to do common data manipulation and analysis tasks, and an excel-like spreadsheet in which to view and edit data frames. The goal of the project is two fold. Provide an intuitive interface so that non-technical users can learn and p
3 0.90412223 1119 andrew gelman stats-2012-01-15-Excellence in Statistical Reporting Award
Introduction: The American Statistical Association is seeking nominations for its annual Excellence in Statistical Reporting Award . The award was created in 2004 to encourage and recognize members of the communications media who have best displayed an informed interest in the science of statistics and its role in public life. The award can be given for a single statistical article or for a body of work. Former winners of the award include: Felix Salmon , financial blogger, 2010; Sharon Begley , Newsweek, 2009; Mark Buchanan, New York Times, 2008; John Berry, Bloomberg News, 2005; and Gina Kolata, New York Times, 2004. If anyone has any suggestions for the 2012 award, feel free to post in the comments or email me.
same-blog 4 0.89181507 372 andrew gelman stats-2010-10-27-A use for tables (really)
Introduction: After our recent discussion of semigraphic displays, Jay Ulfelder sent along a semigraphic table from his recent book. He notes, “When countries are the units of analysis, it’s nice that you can use three-letter codes, so all the proper names have the same visual weight.” Ultimately I think that graphs win over tables for display. However in our work we spend a lot of time looking at raw data, often simply to understand what data we have. This use of tables has, I think, been forgotten in the statistical graphics literature. So I’d like to refocus the eternal tables vs. graphs discussion. If the goal is to present information, comparisons, relationships, models, data, etc etc, graphs win. Forget about tables. But . . . when you’re looking at your data, it can often help to see the raw numbers. Once you’re looking at numbers, it makes sense to organize them. Even a displayed matrix in R is a form of table, after all. And once you’re making a table, it can be sensible to
Introduction: Tyler Cowen links to an interesting article by Terry Teachout on David Mamet’s political conservatism. I don’t think of playwrights as gurus, but I do find it interesting to consider the political orientations of authors and celebrities . I have only one problem with Teachout’s thought-provoking article. He writes: As early as 2002 . . . Arguing that “the Western press [had] embraced antisemitism as the new black,” Mamet drew a sharp contrast between that trendy distaste for Jews and the harsh realities of daily life in Israel . . . In 2006, Mamet published a collection of essays called The Wicked Son: Anti-Semitism, Jewish Self-Hatred and the Jews that made the point even more bluntly. “The Jewish State,” he wrote, “has offered the Arab world peace since 1948; it has received war, and slaughter, and the rhetoric of annihilation.” He went on to argue that secularized Jews who “reject their birthright of ‘connection to the Divine’” succumb in time to a self-hatred tha
6 0.86452687 1282 andrew gelman stats-2012-04-26-Bad news about (some) statisticians
7 0.85018688 1660 andrew gelman stats-2013-01-08-Bayesian, Permutable Symmetries
8 0.84938198 1597 andrew gelman stats-2012-11-29-What is expected of a consultant
9 0.84090912 1858 andrew gelman stats-2013-05-15-Reputations changeable, situations tolerable
10 0.83477867 434 andrew gelman stats-2010-11-28-When Small Numbers Lead to Big Errors
11 0.83234167 840 andrew gelman stats-2011-08-05-An example of Bayesian model averaging
12 0.83214378 239 andrew gelman stats-2010-08-28-The mathematics of democracy
13 0.82589638 1871 andrew gelman stats-2013-05-27-Annals of spam
14 0.82355255 752 andrew gelman stats-2011-06-08-Traffic Prediction
15 0.81391108 1348 andrew gelman stats-2012-05-27-Question 17 of my final exam for Design and Analysis of Sample Surveys
16 0.81104577 244 andrew gelman stats-2010-08-30-Useful models, model checking, and external validation: a mini-discussion
17 0.8056277 2287 andrew gelman stats-2014-04-09-Advice: positive-sum, zero-sum, or negative-sum
18 0.80441689 1564 andrew gelman stats-2012-11-06-Choose your default, or your default will choose you (election forecasting edition)
19 0.80426741 2010 andrew gelman stats-2013-09-06-Would today’s captains of industry be happier in a 1950s-style world?
20 0.80369473 2142 andrew gelman stats-2013-12-21-Chasing the noise