andrew_gelman_stats andrew_gelman_stats-2011 andrew_gelman_stats-2011-496 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: The great statistician John Tukey, in his writings from the 1970s onward (and maybe earlier) was time and again making the implicit argument that you should evaluate a statistical method based on what it does; you should {\em not} be staring at the model that purportedly underlies the method, trying to determine if the model is “true” (or “true enough”). Tukey’s point was that models can be great to inspire methods, but the model is the scaffolding; it is the method that is the building you have to live in. I don’t fully agree with this philosophy–I think models are a good way to understand data and also often connect usefully to scientific models (although not as cleanly as is thought by our friends who work in economics or statistical hypothesis testing). To put it another way: What makes a building good? A building is good if it is useful. If a building is useful, people will use it. Eventually improvements will be needed, partly because the building will get worn down, part
sentIndex sentText sentNum sentScore
1 Tukey’s point was that models can be great to inspire methods, but the model is the scaffolding; it is the method that is the building you have to live in. [sent-2, score-1.068]
2 I don’t fully agree with this philosophy–I think models are a good way to understand data and also often connect usefully to scientific models (although not as cleanly as is thought by our friends who work in economics or statistical hypothesis testing). [sent-3, score-0.649]
3 To put it another way: What makes a building good? [sent-4, score-0.299]
4 Eventually improvements will be needed, partly because the building will get worn down, partly because the interactions between the many users will inspire new, unforeseen uses, partly for the simple reason that if a building is popular, more space will be desired. [sent-7, score-1.868]
5 And, at that point, wouldn’t it be great if some scaffolding were already around? [sent-9, score-0.495]
6 if we now switch the analogy back from buildings to statistical methods, that scaffolding is the model that was used in constructing the method in the first place. [sent-13, score-1.098]
7 In fact, it is the most useful, wonderful statistical methods that get the most use and need improvements most frequently. [sent-15, score-0.461]
8 So I like the model and I don’t see the virtue in hiding it and letting the method stand alone. [sent-16, score-0.587]
9 The model is the basis for future improvements in many directions. [sent-17, score-0.392]
10 And this is one reason why I think that one of the most exciting areas in statistical research is the systematization of model building. [sent-18, score-0.378]
11 But, even though I don’t agree with the implicit philosophy of late Tukey (I don’t agree with the philosophy of early Tukey either, with all that multiple comparisons stuff), I think (of course) that he made hugely important contributions. [sent-20, score-0.794]
12 So I’d like to have this philosophy out there for statisticians and users to evaluate on their own. [sent-21, score-0.435]
13 I have not ever seen Tukey’s ideas expressed in this way before (and they’re just my own imputation; I only met Tukey once, many years ago, and we spoke for about 30 seconds), so I’m posting them here, on the first day of this new decade. [sent-22, score-0.234]
wordName wordTfidf (topN-words)
[('tukey', 0.417), ('scaffolding', 0.408), ('building', 0.299), ('method', 0.223), ('philosophy', 0.216), ('improvements', 0.195), ('partly', 0.163), ('inspire', 0.16), ('model', 0.141), ('statistical', 0.118), ('users', 0.11), ('implicit', 0.109), ('evaluate', 0.109), ('models', 0.102), ('worn', 0.102), ('unforeseen', 0.096), ('cleanly', 0.096), ('onward', 0.096), ('underlies', 0.096), ('purportedly', 0.092), ('agree', 0.09), ('hiding', 0.089), ('methods', 0.089), ('great', 0.087), ('staring', 0.086), ('usefully', 0.079), ('em', 0.075), ('buildings', 0.074), ('hugely', 0.073), ('seconds', 0.072), ('constructing', 0.07), ('virtue', 0.067), ('letting', 0.067), ('spoke', 0.066), ('writings', 0.066), ('useful', 0.065), ('imputation', 0.064), ('switch', 0.064), ('true', 0.063), ('reason', 0.062), ('connect', 0.062), ('decade', 0.061), ('wonderful', 0.059), ('network', 0.058), ('exciting', 0.057), ('met', 0.056), ('expressed', 0.056), ('many', 0.056), ('point', 0.056), ('eventually', 0.056)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 496 andrew gelman stats-2011-01-01-Tukey’s philosophy
Introduction: The great statistician John Tukey, in his writings from the 1970s onward (and maybe earlier) was time and again making the implicit argument that you should evaluate a statistical method based on what it does; you should {\em not} be staring at the model that purportedly underlies the method, trying to determine if the model is “true” (or “true enough”). Tukey’s point was that models can be great to inspire methods, but the model is the scaffolding; it is the method that is the building you have to live in. I don’t fully agree with this philosophy–I think models are a good way to understand data and also often connect usefully to scientific models (although not as cleanly as is thought by our friends who work in economics or statistical hypothesis testing). To put it another way: What makes a building good? A building is good if it is useful. If a building is useful, people will use it. Eventually improvements will be needed, partly because the building will get worn down, part
2 0.21459849 738 andrew gelman stats-2011-05-30-Works well versus well understood
Introduction: John Cook discusses the John Tukey quote, “The test of a good procedure is how well it works, not how well it is understood.” Cook writes: At some level, it’s hard to argue against this. Statistical procedures operate on empirical data, so it makes sense that the procedures themselves be evaluated empirically. But I [Cook] question whether we really know that a statistical procedure works well if it isn’t well understood. Specifically, I’m skeptical of complex statistical methods whose only credentials are a handful of simulations. “We don’t have any theoretical results, buy hey, it works well in practice. Just look at the simulations.” Every method works well on the scenarios its author publishes, almost by definition. If the method didn’t handle a scenario well, the author would publish a different scenario. I agree with Cook but would give a slightly different emphasis. I’d say that a lot of methods can work when they are done well. See the second meta-principle liste
3 0.189026 1719 andrew gelman stats-2013-02-11-Why waste time philosophizing?
Introduction: I’ll answer the above question after first sharing some background and history on the the philosophy of Bayesian statistics, which appeared at the end of our rejoinder to the discussion to which I linked the other day: When we were beginning our statistical educations, the word ‘Bayesian’ conveyed membership in an obscure cult. Statisticians who were outside the charmed circle could ignore the Bayesian subfield, while Bayesians themselves tended to be either apologetic or brazenly defiant. These two extremes manifested themselves in ever more elaborate proposals for non-informative priors, on the one hand, and declarations of the purity of subjective probability, on the other. Much has changed in the past 30 years. ‘Bayesian’ is now often used in casual scientific parlance as a synonym for ‘rational’, the anti-Bayesians have mostly disappeared, and non-Bayesian statisticians feel the need to keep up with developments in Bayesian modelling and computation. Bayesians themselves
Introduction: In response to my remarks on his online book, Think Bayes, Allen Downey wrote: I [Downey] have a question about one of your comments: My [Gelman's] main criticism with both books is that they talk a lot about inference but not so much about model building or model checking (recall the three steps of Bayesian data analysis). I think it’s ok for an introductory book to focus on inference, which of course is central to the data-analytic process—but I’d like them to at least mention that Bayesian ideas arise in model building and model checking as well. This sounds like something I agree with, and one of the things I tried to do in the book is to put modeling decisions front and center. But the word “modeling” is used in lots of ways, so I want to see if we are talking about the same thing. For example, in many chapters, I start with a simple model of the scenario, do some analysis, then check whether the model is good enough, and iterate. Here’s the discussion of modeling
5 0.13750124 998 andrew gelman stats-2011-11-08-Bayes-Godel
Introduction: Sean O’Riordain writes: Your article, “ The holes in my philosophy of Bayesian data analysis ” caused me to wonder whether it was possible to have a consistent philosophy of data analysis and whether it could it be possible that Godel’s incompleteness theorem extends as far as to say that it wasn’t possible? I don’t know but my guess is that this is all related to our lack of a good model for hypothesis generation. Statistics focuses on deductive inference within models and model checking to evaluate models, but we don’t have a good handle on the creation of models. (I’m hoping that some of our network-of-models stuff will be helpful.)
6 0.13330269 1652 andrew gelman stats-2013-01-03-“The Case for Inductive Theory Building”
7 0.13087843 291 andrew gelman stats-2010-09-22-Philosophy of Bayes and non-Bayes: A dialogue with Deborah Mayo
8 0.12890704 110 andrew gelman stats-2010-06-26-Philosophy and the practice of Bayesian statistics
9 0.12796341 244 andrew gelman stats-2010-08-30-Useful models, model checking, and external validation: a mini-discussion
10 0.12604368 781 andrew gelman stats-2011-06-28-The holes in my philosophy of Bayesian data analysis
11 0.12520686 1205 andrew gelman stats-2012-03-09-Coming to agreement on philosophy of statistics
12 0.12496097 746 andrew gelman stats-2011-06-05-An unexpected benefit of Arrow’s other theorem
13 0.11722276 2151 andrew gelman stats-2013-12-27-Should statistics have a Nobel prize?
14 0.11719186 1712 andrew gelman stats-2013-02-07-Philosophy and the practice of Bayesian statistics (with all the discussions!)
15 0.11475419 1406 andrew gelman stats-2012-07-05-Xiao-Li Meng and Xianchao Xie rethink asymptotics
16 0.11337197 754 andrew gelman stats-2011-06-09-Difficulties with Bayesian model averaging
17 0.11126947 109 andrew gelman stats-2010-06-25-Classics of statistics
18 0.10941494 811 andrew gelman stats-2011-07-20-Kind of Bayesian
19 0.10871744 1482 andrew gelman stats-2012-09-04-Model checking and model understanding in machine learning
20 0.10862201 2245 andrew gelman stats-2014-03-12-More on publishing in journals
topicId topicWeight
[(0, 0.19), (1, 0.088), (2, -0.075), (3, 0.046), (4, -0.023), (5, 0.003), (6, -0.075), (7, 0.007), (8, 0.085), (9, 0.002), (10, 0.006), (11, -0.001), (12, -0.051), (13, -0.018), (14, -0.049), (15, -0.013), (16, 0.026), (17, -0.023), (18, -0.01), (19, 0.009), (20, -0.007), (21, -0.061), (22, -0.024), (23, 0.009), (24, -0.009), (25, -0.025), (26, -0.022), (27, 0.025), (28, 0.01), (29, -0.021), (30, 0.022), (31, 0.016), (32, 0.06), (33, 0.012), (34, 0.003), (35, 0.016), (36, 0.023), (37, 0.045), (38, -0.011), (39, -0.043), (40, -0.012), (41, 0.013), (42, -0.013), (43, 0.031), (44, 0.06), (45, 0.008), (46, -0.059), (47, -0.056), (48, -0.009), (49, 0.031)]
simIndex simValue blogId blogTitle
same-blog 1 0.9736172 496 andrew gelman stats-2011-01-01-Tukey’s philosophy
Introduction: The great statistician John Tukey, in his writings from the 1970s onward (and maybe earlier) was time and again making the implicit argument that you should evaluate a statistical method based on what it does; you should {\em not} be staring at the model that purportedly underlies the method, trying to determine if the model is “true” (or “true enough”). Tukey’s point was that models can be great to inspire methods, but the model is the scaffolding; it is the method that is the building you have to live in. I don’t fully agree with this philosophy–I think models are a good way to understand data and also often connect usefully to scientific models (although not as cleanly as is thought by our friends who work in economics or statistical hypothesis testing). To put it another way: What makes a building good? A building is good if it is useful. If a building is useful, people will use it. Eventually improvements will be needed, partly because the building will get worn down, part
Introduction: David Duvenaud writes: I’ve been following your recent discussions about how an AI could do statistics [see also here ]. I was especially excited about your suggestion for new statistical methods using “a language-like approach to recursively creating new models from a specified list of distributions and transformations, and an automatic approach to checking model fit.” Your discussion of these ideas was exciting to me and my colleagues because we recently did some work taking a step in this direction, automatically searching through a grammar over Gaussian process regression models. Roger Grosse previously did the same thing , but over matrix decomposition models using held-out predictive likelihood to check model fit. These are both examples of automatic Bayesian model-building by a search over more and more complex models, as you suggested. One nice thing is that both grammars include lots of standard models for free, and they seem to work pretty well, although the
3 0.8201021 1482 andrew gelman stats-2012-09-04-Model checking and model understanding in machine learning
Introduction: Last month I wrote : Computer scientists are often brilliant but they can be unfamiliar with what is done in the worlds of data collection and analysis. This goes the other way too: statisticians such as myself can look pretty awkward, reinventing (or failing to reinvent) various wheels when we write computer programs or, even worse, try to design software.Andrew MacNamara writes: Andrew MacNamara followed up with some thoughts: I [MacNamara] had some basic statistics training through my MBA program, after having completed an undergrad degree in computer science. Since then I’ve been very interested in learning more about statistical techniques, including things like GLM and censored data analyses as well as machine learning topics like neural nets, SVMs, etc. I began following your blog after some research into Bayesian analysis topics and I am trying to dig deeper on that side of things. One thing I have noticed is that there seems to be a distinction between data analysi
4 0.80571556 614 andrew gelman stats-2011-03-15-Induction within a model, deductive inference for model evaluation
Introduction: Jonathan Livengood writes: I have a couple of questions on your paper with Cosma Shalizi on “Philosophy and the practice of Bayesian statistics.” First, you distinguish between inductive approaches and hypothetico-deductive approaches to inference and locate statistical practice (at least, the practice of model building and checking) on the hypothetico-deductive side. Do you think that there are any interesting elements of statistical practice that are properly inductive? For example, suppose someone is playing around with a system that more or less resembles a toy model, like drawing balls from an urn or some such, and where the person has some well-defined priors. The person makes a number of draws from the urn and applies Bayes theorem to get a posterior. On your view, is that person making an induction? If so, how much space is there in statistical practice for genuine inductions like this? Second, I agree with you that one ought to distinguish induction from other kind
Introduction: David Rohde writes: I have been thinking a lot lately about your Bayesian model checking approach. This is in part because I have been working on exploratory data analysis and wishing to avoid controversy and mathematical statistics we omitted model checking from our discussion. This is something that the refereeing process picked us up on and we ultimately added a critical discussion of null-hypothesis testing to our paper . The exploratory technique we discussed was essentially a 2D histogram approach, but we used Polya models as a formal model for the histogram. We are currently working on a new paper, and we are thinking through how or if we should do “confirmatory analysis” or model checking in the paper. What I find most admirable about your statistical work is that you clearly use the Bayesian approach to do useful applied statistical analysis. My own attempts at applied Bayesian analysis makes me greatly admire your applied successes. On the other hand it may be t
6 0.79671025 1406 andrew gelman stats-2012-07-05-Xiao-Li Meng and Xianchao Xie rethink asymptotics
7 0.79591179 244 andrew gelman stats-2010-08-30-Useful models, model checking, and external validation: a mini-discussion
9 0.77992904 72 andrew gelman stats-2010-06-07-Valencia: Summer of 1991
10 0.76915318 421 andrew gelman stats-2010-11-19-Just chaid
11 0.7627086 964 andrew gelman stats-2011-10-19-An interweaving-transformation strategy for boosting MCMC efficiency
13 0.75400108 744 andrew gelman stats-2011-06-03-Statistical methods for healthcare regulation: rating, screening and surveillance
14 0.74604177 1856 andrew gelman stats-2013-05-14-GPstuff: Bayesian Modeling with Gaussian Processes
15 0.74538594 2007 andrew gelman stats-2013-09-03-Popper and Jaynes
16 0.74514383 1041 andrew gelman stats-2011-12-04-David MacKay and Occam’s Razor
17 0.7435618 1861 andrew gelman stats-2013-05-17-Where do theories come from?
18 0.73853409 738 andrew gelman stats-2011-05-30-Works well versus well understood
20 0.73704469 1392 andrew gelman stats-2012-06-26-Occam
topicId topicWeight
[(5, 0.019), (16, 0.134), (21, 0.018), (24, 0.092), (30, 0.061), (42, 0.014), (45, 0.019), (54, 0.011), (73, 0.14), (76, 0.019), (86, 0.032), (99, 0.315)]
simIndex simValue blogId blogTitle
1 0.967632 794 andrew gelman stats-2011-07-09-The quest for the holy graph
Introduction: Eytan Adar writes: I was just going through the latest draft of your paper with Anthony Unwin . I heard part of it at the talk you gave (remotely) here at UMich. I’m curious about your discussion of the Baby Name Voyager . The tool in itself is simple, attractive, and useful. No argument from me there. It’s an awesome demonstration of how subtle interactions can be very helpful (click and it zooms, type and it filters… falls perfectly into the Shneiderman visualization mantra). It satisfies a very common use case: finding appropriate names for children. That said, I can’t help but feeling that what you are really excited about is the very static analysis on last letters (you spend most of your time on this). This analysis, incidentally, is not possible to infer from the interactive application (which doesn’t support this type of filtering and pivoting). In a sense, the two visualizations don’t have anything to do with each other (other than a shared context/dataset).
2 0.96253878 161 andrew gelman stats-2010-07-24-Differences in color perception by sex, also the Bechdel test for women in movies
Introduction: Here’s a pretty funny example of silly statistics, caught by Lisa Wade : A study published in 2001 . . . asked undergraduate college students their favorite color and presented the results by sex. Men’s favorites are on the left, women’s on the right: The authors of the study, Lee Ellis and Christopher Ficek, wrote: We are inclined to suspect the involvement of neurohormonal factors. Studies of rats have found average sex differences in the number of neurons comprising various parts of the visual cortex. Also, gender differences have been found in rat preferences for the amount of sweetness in drinking water. One experiment demonstrated that the sex differences in rat preferences for sweetness was eliminated by depriving males of male-typical testosterone levels in utero. Perhaps, prenatal exposure to testosterone and other sex hormones operates in a similar way to “bias” preferences for certain colors in humans. As Wade points out, that all seems a bit ridiculous giv
3 0.9555403 1511 andrew gelman stats-2012-09-26-What do statistical p-values mean when the sample = the population?
Introduction: Felipe Nunes writes: I have many friends working with data that they claim to be considered as a ‘population’. For example, the universe of bills presented in a Congress, the roll call votes of all deputies in a legislature, a survey with all deputies in a country, the outcomes of an election, or the set of electoral institutions around the world. Because of the nature of these data, we do not know how to interpret the p-value. I have seen many arguments been made, but I have never seen a formal response to the question. So I don’t know what to say. The most common arguments among the community of young researchers in Brazil are: (1) don’t interpret p-value when you have population, but don’t infer anything either; (2) interpret the p-value because of error measurement which is also present, (3) there is no such a thing as a population, so always look at p-values, (4) don’t worry about p-value, interpret the coefficients substantively, and (5) if you are frequentist you interpret p-
4 0.95457059 497 andrew gelman stats-2011-01-02-Hipmunk update
Introduction: Florence from customer support at Hipmunk writes: Hipmunk now includes American Airlines in our search results. Please note that users will be taken directly to AA.com to complete the booking/transaction. . . . we are steadily increasing the number of flights that we offer on Hipmunk. As you may recall, Hipmunk is a really cool flight-finder that didn’t actually work (as of 16 Sept 2010). At the time, I was a bit annoyed at the NYT columnist who plugged Hipmunk without actually telling his readers that the site didn’t actually do the job. (I discovered the problem myself because I couldn’t believe that my flight options to Raleigh-Durham were really so meager, so I checked on Expedia and found a good flight.) I do think Hipmunk’s graphics are beautiful, though, so I’m rooting for them to catch up. P.S. Apparently they include Amtrak Northeast Corridor trains, so I’ll give them a try, next time I travel. The regular Amtrak website is about as horrible as you’d expect.
5 0.94730413 2238 andrew gelman stats-2014-03-09-Hipmunk worked
Introduction: In the past I’ve categorized Hipmunk as a really cool flight-finder that doesn’t actually work , as worse than Expedia , and as graphics without content . So, I thought it would be only fair to tell you that I bought a flight the other day using Hipmunk and it gave me the same flight as Expedia but at a lower cost (by linking to something called CheapOair, which I hope is legit). So score one for Hipmunk.
same-blog 6 0.94432437 496 andrew gelman stats-2011-01-01-Tukey’s philosophy
7 0.94354129 1925 andrew gelman stats-2013-07-04-“Versatile, affordable chicken has grown in popularity”
8 0.94175953 655 andrew gelman stats-2011-04-10-“Versatile, affordable chicken has grown in popularity”
9 0.93926245 7 andrew gelman stats-2010-04-27-Should Mister P be allowed-encouraged to reside in counter-factual populations?
10 0.93526435 1748 andrew gelman stats-2013-03-04-PyStan!
11 0.93331116 2020 andrew gelman stats-2013-09-12-Samplers for Big Science: emcee and BAT
12 0.92071927 798 andrew gelman stats-2011-07-12-Sometimes a graph really is just ugly
13 0.91374999 631 andrew gelman stats-2011-03-28-Explaining that plot.
14 0.91349459 722 andrew gelman stats-2011-05-20-Why no Wegmania?
15 0.91284966 280 andrew gelman stats-2010-09-16-Meet Hipmunk, a really cool flight-finder that doesn’t actually work
16 0.91217482 320 andrew gelman stats-2010-10-05-Does posterior predictive model checking fit with the operational subjective approach?
17 0.91021287 2280 andrew gelman stats-2014-04-03-As the boldest experiment in journalism history, you admit you made a mistake
18 0.90762347 1846 andrew gelman stats-2013-05-07-Like Casper the ghost, Niall Ferguson is not only white. He is also very, very adorable.
20 0.90684468 154 andrew gelman stats-2010-07-18-Predictive checks for hierarchical models