andrew_gelman_stats andrew_gelman_stats-2011 andrew_gelman_stats-2011-744 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Here is my discussion of a recent article by David Spiegelhalter, Christopher Sherlaw-Johnson, Martin Bardsley, Ian Blunt, Christopher Wood and Olivia Grigg, that is scheduled to appear in the Journal of the Royal Statistical Society: I applaud the authors’ use of a mix of statistical methods to attack an important real-world problem. Policymakers need results right away, and I admire the authors’ ability and willingness to combine several different modeling and significance testing ideas for the purposes of rating and surveillance. That said, I am uncomfortable with the statistical ideas here, for three reasons. First, I feel that the proposed methods, centered as they are around data manipulation and corrections for uncertainty, has serious defects compared to a more model-based approach. My problem with methods based on p-values and z-scores–however they happen to be adjusted–is that they draw discussion toward error rates, sequential analysis, and other technical statistical
sentIndex sentText sentNum sentScore
1 Policymakers need results right away, and I admire the authors’ ability and willingness to combine several different modeling and significance testing ideas for the purposes of rating and surveillance. [sent-2, score-0.248]
2 That said, I am uncomfortable with the statistical ideas here, for three reasons. [sent-3, score-0.125]
3 First, I feel that the proposed methods, centered as they are around data manipulation and corrections for uncertainty, has serious defects compared to a more model-based approach. [sent-4, score-0.171]
4 My problem with methods based on p-values and z-scores–however they happen to be adjusted–is that they draw discussion toward error rates, sequential analysis, and other technical statistical concepts. [sent-5, score-0.784]
5 In contrast, a model-based approach draws discussion toward the model and, from there, the process being modeled. [sent-6, score-0.35]
6 I understand the appeal of p-value adjustments–lots of quantitatively-trained people know about p-values–but I’d much rather draw the statistics toward the data rather than the other way around. [sent-7, score-0.469]
7 Once you have to bring out the funnel plot, this is to me a sign of (partial) failure, that you’re talking about properties of a statistical summary rather than about the underlying process that generates the observed data. [sent-8, score-0.517]
8 My second difficulty is closely related: to me, the mapping seems tenuous from statistical significance to the ultimate healthcare and financial goals. [sent-9, score-0.301]
9 That said, the authors of the article under discussion are doing the work and I’m not. [sent-11, score-0.253]
10 I’m sure they have good reasons for using what I consider to be inferior methods, and I believe that one of the points of this discussion is to give them a chance to give this explanation. [sent-12, score-0.133]
11 Finally, I am glad that these methods result in ratings rather than rankings . [sent-13, score-0.387]
12 (2002), and others, two huge problems arise when constructing ranks from noisy data. [sent-15, score-0.332]
13 First, with unbalanced data (for example, different sample sizes in different hospitals) there is no way to simultaneously get reasonable point estimates of parameters and their rankings. [sent-16, score-0.173]
14 Even with moderately large samples, estimated ranks are unstable and can be misleading, violating well-known principles of quality control by encouraging decision makers to chase noise rather than understanding and reducing variation (Deming, 2000). [sent-18, score-0.877]
15 Thus, although I am unhappy with the components of the methods being used here, I like some aspects of the output. [sent-19, score-0.189]
16 Uncertainty in rank estimation: Implications for Value Added Modeling Accountability Systems. [sent-32, score-0.078]
wordName wordTfidf (topN-words)
[('ranks', 0.257), ('louis', 0.243), ('lockwood', 0.217), ('deming', 0.196), ('methods', 0.189), ('christopher', 0.153), ('toward', 0.135), ('discussion', 0.133), ('ta', 0.13), ('statistical', 0.125), ('authors', 0.12), ('rather', 0.112), ('draw', 0.11), ('mccaffrey', 0.109), ('blunt', 0.109), ('funnel', 0.109), ('unbalanced', 0.098), ('spiegelhalter', 0.098), ('significance', 0.096), ('bayes', 0.096), ('uncertainty', 0.095), ('defects', 0.095), ('chase', 0.092), ('sequential', 0.092), ('accountability', 0.089), ('makers', 0.089), ('generates', 0.089), ('policymakers', 0.087), ('rankings', 0.086), ('hospitals', 0.086), ('violating', 0.086), ('unstable', 0.086), ('cambridge', 0.084), ('wood', 0.084), ('ian', 0.082), ('process', 0.082), ('applaud', 0.081), ('scheduled', 0.08), ('healthcare', 0.08), ('notoriously', 0.079), ('moderately', 0.079), ('royal', 0.079), ('rank', 0.078), ('adjustments', 0.078), ('willingness', 0.077), ('encouraging', 0.076), ('manipulation', 0.076), ('modeling', 0.075), ('constructing', 0.075), ('simultaneously', 0.075)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999964 744 andrew gelman stats-2011-06-03-Statistical methods for healthcare regulation: rating, screening and surveillance
Introduction: Here is my discussion of a recent article by David Spiegelhalter, Christopher Sherlaw-Johnson, Martin Bardsley, Ian Blunt, Christopher Wood and Olivia Grigg, that is scheduled to appear in the Journal of the Royal Statistical Society: I applaud the authors’ use of a mix of statistical methods to attack an important real-world problem. Policymakers need results right away, and I admire the authors’ ability and willingness to combine several different modeling and significance testing ideas for the purposes of rating and surveillance. That said, I am uncomfortable with the statistical ideas here, for three reasons. First, I feel that the proposed methods, centered as they are around data manipulation and corrections for uncertainty, has serious defects compared to a more model-based approach. My problem with methods based on p-values and z-scores–however they happen to be adjusted–is that they draw discussion toward error rates, sequential analysis, and other technical statistical
2 0.15550032 151 andrew gelman stats-2010-07-16-Wanted: Probability distributions for rank orderings
Introduction: Dietrich Stoyan writes: I asked the IMS people for an expert in statistics of voting/elections and they wrote me your name. I am a statistician, but never worked in the field voting/elections. It was my son-in-law who asked me for statistical theories in that field. He posed in particular the following problem: The aim of the voting is to come to a ranking of c candidates. Every vote is a permutation of these c candidates. The problem is to have probability distributions in the set of all permutations of c elements. Are there theories for such distributions? I should be very grateful for a fast answer with hints to literature. (I confess that I do not know your books.) My reply: Rather than trying to model the ranks directly, I’d recommend modeling a latent continuous outcome which then implies a distribution on ranks, if the ranks are of interest. There are lots of distributions of c-dimensional continuous outcomes. In political science, the usual way to start is
3 0.14884807 136 andrew gelman stats-2010-07-09-Using ranks as numbers
Introduction: David Shor writes: I’m dealing with a situation where I have two datasets, one that assigns each participant a discrete score out of five for a set of particular traits (Dog behavior characteristics by breed), and another from an independent source that ranks each breed by each characteristic. It’s also possible to obtain the results of a survey, where experts were asked to rank 7 randomly picked breeds by characteristics. I’m interested in obtaining estimates for each trait, and intuitively, it seems clear that the second and third dataset provide a lot of information. But it’s unclear how to incorporate them to infer latent variables, since only sample ranks are observed. This seems like it is a common problem, do you have any suggestions? My quick answer is that you can treat ranks as numbers (a point we make somewhere in Bayesian Data Analysis, I believe) and just fit an item-response model from there. Val Johnson wrote an article on this in Jasa a few years ago, “Bayesia
4 0.12018223 783 andrew gelman stats-2011-06-30-Don’t stop being a statistician once the analysis is done
Introduction: I received an email from the Royal Statistical Society asking if I wanted to submit a 400-word discussion to the article, Vignettes and health systems responsiveness in cross-country comparative analyses by Nigel Rice, Silvana Robone and Peter C. Smith. My first thought was No, I can’t do it, I don’t know anything about health systems responsiveness etc. But then I thought, sure, I always have something to say. So I skimmed the article and was indeed motivated to write something. Here’s what I sent in: As a frequent user of survey data, I am happy to see this work on improving the reliability and validity of subjective responses. My only comment is to recommend that the statistical sophistication that is evident in the in the design and modeling in this study be applied to the summaries as well. I have three suggestions in particular. First, I believe Figure 1 could be better presented as a series of line plots. As it is, the heights of the purple and blue bars dominat
5 0.11714566 1848 andrew gelman stats-2013-05-09-A tale of two discussion papers
Introduction: Over the years I’ve written a dozen or so journal articles that have appeared with discussions, and I’ve participated in many published discussions of others’ articles as well. I get a lot out of these article-discussion-rejoinder packages, in all three of my roles as reader, writer, and discussant. Part 1: The story of an unsuccessful discussion The first time I had a discussion article was the result of an unfortunate circumstance. I had a research idea that resulted in an article with Don Rubin on monitoring the mixing of Markov chain simulations. I new the idea was great, but back then we worked pretty slowly so it was awhile before we had a final version to submit to a journal. (In retrospect I wish I’d just submitted the draft version as it was.) In the meantime I presented the paper at a conference. Our idea was very well received (I had a sheet of paper so people could write their names and addresses to get preprints, and we got either 50 or 150 (I can’t remembe
6 0.11675341 1695 andrew gelman stats-2013-01-28-Economists argue about Bayes
9 0.10378081 1482 andrew gelman stats-2012-09-04-Model checking and model understanding in machine learning
11 0.10103368 241 andrew gelman stats-2010-08-29-Ethics and statistics in development research
14 0.098505266 1354 andrew gelman stats-2012-05-30-“I didn’t marry a horn, I married a man”
15 0.09798459 1575 andrew gelman stats-2012-11-12-Thinking like a statistician (continuously) rather than like a civilian (discretely)
16 0.097838342 146 andrew gelman stats-2010-07-14-The statistics and the science
17 0.097443745 246 andrew gelman stats-2010-08-31-Somewhat Bayesian multilevel modeling
18 0.096107252 1469 andrew gelman stats-2012-08-25-Ways of knowing
19 0.096013531 2254 andrew gelman stats-2014-03-18-Those wacky anti-Bayesians used to be intimidating, but now they’re just pathetic
20 0.095030747 2245 andrew gelman stats-2014-03-12-More on publishing in journals
topicId topicWeight
[(0, 0.216), (1, 0.071), (2, -0.013), (3, -0.066), (4, -0.008), (5, -0.019), (6, -0.079), (7, 0.024), (8, 0.013), (9, -0.02), (10, -0.006), (11, -0.017), (12, -0.011), (13, -0.0), (14, -0.014), (15, -0.027), (16, -0.032), (17, -0.005), (18, -0.009), (19, -0.013), (20, 0.028), (21, -0.002), (22, 0.014), (23, 0.022), (24, 0.022), (25, 0.014), (26, -0.027), (27, 0.032), (28, 0.025), (29, -0.001), (30, 0.028), (31, 0.034), (32, 0.029), (33, -0.006), (34, -0.014), (35, 0.006), (36, -0.007), (37, 0.017), (38, 0.009), (39, -0.008), (40, 0.013), (41, 0.032), (42, -0.049), (43, 0.023), (44, 0.03), (45, -0.023), (46, -0.03), (47, -0.037), (48, -0.037), (49, 0.008)]
simIndex simValue blogId blogTitle
same-blog 1 0.9784168 744 andrew gelman stats-2011-06-03-Statistical methods for healthcare regulation: rating, screening and surveillance
Introduction: Here is my discussion of a recent article by David Spiegelhalter, Christopher Sherlaw-Johnson, Martin Bardsley, Ian Blunt, Christopher Wood and Olivia Grigg, that is scheduled to appear in the Journal of the Royal Statistical Society: I applaud the authors’ use of a mix of statistical methods to attack an important real-world problem. Policymakers need results right away, and I admire the authors’ ability and willingness to combine several different modeling and significance testing ideas for the purposes of rating and surveillance. That said, I am uncomfortable with the statistical ideas here, for three reasons. First, I feel that the proposed methods, centered as they are around data manipulation and corrections for uncertainty, has serious defects compared to a more model-based approach. My problem with methods based on p-values and z-scores–however they happen to be adjusted–is that they draw discussion toward error rates, sequential analysis, and other technical statistical
Introduction: I received the following note from someone who’d like to remain anonymous: I read your post on ethics and statistics, and the comments therein, with much interest. I did notice, however, that most of the dialogue was about ethical behavior of scientists. Herein I’d like to suggest a different take, one that focuses on the statistical methods of scientists. For example, fitting a line to a scatter plot of data using OLS [linear regression] gives more weight to outliers. If each data point represents a person we are weighting people differently. And surely the ethical implications are different if we use a least absolute deviation estimator. Recently I reviewed a paper where the authors claimed one advantage of non-parametric rank-based tests is their robustness to outliers. Again, maybe that outlier is the 10th person who dies from an otherwise beneficial medicine. Should we ignore him in assessing the effect of the medicine? I guess this gets me partly into loss f
3 0.80560029 241 andrew gelman stats-2010-08-29-Ethics and statistics in development research
Introduction: From Bannerjee and Duflo, “The Experimental Approach to Development Economics,” Annual Review of Economics (2009): One issue with the explicit acknowledgment of randomization as a fair way to allocate the program is that implementers may find that the easiest way to present it to the community is to say that an expansion of the program is planned for the control areas in the future (especially when such is indeed the case, as in phased-in design). I can’t quite figure out whether Bannerjee and Duflo are saying that they would lie and tell people that an expansion is planned when it isn’t, or whether they’re deploring that other people do it. I’m not bothered by a lot of the deception in experimental research–for example, I think the Milgram obedience experiment was just fine–but somehow the above deception bothers me. It just seems wrong to tell people that an expansion is planned if it’s not. P.S. Overall the article is pretty good. My only real problem with it is that
4 0.79185838 556 andrew gelman stats-2011-02-04-Patterns
Introduction: Pete Gries writes: I [Gries] am not sure if what you are suggesting by “doing data analysis in a patternless way” is a pitch for deductive over inductive approaches as a solution to the problem of reporting and publication bias. If so, I may somewhat disagree. A constant quest to prove or disprove theory in a deductive manner is one of the primary causes of both reporting and publication bias. I’m actually becoming a proponent of a remarkably non-existent species – “applied political science” – because there is so much animosity in our discipline to inductive empirical statistical work that seeks to answer real world empirical questions rather than contribute to parsimonious theory building. Anyone want to start a JAPS – Journal of Applied Political Science? Our discipline is in danger of irrelevance. My reply: By “doing data analysis in a patternless way,” I meant statistical methods such as least squares, maximum likelihood, etc., that estimate parameters independently witho
Introduction: Some things I respect When it comes to meta-models of statistics, here are two philosophies that I respect: 1. (My) Bayesian approach, which I associate with E. T. Jaynes, in which you construct models with strong assumptions, ride your models hard, check their fit to data, and then scrap them and improve them as necessary. 2. At the other extreme, model-free statistical procedures that are designed to work well under very weak assumptions—for example, instead of assuming a distribution is Gaussian, you would just want the procedure to work well under some conditions on the smoothness of the second derivative of the log density function. Both the above philosophies recognize that (almost) all important assumptions will be wrong, and they resolve this concern via aggressive model checking or via robustness. And of course there are intermediate positions, such as working with Bayesian models that have been shown to be robust, and then still checking them. Or, to flip it arou
8 0.76752967 738 andrew gelman stats-2011-05-30-Works well versus well understood
9 0.76604903 1195 andrew gelman stats-2012-03-04-Multiple comparisons dispute in the tabloids
10 0.76561344 511 andrew gelman stats-2011-01-11-One more time on that ESP study: The problem of overestimates and the shrinkage solution
11 0.76555282 1880 andrew gelman stats-2013-06-02-Flame bait
12 0.76264107 2151 andrew gelman stats-2013-12-27-Should statistics have a Nobel prize?
13 0.7596072 1859 andrew gelman stats-2013-05-16-How do we choose our default methods?
15 0.75775576 1575 andrew gelman stats-2012-11-12-Thinking like a statistician (continuously) rather than like a civilian (discretely)
16 0.75724757 1690 andrew gelman stats-2013-01-23-When are complicated models helpful in psychology research and when are they overkill?
19 0.74111921 2277 andrew gelman stats-2014-03-31-The most-cited statistics papers ever
20 0.74047899 690 andrew gelman stats-2011-05-01-Peter Huber’s reflections on data analysis
topicId topicWeight
[(2, 0.029), (16, 0.05), (21, 0.054), (24, 0.196), (25, 0.013), (42, 0.022), (51, 0.103), (57, 0.025), (68, 0.012), (77, 0.019), (81, 0.012), (86, 0.048), (95, 0.018), (99, 0.275)]
simIndex simValue blogId blogTitle
1 0.96583128 1594 andrew gelman stats-2012-11-28-My talk on statistical graphics at Mit this Thurs aft
Introduction: Infovis and Statistical Graphics: Different Goals, Different Looks (and here’s the article) Speaker: Andrew Gelman, Columbia University Date: Thursday, November 29 2012 Time: 4:00PM to 5:00PM Location: 32-D463 (Star Conference Room) Host: Polina Golland, CSAIL Contact: Polina Golland, 6172538005, polina@csail.mit.edu The importance of graphical displays in statistical practice has been recognized sporadically in the statistical literature over the past century, with wider awareness following Tukey’s Exploratory Data Analysis (1977) and Tufte’s books in the succeeding decades. But statistical graphics still occupies an awkward in-between position: Within statistics, exploratory and graphical methods represent a minor subfield and are not well-integrated with larger themes of modeling and inference. Outside of statistics, infographics (also called information visualization or Infovis) is huge, but their purveyors and enthusiasts appear largely to be uninterested in statisti
2 0.96244049 1543 andrew gelman stats-2012-10-21-Model complexity as a function of sample size
Introduction: As we get more data, we can fit more model. But at some point we become so overwhelmed by data that, for computational reasons, we can barely do anything at all. Thus, the curve above could be thought of as the product of two curves: a steadily increasing curve showing the statistical ability to fit more complex models with more data, and a steadily decreasing curve showing the computational feasibility of doing so.
3 0.96233964 1147 andrew gelman stats-2012-01-30-Statistical Murder
Introduction: Image via Wikipedia Robert Zubrin writes in “How Much Is an Astronaut’s Life Worth?” ( Reason , Feb 2012 ): …policy analyst John D. Graham and his colleagues at the Harvard Center for Risk Analysis found in 1997 that the median cost for lifesaving expenditures and regulations by the U.S. government in the health care, residential, transportation, and occupational areas ranges from about $1 million to $3 million spent per life saved in today’s dollars. The only marked exception to this pattern occurs in the area of environmental health protection (such as the Superfund program) which costs about $200 million per life saved. Graham and his colleagues call the latter kind of inefficiency “ statistical murder ,” since thousands of additional lives could be saved each year if the money were used more cost-effectively. To avoid such deadly waste, the Department of Transportation has a policy of rejecting any proposed safety expenditure that costs more than $3
same-blog 4 0.96113729 744 andrew gelman stats-2011-06-03-Statistical methods for healthcare regulation: rating, screening and surveillance
Introduction: Here is my discussion of a recent article by David Spiegelhalter, Christopher Sherlaw-Johnson, Martin Bardsley, Ian Blunt, Christopher Wood and Olivia Grigg, that is scheduled to appear in the Journal of the Royal Statistical Society: I applaud the authors’ use of a mix of statistical methods to attack an important real-world problem. Policymakers need results right away, and I admire the authors’ ability and willingness to combine several different modeling and significance testing ideas for the purposes of rating and surveillance. That said, I am uncomfortable with the statistical ideas here, for three reasons. First, I feel that the proposed methods, centered as they are around data manipulation and corrections for uncertainty, has serious defects compared to a more model-based approach. My problem with methods based on p-values and z-scores–however they happen to be adjusted–is that they draw discussion toward error rates, sequential analysis, and other technical statistical
5 0.94319153 1792 andrew gelman stats-2013-04-07-X on JLP
Introduction: Christian Robert writes on the Jeffreys-Lindley paradox. I have nothing to add to this beyond my recent comments : To me, the Lindley paradox falls apart because of its noninformative prior distribution on the parameter of interest. If you really think there’s a high probability the parameter is nearly exactly zero, I don’t see the point of the model saying that you have no prior information at all on the parameter. In short: my criticism of so-called Bayesian hypothesis testing is that it’s insufficiently Bayesian. To clarify, I’m speaking of all the examples I’ve ever worked on in social and environmental science, where in some settings I can imagine a parameter being very close to zero and in other settings I can imagine a parameter taking on just about any value in a wide range, but where I’ve never seen an example where a parameter could be either right at zero or taking on any possible value. But such examples might occur in areas of application that I haven’t worked on.
6 0.94309735 2029 andrew gelman stats-2013-09-18-Understanding posterior p-values
7 0.94280267 1176 andrew gelman stats-2012-02-19-Standardized writing styles and standardized graphing styles
8 0.94239545 2299 andrew gelman stats-2014-04-21-Stan Model of the Week: Hierarchical Modeling of Supernovas
9 0.94153333 1584 andrew gelman stats-2012-11-19-Tradeoffs in information graphics
11 0.93861353 502 andrew gelman stats-2011-01-04-Cash in, cash out graph
12 0.93785429 2086 andrew gelman stats-2013-11-03-How best to compare effects measured in two different time periods?
13 0.93686712 1644 andrew gelman stats-2012-12-30-Fixed effects, followed by Bayes shrinkage?
14 0.93671149 1240 andrew gelman stats-2012-04-02-Blogads update
15 0.93666744 1941 andrew gelman stats-2013-07-16-Priors
16 0.93626571 1208 andrew gelman stats-2012-03-11-Gelman on Hennig on Gelman on Bayes
17 0.93549287 1474 andrew gelman stats-2012-08-29-More on scaled-inverse Wishart and prior independence
18 0.93544066 1465 andrew gelman stats-2012-08-21-D. Buggin