andrew_gelman_stats andrew_gelman_stats-2013 andrew_gelman_stats-2013-1987 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Stuart Hurlbert writes: A colleague recently forwarded to me your 2012 paper with Hill and Yajima on the multiple comparison “non-problem”, as I call it. You and your colleagues might find of interest a 2012 paper [with Celia Lombardi] that reaches similar conclusions by a colleague and myself which is attached. Similar but not identical, as we are a bit Bayesian-shy after seeing so many exaggerated claims made for Bayesian approaches over recent decades. I take pride in having for a few decades defended many colleagues against editors (and many graduate students against faculty members) who demanded “corrections” for multiple comparisons. We’ve gotten no small number of editors and professors to back off their unreasonable demands. Paper tigers all! My reply: I agree that those lopsided tests are too-clever-by-half. I think a lot of statistical methods have this flavor, that they are a solution to a mathematical problem that has been posed without a careful enough se
sentIndex sentText sentNum sentScore
1 Stuart Hurlbert writes: A colleague recently forwarded to me your 2012 paper with Hill and Yajima on the multiple comparison “non-problem”, as I call it. [sent-1, score-0.704]
2 You and your colleagues might find of interest a 2012 paper [with Celia Lombardi] that reaches similar conclusions by a colleague and myself which is attached. [sent-2, score-0.858]
3 Similar but not identical, as we are a bit Bayesian-shy after seeing so many exaggerated claims made for Bayesian approaches over recent decades. [sent-3, score-0.506]
4 I take pride in having for a few decades defended many colleagues against editors (and many graduate students against faculty members) who demanded “corrections” for multiple comparisons. [sent-4, score-1.614]
5 We’ve gotten no small number of editors and professors to back off their unreasonable demands. [sent-5, score-0.572]
6 My reply: I agree that those lopsided tests are too-clever-by-half. [sent-7, score-0.356]
7 I think a lot of statistical methods have this flavor, that they are a solution to a mathematical problem that has been posed without a careful enough sense of whether the problem is worth solving in the first place. [sent-8, score-0.791]
8 Another example are tests of contingency tables with fixed margins. [sent-9, score-0.621]
wordName wordTfidf (topN-words)
[('editors', 0.209), ('colleague', 0.192), ('lopsided', 0.19), ('tigers', 0.179), ('fixed', 0.179), ('many', 0.174), ('reaches', 0.171), ('contingency', 0.171), ('tests', 0.166), ('demanded', 0.165), ('forwarded', 0.165), ('colleagues', 0.158), ('margins', 0.156), ('exaggerated', 0.156), ('defended', 0.15), ('pride', 0.147), ('posed', 0.147), ('multiple', 0.145), ('sophistication', 0.142), ('stuart', 0.142), ('flavor', 0.14), ('unreasonable', 0.134), ('corrections', 0.131), ('similar', 0.127), ('solving', 0.127), ('gotten', 0.12), ('paper', 0.119), ('hill', 0.117), ('identical', 0.114), ('professors', 0.109), ('observational', 0.108), ('tables', 0.105), ('faculty', 0.103), ('rare', 0.1), ('members', 0.1), ('graduate', 0.099), ('table', 0.095), ('approaches', 0.093), ('extremely', 0.092), ('conclusions', 0.091), ('sense', 0.09), ('technical', 0.09), ('decades', 0.09), ('careful', 0.089), ('experiment', 0.087), ('mathematical', 0.086), ('solution', 0.086), ('comparison', 0.083), ('problem', 0.083), ('seeing', 0.083)]
simIndex simValue blogId blogTitle
Introduction: Stuart Hurlbert writes: A colleague recently forwarded to me your 2012 paper with Hill and Yajima on the multiple comparison “non-problem”, as I call it. You and your colleagues might find of interest a 2012 paper [with Celia Lombardi] that reaches similar conclusions by a colleague and myself which is attached. Similar but not identical, as we are a bit Bayesian-shy after seeing so many exaggerated claims made for Bayesian approaches over recent decades. I take pride in having for a few decades defended many colleagues against editors (and many graduate students against faculty members) who demanded “corrections” for multiple comparisons. We’ve gotten no small number of editors and professors to back off their unreasonable demands. Paper tigers all! My reply: I agree that those lopsided tests are too-clever-by-half. I think a lot of statistical methods have this flavor, that they are a solution to a mathematical problem that has been posed without a careful enough se
2 0.12224558 2269 andrew gelman stats-2014-03-27-Beyond the Valley of the Trolls
Introduction: In a further discussion of the discussion about the discussion of a paper in Administrative Science Quarterly, Thomas Basbøll writes: I [Basbøll] feel “entitled”, if that’s the right word (actually, I’d say I feel privileged), to express my opinions to anyone who wants to listen, and while I think it does say something about an author whether or not they answer a question (where what it says depends very much on the quality of the question), I don’t think the author has any obligation to me to respond immediately. If I succeed in raising doubts about something in the minds of many readers, then that’s obviously something an author should take seriously. The point is that an author has a responsibility to the readership of the paper, not any one critic. I agree that the ultimate audience is the scholarly community (and, beyond that, the general public) and that the critic is just serving as a conduit, the person who poses the Q in the Q-and-A. That said, I get frustrated frust
3 0.11835538 1582 andrew gelman stats-2012-11-18-How to teach methods we don’t like?
Introduction: April Galyardt writes: I’m teaching my first graduate class this semester. It’s intro stats for graduate students in the college of education. Most of the students are first year PhD students. Though, there are a number of master’s students who are primarily in-service teachers. The difficulties with teaching an undergraduate intro stats course are still present, in that mathematical preparation and phobia vary widely across the class. I’ve been enjoying the class and the students, but I’d like your take on an issue I’ve been thinking about. How do I balance teaching the standard methods, like hypothesis testing, that these future researchers have to know because they are so standard, with discussing the problems with those methods (e.g. p-value as a measure of sample size , and the decline effect , not to mention multiple testing and other common mistakes). It feels a bit like saying “Ok here’s what everybody does, but really it’s broken” and then there’s not enough time to tal
4 0.11640037 472 andrew gelman stats-2010-12-17-So-called fixed and random effects
Introduction: Someone writes: I am hoping you can give me some advice about when to use fixed and random effects model. I am currently working on a paper that examines the effect of . . . by comparing states . . . It got reviewed . . . by three economists and all suggest that we run a fixed effects model. We ran a hierarchial model in the paper that allow the intercept and slope to vary before and after . . . My question is which is correct? We have ran it both ways and really it makes no difference which model you run, the results are very similar. But for my own learning, I would really like to understand which to use under what circumstances. Is the fact that we use the whole population reason enough to just run a fixed effect model? Perhaps you can suggest a good reference to this question of when to run a fixed vs. random effects model. I’m not always sure what is meant by a “fixed effects model”; see my paper on Anova for discussion of the problems with this terminology: http://w
5 0.1106545 2268 andrew gelman stats-2014-03-26-New research journal on observational studies
Introduction: Dylan Small writes: I am starting an observational studies journal that aims to publish papers on all aspects of observational studies, including study protocols for observational studies, methodologies for observational studies, descriptions of data sets for observational studies, software for observational studies and analyses of observational studies. One of the goals of the journal is to promote the planning of observational studies and to publish study plans for observational studies, like study plans are published for major clinical trials. Regular readers will know my suggestion that scientific journals move away from the idea of being unique publishers of new material and move toward a “newsletter” approach, recommending papers from Arxiv, SSRN, etc. So, instead of going through exhausting review and revision processes, the journal editors would read and review recent preprints on observational studies and then, each month or quarter or whatever, produce a list of pap
6 0.10453378 1878 andrew gelman stats-2013-05-31-How to fix the tabloids? Toward replicable social science research
7 0.10291658 1695 andrew gelman stats-2013-01-28-Economists argue about Bayes
8 0.10121928 1081 andrew gelman stats-2011-12-24-Statistical ethics violation
9 0.099245735 1016 andrew gelman stats-2011-11-17-I got 99 comparisons but multiplicity ain’t one
10 0.09492702 2312 andrew gelman stats-2014-04-29-Ken Rice presents a unifying approach to statistical inference and hypothesis testing
11 0.094473302 1403 andrew gelman stats-2012-07-02-Moving beyond hopeless graphics
12 0.094147354 1241 andrew gelman stats-2012-04-02-Fixed effects and identification
13 0.092669941 466 andrew gelman stats-2010-12-13-“The truth wears off: Is there something wrong with the scientific method?”
14 0.091782659 963 andrew gelman stats-2011-10-18-Question on Type M errors
15 0.089951806 1989 andrew gelman stats-2013-08-20-Correcting for multiple comparisons in a Bayesian regression model
17 0.087722771 2172 andrew gelman stats-2014-01-14-Advice on writing research articles
18 0.087613888 2245 andrew gelman stats-2014-03-12-More on publishing in journals
19 0.087400474 1644 andrew gelman stats-2012-12-30-Fixed effects, followed by Bayes shrinkage?
20 0.086217999 972 andrew gelman stats-2011-10-25-How do you interpret standard errors from a regression fit to the entire population?
topicId topicWeight
[(0, 0.184), (1, 0.006), (2, -0.032), (3, -0.089), (4, -0.01), (5, 0.001), (6, -0.007), (7, 0.016), (8, -0.006), (9, -0.018), (10, 0.042), (11, -0.001), (12, -0.019), (13, -0.015), (14, 0.056), (15, -0.017), (16, -0.006), (17, 0.002), (18, -0.038), (19, 0.031), (20, 0.002), (21, 0.027), (22, 0.013), (23, 0.024), (24, -0.001), (25, -0.037), (26, 0.015), (27, 0.024), (28, -0.016), (29, 0.017), (30, 0.012), (31, 0.014), (32, 0.042), (33, 0.028), (34, -0.02), (35, -0.052), (36, 0.037), (37, 0.009), (38, -0.023), (39, 0.032), (40, 0.016), (41, 0.015), (42, 0.013), (43, 0.006), (44, -0.025), (45, -0.038), (46, -0.005), (47, 0.003), (48, 0.004), (49, -0.013)]
simIndex simValue blogId blogTitle
Introduction: Stuart Hurlbert writes: A colleague recently forwarded to me your 2012 paper with Hill and Yajima on the multiple comparison “non-problem”, as I call it. You and your colleagues might find of interest a 2012 paper [with Celia Lombardi] that reaches similar conclusions by a colleague and myself which is attached. Similar but not identical, as we are a bit Bayesian-shy after seeing so many exaggerated claims made for Bayesian approaches over recent decades. I take pride in having for a few decades defended many colleagues against editors (and many graduate students against faculty members) who demanded “corrections” for multiple comparisons. We’ve gotten no small number of editors and professors to back off their unreasonable demands. Paper tigers all! My reply: I agree that those lopsided tests are too-clever-by-half. I think a lot of statistical methods have this flavor, that they are a solution to a mathematical problem that has been posed without a careful enough se
Introduction: Economist Stan Liebowitz has a longstanding interest in the difficulties of flagging published research errors. Recently he wrote on the related topic of dishonest authorship: While not about direct research fraud, I thought you might be interested in this paper . It discusses the manner in which credit is given for economics articles, and I suspect it applies to many other areas as well. One of the conclusions is that the lack of complete proration per author will lead to excessive coauthorship, reducing overall research output by inducing the use of larger than efficient-sized teams. Under these circumstances, false authorship can be a response to the warped reward system and false authorship might improve research efficiency since it might keep actual research teams (as opposed to nominal teams) from being too large to produce research efficiently. One of the questions I rhetorically ask in the paper is whether anyone has ever been ‘punished’ for having their name included on
Introduction: Benedict Carey writes a follow-up article on ESP studies and Bayesian statistics. ( See here for my previous thoughts on the topic.) Everything Carey writes is fine, and he even uses an example I recommended: The statistical approach that has dominated the social sciences for almost a century is called significance testing. The idea is straightforward. A finding from any well-designed study — say, a correlation between a personality trait and the risk of depression — is considered “significant” if its probability of occurring by chance is less than 5 percent. This arbitrary cutoff makes sense when the effect being studied is a large one — for example, when measuring the so-called Stroop effect. This effect predicts that naming the color of a word is faster and more accurate when the word and color match (“red” in red letters) than when they do not (“red” in blue letters), and is very strong in almost everyone. “But if the true effect of what you are measuring is small,” sai
4 0.76054108 109 andrew gelman stats-2010-06-25-Classics of statistics
Introduction: Christian Robert is planning a graduate seminar in which students read 15 classic articles of statistics. (See here for more details and a slightly different list.) Actually, he just writes “classics,” but based on his list, I assume he only wants articles, not books. If he wanted to include classic books, I’d nominate the following, just for starters: - Fisher’s Statistical Methods for Research Workers - Snedecor and Cochran’s Statistical Methods - Kish’s Survey Sampling - Box, Hunter, and Hunter’s Statistics for Experimenters - Tukey’s Exploratory Data Analysis - Cleveland’s The Elements of Graphing Data - Mosteller and Wallace’s book on the Federalist Papers. Probably Cox and Hinkley, too. That’s a book that I don’t think has aged well, but it seems to have had a big influence. I think there’s a lot more good and accessible material in these classic books than in the equivalent volume of classic articles. Journal articles can be difficult to read and are typicall
Introduction: Chris Masse points me to this response by Daryl Bem and two statisticians (Jessica Utts and Wesley Johnson) to criticisms by Wagenmakers et.al. of Bem’s recent ESP study. I have nothing to add but would like to repeat a couple bits of my discussions of last month, of here : Classical statistical methods that work reasonably well when studying moderate or large effects (see the work of Fisher, Snedecor, Cochran, etc.) fall apart in the presence of small effects. I think it’s naive when people implicitly assume that the study’s claims are correct, or the study’s statistical methods are weak. Generally, the smaller the effects you’re studying, the better the statistics you need. ESP is a field of small effects and so ESP researchers use high-quality statistics. To put it another way: whatever methodological errors happen to be in the paper in question, probably occur in lots of researcher papers in “legitimate” psychology research. The difference is that when you’re studying a
6 0.74578273 2055 andrew gelman stats-2013-10-08-A Bayesian approach for peer-review panels? and a speculation about Bruno Frey
7 0.73874587 1585 andrew gelman stats-2012-11-20-“I know you aren’t the plagiarism police, but . . .”
8 0.73570925 2353 andrew gelman stats-2014-05-30-I posted this as a comment on a sociology blog
9 0.73399282 506 andrew gelman stats-2011-01-06-That silly ESP paper and some silliness in a rebuttal as well
10 0.73101854 1273 andrew gelman stats-2012-04-20-Proposals for alternative review systems for scientific work
11 0.72914481 1889 andrew gelman stats-2013-06-08-Using trends in R-squared to measure progress in criminology??
12 0.72786164 2137 andrew gelman stats-2013-12-17-Replication backlash
13 0.71934199 1272 andrew gelman stats-2012-04-20-More proposals to reform the peer-review system
15 0.71253866 1139 andrew gelman stats-2012-01-26-Suggested resolution of the Bem paradox
16 0.71159369 989 andrew gelman stats-2011-11-03-This post does not mention Wegman
19 0.71037567 854 andrew gelman stats-2011-08-15-A silly paper that tries to make fun of multilevel models
20 0.70918626 1917 andrew gelman stats-2013-06-28-Econ coauthorship update
topicId topicWeight
[(2, 0.033), (12, 0.03), (15, 0.044), (16, 0.067), (21, 0.031), (24, 0.099), (42, 0.015), (45, 0.024), (86, 0.01), (94, 0.137), (95, 0.016), (99, 0.403)]
simIndex simValue blogId blogTitle
Introduction: Stuart Hurlbert writes: A colleague recently forwarded to me your 2012 paper with Hill and Yajima on the multiple comparison “non-problem”, as I call it. You and your colleagues might find of interest a 2012 paper [with Celia Lombardi] that reaches similar conclusions by a colleague and myself which is attached. Similar but not identical, as we are a bit Bayesian-shy after seeing so many exaggerated claims made for Bayesian approaches over recent decades. I take pride in having for a few decades defended many colleagues against editors (and many graduate students against faculty members) who demanded “corrections” for multiple comparisons. We’ve gotten no small number of editors and professors to back off their unreasonable demands. Paper tigers all! My reply: I agree that those lopsided tests are too-clever-by-half. I think a lot of statistical methods have this flavor, that they are a solution to a mathematical problem that has been posed without a careful enough se
2 0.97152621 1943 andrew gelman stats-2013-07-18-Data to use for in-class sampling exercises?
Introduction: Mark Street writes: I teach a high school (grade 11) statistics class outside the USA and I am always looking for hands-on demonstrations. In fact, last week (the start of our school year here), I did the in-class exercise about “guessing ages of ten pictures” (p. 11-13) from your book “Teaching Statistics – A Bag of Tricks”. I am interested in using the “candy weighing” demonstration (p. 120-121) to talk about random sampling. I agree with your advice (p.48, Sec 5.1) that it’s better to have students do sampling from actual populations. I also agree with your suggestion that actual doing personal interviews is not an effective use of time, except for larger projects. To that end, can you suggest some sources of actual population data that I could use in class? As I am outside the USA (in Thailand), we do not have phonebooks here. Certainly, this must soon be a problem for people in the USA with phonebooks going the way of the 8-track tape. I even looked online for di
3 0.95583969 1211 andrew gelman stats-2012-03-13-A personal bit of spam, just for me!
Introduction: Hi Andrew, I came across your site while searching for blogs and posts around American obesity and wanted to reach out to get your readership’s feedback on an infographic my team built which focuses on the obesity of America and where we could end up at the going rate. If you’re interested, let’s connect. Have a great weekend! Thanks. *** I have to say, that’s pretty pitiful, to wish someone a “great weekend” on a Tuesday! This guy’s gotta ratchet up his sophistication a few notches if he ever wants to get a job as a spammer for a major software company , for example.
Introduction: Juli writes: I’m helping a professor out with an analysis, and I was hoping that you might be able to point me to some relevant literature… She has two studies that have been completed already (so we can’t go back to the planning stage in terms of sampling, unfortunately). Both studies are based around the population of adults in LA who attended LA public high schools at some point, so that is the same for both studies. Study #1 uses random digit dialing, so I consider that one to be SRS. Study #2, however, is a convenience sample in which all participants were involved with one of eight community-based organizations (CBOs). Of course, both studies can be analyzed independently, but she was hoping for there to be some way to combine/compare the two studies. Specifically, I am working on looking at the civic engagement of the adults in both studies. In study #1, this means looking at factors such as involvement in student government. In study #2, this means looking at involv
Introduction: My friend Seth, whom I know from Berkeley (we taught a course together on left-handedness), has a blog on topics ranging from thoughtful discussions of scientific evidence, to experiences with his unconventional weight-loss scheme, offbeat self-experimentation, and advocacy of fringe scientific theories, leavened with occasional dollops of cynicism and political extremism . I agree with Seth on some things but not others. ( Here’s Seth’s reason for not attempting a clinical trial of his diet.) Recently I was disturbed (but, I’m sorry to say, not surprised) to see Seth post the following: Predictions of climate models versus reality . I [Seth] have only seen careful prediction-vs-reality comparisons made by AGW [anthropogenic global warming] skeptics. Those who believe humans are dangerously warming the planet appear to be silent on this subject. In response, Phil commented : Funny, on the day you [Seth] made your post saying that you haven’t seen comparis
6 0.94852281 1683 andrew gelman stats-2013-01-19-“Confirmation, on the other hand, is not sexy”
7 0.94501442 582 andrew gelman stats-2011-02-20-Statisticians vs. everybody else
8 0.94340283 2269 andrew gelman stats-2014-03-27-Beyond the Valley of the Trolls
10 0.94208246 2327 andrew gelman stats-2014-05-09-Nicholas Wade and the paradox of racism
12 0.94115508 1832 andrew gelman stats-2013-04-29-The blogroll
13 0.94099665 709 andrew gelman stats-2011-05-13-D. Kahneman serves up a wacky counterfactual
14 0.94097614 2151 andrew gelman stats-2013-12-27-Should statistics have a Nobel prize?
15 0.94087875 2006 andrew gelman stats-2013-09-03-Evaluating evidence from published research
16 0.94057894 2245 andrew gelman stats-2014-03-12-More on publishing in journals
18 0.94039369 1807 andrew gelman stats-2013-04-17-Data problems, coding errors…what can be done?
19 0.94034934 1635 andrew gelman stats-2012-12-22-More Pinker Pinker Pinker
20 0.94032633 1030 andrew gelman stats-2011-11-27-Historian and journalist slug it out