andrew_gelman_stats andrew_gelman_stats-2011 andrew_gelman_stats-2011-540 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Joan Nix writes: Your comments on this paper by Scott Carrell and James West would be most appreciated. I’m afraid the conclusions of this paper are too strong given the data set and other plausible explanations. But given where it is published, this paper is receiving and will continue to receive lots of attention. It will be used to draw deeper conclusions regarding effective teaching and experience. Nix also links to this discussion by Jeff Ely. I don’t completely follow Ely’s criticism, which seems to me to be too clever by half, but I agree with Nix that the findings in the research article don’t seem to fit together very well. For example, Carrell and West estimate that the effects of instructors on performance in the follow-on class is as large as the effects on the class they’re teaching. This seems hard to believe, and it seems central enough to their story that I don’t know what to think about everything else in the paper. My other thought about teaching eva
sentIndex sentText sentNum sentScore
1 I’m afraid the conclusions of this paper are too strong given the data set and other plausible explanations. [sent-2, score-0.284]
2 But given where it is published, this paper is receiving and will continue to receive lots of attention. [sent-3, score-0.242]
3 It will be used to draw deeper conclusions regarding effective teaching and experience. [sent-4, score-0.333]
4 I don’t completely follow Ely’s criticism, which seems to me to be too clever by half, but I agree with Nix that the findings in the research article don’t seem to fit together very well. [sent-6, score-0.508]
5 For example, Carrell and West estimate that the effects of instructors on performance in the follow-on class is as large as the effects on the class they’re teaching. [sent-7, score-0.477]
6 This seems hard to believe, and it seems central enough to their story that I don’t know what to think about everything else in the paper. [sent-8, score-0.192]
7 My other thought about teaching evaluations is from my personal experience. [sent-9, score-0.406]
8 When I feel I’ve taught well–that is, in semesters when it seems that students have really learned something–I tend to get good evaluations. [sent-10, score-0.34]
9 When I don’t think I’ve taught well, my evaluations aren’t so good. [sent-11, score-0.382]
10 And, even when I think my course has gone wonderfully, my evaluations are usually far from perfect. [sent-12, score-0.255]
11 That said, I’d prefer to have objective measures of my teaching effectiveness. [sent-14, score-0.218]
12 Perhaps surprisingly, statisticians aren’t so good about measurement and estimation when applied to their own teaching. [sent-15, score-0.116]
13 ) The trouble is that measurement and evaluation take work! [sent-17, score-0.116]
14 When we’re giving advice to scientists, we’re always yammering on about experimentation and measurement. [sent-18, score-0.202]
15 I continued to be stunned by the way in which tables of numbers are presented in social science research papers with no thought of communication with, for example, tables with interval estimate such as “(. [sent-27, score-0.905]
16 And what do these numbers have to do with anything at all? [sent-31, score-0.147]
17 If the words, sentences, and paragraphs of an article were put together in such a stylized, unthinking way, the article would be completely unreadable. [sent-33, score-0.433]
18 Formal structures with almost no connection to communication or content . [sent-34, score-0.202]
19 it would be like writing the entire research article in iambic pentameter with an a,b,c,b rhyme scheme, or somesuch. [sent-37, score-0.277]
20 I’m not trying to pick on Carrell and West here–this sort of presentation is nearly universal in social science journals. [sent-38, score-0.144]
wordName wordTfidf (topN-words)
[('carrell', 0.386), ('nix', 0.386), ('evaluations', 0.255), ('west', 0.217), ('teaching', 0.151), ('tables', 0.129), ('taught', 0.127), ('communication', 0.121), ('semesters', 0.117), ('rhyme', 0.117), ('yammering', 0.117), ('measurement', 0.116), ('conclusions', 0.112), ('joan', 0.11), ('wonderfully', 0.106), ('paper', 0.098), ('aren', 0.097), ('seems', 0.096), ('digits', 0.094), ('together', 0.093), ('article', 0.091), ('stylized', 0.087), ('completely', 0.087), ('class', 0.087), ('stunned', 0.086), ('experimentation', 0.085), ('instructors', 0.084), ('continued', 0.083), ('numbers', 0.081), ('structures', 0.081), ('blogged', 0.078), ('scheme', 0.077), ('universal', 0.077), ('surprisingly', 0.075), ('receiving', 0.074), ('afraid', 0.074), ('estimate', 0.073), ('sentences', 0.073), ('effects', 0.073), ('clever', 0.072), ('paragraphs', 0.071), ('deeper', 0.07), ('receive', 0.07), ('scott', 0.07), ('research', 0.069), ('social', 0.067), ('interval', 0.067), ('objective', 0.067), ('anything', 0.066), ('formal', 0.066)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999976 540 andrew gelman stats-2011-01-26-Teaching evaluations, instructor effectiveness, the Journal of Political Economy, and the Holy Roman Empire
Introduction: Joan Nix writes: Your comments on this paper by Scott Carrell and James West would be most appreciated. I’m afraid the conclusions of this paper are too strong given the data set and other plausible explanations. But given where it is published, this paper is receiving and will continue to receive lots of attention. It will be used to draw deeper conclusions regarding effective teaching and experience. Nix also links to this discussion by Jeff Ely. I don’t completely follow Ely’s criticism, which seems to me to be too clever by half, but I agree with Nix that the findings in the research article don’t seem to fit together very well. For example, Carrell and West estimate that the effects of instructors on performance in the follow-on class is as large as the effects on the class they’re teaching. This seems hard to believe, and it seems central enough to their story that I don’t know what to think about everything else in the paper. My other thought about teaching eva
2 0.16311336 2048 andrew gelman stats-2013-10-03-A comment on a post at the Monkey Cage
Introduction: The sister blog has moved to the Washington Post. It’s harder to leave comments there, so I’ll post my comments to Monkey Cage posts here instead. Political scientist Lisa Martin wrote a post on student evaluations of teaching, based on a recent paper where she writes: Many female faculty believe that they face prejudice in student evaluations of teaching (SETs), and that this prejudice may be exaggerated by developments such as online evaluations and the prevalence of sites such as RateMyProfessor. However, systematic studies of SETs are mixed in their findings of gender bias. As a statistician, I always like to hear this sort of moderate statement. On the blog, Martin shows the following graph based on data from the political science departments of “publicly available SET data from two large public universities, one in the South and the other on the West Coast”: The graph left me with two questions and a comment: 1. If the data were public, why are the names
3 0.13298798 2172 andrew gelman stats-2014-01-14-Advice on writing research articles
Introduction: From a few years ago : General advice Both the papers sent to me appear to have strong research results. Now that the research has been done, I’d recommend rewriting both articles from scratch, using the following template: 1. Start with the conclusions. Write a couple pages on what you’ve found and what you recommend. In writing these conclusions, you should also be writing some of the introduction, in that you’ll need to give enough background so that general readers can understand what you’re talking about and why they should care. But you want to start with the conclusions, because that will determine what sort of background information you’ll need to give. 2. Now step back. What is the principal evidence for your conclusions? Make some graphs and pull out some key numbers that represent your research findings which back up your claims. 3. Back one more step, now. What are the methods and data you used to obtain your research findings. 4. Now go back and write the l
4 0.1218282 2245 andrew gelman stats-2014-03-12-More on publishing in journals
Introduction: I’m postponing today’s scheduled post (“Empirical implications of Empirical Implications of Theoretical Models”) to continue the lively discussion from yesterday, What if I were to stop publishing in journals? . An example: my papers with Basbøll Thomas Basbøll and I got into a long discussion on our blogs about business school professor Karl Weick and other cases of plagiarism copying text without attribution. We felt it useful to take our ideas to the next level and write them up as a manuscript, which ended up being logical to split into two papers. At that point I put some effort into getting these papers published, which I eventually did: To throw away data: Plagiarism as a statistical crime went into American Scientist and When do stories work? Evidence and illustration in the social sciences will appear in Sociological Methods and Research. The second paper, in particular, took some effort to place; I got some advice from colleagues in sociology as to where
5 0.11880938 1338 andrew gelman stats-2012-05-23-Advice on writing research articles
Introduction: From a few years ago : Both the papers sent to me appear to have strong research results. Now that the research has been done, I’d recommend rewriting both articles from scratch , using the following template: 1. Start with the conclusions. Write a couple pages on what you’ve found and what you recommend. In writing these conclusions, you should also be writing some of the introduction, in that you’ll need to give enough background so that general readers can understand what you’re talking about and why they should care. But you want to start with the conclusions, because that will determine what sort of background information you’ll need to give. 2. Now step back. What is the principal evidence for your conclusions? Make some graphs and pull out some key numbers that represent your research findings which back up your claims. 3. Back one more step, now. What are the methods and data you used to obtain your research findings. 4. Now go back and write the literature review
6 0.099528417 536 andrew gelman stats-2011-01-24-Trends in partisanship by state
7 0.099500775 1435 andrew gelman stats-2012-07-30-Retracted articles and unethical behavior in economics journals?
8 0.099440746 1833 andrew gelman stats-2013-04-30-“Tragedy of the science-communication commons”
11 0.096179619 2287 andrew gelman stats-2014-04-09-Advice: positive-sum, zero-sum, or negative-sum
12 0.095972843 2279 andrew gelman stats-2014-04-02-Am I too negative?
13 0.094933495 1517 andrew gelman stats-2012-10-01-“On Inspiring Students and Being Human”
15 0.093519002 2353 andrew gelman stats-2014-05-30-I posted this as a comment on a sociology blog
16 0.093306407 1611 andrew gelman stats-2012-12-07-Feedback on my Bayesian Data Analysis class at Columbia
17 0.093155824 1928 andrew gelman stats-2013-07-06-How to think about papers published in low-grade journals?
18 0.092551559 785 andrew gelman stats-2011-07-02-Experimental reasoning in social science
19 0.092374884 1264 andrew gelman stats-2012-04-14-Learning from failure
20 0.09171807 51 andrew gelman stats-2010-05-26-If statistics is so significantly great, why don’t statisticians use statistics?
topicId topicWeight
[(0, 0.212), (1, -0.061), (2, -0.046), (3, -0.057), (4, -0.01), (5, 0.004), (6, 0.008), (7, 0.043), (8, -0.034), (9, 0.005), (10, 0.069), (11, 0.04), (12, -0.035), (13, -0.012), (14, 0.022), (15, -0.045), (16, -0.045), (17, 0.023), (18, -0.051), (19, 0.003), (20, 0.033), (21, -0.033), (22, 0.001), (23, 0.003), (24, -0.009), (25, -0.011), (26, 0.008), (27, 0.027), (28, -0.002), (29, 0.034), (30, 0.03), (31, 0.005), (32, -0.038), (33, -0.073), (34, -0.012), (35, -0.001), (36, -0.03), (37, 0.001), (38, 0.004), (39, -0.034), (40, 0.062), (41, 0.013), (42, -0.018), (43, -0.003), (44, -0.001), (45, 0.023), (46, 0.013), (47, 0.011), (48, -0.048), (49, 0.014)]
simIndex simValue blogId blogTitle
same-blog 1 0.97172487 540 andrew gelman stats-2011-01-26-Teaching evaluations, instructor effectiveness, the Journal of Political Economy, and the Holy Roman Empire
Introduction: Joan Nix writes: Your comments on this paper by Scott Carrell and James West would be most appreciated. I’m afraid the conclusions of this paper are too strong given the data set and other plausible explanations. But given where it is published, this paper is receiving and will continue to receive lots of attention. It will be used to draw deeper conclusions regarding effective teaching and experience. Nix also links to this discussion by Jeff Ely. I don’t completely follow Ely’s criticism, which seems to me to be too clever by half, but I agree with Nix that the findings in the research article don’t seem to fit together very well. For example, Carrell and West estimate that the effects of instructors on performance in the follow-on class is as large as the effects on the class they’re teaching. This seems hard to believe, and it seems central enough to their story that I don’t know what to think about everything else in the paper. My other thought about teaching eva
2 0.78109908 516 andrew gelman stats-2011-01-14-A new idea for a science core course based entirely on computer simulation
Introduction: Columbia College has for many years had a Core Curriculum, in which students read classics such as Plato (in translation) etc. A few years ago they created a Science core course. There was always some confusion about this idea: On one hand, how much would college freshmen really learn about science by reading the classic writings of Galileo, Laplace, Darwin, Einstein, etc.? And they certainly wouldn’t get much out by puzzling over the latest issues of Nature, Cell, and Physical Review Letters. On the other hand, what’s the point of having them read Dawkins, Gould, or even Brian Greene? These sorts of popularizations give you a sense of modern science (even to the extent of conveying some of the debates in these fields), but reading them might not give the same intellectual engagement that you’d get from wrestling with the Bible or Shakespeare. I have a different idea. What about structuring the entire course around computer programming and simulation? Start with a few weeks t
Introduction: After posting on David Rubinstein’s remarks on his “cushy life” as a sociology professor at a public university, I read these remarks by some of Rubinstein’s colleagues at the University of Illinois, along with a response from Rubinstein. Before getting to the policy issues, let me first say that I think it must have been so satisfying, first for Rubinstein and then for his colleagues (Barbara Risman, William Bridges, and Anthony Orum) to publish these notes. We all have people we know and hate, but we rarely have a good excuse for blaring our feelings in public. (I remember when I was up for tenure, I was able to read the outside letters on my case (it’s a public university and they have rules), and one of the letter writers really hated my guts. I was surprised–I didn’t know the guy well (the letters were anonymized but it was clear from context who the letter writer was) but the few times we’d met, he’d been cordial enough–but there you have it. He must have been thrilled t
4 0.75755936 2245 andrew gelman stats-2014-03-12-More on publishing in journals
Introduction: I’m postponing today’s scheduled post (“Empirical implications of Empirical Implications of Theoretical Models”) to continue the lively discussion from yesterday, What if I were to stop publishing in journals? . An example: my papers with Basbøll Thomas Basbøll and I got into a long discussion on our blogs about business school professor Karl Weick and other cases of plagiarism copying text without attribution. We felt it useful to take our ideas to the next level and write them up as a manuscript, which ended up being logical to split into two papers. At that point I put some effort into getting these papers published, which I eventually did: To throw away data: Plagiarism as a statistical crime went into American Scientist and When do stories work? Evidence and illustration in the social sciences will appear in Sociological Methods and Research. The second paper, in particular, took some effort to place; I got some advice from colleagues in sociology as to where
5 0.7570464 1254 andrew gelman stats-2012-04-09-In the future, everyone will publish everything.
Introduction: Bob told me the other day (the other week, actually, as I’m stacking up posts here with a roughly one-month delay) that I shouldn’t try to compete with the electrical engineers when it comes to length of C.V.: according to Bob, these dudes can have over two thousand publications! How do they do it? First, an EE prof will have tons of graduate students and postdocs, they’re all writing papers and presenting at conferences, and they all stick his name on the author list. Second, these students and postdocs write up and publish every experiment they do . Including (especially!) computer experiments. And . . . all these people writing paper cite each other, so they quickly rack up thousands of citations. Upon hearing this, my first reaction to this was fear, plain and simple. One of the distinguishing characteristics of my own research record is that I have so many publications and citations. Those electrical engineers . . . how dare they go around devaluing my currency!
6 0.75606769 2006 andrew gelman stats-2013-09-03-Evaluating evidence from published research
7 0.74738538 1224 andrew gelman stats-2012-03-21-Teaching velocity and acceleration
9 0.74077898 1056 andrew gelman stats-2011-12-13-Drawing to Learn in Science
11 0.73658174 109 andrew gelman stats-2010-06-25-Classics of statistics
12 0.73457634 236 andrew gelman stats-2010-08-26-Teaching yourself mathematics
13 0.73132217 1273 andrew gelman stats-2012-04-20-Proposals for alternative review systems for scientific work
14 0.73048699 2269 andrew gelman stats-2014-03-27-Beyond the Valley of the Trolls
17 0.72379446 1139 andrew gelman stats-2012-01-26-Suggested resolution of the Bem paradox
18 0.72316939 1960 andrew gelman stats-2013-07-28-More on that machine learning course
19 0.72309881 2142 andrew gelman stats-2013-12-21-Chasing the noise
20 0.7220304 1028 andrew gelman stats-2011-11-26-Tenure lets you handle students who cheat
topicId topicWeight
[(15, 0.021), (16, 0.094), (24, 0.142), (30, 0.015), (44, 0.015), (55, 0.013), (61, 0.011), (72, 0.013), (85, 0.17), (86, 0.025), (95, 0.032), (99, 0.321)]
simIndex simValue blogId blogTitle
1 0.96349567 2300 andrew gelman stats-2014-04-21-Ticket to Baaaath
Introduction: Ooooooh, I never ever thought I’d have a legitimate excuse to tell this story, and now I do! The story took place many years ago, but first I have to tell you what made me think of it: Rasmus Bååth posted the following comment last month: On airplane tickets a Swedish “å” is written as “aa” resulting in Rasmus Baaaath. Once I bought a ticket online and five minutes later a guy from Lufthansa calls me and asks if I misspelled my name… OK, now here’s my story (which is not nearly as good). A long time ago (but when I was already an adult), I was in England for some reason, and I thought I’d take a day trip from London to Bath. So here I am on line, trying to think of what to say at the ticket counter. I remember that in England, they call Bath, Bahth. So, should I ask for “a ticket to Bahth”? I’m not sure, I’m afraid that it will sound silly, like I’m trying to fake an English accent. So, when I get to the front of the line, I say, hesitantly, “I’d like a ticket to Bath?
2 0.95682609 417 andrew gelman stats-2010-11-17-Clutering and variance components
Introduction: Raymond Lim writes: Do you have any recommendations on clustering and binary models? My particular problem is I’m running a firm fixed effect logit and want to cluster by industry-year (every combination of industry-year). My control variable of interest in measured by industry-year and when I cluster by industry-year, the standard errors are 300x larger than when I don’t cluster. Strangely, this problem only occurs when doing logit and not OLS (linear probability). Also, clustering just by field doesn’t blow up the errors. My hunch is it has something to do with the non-nested structure of year, but I don’t understand why this is only problematic under logit and not OLS. My reply: I’d recommend including four multilevel variance parameters, one for firm, one for industry, one for year, and one for industry-year. (In lmer, that’s (1 | firm) + (1 | industry) + (1 | year) + (1 | industry.year)). No need to include (1 | firm.year) since in your data this is the error term. Try
3 0.9553051 1790 andrew gelman stats-2013-04-06-Calling Jenny Davidson . . .
Introduction: Now that you have some free time again, you’ll have to check out these books and tell us if they’re worth reading. Claire Kirch reports : Lizzie Skurnick Books launches in September with the release of Debutante Hill by Lois Duncan. The novel, which was originally published by Dodd, Mead, in 1958, has been out of print for about three decades. The other books on the initial list, all reissues, are A Long Day in November by Ernest J. Gaines (originally published in 1971), Happy Endings Are All Alike by Sandra Scoppettone (1979), I’ll Love You When You’re More Like Me by M.E. Kerr (1977), Secret Lives by Berthe Amoss (1979), To All My Fans, With Love, From Sylvie by Ellen Conford (1982), and Me and Fat Glenda by Lila Perl (1972). . . . Noting that many of the books of that era beloved by teen boys are still in print – such as Isaac Asimov’s novels and The Chocolate War by Robert Cormier – Skurnick pointed out that, in contrast, many of the books that were embraced by teen gir
Introduction: Ben Hyde sends along this appealing image by Michael Paukner, which represents a nearly perfect distillation of “infographics”: Here are some of the comments on the linked page: Rather than redrawing the picture to make the lines more clear, I’d say: leave the graphic as is, and have a link to a set of statistical graphs that show where the different sorts of old trees are and what they look like. Let’s value the above image for its clean look and its clever Christmas-tree design, and once we have it, take advantage of viewers’ interest in the topic to show them more. P.S. See my comment below which I think further illuminates the appeal of this particular tree.
5 0.95163524 1374 andrew gelman stats-2012-06-11-Convergence Monitoring for Non-Identifiable and Non-Parametric Models
Introduction: Becky Passonneau and colleagues at the Center for Computational Learning Systems (CCLS) at Columbia have been working on a project for ConEd (New York’s major electric utility) to rank structures based on vulnerability to secondary events (e.g., transformer explosions, cable meltdowns, electrical fires). They’ve been using the R implementation BayesTree of Chipman, George and McCulloch’s Bayesian Additive Regression Trees (BART). BART is a Bayesian non-parametric method that is non-identifiable in two ways. Firstly, it is an additive tree model with a fixed number of trees, the indexes of which aren’t identified (you get the same predictions in a model swapping the order of the trees). This is the same kind of non-identifiability you get with any mixture model (additive or interpolated) with an exchangeable prior on the mixture components. Secondly, the trees themselves have varying structure over samples in terms of number of nodes and their topology (depth, branching, etc
6 0.95041931 912 andrew gelman stats-2011-09-15-n = 2
7 0.95035279 734 andrew gelman stats-2011-05-28-Funniest comment ever
8 0.95030487 375 andrew gelman stats-2010-10-28-Matching for preprocessing data for causal inference
9 0.94949466 1187 andrew gelman stats-2012-02-27-“Apple confronts the law of large numbers” . . . huh?
same-blog 10 0.94659787 540 andrew gelman stats-2011-01-26-Teaching evaluations, instructor effectiveness, the Journal of Political Economy, and the Holy Roman Empire
11 0.9409219 843 andrew gelman stats-2011-08-07-Non-rant
12 0.93748158 584 andrew gelman stats-2011-02-22-“Are Wisconsin Public Employees Underpaid?”
13 0.93607062 58 andrew gelman stats-2010-05-29-Stupid legal crap
14 0.93236148 1534 andrew gelman stats-2012-10-15-The strange reappearance of Matthew Klam
15 0.92713428 796 andrew gelman stats-2011-07-10-Matching and regression: two great tastes etc etc
16 0.92145866 1318 andrew gelman stats-2012-05-13-Stolen jokes
17 0.92117715 610 andrew gelman stats-2011-03-13-Secret weapon with rare events
18 0.91994935 1175 andrew gelman stats-2012-02-19-Factual – a new place to find data
19 0.91769278 167 andrew gelman stats-2010-07-27-Why don’t more medical discoveries become cures?
20 0.91200745 2216 andrew gelman stats-2014-02-18-Florida backlash