andrew_gelman_stats andrew_gelman_stats-2010 andrew_gelman_stats-2010-344 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: This one belongs in the statistical lexicon. Kaiser Fung nails it : In reading [news] articles, we must look out for the moment(s) when the reporters announce story time. Much of the article is great propaganda for the statistics lobby, describing an attempt to use observational data to address a practical question, sort of a Freakonomics-style application. We have no problems when they say things like: “There is a substantial gap at year’s end between students whose teachers were in the top 10% in effectiveness and the bottom 10%. The fortunate students ranked 17 percentile points higher in English and 25 points higher in math.” Or this: “On average, Smith’s students slide under his instruction, losing 14 percentile points in math during the school year relative to their peers districtwide, The Times found. Overall, he ranked among the least effective of the district’s elementary school teachers.” Midway through the article (right before the section called “Study in contras
sentIndex sentText sentNum sentScore
1 Kaiser Fung nails it : In reading [news] articles, we must look out for the moment(s) when the reporters announce story time. [sent-2, score-0.608]
2 Much of the article is great propaganda for the statistics lobby, describing an attempt to use observational data to address a practical question, sort of a Freakonomics-style application. [sent-3, score-0.1]
3 We have no problems when they say things like: “There is a substantial gap at year’s end between students whose teachers were in the top 10% in effectiveness and the bottom 10%. [sent-4, score-0.498]
4 The fortunate students ranked 17 percentile points higher in English and 25 points higher in math. [sent-5, score-0.935]
5 ” Or this: “On average, Smith’s students slide under his instruction, losing 14 percentile points in math during the school year relative to their peers districtwide, The Times found. [sent-6, score-0.715]
6 Overall, he ranked among the least effective of the district’s elementary school teachers. [sent-7, score-0.606]
7 But the surest sign of a teacher’s effectiveness was the engagement of his or her students — something that often was obvious from the expressions on their faces. [sent-10, score-0.471]
8 At the very moment they tell readers that engaging students makes teachers more effective, they announce “Story time! [sent-11, score-0.772]
9 ” With barely a fuss, they move from an evidence-based analysis of test scores to a speculation on cause–effect. [sent-12, score-0.077]
10 Their story is no more credible than anybody else’s story, unless they also provide data to support such a causal link. [sent-13, score-0.657]
11 As Jennifer frequently reminds me, we–researchers and also the general public–generally do care about causal inference. [sent-15, score-0.128]
12 So I have a lot of sympathy for researchers and reporters who go beyond the descriptive content of their data and start speculating. [sent-16, score-0.386]
13 The problem, as Kaiser notes, is when the line isn’t drawn clearly, in the short time leading the reader astray and in the longer term, perhaps, discrediting social-scientific research more generally. [sent-17, score-0.1]
14 We also see it in journal articles all the time. [sent-20, score-0.079]
15 It’s that all-too-quick moment when the authors pivot from the causal estimates they’ve proved, to their speculations, which, as Kaiser says, are “no more credible than anybody else’s story. [sent-21, score-0.781]
16 ” Maybe less credible, in fact, because researchers can fool themselves into thinking they’ve proved something when they haven’t. [sent-22, score-0.231]
wordName wordTfidf (topN-words)
[('credible', 0.251), ('kaiser', 0.228), ('reporters', 0.213), ('teachers', 0.187), ('students', 0.176), ('moment', 0.171), ('ranked', 0.167), ('percentile', 0.164), ('announce', 0.161), ('elementary', 0.159), ('effective', 0.157), ('story', 0.147), ('effectiveness', 0.135), ('proved', 0.134), ('anybody', 0.131), ('causal', 0.128), ('school', 0.123), ('astray', 0.1), ('midway', 0.1), ('pivot', 0.1), ('propaganda', 0.1), ('researchers', 0.097), ('lobby', 0.096), ('fortunate', 0.096), ('classrooms', 0.096), ('fuss', 0.096), ('visits', 0.09), ('belongs', 0.09), ('italics', 0.087), ('differed', 0.087), ('arrive', 0.087), ('contrasts', 0.087), ('nails', 0.087), ('slide', 0.085), ('instruction', 0.084), ('strict', 0.084), ('peers', 0.084), ('higher', 0.083), ('points', 0.083), ('angeles', 0.082), ('expressions', 0.081), ('los', 0.079), ('engagement', 0.079), ('articles', 0.079), ('else', 0.077), ('engaging', 0.077), ('barely', 0.077), ('instructors', 0.076), ('sympathy', 0.076), ('speculations', 0.075)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999982 344 andrew gelman stats-2010-10-15-Story time
Introduction: This one belongs in the statistical lexicon. Kaiser Fung nails it : In reading [news] articles, we must look out for the moment(s) when the reporters announce story time. Much of the article is great propaganda for the statistics lobby, describing an attempt to use observational data to address a practical question, sort of a Freakonomics-style application. We have no problems when they say things like: “There is a substantial gap at year’s end between students whose teachers were in the top 10% in effectiveness and the bottom 10%. The fortunate students ranked 17 percentile points higher in English and 25 points higher in math.” Or this: “On average, Smith’s students slide under his instruction, losing 14 percentile points in math during the school year relative to their peers districtwide, The Times found. Overall, he ranked among the least effective of the district’s elementary school teachers.” Midway through the article (right before the section called “Study in contras
2 0.18355638 226 andrew gelman stats-2010-08-23-More on those L.A. Times estimates of teacher effectiveness
Introduction: In discussing the ongoing Los Angeles Times series on teacher effectiveness, Alex Tabarrok and I both were impressed that the newspaper was reporting results on individual teachers, moving beyond the general research findings (“teachers matter,” “KIPP really works, but it requires several extra hours in the school day,” and so forth) that we usually see from value-added analyses in education. My first reaction was that the L.A. Times could get away with this because, unlike academic researchers, they can do whatever they want as long as they don’t break the law. They don’t have to answer to an Institutional Review Board. (By referring to this study by its publication outlet rather than its authors, I’m violating my usual rule (see the last paragraph here ). In this case, I think it’s ok to refer to the “L.A. Times study” because what’s notable is not the analysis (thorough as it may be) but how it is being reported.) Here I’d like to highlight a few other things came up in our
Introduction: Alex Tabarrok reports on an analysis from the Los Angeles Times of teacher performance (as measured by so-called value-added analysis, which is basically compares teachers based on their students’ average test scores at the end of the year, after controlling for pre-test scores. It’s well known that some teachers are much better than others, but, as Alex points out, what’s striking about the L.A. Times study is that they are publishing the estimates for individual teachers . For example, this: Nice graphics, too. To me, this illustrates one of the big advantages of research in a non-academic environment. If you’re writing an article for the L.A. Times, you can do what you want (within the limits of the law). If you’re doing the same research study at a university, there are a million restrictions. For example, from an official documen t, “The primary purpose of an Institutional Review Board (IRB) is to protect the rights and welfare of human subjects participati
4 0.16974491 1803 andrew gelman stats-2013-04-14-Why girls do better in school
Introduction: Wayne Folta writes, “In light of your recent blog post on women in higher education, here’s one I just read about on a techie website regarding elementary education”: Why do girls get better grades in elementary school than boys—even when they perform worse on standardized tests? New research . . . suggests that it’s because of their classroom behavior, which may lead teachers to assign girls higher grades than their male counterparts. . . . The study, co-authored by [Christopher] Cornwell and David Mustard at UGA and Jessica Van Parys at Columbia, analyzed data on more than 5,800 students from kindergarten through fifth grade. It examined students’ performance on standardized tests in three categories—reading, math and science-linking test scores to teachers’ assessments of their students’ progress, both academically and more broadly. The data show, for the first time, that gender disparities in teacher grades start early and uniformly favor girls. In every subject area, bo
5 0.16004148 648 andrew gelman stats-2011-04-04-The Case for More False Positives in Anti-doping Testing
Introduction: No joke. See here (from Kaiser Fung). At the Statistics Forum.
6 0.15576296 543 andrew gelman stats-2011-01-28-NYT shills for personal DNA tests
7 0.14919037 261 andrew gelman stats-2010-09-07-The $900 kindergarten teacher
8 0.14657639 1350 andrew gelman stats-2012-05-28-Value-added assessment: What went wrong?
9 0.14529863 2186 andrew gelman stats-2014-01-26-Infoviz on top of stat graphic on top of spreadsheet
10 0.13764469 1226 andrew gelman stats-2012-03-22-Story time meets the all-else-equal fallacy and the fallacy of measurement
11 0.13728124 1620 andrew gelman stats-2012-12-12-“Teaching effectiveness” as another dimension in cognitive ability
12 0.13655356 2031 andrew gelman stats-2013-09-19-What makes a statistician look like a hero?
13 0.13546734 93 andrew gelman stats-2010-06-17-My proposal for making college admissions fairer
14 0.1300914 1132 andrew gelman stats-2012-01-21-A counterfeit data graphic
15 0.12962504 606 andrew gelman stats-2011-03-10-It’s no fun being graded on a curve
16 0.12074935 388 andrew gelman stats-2010-11-01-The placebo effect in pharma
17 0.1163656 1678 andrew gelman stats-2013-01-17-Wanted: 365 stories of statistics
18 0.11595757 1517 andrew gelman stats-2012-10-01-“On Inspiring Students and Being Human”
19 0.11569393 236 andrew gelman stats-2010-08-26-Teaching yourself mathematics
20 0.11016263 326 andrew gelman stats-2010-10-07-Peer pressure, selection, and educational reform
topicId topicWeight
[(0, 0.181), (1, -0.074), (2, -0.026), (3, -0.064), (4, 0.053), (5, 0.04), (6, 0.024), (7, 0.126), (8, -0.018), (9, 0.027), (10, -0.018), (11, 0.052), (12, -0.058), (13, -0.032), (14, -0.042), (15, -0.061), (16, -0.013), (17, 0.169), (18, -0.051), (19, 0.059), (20, -0.042), (21, 0.027), (22, -0.006), (23, -0.094), (24, 0.11), (25, 0.034), (26, -0.009), (27, -0.002), (28, -0.048), (29, 0.066), (30, -0.031), (31, -0.059), (32, 0.071), (33, 0.068), (34, 0.033), (35, -0.013), (36, 0.005), (37, 0.0), (38, 0.031), (39, -0.033), (40, -0.043), (41, 0.104), (42, -0.005), (43, -0.007), (44, -0.012), (45, 0.023), (46, -0.01), (47, 0.046), (48, -0.027), (49, -0.067)]
simIndex simValue blogId blogTitle
same-blog 1 0.9633587 344 andrew gelman stats-2010-10-15-Story time
Introduction: This one belongs in the statistical lexicon. Kaiser Fung nails it : In reading [news] articles, we must look out for the moment(s) when the reporters announce story time. Much of the article is great propaganda for the statistics lobby, describing an attempt to use observational data to address a practical question, sort of a Freakonomics-style application. We have no problems when they say things like: “There is a substantial gap at year’s end between students whose teachers were in the top 10% in effectiveness and the bottom 10%. The fortunate students ranked 17 percentile points higher in English and 25 points higher in math.” Or this: “On average, Smith’s students slide under his instruction, losing 14 percentile points in math during the school year relative to their peers districtwide, The Times found. Overall, he ranked among the least effective of the district’s elementary school teachers.” Midway through the article (right before the section called “Study in contras
2 0.69902968 543 andrew gelman stats-2011-01-28-NYT shills for personal DNA tests
Introduction: Kaiser nails it . The offending article , by John Tierney, somehow ended up in the Science section rather than the Opinion section. As an opinion piece (or, for that matter, a blog), Tierney’s article would be nothing special. But I agree with Kaiser that it doesn’t work as a newspaper article. As Kaiser notes, this story involves a bunch of statistical and empirical claims that are not well resolved by P.R. and rhetoric.
3 0.69431055 542 andrew gelman stats-2011-01-28-Homework and treatment levels
Introduction: Interesting discussion here by Mark Palko on the difficulty of comparing charter schools to regular schools, even if the slots in the charter schools have been assigned by lottery. Beyond the direct importance of the topic, I found the discussion interesting because I always face a challenge in my own teaching to assign the right amount of homework, given that if I assign too much, students will simply rebel and not do it. To get back to the school-choice issue . . . Mark discussed selection effects: if a charter school is popular, it can require parents to sign a contract agreeing they will supervise their students to do lots of homework. Mark points out that there is a selection issue here, that the sort of parents who would sign that form are different from parents in general. But it seems to me there’s one more twist: These charter schools are popular, right? So that would imply that there is some reservoir of parents who would like to sign the form but don’t have the opp
4 0.68914258 1620 andrew gelman stats-2012-12-12-“Teaching effectiveness” as another dimension in cognitive ability
Introduction: I’m not a great teacher. I can get by because I work hard and I know a lot, and for some students my classes are just great, but it’s not a natural talent of mine. I know people who are amazing teachers, and they have something that I just don’t have. I wrote that book, Teaching Statistics: A Bag of Tricks (with Deb Nolan) because I’m not a good teacher and hence need to develop all sorts of techniques to be able to do what good teachers can do without even trying. I’m not proud of being mediocre at teaching. I don’t think that low teaching skill is some sort of indicator that I’m a great researcher. The other think about teaching ability is that I think it’s hard to detect without actually seeing someone teach a class. If you see me give a seminar presentation or even a guest lecture, you’d think I’m an awesome teacher. But, actually, no. I’m an excellent speaker, not such a great teacher. This all came to mind when I received the following email from anthropologist Hen
5 0.68639654 529 andrew gelman stats-2011-01-21-“City Opens Inquiry on Grading Practices at a Top-Scoring Bronx School”
Introduction: Sharon Otterman reports : When report card grades were released in the fall for the city’s 455 high schools, the highest score went to a small school in a down-and-out section of the Bronx . . . A stunning 94 percent of its seniors graduated, more than 30 points above the citywide average. . . . “When I interviewed for the school,” said Sam Buchbinder, a history teacher, “it was made very clear: this is a school that doesn’t believe in anyone failing.” That statement was not just an exhortation to excellence. It was school policy. By order of the principal, codified in the school’s teacher handbook, all teachers should grade their classes in the same way: 30 percent of students should earn a grade in the A range, 40 percent B’s, 25 percent C’s, and no more than 5 percent D’s. As long as they show up, they should not fail. Hey, that sounds like Harvard and Columbia^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H various selective northeastern colleges I’ve known. Of course, we^H^H
6 0.67702568 93 andrew gelman stats-2010-06-17-My proposal for making college admissions fairer
7 0.67701346 1803 andrew gelman stats-2013-04-14-Why girls do better in school
8 0.67271399 606 andrew gelman stats-2011-03-10-It’s no fun being graded on a curve
9 0.66346657 209 andrew gelman stats-2010-08-16-EdLab at Columbia’s Teachers’ College
10 0.65639818 1265 andrew gelman stats-2012-04-15-Progress in U.S. education; also, a discussion of what it takes to hit the op-ed pages
11 0.65218842 718 andrew gelman stats-2011-05-18-Should kids be able to bring their own lunches to school?
12 0.64897972 388 andrew gelman stats-2010-11-01-The placebo effect in pharma
13 0.63955611 992 andrew gelman stats-2011-11-05-Deadwood in the math curriculum
14 0.63776237 1612 andrew gelman stats-2012-12-08-The Case for More False Positives in Anti-doping Testing
16 0.62887639 1688 andrew gelman stats-2013-01-22-That claim that students whose parents pay for more of college get worse grades
17 0.62788165 2121 andrew gelman stats-2013-12-02-Should personal genetic testing be regulated? Battle of the blogroll
18 0.62580037 226 andrew gelman stats-2010-08-23-More on those L.A. Times estimates of teacher effectiveness
19 0.62528491 326 andrew gelman stats-2010-10-07-Peer pressure, selection, and educational reform
20 0.62489146 1001 andrew gelman stats-2011-11-10-Three hours in the life of a statistician
topicId topicWeight
[(9, 0.018), (10, 0.104), (15, 0.044), (16, 0.055), (17, 0.015), (21, 0.078), (24, 0.066), (40, 0.014), (43, 0.027), (44, 0.029), (53, 0.014), (63, 0.035), (82, 0.02), (86, 0.043), (99, 0.324)]
simIndex simValue blogId blogTitle
same-blog 1 0.96733922 344 andrew gelman stats-2010-10-15-Story time
Introduction: This one belongs in the statistical lexicon. Kaiser Fung nails it : In reading [news] articles, we must look out for the moment(s) when the reporters announce story time. Much of the article is great propaganda for the statistics lobby, describing an attempt to use observational data to address a practical question, sort of a Freakonomics-style application. We have no problems when they say things like: “There is a substantial gap at year’s end between students whose teachers were in the top 10% in effectiveness and the bottom 10%. The fortunate students ranked 17 percentile points higher in English and 25 points higher in math.” Or this: “On average, Smith’s students slide under his instruction, losing 14 percentile points in math during the school year relative to their peers districtwide, The Times found. Overall, he ranked among the least effective of the district’s elementary school teachers.” Midway through the article (right before the section called “Study in contras
2 0.95899135 2215 andrew gelman stats-2014-02-17-The Washington Post reprints university press releases without editing them
Introduction: Somebody points me to this horrifying exposé by Paul Raeburn on a new series by the Washington Post where they reprint press releases as if they are actual news. And the gimmick is, the reason why it’s appearing on this blog, is that these are university press releases on science stories . What could possibly go wrong there? After all, Steve Chaplin, a self-identified “science-writing PIO from an R1,” writes in a comment to Raeburn’s post: We write about peer-reviewed research accepted for publication or published by the world’s leading scientific journals after that research has been determined to be legitimate. Repeatability of new research is a publication requisite. I emphasized that last sentence myself because it was such a stunner. Do people really think that??? So I guess what he’s saying is, they don’t do press releases for articles from Psychological Science or the Journal of Personality and Social Psychology . But I wonder how the profs in the psych d
3 0.9424144 78 andrew gelman stats-2010-06-10-Hey, where’s my kickback?
Introduction: I keep hearing about textbook publishers who practically bribe instructors to assign their textbooks to students. And then I received this (unsolicited) email: You have recently been sent Pearson (Allyn & Bacon, Longman, Prentice Hall) texts to review for your summer and fall courses. As a thank you for reviewing our texts, I would like to invite you to participate in a brief survey (attached). If you have any questions about the survey, are not sure which books you have been sent, or if you would like to receive instructor’s materials, desk copies, etc. please let me know! If you have recently received your course assignments – let me know as well . Additionally, if you have decided to use a Pearson book in your summer or fall courses, I will provide you with an ISBN that will include discounts and resources for your students at no extra cost! All you have to do is answer the 3 simple questions on the attached survey and you will receive a $10.00 Dunkin Donuts gift card.
4 0.94092959 2257 andrew gelman stats-2014-03-20-The candy weighing demonstration, or, the unwisdom of crowds
Introduction: From 2008: The candy weighing demonstration, or, the unwisdom of crowds My favorite statistics demonstration is the one with the bag of candies. I’ve elaborated upon it since including it in the Teaching Statistics book and I thought these tips might be useful to some of you. Preparation Buy 100 candies of different sizes and shapes and put them in a bag (the plastic bag from the store is fine). Get something like 20 large full-sized candy bars, 20 or 30 little things like mini Snickers bars and mini Peppermint Patties. And then 50 or 60 really little things like tiny Tootsie Rolls, lollipops, and individually-wrapped Life Savers. Count and make sure it’s exactly 100. You also need a digital kitchen scale that reads out in grams. Also bring a sealed envelope inside of which is a note (details below). When you get into the room, unobtrusively put the note somewhere, for example between two books on a shelf or behind a window shade. Setup Hold up the back of cand
5 0.94053525 1122 andrew gelman stats-2012-01-16-“Groundbreaking or Definitive? Journals Need to Pick One”
Introduction: Sanjay Srivastava writes : As long as a journal pursues a strategy of publishing “wow” studies, it will inevitably contain more unreplicable findings and unsupportable conclusions than equally rigorous but more “boring” journals. Groundbreaking will always be higher-risk. And definitive will be the territory of journals that publish meta-analyses and reviews. . . . Most conclusions, even those in peer-reviewed papers in rigorous journals, should be regarded as tentative at best; but press releases and other public communication rarely convey that. . . . His message to all of us: Our standard response to a paper in Science, Nature, or Psychological Science should be “wow, that’ll be really interesting if it replicates.” And in our teaching and our engagement with the press and public, we need to make clear why that is the most enthusiastic response we can justify.
6 0.93954867 1859 andrew gelman stats-2013-05-16-How do we choose our default methods?
7 0.93810529 487 andrew gelman stats-2010-12-27-Alfred Kahn
9 0.93456006 1912 andrew gelman stats-2013-06-24-Bayesian quality control?
10 0.93318236 1864 andrew gelman stats-2013-05-20-Evaluating Columbia University’s Frontiers of Science course
11 0.93180817 692 andrew gelman stats-2011-05-03-“Rationality” reinforces, does not compete with, other models of behavior
12 0.92984092 2361 andrew gelman stats-2014-06-06-Hurricanes vs. Himmicanes
13 0.9296782 2301 andrew gelman stats-2014-04-22-Ticket to Baaaaarf
14 0.92955315 2289 andrew gelman stats-2014-04-11-“More research from the lunatic fringe”
15 0.92936802 2158 andrew gelman stats-2014-01-03-Booze: Been There. Done That.
16 0.92845249 578 andrew gelman stats-2011-02-17-Credentialism, elite employment, and career aspirations
17 0.92840546 2173 andrew gelman stats-2014-01-15-Postdoc involving pathbreaking work in MRP, Stan, and the 2014 election!
18 0.92826146 2151 andrew gelman stats-2013-12-27-Should statistics have a Nobel prize?
19 0.92820644 430 andrew gelman stats-2010-11-25-The von Neumann paradox
20 0.92785507 2368 andrew gelman stats-2014-06-11-Bayes in the research conversation