andrew_gelman_stats andrew_gelman_stats-2011 andrew_gelman_stats-2011-609 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: I followed this link from Chris Blattman to an article by economist Roland Fryer, who writes: I [Fryer] find no evidence that teacher incentives increase student performance, attendance, or graduation, nor do I find any evidence that the incentives change student or teacher behavior. What struck me were not the findings (which, as Fryer notes in his article, are plausible enough) but the use of the word “I” rather than “we.” A field experiment is a big deal, and I was surprised to read that Fryer did it all by himself! Here’s the note of acknowledgments (on the first page of the article): This project would not have been possible without the leadership and support of Joel Klein. I am also grateful to Jennifer Bell-Ellwanger, Joanna Cannon, and Dominique West for their cooperation in collecting the data necessary for this project, and to my colleagues Edward Glaeser, Richard Holden, and Lawrence Katz for helpful comments and discussions. Vilsa E. Curto, Meghan L. Howard,
sentIndex sentText sentNum sentScore
1 I followed this link from Chris Blattman to an article by economist Roland Fryer, who writes: I [Fryer] find no evidence that teacher incentives increase student performance, attendance, or graduation, nor do I find any evidence that the incentives change student or teacher behavior. [sent-1, score-0.769]
2 What struck me were not the findings (which, as Fryer notes in his article, are plausible enough) but the use of the word “I” rather than “we. [sent-2, score-0.082]
3 ” A field experiment is a big deal, and I was surprised to read that Fryer did it all by himself! [sent-3, score-0.149]
4 Here’s the note of acknowledgments (on the first page of the article): This project would not have been possible without the leadership and support of Joel Klein. [sent-4, score-0.372]
5 I am also grateful to Jennifer Bell-Ellwanger, Joanna Cannon, and Dominique West for their cooperation in collecting the data necessary for this project, and to my colleagues Edward Glaeser, Richard Holden, and Lawrence Katz for helpful comments and discussions. [sent-5, score-0.227]
6 Howard, Won Hee Park, Jörg Spenkuch, David Toniatti, Rucha Vankudre, and Martha Woerner provided excellent research assistance. [sent-8, score-0.082]
7 I’m surprised that none of the other people ended up as coauthors on the paper. [sent-10, score-0.227]
8 But I guess it makes sense: My colleagues and I will write a paper based on survey data without involving the data collectors as coauthors, so why not do this with experimental data too? [sent-11, score-0.145]
9 I guess I just find field experiments so intimidating that I can’t imagine writing an applied paper on the topic without a lot of serious collaboration. [sent-12, score-0.345]
10 ) Perhaps the implicit rules about coauthorship are different in economics than in political science. [sent-14, score-0.081]
11 On page 1, it says: Despite these reforms to increase achievement, Figure 1 demonstrates that test scores have been largely constant over the past thirty years. [sent-18, score-0.687]
12 Here’s Figure 1: Once you get around the confusingly-labeled lines and the mass of white space on the top and bottom of each graph, you see that math scores have improved a lot! [sent-19, score-0.428]
13 Since 1978, fourth-grade math scores have gone up so much that they’re halfway to where eighth grade scores were in 1978. [sent-20, score-1.222]
14 Eighth grade scores also have increased substantially, and twelfth-grade scores have gone up too (although not by as much). [sent-21, score-0.877]
15 Perhaps Fryer just forgot to add the word “reading” in the sentence above. [sent-23, score-0.151]
16 I only wish that he’d presented the rest of his results graphically. [sent-25, score-0.083]
17 Even a sloppy graph is a lot easier for me to follow than a table full of numbers presented to three decimal places. [sent-26, score-0.336]
18 I know Fryer can do better; his previous papers had excellent graphs (see here and here ). [sent-27, score-0.082]
wordName wordTfidf (topN-words)
[('fryer', 0.593), ('scores', 0.339), ('eighth', 0.18), ('joel', 0.156), ('coauthors', 0.148), ('grade', 0.106), ('teacher', 0.104), ('incentives', 0.104), ('figure', 0.104), ('gone', 0.093), ('acknowledgments', 0.09), ('martha', 0.09), ('math', 0.089), ('klein', 0.085), ('glaeser', 0.085), ('roland', 0.085), ('presented', 0.083), ('word', 0.082), ('excellent', 0.082), ('coauthorship', 0.081), ('intimidating', 0.081), ('surprised', 0.079), ('katz', 0.078), ('cooperation', 0.078), ('increase', 0.077), ('project', 0.077), ('reforms', 0.076), ('halfway', 0.076), ('colleagues', 0.075), ('grateful', 0.074), ('student', 0.073), ('decimal', 0.072), ('blattman', 0.071), ('field', 0.07), ('without', 0.07), ('forgot', 0.069), ('leadership', 0.068), ('attendance', 0.068), ('page', 0.067), ('thirty', 0.066), ('find', 0.065), ('lawrence', 0.065), ('edward', 0.065), ('graph', 0.062), ('demonstrates', 0.062), ('howard', 0.061), ('achievement', 0.061), ('state', 0.06), ('sloppy', 0.06), ('lot', 0.059)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 609 andrew gelman stats-2011-03-13-Coauthorship norms
Introduction: I followed this link from Chris Blattman to an article by economist Roland Fryer, who writes: I [Fryer] find no evidence that teacher incentives increase student performance, attendance, or graduation, nor do I find any evidence that the incentives change student or teacher behavior. What struck me were not the findings (which, as Fryer notes in his article, are plausible enough) but the use of the word “I” rather than “we.” A field experiment is a big deal, and I was surprised to read that Fryer did it all by himself! Here’s the note of acknowledgments (on the first page of the article): This project would not have been possible without the leadership and support of Joel Klein. I am also grateful to Jennifer Bell-Ellwanger, Joanna Cannon, and Dominique West for their cooperation in collecting the data necessary for this project, and to my colleagues Edward Glaeser, Richard Holden, and Lawrence Katz for helpful comments and discussions. Vilsa E. Curto, Meghan L. Howard,
2 0.16695452 1917 andrew gelman stats-2013-06-28-Econ coauthorship update
Introduction: The other day I posted some remarks on Stan Liebowitz’s analysis of coauthorship in economics. Liebowitz followed up with some more thoughts: I [Liebowitz] am not arguing for an increase or decrease in coauthorship, per se. I would prefer an efficient amount of coauthorship, whatever that is, and certainly it will vary by paper and by field. If you feel you are more productive with many coauthors, that is not in contrast to anything in my paper. My point is that you will pick the correct number of coauthors if you and your coauthors are given 1/n credit (assuming you believe each author contributed equally). If, however, all of the coauthors are given full credit for the paper (and I have evidence that, in economics at least, authors are far more likely to receive full credit than 1/n credit), authors will get credit for more papers if they use more coauthors than would otherwise be best for total research productivity. My criticism is in the inefficiency induced by not using 1/n
3 0.15252765 226 andrew gelman stats-2010-08-23-More on those L.A. Times estimates of teacher effectiveness
Introduction: In discussing the ongoing Los Angeles Times series on teacher effectiveness, Alex Tabarrok and I both were impressed that the newspaper was reporting results on individual teachers, moving beyond the general research findings (“teachers matter,” “KIPP really works, but it requires several extra hours in the school day,” and so forth) that we usually see from value-added analyses in education. My first reaction was that the L.A. Times could get away with this because, unlike academic researchers, they can do whatever they want as long as they don’t break the law. They don’t have to answer to an Institutional Review Board. (By referring to this study by its publication outlet rather than its authors, I’m violating my usual rule (see the last paragraph here ). In this case, I think it’s ok to refer to the “L.A. Times study” because what’s notable is not the analysis (thorough as it may be) but how it is being reported.) Here I’d like to highlight a few other things came up in our
4 0.14642486 1093 andrew gelman stats-2011-12-30-Strings Attached: Untangling the Ethics of Incentives
Introduction: Chris Paulse points me to this book by Ruth Grant: Incentives can be found everywhere–in schools, businesses, factories, and government–influencing people’s choices about almost everything, from financial decisions and tobacco use to exercise and child rearing. So long as people have a choice, incentives seem innocuous. But Strings Attached demonstrates that when incentives are viewed as a kind of power rather than as a form of exchange, many ethical questions arise: How do incentives affect character and institutional culture? Can incentives be manipulative or exploitative, even if people are free to refuse them? What are the responsibilities of the powerful in using incentives? Ruth Grant shows that, like all other forms of power, incentives can be subject to abuse, and she identifies their legitimate and illegitimate uses. Grant offers a history of the growth of incentives in early twentieth-century America, identifies standards for judging incentives, and examines incentives
5 0.13242529 1980 andrew gelman stats-2013-08-13-Test scores and grades predict job performance (but maybe not at Google)
Introduction: Eric Loken writes : If you’re used to Google upending conventional wisdom, then yesterday’s interview with Laszlo Bock in the New York Times did not disappoint. Google has determined that test scores and transcripts are useless because they don’t predict performance among its employees. . . . I [Loken] am going to assume they’re well aware of the limits of their claim, and instead I’m going say that as readers of the interview we should not lose sight of a fundamental fact - Across a wide variety of employment settings, one of the most robust findings in organizational psychology is that tests of cognitive ability are strong predictors of job performance. If Google has found otherwise, what they have found is that grades and test scores are not predictive of performance at Google. In general, in the workplace tests are still highly predictive of success. If all the research says that test scores and grades predict performance, why would the people at Google want to igno
6 0.12203711 32 andrew gelman stats-2010-05-14-Causal inference in economics
7 0.1210396 1803 andrew gelman stats-2013-04-14-Why girls do better in school
8 0.1209856 315 andrew gelman stats-2010-10-03-He doesn’t trust the fit . . . r=.999
10 0.1085386 1350 andrew gelman stats-2012-05-28-Value-added assessment: What went wrong?
11 0.097962245 2176 andrew gelman stats-2014-01-19-Transformations for non-normal data
12 0.09669669 606 andrew gelman stats-2011-03-10-It’s no fun being graded on a curve
13 0.091792621 2083 andrew gelman stats-2013-10-31-Value-added modeling in education: Gaming the system by sending kids on a field trip at test time
14 0.088407412 95 andrew gelman stats-2010-06-17-“Rewarding Strivers: Helping Low-Income Students Succeed in College”
15 0.087334685 2172 andrew gelman stats-2014-01-14-Advice on writing research articles
16 0.086693957 326 andrew gelman stats-2010-10-07-Peer pressure, selection, and educational reform
17 0.085949391 61 andrew gelman stats-2010-05-31-A data visualization manifesto
18 0.085725844 1688 andrew gelman stats-2013-01-22-That claim that students whose parents pay for more of college get worse grades
20 0.079071738 957 andrew gelman stats-2011-10-14-Questions about a study of charter schools
topicId topicWeight
[(0, 0.166), (1, -0.067), (2, -0.0), (3, -0.015), (4, 0.056), (5, -0.007), (6, 0.002), (7, 0.048), (8, -0.016), (9, 0.016), (10, 0.033), (11, 0.016), (12, -0.036), (13, -0.016), (14, 0.062), (15, 0.009), (16, 0.052), (17, 0.026), (18, -0.028), (19, 0.003), (20, -0.019), (21, 0.023), (22, -0.013), (23, -0.026), (24, 0.056), (25, -0.019), (26, 0.015), (27, 0.039), (28, 0.01), (29, 0.042), (30, -0.005), (31, 0.008), (32, 0.032), (33, -0.045), (34, -0.023), (35, 0.03), (36, 0.037), (37, -0.035), (38, 0.044), (39, -0.006), (40, 0.041), (41, 0.016), (42, 0.024), (43, 0.012), (44, -0.027), (45, -0.004), (46, -0.003), (47, -0.01), (48, 0.015), (49, 0.026)]
simIndex simValue blogId blogTitle
same-blog 1 0.95689142 609 andrew gelman stats-2011-03-13-Coauthorship norms
Introduction: I followed this link from Chris Blattman to an article by economist Roland Fryer, who writes: I [Fryer] find no evidence that teacher incentives increase student performance, attendance, or graduation, nor do I find any evidence that the incentives change student or teacher behavior. What struck me were not the findings (which, as Fryer notes in his article, are plausible enough) but the use of the word “I” rather than “we.” A field experiment is a big deal, and I was surprised to read that Fryer did it all by himself! Here’s the note of acknowledgments (on the first page of the article): This project would not have been possible without the leadership and support of Joel Klein. I am also grateful to Jennifer Bell-Ellwanger, Joanna Cannon, and Dominique West for their cooperation in collecting the data necessary for this project, and to my colleagues Edward Glaeser, Richard Holden, and Lawrence Katz for helpful comments and discussions. Vilsa E. Curto, Meghan L. Howard,
2 0.77038991 1688 andrew gelman stats-2013-01-22-That claim that students whose parents pay for more of college get worse grades
Introduction: Theodore Vasiloudis writes: I came upon this article by Laura Hamilton, an assistant professor in the University of California at Merced, that claims that “The more money that parents provide for higher education, the lower the grades their children earn.” I can’t help but feel that there something wrong with the basis of the study or a confounding factor causing this apparent correlation, and since you often comment on studies on your blog I thought you might find this study interesting. My reply: I have to admit that the description above made me suspicious of the study before I even looked at it. On first thought, I’d expect the effect of parent’s financial contributions to be positive (as they free the student from the need to get a job during college), but not negative. Hamilton argues that “parental investments create a disincentive for student achievement,” which may be—but I’m generally suspicious of arguments in which the rebound is bigger than the main effect.
Introduction: Howard Wainer writes : When we focus only on the differences between groups, we too easily lose track of the big picture. Nowhere is this more obvious than in the current public discussions of the size of the gap in test scores that is observed between racial groups. It has been noted that in New Jersey the gap between the average scores of white and black students on the well-developed scale of the National Assessment of Educational Progress (NAEP) has shrunk by only about 25 percent over the past two decades. The conclusion drawn was that even though the change is in the right direction, it is far too slow. But focusing on the difference blinds us to what has been a remarkable success in education over the past 20 years. Although the direction and size of student improvements are considered across many subject areas and many age groups, I will describe just one — 4th grade mathematics. . . . there have been steep gains for both racial groups over this period (somewhat steeper g
4 0.75349778 2076 andrew gelman stats-2013-10-24-Chasing the noise: W. Edwards Deming would be spinning in his grave
Introduction: Amy Cohen writes: A surgeon showed me the “report card” his hospital received about his surgical group. The figure below shows what the report card looks like. I am very curious to hear what you think about the “deciles of the odds ratio” approach to evaluate and rank hospitals used by the American College of Surgeons. I replied: I don’t know enough about the substantive context to say too much here. But, speaking generally, I am skeptical of those very high values for some of those odds ratios. I suspect that much of what is plotted in this graph is just noise. But I respect the goals underlying the plot, and I like that it displays a lot of information efficiently. And then she wrote: My concern with the decile approach to identifying opportunities for improvement is that it violates a principle of quality improvement: chasing noise is fruitless or worse, because it leads to tampering with a system in a way that can make it worse. In the paper, the authors admit th
5 0.74234056 95 andrew gelman stats-2010-06-17-“Rewarding Strivers: Helping Low-Income Students Succeed in College”
Introduction: Several years ago, I heard about a project at the Educational Testing Service to identify “strivers”: students from disadvantaged backgrounds who did unexpectedly well on the SAT (the college admissions exam formerly known as the “Scholastic Aptitude Test” but apparently now just “the SAT,” in the same way that Exxon is just “Exxon” and that Harry Truman’s middle name is just “S”), at least 200 points above a predicted score based on demographic and neighborhood information. My ETS colleague and I agreed that this was a silly idea: From a statistical point of view, if student A is expected ahead of time to do better than student B, and then they get identical test scores, then you’d expect student A (the non-”striver”) to do better than student B (the “striver”) later on. Just basic statistics: if a student does much better than expected, then probably some of that improvement is noise. The idea of identifying these “strivers” seemed misguided and not the best use of the SAT.
6 0.73757285 1350 andrew gelman stats-2012-05-28-Value-added assessment: What went wrong?
8 0.71755826 1620 andrew gelman stats-2012-12-12-“Teaching effectiveness” as another dimension in cognitive ability
9 0.71142268 226 andrew gelman stats-2010-08-23-More on those L.A. Times estimates of teacher effectiveness
10 0.7092796 1803 andrew gelman stats-2013-04-14-Why girls do better in school
11 0.70836675 452 andrew gelman stats-2010-12-06-Followup questions
12 0.69714767 825 andrew gelman stats-2011-07-27-Grade inflation: why weren’t the instructors all giving all A’s already??
13 0.68823582 617 andrew gelman stats-2011-03-17-“Why Preschool Shouldn’t Be Like School”?
14 0.68621773 326 andrew gelman stats-2010-10-07-Peer pressure, selection, and educational reform
16 0.68228197 93 andrew gelman stats-2010-06-17-My proposal for making college admissions fairer
17 0.68175334 276 andrew gelman stats-2010-09-14-Don’t look at just one poll number–unless you really know what you’re doing!
18 0.67921823 542 andrew gelman stats-2011-01-28-Homework and treatment levels
19 0.67871785 561 andrew gelman stats-2011-02-06-Poverty, educational performance – and can be done about it
20 0.67506331 296 andrew gelman stats-2010-09-26-A simple semigraphic display
topicId topicWeight
[(16, 0.429), (18, 0.011), (21, 0.019), (24, 0.116), (59, 0.012), (66, 0.013), (86, 0.017), (95, 0.017), (99, 0.237)]
simIndex simValue blogId blogTitle
1 0.99248862 1366 andrew gelman stats-2012-06-05-How do segregation measures change when you change the level of aggregation?
Introduction: In a discussion of workplace segregation, Philip Cohen posts some graphs that led me to a statistical question. I’ll pose my question below, but first the graphs: In a world of zero segregation of jobs by sex, the top graph above would have a spike at 50% (or, whatever the actual percentage is of women in the labor force) and, in the bottom graph, the pink and blue lines would be in the same place and would look like very steep S curves. The difference between the pink and blue lines represents segregation by job. One thing I wonder is how these graphs would change if we redefine occupation. (For example, is my occupation “mathematical scientist,” “statistician,” “teacher,” “university professor,” “statistics professor,” or “tenured statistics professor”?) Finer or coarser classification would give different results, and I wonder how this would work. This is not at all meant as a criticism of Cohen’s claims, it’s just a statistical question. I’m guessing that
2 0.98805481 1279 andrew gelman stats-2012-04-24-ESPN is looking to hire a research analyst
Introduction: This is somebody’s dream job, I’m sure . . . ESPN is looking for a statistician to join the HR department as a Research Analyst . The job will consist of analytical research and producing statistics about the people that work at ESPN. Topics of interest will include productivity, efficiency, and retention of employees, among other items. In addition to data mining and producing reports, we also field surveys and analyze results. The position is located at the headquarters in Bristol, Connecticut, the same campus where nearly all ESPN shows are produced. ESPN is a Disney company, so discounts and free admission to Disney parks are available for employees. Flexible work arrangements are available, along with working in the New York City office part-time if desired. The role is a relatively new function and will have a high impact very quickly on helping the business function. Statistical software, text books, and any other resource needed to get the job done will be provided. T
3 0.98710531 572 andrew gelman stats-2011-02-14-Desecration of valuable real estate
Introduction: Malecki asks: Is this the worst infographic ever to appear in NYT? USA Today is not something to aspire to. To connect to some of our recent themes , I agree this is a pretty horrible data display. But it’s not bad as a series of images. Considering the competition to be a cartoon or series of photos, these images aren’t so bad. One issue, I think, is that designers get credit for creativity and originality (unusual color combinations! Histogram bars shaped like mosques!) , which is often the opposite of what we want in a clear graph. It’s Martin Amis vs. George Orwell all over again.
4 0.9775607 1115 andrew gelman stats-2012-01-12-Where are the larger-than-life athletes?
Introduction: Jonathan Cantor points to this poll estimating rifle-armed QB Tim Tebow as America’s favorite pro athlete: In an ESPN survey of 1,502 Americans age 12 or older, three percent identified Tebow as their favorite professional athlete. Tebow finished in front of Kobe Bryant (2 percent), Aaron Rodgers (1.9 percent), Peyton Manning (1.8 percent), and Tom Brady (1.5 percent). Amusing. What this survey says to me is that there are no super-popular athletes who are active in America today. Which actually sounds about right. No Tiger Woods, no Magic Johnson, Muhammed Ali, John Elway, Pete Rose, Billie Jean King, etc etc. Tebow is an amusing choice, people might as well pick him now while he’s still on top. As a sports celeb, he’s like Bill Lee or the Refrigerator: colorful and a solid pro athlete, but no superstar. When you think about all the colorful superstar athletes of times gone by, it’s perhaps surprising that there’s nobody out there right now to play the role. I supp
5 0.97006875 1487 andrew gelman stats-2012-09-08-Animated drought maps
Introduction: Aleks sends along this dynamic graphic from Mike Bostock: I’m not so happy with the arrangement of years by decade—the lineup of all years ending in 0, or 1, or 2, etc., seems a bit of a distraction—but in many ways the display is impressive. And, as often is the case with such graphs, once it’s out there, other people can do similar things and make their own improvements.
6 0.96814913 1180 andrew gelman stats-2012-02-22-I’m officially no longer a “rogue”
7 0.96788424 528 andrew gelman stats-2011-01-21-Elevator shame is a two-way street
8 0.96770197 1014 andrew gelman stats-2011-11-16-Visualizations of NYPD stop-and-frisk data
9 0.96487629 398 andrew gelman stats-2010-11-06-Quote of the day
10 0.96429634 1304 andrew gelman stats-2012-05-06-Picking on Stephen Wolfram
11 0.96427011 1659 andrew gelman stats-2013-01-07-Some silly things you (didn’t) miss by not reading the sister blog
12 0.96291345 1330 andrew gelman stats-2012-05-19-Cross-validation to check missing-data imputation
13 0.95867825 1598 andrew gelman stats-2012-11-30-A graphics talk with no visuals!
14 0.95833242 1025 andrew gelman stats-2011-11-24-Always check your evidence
15 0.9529686 445 andrew gelman stats-2010-12-03-Getting a job in pro sports… as a statistician
16 0.95242596 700 andrew gelman stats-2011-05-06-Suspicious pattern of too-strong replications of medical research
17 0.94165576 1156 andrew gelman stats-2012-02-06-Bayesian model-building by pure thought: Some principles and examples
18 0.93863362 2 andrew gelman stats-2010-04-23-Modeling heterogenous treatment effects
same-blog 19 0.93401724 609 andrew gelman stats-2011-03-13-Coauthorship norms
20 0.93168783 177 andrew gelman stats-2010-08-02-Reintegrating rebels into civilian life: Quasi-experimental evidence from Burundi