andrew_gelman_stats andrew_gelman_stats-2013 andrew_gelman_stats-2013-1914 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Economist Stan Liebowitz has a longstanding interest in the difficulties of flagging published research errors. Recently he wrote on the related topic of dishonest authorship: While not about direct research fraud, I thought you might be interested in this paper . It discusses the manner in which credit is given for economics articles, and I suspect it applies to many other areas as well. One of the conclusions is that the lack of complete proration per author will lead to excessive coauthorship, reducing overall research output by inducing the use of larger than efficient-sized teams. Under these circumstances, false authorship can be a response to the warped reward system and false authorship might improve research efficiency since it might keep actual research teams (as opposed to nominal teams) from being too large to produce research efficiently. One of the questions I rhetorically ask in the paper is whether anyone has ever been ‘punished’ for having their name included on
sentIndex sentText sentNum sentScore
1 Economist Stan Liebowitz has a longstanding interest in the difficulties of flagging published research errors. [sent-1, score-0.338]
2 Recently he wrote on the related topic of dishonest authorship: While not about direct research fraud, I thought you might be interested in this paper . [sent-2, score-0.416]
3 One of the conclusions is that the lack of complete proration per author will lead to excessive coauthorship, reducing overall research output by inducing the use of larger than efficient-sized teams. [sent-4, score-0.477]
4 Under these circumstances, false authorship can be a response to the warped reward system and false authorship might improve research efficiency since it might keep actual research teams (as opposed to nominal teams) from being too large to produce research efficiently. [sent-5, score-1.559]
5 One of the questions I rhetorically ask in the paper is whether anyone has ever been ‘punished’ for having their name included on a paper for which they did not perform their share of the work (I think the answer is “no” in economics). [sent-6, score-0.269]
6 Much of my most successful research has been collaborative (with varying percentages of contributions from the collaborations). [sent-9, score-0.262]
7 Many times I’ve added a collaborator after most of the work had already been done, just because I think it makes the work stronger. [sent-10, score-0.172]
8 Liebowitz argues that excessive collaboration can harm research, if people would get more done working on their own. [sent-14, score-0.29]
9 I have a different perspective in that I think even a small collaboration by a coauthor can make an article or book much stronger. [sent-15, score-0.378]
10 Given that this seems to hold in statistics, where we publish dozens of papers a year, I’d think it would be even more the case in economics, where researchers take years to publish a single article. [sent-16, score-0.196]
11 True, collaboration did not save economists Carmen Reinhart and Kenneth Rogoff from making three serious errors in a single article , but—no joke—I think that adding a third collaborator might have made a difference and allowed them to have avoided their errors. [sent-17, score-0.547]
12 Either Reinhart and Rogoff had a research assistant do their calculations or they did it on their own. [sent-19, score-0.304]
13 They might have been more open during the research process. [sent-22, score-0.285]
14 Having a coauthor with real responsibility could’ve made all the difference. [sent-28, score-0.272]
15 Liebowitz also writes: As a measure of research success for senior faculty, department chairs, on average, rely most heavily on the journal of publication, with a weight of 40%. [sent-29, score-0.361]
16 Citations, by way of contrast, were in third place, receiving a weight of 26% . [sent-30, score-0.161]
17 This is analogous to a sports team picking its starting lineup based on the ex ante performance of players on athletic tests intended to predict success on the field, instead of using the ex post actual performance in games. [sent-33, score-0.828]
18 This is an issue too, highly relevant to the question of incentives for untenured academic researchers, but to me somewhat distant from the larger concerns about research quality. [sent-34, score-0.34]
19 Don’t get me wrong—I don’t think coauthorship is a panacea for the problems of sloppy research. [sent-35, score-0.346]
20 But for an important project, I see real advantages to having multiple people taking responsibility for the result. [sent-36, score-0.107]
wordName wordTfidf (topN-words)
[('reinhart', 0.355), ('liebowitz', 0.342), ('rogoff', 0.316), ('research', 0.201), ('authorship', 0.182), ('coauthor', 0.165), ('ra', 0.157), ('collaboration', 0.153), ('ex', 0.142), ('coauthorship', 0.142), ('excessive', 0.137), ('collaborator', 0.112), ('responsibility', 0.107), ('assistant', 0.103), ('teams', 0.097), ('economics', 0.095), ('weight', 0.086), ('might', 0.084), ('untenured', 0.079), ('rhetorically', 0.079), ('inducing', 0.079), ('carmen', 0.079), ('panacea', 0.079), ('third', 0.075), ('success', 0.074), ('flagging', 0.074), ('lineup', 0.074), ('collaborations', 0.074), ('false', 0.072), ('performance', 0.071), ('punished', 0.068), ('ante', 0.068), ('chairs', 0.068), ('publish', 0.068), ('ve', 0.067), ('dishonest', 0.066), ('paper', 0.065), ('problems', 0.065), ('avoided', 0.063), ('longstanding', 0.063), ('actual', 0.063), ('athletic', 0.062), ('collaborative', 0.061), ('analogous', 0.061), ('nominal', 0.061), ('larger', 0.06), ('think', 0.06), ('afford', 0.06), ('kenneth', 0.06), ('reward', 0.059)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 1914 andrew gelman stats-2013-06-25-Is there too much coauthorship in economics (and science more generally)? Or too little?
Introduction: Economist Stan Liebowitz has a longstanding interest in the difficulties of flagging published research errors. Recently he wrote on the related topic of dishonest authorship: While not about direct research fraud, I thought you might be interested in this paper . It discusses the manner in which credit is given for economics articles, and I suspect it applies to many other areas as well. One of the conclusions is that the lack of complete proration per author will lead to excessive coauthorship, reducing overall research output by inducing the use of larger than efficient-sized teams. Under these circumstances, false authorship can be a response to the warped reward system and false authorship might improve research efficiency since it might keep actual research teams (as opposed to nominal teams) from being too large to produce research efficiently. One of the questions I rhetorically ask in the paper is whether anyone has ever been ‘punished’ for having their name included on
Introduction: Jeff Ratto points me to this news article by Dean Baker reporting the work of three economists, Thomas Herndon, Michael Ash, and Robert Pollin, who found errors in a much-cited article by Carmen Reinhart and Kenneth Rogoff analyzing historical statistics of economic growth and public debt. Mike Konczal provides a clear summary; that’s where I got the above image. Errors in data processing and data analysis It turns out that Reinhart and Rogoff flubbed it. Herndon et al. write of “spreadsheet errors, omission of available data, weighting, and transcription.” The spreadsheet errors are the most embarrassing, but the other choices in data analysis seem pretty bad too. It can be tough to work with small datasets, so I have sympathy for Reinhart and Rogoff, but it does look like they were jumping to conclusions in their paper. Perhaps the urgency of the topic moved them to publish as fast as possible rather than carefully considering the impact of their data-analytic choi
3 0.28718162 1917 andrew gelman stats-2013-06-28-Econ coauthorship update
Introduction: The other day I posted some remarks on Stan Liebowitz’s analysis of coauthorship in economics. Liebowitz followed up with some more thoughts: I [Liebowitz] am not arguing for an increase or decrease in coauthorship, per se. I would prefer an efficient amount of coauthorship, whatever that is, and certainly it will vary by paper and by field. If you feel you are more productive with many coauthors, that is not in contrast to anything in my paper. My point is that you will pick the correct number of coauthors if you and your coauthors are given 1/n credit (assuming you believe each author contributed equally). If, however, all of the coauthors are given full credit for the paper (and I have evidence that, in economics at least, authors are far more likely to receive full credit than 1/n credit), authors will get credit for more papers if they use more coauthors than would otherwise be best for total research productivity. My criticism is in the inefficiency induced by not using 1/n
4 0.18507037 1835 andrew gelman stats-2013-05-02-7 ways to separate errors from statistics
Introduction: Betsey Stevenson and Justin Wolfers have been inspired by the recent Reinhardt and Rogoff debacle to list “six ways to separate lies from statistics” in economics research: 1. “Focus on how robust a finding is, meaning that different ways of looking at the evidence point to the same conclusion.” 2. Don’t confuse statistical with practical significance. 3. “Be wary of scholars using high-powered statistical techniques as a bludgeon to silence critics who are not specialists.” 4. “Don’t fall into the trap of thinking about an empirical finding as ‘right’ or ‘wrong.’ At best, data provide an imperfect guide.” 5. “Don’t mistake correlation for causation.” 6. “Always ask ‘so what?’” I like all these points, especially #4, which I think doesn’t get said enough. As I wrote a few months ago, high-profile social science research aims for proof, not for understanding—and that’s a problem. My addition to the list If you compare my title above to that of Stevenson
5 0.16172007 1435 andrew gelman stats-2012-07-30-Retracted articles and unethical behavior in economics journals?
Introduction: Stan Liebowitz writes: Have you ever heard of an article being retracted in economics? I know you have only been doing this for a few years but I suspect that the answer is that none or very few are retracted. No economist would ever deceive another. There is virtually no interest in detecting cheating. And what good would that do if there is no form of punishment? I say this because I think I have found a case in one of our top journals but the editor allowed the authors of the original article to write an anonymous referee report defending themselves and used this report to reject my comment even though an independent referee recommended publication. My reply: I wonder how this sort of thing will change in the future as journals become less important. My impression is that, on one side, researchers are increasingly citing NBER reports, Arxiv preprints, and the like; while, from the other direction, journals such as Science and Nature are developing the reputations of being “t
6 0.1446027 409 andrew gelman stats-2010-11-11-“Tiny,” “Large,” “Very,” “Nice,” “Dumbest”
7 0.12473249 1878 andrew gelman stats-2013-05-31-How to fix the tabloids? Toward replicable social science research
9 0.12050187 1844 andrew gelman stats-2013-05-06-Against optimism about social science
10 0.11084355 2245 andrew gelman stats-2014-03-12-More on publishing in journals
12 0.10746059 982 andrew gelman stats-2011-10-30-“There’s at least as much as an 80 percent chance . . .”
13 0.10273872 1252 andrew gelman stats-2012-04-08-Jagdish Bhagwati’s definition of feminist sincerity
14 0.10082079 1139 andrew gelman stats-2012-01-26-Suggested resolution of the Bem paradox
16 0.09204495 2244 andrew gelman stats-2014-03-11-What if I were to stop publishing in journals?
19 0.091530241 901 andrew gelman stats-2011-09-12-Some thoughts on academic cheating, inspired by Frey, Wegman, Fischer, Hauser, Stapel
20 0.09062393 120 andrew gelman stats-2010-06-30-You can’t put Pandora back in the box
topicId topicWeight
[(0, 0.212), (1, -0.069), (2, -0.045), (3, -0.077), (4, -0.01), (5, -0.0), (6, 0.001), (7, -0.05), (8, -0.047), (9, 0.007), (10, 0.008), (11, -0.013), (12, -0.039), (13, -0.032), (14, -0.022), (15, -0.004), (16, 0.027), (17, 0.005), (18, 0.017), (19, 0.008), (20, 0.009), (21, 0.037), (22, 0.02), (23, 0.027), (24, -0.047), (25, -0.024), (26, 0.031), (27, -0.019), (28, -0.009), (29, -0.003), (30, 0.043), (31, 0.0), (32, 0.021), (33, -0.009), (34, 0.006), (35, -0.014), (36, 0.036), (37, -0.007), (38, -0.048), (39, 0.0), (40, 0.005), (41, 0.018), (42, 0.006), (43, -0.003), (44, -0.014), (45, -0.029), (46, 0.016), (47, -0.019), (48, 0.006), (49, 0.035)]
simIndex simValue blogId blogTitle
same-blog 1 0.9722687 1914 andrew gelman stats-2013-06-25-Is there too much coauthorship in economics (and science more generally)? Or too little?
Introduction: Economist Stan Liebowitz has a longstanding interest in the difficulties of flagging published research errors. Recently he wrote on the related topic of dishonest authorship: While not about direct research fraud, I thought you might be interested in this paper . It discusses the manner in which credit is given for economics articles, and I suspect it applies to many other areas as well. One of the conclusions is that the lack of complete proration per author will lead to excessive coauthorship, reducing overall research output by inducing the use of larger than efficient-sized teams. Under these circumstances, false authorship can be a response to the warped reward system and false authorship might improve research efficiency since it might keep actual research teams (as opposed to nominal teams) from being too large to produce research efficiently. One of the questions I rhetorically ask in the paper is whether anyone has ever been ‘punished’ for having their name included on
2 0.86653739 989 andrew gelman stats-2011-11-03-This post does not mention Wegman
Introduction: A correspondent writes: Since you have commented on scientific fraud a lot. I wanted to give you an update on the Diederik Stapel case. I’d rather not see my name on the blog if you would elaborate on this any further. It is long but worth the read I guess. I’ll first give you the horrible details which will fill you with a mixture of horror and stupefied amazement at Stapel’s behavior. Then I’ll share Stapel’s abject apology, which might make you feel sorry for the guy. First the amazing story of how he perpetrated the fraud: There has been an interim report delivered to the rector of Tilburg University. Tilburg University is cooperating with the university of Amsterdam and of Groningen in this case. The results are pretty severe, I provide here a quick and literal translation of some comments by the chairman of the investigation committee. This report is publicly available on the university webpage (along with some other things of interest) but in Dutch: What
3 0.84864521 2137 andrew gelman stats-2013-12-17-Replication backlash
Introduction: Raghuveer Parthasarathy pointed me to an article in Nature by Mina Bissell, who writes , “The push to replicate findings could shelve promising research and unfairly damage the reputations of careful, meticulous scientists.” I can see where she’s coming from: if you work hard day after day in the lab, it’s gotta be a bit frustrating to find all your work questioned, for the frauds of the Dr. Anil Pottis and Diederik Stapels to be treated as a reason for everyone else’s work to be considered guilty until proven innocent. That said, I pretty much disagree with Bissell’s article, and really the best thing I can say about it is that I think it’s a good sign that the push for replication is so strong that now there’s a backlash against it. Traditionally, leading scientists have been able to simply ignore the push for replication. If they are feeling that the replication movement is strong enough that they need to fight it, that to me is good news. I’ll explain a bit in the conte
Introduction: Last month we discussed an opinion piece by Mina Bissell, a nationally-recognized leader in cancer biology. Bissell argued that there was too much of a push to replicate scientific findings. I disagreed , arguing that scientists should want others to be able to replicate their research, that it’s in everyone’s interest if replication can be done as fast and reliably as possible, and that if a published finding cannot be easily replicated, this is at best a failure of communication (in that the conditions for successful replication have not clearly been expressed), or possibly a fragile finding (that is, a phenomenon that appears under some conditions but not others), or at worst a plain old mistake (possibly associated with lab error or maybe with statistical error of some sort, such as jumping to certainty based on a statistically significant claim that arose from multiple comparisons ). So we disagreed. Fair enough. But I got to thinking about a possible source of our diffe
5 0.83835733 1139 andrew gelman stats-2012-01-26-Suggested resolution of the Bem paradox
Introduction: There has been an increasing discussion about the proliferation of flawed research in psychology and medicine, with some landmark events being John Ioannides’s article , “Why most published research findings are false” (according to Google Scholar, cited 973 times since its appearance in 2005), the scandals of Marc Hauser and Diederik Stapel, two leading psychology professors who resigned after disclosures of scientific misconduct, and Daryl Bem’s dubious recent paper on ESP, published to much fanfare in Journal of Personality and Social Psychology, one of the top journals in the field. Alongside all this are the plagiarism scandals, which are uninteresting from a scientific context but are relevant in that, in many cases, neither the institutions housing the plagiarists nor the editors and publishers of the plagiarized material seem to care. Perhaps these universities and publishers are more worried about bad publicity (and maybe lawsuits, given that many of the plagiarism cas
6 0.83592206 1844 andrew gelman stats-2013-05-06-Against optimism about social science
7 0.82861155 1917 andrew gelman stats-2013-06-28-Econ coauthorship update
8 0.8258667 1435 andrew gelman stats-2012-07-30-Retracted articles and unethical behavior in economics journals?
11 0.80894488 1585 andrew gelman stats-2012-11-20-“I know you aren’t the plagiarism police, but . . .”
12 0.80515152 1860 andrew gelman stats-2013-05-17-How can statisticians help psychologists do their research better?
13 0.79861689 1273 andrew gelman stats-2012-04-20-Proposals for alternative review systems for scientific work
14 0.79711801 1878 andrew gelman stats-2013-05-31-How to fix the tabloids? Toward replicable social science research
16 0.78989154 1603 andrew gelman stats-2012-12-03-Somebody listened to me!
17 0.78767759 1272 andrew gelman stats-2012-04-20-More proposals to reform the peer-review system
18 0.78611237 2055 andrew gelman stats-2013-10-08-A Bayesian approach for peer-review panels? and a speculation about Bruno Frey
20 0.77841896 2353 andrew gelman stats-2014-05-30-I posted this as a comment on a sociology blog
topicId topicWeight
[(5, 0.162), (9, 0.011), (15, 0.047), (16, 0.066), (21, 0.016), (24, 0.116), (45, 0.025), (53, 0.011), (61, 0.012), (70, 0.015), (76, 0.026), (82, 0.01), (86, 0.048), (89, 0.036), (95, 0.03), (99, 0.244)]
simIndex simValue blogId blogTitle
1 0.96328664 87 andrew gelman stats-2010-06-15-Statistical analysis and visualization of the drug war in Mexico
Introduction: Christian points me to this interesting (but sad) analysis by Diego Valle with an impressive series of graphs. There are a few things I’d change (notably the R default settings which result in ridiculously over-indexed y-axes, as well as axes for homicide rates which should (but do not) go town to zero (and sometimes, bizarrely, go negative), and a lack of coherent ordering of the 32 states (including D.F.), I’m no expert on Mexico (despite having coauthored a paper on Mexican politics) so I’ll leave it to others to evaluate the substantive claims in Valle’s blog. Just looking at what he’s done, though, it seems impressive to me. To put it another way, it’s like something Nate Silver might do.
2 0.96286339 513 andrew gelman stats-2011-01-12-“Tied for Warmest Year On Record”
Introduction: The National Climatic Data Center has tentatively announced that 2010 is, get this, “tied” for warmest on record. Presumably they mean it’s tied to the precision that they quote (1.12 F above the 20th-century average). The uncertainty in the measurements, as well as some fuzziness about exactly what is being measured (how much of the atmosphere, and the oceans) makes these global-average things really suspect. For instance, if there’s more oceanic turnover one year, that can warm the deep ocean but cool the shallow ocean and atmosphere, so even though the heat content of the atmosphere-ocean system goes up, some of these “global-average” estimates can go down. The reverse can happen too. And of course there are various sources of natural variability that are not, these days, what most people are most interested in. So everybody who knows about the climate professes to hate the emphasis on climate records. And yet, they’re irresistible. I’m sure we’ll see the usual clamor of som
3 0.95601898 665 andrew gelman stats-2011-04-17-Yes, your wish shall be granted (in 25 years)
Introduction: This one was so beautiful I just had to repost it: From the New York Times, 9 Sept 1981: IF I COULD CHANGE PARK SLOPE If I could change Park Slope I would turn it into a palace with queens and kings and princesses to dance the night away at the ball. The trees would look like garden stalks. The lights would look like silver pearls and the dresses would look like soft silver silk. You should see the ball. It looks so luxurious to me. The Park Slope ball is great. Can you guess what street it’s on? “Yes. My street. That’s Carroll Street.” – Jennifer Chatmon, second grade, P.S. 321 This was a few years before my sister told me that she felt safer having a crack house down the block because the cops were surveilling it all the time.
4 0.94972646 1250 andrew gelman stats-2012-04-07-Hangman tips
Introduction: Jeff pointed me to this article by Nick Berry. It’s kind of fun but of course if you know your opponent will be following this strategy you can figure out how to outwit it. Also, Berry writes that ETAOIN SHRDLU CMFWYP VBGKQJ XZ is the “ordering of letter frequency in English language.” Indeed this is the conventional ordering but nobody thinks it’s right anymore. See here (with further discussion here ). I wonder what corpus he’s using. P.S. Klutz was my personal standby.
Introduction: As I wrote a couple years ago: Even though statistical analysis has demonstrated that presidential elections are predictable given economic conditions and previous votes in the states . . . it certainly doesn’t mean that every election can be accurately predicted ahead of time. Presidential general election campaigns have several distinct features that distinguish them from most other elections: 1. Two major candidates; 2. The candidates clearly differ in their political ideologies and in their positions on economic issues; 3. The two sides have roughly equal financial and organizational resources; 4. The current election is the latest in a long series of similar contests (every four years); 5. A long campaign, giving candidates a long time to present their case and giving voters a long time to make up their minds. Other elections look different. . . . Or, as I said in reference to the current NYC mayoral election: Et selon Andrew Gelman, expert de l’universi
7 0.93834794 422 andrew gelman stats-2010-11-20-A Gapminder-like data visualization package
8 0.93704224 1606 andrew gelman stats-2012-12-05-The Grinch Comes Back
9 0.93087196 1052 andrew gelman stats-2011-12-11-Rational Turbulence
10 0.93072462 1512 andrew gelman stats-2012-09-27-A Non-random Walk Down Campaign Street
same-blog 12 0.92482805 1914 andrew gelman stats-2013-06-25-Is there too much coauthorship in economics (and science more generally)? Or too little?
14 0.92395627 1286 andrew gelman stats-2012-04-28-Agreement Groups in US Senate and Dynamic Clustering
15 0.92168796 228 andrew gelman stats-2010-08-24-A new efficient lossless compression algorithm
16 0.91304803 1841 andrew gelman stats-2013-05-04-The Folk Theorem of Statistical Computing
17 0.90398937 131 andrew gelman stats-2010-07-07-A note to John
18 0.90383649 224 andrew gelman stats-2010-08-22-Mister P gets married
19 0.90028822 951 andrew gelman stats-2011-10-11-Data mining efforts for Obama’s campaign
20 0.89398849 391 andrew gelman stats-2010-11-03-Some thoughts on election forecasting