andrew_gelman_stats andrew_gelman_stats-2014 andrew_gelman_stats-2014-2245 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: I’m postponing today’s scheduled post (“Empirical implications of Empirical Implications of Theoretical Models”) to continue the lively discussion from yesterday, What if I were to stop publishing in journals? . An example: my papers with Basbøll Thomas Basbøll and I got into a long discussion on our blogs about business school professor Karl Weick and other cases of plagiarism copying text without attribution. We felt it useful to take our ideas to the next level and write them up as a manuscript, which ended up being logical to split into two papers. At that point I put some effort into getting these papers published, which I eventually did: To throw away data: Plagiarism as a statistical crime went into American Scientist and When do stories work? Evidence and illustration in the social sciences will appear in Sociological Methods and Research. The second paper, in particular, took some effort to place; I got some advice from colleagues in sociology as to where
sentIndex sentText sentNum sentScore
1 An example: my papers with Basbøll Thomas Basbøll and I got into a long discussion on our blogs about business school professor Karl Weick and other cases of plagiarism copying text without attribution. [sent-3, score-0.318]
2 The second paper, in particular, took some effort to place; I got some advice from colleagues in sociology as to where to send it (and I think it got rejected from a couple other places first, but now I can’t remember if I’m confusing that with some other paper). [sent-7, score-0.407]
3 Why wasn’t it enough to post the papers on my website and blog them? [sent-9, score-0.33]
4 The paper will get some additional readers this way, but I don’t know if it was worth the effort (indeed, we are still in the copyediting phase; there was some glitch and we haven’t heard back from the copy editors who I think are in India). [sent-21, score-0.358]
5 These papers express some ideas that have been bugging me for about 20 years. [sent-27, score-0.33]
6 I do think all this got our article more attention and gave it some legitimacy, to the extent that maybe our ideas will have broader influence. [sent-41, score-0.363]
7 Two papers with Guido Next I’ll discuss two articles that are currently in limbo. [sent-42, score-0.346]
8 Guido and I passed this paper back and forth, fixing it up in various ways, then I posted it and it got a lot of attention. [sent-47, score-0.304]
9 Now consider a world in which research journals have disappeared, having been replaced by some sort of beefed-up Arxiv or SSRN (or maybe Plos-One is a better model in that authors would then have to spend a few hundred bucks to defray the cost of organizing review reports). [sent-59, score-0.369]
10 I’d write these papers and post them and blog on them, just as before. [sent-61, score-0.403]
11 In the super-SSRN world, all posted papers would get listed on Google scholar the way published articles do now, so indexing would be no problem. [sent-65, score-0.659]
12 Instead of publishing X articles every quarter, or even Y articles every week, the volunteer editorial board of JASA (along with its volunteer referees) will choose X (or maybe 2X) articles every week to promote, out of all the articles appearing that period in this Arxiv/SSRN/Plos-One space. [sent-68, score-0.742]
13 The top journal is JASA, I think—but others would give the nod to the Annals of Statistics, or maybe Biometrika (although those latter two journals lean toward a no-applied-content rule which restricts their scope). [sent-73, score-0.418]
14 Then there’s Biometrics, and the Royal Statistical Society, and also statistics journals in neighboring fields such as Psychometrika, Econometrica, and JMLR. [sent-74, score-0.408]
15 So statistics is not one of those fields where there’s a clear ranking of top journals. [sent-76, score-0.305]
16 On the other side, the transmission of statistics papers is still centered on journal publication. [sent-77, score-0.422]
17 They have a few journals that are almost universally agreed to be the best, and it seems that papers published in these places automatically get attention. [sent-92, score-0.64]
18 In these social science fields, a paper gets workshopped for awhile before submission, then there can be a grueling review process. [sent-98, score-0.403]
19 But then there’s the whole parallel world of “Psychology Science”-type papers and the culture clash that arises when people naively think that papers published in that top journal must therefore be of top quality. [sent-104, score-1.128]
20 A parallel world of industry-funded studies, so much is published that there are journals that publish nothing but review after review after review of published studies, automated tools for parsing the literature . [sent-111, score-1.024]
wordName wordTfidf (topN-words)
[('guido', 0.273), ('papers', 0.213), ('journals', 0.203), ('published', 0.142), ('paper', 0.14), ('articles', 0.133), ('philosophy', 0.118), ('contacted', 0.118), ('ideas', 0.117), ('parallel', 0.115), ('journal', 0.115), ('fields', 0.111), ('jasa', 0.106), ('got', 0.105), ('top', 0.1), ('review', 0.099), ('submitting', 0.094), ('statistics', 0.094), ('causal', 0.09), ('indexing', 0.089), ('arxiv', 0.088), ('science', 0.087), ('psychology', 0.086), ('get', 0.082), ('publication', 0.079), ('article', 0.078), ('social', 0.077), ('circulation', 0.076), ('write', 0.073), ('effort', 0.073), ('volunteer', 0.072), ('economics', 0.072), ('world', 0.067), ('submission', 0.067), ('publishing', 0.066), ('american', 0.065), ('think', 0.063), ('edited', 0.061), ('send', 0.061), ('blog', 0.059), ('lot', 0.059), ('referees', 0.059), ('tradition', 0.059), ('cosma', 0.059), ('sociological', 0.059), ('book', 0.058), ('bayesian', 0.058), ('publish', 0.058), ('post', 0.058), ('organizations', 0.057)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999928 2245 andrew gelman stats-2014-03-12-More on publishing in journals
Introduction: I’m postponing today’s scheduled post (“Empirical implications of Empirical Implications of Theoretical Models”) to continue the lively discussion from yesterday, What if I were to stop publishing in journals? . An example: my papers with Basbøll Thomas Basbøll and I got into a long discussion on our blogs about business school professor Karl Weick and other cases of plagiarism copying text without attribution. We felt it useful to take our ideas to the next level and write them up as a manuscript, which ended up being logical to split into two papers. At that point I put some effort into getting these papers published, which I eventually did: To throw away data: Plagiarism as a statistical crime went into American Scientist and When do stories work? Evidence and illustration in the social sciences will appear in Sociological Methods and Research. The second paper, in particular, took some effort to place; I got some advice from colleagues in sociology as to where
2 0.35351253 1928 andrew gelman stats-2013-07-06-How to think about papers published in low-grade journals?
Introduction: We’ve had lots of lively discussions of fatally-flawed papers that have been published in top, top journals such as the American Economic Review or the Journal of Personality and Social Psychology or the American Sociological Review or the tabloids . And we also know about mistakes that make their way into mid-ranking outlets such as the Journal of Theoretical Biology. But what about results that appear in the lower tier of legitimate journals? I was thinking about this after reading a post by Dan Kahan slamming a paper that recently appeared in PLOS-One. I won’t discuss the paper itself here because that’s not my point. Rather, I had some thoughts regarding Kahan’s annoyance that a paper with fatal errors was published at all. I commented as follows: Read between the lines. The paper originally was released in 2009 and was published in 2013 in PLOS-One, which is one step above appearing on Arxiv. PLOS-One publishes some good things (so does Arxiv) but it’s the place
3 0.28752166 2244 andrew gelman stats-2014-03-11-What if I were to stop publishing in journals?
Introduction: In our recent discussion of modes of publication, Joseph Wilson wrote, “The single best reform science can make right now is to decouple publication from career advancement, thereby reducing the number of publications by an order of magnitude and then move to an entirely disjointed, informal, online free-for-all communication system for research results.” My first thought on this was: Sure, yeah, that makes sense. But then I got to thinking: what would it really mean to decouple publication from career advancement? This is too late for me—I’m middle-aged and have no career advancement in my future—but it got me thinking more carefully about the role of publication in the research process, and this seemed worth a blog (the simplest sort of publication available to me). However, somewhere between writing the above paragraphs and writing the blog entry, I forgot exactly what I was going to say! I guess I should’ve just typed it all in then. In the old days I just wouldn’t run this
Introduction: The other day we discussed that paper on ovulation and voting (you may recall that the authors reported a scattered bunch of comparisons, significance tests, and p-values, and I recommended that they would’ve done better to simply report complete summaries of their data, so that readers could see the comparisons of interest in full context), and I was thinking a bit more about why I was so bothered that it was published in Psychological Science, which I’d thought of as a serious research journal. My concern isn’t just that that the paper is bad—after all, lots of bad papers get published—but rather that it had nothing really going for it, except that it was headline bait. It was a survey done on Mechanical Turk, that’s it. No clever design, no clever questions, no care in dealing with nonresponse problems, no innovative data analysis, no nothing. The paper had nothing to offer, except that it had no obvious flaws. Psychology is a huge field full of brilliant researchers.
5 0.26350254 1435 andrew gelman stats-2012-07-30-Retracted articles and unethical behavior in economics journals?
Introduction: Stan Liebowitz writes: Have you ever heard of an article being retracted in economics? I know you have only been doing this for a few years but I suspect that the answer is that none or very few are retracted. No economist would ever deceive another. There is virtually no interest in detecting cheating. And what good would that do if there is no form of punishment? I say this because I think I have found a case in one of our top journals but the editor allowed the authors of the original article to write an anonymous referee report defending themselves and used this report to reject my comment even though an independent referee recommended publication. My reply: I wonder how this sort of thing will change in the future as journals become less important. My impression is that, on one side, researchers are increasingly citing NBER reports, Arxiv preprints, and the like; while, from the other direction, journals such as Science and Nature are developing the reputations of being “t
6 0.24999276 834 andrew gelman stats-2011-08-01-I owe it all to the haters
7 0.24882865 902 andrew gelman stats-2011-09-12-The importance of style in academic writing
8 0.24755257 371 andrew gelman stats-2010-10-26-Musical chairs in econ journals
9 0.2365835 110 andrew gelman stats-2010-06-26-Philosophy and the practice of Bayesian statistics
10 0.23421407 1393 andrew gelman stats-2012-06-26-The reverse-journal-submission system
11 0.2312264 1139 andrew gelman stats-2012-01-26-Suggested resolution of the Bem paradox
12 0.23082158 1878 andrew gelman stats-2013-05-31-How to fix the tabloids? Toward replicable social science research
13 0.22969194 746 andrew gelman stats-2011-06-05-An unexpected benefit of Arrow’s other theorem
14 0.22388105 120 andrew gelman stats-2010-06-30-You can’t put Pandora back in the box
15 0.22147401 675 andrew gelman stats-2011-04-22-Arrow’s other theorem
16 0.21823289 109 andrew gelman stats-2010-06-25-Classics of statistics
17 0.21717755 2006 andrew gelman stats-2013-09-03-Evaluating evidence from published research
18 0.21446489 2269 andrew gelman stats-2014-03-27-Beyond the Valley of the Trolls
19 0.20597017 2353 andrew gelman stats-2014-05-30-I posted this as a comment on a sociology blog
topicId topicWeight
[(0, 0.416), (1, -0.115), (2, -0.226), (3, -0.1), (4, -0.169), (5, -0.04), (6, -0.014), (7, -0.149), (8, 0.02), (9, 0.008), (10, 0.193), (11, -0.006), (12, -0.086), (13, 0.066), (14, 0.031), (15, -0.065), (16, -0.014), (17, 0.044), (18, -0.036), (19, -0.006), (20, 0.062), (21, -0.033), (22, 0.041), (23, 0.014), (24, 0.054), (25, 0.041), (26, -0.042), (27, -0.011), (28, -0.042), (29, 0.03), (30, 0.024), (31, -0.065), (32, -0.02), (33, -0.011), (34, -0.007), (35, 0.0), (36, 0.001), (37, 0.054), (38, -0.032), (39, 0.003), (40, -0.005), (41, 0.002), (42, -0.018), (43, -0.003), (44, -0.019), (45, 0.031), (46, -0.007), (47, -0.008), (48, -0.037), (49, 0.009)]
simIndex simValue blogId blogTitle
same-blog 1 0.98409063 2245 andrew gelman stats-2014-03-12-More on publishing in journals
Introduction: I’m postponing today’s scheduled post (“Empirical implications of Empirical Implications of Theoretical Models”) to continue the lively discussion from yesterday, What if I were to stop publishing in journals? . An example: my papers with Basbøll Thomas Basbøll and I got into a long discussion on our blogs about business school professor Karl Weick and other cases of plagiarism copying text without attribution. We felt it useful to take our ideas to the next level and write them up as a manuscript, which ended up being logical to split into two papers. At that point I put some effort into getting these papers published, which I eventually did: To throw away data: Plagiarism as a statistical crime went into American Scientist and When do stories work? Evidence and illustration in the social sciences will appear in Sociological Methods and Research. The second paper, in particular, took some effort to place; I got some advice from colleagues in sociology as to where
2 0.91638756 834 andrew gelman stats-2011-08-01-I owe it all to the haters
Introduction: Sometimes when I submit an article to a journal it is accepted right away or with minor alterations. But many of my favorite articles were rejected or had to go through an exhausting series of revisions. For example, this influential article had a very hostile referee and we had to seriously push the journal editor to accept it. This one was rejected by one or two journals before finally appearing with discussion. This paper was rejected by the American Political Science Review with no chance of revision and we had to publish it in the British Journal of Political Science, which was a bit odd given that the article was 100% about American politics. And when I submitted this instant classic (actually at the invitation of the editor), the referees found it to be trivial, and the editor did me the favor of publishing it but only by officially labeling it as a discussion of another article that appeared in the same issue. Some of my most influential papers were accepted right
3 0.9130162 675 andrew gelman stats-2011-04-22-Arrow’s other theorem
Introduction: I received the following email from someone who’d like to remain anonymous: Lately I [the anonymous correspondent] witnessed that Bruno Frey has published two articles in two well known referreed journals on the Titanic disaster that try to explain survival rates of passenger on board. The articles were published in the Journal of Economic Perspectives and Rationality & Society . While looking up the name of the second journal where I stumbled across the article I even saw that they put the message in a third journal, the Proceedings of the National Academy of Sciences United States of America . To say it in Sopranos like style – with all due respect, I know Bruno Frey from conferences, I really appreciate his take on economics as a social science and he has really published more interesting stuff that most economists ever will. But putting the same message into three journals gives me headaches for at least two reasons: 1) When building a track record and scientific rep
4 0.8948198 2233 andrew gelman stats-2014-03-04-Literal vs. rhetorical
Introduction: Thomas Basbøll pointed me to a discussion on the orgtheory blog in which Jerry Davis, the editor of a journal of business management argued that it is difficult for academic researchers to communicate with the public because “the public prefers Cheetos to a healthy salad” and when serious papers are discussed on the internet, “everyone is a methodologist.” The discussion heated up when an actual methodologist, Steve Morgan, joined in to argue that the salad in question was not so healthy and that the much-derided internet commenters made some valuable points. The final twist was that one of the orgtheory bloggers deleted a comment and then closed the thread entirely when the discussion got too conflictual. In a few days I’ll return to the meta-topic of the discussion, but right now I want to focus on one thing Davis wrote, a particular statement that illustrates to me the gap between the rhetorical and the literal, the way in which a statement can sound good but make no sense. He
5 0.88442677 120 andrew gelman stats-2010-06-30-You can’t put Pandora back in the box
Introduction: Rajiv Sethi writes : I suspect that within a decade, blogs will be a cornerstone of research in economics. Many original and creative contributions to the discipline will first be communicated to the profession (and the world at large) in the form of blog posts, since the medium allows for material of arbitrary length, depth and complexity. Ideas first expressed in this form will make their way (with suitable attribution) into reading lists, doctoral dissertations and more conventionally refereed academic publications. And blogs will come to play a central role in the process of recruitment, promotion and reward at major research universities. This genie is not going back into its bottle. And he thinks this is a good thing: In fact, the refereeing process for blog posts is in some respects more rigorous than that for journal articles. Reports are numerous, non-anonymous, public, rapidly and efficiently produced, and collaboratively constructed. It is not obvious to me [Sethi]
6 0.88049811 2353 andrew gelman stats-2014-05-30-I posted this as a comment on a sociology blog
7 0.87721199 2269 andrew gelman stats-2014-03-27-Beyond the Valley of the Trolls
8 0.87696975 1321 andrew gelman stats-2012-05-15-A statistical research project: Weeding out the fraudulent citations
9 0.87265974 2244 andrew gelman stats-2014-03-11-What if I were to stop publishing in journals?
10 0.86979055 1928 andrew gelman stats-2013-07-06-How to think about papers published in low-grade journals?
11 0.86886793 902 andrew gelman stats-2011-09-12-The importance of style in academic writing
12 0.86642128 1435 andrew gelman stats-2012-07-30-Retracted articles and unethical behavior in economics journals?
14 0.85920542 883 andrew gelman stats-2011-09-01-Arrow’s theorem update
15 0.85343707 1139 andrew gelman stats-2012-01-26-Suggested resolution of the Bem paradox
16 0.84784025 371 andrew gelman stats-2010-10-26-Musical chairs in econ journals
19 0.83205736 2177 andrew gelman stats-2014-01-19-“The British amateur who debunked the mathematics of happiness”
topicId topicWeight
[(9, 0.025), (13, 0.01), (15, 0.067), (16, 0.107), (21, 0.023), (24, 0.139), (27, 0.017), (30, 0.021), (42, 0.023), (43, 0.012), (52, 0.016), (55, 0.013), (64, 0.019), (69, 0.015), (76, 0.011), (86, 0.034), (95, 0.012), (99, 0.341)]
simIndex simValue blogId blogTitle
1 0.98699194 2137 andrew gelman stats-2013-12-17-Replication backlash
Introduction: Raghuveer Parthasarathy pointed me to an article in Nature by Mina Bissell, who writes , “The push to replicate findings could shelve promising research and unfairly damage the reputations of careful, meticulous scientists.” I can see where she’s coming from: if you work hard day after day in the lab, it’s gotta be a bit frustrating to find all your work questioned, for the frauds of the Dr. Anil Pottis and Diederik Stapels to be treated as a reason for everyone else’s work to be considered guilty until proven innocent. That said, I pretty much disagree with Bissell’s article, and really the best thing I can say about it is that I think it’s a good sign that the push for replication is so strong that now there’s a backlash against it. Traditionally, leading scientists have been able to simply ignore the push for replication. If they are feeling that the replication movement is strong enough that they need to fight it, that to me is good news. I’ll explain a bit in the conte
2 0.98549366 2177 andrew gelman stats-2014-01-19-“The British amateur who debunked the mathematics of happiness”
Introduction: Andrew Anthony tells the excellent story of how Nick Brown, Alan Sokal, and Harris Friedman shot down some particularly silly work in psychology. (“According to the graph, it all came down to a specific ratio of positive emotions to negative emotions. If your ratio was greater than 2.9013 positive emotions to 1 negative emotion you were flourishing in life. If your ratio was less than that number you were languishing.” And, yes, the work they were shooting down really is that bad.) If you want to see what the fuss is about, just google “2.9013.” Here’s an example (from 2012) of an uncritical reporting of the claim, here’s another one from 2010, here’s one from 2011 . . . well, you get the idea. And here’s a quick summary posted by Rolf Zwaan after Brown et al. came out with their paper. I know Sokal and Brown and so this story was not news to me. I didn’t post anything about it on this blog because it seemed like it was getting enough coverage elsewhere. I think Ni
Introduction: Jeff Leek points to a post by Alex Holcombe, who disputes the idea that science is self-correcting. Holcombe writes [scroll down to get to his part]: The pace of scientific production has quickened, and self-correction has suffered. Findings that might correct old results are considered less interesting than results from more original research questions. Potential corrections are also more contested. As the competition for space in prestigious journals has become increasingly frenzied, doing and publishing studies that would confirm the rapidly accumulating new discoveries, or would correct them, became a losing proposition. Holcombe picks up on some points that we’ve discussed a lot here in the past year. Here’s Holcombe: In certain subfields, almost all new work appears in only a very few journals, all associated with a single professional society. There is then no way around the senior gatekeepers, who may then suppress corrections with impunity. . . . The bias agai
4 0.98493338 1878 andrew gelman stats-2013-05-31-How to fix the tabloids? Toward replicable social science research
Introduction: This seems to be the topic of the week. Yesterday I posted on the sister blog some further thoughts on those “Psychological Science” papers on menstrual cycles, biceps size, and political attitudes, tied to a horrible press release from the journal Psychological Science hyping the biceps and politics study. Then I was pointed to these suggestions from Richard Lucas and M. Brent Donnellan have on improving the replicability and reproducibility of research published in the Journal of Research in Personality: It goes without saying that editors of scientific journals strive to publish research that is not only theoretically interesting but also methodologically rigorous. The goal is to select papers that advance the field. Accordingly, editors want to publish findings that can be reproduced and replicated by other scientists. Unfortunately, there has been a recent “crisis in confidence” among psychologists about the quality of psychological research (Pashler & Wagenmakers, 2012)
5 0.9848426 431 andrew gelman stats-2010-11-26-One fun thing about physicists . . .
Introduction: . . . they’re not in awe of economists. In contrast, economists sometimes treat each other with the soft bigotry of low expectations. For example, here’s Brad DeLong in defense of Larry Summers: [During a 2005 meeting, Summers] said that in a modern economy with sophisticated financial markets we were likely to have more and bigger financial crises than we had before, just as the worst modern transportation accidents are worse than the worst transportation accidents back in horse-and-buggy days. . . . Indeed, for twenty years one of Larry’s conversation openers has been: “You really should write something else good on positive-feedback trading and its dangers for financial markets.” That’s fine, but, hey, I’ve been going around saying this for many years too, and I’m not even an economist (although I did get an A in the last econ class I took, which was in eleventh grade). Lots and lots of people have been talking for years about the dangers of positive feedback, the risk
6 0.98469228 120 andrew gelman stats-2010-06-30-You can’t put Pandora back in the box
7 0.98427069 1435 andrew gelman stats-2012-07-30-Retracted articles and unethical behavior in economics journals?
8 0.98307198 2353 andrew gelman stats-2014-05-30-I posted this as a comment on a sociology blog
10 0.98147452 2227 andrew gelman stats-2014-02-27-“What Can we Learn from the Many Labs Replication Project?”
11 0.98142624 481 andrew gelman stats-2010-12-22-The Jumpstart financial literacy survey and the different purposes of tests
13 0.98105651 110 andrew gelman stats-2010-06-26-Philosophy and the practice of Bayesian statistics
14 0.98094964 675 andrew gelman stats-2011-04-22-Arrow’s other theorem
15 0.98091549 711 andrew gelman stats-2011-05-14-Steven Rhoads’s book, “The Economist’s View of the World”
same-blog 16 0.98083729 2245 andrew gelman stats-2014-03-12-More on publishing in journals
17 0.97977018 1848 andrew gelman stats-2013-05-09-A tale of two discussion papers
19 0.97943509 2244 andrew gelman stats-2014-03-11-What if I were to stop publishing in journals?