andrew_gelman_stats andrew_gelman_stats-2012 andrew_gelman_stats-2012-1321 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: John Mashey points me to a blog post by Phil Davis on “the emergence of a citation cartel.” Davis tells the story: Cell Transplantation is a medical journal published by the Cognizant Communication Corporation of Putnam Valley, New York. In recent years, its impact factor has been growing rapidly. In 2006, it was 3.482 [I think he means "3.5"---ed.]. In 2010, it had almost doubled to 6.204. When you look at which journals cite Cell Transplantation, two journals stand out noticeably: the Medical Science Monitor, and The Scientific World Journal. According to the JCR, neither of these journals cited Cell Transplantation until 2010. Then, in 2010, a review article was published in the Medical Science Monitor citing 490 articles, 445 of which were to papers published in Cell Transplantation. All 445 citations pointed to papers published in 2008 or 2009 — the citation window from which the journal’s 2010 impact factor was derived. Of the remaining 45 citations, 44 cited the Me
sentIndex sentText sentNum sentScore
1 John Mashey points me to a blog post by Phil Davis on “the emergence of a citation cartel. [sent-1, score-0.247]
2 ” Davis tells the story: Cell Transplantation is a medical journal published by the Cognizant Communication Corporation of Putnam Valley, New York. [sent-2, score-0.477]
3 In recent years, its impact factor has been growing rapidly. [sent-3, score-0.288]
4 When you look at which journals cite Cell Transplantation, two journals stand out noticeably: the Medical Science Monitor, and The Scientific World Journal. [sent-10, score-0.368]
5 According to the JCR, neither of these journals cited Cell Transplantation until 2010. [sent-11, score-0.295]
6 Then, in 2010, a review article was published in the Medical Science Monitor citing 490 articles, 445 of which were to papers published in Cell Transplantation. [sent-12, score-0.54]
7 All 445 citations pointed to papers published in 2008 or 2009 — the citation window from which the journal’s 2010 impact factor was derived. [sent-13, score-0.899]
8 Of the remaining 45 citations, 44 cited the Medical Science Monitor, again, to papers published in 2008 and 2009. [sent-14, score-0.427]
9 Three of the four authors of this paper sit on the editorial board of Cell Transplantation. [sent-15, score-0.402]
10 Two are associate editors, one is the founding editor. [sent-16, score-0.157]
11 The fourth is the CEO of a medical communications company. [sent-17, score-0.381]
12 What’s more, editors can protect these “reviews” from peer review if they are labeled as “editorial material,” as some are. [sent-20, score-0.257]
13 I’d think this sort of thing should be detectable using statistical analysis. [sent-22, score-0.101]
14 This seems like the sort of fight-fire-with-fire situation where there will be at least a partial technological solution. [sent-23, score-0.08]
15 Lots more interesting stuff on that blog, for example this post by Kent Anderson disparaging open-access publishing. [sent-30, score-0.091]
wordName wordTfidf (topN-words)
[('transplantation', 0.333), ('cell', 0.328), ('editorial', 0.282), ('davis', 0.234), ('medical', 0.234), ('monitor', 0.223), ('journals', 0.184), ('citation', 0.156), ('published', 0.146), ('impact', 0.145), ('factor', 0.143), ('citations', 0.135), ('board', 0.12), ('editors', 0.111), ('cited', 0.111), ('papers', 0.103), ('agreements', 0.101), ('cooperative', 0.101), ('detectable', 0.101), ('facilitated', 0.101), ('journal', 0.097), ('emergence', 0.091), ('founding', 0.091), ('doubled', 0.091), ('disparaging', 0.091), ('putnam', 0.091), ('boards', 0.088), ('noticeably', 0.088), ('overlapping', 0.088), ('ease', 0.088), ('gaming', 0.083), ('corporation', 0.083), ('kent', 0.083), ('ceo', 0.083), ('technological', 0.08), ('mashey', 0.08), ('rewards', 0.078), ('communications', 0.075), ('valley', 0.075), ('review', 0.075), ('science', 0.073), ('anderson', 0.073), ('fourth', 0.072), ('window', 0.071), ('protect', 0.071), ('citing', 0.07), ('detect', 0.067), ('remaining', 0.067), ('associate', 0.066), ('greatly', 0.064)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 1321 andrew gelman stats-2012-05-15-A statistical research project: Weeding out the fraudulent citations
Introduction: John Mashey points me to a blog post by Phil Davis on “the emergence of a citation cartel.” Davis tells the story: Cell Transplantation is a medical journal published by the Cognizant Communication Corporation of Putnam Valley, New York. In recent years, its impact factor has been growing rapidly. In 2006, it was 3.482 [I think he means "3.5"---ed.]. In 2010, it had almost doubled to 6.204. When you look at which journals cite Cell Transplantation, two journals stand out noticeably: the Medical Science Monitor, and The Scientific World Journal. According to the JCR, neither of these journals cited Cell Transplantation until 2010. Then, in 2010, a review article was published in the Medical Science Monitor citing 490 articles, 445 of which were to papers published in Cell Transplantation. All 445 citations pointed to papers published in 2008 or 2009 — the citation window from which the journal’s 2010 impact factor was derived. Of the remaining 45 citations, 44 cited the Me
2 0.17760743 2245 andrew gelman stats-2014-03-12-More on publishing in journals
Introduction: I’m postponing today’s scheduled post (“Empirical implications of Empirical Implications of Theoretical Models”) to continue the lively discussion from yesterday, What if I were to stop publishing in journals? . An example: my papers with Basbøll Thomas Basbøll and I got into a long discussion on our blogs about business school professor Karl Weick and other cases of plagiarism copying text without attribution. We felt it useful to take our ideas to the next level and write them up as a manuscript, which ended up being logical to split into two papers. At that point I put some effort into getting these papers published, which I eventually did: To throw away data: Plagiarism as a statistical crime went into American Scientist and When do stories work? Evidence and illustration in the social sciences will appear in Sociological Methods and Research. The second paper, in particular, took some effort to place; I got some advice from colleagues in sociology as to where
3 0.16480924 2233 andrew gelman stats-2014-03-04-Literal vs. rhetorical
Introduction: Thomas Basbøll pointed me to a discussion on the orgtheory blog in which Jerry Davis, the editor of a journal of business management argued that it is difficult for academic researchers to communicate with the public because “the public prefers Cheetos to a healthy salad” and when serious papers are discussed on the internet, “everyone is a methodologist.” The discussion heated up when an actual methodologist, Steve Morgan, joined in to argue that the salad in question was not so healthy and that the much-derided internet commenters made some valuable points. The final twist was that one of the orgtheory bloggers deleted a comment and then closed the thread entirely when the discussion got too conflictual. In a few days I’ll return to the meta-topic of the discussion, but right now I want to focus on one thing Davis wrote, a particular statement that illustrates to me the gap between the rhetorical and the literal, the way in which a statement can sound good but make no sense. He
4 0.16432002 1928 andrew gelman stats-2013-07-06-How to think about papers published in low-grade journals?
Introduction: We’ve had lots of lively discussions of fatally-flawed papers that have been published in top, top journals such as the American Economic Review or the Journal of Personality and Social Psychology or the American Sociological Review or the tabloids . And we also know about mistakes that make their way into mid-ranking outlets such as the Journal of Theoretical Biology. But what about results that appear in the lower tier of legitimate journals? I was thinking about this after reading a post by Dan Kahan slamming a paper that recently appeared in PLOS-One. I won’t discuss the paper itself here because that’s not my point. Rather, I had some thoughts regarding Kahan’s annoyance that a paper with fatal errors was published at all. I commented as follows: Read between the lines. The paper originally was released in 2009 and was published in 2013 in PLOS-One, which is one step above appearing on Arxiv. PLOS-One publishes some good things (so does Arxiv) but it’s the place
Introduction: Jeff Leek points to a post by Alex Holcombe, who disputes the idea that science is self-correcting. Holcombe writes [scroll down to get to his part]: The pace of scientific production has quickened, and self-correction has suffered. Findings that might correct old results are considered less interesting than results from more original research questions. Potential corrections are also more contested. As the competition for space in prestigious journals has become increasingly frenzied, doing and publishing studies that would confirm the rapidly accumulating new discoveries, or would correct them, became a losing proposition. Holcombe picks up on some points that we’ve discussed a lot here in the past year. Here’s Holcombe: In certain subfields, almost all new work appears in only a very few journals, all associated with a single professional society. There is then no way around the senior gatekeepers, who may then suppress corrections with impunity. . . . The bias agai
6 0.14035688 1272 andrew gelman stats-2012-04-20-More proposals to reform the peer-review system
7 0.12574491 371 andrew gelman stats-2010-10-26-Musical chairs in econ journals
8 0.12231244 1435 andrew gelman stats-2012-07-30-Retracted articles and unethical behavior in economics journals?
9 0.12001984 1273 andrew gelman stats-2012-04-20-Proposals for alternative review systems for scientific work
11 0.11327563 675 andrew gelman stats-2011-04-22-Arrow’s other theorem
12 0.1121139 902 andrew gelman stats-2011-09-12-The importance of style in academic writing
14 0.10720898 1878 andrew gelman stats-2013-05-31-How to fix the tabloids? Toward replicable social science research
15 0.10641317 2006 andrew gelman stats-2013-09-03-Evaluating evidence from published research
16 0.1037875 2055 andrew gelman stats-2013-10-08-A Bayesian approach for peer-review panels? and a speculation about Bruno Frey
17 0.10376804 1139 andrew gelman stats-2012-01-26-Suggested resolution of the Bem paradox
19 0.099408083 2353 andrew gelman stats-2014-05-30-I posted this as a comment on a sociology blog
20 0.096243262 1393 andrew gelman stats-2012-06-26-The reverse-journal-submission system
topicId topicWeight
[(0, 0.12), (1, -0.06), (2, -0.077), (3, -0.12), (4, -0.063), (5, -0.053), (6, 0.013), (7, -0.105), (8, -0.029), (9, 0.007), (10, 0.123), (11, 0.015), (12, -0.047), (13, 0.026), (14, -0.005), (15, -0.035), (16, 0.012), (17, 0.051), (18, -0.015), (19, -0.028), (20, 0.027), (21, 0.023), (22, 0.018), (23, 0.014), (24, -0.008), (25, 0.006), (26, -0.034), (27, 0.022), (28, 0.006), (29, 0.006), (30, -0.03), (31, -0.034), (32, -0.006), (33, 0.001), (34, -0.006), (35, -0.003), (36, 0.026), (37, 0.035), (38, 0.024), (39, 0.019), (40, -0.018), (41, -0.054), (42, -0.038), (43, 0.002), (44, 0.014), (45, 0.034), (46, 0.023), (47, 0.011), (48, -0.022), (49, 0.005)]
simIndex simValue blogId blogTitle
same-blog 1 0.98292625 1321 andrew gelman stats-2012-05-15-A statistical research project: Weeding out the fraudulent citations
Introduction: John Mashey points me to a blog post by Phil Davis on “the emergence of a citation cartel.” Davis tells the story: Cell Transplantation is a medical journal published by the Cognizant Communication Corporation of Putnam Valley, New York. In recent years, its impact factor has been growing rapidly. In 2006, it was 3.482 [I think he means "3.5"---ed.]. In 2010, it had almost doubled to 6.204. When you look at which journals cite Cell Transplantation, two journals stand out noticeably: the Medical Science Monitor, and The Scientific World Journal. According to the JCR, neither of these journals cited Cell Transplantation until 2010. Then, in 2010, a review article was published in the Medical Science Monitor citing 490 articles, 445 of which were to papers published in Cell Transplantation. All 445 citations pointed to papers published in 2008 or 2009 — the citation window from which the journal’s 2010 impact factor was derived. Of the remaining 45 citations, 44 cited the Me
2 0.91427344 1928 andrew gelman stats-2013-07-06-How to think about papers published in low-grade journals?
Introduction: We’ve had lots of lively discussions of fatally-flawed papers that have been published in top, top journals such as the American Economic Review or the Journal of Personality and Social Psychology or the American Sociological Review or the tabloids . And we also know about mistakes that make their way into mid-ranking outlets such as the Journal of Theoretical Biology. But what about results that appear in the lower tier of legitimate journals? I was thinking about this after reading a post by Dan Kahan slamming a paper that recently appeared in PLOS-One. I won’t discuss the paper itself here because that’s not my point. Rather, I had some thoughts regarding Kahan’s annoyance that a paper with fatal errors was published at all. I commented as follows: Read between the lines. The paper originally was released in 2009 and was published in 2013 in PLOS-One, which is one step above appearing on Arxiv. PLOS-One publishes some good things (so does Arxiv) but it’s the place
3 0.89953476 1122 andrew gelman stats-2012-01-16-“Groundbreaking or Definitive? Journals Need to Pick One”
Introduction: Sanjay Srivastava writes : As long as a journal pursues a strategy of publishing “wow” studies, it will inevitably contain more unreplicable findings and unsupportable conclusions than equally rigorous but more “boring” journals. Groundbreaking will always be higher-risk. And definitive will be the territory of journals that publish meta-analyses and reviews. . . . Most conclusions, even those in peer-reviewed papers in rigorous journals, should be regarded as tentative at best; but press releases and other public communication rarely convey that. . . . His message to all of us: Our standard response to a paper in Science, Nature, or Psychological Science should be “wow, that’ll be really interesting if it replicates.” And in our teaching and our engagement with the press and public, we need to make clear why that is the most enthusiastic response we can justify.
4 0.85170174 834 andrew gelman stats-2011-08-01-I owe it all to the haters
Introduction: Sometimes when I submit an article to a journal it is accepted right away or with minor alterations. But many of my favorite articles were rejected or had to go through an exhausting series of revisions. For example, this influential article had a very hostile referee and we had to seriously push the journal editor to accept it. This one was rejected by one or two journals before finally appearing with discussion. This paper was rejected by the American Political Science Review with no chance of revision and we had to publish it in the British Journal of Political Science, which was a bit odd given that the article was 100% about American politics. And when I submitted this instant classic (actually at the invitation of the editor), the referees found it to be trivial, and the editor did me the favor of publishing it but only by officially labeling it as a discussion of another article that appeared in the same issue. Some of my most influential papers were accepted right
Introduction: The other day we discussed that paper on ovulation and voting (you may recall that the authors reported a scattered bunch of comparisons, significance tests, and p-values, and I recommended that they would’ve done better to simply report complete summaries of their data, so that readers could see the comparisons of interest in full context), and I was thinking a bit more about why I was so bothered that it was published in Psychological Science, which I’d thought of as a serious research journal. My concern isn’t just that that the paper is bad—after all, lots of bad papers get published—but rather that it had nothing really going for it, except that it was headline bait. It was a survey done on Mechanical Turk, that’s it. No clever design, no clever questions, no care in dealing with nonresponse problems, no innovative data analysis, no nothing. The paper had nothing to offer, except that it had no obvious flaws. Psychology is a huge field full of brilliant researchers.
6 0.83706695 1291 andrew gelman stats-2012-04-30-Systematic review of publication bias in studies on publication bias
7 0.83556503 1393 andrew gelman stats-2012-06-26-The reverse-journal-submission system
8 0.82828099 2233 andrew gelman stats-2014-03-04-Literal vs. rhetorical
9 0.82703716 2353 andrew gelman stats-2014-05-30-I posted this as a comment on a sociology blog
10 0.82378787 371 andrew gelman stats-2010-10-26-Musical chairs in econ journals
11 0.81895888 1118 andrew gelman stats-2012-01-14-A model rejection letter
12 0.81632662 883 andrew gelman stats-2011-09-01-Arrow’s theorem update
14 0.79928941 1429 andrew gelman stats-2012-07-26-Our broken scholarly publishing system
16 0.79553455 1137 andrew gelman stats-2012-01-24-Difficulties in publishing non-replications of implausible findings
17 0.79546756 902 andrew gelman stats-2011-09-12-The importance of style in academic writing
18 0.78942263 1435 andrew gelman stats-2012-07-30-Retracted articles and unethical behavior in economics journals?
19 0.78195626 1774 andrew gelman stats-2013-03-22-Likelihood Ratio ≠ 1 Journal
20 0.77561682 762 andrew gelman stats-2011-06-13-How should journals handle replication studies?
topicId topicWeight
[(2, 0.037), (9, 0.045), (13, 0.017), (15, 0.081), (16, 0.054), (24, 0.103), (42, 0.019), (59, 0.046), (65, 0.015), (81, 0.192), (86, 0.01), (90, 0.019), (99, 0.202)]
simIndex simValue blogId blogTitle
1 0.91961926 915 andrew gelman stats-2011-09-17-(Worst) graph of the year
Introduction: This (forwarded to me from Jeff, from a powerpoint by Willam Gawthrop) wins not on form but on content: Really this graph should stand alone but it’s so wonderful that I can’t resist pointing out a few things: - The gap between 610 and 622 A.D. seems to be about the same as the previous 600 years, and only a little less than the 1400 years before that. - “Pious and devout” Jews are portrayed as having steadily increased in nonviolence up to the present day. Been to Israel lately? - I assume the line labeled “Bible” is referring to Christians? I’m sort of amazed to see pious and devout Christians listed as being maximally violent at the beginning. Huh? I thought Christ was supposed to be a nonviolent, mellow dude. The line starts at 3 B.C., implying that baby Jesus was at the extreme of violence. Gong forward, we can learn from the graph that pious and devout Christians in 1492 or 1618, say, were much more peaceful than Jesus and his crew. - Most amusingly g
Introduction: I just want to share with you the best comment we’ve every had in the nearly ten-year history of this blog. Also it has statistical content! Here’s the story. After seeing an amusing article by Tom Scocca relating how reporter John Lee Anderson called someone as a “little twerp” on twitter: I conjectured that Anderson suffered from “tall person syndrome,” that problem that some people of above-average height have, that they think they’re more important than other people because they literally look down on them. But I had no idea of Anderson’s actual height. Commenter Gary responded with this impressive bit of investigative reporting: Based on this picture: he appears to be fairly tall. But the perspective makes it hard to judge. Based on this picture: he appears to be about 9-10 inches taller than Catalina Garcia. But how tall is Catalina Garcia? Not that tall – she’s shorter than the high-wire artist Phillipe Petit: And he doesn’t appear
same-blog 3 0.91377562 1321 andrew gelman stats-2012-05-15-A statistical research project: Weeding out the fraudulent citations
Introduction: John Mashey points me to a blog post by Phil Davis on “the emergence of a citation cartel.” Davis tells the story: Cell Transplantation is a medical journal published by the Cognizant Communication Corporation of Putnam Valley, New York. In recent years, its impact factor has been growing rapidly. In 2006, it was 3.482 [I think he means "3.5"---ed.]. In 2010, it had almost doubled to 6.204. When you look at which journals cite Cell Transplantation, two journals stand out noticeably: the Medical Science Monitor, and The Scientific World Journal. According to the JCR, neither of these journals cited Cell Transplantation until 2010. Then, in 2010, a review article was published in the Medical Science Monitor citing 490 articles, 445 of which were to papers published in Cell Transplantation. All 445 citations pointed to papers published in 2008 or 2009 — the citation window from which the journal’s 2010 impact factor was derived. Of the remaining 45 citations, 44 cited the Me
4 0.90550125 552 andrew gelman stats-2011-02-03-Model Makers’ Hippocratic Oath
Introduction: Emanuel Derman and Paul Wilmott wonder how to get their fellow modelers to give up their fantasy of perfection. In a Business Week article they proposed, not entirely in jest, a model makers’ Hippocratic Oath: I will remember that I didn’t make the world and that it doesn’t satisfy my equations. Though I will use models boldly to estimate value, I will not be overly impressed by mathematics. I will never sacrifice reality for elegance without explaining why I have done so. Nor will I give the people who use my model false comfort about its accuracy. Instead, I will make explicit its assumptions and oversights. I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension. Found via Abductive Intelligence .
5 0.90230906 1632 andrew gelman stats-2012-12-20-Who exactly are those silly academics who aren’t as smart as a Vegas bookie?
Introduction: I get suspicious when I hear unsourced claims that unnamed experts somewhere are making foolish statements. For example, I recently came across this, from a Super Bowl-themed article from 2006 by Stephen Dubner and Steven Levitt: As it happens, there is one betting strategy that will routinely beat a bookie, and you don’t even have to be smart to use it. One of the most undervalued N.F.L. bets is the home underdog — a team favored to lose but playing in its home stadium. If you had bet $5,000 on the home underdog in every N.F.L. game over the past two decades, you would be up about $150,000 by now (a winning rate of roughly 53 percent). So far, so good. I wonder if this pattern still holds. But then Dubner and Levitt continue: This fact has led some academics to conclude that bookmakers simply aren’t very smart. If an academic researcher can find this loophole, shouldn’t a professional bookie be able to? But the fact is most bookies are doing just fine. So could it be
8 0.86239183 1033 andrew gelman stats-2011-11-28-Greece to head statistician: Tell the truth, go to jail
9 0.85147977 1222 andrew gelman stats-2012-03-20-5 books book
10 0.84884214 1705 andrew gelman stats-2013-02-04-Recently in the sister blog
11 0.84163451 1962 andrew gelman stats-2013-07-30-The Roy causal model?
12 0.83758438 1057 andrew gelman stats-2011-12-14-Hey—I didn’t know that!
14 0.83342493 858 andrew gelman stats-2011-08-17-Jumping off the edge of the world
15 0.83239877 658 andrew gelman stats-2011-04-11-Statistics in high schools: Towards more accessible conceptions of statistical inference
16 0.83042884 556 andrew gelman stats-2011-02-04-Patterns
17 0.79637891 2125 andrew gelman stats-2013-12-05-What predicts whether a school district will participate in a large-scale evaluation?
18 0.78243876 1759 andrew gelman stats-2013-03-12-How tall is Jon Lee Anderson?
19 0.77020615 2250 andrew gelman stats-2014-03-16-“I have no idea who Catalina Garcia is, but she makes a decent ruler”
20 0.76887369 2088 andrew gelman stats-2013-11-04-Recently in the sister blog