andrew_gelman_stats andrew_gelman_stats-2014 andrew_gelman_stats-2014-2269 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: In a further discussion of the discussion about the discussion of a paper in Administrative Science Quarterly, Thomas Basbøll writes: I [Basbøll] feel “entitled”, if that’s the right word (actually, I’d say I feel privileged), to express my opinions to anyone who wants to listen, and while I think it does say something about an author whether or not they answer a question (where what it says depends very much on the quality of the question), I don’t think the author has any obligation to me to respond immediately. If I succeed in raising doubts about something in the minds of many readers, then that’s obviously something an author should take seriously. The point is that an author has a responsibility to the readership of the paper, not any one critic. I agree that the ultimate audience is the scholarly community (and, beyond that, the general public) and that the critic is just serving as a conduit, the person who poses the Q in the Q-and-A. That said, I get frustrated frust
sentIndex sentText sentNum sentScore
1 If I succeed in raising doubts about something in the minds of many readers, then that’s obviously something an author should take seriously. [sent-2, score-0.221]
2 The point is that an author has a responsibility to the readership of the paper, not any one critic. [sent-3, score-0.151]
3 I agree that the ultimate audience is the scholarly community (and, beyond that, the general public) and that the critic is just serving as a conduit, the person who poses the Q in the Q-and-A. [sent-4, score-0.252]
4 That said, I get frustrated frustrated frustrated when people don’t respond to questions and criticisms from me. [sent-5, score-1.05]
5 Perhaps most unreasonable (from my part) is that after I slammed Gregg Easterbrook for an incompetent political column, his editors at Reuters made most of the corrections I suggested (including the links I’d supplied) but without acknowledging me in any way. [sent-7, score-0.299]
6 I still think I was in the right here (it’s only common courtesy to acknowledge the source of your information), but ultimately the information got out there. [sent-8, score-0.149]
7 I got more frustrated after Arthur Brooks garbled some poll data on happiness and David Brooks mainstreamed some anti-Semitic fake stats, with both these errors coming on the New York Times op-ed page. [sent-9, score-0.631]
8 What frustrated me was not the erroneous numbers—after all, I make mistakes too! [sent-10, score-0.394]
9 (Indeed, in neither case was I the one to the discover the mistake; it was other people who pointed these cases out to me, and then I explored them further. [sent-12, score-0.15]
10 In David Brooks’s case in particular, I hate to see him take the dark path of disseminating what is at best sloppy research and what is at worst disinformation, just because it happens to align in some ways with his political views. [sent-14, score-0.163]
11 Or take the case Basbøll is talking about, a paper that recently appeared in a journal of business management. [sent-15, score-0.297]
12 The paper got some favorable publicity, the editor of the journal promoted the paper in a blog, and then the paper came in for some serious criticism. [sent-16, score-1.031]
13 The consensus (at least to me) seems to be that the paper is reasonable if a bit overstated in its conclusions, not quite as good as the journal editor claimed but making a real, if specific, contribution to the literature. [sent-17, score-0.437]
14 ” My point: the “trolling” seemed to be necessary to move the scholarly and scientific discussion forward. [sent-19, score-0.352]
15 Had everyone been super-polite, the journal editor could’ve just remained in a state of complacency about the paper, but instead the strong comments motivated him and the authors of the paper to respond. [sent-20, score-0.437]
16 For example, I’ve occasionally written about a sociologist who’s written a series of papers about sex roles and sex ratios of babies. [sent-22, score-0.27]
17 He makes provocative claims based on very weak data and gets lots of attention making “politically incorrect” statements which would be unremarkable if overheard at the local bar or country club but get attention because of their purported scientific basis. [sent-24, score-0.599]
18 I think they too often represent trolling, of a sort, getting attention based on politically loaded claims which are basically unsupported by data. [sent-26, score-0.422]
19 Just to be clear, I wouldn’t label the administrative science paper discussed above as trolling. [sent-28, score-0.402]
20 I’m just pointing out that, in our discussion of flaws in published papers, we are in many ways living in a world whose parameters are set by scientific publishing and the news media, a world in which the most prestigious scientific journals are known as “the tabloids. [sent-31, score-0.437]
wordName wordTfidf (topN-words)
[('frustrated', 0.321), ('paper', 0.184), ('brooks', 0.165), ('basb', 0.161), ('trolling', 0.154), ('author', 0.151), ('editor', 0.14), ('administrative', 0.132), ('discussion', 0.13), ('unsupported', 0.128), ('attention', 0.123), ('corrections', 0.122), ('scientific', 0.118), ('journal', 0.113), ('scholarly', 0.104), ('politically', 0.1), ('editors', 0.097), ('sex', 0.09), ('papers', 0.09), ('mainstreamed', 0.088), ('tabletop', 0.088), ('disseminating', 0.088), ('respond', 0.087), ('science', 0.086), ('overheard', 0.083), ('incompetent', 0.08), ('got', 0.078), ('poses', 0.077), ('explored', 0.077), ('unremarkable', 0.077), ('provocative', 0.075), ('align', 0.075), ('promoted', 0.075), ('symmetry', 0.075), ('landscape', 0.075), ('privileged', 0.073), ('erroneous', 0.073), ('favorable', 0.073), ('reuters', 0.073), ('garbled', 0.073), ('trolls', 0.073), ('pointed', 0.073), ('arthur', 0.071), ('loaded', 0.071), ('courtesy', 0.071), ('critic', 0.071), ('quarterly', 0.071), ('prestigious', 0.071), ('errors', 0.071), ('doubts', 0.07)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999988 2269 andrew gelman stats-2014-03-27-Beyond the Valley of the Trolls
Introduction: In a further discussion of the discussion about the discussion of a paper in Administrative Science Quarterly, Thomas Basbøll writes: I [Basbøll] feel “entitled”, if that’s the right word (actually, I’d say I feel privileged), to express my opinions to anyone who wants to listen, and while I think it does say something about an author whether or not they answer a question (where what it says depends very much on the quality of the question), I don’t think the author has any obligation to me to respond immediately. If I succeed in raising doubts about something in the minds of many readers, then that’s obviously something an author should take seriously. The point is that an author has a responsibility to the readership of the paper, not any one critic. I agree that the ultimate audience is the scholarly community (and, beyond that, the general public) and that the critic is just serving as a conduit, the person who poses the Q in the Q-and-A. That said, I get frustrated frust
2 0.21446489 2245 andrew gelman stats-2014-03-12-More on publishing in journals
Introduction: I’m postponing today’s scheduled post (“Empirical implications of Empirical Implications of Theoretical Models”) to continue the lively discussion from yesterday, What if I were to stop publishing in journals? . An example: my papers with Basbøll Thomas Basbøll and I got into a long discussion on our blogs about business school professor Karl Weick and other cases of plagiarism copying text without attribution. We felt it useful to take our ideas to the next level and write them up as a manuscript, which ended up being logical to split into two papers. At that point I put some effort into getting these papers published, which I eventually did: To throw away data: Plagiarism as a statistical crime went into American Scientist and When do stories work? Evidence and illustration in the social sciences will appear in Sociological Methods and Research. The second paper, in particular, took some effort to place; I got some advice from colleagues in sociology as to where
Introduction: The other day we discussed that paper on ovulation and voting (you may recall that the authors reported a scattered bunch of comparisons, significance tests, and p-values, and I recommended that they would’ve done better to simply report complete summaries of their data, so that readers could see the comparisons of interest in full context), and I was thinking a bit more about why I was so bothered that it was published in Psychological Science, which I’d thought of as a serious research journal. My concern isn’t just that that the paper is bad—after all, lots of bad papers get published—but rather that it had nothing really going for it, except that it was headline bait. It was a survey done on Mechanical Turk, that’s it. No clever design, no clever questions, no care in dealing with nonresponse problems, no innovative data analysis, no nothing. The paper had nothing to offer, except that it had no obvious flaws. Psychology is a huge field full of brilliant researchers.
4 0.18687254 2233 andrew gelman stats-2014-03-04-Literal vs. rhetorical
Introduction: Thomas Basbøll pointed me to a discussion on the orgtheory blog in which Jerry Davis, the editor of a journal of business management argued that it is difficult for academic researchers to communicate with the public because “the public prefers Cheetos to a healthy salad” and when serious papers are discussed on the internet, “everyone is a methodologist.” The discussion heated up when an actual methodologist, Steve Morgan, joined in to argue that the salad in question was not so healthy and that the much-derided internet commenters made some valuable points. The final twist was that one of the orgtheory bloggers deleted a comment and then closed the thread entirely when the discussion got too conflictual. In a few days I’ll return to the meta-topic of the discussion, but right now I want to focus on one thing Davis wrote, a particular statement that illustrates to me the gap between the rhetorical and the literal, the way in which a statement can sound good but make no sense. He
5 0.17232054 1928 andrew gelman stats-2013-07-06-How to think about papers published in low-grade journals?
Introduction: We’ve had lots of lively discussions of fatally-flawed papers that have been published in top, top journals such as the American Economic Review or the Journal of Personality and Social Psychology or the American Sociological Review or the tabloids . And we also know about mistakes that make their way into mid-ranking outlets such as the Journal of Theoretical Biology. But what about results that appear in the lower tier of legitimate journals? I was thinking about this after reading a post by Dan Kahan slamming a paper that recently appeared in PLOS-One. I won’t discuss the paper itself here because that’s not my point. Rather, I had some thoughts regarding Kahan’s annoyance that a paper with fatal errors was published at all. I commented as follows: Read between the lines. The paper originally was released in 2009 and was published in 2013 in PLOS-One, which is one step above appearing on Arxiv. PLOS-One publishes some good things (so does Arxiv) but it’s the place
6 0.17075098 1844 andrew gelman stats-2013-05-06-Against optimism about social science
8 0.16188627 1878 andrew gelman stats-2013-05-31-How to fix the tabloids? Toward replicable social science research
9 0.16103613 1588 andrew gelman stats-2012-11-23-No one knows what it’s like to be the bad man
10 0.15954246 1139 andrew gelman stats-2012-01-26-Suggested resolution of the Bem paradox
11 0.14828271 1435 andrew gelman stats-2012-07-30-Retracted articles and unethical behavior in economics journals?
12 0.14728932 2232 andrew gelman stats-2014-03-03-What is the appropriate time scale for blogging—the day or the week?
13 0.14455843 2244 andrew gelman stats-2014-03-11-What if I were to stop publishing in journals?
14 0.14364095 2006 andrew gelman stats-2013-09-03-Evaluating evidence from published research
15 0.14266762 1269 andrew gelman stats-2012-04-19-Believe your models (up to the point that you abandon them)
18 0.13909198 2280 andrew gelman stats-2014-04-03-As the boldest experiment in journalism history, you admit you made a mistake
19 0.13822852 1695 andrew gelman stats-2013-01-28-Economists argue about Bayes
20 0.13353562 834 andrew gelman stats-2011-08-01-I owe it all to the haters
topicId topicWeight
[(0, 0.265), (1, -0.103), (2, -0.063), (3, -0.08), (4, -0.095), (5, -0.095), (6, 0.03), (7, -0.1), (8, -0.011), (9, -0.005), (10, 0.085), (11, 0.032), (12, -0.09), (13, 0.063), (14, -0.019), (15, -0.014), (16, -0.042), (17, 0.011), (18, -0.011), (19, 0.001), (20, -0.008), (21, -0.019), (22, -0.001), (23, -0.028), (24, -0.037), (25, 0.046), (26, 0.013), (27, 0.006), (28, 0.005), (29, 0.026), (30, 0.025), (31, -0.016), (32, -0.017), (33, 0.002), (34, 0.022), (35, 0.014), (36, 0.006), (37, 0.006), (38, -0.015), (39, 0.004), (40, 0.013), (41, 0.021), (42, -0.013), (43, -0.031), (44, -0.072), (45, 0.012), (46, 0.004), (47, -0.025), (48, 0.009), (49, 0.002)]
simIndex simValue blogId blogTitle
same-blog 1 0.9816938 2269 andrew gelman stats-2014-03-27-Beyond the Valley of the Trolls
Introduction: In a further discussion of the discussion about the discussion of a paper in Administrative Science Quarterly, Thomas Basbøll writes: I [Basbøll] feel “entitled”, if that’s the right word (actually, I’d say I feel privileged), to express my opinions to anyone who wants to listen, and while I think it does say something about an author whether or not they answer a question (where what it says depends very much on the quality of the question), I don’t think the author has any obligation to me to respond immediately. If I succeed in raising doubts about something in the minds of many readers, then that’s obviously something an author should take seriously. The point is that an author has a responsibility to the readership of the paper, not any one critic. I agree that the ultimate audience is the scholarly community (and, beyond that, the general public) and that the critic is just serving as a conduit, the person who poses the Q in the Q-and-A. That said, I get frustrated frust
Introduction: Jeff Leek points to a post by Alex Holcombe, who disputes the idea that science is self-correcting. Holcombe writes [scroll down to get to his part]: The pace of scientific production has quickened, and self-correction has suffered. Findings that might correct old results are considered less interesting than results from more original research questions. Potential corrections are also more contested. As the competition for space in prestigious journals has become increasingly frenzied, doing and publishing studies that would confirm the rapidly accumulating new discoveries, or would correct them, became a losing proposition. Holcombe picks up on some points that we’ve discussed a lot here in the past year. Here’s Holcombe: In certain subfields, almost all new work appears in only a very few journals, all associated with a single professional society. There is then no way around the senior gatekeepers, who may then suppress corrections with impunity. . . . The bias agai
3 0.86926687 2353 andrew gelman stats-2014-05-30-I posted this as a comment on a sociology blog
Introduction: I discussed two problems: 1. An artificial scarcity applied to journal publication, a scarcity which I believe is being enforced based on a monetary principle of not wanting to reduce the value of publication. The problem is that journals don’t just spread information and improve communication, they also represent chits for hiring and promotion. I’d prefer to separate these two aspects of publication. To keep these functions tied together seems to me like a terrible mistake. It would be as if, instead of using dollar bills as currency, we were to just use paper , and then if the government kept paper artificially scarce to retain the value of money, so that we were reduced to scratching notes to each other on walls and tables. 2. The discontinuous way in which unpublished papers and submissions to journals are taken as highly suspect and requiring a strong justification of all methods and assumptions, but once a paper becomes published its conclusions are taken as true unless
4 0.86589408 2245 andrew gelman stats-2014-03-12-More on publishing in journals
Introduction: I’m postponing today’s scheduled post (“Empirical implications of Empirical Implications of Theoretical Models”) to continue the lively discussion from yesterday, What if I were to stop publishing in journals? . An example: my papers with Basbøll Thomas Basbøll and I got into a long discussion on our blogs about business school professor Karl Weick and other cases of plagiarism copying text without attribution. We felt it useful to take our ideas to the next level and write them up as a manuscript, which ended up being logical to split into two papers. At that point I put some effort into getting these papers published, which I eventually did: To throw away data: Plagiarism as a statistical crime went into American Scientist and When do stories work? Evidence and illustration in the social sciences will appear in Sociological Methods and Research. The second paper, in particular, took some effort to place; I got some advice from colleagues in sociology as to where
5 0.86400735 2177 andrew gelman stats-2014-01-19-“The British amateur who debunked the mathematics of happiness”
Introduction: Andrew Anthony tells the excellent story of how Nick Brown, Alan Sokal, and Harris Friedman shot down some particularly silly work in psychology. (“According to the graph, it all came down to a specific ratio of positive emotions to negative emotions. If your ratio was greater than 2.9013 positive emotions to 1 negative emotion you were flourishing in life. If your ratio was less than that number you were languishing.” And, yes, the work they were shooting down really is that bad.) If you want to see what the fuss is about, just google “2.9013.” Here’s an example (from 2012) of an uncritical reporting of the claim, here’s another one from 2010, here’s one from 2011 . . . well, you get the idea. And here’s a quick summary posted by Rolf Zwaan after Brown et al. came out with their paper. I know Sokal and Brown and so this story was not news to me. I didn’t post anything about it on this blog because it seemed like it was getting enough coverage elsewhere. I think Ni
6 0.86279887 2233 andrew gelman stats-2014-03-04-Literal vs. rhetorical
7 0.85661942 1844 andrew gelman stats-2013-05-06-Against optimism about social science
8 0.85559785 834 andrew gelman stats-2011-08-01-I owe it all to the haters
9 0.8549974 1435 andrew gelman stats-2012-07-30-Retracted articles and unethical behavior in economics journals?
10 0.84712154 1139 andrew gelman stats-2012-01-26-Suggested resolution of the Bem paradox
12 0.83769184 675 andrew gelman stats-2011-04-22-Arrow’s other theorem
13 0.83386582 2006 andrew gelman stats-2013-09-03-Evaluating evidence from published research
14 0.83259439 1928 andrew gelman stats-2013-07-06-How to think about papers published in low-grade journals?
15 0.82988751 883 andrew gelman stats-2011-09-01-Arrow’s theorem update
16 0.82976788 2244 andrew gelman stats-2014-03-11-What if I were to stop publishing in journals?
17 0.8270908 902 andrew gelman stats-2011-09-12-The importance of style in academic writing
19 0.82174969 2137 andrew gelman stats-2013-12-17-Replication backlash
20 0.8149702 1878 andrew gelman stats-2013-05-31-How to fix the tabloids? Toward replicable social science research
topicId topicWeight
[(1, 0.014), (2, 0.03), (3, 0.015), (15, 0.051), (16, 0.118), (21, 0.03), (24, 0.086), (27, 0.014), (29, 0.011), (31, 0.017), (32, 0.02), (42, 0.043), (44, 0.012), (45, 0.054), (47, 0.014), (65, 0.035), (68, 0.042), (86, 0.01), (99, 0.304)]
simIndex simValue blogId blogTitle
same-blog 1 0.97353709 2269 andrew gelman stats-2014-03-27-Beyond the Valley of the Trolls
Introduction: In a further discussion of the discussion about the discussion of a paper in Administrative Science Quarterly, Thomas Basbøll writes: I [Basbøll] feel “entitled”, if that’s the right word (actually, I’d say I feel privileged), to express my opinions to anyone who wants to listen, and while I think it does say something about an author whether or not they answer a question (where what it says depends very much on the quality of the question), I don’t think the author has any obligation to me to respond immediately. If I succeed in raising doubts about something in the minds of many readers, then that’s obviously something an author should take seriously. The point is that an author has a responsibility to the readership of the paper, not any one critic. I agree that the ultimate audience is the scholarly community (and, beyond that, the general public) and that the critic is just serving as a conduit, the person who poses the Q in the Q-and-A. That said, I get frustrated frust
2 0.95909017 47 andrew gelman stats-2010-05-23-Of home runs and grand slams
Introduction: I’ve occasionally mocked academic economists for their discussions of research papers as “singles” or “home runs” (a world in which, counter to the findings of sabermetrics, one of the latter is worth more than four of the former). The best thing, of course, is a “grand slam,” a term that I always found particularly silly as it depends not just on the quality of the hit but also on external considerations (“men on base”). But then I was thinking about this again, and I decided the analogy isn’t so bad: For a research paper to be really influential and important, it has to come at the right time, and the field has to be ready for it. That’s what turns a home run into a grand slam. So in this case the reasoning works pretty well.
Introduction: The pre-NYT David Brooks liked to make fun of the NYT. Here’s one from 1997 : I’m not sure I’d like to be one of the people featured on the New York Times wedding page, but I know I’d like to be the father of one of them. Imagine how happy Stanley J. Kogan must have been, for example, when his daughter Jamie got into Yale. Then imagine his pride when Jamie made Phi Beta Kappa and graduated summa cum laude. . . . he must have enjoyed a gloat or two when his daughter put on that cap and gown. And things only got better. Jamie breezed through Stanford Law School. And then she met a man—Thomas Arena—who appears to be exactly the sort of son-in-law that pediatric urologists dream about. . . . These two awesome resumes collided at a wedding ceremony . . . It must have been one of the happiest days in Stanley J. Kogan’s life. The rest of us got to read about it on the New York Times wedding page. Brooks is reputed to be Jewish himself so I think it’s ok for him to mock Jewish peop
Introduction: Jeff Leek points to a post by Alex Holcombe, who disputes the idea that science is self-correcting. Holcombe writes [scroll down to get to his part]: The pace of scientific production has quickened, and self-correction has suffered. Findings that might correct old results are considered less interesting than results from more original research questions. Potential corrections are also more contested. As the competition for space in prestigious journals has become increasingly frenzied, doing and publishing studies that would confirm the rapidly accumulating new discoveries, or would correct them, became a losing proposition. Holcombe picks up on some points that we’ve discussed a lot here in the past year. Here’s Holcombe: In certain subfields, almost all new work appears in only a very few journals, all associated with a single professional society. There is then no way around the senior gatekeepers, who may then suppress corrections with impunity. . . . The bias agai
5 0.95167804 2301 andrew gelman stats-2014-04-22-Ticket to Baaaaarf
Introduction: A link from the comments here took me to the wonderfully named Barfblog and a report by Don Schaffner on some reporting. First, the background: A university in England issued a press release saying that “Food picked up just a few seconds after being dropped is less likely to contain bacteria than if it is left for longer periods of time . . . The findings suggest there may be some scientific basis to the ‘5 second rule’ – the urban myth about it being fine to eat food that has only had contact with the floor for five seconds or less. Although people have long followed the 5 second rule, until now it was unclear whether it actually helped.” According to the press release, the study was “undertaken by final year Biology students” and led by a professor of microbiology. The press release hit the big time, hitting NPR, Slate, Forbes, the Daily News, etc etc. Some typical headlines: “5-second rule backed up by science” — Atlanta Journal Constitution “Eating food off the floo
6 0.95164347 814 andrew gelman stats-2011-07-21-The powerful consumer?
8 0.9512558 1844 andrew gelman stats-2013-05-06-Against optimism about social science
9 0.95040274 746 andrew gelman stats-2011-06-05-An unexpected benefit of Arrow’s other theorem
10 0.95028979 1921 andrew gelman stats-2013-07-01-Going meta on Niall Ferguson
11 0.95014161 1385 andrew gelman stats-2012-06-20-Reconciling different claims about working-class voters
12 0.94992101 1878 andrew gelman stats-2013-05-31-How to fix the tabloids? Toward replicable social science research
13 0.94982183 2137 andrew gelman stats-2013-12-17-Replication backlash
14 0.94864565 943 andrew gelman stats-2011-10-04-Flip it around
17 0.94656646 666 andrew gelman stats-2011-04-18-American Beliefs about Economic Opportunity and Income Inequality
18 0.94615167 371 andrew gelman stats-2010-10-26-Musical chairs in econ journals
19 0.94607866 598 andrew gelman stats-2011-03-03-Is Harvard hurting poor kids by cutting tuition for the upper middle class?
20 0.94593441 1568 andrew gelman stats-2012-11-07-That last satisfaction at the end of the career