andrew_gelman_stats andrew_gelman_stats-2014 andrew_gelman_stats-2014-2191 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Mike Spagat points to this interview , which, he writes, covers themes that are discussed on the blog such as wrong ideas that don’t die, peer review and the statistics of conflict deaths. I agree. It’s good stuff. Here are some of the things that Spagat says (he’s being interviewed by Joel Wing): In fact, the standard excess-deaths concept leads to an interesting conundrum when combined with an interesting fact exposed in the next-to-latest Human Security Report ; in most countries child mortality rates decline during armed conflict (chapter 6). So if you believe the usual excess-death causality story then you’re forced to conclude that many conflicts actually save the lives of many children. Of course, the idea of wars savings lives is pretty hard to swallow. A much more sensible understanding is that there are a variety of factors that determine child deaths and that in many cases the factors that save the lives of children are stronger than the negative effects that confli
sentIndex sentText sentNum sentScore
1 Mike Spagat points to this interview , which, he writes, covers themes that are discussed on the blog such as wrong ideas that don’t die, peer review and the statistics of conflict deaths. [sent-1, score-0.792]
2 So if you believe the usual excess-death causality story then you’re forced to conclude that many conflicts actually save the lives of many children. [sent-5, score-0.162]
3 A much more sensible understanding is that there are a variety of factors that determine child deaths and that in many cases the factors that save the lives of children are stronger than the negative effects that conflict has on child mortality. [sent-7, score-0.712]
4 We say that if the war is causing non-violent death rates to increase then you would expect non-violent deaths to increase more in the violent parts of Iraq then they do in the non-violent parts of Iraq. [sent-11, score-0.468]
5 This should make us wonder whether there is any reality behind the excess deaths claims that have been based on this Iraq survey. [sent-14, score-0.232]
6 Here’s Spagat again: First of all, saying that something has to be right or is probably right because it has been peer reviewed is quite a weak defense. [sent-18, score-0.669]
7 Peer review is a good thing, and it is a strength of scientific journals that there is that level of scrutiny, but if you look at the list of scientific claims that have turned out to be wrong and that have been published in peer reviewed journals…. [sent-19, score-1.157]
8 Publishing in a peer reviewed journal is no guarantee that something is right. [sent-21, score-0.746]
9 Mostly people just assume what they’re being told by the authors of the paper is correct and valid. [sent-25, score-0.137]
10 Peer review is better than no peer review, but it hardly guarantees that something is going to be correct. [sent-26, score-0.682]
11 Journal peer review is just the beginning of a long peer review process. [sent-30, score-1.364]
12 Thinking that journal peer review is the end of this process is a serious misunderstanding. [sent-31, score-0.828]
13 there have been a number of peer reviewed articles that have critiqued it, and said it is wrong. [sent-37, score-0.669]
14 So if you think peer review has to always be correct then you’re immediately in a logical conundrum because you’ve got peer reviewed articles saying opposite things. [sent-38, score-1.536]
15 And I’m happy to admire the courage of people who do dangerous field work. [sent-41, score-0.242]
16 But doing courageous field work doesn’t make your findings correct and we shouldn’t accept false claims just because someone had the guts to go out in the field and gather data. [sent-42, score-0.442]
17 And I pointed to this blog post from 2006, where I wrote some pretty general comments about cluster sampling. [sent-51, score-0.143]
18 For example, they should be able to compute the probability of selection of each household (based on the selection of province, administrative unit, street, and household). [sent-55, score-0.191]
19 Unfortunately, it is a common problem in research reports in general: to lack details on exact procedures it’s surprisingly difficult for people to simply describe exactly what they did. [sent-57, score-0.294]
20 (I’m always telling this to students when they write up their own research: Just say exactly what you did, and you’ll be mostly there. [sent-58, score-0.148]
wordName wordTfidf (topN-words)
[('peer', 0.481), ('spagat', 0.298), ('iraq', 0.219), ('review', 0.201), ('reviewed', 0.188), ('burnham', 0.153), ('deaths', 0.12), ('conundrum', 0.114), ('claims', 0.112), ('conflict', 0.11), ('death', 0.107), ('courage', 0.105), ('child', 0.105), ('lives', 0.094), ('exactly', 0.093), ('mortality', 0.085), ('household', 0.081), ('cluster', 0.081), ('unfortunately', 0.08), ('surprisingly', 0.078), ('journal', 0.077), ('correct', 0.071), ('field', 0.071), ('serious', 0.069), ('save', 0.068), ('people', 0.066), ('war', 0.064), ('detail', 0.062), ('general', 0.062), ('parts', 0.061), ('surveys', 0.061), ('scientific', 0.06), ('doocy', 0.06), ('lafta', 0.06), ('courageous', 0.06), ('shannon', 0.06), ('fact', 0.059), ('conscientious', 0.057), ('grabs', 0.057), ('guts', 0.057), ('describe', 0.057), ('mostly', 0.055), ('selection', 0.055), ('factors', 0.055), ('journals', 0.055), ('rates', 0.055), ('gilbert', 0.054), ('refereeing', 0.054), ('scrutiny', 0.054), ('invasion', 0.053)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 2191 andrew gelman stats-2014-01-29-“Questioning The Lancet, PLOS, And Other Surveys On Iraqi Deaths, An Interview With Univ. of London Professor Michael Spagat”
Introduction: Mike Spagat points to this interview , which, he writes, covers themes that are discussed on the blog such as wrong ideas that don’t die, peer review and the statistics of conflict deaths. I agree. It’s good stuff. Here are some of the things that Spagat says (he’s being interviewed by Joel Wing): In fact, the standard excess-deaths concept leads to an interesting conundrum when combined with an interesting fact exposed in the next-to-latest Human Security Report ; in most countries child mortality rates decline during armed conflict (chapter 6). So if you believe the usual excess-death causality story then you’re forced to conclude that many conflicts actually save the lives of many children. Of course, the idea of wars savings lives is pretty hard to swallow. A much more sensible understanding is that there are a variety of factors that determine child deaths and that in many cases the factors that save the lives of children are stronger than the negative effects that confli
2 0.28110954 5 andrew gelman stats-2010-04-27-Ethical and data-integrity problems in a study of mortality in Iraq
Introduction: Michael Spagat notifies me that his article criticizing the 2006 study of Burnham, Lafta, Doocy and Roberts has just been published . The Burnham et al. paper (also called, to my irritation (see the last item here ), “the Lancet survey”) used a cluster sample to estimate the number of deaths in Iraq in the three years following the 2003 invasion. In his newly-published paper, Spagat writes: [The Spagat article] presents some evidence suggesting ethical violations to the survey’s respondents including endangerment, privacy breaches and violations in obtaining informed consent. Breaches of minimal disclosure standards examined include non-disclosure of the survey’s questionnaire, data-entry form, data matching anonymised interviewer identifications with households and sample design. The paper also presents some evidence relating to data fabrication and falsification, which falls into nine broad categories. This evidence suggests that this survey cannot be considered a reliable or
Introduction: Mike Spagat sends in an interesting explanation for the noted problems with conflict mortality studies (a topic we’ve discussed on occasion on this blog). Spagat writes: This analysis is based on the fact that conflict violence does not spread out at all uniformly across a map but, rather, tends to concentrate in a few areas. This means that small, headline-grabbing violence surveys are extremely unreliable. There is a second point, based on the work of David Hemenway which you’ve also cited on your blog. Even within exceptionally violent environments most households will still not have a violent death. So a very small false positive rate in a household survey will cause substantial upward bias in violence estimates.
4 0.2187025 141 andrew gelman stats-2010-07-12-Dispute over counts of child deaths in Iraq due to sanctions
Introduction: Mike Spagat writes: Here is yet another debunking article I’ve written, this one in the latest issue of Significance. It shows the Lancet once again publishing spectacularly wrong information that has misinformed public discussion on a crucial issue with ongoing reverberations. For example, there is Tony Blair’s recent justification for the Iraq war offered in front of the Chilcot Inquiry in the UK quoted at the beginning of the paper. I haven’t had a chance to look at this one, but here’s a link to some related Spagat work.
5 0.21839099 2006 andrew gelman stats-2013-09-03-Evaluating evidence from published research
Introduction: Following up on my entry the other day on post-publication peer review, Dan Kahan writes: You give me credit, I think, for merely participating in what I think is a systemic effect in the practice of empirical inquiry that conduces to quality control & hence the advance of knowledge by such means (likely the title conveys that!). I’d say: (a) by far the greatest weakness in the “publication regime” in social sciences today is the systematic disregard for basic principles of valid causal inference, a deficiency either in comprehension or craft that is at the root of scholars’ resort to (and journals’ tolerance for) invalid samples, the employment of designs that don’t generate observations more consistent with a hypothesis than with myriad rival ones, and the resort to deficient statistical modes of analysis that treat detection of “statististically significant difference” rather than “practical corroboration of practical meaningful effect” as the goal of such analysis (especial
6 0.20375997 1798 andrew gelman stats-2013-04-11-Continuing conflict over conflict statistics
7 0.17369285 12 andrew gelman stats-2010-04-30-More on problems with surveys estimating deaths in war zones
9 0.15114711 1371 andrew gelman stats-2012-06-07-Question 28 of my final exam for Design and Analysis of Sample Surveys
10 0.13941829 1854 andrew gelman stats-2013-05-13-A Structural Comparison of Conspicuous Consumption in China and the United States
11 0.1379829 2245 andrew gelman stats-2014-03-12-More on publishing in journals
12 0.13768266 1139 andrew gelman stats-2012-01-26-Suggested resolution of the Bem paradox
13 0.13105801 2135 andrew gelman stats-2013-12-15-The UN Plot to Force Bayesianism on Unsuspecting Americans (penalized B-Spline edition)
14 0.12691101 2239 andrew gelman stats-2014-03-09-Reviewing the peer review process?
16 0.12198137 1928 andrew gelman stats-2013-07-06-How to think about papers published in low-grade journals?
17 0.12012452 1683 andrew gelman stats-2013-01-19-“Confirmation, on the other hand, is not sexy”
18 0.11931266 1435 andrew gelman stats-2012-07-30-Retracted articles and unethical behavior in economics journals?
topicId topicWeight
[(0, 0.233), (1, -0.074), (2, -0.026), (3, -0.12), (4, -0.043), (5, -0.027), (6, 0.022), (7, -0.03), (8, -0.005), (9, -0.029), (10, 0.054), (11, -0.044), (12, -0.026), (13, 0.045), (14, 0.017), (15, -0.02), (16, 0.042), (17, 0.051), (18, 0.014), (19, -0.051), (20, -0.01), (21, 0.006), (22, -0.062), (23, -0.007), (24, -0.011), (25, 0.039), (26, 0.017), (27, -0.006), (28, 0.04), (29, -0.004), (30, -0.031), (31, 0.047), (32, -0.009), (33, -0.008), (34, -0.04), (35, -0.008), (36, 0.057), (37, -0.007), (38, 0.045), (39, 0.017), (40, -0.011), (41, 0.0), (42, 0.012), (43, -0.03), (44, 0.03), (45, 0.003), (46, -0.036), (47, -0.018), (48, 0.044), (49, 0.016)]
simIndex simValue blogId blogTitle
same-blog 1 0.96233553 2191 andrew gelman stats-2014-01-29-“Questioning The Lancet, PLOS, And Other Surveys On Iraqi Deaths, An Interview With Univ. of London Professor Michael Spagat”
Introduction: Mike Spagat points to this interview , which, he writes, covers themes that are discussed on the blog such as wrong ideas that don’t die, peer review and the statistics of conflict deaths. I agree. It’s good stuff. Here are some of the things that Spagat says (he’s being interviewed by Joel Wing): In fact, the standard excess-deaths concept leads to an interesting conundrum when combined with an interesting fact exposed in the next-to-latest Human Security Report ; in most countries child mortality rates decline during armed conflict (chapter 6). So if you believe the usual excess-death causality story then you’re forced to conclude that many conflicts actually save the lives of many children. Of course, the idea of wars savings lives is pretty hard to swallow. A much more sensible understanding is that there are a variety of factors that determine child deaths and that in many cases the factors that save the lives of children are stronger than the negative effects that confli
2 0.78377312 2006 andrew gelman stats-2013-09-03-Evaluating evidence from published research
Introduction: Following up on my entry the other day on post-publication peer review, Dan Kahan writes: You give me credit, I think, for merely participating in what I think is a systemic effect in the practice of empirical inquiry that conduces to quality control & hence the advance of knowledge by such means (likely the title conveys that!). I’d say: (a) by far the greatest weakness in the “publication regime” in social sciences today is the systematic disregard for basic principles of valid causal inference, a deficiency either in comprehension or craft that is at the root of scholars’ resort to (and journals’ tolerance for) invalid samples, the employment of designs that don’t generate observations more consistent with a hypothesis than with myriad rival ones, and the resort to deficient statistical modes of analysis that treat detection of “statististically significant difference” rather than “practical corroboration of practical meaningful effect” as the goal of such analysis (especial
3 0.77340418 5 andrew gelman stats-2010-04-27-Ethical and data-integrity problems in a study of mortality in Iraq
Introduction: Michael Spagat notifies me that his article criticizing the 2006 study of Burnham, Lafta, Doocy and Roberts has just been published . The Burnham et al. paper (also called, to my irritation (see the last item here ), “the Lancet survey”) used a cluster sample to estimate the number of deaths in Iraq in the three years following the 2003 invasion. In his newly-published paper, Spagat writes: [The Spagat article] presents some evidence suggesting ethical violations to the survey’s respondents including endangerment, privacy breaches and violations in obtaining informed consent. Breaches of minimal disclosure standards examined include non-disclosure of the survey’s questionnaire, data-entry form, data matching anonymised interviewer identifications with households and sample design. The paper also presents some evidence relating to data fabrication and falsification, which falls into nine broad categories. This evidence suggests that this survey cannot be considered a reliable or
Introduction: Mike Spagat sends in an interesting explanation for the noted problems with conflict mortality studies (a topic we’ve discussed on occasion on this blog). Spagat writes: This analysis is based on the fact that conflict violence does not spread out at all uniformly across a map but, rather, tends to concentrate in a few areas. This means that small, headline-grabbing violence surveys are extremely unreliable. There is a second point, based on the work of David Hemenway which you’ve also cited on your blog. Even within exceptionally violent environments most households will still not have a violent death. So a very small false positive rate in a household survey will cause substantial upward bias in violence estimates.
5 0.74948305 2137 andrew gelman stats-2013-12-17-Replication backlash
Introduction: Raghuveer Parthasarathy pointed me to an article in Nature by Mina Bissell, who writes , “The push to replicate findings could shelve promising research and unfairly damage the reputations of careful, meticulous scientists.” I can see where she’s coming from: if you work hard day after day in the lab, it’s gotta be a bit frustrating to find all your work questioned, for the frauds of the Dr. Anil Pottis and Diederik Stapels to be treated as a reason for everyone else’s work to be considered guilty until proven innocent. That said, I pretty much disagree with Bissell’s article, and really the best thing I can say about it is that I think it’s a good sign that the push for replication is so strong that now there’s a backlash against it. Traditionally, leading scientists have been able to simply ignore the push for replication. If they are feeling that the replication movement is strong enough that they need to fight it, that to me is good news. I’ll explain a bit in the conte
7 0.74336296 2220 andrew gelman stats-2014-02-22-Quickies
8 0.74277681 1435 andrew gelman stats-2012-07-30-Retracted articles and unethical behavior in economics journals?
9 0.74201 1139 andrew gelman stats-2012-01-26-Suggested resolution of the Bem paradox
10 0.73472536 1854 andrew gelman stats-2013-05-13-A Structural Comparison of Conspicuous Consumption in China and the United States
11 0.73305982 2177 andrew gelman stats-2014-01-19-“The British amateur who debunked the mathematics of happiness”
12 0.73008937 2269 andrew gelman stats-2014-03-27-Beyond the Valley of the Trolls
13 0.72913241 2004 andrew gelman stats-2013-09-01-Post-publication peer review: How it (sometimes) really works
14 0.72902495 1844 andrew gelman stats-2013-05-06-Against optimism about social science
15 0.72821027 2233 andrew gelman stats-2014-03-04-Literal vs. rhetorical
16 0.72769344 1878 andrew gelman stats-2013-05-31-How to fix the tabloids? Toward replicable social science research
17 0.72601771 1683 andrew gelman stats-2013-01-19-“Confirmation, on the other hand, is not sexy”
18 0.72590101 12 andrew gelman stats-2010-04-30-More on problems with surveys estimating deaths in war zones
19 0.72388071 945 andrew gelman stats-2011-10-06-W’man < W’pedia, again
20 0.72347242 1585 andrew gelman stats-2012-11-20-“I know you aren’t the plagiarism police, but . . .”
topicId topicWeight
[(9, 0.037), (15, 0.122), (16, 0.086), (20, 0.01), (21, 0.031), (24, 0.123), (26, 0.037), (27, 0.02), (37, 0.033), (45, 0.025), (52, 0.011), (61, 0.011), (72, 0.026), (95, 0.056), (99, 0.275)]
simIndex simValue blogId blogTitle
same-blog 1 0.96723562 2191 andrew gelman stats-2014-01-29-“Questioning The Lancet, PLOS, And Other Surveys On Iraqi Deaths, An Interview With Univ. of London Professor Michael Spagat”
Introduction: Mike Spagat points to this interview , which, he writes, covers themes that are discussed on the blog such as wrong ideas that don’t die, peer review and the statistics of conflict deaths. I agree. It’s good stuff. Here are some of the things that Spagat says (he’s being interviewed by Joel Wing): In fact, the standard excess-deaths concept leads to an interesting conundrum when combined with an interesting fact exposed in the next-to-latest Human Security Report ; in most countries child mortality rates decline during armed conflict (chapter 6). So if you believe the usual excess-death causality story then you’re forced to conclude that many conflicts actually save the lives of many children. Of course, the idea of wars savings lives is pretty hard to swallow. A much more sensible understanding is that there are a variety of factors that determine child deaths and that in many cases the factors that save the lives of children are stronger than the negative effects that confli
2 0.95635891 945 andrew gelman stats-2011-10-06-W’man < W’pedia, again
Introduction: Blogger Deep Climate looks at another paper by the 2002 recipient of the American Statistical Association’s Founders award. This time it’s not funny, it’s just sad. Here’s Wikipedia on simulated annealing: By analogy with this physical process, each step of the SA algorithm replaces the current solution by a random “nearby” solution, chosen with a probability that depends on the difference between the corresponding function values and on a global parameter T (called the temperature), that is gradually decreased during the process. The dependency is such that the current solution changes almost randomly when T is large, but increasingly “downhill” as T goes to zero. The allowance for “uphill” moves saves the method from becoming stuck at local minima—which are the bane of greedier methods. And here’s Wegman: During each step of the algorithm, the variable that will eventually represent the minimum is replaced by a random solution that is chosen according to a temperature
3 0.95363784 1774 andrew gelman stats-2013-03-22-Likelihood Ratio ≠ 1 Journal
Introduction: Dan Kahan writes : The basic idea . . . is to promote identification of study designs that scholars who disagree about a proposition would agree would generate evidence relevant to their competing conjectures—regardless of what studies based on such designs actually find. Articles proposing designs of this sort would be selected for publication and only then be carried out, by the proposing researchers with funding from the journal, which would publish the results too. Now I [Kahan] am aware of a set of real journals that have a similar motivation. One is the Journal of Articles in Support of the Null Hypothesis, which as its title implies publishes papers reporting studies that fail to “reject” the null. Like JASNH, LR ≠1J would try to offset the “file drawer” bias and like bad consequences associated with the convention of publishing only findings that are “significant at p < 0.05." But it would try to do more. By publishing studies that are deemed to have valid designs an
4 0.95342207 2353 andrew gelman stats-2014-05-30-I posted this as a comment on a sociology blog
Introduction: I discussed two problems: 1. An artificial scarcity applied to journal publication, a scarcity which I believe is being enforced based on a monetary principle of not wanting to reduce the value of publication. The problem is that journals don’t just spread information and improve communication, they also represent chits for hiring and promotion. I’d prefer to separate these two aspects of publication. To keep these functions tied together seems to me like a terrible mistake. It would be as if, instead of using dollar bills as currency, we were to just use paper , and then if the government kept paper artificially scarce to retain the value of money, so that we were reduced to scratching notes to each other on walls and tables. 2. The discontinuous way in which unpublished papers and submissions to journals are taken as highly suspect and requiring a strong justification of all methods and assumptions, but once a paper becomes published its conclusions are taken as true unless
5 0.94880497 1779 andrew gelman stats-2013-03-27-“Two Dogmas of Strong Objective Bayesianism”
Introduction: Prasanta Bandyopadhyay and Gordon Brittan write : We introduce a distinction, unnoticed in the literature, between four varieties of objective Bayesianism. What we call ‘strong objective Bayesianism’ is characterized by two claims, that all scientific inference is ‘logical’ and that, given the same background information two agents will ascribe a unique probability to their priors. We think that neither of these claims can be sustained; in this sense, they are ‘dogmatic’. The first fails to recognize that some scientific inference, in particular that concerning evidential relations, is not (in the appropriate sense) logical, the second fails to provide a non-question-begging account of ‘same background information’. We urge that a suitably objective Bayesian account of scientific inference does not require either of the claims. Finally, we argue that Bayesianism needs to be fine-grained in the same way that Bayesians fine-grain their beliefs. I have not read their paper in detai
6 0.94831467 1541 andrew gelman stats-2012-10-19-Statistical discrimination again
8 0.9468075 902 andrew gelman stats-2011-09-12-The importance of style in academic writing
10 0.94398558 1908 andrew gelman stats-2013-06-21-Interpreting interactions in discrete-data regression
11 0.94370568 1998 andrew gelman stats-2013-08-25-A new Bem theory
13 0.94325018 133 andrew gelman stats-2010-07-08-Gratuitous use of “Bayesian Statistics,” a branding issue?
16 0.93997347 675 andrew gelman stats-2011-04-22-Arrow’s other theorem
17 0.9386282 2177 andrew gelman stats-2014-01-19-“The British amateur who debunked the mathematics of happiness”
18 0.93542629 1435 andrew gelman stats-2012-07-30-Retracted articles and unethical behavior in economics journals?
19 0.93522906 2227 andrew gelman stats-2014-02-27-“What Can we Learn from the Many Labs Replication Project?”
20 0.93403578 1683 andrew gelman stats-2013-01-19-“Confirmation, on the other hand, is not sexy”