andrew_gelman_stats andrew_gelman_stats-2014 andrew_gelman_stats-2014-2328 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Sanjay Kaul writes: I am sure you must be aware of the recent controversy ignited by the 2013 American College of Cardiology/American Heart Association Cholesterol Treatment Guidelines that were released last month. They have been the subject of several newspaper articles and blogs , most of them missing the thrust of the guidelines. There is much to admire about these guidelines as they are more faithfully aligned with high-quality ‘actionable’ evidence than the 3 previous iterations. However, the controversy is focused on the performance of the risk calculator introduced for initiating treatment in individuals without established atherosclerotic disease or diabetes (so-called primary prevention cohort). The guidelines recommend statins for primary prevention in individuals who have a 10-year risk estimated to be 7.5%. The risk calculator was derived from population cohorts studied in the 1990s. The discrimination for predicting the risk of atherosclerotic cardiovascular events
sentIndex sentText sentNum sentScore
1 Sanjay Kaul writes: I am sure you must be aware of the recent controversy ignited by the 2013 American College of Cardiology/American Heart Association Cholesterol Treatment Guidelines that were released last month. [sent-1, score-0.105]
2 They have been the subject of several newspaper articles and blogs , most of them missing the thrust of the guidelines. [sent-2, score-0.081]
3 There is much to admire about these guidelines as they are more faithfully aligned with high-quality ‘actionable’ evidence than the 3 previous iterations. [sent-3, score-0.357]
4 However, the controversy is focused on the performance of the risk calculator introduced for initiating treatment in individuals without established atherosclerotic disease or diabetes (so-called primary prevention cohort). [sent-4, score-1.629]
5 The guidelines recommend statins for primary prevention in individuals who have a 10-year risk estimated to be 7. [sent-5, score-1.058]
6 The risk calculator was derived from population cohorts studied in the 1990s. [sent-7, score-0.997]
7 The discrimination for predicting the risk of atherosclerotic cardiovascular events defined as coronary heart disease deaths, myocardial infarctions and strokes is fair as measured by the area under the ROC curve (c index of 0. [sent-8, score-1.496]
8 [As an aside, very few risk models developed to predict risk of clinical events have c index >0. [sent-11, score-1.24]
9 However, when applied to recent population and randomized controlled trial cohorts, the risk calculator is limited by ‘miscalibration’ (overestimates risk by 75% to 150%). [sent-13, score-1.348]
10 This is of course understandable as the prediction cohort is quite different from the development cohort in terms of baseline risk (many subjects in the former were on statin treatment that modifies risk). [sent-14, score-1.36]
11 The single-event probability estimate (which is essentially what calibration characterizes) is referenced to the ‘state’ of the development cohort. [sent-15, score-0.393]
12 If the state is different (as in the prediction cohort), should it surprise anyone that the risk model miscalibrates risk? [sent-16, score-0.646]
13 So, the question is does calibration trump discrimination in risk prediction? [sent-17, score-0.952]
14 I have attached the Ridker and Cook paper published in Lancet and the response from the chairs of the guidelines ( Stone and Lloyd-Jones ) published in Lancet. [sent-19, score-0.389]
15 The original risk tool publication is also attached ( JACC paper ). [sent-20, score-0.624]
16 This looks interesting but now I’m feeling too overwhelmed to look at it in detail. [sent-22, score-0.065]
17 (It’s too bad that I read all sorts of crappy papers but then feel too busy to read the interesting stuff. [sent-23, score-0.055]
wordName wordTfidf (topN-words)
[('risk', 0.529), ('cohort', 0.238), ('calculator', 0.229), ('guidelines', 0.223), ('atherosclerotic', 0.178), ('cohorts', 0.178), ('calibration', 0.172), ('discrimination', 0.17), ('prevention', 0.141), ('rain', 0.133), ('prediction', 0.117), ('tomorrow', 0.112), ('patient', 0.107), ('heart', 0.106), ('controversy', 0.105), ('treatment', 0.103), ('disease', 0.098), ('index', 0.096), ('attached', 0.095), ('primary', 0.088), ('event', 0.088), ('events', 0.086), ('kaul', 0.081), ('trump', 0.081), ('diabetes', 0.081), ('coronary', 0.081), ('strokes', 0.081), ('thrust', 0.081), ('individuals', 0.077), ('development', 0.077), ('probability', 0.077), ('conundrum', 0.076), ('cholesterol', 0.076), ('roc', 0.073), ('cardiovascular', 0.071), ('faithfully', 0.071), ('chairs', 0.071), ('lancet', 0.068), ('referenced', 0.067), ('overwhelmed', 0.065), ('stone', 0.065), ('overestimates', 0.064), ('sanjay', 0.064), ('aligned', 0.063), ('population', 0.061), ('spectrum', 0.058), ('understandable', 0.058), ('weather', 0.058), ('characterizes', 0.055), ('crappy', 0.055)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 2328 andrew gelman stats-2014-05-10-What property is important in a risk prediction model? Discrimination or calibration?
Introduction: Sanjay Kaul writes: I am sure you must be aware of the recent controversy ignited by the 2013 American College of Cardiology/American Heart Association Cholesterol Treatment Guidelines that were released last month. They have been the subject of several newspaper articles and blogs , most of them missing the thrust of the guidelines. There is much to admire about these guidelines as they are more faithfully aligned with high-quality ‘actionable’ evidence than the 3 previous iterations. However, the controversy is focused on the performance of the risk calculator introduced for initiating treatment in individuals without established atherosclerotic disease or diabetes (so-called primary prevention cohort). The guidelines recommend statins for primary prevention in individuals who have a 10-year risk estimated to be 7.5%. The risk calculator was derived from population cohorts studied in the 1990s. The discrimination for predicting the risk of atherosclerotic cardiovascular events
2 0.22014438 819 andrew gelman stats-2011-07-24-Don’t idealize “risk aversion”
Introduction: Richard Thaler writes (click here and search on Thaler): Both risk and risk aversion are concepts that were once well defined, but are now in danger of becoming Aetherized [this is Thaler's term for adding free parameters to a model to make it work, thus destroying the purity and much of the value of the original model]. Stocks that earn surprisingly high returns are labeled as risky, because in the theory, excess returns must be accompanied by higher risk. If, inconveniently, the traditional measures of risk such as variance or covariance with the market are not high, then the Aetherists tell us there must be some other risk; we just don’t know what it is. Similarly, traditionally the concept of risk aversion was taken to be a primitive; each person had a parameter, gamma, that measured her degree of risk aversion. Now risk aversion is allowed to be time varying, and Aetherists can say with a straight face that the market crashes of 2001 and 2008 were caused by sudden increases
3 0.18116599 2321 andrew gelman stats-2014-05-05-On deck this week
Introduction: Mon: Can we make better graphs of global temperature history? Tues: Priors I don’t believe Wed: Cause he thinks he’s so-phisticated Thurs: Discussion with Steven Pinker on research that is attached to data that are so noisy as to be essentially uninformative Fri: Combining forecasts: Evidence on the relative accuracy of the simple average and Bayesian model averaging for predicting social science problems Sat: What property is important in a risk prediction model? Discrimination or calibration? Sun: “What should you talk about?” Plus whatever the co-bloggers want to throw in. Right now I’m super-excited about wedge sampling but I’ll let you know more about that once the paper is done.
Introduction: I read this front-page New York Times article and was immediately suspicious. Here’s the story (from reporter Gina Kolata): Could exercise actually be bad for some healthy people? A well-known group of researchers, including one who helped write the scientific paper justifying national guidelines that promote exercise for all, say the answer may be a qualified yes. By analyzing data from six rigorous exercise studies involving 1,687 people, the group found that about 10 percent actually got worse on at least one of the measures related to heart disease: blood pressure and levels of insulin, HDL cholesterol or triglycerides. About 7 percent got worse on at least two measures. And the researchers say they do not know why. “It is bizarre,” said Claude Bouchard, lead author of the paper , published on Wednesday in the journal PLoS One . . . Dr. Michael Lauer, director of the Division of Cardiovascular Sciences at the National Heart, Lung, and Blood Institute, the lead federal
Introduction: Commenter Rahul asked what I thought of this note by Scott Firestone ( link from Tyler Cowen) criticizing a recent discussion by Kevin Drum suggesting that lead exposure causes violent crime. Firestone writes: It turns out there was in fact a prospective study done—but its implications for Drum’s argument are mixed. The study was a cohort study done by researchers at the University of Cincinnati. Between 1979 and 1984, 376 infants were recruited. Their parents consented to have lead levels in their blood tested over time; this was matched with records over subsequent decades of the individuals’ arrest records, and specifically arrest for violent crime. Ultimately, some of these individuals were dropped from the study; by the end, 250 were selected for the results. The researchers found that for each increase of 5 micrograms of lead per deciliter of blood, there was a higher risk for being arrested for a violent crime, but a further look at the numbers shows a more mixe
6 0.13746354 1086 andrew gelman stats-2011-12-27-The most dangerous jobs in America
7 0.13354798 1912 andrew gelman stats-2013-06-24-Bayesian quality control?
8 0.13171044 514 andrew gelman stats-2011-01-13-News coverage of statistical issues…how did I do?
9 0.11557411 48 andrew gelman stats-2010-05-23-The bane of many causes
10 0.11401375 1610 andrew gelman stats-2012-12-06-Yes, checking calibration of probability forecasts is part of Bayesian statistics
11 0.11354412 1501 andrew gelman stats-2012-09-18-More studies on the economic effects of climate change
12 0.11207633 1377 andrew gelman stats-2012-06-13-A question about AIC
13 0.10671534 2320 andrew gelman stats-2014-05-05-On deck this month
14 0.10487413 1910 andrew gelman stats-2013-06-22-Struggles over the criticism of the “cannabis users and IQ change” paper
15 0.10270653 2270 andrew gelman stats-2014-03-28-Creating a Lenin-style democracy
16 0.10097405 2265 andrew gelman stats-2014-03-24-On deck this week
17 0.09550909 1038 andrew gelman stats-2011-12-02-Donate Your Data to Science!
18 0.091013171 1766 andrew gelman stats-2013-03-16-“Nightshifts Linked to Increased Risk for Ovarian Cancer”
19 0.090833887 1283 andrew gelman stats-2012-04-26-Let’s play “Guess the smoother”!
20 0.09081769 7 andrew gelman stats-2010-04-27-Should Mister P be allowed-encouraged to reside in counter-factual populations?
topicId topicWeight
[(0, 0.116), (1, 0.003), (2, 0.02), (3, -0.055), (4, -0.004), (5, 0.002), (6, -0.011), (7, -0.015), (8, 0.008), (9, -0.015), (10, -0.017), (11, 0.037), (12, 0.046), (13, 0.041), (14, -0.012), (15, 0.042), (16, 0.065), (17, 0.015), (18, 0.011), (19, -0.006), (20, -0.016), (21, 0.054), (22, -0.021), (23, -0.009), (24, 0.025), (25, 0.046), (26, -0.012), (27, 0.018), (28, 0.041), (29, 0.009), (30, -0.083), (31, 0.016), (32, -0.006), (33, -0.005), (34, 0.004), (35, -0.057), (36, 0.085), (37, 0.022), (38, 0.018), (39, -0.043), (40, -0.036), (41, -0.042), (42, 0.016), (43, -0.071), (44, 0.008), (45, -0.033), (46, -0.013), (47, 0.055), (48, -0.055), (49, 0.002)]
simIndex simValue blogId blogTitle
same-blog 1 0.97643054 2328 andrew gelman stats-2014-05-10-What property is important in a risk prediction model? Discrimination or calibration?
Introduction: Sanjay Kaul writes: I am sure you must be aware of the recent controversy ignited by the 2013 American College of Cardiology/American Heart Association Cholesterol Treatment Guidelines that were released last month. They have been the subject of several newspaper articles and blogs , most of them missing the thrust of the guidelines. There is much to admire about these guidelines as they are more faithfully aligned with high-quality ‘actionable’ evidence than the 3 previous iterations. However, the controversy is focused on the performance of the risk calculator introduced for initiating treatment in individuals without established atherosclerotic disease or diabetes (so-called primary prevention cohort). The guidelines recommend statins for primary prevention in individuals who have a 10-year risk estimated to be 7.5%. The risk calculator was derived from population cohorts studied in the 1990s. The discrimination for predicting the risk of atherosclerotic cardiovascular events
Introduction: Alexander at GiveWell writes : The Disease Control Priorities in Developing Countries (DCP2), a major report funded by the Gates Foundation . . . provides an estimate of $3.41 per disability-adjusted life-year (DALY) for the cost-effectiveness of soil-transmitted-helminth (STH) treatment, implying that STH treatment is one of the most cost-effective interventions for global health. In investigating this figure, we have corresponded, over a period of months, with six scholars who had been directly or indirectly involved in the production of the estimate. Eventually, we were able to obtain the spreadsheet that was used to generate the $3.41/DALY estimate. That spreadsheet contains five separate errors that, when corrected, shift the estimated cost effectiveness of deworming from $3.41 to $326.43. [I think they mean to say $300 -- ed.] We came to this conclusion a year after learning that the DCP2’s published cost-effectiveness estimate for schistosomiasis treatment – another kind of
3 0.67738223 1766 andrew gelman stats-2013-03-16-“Nightshifts Linked to Increased Risk for Ovarian Cancer”
Introduction: Zosia Chustecka writes : Much of the previous work on the link between cancer and nightshifts has focused on breast cancer . . . The latest report, focusing on ovarian cancer, was published in the April issue of Occupational and Environmental Medicine. This increase in the risk for ovarian cancer with nightshift work is consistent with, and of similar magnitude to, the risk for breast cancer, say lead author Parveen Bhatti, PhD, and colleagues from the epidemiology program at the Fred Hutchinson Cancer Research Center in Seattle, Washington. The researchers examined data from a local population-based cancer registry that is part of the Surveillance Epidemiology and End Results (SEER) Program. They identified 1101 women with advanced epithelial ovarian cancer, 389 with borderline disease, and 1832 without ovarian cancer (control group). The women, who were 35 to 74 years of age, were asked about the hours they worked, and specifically whether they had ever worked the nig
4 0.66536534 21 andrew gelman stats-2010-05-07-Environmentally induced cancer “grossly underestimated”? Doubtful.
Introduction: The (U.S.) “President’s Cancer Panel” has released its 2008-2009 annual report, which includes a cover letter that says “the true burden of environmentally induced cancer has been grossly underestimated.” The report itself discusses exposures to various types of industrial chemicals, some of which are known carcinogens, in some detail, but gives nearly no data or analysis to suggest that these exposures are contributing to significant numbers of cancers. In fact, there is pretty good evidence that they are not. The plot above shows age-adjusted cancer mortality for men, by cancer type, in the U.S. The plot below shows the same for women. In both cases, the cancers with the highest mortality rates are shown, but not all cancers (e.g. brain cancer is not shown). For what it’s worth, I’m not sure how trustworthy the rates are from the 1930s — it seems possible that reporting, autopsies, or both, were less careful during the Great Depression — so I suggest focusing on the r
5 0.64946747 2022 andrew gelman stats-2013-09-13-You heard it here first: Intense exercise can suppress appetite
Introduction: This post is by Phil Price. The New York Times recently ran an article entitled “How Exercise Can Help Us Eat Less,” which begins with this: “Strenuous exercise seems to dull the urge to eat afterward better than gentler workouts, several new studies show, adding to a growing body of science suggesting that intense exercise may have unique benefits.” The article is based on a couple of recent studies in which moderately overweight volunteers participated in different types of exercise, and had their food intake monitored at a subsequent meal. The article also says “[The volunteers] also displayed significantly lower levels of the hormone ghrelin, which is known to stimulate appetite, and elevated levels of both blood lactate and blood sugar, which have been shown to lessen the drive to eat, after the most vigorous interval session than after the other workouts. And the appetite-suppressing effect of the highly intense intervals lingered into the next day, according to food diarie
6 0.63497055 1239 andrew gelman stats-2012-04-01-A randomized trial of the set-point diet
7 0.62478971 1741 andrew gelman stats-2013-02-27-Thin scientists say it’s unhealthy to be fat
9 0.60096967 731 andrew gelman stats-2011-05-26-Lottery probability update
10 0.60092521 48 andrew gelman stats-2010-05-23-The bane of many causes
11 0.58685225 1086 andrew gelman stats-2011-12-27-The most dangerous jobs in America
12 0.58466947 819 andrew gelman stats-2011-07-24-Don’t idealize “risk aversion”
13 0.58344352 1147 andrew gelman stats-2012-01-30-Statistical Murder
14 0.58076805 53 andrew gelman stats-2010-05-26-Tumors, on the left, or on the right?
15 0.57509434 1548 andrew gelman stats-2012-10-25-Health disparities are associated with low life expectancy
16 0.56805712 1501 andrew gelman stats-2012-09-18-More studies on the economic effects of climate change
17 0.56748194 1905 andrew gelman stats-2013-06-18-There are no fat sprinters
18 0.56719565 514 andrew gelman stats-2011-01-13-News coverage of statistical issues…how did I do?
20 0.55710864 1546 andrew gelman stats-2012-10-24-Hey—has anybody done this study yet?
topicId topicWeight
[(6, 0.062), (8, 0.029), (11, 0.039), (13, 0.025), (15, 0.043), (16, 0.045), (24, 0.104), (27, 0.028), (41, 0.025), (48, 0.014), (59, 0.018), (63, 0.04), (81, 0.02), (84, 0.032), (86, 0.014), (99, 0.321)]
simIndex simValue blogId blogTitle
same-blog 1 0.9822126 2328 andrew gelman stats-2014-05-10-What property is important in a risk prediction model? Discrimination or calibration?
Introduction: Sanjay Kaul writes: I am sure you must be aware of the recent controversy ignited by the 2013 American College of Cardiology/American Heart Association Cholesterol Treatment Guidelines that were released last month. They have been the subject of several newspaper articles and blogs , most of them missing the thrust of the guidelines. There is much to admire about these guidelines as they are more faithfully aligned with high-quality ‘actionable’ evidence than the 3 previous iterations. However, the controversy is focused on the performance of the risk calculator introduced for initiating treatment in individuals without established atherosclerotic disease or diabetes (so-called primary prevention cohort). The guidelines recommend statins for primary prevention in individuals who have a 10-year risk estimated to be 7.5%. The risk calculator was derived from population cohorts studied in the 1990s. The discrimination for predicting the risk of atherosclerotic cardiovascular events
2 0.96577519 1625 andrew gelman stats-2012-12-15-“I coach the jumpers here at Boise State . . .”
Introduction: Jeff Petersmeyer writes: I coach the jumpers here at Boise State and as a fan of the book Moneyball by Michael Lewis (the book that got my brain initially wired to look further than just recruiting the “best” jumpers out of high school (as listed by Track and Field News, etc), I have tried to delve a lot deeper. While coaching at the Olympics this summer in London I began reading—a lot. I read close to 30 books while there for six weeks—including, Outliers, Thinking, Fast and Slow (amazing), Judgment in Managerial Decision Making (Bazerman), The Power of Habit, Start with Why, Switch, Talent is Overrated, The Talent Code, Freakonomics, The House Advantage, among others and more recently Nate Silver’s The Signal and the Noise. I have been collecting data from past years of NCAA championships in the long and the triple jump—finding out where the All Americans have come from (not too surprising: Texas, Louisiana, N. Carolina, Virginia, California, Florida, etc—warm states = more opp
Introduction: Q. D. Leavis wrote: The answer does seem to be that the academic world, like other worlds, is run by the politicians, and sensitively scrupulous people tend to leave politics to other people, while people with genuine work to do certainly have no time as well as no taste for committee-rigging and the associated techniques. And then of course there are the forces of native stupidity reinforced by that blind hostility to criticism, reform, new ideas and superior ability which is human as well as academic nature. Not that I’ve ever read anything by Mrs. Leavis (or, as the Brits used to write, Mrs Leavis). The above quote is one of the epigraphs to a book by Richard Kostelanetz. Whom I’ve never heard of, except in a footnote in John Rodden’s classic Orwell study, The Politics of Literary Reputation. I’ll have more to say about Orwell in another post, but for now let me return to the above Leavis quote, to which I have three reactions: 1. On a personal level, I’m on Leavis’s s
4 0.95914543 618 andrew gelman stats-2011-03-18-Prior information . . . about the likelihood
Introduction: I read this story by Adrian Chen on Gawker (yeah, yeah, so sue me): Why That ‘NASA Discovers Alien Life’ Story Is Bullshit Fox News has a super-exciting article today: “Exclusive: NASA Scientist claims Evidence of Alien Life on Meteorite.” OMG, aliens exist! Except this NASA scientist has been claiming to have evidence of alien life on meteorites for years. Chen continues with a quote from the Fox News item: [NASA scientist Richard B. Hoover] gave FoxNews.com early access to the out-of-this-world research, published late Friday evening in the March edition of the Journal of Cosmology. In it, Hoover describes the latest findings in his study of an extremely rare class of meteorites, called CI1 carbonaceous chondrites — only nine such meteorites are known to exist on Earth. . . . The bad news is that Hoover reported this same sort of finding in various low-rent venues for several years. Replication, huh? Chen also helpfully points us to the website of the Journal
5 0.95883203 506 andrew gelman stats-2011-01-06-That silly ESP paper and some silliness in a rebuttal as well
Introduction: John Talbott points me to this , which I briefly mocked a couple months ago. I largely agree with the critics of this research, but I want to reiterate my point from earlier that all the statistical sophistication in the world won’t help you if you’re studying a null effect. This is not to say that the actual effect is zero—who am I to say?—just that the comments about the high-quality statistics in the article don’t say much to me. There’s lots of discussion of the lack of science underlying ESP claims. I can’t offer anything useful on that account (not being a psychologist, I could imagine all sorts of stories about brain waves or whatever), but I would like to point out something that usually doesn’t seem to get mentioned in these discussions, which is that lots of people want to believe in ESP. After all, it would be cool to read minds. (It wouldn’t be so cool, maybe, if other people could read your mind and you couldn’t read theirs, but I suspect most people don’t think
6 0.95678312 819 andrew gelman stats-2011-07-24-Don’t idealize “risk aversion”
7 0.95591682 2233 andrew gelman stats-2014-03-04-Literal vs. rhetorical
8 0.95562315 236 andrew gelman stats-2010-08-26-Teaching yourself mathematics
9 0.95556676 563 andrew gelman stats-2011-02-07-Evaluating predictions of political events
10 0.95438689 1690 andrew gelman stats-2013-01-23-When are complicated models helpful in psychology research and when are they overkill?
11 0.9535591 1906 andrew gelman stats-2013-06-19-“Behind a cancer-treatment firm’s rosy survival claims”
12 0.95330709 2332 andrew gelman stats-2014-05-12-“The results (not shown) . . .”
13 0.95312101 421 andrew gelman stats-2010-11-19-Just chaid
14 0.95226866 2279 andrew gelman stats-2014-04-02-Am I too negative?
15 0.95217305 1447 andrew gelman stats-2012-08-07-Reproducible science FAIL (so far): What’s stoppin people from sharin data and code?
16 0.95181042 1620 andrew gelman stats-2012-12-12-“Teaching effectiveness” as another dimension in cognitive ability
17 0.95172673 291 andrew gelman stats-2010-09-22-Philosophy of Bayes and non-Bayes: A dialogue with Deborah Mayo
18 0.95167184 1205 andrew gelman stats-2012-03-09-Coming to agreement on philosophy of statistics
19 0.95151573 458 andrew gelman stats-2010-12-08-Blogging: Is it “fair use”?
20 0.95124471 1165 andrew gelman stats-2012-02-13-Philosophy of Bayesian statistics: my reactions to Wasserman