andrew_gelman_stats andrew_gelman_stats-2011 andrew_gelman_stats-2011-791 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: This post was written by Phil. A medical company is testing a cancer drug. They get a 16 genetically identical (or nearly identical) rats that all have the same kind of tumor, give 8 of them the drug and leave 8 untreated…or maybe they give them a placebo, I don’t know; is there a placebo effect in rats?. Anyway, after a while the rats are killed and examined. If the tumors in the treated rats are smaller than the tumors in the untreated rats, then all of the rats have their blood tested for dozens of different proteins that are known to be associated with tumor growth or suppression. If there is a “significant” difference in one of the protein levels, then the working assumption is that the drug increases or decreases levels of that protein and that may be the mechanism by which the drug affects cancer. All of the above is done on many different cancer types and possibly several different types of rats. It’s just the initial screening: if things look promising, many more tests an
sentIndex sentText sentNum sentScore
1 They get a 16 genetically identical (or nearly identical) rats that all have the same kind of tumor, give 8 of them the drug and leave 8 untreated…or maybe they give them a placebo, I don’t know; is there a placebo effect in rats? [sent-3, score-0.801]
2 Anyway, after a while the rats are killed and examined. [sent-5, score-0.453]
3 If the tumors in the treated rats are smaller than the tumors in the untreated rats, then all of the rats have their blood tested for dozens of different proteins that are known to be associated with tumor growth or suppression. [sent-6, score-1.611]
4 If there is a “significant” difference in one of the protein levels, then the working assumption is that the drug increases or decreases levels of that protein and that may be the mechanism by which the drug affects cancer. [sent-7, score-1.795]
5 All of the above is done on many different cancer types and possibly several different types of rats. [sent-8, score-0.293]
6 It’s just the initial screening: if things look promising, many more tests and different tests are done, potentially culminating (years later) in human tests. [sent-9, score-0.28]
7 So the initial task is to determine, from 8 control and 8 treated rats, which proteins look different. [sent-10, score-0.337]
8 below some level a protein is simply reported as “low”; (2) even above the censoring threshold the data are very uncertain (50% or 30% uncertainty for concentrations up to maybe double the censoring threshold); (3) some proteins are reported only in discrete levels (e. [sent-13, score-1.512]
9 4, but never in between); (4) sometimes instrument problems, chemistry problems, or abnormalities in one or more rats lead to very high measurements of one or more proteins. [sent-17, score-0.624]
10 14, low Protein A, controls: low, low, low, low, 0. [sent-23, score-0.318]
11 24, low, low, low Protein B, cases: 160, 122, 99, 145, 377, 133, 123, 140 Protein B, controls: 94, 107, 139, 135, 152, 120, 111, 118 Note the very high value of Protein B in case rat 5. [sent-24, score-0.375]
12 The drug company would not want to flag Protein B as being affected by their drug just because they got that one big number. [sent-25, score-0.456]
13 Finally, the question: what’s a good algorithm to recognize if the cases tend to have higher levels of a given protein than the controls? [sent-26, score-0.833]
14 A few possibilities that come to mind: (1) generate bootstrap samples from the cases and from the controls, and see how often the medians differ by more than the observed medians do; if it’s a small fraction of the time, then the observed difference is “statistically significant. [sent-27, score-0.555]
15 (3) Discard outliers, then use censored maximum likelihood (or similar) on the rest of the data, thus generating a mean (or geometric mean) and uncertainty for the cases and for the controls. [sent-29, score-0.321]
wordName wordTfidf (topN-words)
[('protein', 0.606), ('rats', 0.41), ('low', 0.318), ('proteins', 0.2), ('drug', 0.17), ('controls', 0.156), ('censoring', 0.133), ('medians', 0.121), ('levels', 0.116), ('cases', 0.111), ('tumors', 0.109), ('untreated', 0.109), ('tumor', 0.102), ('placebo', 0.091), ('threshold', 0.074), ('identical', 0.073), ('measurements', 0.07), ('cancer', 0.069), ('initial', 0.069), ('treated', 0.068), ('types', 0.064), ('company', 0.061), ('rat', 0.057), ('culminating', 0.057), ('abnormalities', 0.057), ('genetically', 0.057), ('observed', 0.057), ('geometric', 0.055), ('censored', 0.055), ('flag', 0.055), ('uncertainty', 0.053), ('reported', 0.053), ('tests', 0.053), ('complications', 0.051), ('concentrations', 0.049), ('discard', 0.049), ('different', 0.048), ('generating', 0.047), ('screening', 0.046), ('bootstrap', 0.046), ('promising', 0.046), ('blood', 0.046), ('decreases', 0.044), ('chemistry', 0.044), ('outliers', 0.044), ('instrument', 0.043), ('killed', 0.043), ('uncertain', 0.042), ('difference', 0.042), ('affects', 0.041)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 791 andrew gelman stats-2011-07-08-Censoring on one end, “outliers” on the other, what can we do with the middle?
Introduction: This post was written by Phil. A medical company is testing a cancer drug. They get a 16 genetically identical (or nearly identical) rats that all have the same kind of tumor, give 8 of them the drug and leave 8 untreated…or maybe they give them a placebo, I don’t know; is there a placebo effect in rats?. Anyway, after a while the rats are killed and examined. If the tumors in the treated rats are smaller than the tumors in the untreated rats, then all of the rats have their blood tested for dozens of different proteins that are known to be associated with tumor growth or suppression. If there is a “significant” difference in one of the protein levels, then the working assumption is that the drug increases or decreases levels of that protein and that may be the mechanism by which the drug affects cancer. All of the above is done on many different cancer types and possibly several different types of rats. It’s just the initial screening: if things look promising, many more tests an
Introduction: Phil Nelson writes in the context of a biostatistics textbook he is writing, “Physical models of living systems”: There are a number of classic statistical problems that arise every day in the lab, and which are discussed in any book: 1. In a control group, M untreated rats out of 20 got a form of cancer. In a test group, N treated rats out of 20 got that cancer. Is this a significant difference? 2. In a control group of 20 untreated rates, their body weights at 2 weeks were w_1,…, w_20. In a test group of 20 treated rats, their body weights at 2 weeks were w’_1,…, w’_20. Are the means significantly different? 3. In a group of 20 rats, each given dose d_i of a drug, their body weights at 2 weeks were w_i. Is there a significant correlation between d and w? I would like to ask: What are some situations in which the classical approach (or a naive implementation of it, based on cookbook recipes) gives worse results than a Bayesian approach, results that actually impeded the scien
3 0.10488276 53 andrew gelman stats-2010-05-26-Tumors, on the left, or on the right?
Introduction: In response to the post The bane of many causes in the context of mobile phone use and brain cancer, Robert Erikson wrote: The true control here is the side of the head of the tumor: same side as phone use or opposite side. If that is the test, the data from the study are scary. Clearly tumors are more likely on the “same” side, at whatever astronomical p value you want to use. That cannot be explained away by misremembering, since an auxiliary study showed misremembering was not biased toward cell phone-tumor consistency. A strong signal in the data pointed by Prof. Erikson is that the tumors are overwhelmingly likelier to appear on the same side of the head as where the phone is held. I’ve converted the ratios into percentages, based on an assumption that the risk for tumors would be apriori equal for both sides of the head. There is a group of people with low-to-moderate exposure and high lateral bias, but the bias does increase quite smoothly with increasing
4 0.097583741 388 andrew gelman stats-2010-11-01-The placebo effect in pharma
Introduction: Bruce McCullough writes: The Sept 2009 issue of Wired had a big article on the increase in the placebo effect, and why it’s been getting bigger. Kaiser Fung has a synopsis . As if you don’t have enough to do, I thought you might be interested in blogging on this. My reply: I thought Kaiser’s discussion was good, especially this point: Effect on treatment group = Effect of the drug + effect of belief in being treated Effect on placebo group = Effect of belief in being treated Thus, the difference between the two groups = effect of the drug, since the effect of belief in being treated affects both groups of patients. Thus, as Kaiser puts it, if the treatment isn’t doing better than placebo, it doesn’t say that the placebo effect is big (let alone “too big”) but that the treatment isn’t showing any additional effect. It’s “treatment + placebo” vs. placebo, not treatment vs. placebo. That said, I’d prefer for Kaiser to make it clear that the additivity he’s assu
5 0.092474572 86 andrew gelman stats-2010-06-14-“Too much data”?
Introduction: Chris Hane writes: I am scientist needing to model a treatment effect on a population of ~500 people. The dependent variable in the model is the difference in a person’s pre-treatment 12 month total medical cost versus post-treatment cost. So there is large variation in costs, but not so much by using the difference between the pre and post treatment costs. The issue I’d like some advice on is that the treatment has already occurred so there is no possibility of creating a fully randomized control now. I do have a very large population of people to use as possible controls via propensity scoring or exact matching. If I had a few thousand people to possibly match, then I would use standard techniques. However, I have a potential population of over a hundred thousand people. An exact match of the possible controls to age, gender and region of the country still leaves a population of 10,000 controls. Even if I use propensity scores to weight the 10,000 observations (understan
6 0.088381462 2115 andrew gelman stats-2013-11-27-Three unblinded mice
8 0.075596057 2180 andrew gelman stats-2014-01-21-Everything I need to know about Bayesian statistics, I learned in eight schools.
9 0.071633533 161 andrew gelman stats-2010-07-24-Differences in color perception by sex, also the Bechdel test for women in movies
10 0.070913151 213 andrew gelman stats-2010-08-17-Matching at two levels
11 0.069387011 1628 andrew gelman stats-2012-12-17-Statistics in a world where nothing is random
12 0.068958424 960 andrew gelman stats-2011-10-15-The bias-variance tradeoff
13 0.066479892 48 andrew gelman stats-2010-05-23-The bane of many causes
14 0.063848831 21 andrew gelman stats-2010-05-07-Environmentally induced cancer “grossly underestimated”? Doubtful.
15 0.062644497 1766 andrew gelman stats-2013-03-16-“Nightshifts Linked to Increased Risk for Ovarian Cancer”
16 0.061950009 411 andrew gelman stats-2010-11-13-Ethical concerns in medical trials
17 0.060521737 1607 andrew gelman stats-2012-12-05-The p-value is not . . .
18 0.058309972 395 andrew gelman stats-2010-11-05-Consulting: how do you figure out what to charge?
19 0.058025688 2210 andrew gelman stats-2014-02-13-Stopping rules and Bayesian analysis
20 0.057832934 923 andrew gelman stats-2011-09-24-What is the normal range of values in a medical test?
topicId topicWeight
[(0, 0.09), (1, 0.01), (2, 0.035), (3, -0.053), (4, 0.009), (5, -0.011), (6, 0.015), (7, 0.02), (8, 0.01), (9, -0.016), (10, -0.046), (11, 0.001), (12, 0.022), (13, -0.043), (14, -0.005), (15, 0.014), (16, 0.027), (17, -0.005), (18, 0.01), (19, -0.002), (20, 0.014), (21, 0.021), (22, -0.012), (23, 0.006), (24, -0.021), (25, 0.033), (26, -0.019), (27, -0.009), (28, 0.018), (29, 0.04), (30, -0.001), (31, 0.017), (32, 0.015), (33, 0.04), (34, -0.004), (35, 0.004), (36, -0.0), (37, 0.012), (38, -0.001), (39, 0.02), (40, -0.021), (41, -0.035), (42, -0.022), (43, -0.006), (44, 0.017), (45, -0.007), (46, 0.005), (47, 0.022), (48, 0.025), (49, 0.003)]
simIndex simValue blogId blogTitle
same-blog 1 0.95312566 791 andrew gelman stats-2011-07-08-Censoring on one end, “outliers” on the other, what can we do with the middle?
Introduction: This post was written by Phil. A medical company is testing a cancer drug. They get a 16 genetically identical (or nearly identical) rats that all have the same kind of tumor, give 8 of them the drug and leave 8 untreated…or maybe they give them a placebo, I don’t know; is there a placebo effect in rats?. Anyway, after a while the rats are killed and examined. If the tumors in the treated rats are smaller than the tumors in the untreated rats, then all of the rats have their blood tested for dozens of different proteins that are known to be associated with tumor growth or suppression. If there is a “significant” difference in one of the protein levels, then the working assumption is that the drug increases or decreases levels of that protein and that may be the mechanism by which the drug affects cancer. All of the above is done on many different cancer types and possibly several different types of rats. It’s just the initial screening: if things look promising, many more tests an
2 0.78656912 923 andrew gelman stats-2011-09-24-What is the normal range of values in a medical test?
Introduction: Geoffrey Sheean writes: I am having trouble thinking Bayesianly about the so-called ‘normal’ or ‘reference’ values that I am supposed to use in some of the tests I perform. These values are obtained from purportedly healthy people. Setting aside concerns about ascertainment bias, non-parametric distributions, and the like, the values are usually obtained by setting the limits at ± 2SD from the mean. In some cases, supposedly because of a non-normal distribution, the third highest and lowest value observed in the healthy group sets the limits, on the assumption that no more than 2 results (out of 20 samples) are allowed to exceed these values: if there are 3 or more, then the test is assumed to be abnormal and the reference range is said to reflect the 90th percentile. The results are binary – normal, abnormal. The relevance to the diseased state is this. People who are known unequivocally to have condition X show Y abnormalities in these tests. Therefore, when people suspected
3 0.784899 2049 andrew gelman stats-2013-10-03-On house arrest for p-hacking
Introduction: People keep pointing me to this excellent news article by David Brown, about a scientist who was convicted of data manipulation: In all, 330 patients were randomly assigned to get either interferon gamma-1b or placebo injections. Disease progression or death occurred in 46 percent of those on the drug and 52 percent of those on placebo. That was not a significant difference, statistically speaking. When only survival was considered, however, the drug looked better: 10 percent of people getting the drug died, compared with 17 percent of those on placebo. However, that difference wasn’t “statistically significant,” either. Specifically, the so-called P value — a mathematical measure of the strength of the evidence that there’s a true difference between a treatment and placebo — was 0.08. . . . Technically, the study was a bust, although the results leaned toward a benefit from interferon gamma-1b. Was there a group of patients in which the results tipped? Harkonen asked the statis
4 0.76621133 314 andrew gelman stats-2010-10-03-Disconnect between drug and medical device approval
Introduction: Sanjay Kaul wrotes: By statute (“the least burdensome” pathway), the approval standard for devices by the US FDA is lower than for drugs. Before a new drug can be marketed, the sponsor must show “substantial evidence of effectiveness” as based on two or more well-controlled clinical studies (which literally means 2 trials, each with a p value of <0.05, or 1 large trial with a robust p value <0.00125). In contrast, the sponsor of a new device, especially those that are designated as high-risk (Class III) device, need only demonstrate "substantial equivalence" to an FDA-approved device via the 510(k) exemption or a "reasonable assurance of safety and effectiveness", evaluated through a pre-market approval and typically based on a single study. What does “reasonable assurance” or “substantial equivalence” imply to you as a Bayesian? These are obviously qualitative constructs, but if one were to quantify them, how would you go about addressing it? The regulatory definitions for
5 0.7489326 53 andrew gelman stats-2010-05-26-Tumors, on the left, or on the right?
Introduction: In response to the post The bane of many causes in the context of mobile phone use and brain cancer, Robert Erikson wrote: The true control here is the side of the head of the tumor: same side as phone use or opposite side. If that is the test, the data from the study are scary. Clearly tumors are more likely on the “same” side, at whatever astronomical p value you want to use. That cannot be explained away by misremembering, since an auxiliary study showed misremembering was not biased toward cell phone-tumor consistency. A strong signal in the data pointed by Prof. Erikson is that the tumors are overwhelmingly likelier to appear on the same side of the head as where the phone is held. I’ve converted the ratios into percentages, based on an assumption that the risk for tumors would be apriori equal for both sides of the head. There is a group of people with low-to-moderate exposure and high lateral bias, but the bias does increase quite smoothly with increasing
6 0.71920627 708 andrew gelman stats-2011-05-12-Improvement of 5 MPG: how many more auto deaths?
7 0.70254636 1910 andrew gelman stats-2013-06-22-Struggles over the criticism of the “cannabis users and IQ change” paper
8 0.70184624 549 andrew gelman stats-2011-02-01-“Roughly 90% of the increase in . . .” Hey, wait a minute!
10 0.6942175 48 andrew gelman stats-2010-05-23-The bane of many causes
12 0.68857425 322 andrew gelman stats-2010-10-06-More on the differences between drugs and medical devices
13 0.68536365 86 andrew gelman stats-2010-06-14-“Too much data”?
14 0.68203336 284 andrew gelman stats-2010-09-18-Continuing efforts to justify false “death panels” claim
15 0.68053442 2155 andrew gelman stats-2013-12-31-No on Yes-No decisions
17 0.67045629 940 andrew gelman stats-2011-10-03-It depends upon what the meaning of the word “firm” is.
18 0.66844982 1741 andrew gelman stats-2013-02-27-Thin scientists say it’s unhealthy to be fat
19 0.66757786 21 andrew gelman stats-2010-05-07-Environmentally induced cancer “grossly underestimated”? Doubtful.
topicId topicWeight
[(2, 0.028), (15, 0.018), (16, 0.075), (21, 0.014), (24, 0.132), (35, 0.013), (45, 0.234), (53, 0.014), (59, 0.021), (72, 0.025), (76, 0.02), (91, 0.04), (95, 0.029), (99, 0.195)]
simIndex simValue blogId blogTitle
1 0.91206682 1407 andrew gelman stats-2012-07-06-Statistical inference and the secret ballot
Introduction: Ring Lardner, Jr.: [In 1936] I was already settled in Southern California, and it may have been that first exercise of the franchise that triggered the FBI surveillance of me that would last for decades. I had assumed, of course, that I was enjoying the vaunted American privilege of the secret ballot. On a wall outside my polling place on Wilshire Boulevard, however, was a compilation of the district’s registered voters: Democrats, a long list of names; Republicans, a somewhat lesser number; and “Declines to State,” one, “Ring W. Lardner, Jr.” The day after the election, alongside those lists were published the results: Roosevelt, so many; Landon, so many; Browder, one.
2 0.90681851 999 andrew gelman stats-2011-11-09-I was at a meeting a couple months ago . . .
Introduction: . . . and I decided to amuse myself by writing down all the management-speak words I heard: “grappling” “early prototypes” “technology platform” “building block” “machine learning” “your team” “workspace” “tagging” “data exhaust” “monitoring a particular population” “collective intelligence” “communities of practice” “hackathon” “human resources . . . technologies” Any one or two or three of these phrases might be fine, but put them all together and what you have is a festival of jargon. A hackathon, indeed.
same-blog 3 0.90117234 791 andrew gelman stats-2011-07-08-Censoring on one end, “outliers” on the other, what can we do with the middle?
Introduction: This post was written by Phil. A medical company is testing a cancer drug. They get a 16 genetically identical (or nearly identical) rats that all have the same kind of tumor, give 8 of them the drug and leave 8 untreated…or maybe they give them a placebo, I don’t know; is there a placebo effect in rats?. Anyway, after a while the rats are killed and examined. If the tumors in the treated rats are smaller than the tumors in the untreated rats, then all of the rats have their blood tested for dozens of different proteins that are known to be associated with tumor growth or suppression. If there is a “significant” difference in one of the protein levels, then the working assumption is that the drug increases or decreases levels of that protein and that may be the mechanism by which the drug affects cancer. All of the above is done on many different cancer types and possibly several different types of rats. It’s just the initial screening: if things look promising, many more tests an
4 0.88445842 543 andrew gelman stats-2011-01-28-NYT shills for personal DNA tests
Introduction: Kaiser nails it . The offending article , by John Tierney, somehow ended up in the Science section rather than the Opinion section. As an opinion piece (or, for that matter, a blog), Tierney’s article would be nothing special. But I agree with Kaiser that it doesn’t work as a newspaper article. As Kaiser notes, this story involves a bunch of statistical and empirical claims that are not well resolved by P.R. and rhetoric.
5 0.87969476 310 andrew gelman stats-2010-10-02-The winner’s curse
Introduction: If an estimate is statistically significant, it’s probably an overestimate of the magnitude of your effect. P.S. I think youall know what I mean here. But could someone rephrase it in a more pithy manner? I’d like to include it in our statistical lexicon.
6 0.86981106 1325 andrew gelman stats-2012-05-17-More on the difficulty of “preaching what you practice”
7 0.85863221 1031 andrew gelman stats-2011-11-27-Richard Stallman and John McCarthy
8 0.83962297 69 andrew gelman stats-2010-06-04-A Wikipedia whitewash
9 0.83377385 206 andrew gelman stats-2010-08-13-Indiemapper makes thematic mapping easy
10 0.83301568 1089 andrew gelman stats-2011-12-28-Path sampling for models of varying dimension
11 0.8324728 1121 andrew gelman stats-2012-01-15-R-squared for multilevel models
12 0.82714736 1504 andrew gelman stats-2012-09-20-Could someone please lock this guy and Niall Ferguson in a room together?
13 0.8230468 1012 andrew gelman stats-2011-11-16-Blog bribes!
14 0.82275069 2189 andrew gelman stats-2014-01-28-History is too important to be left to the history professors
15 0.81083679 362 andrew gelman stats-2010-10-22-A redrawing of the Red-Blue map in November 2010?
16 0.81014788 673 andrew gelman stats-2011-04-20-Upper-income people still don’t realize they’re upper-income
17 0.80372667 1658 andrew gelman stats-2013-01-07-Free advice from an academic writing coach!
18 0.7997281 1767 andrew gelman stats-2013-03-17-The disappearing or non-disappearing middle class
19 0.79669088 192 andrew gelman stats-2010-08-08-Turning pages into data
20 0.79230207 612 andrew gelman stats-2011-03-14-Uh-oh