andrew_gelman_stats andrew_gelman_stats-2013 andrew_gelman_stats-2013-1962 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: A link from Simon Jackman’s blog led me to an article by James Heckman, Hedibert Lopes, and Remi Piatek from 2011, “Treatment effects: A Bayesian perspective.” I was pleasantly surprised to see this, partly because I didn’t know that Heckman was working on Bayesian methods, and partly because the paper explicitly refers to the “potential outcomes model,” a term I associate with Don Rubin. I’ve had the impression that Heckman and Rubin don’t like each other (I was a student of Rubin and have never met Heckman, so I’m only speaking at second hand here), so I was happy to see some convergence. I was curious how Heckman et al. would source the potential outcome model. They do not refer to Rubin’s 1974 paper or to Neyman’s 1923 paper (which was republished in 1990 and is now taken to be the founding document of the Neyman-Rubin approach to causal inference). Nor, for that matter, do Heckman et al. refer to the more recent developments of these theories by Robins, Pearl, and other
sentIndex sentText sentNum sentScore
1 ” I was pleasantly surprised to see this, partly because I didn’t know that Heckman was working on Bayesian methods, and partly because the paper explicitly refers to the “potential outcomes model,” a term I associate with Don Rubin. [sent-2, score-0.557]
2 I’ve had the impression that Heckman and Rubin don’t like each other (I was a student of Rubin and have never met Heckman, so I’m only speaking at second hand here), so I was happy to see some convergence. [sent-3, score-0.044]
3 They do not refer to Rubin’s 1974 paper or to Neyman’s 1923 paper (which was republished in 1990 and is now taken to be the founding document of the Neyman-Rubin approach to causal inference). [sent-6, score-0.416]
4 refer to the more recent developments of these theories by Robins, Pearl, and others. [sent-8, score-0.165]
5 say about causal inference: The well-known fundamental problem in program evaluation derives from the fact that people can never be observed in different treatment states simultaneously, which makes it is impossible to directly observe their outcome gains. [sent-11, score-0.471]
6 The textbook model considered in this paper is an extension of the original Roy model (Roy, 1951; Heckman and Honore, 1990) and assumes a binary treatment decision D that involves two continuous potential outcomes Y1 and Y0 for the treated and untreated states . [sent-15, score-1.063]
7 The relationship of the Roy model to other models of potential outcomes is discussed in Heckman (2008). [sent-18, score-0.505]
8 I wasn’t familiar with Roy (1951), which comes roughly halfway between Neyman’s original work on this area and Rubin’s later developments. [sent-19, score-0.155]
9 The Roy paper is called “Some thoughts on the distribution of earnings” and presents a model of a miniature society whose members can make their living out of some mixture of hunting and fishing. [sent-21, score-0.378]
10 I don’t see the potential outcome model here in any way. [sent-22, score-0.515]
11 Here’s some discussion at Heckman (2008): The Roy model (1951) is another version of this framework with two possible treatment outcomes (S = {0, 1}) and a scalar outcome measure and a particular assignment mechanism . [sent-23, score-0.668]
12 I mean, sure, I see a vague connection, but only very vague. [sent-27, score-0.087]
13 I don’t know the history here—maybe Roy was this brilliant researcher who talked about the potential outcome paper but just didn’t put the details in the article? [sent-28, score-0.492]
14 I also took a look at Heckman and Honore (1990), which did discuss a joint distribution, but not of potential outcomes—they were talking about a bivariate distribution of skills. [sent-29, score-0.382]
15 You want to give credit to early work that inspired important later developments, but you have to be careful not to credit the old stuff for more than it is. [sent-31, score-0.231]
16 I think Heckman’s labeling of potential outcomes as the “Roy model” is going too far. [sent-32, score-0.444]
wordName wordTfidf (topN-words)
[('heckman', 0.625), ('roy', 0.447), ('potential', 0.212), ('outcomes', 0.189), ('rubin', 0.174), ('outcome', 0.155), ('honore', 0.145), ('treatment', 0.112), ('model', 0.104), ('neyman', 0.093), ('developments', 0.09), ('et', 0.084), ('paper', 0.083), ('refer', 0.075), ('partly', 0.071), ('distribution', 0.069), ('credit', 0.068), ('miniature', 0.066), ('robins', 0.066), ('scalar', 0.062), ('jackman', 0.062), ('republished', 0.062), ('founding', 0.06), ('untreated', 0.06), ('pleasantly', 0.056), ('halfway', 0.056), ('hunting', 0.056), ('trace', 0.054), ('earnings', 0.053), ('causal', 0.053), ('took', 0.052), ('states', 0.052), ('assumes', 0.052), ('derives', 0.052), ('later', 0.051), ('bivariate', 0.049), ('simon', 0.048), ('original', 0.048), ('extension', 0.047), ('observe', 0.047), ('assignment', 0.046), ('simultaneously', 0.046), ('pearl', 0.046), ('googled', 0.045), ('inspired', 0.044), ('see', 0.044), ('associate', 0.043), ('labeling', 0.043), ('vague', 0.043), ('brilliant', 0.042)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 1962 andrew gelman stats-2013-07-30-The Roy causal model?
Introduction: A link from Simon Jackman’s blog led me to an article by James Heckman, Hedibert Lopes, and Remi Piatek from 2011, “Treatment effects: A Bayesian perspective.” I was pleasantly surprised to see this, partly because I didn’t know that Heckman was working on Bayesian methods, and partly because the paper explicitly refers to the “potential outcomes model,” a term I associate with Don Rubin. I’ve had the impression that Heckman and Rubin don’t like each other (I was a student of Rubin and have never met Heckman, so I’m only speaking at second hand here), so I was happy to see some convergence. I was curious how Heckman et al. would source the potential outcome model. They do not refer to Rubin’s 1974 paper or to Neyman’s 1923 paper (which was republished in 1990 and is now taken to be the founding document of the Neyman-Rubin approach to causal inference). Nor, for that matter, do Heckman et al. refer to the more recent developments of these theories by Robins, Pearl, and other
2 0.22769798 2033 andrew gelman stats-2013-09-23-More on Bayesian methods and multilevel modeling
Introduction: Ban Chuan Cheah writes: In a previous post, http://andrewgelman.com/2013/07/30/the-roy-causal-model/ you pointed to a paper on Bayesian methods by Heckman. At around the same time I came across another one of his papers, “The Effects of Cognitive and Noncognitive Abilities on Labor Market Outcomes and Social Behavior (2006)” (http://www.nber.org/papers/w12006 or published version http://www.jstor.org/stable/10.1086/504455). In this paper they implement their model as follows: We use Bayesian Markov chain Monte Carlo methods to compute the sample likelihood. Our use of Bayesian methods is only a computational convenience. Our identification analysis is strictly classical. Under our assumptions, the priors we use are asymptotically irrelevant. Some of the authors have also done something similar earlier in: Hansen, Karsten T. & Heckman, James J. & Mullen, K.J.Kathleen J., 2004. “The effect of schooling and ability on achievement test scores,” Journal of Econometrics, Elsevi
3 0.18370137 2362 andrew gelman stats-2014-06-06-Statistically savvy journalism
Introduction: Roy Mendelssohn points me to this excellent bit of statistics reporting by Matt Novak. I have no comment, I just think it’s good to see this sort of high-quality Felix Salmon-style statistically savvy journalism.
4 0.17248061 2223 andrew gelman stats-2014-02-24-“Edlin’s rule” for routinely scaling down published estimates
Introduction: A few months ago I reacted (see further discussion in comments here ) to a recent study on early childhood intervention, in which researchers Paul Gertler, James Heckman, Rodrigo Pinto, Arianna Zanolini, Christel Vermeerch, Susan Walker, Susan M. Chang, and Sally Grantham-McGregor estimated that a particular intervention on young children had raised incomes on young adults by 42%. I wrote: Major decisions on education policy can turn on the statistical interpretation of small, idiosyncratic data sets — in this case, a study of 129 Jamaican children. . . . Overall, I have no reason to doubt the direction of the effect, namely, that psychosocial stimulation should be good. But I’m skeptical of the claim that income differed by 42%, due to the reason of the statistical significance filter . In section 2.3, the authors are doing lots of hypothesizing based on some comparisons being statistically significant and others being non-significant. There’s nothing wrong with speculation, b
Introduction: Hal Pashler wrote in about a recent paper , “Labor Market Returns to Early Childhood Stimulation: a 20-year Followup to an Experimental Intervention in Jamaica,” by Paul Gertler, James Heckman, Rodrigo Pinto, Arianna Zanolini, Christel Vermeerch, Susan Walker, Susan M. Chang, and Sally Grantham-McGregor. Here’s Pashler: Dan Willingham tweeted: @DTWillingham: RCT from Jamaica: Big effects 20 years later of intervention—teaching parenting/child stimulation to moms in poverty http://t.co/rX6904zxvN Browsing pp. 4 ff, it seems the authors are basically saying “hey the stats were challenging, the sample size tiny, other problems, but we solved them all—using innovative methods of our own devising!—and lo and behold, big positive results!”. So this made me think (and tweet) basically that I hope the topic (which is pretty important) will happen to interest Andy Gelman enough to incline him to give us his take. If you happen to have time and interest… My reply became this artic
7 0.11921454 550 andrew gelman stats-2011-02-02-An IV won’t save your life if the line is tangled
8 0.10463364 1133 andrew gelman stats-2012-01-21-Judea Pearl on why he is “only a half-Bayesian”
9 0.10262702 2277 andrew gelman stats-2014-03-31-The most-cited statistics papers ever
12 0.10048907 1675 andrew gelman stats-2013-01-15-“10 Things You Need to Know About Causal Effects”
13 0.095983528 1175 andrew gelman stats-2012-02-19-Factual – a new place to find data
14 0.093675405 1869 andrew gelman stats-2013-05-24-In which I side with Neyman over Fisher
15 0.090374142 1884 andrew gelman stats-2013-06-05-A story of fake-data checking being used to shoot down a flawed analysis at the Farm Credit Agency
16 0.087368228 541 andrew gelman stats-2011-01-27-Why can’t I be more like Bill James, or, The use of default and default-like models
18 0.08067482 781 andrew gelman stats-2011-06-28-The holes in my philosophy of Bayesian data analysis
19 0.080058247 109 andrew gelman stats-2010-06-25-Classics of statistics
topicId topicWeight
[(0, 0.137), (1, 0.061), (2, -0.006), (3, -0.024), (4, -0.02), (5, -0.005), (6, -0.014), (7, -0.017), (8, 0.083), (9, 0.03), (10, -0.002), (11, 0.035), (12, -0.007), (13, -0.03), (14, 0.019), (15, 0.023), (16, 0.031), (17, 0.023), (18, -0.052), (19, 0.038), (20, -0.017), (21, -0.024), (22, 0.03), (23, -0.017), (24, 0.037), (25, 0.077), (26, -0.047), (27, -0.006), (28, 0.008), (29, 0.026), (30, -0.003), (31, -0.001), (32, 0.004), (33, 0.011), (34, -0.023), (35, -0.008), (36, -0.017), (37, -0.025), (38, 0.02), (39, 0.015), (40, 0.002), (41, 0.015), (42, 0.004), (43, 0.009), (44, 0.019), (45, -0.006), (46, 0.006), (47, -0.04), (48, -0.017), (49, 0.016)]
simIndex simValue blogId blogTitle
same-blog 1 0.95693362 1962 andrew gelman stats-2013-07-30-The Roy causal model?
Introduction: A link from Simon Jackman’s blog led me to an article by James Heckman, Hedibert Lopes, and Remi Piatek from 2011, “Treatment effects: A Bayesian perspective.” I was pleasantly surprised to see this, partly because I didn’t know that Heckman was working on Bayesian methods, and partly because the paper explicitly refers to the “potential outcomes model,” a term I associate with Don Rubin. I’ve had the impression that Heckman and Rubin don’t like each other (I was a student of Rubin and have never met Heckman, so I’m only speaking at second hand here), so I was happy to see some convergence. I was curious how Heckman et al. would source the potential outcome model. They do not refer to Rubin’s 1974 paper or to Neyman’s 1923 paper (which was republished in 1990 and is now taken to be the founding document of the Neyman-Rubin approach to causal inference). Nor, for that matter, do Heckman et al. refer to the more recent developments of these theories by Robins, Pearl, and other
2 0.82880384 393 andrew gelman stats-2010-11-04-Estimating the effect of A on B, and also the effect of B on A
Introduction: Lei Liu writes: I am working with clinicians in infectious disease and international health to study the (possible causal) relation between malnutrition and virus infection episodes (e.g., diarrhea) in babies in developing countries. Basically the clinicians are interested in two questions: does malnutrition cause more diarrhea episodes? does diarrhea lead to malnutrition? The malnutrition status is indicated by height and weight (adjusted, HAZ and WAZ measures) observed every 3 months from birth to 1 year. They also recorded the time of each diarrhea episode during the 1 year follow-up period. They have very solid datasets for analysis. As you can see, this is almost like a chicken and egg problem. I am a layman to causal inference. The method I use is just to do some simple regression. For example, to study the causal relation from malnutrition to diarrhea episodes, I use binary variable (diarrhea yes/no during months 0-3) as response, and use the HAZ at month 0 as covariate
3 0.80690598 1136 andrew gelman stats-2012-01-23-Fight! (also a bit of reminiscence at the end)
Introduction: Martin Lindquist and Michael Sobel published a fun little article in Neuroimage on models and assumptions for causal inference with intermediate outcomes. As their subtitle indicates (“A response to the comments on our comment”), this is a topic of some controversy. Lindquist and Sobel write: Our original comment (Lindquist and Sobel, 2011) made explicit the types of assumptions neuroimaging researchers are making when directed graphical models (DGMs), which include certain types of structural equation models (SEMs), are used to estimate causal effects. When these assumptions, which many researchers are not aware of, are not met, parameters of these models should not be interpreted as effects. . . . [Judea] Pearl does not disagree with anything we stated. However, he takes exception to our use of potential outcomes notation, which is the standard notation used in the statistical literature on causal inference, and his comment is devoted to promoting his alternative conventions. [C
Introduction: This material should be familiar to many of you but could be helpful to newcomers. Pearl writes: ALL causal conclusions in nonexperimental settings must be based on untested, judgmental assumptions that investigators are prepared to defend on scientific grounds. . . . To understand what the world should be like for a given procedure to work is of no lesser scientific value than seeking evidence for how the world works . . . Assumptions are self-destructive in their honesty. The more explicit the assumption, the more criticism it invites . . . causal diagrams invite the harshest criticism because they make assumptions more explicit and more transparent than other representation schemes. As regular readers know (for example, search this blog for “Pearl”), I have not got much out of the causal-diagrams approach myself, but in general I think that when there are multiple, mathematically equivalent methods of getting the same answer, we tend to go with the framework we are used
Introduction: Elias Bareinboim asked what I thought about his comment on selection bias in which he referred to a paper by himself and Judea Pearl, “Controlling Selection Bias in Causal Inference.” I replied that I have no problem with what he wrote, but that from my perspective I find it easier to conceptualize such problems in terms of multilevel models. I elaborated on that point in a recent post , “Hierarchical modeling as a framework for extrapolation,” which I think was read by only a few people (I say this because it received only two comments). I don’t think Bareinboim objected to anything I wrote, but like me he is comfortable working within his own framework. He wrote the following to me: In some sense, “not ad hoc” could mean logically consistent. In other words, if one agrees with the assumptions encoded in the model, one must also agree with the conclusions entailed by these assumptions. I am not aware of any other way of doing mathematics. As it turns out, to get causa
7 0.76971912 2097 andrew gelman stats-2013-11-11-Why ask why? Forward causal inference and reverse causal questions
9 0.71093804 518 andrew gelman stats-2011-01-15-Regression discontinuity designs: looking for the keys under the lamppost?
10 0.69999695 1675 andrew gelman stats-2013-01-15-“10 Things You Need to Know About Causal Effects”
11 0.69772476 936 andrew gelman stats-2011-10-02-Covariate Adjustment in RCT - Model Overfitting in Multilevel Regression
13 0.69267875 807 andrew gelman stats-2011-07-17-Macro causality
14 0.69051296 1996 andrew gelman stats-2013-08-24-All inference is about generalizing from sample to population
15 0.6899839 776 andrew gelman stats-2011-06-22-Deviance, DIC, AIC, cross-validation, etc
16 0.68789637 2 andrew gelman stats-2010-04-23-Modeling heterogenous treatment effects
17 0.68662083 614 andrew gelman stats-2011-03-15-Induction within a model, deductive inference for model evaluation
18 0.68210304 1070 andrew gelman stats-2011-12-19-The scope for snooping
19 0.68105525 244 andrew gelman stats-2010-08-30-Useful models, model checking, and external validation: a mini-discussion
20 0.67481863 2336 andrew gelman stats-2014-05-16-How much can we learn about individual-level causal claims from state-level correlations?
topicId topicWeight
[(15, 0.017), (16, 0.093), (21, 0.073), (24, 0.1), (27, 0.012), (30, 0.012), (35, 0.012), (43, 0.017), (45, 0.012), (54, 0.016), (63, 0.013), (81, 0.208), (86, 0.021), (89, 0.011), (99, 0.265)]
simIndex simValue blogId blogTitle
1 0.96846855 915 andrew gelman stats-2011-09-17-(Worst) graph of the year
Introduction: This (forwarded to me from Jeff, from a powerpoint by Willam Gawthrop) wins not on form but on content: Really this graph should stand alone but it’s so wonderful that I can’t resist pointing out a few things: - The gap between 610 and 622 A.D. seems to be about the same as the previous 600 years, and only a little less than the 1400 years before that. - “Pious and devout” Jews are portrayed as having steadily increased in nonviolence up to the present day. Been to Israel lately? - I assume the line labeled “Bible” is referring to Christians? I’m sort of amazed to see pious and devout Christians listed as being maximally violent at the beginning. Huh? I thought Christ was supposed to be a nonviolent, mellow dude. The line starts at 3 B.C., implying that baby Jesus was at the extreme of violence. Gong forward, we can learn from the graph that pious and devout Christians in 1492 or 1618, say, were much more peaceful than Jesus and his crew. - Most amusingly g
2 0.94770885 1632 andrew gelman stats-2012-12-20-Who exactly are those silly academics who aren’t as smart as a Vegas bookie?
Introduction: I get suspicious when I hear unsourced claims that unnamed experts somewhere are making foolish statements. For example, I recently came across this, from a Super Bowl-themed article from 2006 by Stephen Dubner and Steven Levitt: As it happens, there is one betting strategy that will routinely beat a bookie, and you don’t even have to be smart to use it. One of the most undervalued N.F.L. bets is the home underdog — a team favored to lose but playing in its home stadium. If you had bet $5,000 on the home underdog in every N.F.L. game over the past two decades, you would be up about $150,000 by now (a winning rate of roughly 53 percent). So far, so good. I wonder if this pattern still holds. But then Dubner and Levitt continue: This fact has led some academics to conclude that bookmakers simply aren’t very smart. If an academic researcher can find this loophole, shouldn’t a professional bookie be able to? But the fact is most bookies are doing just fine. So could it be
Introduction: I just want to share with you the best comment we’ve every had in the nearly ten-year history of this blog. Also it has statistical content! Here’s the story. After seeing an amusing article by Tom Scocca relating how reporter John Lee Anderson called someone as a “little twerp” on twitter: I conjectured that Anderson suffered from “tall person syndrome,” that problem that some people of above-average height have, that they think they’re more important than other people because they literally look down on them. But I had no idea of Anderson’s actual height. Commenter Gary responded with this impressive bit of investigative reporting: Based on this picture: he appears to be fairly tall. But the perspective makes it hard to judge. Based on this picture: he appears to be about 9-10 inches taller than Catalina Garcia. But how tall is Catalina Garcia? Not that tall – she’s shorter than the high-wire artist Phillipe Petit: And he doesn’t appear
4 0.93964016 552 andrew gelman stats-2011-02-03-Model Makers’ Hippocratic Oath
Introduction: Emanuel Derman and Paul Wilmott wonder how to get their fellow modelers to give up their fantasy of perfection. In a Business Week article they proposed, not entirely in jest, a model makers’ Hippocratic Oath: I will remember that I didn’t make the world and that it doesn’t satisfy my equations. Though I will use models boldly to estimate value, I will not be overly impressed by mathematics. I will never sacrifice reality for elegance without explaining why I have done so. Nor will I give the people who use my model false comfort about its accuracy. Instead, I will make explicit its assumptions and oversights. I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension. Found via Abductive Intelligence .
Introduction: This link on education reform send me to this blog on foreign languages in Canadian public schools: The demand for French immersion education in Vancouver so far outstrips the supply that the school board allocates places by lottery. But why? Is it because French is a useful employment skill? Because learning to speak French makes you a better person? Or is it because parents know intuitively what economists can show econometrically: peer effects matter. Being with high achieving peers raises a student’s own achievement level. . . . Several studies have found that Anglophones who can speak French enjoy an earning premium. The question is: do bilingual Anglophones earn more because speaking French is a valuable skill in the workplace? Or do they earn more because they’re on average smarter and more capable people (after all, they’ve mastered two languages)? And the blog features this comments like this : French immersion classes (as opposed to science, maths or any
same-blog 6 0.93121755 1962 andrew gelman stats-2013-07-30-The Roy causal model?
7 0.9184553 1033 andrew gelman stats-2011-11-28-Greece to head statistician: Tell the truth, go to jail
9 0.90681511 1705 andrew gelman stats-2013-02-04-Recently in the sister blog
10 0.90277469 1222 andrew gelman stats-2012-03-20-5 books book
11 0.89465952 556 andrew gelman stats-2011-02-04-Patterns
12 0.88393724 1321 andrew gelman stats-2012-05-15-A statistical research project: Weeding out the fraudulent citations
14 0.87667453 1057 andrew gelman stats-2011-12-14-Hey—I didn’t know that!
16 0.84706295 1755 andrew gelman stats-2013-03-09-Plaig
17 0.84198761 2002 andrew gelman stats-2013-08-30-Blogging
18 0.83784854 858 andrew gelman stats-2011-08-17-Jumping off the edge of the world
19 0.83720845 1759 andrew gelman stats-2013-03-12-How tall is Jon Lee Anderson?