andrew_gelman_stats andrew_gelman_stats-2011 andrew_gelman_stats-2011-796 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Matthew Bogard writes: Regarding the book Mostly Harmless Econometrics, you state : A casual reader of the book might be left with the unfortunate impression that matching is a competitor to regression rather than a tool for making regression more effective. But in fact isn’t that what they are arguing, that, in a ‘mostly harmless way’ regression is in fact a matching estimator itself? “Our view is that regression can be motivated as a particular sort of weighted matching estimator, and therefore the differences between regression and matching estimates are unlikely to be of major empirical importance” (Chapter 3 p. 70) They seem to be distinguishing regression (without prior matching) from all other types of matching techniques, and therefore implying that regression can be a ‘mostly harmless’ substitute or competitor to matching. My previous understanding, before starting this book was as you say, that matching is a tool that makes regression more effective. I have n
sentIndex sentText sentNum sentScore
1 Matthew Bogard writes: Regarding the book Mostly Harmless Econometrics, you state : A casual reader of the book might be left with the unfortunate impression that matching is a competitor to regression rather than a tool for making regression more effective. [sent-1, score-1.899]
2 But in fact isn’t that what they are arguing, that, in a ‘mostly harmless way’ regression is in fact a matching estimator itself? [sent-2, score-1.489]
3 “Our view is that regression can be motivated as a particular sort of weighted matching estimator, and therefore the differences between regression and matching estimates are unlikely to be of major empirical importance” (Chapter 3 p. [sent-3, score-2.178]
4 70) They seem to be distinguishing regression (without prior matching) from all other types of matching techniques, and therefore implying that regression can be a ‘mostly harmless’ substitute or competitor to matching. [sent-4, score-1.827]
5 My previous understanding, before starting this book was as you say, that matching is a tool that makes regression more effective. [sent-5, score-1.199]
6 I have not finished their book, and have been working at it for a while, but if they do not mean to propose OLS itself as a matching estimator, then I agree that they definitely need some clarification. [sent-6, score-0.748]
7 I actually found your particular post searching for some article that discussed this more formally, as I found my interpretation (misinterpretation) difficult to accept. [sent-7, score-0.101]
8 I’m sorry to report that many users of matching do seem to think of it as a pure substitute for regression: once they decide to use matching, they try to do it perfectly and they often don’t realize they can use regression on the matched data to do even better. [sent-10, score-1.27]
9 In my book with Jennifer, we try to clarify that the primary role of matching is to correct for lack of complete overlap between control and treatment groups. [sent-11, score-0.857]
10 But I think in their comment you quoted above, Angrist and Pischke are just giving a conceptual perspective rather than detailed methodological advice. [sent-12, score-0.048]
11 They’re saying that regression, like matching, is a way of comparing-like-with-like in estimating a comparison. [sent-13, score-0.067]
12 This point seems commonplace from a statistical standpoint but may be news to some economists who might think that regression relies on the linear model being true. [sent-14, score-0.601]
13 Gary King and I discuss this general idea in our 1990 paper on estimating incumbency advantage. [sent-15, score-0.119]
14 Basically, a regression model works if either of two assumptions is satisfied: if the linear model is true, or if the two groups are balanced so that you’re getting an average treatment effect. [sent-16, score-0.62]
15 In many examples, neither regression nor matching works perfectly, which is why it can be better to do both (as Don Rubin discussed in his Ph. [sent-18, score-1.142]
16 thesis in 1970 and subsequently in some published articles with his advisor, William Cochran). [sent-20, score-0.07]
wordName wordTfidf (topN-words)
[('matching', 0.649), ('regression', 0.378), ('harmless', 0.209), ('pischke', 0.2), ('angrist', 0.193), ('estimator', 0.157), ('competitor', 0.129), ('substitute', 0.107), ('mostly', 0.101), ('book', 0.096), ('perfectly', 0.084), ('tool', 0.076), ('therefore', 0.074), ('misinterpretation', 0.074), ('subsequently', 0.07), ('equating', 0.07), ('estimating', 0.067), ('ensuring', 0.067), ('linear', 0.066), ('ols', 0.064), ('treatment', 0.063), ('advisor', 0.061), ('cochran', 0.061), ('distinguishing', 0.061), ('works', 0.06), ('bases', 0.059), ('commonplace', 0.057), ('discussed', 0.055), ('satisfied', 0.054), ('robustness', 0.054), ('balanced', 0.053), ('relies', 0.053), ('matched', 0.052), ('incumbency', 0.052), ('unfortunate', 0.051), ('implying', 0.051), ('propose', 0.05), ('weighted', 0.05), ('finished', 0.049), ('formally', 0.049), ('overlap', 0.049), ('conceptual', 0.048), ('fact', 0.048), ('double', 0.048), ('william', 0.047), ('balance', 0.047), ('standpoint', 0.047), ('casual', 0.046), ('matthew', 0.046), ('searching', 0.046)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 796 andrew gelman stats-2011-07-10-Matching and regression: two great tastes etc etc
Introduction: Matthew Bogard writes: Regarding the book Mostly Harmless Econometrics, you state : A casual reader of the book might be left with the unfortunate impression that matching is a competitor to regression rather than a tool for making regression more effective. But in fact isn’t that what they are arguing, that, in a ‘mostly harmless way’ regression is in fact a matching estimator itself? “Our view is that regression can be motivated as a particular sort of weighted matching estimator, and therefore the differences between regression and matching estimates are unlikely to be of major empirical importance” (Chapter 3 p. 70) They seem to be distinguishing regression (without prior matching) from all other types of matching techniques, and therefore implying that regression can be a ‘mostly harmless’ substitute or competitor to matching. My previous understanding, before starting this book was as you say, that matching is a tool that makes regression more effective. I have n
2 0.5335117 375 andrew gelman stats-2010-10-28-Matching for preprocessing data for causal inference
Introduction: Chris Blattman writes : Matching is not an identification strategy a solution to your endogeneity problem; it is a weighting scheme. Saying matching will reduce endogeneity bias is like saying that the best way to get thin is to weigh yourself in kilos. The statement makes no sense. It confuses technique with substance. . . . When you run a regression, you control for the X you can observe. When you match, you are simply matching based on those same X. . . . I see what Chris is getting at–matching, like regression, won’t help for the variables you’re not controlling for–but I disagree with his characterization of matching as a weighting scheme. I see matching as a way to restrict your analysis to comparable cases. The statistical motivation: robustness. If you had a good enough model, you wouldn’t neet to match, you’d just fit the model to the data. But in common practice we often use simple regression models and so it can be helpful to do some matching first before regress
3 0.16064063 32 andrew gelman stats-2010-05-14-Causal inference in economics
Introduction: Aaron Edlin points me to this issue of the Journal of Economic Perspectives that focuses on statistical methods for causal inference in economics. (Michael Bishop’s page provides some links .) To quickly summarize my reactions to Angrist and Pischke’s book: I pretty much agree with them that the potential-outcomes or natural-experiment approach is the most useful way to think about causality in economics and related fields. My main amendments to Angrist and Pischke would be to recognize that: 1. Modeling is important, especially modeling of interactions . It’s unfortunate to see a debate between experimentalists and modelers. Some experimenters (not Angrist and Pischke) make the mistake of avoiding models: Once they have their experimental data, they check their brains at the door and do nothing but simple differences, not realizing how much more can be learned. Conversely, some modelers are unduly dismissive of experiments and formal observational studies, forgetting t
4 0.13695721 1656 andrew gelman stats-2013-01-05-Understanding regression models and regression coefficients
Introduction: David Hoaglin writes: After seeing it cited, I just read your paper in Technometrics. The home radon levels provide an interesting and instructive example. I [Hoaglin] have a different take on the difficulty of interpreting the estimated coefficient of the county-level basement proportion (gamma-sub-2) on page 434. An important part of the difficulty involves “other things being equal.” That sounds like the widespread interpretation of a regression coefficient as telling how the dependent variable responds to change in that predictor when the other predictors are held constant. Unfortunately, as a general interpretation, that language is oversimplified; it doesn’t reflect how regression actually works. The appropriate general interpretation is that the coefficient tells how the dependent variable responds to change in that predictor after allowing for simultaneous change in the other predictors in the data at hand. Thus, in the county-level regression gamma-sub-2 summarize
5 0.13428499 144 andrew gelman stats-2010-07-13-Hey! Here’s a referee report for you!
Introduction: I just wrote this, and I realized it might be useful more generally: The article looks reasonable to me–but I just did a shallow read and didn’t try to judge whether the conclusions are correct. My main comment is that if they’re doing a Poisson regression, they should really be doing an overdispersed Poisson regression. I don’t know if I’ve ever seen data in my life where the non-overdispersed Poisson is appropriate. Also, I’d like to see a before-after plot with dots for control cases and open circles for treatment cases and fitted regression lines drawn in. Whenever there’s a regression I like to see this scatterplot. The scatterplot isn’t a replacement for the regression, but at the very least it gives me intuition as to the scale of the estimated effect. Finally, all their numbers should be rounded appropriately. Feel free to cut-and-paste this into your own referee reports (and to apply these recommendations in your own applied research).
6 0.13300268 213 andrew gelman stats-2010-08-17-Matching at two levels
7 0.13074116 451 andrew gelman stats-2010-12-05-What do practitioners need to know about regression?
10 0.11825454 2357 andrew gelman stats-2014-06-02-Why we hate stepwise regression
12 0.11596796 957 andrew gelman stats-2011-10-14-Questions about a study of charter schools
13 0.11096927 1967 andrew gelman stats-2013-08-04-What are the key assumptions of linear regression?
14 0.11043686 1735 andrew gelman stats-2013-02-24-F-f-f-fake data
15 0.10761464 1663 andrew gelman stats-2013-01-09-The effects of fiscal consolidation
16 0.10507508 86 andrew gelman stats-2010-06-14-“Too much data”?
17 0.10299093 952 andrew gelman stats-2011-10-11-More reason to like Sims besides just his name
18 0.10236467 146 andrew gelman stats-2010-07-14-The statistics and the science
19 0.10179784 669 andrew gelman stats-2011-04-19-The mysterious Gamma (1.4, 0.4)
20 0.10165693 518 andrew gelman stats-2011-01-15-Regression discontinuity designs: looking for the keys under the lamppost?
topicId topicWeight
[(0, 0.151), (1, 0.068), (2, 0.036), (3, -0.012), (4, 0.047), (5, 0.015), (6, 0.005), (7, -0.029), (8, 0.088), (9, 0.097), (10, 0.054), (11, 0.006), (12, 0.015), (13, 0.002), (14, 0.068), (15, 0.014), (16, -0.023), (17, 0.028), (18, -0.007), (19, -0.014), (20, -0.004), (21, 0.04), (22, 0.034), (23, 0.008), (24, 0.028), (25, 0.067), (26, 0.082), (27, -0.076), (28, -0.082), (29, 0.012), (30, 0.018), (31, 0.045), (32, 0.004), (33, 0.046), (34, -0.02), (35, 0.01), (36, 0.006), (37, 0.018), (38, -0.014), (39, 0.027), (40, 0.003), (41, 0.001), (42, -0.032), (43, -0.044), (44, 0.128), (45, 0.049), (46, -0.016), (47, 0.02), (48, 0.013), (49, 0.01)]
simIndex simValue blogId blogTitle
same-blog 1 0.98182011 796 andrew gelman stats-2011-07-10-Matching and regression: two great tastes etc etc
Introduction: Matthew Bogard writes: Regarding the book Mostly Harmless Econometrics, you state : A casual reader of the book might be left with the unfortunate impression that matching is a competitor to regression rather than a tool for making regression more effective. But in fact isn’t that what they are arguing, that, in a ‘mostly harmless way’ regression is in fact a matching estimator itself? “Our view is that regression can be motivated as a particular sort of weighted matching estimator, and therefore the differences between regression and matching estimates are unlikely to be of major empirical importance” (Chapter 3 p. 70) They seem to be distinguishing regression (without prior matching) from all other types of matching techniques, and therefore implying that regression can be a ‘mostly harmless’ substitute or competitor to matching. My previous understanding, before starting this book was as you say, that matching is a tool that makes regression more effective. I have n
2 0.85507452 2357 andrew gelman stats-2014-06-02-Why we hate stepwise regression
Introduction: Haynes Goddard writes: I have been slowly working my way through the grad program in stats here, and the latest course was a biostats course on categorical and survival analysis. I noticed in the semi-parametric and parametric material (Wang and Lee is the text) that they use stepwise regression a lot. I learned in econometrics that stepwise is poor practice, as it defaults to the “theory of the regression line”, that is no theory at all, just the variation in the data. I don’t find the topic on your blog, and wonder if you have addressed the issue. My reply: Stepwise regression is one of these things, like outlier detection and pie charts, which appear to be popular among non-statisticans but are considered by statisticians to be a bit of a joke. For example, Jennifer and I don’t mention stepwise regression in our book, not even once. To address the issue more directly: the motivation behind stepwise regression is that you have a lot of potential predictors but not e
3 0.84203702 375 andrew gelman stats-2010-10-28-Matching for preprocessing data for causal inference
Introduction: Chris Blattman writes : Matching is not an identification strategy a solution to your endogeneity problem; it is a weighting scheme. Saying matching will reduce endogeneity bias is like saying that the best way to get thin is to weigh yourself in kilos. The statement makes no sense. It confuses technique with substance. . . . When you run a regression, you control for the X you can observe. When you match, you are simply matching based on those same X. . . . I see what Chris is getting at–matching, like regression, won’t help for the variables you’re not controlling for–but I disagree with his characterization of matching as a weighting scheme. I see matching as a way to restrict your analysis to comparable cases. The statistical motivation: robustness. If you had a good enough model, you wouldn’t neet to match, you’d just fit the model to the data. But in common practice we often use simple regression models and so it can be helpful to do some matching first before regress
4 0.81933635 1967 andrew gelman stats-2013-08-04-What are the key assumptions of linear regression?
Introduction: Andy Cooper writes: A link to an article , “Four Assumptions Of Multiple Regression That Researchers Should Always Test”, has been making the rounds on Twitter. Their first rule is “Variables are Normally distributed.” And they seem to be talking about the independent variables – but then later bring in tests on the residuals (while admitting that the normally-distributed error assumption is a weak assumption). I thought we had long-since moved away from transforming our independent variables to make them normally distributed for statistical reasons (as opposed to standardizing them for interpretability, etc.) Am I missing something? I agree that leverage in a influence is important, but normality of the variables? The article is from 2002, so it might be dated, but given the popularity of the tweet, I thought I’d ask your opinion. My response: There’s some useful advice on that page but overall I think the advice was dated even in 2002. In section 3.6 of my book wit
5 0.81203234 1656 andrew gelman stats-2013-01-05-Understanding regression models and regression coefficients
Introduction: David Hoaglin writes: After seeing it cited, I just read your paper in Technometrics. The home radon levels provide an interesting and instructive example. I [Hoaglin] have a different take on the difficulty of interpreting the estimated coefficient of the county-level basement proportion (gamma-sub-2) on page 434. An important part of the difficulty involves “other things being equal.” That sounds like the widespread interpretation of a regression coefficient as telling how the dependent variable responds to change in that predictor when the other predictors are held constant. Unfortunately, as a general interpretation, that language is oversimplified; it doesn’t reflect how regression actually works. The appropriate general interpretation is that the coefficient tells how the dependent variable responds to change in that predictor after allowing for simultaneous change in the other predictors in the data at hand. Thus, in the county-level regression gamma-sub-2 summarize
6 0.78691995 451 andrew gelman stats-2010-12-05-What do practitioners need to know about regression?
8 0.75011063 144 andrew gelman stats-2010-07-13-Hey! Here’s a referee report for you!
9 0.73298305 1849 andrew gelman stats-2013-05-09-Same old same old
10 0.72192359 770 andrew gelman stats-2011-06-15-Still more Mr. P in public health
11 0.72010499 553 andrew gelman stats-2011-02-03-is it possible to “overstratify” when assigning a treatment in a randomized control trial?
12 0.71197903 1870 andrew gelman stats-2013-05-26-How to understand coefficients that reverse sign when you start controlling for things?
13 0.69207716 1535 andrew gelman stats-2012-10-16-Bayesian analogue to stepwise regression?
14 0.68396264 1815 andrew gelman stats-2013-04-20-Displaying inferences from complex models
15 0.67607957 1814 andrew gelman stats-2013-04-20-A mess with which I am comfortable
16 0.67035866 1121 andrew gelman stats-2012-01-15-R-squared for multilevel models
17 0.67034566 1663 andrew gelman stats-2013-01-09-The effects of fiscal consolidation
18 0.66982704 1294 andrew gelman stats-2012-05-01-Modeling y = a + b + c
19 0.66644031 1900 andrew gelman stats-2013-06-15-Exploratory multilevel analysis when group-level variables are of importance
20 0.66009349 1196 andrew gelman stats-2012-03-04-Piss-poor monocausal social science
topicId topicWeight
[(5, 0.029), (7, 0.013), (15, 0.053), (16, 0.048), (21, 0.01), (22, 0.014), (24, 0.126), (34, 0.023), (61, 0.012), (63, 0.019), (76, 0.055), (84, 0.013), (85, 0.086), (86, 0.026), (96, 0.012), (99, 0.304)]
simIndex simValue blogId blogTitle
same-blog 1 0.97283155 796 andrew gelman stats-2011-07-10-Matching and regression: two great tastes etc etc
Introduction: Matthew Bogard writes: Regarding the book Mostly Harmless Econometrics, you state : A casual reader of the book might be left with the unfortunate impression that matching is a competitor to regression rather than a tool for making regression more effective. But in fact isn’t that what they are arguing, that, in a ‘mostly harmless way’ regression is in fact a matching estimator itself? “Our view is that regression can be motivated as a particular sort of weighted matching estimator, and therefore the differences between regression and matching estimates are unlikely to be of major empirical importance” (Chapter 3 p. 70) They seem to be distinguishing regression (without prior matching) from all other types of matching techniques, and therefore implying that regression can be a ‘mostly harmless’ substitute or competitor to matching. My previous understanding, before starting this book was as you say, that matching is a tool that makes regression more effective. I have n
2 0.96407694 2300 andrew gelman stats-2014-04-21-Ticket to Baaaath
Introduction: Ooooooh, I never ever thought I’d have a legitimate excuse to tell this story, and now I do! The story took place many years ago, but first I have to tell you what made me think of it: Rasmus Bååth posted the following comment last month: On airplane tickets a Swedish “å” is written as “aa” resulting in Rasmus Baaaath. Once I bought a ticket online and five minutes later a guy from Lufthansa calls me and asks if I misspelled my name… OK, now here’s my story (which is not nearly as good). A long time ago (but when I was already an adult), I was in England for some reason, and I thought I’d take a day trip from London to Bath. So here I am on line, trying to think of what to say at the ticket counter. I remember that in England, they call Bath, Bahth. So, should I ask for “a ticket to Bahth”? I’m not sure, I’m afraid that it will sound silly, like I’m trying to fake an English accent. So, when I get to the front of the line, I say, hesitantly, “I’d like a ticket to Bath?
Introduction: Joan Nix writes: Your comments on this paper by Scott Carrell and James West would be most appreciated. I’m afraid the conclusions of this paper are too strong given the data set and other plausible explanations. But given where it is published, this paper is receiving and will continue to receive lots of attention. It will be used to draw deeper conclusions regarding effective teaching and experience. Nix also links to this discussion by Jeff Ely. I don’t completely follow Ely’s criticism, which seems to me to be too clever by half, but I agree with Nix that the findings in the research article don’t seem to fit together very well. For example, Carrell and West estimate that the effects of instructors on performance in the follow-on class is as large as the effects on the class they’re teaching. This seems hard to believe, and it seems central enough to their story that I don’t know what to think about everything else in the paper. My other thought about teaching eva
4 0.96101916 1374 andrew gelman stats-2012-06-11-Convergence Monitoring for Non-Identifiable and Non-Parametric Models
Introduction: Becky Passonneau and colleagues at the Center for Computational Learning Systems (CCLS) at Columbia have been working on a project for ConEd (New York’s major electric utility) to rank structures based on vulnerability to secondary events (e.g., transformer explosions, cable meltdowns, electrical fires). They’ve been using the R implementation BayesTree of Chipman, George and McCulloch’s Bayesian Additive Regression Trees (BART). BART is a Bayesian non-parametric method that is non-identifiable in two ways. Firstly, it is an additive tree model with a fixed number of trees, the indexes of which aren’t identified (you get the same predictions in a model swapping the order of the trees). This is the same kind of non-identifiability you get with any mixture model (additive or interpolated) with an exchangeable prior on the mixture components. Secondly, the trees themselves have varying structure over samples in terms of number of nodes and their topology (depth, branching, etc
5 0.9591133 417 andrew gelman stats-2010-11-17-Clutering and variance components
Introduction: Raymond Lim writes: Do you have any recommendations on clustering and binary models? My particular problem is I’m running a firm fixed effect logit and want to cluster by industry-year (every combination of industry-year). My control variable of interest in measured by industry-year and when I cluster by industry-year, the standard errors are 300x larger than when I don’t cluster. Strangely, this problem only occurs when doing logit and not OLS (linear probability). Also, clustering just by field doesn’t blow up the errors. My hunch is it has something to do with the non-nested structure of year, but I don’t understand why this is only problematic under logit and not OLS. My reply: I’d recommend including four multilevel variance parameters, one for firm, one for industry, one for year, and one for industry-year. (In lmer, that’s (1 | firm) + (1 | industry) + (1 | year) + (1 | industry.year)). No need to include (1 | firm.year) since in your data this is the error term. Try
6 0.95848799 167 andrew gelman stats-2010-07-27-Why don’t more medical discoveries become cures?
7 0.95687574 584 andrew gelman stats-2011-02-22-“Are Wisconsin Public Employees Underpaid?”
8 0.95478213 375 andrew gelman stats-2010-10-28-Matching for preprocessing data for causal inference
10 0.95271134 1187 andrew gelman stats-2012-02-27-“Apple confronts the law of large numbers” . . . huh?
11 0.94919544 1610 andrew gelman stats-2012-12-06-Yes, checking calibration of probability forecasts is part of Bayesian statistics
12 0.94913453 2013 andrew gelman stats-2013-09-08-What we need here is some peer review for statistical graphics
13 0.9488371 1269 andrew gelman stats-2012-04-19-Believe your models (up to the point that you abandon them)
14 0.94838566 1683 andrew gelman stats-2013-01-19-“Confirmation, on the other hand, is not sexy”
15 0.94736999 2304 andrew gelman stats-2014-04-24-An open site for researchers to post and share papers
16 0.94725031 32 andrew gelman stats-2010-05-14-Causal inference in economics
18 0.94709176 257 andrew gelman stats-2010-09-04-Question about standard range for social science correlations
19 0.94690067 1850 andrew gelman stats-2013-05-10-The recursion of pop-econ
20 0.94688118 843 andrew gelman stats-2011-08-07-Non-rant