andrew_gelman_stats andrew_gelman_stats-2012 andrew_gelman_stats-2012-1310 knowledge-graph by maker-knowledge-mining

1310 andrew gelman stats-2012-05-09-Varying treatment effects, again


meta infos for this blog

Source: html

Introduction: This time from Bernard Fraga and Eitan Hersh. Once you think about it, it’s hard to imagine any nonzero treatment effects that don’t vary. I’m glad to see this area of research becoming more prominent. ( Here ‘s a discussion of another political science example, also of voter turnout, from a few years ago, from Avi Feller and Chris Holmes.) Some of my fragmentary work on varying treatment effects is here (Treatment Effects in Before-After Data) and here (Estimating Incumbency Advantage and Its Variation, as an Example of a Before–After Study).


Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 Once you think about it, it’s hard to imagine any nonzero treatment effects that don’t vary. [sent-2, score-1.213]

2 I’m glad to see this area of research becoming more prominent. [sent-3, score-0.634]

3 ( Here ‘s a discussion of another political science example, also of voter turnout, from a few years ago, from Avi Feller and Chris Holmes. [sent-4, score-0.641]

4 ) Some of my fragmentary work on varying treatment effects is here (Treatment Effects in Before-After Data) and here (Estimating Incumbency Advantage and Its Variation, as an Example of a Before–After Study). [sent-5, score-0.945]


similar blogs computed by tfidf model

tfidf for this blog:

wordName wordTfidf (topN-words)

[('treatment', 0.411), ('effects', 0.301), ('avi', 0.291), ('bernard', 0.26), ('feller', 0.245), ('nonzero', 0.245), ('turnout', 0.231), ('incumbency', 0.225), ('voter', 0.208), ('becoming', 0.195), ('glad', 0.191), ('varying', 0.178), ('advantage', 0.161), ('chris', 0.161), ('estimating', 0.147), ('variation', 0.145), ('area', 0.141), ('imagine', 0.128), ('example', 0.107), ('hard', 0.093), ('ago', 0.092), ('study', 0.09), ('political', 0.087), ('discussion', 0.079), ('science', 0.078), ('another', 0.077), ('years', 0.07), ('research', 0.064), ('work', 0.055), ('time', 0.053), ('data', 0.044), ('see', 0.043), ('also', 0.042), ('think', 0.035)]

similar blogs list:

simIndex simValue blogId blogTitle

same-blog 1 0.99999994 1310 andrew gelman stats-2012-05-09-Varying treatment effects, again

Introduction: This time from Bernard Fraga and Eitan Hersh. Once you think about it, it’s hard to imagine any nonzero treatment effects that don’t vary. I’m glad to see this area of research becoming more prominent. ( Here ‘s a discussion of another political science example, also of voter turnout, from a few years ago, from Avi Feller and Chris Holmes.) Some of my fragmentary work on varying treatment effects is here (Treatment Effects in Before-After Data) and here (Estimating Incumbency Advantage and Its Variation, as an Example of a Before–After Study).

2 0.2913233 1891 andrew gelman stats-2013-06-09-“Heterogeneity of variance in experimental studies: A challenge to conventional interpretations”

Introduction: Avi sent along this old paper from Bryk and Raudenbush, who write: The presence of heterogeneity of variance across groups indicates that the standard statistical model for treatment effects no longer applies. Specifically, the assumption that treatments add a constant to each subject’s development fails. An alternative model is required to represent how treatment effects are distributed across individuals. We develop in this article a simple statistical model to demonstrate the link between heterogeneity of variance and random treatment effects. Next, we illustrate with results from two previously published studies how a failure to recognize the substantive importance of heterogeneity of variance obscured significant results present in these data. The article concludes with a review and synthesis of techniques for modeling variances. Although these methods have been well established in the statistical literature, they are not widely known by social and behavioral scientists. T

3 0.19248335 2 andrew gelman stats-2010-04-23-Modeling heterogenous treatment effects

Introduction: Don Green and Holger Kern write on one of my favorite topics , treatment interactions (see also here ): We [Green and Kern] present a methodology that largely automates the search for systematic treatment effect heterogeneity in large-scale experiments. We introduce a nonparametric estimator developed in statistical learning, Bayesian Additive Regression Trees (BART), to model treatment effects that vary as a function of covariates. BART has several advantages over commonly employed parametric modeling strategies, in particular its ability to automatically detect and model relevant treatment-covariate interactions in a flexible manner. To increase the reliability and credibility of the resulting conditional treatment effect estimates, we suggest the use of a split sample design. The data are randomly divided into two equally-sized parts, with the first part used to explore treatment effect heterogeneity and the second part used to confirm the results. This approach permits a re

4 0.17056434 1644 andrew gelman stats-2012-12-30-Fixed effects, followed by Bayes shrinkage?

Introduction: Stuart Buck writes: I have a question about fixed effects vs. random effects . Amongst economists who study teacher value-added, it has become common to see people saying that they estimated teacher fixed effects (via least squares dummy variables, so that there is a parameter for each teacher), but that they then applied empirical Bayes shrinkage so that the teacher effects are brought closer to the mean. (See this paper by Jacob and Lefgren, for example.) Can that really be what they are doing? Why wouldn’t they just run random (modeled) effects in the first place? I feel like there’s something I’m missing. My reply: I don’t know the full story here, but I’m thinking there are two goals, first to get an unbiased estimate of an overall treatment effect (and there the econometricians prefer so-called fixed effects; I disagree with them on this but I know where they’re coming from) and second to estimate individual teacher effects (and there it makes sense to use so-called

5 0.16767716 518 andrew gelman stats-2011-01-15-Regression discontinuity designs: looking for the keys under the lamppost?

Introduction: Jas sends along this paper (with Devin Caughey), entitled Regression-Discontinuity Designs and Popular Elections: Implications of Pro-Incumbent Bias in Close U.S. House Races, and writes: The paper shows that regression discontinuity does not work for US House elections. Close House elections are anything but random. It isn’t election recounts or something like that (we collect recount data to show that it isn’t). We have collected much new data to try to hunt down what is going on (e.g., campaign finance data, CQ pre-election forecasts, correct many errors in the Lee dataset). The substantive implications are interesting. We also have a section that compares in details Gelman and King versus the Lee estimand and estimator. I had a few comments: David Lee is not estimating the effect of incumbency; he’s estimating the effect of the incumbent party, which is a completely different thing. The regression discontinuity design is completely inappropriate for estimating the

6 0.13021904 1201 andrew gelman stats-2012-03-07-Inference = data + model

7 0.12903504 1150 andrew gelman stats-2012-02-02-The inevitable problems with statistical significance and 95% intervals

8 0.12006563 1675 andrew gelman stats-2013-01-15-“10 Things You Need to Know About Causal Effects”

9 0.11930387 1571 andrew gelman stats-2012-11-09-The anti-Bayesian moment and its passing

10 0.11904499 898 andrew gelman stats-2011-09-10-Fourteen magic words: an update

11 0.11901044 1869 andrew gelman stats-2013-05-24-In which I side with Neyman over Fisher

12 0.1187346 86 andrew gelman stats-2010-06-14-“Too much data”?

13 0.11176059 1194 andrew gelman stats-2012-03-04-Multilevel modeling even when you’re not interested in predictions for new groups

14 0.10758355 936 andrew gelman stats-2011-10-02-Covariate Adjustment in RCT - Model Overfitting in Multilevel Regression

15 0.10628715 2120 andrew gelman stats-2013-12-02-Does a professor’s intervention in online discussions have the effect of prolonging discussion or cutting it off?

16 0.10421233 678 andrew gelman stats-2011-04-25-Democrats do better among the most and least educated groups

17 0.10353823 388 andrew gelman stats-2010-11-01-The placebo effect in pharma

18 0.10261284 1267 andrew gelman stats-2012-04-17-Hierarchical-multilevel modeling with “big data”

19 0.102281 485 andrew gelman stats-2010-12-25-Unlogging

20 0.10191616 433 andrew gelman stats-2010-11-27-One way that psychology research is different than medical research


similar blogs computed by lsi model

lsi for this blog:

topicId topicWeight

[(0, 0.121), (1, 0.003), (2, 0.049), (3, -0.107), (4, -0.009), (5, -0.011), (6, -0.049), (7, 0.001), (8, 0.032), (9, 0.038), (10, -0.05), (11, 0.017), (12, 0.091), (13, -0.053), (14, 0.068), (15, 0.015), (16, -0.055), (17, 0.015), (18, -0.048), (19, 0.08), (20, -0.052), (21, -0.001), (22, -0.034), (23, -0.015), (24, -0.014), (25, 0.015), (26, -0.105), (27, 0.061), (28, -0.072), (29, 0.023), (30, -0.065), (31, -0.006), (32, -0.094), (33, -0.041), (34, -0.011), (35, -0.028), (36, -0.04), (37, -0.023), (38, 0.005), (39, -0.013), (40, 0.011), (41, 0.074), (42, 0.006), (43, -0.035), (44, -0.003), (45, 0.005), (46, 0.02), (47, -0.057), (48, -0.009), (49, 0.059)]

similar blogs list:

simIndex simValue blogId blogTitle

same-blog 1 0.97299063 1310 andrew gelman stats-2012-05-09-Varying treatment effects, again

Introduction: This time from Bernard Fraga and Eitan Hersh. Once you think about it, it’s hard to imagine any nonzero treatment effects that don’t vary. I’m glad to see this area of research becoming more prominent. ( Here ‘s a discussion of another political science example, also of voter turnout, from a few years ago, from Avi Feller and Chris Holmes.) Some of my fragmentary work on varying treatment effects is here (Treatment Effects in Before-After Data) and here (Estimating Incumbency Advantage and Its Variation, as an Example of a Before–After Study).

2 0.81800544 433 andrew gelman stats-2010-11-27-One way that psychology research is different than medical research

Introduction: Medical researchers care about main effects, psychologists care about interactions. In psychology, the main effects are typically obvious, and it’s only the interactions that are worth studying.

3 0.79392469 1644 andrew gelman stats-2012-12-30-Fixed effects, followed by Bayes shrinkage?

Introduction: Stuart Buck writes: I have a question about fixed effects vs. random effects . Amongst economists who study teacher value-added, it has become common to see people saying that they estimated teacher fixed effects (via least squares dummy variables, so that there is a parameter for each teacher), but that they then applied empirical Bayes shrinkage so that the teacher effects are brought closer to the mean. (See this paper by Jacob and Lefgren, for example.) Can that really be what they are doing? Why wouldn’t they just run random (modeled) effects in the first place? I feel like there’s something I’m missing. My reply: I don’t know the full story here, but I’m thinking there are two goals, first to get an unbiased estimate of an overall treatment effect (and there the econometricians prefer so-called fixed effects; I disagree with them on this but I know where they’re coming from) and second to estimate individual teacher effects (and there it makes sense to use so-called

4 0.79375881 1744 andrew gelman stats-2013-03-01-Why big effects are more important than small effects

Introduction: The title of this post is silly but I have an important point to make, regarding an implicit model which I think many people assume even though it does not really make sense. Following a link from Sanjay Srivastava, I came across a post from David Funder saying that it’s useful to talk about the sizes of effects (I actually prefer the term “comparisons” so as to avoid the causal baggage) rather than just their signs. I agree , and I wanted to elaborate a bit on a point that comes up in Funder’s discussion. He quotes an (unnamed) prominent social psychologist as writing: The key to our research . . . [is not] to accurately estimate effect size. If I were testing an advertisement for a marketing research firm and wanted to be sure that the cost of the ad would produce enough sales to make it worthwhile, effect size would be crucial. But when I am testing a theory about whether, say, positive mood reduces information processing in comparison with negative mood, I am worried abou

5 0.78776366 2 andrew gelman stats-2010-04-23-Modeling heterogenous treatment effects

Introduction: Don Green and Holger Kern write on one of my favorite topics , treatment interactions (see also here ): We [Green and Kern] present a methodology that largely automates the search for systematic treatment effect heterogeneity in large-scale experiments. We introduce a nonparametric estimator developed in statistical learning, Bayesian Additive Regression Trees (BART), to model treatment effects that vary as a function of covariates. BART has several advantages over commonly employed parametric modeling strategies, in particular its ability to automatically detect and model relevant treatment-covariate interactions in a flexible manner. To increase the reliability and credibility of the resulting conditional treatment effect estimates, we suggest the use of a split sample design. The data are randomly divided into two equally-sized parts, with the first part used to explore treatment effect heterogeneity and the second part used to confirm the results. This approach permits a re

6 0.76506203 1891 andrew gelman stats-2013-06-09-“Heterogeneity of variance in experimental studies: A challenge to conventional interpretations”

7 0.75750738 2165 andrew gelman stats-2014-01-09-San Fernando Valley cityscapes: An example of the benefits of fractal devastation?

8 0.73959404 1400 andrew gelman stats-2012-06-29-Decline Effect in Linguistics?

9 0.72916603 1186 andrew gelman stats-2012-02-27-Confusion from illusory precision

10 0.70584589 7 andrew gelman stats-2010-04-27-Should Mister P be allowed-encouraged to reside in counter-factual populations?

11 0.68494719 1241 andrew gelman stats-2012-04-02-Fixed effects and identification

12 0.6808449 963 andrew gelman stats-2011-10-18-Question on Type M errors

13 0.66579521 803 andrew gelman stats-2011-07-14-Subtleties with measurement-error models for the evaluation of wacky claims

14 0.65310246 2185 andrew gelman stats-2014-01-25-Xihong Lin on sparsity and density

15 0.65262198 797 andrew gelman stats-2011-07-11-How do we evaluate a new and wacky claim?

16 0.64369434 1513 andrew gelman stats-2012-09-27-Estimating seasonality with a data set that’s just 52 weeks long

17 0.63887417 64 andrew gelman stats-2010-06-03-Estimates of war deaths: Darfur edition

18 0.63533282 2097 andrew gelman stats-2013-11-11-Why ask why? Forward causal inference and reverse causal questions

19 0.63076758 1929 andrew gelman stats-2013-07-07-Stereotype threat!

20 0.62433171 2243 andrew gelman stats-2014-03-11-The myth of the myth of the myth of the hot hand


similar blogs computed by lda model

lda for this blog:

topicId topicWeight

[(16, 0.034), (24, 0.302), (63, 0.066), (69, 0.19), (99, 0.248)]

similar blogs list:

simIndex simValue blogId blogTitle

same-blog 1 0.93919373 1310 andrew gelman stats-2012-05-09-Varying treatment effects, again

Introduction: This time from Bernard Fraga and Eitan Hersh. Once you think about it, it’s hard to imagine any nonzero treatment effects that don’t vary. I’m glad to see this area of research becoming more prominent. ( Here ‘s a discussion of another political science example, also of voter turnout, from a few years ago, from Avi Feller and Chris Holmes.) Some of my fragmentary work on varying treatment effects is here (Treatment Effects in Before-After Data) and here (Estimating Incumbency Advantage and Its Variation, as an Example of a Before–After Study).

2 0.90715325 1376 andrew gelman stats-2012-06-12-Simple graph WIN: the example of birthday frequencies

Introduction: From Chris Mulligan: The data come from the Center for Disease Control and cover the years 1969-1988. Chris also gives instructions for how to download the data and plot them in R from scratch (in 30 lines of R code)! And now, the background A few months ago I heard about a study reporting that, during a recent eleven-year period, more babies were born on Valentine’s Day and fewer on Halloween compared to neighboring days: I wrote , What I’d really like to see is a graph with all 366 days of the year. It would be easy enough to make. That way we could put the Valentine’s and Halloween data in the context of other possible patterns. While they’re at it, they could also graph births by day of the week and show Thanksgiving, Easter, and other holidays that don’t have fixed dates. It’s so frustrating when people only show part of the story. I was pointed to some tables: and a graph from Matt Stiles: The heatmap is cute but I wanted to se

3 0.89261603 1787 andrew gelman stats-2013-04-04-Wanna be the next Tyler Cowen? It’s not as easy as you might think!

Introduction: Someone told me he ran into someone who said his goal was to be Tyler Cowen. OK, fine, it’s a worthy goal, but I don’t think it’s so easy .

4 0.89000666 898 andrew gelman stats-2011-09-10-Fourteen magic words: an update

Introduction: In the discussion of the fourteen magic words that can increase voter turnout by over 10 percentage points , questions were raised about the methods used to estimate the experimental effects. I sent these on to Chris Bryan, the author of the study, and he gave the following response: We’re happy to address the questions that have come up. It’s always noteworthy when a precise psychological manipulation like this one generates a large effect on a meaningful outcome. Such findings illustrate the power of the underlying psychological process. I’ve provided the contingency tables for the two turnout experiments below. As indicated in the paper, the data are analyzed using logistic regressions. The change in chi-squared statistic represents the significance of the noun vs. verb condition variable in predicting turnout; that is, the change in the model’s significance when the condition variable is added. This is a standard way to analyze dichotomous outcomes. Four outliers were excl

5 0.88850683 278 andrew gelman stats-2010-09-15-Advice that might make sense for individuals but is negative-sum overall

Introduction: There’s a lot of free advice out there. As I wrote a couple years ago, it’s usually presented as advice to individuals, but it’s also interesting to consider the possible total effects if the advice is taken. For example, Nassim Taleb has a webpage that includes a bunch of one-line bits of advice (scroll to item 132 on the linked page). Here’s his final piece of advice: If you dislike someone, leave him alone or eliminate him; don’t attack him verbally. I’m a big Taleb fan (search this blog to see), but this seems like classic negative-sum advice. I can see how it can be a good individual strategy to keep your mouth shut, bide your time, and then sandbag your enemies. But it can’t be good if lots of people are doing this. Verbal attacks are great, as long as there’s a chance to respond. I’ve been in environments where people follow Taleb’s advice, saying nothing and occasionally trying to “eliminate” people, and it’s not pretty. I much prefer for people to be open

6 0.88812613 743 andrew gelman stats-2011-06-03-An argument that can’t possibly make sense

7 0.88777721 1999 andrew gelman stats-2013-08-27-Bayesian model averaging or fitting a larger model

8 0.88771844 1455 andrew gelman stats-2012-08-12-Probabilistic screening to get an approximate self-weighted sample

9 0.88740242 197 andrew gelman stats-2010-08-10-The last great essayist?

10 0.88637108 1167 andrew gelman stats-2012-02-14-Extra babies on Valentine’s Day, fewer on Halloween?

11 0.88572085 1224 andrew gelman stats-2012-03-21-Teaching velocity and acceleration

12 0.88490164 1891 andrew gelman stats-2013-06-09-“Heterogeneity of variance in experimental studies: A challenge to conventional interpretations”

13 0.88464785 2143 andrew gelman stats-2013-12-22-The kluges of today are the textbook solutions of tomorrow.

14 0.8843708 482 andrew gelman stats-2010-12-23-Capitalism as a form of voluntarism

15 0.88402873 1706 andrew gelman stats-2013-02-04-Too many MC’s not enough MIC’s, or What principles should govern attempts to summarize bivariate associations in large multivariate datasets?

16 0.88252544 1092 andrew gelman stats-2011-12-29-More by Berger and me on weakly informative priors

17 0.88220978 414 andrew gelman stats-2010-11-14-“Like a group of teenagers on a bus, they behave in public as if they were in private”

18 0.88179338 847 andrew gelman stats-2011-08-10-Using a “pure infographic” to explore differences between information visualization and statistical graphics

19 0.88170302 938 andrew gelman stats-2011-10-03-Comparing prediction errors

20 0.8813554 1072 andrew gelman stats-2011-12-19-“The difference between . . .”: It’s not just p=.05 vs. p=.06