andrew_gelman_stats andrew_gelman_stats-2014 andrew_gelman_stats-2014-2274 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Jacob Felson writes: Say we have a statistically significant interaction in non-experimental data between two continuous predictors, X and Z and it is unclear which variable is primarily a cause and which variable is primarily a moderator. One person might find it more plausible to think of X as a cause and Z as a moderator and another person may think the reverse more plausible. My question then is whether there is are any set of rules or heuristics you could recommend to help adjudicate between alternate perspectives on such an interaction term. My reply: I think in this setting, it would make sense to think about different interventions, some of which affect X, others of which affect Z, others of which affect both, and go from there. Rather than trying to isolate a single causal path, consider different cases of forward casual inference. My guess is that the different stories regarding moderators etc. could motivate different thought experiments (and, ultimately, differe
sentIndex sentText sentNum sentScore
1 Jacob Felson writes: Say we have a statistically significant interaction in non-experimental data between two continuous predictors, X and Z and it is unclear which variable is primarily a cause and which variable is primarily a moderator. [sent-1, score-1.578]
2 One person might find it more plausible to think of X as a cause and Z as a moderator and another person may think the reverse more plausible. [sent-2, score-0.968]
3 My question then is whether there is are any set of rules or heuristics you could recommend to help adjudicate between alternate perspectives on such an interaction term. [sent-3, score-1.184]
4 My reply: I think in this setting, it would make sense to think about different interventions, some of which affect X, others of which affect Z, others of which affect both, and go from there. [sent-4, score-1.584]
5 Rather than trying to isolate a single causal path, consider different cases of forward casual inference. [sent-5, score-0.702]
6 My guess is that the different stories regarding moderators etc. [sent-6, score-0.764]
7 could motivate different thought experiments (and, ultimately, different observational studies) regarding different potential interventions. [sent-7, score-1.46]
8 So I would not try to “adjudicate� [sent-8, score-0.08]
9 between different stories; rather, I’d recognize that they could all be appropriate, just corresponding to different interventions. [sent-9, score-0.85]
10 Also, all the above would hold even if there are only main effects, no interactions needed. [sent-10, score-0.336]
11 And, for that matter, statistical significance would not be needed either for you to look at these questions. [sent-11, score-0.248]
wordName wordTfidf (topN-words)
[('adjudicate', 0.318), ('different', 0.289), ('affect', 0.283), ('primarily', 0.258), ('interaction', 0.203), ('cause', 0.188), ('moderators', 0.172), ('moderator', 0.165), ('stories', 0.162), ('alternate', 0.15), ('heuristics', 0.15), ('variable', 0.149), ('isolate', 0.141), ('regarding', 0.141), ('unclear', 0.136), ('jacob', 0.131), ('interventions', 0.129), ('person', 0.127), ('perspectives', 0.118), ('motivate', 0.116), ('casual', 0.114), ('reverse', 0.109), ('path', 0.107), ('others', 0.104), ('corresponding', 0.103), ('observational', 0.103), ('interactions', 0.094), ('plausible', 0.094), ('continuous', 0.092), ('hold', 0.091), ('recognize', 0.09), ('rules', 0.087), ('needed', 0.087), ('predictors', 0.086), ('forward', 0.085), ('ultimately', 0.082), ('experiments', 0.081), ('significance', 0.081), ('would', 0.08), ('setting', 0.08), ('could', 0.079), ('recommend', 0.079), ('think', 0.079), ('appropriate', 0.076), ('statistically', 0.076), ('rather', 0.075), ('causal', 0.073), ('potential', 0.073), ('main', 0.071), ('significant', 0.069)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000002 2274 andrew gelman stats-2014-03-30-Adjudicating between alternative interpretations of a statistical interaction?
Introduction: Jacob Felson writes: Say we have a statistically significant interaction in non-experimental data between two continuous predictors, X and Z and it is unclear which variable is primarily a cause and which variable is primarily a moderator. One person might find it more plausible to think of X as a cause and Z as a moderator and another person may think the reverse more plausible. My question then is whether there is are any set of rules or heuristics you could recommend to help adjudicate between alternate perspectives on such an interaction term. My reply: I think in this setting, it would make sense to think about different interventions, some of which affect X, others of which affect Z, others of which affect both, and go from there. Rather than trying to isolate a single causal path, consider different cases of forward casual inference. My guess is that the different stories regarding moderators etc. could motivate different thought experiments (and, ultimately, differe
2 0.17932488 888 andrew gelman stats-2011-09-03-A psychology researcher asks: Is Anova dead?
Introduction: A research psychologist writes in with a question that’s so long that I’ll put my answer first, then put the question itself below the fold. Here’s my reply: As I wrote in my Anova paper and in my book with Jennifer Hill, I do think that multilevel models can completely replace Anova. At the same time, I think the central idea of Anova should persist in our understanding of these models. To me the central idea of Anova is not F-tests or p-values or sums of squares, but rather the idea of predicting an outcome based on factors with discrete levels, and understanding these factors using variance components. The continuous or categorical response thing doesn’t really matter so much to me. I have no problem using a normal linear model for continuous outcomes (perhaps suitably transformed) and a logistic model for binary outcomes. I don’t want to throw away interactions just because they’re not statistically significant. I’d rather partially pool them toward zero using an inform
Introduction: Consider two broad classes of inferential questions : 1. Forward causal inference . What might happen if we do X? What are the effects of smoking on health, the effects of schooling on knowledge, the effect of campaigns on election outcomes, and so forth? 2. Reverse causal inference . What causes Y? Why do more attractive people earn more money? Why do many poor people vote for Republicans and rich people vote for Democrats? Why did the economy collapse? When statisticians and econometricians write about causal inference, they focus on forward causal questions. Rubin always told us: Never ask Why? Only ask What if? And, from the econ perspective, causation is typically framed in terms of manipulations: if x had changed by 1, how much would y be expected to change, holding all else constant? But reverse causal questions are important too. They’re a natural way to think (consider the importance of the word “Why”) and are arguably more important than forward questions.
Introduction: As I’ve written here many times, my experiences in social science and public health research have left me skeptical of statistical methods that hypothesize or try to detect zero relationships between observational data (see, for example, the discussion starting at the bottom of page 960 in my review of causal inference in the American Journal of Sociology). In short, I have a taste for continuous rather than discrete models. As discussed in the above-linked article (with respect to the writings of cognitive scientist Steven Sloman), I think that common-sense thinking about causal inference can often mislead. In many cases, I have found that that the theoretical frameworks of instrumental variables and potential outcomes (for a review see, for example, chapters 9 and 10 of my book with Jennifer) help clarify my thinking. Here is an example that came up in a recent blog discussion. Computer science student Elias Bareinboim gave the following example: “suppose we know nothing a
5 0.14132255 807 andrew gelman stats-2011-07-17-Macro causality
Introduction: David Backus writes: This is from my area of work, macroeconomics. The suggestion here is that the economy is growing slowly because consumers aren’t spending money. But how do we know it’s not the reverse: that consumers are spending less because the economy isn’t doing well. As a teacher, I can tell you that it’s almost impossible to get students to understand that the first statement isn’t obviously true. What I’d call the demand-side story (more spending leads to more output) is everywhere, including this piece, from the usually reliable David Leonhardt. This whole situation reminds me of the story of the village whose inhabitants support themselves by taking in each others’ laundry. I guess we’re rich enough in the U.S. that we can stay afloat for a few decades just buying things from each other? Regarding the causal question, I’d like to move away from the idea of “Does A causes B or does B cause A” and toward a more intervention-based framework (Rubin’s model for
6 0.13114859 1675 andrew gelman stats-2013-01-15-“10 Things You Need to Know About Causal Effects”
7 0.11915234 1908 andrew gelman stats-2013-06-21-Interpreting interactions in discrete-data regression
9 0.10904559 726 andrew gelman stats-2011-05-22-Handling multiple versions of an outcome variable
10 0.10733988 753 andrew gelman stats-2011-06-09-Allowing interaction terms to vary
13 0.10221543 823 andrew gelman stats-2011-07-26-Including interactions or not
14 0.10018904 2268 andrew gelman stats-2014-03-26-New research journal on observational studies
15 0.098271862 1744 andrew gelman stats-2013-03-01-Why big effects are more important than small effects
16 0.096671917 1247 andrew gelman stats-2012-04-05-More philosophy of Bayes
17 0.093780577 255 andrew gelman stats-2010-09-04-How does multilevel modeling affect the estimate of the grand mean?
18 0.092534855 855 andrew gelman stats-2011-08-16-Infovis and statgraphics update update
20 0.08876887 1070 andrew gelman stats-2011-12-19-The scope for snooping
topicId topicWeight
[(0, 0.174), (1, 0.022), (2, 0.036), (3, -0.083), (4, 0.03), (5, -0.029), (6, -0.021), (7, 0.0), (8, 0.066), (9, 0.069), (10, -0.038), (11, 0.041), (12, 0.029), (13, -0.056), (14, 0.051), (15, 0.009), (16, -0.034), (17, -0.024), (18, -0.018), (19, 0.077), (20, -0.022), (21, -0.03), (22, 0.043), (23, 0.02), (24, 0.005), (25, 0.063), (26, 0.048), (27, -0.023), (28, -0.042), (29, 0.01), (30, 0.033), (31, 0.01), (32, -0.02), (33, 0.035), (34, -0.004), (35, 0.001), (36, -0.016), (37, 0.026), (38, -0.017), (39, 0.025), (40, -0.031), (41, -0.041), (42, 0.02), (43, 0.012), (44, -0.027), (45, -0.027), (46, 0.041), (47, 0.037), (48, -0.008), (49, 0.008)]
simIndex simValue blogId blogTitle
same-blog 1 0.96506119 2274 andrew gelman stats-2014-03-30-Adjudicating between alternative interpretations of a statistical interaction?
Introduction: Jacob Felson writes: Say we have a statistically significant interaction in non-experimental data between two continuous predictors, X and Z and it is unclear which variable is primarily a cause and which variable is primarily a moderator. One person might find it more plausible to think of X as a cause and Z as a moderator and another person may think the reverse more plausible. My question then is whether there is are any set of rules or heuristics you could recommend to help adjudicate between alternate perspectives on such an interaction term. My reply: I think in this setting, it would make sense to think about different interventions, some of which affect X, others of which affect Z, others of which affect both, and go from there. Rather than trying to isolate a single causal path, consider different cases of forward casual inference. My guess is that the different stories regarding moderators etc. could motivate different thought experiments (and, ultimately, differe
Introduction: As I’ve written here many times, my experiences in social science and public health research have left me skeptical of statistical methods that hypothesize or try to detect zero relationships between observational data (see, for example, the discussion starting at the bottom of page 960 in my review of causal inference in the American Journal of Sociology). In short, I have a taste for continuous rather than discrete models. As discussed in the above-linked article (with respect to the writings of cognitive scientist Steven Sloman), I think that common-sense thinking about causal inference can often mislead. In many cases, I have found that that the theoretical frameworks of instrumental variables and potential outcomes (for a review see, for example, chapters 9 and 10 of my book with Jennifer) help clarify my thinking. Here is an example that came up in a recent blog discussion. Computer science student Elias Bareinboim gave the following example: “suppose we know nothing a
3 0.79042798 393 andrew gelman stats-2010-11-04-Estimating the effect of A on B, and also the effect of B on A
Introduction: Lei Liu writes: I am working with clinicians in infectious disease and international health to study the (possible causal) relation between malnutrition and virus infection episodes (e.g., diarrhea) in babies in developing countries. Basically the clinicians are interested in two questions: does malnutrition cause more diarrhea episodes? does diarrhea lead to malnutrition? The malnutrition status is indicated by height and weight (adjusted, HAZ and WAZ measures) observed every 3 months from birth to 1 year. They also recorded the time of each diarrhea episode during the 1 year follow-up period. They have very solid datasets for analysis. As you can see, this is almost like a chicken and egg problem. I am a layman to causal inference. The method I use is just to do some simple regression. For example, to study the causal relation from malnutrition to diarrhea episodes, I use binary variable (diarrhea yes/no during months 0-3) as response, and use the HAZ at month 0 as covariate
4 0.78340518 2204 andrew gelman stats-2014-02-09-Keli Liu and Xiao-Li Meng on Simpson’s paradox
Introduction: XL sent me this paper , “A Fruitful Resolution to Simpson’s Paradox via Multi-Resolution Inference.” I told Keli and Xiao-Li that I wasn’t sure I fully understood the paper—as usual, XL is subtle and sophisticated, also I only get about half of his jokes—but I sent along these thoughts: 1. I do not think counterfactuals or potential outcomes are necessary for Simpson’s paradox. I say this because one can set up Simpson’s paradox with variables that cannot be manipulated, or for which manipulations are not directly of interest. 2. Simpson’s paradox is part of a more general issue that regression coefs change if you add more predictors, the flipping of sign is not really necessary. Here’s an example that I use in my teaching that illustrates both points: I can run a regression predicting income from sex and height. I find that the coef of sex is $10,000 (i.e., comparing a man and woman of the same height, on average the man will make $10,000 more) and the coefficient of h
Introduction: Elias Bareinboim asked what I thought about his comment on selection bias in which he referred to a paper by himself and Judea Pearl, “Controlling Selection Bias in Causal Inference.” I replied that I have no problem with what he wrote, but that from my perspective I find it easier to conceptualize such problems in terms of multilevel models. I elaborated on that point in a recent post , “Hierarchical modeling as a framework for extrapolation,” which I think was read by only a few people (I say this because it received only two comments). I don’t think Bareinboim objected to anything I wrote, but like me he is comfortable working within his own framework. He wrote the following to me: In some sense, “not ad hoc” could mean logically consistent. In other words, if one agrees with the assumptions encoded in the model, one must also agree with the conclusions entailed by these assumptions. I am not aware of any other way of doing mathematics. As it turns out, to get causa
7 0.76227826 1996 andrew gelman stats-2013-08-24-All inference is about generalizing from sample to population
8 0.76068974 1732 andrew gelman stats-2013-02-22-Evaluating the impacts of welfare reform?
10 0.75135577 1675 andrew gelman stats-2013-01-15-“10 Things You Need to Know About Causal Effects”
11 0.74993318 1136 andrew gelman stats-2012-01-23-Fight! (also a bit of reminiscence at the end)
12 0.74390602 888 andrew gelman stats-2011-09-03-A psychology researcher asks: Is Anova dead?
13 0.74310392 560 andrew gelman stats-2011-02-06-Education and Poverty
14 0.73724699 1663 andrew gelman stats-2013-01-09-The effects of fiscal consolidation
15 0.73708421 86 andrew gelman stats-2010-06-14-“Too much data”?
16 0.72807139 1910 andrew gelman stats-2013-06-22-Struggles over the criticism of the “cannabis users and IQ change” paper
17 0.72363561 550 andrew gelman stats-2011-02-02-An IV won’t save your life if the line is tangled
18 0.71514374 1070 andrew gelman stats-2011-12-19-The scope for snooping
topicId topicWeight
[(16, 0.042), (21, 0.015), (24, 0.157), (35, 0.086), (53, 0.041), (56, 0.021), (68, 0.031), (79, 0.048), (82, 0.017), (88, 0.017), (89, 0.047), (99, 0.376)]
simIndex simValue blogId blogTitle
same-blog 1 0.98356718 2274 andrew gelman stats-2014-03-30-Adjudicating between alternative interpretations of a statistical interaction?
Introduction: Jacob Felson writes: Say we have a statistically significant interaction in non-experimental data between two continuous predictors, X and Z and it is unclear which variable is primarily a cause and which variable is primarily a moderator. One person might find it more plausible to think of X as a cause and Z as a moderator and another person may think the reverse more plausible. My question then is whether there is are any set of rules or heuristics you could recommend to help adjudicate between alternate perspectives on such an interaction term. My reply: I think in this setting, it would make sense to think about different interventions, some of which affect X, others of which affect Z, others of which affect both, and go from there. Rather than trying to isolate a single causal path, consider different cases of forward casual inference. My guess is that the different stories regarding moderators etc. could motivate different thought experiments (and, ultimately, differe
Introduction: About 12 years ago Greg Wawro, Sy Spilerman, and I started a M.A. program here in Quantitative Methods in Social Sciences, jointly between the departments of history, economics, political science, sociology, psychology, and statistics. We created a bunch of new features for the program, including an interdisciplinary course based on this book . And here’s their new logo: Don’t blame me for the pie-chart motif! Seriously, though, the program is great. I’m proud to have gotten it started, and I’m impressed by the progress that Chris Weiss and others have made in expanding the program during the past decade.
3 0.9745366 1926 andrew gelman stats-2013-07-05-More plain old everyday Bayesianism
Introduction: Following up on this story , Bob Goodman writes: A most recent issue of the New England Journal of Medicine published a study entitled “Biventricular Pacing for Atrioventricular Block and Systolic Dysfunction,” (N Engl J Med 2013; 368:1585-1593), whereby “A hierarchical Bayesian proportional-hazards model was used for analysis of the primary outcome.” It is the first study I can recall in this journal that has reported on Table 2 (primary outcomes) “The Posterior Probability of Hazard Ratio < 1" (which in this case was .9978). This is ok, but to be really picky I will say that there’s typically not so much reason to care about the posterior probability that the effect is greater than 1; I’d rather have an estimate of the effect. Also we should be using informative priors.
4 0.97043794 1443 andrew gelman stats-2012-08-04-Bayesian Learning via Stochastic Gradient Langevin Dynamics
Introduction: Burak Bayramli writes: In this paper by Sunjin Ahn, Anoop Korattikara, and Max Welling and this paper by Welling and Yee Whye The, there are some arguments on big data and the use of MCMC. Both papers have suggested improvements to speed up MCMC computations. I was wondering what your thoughts were, especially on this paragraph: When a dataset has a billion data-cases (as is not uncommon these days) MCMC algorithms will not even have generated a single (burn-in) sample when a clever learning algorithm based on stochastic gradients may already be making fairly good predictions. In fact, the intriguing results of Bottou and Bousquet (2008) seem to indicate that in terms of “number of bits learned per unit of computation”, an algorithm as simple as stochastic gradient descent is almost optimally efficient. We therefore argue that for Bayesian methods to remain useful in an age when the datasets grow at an exponential rate, they need to embrace the ideas of the stochastic optimiz
5 0.96698666 1516 andrew gelman stats-2012-09-30-Computational problems with glm etc.
Introduction: John Mount provides some useful background and follow-up on our discussion from last year on computational instability of the usual logistic regression solver. Just to refresh your memory, here’s a simple logistic regression with only a constant term and no separation, nothing pathological at all: > y <- rep (c(1,0),c(10,5)) > display (glm (y ~ 1, family=binomial(link="logit"))) glm(formula = y ~ 1, family = binomial(link = "logit")) coef.est coef.se (Intercept) 0.69 0.55 --- n = 15, k = 1 residual deviance = 19.1, null deviance = 19.1 (difference = 0.0) And here’s what happens when we give it the not-outrageous starting value of -2: > display (glm (y ~ 1, family=binomial(link="logit"), start=-2)) glm(formula = y ~ 1, family = binomial(link = "logit"), start = -2) coef.est coef.se (Intercept) 71.97 17327434.18 --- n = 15, k = 1 residual deviance = 360.4, null deviance = 19.1 (difference = -341.3) Warning message:
6 0.96439278 296 andrew gelman stats-2010-09-26-A simple semigraphic display
7 0.96234524 392 andrew gelman stats-2010-11-03-Taleb + 3.5 years
8 0.96201158 1482 andrew gelman stats-2012-09-04-Model checking and model understanding in machine learning
9 0.95963669 2359 andrew gelman stats-2014-06-04-All the Assumptions That Are My Life
10 0.95942914 1661 andrew gelman stats-2013-01-08-Software is as software does
11 0.95802814 888 andrew gelman stats-2011-09-03-A psychology researcher asks: Is Anova dead?
12 0.95730436 566 andrew gelman stats-2011-02-09-The boxer, the wrestler, and the coin flip, again
14 0.9571591 1786 andrew gelman stats-2013-04-03-Hierarchical array priors for ANOVA decompositions
15 0.95699954 2210 andrew gelman stats-2014-02-13-Stopping rules and Bayesian analysis
17 0.95630258 972 andrew gelman stats-2011-10-25-How do you interpret standard errors from a regression fit to the entire population?
18 0.95629048 2142 andrew gelman stats-2013-12-21-Chasing the noise
19 0.95627522 295 andrew gelman stats-2010-09-25-Clusters with very small numbers of observations
20 0.95622671 1384 andrew gelman stats-2012-06-19-Slick time series decomposition of the birthdays data