andrew_gelman_stats andrew_gelman_stats-2011 andrew_gelman_stats-2011-823 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Liz Sanders writes: I viewed your 2005 presentation “Interactions in multilevel models” and was hoping you or one of your students/colleagues could point me to some readings about the issue of using all possible vs. only particular interaction terms in regression models with continuous covariates (I think “functional form validity” is the term I have encountered in the past). In particular, I am trying to understand whether I would be mis-specifying a model if I deleted two of its interaction terms (in favor of using only 2-way treatment interaction terms). The general full model, for example, is: Y = intercept + txt + pre1 + pre2 + txt*pre1 + txt*pre2 + pre1*pre2 + txt*pre1*pre2, where txt is effect coded (1=treatment, -1=control) and pre1 and pre2 are two different pretests that are assumed normally distributed. (The model is actually a multilevel model; the error terms are not listed for brevity.) The truncated model, on the other hand, would only test 2-way treatment inte
sentIndex sentText sentNum sentScore
1 Liz Sanders writes: I viewed your 2005 presentation “Interactions in multilevel models” and was hoping you or one of your students/colleagues could point me to some readings about the issue of using all possible vs. [sent-1, score-0.444]
2 only particular interaction terms in regression models with continuous covariates (I think “functional form validity” is the term I have encountered in the past). [sent-2, score-0.971]
3 In particular, I am trying to understand whether I would be mis-specifying a model if I deleted two of its interaction terms (in favor of using only 2-way treatment interaction terms). [sent-3, score-1.528]
4 The general full model, for example, is: Y = intercept + txt + pre1 + pre2 + txt*pre1 + txt*pre2 + pre1*pre2 + txt*pre1*pre2, where txt is effect coded (1=treatment, -1=control) and pre1 and pre2 are two different pretests that are assumed normally distributed. [sent-4, score-1.736]
5 (The model is actually a multilevel model; the error terms are not listed for brevity. [sent-5, score-0.635]
6 ) The truncated model, on the other hand, would only test 2-way treatment interactions (deleting the last two terms). [sent-6, score-0.66]
7 There are plenty of data, and the results indicate that the three-way interaction term is significant for two of six outcomes I modeled. [sent-7, score-0.896]
8 On the one hand, I worry about model mis-specification if I delete the last two interactions. [sent-8, score-0.586]
9 On the other hand, I worry about spurious ‘significant’ results with all of the terms in the model. [sent-9, score-0.553]
10 My reply: The usual advice, which I think is reasonable here, is that your truncated model is ok. [sent-10, score-0.393]
11 But I also think it would fine to include everything and then plot the estimated coefficients and the fitted model to understand what you’ve got. [sent-11, score-0.492]
wordName wordTfidf (topN-words)
[('txt', 0.651), ('terms', 0.279), ('interaction', 0.264), ('truncated', 0.206), ('model', 0.187), ('treatment', 0.151), ('hand', 0.126), ('interactions', 0.123), ('worry', 0.116), ('deleting', 0.112), ('liz', 0.112), ('two', 0.112), ('sanders', 0.103), ('delete', 0.103), ('term', 0.101), ('multilevel', 0.099), ('deleted', 0.095), ('readings', 0.093), ('spurious', 0.092), ('significant', 0.089), ('coded', 0.089), ('intercept', 0.087), ('plenty', 0.08), ('functional', 0.08), ('normally', 0.078), ('covariates', 0.077), ('viewed', 0.075), ('listed', 0.07), ('understand', 0.07), ('encountered', 0.069), ('last', 0.068), ('assumed', 0.068), ('validity', 0.067), ('hoping', 0.066), ('fitted', 0.066), ('indicate', 0.066), ('results', 0.066), ('presentation', 0.062), ('particular', 0.062), ('six', 0.062), ('coefficients', 0.06), ('continuous', 0.06), ('models', 0.059), ('favor', 0.057), ('plot', 0.057), ('outcomes', 0.056), ('estimated', 0.052), ('advice', 0.051), ('using', 0.049), ('control', 0.048)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 823 andrew gelman stats-2011-07-26-Including interactions or not
Introduction: Liz Sanders writes: I viewed your 2005 presentation “Interactions in multilevel models” and was hoping you or one of your students/colleagues could point me to some readings about the issue of using all possible vs. only particular interaction terms in regression models with continuous covariates (I think “functional form validity” is the term I have encountered in the past). In particular, I am trying to understand whether I would be mis-specifying a model if I deleted two of its interaction terms (in favor of using only 2-way treatment interaction terms). The general full model, for example, is: Y = intercept + txt + pre1 + pre2 + txt*pre1 + txt*pre2 + pre1*pre2 + txt*pre1*pre2, where txt is effect coded (1=treatment, -1=control) and pre1 and pre2 are two different pretests that are assumed normally distributed. (The model is actually a multilevel model; the error terms are not listed for brevity.) The truncated model, on the other hand, would only test 2-way treatment inte
2 0.16877016 1908 andrew gelman stats-2013-06-21-Interpreting interactions in discrete-data regression
Introduction: Mike Johns writes: Are you familiar with the work of Ai and Norton on interactions in logit/probit models? I’d be curious to hear your thoughts. Ai, C.R. and Norton E.C. 2003. Interaction terms in logit and probit models. Economics Letters 80(1): 123-129. A peer ref just cited this paper in reaction to a logistic model we tested and claimed that the “only” way to test an interaction in logit/probit regression is to use the cross derivative method of Ai & Norton. I’ve never heard of this issue or method. It leaves me wondering what the interaction term actually tests (something Ai & Norton don’t discuss) and why such an important discovery is not more widely known. Is this an issue that is of particular relevance to econometric analysis because they approach interactions from the difference-in-difference perspective? Full disclosure, I’m coming from a social science/epi background. Thus, i’m not interested in the d-in-d estimator; I want to know if any variables modify the rela
3 0.16665836 888 andrew gelman stats-2011-09-03-A psychology researcher asks: Is Anova dead?
Introduction: A research psychologist writes in with a question that’s so long that I’ll put my answer first, then put the question itself below the fold. Here’s my reply: As I wrote in my Anova paper and in my book with Jennifer Hill, I do think that multilevel models can completely replace Anova. At the same time, I think the central idea of Anova should persist in our understanding of these models. To me the central idea of Anova is not F-tests or p-values or sums of squares, but rather the idea of predicting an outcome based on factors with discrete levels, and understanding these factors using variance components. The continuous or categorical response thing doesn’t really matter so much to me. I have no problem using a normal linear model for continuous outcomes (perhaps suitably transformed) and a logistic model for binary outcomes. I don’t want to throw away interactions just because they’re not statistically significant. I’d rather partially pool them toward zero using an inform
4 0.13966249 1686 andrew gelman stats-2013-01-21-Finite-population Anova calculations for models with interactions
Introduction: Jim Thomson writes: I wonder if you could provide some clarification on the correct way to calculate the finite-population standard deviations for interaction terms in your Bayesian approach to ANOVA (as explained in your 2005 paper, and Gelman and Hill 2007). I understand that it is the SD of the constrained batch coefficients that is of interest, but in most WinBUGS examples I have seen, the SDs are all calculated directly as sd.fin<-sd(beta.main[]) for main effects and sd(beta.int[,]) for interaction effects, where beta.main and beta.int are the unconstrained coefficients, e.g. beta.int[i,j]~dnorm(0,tau). For main effects, I can see that it makes no difference, since the constrained value is calculated by subtracting the mean, and sd(B[]) = sd(B[]-mean(B[])). But the conventional sum-to-zero constraint for interaction terms in linear models is more complicated than subtracting the mean (there are only (n1-1)*(n2-1) free coefficients for an interaction b/w factors with n1 a
5 0.13262534 753 andrew gelman stats-2011-06-09-Allowing interaction terms to vary
Introduction: Zoltan Fazekas writes: I am a 2nd year graduate student in political science at the University of Vienna. In my empirical research I often employ multilevel modeling, and recently I came across a situation that kept me wondering for quite a while. As I did not find much on this in the literature and considering the topics that you work on and blog about, I figured I will try to contact you. The situation is as follows: in a linear multilevel model, there are two important individual level predictors (x1 and x2) and a set of controls. Let us assume that there is a theoretically grounded argument suggesting that an interaction between x1 and x2 should be included in the model (x1 * x2). Both x1 and x2 are let to vary randomly across groups. Would this directly imply that the coefficient of the interaction should also be left to vary across country? This is even more burning if there is no specific hypothesis on the variance of the conditional effect across countries. And then i
6 0.12709519 936 andrew gelman stats-2011-10-02-Covariate Adjustment in RCT - Model Overfitting in Multilevel Regression
9 0.11392979 1786 andrew gelman stats-2013-04-03-Hierarchical array priors for ANOVA decompositions
10 0.10723279 251 andrew gelman stats-2010-09-02-Interactions of predictors in a causal model
11 0.10574644 1999 andrew gelman stats-2013-08-27-Bayesian model averaging or fitting a larger model
13 0.10221543 2274 andrew gelman stats-2014-03-30-Adjudicating between alternative interpretations of a statistical interaction?
14 0.10204502 1144 andrew gelman stats-2012-01-29-How many parameters are in a multilevel model?
15 0.099518962 2 andrew gelman stats-2010-04-23-Modeling heterogenous treatment effects
16 0.096826285 295 andrew gelman stats-2010-09-25-Clusters with very small numbers of observations
17 0.096084118 754 andrew gelman stats-2011-06-09-Difficulties with Bayesian model averaging
18 0.088675275 772 andrew gelman stats-2011-06-17-Graphical tools for understanding multilevel models
19 0.086419418 781 andrew gelman stats-2011-06-28-The holes in my philosophy of Bayesian data analysis
20 0.085889995 1004 andrew gelman stats-2011-11-11-Kaiser Fung on how not to critique models
topicId topicWeight
[(0, 0.142), (1, 0.114), (2, 0.061), (3, -0.031), (4, 0.065), (5, 0.004), (6, -0.01), (7, -0.038), (8, 0.117), (9, 0.085), (10, 0.018), (11, 0.061), (12, 0.002), (13, -0.045), (14, 0.001), (15, 0.008), (16, 0.002), (17, -0.023), (18, -0.031), (19, 0.037), (20, 0.006), (21, 0.006), (22, -0.004), (23, -0.078), (24, -0.045), (25, -0.007), (26, -0.049), (27, -0.012), (28, -0.045), (29, -0.018), (30, -0.041), (31, 0.002), (32, -0.033), (33, 0.032), (34, -0.008), (35, -0.011), (36, -0.042), (37, -0.007), (38, 0.047), (39, -0.022), (40, -0.018), (41, -0.034), (42, 0.024), (43, 0.01), (44, 0.023), (45, -0.021), (46, 0.002), (47, -0.022), (48, -0.019), (49, 0.018)]
simIndex simValue blogId blogTitle
same-blog 1 0.96732146 823 andrew gelman stats-2011-07-26-Including interactions or not
Introduction: Liz Sanders writes: I viewed your 2005 presentation “Interactions in multilevel models” and was hoping you or one of your students/colleagues could point me to some readings about the issue of using all possible vs. only particular interaction terms in regression models with continuous covariates (I think “functional form validity” is the term I have encountered in the past). In particular, I am trying to understand whether I would be mis-specifying a model if I deleted two of its interaction terms (in favor of using only 2-way treatment interaction terms). The general full model, for example, is: Y = intercept + txt + pre1 + pre2 + txt*pre1 + txt*pre2 + pre1*pre2 + txt*pre1*pre2, where txt is effect coded (1=treatment, -1=control) and pre1 and pre2 are two different pretests that are assumed normally distributed. (The model is actually a multilevel model; the error terms are not listed for brevity.) The truncated model, on the other hand, would only test 2-way treatment inte
2 0.85463083 936 andrew gelman stats-2011-10-02-Covariate Adjustment in RCT - Model Overfitting in Multilevel Regression
Introduction: Makoto Hanita writes: We have been discussing the following two issues amongst ourselves, then with our methodological consultant for several days. However, we have not been able to arrive at a consensus. Consequently, we decided to seek an opinion from nationally known experts. FYI, we sent a similar inquiry to Larry Hedges and David Rogosa . . . 1) We are wondering if a post-hoc covariate adjustment is a good practice in the context of RCTs [randomized clinical trials]. We have a situation where we found a significant baseline difference between the treatment and the control groups in 3 variables. Some of us argue that adding those three variables to the original impact analysis model is a good idea, as that would remove the confound from the impact estimate. Others among us, on the other hand, argue that a post-hoc covariate adjustment should never be done, on the ground that those covariates are correlated with the treatment, which makes the analysis model that of quasi
3 0.84780312 77 andrew gelman stats-2010-06-09-Sof[t]
Introduction: Joe Fruehwald writes: I’m working with linguistic data, specifically binomial hits and misses of a certain variable for certain words (specifically whether or not the “t” sound was pronounced at the end of words like “soft”). Word frequency follows a power law, with most words appearing just once, and with some words being hyperfrequent. I’m not interested in specific word effects, but I am interested in the effect of word frequency. A logistic model fit is going to be heavily influenced by the effect of the hyperfrequent words which constitute only one type. To control for the item effect, I would fit a multilevel model with a random intercept by word, but like I said, most of the words appear only once. Is there a principled approach to this problem? My response: It’s ok to fit a multilevel model even if most groups only have one observation each. You’ll want to throw in some word-level predictors too. Think of the multilevel model not as a substitute for the usual thoug
4 0.81071383 1468 andrew gelman stats-2012-08-24-Multilevel modeling and instrumental variables
Introduction: Terence Teo writes: I was wondering if multilevel models can be used as an alternative to 2SLS or IV models to deal with (i) endogeneity and (ii) selection problems. More concretely, I am trying to assess the impact of investment treaties on foreign investment. Aside from the fact that foreign investment is correlated over time, it may be the case that countries that already receive sufficient amounts of foreign investment need not sign treaties, and countries that sign treaties are those that need foreign investment in the first place. Countries thus “select” into treatment; treaty signing is non-random. As such, I argue that to properly estimate the impact of treaties on investment, we must model the determinants of treaty signing. I [Teo] am currently modeling this as two separate models: (1) regress predictors on likelihood of treaty signing, (2) regress treaty (with interactions, etc) on investment (I’ve thought of using propensity score matching for this part of the model)
5 0.79495233 251 andrew gelman stats-2010-09-02-Interactions of predictors in a causal model
Introduction: Michael Bader writes: What is the best way to examine interactions of independent variables in a propensity weights framework? Let’s say we are interested in estimating breathing difficulty (measured on a continuous scale) and our main predictor is age of housing. The object is to estimate whether living in housing 20 years or older is associated with breathing difficulty compared counterfactually to those living in housing less than 20 years old; as a secondary question, we want to know whether that effect differs for those in poverty compared to those not in poverty. In our first-stage propensity model, we include whether the respondent lives in poverty. The weights applied to the other covariates in the propensity model are similar to those living in poverty compared to those who are not. Now, can I simply interact the poverty variable with the age of construction variable to look at the interaction of age of housing and poverty on breathing difficulty? My thought is no —
6 0.78881258 851 andrew gelman stats-2011-08-12-year + (1|year)
7 0.78376049 472 andrew gelman stats-2010-12-17-So-called fixed and random effects
8 0.78230333 753 andrew gelman stats-2011-06-09-Allowing interaction terms to vary
9 0.78157508 1686 andrew gelman stats-2013-01-21-Finite-population Anova calculations for models with interactions
10 0.7703836 888 andrew gelman stats-2011-09-03-A psychology researcher asks: Is Anova dead?
11 0.76829779 1395 andrew gelman stats-2012-06-27-Cross-validation (What is it good for?)
12 0.76673377 417 andrew gelman stats-2010-11-17-Clutering and variance components
14 0.7557739 1267 andrew gelman stats-2012-04-17-Hierarchical-multilevel modeling with “big data”
17 0.74417871 1735 andrew gelman stats-2013-02-24-F-f-f-fake data
18 0.73715347 772 andrew gelman stats-2011-06-17-Graphical tools for understanding multilevel models
19 0.73695111 935 andrew gelman stats-2011-10-01-When should you worry about imputed data?
20 0.73538554 552 andrew gelman stats-2011-02-03-Model Makers’ Hippocratic Oath
topicId topicWeight
[(7, 0.02), (13, 0.024), (16, 0.012), (21, 0.03), (24, 0.139), (29, 0.016), (41, 0.014), (43, 0.012), (48, 0.195), (55, 0.031), (59, 0.015), (79, 0.043), (89, 0.033), (95, 0.034), (99, 0.265)]
simIndex simValue blogId blogTitle
1 0.95037895 332 andrew gelman stats-2010-10-10-Proposed new section of the American Statistical Association on Imaging Sciences
Introduction: Martin Lindquist writes that he and others are trying to start a new ASA section on statistics in imaging. If you’re interested in being a signatory to its formation, please send him an email.
2 0.92562312 2363 andrew gelman stats-2014-06-07-“Does researching casual marijuana use cause brain abnormalities?”
Introduction: David Austin points me to a wonderfully-titled post by Lior Pachter criticizing a recent paper on the purported effects of cannabis use. Not the paper criticized here . Someone should send this all to David Brooks. I’ve heard he’s interested in the latest scientific findings, and I know he’s interested in marijuana.
3 0.91833597 212 andrew gelman stats-2010-08-17-Futures contracts, Granger causality, and my preference for estimation to testing
Introduction: José Iparraguirre writes: There’s a letter in the latest issue of The Economist (July 31st) signed by Sir Richard Branson (Virgin), Michael Masters (Masters Capital Management) and David Frenk (Better Markets) about an “>OECD report on speculation and the prices of commodities, which includes the following: “The report uses a Granger causality test to measure the relationship between the level of commodities futures contracts held by swap dealers, and the prices of those commodities. Granger tests, however, are of dubious applicability to extremely volatile variables like commodities prices.” The report says: Granger causality is a standard statistical technique for determining whether one time series is useful in forecasting another. It is important to bear in mind that the term causality is used in a statistical sense, and not in a philosophical one of structural causation. More precisely a variable A is said to Granger cause B if knowing the time paths of B and A toge
Introduction: John Transue sent it in with the following thoughtful comment: I’d imagine you’ve already received this, but just in case, here’s a cartoon you’d like. At first blush it seems to go against your advice (more nuanced than what I’m about to say by quoting the paper title) to not worry about multiple comparisons. However, if I understand correctly your argument about multiple comparisons in multilevel models, the situation in this comic might have been avoided if shrinkage toward the grand mean (of all colors) had prevented the greens from clearing the .05 threshold. Is that right?
5 0.91412252 1088 andrew gelman stats-2011-12-28-Argument in favor of Ddulites
Introduction: Mark Palko defines a Ddulite as follows: A preference for higher tech solutions even in cases where lower tech alternatives have greater and more appropriate functionality; a person of ddulite tendencies. Though Ddulites are the opposite of Luddites with respect to attitudes toward technology, they occupy more or less the same point with respect to functionality. As a sometime Luddite myself (no cell phone, tv, microwave oven, etc.), I should in fairness point out the logic in favor of being a Ddulite. Old technology is typically pretty stable; new technology is improving. It can make sense to switch early (before the new technology actually performs better than the old) to get the benefits of being familiar with the new technology once it does take off.
6 0.90476382 1234 andrew gelman stats-2012-03-28-The Supreme Court’s Many Median Justices
7 0.90392095 181 andrew gelman stats-2010-08-03-MCMC in Python
same-blog 8 0.89811558 823 andrew gelman stats-2011-07-26-Including interactions or not
9 0.87883687 2147 andrew gelman stats-2013-12-25-Measuring Beauty
10 0.87822568 1496 andrew gelman stats-2012-09-14-Sides and Vavreck on the 2012 election
11 0.87774259 841 andrew gelman stats-2011-08-06-Twitteo killed the bloggio star . . . Not!
12 0.87448746 464 andrew gelman stats-2010-12-12-Finite-population standard deviation in a hierarchical model
14 0.85479665 605 andrew gelman stats-2011-03-09-Does it feel like cheating when I do this? Variation in ethical standards and expectations
15 0.85060769 202 andrew gelman stats-2010-08-12-Job openings in multilevel modeling in Bristol, England
16 0.84560049 681 andrew gelman stats-2011-04-26-Worst statistical graphic I have seen this year
17 0.84241796 2126 andrew gelman stats-2013-12-07-If I could’ve done it all over again
18 0.8372528 1518 andrew gelman stats-2012-10-02-Fighting a losing battle
19 0.83199739 797 andrew gelman stats-2011-07-11-How do we evaluate a new and wacky claim?
20 0.8306129 1910 andrew gelman stats-2013-06-22-Struggles over the criticism of the “cannabis users and IQ change” paper