andrew_gelman_stats andrew_gelman_stats-2011 andrew_gelman_stats-2011-799 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Vincent Yip writes: I have read your paper [with Kobi Abayomi and Marc Levy] regarding multiple imputation application. In order to diagnostic my imputed data, I used Kolmogorov-Smirnov (K-S) tests to compare the distribution differences between the imputed and observed values of a single attribute as mentioned in your paper. My question is: For example I have this attribute X with the following data: (NA = missing) Original dataset: 1, NA, 3, 4, 1, 5, NA Imputed dataset: 1, 2 , 3, 4, 1, 5, 6 a) in order to run the KS test, will I treat the observed data as 1, 3, 4,1, 5? b) and for the observed data, will I treat 1, 2 , 3, 4, 1, 5, 6 as the imputed dataset for the K-S test? or just 2 ,6? c) if I used m=5, I will have 5 set of imputed data sets. How would I apply K-S test to 5 of them and compare to the single observed distribution? Do I combine the 5 imputed data set into one by averaging each imputed values so I get one single imputed data and compare with the ob
sentIndex sentText sentNum sentScore
1 Vincent Yip writes: I have read your paper [with Kobi Abayomi and Marc Levy] regarding multiple imputation application. [sent-1, score-0.199]
2 In order to diagnostic my imputed data, I used Kolmogorov-Smirnov (K-S) tests to compare the distribution differences between the imputed and observed values of a single attribute as mentioned in your paper. [sent-2, score-2.524]
3 My question is: For example I have this attribute X with the following data: (NA = missing) Original dataset: 1, NA, 3, 4, 1, 5, NA Imputed dataset: 1, 2 , 3, 4, 1, 5, 6 a) in order to run the KS test, will I treat the observed data as 1, 3, 4,1, 5? [sent-3, score-0.778]
4 b) and for the observed data, will I treat 1, 2 , 3, 4, 1, 5, 6 as the imputed dataset for the K-S test? [sent-4, score-1.218]
5 c) if I used m=5, I will have 5 set of imputed data sets. [sent-6, score-0.898]
6 How would I apply K-S test to 5 of them and compare to the single observed distribution? [sent-7, score-0.89]
7 Do I combine the 5 imputed data set into one by averaging each imputed values so I get one single imputed data and compare with the observed data? [sent-8, score-3.204]
8 OR will I run KS test to all 5 and averaging the KS test result (i. [sent-9, score-0.711]
9 My reply: I have to admit I have not thought about this in detail. [sent-12, score-0.113]
10 I suppose it would make sense to compare the observed data (1,3,4,1,5) to the imputed (2,6). [sent-13, score-1.316]
11 I would do the test separately for each imputation. [sent-14, score-0.343]
12 I also haven’t thought about what to do with the p-values. [sent-15, score-0.068]
13 My intuition would be to average them but this again is not something I’ve thought much about. [sent-16, score-0.17]
14 Also if the test does reject, this implies a difference between observed and imputed values. [sent-17, score-1.264]
15 It does not show that the imputations are wrong, merely that under the model the data are not missing completely at random. [sent-18, score-0.325]
16 I’m sure there’s a literature on combining hypothesis tests with multiple imputation. [sent-19, score-0.24]
17 Usually I’m not particularly interested in testing–we just threw that Kolmogorov-Smirnov idea into our paper without thinking too hard about what we would do with it. [sent-20, score-0.193]
wordName wordTfidf (topN-words)
[('imputed', 0.679), ('ks', 0.299), ('observed', 0.297), ('test', 0.236), ('averaging', 0.17), ('compare', 0.167), ('na', 0.152), ('dataset', 0.142), ('data', 0.123), ('attribute', 0.119), ('single', 0.1), ('treat', 0.1), ('abayomi', 0.091), ('kobi', 0.091), ('vincent', 0.091), ('tests', 0.08), ('imputations', 0.079), ('levy', 0.077), ('missing', 0.077), ('values', 0.075), ('diagnostic', 0.072), ('threw', 0.07), ('order', 0.07), ('multiple', 0.069), ('run', 0.069), ('thought', 0.068), ('distribution', 0.063), ('marc', 0.063), ('combine', 0.06), ('separately', 0.057), ('imputation', 0.057), ('reject', 0.056), ('combining', 0.054), ('intuition', 0.052), ('implies', 0.052), ('set', 0.052), ('would', 0.05), ('merely', 0.046), ('admit', 0.045), ('used', 0.044), ('mentioned', 0.043), ('testing', 0.041), ('apply', 0.04), ('paper', 0.038), ('hypothesis', 0.037), ('haven', 0.036), ('differences', 0.036), ('regarding', 0.035), ('particularly', 0.035), ('usually', 0.034)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 799 andrew gelman stats-2011-07-13-Hypothesis testing with multiple imputations
Introduction: Vincent Yip writes: I have read your paper [with Kobi Abayomi and Marc Levy] regarding multiple imputation application. In order to diagnostic my imputed data, I used Kolmogorov-Smirnov (K-S) tests to compare the distribution differences between the imputed and observed values of a single attribute as mentioned in your paper. My question is: For example I have this attribute X with the following data: (NA = missing) Original dataset: 1, NA, 3, 4, 1, 5, NA Imputed dataset: 1, 2 , 3, 4, 1, 5, 6 a) in order to run the KS test, will I treat the observed data as 1, 3, 4,1, 5? b) and for the observed data, will I treat 1, 2 , 3, 4, 1, 5, 6 as the imputed dataset for the K-S test? or just 2 ,6? c) if I used m=5, I will have 5 set of imputed data sets. How would I apply K-S test to 5 of them and compare to the single observed distribution? Do I combine the 5 imputed data set into one by averaging each imputed values so I get one single imputed data and compare with the ob
2 0.50112081 935 andrew gelman stats-2011-10-01-When should you worry about imputed data?
Introduction: Majid Ezzati writes: My research group is increasingly focusing on a series of problems that involve data that either have missingness or measurements that may have bias/error. We have at times developed our own approaches to imputation (as simple as interpolating a missing unit and as sophisticated as a problem-specific Bayesian hierarchical model) and at other times, other groups impute the data. The outputs are being used to investigate the basic associations between pairs of variables, Xs and Ys, in regressions; we may or may not interpret these as causal. I am contacting colleagues with relevant expertise to suggest good references on whether having imputed X and/or Y in a subsequent regression is correct or if it could somehow lead to biased/spurious associations. Thinking about this, we can have at least the following situations (these could all be Bayesian or not): 1) X and Y both measured (perhaps with error) 2) Y imputed using some data and a model and X measur
3 0.24751659 1330 andrew gelman stats-2012-05-19-Cross-validation to check missing-data imputation
Introduction: Aureliano Crameri writes: I have questions regarding one technique you and your colleagues described in your papers: the cross validation (Multiple Imputation with Diagnostics (mi) in R: Opening Windows into the Black Box, with reference to Gelman, King, and Liu, 1998). I think this is the technique I need for my purpose, but I am not sure I understand it right. I want to use the multiple imputation to estimate the outcome of psychotherapies based on longitudinal data. First I have to demonstrate that I am able to get unbiased estimates with the multiple imputation. The expected bias is the overestimation of the outcome of dropouts. I will test my imputation strategies by means of a series of simulations (delete values, impute, compare with the original). Due to the complexity of the statistical analyses I think I need at least 200 cases. Now I don’t have so many cases without any missings. My data have missing values in different variables. The proportion of missing values is
4 0.19675247 257 andrew gelman stats-2010-09-04-Question about standard range for social science correlations
Introduction: Andrew Eppig writes: I’m a physicist by training who is transitioning to the social sciences. I recently came across a reference in the Economist to a paper on IQ and parasites which I read as I have more than a passing interest in IQ research (having read much that you and others (e.g., Shalizi, Wicherts) have written). In this paper I note that the authors find a very high correlation between national IQ and parasite prevalence. The strength of the correlation (-0.76 to -0.82) surprised me, as I’m used to much weaker correlations in the social sciences. To me, it’s a bit too high, suggesting that there are other factors at play or that one of the variables is merely a proxy for a large number of other variables. But I have no basis for this other than a gut feeling and a memory of a plot on Language Log about the distribution of correlation coefficients in social psychology. So my question is this: Is a correlation in the range of (-0.82,-0.76) more likely to be a correlatio
5 0.18639579 608 andrew gelman stats-2011-03-12-Single or multiple imputation?
Introduction: Vishnu Ganglani writes: It appears that multiple imputation appears to be the best way to impute missing data because of the more accurate quantification of variance. However, when imputing missing data for income values in national household surveys, would you recommend it would be practical to maintain the multiple datasets associated with multiple imputations, or a single imputation method would suffice. I have worked on household survey projects (in Scotland) and in the past gone with suggesting single methods for ease of implementation, but with the availability of open source R software I am think of performing multiple imputation methodologies, but a bit apprehensive because of the complexity and also the need to maintain multiple datasets (ease of implementation). My reply: In many applications I’ve just used a single random imputation to avoid the awkwardness of working with multiple datasets. But if there’s any concern, I’d recommend doing parallel analyses on multipl
6 0.14424828 704 andrew gelman stats-2011-05-10-Multiple imputation and multilevel analysis
7 0.14147528 321 andrew gelman stats-2010-10-05-Racism!
9 0.12351073 1218 andrew gelman stats-2012-03-18-Check your missing-data imputations using cross-validation
10 0.10105198 404 andrew gelman stats-2010-11-09-“Much of the recent reported drop in interstate migration is a statistical artifact”
12 0.093837813 1605 andrew gelman stats-2012-12-04-Write This Book
13 0.091913797 870 andrew gelman stats-2011-08-25-Why it doesn’t make sense in general to form confidence intervals by inverting hypothesis tests
14 0.087479718 1913 andrew gelman stats-2013-06-24-Why it doesn’t make sense in general to form confidence intervals by inverting hypothesis tests
15 0.08708676 1883 andrew gelman stats-2013-06-04-Interrogating p-values
16 0.086544916 1016 andrew gelman stats-2011-11-17-I got 99 comparisons but multiplicity ain’t one
17 0.084158234 1247 andrew gelman stats-2012-04-05-More philosophy of Bayes
18 0.083793424 754 andrew gelman stats-2011-06-09-Difficulties with Bayesian model averaging
19 0.083621003 929 andrew gelman stats-2011-09-27-Visual diagnostics for discrete-data regressions
20 0.080888815 1289 andrew gelman stats-2012-04-29-We go to war with the data we have, not the data we want
topicId topicWeight
[(0, 0.114), (1, 0.061), (2, 0.027), (3, -0.039), (4, 0.045), (5, -0.002), (6, -0.01), (7, 0.009), (8, 0.044), (9, -0.012), (10, -0.024), (11, 0.051), (12, -0.011), (13, -0.066), (14, -0.005), (15, 0.013), (16, 0.017), (17, -0.025), (18, 0.013), (19, -0.033), (20, 0.026), (21, 0.051), (22, 0.014), (23, -0.026), (24, -0.002), (25, -0.028), (26, 0.014), (27, -0.061), (28, 0.084), (29, 0.05), (30, 0.053), (31, -0.038), (32, 0.065), (33, 0.111), (34, -0.017), (35, 0.037), (36, 0.064), (37, 0.059), (38, 0.023), (39, 0.027), (40, -0.042), (41, -0.017), (42, 0.02), (43, -0.023), (44, -0.044), (45, 0.012), (46, 0.054), (47, -0.052), (48, 0.025), (49, -0.014)]
simIndex simValue blogId blogTitle
same-blog 1 0.93759382 799 andrew gelman stats-2011-07-13-Hypothesis testing with multiple imputations
Introduction: Vincent Yip writes: I have read your paper [with Kobi Abayomi and Marc Levy] regarding multiple imputation application. In order to diagnostic my imputed data, I used Kolmogorov-Smirnov (K-S) tests to compare the distribution differences between the imputed and observed values of a single attribute as mentioned in your paper. My question is: For example I have this attribute X with the following data: (NA = missing) Original dataset: 1, NA, 3, 4, 1, 5, NA Imputed dataset: 1, 2 , 3, 4, 1, 5, 6 a) in order to run the KS test, will I treat the observed data as 1, 3, 4,1, 5? b) and for the observed data, will I treat 1, 2 , 3, 4, 1, 5, 6 as the imputed dataset for the K-S test? or just 2 ,6? c) if I used m=5, I will have 5 set of imputed data sets. How would I apply K-S test to 5 of them and compare to the single observed distribution? Do I combine the 5 imputed data set into one by averaging each imputed values so I get one single imputed data and compare with the ob
2 0.78498709 1330 andrew gelman stats-2012-05-19-Cross-validation to check missing-data imputation
Introduction: Aureliano Crameri writes: I have questions regarding one technique you and your colleagues described in your papers: the cross validation (Multiple Imputation with Diagnostics (mi) in R: Opening Windows into the Black Box, with reference to Gelman, King, and Liu, 1998). I think this is the technique I need for my purpose, but I am not sure I understand it right. I want to use the multiple imputation to estimate the outcome of psychotherapies based on longitudinal data. First I have to demonstrate that I am able to get unbiased estimates with the multiple imputation. The expected bias is the overestimation of the outcome of dropouts. I will test my imputation strategies by means of a series of simulations (delete values, impute, compare with the original). Due to the complexity of the statistical analyses I think I need at least 200 cases. Now I don’t have so many cases without any missings. My data have missing values in different variables. The proportion of missing values is
3 0.70442611 608 andrew gelman stats-2011-03-12-Single or multiple imputation?
Introduction: Vishnu Ganglani writes: It appears that multiple imputation appears to be the best way to impute missing data because of the more accurate quantification of variance. However, when imputing missing data for income values in national household surveys, would you recommend it would be practical to maintain the multiple datasets associated with multiple imputations, or a single imputation method would suffice. I have worked on household survey projects (in Scotland) and in the past gone with suggesting single methods for ease of implementation, but with the availability of open source R software I am think of performing multiple imputation methodologies, but a bit apprehensive because of the complexity and also the need to maintain multiple datasets (ease of implementation). My reply: In many applications I’ve just used a single random imputation to avoid the awkwardness of working with multiple datasets. But if there’s any concern, I’d recommend doing parallel analyses on multipl
4 0.67207253 1081 andrew gelman stats-2011-12-24-Statistical ethics violation
Introduction: A colleague writes: When I was in NYC I went to this party by group of Japanese bio-scientists. There, one guy told me about how the biggest pharmaceutical company in Japan did their statistics. They ran 100 different tests and reported the most significant one. (This was in 2006 and he said they stopped doing this few years back so they were doing this until pretty recently…) I’m not sure if this was 100 multiple comparison or 100 different kinds of test but I’m sure they wouldn’t want to disclose their data… Ouch!
5 0.65335715 2295 andrew gelman stats-2014-04-18-One-tailed or two-tailed?
Introduction: Someone writes: Suppose I have two groups of people, A and B, which differ on some characteristic of interest to me; and for each person I measure a single real-valued quantity X. I have a theory that group A has a higher mean value of X than group B. I test this theory by using a t-test. Am I entitled to use a *one-tailed* t-test? Or should I use a *two-tailed* one (thereby giving a p-value that is twice as large)? I know you will probably answer: Forget the t-test; you should use Bayesian methods instead. But what is the standard frequentist answer to this question? My reply: The quick answer here is that different people will do different things here. I would say the 2-tailed p-value is more standard but some people will insist on the one-tailed version, and it’s hard to make a big stand on this one, given all the other problems with p-values in practice: http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf http://www.stat.columbia.edu/~gelm
6 0.65298617 580 andrew gelman stats-2011-02-19-Weather visualization with WeatherSpark
7 0.65027958 569 andrew gelman stats-2011-02-12-Get the Data
8 0.64609766 527 andrew gelman stats-2011-01-20-Cars vs. trucks
9 0.62242723 212 andrew gelman stats-2010-08-17-Futures contracts, Granger causality, and my preference for estimation to testing
10 0.61592221 935 andrew gelman stats-2011-10-01-When should you worry about imputed data?
11 0.61291587 1178 andrew gelman stats-2012-02-21-How many data points do you really have?
12 0.60900348 544 andrew gelman stats-2011-01-29-Splitting the data
13 0.59821045 1016 andrew gelman stats-2011-11-17-I got 99 comparisons but multiplicity ain’t one
14 0.59697062 791 andrew gelman stats-2011-07-08-Censoring on one end, “outliers” on the other, what can we do with the middle?
15 0.59639817 1142 andrew gelman stats-2012-01-29-Difficulties with the 1-4-power transformation
16 0.59337193 1881 andrew gelman stats-2013-06-03-Boot
17 0.58619708 907 andrew gelman stats-2011-09-14-Reproducibility in Practice
18 0.57615483 1506 andrew gelman stats-2012-09-21-Building a regression model . . . with only 27 data points
19 0.57588512 257 andrew gelman stats-2010-09-04-Question about standard range for social science correlations
20 0.57437825 946 andrew gelman stats-2011-10-07-Analysis of Power Law of Participation
topicId topicWeight
[(2, 0.01), (16, 0.208), (20, 0.011), (23, 0.013), (24, 0.207), (39, 0.016), (48, 0.01), (51, 0.076), (73, 0.025), (84, 0.023), (86, 0.013), (89, 0.026), (99, 0.229)]
simIndex simValue blogId blogTitle
same-blog 1 0.95495391 799 andrew gelman stats-2011-07-13-Hypothesis testing with multiple imputations
Introduction: Vincent Yip writes: I have read your paper [with Kobi Abayomi and Marc Levy] regarding multiple imputation application. In order to diagnostic my imputed data, I used Kolmogorov-Smirnov (K-S) tests to compare the distribution differences between the imputed and observed values of a single attribute as mentioned in your paper. My question is: For example I have this attribute X with the following data: (NA = missing) Original dataset: 1, NA, 3, 4, 1, 5, NA Imputed dataset: 1, 2 , 3, 4, 1, 5, 6 a) in order to run the KS test, will I treat the observed data as 1, 3, 4,1, 5? b) and for the observed data, will I treat 1, 2 , 3, 4, 1, 5, 6 as the imputed dataset for the K-S test? or just 2 ,6? c) if I used m=5, I will have 5 set of imputed data sets. How would I apply K-S test to 5 of them and compare to the single observed distribution? Do I combine the 5 imputed data set into one by averaging each imputed values so I get one single imputed data and compare with the ob
2 0.95474237 177 andrew gelman stats-2010-08-02-Reintegrating rebels into civilian life: Quasi-experimental evidence from Burundi
Introduction: Michael Gilligan, Eric Mvukiyehe, and Cyrus Samii write : We [Gilligan, Mvukiyehe, and Samii] use original survey data, collected in Burundi in the summer of 2007, to show that a World Bank ex-combatant reintegration program implemented after Burundi’s civil war caused significant economic reintegration for its beneficiaries but that this economic reintegration did not translate into greater political and social reintegration. Previous studies of reintegration programs have found them to be ineffective, but these studies have suffered from selection bias: only ex-combatants who self selected into those programs were studied. We avoid such bias with a quasi-experimental research design made possible by an exogenous bureaucratic failure in the implementation of program. One of the World Bank’s implementing partners delayed implementation by almost a year due to an unforeseen contract dispute. As a result, roughly a third of ex-combatants had their program benefits withheld for reas
3 0.95085633 2 andrew gelman stats-2010-04-23-Modeling heterogenous treatment effects
Introduction: Don Green and Holger Kern write on one of my favorite topics , treatment interactions (see also here ): We [Green and Kern] present a methodology that largely automates the search for systematic treatment effect heterogeneity in large-scale experiments. We introduce a nonparametric estimator developed in statistical learning, Bayesian Additive Regression Trees (BART), to model treatment effects that vary as a function of covariates. BART has several advantages over commonly employed parametric modeling strategies, in particular its ability to automatically detect and model relevant treatment-covariate interactions in a flexible manner. To increase the reliability and credibility of the resulting conditional treatment effect estimates, we suggest the use of a split sample design. The data are randomly divided into two equally-sized parts, with the first part used to explore treatment effect heterogeneity and the second part used to confirm the results. This approach permits a re
4 0.94563711 411 andrew gelman stats-2010-11-13-Ethical concerns in medical trials
Introduction: I just read this article on the treatment of medical volunteers, written by doctor and bioethicist Carl Ellliott. As a statistician who has done a small amount of consulting for pharmaceutical companies, I have a slightly different perspective. As a doctor, Elliott focuses on individual patients, whereas, as a statistician, I’ve been trained to focus on the goal of accurately estimate treatment effects. I’ll go through Elliott’s article and give my reactions. Elliott: In Miami, investigative reporters for Bloomberg Markets magazine discovered that a contract research organisation called SFBC International was testing drugs on undocumented immigrants in a rundown motel; since that report, the motel has been demolished for fire and safety violations. . . . SFBC had recently been named one of the best small businesses in America by Forbes magazine. The Holiday Inn testing facility was the largest in North America, and had been operating for nearly ten years before inspecto
5 0.93635571 503 andrew gelman stats-2011-01-04-Clarity on my email policy
Introduction: I never read email before 4. That doesn’t mean I never send email before 4.
6 0.92646372 1293 andrew gelman stats-2012-05-01-Huff the Magic Dragon
7 0.92435372 1871 andrew gelman stats-2013-05-27-Annals of spam
8 0.92397523 1093 andrew gelman stats-2011-12-30-Strings Attached: Untangling the Ethics of Incentives
9 0.92241168 609 andrew gelman stats-2011-03-13-Coauthorship norms
10 0.91693473 434 andrew gelman stats-2010-11-28-When Small Numbers Lead to Big Errors
11 0.91672361 960 andrew gelman stats-2011-10-15-The bias-variance tradeoff
12 0.91337049 42 andrew gelman stats-2010-05-19-Updated solutions to Bayesian Data Analysis homeworks
13 0.91157889 1206 andrew gelman stats-2012-03-10-95% intervals that I don’t believe, because they’re from a flat prior I don’t believe
14 0.91115856 2248 andrew gelman stats-2014-03-15-Problematic interpretations of confidence intervals
15 0.91073596 159 andrew gelman stats-2010-07-23-Popular governor, small state
16 0.91070879 674 andrew gelman stats-2011-04-21-Handbook of Markov Chain Monte Carlo
17 0.90982378 1016 andrew gelman stats-2011-11-17-I got 99 comparisons but multiplicity ain’t one
18 0.90897584 608 andrew gelman stats-2011-03-12-Single or multiple imputation?
19 0.9086436 1019 andrew gelman stats-2011-11-19-Validation of Software for Bayesian Models Using Posterior Quantiles
20 0.90732598 2179 andrew gelman stats-2014-01-20-The AAA Tranche of Subprime Science