andrew_gelman_stats andrew_gelman_stats-2010 andrew_gelman_stats-2010-494 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Type S error: When your estimate is the wrong sign, compared to the true value of the parameter Type M error: When the magnitude of your estimate is far off, compared to the true value of the parameter More here.
sentIndex sentText sentNum sentScore
1 Type S error: When your estimate is the wrong sign, compared to the true value of the parameter Type M error: When the magnitude of your estimate is far off, compared to the true value of the parameter More here. [sent-1, score-3.9]
wordName wordTfidf (topN-words)
[('parameter', 0.4), ('type', 0.395), ('compared', 0.363), ('value', 0.339), ('error', 0.325), ('estimate', 0.288), ('true', 0.286), ('magnitude', 0.256), ('sign', 0.243), ('far', 0.149), ('wrong', 0.143)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 494 andrew gelman stats-2010-12-31-Type S error rates for classical and Bayesian single and multiple comparison procedures
Introduction: Type S error: When your estimate is the wrong sign, compared to the true value of the parameter Type M error: When the magnitude of your estimate is far off, compared to the true value of the parameter More here.
2 0.22790271 1757 andrew gelman stats-2013-03-11-My problem with the Lindley paradox
Introduction: From a couple years ago but still relevant, I think: To me, the Lindley paradox falls apart because of its noninformative prior distribution on the parameter of interest. If you really think there’s a high probability the parameter is nearly exactly zero, I don’t see the point of the model saying that you have no prior information at all on the parameter. In short: my criticism of so-called Bayesian hypothesis testing is that it’s insufficiently Bayesian. P.S. To clarify (in response to Bill’s comment below): I’m speaking of all the examples I’ve ever worked on in social and environmental science, where in some settings I can imagine a parameter being very close to zero and in other settings I can imagine a parameter taking on just about any value in a wide range, but where I’ve never seen an example where a parameter could be either right at zero or taking on any possible value. But such examples might occur in areas of application that I haven’t worked on.
3 0.21323161 1792 andrew gelman stats-2013-04-07-X on JLP
Introduction: Christian Robert writes on the Jeffreys-Lindley paradox. I have nothing to add to this beyond my recent comments : To me, the Lindley paradox falls apart because of its noninformative prior distribution on the parameter of interest. If you really think there’s a high probability the parameter is nearly exactly zero, I don’t see the point of the model saying that you have no prior information at all on the parameter. In short: my criticism of so-called Bayesian hypothesis testing is that it’s insufficiently Bayesian. To clarify, I’m speaking of all the examples I’ve ever worked on in social and environmental science, where in some settings I can imagine a parameter being very close to zero and in other settings I can imagine a parameter taking on just about any value in a wide range, but where I’ve never seen an example where a parameter could be either right at zero or taking on any possible value. But such examples might occur in areas of application that I haven’t worked on.
4 0.20135619 960 andrew gelman stats-2011-10-15-The bias-variance tradeoff
Introduction: Joshua Vogelstein asks for my thoughts as a Bayesian on the above topic. So here they are (briefly): The concept of the bias-variance tradeoff can be useful if you don’t take it too seriously. The basic idea is as follows: if you’re estimating something, you can slice your data finer and finer, or perform more and more adjustments, each time getting a purer—and less biased—estimate. But each subdivision or each adjustment reduces your sample size or increases potential estimation error, hence the variance of your estimate goes up. That story is real. In lots and lots of examples, there’s a continuum between a completely unadjusted general estimate (high bias, low variance) and a specific, focused, adjusted estimate (low bias, high variance). Suppose, for example, you’re using data from a large experiment to estimate the effect of a treatment on a fairly narrow group, say, white men between the ages of 45 and 50. At one extreme, you could just take the estimated treatment e
Introduction: Jeff Leek just posted the discussions of his paper (with Leah Jager), “An estimate of the science-wise false discovery rate and application to the top medical literature,” along with some further comments of his own. Here are my original thoughts on an earlier version of their article. Keith O’Rourke and I expanded these thoughts into a formal comment for the journal. We’re pretty much in agreement with John Ioannidis (you can find his discussion in the top link above). In quick summary, I agree with Jager and Leek that this is an important topic. I think there are two key places where Keith and I disagree with them: 1. They take published p-values at face value whereas we consider them as the result of a complicated process of selection. This is something I didn’t used to think much about, but now I’ve become increasingly convinced that the problems with published p-values is not a simple file-drawer effect or the case of a few p=0.051 values nudged toward p=0.049, bu
6 0.17473516 255 andrew gelman stats-2010-09-04-How does multilevel modeling affect the estimate of the grand mean?
7 0.16450399 803 andrew gelman stats-2011-07-14-Subtleties with measurement-error models for the evaluation of wacky claims
8 0.16406512 1955 andrew gelman stats-2013-07-25-Bayes-respecting experimental design and other things
9 0.1612331 2093 andrew gelman stats-2013-11-07-I’m negative on the expression “false positives”
10 0.1501687 822 andrew gelman stats-2011-07-26-Any good articles on the use of error bars?
11 0.14837998 2102 andrew gelman stats-2013-11-15-“Are all significant p-values created equal?”
12 0.14598998 160 andrew gelman stats-2010-07-23-Unhappy with improvement by a factor of 10^29
13 0.14511801 899 andrew gelman stats-2011-09-10-The statistical significance filter
14 0.14328058 1162 andrew gelman stats-2012-02-11-Adding an error model to a deterministic model
15 0.14057033 777 andrew gelman stats-2011-06-23-Combining survey data obtained using different modes of sampling
16 0.13256958 1038 andrew gelman stats-2011-12-02-Donate Your Data to Science!
17 0.12414662 1348 andrew gelman stats-2012-05-27-Question 17 of my final exam for Design and Analysis of Sample Surveys
18 0.11876051 1377 andrew gelman stats-2012-06-13-A question about AIC
19 0.11392251 184 andrew gelman stats-2010-08-04-That half-Cauchy prior
20 0.11241823 310 andrew gelman stats-2010-10-02-The winner’s curse
topicId topicWeight
[(0, 0.089), (1, 0.095), (2, 0.103), (3, -0.087), (4, 0.021), (5, -0.035), (6, 0.087), (7, 0.026), (8, -0.05), (9, -0.075), (10, -0.039), (11, -0.032), (12, -0.006), (13, 0.024), (14, -0.072), (15, -0.021), (16, -0.059), (17, -0.004), (18, 0.01), (19, -0.026), (20, 0.023), (21, -0.03), (22, 0.075), (23, -0.026), (24, 0.021), (25, 0.006), (26, -0.034), (27, 0.027), (28, -0.023), (29, -0.063), (30, -0.002), (31, 0.061), (32, -0.032), (33, -0.073), (34, 0.001), (35, -0.045), (36, 0.017), (37, -0.076), (38, 0.044), (39, -0.078), (40, -0.067), (41, -0.026), (42, -0.172), (43, 0.073), (44, -0.081), (45, 0.056), (46, 0.071), (47, 0.088), (48, -0.085), (49, 0.037)]
simIndex simValue blogId blogTitle
same-blog 1 0.99833739 494 andrew gelman stats-2010-12-31-Type S error rates for classical and Bayesian single and multiple comparison procedures
Introduction: Type S error: When your estimate is the wrong sign, compared to the true value of the parameter Type M error: When the magnitude of your estimate is far off, compared to the true value of the parameter More here.
2 0.73294312 1377 andrew gelman stats-2012-06-13-A question about AIC
Introduction: Jacob Oaknin asks: Akaike ‘s selection criterion is often justified on the basis of the empirical risk of a ML estimate being a biased estimate of the true generalization error of a parametric family, say the family, S_m, of linear regressors on a m-dimensional variable x=(x_1,..,x_m) with gaussian noise independent of x (for instance in “Unifying the derivations for the Akaike and Corrected Akaike information criteria”, by J.E.Cavanaugh, Statistics and Probability Letters, vol. 33, 1997, pp. 201-208). On the other hand, the family S_m is known to have finite VC-dimension (VC = m+1), and this fact should grant that empirical risk minimizer is asymtotically consistent regardless of the underlying probability distribution, and in particular for the assumed gaussian distribution of noise(“An overview of statistical learning theory”, by V.N.Vapnik, IEEE Transactions On Neural Networks, vol. 10, No. 5, 1999, pp. 988-999) What am I missing? My reply: I’m no expert on AIC so
3 0.68459731 1348 andrew gelman stats-2012-05-27-Question 17 of my final exam for Design and Analysis of Sample Surveys
Introduction: 17. In a survey of n people, half are asked if they support “the health care law recently passed by Congress” and half are asked if they support “the law known as Obamacare.” The goal is to estimate the effect of the wording on the proportion of Yes responses. How large must n be for the effect to be estimated within a standard error of 5 percentage points? Solution to question 16 From yesterday : 16. You are doing a survey in a war-torn country to estimate what percentage of unemployed men support the rebels in a civil war. Express this as a ratio estimation problem, where goal is to estimate Y.bar/X.bar. What are x and y here? Give the estimate and standard error for the percentage of unemployed men who support the rebels. Solution: x is 1 if the respondent is an unemployed man, 0 otherwise. y is 1 if the respondent is an unemployed man and supports the rebels, 0 otherwise. The estimate is y.bar/x.bar [typo fixed], the standard error is (1/x.bar)*(1/sqrt(n))*s.z, whe
Introduction: David Radwin writes: I am seeking a statistic measuring an estimate’s reliability or stability as an alternative to the coefficient of variation (CV), also known as the relative standard error. The CV is the standard error of an estimate (proportion, mean, regression coefficient, etc.) divided by the estimate itself, usually expressed as a percentage. For example, if a survey finds 15% unemployment with a 6% standard error, the CV is .06/.15 = .4 = 40%. Some US government agencies flag or suppress as unreliable any estimate with a CV over a certain threshold such as 30% or 50%. But this standard can be arbitrary (for example, 85% employment would have a much lower CV of .06/.85 = 7%), and the CV has other drawbacks I won’t elaborate here. I don’t need an evaluation of the wisdom of using the CV or anything else for measuring an estimate’s stability, but one of my projects calls for such a measure and I would like to find a better alternative. Can you or your blog readers suggest
5 0.63695103 255 andrew gelman stats-2010-09-04-How does multilevel modeling affect the estimate of the grand mean?
Introduction: Subhadeep Mukhopadhyay writes: I am convinced of the power of hierarchical modeling and individual parameter pooling concept. I was wondering how could multi-level modeling could influence the estimate of grad mean (NOT individual label). My reply: Multilevel modeling will affect the estimate of the grand mean in two ways: 1. If the group-level mean is correlated with group size, then the partial pooling will change the estimate of the grand mean (and, indeed, you might want to include group size or some similar variable as a group-level predictor. 2. In any case, the extra error term(s) in a multilevel model will typically affect the standard error of everything, including the estimate of the grand mean.
6 0.63126332 960 andrew gelman stats-2011-10-15-The bias-variance tradeoff
7 0.59637856 1361 andrew gelman stats-2012-06-02-Question 23 of my final exam for Design and Analysis of Sample Surveys
8 0.58375663 1792 andrew gelman stats-2013-04-07-X on JLP
9 0.57942998 822 andrew gelman stats-2011-07-26-Any good articles on the use of error bars?
10 0.57606554 1757 andrew gelman stats-2013-03-11-My problem with the Lindley paradox
11 0.55068219 1737 andrew gelman stats-2013-02-25-Correlation of 1 . . . too good to be true?
12 0.54189652 1761 andrew gelman stats-2013-03-13-Lame Statistics Patents
13 0.53478765 1345 andrew gelman stats-2012-05-26-Question 16 of my final exam for Design and Analysis of Sample Surveys
14 0.53385085 1358 andrew gelman stats-2012-06-01-Question 22 of my final exam for Design and Analysis of Sample Surveys
15 0.5225116 1349 andrew gelman stats-2012-05-28-Question 18 of my final exam for Design and Analysis of Sample Surveys
16 0.5218057 160 andrew gelman stats-2010-07-23-Unhappy with improvement by a factor of 10^29
17 0.52090871 310 andrew gelman stats-2010-10-02-The winner’s curse
18 0.5198729 803 andrew gelman stats-2011-07-14-Subtleties with measurement-error models for the evaluation of wacky claims
19 0.50375319 918 andrew gelman stats-2011-09-21-Avoiding boundary estimates in linear mixed models
topicId topicWeight
[(15, 0.064), (16, 0.06), (24, 0.247), (86, 0.097), (99, 0.306)]
simIndex simValue blogId blogTitle
same-blog 1 0.99569452 494 andrew gelman stats-2010-12-31-Type S error rates for classical and Bayesian single and multiple comparison procedures
Introduction: Type S error: When your estimate is the wrong sign, compared to the true value of the parameter Type M error: When the magnitude of your estimate is far off, compared to the true value of the parameter More here.
2 0.98497546 1367 andrew gelman stats-2012-06-05-Question 26 of my final exam for Design and Analysis of Sample Surveys
Introduction: 26. You have just graded an an exam with 28 questions and 15 students. You fit a logistic item- response model estimating ability, difficulty, and discrimination parameters. Which of the following statements are basically true? (Indicate all that apply.) (a) If a question is answered correctly by students with very low and very high ability, but is missed by students in the middle, it will have a high value for its discrimination parameter. (b) It is not possible to fit an item-response model when you have more questions than students. In order to fit the model, you either need to reduce the number of questions (for example, by discarding some questions or by putting together some questions into a combined score) or increase the number of students in the dataset. (c) To keep the model identified, you can set one of the difficulty parameters or one of the ability parameters to zero and set one of the discrimination parameters to 1. (d) If two students answer the same number of q
Introduction: Someone who wants to remain anonymous writes: I am working to create a more accurate in-game win probability model for basketball games. My idea is for each timestep in a game (a second, 5 seconds, etc), use the Vegas line, the current score differential, who has the ball, and the number of possessions played already (to account for differences in pace) to create a point estimate probability of the home team winning. This problem would seem to fit a multi-level model structure well. It seems silly to estimate 2,000 regressions (one for each timestep), but the coefficients should vary at each timestep. Do you have suggestions for what type of model this could/would be? Additionally, I believe this needs to be some form of logit/probit given the binary dependent variable (win or loss). Finally, do you have suggestions for what package could accomplish this in Stata or R? To answer the questions in reverse order: 3. I’d hope this could be done in Stan (which can be run from R)
4 0.98012066 779 andrew gelman stats-2011-06-25-Avoiding boundary estimates using a prior distribution as regularization
Introduction: For awhile I’ve been fitting most of my multilevel models using lmer/glmer, which gives point estimates of the group-level variance parameters (maximum marginal likelihood estimate for lmer and an approximation for glmer). I’m usually satisfied with this–sure, point estimation understates the uncertainty in model fitting, but that’s typically the least of our worries. Sometimes, though, lmer/glmer estimates group-level variances at 0 or estimates group-level correlation parameters at +/- 1. Typically, when this happens, it’s not that we’re so sure the variance is close to zero or that the correlation is close to 1 or -1; rather, the marginal likelihood does not provide a lot of information about these parameters of the group-level error distribution. I don’t want point estimates on the boundary. I don’t want to say that the unexplained variance in some dimension is exactly zero. One way to handle this problem is full Bayes: slap a prior on sigma, do your Gibbs and Metropolis
5 0.97984135 899 andrew gelman stats-2011-09-10-The statistical significance filter
Introduction: I’ve talked about this a bit but it’s never had its own blog entry (until now). Statistically significant findings tend to overestimate the magnitude of effects. This holds in general (because E(|x|) > |E(x)|) but even more so if you restrict to statistically significant results. Here’s an example. Suppose a true effect of theta is unbiasedly estimated by y ~ N (theta, 1). Further suppose that we will only consider statistically significant results, that is, cases in which |y| > 2. The estimate “|y| conditional on |y|>2″ is clearly an overestimate of |theta|. First off, if |theta|<2, the estimate |y| conditional on statistical significance is not only too high in expectation, it's always too high. This is a problem, given that |theta| is in reality probably is less than 2. (The low-hangning fruit have already been picked, remember?) But even if |theta|>2, the estimate |y| conditional on statistical significance will still be too high in expectation. For a discussion o
6 0.97937584 1474 andrew gelman stats-2012-08-29-More on scaled-inverse Wishart and prior independence
7 0.97923541 846 andrew gelman stats-2011-08-09-Default priors update?
8 0.97896844 1240 andrew gelman stats-2012-04-02-Blogads update
9 0.97510624 2299 andrew gelman stats-2014-04-21-Stan Model of the Week: Hierarchical Modeling of Supernovas
10 0.97492349 2055 andrew gelman stats-2013-10-08-A Bayesian approach for peer-review panels? and a speculation about Bruno Frey
11 0.97366387 2093 andrew gelman stats-2013-11-07-I’m negative on the expression “false positives”
13 0.97313416 781 andrew gelman stats-2011-06-28-The holes in my philosophy of Bayesian data analysis
14 0.97282505 669 andrew gelman stats-2011-04-19-The mysterious Gamma (1.4, 0.4)
15 0.97118652 1004 andrew gelman stats-2011-11-11-Kaiser Fung on how not to critique models
17 0.97081333 2149 andrew gelman stats-2013-12-26-Statistical evidence for revised standards
18 0.97066396 2208 andrew gelman stats-2014-02-12-How to think about “identifiability” in Bayesian inference?
19 0.97063994 1713 andrew gelman stats-2013-02-08-P-values and statistical practice
20 0.97020018 1637 andrew gelman stats-2012-12-24-Textbook for data visualization?