andrew_gelman_stats andrew_gelman_stats-2014 andrew_gelman_stats-2014-2226 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: This is an echo of yesterday’s post, Basketball Stats: Don’t model the probability of win, model the expected score differential . As with basketball, so with baseball: as the great Bill James wrote, if you want to predict a pitcher’s win-loss record, it’s better to use last year’s ERA than last year’s W-L. As with basketball and baseball, so with epidemiology: as Joseph Delaney points out in my favorite blog that nobody reads, you will see much better prediction if you first model change in the parameter (e.g. blood pressure) and then convert that to the binary disease state (e.g. hypertension) then if you just develop a logistic model for prob(hypertension). As with basketball, baseball, and epidemiology, so with political science: instead of modeling election winners, better to model vote differential, a point that I made back in 1993 (see page 120 here ) but which seems to continually need repeating . A forecasting method should get essentially no credit for correctl
sentIndex sentText sentNum sentScore
1 This is an echo of yesterday’s post, Basketball Stats: Don’t model the probability of win, model the expected score differential . [sent-1, score-0.978]
2 As with basketball, so with baseball: as the great Bill James wrote, if you want to predict a pitcher’s win-loss record, it’s better to use last year’s ERA than last year’s W-L. [sent-2, score-0.138]
3 As with basketball and baseball, so with epidemiology: as Joseph Delaney points out in my favorite blog that nobody reads, you will see much better prediction if you first model change in the parameter (e. [sent-3, score-0.564]
4 blood pressure) and then convert that to the binary disease state (e. [sent-5, score-0.226]
5 hypertension) then if you just develop a logistic model for prob(hypertension). [sent-7, score-0.243]
6 As with basketball, baseball, and epidemiology, so with political science: instead of modeling election winners, better to model vote differential, a point that I made back in 1993 (see page 120 here ) but which seems to continually need repeating . [sent-8, score-0.71]
7 A forecasting method should get essentially no credit for correctly predicting the winner in 1960, 1968, or 2000 and very little for predicting the winner in 1964 or 1964, but there’s information in vote differential, all the same. [sent-9, score-0.606]
8 As with basketball, baseball, and epidemiology, and political science, so with econometrics: Even in recent years, with all the sophistication in economic statistics, you’ll still see people fitting logistic models for binary outcomes even when the continuous variable is readily available. [sent-10, score-0.601]
9 (See, for example, the second-to-last paragraph here , which is actually an economist doing political science, but I’m pretty sure there are lots of examples of this sort of thing in econ too. [sent-11, score-0.07]
10 Why do people keep modeling the discrete variable? [sent-14, score-0.09]
11 Some of the answer is statistical naivety, a simple “like goes with like” attitude that it makes sense to predict W-L from W-L rather than ERA. [sent-15, score-0.166]
12 More generally there’s the attitude that we should be modeling what we ultimately care about. [sent-16, score-0.175]
13 If the objective is to learn about wins, we should study wins directly. [sent-17, score-0.381]
14 To which I reply, sure, study wins, but it will be more statistically efficient to do this in a two-stage process: first study vote differential given X, then study wins given vote differential and X. [sent-18, score-1.797]
15 The key is that vote differential is available, and a simply performing a logit model for wins alone is implicitly taking this differential as latent or missing data, thus throwing away information. [sent-19, score-1.468]
16 Finally, from the econometrics direction, I see a bias or robustness argument. [sent-20, score-0.367]
17 The idea is that it’s safer, in some way, to model the outcome of interest, as this model will not be sensitive to assumptions about the distribution of the intermediate variable. [sent-21, score-0.365]
18 For example, a linear model for score differentials could be inappropriate for games where one team runs up the score (or, conversely, for those games where the team that’s winning sends in the subs so that the score is less lopsided than it would be if both teams were playing their hardest). [sent-22, score-1.301]
19 In response to this, I would make my usual argument that your models already have bias and robustness issues in that, to do your regression at all, you’re already pooling data from many years, many places, many different situations, etc. [sent-23, score-0.427]
20 If the use of continuous data can increase your statistical efficiency—and it will—this in turn will allow you to do less pooling of data to construct estimates that are reliable enough for you to work with. [sent-24, score-0.314]
wordName wordTfidf (topN-words)
[('differential', 0.419), ('basketball', 0.296), ('wins', 0.284), ('baseball', 0.21), ('epidemiology', 0.198), ('vote', 0.192), ('score', 0.185), ('hypertension', 0.174), ('model', 0.154), ('robustness', 0.126), ('winner', 0.116), ('pooling', 0.113), ('econometrics', 0.106), ('games', 0.105), ('binary', 0.097), ('study', 0.097), ('predicting', 0.091), ('modeling', 0.09), ('logistic', 0.089), ('continuous', 0.087), ('lopsided', 0.087), ('attitude', 0.085), ('pitcher', 0.082), ('prob', 0.082), ('differentials', 0.082), ('predict', 0.081), ('team', 0.079), ('bias', 0.078), ('continually', 0.076), ('repeating', 0.071), ('variable', 0.071), ('political', 0.07), ('hardest', 0.07), ('safer', 0.068), ('delaney', 0.066), ('echo', 0.066), ('blood', 0.066), ('readily', 0.065), ('sophistication', 0.065), ('science', 0.063), ('convert', 0.063), ('winners', 0.06), ('ok', 0.059), ('reliable', 0.059), ('intermediate', 0.057), ('see', 0.057), ('better', 0.057), ('construct', 0.055), ('inappropriate', 0.055), ('already', 0.055)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000004 2226 andrew gelman stats-2014-02-26-Econometrics, political science, epidemiology, etc.: Don’t model the probability of a discrete outcome, model the underlying continuous variable
Introduction: This is an echo of yesterday’s post, Basketball Stats: Don’t model the probability of win, model the expected score differential . As with basketball, so with baseball: as the great Bill James wrote, if you want to predict a pitcher’s win-loss record, it’s better to use last year’s ERA than last year’s W-L. As with basketball and baseball, so with epidemiology: as Joseph Delaney points out in my favorite blog that nobody reads, you will see much better prediction if you first model change in the parameter (e.g. blood pressure) and then convert that to the binary disease state (e.g. hypertension) then if you just develop a logistic model for prob(hypertension). As with basketball, baseball, and epidemiology, so with political science: instead of modeling election winners, better to model vote differential, a point that I made back in 1993 (see page 120 here ) but which seems to continually need repeating . A forecasting method should get essentially no credit for correctl
Introduction: Someone who wants to remain anonymous writes: I am working to create a more accurate in-game win probability model for basketball games. My idea is for each timestep in a game (a second, 5 seconds, etc), use the Vegas line, the current score differential, who has the ball, and the number of possessions played already (to account for differences in pace) to create a point estimate probability of the home team winning. This problem would seem to fit a multi-level model structure well. It seems silly to estimate 2,000 regressions (one for each timestep), but the coefficients should vary at each timestep. Do you have suggestions for what type of model this could/would be? Additionally, I believe this needs to be some form of logit/probit given the binary dependent variable (win or loss). Finally, do you have suggestions for what package could accomplish this in Stata or R? To answer the questions in reverse order: 3. I’d hope this could be done in Stan (which can be run from R)
Introduction: Wayne Folta points me to “EigenBracket 2012: Using Graph Theory to Predict NCAA March Madness Basketball” and writes, “I [Folta] have got to believe that he’s simply re-invented a statistical method in a graph-ish context, but don’t know enough to judge.” I have not looked in detail at the method being presented here—I’m not much of college basketball fan—but I’d like to use this as an excuse to make one of my favorite general point, which is that a good way to characterize any statistical method is by what information it uses. The basketball ranking method here uses score differentials between teams in the past season. On the plus side, that is better than simply using one-loss records (which (a) discards score differentials and (b) discards information on who played whom). On the minus side, the method appears to be discretizing the scores (thus throwing away information on the exact score differential) and doesn’t use any external information such as external ratings. A
4 0.25839853 2262 andrew gelman stats-2014-03-23-Win probabilities during a sporting event
Introduction: Todd Schneider writes: Apropos of your recent blog post about modeling score differential of basketball games , I thought you might enjoy a site I built, gambletron2000.com , that gathers real-time win probabilities from betting markets for most major sports (including NBA and college basketball). My original goal was to use the variance of changes in win probabilities to quantify which games were the most exciting, but I got a bit carried away and ended up pursuing a bunch of other ideas, which you can read about in the full writeup here This particular passage from the anonymous someone in your post: My idea is for each timestep in a game (a second, 5 seconds, etc), use the Vegas line, the current score differential, who has the ball, and the number of possessions played already (to account for differences in pace) to create a point estimate probability of the home team winning. reminded me of a graph I made, which shows the mean-reverting tendency of N
5 0.22720867 2311 andrew gelman stats-2014-04-29-Bayesian Uncertainty Quantification for Differential Equations!
Introduction: Mark Girolami points us to this paper and software (with Oksana Chkrebtii, David Campbell, and Ben Calderhead). They write: We develop a general methodology for the probabilistic integration of differential equations via model based updating of a joint prior measure on the space of functions and their temporal and spatial derivatives. This results in a posterior measure over functions reflecting how well they satisfy the system of differential equations and corresponding initial and boundary values. We show how this posterior measure can be naturally incorporated within the Kennedy and O’Hagan framework for uncertainty quantification and provides a fully Bayesian approach to model calibration. . . . A broad variety of examples are provided to illustrate the potential of this framework for characterising discretization uncertainty, including initial value, delay, and boundary value differential equations, as well as partial differential equations. We also demonstrate our methodolo
6 0.21340743 2222 andrew gelman stats-2014-02-24-On deck this week
7 0.17134978 29 andrew gelman stats-2010-05-12-Probability of successive wins in baseball
8 0.15837249 1923 andrew gelman stats-2013-07-03-Bayes pays!
10 0.15556006 697 andrew gelman stats-2011-05-05-A statistician rereads Bill James
11 0.13953438 1544 andrew gelman stats-2012-10-22-Is it meaningful to talk about a probability of “65.7%” that Obama will win the election?
12 0.13866228 541 andrew gelman stats-2011-01-27-Why can’t I be more like Bill James, or, The use of default and default-like models
14 0.13051143 888 andrew gelman stats-2011-09-03-A psychology researcher asks: Is Anova dead?
17 0.12215078 2255 andrew gelman stats-2014-03-19-How Americans vote
18 0.11512344 1339 andrew gelman stats-2012-05-23-Learning Differential Geometry for Hamiltonian Monte Carlo
19 0.11326864 1431 andrew gelman stats-2012-07-27-Overfitting
topicId topicWeight
[(0, 0.221), (1, 0.065), (2, 0.103), (3, 0.07), (4, 0.002), (5, 0.025), (6, -0.039), (7, -0.044), (8, 0.076), (9, 0.008), (10, 0.057), (11, 0.082), (12, -0.032), (13, -0.088), (14, -0.123), (15, 0.017), (16, 0.051), (17, -0.025), (18, 0.044), (19, -0.049), (20, -0.03), (21, 0.05), (22, 0.014), (23, -0.014), (24, 0.01), (25, 0.067), (26, 0.021), (27, -0.0), (28, -0.066), (29, -0.207), (30, -0.032), (31, -0.025), (32, 0.021), (33, -0.005), (34, 0.016), (35, 0.045), (36, 0.07), (37, 0.0), (38, -0.07), (39, 0.015), (40, 0.042), (41, -0.038), (42, -0.014), (43, -0.02), (44, -0.012), (45, 0.03), (46, -0.022), (47, 0.024), (48, -0.113), (49, 0.031)]
simIndex simValue blogId blogTitle
same-blog 1 0.93333805 2226 andrew gelman stats-2014-02-26-Econometrics, political science, epidemiology, etc.: Don’t model the probability of a discrete outcome, model the underlying continuous variable
Introduction: This is an echo of yesterday’s post, Basketball Stats: Don’t model the probability of win, model the expected score differential . As with basketball, so with baseball: as the great Bill James wrote, if you want to predict a pitcher’s win-loss record, it’s better to use last year’s ERA than last year’s W-L. As with basketball and baseball, so with epidemiology: as Joseph Delaney points out in my favorite blog that nobody reads, you will see much better prediction if you first model change in the parameter (e.g. blood pressure) and then convert that to the binary disease state (e.g. hypertension) then if you just develop a logistic model for prob(hypertension). As with basketball, baseball, and epidemiology, so with political science: instead of modeling election winners, better to model vote differential, a point that I made back in 1993 (see page 120 here ) but which seems to continually need repeating . A forecasting method should get essentially no credit for correctl
Introduction: Someone who wants to remain anonymous writes: I am working to create a more accurate in-game win probability model for basketball games. My idea is for each timestep in a game (a second, 5 seconds, etc), use the Vegas line, the current score differential, who has the ball, and the number of possessions played already (to account for differences in pace) to create a point estimate probability of the home team winning. This problem would seem to fit a multi-level model structure well. It seems silly to estimate 2,000 regressions (one for each timestep), but the coefficients should vary at each timestep. Do you have suggestions for what type of model this could/would be? Additionally, I believe this needs to be some form of logit/probit given the binary dependent variable (win or loss). Finally, do you have suggestions for what package could accomplish this in Stata or R? To answer the questions in reverse order: 3. I’d hope this could be done in Stan (which can be run from R)
Introduction: I know next to nothing about golf. My mini-golf scores typically approach the maximum of 7 per hole, and I’ve never actually played macro-golf. I did publish a paper on golf once ( A Probability Model for Golf Putting , with Deb Nolan), but it’s not so rare for people to publish papers on topics they know nothing about. Those who can’t, research. But I certainly have the ability to post other people’s ideas. Charles Murray writes: I [Murray] am playing around with the likelihood of Tiger Woods breaking Nicklaus’s record in the Majors. I’ve already gone on record two years ago with the reason why he won’t, but now I’m looking at it from a non-psychological perspective. Given the history of the majors, what how far above the average _for other great golfers_ does Tiger have to perform? Here’s the procedure I’ve been working on: 1. For all golfers who have won at at least one major since 1934 (the year the Masters began), create 120 lines: one for each Major for each year f
4 0.71819353 355 andrew gelman stats-2010-10-20-Andy vs. the Ideal Point Model of Voting
Introduction: Last week, as I walked into Andrew’s office for a meeting, he was formulating some misgivings about applying an ideal-point model to budgetary bills in the U.S. Senate. Andrew didn’t like that the model of a senator’s position was an indifference point rather than at their optimal point, and that the effect of moving away from a position was automatically modeled as increasing in one direction and decreasing in the other. Executive Summary The monotonicity of inverse logit entails that the expected vote for a bill among any fixed collection of senators’ ideal points is monotonically increasing (or decreasing) with the bill’s position, with direction determined by the outcome coding. The Ideal-Point Model The ideal-point model’s easy to write down, but hard to reason about because of all the polarity shifting going on. To recapitulate from Gelman and Hill’s Regression book (p. 317), using the U.S. Senate instead of the Supreme Court, and ignoring the dis
5 0.71204048 2262 andrew gelman stats-2014-03-23-Win probabilities during a sporting event
Introduction: Todd Schneider writes: Apropos of your recent blog post about modeling score differential of basketball games , I thought you might enjoy a site I built, gambletron2000.com , that gathers real-time win probabilities from betting markets for most major sports (including NBA and college basketball). My original goal was to use the variance of changes in win probabilities to quantify which games were the most exciting, but I got a bit carried away and ended up pursuing a bunch of other ideas, which you can read about in the full writeup here This particular passage from the anonymous someone in your post: My idea is for each timestep in a game (a second, 5 seconds, etc), use the Vegas line, the current score differential, who has the ball, and the number of possessions played already (to account for differences in pace) to create a point estimate probability of the home team winning. reminded me of a graph I made, which shows the mean-reverting tendency of N
6 0.7068181 151 andrew gelman stats-2010-07-16-Wanted: Probability distributions for rank orderings
8 0.68148661 934 andrew gelman stats-2011-09-30-Nooooooooooooooooooo!
10 0.65953881 1284 andrew gelman stats-2012-04-26-Modeling probability data
11 0.64549941 391 andrew gelman stats-2010-11-03-Some thoughts on election forecasting
12 0.64042914 29 andrew gelman stats-2010-05-12-Probability of successive wins in baseball
13 0.63792437 942 andrew gelman stats-2011-10-04-45% hitting, 25% fielding, 25% pitching, and 100% not telling us how they did it
14 0.63116193 623 andrew gelman stats-2011-03-21-Baseball’s greatest fielders
16 0.62524545 1395 andrew gelman stats-2012-06-27-Cross-validation (What is it good for?)
17 0.62448949 1544 andrew gelman stats-2012-10-22-Is it meaningful to talk about a probability of “65.7%” that Obama will win the election?
18 0.62272322 782 andrew gelman stats-2011-06-29-Putting together multinomial discrete regressions by combining simple logits
20 0.60728663 328 andrew gelman stats-2010-10-08-Displaying a fitted multilevel model
topicId topicWeight
[(9, 0.11), (15, 0.025), (16, 0.058), (24, 0.092), (27, 0.018), (36, 0.037), (41, 0.098), (59, 0.013), (86, 0.082), (94, 0.032), (99, 0.316)]
simIndex simValue blogId blogTitle
same-blog 1 0.96671337 2226 andrew gelman stats-2014-02-26-Econometrics, political science, epidemiology, etc.: Don’t model the probability of a discrete outcome, model the underlying continuous variable
Introduction: This is an echo of yesterday’s post, Basketball Stats: Don’t model the probability of win, model the expected score differential . As with basketball, so with baseball: as the great Bill James wrote, if you want to predict a pitcher’s win-loss record, it’s better to use last year’s ERA than last year’s W-L. As with basketball and baseball, so with epidemiology: as Joseph Delaney points out in my favorite blog that nobody reads, you will see much better prediction if you first model change in the parameter (e.g. blood pressure) and then convert that to the binary disease state (e.g. hypertension) then if you just develop a logistic model for prob(hypertension). As with basketball, baseball, and epidemiology, so with political science: instead of modeling election winners, better to model vote differential, a point that I made back in 1993 (see page 120 here ) but which seems to continually need repeating . A forecasting method should get essentially no credit for correctl
2 0.9408015 29 andrew gelman stats-2010-05-12-Probability of successive wins in baseball
Introduction: Dan Goldstein did an informal study asking people the following question: When two baseball teams play each other on two consecutive days, what is the probability that the winner of the first game will be the winner of the second game? You can make your own guess and the continue reading below. Dan writes: We asked two colleagues knowledgeable in baseball and the mathematics of forecasting. The answers came in between 65% and 70%. The true answer [based on Dan's analysis of a database of baseball games]: 51.3%, a little better than a coin toss. I have to say, I’m surprised his colleagues gave such extreme guesses. I was guessing something like 50%, myself, based on the following very crude reasoning: Suppose two unequal teams are playing, and the chance of team A beating team B is 55%. (This seems like a reasonable average of all matchups, which will include some more extreme disparities but also many more equal contests.) Then the chance of the same team
3 0.93498677 454 andrew gelman stats-2010-12-07-Diabetes stops at the state line?
Introduction: From Discover : Razib Khan asks: But follow the gradient from El Paso to the Illinois-Missouri border. The differences are small across state lines, but the consistent differences along the borders really don’t make. Are there state-level policies or regulations causing this? Or, are there state-level differences in measurement? This weird pattern shows up in other CDC data I’ve seen. Turns out that CDC isn’t providing data , they’re providing model . Frank Howland answered: I suspect the answer has to do with the manner in which the county estimates are produced. I went to the original data source, the CDC, and then to the relevant FAQ . There they say that the diabetes prevalence estimates come from the “CDC’s Behavioral Risk Factor Surveillance System (BRFSS) and data from the U.S. Census Bureau’s Population Estimates Program. The BRFSS is an ongoing, monthly, state-based telephone survey of the adult population. The survey provides state-specific informati
4 0.93446702 1142 andrew gelman stats-2012-01-29-Difficulties with the 1-4-power transformation
Introduction: John Hayes writes: I am a fan of the quarter root transform ever since reading about it on your blog . However, today my student and I hit a wall that I’m hoping you might have some insight on. By training, I am a psychophysicist (think SS Stevens), and people in my field often log transform data prior to analysis. However, this data frequently contains zeros, so I’ve tried using quarter root transforms to get around this. But until today, I had never tried to back transform the plot axis for readability. I assumed this would be straightforward – alas it is not. Specifically, we quarter root transformed our data, performed an ANOVA, got what we thought was a reasonable effect, and then plotted the data. So far so good. However, the LS means in question are below 1, meaning that raising them to the 4th power just makes them smaller, and uninterpretable in the original metric. Do you have any thoughts or insights you might share? My reply: I don’t see the problem with pre
5 0.93354464 640 andrew gelman stats-2011-03-31-Why Edit Wikipedia?
Introduction: Zoe Corbyn’s article for The Guardian (UK), titled Wikipedia wants more contributions from academics , and the followup discussion on Slashdot got me thinking about my own Wikipedia edits. The article quotes Dario Taraborelli, a research analyst for the Wikimedia Foundation, as saying “Academics are trapped in this paradox of using Wikipedia but not contributing,” Huh? I’m really wondering what man-in-the-street wrote all the great stats stuff out there. And what’s the paradox? I use lots of things without contributing to them. Taraborelli is further quoted as saying “The Wikimedia Foundation is looking at how it might capture expert conversation about Wikipedia content happening on other websites and feed it back to the community as a way of providing pointers for improvement.” This struck home. I recently went through the entry for latent Dirichlet allocation and found a bug in their derivation. I wrote up a revised derivation and posted it on my own blog .
6 0.93290454 1816 andrew gelman stats-2013-04-21-Exponential increase in the number of stat majors
7 0.9327994 1110 andrew gelman stats-2012-01-10-Jobs in statistics research! In New Jersey!
8 0.93145621 1226 andrew gelman stats-2012-03-22-Story time meets the all-else-equal fallacy and the fallacy of measurement
9 0.93044221 1961 andrew gelman stats-2013-07-29-Postdocs in probabilistic modeling! With David Blei! And Stan!
10 0.92962396 1565 andrew gelman stats-2012-11-06-Why it can be rational to vote
11 0.92961895 389 andrew gelman stats-2010-11-01-Why it can be rational to vote
12 0.92765158 516 andrew gelman stats-2011-01-14-A new idea for a science core course based entirely on computer simulation
13 0.9268145 571 andrew gelman stats-2011-02-13-A departmental wiki page?
14 0.92601871 1337 andrew gelman stats-2012-05-22-Question 12 of my final exam for Design and Analysis of Sample Surveys
15 0.92431784 1715 andrew gelman stats-2013-02-09-Thomas Hobbes would be spinning in his grave
16 0.92405611 903 andrew gelman stats-2011-09-13-Duke postdoctoral fellowships in nonparametric Bayes & high-dimensional data
17 0.92119205 1669 andrew gelman stats-2013-01-12-The power of the puzzlegraph
18 0.92060018 2260 andrew gelman stats-2014-03-22-Postdoc at Rennes on multilevel missing data imputation
19 0.91992337 1009 andrew gelman stats-2011-11-14-Wickham R short course
20 0.91863966 778 andrew gelman stats-2011-06-24-New ideas on DIC from Martyn Plummer and Sumio Watanabe