andrew_gelman_stats andrew_gelman_stats-2012 andrew_gelman_stats-2012-1387 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: I know next to nothing about golf. My mini-golf scores typically approach the maximum of 7 per hole, and I’ve never actually played macro-golf. I did publish a paper on golf once ( A Probability Model for Golf Putting , with Deb Nolan), but it’s not so rare for people to publish papers on topics they know nothing about. Those who can’t, research. But I certainly have the ability to post other people’s ideas. Charles Murray writes: I [Murray] am playing around with the likelihood of Tiger Woods breaking Nicklaus’s record in the Majors. I’ve already gone on record two years ago with the reason why he won’t, but now I’m looking at it from a non-psychological perspective. Given the history of the majors, what how far above the average _for other great golfers_ does Tiger have to perform? Here’s the procedure I’ve been working on: 1. For all golfers who have won at at least one major since 1934 (the year the Masters began), create 120 lines: one for each Major for each year f
sentIndex sentText sentNum sentScore
1 I did publish a paper on golf once ( A Probability Model for Golf Putting , with Deb Nolan), but it’s not so rare for people to publish papers on topics they know nothing about. [sent-3, score-0.132]
2 Charles Murray writes: I [Murray] am playing around with the likelihood of Tiger Woods breaking Nicklaus’s record in the Majors. [sent-6, score-0.084]
3 I’ve already gone on record two years ago with the reason why he won’t, but now I’m looking at it from a non-psychological perspective. [sent-7, score-0.136]
4 For all golfers who have won at at least one major since 1934 (the year the Masters began), create 120 lines: one for each Major for each year from the year the golfer turned 20 through the year he turned 49. [sent-10, score-1.748]
5 Ballesteros won five major championships, the last at age 31. [sent-13, score-0.425]
6 Then he developed back problems, and his ability to win majors (or any other tournament) was effectively ended by the time he was in his late thirties. [sent-14, score-0.613]
7 But golfers tend to develop physical problems as they age. [sent-15, score-0.353]
8 The failure of a championship golfer to compete in major championships is an extremely good indicator of not being able to win if he had competed. [sent-16, score-0.945]
9 Operationally, this means that the database contains 30×4=120 lines for each subject over the entire course of his career. [sent-18, score-0.118]
10 The one that appeals to me a priori consists of golfers who were born after the beginning of 1910 (a way of defining modern golf–it barely gets in both Hogan and Snead, the earliest golfers who intuitively seem to belong) and won at least two Majors (winnowing out the flukes). [sent-21, score-0.957]
11 Create a binary variable WIN for each line scored 0 if the subject did not win and 1 if he did. [sent-24, score-0.259]
12 Create a variable FLOORAGE that is the floor of the age of the subject at the time the tournament occurred. [sent-26, score-0.452]
13 In Stata, I created this with tabstat win if majors>=2 & born2>=3654,by(floorage) stat(mean sum count) These were the results: floorage mean sum N 20 0 0 159 21 . [sent-29, score-0.261]
14 0065789 1 152 49 0 0 80 So among that sample, at age 36, we have an n of 186, 7 wins, for a SUCCESSRATE of . [sent-56, score-0.168]
15 I think my shaky grasp of probabilities tells me that the probability of one of these golfers winning at least one major at age 36 was 14. [sent-58, score-0.941]
16 But that grasp doesn’t extend to calculating the probability of winning an aggregate number of majors over several years. [sent-60, score-0.638]
17 ” There’s information in every tournament that a player competes in. [sent-65, score-0.337]
18 By looking only at a small subset, you’re just reducing your sample size. [sent-66, score-0.124]
19 Instead I think it makes more sense to code the player’s position (rank among the winners, or maybe score compared to the top score in the tournament). [sent-71, score-0.114]
20 There is a logical appeal to this sort of “reduced-form” model but all the logical appeal in the world pales beside the imperative of data. [sent-75, score-0.312]
wordName wordTfidf (topN-words)
[('majors', 0.373), ('golfer', 0.353), ('golfers', 0.353), ('tournament', 0.215), ('win', 0.19), ('floorage', 0.176), ('tiger', 0.172), ('age', 0.168), ('major', 0.16), ('championship', 0.145), ('wins', 0.14), ('golf', 0.132), ('ballesteros', 0.118), ('successrate', 0.118), ('murray', 0.112), ('turned', 0.112), ('create', 0.103), ('year', 0.1), ('won', 0.097), ('championships', 0.097), ('grasp', 0.09), ('woods', 0.088), ('plausibly', 0.084), ('record', 0.084), ('sample', 0.072), ('occurred', 0.071), ('expressing', 0.071), ('sum', 0.071), ('subject', 0.069), ('player', 0.068), ('subset', 0.067), ('appeal', 0.065), ('logical', 0.064), ('number', 0.063), ('winning', 0.061), ('least', 0.058), ('score', 0.057), ('perform', 0.056), ('imperative', 0.054), ('competes', 0.054), ('julius', 0.054), ('predate', 0.054), ('looking', 0.052), ('probability', 0.051), ('ability', 0.05), ('lines', 0.049), ('deb', 0.048), ('appeals', 0.048), ('earliest', 0.048), ('thanksgiving', 0.047)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 1387 andrew gelman stats-2012-06-21-Will Tiger Woods catch Jack Nicklaus? And a discussion of the virtues of using continuous data even if your goal is discrete prediction
Introduction: I know next to nothing about golf. My mini-golf scores typically approach the maximum of 7 per hole, and I’ve never actually played macro-golf. I did publish a paper on golf once ( A Probability Model for Golf Putting , with Deb Nolan), but it’s not so rare for people to publish papers on topics they know nothing about. Those who can’t, research. But I certainly have the ability to post other people’s ideas. Charles Murray writes: I [Murray] am playing around with the likelihood of Tiger Woods breaking Nicklaus’s record in the Majors. I’ve already gone on record two years ago with the reason why he won’t, but now I’m looking at it from a non-psychological perspective. Given the history of the majors, what how far above the average _for other great golfers_ does Tiger have to perform? Here’s the procedure I’ve been working on: 1. For all golfers who have won at at least one major since 1934 (the year the Masters began), create 120 lines: one for each Major for each year f
2 0.24612367 1816 andrew gelman stats-2013-04-21-Exponential increase in the number of stat majors
Introduction: Joe Blitztein sent around the following graph: (The x-axis goes from 2000 to 2012 and the y=axis goes from 0 to 120.) 100 statistics majors (this combines sophomores, juniors, and seniors, but still, that’s a lot more than the 1 or 2 or 3 a year we’re used to seeing). At first I was like, whoa! But then I thought, why not 100 or even 200 or 300 statistics majors? Statistics is important in itself, it’s relatively easy as far as quantitative majors go, it’s applicable to lots of other areas. The real question should be not, What’s been happening that’s made statistics so trendy lately? but rather, What took so long for this to happen, and why isn’t statistics more popular? Both places where I studied as an undergraduate, statistics was just a subset of the math department, and maybe the only reason I ended up in statistics is that I took a probability course one semester because, at 5pm, it fit my schedule.
Introduction: Someone who wants to remain anonymous writes: I am working to create a more accurate in-game win probability model for basketball games. My idea is for each timestep in a game (a second, 5 seconds, etc), use the Vegas line, the current score differential, who has the ball, and the number of possessions played already (to account for differences in pace) to create a point estimate probability of the home team winning. This problem would seem to fit a multi-level model structure well. It seems silly to estimate 2,000 regressions (one for each timestep), but the coefficients should vary at each timestep. Do you have suggestions for what type of model this could/would be? Additionally, I believe this needs to be some form of logit/probit given the binary dependent variable (win or loss). Finally, do you have suggestions for what package could accomplish this in Stata or R? To answer the questions in reverse order: 3. I’d hope this could be done in Stan (which can be run from R)
Introduction: This is an echo of yesterday’s post, Basketball Stats: Don’t model the probability of win, model the expected score differential . As with basketball, so with baseball: as the great Bill James wrote, if you want to predict a pitcher’s win-loss record, it’s better to use last year’s ERA than last year’s W-L. As with basketball and baseball, so with epidemiology: as Joseph Delaney points out in my favorite blog that nobody reads, you will see much better prediction if you first model change in the parameter (e.g. blood pressure) and then convert that to the binary disease state (e.g. hypertension) then if you just develop a logistic model for prob(hypertension). As with basketball, baseball, and epidemiology, so with political science: instead of modeling election winners, better to model vote differential, a point that I made back in 1993 (see page 120 here ) but which seems to continually need repeating . A forecasting method should get essentially no credit for correctl
5 0.11004948 1169 andrew gelman stats-2012-02-15-Charles Murray on the new upper class
Introduction: The other day I posted some comments on the voting patterns of rich and poor in the context of Charles Murray’s recent book, “Coming Apart.” My graphs on income and voting are just fine, but I mischaracterized Murray’s statements. So I want to fix that right away. After that I have some thoughts on the book itself. In brief: 1. I was unfair to call him a Tucker Carlson. 2. Murray talks a lot about upper-class liberals. That’s fine but I think his discussion would be improved by also considering upper-class conservatives, given that I see the big culture war occurring within the upper class. 3. Using the case of Joe Paterno as an example, I discuss why Murray’s “preach what you practice” advice could be difficult to carry out in practice. Murray on the top 5% David Frum quoted Murray as writing that the top 5% “tends to be liberal—right? There’s no getting around it. Every way of answering this question produces a yes.” In response, Frum and I both pointed out t
6 0.10561018 2262 andrew gelman stats-2014-03-23-Win probabilities during a sporting event
7 0.10384195 2105 andrew gelman stats-2013-11-18-What’s my Kasparov number?
8 0.090344593 1819 andrew gelman stats-2013-04-23-Charles Murray’s “Coming Apart” and the measurement of social and political divisions
9 0.086605504 486 andrew gelman stats-2010-12-26-Age and happiness: The pattern isn’t as clear as you might think
10 0.086547039 1836 andrew gelman stats-2013-05-02-Culture clash
11 0.084662728 1544 andrew gelman stats-2012-10-22-Is it meaningful to talk about a probability of “65.7%” that Obama will win the election?
12 0.080949351 982 andrew gelman stats-2011-10-30-“There’s at least as much as an 80 percent chance . . .”
14 0.079005614 20 andrew gelman stats-2010-05-07-Bayesian hierarchical model for the prediction of soccer results
15 0.078979343 437 andrew gelman stats-2010-11-29-The mystery of the U-shaped relationship between happiness and age
16 0.077686608 1881 andrew gelman stats-2013-06-03-Boot
17 0.073711552 1903 andrew gelman stats-2013-06-17-Weak identification provides partial information
19 0.07031779 1899 andrew gelman stats-2013-06-14-Turing chess tournament!
20 0.069639787 1542 andrew gelman stats-2012-10-20-A statistical model for underdispersion
topicId topicWeight
[(0, 0.154), (1, -0.009), (2, 0.049), (3, 0.013), (4, 0.032), (5, 0.008), (6, 0.01), (7, 0.002), (8, 0.017), (9, -0.023), (10, 0.021), (11, 0.016), (12, -0.02), (13, -0.019), (14, -0.041), (15, -0.001), (16, 0.04), (17, -0.017), (18, 0.03), (19, -0.031), (20, -0.009), (21, 0.068), (22, -0.019), (23, 0.003), (24, 0.01), (25, 0.049), (26, -0.014), (27, 0.028), (28, -0.022), (29, -0.074), (30, 0.008), (31, -0.049), (32, -0.013), (33, 0.02), (34, 0.014), (35, 0.003), (36, 0.017), (37, 0.006), (38, -0.037), (39, 0.035), (40, 0.002), (41, -0.036), (42, 0.012), (43, -0.023), (44, 0.009), (45, 0.009), (46, 0.002), (47, 0.037), (48, -0.037), (49, 0.008)]
simIndex simValue blogId blogTitle
same-blog 1 0.9330467 1387 andrew gelman stats-2012-06-21-Will Tiger Woods catch Jack Nicklaus? And a discussion of the virtues of using continuous data even if your goal is discrete prediction
Introduction: I know next to nothing about golf. My mini-golf scores typically approach the maximum of 7 per hole, and I’ve never actually played macro-golf. I did publish a paper on golf once ( A Probability Model for Golf Putting , with Deb Nolan), but it’s not so rare for people to publish papers on topics they know nothing about. Those who can’t, research. But I certainly have the ability to post other people’s ideas. Charles Murray writes: I [Murray] am playing around with the likelihood of Tiger Woods breaking Nicklaus’s record in the Majors. I’ve already gone on record two years ago with the reason why he won’t, but now I’m looking at it from a non-psychological perspective. Given the history of the majors, what how far above the average _for other great golfers_ does Tiger have to perform? Here’s the procedure I’ve been working on: 1. For all golfers who have won at at least one major since 1934 (the year the Masters began), create 120 lines: one for each Major for each year f
2 0.80553681 2262 andrew gelman stats-2014-03-23-Win probabilities during a sporting event
Introduction: Todd Schneider writes: Apropos of your recent blog post about modeling score differential of basketball games , I thought you might enjoy a site I built, gambletron2000.com , that gathers real-time win probabilities from betting markets for most major sports (including NBA and college basketball). My original goal was to use the variance of changes in win probabilities to quantify which games were the most exciting, but I got a bit carried away and ended up pursuing a bunch of other ideas, which you can read about in the full writeup here This particular passage from the anonymous someone in your post: My idea is for each timestep in a game (a second, 5 seconds, etc), use the Vegas line, the current score differential, who has the ball, and the number of possessions played already (to account for differences in pace) to create a point estimate probability of the home team winning. reminded me of a graph I made, which shows the mean-reverting tendency of N
3 0.79684865 562 andrew gelman stats-2011-02-06-Statistician cracks Toronto lottery
Introduction: Christian points me to this amusing story by Jonah Lehrer about Mohan Srivastava, (perhaps the same person as R. Mohan Srivastava, coauthor of a book called Applied Geostatistics) who discovered a flaw in a scratch-off game in which he could figure out which tickets were likely to win based on partial information visible on the ticket. It appears that scratch-off lotteries elsewhere have similar flaws in their design. The obvious question is, why doesn’t the lottery create the patterns on the tickets (including which “teaser” numbers to reveal) completely at random? It shouldn’t be hard to design this so that zero information is supplied from the outside. in which case Srivastava’s trick would be impossible. So why not put down the numbers randomly? Lehrer quotes Srivastava as saying: The tickets are clearly mass-produced, which means there must be some computer program that lays down the numbers. Of course, it would be really nice if the computer could just spit out random
4 0.79024267 559 andrew gelman stats-2011-02-06-Bidding for the kickoff
Introduction: Steven Brams and James Jorash propose a system for reducing the advantage that comes from winning the coin flip in overtime: Dispensing with a coin toss, the teams would bid on where the ball is kicked from by the kicking team. In the NFL, it’s now the 30-yard line. Under Brams and Jorasch’s rule, the kicking team would be the team that bids the lower number, because it is willing to put itself at a disadvantage by kicking from farther back. However, it would not kick from the number it bids, but from the average of the two bids. To illustrate, assume team A bids to kick from the 38-yard line, while team B bids its 32-yard line. Team B would win the bidding and, therefore, be designated as the kick-off team. But B wouldn’t kick from 32, but instead from the average of 38 and 32–its 35-yard line. This is better for B by 3 yards than the 32-yard line that it proposed, because it’s closer to the end zone it is kicking towards. It’s also better for A by 3 yards to have B kick fr
5 0.7811771 29 andrew gelman stats-2010-05-12-Probability of successive wins in baseball
Introduction: Dan Goldstein did an informal study asking people the following question: When two baseball teams play each other on two consecutive days, what is the probability that the winner of the first game will be the winner of the second game? You can make your own guess and the continue reading below. Dan writes: We asked two colleagues knowledgeable in baseball and the mathematics of forecasting. The answers came in between 65% and 70%. The true answer [based on Dan's analysis of a database of baseball games]: 51.3%, a little better than a coin toss. I have to say, I’m surprised his colleagues gave such extreme guesses. I was guessing something like 50%, myself, based on the following very crude reasoning: Suppose two unequal teams are playing, and the chance of team A beating team B is 55%. (This seems like a reasonable average of all matchups, which will include some more extreme disparities but also many more equal contests.) Then the chance of the same team
8 0.74716073 2105 andrew gelman stats-2013-11-18-What’s my Kasparov number?
10 0.70678163 731 andrew gelman stats-2011-05-26-Lottery probability update
11 0.70575517 54 andrew gelman stats-2010-05-27-Hype about conditional probability puzzles
12 0.69566154 2267 andrew gelman stats-2014-03-26-Is a steal really worth 9 points?
13 0.69058555 1547 andrew gelman stats-2012-10-25-College football, voting, and the law of large numbers
14 0.6792447 253 andrew gelman stats-2010-09-03-Gladwell vs Pinker
15 0.67843699 813 andrew gelman stats-2011-07-21-Scrabble!
16 0.67682558 623 andrew gelman stats-2011-03-21-Baseball’s greatest fielders
17 0.66861784 1903 andrew gelman stats-2013-06-17-Weak identification provides partial information
18 0.66652226 1731 andrew gelman stats-2013-02-21-If a lottery is encouraging addictive gambling, don’t expand it!
19 0.66307312 1467 andrew gelman stats-2012-08-23-The pinch-hitter syndrome again
20 0.65448934 526 andrew gelman stats-2011-01-19-“If it saves the life of a single child…” and other nonsense
topicId topicWeight
[(9, 0.02), (11, 0.136), (15, 0.018), (16, 0.064), (24, 0.17), (41, 0.04), (63, 0.026), (77, 0.019), (82, 0.018), (85, 0.02), (86, 0.055), (89, 0.034), (92, 0.02), (96, 0.018), (99, 0.205)]
simIndex simValue blogId blogTitle
1 0.95422733 273 andrew gelman stats-2010-09-13-Update on marathon statistics
Introduction: Frank Hansen updates his story and writes: Here is a link to the new stuff. The update is a little less than half way down the page. 1. used display() instead of summary() 2. include a proxy for [non] newbies — whether I can find their name in a previous Chicago Marathon. 3. graph actual pace vs. fitted pace (color code newbie proxy) 4. estimate the model separately for newbies and non-newbies. some incidental discussion of sd of errors. There are a few things unfinished but I have to get to bed, I’m running the 2010 Chicago Half tomorrow morning, and they moved the start up from 7:30 to 7:00 because it’s the day of the Bears home opener too.
same-blog 2 0.93076611 1387 andrew gelman stats-2012-06-21-Will Tiger Woods catch Jack Nicklaus? And a discussion of the virtues of using continuous data even if your goal is discrete prediction
Introduction: I know next to nothing about golf. My mini-golf scores typically approach the maximum of 7 per hole, and I’ve never actually played macro-golf. I did publish a paper on golf once ( A Probability Model for Golf Putting , with Deb Nolan), but it’s not so rare for people to publish papers on topics they know nothing about. Those who can’t, research. But I certainly have the ability to post other people’s ideas. Charles Murray writes: I [Murray] am playing around with the likelihood of Tiger Woods breaking Nicklaus’s record in the Majors. I’ve already gone on record two years ago with the reason why he won’t, but now I’m looking at it from a non-psychological perspective. Given the history of the majors, what how far above the average _for other great golfers_ does Tiger have to perform? Here’s the procedure I’ve been working on: 1. For all golfers who have won at at least one major since 1934 (the year the Masters began), create 120 lines: one for each Major for each year f
3 0.91184449 1799 andrew gelman stats-2013-04-12-Stan 1.3.0 and RStan 1.3.0 Ready for Action
Introduction: The Stan Development Team is happy to announce that Stan 1.3.0 and RStan 1.3.0 are available for download. Follow the links on: Stan home page: http://mc-stan.org/ Please let us know if you have problems updating. Here’s the full set of release notes. v1.3.0 (12 April 2013) ====================================================================== Enhancements ---------------------------------- Modeling Language * forward sampling (random draws from distributions) in generated quantities * better error messages in parser * new distributions: + exp_mod_normal + gumbel + skew_normal * new special functions: + owenst * new broadcast (repetition) functions for vectors, arrays, matrices + rep_arrray + rep_matrix + rep_row_vector + rep_vector Command-Line * added option to display autocorrelations in the command-line program to print output * changed default point estimation routine from the command line to
4 0.89053893 2262 andrew gelman stats-2014-03-23-Win probabilities during a sporting event
Introduction: Todd Schneider writes: Apropos of your recent blog post about modeling score differential of basketball games , I thought you might enjoy a site I built, gambletron2000.com , that gathers real-time win probabilities from betting markets for most major sports (including NBA and college basketball). My original goal was to use the variance of changes in win probabilities to quantify which games were the most exciting, but I got a bit carried away and ended up pursuing a bunch of other ideas, which you can read about in the full writeup here This particular passage from the anonymous someone in your post: My idea is for each timestep in a game (a second, 5 seconds, etc), use the Vegas line, the current score differential, who has the ball, and the number of possessions played already (to account for differences in pace) to create a point estimate probability of the home team winning. reminded me of a graph I made, which shows the mean-reverting tendency of N
5 0.88996929 1474 andrew gelman stats-2012-08-29-More on scaled-inverse Wishart and prior independence
Introduction: I’ve had a couple of email conversations in the past couple days on dependence in multivariate prior distributions. Modeling the degrees of freedom and scale parameters in the t distribution First, in our Stan group we’ve been discussing the choice of priors for the degrees-of-freedom parameter in the t distribution. I wrote that also there’s the question of parameterization. It does not necessarily make sense to have independent priors on the df and scale parameters. In some sense, the meaning of the scale parameter changes with the df. Prior dependence between correlation and scale parameters in the scaled inverse-Wishart model The second case of parameterization in prior distribution arose from an email I received from Chris Chatham pointing me to this exploration by Matt Simpson of the scaled inverse-Wishart prior distribution for hierarchical covariance matrices. Simpson writes: A popular prior for Σ is the inverse-Wishart distribution [ not the same as the
7 0.87868112 297 andrew gelman stats-2010-09-27-An interesting education and statistics blog
8 0.87819213 1386 andrew gelman stats-2012-06-21-Belief in hell is associated with lower crime rates
9 0.87376088 846 andrew gelman stats-2011-08-09-Default priors update?
10 0.87077671 1225 andrew gelman stats-2012-03-22-Procrastination as a positive productivity strategy
11 0.86991656 1367 andrew gelman stats-2012-06-05-Question 26 of my final exam for Design and Analysis of Sample Surveys
12 0.8697449 1219 andrew gelman stats-2012-03-18-Tips on “great design” from . . . Microsoft!
13 0.86944592 1019 andrew gelman stats-2011-11-19-Validation of Software for Bayesian Models Using Posterior Quantiles
14 0.86821383 1465 andrew gelman stats-2012-08-21-D. Buggin
15 0.86596251 1004 andrew gelman stats-2011-11-11-Kaiser Fung on how not to critique models
16 0.86495483 494 andrew gelman stats-2010-12-31-Type S error rates for classical and Bayesian single and multiple comparison procedures
18 0.86394787 1062 andrew gelman stats-2011-12-16-Mr. Pearson, meet Mr. Mandelbrot: Detecting Novel Associations in Large Data Sets
19 0.86321861 1240 andrew gelman stats-2012-04-02-Blogads update