andrew_gelman_stats andrew_gelman_stats-2011 andrew_gelman_stats-2011-562 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Christian points me to this amusing story by Jonah Lehrer about Mohan Srivastava, (perhaps the same person as R. Mohan Srivastava, coauthor of a book called Applied Geostatistics) who discovered a flaw in a scratch-off game in which he could figure out which tickets were likely to win based on partial information visible on the ticket. It appears that scratch-off lotteries elsewhere have similar flaws in their design. The obvious question is, why doesn’t the lottery create the patterns on the tickets (including which “teaser” numbers to reveal) completely at random? It shouldn’t be hard to design this so that zero information is supplied from the outside. in which case Srivastava’s trick would be impossible. So why not put down the numbers randomly? Lehrer quotes Srivastava as saying: The tickets are clearly mass-produced, which means there must be some computer program that lays down the numbers. Of course, it would be really nice if the computer could just spit out random
sentIndex sentText sentNum sentScore
1 Christian points me to this amusing story by Jonah Lehrer about Mohan Srivastava, (perhaps the same person as R. [sent-1, score-0.075]
2 Mohan Srivastava, coauthor of a book called Applied Geostatistics) who discovered a flaw in a scratch-off game in which he could figure out which tickets were likely to win based on partial information visible on the ticket. [sent-2, score-0.699]
3 It appears that scratch-off lotteries elsewhere have similar flaws in their design. [sent-3, score-0.085]
4 The obvious question is, why doesn’t the lottery create the patterns on the tickets (including which “teaser” numbers to reveal) completely at random? [sent-4, score-0.734]
5 Lehrer quotes Srivastava as saying: The tickets are clearly mass-produced, which means there must be some computer program that lays down the numbers. [sent-8, score-0.497]
6 Of course, it would be really nice if the computer could just spit out random digits. [sent-9, score-0.314]
7 But that’s not possible, since the lottery corporation needs to control the number of winning tickets. [sent-10, score-0.754]
8 Instead, it has to generate the illusion of randomness while actually being carefully determined. [sent-12, score-0.134]
9 We’re talking about $3 payoffs here, so, no, the corporation does not need to control the number of winning tickets. [sent-14, score-0.414]
10 What they do need to control is the probability of a win, but that can be done using a completely random algorithm. [sent-15, score-0.428]
11 From reading the article, I think the real reason the winning tickets could be predicted is that the lottery tickets were designed to be misleadingly appealing. [sent-16, score-1.131]
12 Lehrer writes: Instead of just scratching off the latex and immediately discovering a loser, players have to spend time matching up the revealed numbers with the boards. [sent-17, score-0.604]
13 Ticket designers fill the cards with near-misses (two-in-a-row matchups instead of the necessary three) and players spend tantalizing seconds looking for their win. [sent-18, score-1.092]
14 “Ticket designers fill the cards with near-misses . [sent-20, score-0.459]
15 ”: This doesn’t sound like they’re just slapping down random numbers. [sent-23, score-0.228]
16 Instead, the system seems to be rigged in the fashion of old-time carnival games in order to manipulate one’s intuition that the probability of near-misses should be informative about the underlying probability of hits. [sent-24, score-0.521]
17 ( See here for some general discussion of the use of precursors to estimate the probability of extremely rare events. [sent-25, score-0.115]
18 ) In this sense, the story is slightly more interesting than “Lottery designers made a mistake. [sent-26, score-0.354]
19 ” The mistake they made is directly connected to the manipulations they make in order to sucker people into spend more money. [sent-27, score-0.35]
20 This news story should get him all the business he needs for awhile! [sent-31, score-0.162]
wordName wordTfidf (topN-words)
[('srivastava', 0.365), ('tickets', 0.32), ('lehrer', 0.286), ('lottery', 0.253), ('mohan', 0.199), ('designers', 0.192), ('players', 0.158), ('winning', 0.156), ('corporation', 0.149), ('cards', 0.14), ('random', 0.137), ('fill', 0.127), ('ticket', 0.125), ('spend', 0.118), ('instead', 0.117), ('probability', 0.115), ('control', 0.109), ('numbers', 0.094), ('win', 0.092), ('slapping', 0.091), ('lays', 0.091), ('scratching', 0.091), ('spit', 0.091), ('sucker', 0.091), ('tantalizing', 0.091), ('game', 0.09), ('slightly', 0.087), ('needs', 0.087), ('computer', 0.086), ('matchups', 0.085), ('rigged', 0.085), ('lotteries', 0.085), ('misleadingly', 0.082), ('latex', 0.079), ('loser', 0.077), ('story', 0.075), ('manipulations', 0.071), ('order', 0.07), ('manipulate', 0.069), ('jonah', 0.068), ('completely', 0.067), ('randomness', 0.067), ('visible', 0.067), ('fashion', 0.067), ('illusion', 0.067), ('flaw', 0.067), ('supplied', 0.066), ('discovering', 0.064), ('seconds', 0.064), ('coauthor', 0.063)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 562 andrew gelman stats-2011-02-06-Statistician cracks Toronto lottery
Introduction: Christian points me to this amusing story by Jonah Lehrer about Mohan Srivastava, (perhaps the same person as R. Mohan Srivastava, coauthor of a book called Applied Geostatistics) who discovered a flaw in a scratch-off game in which he could figure out which tickets were likely to win based on partial information visible on the ticket. It appears that scratch-off lotteries elsewhere have similar flaws in their design. The obvious question is, why doesn’t the lottery create the patterns on the tickets (including which “teaser” numbers to reveal) completely at random? It shouldn’t be hard to design this so that zero information is supplied from the outside. in which case Srivastava’s trick would be impossible. So why not put down the numbers randomly? Lehrer quotes Srivastava as saying: The tickets are clearly mass-produced, which means there must be some computer program that lays down the numbers. Of course, it would be really nice if the computer could just spit out random
2 0.14903028 731 andrew gelman stats-2011-05-26-Lottery probability update
Introduction: It was reported last year that the national lottery of Israel featured the exact same 6 numbers (out of 45) twice in the same month, and statistics professor Isaac Meilijson of Tel Aviv University was quoted as saying that “the incident of six numbers repeating themselves within a month is an event of once in 10,000 years.” I shouldn’t mock when it comes to mathematics–after all, I proved a false theorem once! (Or, to be precise, my collaborator and I published a false claim which we thought we’d proved, thus we thought was a theorem.) So let me retract the mockery and move, first to the mathematics and then to the statistics. First, how many possibilities are there in pick 6 out of 45? It’s (45*44*43*42*41*40)/6! = 8,145,060. Let’s call this number N. Second, what’s the probability that the same numbers repeat in a single calendar month? I’ve been told that the Israeli lottery has 2 draws per week, That’s 104/12=8.67 draws per month. Or maybe they skip some holiday
3 0.14116594 1731 andrew gelman stats-2013-02-21-If a lottery is encouraging addictive gambling, don’t expand it!
Introduction: This story from Vivian Yee seems just horrible to me. First the background: Pronto Lotto’s real business takes place in the carpeted, hushed area where its most devoted customers watch video screens from a scattering of tall silver tables, hour after hour, day after day. The players — mostly men, about a dozen at any given time — come on their lunch breaks or after work to study the screens, which are programmed with the Quick Draw lottery game, and flash a new set of winning numbers every four minutes. They have helped make Pronto Lotto the top Quick Draw vendor in the state, selling $3.3 million worth of tickets last year, more than $1 million more than the second busiest location, a World Books shop in Penn Station. Some stay for just a few minutes. Others play for the length of a workday, repeatedly traversing the few yards between their seats and the cash register as they hand the next wager to a clerk with a dollar bill or two, and return to wait. “It’s like my job, 24
4 0.13191873 2300 andrew gelman stats-2014-04-21-Ticket to Baaaath
Introduction: Ooooooh, I never ever thought I’d have a legitimate excuse to tell this story, and now I do! The story took place many years ago, but first I have to tell you what made me think of it: Rasmus Bååth posted the following comment last month: On airplane tickets a Swedish “å” is written as “aa” resulting in Rasmus Baaaath. Once I bought a ticket online and five minutes later a guy from Lufthansa calls me and asks if I misspelled my name… OK, now here’s my story (which is not nearly as good). A long time ago (but when I was already an adult), I was in England for some reason, and I thought I’d take a day trip from London to Bath. So here I am on line, trying to think of what to say at the ticket counter. I remember that in England, they call Bath, Bahth. So, should I ask for “a ticket to Bahth”? I’m not sure, I’m afraid that it will sound silly, like I’m trying to fake an English accent. So, when I get to the front of the line, I say, hesitantly, “I’d like a ticket to Bath?
5 0.1309935 2057 andrew gelman stats-2013-10-10-Chris Chabris is irritated by Malcolm Gladwell
Introduction: Christopher Chabris reviewed the new book by Malcolm Gladwell: One thing “David and Goliath” shows is that Mr. Gladwell has not changed his own strategy, despite serious criticism of his prior work. What he presents are mostly just intriguing possibilities and musings about human behavior, but what his publisher sells them as, and what his readers may incorrectly take them for, are lawful, causal rules that explain how the world really works. Mr. Gladwell should acknowledge when he is speculating or working with thin evidentiary soup. Yet far from abandoning his hand or even standing pat, Mr. Gladwell has doubled down. This will surely bring more success to a Goliath of nonfiction writing, but not to his readers. Afterward he blogged some further thoughts about the popular popular science writer. Good stuff . Chabris has a thoughtful explanation of why the “Gladwell is just an entertainer” alibi doesn’t work for him (Chabris). Some of his discussion reminds me of my articl
6 0.13081244 466 andrew gelman stats-2010-12-13-“The truth wears off: Is there something wrong with the scientific method?”
7 0.12685701 1242 andrew gelman stats-2012-04-03-Best lottery story ever
9 0.12043571 1255 andrew gelman stats-2012-04-10-Amtrak sucks
10 0.11713888 1490 andrew gelman stats-2012-09-09-I’m still wondering . . .
11 0.098899759 2224 andrew gelman stats-2014-02-25-Basketball Stats: Don’t model the probability of win, model the expected score differential.
12 0.095058732 1628 andrew gelman stats-2012-12-17-Statistics in a world where nothing is random
13 0.093966015 2297 andrew gelman stats-2014-04-20-Fooled by randomness
14 0.092712402 1448 andrew gelman stats-2012-08-07-Scientific fraud, double standards and institutions protecting themselves
15 0.091636792 2021 andrew gelman stats-2013-09-13-Swiss Jonah Lehrer
16 0.088080451 762 andrew gelman stats-2011-06-13-How should journals handle replication studies?
17 0.083545767 1467 andrew gelman stats-2012-08-23-The pinch-hitter syndrome again
18 0.082951844 1544 andrew gelman stats-2012-10-22-Is it meaningful to talk about a probability of “65.7%” that Obama will win the election?
19 0.082282528 1105 andrew gelman stats-2012-01-08-Econ debate about prices at a fancy restaurant
20 0.07995566 1847 andrew gelman stats-2013-05-08-Of parsing and chess
topicId topicWeight
[(0, 0.136), (1, -0.017), (2, 0.023), (3, 0.011), (4, 0.023), (5, -0.016), (6, 0.043), (7, 0.01), (8, 0.013), (9, -0.045), (10, -0.0), (11, -0.014), (12, -0.003), (13, -0.018), (14, -0.027), (15, -0.005), (16, 0.015), (17, 0.001), (18, 0.054), (19, -0.009), (20, -0.038), (21, 0.045), (22, 0.023), (23, 0.058), (24, -0.015), (25, 0.041), (26, 0.014), (27, 0.084), (28, -0.063), (29, -0.045), (30, 0.019), (31, -0.041), (32, 0.007), (33, 0.035), (34, 0.001), (35, -0.027), (36, -0.0), (37, -0.015), (38, -0.018), (39, 0.03), (40, 0.006), (41, -0.044), (42, 0.012), (43, -0.027), (44, -0.014), (45, -0.002), (46, 0.025), (47, 0.023), (48, -0.035), (49, -0.019)]
simIndex simValue blogId blogTitle
same-blog 1 0.96179223 562 andrew gelman stats-2011-02-06-Statistician cracks Toronto lottery
Introduction: Christian points me to this amusing story by Jonah Lehrer about Mohan Srivastava, (perhaps the same person as R. Mohan Srivastava, coauthor of a book called Applied Geostatistics) who discovered a flaw in a scratch-off game in which he could figure out which tickets were likely to win based on partial information visible on the ticket. It appears that scratch-off lotteries elsewhere have similar flaws in their design. The obvious question is, why doesn’t the lottery create the patterns on the tickets (including which “teaser” numbers to reveal) completely at random? It shouldn’t be hard to design this so that zero information is supplied from the outside. in which case Srivastava’s trick would be impossible. So why not put down the numbers randomly? Lehrer quotes Srivastava as saying: The tickets are clearly mass-produced, which means there must be some computer program that lays down the numbers. Of course, it would be really nice if the computer could just spit out random
2 0.8054899 2105 andrew gelman stats-2013-11-18-What’s my Kasparov number?
Introduction: A colleague writes: Personally my Kasparov number is two: I beat ** in a regular tournament game, and ** beat Kasparov! That’s pretty impressive, especially given that I didn’t know this guy played chess at all! Anyway, this got me thinking, what’s my Kasparov number? OK, that’s easy. I beat Magnus Carlsen the other day when he was passing through town on vacation, Carlsen beat Anand, . . . OK, just kidding. What is my Kasparov number, though? Note that the definition, unlike that of the Erdos or Bacon numbers, is asymmetric: it has to be that I had a victory over person 1, and person 1 had a victory over person 2, etc., and ultimately person N-1 had a victory over Kasparov. The games don’t have to be in time order, they just all have to be victories. And we’ll further require that the games all be played after childhood and before senility (i.e., it doesn’t count if I happened to play someone who happens to be a cousin of some grandmaster whom he beat when they were b
3 0.77397925 813 andrew gelman stats-2011-07-21-Scrabble!
Introduction: AT writes : Sitting on my [AT's] to-do list for a while now has been an exploration of Scrabble from an experimental design point of view; how to better design a tournament to make the variance as small as possible while still preserving the appearance of the home game to its players. . . . I’m proud (relieved?) to say that I’ve finally finished the first draft of this work for two-player head-to-head games, with a duplication method that ensures that if the game were repeated, each player would receive tiles from the reserve in the same sequence: think of the tiles being laid out in order (but unseen to the players), so that one player draws from the front and the other draws from the back. . . . One goal of this was to figure out how much of the variance in score comes from the tile order and how much comes from the board, given that a tile order would be expected. It turns out to be about half-bag, half-board . . . Some other findings from the simulations: The blank
4 0.76549739 559 andrew gelman stats-2011-02-06-Bidding for the kickoff
Introduction: Steven Brams and James Jorash propose a system for reducing the advantage that comes from winning the coin flip in overtime: Dispensing with a coin toss, the teams would bid on where the ball is kicked from by the kicking team. In the NFL, it’s now the 30-yard line. Under Brams and Jorasch’s rule, the kicking team would be the team that bids the lower number, because it is willing to put itself at a disadvantage by kicking from farther back. However, it would not kick from the number it bids, but from the average of the two bids. To illustrate, assume team A bids to kick from the 38-yard line, while team B bids its 32-yard line. Team B would win the bidding and, therefore, be designated as the kick-off team. But B wouldn’t kick from 32, but instead from the average of 38 and 32–its 35-yard line. This is better for B by 3 yards than the 32-yard line that it proposed, because it’s closer to the end zone it is kicking towards. It’s also better for A by 3 yards to have B kick fr
5 0.76500565 2262 andrew gelman stats-2014-03-23-Win probabilities during a sporting event
Introduction: Todd Schneider writes: Apropos of your recent blog post about modeling score differential of basketball games , I thought you might enjoy a site I built, gambletron2000.com , that gathers real-time win probabilities from betting markets for most major sports (including NBA and college basketball). My original goal was to use the variance of changes in win probabilities to quantify which games were the most exciting, but I got a bit carried away and ended up pursuing a bunch of other ideas, which you can read about in the full writeup here This particular passage from the anonymous someone in your post: My idea is for each timestep in a game (a second, 5 seconds, etc), use the Vegas line, the current score differential, who has the ball, and the number of possessions played already (to account for differences in pace) to create a point estimate probability of the home team winning. reminded me of a graph I made, which shows the mean-reverting tendency of N
7 0.74669474 29 andrew gelman stats-2010-05-12-Probability of successive wins in baseball
8 0.73512846 1467 andrew gelman stats-2012-08-23-The pinch-hitter syndrome again
9 0.72645313 218 andrew gelman stats-2010-08-20-I think you knew this already
10 0.71706367 138 andrew gelman stats-2010-07-10-Creating a good wager based on probability estimates
11 0.70453644 731 andrew gelman stats-2011-05-26-Lottery probability update
12 0.70018142 1847 andrew gelman stats-2013-05-08-Of parsing and chess
14 0.69478911 2267 andrew gelman stats-2014-03-26-Is a steal really worth 9 points?
15 0.69161922 1242 andrew gelman stats-2012-04-03-Best lottery story ever
16 0.68583488 54 andrew gelman stats-2010-05-27-Hype about conditional probability puzzles
17 0.68196571 2322 andrew gelman stats-2014-05-06-Priors I don’t believe
18 0.67135596 1731 andrew gelman stats-2013-02-21-If a lottery is encouraging addictive gambling, don’t expand it!
20 0.64724898 1724 andrew gelman stats-2013-02-16-Zero Dark Thirty and Bayes’ theorem
topicId topicWeight
[(2, 0.015), (9, 0.045), (15, 0.016), (16, 0.08), (21, 0.013), (24, 0.151), (25, 0.015), (27, 0.032), (28, 0.014), (29, 0.029), (72, 0.014), (77, 0.161), (85, 0.026), (86, 0.014), (89, 0.034), (99, 0.223)]
simIndex simValue blogId blogTitle
1 0.94757366 1784 andrew gelman stats-2013-04-01-Wolfram on Mandelbrot
Introduction: The most perfect pairing of author and subject since Nicholson Baker and John Updike. Here’s Wolfram on the great researcher of fractals : In his way, Mandelbrot paid me some great compliments. When I was in my 20s, and he in his 60s, he would ask about my scientific work: “How can so many people take someone so young so seriously?” In 2002, my book “A New Kind of Science”—in which I argued that many phenomena across science are the complex results of relatively simple, program-like rules—appeared. Mandelbrot seemed to see it as a direct threat, once declaring that “Wolfram’s ‘science’ is not new except when it is clearly wrong; it deserves to be completely disregarded.” In private, though, several mutual friends told me, he fretted that in the long view of history it would overwhelm his work. In retrospect, I don’t think Mandelbrot had much to worry about on this account. The link from the above review came from Peter Woit, who also points to a review by Brian Hayes wit
2 0.94604945 1684 andrew gelman stats-2013-01-20-Ugly ugly ugly
Introduction: Denis Cote sends the following , under the heading, “Some bad graphs for your enjoyment”: To start with, they don’t know how to spell “color.” Seriously, though, the graph is a mess. The circular display implies a circular or periodic structure that isn’t actually in the data, the cramped display requires the use of an otherwise-unnecessary color code that makes it difficult to find or make sense of the information, the alphabetical ordering (without even supplying state names, only abbreviations) makes it further difficult to find any patterns. It would be so much better, and even easier, to just display a set of small maps shading states on whether they have different laws. But that’s part of the problem—the clearer graph would also be easier to make! To get a distinctive graph, there needs to be some degree of difficulty. The designers continue with these monstrosities: Here they decide to display only 5 states at a time so that it’s really hard to see any big pi
3 0.93945765 1604 andrew gelman stats-2012-12-04-An epithet I can live with
Introduction: Here . Indeed, I’d much rather be a legend than a myth. I just want to clarify one thing. Walter Hickey writes: [Antony Unwin and Andrew Gelman] collaborated on this presentation where they take a hard look at what’s wrong with the recent trends of data visualization and infographics. The takeaway is that while there have been great leaps in visualization technology, some of the visualizations that have garnered the highest praises have actually been lacking in a number of key areas. Specifically, the pair does a takedown of the top visualizations of 2008 as decided by the popular statistics blog Flowing Data. This is a fair summary, but I want to emphasize that, although our dislike of some award-winning visualizations is central to our argument, it is only the first part of our story. As Antony and I worked more on our paper, and especially after seeing the discussions by Robert Kosara, Stephen Few, Hadley Wickham, and Paul Murrell (all to appear in Journal of Computati
4 0.93928695 978 andrew gelman stats-2011-10-28-Cool job opening with brilliant researchers at Yahoo
Introduction: Duncan Watts writes: The Human Social Dynamics Group in Yahoo Research is seeking highly qualified candidates for a post-doctoral research scientist position. The Human and Social Dynamics group is devoted to understanding the interplay between individual-level behavior (e.g. how people make decisions about what music they like, which dates to go on, or which groups to join) and the social environment in which individual behavior necessarily plays itself out. In particular, we are interested in: * Structure and evolution of social groups and networks * Decision making, social influence, diffusion, and collective decisions * Networking and collaborative problem solving. The intrinsically multi-disciplinary and cross-cutting nature of the subject demands an eclectic range of researchers, both in terms of domain-expertise (e.g. decision sciences, social psychology, sociology) and technical skills (e.g. statistical analysis, mathematical modeling, computer simulations, design o
same-blog 5 0.93595809 562 andrew gelman stats-2011-02-06-Statistician cracks Toronto lottery
Introduction: Christian points me to this amusing story by Jonah Lehrer about Mohan Srivastava, (perhaps the same person as R. Mohan Srivastava, coauthor of a book called Applied Geostatistics) who discovered a flaw in a scratch-off game in which he could figure out which tickets were likely to win based on partial information visible on the ticket. It appears that scratch-off lotteries elsewhere have similar flaws in their design. The obvious question is, why doesn’t the lottery create the patterns on the tickets (including which “teaser” numbers to reveal) completely at random? It shouldn’t be hard to design this so that zero information is supplied from the outside. in which case Srivastava’s trick would be impossible. So why not put down the numbers randomly? Lehrer quotes Srivastava as saying: The tickets are clearly mass-produced, which means there must be some computer program that lays down the numbers. Of course, it would be really nice if the computer could just spit out random
8 0.9132098 1124 andrew gelman stats-2012-01-17-How to map geographically-detailed survey responses?
9 0.90872449 401 andrew gelman stats-2010-11-08-Silly old chi-square!
10 0.88966227 1976 andrew gelman stats-2013-08-10-The birthday problem
11 0.88793385 1438 andrew gelman stats-2012-07-31-What is a Bayesian?
12 0.8854388 93 andrew gelman stats-2010-06-17-My proposal for making college admissions fairer
13 0.88350344 207 andrew gelman stats-2010-08-14-Pourquoi Google search est devenu plus raisonnable?
14 0.87911421 57 andrew gelman stats-2010-05-29-Roth and Amsterdam
15 0.87315756 1219 andrew gelman stats-2012-03-18-Tips on “great design” from . . . Microsoft!
16 0.87229264 1980 andrew gelman stats-2013-08-13-Test scores and grades predict job performance (but maybe not at Google)
17 0.87067747 1247 andrew gelman stats-2012-04-05-More philosophy of Bayes
18 0.87016529 2054 andrew gelman stats-2013-10-07-Bing is preferred to Google by people who aren’t like me
19 0.86975461 1561 andrew gelman stats-2012-11-04-Someone is wrong on the internet
20 0.8682003 1296 andrew gelman stats-2012-05-03-Google Translate for code, and an R help-list bot