andrew_gelman_stats andrew_gelman_stats-2011 andrew_gelman_stats-2011-628 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: According to the National Weather Service : What is a 100 year flood? A 100 year flood is an event that statistically has a 1% chance of occurring in any given year. A 500 year flood has a .2% chance of occurring and a 1000 year flood has a .1% chance of occurring. The accompanying map shows a part of Tennessee that in May 2010 had 1000-year levels of flooding. At first, it seems hard to believe that a 1000-year flood would have just happened to occur last year. But then, this is just a 1000-year flood for that particular place. I don’t really have a sense of the statistics of these events. How many 100-year, 500-year, and 1000-year flood events have been recorded by the Weather Service, and when have they occurred?
sentIndex sentText sentNum sentScore
1 According to the National Weather Service : What is a 100 year flood? [sent-1, score-0.178]
2 A 100 year flood is an event that statistically has a 1% chance of occurring in any given year. [sent-2, score-1.551]
3 2% chance of occurring and a 1000 year flood has a . [sent-4, score-1.378]
4 The accompanying map shows a part of Tennessee that in May 2010 had 1000-year levels of flooding. [sent-6, score-0.365]
5 At first, it seems hard to believe that a 1000-year flood would have just happened to occur last year. [sent-7, score-1.168]
6 But then, this is just a 1000-year flood for that particular place. [sent-8, score-0.889]
7 I don’t really have a sense of the statistics of these events. [sent-9, score-0.088]
8 How many 100-year, 500-year, and 1000-year flood events have been recorded by the Weather Service, and when have they occurred? [sent-10, score-1.06]
wordName wordTfidf (topN-words)
[('flood', 0.851), ('weather', 0.206), ('occurring', 0.188), ('year', 0.178), ('chance', 0.161), ('service', 0.149), ('tennessee', 0.144), ('accompanying', 0.116), ('recorded', 0.107), ('occurred', 0.096), ('occur', 0.086), ('map', 0.079), ('event', 0.078), ('events', 0.076), ('levels', 0.069), ('shows', 0.062), ('statistically', 0.06), ('according', 0.059), ('happened', 0.059), ('national', 0.058), ('believe', 0.045), ('hard', 0.041), ('last', 0.041), ('part', 0.039), ('may', 0.038), ('particular', 0.038), ('given', 0.035), ('sense', 0.034), ('statistics', 0.03), ('seems', 0.029), ('first', 0.028), ('many', 0.026), ('really', 0.024), ('would', 0.016)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 628 andrew gelman stats-2011-03-25-100-year floods
Introduction: According to the National Weather Service : What is a 100 year flood? A 100 year flood is an event that statistically has a 1% chance of occurring in any given year. A 500 year flood has a .2% chance of occurring and a 1000 year flood has a .1% chance of occurring. The accompanying map shows a part of Tennessee that in May 2010 had 1000-year levels of flooding. At first, it seems hard to believe that a 1000-year flood would have just happened to occur last year. But then, this is just a 1000-year flood for that particular place. I don’t really have a sense of the statistics of these events. How many 100-year, 500-year, and 1000-year flood events have been recorded by the Weather Service, and when have they occurred?
2 0.26023784 1319 andrew gelman stats-2012-05-14-I hate to get all Gerd Gigerenzer on you here, but . . .
Introduction: Jonathan Cantor points me to an opinion piece by psychologist Reid Hastie, “Our Gift for Good Stories Blinds Us to the Truth.” I have mixed feelings about Hastie’s article. On one hand I do think his point is important. It’s not new to me, but presumably it’s new to many readers of bloomberg.com. I like Hastie’s book (with Robyn Dawes), Rational Choice in an Uncertain World, and I’m predisposed to like anything new that he writes. On the other hand, there’s something about Hastie’s article that bothered me. It seemed a bit smug, as if he thinks he understands the world and wants to just explain it to the rest of us. That could be fine—after all, Hastie is a distinguished psychology researcher—but I wasn’t so clear that he’s so clear on what he’s saying. For example: The human brain is designed to support two modes of thought: visual and narrative. These forms of thinking are universal across human societies throughout history, develop reliably early in individuals’ lives
3 0.092430949 1336 andrew gelman stats-2012-05-22-Battle of the Repo Man quotes: Reid Hastie’s turn
Introduction: In response to my comments on his recent opinion article on the the human tendency to overvalue information presented as stories, Reid Hastie writes : Andrew (and Commenters) … I’d like to try to clarify some of the statements and implications in my Bloomberg article on “Our Gift for Good Stories …” The essay is what it is, but some of the implications that I intended to convey do not seem to have been communicated effectively. So, let me take a shot at clarification here. (Of course I am not assuming that, once clarified, my statements are necessarily correct or that you will agree with them.) 1. What I meant in the sections of the paper that claimed the brain is naturally good at visual and causal (narrative) thinking, is that that the brain was probably selected, through evolutionary processes to be adaptively successful at those capacities. I don’t have good evidence for this claim … but, we do a lot of those kinds of thinking, we’re distinctive as a species in the ways
4 0.075873859 1237 andrew gelman stats-2012-03-30-Statisticians: When We Teach, We Don’t Practice What We Preach
Introduction: My new Chance ethics column (cowritten with Eric Loken). Click through and take a look. It’s a short article and I really like it. And here’s more Chance.
5 0.067627221 1501 andrew gelman stats-2012-09-18-More studies on the economic effects of climate change
Introduction: After writing yesterday’s post , I was going through Solomon Hsiang’s blog and found a post pointing to three studies from researchers at business schools: Severe Weather and Automobile Assembly Productivity Gérard P. Cachon, Santiago Gallino and Marcelo Olivares Abstract: It is expected that climate change could lead to an increased frequency of severe weather. In turn, severe weather intuitively should hamper the productivity of work that occurs outside. But what is the effect of rain, snow, fog, heat and wind on work that occurs indoors, such as the production of automobiles? Using weekly production data from 64 automobile plants in the United States over a ten-year period, we find that adverse weather conditions lead to a significant reduction in production. For example, one additional day of high wind advisory by the National Weather Service (i.e., maximum winds generally in excess of 44 miles per hour) reduces production by 26%, which is comparable in order of magnitude t
6 0.064398743 1558 andrew gelman stats-2012-11-02-Not so fast on levees and seawalls for NY harbor?
7 0.061726615 1897 andrew gelman stats-2013-06-13-When’s that next gamma-ray blast gonna come, already?
8 0.061679892 677 andrew gelman stats-2011-04-24-My NOAA story
9 0.059480578 851 andrew gelman stats-2011-08-12-year + (1|year)
10 0.05744018 2232 andrew gelman stats-2014-03-03-What is the appropriate time scale for blogging—the day or the week?
11 0.056652755 481 andrew gelman stats-2010-12-22-The Jumpstart financial literacy survey and the different purposes of tests
12 0.055707723 2181 andrew gelman stats-2014-01-21-The Commissar for Traffic presents the latest Five-Year Plan
13 0.04770568 389 andrew gelman stats-2010-11-01-Why it can be rational to vote
14 0.04770568 1565 andrew gelman stats-2012-11-06-Why it can be rational to vote
15 0.04248419 138 andrew gelman stats-2010-07-10-Creating a good wager based on probability estimates
16 0.042273268 797 andrew gelman stats-2011-07-11-How do we evaluate a new and wacky claim?
17 0.041394748 2288 andrew gelman stats-2014-04-10-Small multiples of lineplots > maps (ok, not always, but yes in this case)
18 0.041352391 563 andrew gelman stats-2011-02-07-Evaluating predictions of political events
19 0.040734842 981 andrew gelman stats-2011-10-30-rms2
20 0.040032096 142 andrew gelman stats-2010-07-12-God, Guns, and Gaydar: The Laws of Probability Push You to Overestimate Small Groups
topicId topicWeight
[(0, 0.047), (1, -0.022), (2, 0.014), (3, 0.005), (4, -0.007), (5, -0.001), (6, -0.002), (7, 0.009), (8, -0.006), (9, -0.003), (10, -0.0), (11, 0.005), (12, 0.006), (13, -0.017), (14, -0.032), (15, 0.021), (16, 0.022), (17, 0.021), (18, 0.009), (19, 0.008), (20, -0.022), (21, 0.046), (22, 0.002), (23, -0.009), (24, 0.003), (25, 0.019), (26, -0.02), (27, 0.003), (28, -0.002), (29, -0.011), (30, -0.006), (31, 0.009), (32, -0.02), (33, 0.005), (34, -0.009), (35, -0.018), (36, 0.016), (37, 0.013), (38, -0.027), (39, 0.005), (40, -0.026), (41, 0.002), (42, -0.013), (43, -0.014), (44, -0.027), (45, -0.015), (46, -0.046), (47, 0.003), (48, -0.024), (49, -0.016)]
simIndex simValue blogId blogTitle
same-blog 1 0.95603818 628 andrew gelman stats-2011-03-25-100-year floods
Introduction: According to the National Weather Service : What is a 100 year flood? A 100 year flood is an event that statistically has a 1% chance of occurring in any given year. A 500 year flood has a .2% chance of occurring and a 1000 year flood has a .1% chance of occurring. The accompanying map shows a part of Tennessee that in May 2010 had 1000-year levels of flooding. At first, it seems hard to believe that a 1000-year flood would have just happened to occur last year. But then, this is just a 1000-year flood for that particular place. I don’t really have a sense of the statistics of these events. How many 100-year, 500-year, and 1000-year flood events have been recorded by the Weather Service, and when have they occurred?
2 0.65233278 731 andrew gelman stats-2011-05-26-Lottery probability update
Introduction: It was reported last year that the national lottery of Israel featured the exact same 6 numbers (out of 45) twice in the same month, and statistics professor Isaac Meilijson of Tel Aviv University was quoted as saying that “the incident of six numbers repeating themselves within a month is an event of once in 10,000 years.” I shouldn’t mock when it comes to mathematics–after all, I proved a false theorem once! (Or, to be precise, my collaborator and I published a false claim which we thought we’d proved, thus we thought was a theorem.) So let me retract the mockery and move, first to the mathematics and then to the statistics. First, how many possibilities are there in pick 6 out of 45? It’s (45*44*43*42*41*40)/6! = 8,145,060. Let’s call this number N. Second, what’s the probability that the same numbers repeat in a single calendar month? I’ve been told that the Israeli lottery has 2 draws per week, That’s 104/12=8.67 draws per month. Or maybe they skip some holiday
3 0.60292792 873 andrew gelman stats-2011-08-26-Luck or knowledge?
Introduction: Joan Ginther has won the Texas lottery four times. First, she won $5.4 million, then a decade later, she won $2million, then two years later $3million and in the summer of 2010, she hit a $10million jackpot. The odds of this has been calculated at one in eighteen septillion and luck like this could only come once every quadrillion years. According to Forbes, the residents of Bishop, Texas, seem to believe God was behind it all. The Texas Lottery Commission told Mr Rich that Ms Ginther must have been ‘born under a lucky star’, and that they don’t suspect foul play. Harper’s reporter Nathanial Rich recently wrote an article about Ms Ginther, which calls the the validity of her ‘luck’ into question. First, he points out, Ms Ginther is a former math professor with a PhD from Stanford University specialising in statistics. More at Daily Mail. [Edited Saturday] In comments, C Ryan King points to the original article at Harper’s and Bill Jefferys to Wired .
4 0.59288222 180 andrew gelman stats-2010-08-03-Climate Change News
Introduction: I. State of the Climate report The National Oceanic and Atmospheric Administration recently released their “State of the Climate Report” for 2009 . The report has chapters discussing global climate (temperatures, water vapor, cloudiness, alpine glaciers,…); oceans (ocean heat content, sea level, sea surface temperatures, etc.); the arctic (sea ice extent, permafrost, vegetation, and so on); Antarctica (weather observations, sea ice extent,…), and regional climates. NOAA also provides a nice page that lets you display any of 11 relevant time-series datasets (land-surface air temperature, sea level, ocean heat content, September arctic sea-ice extent, sea-surface temperature, northern hemisphere snow cover, specific humidity, glacier mass balance, marine air temperature, tropospheric temperature, and stratospheric temperature). Each of the plots overlays data from several databases (not necessarily indepenedent of each other), and you can select which ones to include or leave
Introduction: The other day we had a fun little discussion in the comments section of the sister blog about the appropriateness of stating forecast probabilities to the nearest tenth of a percentage point. It started when Josh Tucker posted this graph from Nate Silver : My first reaction was: this looks pretty but it’s hyper-precise. I’m a big fan of Nate’s work, but all those little wiggles on the graph can’t really mean anything. And what could it possibly mean to compute this probability to that level of precision? In the comments, people came at me from two directions. From one side, Jeffrey Friedman expressed a hard core attitude that it’s meaningless to give a probability forecast of a unique event: What could it possibly mean, period, given that this election will never be repeated? . . . I know there’s a vast literature on this, but I’m still curious, as a non-statistician, what it could mean for there to be a meaningful 65% probability (as opposed to a non-quantifiab
6 0.56451225 549 andrew gelman stats-2011-02-01-“Roughly 90% of the increase in . . .” Hey, wait a minute!
7 0.55975437 474 andrew gelman stats-2010-12-18-The kind of frustration we could all use more of
9 0.55045509 1623 andrew gelman stats-2012-12-14-GiveWell charity recommendations
10 0.54962122 1897 andrew gelman stats-2013-06-13-When’s that next gamma-ray blast gonna come, already?
11 0.54631793 1547 andrew gelman stats-2012-10-25-College football, voting, and the law of large numbers
13 0.54042596 473 andrew gelman stats-2010-12-17-Why a bonobo won’t play poker with you
14 0.53982651 1501 andrew gelman stats-2012-09-18-More studies on the economic effects of climate change
15 0.53388244 108 andrew gelman stats-2010-06-24-Sometimes the raw numbers are better than a percentage
17 0.52543032 138 andrew gelman stats-2010-07-10-Creating a good wager based on probability estimates
18 0.52530551 1058 andrew gelman stats-2011-12-14-Higgs bozos: Rosencrantz and Guildenstern are spinning in their graves
19 0.52252555 1086 andrew gelman stats-2011-12-27-The most dangerous jobs in America
topicId topicWeight
[(10, 0.02), (16, 0.016), (24, 0.039), (63, 0.46), (99, 0.246)]
simIndex simValue blogId blogTitle
same-blog 1 0.91986984 628 andrew gelman stats-2011-03-25-100-year floods
Introduction: According to the National Weather Service : What is a 100 year flood? A 100 year flood is an event that statistically has a 1% chance of occurring in any given year. A 500 year flood has a .2% chance of occurring and a 1000 year flood has a .1% chance of occurring. The accompanying map shows a part of Tennessee that in May 2010 had 1000-year levels of flooding. At first, it seems hard to believe that a 1000-year flood would have just happened to occur last year. But then, this is just a 1000-year flood for that particular place. I don’t really have a sense of the statistics of these events. How many 100-year, 500-year, and 1000-year flood events have been recorded by the Weather Service, and when have they occurred?
2 0.91768289 568 andrew gelman stats-2011-02-11-Calibration in chess
Introduction: Has anybody done this study yet? I’m curious about the results. Perhaps there’s some chess-playing cognitive psychologist who’d like to collaborate on this?
3 0.88265693 739 andrew gelman stats-2011-05-31-When Did Girls Start Wearing Pink?
Introduction: That cute picture is of toddler FDR in a dress, from 1884. Jeanne Maglaty writes : A Ladies’ Home Journal article [or maybe from a different source, according to a commenter] in June 1918 said, “The generally accepted rule is pink for the boys, and blue for the girls. The reason is that pink, being a more decided and stronger color, is more suitable for the boy, while blue, which is more delicate and dainty, is prettier for the girl.” Other sources said blue was flattering for blonds, pink for brunettes; or blue was for blue-eyed babies, pink for brown-eyed babies, according to Paoletti. In 1927, Time magazine printed a chart showing sex-appropriate colors for girls and boys according to leading U.S. stores. In Boston, Filene’s told parents to dress boys in pink. So did Best & Co. in New York City, Halle’s in Cleveland and Marshall Field in Chicago. Today’s color dictate wasn’t established until the 1940s . . . When the women’s liberation movement arrived in the mid-1960s, w
4 0.84596896 684 andrew gelman stats-2011-04-28-Hierarchical ordered logit or probit
Introduction: Jeff writes: How far off is bglmer and can it handle ordered logit or multinom logit? My reply: bglmer is very close. No ordered logit but I was just talking about it with Sophia today. My guess is that the easiest way to fit a hierarchical ordered logit or multinom logit will be to use stan. For right now I’d recommend using glmer/bglmer to fit the ordered logits in order (e.g., 1 vs. 2,3,4, then 2 vs. 3,4, then 3 vs. 4). Or maybe there’s already a hierarchical multinomial logit in mcmcpack or somewhere?
Introduction: The official announcement: The Excellence in Statistical Reporting Award for 2010 is presented to Felix Salmon for his body of work, which exemplifies the highest standards of scientific reporting. His insightful use of statistics as a tool to understanding the world of business and economics, areas that are critical in today’s economy, sets a new standard in statistical investigative reporting. Here are some examples: Tiger Woods Nigerian spammers How the government fudges job statistics This one is important to me. The idea is that “statistical reporting” is not just traditional science reporting (journalist talks with scientists and tries to understand the consensus) or science popularization or silly feature stories about the lottery. Salmon is doing investigative reporting using statistical thinking. Also, from a political angle, Salmon’s smart and quantitatively sophisticated work (as well as that of others such as Nate Silver) is an important counterweigh
6 0.83715814 745 andrew gelman stats-2011-06-04-High-level intellectual discussions in the Columbia statistics department
7 0.82465917 313 andrew gelman stats-2010-10-03-A question for psychometricians
8 0.82159442 126 andrew gelman stats-2010-07-03-Graphical presentation of risk ratios
9 0.78328967 1078 andrew gelman stats-2011-12-22-Tables as graphs: The Ramanujan principle
10 0.7701093 782 andrew gelman stats-2011-06-29-Putting together multinomial discrete regressions by combining simple logits
11 0.74837983 1621 andrew gelman stats-2012-12-13-Puzzles of criminal justice
12 0.72435689 293 andrew gelman stats-2010-09-23-Lowess is great
13 0.70753872 102 andrew gelman stats-2010-06-21-Why modern art is all in the mind
14 0.70619351 1480 andrew gelman stats-2012-09-02-“If our product is harmful . . . we’ll stop making it.”
15 0.70366186 1484 andrew gelman stats-2012-09-05-Two exciting movie ideas: “Second Chance U” and “The New Dirty Dozen”
16 0.69203222 2163 andrew gelman stats-2014-01-08-How to display multinominal logit results graphically?
17 0.67595923 1316 andrew gelman stats-2012-05-12-black and Black, white and White
18 0.66800636 1201 andrew gelman stats-2012-03-07-Inference = data + model
19 0.66654813 2249 andrew gelman stats-2014-03-15-Recently in the sister blog
20 0.65766561 286 andrew gelman stats-2010-09-20-Are the Democrats avoiding a national campaign?