andrew_gelman_stats andrew_gelman_stats-2014 andrew_gelman_stats-2014-2358 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Please answer the above question before reading on . . . I’m curious after reading Leif Nelson’s report that, based on research with Minah Jung, approximately 42% of the people they surveyed said they bought laundry detergent on their most recent trip to the store. I’m stunned that the number is so high. 42%??? That’s almost half the time. If we bought laundry detergent half the time we went to the store, our apartment would be stacked so full with the stuff, we wouldn’t be able to enter the door. I think we buy laundry detergent . . . ummm, how often? There are 40 of those little laundry packets in the box, we do laundry once a day, sometimes twice, let’s say 10 times a week, so this means we buy detergent about once every 4 weeks. We go to the store, hmmm, about once a day, let’s say 5 times a week to put our guess on the conservative side. So, 20 trips to the store for each purchase of detergent, that’s 5% of the time. Compared to us, lots of people must (a) go to
sentIndex sentText sentNum sentScore
1 I’m curious after reading Leif Nelson’s report that, based on research with Minah Jung, approximately 42% of the people they surveyed said they bought laundry detergent on their most recent trip to the store. [sent-4, score-1.407]
2 If we bought laundry detergent half the time we went to the store, our apartment would be stacked so full with the stuff, we wouldn’t be able to enter the door. [sent-10, score-1.414]
3 There are 40 of those little laundry packets in the box, we do laundry once a day, sometimes twice, let’s say 10 times a week, so this means we buy detergent about once every 4 weeks. [sent-15, score-1.799]
4 We go to the store, hmmm, about once a day, let’s say 5 times a week to put our guess on the conservative side. [sent-16, score-0.164]
5 So, 20 trips to the store for each purchase of detergent, that’s 5% of the time. [sent-17, score-0.371]
6 Compared to us, lots of people must (a) go to the store very rarely, (b) buy really small containers of laundry detergent, (c) do the laundry all the time. [sent-18, score-1.509]
7 I suppose that, all the time they lose by doing the laundry all the time, they save by not having to go the store every day. [sent-19, score-0.774]
8 Regarding their research, Nelson writes: Minah and I could cheerfully use the same data to write one of two papers. [sent-23, score-0.118]
9 The first could use a pervasive judgmental bias (18 out of 18 products show the effect! [sent-24, score-0.114]
10 Fortunately, this is a blog post, so I get to comfortably write about both. [sent-29, score-0.114]
11 I guess Nelson is joking here, but just in case he’s not: of course you should convey both results in your paper! [sent-30, score-0.131]
12 One good thing about working in political science is that it does seem that our field has a good tolerance for complexity and ambiguity. [sent-31, score-0.051]
13 It’s ok for us to write papers that convey mixed messages, if that is what we find. [sent-32, score-0.174]
14 If it’s really the case that psychology papers need to have a single-minded focus, that’s too bad. [sent-33, score-0.044]
wordName wordTfidf (topN-words)
[('detergent', 0.645), ('laundry', 0.466), ('store', 0.264), ('buy', 0.16), ('nelson', 0.142), ('minah', 0.142), ('frequency', 0.121), ('highlight', 0.09), ('bought', 0.09), ('convey', 0.077), ('rarely', 0.074), ('recent', 0.067), ('cheerfully', 0.065), ('containers', 0.065), ('jung', 0.065), ('times', 0.062), ('purchases', 0.061), ('buyers', 0.061), ('stacked', 0.061), ('leif', 0.061), ('decomposing', 0.061), ('comfortably', 0.061), ('cosmic', 0.058), ('judgmental', 0.058), ('week', 0.058), ('half', 0.058), ('trips', 0.056), ('pervasive', 0.056), ('joking', 0.054), ('strikingly', 0.054), ('ummm', 0.054), ('shopping', 0.053), ('underestimate', 0.053), ('write', 0.053), ('human', 0.052), ('purchase', 0.051), ('habits', 0.051), ('tolerance', 0.051), ('fortunately', 0.05), ('apartment', 0.05), ('surveyed', 0.049), ('stunned', 0.047), ('judgments', 0.046), ('trip', 0.046), ('irrational', 0.044), ('people', 0.044), ('overestimate', 0.044), ('enter', 0.044), ('papers', 0.044), ('go', 0.044)]
simIndex simValue blogId blogTitle
Introduction: Please answer the above question before reading on . . . I’m curious after reading Leif Nelson’s report that, based on research with Minah Jung, approximately 42% of the people they surveyed said they bought laundry detergent on their most recent trip to the store. I’m stunned that the number is so high. 42%??? That’s almost half the time. If we bought laundry detergent half the time we went to the store, our apartment would be stacked so full with the stuff, we wouldn’t be able to enter the door. I think we buy laundry detergent . . . ummm, how often? There are 40 of those little laundry packets in the box, we do laundry once a day, sometimes twice, let’s say 10 times a week, so this means we buy detergent about once every 4 weeks. We go to the store, hmmm, about once a day, let’s say 5 times a week to put our guess on the conservative side. So, 20 trips to the store for each purchase of detergent, that’s 5% of the time. Compared to us, lots of people must (a) go to
2 0.31795397 2356 andrew gelman stats-2014-06-02-On deck this week
Introduction: Mon: Why we hate stepwise regression Tues: Did you buy laundry detergent on their most recent trip to the store? Also comments on scientific publication and yet another suggestion to do a study that allows within-person comparisons Wed: All the Assumptions That Are My Life Thurs: Identifying pathways for managing multiple disturbances to limit plant invasions Fri: Statistically savvy journalism Sat: “Does researching casual marijuana use cause brain abnormalities?” Sun: Regression and causality and variable ordering
3 0.080812655 259 andrew gelman stats-2010-09-06-Inbox zero. Really.
Introduction: Just in time for the new semester: This time I’m sticking with the plan : 1. Don’t open a message until I’m ready to deal with it. 2. Don’t store anything–anything–in the inbox. 3. Put to-do items in the (physical) bookje rather than the (computer) “desktop.” 4. Never read email before 4pm. (This is the one rule I have been following. 5. Only one email session per day. (I’ll have to see how this one works.)
4 0.079208203 1369 andrew gelman stats-2012-06-06-Your conclusion is only as good as your data
Introduction: Jay Livingston points to an excellent rant from Peter Moskos, trashing a study about “food deserts” (which I kept reading as “food desserts”) in inner-city neighborhoods. Here’s Moskos: From the Times: There is no relationship between the type of food being sold in a neighborhood and obesity among its children and adolescents. Within a couple of miles of almost any urban neighborhood, “you can get basically any type of food,” said Roland Sturm of the RAND Corporation, lead author of one of the studies. “Maybe we should call it a food swamp rather than a desert,” he said. Sure thing, Sturm. But I suspect you wouldn’t think certain neighborhoods are swamped with good food if you actually got out of your office and went to one of the neighborhoods. After all, what are going to believe: A nice data set or your lying eyes? “Food outlet data … are classifıed using the North American Industry Classifıcation System (NAICS)” (p. 130). Assuming validity and reliability of NAICS
5 0.072884105 2065 andrew gelman stats-2013-10-17-Cool dynamic demographic maps provide beautiful illustration of Chris Rock effect
Introduction: Robert Gonzalez reports on some beautiful graphs from John Nelson. Here’s Nelson: The sexes start out homogenous, go super segregated in the teen years, segregate for business in the twenty-somethings, and re-couple for co-habitation years. Then the lights fade into faint pockets of pink. I [Nelson] am using simple tract-level population/gender counts from the US Census Bureau. Because their tract boundaries extend into the water and vacant area, I used NYC’s Bytes of the Big Apple zoning shapes to clip the census tracts to residentially zoned areas -giving me a more realistic (and more recognizable) definition of populated areas. The census breaks out their population counts by gender for five-year age spans ranging from teeny tiny infants through esteemed 85+ year-olds. And here’s Gonzalez: Between ages 0 and 14, the entire map is more or less an evenly mixed purple landscape; newborns, children and adolescents, after all, can’t really choose where the
6 0.05915406 2255 andrew gelman stats-2014-03-19-How Americans vote
7 0.058110446 1844 andrew gelman stats-2013-05-06-Against optimism about social science
8 0.055006001 1897 andrew gelman stats-2013-06-13-When’s that next gamma-ray blast gonna come, already?
9 0.054559749 1495 andrew gelman stats-2012-09-13-Win $5000 in the Economist’s data visualization competition
10 0.054541245 2245 andrew gelman stats-2014-03-12-More on publishing in journals
11 0.054045543 1171 andrew gelman stats-2012-02-16-“False-positive psychology”
12 0.053161744 8 andrew gelman stats-2010-04-28-Advice to help the rich get richer
13 0.052199781 980 andrew gelman stats-2011-10-29-When people meet this guy, can they resist the temptation to ask him what he’s doing for breakfast??
15 0.051046871 719 andrew gelman stats-2011-05-19-Everything is Obvious (once you know the answer)
16 0.050392553 1255 andrew gelman stats-2012-04-10-Amtrak sucks
18 0.04992808 1076 andrew gelman stats-2011-12-21-Derman, Rodrik and the nature of statistical models
19 0.049729802 1479 andrew gelman stats-2012-09-01-Mothers and Moms
20 0.048775245 157 andrew gelman stats-2010-07-21-Roller coasters, charity, profit, hmmm
topicId topicWeight
[(0, 0.107), (1, -0.04), (2, -0.005), (3, -0.016), (4, 0.004), (5, -0.011), (6, 0.021), (7, -0.02), (8, 0.01), (9, -0.02), (10, 0.009), (11, 0.02), (12, 0.027), (13, 0.017), (14, -0.0), (15, -0.005), (16, -0.004), (17, -0.003), (18, 0.01), (19, 0.002), (20, 0.005), (21, 0.005), (22, -0.008), (23, 0.005), (24, -0.011), (25, 0.005), (26, 0.005), (27, -0.001), (28, 0.013), (29, 0.021), (30, -0.005), (31, 0.005), (32, 0.005), (33, -0.018), (34, 0.019), (35, -0.024), (36, -0.012), (37, -0.0), (38, -0.025), (39, -0.005), (40, 0.027), (41, -0.0), (42, -0.012), (43, -0.026), (44, -0.002), (45, -0.007), (46, 0.004), (47, 0.004), (48, 0.016), (49, -0.039)]
simIndex simValue blogId blogTitle
Introduction: Please answer the above question before reading on . . . I’m curious after reading Leif Nelson’s report that, based on research with Minah Jung, approximately 42% of the people they surveyed said they bought laundry detergent on their most recent trip to the store. I’m stunned that the number is so high. 42%??? That’s almost half the time. If we bought laundry detergent half the time we went to the store, our apartment would be stacked so full with the stuff, we wouldn’t be able to enter the door. I think we buy laundry detergent . . . ummm, how often? There are 40 of those little laundry packets in the box, we do laundry once a day, sometimes twice, let’s say 10 times a week, so this means we buy detergent about once every 4 weeks. We go to the store, hmmm, about once a day, let’s say 5 times a week to put our guess on the conservative side. So, 20 trips to the store for each purchase of detergent, that’s 5% of the time. Compared to us, lots of people must (a) go to
Introduction: I had a brief email exchange with Jeff Leek regarding our recent discussions of replication, criticism, and the self-correcting process of science. Jeff writes: (1) I can see the problem with serious, evidence-based criticisms not being published in the same journal (and linked to) studies that are shown to be incorrect. I have been mostly seeing these sorts of things show up in blogs. But I’m not sure that is a bad thing. I think people read blogs more than they read the literature. I wonder if this means that blogs will eventually be a sort of “shadow literature”? (2) I think there is a ton of bad literature out there, just like there is a ton of bad stuff on Google. If we focus too much on the bad stuff we will be paralyzed. I still manage to find good papers despite all the bad papers. (3) I think one positive solution to this problem is to incentivize/publish referee reports and give people credit for a good referee report just like they get credit for a good paper. T
3 0.80008852 514 andrew gelman stats-2011-01-13-News coverage of statistical issues…how did I do?
Introduction: This post is by Phil Price. A reporter once told me that the worst-kept secret of journalism is that every story has errors. And it’s true that just about every time I know about something first-hand, the news stories about it have some mistakes. Reporters aren’t subject-matter experts, they have limited time, and they generally can’t keep revisiting the things they are saying and checking them for accuracy. Many of us have published papers with errors — my most recent paper has an incorrect figure — and that’s after working on them carefully for weeks! One way that reporters can try to get things right is by quoting experts. Even then, there are problems with taking quotes out of context, or with making poor choices about what material to include or exclude, or, of course, with making a poor selection of experts. Yesterday, I was interviewed by an NPR reporter about the risks of breathing radon (a naturally occurring radioactive gas): who should test for it, how dangerous
4 0.77635348 1031 andrew gelman stats-2011-11-27-Richard Stallman and John McCarthy
Introduction: After blogging on quirky software pioneer Richard Stallman , I thought it appropriate to write something about recently deceased quirky software pioneer John McCarthy, who, with the exception of being bearded, seems like he was the personal and political opposite of Stallman. Here’s a page I found of Stallman McCarthy quotes (compiled by Neil Craig). It’s a mixture of the reasonable and the unreasonable (ok, I suppose the same could be said of this blog!). I wonder if he and Stallman ever met and, if so, whether they had an extended conversation. It would be like matter and anti-matter! P.S. I flipped through McCarthy’s pages and found one of my pet peeves. See item 3 here , which sounds so plausible but is in fact not true (at least, not according to the National Election Study). As McCarthy’s Stanford colleague Mo Fiorina can tell you, otherwise well-informed people believe all sorts of things about polarization that aren’t so. Labeling groups of Americans as “
5 0.77264738 2337 andrew gelman stats-2014-05-18-Never back down: The culture of poverty and the culture of journalism
Introduction: Ta-Nehisi Coates recently published a fascinating column on the “culture of poverty,” in particular focusing on the idea that behavior that is rational and adaptive in some settings is not so appropriate in others: The set of practices required for a young man to secure his safety on the streets of his troubled neighborhood are not the same as those required to place him on an honor roll . . . The way to guide him through this transition is not to insult his native language. . . . For black men like us, the feeling of having something to lose, beyond honor and face, is foreign. We grew up in communities—New York, Baltimore, Chicago—where the Code of the Streets was the first code we learned. Respect and reputation are everything there. These values are often denigrated by people who have never been punched in the face. But when you live around violence there is no opting out. A reputation for meeting violence with violence is a shield. That protection increases when you are part
7 0.77029157 1369 andrew gelman stats-2012-06-06-Your conclusion is only as good as your data
10 0.75582117 2287 andrew gelman stats-2014-04-09-Advice: positive-sum, zero-sum, or negative-sum
11 0.75484329 1561 andrew gelman stats-2012-11-04-Someone is wrong on the internet
12 0.7542432 278 andrew gelman stats-2010-09-15-Advice that might make sense for individuals but is negative-sum overall
14 0.75239736 1933 andrew gelman stats-2013-07-10-Please send all comments to -dev-ripley
15 0.74841499 2137 andrew gelman stats-2013-12-17-Replication backlash
16 0.74730349 641 andrew gelman stats-2011-04-01-So many topics, so little time
17 0.74557567 189 andrew gelman stats-2010-08-06-Proposal for a moratorium on the use of the words “fashionable” and “trendy”
18 0.74442077 2053 andrew gelman stats-2013-10-06-Ideas that spread fast and slow
19 0.7428019 2158 andrew gelman stats-2014-01-03-Booze: Been There. Done That.
20 0.74146038 2241 andrew gelman stats-2014-03-10-Preregistration: what’s in it for you?
topicId topicWeight
[(2, 0.021), (5, 0.028), (9, 0.012), (15, 0.013), (16, 0.07), (17, 0.015), (21, 0.024), (24, 0.319), (27, 0.021), (39, 0.027), (68, 0.015), (76, 0.024), (89, 0.014), (91, 0.018), (95, 0.013), (99, 0.196)]
simIndex simValue blogId blogTitle
1 0.9794144 1479 andrew gelman stats-2012-09-01-Mothers and Moms
Introduction: Philip Cohen asks , “Why are mothers becoming moms?” These aren’t just two words for the same thing: in political terms “mother” is merely descriptive while “mom” is more positive. Indeed, we speak of “mom and apple pie” as unquestionable American icons. Cohen points out that motherhood is sometimes but not always respected in political discourse: On the one hand, both President Obama and pundit Hilary Rosen have now called motherhood the world’s hardest job. And with the Romneys flopping onto the all-mothers-work bandwagon, it appears we’re reaching a rare rhetorical consensus. On the other hand, the majority in both major political parties agrees that poor single mothers and their children need one thing above all – a (real) job, one that provides the “dignity of an honest day’s work.” For welfare purposes, taking care of children is not only not the toughest job in the world, it is more akin to nothing at all. When Bill Clinton’s endorsed welfare-to-work he famously decla
2 0.97853804 1092 andrew gelman stats-2011-12-29-More by Berger and me on weakly informative priors
Introduction: A couple days ago we discussed some remarks by Tony O’Hagan and Jim Berger on weakly informative priors. Jim followed up on Deborah Mayo’s blog with this: Objective Bayesian priors are often improper (i.e., have infinite total mass), but this is not a problem when they are developed correctly. But not every improper prior is satisfactory. For instance, the constant prior is known to be unsatisfactory in many situations. The ‘solution’ pseudo-Bayesians often use is to choose a constant prior over a large but bounded set (a ‘weakly informative’ prior), saying it is now proper and so all is well. This is not true; if the constant prior on the whole parameter space is bad, so will be the constant prior over the bounded set. The problem is, in part, that some people confuse proper priors with subjective priors and, having learned that true subjective priors are fine, incorrectly presume that weakly informative proper priors are fine. I have a few reactions to this: 1. I agree
3 0.97816443 482 andrew gelman stats-2010-12-23-Capitalism as a form of voluntarism
Introduction: Interesting discussion by Alex Tabarrok (following up on an article by Rebecca Solnit) on the continuum between voluntarism (or, more generally, non-cash transactions) and markets with monetary exchange. I just have a few comments of my own: 1. Solnit writes of “the iceberg economy,” which she characterizes as “based on gift economies, barter, mutual aid, and giving without hope of return . . . the relations between friends, between family members, the activities of volunteers or those who have chosen their vocation on principle rather than for profit.” I just wonder whether “barter” completely fits in here. Maybe it depends on context. Sometimes barter is an informal way of keeping track (you help me and I help you), but in settings of low liquidity I could imagine barter being simply an inefficient way of performing an economic transaction. 2. I am no expert on capitalism but my impression is that it’s not just about “competition and selfishness” but also is related to the
Introduction: Justin Kinney writes: Since your blog has discussed the “maximal information coefficient” (MIC) of Reshef et al., I figured you might want to see the critique that Gurinder Atwal and I have posted. In short, Reshef et al.’s central claim that MIC is “equitable” is incorrect. We [Kinney and Atwal] offer mathematical proof that the definition of “equitability” Reshef et al. propose is unsatisfiable—no nontrivial dependence measure, including MIC, has this property. Replicating the simulations in their paper with modestly larger data sets validates this finding. The heuristic notion of equitability, however, can be formalized instead as a self-consistency condition closely related to the Data Processing Inequality. Mutual information satisfies this new definition of equitability but MIC does not. We therefore propose that simply estimating mutual information will, in many cases, provide the sort of dependence measure Reshef et al. seek. For background, here are my two p
5 0.97667956 938 andrew gelman stats-2011-10-03-Comparing prediction errors
Introduction: Someone named James writes: I’m working on a classification task, sentence segmentation. The classifier algorithm we use (BoosTexter, a boosted learning algorithm) classifies each word independently conditional on its features, i.e. a bag-of-words model, so any contextual clues need to be encoded into the features. The feature extraction system I am proposing in my thesis uses a heteroscedastic LDA to transform data to produce the features the classifier runs on. The HLDA system has a couple parameters I’m testing, and I’m running a 3×2 full factorial experiment. That’s the background which may or may not be relevant to the question. The output of each trial is a class (there are only 2 classes, right now) for every word in the dataset. Because of the nature of the task, one class strongly predominates, say 90-95% of the data. My question is this: in terms of overall performance (we use F1 score), many of these trials are pretty close together, which leads me to ask whethe
6 0.97494501 743 andrew gelman stats-2011-06-03-An argument that can’t possibly make sense
7 0.97409469 1787 andrew gelman stats-2013-04-04-Wanna be the next Tyler Cowen? It’s not as easy as you might think!
8 0.97236931 1376 andrew gelman stats-2012-06-12-Simple graph WIN: the example of birthday frequencies
9 0.97205925 241 andrew gelman stats-2010-08-29-Ethics and statistics in development research
10 0.97109503 1978 andrew gelman stats-2013-08-12-Fixing the race, ethnicity, and national origin questions on the U.S. Census
11 0.96868002 38 andrew gelman stats-2010-05-18-Breastfeeding, infant hyperbilirubinemia, statistical graphics, and modern medicine
12 0.96781933 643 andrew gelman stats-2011-04-02-So-called Bayesian hypothesis testing is just as bad as regular hypothesis testing
13 0.96712327 1455 andrew gelman stats-2012-08-12-Probabilistic screening to get an approximate self-weighted sample
14 0.96691513 278 andrew gelman stats-2010-09-15-Advice that might make sense for individuals but is negative-sum overall
16 0.96331322 2143 andrew gelman stats-2013-12-22-The kluges of today are the textbook solutions of tomorrow.
17 0.9607144 1999 andrew gelman stats-2013-08-27-Bayesian model averaging or fitting a larger model
18 0.95732993 197 andrew gelman stats-2010-08-10-The last great essayist?
19 0.95489484 1584 andrew gelman stats-2012-11-19-Tradeoffs in information graphics
20 0.95443296 2229 andrew gelman stats-2014-02-28-God-leaf-tree