andrew_gelman_stats andrew_gelman_stats-2010 andrew_gelman_stats-2010-27 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: A few days ago I reported on the spam email that I received from two business school professors (one at Columbia)! As noted on the blog, I sent an email directly to the study’s authors at the time of reading the email, but they have yet to respond. This surprises me a bit. Certainly if 6300 faculty each have time to respond to one email on this study, the two faculty have time to respond to 6300 email replies, no? I was actually polite enough to respond to both of their emails! If I do hear back, I’ll let youall know! P.S. Paul Basken interviewed me briefly for a story in the Chronicle of Higher Education on the now-notorious spam email study. Basken’s article is reasonable–he points out that (a) the study irritated a lot of people, but (b) is ultimately no big deal. One interesting thing about the article is that, although some people felt that the spam email study was ethical, nobody came forth with an argument that the study was actually worth doing. P.P.S. In
sentIndex sentText sentNum sentScore
1 A few days ago I reported on the spam email that I received from two business school professors (one at Columbia)! [sent-1, score-1.181]
2 As noted on the blog, I sent an email directly to the study’s authors at the time of reading the email, but they have yet to respond. [sent-2, score-0.61]
3 Certainly if 6300 faculty each have time to respond to one email on this study, the two faculty have time to respond to 6300 email replies, no? [sent-4, score-1.83]
4 I was actually polite enough to respond to both of their emails! [sent-5, score-0.383]
5 Paul Basken interviewed me briefly for a story in the Chronicle of Higher Education on the now-notorious spam email study. [sent-9, score-1.032]
6 Basken’s article is reasonable–he points out that (a) the study irritated a lot of people, but (b) is ultimately no big deal. [sent-10, score-0.61]
7 One interesting thing about the article is that, although some people felt that the spam email study was ethical, nobody came forth with an argument that the study was actually worth doing. [sent-11, score-1.533]
8 Heck, I’m a full professor of statistics but I published a false theorem once. [sent-19, score-0.272]
9 And the notorious Frank Flynn is a professor of usiness at Stanford, so I’m sure he’s done a lot of great stuff. [sent-20, score-0.294]
10 I also imagine the designers of the spam email survey have a lot of good ideas in them: they’re young researchers and they made a silly mistake, it’s ultimately no big deal. [sent-21, score-1.456]
wordName wordTfidf (topN-words)
[('email', 0.475), ('spam', 0.375), ('basken', 0.317), ('respond', 0.212), ('study', 0.201), ('faculty', 0.157), ('youall', 0.13), ('ultimately', 0.13), ('chronicle', 0.126), ('heck', 0.126), ('surprises', 0.126), ('professor', 0.123), ('flynn', 0.122), ('polite', 0.108), ('irritated', 0.106), ('designers', 0.102), ('seriousness', 0.102), ('replies', 0.099), ('interviewed', 0.098), ('stanford', 0.098), ('lot', 0.096), ('emails', 0.094), ('ethical', 0.089), ('frank', 0.087), ('forth', 0.086), ('briefly', 0.084), ('professors', 0.083), ('theorem', 0.082), ('young', 0.079), ('big', 0.077), ('notorious', 0.075), ('time', 0.071), ('paul', 0.07), ('felt', 0.068), ('false', 0.067), ('mistake', 0.066), ('hear', 0.065), ('columbia', 0.065), ('silly', 0.065), ('nobody', 0.064), ('noted', 0.064), ('reported', 0.063), ('actually', 0.063), ('days', 0.063), ('deal', 0.061), ('received', 0.061), ('education', 0.061), ('business', 0.061), ('everyone', 0.058), ('imagine', 0.057)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 27 andrew gelman stats-2010-05-11-Update on the spam email study
Introduction: A few days ago I reported on the spam email that I received from two business school professors (one at Columbia)! As noted on the blog, I sent an email directly to the study’s authors at the time of reading the email, but they have yet to respond. This surprises me a bit. Certainly if 6300 faculty each have time to respond to one email on this study, the two faculty have time to respond to 6300 email replies, no? I was actually polite enough to respond to both of their emails! If I do hear back, I’ll let youall know! P.S. Paul Basken interviewed me briefly for a story in the Chronicle of Higher Education on the now-notorious spam email study. Basken’s article is reasonable–he points out that (a) the study irritated a lot of people, but (b) is ultimately no big deal. One interesting thing about the article is that, although some people felt that the spam email study was ethical, nobody came forth with an argument that the study was actually worth doing. P.P.S. In
2 0.33171332 503 andrew gelman stats-2011-01-04-Clarity on my email policy
Introduction: I never read email before 4. That doesn’t mean I never send email before 4.
3 0.28402677 18 andrew gelman stats-2010-05-06-$63,000 worth of abusive research . . . or just a really stupid waste of time?
Introduction: As someone who relies strongly on survey research, it’s good for me to be reminded that some surveys are useful, some are useless, but one thing they almost all have in common is . . . they waste the respondents’ time. I thought of this after receiving the following email, which I shall reproduce here. My own comments appear after. Recently, you received an email from a student asking for 10 minutes of your time to discuss your Ph.D. program (the body of the email appears below). We are emailing you today to debrief you on the actual purpose of that email, as it was part of a research study. We sincerely hope our study did not cause you any disruption and we apologize if you were at all inconvenienced. Our hope is that this letter will provide a sufficient explanation of the purpose and design of our study to alleviate any concerns you may have about your involvement. We want to thank you for your time and for reading further if you are interested in understanding why you rece
4 0.27850839 425 andrew gelman stats-2010-11-21-If your comment didn’t get through . . .
Introduction: It probably got caught in the spam filter. We get tons and tons of spam (including the annoying spam that I have to remove by hand). If your comment was accompanied by an ad or a spam link, then maybe I just deleted it.
5 0.26948658 619 andrew gelman stats-2011-03-19-If a comment is flagged as spam, it will disappear forever
Introduction: A commenter wrote (by email): I’ve noticed that you’ve quit approving my comments on your blog. I hope I didn’t anger you in some way or write something you felt was inappropriate. My reply: I have not been unapproving any comments. If you have comments that have not appeared, they have probably been going into the spam filter. I get literally thousands of spam comments a day and so anything that hits the spam filter is gone forever. I think there is a way to register as a commenter; that could help.
6 0.24211589 1488 andrew gelman stats-2012-09-08-Annals of spam
7 0.22546299 132 andrew gelman stats-2010-07-07-Note to “Cigarettes”
8 0.1902055 35 andrew gelman stats-2010-05-16-Another update on the spam email study
9 0.18782334 259 andrew gelman stats-2010-09-06-Inbox zero. Really.
10 0.18247218 605 andrew gelman stats-2011-03-09-Does it feel like cheating when I do this? Variation in ethical standards and expectations
12 0.15647367 2160 andrew gelman stats-2014-01-06-Spam names
13 0.15530631 839 andrew gelman stats-2011-08-04-To commenters who are trying to sell something
14 0.15390791 530 andrew gelman stats-2011-01-22-MS-Bayes?
15 0.14900102 817 andrew gelman stats-2011-07-23-New blog home
16 0.14579698 771 andrew gelman stats-2011-06-16-30 days of statistics
17 0.14568292 545 andrew gelman stats-2011-01-30-New innovations in spam
18 0.14080882 523 andrew gelman stats-2011-01-18-Spam is out of control
19 0.13051552 1841 andrew gelman stats-2013-05-04-The Folk Theorem of Statistical Computing
20 0.12453035 2111 andrew gelman stats-2013-11-23-Tables > figures yet again
topicId topicWeight
[(0, 0.151), (1, -0.099), (2, -0.047), (3, -0.042), (4, 0.028), (5, 0.047), (6, 0.073), (7, -0.033), (8, -0.017), (9, -0.055), (10, 0.02), (11, -0.039), (12, 0.151), (13, 0.045), (14, -0.059), (15, 0.106), (16, 0.041), (17, -0.107), (18, -0.031), (19, 0.072), (20, 0.053), (21, -0.087), (22, 0.015), (23, -0.176), (24, 0.014), (25, -0.022), (26, 0.055), (27, 0.073), (28, -0.072), (29, -0.008), (30, -0.055), (31, 0.096), (32, -0.084), (33, 0.051), (34, -0.084), (35, 0.067), (36, 0.012), (37, -0.005), (38, -0.031), (39, -0.022), (40, 0.003), (41, 0.13), (42, -0.056), (43, -0.084), (44, 0.061), (45, -0.133), (46, 0.076), (47, -0.017), (48, 0.082), (49, -0.105)]
simIndex simValue blogId blogTitle
same-blog 1 0.96130288 27 andrew gelman stats-2010-05-11-Update on the spam email study
Introduction: A few days ago I reported on the spam email that I received from two business school professors (one at Columbia)! As noted on the blog, I sent an email directly to the study’s authors at the time of reading the email, but they have yet to respond. This surprises me a bit. Certainly if 6300 faculty each have time to respond to one email on this study, the two faculty have time to respond to 6300 email replies, no? I was actually polite enough to respond to both of their emails! If I do hear back, I’ll let youall know! P.S. Paul Basken interviewed me briefly for a story in the Chronicle of Higher Education on the now-notorious spam email study. Basken’s article is reasonable–he points out that (a) the study irritated a lot of people, but (b) is ultimately no big deal. One interesting thing about the article is that, although some people felt that the spam email study was ethical, nobody came forth with an argument that the study was actually worth doing. P.P.S. In
2 0.80125356 619 andrew gelman stats-2011-03-19-If a comment is flagged as spam, it will disappear forever
Introduction: A commenter wrote (by email): I’ve noticed that you’ve quit approving my comments on your blog. I hope I didn’t anger you in some way or write something you felt was inappropriate. My reply: I have not been unapproving any comments. If you have comments that have not appeared, they have probably been going into the spam filter. I get literally thousands of spam comments a day and so anything that hits the spam filter is gone forever. I think there is a way to register as a commenter; that could help.
3 0.78005296 1488 andrew gelman stats-2012-09-08-Annals of spam
Introduction: I have to go through the inbox to approve new comments. When I set to auto-approve, I get overwhelmed with spam. As is, I still get spam but it’s manageable. Usually the spam is uninteresting but this one caught my eye: At first this seemed reasonable enough: law firm is desperate for business, spams blogs to raise its Google ranking. But what’s with the writing in the actual comment? It’s incoherent but it doesn’t look computer-generated. My guess is that the law firm in Massachusetts hired a company that promised to raise their Google rankings, and that this company hired some non-English-speaking foreigners to search through the web and write some spam comments. If anyone actually reads the comments, they might get the impression that this law firm is staffed by illiterates . . . but, as we all know, nobody reads blog comments! P.S. I followed the link (sorry!) and came across this: I guess if they’re going to use a tragedy as an excuse to troll for Faceb
4 0.74909025 503 andrew gelman stats-2011-01-04-Clarity on my email policy
Introduction: I never read email before 4. That doesn’t mean I never send email before 4.
5 0.7301532 425 andrew gelman stats-2010-11-21-If your comment didn’t get through . . .
Introduction: It probably got caught in the spam filter. We get tons and tons of spam (including the annoying spam that I have to remove by hand). If your comment was accompanied by an ad or a spam link, then maybe I just deleted it.
6 0.72198534 523 andrew gelman stats-2011-01-18-Spam is out of control
7 0.7023074 2160 andrew gelman stats-2014-01-06-Spam names
8 0.6992293 132 andrew gelman stats-2010-07-07-Note to “Cigarettes”
9 0.6899963 545 andrew gelman stats-2011-01-30-New innovations in spam
10 0.67903179 18 andrew gelman stats-2010-05-06-$63,000 worth of abusive research . . . or just a really stupid waste of time?
11 0.6779207 839 andrew gelman stats-2011-08-04-To commenters who are trying to sell something
12 0.64974254 530 andrew gelman stats-2011-01-22-MS-Bayes?
13 0.64529639 605 andrew gelman stats-2011-03-09-Does it feel like cheating when I do this? Variation in ethical standards and expectations
14 0.62942129 1573 andrew gelman stats-2012-11-11-Incredibly strange spam
16 0.61350536 817 andrew gelman stats-2011-07-23-New blog home
17 0.61237717 332 andrew gelman stats-2010-10-10-Proposed new section of the American Statistical Association on Imaging Sciences
18 0.60701799 259 andrew gelman stats-2010-09-06-Inbox zero. Really.
19 0.57741284 1589 andrew gelman stats-2012-11-25-Life as a blogger: the emails just get weirder and weirder
20 0.5772627 35 andrew gelman stats-2010-05-16-Another update on the spam email study
topicId topicWeight
[(1, 0.016), (2, 0.174), (5, 0.015), (15, 0.024), (16, 0.053), (21, 0.044), (24, 0.148), (25, 0.014), (45, 0.017), (57, 0.011), (63, 0.013), (98, 0.047), (99, 0.312)]
simIndex simValue blogId blogTitle
1 0.98111391 1663 andrew gelman stats-2013-01-09-The effects of fiscal consolidation
Introduction: José Iparraguirre writes: I’ve read a recent paper by the International Monetary Fund on the effects of fiscal consolidation measures on income inequality (Fiscal Monitor October 2012, Appendix 1). They run a panel regression with 48 countries and 30 years (annual data) of a measure of income inequality (Gini coefficient) on a number of covariates, including a measure of fiscal consolidation. Footnote 39 (page 51) informs that they’ve employed seemingly unrelated regression and panel-corrected standard errors, and that to double-check they’ve also run ordinary least squares and fixed-effects panel regressions—all with similar results. So far, so good. However, the footnote goes on to explain that “Some of the results (e.g. the causal relationship between consolidation and inequality) may be subject to endogeneity and should be interpreted with caution”. (Italics are mine). Therefore, it seems that the crux of the exercise—i.e. estimating the relationship between fiscal con
2 0.97664887 17 andrew gelman stats-2010-05-05-Taking philosophical arguments literally
Introduction: Aaron Swartz writes the following, as a lead-in to an argument in favor of vegetarianism: Imagine you were an early settler of what is now the United States. It seems likely you would have killed native Americans. After all, your parents killed them, your siblings killed them, your friends killed them, the leaders of the community killed them, the President killed them. Chances are, you would have killed them too . . . Or if you see nothing wrong with killing native Americans, take the example of slavery. Again, everyone had slaves and probably didn’t think too much about the morality of it. . . . Are these statements true, though? It’s hard for me to believe that most early settlers (from the context, it looks like Swartz is discussing the 1500s-1700s here) killed native Americans. That is, if N is the number of early settlers, and Y is the number of these settlers who killed at least one Indian, I suspect Y/N is much closer to 0 than to 1. Similarly, it’s not even cl
3 0.97580177 549 andrew gelman stats-2011-02-01-“Roughly 90% of the increase in . . .” Hey, wait a minute!
Introduction: Matthew Yglesias links approvingly to the following statement by Michael Mandel: Homeland Security accounts for roughly 90% of the increase in federal regulatory employment over the past ten years. Roughly 90%, huh? That sounds pretty impressive. But wait a minute . . . what if total federal regulatory employment had increased a bit less. Then Homeland Security could’ve accounted for 105% of the increase, or 500% of the increase, or whatever. The point is the change in total employment is the sum of a bunch of pluses and minuses. It happens that, if you don’t count Homeland Security, the total hasn’t changed much–I’m assuming Mandel’s numbers are correct here–and that could be interesting. The “roughly 90%” figure is misleading because, when written as a percent of the total increase, it’s natural to quickly envision it as a percentage that is bounded by 100%. There is a total increase in regulatory employment that the individual agencies sum to, but some margins are p
4 0.97564465 1189 andrew gelman stats-2012-02-28-Those darn physicists
Introduction: X pointed me to this atrocity: The data on obesity are pretty unequivocal: we’re fat, and we’re getting fatter. Explanations for this trend, however, vary widely, with the blame alternately pinned on individual behaviour, genetics and the environment. In other words, it’s a race between “we eat too much”, “we’re born that way” and “it’s society’s fault”. Now, research by Lazaros Gallos has come down strongly in favour of the third option. Gallos and his colleagues at City College of New York treated the obesity rates in some 3000 US counties as “particles” in a physical system, and calculated the correlation between pairs of “particles” as a function of the distance between them. . . . the data indicated that the size of the “obesity cities” – geographic regions with correlated obesity rates – was huge, up to 1000 km. . . . Just to be clear: I have no problem with people calculating spatial autocorrelations (or even with them using quaint terminology such as referring to coun
5 0.97270739 1017 andrew gelman stats-2011-11-18-Lack of complete overlap
Introduction: Evens Salies writes: I have a question regarding a randomizing constraint in my current funded electricity experiment. After elimination of missing data we have 110 voluntary households from a larger population (resource constraints do not allow us to have more households!). I randomly assign them to threated and non treated where the treatment variable is some ICT that allows the treated to track their electricity consumption in real tim. The ICT is made of two devices, one that is plugged on the household’s modem and the other on the electric meter. A necessary condition for being treated is that the distance between the box and the meter be below some threshold (d), the value of which is 20 meters approximately. 50 ICTs can be installed. 60 households will be in the control group. But, I can only assign 6 households in the control group for whom d is less than 20. Therefore, I have only 6 households in the control group who have a counterfactual in the group of treated.
6 0.97091258 1508 andrew gelman stats-2012-09-23-Speaking frankly
7 0.96707821 1893 andrew gelman stats-2013-06-11-Folic acid and autism
8 0.96642715 1872 andrew gelman stats-2013-05-27-More spam!
9 0.96024472 1254 andrew gelman stats-2012-04-09-In the future, everyone will publish everything.
10 0.96001947 97 andrew gelman stats-2010-06-18-Economic Disparities and Life Satisfaction in European Regions
12 0.95754933 1954 andrew gelman stats-2013-07-24-Too Good To Be True: The Scientific Mass Production of Spurious Statistical Significance
same-blog 13 0.95177412 27 andrew gelman stats-2010-05-11-Update on the spam email study
14 0.95104754 1196 andrew gelman stats-2012-03-04-Piss-poor monocausal social science
15 0.94853246 1171 andrew gelman stats-2012-02-16-“False-positive psychology”
16 0.94770169 489 andrew gelman stats-2010-12-28-Brow inflation
17 0.94627404 1567 andrew gelman stats-2012-11-07-Election reports
18 0.94449544 1698 andrew gelman stats-2013-01-30-The spam just gets weirder and weirder
19 0.93767822 2336 andrew gelman stats-2014-05-16-How much can we learn about individual-level causal claims from state-level correlations?
20 0.93405753 1260 andrew gelman stats-2012-04-11-Hunger Games survival analysis