andrew_gelman_stats andrew_gelman_stats-2010 andrew_gelman_stats-2010-18 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: As someone who relies strongly on survey research, it’s good for me to be reminded that some surveys are useful, some are useless, but one thing they almost all have in common is . . . they waste the respondents’ time. I thought of this after receiving the following email, which I shall reproduce here. My own comments appear after. Recently, you received an email from a student asking for 10 minutes of your time to discuss your Ph.D. program (the body of the email appears below). We are emailing you today to debrief you on the actual purpose of that email, as it was part of a research study. We sincerely hope our study did not cause you any disruption and we apologize if you were at all inconvenienced. Our hope is that this letter will provide a sufficient explanation of the purpose and design of our study to alleviate any concerns you may have about your involvement. We want to thank you for your time and for reading further if you are interested in understanding why you rece
sentIndex sentText sentNum sentScore
1 Recently, you received an email from a student asking for 10 minutes of your time to discuss your Ph. [sent-7, score-0.584]
2 We sincerely hope our study did not cause you any disruption and we apologize if you were at all inconvenienced. [sent-11, score-0.369]
3 Our hope is that this letter will provide a sufficient explanation of the purpose and design of our study to alleviate any concerns you may have about your involvement. [sent-12, score-0.283]
4 We want to thank you for your time and for reading further if you are interested in understanding why you received this message. [sent-13, score-0.28]
5 The email you received from a student asked for a meeting with you either today (if you were randomly assigned to the “now” condition) or in a week (if you were randomly assigned to the “later” condition). [sent-17, score-1.138]
6 It was designed for a study of the responsiveness of University faculty to meeting requests from prospective students of various backgrounds made on short notice versus well in advance. [sent-19, score-1.15]
7 programs were identified as potential participants in this study, and a random sample (6,300 faculty in total – one per Ph. [sent-27, score-0.233]
8 We expected that students from underrepresented groups would receive fewer meeting acceptances than other students, though we have competing hypotheses about whether this would effect would be stronger in the “now” or the “later” condition. [sent-31, score-0.408]
9 The email you received as a part of this study contained the following message: “I am writing you because I am a prospective Ph. [sent-32, score-0.662]
10 programs this coming fall, and I am eager to learn as much as I can about research opportunities in the meantime. [sent-37, score-0.285]
11 I will be on campus today/(next Monday), and although I know it is short notice, I was wondering if you might have 10 minutes when you would be willing to meet with me to briefly talk about your work and any possible opportunities for me to get involved in your research. [sent-38, score-0.233]
12 Any time that would be convenient for you would be fine with me, as meeting with you is my first priority during this campus visit. [sent-39, score-0.353]
13 Please rest assured that no identifiable data will ever be reported from this study, and our between subject design ensures that we will only be able to identify email responsiveness patterns in aggregate – not at the individual level. [sent-42, score-0.759]
14 No individual or university will be identifiable in any of the research or data we publish. [sent-43, score-0.573]
15 Of course, any one individual email response is not meaningful as there are multiple reasons why an individual faculty member might accept or decline a meeting request. [sent-44, score-0.916]
16 All data has already been de-identified and the identifiable email responses have already been deleted from our databases and related server. [sent-45, score-0.514]
17 And as is always the case when academics conduct research involving human subjects, our research protocols were approved by our universities’ ! [sent-47, score-0.284]
18 If you have any questions about your rights as a research subject, you may contact the Columbia University Morningside Institutional Review Board at 212-851-7040 or by email at askirb@columbia. [sent-49, score-0.463]
19 My understanding is that standard ethical guidelines require that research subjects (which, in this case, include me) be compensated for their time. [sent-55, score-0.382]
20 Some might say that it is mean of me to send such a sarcastic email to two evidently serious researchers. [sent-62, score-0.321]
wordName wordTfidf (topN-words)
[('email', 0.321), ('meeting', 0.256), ('identifiable', 0.193), ('milkman', 0.18), ('faculty', 0.163), ('responsiveness', 0.157), ('study', 0.151), ('university', 0.15), ('pennsylvania', 0.145), ('research', 0.142), ('irb', 0.142), ('institutional', 0.123), ('akinola', 0.12), ('later', 0.116), ('morningside', 0.113), ('thank', 0.113), ('ethical', 0.11), ('columbia', 0.109), ('versus', 0.105), ('received', 0.102), ('student', 0.098), ('campus', 0.097), ('review', 0.089), ('prospective', 0.088), ('sincerely', 0.088), ('individual', 0.088), ('requests', 0.087), ('students', 0.08), ('assistant', 0.079), ('assigned', 0.078), ('condition', 0.075), ('opportunities', 0.073), ('management', 0.072), ('receive', 0.072), ('universities', 0.072), ('board', 0.071), ('hope', 0.07), ('programs', 0.07), ('school', 0.069), ('randomly', 0.069), ('asked', 0.067), ('understanding', 0.065), ('subjects', 0.065), ('notice', 0.063), ('survey', 0.063), ('minutes', 0.063), ('purpose', 0.062), ('affiliated', 0.06), ('bazerman', 0.06), ('disruption', 0.06)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 18 andrew gelman stats-2010-05-06-$63,000 worth of abusive research . . . or just a really stupid waste of time?
Introduction: As someone who relies strongly on survey research, it’s good for me to be reminded that some surveys are useful, some are useless, but one thing they almost all have in common is . . . they waste the respondents’ time. I thought of this after receiving the following email, which I shall reproduce here. My own comments appear after. Recently, you received an email from a student asking for 10 minutes of your time to discuss your Ph.D. program (the body of the email appears below). We are emailing you today to debrief you on the actual purpose of that email, as it was part of a research study. We sincerely hope our study did not cause you any disruption and we apologize if you were at all inconvenienced. Our hope is that this letter will provide a sufficient explanation of the purpose and design of our study to alleviate any concerns you may have about your involvement. We want to thank you for your time and for reading further if you are interested in understanding why you rece
2 0.32705829 35 andrew gelman stats-2010-05-16-Another update on the spam email study
Introduction: I think youall are probably getting sick of this by now so I’ll put it all below the fold. Akinola Modupe and Katherine Milkman responded to my email about their study : We want to clarify the reason we believe that the use of deception and a lack of informed consent were appropriate and ethical for this research study. In this project, we were studying how the timing of a decision affects discrimination based on race and/or gender. The emails all participants in our study received were identical except for a) the sender’s name (we used 20 names that pretesting revealed were strongly associated with being either Caucasian, Black, Indian, Chinese or Hispanic, as well as associated with being male or female) and b) whether the meeting requested was for today or for a week from today. Recipients were randomly selected and were randomly assigned to one of the race/gender/timing conditions. This study design will allow us to test for baseline levels of discrimination in acade
3 0.28402677 27 andrew gelman stats-2010-05-11-Update on the spam email study
Introduction: A few days ago I reported on the spam email that I received from two business school professors (one at Columbia)! As noted on the blog, I sent an email directly to the study’s authors at the time of reading the email, but they have yet to respond. This surprises me a bit. Certainly if 6300 faculty each have time to respond to one email on this study, the two faculty have time to respond to 6300 email replies, no? I was actually polite enough to respond to both of their emails! If I do hear back, I’ll let youall know! P.S. Paul Basken interviewed me briefly for a story in the Chronicle of Higher Education on the now-notorious spam email study. Basken’s article is reasonable–he points out that (a) the study irritated a lot of people, but (b) is ultimately no big deal. One interesting thing about the article is that, although some people felt that the spam email study was ethical, nobody came forth with an argument that the study was actually worth doing. P.P.S. In
4 0.23883744 503 andrew gelman stats-2011-01-04-Clarity on my email policy
Introduction: I never read email before 4. That doesn’t mean I never send email before 4.
Introduction: John Sides points to this discussion (with over 200 comments!) by political scientist Charli Carpenter of her response to a student from another university who emailed with questions that look like they come from a homework assignment. Here’s the student’s original email : Hi Mr. Carpenter, I am a fourth year college student and I have the honor of reading one of your books and I just had a few questions… I am very fascinated by your work and I am just trying to understand everything. Can you please address some of my questions? I would greatly appreciate it. It certainly help me understand your wonderful article better. Thank you very much! :) 1. What is the fundamental purpose of your article? 2. What is your fundamental thesis? 3. What evidence do you use to support your thesis? 4. What is the overall conclusion? 5. Do you feel that you have a fair balance of opposing viewpoints? Sincerely, After a series of emails in which Carpenter explained why she thought
6 0.19351666 866 andrew gelman stats-2011-08-23-Participate in a research project on combining information for prediction
7 0.17523782 1539 andrew gelman stats-2012-10-18-IRB nightmares
8 0.14454804 1055 andrew gelman stats-2011-12-13-Data sharing update
9 0.14428128 530 andrew gelman stats-2011-01-22-MS-Bayes?
10 0.13691889 571 andrew gelman stats-2011-02-13-A departmental wiki page?
11 0.13366903 259 andrew gelman stats-2010-09-06-Inbox zero. Really.
14 0.1233569 2148 andrew gelman stats-2013-12-25-Spam!
15 0.12010624 1502 andrew gelman stats-2012-09-19-Scalability in education
16 0.11851279 2301 andrew gelman stats-2014-04-22-Ticket to Baaaaarf
17 0.11670571 1053 andrew gelman stats-2011-12-11-This one is so dumb it makes me want to barf
19 0.11533605 2111 andrew gelman stats-2013-11-23-Tables > figures yet again
20 0.11300553 350 andrew gelman stats-2010-10-18-Subtle statistical issues to be debated on TV.
topicId topicWeight
[(0, 0.217), (1, -0.097), (2, -0.035), (3, -0.112), (4, 0.047), (5, 0.147), (6, 0.017), (7, 0.02), (8, -0.122), (9, -0.008), (10, 0.016), (11, -0.046), (12, 0.074), (13, 0.017), (14, -0.051), (15, 0.048), (16, 0.093), (17, -0.085), (18, 0.005), (19, 0.082), (20, -0.006), (21, -0.008), (22, 0.036), (23, -0.098), (24, 0.016), (25, -0.035), (26, 0.053), (27, -0.031), (28, -0.004), (29, 0.016), (30, -0.106), (31, 0.011), (32, -0.079), (33, 0.105), (34, -0.06), (35, -0.036), (36, 0.034), (37, -0.075), (38, -0.005), (39, -0.005), (40, 0.111), (41, 0.031), (42, -0.014), (43, -0.083), (44, 0.036), (45, -0.085), (46, 0.037), (47, -0.057), (48, 0.07), (49, -0.021)]
simIndex simValue blogId blogTitle
same-blog 1 0.98501921 18 andrew gelman stats-2010-05-06-$63,000 worth of abusive research . . . or just a really stupid waste of time?
Introduction: As someone who relies strongly on survey research, it’s good for me to be reminded that some surveys are useful, some are useless, but one thing they almost all have in common is . . . they waste the respondents’ time. I thought of this after receiving the following email, which I shall reproduce here. My own comments appear after. Recently, you received an email from a student asking for 10 minutes of your time to discuss your Ph.D. program (the body of the email appears below). We are emailing you today to debrief you on the actual purpose of that email, as it was part of a research study. We sincerely hope our study did not cause you any disruption and we apologize if you were at all inconvenienced. Our hope is that this letter will provide a sufficient explanation of the purpose and design of our study to alleviate any concerns you may have about your involvement. We want to thank you for your time and for reading further if you are interested in understanding why you rece
2 0.82320255 866 andrew gelman stats-2011-08-23-Participate in a research project on combining information for prediction
Introduction: Thomas Wallsten writes: To viewers of Dr. Andrew Gelman’s blog, I [Wallsten] am pleased to invite you to participate in an important research project to develop improved methods for predicting future events and outcomes. More specifically, our goal is to develop methods for aggregating many individual judgments in a manner that yields more accurate predictions than any one person or small group alone could provide. Our research is funded by the Intelligence Advanced Research Project Activity (IARPA, iarpa.gov ), but its application will extend far beyond intelligence to such areas as business forecasting or medical diagnosis. Our team consists of researchers at ARA, a private company; as well as researchers at the University of Maryland-College Park, University of Michigan, Ohio State University, Fordham University, University of California-Irvine, Wake Forest University, and the University of Missouri. Details can be found at forecastingace.com/ . We are seeking to recruit ind
3 0.81055766 1618 andrew gelman stats-2012-12-11-The consulting biz
Introduction: I received the following (unsolicited) email: Hello, *** LLC, a ***-based market research company, has a financial client who is interested in speaking with a statistician who has done research in the field of Alzheimer’s Disease and preferably familiar with the SOLA and BAPI trials. We offer an honorarium of $200 for a 30 minute telephone interview. Please advise us if you have an employment or consulting agreement with any organization or operate professionally pursuant to an organization’s code of conduct or employee manual that may control activities by you outside of your regular present and former employment, such as participating in this consulting project for MedPanel. If there are such contracts or other documents that do apply to you, please forward MedPanel a copy of each such document asap as we are obligated to review such documents to determine if you are permitted to participate as a consultant for MedPanel on a project with this particular client. If you are
4 0.80430061 35 andrew gelman stats-2010-05-16-Another update on the spam email study
Introduction: I think youall are probably getting sick of this by now so I’ll put it all below the fold. Akinola Modupe and Katherine Milkman responded to my email about their study : We want to clarify the reason we believe that the use of deception and a lack of informed consent were appropriate and ethical for this research study. In this project, we were studying how the timing of a decision affects discrimination based on race and/or gender. The emails all participants in our study received were identical except for a) the sender’s name (we used 20 names that pretesting revealed were strongly associated with being either Caucasian, Black, Indian, Chinese or Hispanic, as well as associated with being male or female) and b) whether the meeting requested was for today or for a week from today. Recipients were randomly selected and were randomly assigned to one of the race/gender/timing conditions. This study design will allow us to test for baseline levels of discrimination in acade
5 0.78394896 2118 andrew gelman stats-2013-11-30-???
Introduction: I received the following unsolicited email, subject line Technology and Engineering Research: Dear Editor We have done research in some of the cutting edge technology and engineering field and would like to if you will be able to write about it in your news section. Our Primarily research focus on building high performance systems that are helping in social networks, web, finding disease, cancer and sports using BIG DATA . Hope to hear from you some time soon. Thanks, ***, PhD Chartered Scientist IBM Corportation ***@us.ibm.com 916 *** **** I thought IBM was a professional operation—don’t they have their own public relations department?
6 0.77508479 27 andrew gelman stats-2010-05-11-Update on the spam email study
8 0.73735505 350 andrew gelman stats-2010-10-18-Subtle statistical issues to be debated on TV.
9 0.71805686 2148 andrew gelman stats-2013-12-25-Spam!
10 0.70374167 343 andrew gelman stats-2010-10-15-?
11 0.69441462 1539 andrew gelman stats-2012-10-18-IRB nightmares
12 0.69008863 882 andrew gelman stats-2011-08-31-Meanwhile, on the sister blog . . .
13 0.67532259 503 andrew gelman stats-2011-01-04-Clarity on my email policy
14 0.67517602 282 andrew gelman stats-2010-09-17-I can’t escape it
15 0.66764343 1191 andrew gelman stats-2012-03-01-Hoe noem je?
16 0.66742736 1922 andrew gelman stats-2013-07-02-They want me to send them free material and pay for the privilege
17 0.64112622 259 andrew gelman stats-2010-09-06-Inbox zero. Really.
18 0.64001787 2239 andrew gelman stats-2014-03-09-Reviewing the peer review process?
19 0.63585931 530 andrew gelman stats-2011-01-22-MS-Bayes?
20 0.63430309 332 andrew gelman stats-2010-10-10-Proposed new section of the American Statistical Association on Imaging Sciences
topicId topicWeight
[(2, 0.021), (9, 0.042), (15, 0.022), (16, 0.089), (21, 0.021), (24, 0.149), (29, 0.02), (30, 0.01), (47, 0.023), (71, 0.016), (73, 0.012), (86, 0.032), (87, 0.011), (97, 0.08), (99, 0.286)]
simIndex simValue blogId blogTitle
Introduction: Earlier today, Nate criticized a U.S. military survey that asks troops the question, “Do you currently serve with a male or female Service member you believe to be homosexual.” [emphasis added] As Nate points out, by asking this question in such a speculative way, “it would seem that you’ll be picking up a tremendous number of false positives–soldiers who are believed to be gay, but aren’t–and that these false positives will swamp any instances in which soldiers (in spite of DADT) are actually somewhat open about their same-sex attractions.” This is a general problem in survey research. In an article in Chance magazine in 1997, “The myth of millions of annual self-defense gun uses: a case study of survey overestimates of rare events” [see here for related references], David Hemenway uses the false-positive, false-negative reasoning to explain this bias in terms of probability theory. Misclassifications that induce seemingly minor biases in estimates of certain small probab
same-blog 2 0.97455096 18 andrew gelman stats-2010-05-06-$63,000 worth of abusive research . . . or just a really stupid waste of time?
Introduction: As someone who relies strongly on survey research, it’s good for me to be reminded that some surveys are useful, some are useless, but one thing they almost all have in common is . . . they waste the respondents’ time. I thought of this after receiving the following email, which I shall reproduce here. My own comments appear after. Recently, you received an email from a student asking for 10 minutes of your time to discuss your Ph.D. program (the body of the email appears below). We are emailing you today to debrief you on the actual purpose of that email, as it was part of a research study. We sincerely hope our study did not cause you any disruption and we apologize if you were at all inconvenienced. Our hope is that this letter will provide a sufficient explanation of the purpose and design of our study to alleviate any concerns you may have about your involvement. We want to thank you for your time and for reading further if you are interested in understanding why you rece
Introduction: Peter Bergman writes: is it possible to “overstratify” when assigning a treatment in a randomized control trial? I [Bergman] have a sample size of roughly 400 people, and several binary variables correlate strongly with the outcome of interest and would also define interesting subgroups for analysis. The problem is, stratifying over all of these (five or six) variables leaves me with strata that have only 1 person in them. I have done some background reading on whether there is a rule of thumb for the maximum number of variables to stratify. There does not seem to be much agreement (some say there should be between N/50-N/100 strata, others say as few as possible). In economics, the paper I looked to is here, which seems to summarize literature related to clinical trials. In short, my question is: is it bad to have several strata with 1 person in them? Should I group these people in with another stratum? P.S. In the paper I mention above, they also say it is important to inc
4 0.96878445 2118 andrew gelman stats-2013-11-30-???
Introduction: I received the following unsolicited email, subject line Technology and Engineering Research: Dear Editor We have done research in some of the cutting edge technology and engineering field and would like to if you will be able to write about it in your news section. Our Primarily research focus on building high performance systems that are helping in social networks, web, finding disease, cancer and sports using BIG DATA . Hope to hear from you some time soon. Thanks, ***, PhD Chartered Scientist IBM Corportation ***@us.ibm.com 916 *** **** I thought IBM was a professional operation—don’t they have their own public relations department?
5 0.96653932 996 andrew gelman stats-2011-11-07-Chi-square FAIL when many cells have small expected values
Introduction: William Perkins, Mark Tygert, and Rachel Ward write : If a discrete probability distribution in a model being tested for goodness-of-fit is not close to uniform, then forming the Pearson χ2 statistic can involve division by nearly zero. This often leads to serious trouble in practice — even in the absence of round-off errors . . . The problem is not merely that the chi-squared statistic doesn’t have the advertised chi-squared distribution —a reference distribution can always be computed via simulation, either using the posterior predictive distribution or by conditioning on a point estimate of the cell expectations and then making a degrees-of-freedom sort of adjustment. Rather, the problem is that, when there are lots of cells with near-zero expectation, the chi-squared test is mostly noise. And this is not merely a theoretical problem. It comes up in real examples. Here’s one, taken from the classic 1992 genetics paper of Guo and Thomspson: And here are the e
6 0.96201956 526 andrew gelman stats-2011-01-19-“If it saves the life of a single child…” and other nonsense
7 0.9600566 13 andrew gelman stats-2010-04-30-Things I learned from the Mickey Kaus for Senate campaign
9 0.95764685 820 andrew gelman stats-2011-07-25-Design of nonrandomized cluster sample study
10 0.95687836 160 andrew gelman stats-2010-07-23-Unhappy with improvement by a factor of 10^29
11 0.95652103 1218 andrew gelman stats-2012-03-18-Check your missing-data imputations using cross-validation
12 0.95319384 2137 andrew gelman stats-2013-12-17-Replication backlash
13 0.95272708 2183 andrew gelman stats-2014-01-23-Discussion on preregistration of research studies
14 0.95266479 167 andrew gelman stats-2010-07-27-Why don’t more medical discoveries become cures?
15 0.95152777 1910 andrew gelman stats-2013-06-22-Struggles over the criticism of the “cannabis users and IQ change” paper
16 0.95138335 2065 andrew gelman stats-2013-10-17-Cool dynamic demographic maps provide beautiful illustration of Chris Rock effect
17 0.95134544 1760 andrew gelman stats-2013-03-12-Misunderstanding the p-value
18 0.95109534 2246 andrew gelman stats-2014-03-13-An Economist’s Guide to Visualizing Data
19 0.95085686 2244 andrew gelman stats-2014-03-11-What if I were to stop publishing in journals?
20 0.95065415 1097 andrew gelman stats-2012-01-03-Libertarians in Space