andrew_gelman_stats andrew_gelman_stats-2012 andrew_gelman_stats-2012-1184 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Eric Loken explains : Some newspapers and radio stations recently picked up a story that Facebook profiles can be revealing, and can yield information more predictive of job performance than typical self-report personality questionnaires or even an IQ test. . . . A most consistent finding from the last 50 years of organizational psychology research is that cognitive ability is the strongest predictor of job performance, sometimes followed closely by measures of conscientiousness (and recently there has been interest in perseverance or grit). So has the Facebook study upended all this established research? Not at all, and the reason lies in the enormous gap between the claims about the study’s outcomes, and the details of what was actually done. The researchers had two college population samples. In Study 1 they had job performance ratings for the part-time college jobs of about 10% of the original sample. But in study 1 they did not have any IQ or cognitive ability measure.
sentIndex sentText sentNum sentScore
1 Eric Loken explains : Some newspapers and radio stations recently picked up a story that Facebook profiles can be revealing, and can yield information more predictive of job performance than typical self-report personality questionnaires or even an IQ test. [sent-1, score-1.949]
2 A most consistent finding from the last 50 years of organizational psychology research is that cognitive ability is the strongest predictor of job performance, sometimes followed closely by measures of conscientiousness (and recently there has been interest in perseverance or grit). [sent-5, score-1.93]
3 So has the Facebook study upended all this established research? [sent-6, score-0.318]
4 Not at all, and the reason lies in the enormous gap between the claims about the study’s outcomes, and the details of what was actually done. [sent-7, score-0.383]
5 The researchers had two college population samples. [sent-8, score-0.257]
6 In Study 1 they had job performance ratings for the part-time college jobs of about 10% of the original sample. [sent-9, score-1.086]
7 But in study 1 they did not have any IQ or cognitive ability measure. [sent-10, score-0.693]
8 In Study 2 they gathered Wonderlic’s measure of cognitive ability, but this time they had no job performance data but rather college GPA which they say is correlated with job performance. [sent-11, score-1.783]
9 All in all this particular research has very little of value to add about predicting job performance in any real world setting. [sent-15, score-0.935]
wordName wordTfidf (topN-words)
[('performance', 0.355), ('job', 0.344), ('cognitive', 0.256), ('iq', 0.238), ('ability', 0.219), ('facebook', 0.219), ('study', 0.218), ('college', 0.198), ('conscientiousness', 0.148), ('gathered', 0.141), ('stations', 0.141), ('gpa', 0.141), ('profiles', 0.132), ('questionnaires', 0.132), ('organizational', 0.123), ('strongest', 0.121), ('radio', 0.115), ('lies', 0.114), ('revealing', 0.114), ('loken', 0.112), ('enormous', 0.108), ('newspapers', 0.108), ('ratings', 0.106), ('gap', 0.1), ('established', 0.1), ('recently', 0.099), ('personality', 0.097), ('yield', 0.093), ('research', 0.093), ('picked', 0.092), ('predictor', 0.09), ('eric', 0.089), ('closely', 0.088), ('explains', 0.085), ('typical', 0.085), ('jobs', 0.083), ('predicting', 0.082), ('correlated', 0.08), ('measures', 0.078), ('outcomes', 0.074), ('consistent', 0.071), ('predictive', 0.071), ('finding', 0.07), ('followed', 0.069), ('setting', 0.068), ('measure', 0.065), ('details', 0.061), ('add', 0.061), ('psychology', 0.061), ('population', 0.059)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 1184 andrew gelman stats-2012-02-25-Facebook Profiles as Predictors of Job Performance? Maybe…but not yet.
Introduction: Eric Loken explains : Some newspapers and radio stations recently picked up a story that Facebook profiles can be revealing, and can yield information more predictive of job performance than typical self-report personality questionnaires or even an IQ test. . . . A most consistent finding from the last 50 years of organizational psychology research is that cognitive ability is the strongest predictor of job performance, sometimes followed closely by measures of conscientiousness (and recently there has been interest in perseverance or grit). So has the Facebook study upended all this established research? Not at all, and the reason lies in the enormous gap between the claims about the study’s outcomes, and the details of what was actually done. The researchers had two college population samples. In Study 1 they had job performance ratings for the part-time college jobs of about 10% of the original sample. But in study 1 they did not have any IQ or cognitive ability measure.
2 0.23292395 1980 andrew gelman stats-2013-08-13-Test scores and grades predict job performance (but maybe not at Google)
Introduction: Eric Loken writes : If you’re used to Google upending conventional wisdom, then yesterday’s interview with Laszlo Bock in the New York Times did not disappoint. Google has determined that test scores and transcripts are useless because they don’t predict performance among its employees. . . . I [Loken] am going to assume they’re well aware of the limits of their claim, and instead I’m going say that as readers of the interview we should not lose sight of a fundamental fact - Across a wide variety of employment settings, one of the most robust findings in organizational psychology is that tests of cognitive ability are strong predictors of job performance. If Google has found otherwise, what they have found is that grades and test scores are not predictive of performance at Google. In general, in the workplace tests are still highly predictive of success. If all the research says that test scores and grades predict performance, why would the people at Google want to igno
3 0.15980411 1910 andrew gelman stats-2013-06-22-Struggles over the criticism of the “cannabis users and IQ change” paper
Introduction: Ole Rogeberg points me to a discussion of a discussion of a paper: Did pre-release of my [Rogeberg's] PNAS paper on methodological problems with Meier et al’s 2012 paper on cannabis and IQ reduce the chances that it will have its intended effect? In my case, serious methodological issues related to causal inference from non-random observational data became framed as a conflict over conclusions, forcing the original research team to respond rapidly and insufficiently to my concerns, and prompting them to defend their conclusions and original paper in a way that makes a later, more comprehensive reanalysis of their data less likely. This fits with a recurring theme on this blog: the defensiveness of researchers who don’t want to admit they were wrong. Setting aside cases of outright fraud and plagiarism, I think the worst case remains that of psychologists Neil Anderson and Deniz Ones, who denied any problems even in the presence of a smoking gun of a graph revealing their data
4 0.14851987 1620 andrew gelman stats-2012-12-12-“Teaching effectiveness” as another dimension in cognitive ability
Introduction: I’m not a great teacher. I can get by because I work hard and I know a lot, and for some students my classes are just great, but it’s not a natural talent of mine. I know people who are amazing teachers, and they have something that I just don’t have. I wrote that book, Teaching Statistics: A Bag of Tricks (with Deb Nolan) because I’m not a good teacher and hence need to develop all sorts of techniques to be able to do what good teachers can do without even trying. I’m not proud of being mediocre at teaching. I don’t think that low teaching skill is some sort of indicator that I’m a great researcher. The other think about teaching ability is that I think it’s hard to detect without actually seeing someone teach a class. If you see me give a seminar presentation or even a guest lecture, you’d think I’m an awesome teacher. But, actually, no. I’m an excellent speaker, not such a great teacher. This all came to mind when I received the following email from anthropologist Hen
5 0.14674841 277 andrew gelman stats-2010-09-14-In an introductory course, when does learning occur?
Introduction: Now that September has arrived, it’s time for us to think teaching. Here’s something from Andrew Heckler and Eleanor Sayre. Heckler writes: The article describes a project studying the performance of university level students taking an intro physics course. Every week for ten weeks we took 1/10th of the students (randomly selected only once) and gave them the same set of questions relevant to the course. This allowed us to plot the evolution of average performance in the class during the quarter. We can then determine when learning occurs: For example, do they learn the material in a relevant lecture or lab or homework? Since we had about 350 students taking the course, we could get some reasonable stats. In particular, you might be interested in Figure 10 (page 774) which shows student performance day-by-day on a particular question. The performance does not change directly after lecture, but rather only when the homework was due. [emphasis added] We could not find any oth
6 0.13846052 561 andrew gelman stats-2011-02-06-Poverty, educational performance – and can be done about it
7 0.1332611 568 andrew gelman stats-2011-02-11-Calibration in chess
8 0.13151401 2082 andrew gelman stats-2013-10-30-Berri Gladwell Loken football update
10 0.10637283 1923 andrew gelman stats-2013-07-03-Bayes pays!
11 0.10544902 1367 andrew gelman stats-2012-06-05-Question 26 of my final exam for Design and Analysis of Sample Surveys
12 0.10300062 1688 andrew gelman stats-2013-01-22-That claim that students whose parents pay for more of college get worse grades
13 0.096843332 578 andrew gelman stats-2011-02-17-Credentialism, elite employment, and career aspirations
14 0.094681785 361 andrew gelman stats-2010-10-21-Tenure-track statistics job at Teachers College, here at Columbia!
15 0.0930437 1086 andrew gelman stats-2011-12-27-The most dangerous jobs in America
16 0.088113472 1271 andrew gelman stats-2012-04-20-Education could use some systematic evaluation
18 0.085661203 760 andrew gelman stats-2011-06-12-How To Party Your Way Into a Multi-Million Dollar Facebook Job
19 0.085473441 1981 andrew gelman stats-2013-08-14-The robust beauty of improper linear models in decision making
20 0.084801927 1053 andrew gelman stats-2011-12-11-This one is so dumb it makes me want to barf
topicId topicWeight
[(0, 0.12), (1, -0.04), (2, 0.022), (3, -0.081), (4, 0.011), (5, 0.049), (6, -0.004), (7, 0.029), (8, -0.042), (9, 0.046), (10, -0.04), (11, 0.026), (12, -0.0), (13, -0.048), (14, -0.042), (15, 0.026), (16, 0.08), (17, -0.007), (18, 0.005), (19, 0.021), (20, -0.022), (21, 0.016), (22, -0.013), (23, -0.026), (24, 0.04), (25, 0.012), (26, -0.051), (27, 0.026), (28, -0.003), (29, -0.028), (30, -0.003), (31, -0.043), (32, 0.057), (33, -0.002), (34, 0.033), (35, 0.055), (36, 0.057), (37, -0.007), (38, -0.009), (39, -0.056), (40, 0.084), (41, 0.015), (42, 0.025), (43, 0.038), (44, -0.017), (45, -0.072), (46, 0.015), (47, 0.055), (48, 0.083), (49, 0.008)]
simIndex simValue blogId blogTitle
same-blog 1 0.97631556 1184 andrew gelman stats-2012-02-25-Facebook Profiles as Predictors of Job Performance? Maybe…but not yet.
Introduction: Eric Loken explains : Some newspapers and radio stations recently picked up a story that Facebook profiles can be revealing, and can yield information more predictive of job performance than typical self-report personality questionnaires or even an IQ test. . . . A most consistent finding from the last 50 years of organizational psychology research is that cognitive ability is the strongest predictor of job performance, sometimes followed closely by measures of conscientiousness (and recently there has been interest in perseverance or grit). So has the Facebook study upended all this established research? Not at all, and the reason lies in the enormous gap between the claims about the study’s outcomes, and the details of what was actually done. The researchers had two college population samples. In Study 1 they had job performance ratings for the part-time college jobs of about 10% of the original sample. But in study 1 they did not have any IQ or cognitive ability measure.
2 0.73127818 1980 andrew gelman stats-2013-08-13-Test scores and grades predict job performance (but maybe not at Google)
Introduction: Eric Loken writes : If you’re used to Google upending conventional wisdom, then yesterday’s interview with Laszlo Bock in the New York Times did not disappoint. Google has determined that test scores and transcripts are useless because they don’t predict performance among its employees. . . . I [Loken] am going to assume they’re well aware of the limits of their claim, and instead I’m going say that as readers of the interview we should not lose sight of a fundamental fact - Across a wide variety of employment settings, one of the most robust findings in organizational psychology is that tests of cognitive ability are strong predictors of job performance. If Google has found otherwise, what they have found is that grades and test scores are not predictive of performance at Google. In general, in the workplace tests are still highly predictive of success. If all the research says that test scores and grades predict performance, why would the people at Google want to igno
3 0.71038508 303 andrew gelman stats-2010-09-28-“Genomics” vs. genetics
Introduction: John Cook and Joseph Delaney point to an article by Yurii Aulchenko et al., who write: 54 loci showing strong statistical evidence for association to human height were described, providing us with potential genomic means of human height prediction. In a population-based study of 5748 people, we find that a 54-loci genomic profile explained 4-6% of the sex- and age-adjusted height variance, and had limited ability to discriminate tall/short people. . . . In a family-based study of 550 people, with both parents having height measurements, we find that the Galtonian mid-parental prediction method explained 40% of the sex- and age-adjusted height variance, and showed high discriminative accuracy. . . . The message is that the simple approach of predicting child’s height using a regression model given parents’ average height performs much better than the method they have based on combining 54 genes. They also find that, if you start with the prediction based on parents’ heigh
4 0.6854673 1053 andrew gelman stats-2011-12-11-This one is so dumb it makes me want to barf
Introduction: Dan Kahan sends in this horror story: A new study finds that atheists are among society’s most distrusted group, comparable even to rapists in certain circumstances. Psychologists at the University of British Columbia and the University of Oregon say that their study demonstrates that anti-atheist prejudice stems from moral distrust, not dislike, of nonbelievers. “It’s pretty remarkable,” said Azim Shariff, an assistant professor of psychology at the University of Oregon and a co-author of the study, which appears in the current issue of Journal of Personality and Social Psychology. The study, conducted among 350 Americans adults and 420 Canadian college students, asked participants to decide if a fictional driver damaged a parked car and left the scene, then found a wallet and took the money, was the driver more likely to be a teacher, an atheist teacher, or a rapist teacher? The participants, who were from religious and nonreligious backgrounds, most often chose the athe
Introduction: Alex Tabarrok reports on an analysis from the Los Angeles Times of teacher performance (as measured by so-called value-added analysis, which is basically compares teachers based on their students’ average test scores at the end of the year, after controlling for pre-test scores. It’s well known that some teachers are much better than others, but, as Alex points out, what’s striking about the L.A. Times study is that they are publishing the estimates for individual teachers . For example, this: Nice graphics, too. To me, this illustrates one of the big advantages of research in a non-academic environment. If you’re writing an article for the L.A. Times, you can do what you want (within the limits of the law). If you’re doing the same research study at a university, there are a million restrictions. For example, from an official documen t, “The primary purpose of an Institutional Review Board (IRB) is to protect the rights and welfare of human subjects participati
6 0.64739656 561 andrew gelman stats-2011-02-06-Poverty, educational performance – and can be done about it
7 0.6333304 568 andrew gelman stats-2011-02-11-Calibration in chess
8 0.6117512 1929 andrew gelman stats-2013-07-07-Stereotype threat!
9 0.61017787 1688 andrew gelman stats-2013-01-22-That claim that students whose parents pay for more of college get worse grades
10 0.60356915 1643 andrew gelman stats-2012-12-29-Sexism in science (as elsewhere)
11 0.60285038 148 andrew gelman stats-2010-07-15-“Gender Bias Still Exists in Modern Children’s Literature, Say Centre Researchers”
12 0.59371066 35 andrew gelman stats-2010-05-16-Another update on the spam email study
13 0.58445823 1803 andrew gelman stats-2013-04-14-Why girls do better in school
14 0.58192283 370 andrew gelman stats-2010-10-25-Who gets wedding announcements in the Times?
15 0.57504827 2054 andrew gelman stats-2013-10-07-Bing is preferred to Google by people who aren’t like me
16 0.57403809 2301 andrew gelman stats-2014-04-22-Ticket to Baaaaarf
17 0.56758255 48 andrew gelman stats-2010-05-23-The bane of many causes
19 0.55732232 1910 andrew gelman stats-2013-06-22-Struggles over the criticism of the “cannabis users and IQ change” paper
20 0.55259174 1427 andrew gelman stats-2012-07-24-More from the sister blog
topicId topicWeight
[(4, 0.019), (15, 0.012), (21, 0.078), (24, 0.14), (47, 0.154), (61, 0.031), (68, 0.03), (76, 0.015), (82, 0.015), (86, 0.069), (99, 0.318)]
simIndex simValue blogId blogTitle
same-blog 1 0.95564657 1184 andrew gelman stats-2012-02-25-Facebook Profiles as Predictors of Job Performance? Maybe…but not yet.
Introduction: Eric Loken explains : Some newspapers and radio stations recently picked up a story that Facebook profiles can be revealing, and can yield information more predictive of job performance than typical self-report personality questionnaires or even an IQ test. . . . A most consistent finding from the last 50 years of organizational psychology research is that cognitive ability is the strongest predictor of job performance, sometimes followed closely by measures of conscientiousness (and recently there has been interest in perseverance or grit). So has the Facebook study upended all this established research? Not at all, and the reason lies in the enormous gap between the claims about the study’s outcomes, and the details of what was actually done. The researchers had two college population samples. In Study 1 they had job performance ratings for the part-time college jobs of about 10% of the original sample. But in study 1 they did not have any IQ or cognitive ability measure.
2 0.94701636 1668 andrew gelman stats-2013-01-11-My talk at the NY data visualization meetup this Monday!
Introduction: It’s in midtown at 7pm (on Mon 14 Jan 2013). Last time I talked for this group, I spoke on Infovis vs. Statistical Graphics . This time I plan to just go thru the choices involved in a few zillion graphs I’ve published over the years, to give a sense of the options and choices involved in graphical communication. For this talk there will be no single theme (except, perhaps, my usual “Graphs as comparisons,” “All of statistics as comparisons,” and “Exploratory data analysis as hypothesis testing”), just a bunch of open discussion about what I tried, why I tried it, what worked and what didn’t work, etc. I’ve discussed these sorts of decisions on occasion (and am now writing a paper with Yair about some of this for our voting models), but I’ve never tried to make a talk out of it before. Could be fun.
3 0.9462018 1261 andrew gelman stats-2012-04-12-The Naval Research Lab
Introduction: I worked at the U.S. Naval Research Laboratory for four summers during high school and college. I spent much of my time writing a computer program to do thermal analysis for an experiment that we put on the space shuttle. The facility I developed with the finite-element method came in handy in my job at Bell Labs the following summers. I was working for C. H. Tsao and Jim Adams in the Laboratory for Cosmic Ray Physics. We were estimating the distribution of isotopes in cosmic rays using a pile of track detectors. To get accurate measurements, you want these plastic disks to be as close as possible to a constant temperature, so we designed an elaborate wrapping of thermal blankets. My program computed the temperature of the detectors during the year that the Long Duration Exposure Facility (including our experiment and a bunch of others) was scheduled to be in orbit. The input is the heat from solar radiation (easy enough to compute given the trajectory). On the computer I tr
4 0.94529247 1055 andrew gelman stats-2011-12-13-Data sharing update
Introduction: Fred Oswald reports that Sian Beilock sent him sufficient amounts of raw data from her research study so allow him to answer his questions about the large effects that were observed. This sort of collegiality is central to the collective scientific enterprise. The bad news is that IRB’s are still getting in the way. Beilock was very helpful but she had to work within the constraints of her IRB, which apparently advised her not to share data—even if de-identified—without getting lots more permissions. Oswald writes: It is a little concerning that the IRB bars the sharing of de-identified data, particularly in light of the specific guidelines of the journal Science, which appears to say that when you submit a study to the journal for publication, you are allowing for the sharing of de-identified data — unless you expressly say otherwise at the point that you submit the paper for consideration. Again, I don’t blame Beilock and Ramirez—they appear to have been as helpful as
Introduction: Remember How to Lie With Statistics? It turns out that the author worked for the cigarette companies. John Mashey points to this, from Robert Proctor’s book, “Golden Holocaust: Origins of the Cigarette Catastrophe and the Case for Abolition”: Darrell Huff, author of the wildly popular (and aptly named) How to Lie With Statistics, was paid to testify before Congress in the 1950s and then again in the 1960s, with the assigned task of ridiculing any notion of a cigarette-disease link. On March 22, 1965, Huff testified at hearings on cigarette labeling and advertising, accusing the recent Surgeon General’s report of myriad failures and “fallacies.” Huff peppered his attack with with amusing asides and anecdotes, lampooning spurious correlations like that between the size of Dutch families and the number of storks nesting on rooftops–which proves not that storks bring babies but rather that people with large families tend to have larger houses (which therefore attract more storks).
6 0.94352782 1050 andrew gelman stats-2011-12-10-Presenting at the econ seminar
7 0.9417612 2275 andrew gelman stats-2014-03-31-Just gave a talk
8 0.93064481 275 andrew gelman stats-2010-09-14-Data visualization at the American Evaluation Association
9 0.93047804 95 andrew gelman stats-2010-06-17-“Rewarding Strivers: Helping Low-Income Students Succeed in College”
10 0.92914331 1897 andrew gelman stats-2013-06-13-When’s that next gamma-ray blast gonna come, already?
11 0.92721099 1349 andrew gelman stats-2012-05-28-Question 18 of my final exam for Design and Analysis of Sample Surveys
12 0.92165101 1143 andrew gelman stats-2012-01-29-G+ > Skype
13 0.91809005 1486 andrew gelman stats-2012-09-07-Prior distributions for regression coefficients
15 0.91384304 2068 andrew gelman stats-2013-10-18-G+ hangout for Bayesian Data Analysis course now! (actually, in 5 minutes)
16 0.91253757 1730 andrew gelman stats-2013-02-20-Unz on Unz
17 0.91132319 2322 andrew gelman stats-2014-05-06-Priors I don’t believe
18 0.90861982 1218 andrew gelman stats-2012-03-18-Check your missing-data imputations using cross-validation
19 0.90825063 1450 andrew gelman stats-2012-08-08-My upcoming talk for the data visualization meetup
20 0.90685451 1273 andrew gelman stats-2012-04-20-Proposals for alternative review systems for scientific work