andrew_gelman_stats andrew_gelman_stats-2013 andrew_gelman_stats-2013-1980 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Eric Loken writes : If you’re used to Google upending conventional wisdom, then yesterday’s interview with Laszlo Bock in the New York Times did not disappoint. Google has determined that test scores and transcripts are useless because they don’t predict performance among its employees. . . . I [Loken] am going to assume they’re well aware of the limits of their claim, and instead I’m going say that as readers of the interview we should not lose sight of a fundamental fact - Across a wide variety of employment settings, one of the most robust findings in organizational psychology is that tests of cognitive ability are strong predictors of job performance. If Google has found otherwise, what they have found is that grades and test scores are not predictive of performance at Google. In general, in the workplace tests are still highly predictive of success. If all the research says that test scores and grades predict performance, why would the people at Google want to igno
sentIndex sentText sentNum sentScore
1 Eric Loken writes : If you’re used to Google upending conventional wisdom, then yesterday’s interview with Laszlo Bock in the New York Times did not disappoint. [sent-1, score-0.105]
2 Google has determined that test scores and transcripts are useless because they don’t predict performance among its employees. [sent-2, score-1.095]
3 If Google has found otherwise, what they have found is that grades and test scores are not predictive of performance at Google. [sent-7, score-1.049]
4 In general, in the workplace tests are still highly predictive of success. [sent-8, score-0.415]
5 If all the research says that test scores and grades predict performance, why would the people at Google want to ignore this information? [sent-9, score-0.794]
6 Loken continues: There are at least two factors in play here (and again I’m assuming the folks at Google are well aware of both of these points). [sent-10, score-0.161]
7 First, when a company has built its brand on attracting only the brightest prospective employees, through self-selection and through sheer volume of applicants the pool will be extremely competitive. [sent-11, score-0.6]
8 Google likely doesn’t have much variability among those hired with respect to test scores and grades. [sent-12, score-0.907]
9 And when there is no variability, there is no correlation with anything. [sent-13, score-0.179]
10 It’s a similar argument to MIT saying that the SAT is useless for their admissions. [sent-14, score-0.115]
11 Their applicant pool is so vast and highly qualified that the incoming class is largely homogeneous on those measures. [sent-15, score-0.583]
12 We of course need to wonder how all those people with lower test scores and grades would have fared at Google had they been hired. [sent-17, score-0.787]
13 But furthermore, when an organization has used a certain instrument to select their employees, the correlation with job performance goes down. [sent-18, score-0.667]
14 It’s also that the information has already been acted upon. [sent-20, score-0.085]
15 If someone is hired despite their lower test scores, it usually means some compelling compensating characteristics made that person look like a good bet. [sent-21, score-0.626]
16 That is why the correlation between a valid selection instrument and job performance can be dramatically depressed when only looking at the hired sample. [sent-22, score-1.029]
17 Loken concludes: It’s common to misinterpret that low correlation as a sign of poor prediction. [sent-23, score-0.274]
18 Again – thousands of research studies have confirmed the predictive validity of tests of cognitive ability for job performance. [sent-24, score-0.651]
wordName wordTfidf (topN-words)
[('google', 0.342), ('scores', 0.307), ('loken', 0.271), ('performance', 0.214), ('test', 0.211), ('hired', 0.196), ('grades', 0.188), ('correlation', 0.179), ('job', 0.139), ('employees', 0.137), ('instrument', 0.135), ('predictive', 0.129), ('variability', 0.128), ('tests', 0.124), ('pool', 0.123), ('useless', 0.115), ('interview', 0.105), ('cognitive', 0.103), ('homogeneous', 0.095), ('misinterpret', 0.095), ('transcripts', 0.095), ('attracting', 0.095), ('aware', 0.092), ('brightest', 0.089), ('ability', 0.088), ('predict', 0.088), ('selection', 0.086), ('acted', 0.085), ('sight', 0.082), ('highly', 0.082), ('lower', 0.081), ('sheer', 0.08), ('workplace', 0.08), ('depressed', 0.08), ('applicant', 0.078), ('brand', 0.078), ('compensating', 0.075), ('incoming', 0.075), ('organizational', 0.075), ('prospective', 0.07), ('folks', 0.069), ('confirmed', 0.068), ('qualified', 0.068), ('furthermore', 0.067), ('applicants', 0.065), ('among', 0.065), ('mit', 0.065), ('compelling', 0.063), ('wisdom', 0.063), ('vast', 0.062)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 1980 andrew gelman stats-2013-08-13-Test scores and grades predict job performance (but maybe not at Google)
Introduction: Eric Loken writes : If you’re used to Google upending conventional wisdom, then yesterday’s interview with Laszlo Bock in the New York Times did not disappoint. Google has determined that test scores and transcripts are useless because they don’t predict performance among its employees. . . . I [Loken] am going to assume they’re well aware of the limits of their claim, and instead I’m going say that as readers of the interview we should not lose sight of a fundamental fact - Across a wide variety of employment settings, one of the most robust findings in organizational psychology is that tests of cognitive ability are strong predictors of job performance. If Google has found otherwise, what they have found is that grades and test scores are not predictive of performance at Google. In general, in the workplace tests are still highly predictive of success. If all the research says that test scores and grades predict performance, why would the people at Google want to igno
2 0.23292395 1184 andrew gelman stats-2012-02-25-Facebook Profiles as Predictors of Job Performance? Maybe…but not yet.
Introduction: Eric Loken explains : Some newspapers and radio stations recently picked up a story that Facebook profiles can be revealing, and can yield information more predictive of job performance than typical self-report personality questionnaires or even an IQ test. . . . A most consistent finding from the last 50 years of organizational psychology research is that cognitive ability is the strongest predictor of job performance, sometimes followed closely by measures of conscientiousness (and recently there has been interest in perseverance or grit). So has the Facebook study upended all this established research? Not at all, and the reason lies in the enormous gap between the claims about the study’s outcomes, and the details of what was actually done. The researchers had two college population samples. In Study 1 they had job performance ratings for the part-time college jobs of about 10% of the original sample. But in study 1 they did not have any IQ or cognitive ability measure.
3 0.20353676 2082 andrew gelman stats-2013-10-30-Berri Gladwell Loken football update
Introduction: Sports researcher Dave Berri had a disagreement with a remark in our recent discussion of Malcolm Gladwell. Berri writes: This post [from Gelman] contains the following paragraph: Similarly, when Gladwell claimed that NFL quarterback performance is unrelated to the order they were drafted out of college, he appears to have been wrong. But if you take his writing as stone soup, maybe it’s valuable: just retreat to the statement that there’s only a weak relationship between draft order and NFL performance. That alone is interesting. It’s too bad that Gladwell sometimes has to make false general statements in order to get our attention, but maybe that’s what is needed to shake people out of their mental complacency. The above paragraph links to a blog post by Eric Loken. This is something you have linked to before. And when you linked to it before I tried to explain why Loken’s work is not very good. Since you still think this work shows that Gladwell – and therefore Rob
4 0.18767433 911 andrew gelman stats-2011-09-15-More data tools worth using from Google
Introduction: Speaking of open data and google tools, see this post from Revolution R: How to use a Google Spreadsheet as data in R .
5 0.17270863 1803 andrew gelman stats-2013-04-14-Why girls do better in school
Introduction: Wayne Folta writes, “In light of your recent blog post on women in higher education, here’s one I just read about on a techie website regarding elementary education”: Why do girls get better grades in elementary school than boys—even when they perform worse on standardized tests? New research . . . suggests that it’s because of their classroom behavior, which may lead teachers to assign girls higher grades than their male counterparts. . . . The study, co-authored by [Christopher] Cornwell and David Mustard at UGA and Jessica Van Parys at Columbia, analyzed data on more than 5,800 students from kindergarten through fifth grade. It examined students’ performance on standardized tests in three categories—reading, math and science-linking test scores to teachers’ assessments of their students’ progress, both academically and more broadly. The data show, for the first time, that gender disparities in teacher grades start early and uniformly favor girls. In every subject area, bo
6 0.14987063 207 andrew gelman stats-2010-08-14-Pourquoi Google search est devenu plus raisonnable?
7 0.14231251 825 andrew gelman stats-2011-07-27-Grade inflation: why weren’t the instructors all giving all A’s already??
8 0.14029361 1237 andrew gelman stats-2012-03-30-Statisticians: When We Teach, We Don’t Practice What We Preach
9 0.13906862 1559 andrew gelman stats-2012-11-02-The blog is back
10 0.13883978 1507 andrew gelman stats-2012-09-22-Grade inflation: why weren’t the instructors all giving all A’s already??
11 0.13632737 315 andrew gelman stats-2010-10-03-He doesn’t trust the fit . . . r=.999
12 0.13448803 226 andrew gelman stats-2010-08-23-More on those L.A. Times estimates of teacher effectiveness
13 0.13242529 609 andrew gelman stats-2011-03-13-Coauthorship norms
14 0.13173792 505 andrew gelman stats-2011-01-05-Wacky interview questions: An exploration into the nature of evidence on the internet
15 0.12657726 32 andrew gelman stats-2010-05-14-Causal inference in economics
16 0.12577927 1688 andrew gelman stats-2013-01-22-That claim that students whose parents pay for more of college get worse grades
17 0.119658 253 andrew gelman stats-2010-09-03-Gladwell vs Pinker
18 0.11789081 1605 andrew gelman stats-2012-12-04-Write This Book
20 0.10753417 1488 andrew gelman stats-2012-09-08-Annals of spam
topicId topicWeight
[(0, 0.149), (1, -0.025), (2, 0.032), (3, -0.042), (4, 0.043), (5, 0.054), (6, 0.034), (7, 0.042), (8, -0.013), (9, 0.008), (10, -0.047), (11, 0.031), (12, 0.008), (13, -0.078), (14, -0.024), (15, 0.034), (16, 0.08), (17, -0.003), (18, -0.022), (19, 0.002), (20, 0.002), (21, 0.031), (22, 0.027), (23, -0.058), (24, 0.058), (25, -0.018), (26, -0.038), (27, 0.021), (28, 0.002), (29, 0.023), (30, 0.047), (31, -0.063), (32, 0.116), (33, 0.016), (34, 0.001), (35, 0.046), (36, 0.072), (37, -0.036), (38, 0.032), (39, -0.034), (40, 0.076), (41, -0.015), (42, 0.076), (43, 0.075), (44, -0.107), (45, 0.0), (46, 0.003), (47, -0.012), (48, 0.054), (49, -0.072)]
simIndex simValue blogId blogTitle
same-blog 1 0.97900838 1980 andrew gelman stats-2013-08-13-Test scores and grades predict job performance (but maybe not at Google)
Introduction: Eric Loken writes : If you’re used to Google upending conventional wisdom, then yesterday’s interview with Laszlo Bock in the New York Times did not disappoint. Google has determined that test scores and transcripts are useless because they don’t predict performance among its employees. . . . I [Loken] am going to assume they’re well aware of the limits of their claim, and instead I’m going say that as readers of the interview we should not lose sight of a fundamental fact - Across a wide variety of employment settings, one of the most robust findings in organizational psychology is that tests of cognitive ability are strong predictors of job performance. If Google has found otherwise, what they have found is that grades and test scores are not predictive of performance at Google. In general, in the workplace tests are still highly predictive of success. If all the research says that test scores and grades predict performance, why would the people at Google want to igno
2 0.68335801 1184 andrew gelman stats-2012-02-25-Facebook Profiles as Predictors of Job Performance? Maybe…but not yet.
Introduction: Eric Loken explains : Some newspapers and radio stations recently picked up a story that Facebook profiles can be revealing, and can yield information more predictive of job performance than typical self-report personality questionnaires or even an IQ test. . . . A most consistent finding from the last 50 years of organizational psychology research is that cognitive ability is the strongest predictor of job performance, sometimes followed closely by measures of conscientiousness (and recently there has been interest in perseverance or grit). So has the Facebook study upended all this established research? Not at all, and the reason lies in the enormous gap between the claims about the study’s outcomes, and the details of what was actually done. The researchers had two college population samples. In Study 1 they had job performance ratings for the part-time college jobs of about 10% of the original sample. But in study 1 they did not have any IQ or cognitive ability measure.
3 0.66545713 1559 andrew gelman stats-2012-11-02-The blog is back
Introduction: We had some security problem: not an actual virus or anything, but a potential leak which caused Google to blacklist us. Cord fixed us and now we’re fine. Good job, Google! Better to find the potential problem before there is any harm!
4 0.64853984 1803 andrew gelman stats-2013-04-14-Why girls do better in school
Introduction: Wayne Folta writes, “In light of your recent blog post on women in higher education, here’s one I just read about on a techie website regarding elementary education”: Why do girls get better grades in elementary school than boys—even when they perform worse on standardized tests? New research . . . suggests that it’s because of their classroom behavior, which may lead teachers to assign girls higher grades than their male counterparts. . . . The study, co-authored by [Christopher] Cornwell and David Mustard at UGA and Jessica Van Parys at Columbia, analyzed data on more than 5,800 students from kindergarten through fifth grade. It examined students’ performance on standardized tests in three categories—reading, math and science-linking test scores to teachers’ assessments of their students’ progress, both academically and more broadly. The data show, for the first time, that gender disparities in teacher grades start early and uniformly favor girls. In every subject area, bo
5 0.64524186 2054 andrew gelman stats-2013-10-07-Bing is preferred to Google by people who aren’t like me
Introduction: This one is fun because I have a double conflict of interest: I’ve been paid (at different times) both by Google and by Microsoft. Here’s the story: Microsoft, September 2012 : An independent research company, Answers Research based in San Diego, CA, conducted a study using a representative online sample of nearly 1000 people, ages 18 and older from across the US. The participants were chosen from a random survey panel and were required to have used a major search engine in the past month. Participants were not aware that Microsoft was involved. In the test, participants were shown the main web search results pane of both Bing and Google for 10 search queries of their choice. Bing and Google search results were shown side-by-side on one page for easy comparison – with all branding removed from both search engines. The test did not include ads or content in other parts of the page such as Bing’s Snapshot and Social Search panes and Google’s Knowledge Graph. For each search,
8 0.61775959 561 andrew gelman stats-2011-02-06-Poverty, educational performance – and can be done about it
9 0.61642909 226 andrew gelman stats-2010-08-23-More on those L.A. Times estimates of teacher effectiveness
10 0.61092293 1688 andrew gelman stats-2013-01-22-That claim that students whose parents pay for more of college get worse grades
11 0.60514235 1350 andrew gelman stats-2012-05-28-Value-added assessment: What went wrong?
12 0.60407454 315 andrew gelman stats-2010-10-03-He doesn’t trust the fit . . . r=.999
14 0.60377514 748 andrew gelman stats-2011-06-06-Why your Klout score is meaningless
15 0.58529425 1620 andrew gelman stats-2012-12-12-“Teaching effectiveness” as another dimension in cognitive ability
16 0.57533425 484 andrew gelman stats-2010-12-24-Foreign language skills as an intrinsic good; also, beware the tyranny of measurement
17 0.57408851 1612 andrew gelman stats-2012-12-08-The Case for More False Positives in Anti-doping Testing
18 0.56340593 1918 andrew gelman stats-2013-06-29-Going negative
19 0.56127959 505 andrew gelman stats-2011-01-05-Wacky interview questions: An exploration into the nature of evidence on the internet
20 0.55927825 825 andrew gelman stats-2011-07-27-Grade inflation: why weren’t the instructors all giving all A’s already??
topicId topicWeight
[(5, 0.013), (9, 0.015), (15, 0.01), (16, 0.119), (21, 0.02), (24, 0.137), (59, 0.016), (61, 0.067), (66, 0.015), (77, 0.044), (78, 0.014), (82, 0.032), (86, 0.083), (88, 0.022), (89, 0.026), (95, 0.027), (99, 0.21)]
simIndex simValue blogId blogTitle
same-blog 1 0.95896673 1980 andrew gelman stats-2013-08-13-Test scores and grades predict job performance (but maybe not at Google)
Introduction: Eric Loken writes : If you’re used to Google upending conventional wisdom, then yesterday’s interview with Laszlo Bock in the New York Times did not disappoint. Google has determined that test scores and transcripts are useless because they don’t predict performance among its employees. . . . I [Loken] am going to assume they’re well aware of the limits of their claim, and instead I’m going say that as readers of the interview we should not lose sight of a fundamental fact - Across a wide variety of employment settings, one of the most robust findings in organizational psychology is that tests of cognitive ability are strong predictors of job performance. If Google has found otherwise, what they have found is that grades and test scores are not predictive of performance at Google. In general, in the workplace tests are still highly predictive of success. If all the research says that test scores and grades predict performance, why would the people at Google want to igno
2 0.91817534 1016 andrew gelman stats-2011-11-17-I got 99 comparisons but multiplicity ain’t one
Introduction: After I gave my talk at an econ seminar on Why We (Usually) Don’t Care About Multiple Comparisons, I got the following comment: One question that came up later was whether your argument is really with testing in general, rather than only with testing in multiple comparison settings. My reply: Yes, my argument is with testing in general. But it arises with particular force in multiple comparisons. With a single test, we can just say we dislike testing so we use confidence intervals or Bayesian inference instead, and it’s no problem—really more of a change in emphasis than a change in methods. But with multiple tests, the classical advice is not simply to look at type 1 error rates but more specifically to make a multiplicity adjustment, for example to make confidence intervals wider to account for multiplicity. I don’t want to do this! So here there is a real battle to fight. P.S. Here’s the article (with Jennifer and Masanao), to appear in the Journal of Research on
3 0.91810358 1019 andrew gelman stats-2011-11-19-Validation of Software for Bayesian Models Using Posterior Quantiles
Introduction: I love this stuff : This article presents a simulation-based method designed to establish the computational correctness of software developed to fit a specific Bayesian model, capitalizing on properties of Bayesian posterior distributions. We illustrate the validation technique with two examples. The validation method is shown to find errors in software when they exist and, moreover, the validation output can be informative about the nature and location of such errors. We also compare our method with that of an earlier approach. I hope we can put it into Stan.
4 0.91604459 185 andrew gelman stats-2010-08-04-Why does anyone support private macroeconomic forecasts?
Introduction: Tyler Cowen asks the above question. I don’t have a full answer, but, in the Economics section of A Quantitative Tour of the Social Sciences , Richard Clarida discusses in detail the ways that researchers have tried to estimate the extent to which government or private forecasts supply additional information.
Introduction: Etienne LeBel writes: You’ve probably already seen it, but I thought you could have a lot of fun with this one!! The article , with the admirably clear title given above, is by James McNulty, Michael Olson, Andrea Meltzer, Matthew Shaffer, and begins as follows: For decades, social psychological theories have posited that the automatic processes captured by implicit measures have implications for social outcomes. Yet few studies have demonstrated any long-term implications of automatic processes, and some scholars have begun to question the relevance and even the validity of these theories. At baseline of our longitudinal study, 135 newlywed couples (270 individuals) completed an explicit measure of their conscious attitudes toward their relationship and an implicit measure of their automatic attitudes toward their partner. They then reported their marital satisfaction every 6 months for the next 4 years. We found no correlation between spouses’ automatic and conscious attitu
6 0.90946031 599 andrew gelman stats-2011-03-03-Two interesting posts elsewhere on graphics
7 0.90902269 16 andrew gelman stats-2010-05-04-Burgess on Kipling
8 0.90825397 301 andrew gelman stats-2010-09-28-Correlation, prediction, variation, etc.
9 0.90778065 2182 andrew gelman stats-2014-01-22-Spell-checking example demonstrates key aspects of Bayesian data analysis
10 0.90756088 1266 andrew gelman stats-2012-04-16-Another day, another plagiarist
11 0.90705019 1278 andrew gelman stats-2012-04-23-“Any old map will do” meets “God is in every leaf of every tree”
12 0.90674901 827 andrew gelman stats-2011-07-28-Amusing case of self-defeating science writing
13 0.9059999 1975 andrew gelman stats-2013-08-09-Understanding predictive information criteria for Bayesian models
14 0.90486717 2299 andrew gelman stats-2014-04-21-Stan Model of the Week: Hierarchical Modeling of Supernovas
15 0.90440416 1206 andrew gelman stats-2012-03-10-95% intervals that I don’t believe, because they’re from a flat prior I don’t believe
16 0.90423477 411 andrew gelman stats-2010-11-13-Ethical concerns in medical trials
17 0.90392178 187 andrew gelman stats-2010-08-05-Update on state size and governors’ popularity
18 0.90284121 2179 andrew gelman stats-2014-01-20-The AAA Tranche of Subprime Science
19 0.90220803 777 andrew gelman stats-2011-06-23-Combining survey data obtained using different modes of sampling
20 0.90138686 2248 andrew gelman stats-2014-03-15-Problematic interpretations of confidence intervals