andrew_gelman_stats andrew_gelman_stats-2014 andrew_gelman_stats-2014-2270 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Mark Palko explains why a penalty for getting the wrong answer on a test (the SAT, which is used in college admissions and which is used in the famous 8 schools example) is not a “penalty for guessing.” Then the very next day he catches this from Todd Balf in the New York Times Magazine: Students were docked one-quarter point for every multiple-choice question they got wrong, requiring a time-consuming risk analysis to determine which questions to answer and which to leave blank. Ugh! That just makes me want to . . . ok, I won’t go there. Anyway, Palko goes to the trouble to explain: While time management for a test like the SAT can be complicated, the rule for guessing is embarrassingly simple: give your best guess for questions you read; don’t waste time guessing on questions that you didn’t have time to read. The risk analysis actually becomes much more complicated when you take away the penalty for guessing. On the ACT (or the new SAT), there is a positive
sentIndex sentText sentNum sentScore
1 Mark Palko explains why a penalty for getting the wrong answer on a test (the SAT, which is used in college admissions and which is used in the famous 8 schools example) is not a “penalty for guessing. [sent-1, score-0.533]
2 ” Then the very next day he catches this from Todd Balf in the New York Times Magazine: Students were docked one-quarter point for every multiple-choice question they got wrong, requiring a time-consuming risk analysis to determine which questions to answer and which to leave blank. [sent-2, score-0.667]
3 Anyway, Palko goes to the trouble to explain: While time management for a test like the SAT can be complicated, the rule for guessing is embarrassingly simple: give your best guess for questions you read; don’t waste time guessing on questions that you didn’t have time to read. [sent-8, score-1.334]
4 The risk analysis actually becomes much more complicated when you take away the penalty for guessing. [sent-9, score-0.584]
5 On the ACT (or the new SAT), there is a positive expected value associated with blind guessing and that value is large enough to cause trouble. [sent-10, score-0.459]
6 Under severe time constraints (a fairly common occurrence with these tests), the minute it would take you to attempt a problem, even if you get it right, would be better spent filling in bubbles for questions you haven’t read. [sent-11, score-0.725]
7 Putting aside what this does to the validity of the test, trying to decide when to start guessing is a real and needless distraction for test takers. [sent-12, score-0.565]
8 In summary: The claim about the effects of the correction for guessing aren’t just wrong; they are the opposite of right. [sent-13, score-0.329]
9 The old system didn’t require time-consuming risk analysis but the new one does. [sent-14, score-0.392]
10 Without time control, you look at each question and make your best guess. [sent-16, score-0.187]
11 And if everybody attempts every question, then taking off points for wrong answers has exactly zero effect. [sent-18, score-0.222]
12 But if not everyone gets to every question, then the new format creates a clear incentive for people to spend time filling in bubbles to questions they haven’t looked at. [sent-19, score-0.844]
13 I also clicked through to Balf’s New York Times magazine article and noticed this: When the Scholastic Aptitude Test was created in 1926, it was promoted as a tool to create a classless, Jeffersonian-style meritocracy. [sent-21, score-0.512]
14 Is a Jeffersonian-style meritocracy the system where everyone is equal and so we all own slaves? [sent-23, score-0.219]
15 I guess if this article were written in Russia it would all about how the SAT was promoted as a tool to create a Lenin-style democracy. [sent-24, score-0.416]
16 The problem I see here is that Balf seems to be dealing in images and impressions rather than thinking through his ideas. [sent-25, score-0.183]
17 “Jefferson” and “meritocracy” have positive images, so they go together, the old SAT was bad so therefore it required “a time-consuming risk analysis,” etc. [sent-26, score-0.305]
18 One journalistic convention that I really can’t stand is the push toward giving every story a single coherent perspective. [sent-27, score-0.181]
19 Another article by someone else might take the opposite stance. [sent-29, score-0.245]
20 I’m not saying the article needs “balance,” I’d just like to see a bit of questioning. [sent-30, score-0.087]
wordName wordTfidf (topN-words)
[('balf', 0.392), ('sat', 0.246), ('guessing', 0.24), ('penalty', 0.181), ('bubbles', 0.179), ('risk', 0.167), ('test', 0.166), ('palko', 0.153), ('promoted', 0.151), ('filling', 0.151), ('questions', 0.149), ('meritocracy', 0.147), ('images', 0.113), ('every', 0.111), ('wrong', 0.111), ('time', 0.102), ('magazine', 0.096), ('tool', 0.092), ('complicated', 0.09), ('slaves', 0.089), ('coleman', 0.089), ('opposite', 0.089), ('article', 0.087), ('create', 0.086), ('question', 0.085), ('aptitude', 0.084), ('scholastic', 0.084), ('embarrassingly', 0.084), ('jefferson', 0.084), ('todd', 0.081), ('needless', 0.081), ('new', 0.08), ('distraction', 0.078), ('catches', 0.078), ('analysis', 0.077), ('occurrence', 0.075), ('college', 0.075), ('york', 0.075), ('ugh', 0.074), ('everyone', 0.072), ('horribly', 0.072), ('control', 0.072), ('haven', 0.071), ('positive', 0.07), ('journalistic', 0.07), ('impressions', 0.07), ('blind', 0.069), ('russia', 0.069), ('take', 0.069), ('old', 0.068)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 2270 andrew gelman stats-2014-03-28-Creating a Lenin-style democracy
Introduction: Mark Palko explains why a penalty for getting the wrong answer on a test (the SAT, which is used in college admissions and which is used in the famous 8 schools example) is not a “penalty for guessing.” Then the very next day he catches this from Todd Balf in the New York Times Magazine: Students were docked one-quarter point for every multiple-choice question they got wrong, requiring a time-consuming risk analysis to determine which questions to answer and which to leave blank. Ugh! That just makes me want to . . . ok, I won’t go there. Anyway, Palko goes to the trouble to explain: While time management for a test like the SAT can be complicated, the rule for guessing is embarrassingly simple: give your best guess for questions you read; don’t waste time guessing on questions that you didn’t have time to read. The risk analysis actually becomes much more complicated when you take away the penalty for guessing. On the ACT (or the new SAT), there is a positive
2 0.1493492 93 andrew gelman stats-2010-06-17-My proposal for making college admissions fairer
Introduction: After reading the Rewarding Strivers book , I had some thoughts about how to make the college admissions system more fair to students from varying socioeconomic backgrounds. Instead of boosting up the disadvantaged students, why not pull down the advantaged students? Here’s the idea. Disadvantaged students are defined typically not by a bad thing that they have, but rather by good things that they don’t have: financial resources, a high-quality education, and so forth. In contrast, advantaged students get all sorts of freebies. So here are my suggestions: 1. All high school grades on a 4-point scale (A=4, B=3, etc). No more of this 5-points-for-an-A-in-an-AP course, which gives the ridiculous outcomes of kids graduating with a 4.3 average, not so fair to kids in schools that don’t offer a lot of AP classes. 2. Subtract points for taking the SAT multiple times. A simple rule would be: You can use your highest SAT score, but you lose 50 points for every other time
3 0.12113845 95 andrew gelman stats-2010-06-17-“Rewarding Strivers: Helping Low-Income Students Succeed in College”
Introduction: Several years ago, I heard about a project at the Educational Testing Service to identify “strivers”: students from disadvantaged backgrounds who did unexpectedly well on the SAT (the college admissions exam formerly known as the “Scholastic Aptitude Test” but apparently now just “the SAT,” in the same way that Exxon is just “Exxon” and that Harry Truman’s middle name is just “S”), at least 200 points above a predicted score based on demographic and neighborhood information. My ETS colleague and I agreed that this was a silly idea: From a statistical point of view, if student A is expected ahead of time to do better than student B, and then they get identical test scores, then you’d expect student A (the non-”striver”) to do better than student B (the “striver”) later on. Just basic statistics: if a student does much better than expected, then probably some of that improvement is noise. The idea of identifying these “strivers” seemed misguided and not the best use of the SAT.
4 0.11338206 94 andrew gelman stats-2010-06-17-SAT stories
Introduction: I received a bunch of interesting comments on my blog on adjusting SAT scores. Below I have a long comment from a colleague with experience in the field. But first, this hilarious (from a statistical perspective) story from Howard Wainer: Some years ago when we were visiting Harvard [as a parent of a potential student, not in Howard's role as educational researcher], an admissions director said two things of relevance (i) the SAT hasn’t got enough ‘top’ for Harvard — it doesn’t discriminate well enough at the high end. To prove this she said (ii) that Harvard had more than 1500 ‘perfect 1600s’ apply. Some were rejected. I mentioned that there were only about 750 1600s from HS seniors in the US — about 400 had 1600 in their junior year (and obviously didn’t retake) and about 350 from their senior year. So, I concluded, she must be mistaken. Then I found out that they allowed applicants to pick and choose their highest SAT-V score and their highest SAT-M score from separate adm
5 0.10805772 943 andrew gelman stats-2011-10-04-Flip it around
Introduction: Mark Palko discusses a radio interview on the effect of parents on children’s education. In short, the interviewer (Stephen Dubner of Freakonomics fame) claims that the research shows that parents don’t have much influence on whether their children go to college. The evidence is based on a comparison of adopted and non-adopted children. Palko makes a convincing case that the statistical analysis (by economist Bruce Sacerdote) doesn’t show what Dubner says it shows. I looked over the linked transcript, and overall I’m less unhappy than Palko is about the interview. I agree that some of the causal implications are sloppy, and I think it’s a bit silly for the interviewer (Kai Ryssdal) to use celebrities as a benchmark. (Ryssdal says, “if [a certain parenting style is] good enough for Steven Levitt, it’s good enough for me.” But Levitt is a multimillionaire—he’ll always have a huge financial cushion. It’s not clear that what works for him would work for others who are not so wel
7 0.10363492 1318 andrew gelman stats-2012-05-13-Stolen jokes
8 0.1034283 1683 andrew gelman stats-2013-01-19-“Confirmation, on the other hand, is not sexy”
9 0.10281398 962 andrew gelman stats-2011-10-17-Death!
10 0.10270653 2328 andrew gelman stats-2014-05-10-What property is important in a risk prediction model? Discrimination or calibration?
11 0.1020509 481 andrew gelman stats-2010-12-22-The Jumpstart financial literacy survey and the different purposes of tests
12 0.10190184 1464 andrew gelman stats-2012-08-20-Donald E. Westlake on George W. Bush
13 0.094414331 1605 andrew gelman stats-2012-12-04-Write This Book
14 0.090872593 666 andrew gelman stats-2011-04-18-American Beliefs about Economic Opportunity and Income Inequality
16 0.087441869 1037 andrew gelman stats-2011-12-01-Lamentably common misunderstanding of meritocracy
17 0.087276779 1311 andrew gelman stats-2012-05-10-My final exam for Design and Analysis of Sample Surveys
18 0.085642979 401 andrew gelman stats-2010-11-08-Silly old chi-square!
20 0.084752373 1690 andrew gelman stats-2013-01-23-When are complicated models helpful in psychology research and when are they overkill?
topicId topicWeight
[(0, 0.19), (1, -0.059), (2, 0.008), (3, -0.02), (4, 0.032), (5, 0.02), (6, 0.07), (7, 0.054), (8, 0.03), (9, -0.035), (10, -0.049), (11, 0.087), (12, 0.009), (13, 0.006), (14, -0.009), (15, 0.001), (16, 0.045), (17, 0.001), (18, 0.017), (19, 0.017), (20, -0.015), (21, 0.011), (22, 0.009), (23, -0.006), (24, 0.003), (25, -0.032), (26, 0.02), (27, 0.0), (28, 0.022), (29, 0.002), (30, -0.018), (31, -0.021), (32, 0.057), (33, 0.034), (34, 0.03), (35, -0.019), (36, -0.026), (37, 0.049), (38, 0.005), (39, 0.024), (40, -0.021), (41, 0.009), (42, -0.017), (43, -0.0), (44, -0.004), (45, 0.036), (46, -0.008), (47, 0.015), (48, 0.014), (49, -0.01)]
simIndex simValue blogId blogTitle
same-blog 1 0.95822686 2270 andrew gelman stats-2014-03-28-Creating a Lenin-style democracy
Introduction: Mark Palko explains why a penalty for getting the wrong answer on a test (the SAT, which is used in college admissions and which is used in the famous 8 schools example) is not a “penalty for guessing.” Then the very next day he catches this from Todd Balf in the New York Times Magazine: Students were docked one-quarter point for every multiple-choice question they got wrong, requiring a time-consuming risk analysis to determine which questions to answer and which to leave blank. Ugh! That just makes me want to . . . ok, I won’t go there. Anyway, Palko goes to the trouble to explain: While time management for a test like the SAT can be complicated, the rule for guessing is embarrassingly simple: give your best guess for questions you read; don’t waste time guessing on questions that you didn’t have time to read. The risk analysis actually becomes much more complicated when you take away the penalty for guessing. On the ACT (or the new SAT), there is a positive
2 0.81853497 94 andrew gelman stats-2010-06-17-SAT stories
Introduction: I received a bunch of interesting comments on my blog on adjusting SAT scores. Below I have a long comment from a colleague with experience in the field. But first, this hilarious (from a statistical perspective) story from Howard Wainer: Some years ago when we were visiting Harvard [as a parent of a potential student, not in Howard's role as educational researcher], an admissions director said two things of relevance (i) the SAT hasn’t got enough ‘top’ for Harvard — it doesn’t discriminate well enough at the high end. To prove this she said (ii) that Harvard had more than 1500 ‘perfect 1600s’ apply. Some were rejected. I mentioned that there were only about 750 1600s from HS seniors in the US — about 400 had 1600 in their junior year (and obviously didn’t retake) and about 350 from their senior year. So, I concluded, she must be mistaken. Then I found out that they allowed applicants to pick and choose their highest SAT-V score and their highest SAT-M score from separate adm
3 0.77872962 2202 andrew gelman stats-2014-02-07-Outrage of the week
Introduction: Mark Palko passes this one along from high school principal Carol Burris: My music teacher, Doreen, brought me her second-grade daughter’s math homework. She was already fuming over Education Secretary Arne Duncan ’s remark about why “white suburban moms” oppose the Common Core, and the homework added fuel to the fire. The problem that disturbed her the most was the following: 3. Sally did some counting. Look at her work. Explain why you think Sally counted this way. 177, 178, 179, 180, 190, 200, 210, 211, 212, 213, 214. It was on a homework sheet from the New York State Common Core Mathematics Curriculum for Grade 2, which you can find here . Doreen’s daughter had no idea how to answer this odd question. I’m with Doreen’s daughter on this one. Actually, it’s worse than that. I clicked on the link, searched on *Sally* to check that the problem was really there as stated, then I looked at the two previous problems: I can’t be sure of the answer to either of
4 0.7646094 718 andrew gelman stats-2011-05-18-Should kids be able to bring their own lunches to school?
Introduction: I encountered this news article , “Chicago school bans some lunches brought from home”: At Little Village, most students must take the meals served in the cafeteria or go hungry or both. . . . students are not allowed to pack lunches from home. Unless they have a medical excuse, they must eat the food served in the cafeteria. . . . Such discussions over school lunches and healthy eating echo a larger national debate about the role government should play in individual food choices. “This is such a fundamental infringement on parental responsibility,” said J. Justin Wilson, a senior researcher at the Washington-based Center for Consumer Freedom, which is partially funded by the food industry. . . . For many CPS parents, the idea of forbidding home-packed lunches would be unthinkable. . . . If I had read this two years ago, I’d be at one with J. Justin Wilson and the outraged kids and parents. But last year we spent a sabbatical in Paris, where . . . kids aren’t allowed to bring
Introduction: The title of this blog post quotes the second line of the abstract of Goldstein et al.’s much ballyhooed 2008 tech report, Do More Expensive Wines Taste Better? Evidence from a Large Sample of Blind Tastings . The first sentence of the abstract is Individuals who are unaware of the price do not derive more enjoyment from more expensive wine. Perhaps not surprisingly, given the easy target wine snobs make, the popular press has picked up on the first sentence of the tech report. For example, the Freakonomics blog/radio entry of the same name quotes the first line, ignores the qualification, then concludes Wishing you the happiest of holiday seasons, and urging you to spend $15 instead of $50 on your next bottle of wine. Go ahead, take the money you save and blow it on the lottery. In case you’re wondering about whether to buy me a cheap or expensive bottle of wine, keep in mind I’ve had classical “wine training”. After ten minutes of training with some side by
6 0.7567755 992 andrew gelman stats-2011-11-05-Deadwood in the math curriculum
7 0.75596619 1724 andrew gelman stats-2013-02-16-Zero Dark Thirty and Bayes’ theorem
8 0.75535929 732 andrew gelman stats-2011-05-26-What Do We Learn from Narrow Randomized Studies?
9 0.75136375 1307 andrew gelman stats-2012-05-07-The hare, the pineapple, and Ed Wegman
10 0.75003546 481 andrew gelman stats-2010-12-22-The Jumpstart financial literacy survey and the different purposes of tests
11 0.74637461 326 andrew gelman stats-2010-10-07-Peer pressure, selection, and educational reform
13 0.74169636 1265 andrew gelman stats-2012-04-15-Progress in U.S. education; also, a discussion of what it takes to hit the op-ed pages
14 0.73881537 2352 andrew gelman stats-2014-05-29-When you believe in things that you don’t understand
15 0.73743117 2216 andrew gelman stats-2014-02-18-Florida backlash
16 0.72766572 1789 andrew gelman stats-2013-04-05-Elites have alcohol problems too!
17 0.72669142 284 andrew gelman stats-2010-09-18-Continuing efforts to justify false “death panels” claim
18 0.7205016 17 andrew gelman stats-2010-05-05-Taking philosophical arguments literally
19 0.72020525 526 andrew gelman stats-2011-01-19-“If it saves the life of a single child…” and other nonsense
20 0.71631926 2053 andrew gelman stats-2013-10-06-Ideas that spread fast and slow
topicId topicWeight
[(4, 0.022), (15, 0.014), (16, 0.087), (21, 0.029), (22, 0.019), (24, 0.146), (25, 0.022), (27, 0.014), (31, 0.04), (32, 0.024), (40, 0.024), (47, 0.065), (48, 0.014), (63, 0.03), (94, 0.043), (99, 0.28)]
simIndex simValue blogId blogTitle
same-blog 1 0.97299862 2270 andrew gelman stats-2014-03-28-Creating a Lenin-style democracy
Introduction: Mark Palko explains why a penalty for getting the wrong answer on a test (the SAT, which is used in college admissions and which is used in the famous 8 schools example) is not a “penalty for guessing.” Then the very next day he catches this from Todd Balf in the New York Times Magazine: Students were docked one-quarter point for every multiple-choice question they got wrong, requiring a time-consuming risk analysis to determine which questions to answer and which to leave blank. Ugh! That just makes me want to . . . ok, I won’t go there. Anyway, Palko goes to the trouble to explain: While time management for a test like the SAT can be complicated, the rule for guessing is embarrassingly simple: give your best guess for questions you read; don’t waste time guessing on questions that you didn’t have time to read. The risk analysis actually becomes much more complicated when you take away the penalty for guessing. On the ACT (or the new SAT), there is a positive
Introduction: Remember How to Lie With Statistics? It turns out that the author worked for the cigarette companies. John Mashey points to this, from Robert Proctor’s book, “Golden Holocaust: Origins of the Cigarette Catastrophe and the Case for Abolition”: Darrell Huff, author of the wildly popular (and aptly named) How to Lie With Statistics, was paid to testify before Congress in the 1950s and then again in the 1960s, with the assigned task of ridiculing any notion of a cigarette-disease link. On March 22, 1965, Huff testified at hearings on cigarette labeling and advertising, accusing the recent Surgeon General’s report of myriad failures and “fallacies.” Huff peppered his attack with with amusing asides and anecdotes, lampooning spurious correlations like that between the size of Dutch families and the number of storks nesting on rooftops–which proves not that storks bring babies but rather that people with large families tend to have larger houses (which therefore attract more storks).
3 0.96268606 1218 andrew gelman stats-2012-03-18-Check your missing-data imputations using cross-validation
Introduction: Elena Grewal writes: I am currently using the iterative regression imputation model as implemented in the Stata ICE package. I am using data from a survey of about 90,000 students in 142 schools and my variable of interest is parent level of education. I want only this variable to be imputed with as little bias as possible as I am not using any other variable. So I scoured the survey for every variable I thought could possibly predict parent education. The main variable I found is parent occupation, which explains about 35% of the variance in parent education for the students with complete data on both. I then include the 20 other variables I found in the survey in a regression predicting parent education, which explains about 40% of the variance in parent education for students with complete data on all the variables. My question is this: many of the other variables I found have more missing values than the parent education variable, and also, although statistically significant
4 0.95656192 2183 andrew gelman stats-2014-01-23-Discussion on preregistration of research studies
Introduction: Chris Chambers and I had an enlightening discussion the other day at the blog of Rolf Zwaan, regarding the Garden of Forking Paths ( go here and scroll down through the comments). Chris sent me the following note: I’m writing a book at the moment about reforming practices in psychological research (focusing on various bad practices such as p-hacking, HARKing, low statistical power, publication bias, lack of data sharing etc. – and posing solutions such as pre-registration, Bayesian hypothesis testing, mandatory data archiving etc.) and I am arriving at rather unsettling conclusion: that null hypothesis significance testing (NHST) simply isn’t valid for observational research. If this is true then most of the psychological literature is statistically flawed. I was wonder what your thoughts were on this, both from a statistical point of view and from your experience working in an observational field. We all know about the dangers of researcher degrees of freedom. We also know
5 0.95394254 1730 andrew gelman stats-2013-02-20-Unz on Unz
Introduction: Last week I posted skeptical remarks about Ron Unz’s claim that Harvard admissions discriminate in favor of Jews. The comment thread was getting long enough there that I thought it most fair to give Unz a chance to present his thoughts here as a new post. I’ve done that before in cases where I’ve disagreed with someone and he wanted to make his views clear. I will post Unz’s email and my brief response. This is what Unz wrote to me: Since there’s been a great deal of dispute over the numerator and the denominator, it might be useful for each of us should provide our own estimate-range of what we believe are the true figures, and the justification. Perhaps if our ranges actually overlap substantially, then we don’t really disagree much after all. I’d think if you’ve been reading most of the endless comments and refreshing your memory about my claims, you’ve probably now developed your own mental model about the likely reality of the values whereas initially you may have simpl
6 0.95154911 1897 andrew gelman stats-2013-06-13-When’s that next gamma-ray blast gonna come, already?
7 0.95095944 1760 andrew gelman stats-2013-03-12-Misunderstanding the p-value
8 0.9504509 548 andrew gelman stats-2011-02-01-What goes around . . .
9 0.94894874 1261 andrew gelman stats-2012-04-12-The Naval Research Lab
10 0.94827843 716 andrew gelman stats-2011-05-17-Is the internet causing half the rapes in Norway? I wanna see the scatterplot.
12 0.94722694 1450 andrew gelman stats-2012-08-08-My upcoming talk for the data visualization meetup
14 0.94707042 2266 andrew gelman stats-2014-03-25-A statistical graphics course and statistical graphics advice
15 0.94668305 586 andrew gelman stats-2011-02-23-A statistical version of Arrow’s paradox
16 0.94667423 291 andrew gelman stats-2010-09-22-Philosophy of Bayes and non-Bayes: A dialogue with Deborah Mayo
17 0.9457978 18 andrew gelman stats-2010-05-06-$63,000 worth of abusive research . . . or just a really stupid waste of time?
19 0.94507635 807 andrew gelman stats-2011-07-17-Macro causality
20 0.94501776 94 andrew gelman stats-2010-06-17-SAT stories