andrew_gelman_stats andrew_gelman_stats-2010 andrew_gelman_stats-2010-95 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Several years ago, I heard about a project at the Educational Testing Service to identify “strivers”: students from disadvantaged backgrounds who did unexpectedly well on the SAT (the college admissions exam formerly known as the “Scholastic Aptitude Test” but apparently now just “the SAT,” in the same way that Exxon is just “Exxon” and that Harry Truman’s middle name is just “S”), at least 200 points above a predicted score based on demographic and neighborhood information. My ETS colleague and I agreed that this was a silly idea: From a statistical point of view, if student A is expected ahead of time to do better than student B, and then they get identical test scores, then you’d expect student A (the non-”striver”) to do better than student B (the “striver”) later on. Just basic statistics: if a student does much better than expected, then probably some of that improvement is noise. The idea of identifying these “strivers” seemed misguided and not the best use of the SAT.
sentIndex sentText sentNum sentScore
1 Just basic statistics: if a student does much better than expected, then probably some of that improvement is noise. [sent-3, score-0.134]
2 The idea of identifying these “strivers” seemed misguided and not the best use of the SAT. [sent-4, score-0.088]
3 But then I thought that maybe I was being unfair: Even if there are statistical problems with the original idea, there may be some policy benefits that I was missing. [sent-7, score-0.071]
4 I sent off a request for a review copy of the book, warning the book’s editor, Richard Kahlenberg, ahead of time that I was likely to be critical, given my earlier discussions on the topic, but that the review would be thoughtful in any case. [sent-8, score-0.14]
5 To Kahlenberg’s credit, he sent me a copy anyway. [sent-9, score-0.073]
6 So I looked up “strivers” in the index, and this is all they’ve got: Strivers: definition of, 10, 174; ETS project to identify, 10-11 They seem to be downplaying the original “strivers” idea in favor of a more general approach of affirmative action for disadvantaged students. [sent-11, score-0.464]
7 The plan seems to be to use a regression analysis on SAT scores, identify the background variables (including family income, parents’ education and occupation, neighborhood characteristics, and ethnicity) that predict SAT scores, and then give points for having a low predicted value. [sent-12, score-0.469]
8 Universities and colleagues should customize information about the various obstacles faced by applicants in the context of their own applicant pools and their own standards of readiness, based on past experience. [sent-14, score-0.424]
9 In practice, I’m not so sure how this differs from what colleges already do–although I don’t know how a large university is going to have the resources to “assess the merits of an individual applicant. [sent-15, score-0.098]
10 ” Actually, I don’t know how a small college can do that either. [sent-16, score-0.11]
11 I guess they could have a checklist of “obstacles faced by applicants,” but I don’t know how they’d measure “readiness” other than through test scores and school grades. [sent-17, score-0.376]
12 To get back to the original “strivers” idea, which seems to have receded into the background, I can see the appeal of giving extra points to applicants with disadvantaged backgrounds. [sent-18, score-0.568]
13 But I don’t see that it makes sense to have any discrete cutoff–for example, getting extra credit for having a predicted score more than 200 points below your actual SAT score. [sent-19, score-0.417]
14 I think it would be more reasonable just to openly give points for being from a low-income family, having non-college-educated parents, etc. [sent-20, score-0.103]
15 Some interesting data from the report Postsecondary destination of high-scoring students, by socioeconomic status: Among the top-scoring quartile of students on the SAT: 80% of high-SES students went on to a four-year college, compared to only 44% of low-SES students. [sent-21, score-0.449]
16 Among the top-scoring students who do go to college, 80% of the high-SES kids graduate, compared to 45% of the kids of low socio-economic status. [sent-22, score-0.243]
wordName wordTfidf (topN-words)
[('strivers', 0.557), ('sat', 0.253), ('kahlenberg', 0.197), ('disadvantaged', 0.186), ('ets', 0.144), ('striver', 0.144), ('scores', 0.142), ('applicants', 0.136), ('student', 0.134), ('exxon', 0.131), ('obstacles', 0.131), ('readiness', 0.131), ('college', 0.11), ('students', 0.109), ('socioeconomic', 0.108), ('predicted', 0.105), ('points', 0.103), ('identify', 0.099), ('merits', 0.098), ('neighborhood', 0.097), ('faced', 0.095), ('idea', 0.088), ('universities', 0.078), ('test', 0.073), ('copy', 0.073), ('extra', 0.072), ('original', 0.071), ('parents', 0.07), ('status', 0.07), ('richard', 0.07), ('score', 0.07), ('kids', 0.067), ('ahead', 0.067), ('credit', 0.067), ('checklist', 0.066), ('quartile', 0.066), ('truman', 0.066), ('family', 0.065), ('customize', 0.062), ('aptitude', 0.062), ('downplaying', 0.062), ('scholastic', 0.062), ('rewarding', 0.059), ('formerly', 0.059), ('harry', 0.057), ('affirmative', 0.057), ('destination', 0.057), ('occupation', 0.057), ('subtracting', 0.057), ('unexpectedly', 0.057)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 95 andrew gelman stats-2010-06-17-“Rewarding Strivers: Helping Low-Income Students Succeed in College”
Introduction: Several years ago, I heard about a project at the Educational Testing Service to identify “strivers”: students from disadvantaged backgrounds who did unexpectedly well on the SAT (the college admissions exam formerly known as the “Scholastic Aptitude Test” but apparently now just “the SAT,” in the same way that Exxon is just “Exxon” and that Harry Truman’s middle name is just “S”), at least 200 points above a predicted score based on demographic and neighborhood information. My ETS colleague and I agreed that this was a silly idea: From a statistical point of view, if student A is expected ahead of time to do better than student B, and then they get identical test scores, then you’d expect student A (the non-”striver”) to do better than student B (the “striver”) later on. Just basic statistics: if a student does much better than expected, then probably some of that improvement is noise. The idea of identifying these “strivers” seemed misguided and not the best use of the SAT.
2 0.2905688 93 andrew gelman stats-2010-06-17-My proposal for making college admissions fairer
Introduction: After reading the Rewarding Strivers book , I had some thoughts about how to make the college admissions system more fair to students from varying socioeconomic backgrounds. Instead of boosting up the disadvantaged students, why not pull down the advantaged students? Here’s the idea. Disadvantaged students are defined typically not by a bad thing that they have, but rather by good things that they don’t have: financial resources, a high-quality education, and so forth. In contrast, advantaged students get all sorts of freebies. So here are my suggestions: 1. All high school grades on a 4-point scale (A=4, B=3, etc). No more of this 5-points-for-an-A-in-an-AP course, which gives the ridiculous outcomes of kids graduating with a 4.3 average, not so fair to kids in schools that don’t offer a lot of AP classes. 2. Subtract points for taking the SAT multiple times. A simple rule would be: You can use your highest SAT score, but you lose 50 points for every other time
3 0.17068096 94 andrew gelman stats-2010-06-17-SAT stories
Introduction: I received a bunch of interesting comments on my blog on adjusting SAT scores. Below I have a long comment from a colleague with experience in the field. But first, this hilarious (from a statistical perspective) story from Howard Wainer: Some years ago when we were visiting Harvard [as a parent of a potential student, not in Howard's role as educational researcher], an admissions director said two things of relevance (i) the SAT hasn’t got enough ‘top’ for Harvard — it doesn’t discriminate well enough at the high end. To prove this she said (ii) that Harvard had more than 1500 ‘perfect 1600s’ apply. Some were rejected. I mentioned that there were only about 750 1600s from HS seniors in the US — about 400 had 1600 in their junior year (and obviously didn’t retake) and about 350 from their senior year. So, I concluded, she must be mistaken. Then I found out that they allowed applicants to pick and choose their highest SAT-V score and their highest SAT-M score from separate adm
4 0.15157115 1688 andrew gelman stats-2013-01-22-That claim that students whose parents pay for more of college get worse grades
Introduction: Theodore Vasiloudis writes: I came upon this article by Laura Hamilton, an assistant professor in the University of California at Merced, that claims that “The more money that parents provide for higher education, the lower the grades their children earn.” I can’t help but feel that there something wrong with the basis of the study or a confounding factor causing this apparent correlation, and since you often comment on studies on your blog I thought you might find this study interesting. My reply: I have to admit that the description above made me suspicious of the study before I even looked at it. On first thought, I’d expect the effect of parent’s financial contributions to be positive (as they free the student from the need to get a job during college), but not negative. Hamilton argues that “parental investments create a disincentive for student achievement,” which may be—but I’m generally suspicious of arguments in which the rebound is bigger than the main effect.
5 0.12113845 2270 andrew gelman stats-2014-03-28-Creating a Lenin-style democracy
Introduction: Mark Palko explains why a penalty for getting the wrong answer on a test (the SAT, which is used in college admissions and which is used in the famous 8 schools example) is not a “penalty for guessing.” Then the very next day he catches this from Todd Balf in the New York Times Magazine: Students were docked one-quarter point for every multiple-choice question they got wrong, requiring a time-consuming risk analysis to determine which questions to answer and which to leave blank. Ugh! That just makes me want to . . . ok, I won’t go there. Anyway, Palko goes to the trouble to explain: While time management for a test like the SAT can be complicated, the rule for guessing is embarrassingly simple: give your best guess for questions you read; don’t waste time guessing on questions that you didn’t have time to read. The risk analysis actually becomes much more complicated when you take away the penalty for guessing. On the ACT (or the new SAT), there is a positive
6 0.12067655 315 andrew gelman stats-2010-10-03-He doesn’t trust the fit . . . r=.999
7 0.11523038 853 andrew gelman stats-2011-08-14-Preferential admissions for children of elite colleges
8 0.11499738 261 andrew gelman stats-2010-09-07-The $900 kindergarten teacher
9 0.11386818 943 andrew gelman stats-2011-10-04-Flip it around
10 0.1047764 1980 andrew gelman stats-2013-08-13-Test scores and grades predict job performance (but maybe not at Google)
11 0.095802389 1517 andrew gelman stats-2012-10-01-“On Inspiring Students and Being Human”
12 0.093634464 462 andrew gelman stats-2010-12-10-Who’s holding the pen?, The split screen, and other ideas for one-on-one instruction
13 0.093137965 1265 andrew gelman stats-2012-04-15-Progress in U.S. education; also, a discussion of what it takes to hit the op-ed pages
14 0.091769986 1595 andrew gelman stats-2012-11-28-Should Harvard start admitting kids at random?
15 0.091010973 606 andrew gelman stats-2011-03-10-It’s no fun being graded on a curve
16 0.090025932 1271 andrew gelman stats-2012-04-20-Education could use some systematic evaluation
17 0.089577138 226 andrew gelman stats-2010-08-23-More on those L.A. Times estimates of teacher effectiveness
18 0.088407412 609 andrew gelman stats-2011-03-13-Coauthorship norms
19 0.087907806 71 andrew gelman stats-2010-06-07-Pay for an A?
20 0.085068978 836 andrew gelman stats-2011-08-03-Another plagiarism mystery
topicId topicWeight
[(0, 0.157), (1, -0.049), (2, 0.002), (3, -0.02), (4, 0.051), (5, 0.09), (6, 0.045), (7, 0.085), (8, -0.025), (9, -0.001), (10, 0.018), (11, 0.111), (12, -0.024), (13, -0.02), (14, 0.016), (15, -0.01), (16, 0.062), (17, 0.027), (18, -0.032), (19, -0.024), (20, -0.006), (21, 0.054), (22, 0.018), (23, -0.012), (24, 0.052), (25, -0.031), (26, 0.028), (27, -0.003), (28, -0.018), (29, -0.011), (30, -0.018), (31, 0.014), (32, 0.055), (33, 0.038), (34, -0.001), (35, 0.038), (36, 0.033), (37, -0.029), (38, -0.009), (39, -0.0), (40, -0.016), (41, 0.017), (42, 0.035), (43, 0.01), (44, -0.012), (45, -0.011), (46, 0.023), (47, -0.044), (48, 0.001), (49, 0.043)]
simIndex simValue blogId blogTitle
same-blog 1 0.95970768 95 andrew gelman stats-2010-06-17-“Rewarding Strivers: Helping Low-Income Students Succeed in College”
Introduction: Several years ago, I heard about a project at the Educational Testing Service to identify “strivers”: students from disadvantaged backgrounds who did unexpectedly well on the SAT (the college admissions exam formerly known as the “Scholastic Aptitude Test” but apparently now just “the SAT,” in the same way that Exxon is just “Exxon” and that Harry Truman’s middle name is just “S”), at least 200 points above a predicted score based on demographic and neighborhood information. My ETS colleague and I agreed that this was a silly idea: From a statistical point of view, if student A is expected ahead of time to do better than student B, and then they get identical test scores, then you’d expect student A (the non-”striver”) to do better than student B (the “striver”) later on. Just basic statistics: if a student does much better than expected, then probably some of that improvement is noise. The idea of identifying these “strivers” seemed misguided and not the best use of the SAT.
2 0.89267182 93 andrew gelman stats-2010-06-17-My proposal for making college admissions fairer
Introduction: After reading the Rewarding Strivers book , I had some thoughts about how to make the college admissions system more fair to students from varying socioeconomic backgrounds. Instead of boosting up the disadvantaged students, why not pull down the advantaged students? Here’s the idea. Disadvantaged students are defined typically not by a bad thing that they have, but rather by good things that they don’t have: financial resources, a high-quality education, and so forth. In contrast, advantaged students get all sorts of freebies. So here are my suggestions: 1. All high school grades on a 4-point scale (A=4, B=3, etc). No more of this 5-points-for-an-A-in-an-AP course, which gives the ridiculous outcomes of kids graduating with a 4.3 average, not so fair to kids in schools that don’t offer a lot of AP classes. 2. Subtract points for taking the SAT multiple times. A simple rule would be: You can use your highest SAT score, but you lose 50 points for every other time
Introduction: Howard Wainer writes : When we focus only on the differences between groups, we too easily lose track of the big picture. Nowhere is this more obvious than in the current public discussions of the size of the gap in test scores that is observed between racial groups. It has been noted that in New Jersey the gap between the average scores of white and black students on the well-developed scale of the National Assessment of Educational Progress (NAEP) has shrunk by only about 25 percent over the past two decades. The conclusion drawn was that even though the change is in the right direction, it is far too slow. But focusing on the difference blinds us to what has been a remarkable success in education over the past 20 years. Although the direction and size of student improvements are considered across many subject areas and many age groups, I will describe just one — 4th grade mathematics. . . . there have been steep gains for both racial groups over this period (somewhat steeper g
4 0.81961775 1688 andrew gelman stats-2013-01-22-That claim that students whose parents pay for more of college get worse grades
Introduction: Theodore Vasiloudis writes: I came upon this article by Laura Hamilton, an assistant professor in the University of California at Merced, that claims that “The more money that parents provide for higher education, the lower the grades their children earn.” I can’t help but feel that there something wrong with the basis of the study or a confounding factor causing this apparent correlation, and since you often comment on studies on your blog I thought you might find this study interesting. My reply: I have to admit that the description above made me suspicious of the study before I even looked at it. On first thought, I’d expect the effect of parent’s financial contributions to be positive (as they free the student from the need to get a job during college), but not negative. Hamilton argues that “parental investments create a disincentive for student achievement,” which may be—but I’m generally suspicious of arguments in which the rebound is bigger than the main effect.
5 0.81453747 71 andrew gelman stats-2010-06-07-Pay for an A?
Introduction: Judah Guber writes about his new company : What we have done with Ultrinsic is created a system of incentives for students to allow them to invest in their ability to achieve a certain grade and when they achieve that grade we reward them with a cash incentive on top of receiving their original investment. This helps remove one of the large barriers students have to studying and staying motivated over the course of long semesters of college by giving them rewards on a much more immediate basis. We have been doing a pilot program in 2 schools, NYU and Penn, for the past year or so, and are currently in the process of a major roll out of our services to 37 schools all across the country. This is due to our popularity and inquiries from students in tons of schools all around the country regarding getting Ultrinsic’s services in their school. In the Fall 2010 semester, Ultrinsic will be revolutionizing student motivation on a grand scale . This is the dream of many economists: to c
6 0.81357634 606 andrew gelman stats-2011-03-10-It’s no fun being graded on a curve
7 0.78093916 825 andrew gelman stats-2011-07-27-Grade inflation: why weren’t the instructors all giving all A’s already??
8 0.77874422 326 andrew gelman stats-2010-10-07-Peer pressure, selection, and educational reform
9 0.76771754 1507 andrew gelman stats-2012-09-22-Grade inflation: why weren’t the instructors all giving all A’s already??
10 0.76336718 542 andrew gelman stats-2011-01-28-Homework and treatment levels
11 0.75750482 617 andrew gelman stats-2011-03-17-“Why Preschool Shouldn’t Be Like School”?
12 0.75635588 462 andrew gelman stats-2010-12-10-Who’s holding the pen?, The split screen, and other ideas for one-on-one instruction
13 0.75592166 1803 andrew gelman stats-2013-04-14-Why girls do better in school
14 0.75563252 94 andrew gelman stats-2010-06-17-SAT stories
15 0.73468828 261 andrew gelman stats-2010-09-07-The $900 kindergarten teacher
16 0.72935927 956 andrew gelman stats-2011-10-13-Hey, you! Don’t take that class!
17 0.71414828 1350 andrew gelman stats-2012-05-28-Value-added assessment: What went wrong?
18 0.70828772 529 andrew gelman stats-2011-01-21-“City Opens Inquiry on Grading Practices at a Top-Scoring Bronx School”
19 0.70455861 718 andrew gelman stats-2011-05-18-Should kids be able to bring their own lunches to school?
20 0.69976908 1620 andrew gelman stats-2012-12-12-“Teaching effectiveness” as another dimension in cognitive ability
topicId topicWeight
[(2, 0.016), (9, 0.037), (15, 0.047), (16, 0.087), (24, 0.091), (31, 0.017), (45, 0.01), (47, 0.251), (59, 0.018), (77, 0.015), (86, 0.023), (95, 0.018), (99, 0.237)]
simIndex simValue blogId blogTitle
1 0.93604183 275 andrew gelman stats-2010-09-14-Data visualization at the American Evaluation Association
Introduction: Stephanie Evergreen writes: Media, web design, and marketing have all created an environment where stakeholders – clients, program participants, funders – all expect high quality graphics and reporting that effectively conveys the valuable insights from evaluation work. Some in statistics and mathematics have used data visualization strategies to support more useful reporting of complex ideas. Global growing interest in improving communications has begun to take root in the evaluation field as well. But as anyone who has sat through a day’s worth of a conference or had to endure a dissertation-worthy evaluation report knows, evaluators still have a long way to go. To support the development of researchers and evaluators, some members of the American Evaluation Association are proposing a new TIG (Topical Interest Group) on Data Visualization and Reporting. If you are a member of AEA (or want to be) and you are interested in joining this TIG, contact Stephanie Evergreen.
Introduction: Adam Marcus at Retraction Watch reports on a physicist at the University of Toronto who had this unfortunate thing happen to him: This article has been retracted at the request of the Editor-in-Chief and first and corresponding author. The article was largely a duplication of a paper that had already appeared in ACS Nano, 4 (2010) 3374–3380, http://dx.doi.org/10.1021/nn100335g. The first and the corresponding authors (Kramer and Sargent) would like to apologize for this administrative error on their part . . . “Administrative error” . . . I love that! Is that what the robber says when he knocks over a liquor store and gets caught? As Marcus points out, the two papers have different titles and a different order of authors, which makes it less plausible that this was an administrative mistake (as could happen, for example, if a secretary was given a list of journals to submit the paper to, and accidentally submitted it to the second journal on the list without realizing it
3 0.89674205 1055 andrew gelman stats-2011-12-13-Data sharing update
Introduction: Fred Oswald reports that Sian Beilock sent him sufficient amounts of raw data from her research study so allow him to answer his questions about the large effects that were observed. This sort of collegiality is central to the collective scientific enterprise. The bad news is that IRB’s are still getting in the way. Beilock was very helpful but she had to work within the constraints of her IRB, which apparently advised her not to share data—even if de-identified—without getting lots more permissions. Oswald writes: It is a little concerning that the IRB bars the sharing of de-identified data, particularly in light of the specific guidelines of the journal Science, which appears to say that when you submit a study to the journal for publication, you are allowing for the sharing of de-identified data — unless you expressly say otherwise at the point that you submit the paper for consideration. Again, I don’t blame Beilock and Ramirez—they appear to have been as helpful as
same-blog 4 0.89340484 95 andrew gelman stats-2010-06-17-“Rewarding Strivers: Helping Low-Income Students Succeed in College”
Introduction: Several years ago, I heard about a project at the Educational Testing Service to identify “strivers”: students from disadvantaged backgrounds who did unexpectedly well on the SAT (the college admissions exam formerly known as the “Scholastic Aptitude Test” but apparently now just “the SAT,” in the same way that Exxon is just “Exxon” and that Harry Truman’s middle name is just “S”), at least 200 points above a predicted score based on demographic and neighborhood information. My ETS colleague and I agreed that this was a silly idea: From a statistical point of view, if student A is expected ahead of time to do better than student B, and then they get identical test scores, then you’d expect student A (the non-”striver”) to do better than student B (the “striver”) later on. Just basic statistics: if a student does much better than expected, then probably some of that improvement is noise. The idea of identifying these “strivers” seemed misguided and not the best use of the SAT.
Introduction: Remember How to Lie With Statistics? It turns out that the author worked for the cigarette companies. John Mashey points to this, from Robert Proctor’s book, “Golden Holocaust: Origins of the Cigarette Catastrophe and the Case for Abolition”: Darrell Huff, author of the wildly popular (and aptly named) How to Lie With Statistics, was paid to testify before Congress in the 1950s and then again in the 1960s, with the assigned task of ridiculing any notion of a cigarette-disease link. On March 22, 1965, Huff testified at hearings on cigarette labeling and advertising, accusing the recent Surgeon General’s report of myriad failures and “fallacies.” Huff peppered his attack with with amusing asides and anecdotes, lampooning spurious correlations like that between the size of Dutch families and the number of storks nesting on rooftops–which proves not that storks bring babies but rather that people with large families tend to have larger houses (which therefore attract more storks).
6 0.84826648 2275 andrew gelman stats-2014-03-31-Just gave a talk
7 0.84740496 1143 andrew gelman stats-2012-01-29-G+ > Skype
8 0.84732181 1050 andrew gelman stats-2011-12-10-Presenting at the econ seminar
9 0.84661758 1261 andrew gelman stats-2012-04-12-The Naval Research Lab
10 0.83152354 1668 andrew gelman stats-2013-01-11-My talk at the NY data visualization meetup this Monday!
11 0.82402629 2131 andrew gelman stats-2013-12-12-My talk at Leuven, Sat 14 Dec
12 0.8151443 1897 andrew gelman stats-2013-06-13-When’s that next gamma-ray blast gonna come, already?
13 0.81383353 716 andrew gelman stats-2011-05-17-Is the internet causing half the rapes in Norway? I wanna see the scatterplot.
14 0.80670667 548 andrew gelman stats-2011-02-01-What goes around . . .
15 0.79945809 1273 andrew gelman stats-2012-04-20-Proposals for alternative review systems for scientific work
17 0.79398483 1730 andrew gelman stats-2013-02-20-Unz on Unz
18 0.7910856 438 andrew gelman stats-2010-11-30-I just skyped in from Kentucky, and boy are my arms tired
19 0.79054713 602 andrew gelman stats-2011-03-06-Assumptions vs. conditions
20 0.79000831 2290 andrew gelman stats-2014-04-14-On deck this week