andrew_gelman_stats andrew_gelman_stats-2013 andrew_gelman_stats-2013-2083 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Just in time for Halloween, here’s a horror story for you . . . Howard Wainer writes: In my book “Uneducated Guesses” in the chapter on value-added models, I discuss how the treatment of missing data can have a profound effect on the estimates of teacher scores. I made up how a principal might send the best students on a field trip at the beginning of the year when the ‘pre-test’ was given (and their scores would be imputed from the students who showed up) and that the bottom half of the class would have a matching field trip on the day of the post test. Everyone laughed. But apparently someone decided to take it seriously. http://www.amren.com/news/2012/10/el-paso-schools-confront-scandal-of-students-who-disappeared-at-test-time/ http://www.elpasotimes.com/episd/ci_20848628/former-episd-superintendent-lorenzo-garcia-enter-plea-aggreement You can’t make this stuff up. This sort of thing is not surprising but it’s worth keeping in mind. That a measurement system c
sentIndex sentText sentNum sentScore
1 Just in time for Halloween, here’s a horror story for you . [sent-1, score-0.261]
2 Howard Wainer writes: In my book “Uneducated Guesses” in the chapter on value-added models, I discuss how the treatment of missing data can have a profound effect on the estimates of teacher scores. [sent-4, score-0.914]
3 I made up how a principal might send the best students on a field trip at the beginning of the year when the ‘pre-test’ was given (and their scores would be imputed from the students who showed up) and that the bottom half of the class would have a matching field trip on the day of the post test. [sent-5, score-2.903]
4 But apparently someone decided to take it seriously. [sent-7, score-0.294]
5 This sort of thing is not surprising but it’s worth keeping in mind. [sent-13, score-0.353]
6 That a measurement system can be gamed, does not mean it’s useless, but part of good measurement is to consider these problems. [sent-14, score-0.749]
wordName wordTfidf (topN-words)
[('trip', 0.334), ('uneducated', 0.233), ('gamed', 0.233), ('measurement', 0.23), ('http', 0.21), ('horror', 0.197), ('guesses', 0.188), ('profound', 0.184), ('field', 0.183), ('halloween', 0.177), ('imputed', 0.174), ('wainer', 0.172), ('principal', 0.161), ('howard', 0.159), ('students', 0.155), ('matching', 0.146), ('useless', 0.142), ('keeping', 0.135), ('teacher', 0.135), ('surprising', 0.132), ('beginning', 0.129), ('bottom', 0.128), ('showed', 0.127), ('scores', 0.126), ('decided', 0.115), ('apparently', 0.113), ('send', 0.106), ('half', 0.104), ('treatment', 0.099), ('chapter', 0.099), ('missing', 0.098), ('everyone', 0.094), ('stuff', 0.09), ('discuss', 0.089), ('system', 0.088), ('class', 0.086), ('worth', 0.086), ('estimates', 0.08), ('day', 0.077), ('year', 0.072), ('consider', 0.072), ('effect', 0.069), ('mean', 0.066), ('someone', 0.066), ('story', 0.064), ('problems', 0.064), ('part', 0.063), ('best', 0.063), ('book', 0.061), ('post', 0.06)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 2083 andrew gelman stats-2013-10-31-Value-added modeling in education: Gaming the system by sending kids on a field trip at test time
Introduction: Just in time for Halloween, here’s a horror story for you . . . Howard Wainer writes: In my book “Uneducated Guesses” in the chapter on value-added models, I discuss how the treatment of missing data can have a profound effect on the estimates of teacher scores. I made up how a principal might send the best students on a field trip at the beginning of the year when the ‘pre-test’ was given (and their scores would be imputed from the students who showed up) and that the bottom half of the class would have a matching field trip on the day of the post test. Everyone laughed. But apparently someone decided to take it seriously. http://www.amren.com/news/2012/10/el-paso-schools-confront-scandal-of-students-who-disappeared-at-test-time/ http://www.elpasotimes.com/episd/ci_20848628/former-episd-superintendent-lorenzo-garcia-enter-plea-aggreement You can’t make this stuff up. This sort of thing is not surprising but it’s worth keeping in mind. That a measurement system c
2 0.13333076 799 andrew gelman stats-2011-07-13-Hypothesis testing with multiple imputations
Introduction: Vincent Yip writes: I have read your paper [with Kobi Abayomi and Marc Levy] regarding multiple imputation application. In order to diagnostic my imputed data, I used Kolmogorov-Smirnov (K-S) tests to compare the distribution differences between the imputed and observed values of a single attribute as mentioned in your paper. My question is: For example I have this attribute X with the following data: (NA = missing) Original dataset: 1, NA, 3, 4, 1, 5, NA Imputed dataset: 1, 2 , 3, 4, 1, 5, 6 a) in order to run the KS test, will I treat the observed data as 1, 3, 4,1, 5? b) and for the observed data, will I treat 1, 2 , 3, 4, 1, 5, 6 as the imputed dataset for the K-S test? or just 2 ,6? c) if I used m=5, I will have 5 set of imputed data sets. How would I apply K-S test to 5 of them and compare to the single observed distribution? Do I combine the 5 imputed data set into one by averaging each imputed values so I get one single imputed data and compare with the ob
3 0.12920161 935 andrew gelman stats-2011-10-01-When should you worry about imputed data?
Introduction: Majid Ezzati writes: My research group is increasingly focusing on a series of problems that involve data that either have missingness or measurements that may have bias/error. We have at times developed our own approaches to imputation (as simple as interpolating a missing unit and as sophisticated as a problem-specific Bayesian hierarchical model) and at other times, other groups impute the data. The outputs are being used to investigate the basic associations between pairs of variables, Xs and Ys, in regressions; we may or may not interpret these as causal. I am contacting colleagues with relevant expertise to suggest good references on whether having imputed X and/or Y in a subsequent regression is correct or if it could somehow lead to biased/spurious associations. Thinking about this, we can have at least the following situations (these could all be Bayesian or not): 1) X and Y both measured (perhaps with error) 2) Y imputed using some data and a model and X measur
4 0.12084465 796 andrew gelman stats-2011-07-10-Matching and regression: two great tastes etc etc
Introduction: Matthew Bogard writes: Regarding the book Mostly Harmless Econometrics, you state : A casual reader of the book might be left with the unfortunate impression that matching is a competitor to regression rather than a tool for making regression more effective. But in fact isn’t that what they are arguing, that, in a ‘mostly harmless way’ regression is in fact a matching estimator itself? “Our view is that regression can be motivated as a particular sort of weighted matching estimator, and therefore the differences between regression and matching estimates are unlikely to be of major empirical importance” (Chapter 3 p. 70) They seem to be distinguishing regression (without prior matching) from all other types of matching techniques, and therefore implying that regression can be a ‘mostly harmless’ substitute or competitor to matching. My previous understanding, before starting this book was as you say, that matching is a tool that makes regression more effective. I have n
5 0.11949629 606 andrew gelman stats-2011-03-10-It’s no fun being graded on a curve
Introduction: Mark Palko points to a news article by Michael Winerip on teacher assessment: No one at the Lab Middle School for Collaborative Studies works harder than Stacey Isaacson, a seventh-grade English and social studies teacher. She is out the door of her Queens home by 6:15 a.m., takes the E train into Manhattan and is standing out front when the school doors are unlocked, at 7. Nights, she leaves her classroom at 5:30. . . . Her principal, Megan Adams, has given her terrific reviews during the two and a half years Ms. Isaacson has been a teacher. . . . The Lab School has selective admissions, and Ms. Isaacson’s students have excelled. Her first year teaching, 65 of 66 scored proficient on the state language arts test, meaning they got 3′s or 4′s; only one scored below grade level with a 2. More than two dozen students from her first two years teaching have gone on to . . . the city’s most competitive high schools. . . . You would think the Department of Education would want to r
6 0.11700442 375 andrew gelman stats-2010-10-28-Matching for preprocessing data for causal inference
7 0.11583018 1350 andrew gelman stats-2012-05-28-Value-added assessment: What went wrong?
8 0.11301627 226 andrew gelman stats-2010-08-23-More on those L.A. Times estimates of teacher effectiveness
9 0.11111841 1517 andrew gelman stats-2012-10-01-“On Inspiring Students and Being Human”
11 0.10811468 894 andrew gelman stats-2011-09-07-Hipmunk FAIL: Graphics without content is not enough
12 0.10518326 1118 andrew gelman stats-2012-01-14-A model rejection letter
13 0.1028197 1380 andrew gelman stats-2012-06-15-Coaching, teaching, and writing
15 0.098865643 700 andrew gelman stats-2011-05-06-Suspicious pattern of too-strong replications of medical research
16 0.098493621 2286 andrew gelman stats-2014-04-08-Understanding Simpson’s paradox using a graph
17 0.097809456 6 andrew gelman stats-2010-04-27-Jelte Wicherts lays down the stats on IQ
18 0.094164759 1265 andrew gelman stats-2012-04-15-Progress in U.S. education; also, a discussion of what it takes to hit the op-ed pages
19 0.092067897 995 andrew gelman stats-2011-11-06-Statistical models and actual models
20 0.091792621 609 andrew gelman stats-2011-03-13-Coauthorship norms
topicId topicWeight
[(0, 0.139), (1, -0.026), (2, -0.02), (3, -0.039), (4, 0.069), (5, 0.089), (6, 0.036), (7, 0.07), (8, 0.018), (9, 0.009), (10, 0.035), (11, 0.051), (12, 0.013), (13, -0.061), (14, 0.048), (15, -0.013), (16, 0.015), (17, 0.027), (18, -0.049), (19, 0.024), (20, -0.0), (21, 0.022), (22, 0.013), (23, -0.016), (24, 0.019), (25, 0.029), (26, 0.022), (27, 0.049), (28, 0.01), (29, 0.024), (30, -0.006), (31, -0.0), (32, 0.029), (33, -0.001), (34, 0.004), (35, 0.009), (36, -0.022), (37, -0.051), (38, 0.032), (39, 0.032), (40, -0.012), (41, -0.008), (42, 0.004), (43, -0.032), (44, 0.035), (45, 0.046), (46, 0.038), (47, -0.044), (48, -0.041), (49, -0.006)]
simIndex simValue blogId blogTitle
same-blog 1 0.96777016 2083 andrew gelman stats-2013-10-31-Value-added modeling in education: Gaming the system by sending kids on a field trip at test time
Introduction: Just in time for Halloween, here’s a horror story for you . . . Howard Wainer writes: In my book “Uneducated Guesses” in the chapter on value-added models, I discuss how the treatment of missing data can have a profound effect on the estimates of teacher scores. I made up how a principal might send the best students on a field trip at the beginning of the year when the ‘pre-test’ was given (and their scores would be imputed from the students who showed up) and that the bottom half of the class would have a matching field trip on the day of the post test. Everyone laughed. But apparently someone decided to take it seriously. http://www.amren.com/news/2012/10/el-paso-schools-confront-scandal-of-students-who-disappeared-at-test-time/ http://www.elpasotimes.com/episd/ci_20848628/former-episd-superintendent-lorenzo-garcia-enter-plea-aggreement You can’t make this stuff up. This sort of thing is not surprising but it’s worth keeping in mind. That a measurement system c
2 0.77413386 606 andrew gelman stats-2011-03-10-It’s no fun being graded on a curve
Introduction: Mark Palko points to a news article by Michael Winerip on teacher assessment: No one at the Lab Middle School for Collaborative Studies works harder than Stacey Isaacson, a seventh-grade English and social studies teacher. She is out the door of her Queens home by 6:15 a.m., takes the E train into Manhattan and is standing out front when the school doors are unlocked, at 7. Nights, she leaves her classroom at 5:30. . . . Her principal, Megan Adams, has given her terrific reviews during the two and a half years Ms. Isaacson has been a teacher. . . . The Lab School has selective admissions, and Ms. Isaacson’s students have excelled. Her first year teaching, 65 of 66 scored proficient on the state language arts test, meaning they got 3′s or 4′s; only one scored below grade level with a 2. More than two dozen students from her first two years teaching have gone on to . . . the city’s most competitive high schools. . . . You would think the Department of Education would want to r
3 0.75713378 95 andrew gelman stats-2010-06-17-“Rewarding Strivers: Helping Low-Income Students Succeed in College”
Introduction: Several years ago, I heard about a project at the Educational Testing Service to identify “strivers”: students from disadvantaged backgrounds who did unexpectedly well on the SAT (the college admissions exam formerly known as the “Scholastic Aptitude Test” but apparently now just “the SAT,” in the same way that Exxon is just “Exxon” and that Harry Truman’s middle name is just “S”), at least 200 points above a predicted score based on demographic and neighborhood information. My ETS colleague and I agreed that this was a silly idea: From a statistical point of view, if student A is expected ahead of time to do better than student B, and then they get identical test scores, then you’d expect student A (the non-”striver”) to do better than student B (the “striver”) later on. Just basic statistics: if a student does much better than expected, then probably some of that improvement is noise. The idea of identifying these “strivers” seemed misguided and not the best use of the SAT.
4 0.75013256 542 andrew gelman stats-2011-01-28-Homework and treatment levels
Introduction: Interesting discussion here by Mark Palko on the difficulty of comparing charter schools to regular schools, even if the slots in the charter schools have been assigned by lottery. Beyond the direct importance of the topic, I found the discussion interesting because I always face a challenge in my own teaching to assign the right amount of homework, given that if I assign too much, students will simply rebel and not do it. To get back to the school-choice issue . . . Mark discussed selection effects: if a charter school is popular, it can require parents to sign a contract agreeing they will supervise their students to do lots of homework. Mark points out that there is a selection issue here, that the sort of parents who would sign that form are different from parents in general. But it seems to me there’s one more twist: These charter schools are popular, right? So that would imply that there is some reservoir of parents who would like to sign the form but don’t have the opp
5 0.74858338 402 andrew gelman stats-2010-11-09-Kaggle: forecasting competitions in the classroom
Introduction: Anthony Goldbloom writes: For those who haven’t come across Kaggle, we are a new platform for data prediction competitions. Companies and researchers put up a dataset and a problem and data scientists compete to produce the best solutions. We’ve just launched a new initiative called Kaggle in Class, allowing instructors to host competitions for their students. Competitions are a neat way to engage students, giving them the opportunity to put into practice what they learn. The platform offers live leaderboards, so students get instant feedback on the accuracy of their work. And since competitions are judged on objective criteria (predictions are compared with outcomes), the platform offers unique assessment opportunities. The first Kaggle in Class competition is being hosted by Stanford University’s Stats 202 class and requires students to predict the price of different wines based on vintage, country, ratings and other information. Those interested in hosting a competition f
7 0.73606539 93 andrew gelman stats-2010-06-17-My proposal for making college admissions fairer
8 0.73322427 956 andrew gelman stats-2011-10-13-Hey, you! Don’t take that class!
9 0.73157698 326 andrew gelman stats-2010-10-07-Peer pressure, selection, and educational reform
10 0.73054117 1517 andrew gelman stats-2012-10-01-“On Inspiring Students and Being Human”
11 0.73003632 71 andrew gelman stats-2010-06-07-Pay for an A?
12 0.71763277 1752 andrew gelman stats-2013-03-06-Online Education and Jazz
13 0.71338844 1265 andrew gelman stats-2012-04-15-Progress in U.S. education; also, a discussion of what it takes to hit the op-ed pages
14 0.71026862 1620 andrew gelman stats-2012-12-12-“Teaching effectiveness” as another dimension in cognitive ability
15 0.70937598 1350 andrew gelman stats-2012-05-28-Value-added assessment: What went wrong?
17 0.70200974 957 andrew gelman stats-2011-10-14-Questions about a study of charter schools
18 0.69741178 1688 andrew gelman stats-2013-01-22-That claim that students whose parents pay for more of college get worse grades
19 0.69198519 1582 andrew gelman stats-2012-11-18-How to teach methods we don’t like?
20 0.68348801 277 andrew gelman stats-2010-09-14-In an introductory course, when does learning occur?
topicId topicWeight
[(0, 0.047), (16, 0.234), (24, 0.024), (53, 0.099), (69, 0.024), (71, 0.026), (83, 0.031), (85, 0.02), (95, 0.03), (96, 0.038), (99, 0.316)]
simIndex simValue blogId blogTitle
Introduction: Our apartment is from earlier in the century, so I can’t give Tyler Cowen’s first answer , but, after that, I follow him in thinking of the several books I have from that decade. Beyond that, lemme think . . . We occasionally play Risk , and our set dates from the 50s. Some kitchen implements (a mixmaster, a couple of cookbooks, who knows which old bowls, forks, etc). Probably some of the furniture, although I don’t know which. Probably some of the items in our building (the boiler?) What else, I wonder? There are probably a few things I’m forgetting. 50-60 years is a long time, I guess. P.S. to the commenters: I’m taking the question to refer to things manufactured in the 1950s and not before!
2 0.94606888 1168 andrew gelman stats-2012-02-14-The tabloids strike again
Introduction: See comments #2,3,4 here . I guess that’s why Science and Nature are known as “the tabloids.” As the commenter writes, “you can’t have people look at too many images of maggot-infested wounds.”
3 0.9448421 1495 andrew gelman stats-2012-09-13-Win $5000 in the Economist’s data visualization competition
Introduction: Michael Nelson points me to this . OK, $5,000 isn’t a lot of money (I’m not expecting Niall Ferguson in the competition), but I’m still glad to see this, given that the Economist is known for its excellent graphics.
4 0.94391388 1022 andrew gelman stats-2011-11-21-Progress for the Poor
Introduction: Lane Kenworthy writes : The book is full of graphs that support the above claims. One thing I like about Kenworthy’s approach is that he performs a separate analysis to examine each of his hypotheses. A lot of social scientists seem to think that the ideal analysis will conclude with a big regression where each coefficient tells a story and you can address all your hypotheses by looking at which predictors and interactions have statistically significant coefficients. Really, though, I think you need a separate analysis for each causal question (see chapters 9 and 10 of my book with Jennifer, follow this link ). Kenworthy’s overall recommendation is to increase transfer payments to low-income families and to increase overall government spending on social services, and to fund this through general tax increases. What will it take for this to happen? After a review of the evidence from economic trends and opinion polls, Kenworthy writes, “Americans are potentially recepti
5 0.94219375 1928 andrew gelman stats-2013-07-06-How to think about papers published in low-grade journals?
Introduction: We’ve had lots of lively discussions of fatally-flawed papers that have been published in top, top journals such as the American Economic Review or the Journal of Personality and Social Psychology or the American Sociological Review or the tabloids . And we also know about mistakes that make their way into mid-ranking outlets such as the Journal of Theoretical Biology. But what about results that appear in the lower tier of legitimate journals? I was thinking about this after reading a post by Dan Kahan slamming a paper that recently appeared in PLOS-One. I won’t discuss the paper itself here because that’s not my point. Rather, I had some thoughts regarding Kahan’s annoyance that a paper with fatal errors was published at all. I commented as follows: Read between the lines. The paper originally was released in 2009 and was published in 2013 in PLOS-One, which is one step above appearing on Arxiv. PLOS-One publishes some good things (so does Arxiv) but it’s the place
6 0.93980765 321 andrew gelman stats-2010-10-05-Racism!
same-blog 7 0.9375453 2083 andrew gelman stats-2013-10-31-Value-added modeling in education: Gaming the system by sending kids on a field trip at test time
8 0.93411803 1156 andrew gelman stats-2012-02-06-Bayesian model-building by pure thought: Some principles and examples
9 0.93311536 1025 andrew gelman stats-2011-11-24-Always check your evidence
10 0.93293405 1598 andrew gelman stats-2012-11-30-A graphics talk with no visuals!
11 0.92895287 1330 andrew gelman stats-2012-05-19-Cross-validation to check missing-data imputation
12 0.92893052 159 andrew gelman stats-2010-07-23-Popular governor, small state
13 0.9286114 700 andrew gelman stats-2011-05-06-Suspicious pattern of too-strong replications of medical research
14 0.92840672 564 andrew gelman stats-2011-02-08-Different attitudes about parenting, possibly deriving from different attitudes about self
15 0.92724383 609 andrew gelman stats-2011-03-13-Coauthorship norms
16 0.92285502 960 andrew gelman stats-2011-10-15-The bias-variance tradeoff
17 0.92278576 1487 andrew gelman stats-2012-09-08-Animated drought maps
18 0.91885281 1180 andrew gelman stats-2012-02-22-I’m officially no longer a “rogue”
19 0.91798812 377 andrew gelman stats-2010-10-28-The incoming moderate Republican congressmembers
20 0.91592038 445 andrew gelman stats-2010-12-03-Getting a job in pro sports… as a statistician