andrew_gelman_stats andrew_gelman_stats-2011 andrew_gelman_stats-2011-560 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Jonathan Livengood writes: There has been some discussion about the recent PISA results (in which the U.S. comes out pretty badly), for example here and here . The claim being made is that the poor U.S. scores are due to rampant individual- or family-level poverty in the U.S. They claim that when one controls for poverty, the U.S. comes out on top in the PISA standings, and then they infer that poverty causes poor test scores. The further inference is then that the U.S. could improve education by the “simple” action of reducing poverty. Anyway, I was wondering what you thought about their analysis. My reply: I agree this is interesting and I agree it’s hard to know exactly what to say about these comparisons. When I’m stuck in this sort of question, I ask, WWJD? In this case, I think Jennifer would ask what are the potential interventions being considered. Various ideas for changing the school system would perhaps have different effects on different groups of students.
sentIndex sentText sentNum sentScore
1 Jonathan Livengood writes: There has been some discussion about the recent PISA results (in which the U. [sent-1, score-0.139]
2 comes out pretty badly), for example here and here . [sent-3, score-0.125]
3 scores are due to rampant individual- or family-level poverty in the U. [sent-6, score-0.763]
4 They claim that when one controls for poverty, the U. [sent-8, score-0.254]
5 comes out on top in the PISA standings, and then they infer that poverty causes poor test scores. [sent-10, score-1.083]
6 could improve education by the “simple” action of reducing poverty. [sent-13, score-0.398]
7 Anyway, I was wondering what you thought about their analysis. [sent-14, score-0.089]
8 My reply: I agree this is interesting and I agree it’s hard to know exactly what to say about these comparisons. [sent-15, score-0.298]
9 When I’m stuck in this sort of question, I ask, WWJD? [sent-16, score-0.103]
10 In this case, I think Jennifer would ask what are the potential interventions being considered. [sent-17, score-0.417]
11 Various ideas for changing the school system would perhaps have different effects on different groups of students. [sent-18, score-0.721]
12 I think that would a useful way to focus discussion, to consider the effects of possible reforms in the U. [sent-19, score-0.593]
13 Livengood has some graphs and discussion here . [sent-25, score-0.209]
wordName wordTfidf (topN-words)
[('poverty', 0.381), ('pisa', 0.38), ('livengood', 0.358), ('standings', 0.19), ('rampant', 0.19), ('poor', 0.179), ('wwjd', 0.171), ('reforms', 0.161), ('badly', 0.147), ('infer', 0.147), ('ask', 0.144), ('discussion', 0.139), ('interventions', 0.134), ('claim', 0.132), ('comes', 0.125), ('controls', 0.122), ('effects', 0.118), ('action', 0.113), ('agree', 0.112), ('reducing', 0.112), ('causes', 0.11), ('jonathan', 0.106), ('stuck', 0.103), ('scores', 0.103), ('jennifer', 0.101), ('changing', 0.101), ('improve', 0.093), ('wondering', 0.089), ('due', 0.089), ('groups', 0.082), ('education', 0.08), ('anyway', 0.078), ('potential', 0.076), ('focus', 0.075), ('different', 0.075), ('exactly', 0.074), ('school', 0.073), ('system', 0.072), ('top', 0.071), ('test', 0.07), ('graphs', 0.07), ('would', 0.063), ('ideas', 0.062), ('useful', 0.06), ('inference', 0.06), ('various', 0.059), ('simple', 0.059), ('consider', 0.058), ('possible', 0.058), ('reply', 0.055)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 560 andrew gelman stats-2011-02-06-Education and Poverty
Introduction: Jonathan Livengood writes: There has been some discussion about the recent PISA results (in which the U.S. comes out pretty badly), for example here and here . The claim being made is that the poor U.S. scores are due to rampant individual- or family-level poverty in the U.S. They claim that when one controls for poverty, the U.S. comes out on top in the PISA standings, and then they infer that poverty causes poor test scores. The further inference is then that the U.S. could improve education by the “simple” action of reducing poverty. Anyway, I was wondering what you thought about their analysis. My reply: I agree this is interesting and I agree it’s hard to know exactly what to say about these comparisons. When I’m stuck in this sort of question, I ask, WWJD? In this case, I think Jennifer would ask what are the potential interventions being considered. Various ideas for changing the school system would perhaps have different effects on different groups of students.
2 0.18487248 251 andrew gelman stats-2010-09-02-Interactions of predictors in a causal model
Introduction: Michael Bader writes: What is the best way to examine interactions of independent variables in a propensity weights framework? Let’s say we are interested in estimating breathing difficulty (measured on a continuous scale) and our main predictor is age of housing. The object is to estimate whether living in housing 20 years or older is associated with breathing difficulty compared counterfactually to those living in housing less than 20 years old; as a secondary question, we want to know whether that effect differs for those in poverty compared to those not in poverty. In our first-stage propensity model, we include whether the respondent lives in poverty. The weights applied to the other covariates in the propensity model are similar to those living in poverty compared to those who are not. Now, can I simply interact the poverty variable with the age of construction variable to look at the interaction of age of housing and poverty on breathing difficulty? My thought is no —
3 0.14522828 561 andrew gelman stats-2011-02-06-Poverty, educational performance – and can be done about it
Introduction: Andrew has pointed to Jonathan Livengood’s analysis of the correlation between poverty and PISA results, whereby schools with poorer students get poorer test results. I’d have written a comment, but then I couldn’t have inserted a chart. Andrew points out that a causal analysis is needed. This reminds me of an intervention that has been done before: take a child out of poverty, and bring him up in a better-off family. What’s going to happen? There have been several studies examining correlations between adoptive and biological parents’ IQ (assuming IQ is a test analogous to the math and verbal tests, and that parent IQ is analogous to the quality of instruction – but the point is in the analysis not in the metric). This is the result (from Adoption Strategies by Robin P Corley in Encyclopedia of Life Sciences): So, while it did make a difference at an early age, with increasing age of the adopted child, the intelligence of adoptive parents might not be making any difference
4 0.10845328 1675 andrew gelman stats-2013-01-15-“10 Things You Need to Know About Causal Effects”
Introduction: Macartan Humphreys pointed me to this excellent guide . Here are the 10 items: 1. A causal claim is a statement about what didn’t happen. 2. There is a fundamental problem of causal inference. 3. You can estimate average causal effects even if you cannot observe any individual causal effects. 4. If you know that, on average, A causes B and that B causes C, this does not mean that you know that A causes C. 5. The counterfactual model is all about contribution, not attribution. 6. X can cause Y even if there is no “causal path” connecting X and Y. 7. Correlation is not causation. 8. X can cause Y even if X is not a necessary condition or a sufficient condition for Y. 9. Estimating average causal effects does not require that treatment and control groups are identical. 10. There is no causation without manipulation. The article follows with crisp discussions of each point. My favorite is item #6, not because it’s the most important but because it brings in some real s
5 0.10745077 2097 andrew gelman stats-2013-11-11-Why ask why? Forward causal inference and reverse causal questions
Introduction: Guido Imbens and I write : The statistical and econometrics literature on causality is more focused on “effects of causes” than on “causes of effects.” That is, in the standard approach it is natural to study the effect of a treatment, but it is not in general possible to define the causes of any particular outcome. This has led some researchers to dismiss the search for causes as “cocktail party chatter” that is outside the realm of science. We argue here that the search for causes can be understood within traditional statistical frameworks as a part of model checking and hypothesis generation. We argue that it can make sense to ask questions about the causes of effects, but the answers to these questions will be in terms of effects of causes. We also posted the paper on NBER so I’m hoping it will get some attention from economists. [Again, here's the open link to the paper.] I think what we have here is an important idea linking statistical and econometric models of caus
6 0.10096861 139 andrew gelman stats-2010-07-10-Life in New York, Then and Now
7 0.095768906 315 andrew gelman stats-2010-10-03-He doesn’t trust the fit . . . r=.999
8 0.091870219 807 andrew gelman stats-2011-07-17-Macro causality
9 0.088546932 614 andrew gelman stats-2011-03-15-Induction within a model, deductive inference for model evaluation
10 0.088299729 2337 andrew gelman stats-2014-05-18-Never back down: The culture of poverty and the culture of journalism
11 0.085358925 86 andrew gelman stats-2010-06-14-“Too much data”?
12 0.081382021 624 andrew gelman stats-2011-03-22-A question about the economic benefits of universities
13 0.07975705 226 andrew gelman stats-2010-08-23-More on those L.A. Times estimates of teacher effectiveness
14 0.076597057 2283 andrew gelman stats-2014-04-06-An old discussion of food deserts
15 0.075564429 1070 andrew gelman stats-2011-12-19-The scope for snooping
16 0.075226545 125 andrew gelman stats-2010-07-02-The moral of the story is, Don’t look yourself up on Google
17 0.074368633 2274 andrew gelman stats-2014-03-30-Adjudicating between alternative interpretations of a statistical interaction?
18 0.069147758 2331 andrew gelman stats-2014-05-12-On deck this week
19 0.068102203 1492 andrew gelman stats-2012-09-11-Using the “instrumental variables” or “potential outcomes” approach to clarify causal thinking
20 0.067397892 609 andrew gelman stats-2011-03-13-Coauthorship norms
topicId topicWeight
[(0, 0.137), (1, -0.01), (2, 0.019), (3, -0.033), (4, 0.02), (5, -0.005), (6, 0.002), (7, 0.031), (8, 0.041), (9, 0.031), (10, -0.021), (11, 0.049), (12, 0.002), (13, -0.034), (14, 0.058), (15, 0.011), (16, 0.005), (17, 0.02), (18, -0.03), (19, 0.042), (20, -0.014), (21, -0.011), (22, 0.025), (23, -0.026), (24, 0.027), (25, 0.001), (26, -0.02), (27, 0.034), (28, -0.056), (29, 0.049), (30, 0.031), (31, -0.013), (32, -0.005), (33, 0.038), (34, -0.019), (35, -0.003), (36, 0.003), (37, 0.038), (38, 0.005), (39, 0.054), (40, 0.008), (41, -0.006), (42, 0.021), (43, -0.007), (44, -0.047), (45, 0.006), (46, 0.001), (47, -0.034), (48, -0.017), (49, 0.009)]
simIndex simValue blogId blogTitle
same-blog 1 0.96283942 560 andrew gelman stats-2011-02-06-Education and Poverty
Introduction: Jonathan Livengood writes: There has been some discussion about the recent PISA results (in which the U.S. comes out pretty badly), for example here and here . The claim being made is that the poor U.S. scores are due to rampant individual- or family-level poverty in the U.S. They claim that when one controls for poverty, the U.S. comes out on top in the PISA standings, and then they infer that poverty causes poor test scores. The further inference is then that the U.S. could improve education by the “simple” action of reducing poverty. Anyway, I was wondering what you thought about their analysis. My reply: I agree this is interesting and I agree it’s hard to know exactly what to say about these comparisons. When I’m stuck in this sort of question, I ask, WWJD? In this case, I think Jennifer would ask what are the potential interventions being considered. Various ideas for changing the school system would perhaps have different effects on different groups of students.
2 0.74359071 2274 andrew gelman stats-2014-03-30-Adjudicating between alternative interpretations of a statistical interaction?
Introduction: Jacob Felson writes: Say we have a statistically significant interaction in non-experimental data between two continuous predictors, X and Z and it is unclear which variable is primarily a cause and which variable is primarily a moderator. One person might find it more plausible to think of X as a cause and Z as a moderator and another person may think the reverse more plausible. My question then is whether there is are any set of rules or heuristics you could recommend to help adjudicate between alternate perspectives on such an interaction term. My reply: I think in this setting, it would make sense to think about different interventions, some of which affect X, others of which affect Z, others of which affect both, and go from there. Rather than trying to isolate a single causal path, consider different cases of forward casual inference. My guess is that the different stories regarding moderators etc. could motivate different thought experiments (and, ultimately, differe
Introduction: As I’ve written here many times, my experiences in social science and public health research have left me skeptical of statistical methods that hypothesize or try to detect zero relationships between observational data (see, for example, the discussion starting at the bottom of page 960 in my review of causal inference in the American Journal of Sociology). In short, I have a taste for continuous rather than discrete models. As discussed in the above-linked article (with respect to the writings of cognitive scientist Steven Sloman), I think that common-sense thinking about causal inference can often mislead. In many cases, I have found that that the theoretical frameworks of instrumental variables and potential outcomes (for a review see, for example, chapters 9 and 10 of my book with Jennifer) help clarify my thinking. Here is an example that came up in a recent blog discussion. Computer science student Elias Bareinboim gave the following example: “suppose we know nothing a
Introduction: Hey, we all know the answer: “correlation does not imply causation”—but of course life is more complicated than that. As philosophers, economists, statisticians, and others have repeatedly noted, most of our information about the world is observational not experimental, yet we manage to draw causal conclusions all the time. Sure, some of these conclusions are wrong (more often than 5% of the time, I’m sure) but that’s an accepted part of life. Challenges in this regard arise in the design of a study, in the statistical analysis, in how you write it up for a peer-reviewed journal, and finally in how you present it to the world. School sports and life outcomes An interesting case of all this came up recently in a post on Freakonomics that pointed to a post on Deadspin that pointed to a research article . The claim was that “sports participation [in high school] causes women to be less likely to be religious . . . more likely to have children . . . more likely to be singl
5 0.70612818 561 andrew gelman stats-2011-02-06-Poverty, educational performance – and can be done about it
Introduction: Andrew has pointed to Jonathan Livengood’s analysis of the correlation between poverty and PISA results, whereby schools with poorer students get poorer test results. I’d have written a comment, but then I couldn’t have inserted a chart. Andrew points out that a causal analysis is needed. This reminds me of an intervention that has been done before: take a child out of poverty, and bring him up in a better-off family. What’s going to happen? There have been several studies examining correlations between adoptive and biological parents’ IQ (assuming IQ is a test analogous to the math and verbal tests, and that parent IQ is analogous to the quality of instruction – but the point is in the analysis not in the metric). This is the result (from Adoption Strategies by Robin P Corley in Encyclopedia of Life Sciences): So, while it did make a difference at an early age, with increasing age of the adopted child, the intelligence of adoptive parents might not be making any difference
6 0.69844651 1910 andrew gelman stats-2013-06-22-Struggles over the criticism of the “cannabis users and IQ change” paper
7 0.69191051 213 andrew gelman stats-2010-08-17-Matching at two levels
8 0.69151264 1350 andrew gelman stats-2012-05-28-Value-added assessment: What went wrong?
10 0.68637657 1688 andrew gelman stats-2013-01-22-That claim that students whose parents pay for more of college get worse grades
11 0.68317902 1793 andrew gelman stats-2013-04-08-The Supreme Court meets the fallacy of the one-sided bet
12 0.68109459 226 andrew gelman stats-2010-08-23-More on those L.A. Times estimates of teacher effectiveness
13 0.67946988 393 andrew gelman stats-2010-11-04-Estimating the effect of A on B, and also the effect of B on A
14 0.67940331 315 andrew gelman stats-2010-10-03-He doesn’t trust the fit . . . r=.999
15 0.67474574 610 andrew gelman stats-2011-03-13-Secret weapon with rare events
16 0.67175257 1732 andrew gelman stats-2013-02-22-Evaluating the impacts of welfare reform?
17 0.67159534 1675 andrew gelman stats-2013-01-15-“10 Things You Need to Know About Causal Effects”
18 0.66806871 1410 andrew gelman stats-2012-07-09-Experimental work on market-based or non-market-based incentives
19 0.66802853 1136 andrew gelman stats-2012-01-23-Fight! (also a bit of reminiscence at the end)
20 0.66673392 1663 andrew gelman stats-2013-01-09-The effects of fiscal consolidation
topicId topicWeight
[(4, 0.019), (9, 0.33), (16, 0.078), (21, 0.016), (24, 0.15), (95, 0.015), (99, 0.272)]
simIndex simValue blogId blogTitle
1 0.99403489 577 andrew gelman stats-2011-02-16-Annals of really really stupid spam
Introduction: This came in the inbox today: Dear Dr. Gelman, GenWay recently found your article titled “Multiple imputation for model checking: completed-data plots with missing and latent data.” (Biometrics. 2005 Mar;61(1):74-85.) and thought you might be interested in learning about our superior quality signaling proteins. GenWay prides itself on being a leader in customer service aiming to exceed your expectations with the quality and price of our products. With more than 60,000 reagents backed by our outstanding guarantee you are sure to find the products you have been searching for. Please feel free to visit the following resource pages: * Apoptosis Pathway (product list) * Adipocytokine (product list) * Cell Cycle Pathway (product list) * Jak STAT (product list) * GnRH (product list) * MAPK (product list) * mTOR (product list) * T Cell Receptor (product list) * TGF-beta (product list) * Wnt (product list) * View All Pathways
2 0.98950005 993 andrew gelman stats-2011-11-05-The sort of thing that gives technocratic reasoning a bad name
Introduction: 1. Freakonomics characterizes drunk driving as an example of “the human tendency to worry about rare problems that are unlikely to happen.” 2. The CDC reports , “Alcohol-impaired drivers are involved in about 1 in 3 crash deaths, resulting in nearly 11,000 deaths in 2009.” No offense to the tenured faculty at the University of Chicago, but I’m going with the CDC on this one. P.S. The Freakonomics blog deserves to be dinged another time, not just for claiming, based on implausible assumptions and making the all-else-equal fallacy that “drunk walking is 8 times more likely to result in your death than drunk driving” but for presenting this weak inference as a fact rather than as a speculation. When doing “Freakonomics,” you can be counterintuitive, or you can be sensible, but it’s hard to be both. I mean, sure, sometimes you can be. But there’s a tradeoff, and in this case, they’re choosing to push the envelope on counterintuitiveness.
3 0.9689014 1356 andrew gelman stats-2012-05-31-Question 21 of my final exam for Design and Analysis of Sample Surveys
Introduction: 21. A country is divided into three regions with populations of 2 million, 2 million, and 0.5 million, respectively. A survey is done asking about foreign policy opinions.. Somebody proposes taking a sample of 50 people from each reason. Give a reason why this non-proportional sample would not usually be done, and also a reason why it might actually be a good idea. Solution to question 20 From yesterday : 20. Explain in two sentences why we expect survey respondents to be honest about vote preferences but possibly dishonest about reporting unhealty behaviors. Solution: Respondents tend to be sincere about vote preferences because this affects the outcome of the poll, and people are motivated to have their candidate poll well. This motivation is typically not present in reporting behaviors; you have no particular reason for wanting to affect the average survey response.
4 0.95511413 1291 andrew gelman stats-2012-04-30-Systematic review of publication bias in studies on publication bias
Introduction: Via Yalda Afshar , a 2005 paper by Hans-Hermann Dubben and Hans-Peter Beck-Bornholdt: Publication bias is a well known phenomenon in clinical literature, in which positive results have a better chance of being published, are published earlier, and are published in journals with higher impact factors. Conclusions exclusively based on published studies, therefore, can be misleading. Selective under-reporting of research might be more widespread and more likely to have adverse consequences for patients than publication of deliberately falsified data. We investigated whether there is preferential publication of positive papers on publication bias. They conclude, “We found no evidence of publication bias in reports on publication bias.” But of course that’s the sort of finding regarding publication bias of findings on publication bias that you’d expect would get published. What we really need is a careful meta-analysis to estimate the level of publication bias in studies of publi
5 0.94651556 529 andrew gelman stats-2011-01-21-“City Opens Inquiry on Grading Practices at a Top-Scoring Bronx School”
Introduction: Sharon Otterman reports : When report card grades were released in the fall for the city’s 455 high schools, the highest score went to a small school in a down-and-out section of the Bronx . . . A stunning 94 percent of its seniors graduated, more than 30 points above the citywide average. . . . “When I interviewed for the school,” said Sam Buchbinder, a history teacher, “it was made very clear: this is a school that doesn’t believe in anyone failing.” That statement was not just an exhortation to excellence. It was school policy. By order of the principal, codified in the school’s teacher handbook, all teachers should grade their classes in the same way: 30 percent of students should earn a grade in the A range, 40 percent B’s, 25 percent C’s, and no more than 5 percent D’s. As long as they show up, they should not fail. Hey, that sounds like Harvard and Columbia^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H various selective northeastern colleges I’ve known. Of course, we^H^H
6 0.94374037 1532 andrew gelman stats-2012-10-13-A real-life dollar auction game!
7 0.93703783 1332 andrew gelman stats-2012-05-20-Problemen met het boek
8 0.93482065 1424 andrew gelman stats-2012-07-22-Extreme events as evidence for differences in distributions
9 0.92084569 1664 andrew gelman stats-2013-01-10-Recently in the sister blog: Brussels sprouts, ugly graphs, and switched at birth
10 0.89316189 1961 andrew gelman stats-2013-07-29-Postdocs in probabilistic modeling! With David Blei! And Stan!
11 0.88803709 389 andrew gelman stats-2010-11-01-Why it can be rational to vote
12 0.88803506 1565 andrew gelman stats-2012-11-06-Why it can be rational to vote
same-blog 13 0.88155335 560 andrew gelman stats-2011-02-06-Education and Poverty
14 0.87511706 1142 andrew gelman stats-2012-01-29-Difficulties with the 1-4-power transformation
15 0.87239319 1110 andrew gelman stats-2012-01-10-Jobs in statistics research! In New Jersey!
16 0.86658376 29 andrew gelman stats-2010-05-12-Probability of successive wins in baseball
17 0.86462981 1226 andrew gelman stats-2012-03-22-Story time meets the all-else-equal fallacy and the fallacy of measurement
18 0.86365163 1566 andrew gelman stats-2012-11-07-A question about voting systems—unrelated to U.S. elections!
19 0.84411401 1715 andrew gelman stats-2013-02-09-Thomas Hobbes would be spinning in his grave
20 0.83874333 640 andrew gelman stats-2011-03-31-Why Edit Wikipedia?