andrew_gelman_stats andrew_gelman_stats-2010 andrew_gelman_stats-2010-309 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Robert Neumann writes: in the JEP 24(3), page18, Daron Acemoglu states: Why Development Economics Needs Theory There is no general agreement on how much we should rely on economic theory in motivating empirical work and whether we should try to formulate and estimate “structural parameters.” I (Acemoglu) argue that the answer is largely “yes” because otherwise econometric estimates would lack external validity, in which case they can neither inform us about whether a particular model or theory is a useful approximation to reality, nor would they be useful in providing us guidance on what the effects of similar shocks and policies would be in different circumstances or if implemented in different scales. I therefore define “structural parameters” as those that provide external validity and would thus be useful in testing theories or in policy analysis beyond the specific environment and sample from which they are derived. External validity becomes a particularly challenging t
sentIndex sentText sentNum sentScore
1 I therefore define “structural parameters” as those that provide external validity and would thus be useful in testing theories or in policy analysis beyond the specific environment and sample from which they are derived. [sent-3, score-1.113]
2 External validity becomes a particularly challenging task in the presence of general equilibrium and political economy considerations, and a major role of economic theory is in helping us overcome these problems or at the very least alerting us to their importance. [sent-4, score-1.321]
3 Leaving aside the equilibrium debate, what do you think of his remark that the external validity of estimates refers to an underlying model. [sent-5, score-0.863]
4 My reply: This reminds me a lot of Heckman’s argument of why randomized experiments are not a gold standard. [sent-7, score-0.065]
5 I see the point but, on the other hand, as Don Green and others have noted, observational studies have external validity problems too! [sent-8, score-0.653]
6 Whether or not a model is motivated by economic theory, you’ll have to make assumptions to generalize your inferences beyond the population under study. [sent-9, score-0.226]
7 We used statistical language and Acemoglu is using econometric language but it’s the same idea, I think, and a point worth making in as many languages as it takes. [sent-11, score-0.376]
8 Theory is great—and we had it in abundance in our toxicology analysis—but I’d think you could have generalizable parameters without formal theory, if you’re careful enough to define what you’re measuring. [sent-13, score-0.656]
wordName wordTfidf (topN-words)
[('acemoglu', 0.457), ('external', 0.342), ('validity', 0.311), ('theory', 0.248), ('structural', 0.187), ('generalizable', 0.165), ('toxicology', 0.155), ('equilibrium', 0.147), ('define', 0.14), ('econometric', 0.137), ('parameters', 0.11), ('economic', 0.101), ('therefore', 0.092), ('us', 0.092), ('jep', 0.091), ('physiological', 0.091), ('useful', 0.087), ('language', 0.087), ('jiang', 0.086), ('abundance', 0.086), ('alerting', 0.083), ('phenomenological', 0.083), ('shocks', 0.083), ('parameter', 0.079), ('whether', 0.079), ('neumann', 0.075), ('provide', 0.074), ('bois', 0.074), ('heckman', 0.072), ('usefully', 0.071), ('motivating', 0.071), ('formulate', 0.069), ('overcome', 0.068), ('considerations', 0.068), ('inform', 0.068), ('beyond', 0.067), ('guidance', 0.066), ('gold', 0.065), ('languages', 0.065), ('challenging', 0.064), ('estimates', 0.063), ('leaving', 0.062), ('incorporate', 0.061), ('green', 0.059), ('presence', 0.059), ('generalize', 0.058), ('approximation', 0.058), ('implemented', 0.057), ('circumstances', 0.057), ('helping', 0.056)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 309 andrew gelman stats-2010-10-01-Why Development Economics Needs Theory?
Introduction: Robert Neumann writes: in the JEP 24(3), page18, Daron Acemoglu states: Why Development Economics Needs Theory There is no general agreement on how much we should rely on economic theory in motivating empirical work and whether we should try to formulate and estimate “structural parameters.” I (Acemoglu) argue that the answer is largely “yes” because otherwise econometric estimates would lack external validity, in which case they can neither inform us about whether a particular model or theory is a useful approximation to reality, nor would they be useful in providing us guidance on what the effects of similar shocks and policies would be in different circumstances or if implemented in different scales. I therefore define “structural parameters” as those that provide external validity and would thus be useful in testing theories or in policy analysis beyond the specific environment and sample from which they are derived. External validity becomes a particularly challenging t
Introduction: Elias Bareinboim asked what I thought about his comment on selection bias in which he referred to a paper by himself and Judea Pearl, “Controlling Selection Bias in Causal Inference.” I replied that I have no problem with what he wrote, but that from my perspective I find it easier to conceptualize such problems in terms of multilevel models. I elaborated on that point in a recent post , “Hierarchical modeling as a framework for extrapolation,” which I think was read by only a few people (I say this because it received only two comments). I don’t think Bareinboim objected to anything I wrote, but like me he is comfortable working within his own framework. He wrote the following to me: In some sense, “not ad hoc” could mean logically consistent. In other words, if one agrees with the assumptions encoded in the model, one must also agree with the conclusions entailed by these assumptions. I am not aware of any other way of doing mathematics. As it turns out, to get causa
3 0.12175785 32 andrew gelman stats-2010-05-14-Causal inference in economics
Introduction: Aaron Edlin points me to this issue of the Journal of Economic Perspectives that focuses on statistical methods for causal inference in economics. (Michael Bishop’s page provides some links .) To quickly summarize my reactions to Angrist and Pischke’s book: I pretty much agree with them that the potential-outcomes or natural-experiment approach is the most useful way to think about causality in economics and related fields. My main amendments to Angrist and Pischke would be to recognize that: 1. Modeling is important, especially modeling of interactions . It’s unfortunate to see a debate between experimentalists and modelers. Some experimenters (not Angrist and Pischke) make the mistake of avoiding models: Once they have their experimental data, they check their brains at the door and do nothing but simple differences, not realizing how much more can be learned. Conversely, some modelers are unduly dismissive of experiments and formal observational studies, forgetting t
4 0.11724643 1469 andrew gelman stats-2012-08-25-Ways of knowing
Introduction: In this discussion from last month, computer science student and Judea Pearl collaborator Elias Barenboim expressed an attitude that hierarchical Bayesian methods might be fine in practice but that they lack theory, that Bayesians can’t succeed in toy problems. I posted a P.S. there which might not have been noticed so I will put it here: I now realize that there is some disagreement about what constitutes a “guarantee.” In one of his comments, Barenboim writes, “the assurance we have that the result must hold as long as the assumptions in the model are correct should be regarded as a guarantee.” In that sense, yes, we have guarantees! It is fundamental to Bayesian inference that the result must hold if the assumptions in the model are correct. We have lots of that in Bayesian Data Analysis (particularly in the first four chapters but implicitly elsewhere as well), and this is also covered in the classic books by Lindley, Jaynes, and others. This sort of guarantee is indeed p
Introduction: I pointed Steven Pinker to my post, How much time (if any) should we spend criticizing research that’s fraudulent, crappy, or just plain pointless? , and he responded: Clearly it *is* important to call out publicized research whose conclusions are likely to be false. The only danger is that it’s so easy and fun to criticize, with all the perks of intellectual and moral superiority for so little cost, that there is a moral hazard to go overboard and become a professional slasher and snarker. (That’s a common phenomenon among literary critics, especially in the UK.) There’s also the risk of altering the incentive structure for innovative research, so that researchers stick to the safest kinds of paradigm-twiddling. I think these two considerations were what my late colleague Dan Wegner had in mind when he made the bumbler-pointer contrast — he himself was certainly a discerning critic of social science research. [Just to clarify: Wegner is the person who talked about bumblers and po
6 0.10338474 2359 andrew gelman stats-2014-06-04-All the Assumptions That Are My Life
7 0.10161027 2033 andrew gelman stats-2013-09-23-More on Bayesian methods and multilevel modeling
10 0.096556999 754 andrew gelman stats-2011-06-09-Difficulties with Bayesian model averaging
12 0.095721923 792 andrew gelman stats-2011-07-08-The virtues of incoherence?
13 0.093578555 1652 andrew gelman stats-2013-01-03-“The Case for Inductive Theory Building”
14 0.091679424 2127 andrew gelman stats-2013-12-08-The never-ending (and often productive) race between theory and practice
15 0.091366105 1149 andrew gelman stats-2012-02-01-Philosophy of Bayesian statistics: my reactions to Cox and Mayo
16 0.08972729 1204 andrew gelman stats-2012-03-08-The politics of economic and statistical models
17 0.08879035 2330 andrew gelman stats-2014-05-12-Historical Arc of Universities
18 0.087454535 1208 andrew gelman stats-2012-03-11-Gelman on Hennig on Gelman on Bayes
19 0.083695091 674 andrew gelman stats-2011-04-21-Handbook of Markov Chain Monte Carlo
20 0.078490235 2037 andrew gelman stats-2013-09-25-Classical probability does not apply to quantum systems (causal inference edition)
topicId topicWeight
[(0, 0.152), (1, 0.052), (2, 0.026), (3, -0.022), (4, -0.04), (5, 0.006), (6, -0.015), (7, -0.003), (8, 0.012), (9, 0.051), (10, -0.058), (11, 0.005), (12, -0.016), (13, -0.012), (14, -0.016), (15, 0.006), (16, -0.025), (17, -0.02), (18, -0.016), (19, -0.0), (20, -0.007), (21, -0.057), (22, -0.01), (23, 0.042), (24, -0.019), (25, 0.027), (26, 0.04), (27, 0.043), (28, -0.001), (29, 0.016), (30, -0.008), (31, -0.035), (32, 0.023), (33, -0.026), (34, -0.029), (35, -0.033), (36, -0.035), (37, 0.006), (38, 0.035), (39, 0.021), (40, 0.0), (41, -0.004), (42, -0.012), (43, 0.013), (44, 0.003), (45, -0.034), (46, 0.036), (47, -0.049), (48, -0.002), (49, -0.008)]
simIndex simValue blogId blogTitle
same-blog 1 0.9689644 309 andrew gelman stats-2010-10-01-Why Development Economics Needs Theory?
Introduction: Robert Neumann writes: in the JEP 24(3), page18, Daron Acemoglu states: Why Development Economics Needs Theory There is no general agreement on how much we should rely on economic theory in motivating empirical work and whether we should try to formulate and estimate “structural parameters.” I (Acemoglu) argue that the answer is largely “yes” because otherwise econometric estimates would lack external validity, in which case they can neither inform us about whether a particular model or theory is a useful approximation to reality, nor would they be useful in providing us guidance on what the effects of similar shocks and policies would be in different circumstances or if implemented in different scales. I therefore define “structural parameters” as those that provide external validity and would thus be useful in testing theories or in policy analysis beyond the specific environment and sample from which they are derived. External validity becomes a particularly challenging t
2 0.73478407 789 andrew gelman stats-2011-07-07-Descriptive statistics, causal inference, and story time
Introduction: Dave Backus points me to this review by anthropologist Mike McGovern of two books by economist Paul Collier on the politics of economic development in Africa. My first reaction was that this was interesting but non-statistical so I’d have to either post it on the sister blog or wait until the 30 days of statistics was over. But then I looked more carefully and realized that this discussion is very relevant to applied statistics. Here’s McGovern’s substantive critique: Much of the fundamental intellectual work in Collier’s analyses is, in fact, ethnographic. Because it is not done very self-consciously and takes place within a larger econometric rhetoric in which such forms of knowledge are dismissed as “subjective” or worse still biased by the political (read “leftist”) agendas of the academics who create them, it is often ethnography of a low quality. . . . Despite the adoption of a Naipaulian unsentimental-dispatches-from-the-trenches rhetoric, the story told in Collier’s
3 0.71941483 1861 andrew gelman stats-2013-05-17-Where do theories come from?
Introduction: Lee Sechrest sends along this article by Brian Haig and writes that it “presents what seems to me a useful perspective on much of what scientists/statisticians do and how science works, at least in the fields in which I work.” Here’s Haig’s abstract: A broad theory of scientific method is sketched that has particular relevance for the behavioral sciences. This theory of method assembles a complex of specific strategies and methods that are used in the detection of empirical phenomena and the subsequent construction of explanatory theories. A characterization of the nature of phenomena is given, and the process of their detection is briefly described in terms of a multistage model of data analysis. The construction of explanatory theories is shown to involve their generation through abductive, or explanatory, reasoning, their development through analogical modeling, and their fuller appraisal in terms of judgments of the best of competing explanations. The nature and limits of
4 0.70447159 60 andrew gelman stats-2010-05-30-What Auteur Theory and Freshwater Economics have in common
Introduction: Mark Palko writes : We’ll define freshwater economics as the theory that economic behavior (and perhaps most non-economic behavior) can be explained using the concepts of rational actors and efficient markets and auteur theory as the idea that most films (particularly great films) represent the artistic vision of a single author (almost always the director) and the best way to approach one of those films is through the body of work of its author. Both of these definitions are oversimplified and a bit unfair but they will get the discussion started. . . . Compared to their nearest neighbors, film criticism and economics (particularly macroeconomics) are both difficult, messy fields. Films are collaborative efforts where individual contributions defy attribution and creative decisions often can’t be distinguished from accidents of filming. Worse yet, most films are the product of large corporations which means that dozens of VPs and executives might have played a role (sometimes
Introduction: Elias Bareinboim asked what I thought about his comment on selection bias in which he referred to a paper by himself and Judea Pearl, “Controlling Selection Bias in Causal Inference.” I replied that I have no problem with what he wrote, but that from my perspective I find it easier to conceptualize such problems in terms of multilevel models. I elaborated on that point in a recent post , “Hierarchical modeling as a framework for extrapolation,” which I think was read by only a few people (I say this because it received only two comments). I don’t think Bareinboim objected to anything I wrote, but like me he is comfortable working within his own framework. He wrote the following to me: In some sense, “not ad hoc” could mean logically consistent. In other words, if one agrees with the assumptions encoded in the model, one must also agree with the conclusions entailed by these assumptions. I am not aware of any other way of doing mathematics. As it turns out, to get causa
8 0.67843038 241 andrew gelman stats-2010-08-29-Ethics and statistics in development research
9 0.67775816 556 andrew gelman stats-2011-02-04-Patterns
10 0.67516547 177 andrew gelman stats-2010-08-02-Reintegrating rebels into civilian life: Quasi-experimental evidence from Burundi
11 0.67212158 2330 andrew gelman stats-2014-05-12-Historical Arc of Universities
12 0.6685555 1195 andrew gelman stats-2012-03-04-Multiple comparisons dispute in the tabloids
13 0.66687745 994 andrew gelman stats-2011-11-06-Josh Tenenbaum presents . . . a model of folk physics!
14 0.66670793 2251 andrew gelman stats-2014-03-17-In the best alternative histories, the real world is what’s ultimately real
15 0.66652024 242 andrew gelman stats-2010-08-29-The Subtle Micro-Effects of Peacekeeping
16 0.66609097 32 andrew gelman stats-2010-05-14-Causal inference in economics
18 0.66305971 1802 andrew gelman stats-2013-04-14-Detecting predictability in complex ecosystems
19 0.65611708 1425 andrew gelman stats-2012-07-23-Examples of the use of hierarchical modeling to generalize to new settings
20 0.64841682 1101 andrew gelman stats-2012-01-05-What are the standards for reliability in experimental psychology?
topicId topicWeight
[(9, 0.021), (13, 0.023), (15, 0.011), (16, 0.041), (17, 0.235), (21, 0.043), (24, 0.138), (55, 0.012), (65, 0.016), (84, 0.013), (86, 0.05), (99, 0.259)]
simIndex simValue blogId blogTitle
1 0.91784656 2314 andrew gelman stats-2014-05-01-Heller, Heller, and Gorfine on univariate and multivariate information measures
Introduction: Malka Gorfine writes: We noticed that the important topic of association measures and tests came up again in your blog, and we have few comments in this regard. It is useful to distinguish between the univariate and multivariate methods. A consistent multivariate method can recognise dependence between two vectors of random variables, while a univariate method can only loop over pairs of components and check for dependency between them. There are very few consistent multivariate methods. To the best of our knowledge there are three practical methods: 1) HSIC by Gretton et al. (http://www.gatsby.ucl.ac.uk/~gretton/papers/GreBouSmoSch05.pdf) 2) dcov by Szekely et al. (http://projecteuclid.org/euclid.aoas/1267453933) 3) A method we introduced in Heller et al (Biometrika, 2013, 503—510, http://biomet.oxfordjournals.org/content/early/2012/12/04/biomet.ass070.full.pdf+html, and an R package, HHG, is available as well http://cran.r-project.org/web/packages/HHG/index.html). A
2 0.91377914 705 andrew gelman stats-2011-05-10-Some interesting unpublished ideas on survey weighting
Introduction: A couple years ago we had an amazing all-star session at the Joint Statistical Meetings. The topic was new approaches to survey weighting (which is a mess , as I’m sure you’ve heard). Xiao-Li Meng recommended shrinking weights by taking them to a fractional power (such as square root) instead of trimming the extremes. Rod Little combined design-based and model-based survey inference. Michael Elliott used mixture models for complex survey design. And here’s my introduction to the session.
3 0.90744245 1230 andrew gelman stats-2012-03-26-Further thoughts on nonparametric correlation measures
Introduction: Malka Gorfine, Ruth Heller, and Yair Heller write a comment on the paper of Reshef et al. that we discussed a few months ago. Just to remind you what’s going on here, here’s my quick summary from December: Reshef et al. propose a new nonlinear R-squared-like measure. Unlike R-squared, this new method depends on a tuning parameter that controls the level of discretization, in a “How long is the coast of Britain” sort of way. The dependence on scale is inevitable for such a general method. Just consider: if you sample 1000 points from the unit bivariate normal distribution, (x,y) ~ N(0,I), you’ll be able to fit them perfectly by a 999-degree polynomial fit to the data. So the scale of the fit matters. The clever idea of the paper is that, instead of going for an absolute measure (which, as we’ve seen, will be scale-dependent), they focus on the problem of summarizing the grid of pairwise dependences in a large set of variables. As they put it: “Imagine a data set with hundreds
same-blog 4 0.90516019 309 andrew gelman stats-2010-10-01-Why Development Economics Needs Theory?
Introduction: Robert Neumann writes: in the JEP 24(3), page18, Daron Acemoglu states: Why Development Economics Needs Theory There is no general agreement on how much we should rely on economic theory in motivating empirical work and whether we should try to formulate and estimate “structural parameters.” I (Acemoglu) argue that the answer is largely “yes” because otherwise econometric estimates would lack external validity, in which case they can neither inform us about whether a particular model or theory is a useful approximation to reality, nor would they be useful in providing us guidance on what the effects of similar shocks and policies would be in different circumstances or if implemented in different scales. I therefore define “structural parameters” as those that provide external validity and would thus be useful in testing theories or in policy analysis beyond the specific environment and sample from which they are derived. External validity becomes a particularly challenging t
5 0.89303029 2324 andrew gelman stats-2014-05-07-Once more on nonparametric measures of mutual information
Introduction: Ben Murell writes: Our reply to Kinney and Atwal has come out (http://www.pnas.org/content/early/2014/04/29/1403623111.full.pdf) along with their response (http://www.pnas.org/content/early/2014/04/29/1404661111.full.pdf). I feel like they somewhat missed the point. If you’re still interested in this line of discussion, feel free to post, and maybe the Murrells and Kinney can bash it out in your comments! Background: Too many MC’s not enough MIC’s, or What principles should govern attempts to summarize bivariate associations in large multivariate datasets? Heller, Heller, and Gorfine on univariate and multivariate information measures Kinney and Atwal on the maximal information coefficient Mr. Pearson, meet Mr. Mandelbrot: Detecting Novel Associations in Large Data Sets Gorfine, Heller, Heller, Simon, and Tibshirani don’t like MIC The fun thing is that all these people are sending me their papers, and I’m enough of an outsider in this field that each of the
6 0.87461984 1557 andrew gelman stats-2012-11-01-‘Researcher Degrees of Freedom’
7 0.87377256 1136 andrew gelman stats-2012-01-23-Fight! (also a bit of reminiscence at the end)
8 0.86507368 1467 andrew gelman stats-2012-08-23-The pinch-hitter syndrome again
9 0.8631407 1616 andrew gelman stats-2012-12-10-John McAfee is a Heinlein hero
10 0.8594141 1362 andrew gelman stats-2012-06-03-Question 24 of my final exam for Design and Analysis of Sample Surveys
11 0.85075641 1076 andrew gelman stats-2011-12-21-Derman, Rodrik and the nature of statistical models
12 0.84428412 397 andrew gelman stats-2010-11-06-Multilevel quantile regression
13 0.82748306 1422 andrew gelman stats-2012-07-20-Likelihood thresholds and decisions
14 0.82719564 2315 andrew gelman stats-2014-05-02-Discovering general multidimensional associations
15 0.82472843 2359 andrew gelman stats-2014-06-04-All the Assumptions That Are My Life
16 0.8215034 2136 andrew gelman stats-2013-12-16-Whither the “bet on sparsity principle” in a nonsparse world?
17 0.81069469 1383 andrew gelman stats-2012-06-18-Hierarchical modeling as a framework for extrapolation
18 0.80906188 2097 andrew gelman stats-2013-11-11-Why ask why? Forward causal inference and reverse causal questions
19 0.80686939 1361 andrew gelman stats-2012-06-02-Question 23 of my final exam for Design and Analysis of Sample Surveys
20 0.80469632 1228 andrew gelman stats-2012-03-25-Continuous variables in Bayesian networks