andrew_gelman_stats andrew_gelman_stats-2012 andrew_gelman_stats-2012-1339 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: You can get a taste of Hamiltonian Monte Carlo (HMC) by reading the very gentle introduction in David MacKay’s general text on information theory: MacKay, D. 2003. Information Theory, Inference, and Learning Algorithms . Cambridge University Press. [see Chapter 31, which is relatively standalone and can be downloaded separately.] Follow this up with Radford Neal’s much more thorough introduction to HMC: Neal, R. 2011. MCMC Using Hamiltonian Dynamics . In Brooks, Gelman, Jones and Meng, eds., Handbook of Markov Chain Monte Carlo . Chapman and Hall/CRC Press. To understand why HMC works and set yourself on the path to understanding generalizations like Riemann manifold HMC , you’ll need to know a bit about differential geometry. I really liked the combination of these two books: Magnus, J. R. and H. Neudecker. 2007. Matrix Differential Calculus with Application in Statistics and Econometrics . 3rd Edition. Wiley? and Leimkuhler, B. and S.
sentIndex sentText sentNum sentScore
1 You can get a taste of Hamiltonian Monte Carlo (HMC) by reading the very gentle introduction in David MacKay’s general text on information theory: MacKay, D. [sent-1, score-0.497]
2 [see Chapter 31, which is relatively standalone and can be downloaded separately. [sent-5, score-0.26]
3 ] Follow this up with Radford Neal’s much more thorough introduction to HMC: Neal, R. [sent-6, score-0.297]
4 To understand why HMC works and set yourself on the path to understanding generalizations like Riemann manifold HMC , you’ll need to know a bit about differential geometry. [sent-12, score-0.488]
5 I really liked the combination of these two books: Magnus, J. [sent-13, score-0.061]
6 As a bonus, Magnus and Neudecker also provide an excellent introduction to matrix algebra and real analysis before mashing them up. [sent-27, score-0.49]
7 The question mark after “Wiley” is due to the fact that the preface says that the third-edition is self-published and copyright the authors and and available from the first author’s home page . [sent-28, score-0.507]
8 It’s no longer available on Magnus’s home page, nor is it available for sale by Wiley. [sent-29, score-0.471]
9 It can be found in PDF form on the web, though; try Googling [ matrix differential calculus magnus ]. [sent-30, score-1.027]
wordName wordTfidf (topN-words)
[('magnus', 0.441), ('hmc', 0.311), ('differential', 0.228), ('introduction', 0.217), ('hamiltonian', 0.217), ('matrix', 0.196), ('wiley', 0.186), ('mackay', 0.186), ('cambridge', 0.17), ('neal', 0.167), ('calculus', 0.162), ('dynamics', 0.16), ('carlo', 0.137), ('available', 0.133), ('monte', 0.13), ('manifold', 0.11), ('standalone', 0.11), ('home', 0.106), ('preface', 0.099), ('sale', 0.099), ('gentle', 0.096), ('chapman', 0.093), ('downloaded', 0.091), ('copyright', 0.087), ('handbook', 0.087), ('radford', 0.087), ('jones', 0.087), ('generalizations', 0.085), ('simulating', 0.084), ('page', 0.082), ('thorough', 0.08), ('bonus', 0.079), ('googling', 0.079), ('university', 0.078), ('meng', 0.078), ('algebra', 0.077), ('markov', 0.076), ('theory', 0.075), ('pdf', 0.074), ('brooks', 0.069), ('taste', 0.069), ('econometrics', 0.068), ('mcmc', 0.066), ('algorithms', 0.066), ('chain', 0.065), ('path', 0.065), ('liked', 0.061), ('relatively', 0.059), ('information', 0.058), ('text', 0.057)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 1339 andrew gelman stats-2012-05-23-Learning Differential Geometry for Hamiltonian Monte Carlo
Introduction: You can get a taste of Hamiltonian Monte Carlo (HMC) by reading the very gentle introduction in David MacKay’s general text on information theory: MacKay, D. 2003. Information Theory, Inference, and Learning Algorithms . Cambridge University Press. [see Chapter 31, which is relatively standalone and can be downloaded separately.] Follow this up with Radford Neal’s much more thorough introduction to HMC: Neal, R. 2011. MCMC Using Hamiltonian Dynamics . In Brooks, Gelman, Jones and Meng, eds., Handbook of Markov Chain Monte Carlo . Chapman and Hall/CRC Press. To understand why HMC works and set yourself on the path to understanding generalizations like Riemann manifold HMC , you’ll need to know a bit about differential geometry. I really liked the combination of these two books: Magnus, J. R. and H. Neudecker. 2007. Matrix Differential Calculus with Application in Statistics and Econometrics . 3rd Edition. Wiley? and Leimkuhler, B. and S.
2 0.32958895 931 andrew gelman stats-2011-09-29-Hamiltonian Monte Carlo stories
Introduction: Tomas Iesmantas had asked me for advice on a regression problem with 50 parameters, and I’d recommended Hamiltonian Monte Carlo. A few weeks later he reported back: After trying several modifications (HMC for all parameters at once, HMC just for first level parameters and Riemman manifold Hamiltonian Monte Carlo method), I finally got it running with HMC just for first level parameters and for others using direct sampling, since conditional distributions turned out to have closed form. However, even in this case it is quite tricky, since I had to employ mass matrix and not just diagonal but at the beginning of algorithm generated it randomly (ensuring it is positive definite). Such random generation of mass matrix is quite blind step, but it proved to be quite helpful. Riemman manifold HMC is quite vagarious, or to be more specific, metric of manifold is very sensitive. In my model log-likelihood I had exponents and values of metrics matrix elements was very large and wh
3 0.21947649 1772 andrew gelman stats-2013-03-20-Stan at Google this Thurs and at Berkeley this Fri noon
Introduction: Michael Betancourt will be speaking at Google and at the University of California, Berkeley. The Google talk is closed to outsiders (but if you work at Google, you should go!); the Berkeley talk is open to all: Friday March 22, 12:10 pm, Evans Hall 1011. Title of talk: Stan : Practical Bayesian Inference with Hamiltonian Monte Carlo Abstract: Practical implementations of Bayesian inference are often limited to approximation methods that only slowly explore the posterior distribution. By taking advantage of the curvature of the posterior, however, Hamiltonian Monte Carlo (HMC) efficiently explores even the most highly contorted distributions. In this talk I will review the foundations of and recent developments within HMC, concluding with a discussion of Stan, a powerful inference engine that utilizes HMC, automatic differentiation, and adaptive methods to minimize user input. This is cool stuff. And he’ll be showing the whirlpool movie!
4 0.19028232 1749 andrew gelman stats-2013-03-04-Stan in L.A. this Wed 3:30pm
Introduction: Michael Betancourt will be speaking at UCLA: The location for refreshment is in room 51-254 CHS at 3:00 PM. The place for the seminar is at CHS 33-105A at 3:30pm – 4:30pm, Wed 6 Mar. ["CHS" stands for Center for Health Sciences, the building of the UCLA schools of medicine and public health. Here's a map with directions .] Title of talk: Stan : Practical Bayesian Inference with Hamiltonian Monte Carlo Abstract: Practical implementations of Bayesian inference are often limited to approximation methods that only slowly explore the posterior distribution. By taking advantage of the curvature of the posterior, however, Hamiltonian Monte Carlo (HMC) efficiently explores even the most highly contorted distributions. In this talk I will review the foundations of and recent developments within HMC, concluding with a discussion of Stan, a powerful inference engine that utilizes HMC, automatic differentiation, and adaptive methods to minimize user input. This is cool stuff.
5 0.17121848 844 andrew gelman stats-2011-08-07-Update on the new Handbook of MCMC
Introduction: It’s edited by Steve Brooks, Galin Jones, Xiao-Li Meng, and myself. Here’s the information and some sample chapters (including my own chapter with Ken Shirley on inference and monitoring convergence and Radford’s instant classic on Hamiltonian Monte Carlo). Sorry about the $100 price tag–nobody asked me about that! But if you’re doing these computations as part of your work, I think the book will be well worth it.
6 0.16005999 984 andrew gelman stats-2011-11-01-David MacKay sez . . . 12??
7 0.14713591 2231 andrew gelman stats-2014-03-03-Running into a Stan Reference by Accident
8 0.13629042 1036 andrew gelman stats-2011-11-30-Stan uses Nuts!
9 0.13606007 2003 andrew gelman stats-2013-08-30-Stan Project: Continuous Relaxations for Discrete MRFs
10 0.11523924 674 andrew gelman stats-2011-04-21-Handbook of Markov Chain Monte Carlo
12 0.11166175 1041 andrew gelman stats-2011-12-04-David MacKay and Occam’s Razor
13 0.11034904 2311 andrew gelman stats-2014-04-29-Bayesian Uncertainty Quantification for Differential Equations!
14 0.10630602 2296 andrew gelman stats-2014-04-19-Index or indicator variables
15 0.10031677 535 andrew gelman stats-2011-01-24-Bleg: Automatic Differentiation for Log Prob Gradients?
16 0.098690853 555 andrew gelman stats-2011-02-04-Handy Matrix Cheat Sheet, with Gradients
17 0.095606007 1309 andrew gelman stats-2012-05-09-The first version of my “inference from iterative simulation using parallel sequences” paper!
18 0.094914816 2224 andrew gelman stats-2014-02-25-Basketball Stats: Don’t model the probability of win, model the expected score differential.
19 0.094442196 1991 andrew gelman stats-2013-08-21-BDA3 table of contents (also a new paper on visualization)
20 0.093397982 884 andrew gelman stats-2011-09-01-My course this fall on Bayesian Computation
topicId topicWeight
[(0, 0.089), (1, 0.026), (2, -0.044), (3, 0.042), (4, 0.031), (5, 0.062), (6, 0.002), (7, -0.069), (8, -0.038), (9, -0.047), (10, -0.025), (11, -0.028), (12, -0.057), (13, 0.002), (14, 0.033), (15, -0.023), (16, -0.007), (17, 0.018), (18, 0.03), (19, -0.041), (20, 0.004), (21, 0.018), (22, 0.05), (23, -0.0), (24, 0.066), (25, 0.063), (26, 0.01), (27, 0.062), (28, 0.041), (29, -0.028), (30, -0.039), (31, 0.012), (32, -0.002), (33, -0.007), (34, 0.02), (35, -0.024), (36, -0.01), (37, -0.047), (38, 0.017), (39, 0.018), (40, -0.039), (41, -0.012), (42, 0.018), (43, -0.019), (44, 0.006), (45, -0.019), (46, -0.046), (47, -0.021), (48, 0.034), (49, -0.068)]
simIndex simValue blogId blogTitle
same-blog 1 0.95512897 1339 andrew gelman stats-2012-05-23-Learning Differential Geometry for Hamiltonian Monte Carlo
Introduction: You can get a taste of Hamiltonian Monte Carlo (HMC) by reading the very gentle introduction in David MacKay’s general text on information theory: MacKay, D. 2003. Information Theory, Inference, and Learning Algorithms . Cambridge University Press. [see Chapter 31, which is relatively standalone and can be downloaded separately.] Follow this up with Radford Neal’s much more thorough introduction to HMC: Neal, R. 2011. MCMC Using Hamiltonian Dynamics . In Brooks, Gelman, Jones and Meng, eds., Handbook of Markov Chain Monte Carlo . Chapman and Hall/CRC Press. To understand why HMC works and set yourself on the path to understanding generalizations like Riemann manifold HMC , you’ll need to know a bit about differential geometry. I really liked the combination of these two books: Magnus, J. R. and H. Neudecker. 2007. Matrix Differential Calculus with Application in Statistics and Econometrics . 3rd Edition. Wiley? and Leimkuhler, B. and S.
2 0.72399199 555 andrew gelman stats-2011-02-04-Handy Matrix Cheat Sheet, with Gradients
Introduction: This post is an (unpaid) advertisement for the following extremely useful resource: Petersen, K. B. and M. S. Pedersen. 2008. The Matrix Cookbook . Tehcnical Report, Technical University of Denmark. It contains 70+ pages of useful relations and derivations involving matrices. What grabbed my eye was the computation of gradients for matrix operations ranging from eigenvalues and determinants to multivariate normal density functions. I had no idea the multivariate normal had such a clean gradient (see section 8). We’ve been playing around with Hamiltonian (aka Hybrid) Monte Carlo for sampling from the posterior of hierarchical generalized linear models with lots of interactions. HMC speeds up Metropolis sampling by using the gradient of the log probability to drive samples in the direction of higher probability density, which is particularly useful for correlated parameters that mix slowly with standard Gibbs sampling. Matt “III” Hoffman ‘s already got it workin
3 0.7035026 674 andrew gelman stats-2011-04-21-Handbook of Markov Chain Monte Carlo
Introduction: Galin Jones, Steve Brooks, Xiao-Li Meng and I edited a handbook of Markov Chain Monte Carlo that has just been published . My chapter (with Kenny Shirley) is here , and it begins like this: Convergence of Markov chain simulations can be monitored by measuring the diffusion and mixing of multiple independently-simulated chains, but different levels of convergence are appropriate for different goals. When considering inference from stochastic simulation, we need to separate two tasks: (1) inference about parameters and functions of parameters based on broad characteristics of their distribution, and (2) more precise computation of expectations and other functions of probability distributions. For the first task, there is a natural limit to precision beyond which additional simulations add essentially nothing; for the second task, the appropriate precision must be decided from external considerations. We illustrate with an example from our current research, a hierarchical model of t
4 0.69221622 931 andrew gelman stats-2011-09-29-Hamiltonian Monte Carlo stories
Introduction: Tomas Iesmantas had asked me for advice on a regression problem with 50 parameters, and I’d recommended Hamiltonian Monte Carlo. A few weeks later he reported back: After trying several modifications (HMC for all parameters at once, HMC just for first level parameters and Riemman manifold Hamiltonian Monte Carlo method), I finally got it running with HMC just for first level parameters and for others using direct sampling, since conditional distributions turned out to have closed form. However, even in this case it is quite tricky, since I had to employ mass matrix and not just diagonal but at the beginning of algorithm generated it randomly (ensuring it is positive definite). Such random generation of mass matrix is quite blind step, but it proved to be quite helpful. Riemman manifold HMC is quite vagarious, or to be more specific, metric of manifold is very sensitive. In my model log-likelihood I had exponents and values of metrics matrix elements was very large and wh
5 0.67907 1991 andrew gelman stats-2013-08-21-BDA3 table of contents (also a new paper on visualization)
Introduction: In response to our recent posting of Amazon’s offer of Bayesian Data Analysis 3rd edition at 40% off, some people asked what was in this new edition, with more information beyond the beautiful cover image and the brief paragraph I’d posted earlier. Here’s the table of contents. The following sections have all-new material: 1.4 New introduction of BDA principles using a simple spell checking example 2.9 Weakly informative prior distributions 5.7 Weakly informative priors for hierarchical variance parameters 7.1-7.4 Predictive accuracy for model evaluation and comparison 10.6 Computing environments 11.4 Split R-hat 11.5 New measure of effective number of simulation draws 13.7 Variational inference 13.8 Expectation propagation 13.9 Other approximations 14.6 Regularization for regression models C.1 Getting started with R and Stan C.2 Fitting a hierarchical model in Stan C.4 Programming Hamiltonian Monte Carlo in R And the new chapters: 20 Basis function models 2
6 0.67901969 535 andrew gelman stats-2011-01-24-Bleg: Automatic Differentiation for Log Prob Gradients?
7 0.66372639 844 andrew gelman stats-2011-08-07-Update on the new Handbook of MCMC
8 0.65348446 1477 andrew gelman stats-2012-08-30-Visualizing Distributions of Covariance Matrices
9 0.63602829 1710 andrew gelman stats-2013-02-06-The new Stan 1.1.1, featuring Gaussian processes!
10 0.6261571 1753 andrew gelman stats-2013-03-06-Stan 1.2.0 and RStan 1.2.0
11 0.62602681 1749 andrew gelman stats-2013-03-04-Stan in L.A. this Wed 3:30pm
12 0.62118399 1036 andrew gelman stats-2011-11-30-Stan uses Nuts!
13 0.59533328 1799 andrew gelman stats-2013-04-12-Stan 1.3.0 and RStan 1.3.0 Ready for Action
14 0.58905739 2003 andrew gelman stats-2013-08-30-Stan Project: Continuous Relaxations for Discrete MRFs
15 0.57925189 1772 andrew gelman stats-2013-03-20-Stan at Google this Thurs and at Berkeley this Fri noon
16 0.5770672 2231 andrew gelman stats-2014-03-03-Running into a Stan Reference by Accident
17 0.56971055 1627 andrew gelman stats-2012-12-17-Stan and RStan 1.1.0
18 0.54835296 1682 andrew gelman stats-2013-01-19-R package for Bayes factors
20 0.54129595 2150 andrew gelman stats-2013-12-27-(R-Py-Cmd)Stan 2.1.0
topicId topicWeight
[(0, 0.012), (7, 0.011), (9, 0.011), (15, 0.011), (16, 0.047), (19, 0.013), (21, 0.041), (24, 0.046), (27, 0.069), (29, 0.044), (38, 0.093), (39, 0.032), (40, 0.01), (41, 0.028), (56, 0.035), (68, 0.021), (73, 0.014), (77, 0.031), (82, 0.076), (85, 0.032), (96, 0.026), (98, 0.011), (99, 0.189)]
simIndex simValue blogId blogTitle
same-blog 1 0.92200428 1339 andrew gelman stats-2012-05-23-Learning Differential Geometry for Hamiltonian Monte Carlo
Introduction: You can get a taste of Hamiltonian Monte Carlo (HMC) by reading the very gentle introduction in David MacKay’s general text on information theory: MacKay, D. 2003. Information Theory, Inference, and Learning Algorithms . Cambridge University Press. [see Chapter 31, which is relatively standalone and can be downloaded separately.] Follow this up with Radford Neal’s much more thorough introduction to HMC: Neal, R. 2011. MCMC Using Hamiltonian Dynamics . In Brooks, Gelman, Jones and Meng, eds., Handbook of Markov Chain Monte Carlo . Chapman and Hall/CRC Press. To understand why HMC works and set yourself on the path to understanding generalizations like Riemann manifold HMC , you’ll need to know a bit about differential geometry. I really liked the combination of these two books: Magnus, J. R. and H. Neudecker. 2007. Matrix Differential Calculus with Application in Statistics and Econometrics . 3rd Edition. Wiley? and Leimkuhler, B. and S.
2 0.82105851 393 andrew gelman stats-2010-11-04-Estimating the effect of A on B, and also the effect of B on A
Introduction: Lei Liu writes: I am working with clinicians in infectious disease and international health to study the (possible causal) relation between malnutrition and virus infection episodes (e.g., diarrhea) in babies in developing countries. Basically the clinicians are interested in two questions: does malnutrition cause more diarrhea episodes? does diarrhea lead to malnutrition? The malnutrition status is indicated by height and weight (adjusted, HAZ and WAZ measures) observed every 3 months from birth to 1 year. They also recorded the time of each diarrhea episode during the 1 year follow-up period. They have very solid datasets for analysis. As you can see, this is almost like a chicken and egg problem. I am a layman to causal inference. The method I use is just to do some simple regression. For example, to study the causal relation from malnutrition to diarrhea episodes, I use binary variable (diarrhea yes/no during months 0-3) as response, and use the HAZ at month 0 as covariate
3 0.81806391 527 andrew gelman stats-2011-01-20-Cars vs. trucks
Introduction: Anupam Agrawal writes: I am an Assistant Professor of Operations Management at the University of Illinois. . . . My main work is in supply chain area, and empirical in nature. . . . I am working with a firm that has two separate divisions – one making cars, and the other makes trucks. Four years back, the firm made an interesting organizational change. They created a separate group of ~25 engineers, in their car division (from within their quality and production engineers). This group was focused on improving supplier quality and reported to car plant head . The truck division did not (and still does not) have such an independent “supplier improvement group”. Other than this unit in car, the organizational arrangements in the two divisions mimic each other. There are many common suppliers to the car and truck division. Data on quality of components coming from suppliers has been collected (for the last four years). The organizational change happened in January 2007. My focus is
4 0.81531221 1874 andrew gelman stats-2013-05-28-Nostalgia
Introduction: Saw Argo the other day, was impressed by the way it was filmed in such a 70s style, sorta like that movie The Limey or an episode of the Rockford Files. I also felt nostalgia for that relatively nonviolent era. All those hostages and nobody was killed. It’s a good thing the Ayatollah didn’t have some fundamentalist Shiite equivalent of John Yoo telling him to waterboard everybody. At the time we were all so angry and upset about the hostage-taking, but from the perspective of our suicide-bomber era, that whole hostage episode seems so comfortingly mild.
5 0.8127929 173 andrew gelman stats-2010-07-31-Editing and clutch hitting
Introduction: Regarding editing : The only serious editing I’ve ever received has been for my New York Times op-eds and my article in the American Scientist. My book editors have all been nice people, and they’ve helped me with many things (including suggestions of what my priorities should be in communicating with readers)–they’ve been great–but they’ve not given (nor have I expected or asked for) serious editing. Maybe I should’ve asked for it, I don’t know. I’ve had time-wasting experiences with copy editors and a particularly annoying experience with a production editor (who was so difficult that my coauthors and I actually contacted our agent and a lawyer about the possibility of getting out of our contract), but that’s another story. Regarding clutch hitting , Bill James once noted that it’s great when a Bucky Dent hits an unexpected home run, but what’s really special is being able to get the big hit when it’s expected of you. The best players can do their best every time they come t
6 0.81225097 340 andrew gelman stats-2010-10-13-Randomized experiments, non-randomized experiments, and observational studies
7 0.81038505 699 andrew gelman stats-2011-05-06-Another stereotype demolished
8 0.80988991 1498 andrew gelman stats-2012-09-16-Choices in graphing parallel time series
9 0.80859303 193 andrew gelman stats-2010-08-09-Besag
10 0.80748069 1958 andrew gelman stats-2013-07-27-Teaching is hard
11 0.80692893 1440 andrew gelman stats-2012-08-02-“A Christmas Carol” as applied to plagiarism
12 0.80687827 67 andrew gelman stats-2010-06-03-More on that Dartmouth health care study
13 0.80644625 178 andrew gelman stats-2010-08-03-(Partisan) visualization of health care legislation
14 0.80604649 251 andrew gelman stats-2010-09-02-Interactions of predictors in a causal model
15 0.80502021 686 andrew gelman stats-2011-04-29-What are the open problems in Bayesian statistics??
16 0.80474722 600 andrew gelman stats-2011-03-04-“Social Psychologists Detect Liberal Bias Within”
17 0.80225611 606 andrew gelman stats-2011-03-10-It’s no fun being graded on a curve
18 0.79906398 326 andrew gelman stats-2010-10-07-Peer pressure, selection, and educational reform
19 0.79850477 1553 andrew gelman stats-2012-10-30-Real rothko, fake rothko