andrew_gelman_stats andrew_gelman_stats-2013 andrew_gelman_stats-2013-1889 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Torbjørn Skardhamar writes: I am a sociologist/criminologist working at Statistics Norway. As I am not a trained statistician, I find myself sometimes in need to check basic statistical concepts. Recently, I came across an article which I found a bit strange, but I needed to check up on my statistical understanding of a very basic concept: the r-squared. When doing so, I realized that this was also an interesting case of research ethics. Given your interest in research ethics, I though this might be interesting to you. Here’s the mentioned article, by Weisburd and Piquero, is attached. What they do is to analyzed reported results from all articles published in the highest ranking criminological journal since 1968 through 2005 to determine whether there are any progress in the field of criminology. Their approach is basically to calculate the average r-square from linear models in published articles. For example, they state that “variance explained provides one way to assess
sentIndex sentText sentNum sentScore
1 As I am not a trained statistician, I find myself sometimes in need to check basic statistical concepts. [sent-2, score-0.325]
2 Recently, I came across an article which I found a bit strange, but I needed to check up on my statistical understanding of a very basic concept: the r-squared. [sent-3, score-0.435]
3 When doing so, I realized that this was also an interesting case of research ethics. [sent-4, score-0.169]
4 What they do is to analyzed reported results from all articles published in the highest ranking criminological journal since 1968 through 2005 to determine whether there are any progress in the field of criminology. [sent-7, score-0.411]
5 Their approach is basically to calculate the average r-square from linear models in published articles. [sent-8, score-0.168]
6 For example, they state that “variance explained provides one way to assess the state of the science of criminology and its relevance for public policy, and how that science has changed over time” (page 455, final paragraph). [sent-9, score-0.459]
7 They find that the “explained variance” is generally low – and even on the decline, so there has not been much progress, and they conclude: “That criminology is not developing models of crime with more explanatory power over time is troubling” (page 491, first sentence). [sent-10, score-0.43]
8 I needed to look up in my old statistical text books to find out if interpreting the r-squared statistics in this way made much sense. [sent-11, score-0.272]
9 I think it doesn’t, but I’m not entirely sure if there might be some circumstances where it might be meaningful after all. [sent-12, score-0.215]
10 The research ethical issue is related to the statistical issue. [sent-15, score-0.356]
11 While trying to find out if I had misunderstood something about r-squard, I came across Gary King’s article (1986) “How to not lie with statistics” where he also discusses the direct interpretation of r-squared. [sent-16, score-0.364]
12 If I got it right, King argues that r-squared is only meaningful for comparing models on the same data with the same outcome variable. [sent-17, score-0.16]
13 As far as I can understand, then, King’s argument implies that calculating the average r-squared across studies does not make much sense. [sent-18, score-0.259]
14 com/2012/10/r-squared-of-1/ Then I realized that Weisburd and Piquero actually cites King’s article (on page 464), but only on a side note that one can easily manipulate to get a higher r-squared. [sent-21, score-0.382]
15 As Weisburd and Piquero cite King’s article, we must assume they have read the main arguments too. [sent-22, score-0.217]
16 The ethical issue is then when authors delibrately ignore highly relevant arguments that might undermine their own publication. [sent-24, score-0.522]
17 At the least, they should have discussed the counter arguments properly. [sent-26, score-0.216]
18 By the way, it should be noted that Weisburd and Piquero are much cited criminologists having some impact in the criminological literature. [sent-27, score-0.259]
19 But I question the premise that progress in criminology research would be characterized by an increase of explained variance. [sent-30, score-0.723]
20 When I think of my own applied research, progress is typically not measured by pure prediction. [sent-31, score-0.171]
wordName wordTfidf (topN-words)
[('piquero', 0.446), ('weisburd', 0.446), ('king', 0.296), ('criminology', 0.23), ('criminological', 0.178), ('progress', 0.171), ('arguments', 0.154), ('explained', 0.121), ('ethical', 0.099), ('meaningful', 0.099), ('page', 0.091), ('ignore', 0.09), ('realized', 0.089), ('across', 0.086), ('purpose', 0.084), ('criminologists', 0.081), ('beginners', 0.081), ('research', 0.08), ('article', 0.079), ('find', 0.079), ('needed', 0.077), ('variance', 0.073), ('http', 0.073), ('troubling', 0.069), ('undermine', 0.067), ('basic', 0.066), ('premise', 0.065), ('misunderstood', 0.065), ('check', 0.065), ('main', 0.063), ('statistical', 0.062), ('counter', 0.062), ('manipulate', 0.062), ('ranking', 0.062), ('related', 0.061), ('models', 0.061), ('argument', 0.061), ('cites', 0.061), ('sole', 0.06), ('explanatory', 0.06), ('might', 0.058), ('calculating', 0.057), ('characterized', 0.056), ('average', 0.055), ('lie', 0.055), ('state', 0.054), ('interpreting', 0.054), ('issue', 0.054), ('trained', 0.053), ('calculate', 0.052)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999988 1889 andrew gelman stats-2013-06-08-Using trends in R-squared to measure progress in criminology??
Introduction: Torbjørn Skardhamar writes: I am a sociologist/criminologist working at Statistics Norway. As I am not a trained statistician, I find myself sometimes in need to check basic statistical concepts. Recently, I came across an article which I found a bit strange, but I needed to check up on my statistical understanding of a very basic concept: the r-squared. When doing so, I realized that this was also an interesting case of research ethics. Given your interest in research ethics, I though this might be interesting to you. Here’s the mentioned article, by Weisburd and Piquero, is attached. What they do is to analyzed reported results from all articles published in the highest ranking criminological journal since 1968 through 2005 to determine whether there are any progress in the field of criminology. Their approach is basically to calculate the average r-square from linear models in published articles. For example, they state that “variance explained provides one way to assess
2 0.1291815 365 andrew gelman stats-2010-10-24-Erving Goffman archives
Introduction: Brayden King points to this page of materials on sociologist Erving Goffman. Whenever I’ve read about Goffman, it always seems to be in conjunction with some story about his bad behavior–in that respect, King’s link above does not disappoint. In the absence of any context, it all seems mysterious to me Once or twice I’ve tried to read passages in books by Goffman but have never manage to get through any of it. (This is not mean as any kind of criticism, it’s just a statement of my lack of knowledge.) I was amused enough by the stories reported by King that I clicked through to the Biographical Materials section of the Goffman page and read a few. I still couldn’t really quite get the point, though, perhaps in part because I only know one of the many people on that list.
3 0.10772339 2279 andrew gelman stats-2014-04-02-Am I too negative?
Introduction: For background, you can start by reading my recent article, Is It Possible to Be an Ethicist Without Being Mean to People? and then a blog post, Quality over Quantity , by John Cook, who writes: At one point [Ed] Tufte spoke more generally and more personally about pursuing quality over quantity. He said most papers are not worth reading and that he learned early on to concentrate on the great papers, maybe one in 500, that are worth reading and rereading rather than trying to “keep up with the literature.” He also explained how over time he has concentrated more on showcasing excellent work than on criticizing bad work. You can see this in the progression from his first book to his latest. (Criticizing bad work is important too, but you’ll have to read his early books to find more of that. He won’t spend as much time talking about it in his course.) That reminded me of Jesse Robbins’ line: “Don’t fight stupid. You are better than that. Make more awesome.” This made me stop an
4 0.087209918 2245 andrew gelman stats-2014-03-12-More on publishing in journals
Introduction: I’m postponing today’s scheduled post (“Empirical implications of Empirical Implications of Theoretical Models”) to continue the lively discussion from yesterday, What if I were to stop publishing in journals? . An example: my papers with Basbøll Thomas Basbøll and I got into a long discussion on our blogs about business school professor Karl Weick and other cases of plagiarism copying text without attribution. We felt it useful to take our ideas to the next level and write them up as a manuscript, which ended up being logical to split into two papers. At that point I put some effort into getting these papers published, which I eventually did: To throw away data: Plagiarism as a statistical crime went into American Scientist and When do stories work? Evidence and illustration in the social sciences will appear in Sociological Methods and Research. The second paper, in particular, took some effort to place; I got some advice from colleagues in sociology as to where
5 0.082310654 1561 andrew gelman stats-2012-11-04-Someone is wrong on the internet
Introduction: I made the mistake of googling myself (I know, I know . . .) and came across a couple of rude bloggers criticizing something I’d written. I don’t mind criticism, and lord knows I can be a rude blogger myself at times, but these criticisms were really bad, a mix of already-refuted arguments and new claims that were just flat-out ridiculous. Really bad stuff. I then spent about an hour, on and off, writing a long long post explaining why they were wrong and how they could make their arguments better. But then, before I hit Send, I realized it would a mistake to post my response. Getting into a fight with these people whom I’d never heard of before . . . what’s the point? If they want to comment on my blog, I will respond (within reason), or if they are well known researchers or journalists, it’s perhaps worth correcting them. Or if they made an interesting argument, sure. But there’s no point in scouring the web looking for bad arguments to refute. That way lies madness. I w
6 0.080818839 1435 andrew gelman stats-2012-07-30-Retracted articles and unethical behavior in economics journals?
7 0.077644423 1080 andrew gelman stats-2011-12-24-Latest in blog advertising
8 0.077207088 1996 andrew gelman stats-2013-08-24-All inference is about generalizing from sample to population
9 0.073875807 1395 andrew gelman stats-2012-06-27-Cross-validation (What is it good for?)
10 0.072294585 2172 andrew gelman stats-2014-01-14-Advice on writing research articles
12 0.070311271 1695 andrew gelman stats-2013-01-28-Economists argue about Bayes
13 0.069972381 2117 andrew gelman stats-2013-11-29-The gradual transition to replicable science
14 0.069944724 2232 andrew gelman stats-2014-03-03-What is the appropriate time scale for blogging—the day or the week?
15 0.069444887 765 andrew gelman stats-2011-06-14-How the ignorant idiots win, explained. Maybe.
16 0.068662569 1139 andrew gelman stats-2012-01-26-Suggested resolution of the Bem paradox
17 0.067764856 499 andrew gelman stats-2011-01-03-5 books
18 0.067760088 846 andrew gelman stats-2011-08-09-Default priors update?
19 0.067449793 785 andrew gelman stats-2011-07-02-Experimental reasoning in social science
20 0.066982873 120 andrew gelman stats-2010-06-30-You can’t put Pandora back in the box
topicId topicWeight
[(0, 0.178), (1, -0.014), (2, -0.022), (3, -0.038), (4, -0.007), (5, -0.003), (6, -0.026), (7, -0.024), (8, 0.006), (9, 0.04), (10, 0.018), (11, -0.016), (12, 0.014), (13, -0.006), (14, 0.009), (15, 0.03), (16, -0.015), (17, 0.019), (18, -0.003), (19, -0.011), (20, 0.01), (21, -0.009), (22, 0.024), (23, 0.018), (24, 0.018), (25, 0.002), (26, 0.022), (27, 0.027), (28, -0.014), (29, -0.002), (30, 0.013), (31, 0.019), (32, 0.004), (33, -0.011), (34, 0.004), (35, -0.019), (36, 0.015), (37, -0.014), (38, 0.054), (39, -0.006), (40, 0.035), (41, -0.037), (42, 0.015), (43, 0.002), (44, 0.006), (45, -0.036), (46, -0.017), (47, -0.029), (48, -0.009), (49, -0.002)]
simIndex simValue blogId blogTitle
same-blog 1 0.96476346 1889 andrew gelman stats-2013-06-08-Using trends in R-squared to measure progress in criminology??
Introduction: Torbjørn Skardhamar writes: I am a sociologist/criminologist working at Statistics Norway. As I am not a trained statistician, I find myself sometimes in need to check basic statistical concepts. Recently, I came across an article which I found a bit strange, but I needed to check up on my statistical understanding of a very basic concept: the r-squared. When doing so, I realized that this was also an interesting case of research ethics. Given your interest in research ethics, I though this might be interesting to you. Here’s the mentioned article, by Weisburd and Piquero, is attached. What they do is to analyzed reported results from all articles published in the highest ranking criminological journal since 1968 through 2005 to determine whether there are any progress in the field of criminology. Their approach is basically to calculate the average r-square from linear models in published articles. For example, they state that “variance explained provides one way to assess
Introduction: David Jinkins writes : The objective of this paper is to measure the relative importance of conspicous consumption to Americans and Chinese. To this end, I estimate the parameters of a utility function borrowed from recent theoretical work using American and Chinese data. The main parameter of interest governs the amount that individuals care about peer group beliefs regarding their welfare. Using survey data on the visibility of different good categories along with household budget surveys, I find that Chinese consumers care twice as much as American consumers about the beliefs of their peer group. I came across this draft research manuscript by following the links back after Jinkins commented on our blog. The framing of the paper is a bit more foundation-y and a bit less statistic-y than I’d prefer, but I guess that’s just the way they do things in economics, compared to statistics or (some) political science. In any case, I wanted to point you to this paper, partly to let y
3 0.7778821 32 andrew gelman stats-2010-05-14-Causal inference in economics
Introduction: Aaron Edlin points me to this issue of the Journal of Economic Perspectives that focuses on statistical methods for causal inference in economics. (Michael Bishop’s page provides some links .) To quickly summarize my reactions to Angrist and Pischke’s book: I pretty much agree with them that the potential-outcomes or natural-experiment approach is the most useful way to think about causality in economics and related fields. My main amendments to Angrist and Pischke would be to recognize that: 1. Modeling is important, especially modeling of interactions . It’s unfortunate to see a debate between experimentalists and modelers. Some experimenters (not Angrist and Pischke) make the mistake of avoiding models: Once they have their experimental data, they check their brains at the door and do nothing but simple differences, not realizing how much more can be learned. Conversely, some modelers are unduly dismissive of experiments and formal observational studies, forgetting t
4 0.76828504 490 andrew gelman stats-2010-12-29-Brain Structure and the Big Five
Introduction: Many years ago, a research psychologist whose judgment I greatly respect told me that the characterization of personality by the so-called Big Five traits (extraversion, etc.) was old-fashioned. So I’m always surprised to see that the Big Five keeps cropping up. I guess not everyone agrees that it’s a bad idea. For example, Hamdan Azhar wrote to me: I was wondering if you’d seen this recent paper (De Young et al. 2010) that finds significant correlations between brain volume in selected regions and personality trait measures (from the Big Five). This is quite a ground-breaking finding and it was covered extensively in the mainstream media. I think readers of your blog would be interested in your thoughts, statistically speaking, on their methodology and findings. My reply: I’d be interested in my thoughts on this too! But I don’t know enough to say anything useful. From the abstract of the paper under discussion: Controlling for age, sex, and whole-brain volume
Introduction: The other day, Nicholas Christakis wrote an article in the newspaper criticizing academic social science departments: The social sciences have stagnated. . . . This is not only boring but also counterproductive, constraining engagement with the scientific cutting edge and stifling the creation of new and useful knowledge. . . . I’m not suggesting that social scientists stop teaching and investigating classic topics like monopoly power, racial profiling and health inequality. But everyone knows that monopoly power is bad for markets, that people are racially biased and that illness is unequally distributed by social class. There are diminishing returns from the continuing study of many such topics. And repeatedly observing these phenomena does not help us fix them. I disagreed , saying that Christakis wasn’t giving social science research enough credit: I’m no economist so I can let others discuss the bit about “monopoly power is bad for markets.” I assume that the study by
6 0.75058627 69 andrew gelman stats-2010-06-04-A Wikipedia whitewash
7 0.74977869 1585 andrew gelman stats-2012-11-20-“I know you aren’t the plagiarism police, but . . .”
8 0.74033451 789 andrew gelman stats-2011-07-07-Descriptive statistics, causal inference, and story time
9 0.73832506 1096 andrew gelman stats-2012-01-02-Graphical communication for legal scholarship
10 0.73741168 2172 andrew gelman stats-2014-01-14-Advice on writing research articles
11 0.73497671 1031 andrew gelman stats-2011-11-27-Richard Stallman and John McCarthy
12 0.7340219 2057 andrew gelman stats-2013-10-10-Chris Chabris is irritated by Malcolm Gladwell
13 0.73353893 1861 andrew gelman stats-2013-05-17-Where do theories come from?
14 0.73344308 989 andrew gelman stats-2011-11-03-This post does not mention Wegman
15 0.72872937 2330 andrew gelman stats-2014-05-12-Historical Arc of Universities
16 0.72670108 1680 andrew gelman stats-2013-01-18-“If scientists wrote horoscopes, this is what yours would say”
18 0.72385788 1645 andrew gelman stats-2012-12-31-Statistical modeling, causal inference, and social science
20 0.72259414 1714 andrew gelman stats-2013-02-09-Partial least squares path analysis
topicId topicWeight
[(15, 0.024), (16, 0.126), (21, 0.014), (24, 0.143), (52, 0.012), (53, 0.016), (54, 0.162), (84, 0.013), (86, 0.016), (99, 0.322)]
simIndex simValue blogId blogTitle
1 0.97680938 322 andrew gelman stats-2010-10-06-More on the differences between drugs and medical devices
Introduction: Someone who works in statistics in the pharmaceutical industry (but prefers to remain anonymous) sent me this update to our discussion on the differences between approvals of drugs and medical devices: The ‘substantial equivalence’ threshold is a very outdated. Basically the FDA has to follow federal law and the law is antiquated and leads to two extraordinarily different paths for device approval. You could have a very simple but first-in-kind device with an easy to understand physiological mechanism of action (e.g. the FDA approved a simple tiny stent that would relieve pressure from a glaucoma patient’s eye this summer). This device would require a standard (likely controlled) trial at the one-sided 0.025 level. Even after the trial it would likely go to a panel where outside experts (e.g.practicing & academic MDs and statisticians) hear evidence from the company and FDA and vote on its safety and efficacy. FDA would then rule, consider the panel’s vote, on whether to appro
2 0.95725811 1938 andrew gelman stats-2013-07-14-Learning how to speak
Introduction: I’ve been trying to reduce my American accent when speaking French. I tried taping my voice and playing it back, but that didn’t help. I couldn’t actually tell that I had a strong accent by listening to myself. My own voice is just too familiar to me. Then Malecki told me about the international phonetic alphabet, which is just great. And there’s even a convenient website that translates. For example, le loup est revenu -> lə lu ε ʀəvny I stared at Malecki’s mouth while he said the phrase, and I finally understood the difference between the two different “oo” sounds. That evening at home I tried it out on the local expert and he laughed at my attempts but grudgingly admitted I was getting better. On about the 10th try, after watching him say it over and over and staring at his mouth, I was finally able to do it! I know this is going to sound stupid to all you linguistics experts out there, but I had no idea that you could figure out how to speak better by staring at s
3 0.95078731 615 andrew gelman stats-2011-03-16-Chess vs. checkers
Introduction: Mark Palko writes : Chess derives most of its complexity through differentiated pieces; with checkers the complexity comes from the interaction between pieces. The result is a series of elegant graph problems where the viable paths change with each move of your opponent. To draw an analogy with chess, imagine if moving your knight could allow your opponent’s bishop to move like a rook. Add to that the potential for traps and manipulation that come with forced capture and you have one of the most remarkable games of all time. . . . It’s not unusual to hear masters of both chess and checkers (draughts) to admit that they prefer the latter. So why does chess get all the respect? Why do you never see a criminal mastermind or a Bond villain playing in a checkers tournament? Part of the problem is that we learn the game as children so we tend to think of it as a children’s game. We focus on how simple the rules are and miss how much complexity and subtlety you can get out of those ru
same-blog 4 0.94926345 1889 andrew gelman stats-2013-06-08-Using trends in R-squared to measure progress in criminology??
Introduction: Torbjørn Skardhamar writes: I am a sociologist/criminologist working at Statistics Norway. As I am not a trained statistician, I find myself sometimes in need to check basic statistical concepts. Recently, I came across an article which I found a bit strange, but I needed to check up on my statistical understanding of a very basic concept: the r-squared. When doing so, I realized that this was also an interesting case of research ethics. Given your interest in research ethics, I though this might be interesting to you. Here’s the mentioned article, by Weisburd and Piquero, is attached. What they do is to analyzed reported results from all articles published in the highest ranking criminological journal since 1968 through 2005 to determine whether there are any progress in the field of criminology. Their approach is basically to calculate the average r-square from linear models in published articles. For example, they state that “variance explained provides one way to assess
5 0.94902062 1676 andrew gelman stats-2013-01-16-Detecting cheating in chess
Introduction: Three different people have pointed me to this post by Ken Regan on statistical evaluation of claims of cheating in chess. So I figured I have to satisfy demand and post something on this. But I have nothing to say. All these topics interest me, but I somehow had difficulty reading through the entire post. I scanned through but what I really wanted to see was some data. Show me a scatterplot, then I’ll get interested. P.S. This is meant as no disparagement of Regan or his blog. I just couldn’t quite get into this particular example.
6 0.94835079 839 andrew gelman stats-2011-08-04-To commenters who are trying to sell something
7 0.94606602 358 andrew gelman stats-2010-10-20-When Kerry Met Sally: Politics and Perceptions in the Demand for Movies
8 0.9384675 94 andrew gelman stats-2010-06-17-SAT stories
9 0.92770815 1105 andrew gelman stats-2012-01-08-Econ debate about prices at a fancy restaurant
10 0.92737579 1083 andrew gelman stats-2011-12-26-The quals and the quants
11 0.92627347 2121 andrew gelman stats-2013-12-02-Should personal genetic testing be regulated? Battle of the blogroll
12 0.9254896 1473 andrew gelman stats-2012-08-28-Turing chess run update
13 0.92512894 1656 andrew gelman stats-2013-01-05-Understanding regression models and regression coefficients
14 0.92287457 1721 andrew gelman stats-2013-02-13-A must-read paper on statistical analysis of experimental data
15 0.91232288 1578 andrew gelman stats-2012-11-15-Outta control political incorrectness
16 0.91050482 2137 andrew gelman stats-2013-12-17-Replication backlash
17 0.90956241 867 andrew gelman stats-2011-08-23-The economics of the mac? A paradox of competition
18 0.90918636 1878 andrew gelman stats-2013-05-31-How to fix the tabloids? Toward replicable social science research
19 0.90764672 503 andrew gelman stats-2011-01-04-Clarity on my email policy
20 0.9074564 1163 andrew gelman stats-2012-02-12-Meta-analysis, game theory, and incentives to do replicable research