andrew_gelman_stats andrew_gelman_stats-2013 andrew_gelman_stats-2013-1959 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: This is a killer story (from Brian Nosek, Jeffrey Spies, and Matt Motyl). Part 1: Two of the present authors, Motyl and Nosek, share interests in political ideology. We were inspired by the fast growing literature on embodiment that demonstrates surprising links between body and mind (Markman & Brendl, 2005; Proffitt, 2006) to investigate embodiment of political extremism. Participants from the political left, right and center (N = 1,979) completed a perceptual judgment task in which words were presented in different shades of gray. Participants had to click along a gradient representing grays from near black to near white to select a shade that matched the shade of the word. We calculated accuracy: How close to the actual shade did participants get? The results were stunning. Moderates perceived the shades of gray more accurately than extremists on the left and right (p = .01). Our conclusion: political extremists perceive the world in black-and-white, figuratively and literally
sentIndex sentText sentNum sentScore
1 Part 1: Two of the present authors, Motyl and Nosek, share interests in political ideology. [sent-2, score-0.098]
2 We were inspired by the fast growing literature on embodiment that demonstrates surprising links between body and mind (Markman & Brendl, 2005; Proffitt, 2006) to investigate embodiment of political extremism. [sent-3, score-0.597]
3 Participants from the political left, right and center (N = 1,979) completed a perceptual judgment task in which words were presented in different shades of gray. [sent-4, score-0.438]
4 Participants had to click along a gradient representing grays from near black to near white to select a shade that matched the shade of the word. [sent-5, score-0.889]
5 We calculated accuracy: How close to the actual shade did participants get? [sent-6, score-0.437]
6 Moderates perceived the shades of gray more accurately than extremists on the left and right (p = . [sent-8, score-0.405]
7 Our conclusion: political extremists perceive the world in black-and-white, figuratively and literally. [sent-10, score-0.352]
8 Our design and follow-up analyses ruled out obvious alternative explanations such as time spent on task and a tendency to select extreme responses. [sent-11, score-0.303]
9 The ultimate publication, Motyl and Nosek (2012) served as one of Motyl’s signature publications as he finished graduate school and entered the job market. [sent-13, score-0.158]
10 Part 2: The story is all true, except for the last sentence; we did not publish the finding. [sent-14, score-0.079]
11 Two recent papers highlighted the possibility that research practices spuriously inflate the presence of positive results in the published literature (John, Loewenstein, & Prelec, 2012; Simmons, Nelson, & Simonsohn, 2011). [sent-16, score-0.398]
12 But, we had been discussing reproducibility, and we had declared to our lab mates the importance of replication for increasing certainty of research results. [sent-19, score-0.231]
13 We could not justify skipping replication on the grounds of feasibility or resource constraints. [sent-23, score-0.399]
14 Finally, the procedure had been created by someone else for another purpose, and we had not laid out our analysis strategy in advance. [sent-24, score-0.065]
15 We could have made analysis decisions that increased the likelihood of obtaining results aligned with our hypothesis. [sent-25, score-0.147]
16 We conducted a direct replication while we prepared the manuscript. [sent-27, score-0.222]
17 995 power to detect an effect of the original effect size at alpha = . [sent-29, score-0.216]
18 Their paper is all about how to provide incentives for this sort of good behavior, in contrast to the ample incentives that researchers have to publish their tentative findings attached to grandiose claims. [sent-33, score-0.455]
19 I wrote this a couple months ago (as regular readers know, most of the posts on this blog are on a delay of two months or so) but now that it appears I realize it is very relevant to our discussion of statistical significance from the other day. [sent-36, score-0.229]
wordName wordTfidf (topN-words)
[('motyl', 0.366), ('nosek', 0.294), ('shade', 0.259), ('embodiment', 0.183), ('extremists', 0.183), ('participants', 0.178), ('replication', 0.156), ('shades', 0.141), ('select', 0.112), ('effect', 0.108), ('task', 0.108), ('incentives', 0.105), ('political', 0.098), ('near', 0.095), ('spuriously', 0.091), ('perceptual', 0.091), ('feasibility', 0.086), ('inflate', 0.086), ('skipping', 0.086), ('ruled', 0.083), ('ample', 0.083), ('signature', 0.083), ('tentative', 0.083), ('spies', 0.083), ('left', 0.081), ('months', 0.081), ('moderates', 0.08), ('publish', 0.079), ('highlighted', 0.077), ('results', 0.076), ('declared', 0.075), ('entered', 0.075), ('part', 0.074), ('reproducibility', 0.074), ('aligned', 0.071), ('hypothesized', 0.071), ('resource', 0.071), ('perceive', 0.071), ('gradient', 0.069), ('browser', 0.068), ('simmons', 0.068), ('literature', 0.068), ('jeffrey', 0.067), ('delay', 0.067), ('pnas', 0.067), ('nelson', 0.067), ('laboratory', 0.066), ('prepared', 0.066), ('laid', 0.065), ('investigate', 0.065)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 1959 andrew gelman stats-2013-07-28-50 shades of gray: A research story
Introduction: This is a killer story (from Brian Nosek, Jeffrey Spies, and Matt Motyl). Part 1: Two of the present authors, Motyl and Nosek, share interests in political ideology. We were inspired by the fast growing literature on embodiment that demonstrates surprising links between body and mind (Markman & Brendl, 2005; Proffitt, 2006) to investigate embodiment of political extremism. Participants from the political left, right and center (N = 1,979) completed a perceptual judgment task in which words were presented in different shades of gray. Participants had to click along a gradient representing grays from near black to near white to select a shade that matched the shade of the word. We calculated accuracy: How close to the actual shade did participants get? The results were stunning. Moderates perceived the shades of gray more accurately than extremists on the left and right (p = .01). Our conclusion: political extremists perceive the world in black-and-white, figuratively and literally
2 0.15098128 1329 andrew gelman stats-2012-05-18-Those mean psychologists, making fun of dodgy research!
Introduction: Two people separately sent me this amusing mock-research paper by Brian A. Nosek (I assume that’s what’s meant by “Arina K. Bones”). The article is pretty funny, but this poster (by Nosek and Samuel Gosling) is even better! Check it out: I remarked that this was almost as good as my zombies paper, and my correspondent pointed me to this page of (I assume) Nosek’s research on aliens. P.S. I clicked through to take the test to see if I’m dead or alive, but I got bored after a few minutes. I gotta say, if Gosling can come up with a 10-item measure of the Big Five, this crew should be able to come up with a reasonably valid alive-or-dead test that doesn’t require dozens and dozens of questions!
Introduction: Brendan Nyhan sends me this article from the research-methods all-star team of Katherine Button, John Ioannidis, Claire Mokrysz, Brian Nosek , Jonathan Flint, Emma Robinson, and Marcus Munafo: A study with low statistical power has a reduced chance of detecting a true effect, but it is less well appreciated that low power also reduces the likelihood that a statistically significant result reflects a true effect. Here, we show that the average statistical power of studies in the neurosciences is very low. The consequences of this include overestimates of effect size and low reproducibility of results. There are also ethical dimensions to this problem, as unreliable research is inefficient and wasteful. Improving reproducibility in neuroscience is a key priority and requires attention to well-established but often ignored methodological principles. I agree completely. In my terminology, with small sample size, the classical approach of looking for statistical significance leads
4 0.1172635 1878 andrew gelman stats-2013-05-31-How to fix the tabloids? Toward replicable social science research
Introduction: This seems to be the topic of the week. Yesterday I posted on the sister blog some further thoughts on those “Psychological Science” papers on menstrual cycles, biceps size, and political attitudes, tied to a horrible press release from the journal Psychological Science hyping the biceps and politics study. Then I was pointed to these suggestions from Richard Lucas and M. Brent Donnellan have on improving the replicability and reproducibility of research published in the Journal of Research in Personality: It goes without saying that editors of scientific journals strive to publish research that is not only theoretically interesting but also methodologically rigorous. The goal is to select papers that advance the field. Accordingly, editors want to publish findings that can be reproduced and replicated by other scientists. Unfortunately, there has been a recent “crisis in confidence” among psychologists about the quality of psychological research (Pashler & Wagenmakers, 2012)
Introduction: Last week I published in Slate a critique of a paper that appeared in the journal Psychological Science. That paper, by Alec Beall and Jessica Tracy, found that women who were at peak fertility were three times more likely to wear red or pink shirts, compared to women at other points in their menstrual cycles. The study was based an 100 participants on the internet and 24 college students. In my critique, I argued that we had no reason to believe the results generalized to the larger population, because (1) the samples were not representative, (2) the measurements were noisy, (3) the researchers did not use the correct dates of peak fertility, and (4) there were many different comparisons that could have been reported in the data, so there was nothing special about a particular comparison being statistically significant. I likened their paper to other work which I considered flawed for multiple comparisons (too many researcher degrees of freedom), including a claimed relation bet
6 0.10751383 2137 andrew gelman stats-2013-12-17-Replication backlash
7 0.10155889 1171 andrew gelman stats-2012-02-16-“False-positive psychology”
8 0.094625048 1093 andrew gelman stats-2011-12-30-Strings Attached: Untangling the Ethics of Incentives
9 0.09378159 1844 andrew gelman stats-2013-05-06-Against optimism about social science
11 0.083234012 2244 andrew gelman stats-2014-03-11-What if I were to stop publishing in journals?
12 0.081099391 1191 andrew gelman stats-2012-03-01-Hoe noem je?
14 0.077427797 797 andrew gelman stats-2011-07-11-How do we evaluate a new and wacky claim?
15 0.076337419 1883 andrew gelman stats-2013-06-04-Interrogating p-values
16 0.076313846 1139 andrew gelman stats-2012-01-26-Suggested resolution of the Bem paradox
17 0.074586526 2245 andrew gelman stats-2014-03-12-More on publishing in journals
18 0.074519366 2232 andrew gelman stats-2014-03-03-What is the appropriate time scale for blogging—the day or the week?
19 0.074493468 1273 andrew gelman stats-2012-04-20-Proposals for alternative review systems for scientific work
topicId topicWeight
[(0, 0.154), (1, -0.055), (2, -0.012), (3, -0.106), (4, -0.035), (5, -0.029), (6, -0.019), (7, -0.036), (8, -0.043), (9, -0.006), (10, 0.001), (11, 0.024), (12, 0.006), (13, -0.019), (14, 0.003), (15, -0.009), (16, -0.02), (17, -0.016), (18, 0.004), (19, -0.008), (20, -0.008), (21, 0.01), (22, -0.026), (23, -0.004), (24, -0.022), (25, 0.005), (26, 0.016), (27, 0.001), (28, 0.036), (29, -0.01), (30, -0.029), (31, -0.018), (32, 0.005), (33, 0.031), (34, 0.009), (35, -0.014), (36, -0.048), (37, -0.015), (38, 0.039), (39, -0.019), (40, 0.021), (41, -0.011), (42, 0.014), (43, 0.006), (44, 0.018), (45, -0.018), (46, -0.002), (47, 0.006), (48, 0.033), (49, 0.033)]
simIndex simValue blogId blogTitle
same-blog 1 0.9802705 1959 andrew gelman stats-2013-07-28-50 shades of gray: A research story
Introduction: This is a killer story (from Brian Nosek, Jeffrey Spies, and Matt Motyl). Part 1: Two of the present authors, Motyl and Nosek, share interests in political ideology. We were inspired by the fast growing literature on embodiment that demonstrates surprising links between body and mind (Markman & Brendl, 2005; Proffitt, 2006) to investigate embodiment of political extremism. Participants from the political left, right and center (N = 1,979) completed a perceptual judgment task in which words were presented in different shades of gray. Participants had to click along a gradient representing grays from near black to near white to select a shade that matched the shade of the word. We calculated accuracy: How close to the actual shade did participants get? The results were stunning. Moderates perceived the shades of gray more accurately than extremists on the left and right (p = .01). Our conclusion: political extremists perceive the world in black-and-white, figuratively and literally
2 0.86593837 1163 andrew gelman stats-2012-02-12-Meta-analysis, game theory, and incentives to do replicable research
Introduction: One of the key insights of game theory is to solve problems in reverse time order. You first figure out what you would do in the endgame, then decide a middle-game strategy to get you where you want to be at the end, then you choose an opening that will take you on your desired path. All conditional on what the other players do in their turn. In an article from 1989, “Meta-analysis in medical research: Strong encouragement for higher quality in individual research efforts,” Keith O’Rourke and Allan Detsky apply this principle to the process of publication of scientific research: From the statistical point of view, there really is no escape from performing a de facto meta-analysis. One can either judge the effectiveness of a therapy based solely on the most recent study and ignore all previous studies, a method which is equivalent to giving the most recent study weight 1.Oand all previous studies weight 0, or try to choose the weights on some scientific basis . . . If impo
3 0.86341691 1671 andrew gelman stats-2013-01-13-Preregistration of Studies and Mock Reports
Introduction: The traditional system of scientific and scholarly publishing is breaking down in two different directions. On one hand, we are moving away from relying on a small set of journals as gatekeepers: the number of papers and research projects is increasing, the number of publication outlets is increasing, and important manuscripts are being posted on SSRN, Arxiv, and other nonrefereed sites. At the same time, many researchers are worried about the profusion of published claims that turn out to not replicate or in plain language, to be false. This concern is not new–some prominent discussions include Rosenthal (1979), Ioannidis (2005), and Vul et al. (2009)–but there is a growing sense that the scientific signal is being swamped by noise. I recently had the opportunity to comment in the journal Political Analysis on two papers, one by Humphreys, Sierra, and Windt, and one by Monogan, on the preregistration of studies and mock reports. Here’s the issue of the journal. Given the hi
4 0.83284402 1878 andrew gelman stats-2013-05-31-How to fix the tabloids? Toward replicable social science research
Introduction: This seems to be the topic of the week. Yesterday I posted on the sister blog some further thoughts on those “Psychological Science” papers on menstrual cycles, biceps size, and political attitudes, tied to a horrible press release from the journal Psychological Science hyping the biceps and politics study. Then I was pointed to these suggestions from Richard Lucas and M. Brent Donnellan have on improving the replicability and reproducibility of research published in the Journal of Research in Personality: It goes without saying that editors of scientific journals strive to publish research that is not only theoretically interesting but also methodologically rigorous. The goal is to select papers that advance the field. Accordingly, editors want to publish findings that can be reproduced and replicated by other scientists. Unfortunately, there has been a recent “crisis in confidence” among psychologists about the quality of psychological research (Pashler & Wagenmakers, 2012)
Introduction: Last week I published in Slate a critique of a paper that appeared in the journal Psychological Science. That paper, by Alec Beall and Jessica Tracy, found that women who were at peak fertility were three times more likely to wear red or pink shirts, compared to women at other points in their menstrual cycles. The study was based an 100 participants on the internet and 24 college students. In my critique, I argued that we had no reason to believe the results generalized to the larger population, because (1) the samples were not representative, (2) the measurements were noisy, (3) the researchers did not use the correct dates of peak fertility, and (4) there were many different comparisons that could have been reported in the data, so there was nothing special about a particular comparison being statistically significant. I likened their paper to other work which I considered flawed for multiple comparisons (too many researcher degrees of freedom), including a claimed relation bet
6 0.80217791 2301 andrew gelman stats-2014-04-22-Ticket to Baaaaarf
7 0.79717439 2179 andrew gelman stats-2014-01-20-The AAA Tranche of Subprime Science
9 0.79313856 2241 andrew gelman stats-2014-03-10-Preregistration: what’s in it for you?
10 0.79145193 1054 andrew gelman stats-2011-12-12-More frustrations trying to replicate an analysis published in a reputable journal
11 0.78377396 908 andrew gelman stats-2011-09-14-Type M errors in the lab
12 0.78106219 1774 andrew gelman stats-2013-03-22-Likelihood Ratio ≠ 1 Journal
13 0.77812952 2137 andrew gelman stats-2013-12-17-Replication backlash
14 0.77176654 1171 andrew gelman stats-2012-02-16-“False-positive psychology”
15 0.7709533 2093 andrew gelman stats-2013-11-07-I’m negative on the expression “false positives”
17 0.76830834 2220 andrew gelman stats-2014-02-22-Quickies
18 0.76465601 2227 andrew gelman stats-2014-02-27-“What Can we Learn from the Many Labs Replication Project?”
19 0.76270223 350 andrew gelman stats-2010-10-18-Subtle statistical issues to be debated on TV.
20 0.76265937 2361 andrew gelman stats-2014-06-06-Hurricanes vs. Himmicanes
topicId topicWeight
[(2, 0.011), (15, 0.031), (16, 0.121), (18, 0.023), (21, 0.069), (24, 0.162), (25, 0.016), (49, 0.012), (70, 0.029), (86, 0.025), (89, 0.021), (93, 0.167), (99, 0.175)]
simIndex simValue blogId blogTitle
same-blog 1 0.92443359 1959 andrew gelman stats-2013-07-28-50 shades of gray: A research story
Introduction: This is a killer story (from Brian Nosek, Jeffrey Spies, and Matt Motyl). Part 1: Two of the present authors, Motyl and Nosek, share interests in political ideology. We were inspired by the fast growing literature on embodiment that demonstrates surprising links between body and mind (Markman & Brendl, 2005; Proffitt, 2006) to investigate embodiment of political extremism. Participants from the political left, right and center (N = 1,979) completed a perceptual judgment task in which words were presented in different shades of gray. Participants had to click along a gradient representing grays from near black to near white to select a shade that matched the shade of the word. We calculated accuracy: How close to the actual shade did participants get? The results were stunning. Moderates perceived the shades of gray more accurately than extremists on the left and right (p = .01). Our conclusion: political extremists perceive the world in black-and-white, figuratively and literally
2 0.91559923 1432 andrew gelman stats-2012-07-27-“Get off my lawn”-blogging
Introduction: Jay Livingston critiques the recent pronouncements of sociologist and cigarette shill Peter Berger, who recently has moved into cultural criticism of New York’s mayor for living with “a woman to whom he is not married” (this is apparently a European sort of thing, I guess they don’t have unmarried partners in the parts of the U.S. where Berger hangs out). But what impresses me is that Berger is doing regular blogging at the age of 84 , writing a long essay each week. That’s really amazing to me. Some of the blogging is a bit suspect, for example the bit where he claims that he personally could convert gays to heterosexual orientation (“A few stubborn individuals may resist the Berger conversion program. The majority will succumb”)—but, really, you gotta admire that he’s doing this. I hope I’m that active when (if) I reach my mid-80s. (As a nonsmoker, I should have a pretty good chance of reaching that point.) P.S. More rant at the sister blog. P.P.S. In comments,
3 0.90220273 1569 andrew gelman stats-2012-11-08-30-30-40 Nation
Introduction: Barack Obama’s win has a potentially huge effect on policy. The current budget negotiations will affect the level and direction of government spending and on the mix of taxes paid by different groups of Americans. We can guess that a President Romney would have fought hard against upper-income tax increases. Other areas of long-term impact include the government’s stance on global warming, foreign policy, and the likelihood that Obama will nominate new Supreme Court justices who will uphold the right to abortion announced in Roe v. Wade. When it comes to public opinion, the story is different. The Democrats may well benefit in 2014 and 2016 from the anticipated slow but steady recovery of the economy over the next few years—but, as of November 6, 2012, the parties are essentially tied, with Barack Obama receiving 51% of the two-party vote, compared to Mitt Romney’s 49%, a split comparable to Al Gore’s narrow victory in 2000, Richard Nixon’s in 1968, and John Kennedy’s in 1960.
4 0.88713241 1116 andrew gelman stats-2012-01-13-Infographic on the economy
Introduction: Gabriel Bergin writes: Just thought I’d share an infographic you might enjoy. I [Bergin] quite like what they did with the colored ranges of previous curves in the two middle graphs: I like it. Would it be possible to put the two long time series on the same scale? As it is, one starts in 1948 and the other starts in 1980. The only thing about the display that I really don’t like are those balls on the top indicating the duration of recessions. It looks weird to me to display a time duration in the form of the area of a ball.
5 0.88571441 1397 andrew gelman stats-2012-06-27-Stand Your Ground laws and homicides
Introduction: Jeff points me to a paper by Chandler McClellan and Erdal Tekin which begins as follows: The controversies surrounding Stand Your Ground laws have recently captured the nation’s attention. Since 2005, eighteen states have passed laws extending the right to self-defense with no duty to retreat to any place a person has a legal right to be, and several additional states are debating the adoption of similar legislation. Despite the implications that these laws may have for public safety, there has been little empirical investigation of their impact on crime and victimization. In this paper, we use monthly data from the U.S. Vital Statistics to examine how Stand Your Ground laws affect homicides. We identify the impact of these laws by exploiting variation in the effective date of these laws across states. Our results indicate that Stand Your Ground laws are associated with a significant increase in the number of homicides among whites, especially white males. According to our estimat
6 0.88434553 683 andrew gelman stats-2011-04-28-Asymmetry in Political Bias
7 0.87220705 1281 andrew gelman stats-2012-04-25-Dyson’s baffling love of crackpots
8 0.87105143 1503 andrew gelman stats-2012-09-19-“Poor Smokers in New York State Spend 25% of Income on Cigarettes, Study Finds”
9 0.86973029 1123 andrew gelman stats-2012-01-17-Big corporations are more popular than you might realize
10 0.86500728 1210 andrew gelman stats-2012-03-12-Plagiarists are in the habit of lying
11 0.85890543 1711 andrew gelman stats-2013-02-07-How Open Should Academic Papers Be?
13 0.82578778 1693 andrew gelman stats-2013-01-25-Subsidized driving
14 0.82034779 586 andrew gelman stats-2011-02-23-A statistical version of Arrow’s paradox
17 0.80990666 1607 andrew gelman stats-2012-12-05-The p-value is not . . .
18 0.80491918 1080 andrew gelman stats-2011-12-24-Latest in blog advertising
19 0.80418181 1293 andrew gelman stats-2012-05-01-Huff the Magic Dragon
20 0.80384624 898 andrew gelman stats-2011-09-10-Fourteen magic words: an update