andrew_gelman_stats andrew_gelman_stats-2013 andrew_gelman_stats-2013-1842 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: David Hogg pointed me to this post by Gary Marcus, reviewing this skeptics’ all-star issue of Perspectives on Psychological Science that features replication culture heroes Jelte Wicherts, Hal Pashler, Arina Bones, E. J. Wagenmakers, Gregory Francis, Hal Pashler, John Ioannidis, and Uri Simonsohn. I agree with pretty much everything Marcus has to say. In addition to Marcus’s suggestions, which might be called cultural or psychological, I also have various statistical ideas that might help move the field forward. Most notably I think we need to go beyond uniform priors and null-hypothesis testing to a more realistic set of models for effects and variation. I’ll discuss more at some other time, but in the meantime I thought I’d share these links. P.S. Marcus updates with a glass-is-half-full take.
sentIndex sentText sentNum sentScore
1 David Hogg pointed me to this post by Gary Marcus, reviewing this skeptics’ all-star issue of Perspectives on Psychological Science that features replication culture heroes Jelte Wicherts, Hal Pashler, Arina Bones, E. [sent-1, score-0.666]
2 I agree with pretty much everything Marcus has to say. [sent-4, score-0.118]
3 In addition to Marcus’s suggestions, which might be called cultural or psychological, I also have various statistical ideas that might help move the field forward. [sent-5, score-0.681]
4 Most notably I think we need to go beyond uniform priors and null-hypothesis testing to a more realistic set of models for effects and variation. [sent-6, score-0.662]
5 I’ll discuss more at some other time, but in the meantime I thought I’d share these links. [sent-7, score-0.253]
6 Marcus updates with a glass-is-half-full take. [sent-10, score-0.131]
wordName wordTfidf (topN-words)
[('marcus', 0.552), ('pashler', 0.316), ('hal', 0.254), ('arina', 0.175), ('psychological', 0.172), ('jelte', 0.158), ('gregory', 0.153), ('bones', 0.153), ('wicherts', 0.148), ('heroes', 0.141), ('skeptics', 0.138), ('updates', 0.131), ('francis', 0.131), ('wagenmakers', 0.131), ('ioannidis', 0.129), ('hogg', 0.125), ('uri', 0.117), ('notably', 0.115), ('perspectives', 0.114), ('realistic', 0.113), ('reviewing', 0.111), ('meantime', 0.107), ('gary', 0.104), ('uniform', 0.104), ('cultural', 0.101), ('replication', 0.1), ('culture', 0.094), ('features', 0.09), ('suggestions', 0.087), ('priors', 0.082), ('addition', 0.081), ('testing', 0.08), ('share', 0.079), ('move', 0.073), ('pointed', 0.072), ('field', 0.069), ('discuss', 0.067), ('everything', 0.066), ('beyond', 0.064), ('david', 0.064), ('called', 0.063), ('might', 0.062), ('john', 0.062), ('issue', 0.058), ('ideas', 0.058), ('help', 0.057), ('various', 0.055), ('effects', 0.054), ('agree', 0.052), ('set', 0.05)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 1842 andrew gelman stats-2013-05-05-Cleaning up science
Introduction: David Hogg pointed me to this post by Gary Marcus, reviewing this skeptics’ all-star issue of Perspectives on Psychological Science that features replication culture heroes Jelte Wicherts, Hal Pashler, Arina Bones, E. J. Wagenmakers, Gregory Francis, Hal Pashler, John Ioannidis, and Uri Simonsohn. I agree with pretty much everything Marcus has to say. In addition to Marcus’s suggestions, which might be called cultural or psychological, I also have various statistical ideas that might help move the field forward. Most notably I think we need to go beyond uniform priors and null-hypothesis testing to a more realistic set of models for effects and variation. I’ll discuss more at some other time, but in the meantime I thought I’d share these links. P.S. Marcus updates with a glass-is-half-full take.
2 0.18561548 1844 andrew gelman stats-2013-05-06-Against optimism about social science
Introduction: Social science research has been getting pretty bad press recently, what with the Excel buccaneers who didn’t know how to handle data with different numbers of observations per country, and the psychologist who published dozens of papers based on fabricated data, and the Evilicious guy who wouldn’t let people review his data tapes, etc etc. And that’s not even considering Dr. Anil Potti. On the other hand, the revelation of all these problems can be taken as evidence that things are getting better. Psychology researcher Gary Marcus writes : There is something positive that has come out of the crisis of replicability—something vitally important for all experimental sciences. For years, it was extremely difficult to publish a direct replication, or a failure to replicate an experiment, in a good journal. . . . Now, happily, the scientific culture has changed. . . . The Reproducibility Project, from the Center for Open Science is now underway . . . And sociologist Fabio Rojas
3 0.14826185 933 andrew gelman stats-2011-09-30-More bad news: The (mis)reporting of statistical results in psychology journals
Introduction: Another entry in the growing literature on systematic flaws in the scientific research literature. This time the bad tidings come from Marjan Bakker and Jelte Wicherts, who write : Around 18% of statistical results in the psychological literature are incorrectly reported. Inconsistencies were more common in low-impact journals than in high-impact journals. Moreover, around 15% of the articles contained at least one statistical conclusion that proved, upon recalculation, to be incorrect; that is, recalculation rendered the previously significant result insignificant, or vice versa. These errors were often in line with researchers’ expectations. Their research also had a qualitative component: To obtain a better understanding of the origins of the errors made in the reporting of statistics, we contacted the authors of the articles with errors in the second study and asked them to send us the raw data. Regrettably, only 24% of the authors shared their data, despite our request
Introduction: Adam Marcus at Retraction Watch reports on a physicist at the University of Toronto who had this unfortunate thing happen to him: This article has been retracted at the request of the Editor-in-Chief and first and corresponding author. The article was largely a duplication of a paper that had already appeared in ACS Nano, 4 (2010) 3374–3380, http://dx.doi.org/10.1021/nn100335g. The first and the corresponding authors (Kramer and Sargent) would like to apologize for this administrative error on their part . . . “Administrative error” . . . I love that! Is that what the robber says when he knocks over a liquor store and gets caught? As Marcus points out, the two papers have different titles and a different order of authors, which makes it less plausible that this was an administrative mistake (as could happen, for example, if a secretary was given a list of journals to submit the paper to, and accidentally submitted it to the second journal on the list without realizing it
5 0.11878934 897 andrew gelman stats-2011-09-09-The difference between significant and not significant…
Introduction: E. J. Wagenmakers writes: You may be interested in a recent article [by Nieuwenhuis, Forstmann, and Wagenmakers] showing how often researchers draw conclusions by comparing p-values. As you and Hal Stern have pointed out, this is potentially misleading because the difference between significant and not significant is not necessarily significant. We were really suprised to see how often researchers in the neurosciences make this mistake. In the paper we speculate a little bit on the cause of the error. From their paper: In theory, a comparison of two experimental effects requires a statistical test on their difference. In practice, this comparison is often based on an incorrect procedure involving two separate tests in which researchers conclude that effects differ when one effect is significant (P < 0.05) but the other is not (P > 0.05). We reviewed 513 behavioral, systems and cognitive neuroscience articles in five top-ranking journals (Science, Nature, Nature Neuroscien
6 0.10883132 1883 andrew gelman stats-2013-06-04-Interrogating p-values
8 0.091673106 991 andrew gelman stats-2011-11-04-Insecure researchers aren’t sharing their data
9 0.090197548 2278 andrew gelman stats-2014-04-01-Association for Psychological Science announces a new journal
10 0.08880803 758 andrew gelman stats-2011-06-11-Hey, good news! Your p-value just passed the 0.05 threshold!
11 0.084697403 424 andrew gelman stats-2010-11-21-Data cleaning tool!
12 0.083493426 1878 andrew gelman stats-2013-05-31-How to fix the tabloids? Toward replicable social science research
13 0.082199015 1833 andrew gelman stats-2013-04-30-“Tragedy of the science-communication commons”
14 0.079792596 268 andrew gelman stats-2010-09-10-Fighting Migraine with Multilevel Modeling
15 0.0784721 450 andrew gelman stats-2010-12-04-The Joy of Stats
16 0.078057542 2343 andrew gelman stats-2014-05-22-Big Data needs Big Model
17 0.077994429 2090 andrew gelman stats-2013-11-05-How much do we trust a new claim that early childhood stimulation raised earnings by 42%?
18 0.071948305 1072 andrew gelman stats-2011-12-19-“The difference between . . .”: It’s not just p=.05 vs. p=.06
19 0.069684699 1329 andrew gelman stats-2012-05-18-Those mean psychologists, making fun of dodgy research!
topicId topicWeight
[(0, 0.09), (1, 0.007), (2, -0.028), (3, -0.033), (4, -0.025), (5, -0.032), (6, -0.017), (7, -0.022), (8, -0.033), (9, 0.023), (10, 0.002), (11, 0.029), (12, 0.026), (13, -0.019), (14, 0.014), (15, -0.022), (16, -0.034), (17, -0.006), (18, -0.009), (19, -0.015), (20, -0.019), (21, -0.027), (22, -0.037), (23, 0.012), (24, -0.036), (25, -0.021), (26, 0.038), (27, 0.011), (28, 0.016), (29, -0.03), (30, -0.014), (31, -0.028), (32, -0.023), (33, -0.013), (34, 0.019), (35, -0.003), (36, -0.049), (37, -0.002), (38, 0.012), (39, 0.022), (40, -0.015), (41, 0.041), (42, 0.001), (43, 0.02), (44, -0.019), (45, 0.004), (46, -0.049), (47, -0.002), (48, 0.003), (49, 0.006)]
simIndex simValue blogId blogTitle
same-blog 1 0.95816267 1842 andrew gelman stats-2013-05-05-Cleaning up science
Introduction: David Hogg pointed me to this post by Gary Marcus, reviewing this skeptics’ all-star issue of Perspectives on Psychological Science that features replication culture heroes Jelte Wicherts, Hal Pashler, Arina Bones, E. J. Wagenmakers, Gregory Francis, Hal Pashler, John Ioannidis, and Uri Simonsohn. I agree with pretty much everything Marcus has to say. In addition to Marcus’s suggestions, which might be called cultural or psychological, I also have various statistical ideas that might help move the field forward. Most notably I think we need to go beyond uniform priors and null-hypothesis testing to a more realistic set of models for effects and variation. I’ll discuss more at some other time, but in the meantime I thought I’d share these links. P.S. Marcus updates with a glass-is-half-full take.
2 0.66408741 1833 andrew gelman stats-2013-04-30-“Tragedy of the science-communication commons”
Introduction: I’ve earlier written that science is science communication —that is, the act of communicating scientific ideas and findings to ourselves and others is itself a central part of science. My point was to push against a conventional separation between the act of science and the act of communication, the idea that science is done by scientists and communication is done by communicators. It’s a rare bit of science that does not include communication as part of it. As a scientist and science communicator myself, I’m particularly sensitive to devaluing of communication. (For example, Bayesian Data Analysis is full of original research that was done in order to communicate; or, to put it another way, we often think we understand a scientific idea, but once we try to communicate it, we recognize gaps in our understanding that motivate further research.) I once saw the following on one of those inspirational-sayings-for-every-day desk calendars: “To have ideas is to gather flowers. To thin
Introduction: I pointed Steven Pinker to my post, How much time (if any) should we spend criticizing research that’s fraudulent, crappy, or just plain pointless? , and he responded: Clearly it *is* important to call out publicized research whose conclusions are likely to be false. The only danger is that it’s so easy and fun to criticize, with all the perks of intellectual and moral superiority for so little cost, that there is a moral hazard to go overboard and become a professional slasher and snarker. (That’s a common phenomenon among literary critics, especially in the UK.) There’s also the risk of altering the incentive structure for innovative research, so that researchers stick to the safest kinds of paradigm-twiddling. I think these two considerations were what my late colleague Dan Wegner had in mind when he made the bumbler-pointer contrast — he himself was certainly a discerning critic of social science research. [Just to clarify: Wegner is the person who talked about bumblers and po
4 0.65858412 897 andrew gelman stats-2011-09-09-The difference between significant and not significant…
Introduction: E. J. Wagenmakers writes: You may be interested in a recent article [by Nieuwenhuis, Forstmann, and Wagenmakers] showing how often researchers draw conclusions by comparing p-values. As you and Hal Stern have pointed out, this is potentially misleading because the difference between significant and not significant is not necessarily significant. We were really suprised to see how often researchers in the neurosciences make this mistake. In the paper we speculate a little bit on the cause of the error. From their paper: In theory, a comparison of two experimental effects requires a statistical test on their difference. In practice, this comparison is often based on an incorrect procedure involving two separate tests in which researchers conclude that effects differ when one effect is significant (P < 0.05) but the other is not (P > 0.05). We reviewed 513 behavioral, systems and cognitive neuroscience articles in five top-ranking journals (Science, Nature, Nature Neuroscien
5 0.65374535 1959 andrew gelman stats-2013-07-28-50 shades of gray: A research story
Introduction: This is a killer story (from Brian Nosek, Jeffrey Spies, and Matt Motyl). Part 1: Two of the present authors, Motyl and Nosek, share interests in political ideology. We were inspired by the fast growing literature on embodiment that demonstrates surprising links between body and mind (Markman & Brendl, 2005; Proffitt, 2006) to investigate embodiment of political extremism. Participants from the political left, right and center (N = 1,979) completed a perceptual judgment task in which words were presented in different shades of gray. Participants had to click along a gradient representing grays from near black to near white to select a shade that matched the shade of the word. We calculated accuracy: How close to the actual shade did participants get? The results were stunning. Moderates perceived the shades of gray more accurately than extremists on the left and right (p = .01). Our conclusion: political extremists perceive the world in black-and-white, figuratively and literally
7 0.63866055 2188 andrew gelman stats-2014-01-27-“Disappointed with your results? Boost your scientific paper”
8 0.63702881 2241 andrew gelman stats-2014-03-10-Preregistration: what’s in it for you?
9 0.6280269 1878 andrew gelman stats-2013-05-31-How to fix the tabloids? Toward replicable social science research
10 0.6270982 1671 andrew gelman stats-2013-01-13-Preregistration of Studies and Mock Reports
11 0.62060744 1880 andrew gelman stats-2013-06-02-Flame bait
12 0.6190412 2093 andrew gelman stats-2013-11-07-I’m negative on the expression “false positives”
13 0.6172117 1998 andrew gelman stats-2013-08-25-A new Bem theory
14 0.61435789 1883 andrew gelman stats-2013-06-04-Interrogating p-values
15 0.61168188 2227 andrew gelman stats-2014-02-27-“What Can we Learn from the Many Labs Replication Project?”
17 0.60430777 1974 andrew gelman stats-2013-08-08-Statistical significance and the dangerous lure of certainty
18 0.60301852 1844 andrew gelman stats-2013-05-06-Against optimism about social science
19 0.60074127 1171 andrew gelman stats-2012-02-16-“False-positive psychology”
20 0.59745848 898 andrew gelman stats-2011-09-10-Fourteen magic words: an update
topicId topicWeight
[(4, 0.017), (6, 0.022), (15, 0.056), (16, 0.053), (21, 0.076), (24, 0.097), (34, 0.109), (53, 0.019), (56, 0.044), (85, 0.021), (86, 0.016), (90, 0.138), (99, 0.213)]
simIndex simValue blogId blogTitle
same-blog 1 0.92482793 1842 andrew gelman stats-2013-05-05-Cleaning up science
Introduction: David Hogg pointed me to this post by Gary Marcus, reviewing this skeptics’ all-star issue of Perspectives on Psychological Science that features replication culture heroes Jelte Wicherts, Hal Pashler, Arina Bones, E. J. Wagenmakers, Gregory Francis, Hal Pashler, John Ioannidis, and Uri Simonsohn. I agree with pretty much everything Marcus has to say. In addition to Marcus’s suggestions, which might be called cultural or psychological, I also have various statistical ideas that might help move the field forward. Most notably I think we need to go beyond uniform priors and null-hypothesis testing to a more realistic set of models for effects and variation. I’ll discuss more at some other time, but in the meantime I thought I’d share these links. P.S. Marcus updates with a glass-is-half-full take.
Introduction: Solomon Hsiang writes : I [Hsiang] have posted about high temperature inducing individuals to exhibit more violent behavior when driving, playing baseball and prowling bars. These cases are neat anecdotes that let us see the “pure aggression” response in lab-like conditions. But they don’t affect most of us too much. But violent crime in the real world affects everyone. Earlier, I posted a paper by Jacob et al. that looked at assault in the USA for about a decade – they found that higher temperatures lead to more assault and that the rise in violent crimes rose more quickly than the analogous rise in non-violent property-crime, an indicator that there is a “pure aggression” component to the rise in violent crime. A new working paper “Crime, Weather, and Climate Change” by recent Harvard grad Matthew Ranson puts together an impressive data set of all types of crime in USA counties for 50 years. The results tell the aggression story using street-level data very clearly [click to
3 0.87407416 1932 andrew gelman stats-2013-07-10-Don’t trust the Turk
Introduction: Dan Kahan gives a bunch of reasons not to trust Mechanical Turk in psychology experiments, in particular when studying “hypotheses about cognition and political conflict over societal risks and other policy-relevant facts.”
4 0.86601889 475 andrew gelman stats-2010-12-19-All politics are local — not
Introduction: Mickey Kaus does a public service by trashing Tip O’Neill’s famous dictum that “all politics are local.” As Kaus point out, all the congressional elections in recent decades have been nationalized. I’d go one step further and say that, sure, all politics are local–if you’re Tip O’Neill and represent a ironclad Democratic seat in Congress. It’s easy to be smug about your political skills if you’re in a safe seat and have enough pull in state politics to avoid your district getting gerrymandered. Then you can sit there and sagely attribute your success to your continuing mastery of local politics rather than to whatever it took to get the seat in the first place.
5 0.86299717 1417 andrew gelman stats-2012-07-15-Some decision analysis problems are pretty easy, no?
Introduction: Cassie Murdoch reports : A 47-year-old woman in Uxbridge, Massachusetts, got behind the wheel of her car after having a bit too much to drink, but instead of wreaking havoc on the road, she ended up lodged in a sand trap at a local golf course. Why? Because her GPS made her do it—obviously! She said the GPS told her to turn left, and she did, right into a cornfield. That didn’t faze her, and she just kept on going until she ended up on the golf course and got stuck in the sand. There were people on the course at the time, but thankfully nobody was injured. Police found a cup full of alcohol in her car and arrested her for driving drunk. Here’s the punchline: This is the fourth time she’s been arrested for a DUI. Assuming this story is accurate, I guess they don’t have one of those “three strikes” laws in Massachusetts? Personally, I’m a lot more afraid of a dangerous driver than of some drug dealer. I’d think a simple cost-benefit calculation would recommend taking away
6 0.86231232 1911 andrew gelman stats-2013-06-23-AI Stats conference on Stan etc.
8 0.85442674 15 andrew gelman stats-2010-05-03-Public Opinion on Health Care Reform
9 0.85383219 762 andrew gelman stats-2011-06-13-How should journals handle replication studies?
10 0.85166043 1411 andrew gelman stats-2012-07-10-Defining ourselves arbitrarily
11 0.84940398 1734 andrew gelman stats-2013-02-23-Life in the C-suite: A graph that is both ugly and bad, and an unrelated story
12 0.84198022 1947 andrew gelman stats-2013-07-20-We are what we are studying
13 0.84044302 292 andrew gelman stats-2010-09-23-Doug Hibbs on the fundamentals in 2010
14 0.84020311 1655 andrew gelman stats-2013-01-05-The statistics software signal
15 0.83777535 1501 andrew gelman stats-2012-09-18-More studies on the economic effects of climate change
16 0.83644819 956 andrew gelman stats-2011-10-13-Hey, you! Don’t take that class!
17 0.83221233 1111 andrew gelman stats-2012-01-10-The blog of the Cultural Cognition Project
19 0.82618439 766 andrew gelman stats-2011-06-14-Last Wegman post (for now)
20 0.82599556 1144 andrew gelman stats-2012-01-29-How many parameters are in a multilevel model?