hunch_net hunch_net-2010 hunch_net-2010-411 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Nikos pointed out this new york times article about poor clinical design killing people . For those of us who study learning from exploration information this is a reminder that low regret algorithms are particularly important, as regret in clinical trials is measured by patient deaths. Two obvious improvements on the experimental design are: With reasonable record keeping of existing outcomes for the standard treatments, there is no need to explicitly assign people to a control group with the standard treatment, as that approach is effectively explored with great certainty. Asserting otherwise would imply that the nature of effective treatments for cancer has changed between now and a year ago, which denies the value of any clinical trial. An optimal experimental design will smoothly phase between exploration and exploitation as evidence for a new treatment shows that it can be effective. This is old tech, for example in the EXP3.P algorithm (page 12 aka 59) although
sentIndex sentText sentNum sentScore
1 Nikos pointed out this new york times article about poor clinical design killing people . [sent-1, score-0.849]
2 For those of us who study learning from exploration information this is a reminder that low regret algorithms are particularly important, as regret in clinical trials is measured by patient deaths. [sent-2, score-0.949]
3 Asserting otherwise would imply that the nature of effective treatments for cancer has changed between now and a year ago, which denies the value of any clinical trial. [sent-4, score-1.02]
4 An optimal experimental design will smoothly phase between exploration and exploitation as evidence for a new treatment shows that it can be effective. [sent-5, score-1.026]
5 P algorithm (page 12 aka 59) although I prefer the generalized and somewhat clearer analysis of EXP4. [sent-7, score-0.143]
6 Done the right way, the clinical trial for a successful treatment would start with some initial small pool (equivalent to “phase 1″ in the article) and then simply expanded the pool of participants over time as it proved superior to the existing treatment, until the pool is everyone. [sent-9, score-1.845]
7 And as a bonus, you can even compete with policies on treatments rather than raw treatments (i. [sent-10, score-1.027]
8 P was first published, and the progress in clinical trial design seems glacial to us outsiders. [sent-15, score-0.819]
9 Partly, I think this is a communication and education failure, but partly, it’s also a failure of imagination within our own field. [sent-16, score-0.3]
10 When we design algorithms, we often don’t think about all the applications, where a little massaging of the design in obvious-to-us ways so as to suit these applications would go a long ways. [sent-17, score-0.648]
11 Getting this right here has a substantial moral aspect, potentially saving millions of lives over time through more precise and fast deployments of new treatments. [sent-18, score-0.327]
wordName wordTfidf (topN-words)
[('clinical', 0.448), ('treatments', 0.415), ('treatment', 0.299), ('design', 0.214), ('pool', 0.195), ('trial', 0.157), ('phase', 0.143), ('article', 0.118), ('partly', 0.104), ('exploration', 0.104), ('experimental', 0.098), ('failure', 0.092), ('asserting', 0.09), ('moral', 0.09), ('expanded', 0.09), ('exploitation', 0.09), ('cancer', 0.09), ('saving', 0.09), ('regret', 0.088), ('getting', 0.086), ('bonus', 0.083), ('imagination', 0.083), ('explored', 0.083), ('patient', 0.083), ('tech', 0.083), ('massaging', 0.083), ('personalized', 0.083), ('aka', 0.078), ('outcomes', 0.078), ('millions', 0.078), ('reminder', 0.078), ('smoothly', 0.078), ('medicine', 0.078), ('existing', 0.074), ('raw', 0.072), ('nikos', 0.072), ('standard', 0.07), ('applications', 0.07), ('lives', 0.069), ('assign', 0.069), ('pointed', 0.069), ('would', 0.067), ('generalized', 0.065), ('education', 0.063), ('successful', 0.063), ('policies', 0.063), ('communication', 0.062), ('compete', 0.062), ('proved', 0.062), ('measured', 0.06)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 411 hunch net-2010-09-21-Regretting the dead
Introduction: Nikos pointed out this new york times article about poor clinical design killing people . For those of us who study learning from exploration information this is a reminder that low regret algorithms are particularly important, as regret in clinical trials is measured by patient deaths. Two obvious improvements on the experimental design are: With reasonable record keeping of existing outcomes for the standard treatments, there is no need to explicitly assign people to a control group with the standard treatment, as that approach is effectively explored with great certainty. Asserting otherwise would imply that the nature of effective treatments for cancer has changed between now and a year ago, which denies the value of any clinical trial. An optimal experimental design will smoothly phase between exploration and exploitation as evidence for a new treatment shows that it can be effective. This is old tech, for example in the EXP3.P algorithm (page 12 aka 59) although
2 0.2090722 314 hunch net-2008-08-24-Mass Customized Medicine in the Future?
Introduction: This post is about a technology which could develop in the future. Right now, a new drug might be tested by finding patients with some diagnosis and giving or not giving them a drug according to a secret randomization. The outcome is observed, and if the average outcome for those treated is measurably better than the average outcome for those not treated, the drug might become a standard treatment. Generalizing this, a filter F sorts people into two groups: those for treatment A and those not for treatment B based upon observations x . To measure the outcome, you randomize between treatment and nontreatment of group A and measure the relative performance of the treatment. A problem often arises: in many cases the treated group does not do better than the nontreated group. A basic question is: does this mean the treatment is bad? With respect to the filter F it may mean that, but with respect to another filter F’ , the treatment might be very effective. For exampl
3 0.087028295 345 hunch net-2009-03-08-Prediction Science
Introduction: One view of machine learning is that it’s about how to program computers to predict well. This suggests a broader research program centered around the more pervasive goal of simply predicting well. There are many distinct strands of this broader research program which are only partially unified. Here are the ones that I know of: Learning Theory . Learning theory focuses on several topics related to the dynamics and process of prediction. Convergence bounds like the VC bound give an intellectual foundation to many learning algorithms. Online learning algorithms like Weighted Majority provide an alternate purely game theoretic foundation for learning. Boosting algorithms yield algorithms for purifying prediction abiliity. Reduction algorithms provide means for changing esoteric problems into well known ones. Machine Learning . A great deal of experience has accumulated in practical algorithm design from a mixture of paradigms, including bayesian, biological, opt
4 0.085947141 265 hunch net-2007-10-14-NIPS workshp: Learning Problem Design
Introduction: Alina and I are organizing a workshop on Learning Problem Design at NIPS . What is learning problem design? It’s about being clever in creating learning problems from otherwise unlabeled data. Read the webpage above for examples. I want to participate! Email us before Nov. 1 with a description of what you want to talk about.
5 0.084387735 217 hunch net-2006-11-06-Data Linkage Problems
Introduction: Data linkage is a problem which seems to come up in various applied machine learning problems. I have heard it mentioned in various data mining contexts, but it seems relatively less studied for systemic reasons. A very simple version of the data linkage problem is a cross hospital patient record merge. Suppose a patient (John Doe) is admitted to a hospital (General Health), treated, and released. Later, John Doe is admitted to a second hospital (Health General), treated, and released. Given a large number of records of this sort, it becomes very tempting to try and predict the outcomes of treatments. This is reasonably straightforward as a machine learning problem if there is a shared unique identifier for John Doe used by General Health and Health General along with time stamps. We can merge the records and create examples of the form “Given symptoms and treatment, did the patient come back to a hospital within the next year?” These examples could be fed into a learning algo
6 0.083599009 205 hunch net-2006-09-07-Objective and subjective interpretations of probability
7 0.079851985 332 hunch net-2008-12-23-Use of Learning Theory
8 0.076154664 183 hunch net-2006-06-14-Explorations of Exploration
9 0.071548454 48 hunch net-2005-03-29-Academic Mechanism Design
10 0.071142338 115 hunch net-2005-09-26-Prediction Bounds as the Mathematics of Science
11 0.071141466 235 hunch net-2007-03-03-All Models of Learning have Flaws
12 0.069324598 435 hunch net-2011-05-16-Research Directions for Machine Learning and Algorithms
13 0.068171233 388 hunch net-2010-01-24-Specializations of the Master Problem
14 0.067974344 125 hunch net-2005-10-20-Machine Learning in the News
15 0.067109525 444 hunch net-2011-09-07-KDD and MUCMD 2011
16 0.066984087 375 hunch net-2009-10-26-NIPS workshops
17 0.066311412 351 hunch net-2009-05-02-Wielding a New Abstraction
18 0.061726641 106 hunch net-2005-09-04-Science in the Government
19 0.060597178 98 hunch net-2005-07-27-Not goal metrics
20 0.059638362 311 hunch net-2008-07-26-Compositional Machine Learning Algorithm Design
topicId topicWeight
[(0, 0.146), (1, 0.011), (2, -0.039), (3, 0.026), (4, 0.01), (5, -0.001), (6, 0.032), (7, 0.037), (8, -0.023), (9, 0.01), (10, -0.004), (11, 0.006), (12, 0.011), (13, 0.017), (14, -0.053), (15, 0.013), (16, -0.019), (17, 0.027), (18, -0.008), (19, 0.011), (20, -0.038), (21, -0.012), (22, -0.029), (23, 0.028), (24, -0.11), (25, -0.013), (26, 0.053), (27, 0.018), (28, 0.089), (29, -0.072), (30, -0.096), (31, 0.002), (32, -0.008), (33, -0.032), (34, -0.037), (35, -0.069), (36, -0.038), (37, 0.002), (38, -0.087), (39, -0.023), (40, 0.048), (41, 0.056), (42, 0.01), (43, -0.009), (44, 0.009), (45, 0.001), (46, -0.008), (47, -0.022), (48, 0.04), (49, -0.016)]
simIndex simValue blogId blogTitle
same-blog 1 0.96044272 411 hunch net-2010-09-21-Regretting the dead
Introduction: Nikos pointed out this new york times article about poor clinical design killing people . For those of us who study learning from exploration information this is a reminder that low regret algorithms are particularly important, as regret in clinical trials is measured by patient deaths. Two obvious improvements on the experimental design are: With reasonable record keeping of existing outcomes for the standard treatments, there is no need to explicitly assign people to a control group with the standard treatment, as that approach is effectively explored with great certainty. Asserting otherwise would imply that the nature of effective treatments for cancer has changed between now and a year ago, which denies the value of any clinical trial. An optimal experimental design will smoothly phase between exploration and exploitation as evidence for a new treatment shows that it can be effective. This is old tech, for example in the EXP3.P algorithm (page 12 aka 59) although
2 0.68442184 314 hunch net-2008-08-24-Mass Customized Medicine in the Future?
Introduction: This post is about a technology which could develop in the future. Right now, a new drug might be tested by finding patients with some diagnosis and giving or not giving them a drug according to a secret randomization. The outcome is observed, and if the average outcome for those treated is measurably better than the average outcome for those not treated, the drug might become a standard treatment. Generalizing this, a filter F sorts people into two groups: those for treatment A and those not for treatment B based upon observations x . To measure the outcome, you randomize between treatment and nontreatment of group A and measure the relative performance of the treatment. A problem often arises: in many cases the treated group does not do better than the nontreated group. A basic question is: does this mean the treatment is bad? With respect to the filter F it may mean that, but with respect to another filter F’ , the treatment might be very effective. For exampl
3 0.66411752 205 hunch net-2006-09-07-Objective and subjective interpretations of probability
Introduction: An amusing tidbit (reproduced without permission) from Herman Chernoff’s delightful monograph, “Sequential analysis and optimal design”: The use of randomization raises a philosophical question which is articulated by the following probably apocryphal anecdote. The metallurgist told his friend the statistician how he planned to test the effect of heat on the strength of a metal bar by sawing the bar into six pieces. The first two would go into the hot oven, the next two into the medium oven, and the last two into the cool oven. The statistician, horrified, explained how he should randomize to avoid the effect of a possible gradient of strength in the metal bar. The method of randomization was applied, and it turned out that the randomized experiment called for putting the first two pieces into the hot oven, the next two into the medium oven, and the last two into the cool oven. “Obviously, we can’t do that,” said the metallurgist. “On the contrary, you have to do that,” said the st
4 0.60476702 106 hunch net-2005-09-04-Science in the Government
Introduction: I found the article on “ Political Science ” at the New York Times interesting. Essentially the article is about allegations that the US government has been systematically distorting scientific views. With a petition by some 7000+ scientists alleging such behavior this is clearly a significant concern. One thing not mentioned explicitly in this discussion is that there are fundamental cultural differences between academic research and the rest of the world. In academic research, careful, clear thought is valued. This value is achieved by both formal and informal mechanisms. One example of a formal mechanism is peer review. In contrast, in the land of politics, the basic value is agreement. It is only with some amount of agreement that a new law can be passed or other actions can be taken. Since Science (with a capitol ‘S’) has accomplished many things, it can be a significant tool in persuading people. This makes it compelling for a politician to use science as a mec
5 0.59204477 241 hunch net-2007-04-28-The Coming Patent Apocalypse
Introduction: Many people in computer science believe that patents are problematic. The truth is even worse—the patent system in the US is fundamentally broken in ways that will require much more significant reform than is being considered now . The myth of the patent is the following: Patents are a mechanism for inventors to be compensated according to the value of their inventions while making the invention available to all. This myth sounds pretty desirable, but the reality is a strange distortion slowly leading towards collapse. There are many problems associated with patents, but I would like to focus on just two of them: Patent Trolls The way that patents have generally worked over the last several decades is that they were a tool of large companies. Large companies would amass a large number of patents and then cross-license each other’s patents—in effect saying “we agree to owe each other nothing”. Smaller companies would sometimes lose in this game, essentially because they
6 0.5658316 282 hunch net-2008-01-06-Research Political Issues
7 0.54604912 345 hunch net-2009-03-08-Prediction Science
8 0.54434794 260 hunch net-2007-08-25-The Privacy Problem
9 0.54171652 334 hunch net-2009-01-07-Interesting Papers at SODA 2009
10 0.52645755 265 hunch net-2007-10-14-NIPS workshp: Learning Problem Design
11 0.51741844 126 hunch net-2005-10-26-Fallback Analysis is a Secret to Useful Algorithms
12 0.5122233 120 hunch net-2005-10-10-Predictive Search is Coming
13 0.51161945 348 hunch net-2009-04-02-Asymmophobia
14 0.48928872 125 hunch net-2005-10-20-Machine Learning in the News
15 0.48687539 217 hunch net-2006-11-06-Data Linkage Problems
16 0.48408568 167 hunch net-2006-03-27-Gradients everywhere
17 0.48194593 397 hunch net-2010-05-02-What’s the difference between gambling and rewarding good prediction?
18 0.47960037 370 hunch net-2009-09-18-Necessary and Sufficient Research
19 0.47798514 161 hunch net-2006-03-05-“Structural” Learning
20 0.47416395 366 hunch net-2009-08-03-Carbon in Computer Science Research
topicId topicWeight
[(0, 0.022), (3, 0.021), (10, 0.021), (27, 0.182), (30, 0.011), (38, 0.034), (44, 0.011), (53, 0.012), (55, 0.09), (68, 0.036), (74, 0.018), (84, 0.284), (94, 0.099), (95, 0.052)]
simIndex simValue blogId blogTitle
1 0.94170845 383 hunch net-2009-12-09-Inherent Uncertainty
Introduction: I’d like to point out Inherent Uncertainty , which I’ve added to the ML blog post scanner on the right. My understanding from Jake is that the intention is to have a multiauthor blog which is more specialized towards learning theory/game theory than this one. Nevertheless, several of the posts seem to be of wider interest.
2 0.93094271 121 hunch net-2005-10-12-The unrealized potential of the research lab
Introduction: I attended the IBM research 60th anniversary . IBM research is, by any reasonable account, the industrial research lab which has managed to bring the most value to it’s parent company over the long term. This can be seen by simply counting the survivors: IBM research is the only older research lab which has not gone through a period of massive firing. (Note that there are also new research labs .) Despite this impressive record, IBM research has failed, by far, to achieve it’s potential. Examples which came up in this meeting include: It took about a decade to produce DRAM after it was invented in the lab. (In fact, Intel produced it first.) Relational databases and SQL were invented and then languished. It was only under external competition that IBM released it’s own relational database. Why didn’t IBM grow an Oracle division ? An early lead in IP networking hardware did not result in IBM growing a Cisco division . Why not? And remember … IBM research is a s
3 0.91744435 467 hunch net-2012-06-15-Normal Deviate and the UCSC Machine Learning Summer School
Introduction: Larry Wasserman has started the Normal Deviate blog which I added to the blogroll on the right. Manfred Warmuth points out the UCSC machine learning summer school running July 9-20 which may be of particular interest to those in silicon valley.
same-blog 4 0.91512597 411 hunch net-2010-09-21-Regretting the dead
Introduction: Nikos pointed out this new york times article about poor clinical design killing people . For those of us who study learning from exploration information this is a reminder that low regret algorithms are particularly important, as regret in clinical trials is measured by patient deaths. Two obvious improvements on the experimental design are: With reasonable record keeping of existing outcomes for the standard treatments, there is no need to explicitly assign people to a control group with the standard treatment, as that approach is effectively explored with great certainty. Asserting otherwise would imply that the nature of effective treatments for cancer has changed between now and a year ago, which denies the value of any clinical trial. An optimal experimental design will smoothly phase between exploration and exploitation as evidence for a new treatment shows that it can be effective. This is old tech, for example in the EXP3.P algorithm (page 12 aka 59) although
5 0.90105498 200 hunch net-2006-08-03-AOL’s data drop
Introduction: AOL has released several large search engine related datasets. This looks like a pretty impressive data release, and it is a big opportunity for people everywhere to worry about search engine related learning problems, if they want.
6 0.83386159 142 hunch net-2005-12-22-Yes , I am applying
7 0.79189724 219 hunch net-2006-11-22-Explicit Randomization in Learning algorithms
8 0.71854937 337 hunch net-2009-01-21-Nearly all natural problems require nonlinearity
9 0.66386998 464 hunch net-2012-05-03-Microsoft Research, New York City
10 0.6577242 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
11 0.63389236 36 hunch net-2005-03-05-Funding Research
12 0.62151575 314 hunch net-2008-08-24-Mass Customized Medicine in the Future?
13 0.62077439 281 hunch net-2007-12-21-Vowpal Wabbit Code Release
14 0.61743909 217 hunch net-2006-11-06-Data Linkage Problems
15 0.61366743 345 hunch net-2009-03-08-Prediction Science
16 0.61133862 343 hunch net-2009-02-18-Decision by Vetocracy
17 0.60721999 51 hunch net-2005-04-01-The Producer-Consumer Model of Research
18 0.60698676 360 hunch net-2009-06-15-In Active Learning, the question changes
19 0.60642523 110 hunch net-2005-09-10-“Failure” is an option
20 0.60540533 95 hunch net-2005-07-14-What Learning Theory might do