hunch_net hunch_net-2005 hunch_net-2005-112 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Virtually every discipline of significant human endeavor has a way explaining itself as fundamental and important. In all the cases I know of, they are both right (they are vital) and wrong (they are not solely vital). Politics. This is the one that everyone is familiar with at the moment. “What could be more important than the process of making decisions?” Science and Technology. This is the one that we-the-academics are familiar with. “The loss of modern science and technology would be catastrophic.” Military. “Without the military, a nation will be invaded and destroyed.” (insert your favorite here) Within science and technology, the same thing happens again. Mathematics. “What could be more important than a precise language for establishing truths?” Physics. “Nothing is more fundamental than the laws which govern the universe. Understanding them is the key to understanding everything else.” Biology. “Without life, we wouldn’t be here, so clearly the s
sentIndex sentText sentNum sentScore
1 Virtually every discipline of significant human endeavor has a way explaining itself as fundamental and important. [sent-1, score-0.469]
2 This is the one that everyone is familiar with at the moment. [sent-4, score-0.128]
3 This is the one that we-the-academics are familiar with. [sent-7, score-0.128]
4 “The loss of modern science and technology would be catastrophic. [sent-8, score-0.283]
5 ” (insert your favorite here) Within science and technology, the same thing happens again. [sent-11, score-0.151]
6 “What could be more important than a precise language for establishing truths? [sent-13, score-0.119]
7 “Nothing is more fundamental than the laws which govern the universe. [sent-15, score-0.256]
8 Understanding them is the key to understanding everything else. [sent-16, score-0.211]
9 “Without life, we wouldn’t be here, so clearly the study of life is fundamental. [sent-18, score-0.157]
10 Controlling computation is fundamental to controlling the world. [sent-21, score-0.342]
11 In particular, for any agent (human or machine), there are things which are sensed and the goal is make good predictions about which actions to take. [sent-24, score-0.362]
12 The ability to predict what will happen in the future is a huge edge in games. [sent-27, score-0.221]
13 You, as a driver, attempt to predict how the other cars around you can mess up, and take that into account in your own driving style. [sent-30, score-0.62]
14 Predicting well can make you very wealthy by playing the stock market. [sent-31, score-0.541]
15 Some companies have been formed around the idea of automated stock picking, with partial success. [sent-32, score-0.468]
16 More generally, the idea of prediction as the essential ingredient is very common when gambling with stocks. [sent-33, score-0.324]
17 Information markets generalize the notion of stock picking to make predictions about arbitrary facts. [sent-34, score-0.793]
18 Prediction problems are prevalent throughout our lives so studying the problems and their solution, which is a core goal of machine learning, is essential. [sent-35, score-0.5]
19 From the predictionist viewpoint, it is not about what you know, what you can prove or infer, who your friends are, or how much wealth you have. [sent-36, score-0.181]
20 Instead, it’s about how well you can predict (and act on predictions of) the future. [sent-37, score-0.397]
wordName wordTfidf (topN-words)
[('stock', 0.259), ('controlling', 0.19), ('vital', 0.19), ('predictions', 0.179), ('playing', 0.172), ('picking', 0.168), ('driving', 0.16), ('life', 0.157), ('fundamental', 0.152), ('science', 0.151), ('technology', 0.132), ('predict', 0.132), ('familiar', 0.128), ('everything', 0.127), ('stone', 0.119), ('defensive', 0.119), ('endeavor', 0.119), ('establishing', 0.119), ('mess', 0.119), ('soccer', 0.119), ('human', 0.114), ('prediction', 0.11), ('studying', 0.11), ('nation', 0.11), ('ingredient', 0.11), ('wealthy', 0.11), ('driver', 0.11), ('around', 0.105), ('gambling', 0.104), ('formed', 0.104), ('cars', 0.104), ('laws', 0.104), ('throughout', 0.104), ('infer', 0.099), ('goal', 0.099), ('generalize', 0.095), ('wealth', 0.095), ('prevalent', 0.095), ('without', 0.093), ('wouldn', 0.092), ('insert', 0.092), ('lives', 0.092), ('markets', 0.092), ('future', 0.089), ('friends', 0.086), ('act', 0.086), ('explaining', 0.084), ('peter', 0.084), ('agent', 0.084), ('understanding', 0.084)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999988 112 hunch net-2005-09-14-The Predictionist Viewpoint
Introduction: Virtually every discipline of significant human endeavor has a way explaining itself as fundamental and important. In all the cases I know of, they are both right (they are vital) and wrong (they are not solely vital). Politics. This is the one that everyone is familiar with at the moment. “What could be more important than the process of making decisions?” Science and Technology. This is the one that we-the-academics are familiar with. “The loss of modern science and technology would be catastrophic.” Military. “Without the military, a nation will be invaded and destroyed.” (insert your favorite here) Within science and technology, the same thing happens again. Mathematics. “What could be more important than a precise language for establishing truths?” Physics. “Nothing is more fundamental than the laws which govern the universe. Understanding them is the key to understanding everything else.” Biology. “Without life, we wouldn’t be here, so clearly the s
2 0.13623413 193 hunch net-2006-07-09-The Stock Prediction Machine Learning Problem
Introduction: …is discussed in this nytimes article . I generally expect such approaches to become more common since computers are getting faster, machine learning is getting better, and data is becoming more plentiful. This is another example where machine learning technology may have a huge economic impact. Some side notes: We-in-research know almost nothing about how these things are done (because it is typically a corporate secret). … but the limited discussion in the article seem naive from a machine learning viewpoint. The learning process used apparently often fails to take into account transaction costs. What little of the approaches is discussed appears modeling based. It seems plausible that more direct prediction methods can yield an edge. One difficulty with stock picking as a research topic is that it is inherently a zero sum game (for every winner, there is a loser). Much of the rest of research is positive sum (basically, everyone wins).
3 0.13363545 282 hunch net-2008-01-06-Research Political Issues
Introduction: I’ve avoided discussing politics here, although not for lack of interest. The problem with discussing politics is that it’s customary for people to say much based upon little information. Nevertheless, politics can have a substantial impact on science (and we might hope for the vice-versa). It’s primary election time in the United States, so the topic is timely, although the issues are not. There are several policy decisions which substantially effect development of science and technology in the US. Education The US has great contrasts in education. The top universities are very good places, yet the grade school education system produces mediocre results. For me, the contrast between a public education and Caltech was bracing. For many others attending Caltech, it clearly was not. Upgrading the k-12 education system in the US is a long-standing chronic problem which I know relatively little about. My own experience is that a basic attitude of “no child unrealized” i
4 0.12665066 109 hunch net-2005-09-08-Online Learning as the Mathematics of Accountability
Introduction: Accountability is a social problem. When someone screws up, do you fire them? Or do you accept the error and let them continue? This is a very difficult problem and we all know of stories where the wrong decision was made. Online learning (as meant here), is a subfield of learning theory which analyzes the online learning model. In the online learning model, there are a set of hypotheses or “experts”. On any instantance x , each expert makes a prediction y . A master algorithm A uses these predictions to form it’s own prediction y A and then learns the correct prediction y * . This process repeats. The goal of online learning is to find a master algorithm A which uses the advice of the experts to make good predictions. In particular, we typically want to guarantee that the master algorithm performs almost as well as the best expert. If L(e) is the loss of expert e and L(A) is the loss of the master algorithm, it is often possible to prove: L(A) les
5 0.12242673 397 hunch net-2010-05-02-What’s the difference between gambling and rewarding good prediction?
Introduction: After a major financial crisis , there is much discussion about how finance has become a casino gambling with other’s money, keeping the winnings, and walking away when the money is lost. When thinking about financial reform, all the many losers in the above scenario are apt to take the view that this activity should be completely, or nearly completely curtailed. But, a more thoughtful view is that sometimes there is a real sense in which there are right and wrong decisions, and we as a society would really prefer that the people most likely to make right decisions are making them. A crucial question then is: “What is the difference between gambling and rewarding good prediction?” We discussed this before the financial crisis . The cheat-sheet sketch is that the online learning against an adversary problem, algorithm, and theorems, provide a good mathematical model for thinking about this question. What I would like to do here is map this onto various types of financial t
6 0.11273359 235 hunch net-2007-03-03-All Models of Learning have Flaws
7 0.11257093 345 hunch net-2009-03-08-Prediction Science
8 0.10637586 3 hunch net-2005-01-24-The Humanloop Spectrum of Machine Learning
9 0.10480262 228 hunch net-2007-01-15-The Machine Learning Department
10 0.097425327 68 hunch net-2005-05-10-Learning Reductions are Reductionist
11 0.093258321 22 hunch net-2005-02-18-What it means to do research.
12 0.08658874 161 hunch net-2006-03-05-“Structural” Learning
13 0.085189864 424 hunch net-2011-02-17-What does Watson mean?
14 0.083430693 64 hunch net-2005-04-28-Science Fiction and Research
15 0.083361305 12 hunch net-2005-02-03-Learning Theory, by assumption
16 0.083025374 287 hunch net-2008-01-28-Sufficient Computation
17 0.082775019 118 hunch net-2005-10-07-On-line learning of regular decision rules
18 0.081402868 5 hunch net-2005-01-26-Watchword: Probability
19 0.079927087 335 hunch net-2009-01-08-Predictive Analytics World
20 0.079475485 96 hunch net-2005-07-21-Six Months
topicId topicWeight
[(0, 0.198), (1, 0.051), (2, -0.042), (3, 0.08), (4, -0.088), (5, -0.05), (6, 0.015), (7, 0.048), (8, 0.081), (9, -0.01), (10, 0.029), (11, -0.039), (12, 0.006), (13, 0.003), (14, -0.059), (15, -0.022), (16, 0.05), (17, -0.057), (18, 0.066), (19, 0.032), (20, 0.011), (21, -0.091), (22, -0.067), (23, 0.085), (24, -0.007), (25, -0.027), (26, 0.088), (27, 0.032), (28, 0.036), (29, 0.04), (30, 0.02), (31, 0.101), (32, -0.012), (33, -0.037), (34, 0.006), (35, 0.059), (36, -0.076), (37, 0.05), (38, 0.039), (39, -0.043), (40, 0.062), (41, 0.062), (42, -0.053), (43, 0.063), (44, -0.008), (45, 0.113), (46, -0.038), (47, -0.027), (48, 0.09), (49, 0.068)]
simIndex simValue blogId blogTitle
same-blog 1 0.97546935 112 hunch net-2005-09-14-The Predictionist Viewpoint
Introduction: Virtually every discipline of significant human endeavor has a way explaining itself as fundamental and important. In all the cases I know of, they are both right (they are vital) and wrong (they are not solely vital). Politics. This is the one that everyone is familiar with at the moment. “What could be more important than the process of making decisions?” Science and Technology. This is the one that we-the-academics are familiar with. “The loss of modern science and technology would be catastrophic.” Military. “Without the military, a nation will be invaded and destroyed.” (insert your favorite here) Within science and technology, the same thing happens again. Mathematics. “What could be more important than a precise language for establishing truths?” Physics. “Nothing is more fundamental than the laws which govern the universe. Understanding them is the key to understanding everything else.” Biology. “Without life, we wouldn’t be here, so clearly the s
2 0.70957035 282 hunch net-2008-01-06-Research Political Issues
Introduction: I’ve avoided discussing politics here, although not for lack of interest. The problem with discussing politics is that it’s customary for people to say much based upon little information. Nevertheless, politics can have a substantial impact on science (and we might hope for the vice-versa). It’s primary election time in the United States, so the topic is timely, although the issues are not. There are several policy decisions which substantially effect development of science and technology in the US. Education The US has great contrasts in education. The top universities are very good places, yet the grade school education system produces mediocre results. For me, the contrast between a public education and Caltech was bracing. For many others attending Caltech, it clearly was not. Upgrading the k-12 education system in the US is a long-standing chronic problem which I know relatively little about. My own experience is that a basic attitude of “no child unrealized” i
3 0.6802671 64 hunch net-2005-04-28-Science Fiction and Research
Introduction: A big part of doing research is imagining how things could be different, and then trying to figure out how to get there. A big part of science fiction is imagining how things could be different, and then working through the implications. Because of the similarity here, reading science fiction can sometimes be helpful in understanding and doing research. (And, hey, it’s fun.) Here’s some list of science fiction books I enjoyed which seem particularly relevant to computer science and (sometimes) learning systems: Vernor Vinge, “True Names”, “A Fire Upon the Deep” Marc Stiegler, “David’s Sling”, “Earthweb” Charles Stross, “Singularity Sky” Greg Egan, “Diaspora” Joe Haldeman, “Forever Peace” (There are surely many others.) Incidentally, the nature of science fiction itself has changed. Decades ago, science fiction projected great increases in the power humans control (example: E.E. Smith Lensman series). That didn’t really happen in the last 50 years. Inste
4 0.6600669 106 hunch net-2005-09-04-Science in the Government
Introduction: I found the article on “ Political Science ” at the New York Times interesting. Essentially the article is about allegations that the US government has been systematically distorting scientific views. With a petition by some 7000+ scientists alleging such behavior this is clearly a significant concern. One thing not mentioned explicitly in this discussion is that there are fundamental cultural differences between academic research and the rest of the world. In academic research, careful, clear thought is valued. This value is achieved by both formal and informal mechanisms. One example of a formal mechanism is peer review. In contrast, in the land of politics, the basic value is agreement. It is only with some amount of agreement that a new law can be passed or other actions can be taken. Since Science (with a capitol ‘S’) has accomplished many things, it can be a significant tool in persuading people. This makes it compelling for a politician to use science as a mec
5 0.63722223 193 hunch net-2006-07-09-The Stock Prediction Machine Learning Problem
Introduction: …is discussed in this nytimes article . I generally expect such approaches to become more common since computers are getting faster, machine learning is getting better, and data is becoming more plentiful. This is another example where machine learning technology may have a huge economic impact. Some side notes: We-in-research know almost nothing about how these things are done (because it is typically a corporate secret). … but the limited discussion in the article seem naive from a machine learning viewpoint. The learning process used apparently often fails to take into account transaction costs. What little of the approaches is discussed appears modeling based. It seems plausible that more direct prediction methods can yield an edge. One difficulty with stock picking as a research topic is that it is inherently a zero sum game (for every winner, there is a loser). Much of the rest of research is positive sum (basically, everyone wins).
6 0.60005188 345 hunch net-2009-03-08-Prediction Science
7 0.5955587 228 hunch net-2007-01-15-The Machine Learning Department
8 0.59143335 397 hunch net-2010-05-02-What’s the difference between gambling and rewarding good prediction?
9 0.56039864 68 hunch net-2005-05-10-Learning Reductions are Reductionist
10 0.54622203 366 hunch net-2009-08-03-Carbon in Computer Science Research
11 0.54425246 120 hunch net-2005-10-10-Predictive Search is Coming
12 0.53425431 314 hunch net-2008-08-24-Mass Customized Medicine in the Future?
13 0.53285378 3 hunch net-2005-01-24-The Humanloop Spectrum of Machine Learning
14 0.52880633 118 hunch net-2005-10-07-On-line learning of regular decision rules
15 0.49868926 109 hunch net-2005-09-08-Online Learning as the Mathematics of Accountability
16 0.49758056 295 hunch net-2008-04-12-It Doesn’t Stop
17 0.49649337 237 hunch net-2007-04-02-Contextual Scaling
18 0.49551016 171 hunch net-2006-04-09-Progress in Machine Translation
19 0.49423024 296 hunch net-2008-04-21-The Science 2.0 article
20 0.48690021 353 hunch net-2009-05-08-Computability in Artificial Intelligence
topicId topicWeight
[(10, 0.013), (27, 0.214), (53, 0.072), (55, 0.071), (93, 0.415), (94, 0.1), (95, 0.029)]
simIndex simValue blogId blogTitle
1 0.8636775 381 hunch net-2009-12-07-Vowpal Wabbit version 4.0, and a NIPS heresy
Introduction: I’m releasing version 4.0 ( tarball ) of Vowpal Wabbit . The biggest change (by far) in this release is experimental support for cluster parallelism, with notable help from Daniel Hsu . I also took advantage of the major version number to introduce some incompatible changes, including switching to murmurhash 2 , and other alterations to cachefiles. You’ll need to delete and regenerate them. In addition, the precise specification for a “tag” (i.e. string that can be used to identify an example) changed—you can’t have a space between the tag and the ‘|’ at the beginning of the feature namespace. And, of course, we made it faster. For the future, I put up my todo list outlining the major future improvements I want to see in the code. I’m planning to discuss the current mechanism and results of the cluster parallel implementation at the large scale machine learning workshop at NIPS later this week. Several people have asked me to do a tutorial/walkthrough of VW, wh
same-blog 2 0.84107774 112 hunch net-2005-09-14-The Predictionist Viewpoint
Introduction: Virtually every discipline of significant human endeavor has a way explaining itself as fundamental and important. In all the cases I know of, they are both right (they are vital) and wrong (they are not solely vital). Politics. This is the one that everyone is familiar with at the moment. “What could be more important than the process of making decisions?” Science and Technology. This is the one that we-the-academics are familiar with. “The loss of modern science and technology would be catastrophic.” Military. “Without the military, a nation will be invaded and destroyed.” (insert your favorite here) Within science and technology, the same thing happens again. Mathematics. “What could be more important than a precise language for establishing truths?” Physics. “Nothing is more fundamental than the laws which govern the universe. Understanding them is the key to understanding everything else.” Biology. “Without life, we wouldn’t be here, so clearly the s
3 0.82705599 363 hunch net-2009-07-09-The Machine Learning Forum
Introduction: Dear Fellow Machine Learners, For the past year or so I have become increasingly frustrated with the peer review system in our field. I constantly get asked to review papers in which I have no interest. At the same time, as an action editor in JMLR, I constantly have to harass people to review papers. When I send papers to conferences and to journals I often get rejected with reviews that, at least in my mind, make no sense. Finally, I have a very hard time keeping up with the best new work, because I don’t know where to look for it… I decided to try an do something to improve the situation. I started a new web site, which I decided to call “The machine learning forum” the URL is http://themachinelearningforum.org The main idea behind this web site is to remove anonymity from the review process. In this site, all opinions are attributed to the actual person that expressed them. I expect that this will improve the quality of the reviews. An obvious other effect is that there wil
4 0.52252847 419 hunch net-2010-12-04-Vowpal Wabbit, version 5.0, and the second heresy
Introduction: I’ve released version 5.0 of the Vowpal Wabbit online learning software. The major number has changed since the last release because I regard all earlier versions as obsolete—there are several new algorithms & features including substantial changes and upgrades to the default learning algorithm. The biggest changes are new algorithms: Nikos and I improved the default algorithm. The basic update rule still uses gradient descent, but the size of the update is carefully controlled so that it’s impossible to overrun the label. In addition, the normalization has changed. Computationally, these changes are virtually free and yield better results, sometimes much better. Less careful updates can be reenabled with –loss_function classic, although results are still not identical to previous due to normalization changes. Nikos also implemented the per-feature learning rates as per these two papers . Often, this works better than the default algorithm. It isn’t the defa
5 0.52069259 286 hunch net-2008-01-25-Turing’s Club for Machine Learning
Introduction: Many people in Machine Learning don’t fully understand the impact of computation, as demonstrated by a lack of big-O analysis of new learning algorithms. This is important—some current active research programs are fundamentally flawed w.r.t. computation, and other research programs are directly motivated by it. When considering a learning algorithm, I think about the following questions: How does the learning algorithm scale with the number of examples m ? Any algorithm using all of the data is at least O(m) , but in many cases this is O(m 2 ) (naive nearest neighbor for self-prediction) or unknown (k-means or many other optimization algorithms). The unknown case is very common, and it can mean (for example) that the algorithm isn’t convergent or simply that the amount of computation isn’t controlled. The above question can also be asked for test cases. In some applications, test-time performance is of great importance. How does the algorithm scale with the number of
6 0.51879692 435 hunch net-2011-05-16-Research Directions for Machine Learning and Algorithms
7 0.51746154 347 hunch net-2009-03-26-Machine Learning is too easy
8 0.51560992 95 hunch net-2005-07-14-What Learning Theory might do
9 0.5135119 351 hunch net-2009-05-02-Wielding a New Abstraction
10 0.51307988 158 hunch net-2006-02-24-A Fundamentalist Organization of Machine Learning
11 0.51227796 337 hunch net-2009-01-21-Nearly all natural problems require nonlinearity
12 0.51206565 258 hunch net-2007-08-12-Exponentiated Gradient
13 0.51171011 371 hunch net-2009-09-21-Netflix finishes (and starts)
14 0.51169705 359 hunch net-2009-06-03-Functionally defined Nonlinear Dynamic Models
15 0.51120365 360 hunch net-2009-06-15-In Active Learning, the question changes
16 0.51118892 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
17 0.51047248 370 hunch net-2009-09-18-Necessary and Sufficient Research
18 0.5092597 237 hunch net-2007-04-02-Contextual Scaling
19 0.50846392 235 hunch net-2007-03-03-All Models of Learning have Flaws
20 0.50805146 22 hunch net-2005-02-18-What it means to do research.