hunch_net hunch_net-2013 hunch_net-2013-483 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: The large scale machine learning class I taught with Yann LeCun has finished. As I expected, it took quite a bit of time . We had about 25 people attending in person on average and 400 regularly watching the recorded lectures which is substantially more sustained interest than I expected for an advanced ML class. We also had some fun with class projects—I’m hopeful that several will eventually turn into papers. I expect there are a number of professors interested in lecturing on this and related topics. Everyone will have their personal taste in subjects of course, but hopefully there will be some convergence to common course materials as well. To help with this, I am making the sources to my presentations available . Feel free to use/improve/embelish/ridicule/etc… in the pursuit of the perfect course.
sentIndex sentText sentNum sentScore
1 The large scale machine learning class I taught with Yann LeCun has finished. [sent-1, score-0.501]
2 We had about 25 people attending in person on average and 400 regularly watching the recorded lectures which is substantially more sustained interest than I expected for an advanced ML class. [sent-3, score-1.639]
3 We also had some fun with class projects—I’m hopeful that several will eventually turn into papers. [sent-4, score-0.79]
4 I expect there are a number of professors interested in lecturing on this and related topics. [sent-5, score-0.477]
5 Everyone will have their personal taste in subjects of course, but hopefully there will be some convergence to common course materials as well. [sent-6, score-1.294]
6 To help with this, I am making the sources to my presentations available . [sent-7, score-0.53]
7 Feel free to use/improve/embelish/ridicule/etc… in the pursuit of the perfect course. [sent-8, score-0.443]
wordName wordTfidf (topN-words)
[('course', 0.303), ('taste', 0.219), ('expected', 0.204), ('watching', 0.203), ('class', 0.194), ('regularly', 0.191), ('pursuit', 0.191), ('hopeful', 0.191), ('professors', 0.182), ('materials', 0.182), ('recorded', 0.175), ('lectures', 0.169), ('projects', 0.159), ('advanced', 0.159), ('took', 0.155), ('taught', 0.155), ('lecun', 0.151), ('perfect', 0.151), ('eventually', 0.142), ('subjects', 0.142), ('presentations', 0.142), ('attending', 0.142), ('convergence', 0.139), ('yann', 0.139), ('fun', 0.136), ('turn', 0.127), ('sources', 0.125), ('hopefully', 0.125), ('personal', 0.12), ('person', 0.114), ('feel', 0.113), ('average', 0.11), ('everyone', 0.104), ('free', 0.101), ('scale', 0.1), ('ml', 0.099), ('available', 0.095), ('substantially', 0.09), ('help', 0.089), ('expect', 0.084), ('related', 0.083), ('interest', 0.082), ('making', 0.079), ('interested', 0.076), ('bit', 0.066), ('common', 0.064), ('quite', 0.06), ('number', 0.052), ('large', 0.052), ('time', 0.044)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 483 hunch net-2013-06-10-The Large Scale Learning class notes
Introduction: The large scale machine learning class I taught with Yann LeCun has finished. As I expected, it took quite a bit of time . We had about 25 people attending in person on average and 400 regularly watching the recorded lectures which is substantially more sustained interest than I expected for an advanced ML class. We also had some fun with class projects—I’m hopeful that several will eventually turn into papers. I expect there are a number of professors interested in lecturing on this and related topics. Everyone will have their personal taste in subjects of course, but hopefully there will be some convergence to common course materials as well. To help with this, I am making the sources to my presentations available . Feel free to use/improve/embelish/ridicule/etc… in the pursuit of the perfect course.
2 0.20349143 478 hunch net-2013-01-07-NYU Large Scale Machine Learning Class
Introduction: Yann LeCun and I are coteaching a class on Large Scale Machine Learning starting late January at NYU . This class will cover many tricks to get machine learning working well on datasets with many features, examples, and classes, along with several elements of deep learning and support systems enabling the previous. This is not a beginning class—you really need to have taken a basic machine learning class previously to follow along. Students will be able to run and experiment with large scale learning algorithms since Yahoo! has donated servers which are being configured into a small scale Hadoop cluster. We are planning to cover the frontier of research in scalable learning algorithms, so good class projects could easily lead to papers. For me, this is a chance to teach on many topics of past research. In general, it seems like researchers should engage in at least occasional teaching of research, both as a proof of teachability and to see their own research through th
3 0.15006958 240 hunch net-2007-04-21-Videolectures.net
Introduction: Davor has been working to setup videolectures.net which is the new site for the many lectures mentioned here . (Tragically, they seem to only be available in windows media format.) I went through my own projects and added a few links to the videos. The day when every result is a set of {paper, slides, video} isn’t quite here yet, but it’s within sight. (For many papers, of course, code is a 4th component.)
4 0.13356975 469 hunch net-2012-07-09-Videolectures
Introduction: Yaser points out some nicely videotaped machine learning lectures at Caltech . Yaser taught me machine learning, and I always found the lectures clear and interesting, so I expect many people can benefit from watching. Relative to Andrew Ng ‘s ML class there are somewhat different areas of emphasis but the topic is the same, so picking and choosing the union may be helpful.
5 0.1110186 16 hunch net-2005-02-09-Intuitions from applied learning
Introduction: Since learning is far from an exact science, it’s good to pay attention to basic intuitions of applied learning. Here are a few I’ve collected. Integration In Bayesian learning, the posterior is computed by an integral, and the optimal thing to do is to predict according to this integral. This phenomena seems to be far more general. Bagging, Boosting, SVMs, and Neural Networks all take advantage of this idea to some extent. The phenomena is more general: you can average over many different classification predictors to improve performance. Sources: Zoubin , Caruana Differentiation Different pieces of an average should differentiate to achieve good performance by different methods. This is know as the ‘symmetry breaking’ problem for neural networks, and it’s why weights are initialized randomly. Boosting explicitly attempts to achieve good differentiation by creating new, different, learning problems. Sources: Yann LeCun , Phil Long Deep Representation Ha
6 0.092183441 428 hunch net-2011-03-27-Vowpal Wabbit, v5.1
7 0.084171958 402 hunch net-2010-07-02-MetaOptimize
8 0.082787216 264 hunch net-2007-09-30-NIPS workshops are out.
9 0.080136053 378 hunch net-2009-11-15-The Other Online Learning
10 0.068824045 382 hunch net-2009-12-09-Future Publication Models @ NIPS
11 0.067383319 464 hunch net-2012-05-03-Microsoft Research, New York City
12 0.066858031 445 hunch net-2011-09-28-Somebody’s Eating Your Lunch
13 0.066739544 479 hunch net-2013-01-31-Remote large scale learning class participation
14 0.064554185 493 hunch net-2014-02-16-Metacademy: a package manager for knowledge
15 0.063678481 381 hunch net-2009-12-07-Vowpal Wabbit version 4.0, and a NIPS heresy
16 0.062686756 142 hunch net-2005-12-22-Yes , I am applying
17 0.062172115 468 hunch net-2012-06-29-ICML survey and comments
18 0.061219491 286 hunch net-2008-01-25-Turing’s Club for Machine Learning
19 0.060988657 300 hunch net-2008-04-30-Concerns about the Large Scale Learning Challenge
20 0.060631514 410 hunch net-2010-09-17-New York Area Machine Learning Events
topicId topicWeight
[(0, 0.125), (1, -0.03), (2, -0.077), (3, 0.022), (4, -0.01), (5, 0.054), (6, -0.041), (7, -0.03), (8, -0.045), (9, -0.016), (10, -0.029), (11, -0.016), (12, 0.048), (13, 0.008), (14, 0.025), (15, 0.029), (16, -0.01), (17, -0.0), (18, 0.03), (19, 0.03), (20, 0.138), (21, 0.004), (22, -0.009), (23, -0.066), (24, 0.115), (25, 0.005), (26, -0.044), (27, 0.01), (28, -0.115), (29, 0.068), (30, 0.008), (31, -0.19), (32, -0.036), (33, -0.057), (34, 0.057), (35, -0.146), (36, 0.017), (37, 0.059), (38, -0.017), (39, -0.088), (40, -0.026), (41, -0.052), (42, -0.003), (43, 0.018), (44, 0.081), (45, -0.107), (46, -0.065), (47, -0.241), (48, 0.05), (49, 0.086)]
simIndex simValue blogId blogTitle
same-blog 1 0.98568141 483 hunch net-2013-06-10-The Large Scale Learning class notes
Introduction: The large scale machine learning class I taught with Yann LeCun has finished. As I expected, it took quite a bit of time . We had about 25 people attending in person on average and 400 regularly watching the recorded lectures which is substantially more sustained interest than I expected for an advanced ML class. We also had some fun with class projects—I’m hopeful that several will eventually turn into papers. I expect there are a number of professors interested in lecturing on this and related topics. Everyone will have their personal taste in subjects of course, but hopefully there will be some convergence to common course materials as well. To help with this, I am making the sources to my presentations available . Feel free to use/improve/embelish/ridicule/etc… in the pursuit of the perfect course.
2 0.76238996 479 hunch net-2013-01-31-Remote large scale learning class participation
Introduction: Yann and I have arranged so that people who are interested in our large scale machine learning class and not able to attend in person can follow along via two methods. Videos will be posted with about a 1 day delay on techtalks . This is a side-by-side capture of video+slides from Weyond . We are experimenting with Piazza as a discussion forum. Anyone is welcome to subscribe to Piazza and ask questions there, where I will be monitoring things. update2 : Sign up here . The first lecture is up now, including the revised version of the slides which fixes a few typos and rounds out references.
3 0.75359493 478 hunch net-2013-01-07-NYU Large Scale Machine Learning Class
Introduction: Yann LeCun and I are coteaching a class on Large Scale Machine Learning starting late January at NYU . This class will cover many tricks to get machine learning working well on datasets with many features, examples, and classes, along with several elements of deep learning and support systems enabling the previous. This is not a beginning class—you really need to have taken a basic machine learning class previously to follow along. Students will be able to run and experiment with large scale learning algorithms since Yahoo! has donated servers which are being configured into a small scale Hadoop cluster. We are planning to cover the frontier of research in scalable learning algorithms, so good class projects could easily lead to papers. For me, this is a chance to teach on many topics of past research. In general, it seems like researchers should engage in at least occasional teaching of research, both as a proof of teachability and to see their own research through th
4 0.72619301 469 hunch net-2012-07-09-Videolectures
Introduction: Yaser points out some nicely videotaped machine learning lectures at Caltech . Yaser taught me machine learning, and I always found the lectures clear and interesting, so I expect many people can benefit from watching. Relative to Andrew Ng ‘s ML class there are somewhat different areas of emphasis but the topic is the same, so picking and choosing the union may be helpful.
5 0.66570187 240 hunch net-2007-04-21-Videolectures.net
Introduction: Davor has been working to setup videolectures.net which is the new site for the many lectures mentioned here . (Tragically, they seem to only be available in windows media format.) I went through my own projects and added a few links to the videos. The day when every result is a set of {paper, slides, video} isn’t quite here yet, but it’s within sight. (For many papers, of course, code is a 4th component.)
6 0.4779537 261 hunch net-2007-08-28-Live ML Class
7 0.47726339 493 hunch net-2014-02-16-Metacademy: a package manager for knowledge
8 0.45125616 382 hunch net-2009-12-09-Future Publication Models @ NIPS
9 0.42929581 172 hunch net-2006-04-14-JMLR is a success
10 0.42231065 6 hunch net-2005-01-27-Learning Complete Problems
11 0.38374248 445 hunch net-2011-09-28-Somebody’s Eating Your Lunch
12 0.37925091 428 hunch net-2011-03-27-Vowpal Wabbit, v5.1
13 0.36917698 322 hunch net-2008-10-20-New York’s ML Day
14 0.35279712 37 hunch net-2005-03-08-Fast Physics for Learning
15 0.34711954 250 hunch net-2007-06-23-Machine Learning Jobs are Growing on Trees
16 0.34678158 31 hunch net-2005-02-26-Problem: Reductions and Relative Ranking Metrics
17 0.34621298 487 hunch net-2013-07-24-ICML 2012 videos lost
18 0.34394106 13 hunch net-2005-02-04-JMLG
19 0.335976 349 hunch net-2009-04-21-Interesting Presentations at Snowbird
20 0.33567336 16 hunch net-2005-02-09-Intuitions from applied learning
topicId topicWeight
[(6, 0.11), (11, 0.026), (27, 0.474), (53, 0.132), (55, 0.036), (95, 0.1)]
simIndex simValue blogId blogTitle
same-blog 1 0.99010658 483 hunch net-2013-06-10-The Large Scale Learning class notes
Introduction: The large scale machine learning class I taught with Yann LeCun has finished. As I expected, it took quite a bit of time . We had about 25 people attending in person on average and 400 regularly watching the recorded lectures which is substantially more sustained interest than I expected for an advanced ML class. We also had some fun with class projects—I’m hopeful that several will eventually turn into papers. I expect there are a number of professors interested in lecturing on this and related topics. Everyone will have their personal taste in subjects of course, but hopefully there will be some convergence to common course materials as well. To help with this, I am making the sources to my presentations available . Feel free to use/improve/embelish/ridicule/etc… in the pursuit of the perfect course.
2 0.94981664 9 hunch net-2005-02-01-Watchword: Loss
Introduction: A loss function is some function which, for any example, takes a prediction and the correct prediction, and determines how much loss is incurred. (People sometimes attempt to optimize functions of more than one example such as “area under the ROC curve” or “harmonic mean of precision and recall”.) Typically we try to find predictors that minimize loss. There seems to be a strong dichotomy between two views of what “loss” means in learning. Loss is determined by the problem. Loss is a part of the specification of the learning problem. Examples of problems specified by the loss function include “binary classification”, “multiclass classification”, “importance weighted classification”, “l 2 regression”, etc… This is the decision theory view of what loss means, and the view that I prefer. Loss is determined by the solution. To solve a problem, you optimize some particular loss function not given by the problem. Examples of these loss functions are “hinge loss” (for SV
3 0.93679374 308 hunch net-2008-07-06-To Dual or Not
Introduction: Yoram and Shai ‘s online learning tutorial at ICML brings up a question for me, “Why use the dual ?” The basic setting is learning a weight vector w i so that the function f(x)= sum i w i x i optimizes some convex loss function. The functional view of the dual is that instead of (or in addition to) keeping track of w i over the feature space, you keep track of a vector a j over the examples and define w i = sum j a j x ji . The above view of duality makes operating in the dual appear unnecessary, because in the end a weight vector is always used. The tutorial suggests that thinking about the dual gives a unified algorithmic font for deriving online learning algorithms. I haven’t worked with the dual representation much myself, but I have seen a few examples where it appears helpful. Noise When doing online optimization (i.e. online learning where you are allowed to look at individual examples multiple times), the dual representation may be helpfu
4 0.93593305 172 hunch net-2006-04-14-JMLR is a success
Introduction: In 2001, the “ Journal of Machine Learning Research ” was created in reaction to unadaptive publisher policies at MLJ . Essentially, with the creation of the internet, the bottleneck in publishing research shifted from publishing to research. The declaration of independence accompanying this move expresses the reasons why in greater detail. MLJ has strongly changed its policy in reaction to this. In particular, there is no longer an assignment of copyright to the publisher (*), and MLJ regularly sponsors many student “best paper awards” across several conferences with cash prizes. This is an advantage of MLJ over JMLR: MLJ can afford to sponsor cash prizes for the machine learning community. The remaining disadvantage is that reading papers in MLJ sometimes requires searching for the author’s website where the free version is available. In contrast, JMLR articles are freely available to everyone off the JMLR website. Whether or not this disadvantage cancels the advantage i
5 0.93220258 309 hunch net-2008-07-10-Interesting papers, ICML 2008
Introduction: Here are some papers from ICML 2008 that I found interesting. Risi Kondor and Karsten Borgwardt , The Skew Spectrum of Graphs . This paper is about a new family of functions on graphs which is invariant under node label permutation. They show that these quantities appear to yield good features for learning. Sanjoy Dasgupta and Daniel Hsu . Hierarchical sampling for active learning. This is the first published practical consistent active learning algorithm. The abstract is also pretty impressive. Lihong Li , Michael Littman , and Thomas Walsh Knows What It Knows: A Framework For Self-Aware Learning. This is an attempt to create learning algorithms that know when they err, (other work includes Vovk ). It’s not yet clear to me what the right model for feature-dependent confidence intervals is. Novi Quadrianto , Alex Smola , TIberio Caetano , and Quoc Viet Le Estimating Labels from Label Proportions . This is an example of learning in a speciali
6 0.93102551 400 hunch net-2010-06-13-The Good News on Exploration and Learning
7 0.93072909 352 hunch net-2009-05-06-Machine Learning to AI
8 0.9306109 288 hunch net-2008-02-10-Complexity Illness
9 0.93051857 247 hunch net-2007-06-14-Interesting Papers at COLT 2007
10 0.92933911 67 hunch net-2005-05-06-Don’t mix the solution into the problem
11 0.92919409 245 hunch net-2007-05-12-Loss Function Semantics
12 0.92862457 341 hunch net-2009-02-04-Optimal Proxy Loss for Classification
13 0.92861331 45 hunch net-2005-03-22-Active learning
14 0.92652363 8 hunch net-2005-02-01-NIPS: Online Bayes
15 0.92525804 304 hunch net-2008-06-27-Reviewing Horror Stories
16 0.92438531 12 hunch net-2005-02-03-Learning Theory, by assumption
17 0.92399907 478 hunch net-2013-01-07-NYU Large Scale Machine Learning Class
18 0.92079747 274 hunch net-2007-11-28-Computational Consequences of Classification
19 0.91955978 158 hunch net-2006-02-24-A Fundamentalist Organization of Machine Learning
20 0.91887575 166 hunch net-2006-03-24-NLPers