hunch_net hunch_net-2005 hunch_net-2005-54 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: There was a presentation at snowbird about parallelized support vector machines. In many cases, people parallelize by ignoring serial operations, but that is not what happened here—they parallelize with optimizations. Consequently, this seems to be the fastest SVM in existence. There is a related paper here .
sentIndex sentText sentNum sentScore
1 There was a presentation at snowbird about parallelized support vector machines. [sent-1, score-0.798]
2 In many cases, people parallelize by ignoring serial operations, but that is not what happened here—they parallelize with optimizations. [sent-2, score-1.997]
3 Consequently, this seems to be the fastest SVM in existence. [sent-3, score-0.378]
wordName wordTfidf (topN-words)
[('parallelize', 0.535), ('serial', 0.334), ('fastest', 0.31), ('ignoring', 0.279), ('snowbird', 0.279), ('operations', 0.25), ('svm', 0.231), ('happened', 0.221), ('consequently', 0.198), ('presentation', 0.184), ('vector', 0.181), ('cases', 0.159), ('support', 0.154), ('related', 0.127), ('paper', 0.08), ('seems', 0.068), ('people', 0.052), ('many', 0.041)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 54 hunch net-2005-04-08-Fast SVMs
Introduction: There was a presentation at snowbird about parallelized support vector machines. In many cases, people parallelize by ignoring serial operations, but that is not what happened here—they parallelize with optimizations. Consequently, this seems to be the fastest SVM in existence. There is a related paper here .
2 0.15417261 431 hunch net-2011-04-18-A paper not at Snowbird
Introduction: Unfortunately, a scheduling failure meant I missed all of AIStat and most of the learning workshop , otherwise known as Snowbird, when it’s at Snowbird . At snowbird, the talk on Sum-Product networks by Hoifung Poon stood out to me ( Pedro Domingos is a coauthor.). The basic point was that by appropriately constructing networks based on sums and products, the normalization problem in probabilistic models is eliminated, yielding a highly tractable yet flexible representation+learning algorithm. As an algorithm, this is noticeably cleaner than deep belief networks with a claim to being an order of magnitude faster and working better on an image completion task. Snowbird doesn’t have real papers—just the abstract above. I look forward to seeing the paper. (added: Rodrigo points out the deep learning workshop draft .)
3 0.12437398 229 hunch net-2007-01-26-Parallel Machine Learning Problems
Introduction: Parallel machine learning is a subject rarely addressed at machine learning conferences. Nevertheless, it seems likely to increase in importance because: Data set sizes appear to be growing substantially faster than computation. Essentially, this happens because more and more sensors of various sorts are being hooked up to the internet. Serial speedups of processors seem are relatively stalled. The new trend is to make processors more powerful by making them multicore . Both AMD and Intel are making dual core designs standard, with plans for more parallelism in the future. IBM’s Cell processor has (essentially) 9 cores. Modern graphics chips can have an order of magnitude more separate execution units. The meaning of ‘core’ varies a bit from processor to processor, but the overall trend seems quite clear. So, how do we parallelize machine learning algorithms? The simplest and most common technique is to simply run the same learning algorithm with di
4 0.12386163 442 hunch net-2011-08-20-The Large Scale Learning Survey Tutorial
Introduction: Ron Bekkerman initiated an effort to create an edited book on parallel machine learning that Misha and I have been helping with. The breadth of efforts to parallelize machine learning surprised me: I was only aware of a small fraction initially. This put us in a unique position, with knowledge of a wide array of different efforts, so it is natural to put together a survey tutorial on the subject of parallel learning for KDD , tomorrow. This tutorial is not limited to the book itself however, as several interesting new algorithms have come out since we started inviting chapters. This tutorial should interest anyone trying to use machine learning on significant quantities of data, anyone interested in developing algorithms for such, and of course who has bragging rights to the fastest learning algorithm on planet earth (Also note the Modeling with Hadoop tutorial just before ours which deals with one way of trying to speed up learning algorithms. We have almost no
5 0.12353014 346 hunch net-2009-03-18-Parallel ML primitives
Introduction: Previously, we discussed parallel machine learning a bit. As parallel ML is rather difficult, I’d like to describe my thinking at the moment, and ask for advice from the rest of the world. This is particularly relevant right now, as I’m attending a workshop tomorrow on parallel ML. Parallelizing slow algorithms seems uncompelling. Parallelizing many algorithms also seems uncompelling, because the effort required to parallelize is substantial. This leaves the question: Which one fast algorithm is the best to parallelize? What is a substantially different second? One compellingly fast simple algorithm is online gradient descent on a linear representation. This is the core of Leon’s sgd code and Vowpal Wabbit . Antoine Bordes showed a variant was competitive in the large scale learning challenge . It’s also a decades old primitive which has been reused in many algorithms, and continues to be reused. It also applies to online learning rather than just online optimiz
6 0.09556289 307 hunch net-2008-07-04-More Presentation Preparation
7 0.093406558 53 hunch net-2005-04-06-Structured Regret Minimization
8 0.072721854 136 hunch net-2005-12-07-Is the Google way the way for machine learning?
9 0.071569145 135 hunch net-2005-12-04-Watchword: model
10 0.071430929 80 hunch net-2005-06-10-Workshops are not Conferences
11 0.070925869 201 hunch net-2006-08-07-The Call of the Deep
12 0.061114788 23 hunch net-2005-02-19-Loss Functions for Discriminative Training of Energy-Based Models
13 0.059716411 152 hunch net-2006-01-30-Should the Input Representation be a Vector?
14 0.058179088 37 hunch net-2005-03-08-Fast Physics for Learning
15 0.057987303 347 hunch net-2009-03-26-Machine Learning is too easy
16 0.057342775 441 hunch net-2011-08-15-Vowpal Wabbit 6.0
17 0.052075218 308 hunch net-2008-07-06-To Dual or Not
18 0.051387798 86 hunch net-2005-06-28-The cross validation problem: cash reward
19 0.042291891 352 hunch net-2009-05-06-Machine Learning to AI
20 0.041562367 349 hunch net-2009-04-21-Interesting Presentations at Snowbird
topicId topicWeight
[(0, 0.06), (1, -0.006), (2, -0.013), (3, -0.0), (4, 0.039), (5, 0.044), (6, -0.053), (7, -0.006), (8, -0.001), (9, 0.029), (10, -0.076), (11, -0.036), (12, -0.002), (13, -0.004), (14, 0.03), (15, -0.002), (16, 0.024), (17, 0.078), (18, -0.069), (19, -0.022), (20, -0.022), (21, 0.034), (22, -0.04), (23, 0.067), (24, 0.027), (25, 0.076), (26, 0.023), (27, -0.014), (28, -0.056), (29, 0.062), (30, -0.052), (31, 0.042), (32, -0.064), (33, 0.075), (34, 0.131), (35, -0.089), (36, 0.041), (37, 0.067), (38, 0.018), (39, -0.026), (40, -0.013), (41, -0.11), (42, -0.081), (43, 0.118), (44, -0.051), (45, -0.066), (46, -0.005), (47, 0.064), (48, 0.095), (49, 0.065)]
simIndex simValue blogId blogTitle
same-blog 1 0.97293609 54 hunch net-2005-04-08-Fast SVMs
Introduction: There was a presentation at snowbird about parallelized support vector machines. In many cases, people parallelize by ignoring serial operations, but that is not what happened here—they parallelize with optimizations. Consequently, this seems to be the fastest SVM in existence. There is a related paper here .
2 0.57027507 229 hunch net-2007-01-26-Parallel Machine Learning Problems
Introduction: Parallel machine learning is a subject rarely addressed at machine learning conferences. Nevertheless, it seems likely to increase in importance because: Data set sizes appear to be growing substantially faster than computation. Essentially, this happens because more and more sensors of various sorts are being hooked up to the internet. Serial speedups of processors seem are relatively stalled. The new trend is to make processors more powerful by making them multicore . Both AMD and Intel are making dual core designs standard, with plans for more parallelism in the future. IBM’s Cell processor has (essentially) 9 cores. Modern graphics chips can have an order of magnitude more separate execution units. The meaning of ‘core’ varies a bit from processor to processor, but the overall trend seems quite clear. So, how do we parallelize machine learning algorithms? The simplest and most common technique is to simply run the same learning algorithm with di
3 0.56984651 431 hunch net-2011-04-18-A paper not at Snowbird
Introduction: Unfortunately, a scheduling failure meant I missed all of AIStat and most of the learning workshop , otherwise known as Snowbird, when it’s at Snowbird . At snowbird, the talk on Sum-Product networks by Hoifung Poon stood out to me ( Pedro Domingos is a coauthor.). The basic point was that by appropriately constructing networks based on sums and products, the normalization problem in probabilistic models is eliminated, yielding a highly tractable yet flexible representation+learning algorithm. As an algorithm, this is noticeably cleaner than deep belief networks with a claim to being an order of magnitude faster and working better on an image completion task. Snowbird doesn’t have real papers—just the abstract above. I look forward to seeing the paper. (added: Rodrigo points out the deep learning workshop draft .)
4 0.54948652 53 hunch net-2005-04-06-Structured Regret Minimization
Introduction: Geoff Gordon made an interesting presentation at the snowbird learning workshop discussing the use of no-regret algorithms for the use of several robot-related learning problems. There seems to be a draft here . This seems interesting in two ways: Drawback Removal One of the significant problems with these online algorithms is that they can’t cope with structure very easily. This drawback is addressed for certain structures. Experiments One criticism of such algorithms is that they are too “worst case”. Several experiments suggest that protecting yourself against this worst case does not necessarily incur a great loss.
5 0.47887173 307 hunch net-2008-07-04-More Presentation Preparation
Introduction: We’ve discussed presentation preparation before , but I have one more thing to add: transitioning . For a research presentation, it is substantially helpful for the audience if transitions are clear. A common outline for a research presentation in machine leanring is: The problem . Presentations which don’t describe the problem almost immediately lose people, because the context is missing to understand the detail. Prior relevant work . In many cases, a paper builds on some previous bit of work which must be understood in order to understand what the paper does. A common failure mode seems to be spending too much time on prior work. Discuss just the relevant aspects of prior work in the language of your work. Sometimes this is missing when unneeded. What we did . For theory papers in particular, it is often not possible to really cover the details. Prioritizing what you present can be very important. How it worked . Many papers in Machine Learning have some sor
6 0.47285342 404 hunch net-2010-08-20-The Workshop on Cores, Clusters, and Clouds
7 0.41196862 442 hunch net-2011-08-20-The Large Scale Learning Survey Tutorial
8 0.41132569 349 hunch net-2009-04-21-Interesting Presentations at Snowbird
9 0.40163073 346 hunch net-2009-03-18-Parallel ML primitives
10 0.39775079 249 hunch net-2007-06-21-Presentation Preparation
11 0.39271945 80 hunch net-2005-06-10-Workshops are not Conferences
12 0.38567421 128 hunch net-2005-11-05-The design of a computing cluster
13 0.35798535 440 hunch net-2011-08-06-Interesting thing at UAI 2011
14 0.35553247 37 hunch net-2005-03-08-Fast Physics for Learning
15 0.34751451 23 hunch net-2005-02-19-Loss Functions for Discriminative Training of Energy-Based Models
16 0.33930907 451 hunch net-2011-12-13-Vowpal Wabbit version 6.1 & the NIPS tutorial
17 0.32399571 381 hunch net-2009-12-07-Vowpal Wabbit version 4.0, and a NIPS heresy
18 0.32191354 187 hunch net-2006-06-25-Presentation of Proofs is Hard.
19 0.32011575 114 hunch net-2005-09-20-Workshop Proposal: Atomic Learning
20 0.30433765 438 hunch net-2011-07-11-Interesting Neural Network Papers at ICML 2011
topicId topicWeight
[(27, 0.159), (86, 0.632)]
simIndex simValue blogId blogTitle
same-blog 1 0.92223001 54 hunch net-2005-04-08-Fast SVMs
Introduction: There was a presentation at snowbird about parallelized support vector machines. In many cases, people parallelize by ignoring serial operations, but that is not what happened here—they parallelize with optimizations. Consequently, this seems to be the fastest SVM in existence. There is a related paper here .
2 0.75125378 477 hunch net-2013-01-01-Deep Learning 2012
Introduction: 2012 was a tumultuous year for me, but it was undeniably a great year for deep learning efforts. Signs of this include: Winning a Kaggle competition . Wide adoption of deep learning for speech recognition . Significant industry support . Gains in image recognition . This is a rare event in research: a significant capability breakout. Congratulations are definitely in order for those who managed to achieve it. At this point, deep learning algorithms seem like a choice undeniably worth investigating for real applications with significant data.
3 0.72105068 273 hunch net-2007-11-16-MLSS 2008
Introduction: … is in Kioloa, Australia from March 3 to March 14. It’s a great chance to learn something about Machine Learning and I’ve enjoyed several previous Machine Learning Summer Schools . The website has many more details , but registration is open now for the first 80 to sign up.
4 0.45627716 281 hunch net-2007-12-21-Vowpal Wabbit Code Release
Introduction: We are releasing the Vowpal Wabbit (Fast Online Learning) code as open source under a BSD (revised) license. This is a project at Yahoo! Research to build a useful large scale learning algorithm which Lihong Li , Alex Strehl , and I have been working on. To appreciate the meaning of “large”, it’s useful to define “small” and “medium”. A “small” supervised learning problem is one where a human could use a labeled dataset and come up with a reasonable predictor. A “medium” supervised learning problem dataset fits into the RAM of a modern desktop computer. A “large” supervised learning problem is one which does not fit into the RAM of a normal machine. VW tackles large scale learning problems by this definition of large. I’m not aware of any other open source Machine Learning tools which can handle this scale (although they may exist). A few close ones are: IBM’s Parallel Machine Learning Toolbox isn’t quite open source . The approach used by this toolbox is essenti
5 0.24396852 166 hunch net-2006-03-24-NLPers
Introduction: Hal Daume has started the NLPers blog to discuss learning for language problems.
6 0.24396852 246 hunch net-2007-06-13-Not Posting
7 0.24396852 418 hunch net-2010-12-02-Traffic Prediction Problem
8 0.24371712 274 hunch net-2007-11-28-Computational Consequences of Classification
9 0.24331504 247 hunch net-2007-06-14-Interesting Papers at COLT 2007
10 0.24276808 308 hunch net-2008-07-06-To Dual or Not
11 0.24228516 400 hunch net-2010-06-13-The Good News on Exploration and Learning
12 0.24218087 245 hunch net-2007-05-12-Loss Function Semantics
13 0.24213678 172 hunch net-2006-04-14-JMLR is a success
14 0.24191751 288 hunch net-2008-02-10-Complexity Illness
15 0.24064109 45 hunch net-2005-03-22-Active learning
16 0.23786905 9 hunch net-2005-02-01-Watchword: Loss
17 0.23656152 341 hunch net-2009-02-04-Optimal Proxy Loss for Classification
18 0.23613386 352 hunch net-2009-05-06-Machine Learning to AI
19 0.23394345 304 hunch net-2008-06-27-Reviewing Horror Stories
20 0.23244806 196 hunch net-2006-07-13-Regression vs. Classification as a Primitive