hunch_net hunch_net-2005 hunch_net-2005-37 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: While everyone is silently working on ICML submissions, I found this discussion about a fast physics simulator chip interesting from a learning viewpoint. In many cases, learning attempts to predict the outcome of physical processes. Access to a fast simulator for these processes might be quite helpful in predicting the outcome. Bayesian learning in particular may directly benefit while many other algorithms (like support vector machines) might have their speed greatly increased. The biggest drawback is that writing software for these odd architectures is always difficult and time consuming, but a several-orders-of-magnitude speedup might make that worthwhile.
sentIndex sentText sentNum sentScore
1 While everyone is silently working on ICML submissions, I found this discussion about a fast physics simulator chip interesting from a learning viewpoint. [sent-1, score-1.708]
2 In many cases, learning attempts to predict the outcome of physical processes. [sent-2, score-0.658]
3 Access to a fast simulator for these processes might be quite helpful in predicting the outcome. [sent-3, score-1.193]
4 Bayesian learning in particular may directly benefit while many other algorithms (like support vector machines) might have their speed greatly increased. [sent-4, score-1.133]
5 The biggest drawback is that writing software for these odd architectures is always difficult and time consuming, but a several-orders-of-magnitude speedup might make that worthwhile. [sent-5, score-1.556]
wordName wordTfidf (topN-words)
[('simulator', 0.424), ('consuming', 0.229), ('silently', 0.229), ('chip', 0.212), ('speedup', 0.212), ('fast', 0.211), ('architectures', 0.177), ('biggest', 0.177), ('physical', 0.177), ('odd', 0.171), ('physics', 0.158), ('outcome', 0.154), ('might', 0.149), ('software', 0.148), ('attempts', 0.145), ('benefit', 0.145), ('drawback', 0.145), ('worthwhile', 0.14), ('submissions', 0.14), ('processes', 0.137), ('machines', 0.133), ('speed', 0.131), ('writing', 0.126), ('vector', 0.124), ('access', 0.121), ('predicting', 0.114), ('greatly', 0.11), ('cases', 0.109), ('everyone', 0.109), ('support', 0.105), ('directly', 0.102), ('bayesian', 0.095), ('helpful', 0.095), ('discussion', 0.091), ('predict', 0.085), ('found', 0.084), ('working', 0.082), ('always', 0.081), ('particular', 0.073), ('difficult', 0.073), ('icml', 0.071), ('interesting', 0.067), ('quite', 0.063), ('many', 0.056), ('algorithms', 0.052), ('make', 0.051), ('time', 0.046), ('may', 0.045), ('like', 0.045), ('learning', 0.041)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 37 hunch net-2005-03-08-Fast Physics for Learning
Introduction: While everyone is silently working on ICML submissions, I found this discussion about a fast physics simulator chip interesting from a learning viewpoint. In many cases, learning attempts to predict the outcome of physical processes. Access to a fast simulator for these processes might be quite helpful in predicting the outcome. Bayesian learning in particular may directly benefit while many other algorithms (like support vector machines) might have their speed greatly increased. The biggest drawback is that writing software for these odd architectures is always difficult and time consuming, but a several-orders-of-magnitude speedup might make that worthwhile.
2 0.074346513 433 hunch net-2011-04-23-ICML workshops due
Introduction: Lihong points out that ICML workshop submissions are due April 29.
3 0.073090494 136 hunch net-2005-12-07-Is the Google way the way for machine learning?
Introduction: Urs Hoelzle from Google gave an invited presentation at NIPS . In the presentation, he strongly advocates interacting with data in a particular scalable manner which is something like the following: Make a cluster of machines. Build a unified filesystem. (Google uses GFS, but NFS or other approaches work reasonably well for smaller clusters.) Interact with data via MapReduce . Creating a cluster of machines is, by this point, relatively straightforward. Unified filesystems are a little bit tricky—GFS is capable by design of essentially unlimited speed throughput to disk. NFS can bottleneck because all of the data has to move through one machine. Nevertheless, this may not be a limiting factor for smaller clusters. MapReduce is a programming paradigm. Essentially, it is a combination of a data element transform (map) and an agreggator/selector (reduce). These operations are highly parallelizable and the claim is that they support the forms of data interacti
4 0.072648749 454 hunch net-2012-01-30-ICML Posters and Scope
Introduction: Normally, I don’t indulge in posters for ICML , but this year is naturally an exception for me. If you want one, there are a small number left here , if you sign up before February. It also seems worthwhile to give some sense of the scope and reviewing criteria for ICML for authors considering submitting papers. At ICML, the (very large) program committee does the reviewing which informs final decisions by area chairs on most papers. Program chairs setup the process, deal with exceptions or disagreements, and provide advice for the reviewing process. Providing advice is tricky (and easily misleading) because a conference is a community, and in the end the aggregate interests of the community determine the conference. Nevertheless, as a program chair this year it seems worthwhile to state the overall philosophy I have and what I plan to encourage (and occasionally discourage). At the highest level, I believe ICML exists to further research into machine learning, which I gene
5 0.071891606 95 hunch net-2005-07-14-What Learning Theory might do
Introduction: I wanted to expand on this post and some of the previous problems/research directions about where learning theory might make large strides. Why theory? The essential reason for theory is “intuition extension”. A very good applied learning person can master some particular application domain yielding the best computer algorithms for solving that problem. A very good theory can take the intuitions discovered by this and other applied learning people and extend them to new domains in a relatively automatic fashion. To do this, we take these basic intuitions and try to find a mathematical model that: Explains the basic intuitions. Makes new testable predictions about how to learn. Succeeds in so learning. This is “intuition extension”: taking what we have learned somewhere else and applying it in new domains. It is fundamentally useful to everyone because it increases the level of automation in solving problems. Where next for learning theory? I like the a
6 0.070701174 152 hunch net-2006-01-30-Should the Input Representation be a Vector?
7 0.070512429 60 hunch net-2005-04-23-Advantages and Disadvantages of Bayesian Learning
8 0.068349361 53 hunch net-2005-04-06-Structured Regret Minimization
9 0.066193305 120 hunch net-2005-10-10-Predictive Search is Coming
10 0.062798992 65 hunch net-2005-05-02-Reviewing techniques for conferences
11 0.062742069 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
12 0.060985077 277 hunch net-2007-12-12-Workshop Summary—Principles of Learning Problem Design
13 0.060766354 362 hunch net-2009-06-26-Netflix nearly done
14 0.0607436 210 hunch net-2006-09-28-Programming Languages for Machine Learning Implementations
15 0.060548373 308 hunch net-2008-07-06-To Dual or Not
16 0.060325492 347 hunch net-2009-03-26-Machine Learning is too easy
17 0.059988823 314 hunch net-2008-08-24-Mass Customized Medicine in the Future?
18 0.059139539 452 hunch net-2012-01-04-Why ICML? and the summer conferences
19 0.058454916 346 hunch net-2009-03-18-Parallel ML primitives
20 0.058179088 54 hunch net-2005-04-08-Fast SVMs
topicId topicWeight
[(0, 0.132), (1, -0.007), (2, -0.027), (3, 0.012), (4, 0.03), (5, 0.009), (6, -0.032), (7, 0.015), (8, 0.048), (9, 0.0), (10, -0.053), (11, -0.059), (12, -0.022), (13, -0.025), (14, 0.059), (15, 0.014), (16, -0.028), (17, -0.066), (18, -0.009), (19, -0.017), (20, -0.013), (21, -0.018), (22, -0.038), (23, 0.071), (24, 0.026), (25, -0.048), (26, -0.011), (27, -0.025), (28, -0.054), (29, 0.058), (30, -0.064), (31, -0.015), (32, -0.006), (33, 0.051), (34, -0.038), (35, -0.01), (36, -0.052), (37, 0.001), (38, -0.028), (39, -0.03), (40, 0.058), (41, -0.065), (42, -0.037), (43, 0.072), (44, -0.036), (45, -0.027), (46, 0.035), (47, -0.044), (48, 0.022), (49, 0.046)]
simIndex simValue blogId blogTitle
same-blog 1 0.9422816 37 hunch net-2005-03-08-Fast Physics for Learning
Introduction: While everyone is silently working on ICML submissions, I found this discussion about a fast physics simulator chip interesting from a learning viewpoint. In many cases, learning attempts to predict the outcome of physical processes. Access to a fast simulator for these processes might be quite helpful in predicting the outcome. Bayesian learning in particular may directly benefit while many other algorithms (like support vector machines) might have their speed greatly increased. The biggest drawback is that writing software for these odd architectures is always difficult and time consuming, but a several-orders-of-magnitude speedup might make that worthwhile.
2 0.51751572 346 hunch net-2009-03-18-Parallel ML primitives
Introduction: Previously, we discussed parallel machine learning a bit. As parallel ML is rather difficult, I’d like to describe my thinking at the moment, and ask for advice from the rest of the world. This is particularly relevant right now, as I’m attending a workshop tomorrow on parallel ML. Parallelizing slow algorithms seems uncompelling. Parallelizing many algorithms also seems uncompelling, because the effort required to parallelize is substantial. This leaves the question: Which one fast algorithm is the best to parallelize? What is a substantially different second? One compellingly fast simple algorithm is online gradient descent on a linear representation. This is the core of Leon’s sgd code and Vowpal Wabbit . Antoine Bordes showed a variant was competitive in the large scale learning challenge . It’s also a decades old primitive which has been reused in many algorithms, and continues to be reused. It also applies to online learning rather than just online optimiz
3 0.5120948 152 hunch net-2006-01-30-Should the Input Representation be a Vector?
Introduction: Let’s suppose that we are trying to create a general purpose machine learning box. The box is fed many examples of the function it is supposed to learn and (hopefully) succeeds. To date, most such attempts to produce a box of this form take a vector as input. The elements of the vector might be bits, real numbers, or ‘categorical’ data (a discrete set of values). On the other hand, there are a number of succesful applications of machine learning which do not seem to use a vector representation as input. For example, in vision, convolutional neural networks have been used to solve several vision problems. The input to the convolutional neural network is essentially the raw camera image as a matrix . In learning for natural languages, several people have had success on problems like parts-of-speech tagging using predictors restricted to a window surrounding the word to be predicted. A vector window and a matrix both imply a notion of locality which is being actively and
4 0.51163965 210 hunch net-2006-09-28-Programming Languages for Machine Learning Implementations
Introduction: Machine learning algorithms have a much better chance of being widely adopted if they are implemented in some easy-to-use code. There are several important concerns associated with machine learning which stress programming languages on the ease-of-use vs. speed frontier. Speed The rate at which data sources are growing seems to be outstripping the rate at which computational power is growing, so it is important that we be able to eak out every bit of computational power. Garbage collected languages ( java , ocaml , perl and python ) often have several issues here. Garbage collection often implies that floating point numbers are “boxed”: every float is represented by a pointer to a float. Boxing can cause an order of magnitude slowdown because an extra nonlocalized memory reference is made, and accesses to main memory can are many CPU cycles long. Garbage collection often implies that considerably more memory is used than is necessary. This has a variable effect. I
5 0.51023811 262 hunch net-2007-09-16-Optimizing Machine Learning Programs
Introduction: Machine learning is often computationally bounded which implies that the ability to write fast code becomes important if you ever want to implement a machine learning algorithm. Basic tactical optimizations are covered well elsewhere , but I haven’t seen a reasonable guide to higher level optimizations, which are the most important in my experience. Here are some of the higher level optimizations I’ve often found useful. Algorithmic Improvement First . This is Hard, but it is the most important consideration, and typically yields the most benefits. Good optimizations here are publishable. In the context of machine learning, you should be familiar with the arguments for online vs. batch learning. Choice of Language . There are many arguments about the choice of language . Sometimes you don’t have a choice when interfacing with other people. Personally, I favor C/C++ when I want to write fast code. This (admittedly) makes me a slower programmer than when using higher lev
6 0.4985114 254 hunch net-2007-07-12-ICML Trends
7 0.49842852 469 hunch net-2012-07-09-Videolectures
8 0.49608383 13 hunch net-2005-02-04-JMLG
9 0.49246815 250 hunch net-2007-06-23-Machine Learning Jobs are Growing on Trees
10 0.49132812 90 hunch net-2005-07-07-The Limits of Learning Theory
11 0.48959053 305 hunch net-2008-06-30-ICML has a comment system
12 0.48926032 366 hunch net-2009-08-03-Carbon in Computer Science Research
13 0.48141384 84 hunch net-2005-06-22-Languages of Learning
14 0.48088104 229 hunch net-2007-01-26-Parallel Machine Learning Problems
15 0.47541368 114 hunch net-2005-09-20-Workshop Proposal: Atomic Learning
16 0.47055468 348 hunch net-2009-04-02-Asymmophobia
17 0.47045434 314 hunch net-2008-08-24-Mass Customized Medicine in the Future?
18 0.45985553 382 hunch net-2009-12-09-Future Publication Models @ NIPS
19 0.45891929 168 hunch net-2006-04-02-Mad (Neuro)science
20 0.45848659 454 hunch net-2012-01-30-ICML Posters and Scope
topicId topicWeight
[(27, 0.156), (49, 0.295), (53, 0.09), (55, 0.141), (94, 0.099), (95, 0.09)]
simIndex simValue blogId blogTitle
1 0.92714459 224 hunch net-2006-12-12-Interesting Papers at NIPS 2006
Introduction: Here are some papers that I found surprisingly interesting. Yoshua Bengio , Pascal Lamblin, Dan Popovici, Hugo Larochelle, Greedy Layer-wise Training of Deep Networks . Empirically investigates some of the design choices behind deep belief networks. Long Zhu , Yuanhao Chen, Alan Yuille Unsupervised Learning of a Probabilistic Grammar for Object Detection and Parsing. An unsupervised method for detecting objects using simple feature filters that works remarkably well on the (supervised) caltech-101 dataset . Shai Ben-David , John Blitzer , Koby Crammer , and Fernando Pereira , Analysis of Representations for Domain Adaptation . This is the first analysis I’ve seen of learning with respect to samples drawn differently from the evaluation distribution which depends on reasonable measurable quantities. All of these papers turn out to have a common theme—the power of unlabeled data to do generically useful things.
same-blog 2 0.9072817 37 hunch net-2005-03-08-Fast Physics for Learning
Introduction: While everyone is silently working on ICML submissions, I found this discussion about a fast physics simulator chip interesting from a learning viewpoint. In many cases, learning attempts to predict the outcome of physical processes. Access to a fast simulator for these processes might be quite helpful in predicting the outcome. Bayesian learning in particular may directly benefit while many other algorithms (like support vector machines) might have their speed greatly increased. The biggest drawback is that writing software for these odd architectures is always difficult and time consuming, but a several-orders-of-magnitude speedup might make that worthwhile.
3 0.90163279 122 hunch net-2005-10-13-Site tweak
Introduction: Several people have had difficulty with comments which seem to have an allowed language significantly poorer than posts. The set of allowed html tags has been increased and the markdown filter has been put in place to try to make commenting easier. I’ll put some examples into the comments of this post.
4 0.87748474 365 hunch net-2009-07-31-Vowpal Wabbit Open Source Project
Introduction: Today brings a new release of the Vowpal Wabbit fast online learning software. This time, unlike the previous release, the project itself is going open source, developing via github . For example, the lastest and greatest can be downloaded via: git clone git://github.com/JohnLangford/vowpal_wabbit.git If you aren’t familiar with git , it’s a distributed version control system which supports quick and easy branching, as well as reconciliation. This version of the code is confirmed to compile without complaint on at least some flavors of OSX as well as Linux boxes. As much of the point of this project is pushing the limits of fast and effective machine learning, let me mention a few datapoints from my experience. The program can effectively scale up to batch-style training on sparse terafeature (i.e. 10 12 sparse feature) size datasets. The limiting factor is typically i/o. I started using the the real datasets from the large-scale learning workshop as a conve
5 0.82169586 338 hunch net-2009-01-23-An Active Learning Survey
Introduction: Burr Settles wrote a fairly comprehensive survey of active learning . He intends to maintain and update the survey, so send him any suggestions you have.
6 0.78751218 23 hunch net-2005-02-19-Loss Functions for Discriminative Training of Energy-Based Models
7 0.78142238 348 hunch net-2009-04-02-Asymmophobia
8 0.69835031 359 hunch net-2009-06-03-Functionally defined Nonlinear Dynamic Models
9 0.65536308 141 hunch net-2005-12-17-Workshops as Franchise Conferences
10 0.65111059 438 hunch net-2011-07-11-Interesting Neural Network Papers at ICML 2011
11 0.6468969 452 hunch net-2012-01-04-Why ICML? and the summer conferences
12 0.63679421 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
13 0.62764937 75 hunch net-2005-05-28-Running A Machine Learning Summer School
14 0.62722743 105 hunch net-2005-08-23-(Dis)similarities between academia and open source programmers
15 0.62631553 297 hunch net-2008-04-22-Taking the next step
16 0.62534517 416 hunch net-2010-10-29-To Vidoelecture or not
17 0.62270093 151 hunch net-2006-01-25-1 year
18 0.62182522 437 hunch net-2011-07-10-ICML 2011 and the future
19 0.62150902 423 hunch net-2011-02-02-User preferences for search engines
20 0.62010199 40 hunch net-2005-03-13-Avoiding Bad Reviewing