hunch_net hunch_net-2008 hunch_net-2008-329 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: My impression is that this is a particularly strong year for machine learning graduates. Here’s my short list of the strong graduates I know. Analpha (for perversity’s sake) by last name: Jenn Wortmann . When Jenn visited us for the summer, she had one , two , three , four papers. That is typical—she’s smart, capable, and follows up many directions of research. I believe approximately all of her many papers are on different subjects. Ruslan Salakhutdinov . A Science paper on bijective dimensionality reduction , mastered and improved on deep belief nets which seems like an important flavor of nonlinear learning, and in my experience he’s very fast, capable and creative at problem solving. Marc’Aurelio Ranzato . I haven’t spoken with Marc very much, but he had a great visit at Yahoo! this summer, and has an impressive portfolio of applications and improvements on convolutional neural networks and other deep learning algorithms. Lihong Li . Lihong developed the
sentIndex sentText sentNum sentScore
1 My impression is that this is a particularly strong year for machine learning graduates. [sent-1, score-0.097]
2 Here’s my short list of the strong graduates I know. [sent-2, score-0.204]
3 When Jenn visited us for the summer, she had one , two , three , four papers. [sent-4, score-0.2]
4 That is typical—she’s smart, capable, and follows up many directions of research. [sent-5, score-0.107]
5 I believe approximately all of her many papers are on different subjects. [sent-6, score-0.087]
6 A Science paper on bijective dimensionality reduction , mastered and improved on deep belief nets which seems like an important flavor of nonlinear learning, and in my experience he’s very fast, capable and creative at problem solving. [sent-8, score-1.042]
7 I haven’t spoken with Marc very much, but he had a great visit at Yahoo! [sent-10, score-0.089]
8 this summer, and has an impressive portfolio of applications and improvements on convolutional neural networks and other deep learning algorithms. [sent-11, score-0.392]
9 Lihong developed the KWIK (“Knows what it Knows”) learning framework , for analyzing and creating uncertainty-aware learning algorithms. [sent-13, score-0.084]
10 New mathematical models of learning are rare, and the topic is of substantial interest, so this is pretty cool. [sent-14, score-0.086]
11 He’s also worked on a wide variety of other subjects and in my experience is broadly capable. [sent-15, score-0.281]
12 Steve Hanneke : When the chapter on active learning is written in a machine learning textbook, I expect the disagreement coefficient to be in it. [sent-16, score-0.419]
13 Steve’s work is strongly distinguished from his adviser’s, so he is guaranteed capable of independent research. [sent-17, score-0.43]
14 There are a couple others such as Daniel and Jake for whom I’m unsure of their graduation plans, although they have already done good work. [sent-18, score-0.137]
15 In addition, I’m sure there are several others that I don’t know—feel free to mention others I don’t know in comments. [sent-19, score-0.365]
16 It’s traditional to imagine that one is best overall for hiring purposes, but I have substantial difficulty with that—the field of ML is simply to broad. [sent-20, score-0.41]
wordName wordTfidf (topN-words)
[('marc', 0.245), ('hiring', 0.245), ('jenn', 0.227), ('capable', 0.204), ('lihong', 0.196), ('steve', 0.189), ('knows', 0.183), ('others', 0.137), ('summer', 0.131), ('sake', 0.122), ('chapter', 0.122), ('coefficient', 0.122), ('adviser', 0.122), ('creative', 0.122), ('flavor', 0.122), ('kwik', 0.122), ('distinguished', 0.113), ('four', 0.113), ('guaranteed', 0.113), ('deep', 0.112), ('follows', 0.107), ('textbook', 0.107), ('portfolio', 0.107), ('mastered', 0.107), ('hanneke', 0.107), ('graduates', 0.107), ('variety', 0.102), ('nets', 0.102), ('plans', 0.098), ('strong', 0.097), ('experience', 0.097), ('disagreement', 0.094), ('jake', 0.094), ('convolutional', 0.091), ('mention', 0.091), ('rare', 0.091), ('dimensionality', 0.089), ('visit', 0.089), ('li', 0.089), ('purposes', 0.087), ('approximately', 0.087), ('visited', 0.087), ('nonlinear', 0.087), ('substantial', 0.086), ('analyzing', 0.084), ('broadly', 0.082), ('impressive', 0.082), ('written', 0.081), ('smart', 0.081), ('traditional', 0.079)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000002 329 hunch net-2008-11-28-A Bumper Crop of Machine Learning Graduates
Introduction: My impression is that this is a particularly strong year for machine learning graduates. Here’s my short list of the strong graduates I know. Analpha (for perversity’s sake) by last name: Jenn Wortmann . When Jenn visited us for the summer, she had one , two , three , four papers. That is typical—she’s smart, capable, and follows up many directions of research. I believe approximately all of her many papers are on different subjects. Ruslan Salakhutdinov . A Science paper on bijective dimensionality reduction , mastered and improved on deep belief nets which seems like an important flavor of nonlinear learning, and in my experience he’s very fast, capable and creative at problem solving. Marc’Aurelio Ranzato . I haven’t spoken with Marc very much, but he had a great visit at Yahoo! this summer, and has an impressive portfolio of applications and improvements on convolutional neural networks and other deep learning algorithms. Lihong Li . Lihong developed the
2 0.12670222 313 hunch net-2008-08-18-Radford Neal starts a blog
Introduction: here on statistics, ML, CS, and other things he knows well.
3 0.12484159 201 hunch net-2006-08-07-The Call of the Deep
Introduction: Many learning algorithms used in practice are fairly simple. Viewed representationally, many prediction algorithms either compute a linear separator of basic features (perceptron, winnow, weighted majority, SVM) or perhaps a linear separator of slightly more complex features (2-layer neural networks or kernelized SVMs). Should we go beyond this, and start using “deep” representations? What is deep learning? Intuitively, deep learning is about learning to predict in ways which can involve complex dependencies between the input (observed) features. Specifying this more rigorously turns out to be rather difficult. Consider the following cases: SVM with Gaussian Kernel. This is not considered deep learning, because an SVM with a gaussian kernel can’t succinctly represent certain decision surfaces. One of Yann LeCun ‘s examples is recognizing objects based on pixel values. An SVM will need a new support vector for each significantly different background. Since the number
4 0.11739264 407 hunch net-2010-08-23-Boosted Decision Trees for Deep Learning
Introduction: About 4 years ago, I speculated that decision trees qualify as a deep learning algorithm because they can make decisions which are substantially nonlinear in the input representation. Ping Li has proved this correct, empirically at UAI by showing that boosted decision trees can beat deep belief networks on versions of Mnist which are artificially hardened so as to make them solvable only by deep learning algorithms. This is an important point, because the ability to solve these sorts of problems is probably the best objective definition of a deep learning algorithm we have. I’m not that surprised. In my experience, if you can accept the computational drawbacks of a boosted decision tree, they can achieve pretty good performance. Geoff Hinton once told me that the great thing about deep belief networks is that they work. I understand that Ping had very substantial difficulty in getting this published, so I hope some reviewers step up to the standard of valuing wha
5 0.11341457 310 hunch net-2008-07-15-Interesting papers at COLT (and a bit of UAI & workshops)
Introduction: Here are a few papers from COLT 2008 that I found interesting. Maria-Florina Balcan , Steve Hanneke , and Jenn Wortman , The True Sample Complexity of Active Learning . This paper shows that in an asymptotic setting, active learning is always better than supervised learning (although the gap may be small). This is evidence that the only thing in the way of universal active learning is us knowing how to do it properly. Nir Ailon and Mehryar Mohri , An Efficient Reduction of Ranking to Classification . This paper shows how to robustly rank n objects with n log(n) classifications using a quicksort based algorithm. The result is applicable to many ranking loss functions and has implications for others. Michael Kearns and Jennifer Wortman . Learning from Collective Behavior . This is about learning in a new model, where the goal is to predict how a collection of interacting agents behave. One claim is that learning in this setting can be reduced to IID lear
6 0.10823686 433 hunch net-2011-04-23-ICML workshops due
7 0.10119346 384 hunch net-2009-12-24-Top graduates this season
8 0.10066045 309 hunch net-2008-07-10-Interesting papers, ICML 2008
9 0.096452214 490 hunch net-2013-11-09-Graduates and Postdocs
10 0.092215627 251 hunch net-2007-06-24-Interesting Papers at ICML 2007
11 0.084765166 448 hunch net-2011-10-24-2011 ML symposium and the bears
12 0.079860739 437 hunch net-2011-07-10-ICML 2011 and the future
13 0.079584047 355 hunch net-2009-05-19-CI Fellows
14 0.078570232 339 hunch net-2009-01-27-Key Scientific Challenges
15 0.078086339 477 hunch net-2013-01-01-Deep Learning 2012
16 0.076169178 432 hunch net-2011-04-20-The End of the Beginning of Active Learning
17 0.075631008 438 hunch net-2011-07-11-Interesting Neural Network Papers at ICML 2011
18 0.07558094 325 hunch net-2008-11-10-ICML Reviewing Criteria
19 0.074780598 466 hunch net-2012-06-05-ICML acceptance statistics
20 0.071902432 403 hunch net-2010-07-18-ICML & COLT 2010
topicId topicWeight
[(0, 0.183), (1, -0.031), (2, -0.079), (3, -0.003), (4, 0.057), (5, -0.014), (6, -0.039), (7, -0.011), (8, -0.043), (9, -0.061), (10, 0.067), (11, -0.058), (12, -0.104), (13, -0.075), (14, -0.065), (15, 0.113), (16, 0.026), (17, 0.074), (18, -0.012), (19, 0.018), (20, 0.061), (21, 0.072), (22, -0.022), (23, -0.051), (24, -0.053), (25, -0.059), (26, 0.059), (27, -0.022), (28, -0.061), (29, 0.146), (30, 0.015), (31, 0.021), (32, -0.066), (33, 0.014), (34, 0.104), (35, -0.026), (36, -0.029), (37, -0.07), (38, -0.027), (39, 0.04), (40, -0.033), (41, 0.049), (42, -0.048), (43, 0.014), (44, -0.045), (45, -0.031), (46, 0.014), (47, 0.007), (48, -0.038), (49, 0.002)]
simIndex simValue blogId blogTitle
same-blog 1 0.95170552 329 hunch net-2008-11-28-A Bumper Crop of Machine Learning Graduates
Introduction: My impression is that this is a particularly strong year for machine learning graduates. Here’s my short list of the strong graduates I know. Analpha (for perversity’s sake) by last name: Jenn Wortmann . When Jenn visited us for the summer, she had one , two , three , four papers. That is typical—she’s smart, capable, and follows up many directions of research. I believe approximately all of her many papers are on different subjects. Ruslan Salakhutdinov . A Science paper on bijective dimensionality reduction , mastered and improved on deep belief nets which seems like an important flavor of nonlinear learning, and in my experience he’s very fast, capable and creative at problem solving. Marc’Aurelio Ranzato . I haven’t spoken with Marc very much, but he had a great visit at Yahoo! this summer, and has an impressive portfolio of applications and improvements on convolutional neural networks and other deep learning algorithms. Lihong Li . Lihong developed the
2 0.65610081 414 hunch net-2010-10-17-Partha Niyogi has died
Introduction: from brain cancer. I asked Misha who worked with him to write about it. Partha Niyogi, Louis Block Professor in Computer Science and Statistics at the University of Chicago passed away on October 1, 2010, aged 43. I first met Partha Niyogi almost exactly ten years ago when I was a graduate student in math and he had just started as a faculty in Computer Science and Statistics at the University of Chicago. Strangely, we first talked at length due to a somewhat convoluted mathematical argument in a paper on pattern recognition. I asked him some questions about the paper, and, even though the topic was new to him, he had put serious thought into it and we started regular meetings. We made significant progress and developed a line of research stemming initially just from trying to understand that one paper and to simplify one derivation. I think this was typical of Partha, showing both his intellectual curiosity and his intuition for the serendipitous; having a sense and focus fo
3 0.62495661 384 hunch net-2009-12-24-Top graduates this season
Introduction: I would like to point out 3 graduates this season as having my confidence they are capable of doing great things. Daniel Hsu has diverse papers with diverse coauthors on {active learning, mulitlabeling, temporal learning, …} each covering new algorithms and methods of analysis. He is also a capable programmer, having helped me with some nitty-gritty details of cluster parallel Vowpal Wabbit this summer. He has an excellent tendency to just get things done. Nicolas Lambert doesn’t nominally work in machine learning, but I’ve found his work in elicitation relevant nevertheless. In essence, elicitable properties are closely related to learnable properties, and the elicitation complexity is related to a notion of learning complexity. See the Surrogate regret bounds paper for some related discussion. Few people successfully work at such a general level that it crosses fields, but he’s one of them. Yisong Yue is deeply focused on interactive learning, which he has a
4 0.61808759 407 hunch net-2010-08-23-Boosted Decision Trees for Deep Learning
Introduction: About 4 years ago, I speculated that decision trees qualify as a deep learning algorithm because they can make decisions which are substantially nonlinear in the input representation. Ping Li has proved this correct, empirically at UAI by showing that boosted decision trees can beat deep belief networks on versions of Mnist which are artificially hardened so as to make them solvable only by deep learning algorithms. This is an important point, because the ability to solve these sorts of problems is probably the best objective definition of a deep learning algorithm we have. I’m not that surprised. In my experience, if you can accept the computational drawbacks of a boosted decision tree, they can achieve pretty good performance. Geoff Hinton once told me that the great thing about deep belief networks is that they work. I understand that Ping had very substantial difficulty in getting this published, so I hope some reviewers step up to the standard of valuing wha
5 0.61172301 477 hunch net-2013-01-01-Deep Learning 2012
Introduction: 2012 was a tumultuous year for me, but it was undeniably a great year for deep learning efforts. Signs of this include: Winning a Kaggle competition . Wide adoption of deep learning for speech recognition . Significant industry support . Gains in image recognition . This is a rare event in research: a significant capability breakout. Congratulations are definitely in order for those who managed to achieve it. At this point, deep learning algorithms seem like a choice undeniably worth investigating for real applications with significant data.
6 0.61151779 201 hunch net-2006-08-07-The Call of the Deep
7 0.59423554 490 hunch net-2013-11-09-Graduates and Postdocs
8 0.58191413 438 hunch net-2011-07-11-Interesting Neural Network Papers at ICML 2011
9 0.53930944 168 hunch net-2006-04-02-Mad (Neuro)science
10 0.5377211 466 hunch net-2012-06-05-ICML acceptance statistics
11 0.53053474 279 hunch net-2007-12-19-Cool and interesting things seen at NIPS
12 0.51992613 250 hunch net-2007-06-23-Machine Learning Jobs are Growing on Trees
13 0.51662982 199 hunch net-2006-07-26-Two more UAI papers of interest
14 0.50905102 431 hunch net-2011-04-18-A paper not at Snowbird
15 0.50041103 228 hunch net-2007-01-15-The Machine Learning Department
16 0.49512058 158 hunch net-2006-02-24-A Fundamentalist Organization of Machine Learning
17 0.49273601 456 hunch net-2012-02-24-ICML+50%
18 0.48974162 224 hunch net-2006-12-12-Interesting Papers at NIPS 2006
19 0.48811397 293 hunch net-2008-03-23-Interactive Machine Learning
20 0.48666474 64 hunch net-2005-04-28-Science Fiction and Research
topicId topicWeight
[(7, 0.019), (10, 0.066), (27, 0.176), (38, 0.071), (47, 0.287), (53, 0.076), (55, 0.088), (84, 0.034), (94, 0.049), (95, 0.05)]
simIndex simValue blogId blogTitle
1 0.86739099 372 hunch net-2009-09-29-Machine Learning Protests at the G20
Introduction: The machine learning department at CMU turned out en masse to protest the G20 summit in Pittsburgh. Arthur Gretton uploaded some great photos covering the event
same-blog 2 0.86420459 329 hunch net-2008-11-28-A Bumper Crop of Machine Learning Graduates
Introduction: My impression is that this is a particularly strong year for machine learning graduates. Here’s my short list of the strong graduates I know. Analpha (for perversity’s sake) by last name: Jenn Wortmann . When Jenn visited us for the summer, she had one , two , three , four papers. That is typical—she’s smart, capable, and follows up many directions of research. I believe approximately all of her many papers are on different subjects. Ruslan Salakhutdinov . A Science paper on bijective dimensionality reduction , mastered and improved on deep belief nets which seems like an important flavor of nonlinear learning, and in my experience he’s very fast, capable and creative at problem solving. Marc’Aurelio Ranzato . I haven’t spoken with Marc very much, but he had a great visit at Yahoo! this summer, and has an impressive portfolio of applications and improvements on convolutional neural networks and other deep learning algorithms. Lihong Li . Lihong developed the
3 0.73355234 36 hunch net-2005-03-05-Funding Research
Introduction: The funding of research (and machine learning research) is an issue which seems to have become more significant in the United States over the last decade. The word “research” is applied broadly here to science, mathematics, and engineering. There are two essential difficulties with funding research: Longshot Paying a researcher is often a big gamble. Most research projects don’t pan out, but a few big payoffs can make it all worthwhile. Information Only Much of research is about finding the right way to think about or do something. The Longshot difficulty means that there is high variance in payoffs. This can be compensated for by funding many different research projects, reducing variance. The Information-Only difficulty means that it’s hard to extract a profit directly from many types of research, so companies have difficulty justifying basic research. (Patents are a mechanism for doing this. They are often extraordinarily clumsy or simply not applicable.) T
4 0.73057467 320 hunch net-2008-10-14-Who is Responsible for a Bad Review?
Introduction: Although I’m greatly interested in machine learning, I think it must be admitted that there is a large amount of low quality logic being used in reviews. The problem is bad enough that sometimes I wonder if the Byzantine generals limit has been exceeded. For example, I’ve seen recent reviews where the given reasons for rejecting are: [ NIPS ] Theorem A is uninteresting because Theorem B is uninteresting. [ UAI ] When you learn by memorization, the problem addressed is trivial. [NIPS] The proof is in the appendix. [NIPS] This has been done before. (… but not giving any relevant citations) Just for the record I want to point out what’s wrong with these reviews. A future world in which such reasons never come up again would be great, but I’m sure these errors will be committed many times more in the future. This is nonsense. A theorem should be evaluated based on it’s merits, rather than the merits of another theorem. Learning by memorization requires an expon
5 0.61721146 454 hunch net-2012-01-30-ICML Posters and Scope
Introduction: Normally, I don’t indulge in posters for ICML , but this year is naturally an exception for me. If you want one, there are a small number left here , if you sign up before February. It also seems worthwhile to give some sense of the scope and reviewing criteria for ICML for authors considering submitting papers. At ICML, the (very large) program committee does the reviewing which informs final decisions by area chairs on most papers. Program chairs setup the process, deal with exceptions or disagreements, and provide advice for the reviewing process. Providing advice is tricky (and easily misleading) because a conference is a community, and in the end the aggregate interests of the community determine the conference. Nevertheless, as a program chair this year it seems worthwhile to state the overall philosophy I have and what I plan to encourage (and occasionally discourage). At the highest level, I believe ICML exists to further research into machine learning, which I gene
6 0.60835749 343 hunch net-2009-02-18-Decision by Vetocracy
7 0.6071493 437 hunch net-2011-07-10-ICML 2011 and the future
8 0.6019789 332 hunch net-2008-12-23-Use of Learning Theory
9 0.60136515 464 hunch net-2012-05-03-Microsoft Research, New York City
10 0.60113388 207 hunch net-2006-09-12-Incentive Compatible Reviewing
11 0.59960598 95 hunch net-2005-07-14-What Learning Theory might do
12 0.59913117 19 hunch net-2005-02-14-Clever Methods of Overfitting
13 0.59879011 194 hunch net-2006-07-11-New Models
14 0.59832764 225 hunch net-2007-01-02-Retrospective
15 0.59783143 204 hunch net-2006-08-28-Learning Theory standards for NIPS 2006
16 0.59767026 478 hunch net-2013-01-07-NYU Large Scale Machine Learning Class
17 0.59649456 466 hunch net-2012-06-05-ICML acceptance statistics
18 0.59626776 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
19 0.59561718 297 hunch net-2008-04-22-Taking the next step
20 0.59430563 131 hunch net-2005-11-16-The Everything Ensemble Edge