hunch_net hunch_net-2006 hunch_net-2006-199 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: In addition to Ed Snelson’s paper, there were (at least) two other papers that caught my eye at UAI. One was this paper by Sanjoy Dasgupta, Daniel Hsu and Nakul Verma at UCSD which shows in a surprisingly general and strong way that almost all linear projections of any jointly distributed vector random variable with finite first and second moments look sphereical and unimodal (in fact look like a scale mixture of Gaussians). Great result, as you’d expect from Sanjoy. The other paper which I found intriguing but which I just haven’t groked yet is this beast by Manfred and Dima Kuzmin. You can check out the (beautiful) slides if that helps. I feel like there is something deep here, but my brain is too small to understand it. The COLT and last NIPS papers/slides are also on Manfred’s page. Hopefully someone here can illuminate.
sentIndex sentText sentNum sentScore
1 In addition to Ed Snelson’s paper, there were (at least) two other papers that caught my eye at UAI. [sent-1, score-0.463]
2 The other paper which I found intriguing but which I just haven’t groked yet is this beast by Manfred and Dima Kuzmin. [sent-4, score-0.429]
3 You can check out the (beautiful) slides if that helps. [sent-5, score-0.268]
4 I feel like there is something deep here, but my brain is too small to understand it. [sent-6, score-0.557]
5 The COLT and last NIPS papers/slides are also on Manfred’s page. [sent-7, score-0.069]
wordName wordTfidf (topN-words)
[('manfred', 0.344), ('intriguing', 0.206), ('gaussians', 0.206), ('ed', 0.206), ('snelson', 0.206), ('moments', 0.206), ('eye', 0.191), ('look', 0.185), ('jointly', 0.18), ('projections', 0.18), ('beautiful', 0.165), ('caught', 0.165), ('mixture', 0.154), ('dasgupta', 0.15), ('paper', 0.147), ('surprisingly', 0.146), ('slides', 0.146), ('hsu', 0.142), ('sanjoy', 0.136), ('brain', 0.136), ('distributed', 0.133), ('variable', 0.128), ('daniel', 0.128), ('check', 0.122), ('finite', 0.12), ('hopefully', 0.118), ('vector', 0.112), ('shows', 0.109), ('addition', 0.107), ('feel', 0.106), ('haven', 0.104), ('fact', 0.099), ('someone', 0.095), ('scale', 0.094), ('deep', 0.094), ('random', 0.091), ('linear', 0.088), ('result', 0.086), ('colt', 0.085), ('nips', 0.084), ('strong', 0.082), ('like', 0.081), ('second', 0.081), ('expect', 0.079), ('found', 0.076), ('almost', 0.075), ('small', 0.071), ('last', 0.069), ('understand', 0.069), ('least', 0.066)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999982 199 hunch net-2006-07-26-Two more UAI papers of interest
Introduction: In addition to Ed Snelson’s paper, there were (at least) two other papers that caught my eye at UAI. One was this paper by Sanjoy Dasgupta, Daniel Hsu and Nakul Verma at UCSD which shows in a surprisingly general and strong way that almost all linear projections of any jointly distributed vector random variable with finite first and second moments look sphereical and unimodal (in fact look like a scale mixture of Gaussians). Great result, as you’d expect from Sanjoy. The other paper which I found intriguing but which I just haven’t groked yet is this beast by Manfred and Dima Kuzmin. You can check out the (beautiful) slides if that helps. I feel like there is something deep here, but my brain is too small to understand it. The COLT and last NIPS papers/slides are also on Manfred’s page. Hopefully someone here can illuminate.
2 0.16265567 248 hunch net-2007-06-19-How is Compressed Sensing going to change Machine Learning ?
Introduction: Compressed Sensing (CS) is a new framework developed by Emmanuel Candes , Terry Tao and David Donoho . To summarize, if you acquire a signal in some basis that is incoherent with the basis in which you know the signal to be sparse in, it is very likely you will be able to reconstruct the signal from these incoherent projections. Terry Tao, the recent Fields medalist , does a very nice job at explaining the framework here . He goes further in the theory description in this post where he mentions the central issue of the Uniform Uncertainty Principle. It so happens that random projections are on average incoherent, within the UUP meaning, with most known basis (sines, polynomials, splines, wavelets, curvelets …) and are therefore an ideal basis for Compressed Sensing. [ For more in-depth information on the subject, the Rice group has done a very good job at providing a central library of papers relevant to the growing subject: http://www.dsp.ece.rice.edu/cs/ ] The Machine
3 0.12496643 361 hunch net-2009-06-24-Interesting papers at UAICMOLT 2009
Introduction: Here’s a list of papers that I found interesting at ICML / COLT / UAI in 2009. Elad Hazan and Comandur Seshadhri Efficient learning algorithms for changing environments at ICML. This paper shows how to adapt learning algorithms that compete with fixed predictors to compete with changing policies. The definition of regret they deal with seems particularly useful in many situation. Hal Daume , Unsupervised Search-based Structured Prediction at ICML. This paper shows a technique for reducing unsupervised learning to supervised learning which (a) make a fast unsupervised learning algorithm and (b) makes semisupervised learning both easy and highly effective. There were two papers with similar results on active learning in the KWIK framework for linear regression, both reducing the sample complexity to . One was Nicolo Cesa-Bianchi , Claudio Gentile , and Francesco Orabona Robust Bounds for Classification via Selective Sampling at ICML and the other was Thoma
4 0.12383645 279 hunch net-2007-12-19-Cool and interesting things seen at NIPS
Introduction: I learned a number of things at NIPS . The financial people were there in greater force than previously. Two Sigma sponsored NIPS while DRW Trading had a booth. The adversarial machine learning workshop had a number of talks about interesting applications where an adversary really is out to try and mess up your learning algorithm. This is very different from the situation we often think of where the world is oblivious to our learning. This may present new and convincing applications for the learning-against-an-adversary work common at COLT . There were several interesing papers. Sanjoy Dasgupta , Daniel Hsu , and Claire Monteleoni had a paper on General Agnostic Active Learning . The basic idea is that active learning can be done via reduction to a form of supervised learning problem. This is great, because we have many supervised learning algorithms from which the benefits of active learning may be derived. Joseph Bradley and Robert Schapire had a P
5 0.1142295 197 hunch net-2006-07-17-A Winner
Introduction: Ed Snelson won the Predictive Uncertainty in Environmental Modelling Competition in the temp(erature) category using this algorithm . Some characteristics of the algorithm are: Gradient descent … on about 600 parameters … with local minima … to solve regression. This bears a strong resemblance to a neural network. The two main differences seem to be: The system has a probabilistic interpretation (which may aid design). There are (perhaps) fewer parameters than a typical neural network might have for the same problem (aiding speed).
6 0.097353317 467 hunch net-2012-06-15-Normal Deviate and the UCSC Machine Learning Summer School
7 0.08402767 309 hunch net-2008-07-10-Interesting papers, ICML 2008
8 0.081474259 139 hunch net-2005-12-11-More NIPS Papers
9 0.077782094 201 hunch net-2006-08-07-The Call of the Deep
10 0.075558901 406 hunch net-2010-08-22-KDD 2010
11 0.072144099 453 hunch net-2012-01-28-Why COLT?
12 0.071802013 403 hunch net-2010-07-18-ICML & COLT 2010
13 0.070805095 337 hunch net-2009-01-21-Nearly all natural problems require nonlinearity
14 0.070784032 240 hunch net-2007-04-21-Videolectures.net
15 0.069646433 398 hunch net-2010-05-10-Aggregation of estimators, sparsity in high dimension and computational feasibility
16 0.069333561 87 hunch net-2005-06-29-Not EM for clustering at COLT
17 0.067902103 327 hunch net-2008-11-16-Observations on Linearity for Reductions to Regression
18 0.0676466 329 hunch net-2008-11-28-A Bumper Crop of Machine Learning Graduates
19 0.066604257 462 hunch net-2012-04-20-Both new: STOC workshops and NEML
20 0.066581592 238 hunch net-2007-04-13-What to do with an unreasonable conditional accept
topicId topicWeight
[(0, 0.145), (1, -0.041), (2, -0.002), (3, -0.035), (4, 0.101), (5, 0.043), (6, -0.048), (7, -0.06), (8, -0.025), (9, -0.006), (10, 0.033), (11, -0.007), (12, -0.124), (13, -0.079), (14, 0.024), (15, 0.049), (16, -0.001), (17, 0.112), (18, 0.089), (19, 0.034), (20, -0.013), (21, 0.135), (22, 0.009), (23, -0.014), (24, -0.025), (25, 0.002), (26, -0.025), (27, 0.005), (28, -0.05), (29, -0.016), (30, 0.0), (31, -0.079), (32, -0.035), (33, 0.063), (34, 0.048), (35, 0.011), (36, -0.085), (37, 0.068), (38, 0.049), (39, 0.07), (40, 0.026), (41, 0.04), (42, -0.005), (43, 0.052), (44, -0.097), (45, -0.115), (46, -0.026), (47, -0.053), (48, 0.027), (49, -0.06)]
simIndex simValue blogId blogTitle
same-blog 1 0.98014671 199 hunch net-2006-07-26-Two more UAI papers of interest
Introduction: In addition to Ed Snelson’s paper, there were (at least) two other papers that caught my eye at UAI. One was this paper by Sanjoy Dasgupta, Daniel Hsu and Nakul Verma at UCSD which shows in a surprisingly general and strong way that almost all linear projections of any jointly distributed vector random variable with finite first and second moments look sphereical and unimodal (in fact look like a scale mixture of Gaussians). Great result, as you’d expect from Sanjoy. The other paper which I found intriguing but which I just haven’t groked yet is this beast by Manfred and Dima Kuzmin. You can check out the (beautiful) slides if that helps. I feel like there is something deep here, but my brain is too small to understand it. The COLT and last NIPS papers/slides are also on Manfred’s page. Hopefully someone here can illuminate.
2 0.62741297 439 hunch net-2011-08-01-Interesting papers at COLT 2011
Introduction: Since John did not attend COLT this year, I have been volunteered to report back on the hot stuff at this year’s meeting. The conference seemed to have pretty high quality stuff this year, and I found plenty of interesting papers on all the three days. I’m gonna pick some of my favorites going through the program in a chronological order. The first session on matrices seemed interesting for two reasons. First, the papers were quite nice. But more interestingly, this is a topic that has had a lot of presence in Statistics and Compressed sensing literature recently. So it was good to see high-dimensional matrices finally make their entry at COLT. The paper of Ohad and Shai on Collaborative Filtering with the Trace Norm: Learning, Bounding, and Transducing provides non-trivial guarantees on trace norm regularization in an agnostic setup, while Rina and Nati show how Rademacher averages can be used to get sharper results for matrix completion problems in their paper Concentr
3 0.61100596 361 hunch net-2009-06-24-Interesting papers at UAICMOLT 2009
Introduction: Here’s a list of papers that I found interesting at ICML / COLT / UAI in 2009. Elad Hazan and Comandur Seshadhri Efficient learning algorithms for changing environments at ICML. This paper shows how to adapt learning algorithms that compete with fixed predictors to compete with changing policies. The definition of regret they deal with seems particularly useful in many situation. Hal Daume , Unsupervised Search-based Structured Prediction at ICML. This paper shows a technique for reducing unsupervised learning to supervised learning which (a) make a fast unsupervised learning algorithm and (b) makes semisupervised learning both easy and highly effective. There were two papers with similar results on active learning in the KWIK framework for linear regression, both reducing the sample complexity to . One was Nicolo Cesa-Bianchi , Claudio Gentile , and Francesco Orabona Robust Bounds for Classification via Selective Sampling at ICML and the other was Thoma
4 0.59996396 248 hunch net-2007-06-19-How is Compressed Sensing going to change Machine Learning ?
Introduction: Compressed Sensing (CS) is a new framework developed by Emmanuel Candes , Terry Tao and David Donoho . To summarize, if you acquire a signal in some basis that is incoherent with the basis in which you know the signal to be sparse in, it is very likely you will be able to reconstruct the signal from these incoherent projections. Terry Tao, the recent Fields medalist , does a very nice job at explaining the framework here . He goes further in the theory description in this post where he mentions the central issue of the Uniform Uncertainty Principle. It so happens that random projections are on average incoherent, within the UUP meaning, with most known basis (sines, polynomials, splines, wavelets, curvelets …) and are therefore an ideal basis for Compressed Sensing. [ For more in-depth information on the subject, the Rice group has done a very good job at providing a central library of papers relevant to the growing subject: http://www.dsp.ece.rice.edu/cs/ ] The Machine
5 0.57366627 23 hunch net-2005-02-19-Loss Functions for Discriminative Training of Energy-Based Models
Introduction: This is a paper by Yann LeCun and Fu Jie Huang published at AISTAT 2005 . I found this paper very difficult to read, but it does have some point about a computational shortcut. This paper takes for granted that the method of solving a problem is gradient descent on parameters. Given this assumption, the question arises: Do you want to do gradient descent on a probabilistic model or something else? All (conditional) probabilistic models have the form p(y|x) = f(x,y)/Z(x) where Z(x) = sum y f(x,y) (the paper calls - log f(x,y) an “energy”). If f is parameterized by some w , the gradient has a term for Z(x) , and hence for every value of y . The paper claims, that such models can be optimized for classification purposes using only the correct y and the other y’ not y which maximizes f(x,y) . This can even be done on unnormalizable models. The paper further claims that this can be done with an approximate maximum. These claims are plausible based on experimen
6 0.51289332 406 hunch net-2010-08-22-KDD 2010
7 0.50100446 185 hunch net-2006-06-16-Regularization = Robustness
8 0.49561453 180 hunch net-2006-05-21-NIPS paper evaluation criteria
9 0.48559037 304 hunch net-2008-06-27-Reviewing Horror Stories
10 0.48085696 329 hunch net-2008-11-28-A Bumper Crop of Machine Learning Graduates
11 0.47683424 238 hunch net-2007-04-13-What to do with an unreasonable conditional accept
12 0.47307357 140 hunch net-2005-12-14-More NIPS Papers II
13 0.47192091 438 hunch net-2011-07-11-Interesting Neural Network Papers at ICML 2011
14 0.47090229 334 hunch net-2009-01-07-Interesting Papers at SODA 2009
15 0.47005653 309 hunch net-2008-07-10-Interesting papers, ICML 2008
16 0.46884677 188 hunch net-2006-06-30-ICML papers
17 0.45982388 256 hunch net-2007-07-20-Motivation should be the Responsibility of the Reviewer
18 0.45570585 280 hunch net-2007-12-20-Cool and Interesting things at NIPS, take three
19 0.4442932 279 hunch net-2007-12-19-Cool and interesting things seen at NIPS
20 0.44296318 364 hunch net-2009-07-11-Interesting papers at KDD
topicId topicWeight
[(10, 0.466), (27, 0.223), (53, 0.045), (55, 0.058), (95, 0.09)]
simIndex simValue blogId blogTitle
1 0.97447687 474 hunch net-2012-10-18-7th Annual Machine Learning Symposium
Introduction: A reminder that the New York Academy of Sciences will be hosting the 7th Annual Machine Learning Symposium tomorrow from 9:30am. The main program will feature invited talks from Peter Bartlett , William Freeman , and Vladimir Vapnik , along with numerous spotlight talks and a poster session. Following the main program, hackNY and Microsoft Research are sponsoring a networking hour with talks from machine learning practitioners at NYC startups (specifically bit.ly , Buzzfeed , Chartbeat , and Sense Networks , Visual Revenue ). This should be of great interest to everyone considering working in machine learning.
same-blog 2 0.9432317 199 hunch net-2006-07-26-Two more UAI papers of interest
Introduction: In addition to Ed Snelson’s paper, there were (at least) two other papers that caught my eye at UAI. One was this paper by Sanjoy Dasgupta, Daniel Hsu and Nakul Verma at UCSD which shows in a surprisingly general and strong way that almost all linear projections of any jointly distributed vector random variable with finite first and second moments look sphereical and unimodal (in fact look like a scale mixture of Gaussians). Great result, as you’d expect from Sanjoy. The other paper which I found intriguing but which I just haven’t groked yet is this beast by Manfred and Dima Kuzmin. You can check out the (beautiful) slides if that helps. I feel like there is something deep here, but my brain is too small to understand it. The COLT and last NIPS papers/slides are also on Manfred’s page. Hopefully someone here can illuminate.
3 0.92676371 38 hunch net-2005-03-09-Bad Reviewing
Introduction: This is a difficult subject to talk about for many reasons, but a discussion may be helpful. Bad reviewing is a problem in academia. The first step in understanding this is admitting to the problem, so here is a short list of examples of bad reviewing. Reviewer disbelieves theorem proof (ICML), or disbelieve theorem with a trivially false counterexample. (COLT) Reviewer internally swaps quantifiers in a theorem, concludes it has been done before and is trivial. (NIPS) Reviewer believes a technique will not work despite experimental validation. (COLT) Reviewers fail to notice flaw in theorem statement (CRYPTO). Reviewer erroneously claims that it has been done before (NIPS, SODA, JMLR)—(complete with references!) Reviewer inverts the message of a paper and concludes it says nothing important. (NIPS*2) Reviewer fails to distinguish between a DAG and a tree (SODA). Reviewer is enthusiastic about paper but clearly does not understand (ICML). Reviewer erroneously
4 0.86833429 55 hunch net-2005-04-10-Is the Goal Understanding or Prediction?
Introduction: Steve Smale and I have a debate about goals of learning theory. Steve likes theorems with a dependence on unobservable quantities. For example, if D is a distribution over a space X x [0,1] , you can state a theorem about the error rate dependent on the variance, E (x,y)~D (y-E y’~D|x [y']) 2 . I dislike this, because I want to use the theorems to produce code solving learning problems. Since I don’t know (and can’t measure) the variance, a theorem depending on the variance does not help me—I would not know what variance to plug into the learning algorithm. Recast more broadly, this is a debate between “declarative” and “operative” mathematics. A strong example of “declarative” mathematics is “a new kind of science” . Roughly speaking, the goal of this kind of approach seems to be finding a way to explain the observations we make. Examples include “some things are unpredictable”, “a phase transition exists”, etc… “Operative” mathematics helps you make predictions a
5 0.8156333 434 hunch net-2011-05-09-CI Fellows, again
Introduction: Lev and Hal point out the CI Fellows program is on again for this year. Lev visited me for a year under this program, and I quite enjoyed it. Due May 31.
6 0.77249116 240 hunch net-2007-04-21-Videolectures.net
7 0.75068647 182 hunch net-2006-06-05-Server Shift, Site Tweaks, Suggestions?
8 0.67424774 332 hunch net-2008-12-23-Use of Learning Theory
9 0.63122016 454 hunch net-2012-01-30-ICML Posters and Scope
10 0.53392667 343 hunch net-2009-02-18-Decision by Vetocracy
11 0.53274184 354 hunch net-2009-05-17-Server Update
12 0.53007972 207 hunch net-2006-09-12-Incentive Compatible Reviewing
13 0.52222645 279 hunch net-2007-12-19-Cool and interesting things seen at NIPS
14 0.52004027 329 hunch net-2008-11-28-A Bumper Crop of Machine Learning Graduates
15 0.51202518 244 hunch net-2007-05-09-The Missing Bound
16 0.50716531 432 hunch net-2011-04-20-The End of the Beginning of Active Learning
17 0.50705707 51 hunch net-2005-04-01-The Producer-Consumer Model of Research
18 0.50635523 5 hunch net-2005-01-26-Watchword: Probability
19 0.50214946 428 hunch net-2011-03-27-Vowpal Wabbit, v5.1
20 0.5004673 36 hunch net-2005-03-05-Funding Research