hunch_net hunch_net-2005 hunch_net-2005-135 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: In everyday use a model is a system which explains the behavior of some system, hopefully at the level where some alteration of the model predicts some alteration of the real-world system. In machine learning “model” has several variant definitions. Everyday . The common definition is sometimes used. Parameterized . Sometimes model is a short-hand for “parameterized model”. Here, it refers to a model with unspecified free parameters. In the Bayesian learning approach, you typically have a prior over (everyday) models. Predictive . Even further from everyday use is the predictive model. Examples of this are “my model is a decision tree” or “my model is a support vector machine”. Here, there is no real sense in which an SVM explains the underlying process. For example, an SVM tells us nothing in particular about how alterations to the real-world system would create a change. Which definition is being used at any particular time is important information. For examp
sentIndex sentText sentNum sentScore
1 In everyday use a model is a system which explains the behavior of some system, hopefully at the level where some alteration of the model predicts some alteration of the real-world system. [sent-1, score-2.53]
2 In machine learning “model” has several variant definitions. [sent-2, score-0.072]
3 Here, it refers to a model with unspecified free parameters. [sent-7, score-0.58]
4 In the Bayesian learning approach, you typically have a prior over (everyday) models. [sent-8, score-0.085]
5 Even further from everyday use is the predictive model. [sent-10, score-0.881]
6 Examples of this are “my model is a decision tree” or “my model is a support vector machine”. [sent-11, score-1.028]
7 Here, there is no real sense in which an SVM explains the underlying process. [sent-12, score-0.317]
8 For example, an SVM tells us nothing in particular about how alterations to the real-world system would create a change. [sent-13, score-0.516]
9 Which definition is being used at any particular time is important information. [sent-14, score-0.21]
10 For example, if it’s a parameterized or predictive model, this implies some learning is required. [sent-15, score-0.534]
11 If it’s a predictive model, then the set of operations which can be done to the model are restricted with respect to everyday usage. [sent-16, score-1.513]
12 I don’t have any particular advice here other than “watch out”—be aware of the distinctions, watch for this source of ambiguity, and clarify when necessary. [sent-17, score-0.553]
wordName wordTfidf (topN-words)
[('everyday', 0.597), ('model', 0.443), ('parameterized', 0.255), ('predictive', 0.239), ('alteration', 0.239), ('watch', 0.185), ('explains', 0.164), ('svm', 0.146), ('definition', 0.108), ('ambiguity', 0.106), ('system', 0.103), ('particular', 0.102), ('clarify', 0.098), ('alterations', 0.098), ('refers', 0.088), ('distinctions', 0.088), ('predicts', 0.079), ('underlying', 0.079), ('operations', 0.079), ('restricted', 0.077), ('behavior', 0.073), ('variant', 0.072), ('tells', 0.072), ('sometimes', 0.066), ('nothing', 0.065), ('advice', 0.065), ('hopefully', 0.061), ('aware', 0.057), ('vector', 0.057), ('tree', 0.055), ('free', 0.049), ('support', 0.049), ('prior', 0.048), ('necessary', 0.047), ('source', 0.046), ('use', 0.045), ('level', 0.044), ('bayesian', 0.044), ('sense', 0.043), ('respect', 0.042), ('create', 0.041), ('example', 0.04), ('implies', 0.04), ('typically', 0.037), ('decision', 0.036), ('done', 0.036), ('us', 0.035), ('approach', 0.032), ('common', 0.031), ('real', 0.031)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 135 hunch net-2005-12-04-Watchword: model
Introduction: In everyday use a model is a system which explains the behavior of some system, hopefully at the level where some alteration of the model predicts some alteration of the real-world system. In machine learning “model” has several variant definitions. Everyday . The common definition is sometimes used. Parameterized . Sometimes model is a short-hand for “parameterized model”. Here, it refers to a model with unspecified free parameters. In the Bayesian learning approach, you typically have a prior over (everyday) models. Predictive . Even further from everyday use is the predictive model. Examples of this are “my model is a decision tree” or “my model is a support vector machine”. Here, there is no real sense in which an SVM explains the underlying process. For example, an SVM tells us nothing in particular about how alterations to the real-world system would create a change. Which definition is being used at any particular time is important information. For examp
2 0.19835004 194 hunch net-2006-07-11-New Models
Introduction: How should we, as researchers in machine learning, organize ourselves? The most immediate measurable objective of computer science research is publishing a paper. The most difficult aspect of publishing a paper is having reviewers accept and recommend it for publication. The simplest mechanism for doing this is to show theoretical progress on some standard, well-known easily understood problem. In doing this, we often fall into a local minima of the research process. The basic problem in machine learning is that it is very unclear that the mathematical model is the right one for the (or some) real problem. A good mathematical model in machine learning should have one fundamental trait: it should aid the design of effective learning algorithms. To date, our ability to solve interesting learning problems (speech recognition, machine translation, object recognition, etc…) remains limited (although improving), so the “rightness” of our models is in doubt. If our mathematical mod
3 0.14629947 301 hunch net-2008-05-23-Three levels of addressing the Netflix Prize
Introduction: In October 2006, the online movie renter, Netflix, announced the Netflix Prize contest. They published a comprehensive dataset including more than 100 million movie ratings, which were performed by about 480,000 real customers on 17,770 movies.  Competitors in the challenge are required to estimate a few million ratings.  To win the “grand prize,” they need to deliver a 10% improvement in the prediction error compared with the results of Cinematch, Netflix’s proprietary recommender system. Best current results deliver 9.12% improvement , which is quite close to the 10% goal, yet painfully distant.  The Netflix Prize breathed new life and excitement into recommender systems research. The competition allowed the wide research community to access a large scale, real life dataset. Beyond this, the competition changed the rules of the game. Claiming that your nice idea could outperform some mediocre algorithms on some toy dataset is no longer acceptable. Now researcher
4 0.14016341 95 hunch net-2005-07-14-What Learning Theory might do
Introduction: I wanted to expand on this post and some of the previous problems/research directions about where learning theory might make large strides. Why theory? The essential reason for theory is “intuition extension”. A very good applied learning person can master some particular application domain yielding the best computer algorithms for solving that problem. A very good theory can take the intuitions discovered by this and other applied learning people and extend them to new domains in a relatively automatic fashion. To do this, we take these basic intuitions and try to find a mathematical model that: Explains the basic intuitions. Makes new testable predictions about how to learn. Succeeds in so learning. This is “intuition extension”: taking what we have learned somewhere else and applying it in new domains. It is fundamentally useful to everyone because it increases the level of automation in solving problems. Where next for learning theory? I like the a
5 0.12033739 97 hunch net-2005-07-23-Interesting papers at ACL
Introduction: A recent discussion indicated that one goal of this blog might be to allow people to post comments about recent papers that they liked. I think this could potentially be very useful, especially for those with diverse interests but only finite time to read through conference proceedings. ACL 2005 recently completed, and here are four papers from that conference that I thought were either good or perhaps of interest to a machine learning audience. David Chiang, A Hierarchical Phrase-Based Model for Statistical Machine Translation . (Best paper award.) This paper takes the standard phrase-based MT model that is popular in our field (basically, translate a sentence by individually translating phrases and reordering them according to a complicated statistical model) and extends it to take into account hierarchy in phrases, so that you can learn things like “X ‘s Y” -> “Y de X” in chinese, where X and Y are arbitrary phrases. This takes a step toward linguistic syntax for MT, whic
6 0.10821897 27 hunch net-2005-02-23-Problem: Reinforcement Learning with Classification
7 0.10344133 51 hunch net-2005-04-01-The Producer-Consumer Model of Research
8 0.089868478 60 hunch net-2005-04-23-Advantages and Disadvantages of Bayesian Learning
9 0.088632397 235 hunch net-2007-03-03-All Models of Learning have Flaws
10 0.086004063 23 hunch net-2005-02-19-Loss Functions for Discriminative Training of Energy-Based Models
11 0.08255209 347 hunch net-2009-03-26-Machine Learning is too easy
12 0.079920739 8 hunch net-2005-02-01-NIPS: Online Bayes
13 0.07379996 204 hunch net-2006-08-28-Learning Theory standards for NIPS 2006
14 0.071793906 398 hunch net-2010-05-10-Aggregation of estimators, sparsity in high dimension and computational feasibility
15 0.071569145 54 hunch net-2005-04-08-Fast SVMs
16 0.070887305 335 hunch net-2009-01-08-Predictive Analytics World
17 0.070021674 368 hunch net-2009-08-26-Another 10-year paper in Machine Learning
18 0.064509302 252 hunch net-2007-07-01-Watchword: Online Learning
19 0.059455991 435 hunch net-2011-05-16-Research Directions for Machine Learning and Algorithms
20 0.058070935 440 hunch net-2011-08-06-Interesting thing at UAI 2011
topicId topicWeight
[(0, 0.113), (1, 0.053), (2, -0.003), (3, 0.021), (4, 0.023), (5, -0.015), (6, -0.026), (7, 0.033), (8, 0.069), (9, -0.047), (10, 0.011), (11, -0.052), (12, -0.053), (13, -0.001), (14, 0.077), (15, 0.01), (16, 0.037), (17, 0.04), (18, 0.127), (19, -0.125), (20, -0.116), (21, -0.043), (22, -0.056), (23, -0.027), (24, 0.049), (25, 0.057), (26, -0.02), (27, -0.04), (28, -0.032), (29, -0.028), (30, 0.033), (31, 0.084), (32, -0.006), (33, 0.017), (34, 0.026), (35, 0.012), (36, 0.022), (37, -0.139), (38, -0.004), (39, -0.139), (40, -0.091), (41, -0.094), (42, -0.085), (43, 0.048), (44, 0.048), (45, -0.035), (46, 0.032), (47, -0.034), (48, -0.024), (49, 0.026)]
simIndex simValue blogId blogTitle
same-blog 1 0.98595375 135 hunch net-2005-12-04-Watchword: model
Introduction: In everyday use a model is a system which explains the behavior of some system, hopefully at the level where some alteration of the model predicts some alteration of the real-world system. In machine learning “model” has several variant definitions. Everyday . The common definition is sometimes used. Parameterized . Sometimes model is a short-hand for “parameterized model”. Here, it refers to a model with unspecified free parameters. In the Bayesian learning approach, you typically have a prior over (everyday) models. Predictive . Even further from everyday use is the predictive model. Examples of this are “my model is a decision tree” or “my model is a support vector machine”. Here, there is no real sense in which an SVM explains the underlying process. For example, an SVM tells us nothing in particular about how alterations to the real-world system would create a change. Which definition is being used at any particular time is important information. For examp
2 0.77866089 194 hunch net-2006-07-11-New Models
Introduction: How should we, as researchers in machine learning, organize ourselves? The most immediate measurable objective of computer science research is publishing a paper. The most difficult aspect of publishing a paper is having reviewers accept and recommend it for publication. The simplest mechanism for doing this is to show theoretical progress on some standard, well-known easily understood problem. In doing this, we often fall into a local minima of the research process. The basic problem in machine learning is that it is very unclear that the mathematical model is the right one for the (or some) real problem. A good mathematical model in machine learning should have one fundamental trait: it should aid the design of effective learning algorithms. To date, our ability to solve interesting learning problems (speech recognition, machine translation, object recognition, etc…) remains limited (although improving), so the “rightness” of our models is in doubt. If our mathematical mod
3 0.72838026 301 hunch net-2008-05-23-Three levels of addressing the Netflix Prize
Introduction: In October 2006, the online movie renter, Netflix, announced the Netflix Prize contest. They published a comprehensive dataset including more than 100 million movie ratings, which were performed by about 480,000 real customers on 17,770 movies.  Competitors in the challenge are required to estimate a few million ratings.  To win the “grand prize,” they need to deliver a 10% improvement in the prediction error compared with the results of Cinematch, Netflix’s proprietary recommender system. Best current results deliver 9.12% improvement , which is quite close to the 10% goal, yet painfully distant.  The Netflix Prize breathed new life and excitement into recommender systems research. The competition allowed the wide research community to access a large scale, real life dataset. Beyond this, the competition changed the rules of the game. Claiming that your nice idea could outperform some mediocre algorithms on some toy dataset is no longer acceptable. Now researcher
4 0.68385917 97 hunch net-2005-07-23-Interesting papers at ACL
Introduction: A recent discussion indicated that one goal of this blog might be to allow people to post comments about recent papers that they liked. I think this could potentially be very useful, especially for those with diverse interests but only finite time to read through conference proceedings. ACL 2005 recently completed, and here are four papers from that conference that I thought were either good or perhaps of interest to a machine learning audience. David Chiang, A Hierarchical Phrase-Based Model for Statistical Machine Translation . (Best paper award.) This paper takes the standard phrase-based MT model that is popular in our field (basically, translate a sentence by individually translating phrases and reordering them according to a complicated statistical model) and extends it to take into account hierarchy in phrases, so that you can learn things like “X ‘s Y” -> “Y de X” in chinese, where X and Y are arbitrary phrases. This takes a step toward linguistic syntax for MT, whic
5 0.63337666 95 hunch net-2005-07-14-What Learning Theory might do
Introduction: I wanted to expand on this post and some of the previous problems/research directions about where learning theory might make large strides. Why theory? The essential reason for theory is “intuition extension”. A very good applied learning person can master some particular application domain yielding the best computer algorithms for solving that problem. A very good theory can take the intuitions discovered by this and other applied learning people and extend them to new domains in a relatively automatic fashion. To do this, we take these basic intuitions and try to find a mathematical model that: Explains the basic intuitions. Makes new testable predictions about how to learn. Succeeds in so learning. This is “intuition extension”: taking what we have learned somewhere else and applying it in new domains. It is fundamentally useful to everyone because it increases the level of automation in solving problems. Where next for learning theory? I like the a
6 0.55943716 440 hunch net-2011-08-06-Interesting thing at UAI 2011
7 0.50876039 189 hunch net-2006-07-05-more icml papers
8 0.45494023 60 hunch net-2005-04-23-Advantages and Disadvantages of Bayesian Learning
9 0.43652505 23 hunch net-2005-02-19-Loss Functions for Discriminative Training of Energy-Based Models
10 0.43135518 27 hunch net-2005-02-23-Problem: Reinforcement Learning with Classification
11 0.42712089 398 hunch net-2010-05-10-Aggregation of estimators, sparsity in high dimension and computational feasibility
12 0.42157623 101 hunch net-2005-08-08-Apprenticeship Reinforcement Learning for Control
13 0.41182479 172 hunch net-2006-04-14-JMLR is a success
14 0.41002715 330 hunch net-2008-12-07-A NIPS paper
15 0.39702755 51 hunch net-2005-04-01-The Producer-Consumer Model of Research
16 0.39431995 204 hunch net-2006-08-28-Learning Theory standards for NIPS 2006
17 0.38792771 188 hunch net-2006-06-30-ICML papers
18 0.38289875 3 hunch net-2005-01-24-The Humanloop Spectrum of Machine Learning
19 0.38166022 275 hunch net-2007-11-29-The Netflix Crack
20 0.38148332 8 hunch net-2005-02-01-NIPS: Online Bayes
topicId topicWeight
[(27, 0.245), (53, 0.058), (55, 0.066), (83, 0.364), (94, 0.073), (95, 0.033)]
simIndex simValue blogId blogTitle
1 0.93837398 261 hunch net-2007-08-28-Live ML Class
Introduction: Davor and Chunnan point out that MLSS 2007 in Tuebingen has live video for the majority of the world that is not there (heh).
same-blog 2 0.89787346 135 hunch net-2005-12-04-Watchword: model
Introduction: In everyday use a model is a system which explains the behavior of some system, hopefully at the level where some alteration of the model predicts some alteration of the real-world system. In machine learning “model” has several variant definitions. Everyday . The common definition is sometimes used. Parameterized . Sometimes model is a short-hand for “parameterized model”. Here, it refers to a model with unspecified free parameters. In the Bayesian learning approach, you typically have a prior over (everyday) models. Predictive . Even further from everyday use is the predictive model. Examples of this are “my model is a decision tree” or “my model is a support vector machine”. Here, there is no real sense in which an SVM explains the underlying process. For example, an SVM tells us nothing in particular about how alterations to the real-world system would create a change. Which definition is being used at any particular time is important information. For examp
3 0.8887009 52 hunch net-2005-04-04-Grounds for Rejection
Introduction: It’s reviewing season right now, so I thought I would list (at a high level) the sorts of problems which I see in papers. Hopefully, this will help us all write better papers. The following flaws are fatal to any paper: Incorrect theorem or lemma statements A typo might be “ok”, if it can be understood. Any theorem or lemma which indicates an incorrect understanding of reality must be rejected. Not doing so would severely harm the integrity of the conference. A paper rejected for this reason must be fixed. Lack of Understanding If a paper is understood by none of the (typically 3) reviewers then it must be rejected for the same reason. This is more controversial than it sounds because there are some people who maximize paper complexity in the hope of impressing the reviewer. The tactic sometimes succeeds with some reviewers (but not with me). As a reviewer, I sometimes get lost for stupid reasons. This is why an anonymized communication channel with the author can
4 0.8875671 321 hunch net-2008-10-19-NIPS 2008 workshop on Kernel Learning
Introduction: We’d like to invite hunch.net readers to participate in the NIPS 2008 workshop on kernel learning. While the main focus is on automatically learning kernels from data, we are also also looking at the broader questions of feature selection, multi-task learning and multi-view learning. There are no restrictions on the learning problem being addressed (regression, classification, etc), and both theoretical and applied work will be considered. The deadline for submissions is October 24 . More detail can be found here . Corinna Cortes, Arthur Gretton, Gert Lanckriet, Mehryar Mohri, Afshin Rostamizadeh
5 0.8249076 228 hunch net-2007-01-15-The Machine Learning Department
Introduction: Carnegie Mellon School of Computer Science has the first academic Machine Learning department . This department already existed as the Center for Automated Learning and Discovery , but recently changed it’s name. The reason for changing the name is obvious: very few people think of themselves as “Automated Learner and Discoverers”, but there are number of people who think of themselves as “Machine Learners”. Machine learning is both more succinct and recognizable—good properties for a name. A more interesting question is “Should there be a Machine Learning Department?”. Tom Mitchell has a relevant whitepaper claiming that machine learning is answering a different question than other fields or departments. The fundamental debate here is “Is machine learning different from statistics?” At a cultural level, there is no real debate: they are different. Machine learning is characterized by several very active large peer reviewed conferences, operating in a computer
6 0.74935186 98 hunch net-2005-07-27-Not goal metrics
7 0.60823691 158 hunch net-2006-02-24-A Fundamentalist Organization of Machine Learning
8 0.60494024 345 hunch net-2009-03-08-Prediction Science
9 0.59716249 225 hunch net-2007-01-02-Retrospective
10 0.58813488 347 hunch net-2009-03-26-Machine Learning is too easy
11 0.58704263 435 hunch net-2011-05-16-Research Directions for Machine Learning and Algorithms
12 0.58664393 360 hunch net-2009-06-15-In Active Learning, the question changes
13 0.58587724 258 hunch net-2007-08-12-Exponentiated Gradient
14 0.58476114 351 hunch net-2009-05-02-Wielding a New Abstraction
15 0.5834403 220 hunch net-2006-11-27-Continuizing Solutions
16 0.58276993 337 hunch net-2009-01-21-Nearly all natural problems require nonlinearity
17 0.5816251 304 hunch net-2008-06-27-Reviewing Horror Stories
18 0.58126378 352 hunch net-2009-05-06-Machine Learning to AI
19 0.58114129 370 hunch net-2009-09-18-Necessary and Sufficient Research
20 0.58028084 3 hunch net-2005-01-24-The Humanloop Spectrum of Machine Learning