hunch_net hunch_net-2009 hunch_net-2009-378 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: If you search for “online learning” with any major search engine , it’s interesting to note that zero of the results are for online machine learning. This may not be a mistake if you are committed to a global ordering. In other words, the number of people specifically interested in the least interesting top-10 online human learning result might exceed the number of people interested in online machine learning, even given the presence of the other 9 results. The essential observation here is that the process of human learning is a big business (around 5% of GDP) effecting virtually everyone. The internet is changing this dramatically, by altering the economics of teaching. Consider two possibilities: The classroom-style teaching environment continues as is, with many teachers for the same subject. All the teachers for one subject get together, along with perhaps a factor of 2 more people who are experts in online delivery. They spend a factor of 4 more time designing
sentIndex sentText sentNum sentScore
1 If you search for “online learning” with any major search engine , it’s interesting to note that zero of the results are for online machine learning. [sent-1, score-0.476]
2 In other words, the number of people specifically interested in the least interesting top-10 online human learning result might exceed the number of people interested in online machine learning, even given the presence of the other 9 results. [sent-3, score-0.897]
3 Consider two possibilities: The classroom-style teaching environment continues as is, with many teachers for the same subject. [sent-6, score-0.419]
4 All the teachers for one subject get together, along with perhaps a factor of 2 more people who are experts in online delivery. [sent-7, score-0.505]
5 They spend a factor of 4 more time designing the perfect lecture & learning environment as verified by extensive study. [sent-8, score-0.436]
6 These two approaches have a similar economic cost, with the additional effort in the second approach being offset by the fact that it is a one-time effort rather than an annual effort. [sent-9, score-0.475]
7 I’m sure many people prefer the classroom approach, because it’s traditional, because a teacher can adjust dynamically and intelligently to the student, and because a teacher provides ancillary benefits such as day care and child abuse detection. [sent-10, score-0.582]
8 For classes commonly taught through high school, it’s difficult to imagine how good a learning experience could be after millions of hours spent refining to create the perfect approach. [sent-12, score-0.546]
9 Imagine repeating a lecture over-and-over, testing the resulting student understanding a {day, week, month, year, decade} later to such an extent that every slide, every sentence, and every exercise is optimized for excellent learning. [sent-13, score-0.662]
10 If a student can ace these tests after taking online learning classes, then there is a real sense in which colleges accepting students are satisfied by their qualifications. [sent-24, score-0.797]
11 We can also expect that testable subjects have an inherent advantage in online learning. [sent-26, score-0.567]
12 As centralized testing is a difficult market to break into, the existing systems have a substantial advantage here. [sent-27, score-0.446]
13 These include perfect replicability, essentially free distribution, and difficult economics—on one hand the approach could be vastly valuable while on the other it’s difficult to charge someone for something they can get free. [sent-30, score-0.485]
14 The economics imply that there is room for a major charity or state government to accomplish a great deal which might be difficult to accomplish in a business model. [sent-31, score-0.639]
15 There are areas of teaching which are not amenable to online instruction. [sent-33, score-0.458]
16 Similarly, many elements of our current education system are not related to formal education, but rather are about students meeting students, teachers acting as daycare for students, or simply structuring the day for learning. [sent-37, score-0.573]
17 Mechanisms achieving the same ends with online human learning systems are necessary, and the conflation of goals represented by the traditional education approach will retard (but not stop) the adoption of online learning approaches. [sent-38, score-1.163]
18 For those of us interested in online machine learning, it’s natural to question the relationship with online human learning. [sent-40, score-0.777]
19 Much of our other theory about the process of online learning may be helpful in a heuristic-motivating manner, but it appears typically too pessimistic to accurately capture what is possible. [sent-46, score-0.477]
20 A setting more suitable for student and teacher has been studied in learning theory (see the bibliography here for a link into the citation tree). [sent-50, score-0.531]
wordName wordTfidf (topN-words)
[('online', 0.28), ('teacher', 0.248), ('student', 0.221), ('teaching', 0.178), ('centralized', 0.165), ('teachers', 0.165), ('human', 0.159), ('economics', 0.144), ('students', 0.142), ('testing', 0.137), ('approach', 0.132), ('apprentice', 0.124), ('lecture', 0.124), ('education', 0.117), ('perfect', 0.114), ('expect', 0.105), ('room', 0.095), ('tests', 0.092), ('similarly', 0.088), ('difficult', 0.088), ('day', 0.086), ('process', 0.08), ('decade', 0.078), ('universities', 0.078), ('accomplish', 0.078), ('business', 0.078), ('taught', 0.078), ('major', 0.078), ('environment', 0.076), ('effort', 0.075), ('traditional', 0.071), ('subjects', 0.071), ('imagine', 0.071), ('classes', 0.07), ('statements', 0.066), ('approaches', 0.065), ('second', 0.065), ('could', 0.063), ('rather', 0.063), ('learning', 0.062), ('research', 0.061), ('factor', 0.06), ('every', 0.06), ('extreme', 0.06), ('search', 0.059), ('interested', 0.058), ('advantage', 0.056), ('accurately', 0.055), ('testable', 0.055), ('exercises', 0.055)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999958 378 hunch net-2009-11-15-The Other Online Learning
Introduction: If you search for “online learning” with any major search engine , it’s interesting to note that zero of the results are for online machine learning. This may not be a mistake if you are committed to a global ordering. In other words, the number of people specifically interested in the least interesting top-10 online human learning result might exceed the number of people interested in online machine learning, even given the presence of the other 9 results. The essential observation here is that the process of human learning is a big business (around 5% of GDP) effecting virtually everyone. The internet is changing this dramatically, by altering the economics of teaching. Consider two possibilities: The classroom-style teaching environment continues as is, with many teachers for the same subject. All the teachers for one subject get together, along with perhaps a factor of 2 more people who are experts in online delivery. They spend a factor of 4 more time designing
2 0.27435923 445 hunch net-2011-09-28-Somebody’s Eating Your Lunch
Introduction: Since we last discussed the other online learning , Stanford has very visibly started pushing mass teaching in AI , Machine Learning , and Databases . In retrospect, it’s not too surprising that the next step up in serious online teaching experiments are occurring at the computer science department of a university embedded in the land of startups. Numbers on the order of 100000 are quite significant—similar in scale to the number of computer science undergraduate students/year in the US. Although these populations surely differ, the fact that they could overlap is worth considering for the future. It’s too soon to say how successful these classes will be and there are many easy criticisms to make: Registration != Learning … but if only 1/10th complete these classes, the scale of teaching still surpasses the scale of any traditional process. 1st year excitement != nth year routine … but if only 1/10th take future classes, the scale of teaching still surpass
3 0.19450614 267 hunch net-2007-10-17-Online as the new adjective
Introduction: Online learning is in vogue, which means we should expect to see in the near future: Online boosting. Online decision trees. Online SVMs. (actually, we’ve already seen) Online deep learning. Online parallel learning. etc… There are three fundamental drivers of this trend. Increasing size of datasets makes online algorithms attractive. Online learning can simply be more efficient than batch learning. Here is a picture from a class on online learning: The point of this picture is that even in 3 dimensions and even with linear constraints, finding the minima of a set in an online fashion can be typically faster than finding the minima in a batch fashion. To see this, note that there is a minimal number of gradient updates (i.e. 2) required in order to reach the minima in the typical case. Given this, it’s best to do these updates as quickly as possible, which implies doing the first update online (i.e. before seeing all the examples) is preferred. Note
4 0.16775382 252 hunch net-2007-07-01-Watchword: Online Learning
Introduction: It turns out that many different people use the term “Online Learning”, and often they don’t have the same definition in mind. Here’s a list of the possibilities I know of. Online Information Setting Online learning refers to a problem in which unlabeled data comes, a prediction is made, and then feedback is acquired. Online Adversarial Setting Online learning refers to algorithms in the Online Information Setting which satisfy guarantees of the form: “For all possible sequences of observations, the algorithim has regret at most log ( number of strategies) with respect to the best strategy in a set.” This is sometimes called online learning with experts. Online Optimization Constraint Online learning refers to optimizing a predictor via a learning algorithm tunes parameters on a per-example basis. This may or may not be applied in the Online Information Setting, and the strategy may or may not satisfy Adversarial setting theory. Online Computational Constra
5 0.16494179 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
Introduction: How do you create an optimal environment for research? Here are some essential ingredients that I see. Stability . University-based research is relatively good at this. On any particular day, researchers face choices in what they will work on. A very common tradeoff is between: easy small difficult big For researchers without stability, the ‘easy small’ option wins. This is often “ok”—a series of incremental improvements on the state of the art can add up to something very beneficial. However, it misses one of the big potentials of research: finding entirely new and better ways of doing things. Stability comes in many forms. The prototypical example is tenure at a university—a tenured professor is almost imposssible to fire which means that the professor has the freedom to consider far horizon activities. An iron-clad guarantee of a paycheck is not necessary—industrial research labs have succeeded well with research positions of indefinite duration. Atnt rese
6 0.14096273 449 hunch net-2011-11-26-Giving Thanks
7 0.13996728 109 hunch net-2005-09-08-Online Learning as the Mathematics of Accountability
8 0.13260588 3 hunch net-2005-01-24-The Humanloop Spectrum of Machine Learning
9 0.13187221 343 hunch net-2009-02-18-Decision by Vetocracy
10 0.13155726 347 hunch net-2009-03-26-Machine Learning is too easy
11 0.12715858 478 hunch net-2013-01-07-NYU Large Scale Machine Learning Class
12 0.12544337 370 hunch net-2009-09-18-Necessary and Sufficient Research
13 0.12319142 344 hunch net-2009-02-22-Effective Research Funding
14 0.11759284 134 hunch net-2005-12-01-The Webscience Future
15 0.11618149 332 hunch net-2008-12-23-Use of Learning Theory
16 0.11187029 424 hunch net-2011-02-17-What does Watson mean?
17 0.1104573 110 hunch net-2005-09-10-“Failure” is an option
18 0.11009313 237 hunch net-2007-04-02-Contextual Scaling
19 0.10973816 288 hunch net-2008-02-10-Complexity Illness
20 0.10702712 311 hunch net-2008-07-26-Compositional Machine Learning Algorithm Design
topicId topicWeight
[(0, 0.29), (1, 0.014), (2, -0.126), (3, 0.094), (4, -0.014), (5, -0.016), (6, -0.033), (7, 0.016), (8, -0.06), (9, 0.137), (10, 0.123), (11, 0.023), (12, 0.067), (13, -0.074), (14, 0.069), (15, 0.034), (16, 0.024), (17, -0.042), (18, 0.117), (19, 0.045), (20, 0.054), (21, -0.058), (22, -0.033), (23, -0.006), (24, 0.162), (25, 0.057), (26, 0.111), (27, -0.041), (28, 0.027), (29, 0.066), (30, 0.097), (31, -0.047), (32, -0.067), (33, -0.05), (34, -0.016), (35, 0.021), (36, 0.092), (37, -0.001), (38, 0.029), (39, -0.083), (40, 0.057), (41, 0.069), (42, 0.053), (43, -0.006), (44, 0.029), (45, -0.01), (46, -0.065), (47, 0.023), (48, -0.023), (49, -0.055)]
simIndex simValue blogId blogTitle
same-blog 1 0.96185738 378 hunch net-2009-11-15-The Other Online Learning
Introduction: If you search for “online learning” with any major search engine , it’s interesting to note that zero of the results are for online machine learning. This may not be a mistake if you are committed to a global ordering. In other words, the number of people specifically interested in the least interesting top-10 online human learning result might exceed the number of people interested in online machine learning, even given the presence of the other 9 results. The essential observation here is that the process of human learning is a big business (around 5% of GDP) effecting virtually everyone. The internet is changing this dramatically, by altering the economics of teaching. Consider two possibilities: The classroom-style teaching environment continues as is, with many teachers for the same subject. All the teachers for one subject get together, along with perhaps a factor of 2 more people who are experts in online delivery. They spend a factor of 4 more time designing
2 0.88167447 445 hunch net-2011-09-28-Somebody’s Eating Your Lunch
Introduction: Since we last discussed the other online learning , Stanford has very visibly started pushing mass teaching in AI , Machine Learning , and Databases . In retrospect, it’s not too surprising that the next step up in serious online teaching experiments are occurring at the computer science department of a university embedded in the land of startups. Numbers on the order of 100000 are quite significant—similar in scale to the number of computer science undergraduate students/year in the US. Although these populations surely differ, the fact that they could overlap is worth considering for the future. It’s too soon to say how successful these classes will be and there are many easy criticisms to make: Registration != Learning … but if only 1/10th complete these classes, the scale of teaching still surpasses the scale of any traditional process. 1st year excitement != nth year routine … but if only 1/10th take future classes, the scale of teaching still surpass
3 0.74088377 267 hunch net-2007-10-17-Online as the new adjective
Introduction: Online learning is in vogue, which means we should expect to see in the near future: Online boosting. Online decision trees. Online SVMs. (actually, we’ve already seen) Online deep learning. Online parallel learning. etc… There are three fundamental drivers of this trend. Increasing size of datasets makes online algorithms attractive. Online learning can simply be more efficient than batch learning. Here is a picture from a class on online learning: The point of this picture is that even in 3 dimensions and even with linear constraints, finding the minima of a set in an online fashion can be typically faster than finding the minima in a batch fashion. To see this, note that there is a minimal number of gradient updates (i.e. 2) required in order to reach the minima in the typical case. Given this, it’s best to do these updates as quickly as possible, which implies doing the first update online (i.e. before seeing all the examples) is preferred. Note
4 0.707192 252 hunch net-2007-07-01-Watchword: Online Learning
Introduction: It turns out that many different people use the term “Online Learning”, and often they don’t have the same definition in mind. Here’s a list of the possibilities I know of. Online Information Setting Online learning refers to a problem in which unlabeled data comes, a prediction is made, and then feedback is acquired. Online Adversarial Setting Online learning refers to algorithms in the Online Information Setting which satisfy guarantees of the form: “For all possible sequences of observations, the algorithim has regret at most log ( number of strategies) with respect to the best strategy in a set.” This is sometimes called online learning with experts. Online Optimization Constraint Online learning refers to optimizing a predictor via a learning algorithm tunes parameters on a per-example basis. This may or may not be applied in the Online Information Setting, and the strategy may or may not satisfy Adversarial setting theory. Online Computational Constra
5 0.67303973 109 hunch net-2005-09-08-Online Learning as the Mathematics of Accountability
Introduction: Accountability is a social problem. When someone screws up, do you fire them? Or do you accept the error and let them continue? This is a very difficult problem and we all know of stories where the wrong decision was made. Online learning (as meant here), is a subfield of learning theory which analyzes the online learning model. In the online learning model, there are a set of hypotheses or “experts”. On any instantance x , each expert makes a prediction y . A master algorithm A uses these predictions to form it’s own prediction y A and then learns the correct prediction y * . This process repeats. The goal of online learning is to find a master algorithm A which uses the advice of the experts to make good predictions. In particular, we typically want to guarantee that the master algorithm performs almost as well as the best expert. If L(e) is the loss of expert e and L(A) is the loss of the master algorithm, it is often possible to prove: L(A) les
6 0.67008317 478 hunch net-2013-01-07-NYU Large Scale Machine Learning Class
7 0.64070946 397 hunch net-2010-05-02-What’s the difference between gambling and rewarding good prediction?
8 0.61004692 493 hunch net-2014-02-16-Metacademy: a package manager for knowledge
9 0.60538918 373 hunch net-2009-10-03-Static vs. Dynamic multiclass prediction
10 0.60028708 134 hunch net-2005-12-01-The Webscience Future
11 0.59025091 385 hunch net-2009-12-27-Interesting things at NIPS 2009
12 0.58396786 449 hunch net-2011-11-26-Giving Thanks
13 0.58362192 28 hunch net-2005-02-25-Problem: Online Learning
14 0.58273715 69 hunch net-2005-05-11-Visa Casualties
15 0.57722205 73 hunch net-2005-05-17-A Short Guide to PhD Graduate Study
16 0.56974691 208 hunch net-2006-09-18-What is missing for online collaborative research?
17 0.56642109 448 hunch net-2011-10-24-2011 ML symposium and the bears
18 0.56233805 8 hunch net-2005-02-01-NIPS: Online Bayes
19 0.54186386 370 hunch net-2009-09-18-Necessary and Sufficient Research
20 0.53805721 290 hunch net-2008-02-27-The Stats Handicap
topicId topicWeight
[(0, 0.026), (3, 0.03), (21, 0.2), (27, 0.253), (38, 0.031), (42, 0.015), (48, 0.014), (53, 0.036), (55, 0.109), (56, 0.014), (64, 0.023), (67, 0.018), (84, 0.011), (94, 0.056), (95, 0.067)]
simIndex simValue blogId blogTitle
1 0.96834689 58 hunch net-2005-04-21-Dynamic Programming Generalizations and Their Use
Introduction: David Mcallester gave a talk about this paper (with Pedro Felzenszwalb ). I’ll try to give a high level summary of why it’s interesting. Dynamic programming is most familiar as instantiated by Viterbi decoding in a hidden markov model. It is a general paradigm for problem solving where subproblems are solved and used to solve larger problems. In the Viterbi decoding example, the subproblem is “What is the most probable path ending at each state at timestep t ?”, and the larger problem is the same except at timestep t+1 . There are a few optimizations you can do here: Dynamic Programming -> queued Dynamic Programming . Keep track of the “cost so far” (or “most probable path”) and (carefully) only look at extensions to paths likely to yield the shortest path. “Carefully” here is defined by Dijkstra’s shortest path algorithm . queued Dynamic programming -> A * Add a lower bound on the cost to complete a path (or an upper bound on the probability of a completion) for
2 0.96140522 369 hunch net-2009-08-27-New York Area Machine Learning Events
Introduction: Several events are happening in the NY area. Barriers in Computational Learning Theory Workshop, Aug 28. That’s tomorrow near Princeton. I’m looking forward to speaking at this one on “Getting around Barriers in Learning Theory”, but several other talks are of interest, particularly to the CS theory inclined. Claudia Perlich is running the INFORMS Data Mining Contest with a deadline of Sept. 25. This is a contest using real health record data (they partnered with HealthCare Intelligence ) to predict transfers and mortality. In the current US health care reform debate, the case studies of high costs we hear strongly suggest machine learning & statistics can save many billions. The Singularity Summit October 3&4 . This is for the AIists out there. Several of the talks look interesting, although unfortunately I’ll miss it for ALT . Predictive Analytics World, Oct 20-21 . This is stretching the definition of “New York Area” a bit, but the train to DC is reasonable.
3 0.95212698 126 hunch net-2005-10-26-Fallback Analysis is a Secret to Useful Algorithms
Introduction: The ideal of theoretical algorithm analysis is to construct an algorithm with accompanying optimality theorems proving that it is a useful algorithm. This ideal often fails, particularly for learning algorithms and theory. The general form of a theorem is: If preconditions Then postconditions When we design learning algorithms it is very common to come up with precondition assumptions such as “the data is IID”, “the learning problem is drawn from a known distribution over learning problems”, or “there is a perfect classifier”. All of these example preconditions can be false for real-world problems in ways that are not easily detectable. This means that algorithms derived and justified by these very common forms of analysis may be prone to catastrophic failure in routine (mis)application. We can hope for better. Several different kinds of learning algorithm analysis have been developed some of which have fewer preconditions. Simply demanding that these forms of analysi
same-blog 4 0.90491736 378 hunch net-2009-11-15-The Other Online Learning
Introduction: If you search for “online learning” with any major search engine , it’s interesting to note that zero of the results are for online machine learning. This may not be a mistake if you are committed to a global ordering. In other words, the number of people specifically interested in the least interesting top-10 online human learning result might exceed the number of people interested in online machine learning, even given the presence of the other 9 results. The essential observation here is that the process of human learning is a big business (around 5% of GDP) effecting virtually everyone. The internet is changing this dramatically, by altering the economics of teaching. Consider two possibilities: The classroom-style teaching environment continues as is, with many teachers for the same subject. All the teachers for one subject get together, along with perhaps a factor of 2 more people who are experts in online delivery. They spend a factor of 4 more time designing
5 0.90000397 267 hunch net-2007-10-17-Online as the new adjective
Introduction: Online learning is in vogue, which means we should expect to see in the near future: Online boosting. Online decision trees. Online SVMs. (actually, we’ve already seen) Online deep learning. Online parallel learning. etc… There are three fundamental drivers of this trend. Increasing size of datasets makes online algorithms attractive. Online learning can simply be more efficient than batch learning. Here is a picture from a class on online learning: The point of this picture is that even in 3 dimensions and even with linear constraints, finding the minima of a set in an online fashion can be typically faster than finding the minima in a batch fashion. To see this, note that there is a minimal number of gradient updates (i.e. 2) required in order to reach the minima in the typical case. Given this, it’s best to do these updates as quickly as possible, which implies doing the first update online (i.e. before seeing all the examples) is preferred. Note
6 0.89346403 3 hunch net-2005-01-24-The Humanloop Spectrum of Machine Learning
7 0.82950699 194 hunch net-2006-07-11-New Models
8 0.81171858 343 hunch net-2009-02-18-Decision by Vetocracy
9 0.80594856 360 hunch net-2009-06-15-In Active Learning, the question changes
10 0.8059113 309 hunch net-2008-07-10-Interesting papers, ICML 2008
11 0.80534917 220 hunch net-2006-11-27-Continuizing Solutions
12 0.80331731 379 hunch net-2009-11-23-ICML 2009 Workshops (and Tutorials)
13 0.7997691 225 hunch net-2007-01-02-Retrospective
14 0.79670465 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
15 0.79637516 406 hunch net-2010-08-22-KDD 2010
16 0.79433614 51 hunch net-2005-04-01-The Producer-Consumer Model of Research
17 0.79364318 230 hunch net-2007-02-02-Thoughts regarding “Is machine learning different from statistics?”
18 0.79312992 359 hunch net-2009-06-03-Functionally defined Nonlinear Dynamic Models
19 0.79263383 320 hunch net-2008-10-14-Who is Responsible for a Bad Review?
20 0.79052222 36 hunch net-2005-03-05-Funding Research