hunch_net hunch_net-2011 hunch_net-2011-445 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Since we last discussed the other online learning , Stanford has very visibly started pushing mass teaching in AI , Machine Learning , and Databases . In retrospect, it’s not too surprising that the next step up in serious online teaching experiments are occurring at the computer science department of a university embedded in the land of startups. Numbers on the order of 100000 are quite significant—similar in scale to the number of computer science undergraduate students/year in the US. Although these populations surely differ, the fact that they could overlap is worth considering for the future. It’s too soon to say how successful these classes will be and there are many easy criticisms to make: Registration != Learning … but if only 1/10th complete these classes, the scale of teaching still surpasses the scale of any traditional process. 1st year excitement != nth year routine … but if only 1/10th take future classes, the scale of teaching still surpass
sentIndex sentText sentNum sentScore
1 Since we last discussed the other online learning , Stanford has very visibly started pushing mass teaching in AI , Machine Learning , and Databases . [sent-1, score-0.766]
2 In retrospect, it’s not too surprising that the next step up in serious online teaching experiments are occurring at the computer science department of a university embedded in the land of startups. [sent-2, score-1.07]
3 Numbers on the order of 100000 are quite significant—similar in scale to the number of computer science undergraduate students/year in the US. [sent-3, score-0.333]
4 It’s too soon to say how successful these classes will be and there are many easy criticisms to make: Registration ! [sent-5, score-0.513]
5 = Learning … but if only 1/10th complete these classes, the scale of teaching still surpasses the scale of any traditional process. [sent-6, score-1.261]
6 = nth year routine … but if only 1/10th take future classes, the scale of teaching still surpasses the scale of any traditional process. [sent-8, score-1.261]
7 Hello, cheating … but teaching is much harder than testing in general, and we already have recognized systems for mass testing. [sent-9, score-0.817]
8 Online misses out … sure, but for students not enrolled in a high quality university program, this is simply not a relevant comparison. [sent-10, score-0.423]
9 Anecdotally, at Caltech , they let us take two classes at the same time, which I did a few times. [sent-12, score-0.283]
10 And, if you first wait until it’s clear how to make money, you won’t make any. [sent-17, score-0.329]
11 The prospect of teaching 1 student means you might review some notes. [sent-20, score-0.784]
12 The prospect of teaching ~10 students means you prepare some slides. [sent-21, score-1.044]
13 The prospect of teaching ~100 students means you polish your slides well, trying to anticipate questions, and hopefully drawing on experience from previous presentations. [sent-22, score-1.044]
14 I’ve never directly taught ~1000 students, but at that scale you must try very hard to make the presentation perfect, including serious testing with dry runs. [sent-23, score-0.539]
15 10 5 students must make getting out of bed in the morning quite easy. [sent-24, score-0.392]
16 Stanford has a significant first-mover advantage amongst top research universities, but it’s easy to imagine a few other (but not many) universities operating at a similar scale. [sent-25, score-0.415]
17 Those that have the foresight to start a serious online teaching program soon will have a chance of being among the few. [sent-26, score-0.876]
18 For other research universities, we can expect boutique traditional classes to continue for some time. [sent-27, score-0.594]
19 These boutique classes may have some significant social value, because it’s easy to imagine that the few megaclasses miss important things in developing research areas. [sent-28, score-0.759]
20 And for everyone working at teaching universities, someone is eating your lunch. [sent-29, score-0.481]
wordName wordTfidf (topN-words)
[('teaching', 0.481), ('classes', 0.283), ('prospect', 0.223), ('universities', 0.211), ('scale', 0.204), ('students', 0.191), ('boutique', 0.167), ('surpasses', 0.167), ('traditional', 0.144), ('stanford', 0.138), ('make', 0.132), ('online', 0.118), ('serious', 0.11), ('miss', 0.105), ('mass', 0.105), ('university', 0.098), ('soon', 0.098), ('testing', 0.093), ('money', 0.091), ('means', 0.08), ('embedded', 0.074), ('populations', 0.074), ('excitement', 0.074), ('retrospect', 0.074), ('enrolled', 0.074), ('skipped', 0.074), ('significant', 0.073), ('recognized', 0.069), ('anticipate', 0.069), ('pan', 0.069), ('cheating', 0.069), ('databases', 0.069), ('morning', 0.069), ('among', 0.069), ('prepare', 0.069), ('motivates', 0.069), ('decent', 0.069), ('easy', 0.067), ('computer', 0.066), ('wait', 0.065), ('anecdotally', 0.065), ('criticisms', 0.065), ('imagine', 0.064), ('science', 0.063), ('grade', 0.062), ('pushing', 0.062), ('still', 0.061), ('misses', 0.06), ('inevitable', 0.06), ('land', 0.06)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000005 445 hunch net-2011-09-28-Somebody’s Eating Your Lunch
Introduction: Since we last discussed the other online learning , Stanford has very visibly started pushing mass teaching in AI , Machine Learning , and Databases . In retrospect, it’s not too surprising that the next step up in serious online teaching experiments are occurring at the computer science department of a university embedded in the land of startups. Numbers on the order of 100000 are quite significant—similar in scale to the number of computer science undergraduate students/year in the US. Although these populations surely differ, the fact that they could overlap is worth considering for the future. It’s too soon to say how successful these classes will be and there are many easy criticisms to make: Registration != Learning … but if only 1/10th complete these classes, the scale of teaching still surpasses the scale of any traditional process. 1st year excitement != nth year routine … but if only 1/10th take future classes, the scale of teaching still surpass
2 0.27435923 378 hunch net-2009-11-15-The Other Online Learning
Introduction: If you search for “online learning” with any major search engine , it’s interesting to note that zero of the results are for online machine learning. This may not be a mistake if you are committed to a global ordering. In other words, the number of people specifically interested in the least interesting top-10 online human learning result might exceed the number of people interested in online machine learning, even given the presence of the other 9 results. The essential observation here is that the process of human learning is a big business (around 5% of GDP) effecting virtually everyone. The internet is changing this dramatically, by altering the economics of teaching. Consider two possibilities: The classroom-style teaching environment continues as is, with many teachers for the same subject. All the teachers for one subject get together, along with perhaps a factor of 2 more people who are experts in online delivery. They spend a factor of 4 more time designing
3 0.17956743 478 hunch net-2013-01-07-NYU Large Scale Machine Learning Class
Introduction: Yann LeCun and I are coteaching a class on Large Scale Machine Learning starting late January at NYU . This class will cover many tricks to get machine learning working well on datasets with many features, examples, and classes, along with several elements of deep learning and support systems enabling the previous. This is not a beginning class—you really need to have taken a basic machine learning class previously to follow along. Students will be able to run and experiment with large scale learning algorithms since Yahoo! has donated servers which are being configured into a small scale Hadoop cluster. We are planning to cover the frontier of research in scalable learning algorithms, so good class projects could easily lead to papers. For me, this is a chance to teach on many topics of past research. In general, it seems like researchers should engage in at least occasional teaching of research, both as a proof of teachability and to see their own research through th
4 0.16364135 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
Introduction: How do you create an optimal environment for research? Here are some essential ingredients that I see. Stability . University-based research is relatively good at this. On any particular day, researchers face choices in what they will work on. A very common tradeoff is between: easy small difficult big For researchers without stability, the ‘easy small’ option wins. This is often “ok”—a series of incremental improvements on the state of the art can add up to something very beneficial. However, it misses one of the big potentials of research: finding entirely new and better ways of doing things. Stability comes in many forms. The prototypical example is tenure at a university—a tenured professor is almost imposssible to fire which means that the professor has the freedom to consider far horizon activities. An iron-clad guarantee of a paycheck is not necessary—industrial research labs have succeeded well with research positions of indefinite duration. Atnt rese
5 0.15155545 344 hunch net-2009-02-22-Effective Research Funding
Introduction: With a worldwide recession on, my impression is that the carnage in research has not been as severe as might be feared, at least in the United States. I know of two notable negative impacts: It’s quite difficult to get a job this year, as many companies and universities simply aren’t hiring. This is particularly tough on graduating students. Perhaps 10% of IBM research was fired. In contrast, around the time of the dot com bust, ATnT Research and Lucent had one or several 50% size firings wiping out much of the remainder of Bell Labs , triggering a notable diaspora for the respected machine learning group there. As the recession progresses, we may easily see more firings as companies in particular reach a point where they can no longer support research. There are a couple positives to the recession as well. Both the implosion of Wall Street (which siphoned off smart people) and the general difficulty of getting a job coming out of an undergraduate education s
6 0.1476627 373 hunch net-2009-10-03-Static vs. Dynamic multiclass prediction
7 0.13158281 228 hunch net-2007-01-15-The Machine Learning Department
8 0.12463093 271 hunch net-2007-11-05-CMU wins DARPA Urban Challenge
9 0.11749064 288 hunch net-2008-02-10-Complexity Illness
10 0.11742616 22 hunch net-2005-02-18-What it means to do research.
11 0.10708859 110 hunch net-2005-09-10-“Failure” is an option
12 0.10341548 75 hunch net-2005-05-28-Running A Machine Learning Summer School
13 0.10320009 286 hunch net-2008-01-25-Turing’s Club for Machine Learning
14 0.1019363 449 hunch net-2011-11-26-Giving Thanks
15 0.096487328 290 hunch net-2008-02-27-The Stats Handicap
16 0.094470002 493 hunch net-2014-02-16-Metacademy: a package manager for knowledge
17 0.094163254 267 hunch net-2007-10-17-Online as the new adjective
18 0.086584538 73 hunch net-2005-05-17-A Short Guide to PhD Graduate Study
19 0.086280644 36 hunch net-2005-03-05-Funding Research
20 0.083657153 119 hunch net-2005-10-08-We have a winner
topicId topicWeight
[(0, 0.198), (1, -0.041), (2, -0.114), (3, 0.103), (4, -0.065), (5, -0.014), (6, -0.021), (7, 0.052), (8, -0.095), (9, 0.084), (10, 0.095), (11, 0.023), (12, 0.027), (13, -0.118), (14, 0.063), (15, -0.022), (16, 0.044), (17, 0.005), (18, 0.055), (19, 0.109), (20, 0.041), (21, 0.016), (22, 0.016), (23, -0.01), (24, 0.148), (25, -0.043), (26, 0.11), (27, -0.038), (28, 0.011), (29, 0.123), (30, 0.056), (31, -0.105), (32, -0.049), (33, -0.004), (34, -0.042), (35, -0.015), (36, 0.098), (37, -0.006), (38, 0.061), (39, -0.038), (40, 0.098), (41, 0.09), (42, 0.049), (43, -0.013), (44, 0.057), (45, -0.003), (46, 0.005), (47, -0.039), (48, 0.027), (49, -0.016)]
simIndex simValue blogId blogTitle
same-blog 1 0.97905421 445 hunch net-2011-09-28-Somebody’s Eating Your Lunch
Introduction: Since we last discussed the other online learning , Stanford has very visibly started pushing mass teaching in AI , Machine Learning , and Databases . In retrospect, it’s not too surprising that the next step up in serious online teaching experiments are occurring at the computer science department of a university embedded in the land of startups. Numbers on the order of 100000 are quite significant—similar in scale to the number of computer science undergraduate students/year in the US. Although these populations surely differ, the fact that they could overlap is worth considering for the future. It’s too soon to say how successful these classes will be and there are many easy criticisms to make: Registration != Learning … but if only 1/10th complete these classes, the scale of teaching still surpasses the scale of any traditional process. 1st year excitement != nth year routine … but if only 1/10th take future classes, the scale of teaching still surpass
2 0.76198041 378 hunch net-2009-11-15-The Other Online Learning
Introduction: If you search for “online learning” with any major search engine , it’s interesting to note that zero of the results are for online machine learning. This may not be a mistake if you are committed to a global ordering. In other words, the number of people specifically interested in the least interesting top-10 online human learning result might exceed the number of people interested in online machine learning, even given the presence of the other 9 results. The essential observation here is that the process of human learning is a big business (around 5% of GDP) effecting virtually everyone. The internet is changing this dramatically, by altering the economics of teaching. Consider two possibilities: The classroom-style teaching environment continues as is, with many teachers for the same subject. All the teachers for one subject get together, along with perhaps a factor of 2 more people who are experts in online delivery. They spend a factor of 4 more time designing
3 0.66322649 478 hunch net-2013-01-07-NYU Large Scale Machine Learning Class
Introduction: Yann LeCun and I are coteaching a class on Large Scale Machine Learning starting late January at NYU . This class will cover many tricks to get machine learning working well on datasets with many features, examples, and classes, along with several elements of deep learning and support systems enabling the previous. This is not a beginning class—you really need to have taken a basic machine learning class previously to follow along. Students will be able to run and experiment with large scale learning algorithms since Yahoo! has donated servers which are being configured into a small scale Hadoop cluster. We are planning to cover the frontier of research in scalable learning algorithms, so good class projects could easily lead to papers. For me, this is a chance to teach on many topics of past research. In general, it seems like researchers should engage in at least occasional teaching of research, both as a proof of teachability and to see their own research through th
4 0.65230793 449 hunch net-2011-11-26-Giving Thanks
Introduction: Thanksgiving is perhaps my favorite holiday, because pausing your life and giving thanks provides a needed moment of perspective. As a researcher, I am most thankful for my education, without which I could not function. I want to share this, because it provides some sense of how a researcher starts. My long term memory seems to function particularly well, which makes any education I get is particularly useful. I am naturally obsessive, which makes me chase down details until I fully understand things. Natural obsessiveness can go wrong, of course, but it’s a great ally when you absolutely must get things right. My childhood was all in one hometown, which was a conscious sacrifice on the part of my father, implying disruptions from moving around were eliminated. I’m not sure how important this was since travel has it’s own benefits, but it bears thought. I had several great teachers in grade school, and naturally gravitated towards teachers over classmates, as they seemed
5 0.60388309 290 hunch net-2008-02-27-The Stats Handicap
Introduction: Graduating students in Statistics appear to be at a substantial handicap compared to graduating students in Machine Learning, despite being in substantially overlapping subjects. The problem seems to be cultural. Statistics comes from a mathematics background which emphasizes large publications slowly published under review at journals. Machine Learning comes from a Computer Science background which emphasizes quick publishing at reviewed conferences. This has a number of implications: Graduating statistics PhDs often have 0-2 publications while graduating machine learning PhDs might have 5-15. Graduating ML students have had a chance for others to build on their work. Stats students have had no such chance. Graduating ML students have attended a number of conferences and presented their work, giving them a chance to meet people. Stats students have had fewer chances of this sort. In short, Stats students have had relatively few chances to distinguish themselves and
6 0.60170633 73 hunch net-2005-05-17-A Short Guide to PhD Graduate Study
7 0.55979961 493 hunch net-2014-02-16-Metacademy: a package manager for knowledge
8 0.55662018 69 hunch net-2005-05-11-Visa Casualties
9 0.55135596 75 hunch net-2005-05-28-Running A Machine Learning Summer School
10 0.53215414 110 hunch net-2005-09-10-“Failure” is an option
11 0.53138942 448 hunch net-2011-10-24-2011 ML symposium and the bears
12 0.50547576 397 hunch net-2010-05-02-What’s the difference between gambling and rewarding good prediction?
13 0.5052132 373 hunch net-2009-10-03-Static vs. Dynamic multiclass prediction
14 0.50043398 344 hunch net-2009-02-22-Effective Research Funding
15 0.48694605 267 hunch net-2007-10-17-Online as the new adjective
16 0.48157555 424 hunch net-2011-02-17-What does Watson mean?
17 0.48069328 414 hunch net-2010-10-17-Partha Niyogi has died
18 0.47841245 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
19 0.46894017 76 hunch net-2005-05-29-Bad ideas
20 0.46851265 134 hunch net-2005-12-01-The Webscience Future
topicId topicWeight
[(27, 0.184), (38, 0.045), (48, 0.284), (53, 0.054), (55, 0.089), (64, 0.011), (94, 0.114), (95, 0.124)]
simIndex simValue blogId blogTitle
1 0.92345965 468 hunch net-2012-06-29-ICML survey and comments
Introduction: Just about nothing could keep me from attending ICML , except for Dora who arrived on Monday. Consequently, I have only secondhand reports that the conference is going well. For those who are remote (like me) or after the conference (like everyone), Mark Reid has setup the ICML discussion site where you can comment on any paper or subscribe to papers. Authors are automatically subscribed to their own papers, so it should be possible to have a discussion significantly after the fact, as people desire. We also conducted a survey before the conference and have the survey results now. This can be compared with the ICML 2010 survey results . Looking at the comparable questions, we can sometimes order the answers to have scores ranging from 0 to 3 or 0 to 4 with 3 or 4 being best and 0 worst, then compute the average difference between 2012 and 2010. Glancing through them, I see: Most people found the papers they reviewed a good fit for their expertise (-.037 w.r.t 20
same-blog 2 0.90289056 445 hunch net-2011-09-28-Somebody’s Eating Your Lunch
Introduction: Since we last discussed the other online learning , Stanford has very visibly started pushing mass teaching in AI , Machine Learning , and Databases . In retrospect, it’s not too surprising that the next step up in serious online teaching experiments are occurring at the computer science department of a university embedded in the land of startups. Numbers on the order of 100000 are quite significant—similar in scale to the number of computer science undergraduate students/year in the US. Although these populations surely differ, the fact that they could overlap is worth considering for the future. It’s too soon to say how successful these classes will be and there are many easy criticisms to make: Registration != Learning … but if only 1/10th complete these classes, the scale of teaching still surpasses the scale of any traditional process. 1st year excitement != nth year routine … but if only 1/10th take future classes, the scale of teaching still surpass
3 0.85570771 232 hunch net-2007-02-11-24
Introduction: To commemorate the Twenty Fourth Annual International Conference on Machine Learning (ICML-07), the FOX Network has decided to launch a new spin-off series in prime time. Through unofficial sources, I have obtained the story arc for the first season, which appears frighteningly realistic.
4 0.85156727 303 hunch net-2008-06-09-The Minimum Sample Complexity of Importance Weighting
Introduction: This post is about a trick that I learned from Dale Schuurmans which has been repeatedly useful for me over time. The basic trick has to do with importance weighting for monte carlo integration. Consider the problem of finding: N = E x ~ D f(x) given samples from D and knowledge of f . Often, we don’t have samples from D available. Instead, we must make do with samples from some other distribution Q . In that case, we can still often solve the problem, as long as Q(x) isn’t 0 when D(x) is nonzero, using the importance weighting formula: E x ~ Q f(x) D(x)/Q(x) A basic question is: How many samples from Q are required in order to estimate N to some precision? In general the convergence rate is not bounded, because f(x) D(x)/Q(x) is not bounded given the assumptions. Nevertheless, there is one special value Q(x) = f(x) D(x) / N where the sample complexity turns out to be 1 , which is typically substantially better than the sample complexity of the orig
5 0.84671998 318 hunch net-2008-09-26-The SODA Program Committee
Introduction: Claire asked me to be on the SODA program committee this year, which was quite a bit of work. I had a relatively light load—merely 49 theory papers. Many of these papers were not on subjects that I was expert about, so (as is common for theory conferences) I found various reviewers that I trusted to help review the papers. I ended up reviewing about 1/3 personally. There were a couple instances where I ended up overruling a subreviewer whose logic seemed off, but otherwise I generally let their reviews stand. There are some differences in standards for paper reviews between the machine learning and theory communities. In machine learning it is expected that a review be detailed, while in the theory community this is often not the case. Every paper given to me ended up with a review varying between somewhat and very detailed. I’m sure not every author was happy with the outcome. While we did our best to make good decisions, they were difficult decisions to make. For exam
6 0.79219741 46 hunch net-2005-03-24-The Role of Workshops
7 0.77556562 327 hunch net-2008-11-16-Observations on Linearity for Reductions to Regression
8 0.71530652 466 hunch net-2012-06-05-ICML acceptance statistics
9 0.68667477 105 hunch net-2005-08-23-(Dis)similarities between academia and open source programmers
10 0.68517655 75 hunch net-2005-05-28-Running A Machine Learning Summer School
11 0.68243951 449 hunch net-2011-11-26-Giving Thanks
12 0.67853892 464 hunch net-2012-05-03-Microsoft Research, New York City
13 0.66964507 403 hunch net-2010-07-18-ICML & COLT 2010
14 0.66422367 456 hunch net-2012-02-24-ICML+50%
15 0.66261768 371 hunch net-2009-09-21-Netflix finishes (and starts)
16 0.66185409 344 hunch net-2009-02-22-Effective Research Funding
17 0.6611644 359 hunch net-2009-06-03-Functionally defined Nonlinear Dynamic Models
18 0.66001457 57 hunch net-2005-04-16-Which Assumptions are Reasonable?
19 0.65696132 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
20 0.65576112 343 hunch net-2009-02-18-Decision by Vetocracy