hunch_net hunch_net-2011 hunch_net-2011-449 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Thanksgiving is perhaps my favorite holiday, because pausing your life and giving thanks provides a needed moment of perspective. As a researcher, I am most thankful for my education, without which I could not function. I want to share this, because it provides some sense of how a researcher starts. My long term memory seems to function particularly well, which makes any education I get is particularly useful. I am naturally obsessive, which makes me chase down details until I fully understand things. Natural obsessiveness can go wrong, of course, but it’s a great ally when you absolutely must get things right. My childhood was all in one hometown, which was a conscious sacrifice on the part of my father, implying disruptions from moving around were eliminated. I’m not sure how important this was since travel has it’s own benefits, but it bears thought. I had several great teachers in grade school, and naturally gravitated towards teachers over classmates, as they seemed
sentIndex sentText sentNum sentScore
1 Thanksgiving is perhaps my favorite holiday, because pausing your life and giving thanks provides a needed moment of perspective. [sent-1, score-0.217]
2 As a researcher, I am most thankful for my education, without which I could not function. [sent-2, score-0.168]
3 My long term memory seems to function particularly well, which makes any education I get is particularly useful. [sent-4, score-0.427]
4 I am naturally obsessive, which makes me chase down details until I fully understand things. [sent-5, score-0.181]
5 I had several great teachers in grade school, and naturally gravitated towards teachers over classmates, as they seemed more interesting. [sent-9, score-0.39]
6 The frustration of not getting to the ending drove me into reading books on my own, including just about every science fiction book in Lebanon Oregon. [sent-12, score-0.375]
7 It’s great motivation to not do that sort of thing for a living. [sent-14, score-0.179]
8 Lebanon school district was willing to bend the rules for me, so I could skip unnecessary math classes. [sent-15, score-0.487]
9 I ended up a year advanced, taking math from our local community college during senior year in high school. [sent-16, score-0.345]
10 College applications was a very nervous time, because high quality colleges cost much more than we could reasonably expect to pay. [sent-17, score-0.344]
11 ; Between a few scholarships and plentiful summer research opportunities, I managed to graduate debt free. [sent-22, score-0.339]
12 Caltech was also an exceptional place to study, because rules like “no taking two classes at the same time” were never enforced them. [sent-23, score-0.186]
13 At the time, I knew I wanted to do research in some sort of ML/AI subject area, but not really what, so the breadth of possibilities at CMU was excellent. [sent-27, score-0.181]
14 In graduate school, your advisor is much more important, and between Avrim and Sebastian , I learned quite a bit. [sent-28, score-0.152]
15 The funding which made this all work out was mostly hidden from me at CMU, but there was surely a strong dependence on NSF and DARPA . [sent-29, score-0.188]
16 Figuring out what to do next was again a nervous time, but it did work out, first in a summer postdoc with Michael Kearns , then at IBM research as a Herman Goldstine Fellow, then at TTI-Chicago , and now at Yahoo! [sent-31, score-0.441]
17 But trying to abstract the details, it seems like the critical elements for success are a good memory, an interest in getting the details right, motivation, and huge amounts of time to learn and then to do research. [sent-34, score-0.475]
18 Given that many of the steps in this process winnow out large fractions of people, some amount of determination and sheer luck is involved. [sent-35, score-0.228]
19 But mostly I’d like to give thanks for the “huge amounts of time” which in practical terms translates into access to other smart people and funding. [sent-37, score-0.313]
20 In education and research funding is something like oxygen—you really miss it when it’s not there, so Thanksgiving is a good time to remember it. [sent-38, score-0.573]
wordName wordTfidf (topN-words)
[('school', 0.188), ('caltech', 0.188), ('lebanon', 0.176), ('nervous', 0.176), ('siebel', 0.176), ('thanksgiving', 0.176), ('education', 0.166), ('teachers', 0.156), ('college', 0.156), ('graduate', 0.152), ('avrim', 0.117), ('thanks', 0.114), ('memory', 0.111), ('cmu', 0.111), ('time', 0.11), ('amounts', 0.108), ('researcher', 0.106), ('math', 0.106), ('life', 0.103), ('rules', 0.103), ('research', 0.103), ('details', 0.103), ('motivation', 0.101), ('funding', 0.097), ('remember', 0.097), ('mostly', 0.091), ('could', 0.09), ('summer', 0.084), ('taking', 0.083), ('huge', 0.079), ('ending', 0.078), ('fractions', 0.078), ('tuition', 0.078), ('determination', 0.078), ('carnegie', 0.078), ('mellon', 0.078), ('conscious', 0.078), ('frustration', 0.078), ('colleges', 0.078), ('thankful', 0.078), ('goldstine', 0.078), ('package', 0.078), ('postdoc', 0.078), ('naturally', 0.078), ('sort', 0.078), ('particularly', 0.075), ('getting', 0.075), ('books', 0.072), ('fiction', 0.072), ('luck', 0.072)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999923 449 hunch net-2011-11-26-Giving Thanks
Introduction: Thanksgiving is perhaps my favorite holiday, because pausing your life and giving thanks provides a needed moment of perspective. As a researcher, I am most thankful for my education, without which I could not function. I want to share this, because it provides some sense of how a researcher starts. My long term memory seems to function particularly well, which makes any education I get is particularly useful. I am naturally obsessive, which makes me chase down details until I fully understand things. Natural obsessiveness can go wrong, of course, but it’s a great ally when you absolutely must get things right. My childhood was all in one hometown, which was a conscious sacrifice on the part of my father, implying disruptions from moving around were eliminated. I’m not sure how important this was since travel has it’s own benefits, but it bears thought. I had several great teachers in grade school, and naturally gravitated towards teachers over classmates, as they seemed
2 0.15228739 36 hunch net-2005-03-05-Funding Research
Introduction: The funding of research (and machine learning research) is an issue which seems to have become more significant in the United States over the last decade. The word “research” is applied broadly here to science, mathematics, and engineering. There are two essential difficulties with funding research: Longshot Paying a researcher is often a big gamble. Most research projects don’t pan out, but a few big payoffs can make it all worthwhile. Information Only Much of research is about finding the right way to think about or do something. The Longshot difficulty means that there is high variance in payoffs. This can be compensated for by funding many different research projects, reducing variance. The Information-Only difficulty means that it’s hard to extract a profit directly from many types of research, so companies have difficulty justifying basic research. (Patents are a mechanism for doing this. They are often extraordinarily clumsy or simply not applicable.) T
3 0.14096273 378 hunch net-2009-11-15-The Other Online Learning
Introduction: If you search for “online learning” with any major search engine , it’s interesting to note that zero of the results are for online machine learning. This may not be a mistake if you are committed to a global ordering. In other words, the number of people specifically interested in the least interesting top-10 online human learning result might exceed the number of people interested in online machine learning, even given the presence of the other 9 results. The essential observation here is that the process of human learning is a big business (around 5% of GDP) effecting virtually everyone. The internet is changing this dramatically, by altering the economics of teaching. Consider two possibilities: The classroom-style teaching environment continues as is, with many teachers for the same subject. All the teachers for one subject get together, along with perhaps a factor of 2 more people who are experts in online delivery. They spend a factor of 4 more time designing
4 0.13090563 344 hunch net-2009-02-22-Effective Research Funding
Introduction: With a worldwide recession on, my impression is that the carnage in research has not been as severe as might be feared, at least in the United States. I know of two notable negative impacts: It’s quite difficult to get a job this year, as many companies and universities simply aren’t hiring. This is particularly tough on graduating students. Perhaps 10% of IBM research was fired. In contrast, around the time of the dot com bust, ATnT Research and Lucent had one or several 50% size firings wiping out much of the remainder of Bell Labs , triggering a notable diaspora for the respected machine learning group there. As the recession progresses, we may easily see more firings as companies in particular reach a point where they can no longer support research. There are a couple positives to the recession as well. Both the implosion of Wall Street (which siphoned off smart people) and the general difficulty of getting a job coming out of an undergraduate education s
5 0.13014567 73 hunch net-2005-05-17-A Short Guide to PhD Graduate Study
Introduction: Graduate study is a mysterious and uncertain process. This easiest way to see this is by noting that a very old advisor/student mechanism is preferred. There is no known succesful mechanism for “mass producing” PhDs as is done (in some sense) for undergraduate and masters study. Here are a few hints that might be useful to prospective or current students based on my own experience. Masters or PhD (a) You want a PhD if you want to do research. (b) You want a masters if you want to make money. People wanting (b) will be manifestly unhappy with (a) because it typically means years of low pay. People wanting (a) should try to avoid (b) because it prolongs an already long process. Attitude . Many students struggle for awhile with the wrong attitude towards research. Most students come into graduate school with 16-19 years of schooling where the principle means of success is proving that you know something via assignments, tests, etc… Research does not work this way. Re
6 0.12532413 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
7 0.12532382 421 hunch net-2011-01-03-Herman Goldstine 2011
8 0.12092203 75 hunch net-2005-05-28-Running A Machine Learning Summer School
9 0.11828247 4 hunch net-2005-01-26-Summer Schools
10 0.11416535 110 hunch net-2005-09-10-“Failure” is an option
11 0.1111567 437 hunch net-2011-07-10-ICML 2011 and the future
12 0.1019363 445 hunch net-2011-09-28-Somebody’s Eating Your Lunch
13 0.10191292 64 hunch net-2005-04-28-Science Fiction and Research
14 0.10158391 146 hunch net-2006-01-06-MLTV
15 0.10045592 105 hunch net-2005-08-23-(Dis)similarities between academia and open source programmers
16 0.10043518 50 hunch net-2005-04-01-Basic computer science research takes a hit
17 0.1002062 121 hunch net-2005-10-12-The unrealized potential of the research lab
18 0.099922255 256 hunch net-2007-07-20-Motivation should be the Responsibility of the Reviewer
19 0.097116806 154 hunch net-2006-02-04-Research Budget Changes
20 0.096656471 282 hunch net-2008-01-06-Research Political Issues
topicId topicWeight
[(0, 0.221), (1, -0.078), (2, -0.125), (3, 0.114), (4, -0.114), (5, -0.035), (6, 0.029), (7, 0.042), (8, -0.071), (9, 0.061), (10, 0.084), (11, -0.004), (12, -0.034), (13, -0.079), (14, -0.002), (15, 0.0), (16, -0.012), (17, 0.111), (18, 0.008), (19, 0.031), (20, 0.04), (21, -0.01), (22, 0.002), (23, -0.023), (24, 0.023), (25, -0.032), (26, 0.044), (27, -0.042), (28, 0.016), (29, 0.019), (30, -0.018), (31, -0.093), (32, 0.028), (33, 0.035), (34, -0.073), (35, 0.036), (36, -0.016), (37, -0.028), (38, 0.044), (39, -0.017), (40, 0.011), (41, -0.004), (42, 0.03), (43, 0.1), (44, 0.0), (45, 0.081), (46, 0.011), (47, 0.001), (48, -0.014), (49, -0.028)]
simIndex simValue blogId blogTitle
same-blog 1 0.96404803 449 hunch net-2011-11-26-Giving Thanks
Introduction: Thanksgiving is perhaps my favorite holiday, because pausing your life and giving thanks provides a needed moment of perspective. As a researcher, I am most thankful for my education, without which I could not function. I want to share this, because it provides some sense of how a researcher starts. My long term memory seems to function particularly well, which makes any education I get is particularly useful. I am naturally obsessive, which makes me chase down details until I fully understand things. Natural obsessiveness can go wrong, of course, but it’s a great ally when you absolutely must get things right. My childhood was all in one hometown, which was a conscious sacrifice on the part of my father, implying disruptions from moving around were eliminated. I’m not sure how important this was since travel has it’s own benefits, but it bears thought. I had several great teachers in grade school, and naturally gravitated towards teachers over classmates, as they seemed
2 0.75997925 73 hunch net-2005-05-17-A Short Guide to PhD Graduate Study
Introduction: Graduate study is a mysterious and uncertain process. This easiest way to see this is by noting that a very old advisor/student mechanism is preferred. There is no known succesful mechanism for “mass producing” PhDs as is done (in some sense) for undergraduate and masters study. Here are a few hints that might be useful to prospective or current students based on my own experience. Masters or PhD (a) You want a PhD if you want to do research. (b) You want a masters if you want to make money. People wanting (b) will be manifestly unhappy with (a) because it typically means years of low pay. People wanting (a) should try to avoid (b) because it prolongs an already long process. Attitude . Many students struggle for awhile with the wrong attitude towards research. Most students come into graduate school with 16-19 years of schooling where the principle means of success is proving that you know something via assignments, tests, etc… Research does not work this way. Re
3 0.74506724 344 hunch net-2009-02-22-Effective Research Funding
Introduction: With a worldwide recession on, my impression is that the carnage in research has not been as severe as might be feared, at least in the United States. I know of two notable negative impacts: It’s quite difficult to get a job this year, as many companies and universities simply aren’t hiring. This is particularly tough on graduating students. Perhaps 10% of IBM research was fired. In contrast, around the time of the dot com bust, ATnT Research and Lucent had one or several 50% size firings wiping out much of the remainder of Bell Labs , triggering a notable diaspora for the respected machine learning group there. As the recession progresses, we may easily see more firings as companies in particular reach a point where they can no longer support research. There are a couple positives to the recession as well. Both the implosion of Wall Street (which siphoned off smart people) and the general difficulty of getting a job coming out of an undergraduate education s
4 0.70567209 445 hunch net-2011-09-28-Somebody’s Eating Your Lunch
Introduction: Since we last discussed the other online learning , Stanford has very visibly started pushing mass teaching in AI , Machine Learning , and Databases . In retrospect, it’s not too surprising that the next step up in serious online teaching experiments are occurring at the computer science department of a university embedded in the land of startups. Numbers on the order of 100000 are quite significant—similar in scale to the number of computer science undergraduate students/year in the US. Although these populations surely differ, the fact that they could overlap is worth considering for the future. It’s too soon to say how successful these classes will be and there are many easy criticisms to make: Registration != Learning … but if only 1/10th complete these classes, the scale of teaching still surpasses the scale of any traditional process. 1st year excitement != nth year routine … but if only 1/10th take future classes, the scale of teaching still surpass
5 0.69776398 110 hunch net-2005-09-10-“Failure” is an option
Introduction: This is about the hard choices that graduate students must make. The cultural definition of success in academic research is to: Produce good research which many other people appreciate. Produce many students who go on to do the same. There are fundamental reasons why this is success in the local culture. Good research appreciated by others means access to jobs. Many students succesful in the same way implies that there are a number of people who think in a similar way and appreciate your work. In order to graduate, a phd student must live in an academic culture for a period of several years. It is common to adopt the culture’s definition of success during this time. It’s also common for many phd students discover they are not suited to an academic research lifestyle. This collision of values and abilities naturally results in depression. The most fundamental advice when this happens is: change something. Pick a new advisor. Pick a new research topic. Or leave th
6 0.68962914 75 hunch net-2005-05-28-Running A Machine Learning Summer School
7 0.68248391 36 hunch net-2005-03-05-Funding Research
8 0.68225646 414 hunch net-2010-10-17-Partha Niyogi has died
9 0.67832828 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
10 0.66328335 69 hunch net-2005-05-11-Visa Casualties
11 0.62702823 106 hunch net-2005-09-04-Science in the Government
12 0.60979038 464 hunch net-2012-05-03-Microsoft Research, New York City
13 0.60891598 282 hunch net-2008-01-06-Research Political Issues
14 0.60473311 154 hunch net-2006-02-04-Research Budget Changes
15 0.60215104 48 hunch net-2005-03-29-Academic Mechanism Design
16 0.60039586 471 hunch net-2012-08-24-Patterns for research in machine learning
17 0.59478414 121 hunch net-2005-10-12-The unrealized potential of the research lab
18 0.58612031 203 hunch net-2006-08-18-Report of MLSS 2006 Taipei
19 0.5753457 241 hunch net-2007-04-28-The Coming Patent Apocalypse
20 0.56812835 357 hunch net-2009-05-30-Many ways to Learn this summer
topicId topicWeight
[(3, 0.014), (12, 0.023), (27, 0.178), (38, 0.051), (48, 0.04), (51, 0.016), (53, 0.038), (55, 0.094), (59, 0.234), (68, 0.011), (89, 0.017), (94, 0.121), (95, 0.052)]
simIndex simValue blogId blogTitle
same-blog 1 0.88118726 449 hunch net-2011-11-26-Giving Thanks
Introduction: Thanksgiving is perhaps my favorite holiday, because pausing your life and giving thanks provides a needed moment of perspective. As a researcher, I am most thankful for my education, without which I could not function. I want to share this, because it provides some sense of how a researcher starts. My long term memory seems to function particularly well, which makes any education I get is particularly useful. I am naturally obsessive, which makes me chase down details until I fully understand things. Natural obsessiveness can go wrong, of course, but it’s a great ally when you absolutely must get things right. My childhood was all in one hometown, which was a conscious sacrifice on the part of my father, implying disruptions from moving around were eliminated. I’m not sure how important this was since travel has it’s own benefits, but it bears thought. I had several great teachers in grade school, and naturally gravitated towards teachers over classmates, as they seemed
2 0.87136251 459 hunch net-2012-03-13-The Submodularity workshop and Lucca Professorship
Introduction: Nina points out the Submodularity Workshop March 19-20 next week at Georgia Tech . Many people want to make Submodularity the new Convexity in machine learning, and it certainly seems worth exploring. Sara Olson also points out a tenured faculty position at IMT Lucca with a deadline of May 15th . Lucca happens to be the ancestral home of 1/4 of my heritage
3 0.72146904 258 hunch net-2007-08-12-Exponentiated Gradient
Introduction: The Exponentiated Gradient algorithm by Manfred Warmuth and Jyrki Kivinen came out just as I was starting graduate school, so I missed it both at a conference and in class. It’s a fine algorithm which has a remarkable theoretical statement accompanying it. The essential statement holds in the “online learning with an adversary” setting. Initially, there are of set of n weights, which might have values (1/n,…,1/n) , (or any other values from a probability distribution). Everything happens in a round-by-round fashion. On each round, the following happens: The world reveals a set of features x in {0,1} n . In the online learning with an adversary literature, the features are called “experts” and thought of as subpredictors, but this interpretation isn’t necessary—you can just use feature values as experts (or maybe the feature value and the negation of the feature value as two experts). EG makes a prediction according to y’ = w . x (dot product). The world reve
4 0.70293278 221 hunch net-2006-12-04-Structural Problems in NIPS Decision Making
Introduction: This is a very difficult post to write, because it is about a perenially touchy subject. Nevertheless, it is an important one which needs to be thought about carefully. There are a few things which should be understood: The system is changing and responsive. We-the-authors are we-the-reviewers, we-the-PC, and even we-the-NIPS-board. NIPS has implemented ‘secondary program chairs’, ‘author response’, and ‘double blind reviewing’ in the last few years to help with the decision process, and more changes may happen in the future. Agreement creates a perception of correctness. When any PC meets and makes a group decision about a paper, there is a strong tendency for the reinforcement inherent in a group decision to create the perception of correctness. For the many people who have been on the NIPS PC it’s reasonable to entertain a healthy skepticism in the face of this reinforcing certainty. This post is about structural problems. What problems arise because of the structure
5 0.70242381 286 hunch net-2008-01-25-Turing’s Club for Machine Learning
Introduction: Many people in Machine Learning don’t fully understand the impact of computation, as demonstrated by a lack of big-O analysis of new learning algorithms. This is important—some current active research programs are fundamentally flawed w.r.t. computation, and other research programs are directly motivated by it. When considering a learning algorithm, I think about the following questions: How does the learning algorithm scale with the number of examples m ? Any algorithm using all of the data is at least O(m) , but in many cases this is O(m 2 ) (naive nearest neighbor for self-prediction) or unknown (k-means or many other optimization algorithms). The unknown case is very common, and it can mean (for example) that the algorithm isn’t convergent or simply that the amount of computation isn’t controlled. The above question can also be asked for test cases. In some applications, test-time performance is of great importance. How does the algorithm scale with the number of
6 0.69994754 445 hunch net-2011-09-28-Somebody’s Eating Your Lunch
7 0.69415438 95 hunch net-2005-07-14-What Learning Theory might do
8 0.69369948 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
9 0.69259065 136 hunch net-2005-12-07-Is the Google way the way for machine learning?
10 0.69061571 423 hunch net-2011-02-02-User preferences for search engines
11 0.68994248 371 hunch net-2009-09-21-Netflix finishes (and starts)
12 0.68857312 359 hunch net-2009-06-03-Functionally defined Nonlinear Dynamic Models
13 0.68755859 343 hunch net-2009-02-18-Decision by Vetocracy
14 0.68635184 229 hunch net-2007-01-26-Parallel Machine Learning Problems
15 0.68612874 75 hunch net-2005-05-28-Running A Machine Learning Summer School
16 0.68519264 450 hunch net-2011-12-02-Hadoop AllReduce and Terascale Learning
17 0.68502581 300 hunch net-2008-04-30-Concerns about the Large Scale Learning Challenge
18 0.68337256 235 hunch net-2007-03-03-All Models of Learning have Flaws
19 0.6825887 306 hunch net-2008-07-02-Proprietary Data in Academic Research?
20 0.68213975 109 hunch net-2005-09-08-Online Learning as the Mathematics of Accountability