hunch_net hunch_net-2005 hunch_net-2005-91 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: One thing common to much research is that the researcher must be the first person ever to have some thought. How do you think of something that has never been thought of? There seems to be no methodical manner of doing this, but there are some tricks. The easiest method is to just have some connection come to you. There is a trick here however: you should write it down and fill out the idea immediately because it can just as easily go away. A harder method is to set aside a block of time and simply think about an idea. Distraction elimination is essential here because thinking about the unthought is hard work which your mind will avoid. Another common method is in conversation. Sometimes the process of verbalizing implies new ideas come up and sometimes whoever you are talking to replies just the right way. This method is dangerous though—you must speak to someone who helps you think rather than someone who occupies your thoughts. Try to rephrase the problem so the a
sentIndex sentText sentNum sentScore
1 One thing common to much research is that the researcher must be the first person ever to have some thought. [sent-1, score-0.527]
2 How do you think of something that has never been thought of? [sent-2, score-0.22]
3 There seems to be no methodical manner of doing this, but there are some tricks. [sent-3, score-0.102]
4 The easiest method is to just have some connection come to you. [sent-4, score-0.652]
5 There is a trick here however: you should write it down and fill out the idea immediately because it can just as easily go away. [sent-5, score-0.44]
6 A harder method is to set aside a block of time and simply think about an idea. [sent-6, score-0.731]
7 Distraction elimination is essential here because thinking about the unthought is hard work which your mind will avoid. [sent-7, score-0.247]
8 Sometimes the process of verbalizing implies new ideas come up and sometimes whoever you are talking to replies just the right way. [sent-9, score-0.578]
9 This method is dangerous though—you must speak to someone who helps you think rather than someone who occupies your thoughts. [sent-10, score-1.145]
10 Try to rephrase the problem so the answer is simple. [sent-11, score-0.167]
11 There are also general ‘context development’ techniques which are not specifically helpful for your problem, but which are generally helpful for related problems. [sent-14, score-0.527]
12 Understand the multiple motivations for working on some topic, when they exist. [sent-15, score-0.218]
13 Question the “rightness” of every related thing. [sent-16, score-0.127]
14 This is fundamental to finding good judgement in what you work on. [sent-17, score-0.129]
wordName wordTfidf (topN-words)
[('failing', 0.293), ('method', 0.26), ('rephrase', 0.167), ('distraction', 0.167), ('rightness', 0.167), ('whoever', 0.155), ('chaos', 0.155), ('someone', 0.154), ('elimination', 0.146), ('connection', 0.146), ('talk', 0.146), ('motivations', 0.14), ('dangerous', 0.14), ('helpful', 0.139), ('think', 0.138), ('block', 0.129), ('judgement', 0.129), ('fill', 0.129), ('related', 0.127), ('ever', 0.125), ('easiest', 0.125), ('specifically', 0.122), ('come', 0.121), ('awhile', 0.116), ('aside', 0.116), ('researcher', 0.113), ('trick', 0.113), ('attend', 0.113), ('talking', 0.111), ('development', 0.108), ('sometimes', 0.105), ('speak', 0.104), ('aspect', 0.104), ('must', 0.104), ('immediately', 0.102), ('manner', 0.102), ('mind', 0.101), ('common', 0.098), ('though', 0.097), ('write', 0.096), ('helps', 0.091), ('giving', 0.091), ('harder', 0.088), ('person', 0.087), ('ideas', 0.086), ('topic', 0.085), ('context', 0.084), ('never', 0.082), ('otherwise', 0.081), ('multiple', 0.078)]
simIndex simValue blogId blogTitle
same-blog 1 1.0 91 hunch net-2005-07-10-Thinking the Unthought
Introduction: One thing common to much research is that the researcher must be the first person ever to have some thought. How do you think of something that has never been thought of? There seems to be no methodical manner of doing this, but there are some tricks. The easiest method is to just have some connection come to you. There is a trick here however: you should write it down and fill out the idea immediately because it can just as easily go away. A harder method is to set aside a block of time and simply think about an idea. Distraction elimination is essential here because thinking about the unthought is hard work which your mind will avoid. Another common method is in conversation. Sometimes the process of verbalizing implies new ideas come up and sometimes whoever you are talking to replies just the right way. This method is dangerous though—you must speak to someone who helps you think rather than someone who occupies your thoughts. Try to rephrase the problem so the a
2 0.13637964 151 hunch net-2006-01-25-1 year
Introduction: At the one year (+5 days) anniversary, the natural question is: “Was it helpful for research?” Answer: Yes, and so it shall continue. Some evidence is provided by noticing that I am about a factor of 2 more overloaded with paper ideas than I’ve ever previously been. It is always hard to estimate counterfactual worlds, but I expect that this is also a factor of 2 more than “What if I had not started the blog?” As for “Why?”, there seem to be two primary effects. A blog is a mechanism for connecting with people who either think like you or are interested in the same problems. This allows for concentration of thinking which is very helpful in solving problems. The process of stating things you don’t understand publicly is very helpful in understanding them. Sometimes you are simply forced to express them in a way which aids understanding. Sometimes someone else says something which helps. And sometimes you discover that someone else has already solved the problem. The
3 0.12705375 358 hunch net-2009-06-01-Multitask Poisoning
Introduction: There are many ways that interesting research gets done. For example it’s common at a conference for someone to discuss a problem with a partial solution, and for someone else to know how to solve a piece of it, resulting in a paper. In some sense, these are the easiest results we can achieve, so we should ask: Can all research be this easy? The answer is certainly no for fields where research inherently requires experimentation to discover how the real world works. However, mathematics, including parts of physics, computer science, statistics, etc… which are effectively mathematics don’t require experimentation. In effect, a paper can be simply a pure expression of thinking. Can all mathematical-style research be this easy? What’s going on here is research-by-communication. Someone knows something, someone knows something else, and as soon as someone knows both things, a problem is solved. The interesting thing about research-by-communication is that it is becoming radic
4 0.11530601 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
Introduction: How do you create an optimal environment for research? Here are some essential ingredients that I see. Stability . University-based research is relatively good at this. On any particular day, researchers face choices in what they will work on. A very common tradeoff is between: easy small difficult big For researchers without stability, the ‘easy small’ option wins. This is often “ok”—a series of incremental improvements on the state of the art can add up to something very beneficial. However, it misses one of the big potentials of research: finding entirely new and better ways of doing things. Stability comes in many forms. The prototypical example is tenure at a university—a tenured professor is almost imposssible to fire which means that the professor has the freedom to consider far horizon activities. An iron-clad guarantee of a paycheck is not necessary—industrial research labs have succeeded well with research positions of indefinite duration. Atnt rese
5 0.11004187 141 hunch net-2005-12-17-Workshops as Franchise Conferences
Introduction: Founding a successful new conference is extraordinarily difficult. As a conference founder, you must manage to attract a significant number of good papers—enough to entice the participants into participating next year and to (generally) to grow the conference. For someone choosing to participate in a new conference, there is a very significant decision to make: do you send a paper to some new conference with no guarantee that the conference will work out? Or do you send it to another (possibly less related) conference that you are sure will work? The conference founding problem is a joint agreement problem with a very significant barrier. Workshops are a way around this problem, and workshops attached to conferences are a particularly effective means for this. A workshop at a conference is sure to have people available to speak and attend and is sure to have a large audience available. Presenting work at a workshop is not generally exclusive: it can also be presented at a confe
6 0.1002575 454 hunch net-2012-01-30-ICML Posters and Scope
7 0.098486565 435 hunch net-2011-05-16-Research Directions for Machine Learning and Algorithms
8 0.095236465 343 hunch net-2009-02-18-Decision by Vetocracy
9 0.094533347 249 hunch net-2007-06-21-Presentation Preparation
10 0.093027368 115 hunch net-2005-09-26-Prediction Bounds as the Mathematics of Science
11 0.087985009 96 hunch net-2005-07-21-Six Months
12 0.087930992 347 hunch net-2009-03-26-Machine Learning is too easy
13 0.087889686 256 hunch net-2007-07-20-Motivation should be the Responsibility of the Reviewer
14 0.086272888 134 hunch net-2005-12-01-The Webscience Future
15 0.085869275 30 hunch net-2005-02-25-Why Papers?
16 0.083164513 307 hunch net-2008-07-04-More Presentation Preparation
17 0.083065018 395 hunch net-2010-04-26-Compassionate Reviewing
18 0.081662923 116 hunch net-2005-09-30-Research in conferences
19 0.079853214 82 hunch net-2005-06-17-Reopening RL->Classification
20 0.07946226 22 hunch net-2005-02-18-What it means to do research.
topicId topicWeight
[(0, 0.212), (1, -0.017), (2, -0.019), (3, 0.1), (4, -0.061), (5, -0.008), (6, 0.063), (7, 0.033), (8, 0.058), (9, -0.004), (10, -0.026), (11, 0.014), (12, 0.007), (13, 0.065), (14, 0.059), (15, -0.047), (16, 0.02), (17, 0.072), (18, 0.059), (19, 0.041), (20, -0.051), (21, 0.003), (22, -0.026), (23, 0.007), (24, 0.073), (25, 0.003), (26, -0.02), (27, 0.042), (28, -0.058), (29, -0.05), (30, -0.095), (31, -0.06), (32, -0.069), (33, 0.039), (34, 0.031), (35, -0.086), (36, -0.016), (37, 0.09), (38, -0.126), (39, 0.084), (40, 0.04), (41, 0.047), (42, 0.004), (43, -0.027), (44, 0.006), (45, 0.009), (46, 0.006), (47, -0.031), (48, -0.029), (49, -0.07)]
simIndex simValue blogId blogTitle
same-blog 1 0.98503125 91 hunch net-2005-07-10-Thinking the Unthought
Introduction: One thing common to much research is that the researcher must be the first person ever to have some thought. How do you think of something that has never been thought of? There seems to be no methodical manner of doing this, but there are some tricks. The easiest method is to just have some connection come to you. There is a trick here however: you should write it down and fill out the idea immediately because it can just as easily go away. A harder method is to set aside a block of time and simply think about an idea. Distraction elimination is essential here because thinking about the unthought is hard work which your mind will avoid. Another common method is in conversation. Sometimes the process of verbalizing implies new ideas come up and sometimes whoever you are talking to replies just the right way. This method is dangerous though—you must speak to someone who helps you think rather than someone who occupies your thoughts. Try to rephrase the problem so the a
2 0.7274493 249 hunch net-2007-06-21-Presentation Preparation
Introduction: A big part of doing research is presenting it at a conference. Since many people start out shy of public presentations, this can be a substantial challenge. Here are a few notes which might be helpful when thinking about preparing a presentation on research. Motivate . Talks which don’t start by describing the problem to solve cause many people to zone out. Prioritize . It is typical that you have more things to say than time to say them, and many presenters fall into the failure mode of trying to say too much. This is an easy-to-understand failure mode as it’s very natural to want to include everything. A basic fact is: you can’t. Example of this are: Your slides are so densely full of equations and words that you can’t cover them. Your talk runs over and a moderator prioritizes for you by cutting you off. You motor-mouth through the presentation, and the information absorption rate of the audience prioritizes in some uncontrolled fashion. The rate of flow of c
3 0.70017976 358 hunch net-2009-06-01-Multitask Poisoning
Introduction: There are many ways that interesting research gets done. For example it’s common at a conference for someone to discuss a problem with a partial solution, and for someone else to know how to solve a piece of it, resulting in a paper. In some sense, these are the easiest results we can achieve, so we should ask: Can all research be this easy? The answer is certainly no for fields where research inherently requires experimentation to discover how the real world works. However, mathematics, including parts of physics, computer science, statistics, etc… which are effectively mathematics don’t require experimentation. In effect, a paper can be simply a pure expression of thinking. Can all mathematical-style research be this easy? What’s going on here is research-by-communication. Someone knows something, someone knows something else, and as soon as someone knows both things, a problem is solved. The interesting thing about research-by-communication is that it is becoming radic
4 0.68741262 307 hunch net-2008-07-04-More Presentation Preparation
Introduction: We’ve discussed presentation preparation before , but I have one more thing to add: transitioning . For a research presentation, it is substantially helpful for the audience if transitions are clear. A common outline for a research presentation in machine leanring is: The problem . Presentations which don’t describe the problem almost immediately lose people, because the context is missing to understand the detail. Prior relevant work . In many cases, a paper builds on some previous bit of work which must be understood in order to understand what the paper does. A common failure mode seems to be spending too much time on prior work. Discuss just the relevant aspects of prior work in the language of your work. Sometimes this is missing when unneeded. What we did . For theory papers in particular, it is often not possible to really cover the details. Prioritizing what you present can be very important. How it worked . Many papers in Machine Learning have some sor
5 0.6686793 257 hunch net-2007-07-28-Asking questions
Introduction: There are very substantial differences in how question asking is viewed culturally. For example, all of the following are common: If no one asks a question, then no one is paying attention. To ask a question is disrespectful of the speaker. Asking a question is admitting your own ignorance. The first view seems to be the right one for research, for several reasons. Research is quite hard—it’s difficult to guess how people won’t understand something in advance while preparing a presentation. Consequently, it’s very common to lose people. No worthwhile presenter wants that. Real understanding is precious. By asking a question, you are really declaring “I want to understand”, and everyone should respect that. Asking a question wakes you up. I don’t mean from “asleep” to “awake” but from “awake” to “really awake”. It’s easy to drift through something sort-of-understanding. When you ask a question, especially because you are on the spot, you will do much better.
6 0.65499717 231 hunch net-2007-02-10-Best Practices for Collaboration
7 0.62014377 256 hunch net-2007-07-20-Motivation should be the Responsibility of the Reviewer
8 0.61592996 241 hunch net-2007-04-28-The Coming Patent Apocalypse
9 0.61417389 22 hunch net-2005-02-18-What it means to do research.
10 0.6094389 187 hunch net-2006-06-25-Presentation of Proofs is Hard.
11 0.60722566 162 hunch net-2006-03-09-Use of Notation
12 0.59775078 151 hunch net-2006-01-25-1 year
13 0.58944505 76 hunch net-2005-05-29-Bad ideas
14 0.5723421 202 hunch net-2006-08-10-Precision is not accuracy
15 0.56328422 295 hunch net-2008-04-12-It Doesn’t Stop
16 0.54842412 153 hunch net-2006-02-02-Introspectionism as a Disease
17 0.54711616 73 hunch net-2005-05-17-A Short Guide to PhD Graduate Study
18 0.54317814 98 hunch net-2005-07-27-Not goal metrics
19 0.53977185 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
20 0.53805357 370 hunch net-2009-09-18-Necessary and Sufficient Research
topicId topicWeight
[(27, 0.178), (53, 0.617), (55, 0.06), (94, 0.047)]
simIndex simValue blogId blogTitle
1 0.99861324 107 hunch net-2005-09-05-Site Update
Introduction: I tweaked the site in a number of ways today, including: Updating to WordPress 1.5. Installing and heavily tweaking the Geekniche theme. Update: I switched back to a tweaked version of the old theme. Adding the Customizable Post Listings plugin. Installing the StatTraq plugin. Updating some of the links. I particularly recommend looking at the computer research policy blog. Adding threaded comments . This doesn’t thread old comments obviously, but the extra structure may be helpful for new ones. Overall, I think this is an improvement, and it addresses a few of my earlier problems . If you have any difficulties or anything seems “not quite right”, please speak up. A few other tweaks to the site may happen in the near future.
2 0.9949249 16 hunch net-2005-02-09-Intuitions from applied learning
Introduction: Since learning is far from an exact science, it’s good to pay attention to basic intuitions of applied learning. Here are a few I’ve collected. Integration In Bayesian learning, the posterior is computed by an integral, and the optimal thing to do is to predict according to this integral. This phenomena seems to be far more general. Bagging, Boosting, SVMs, and Neural Networks all take advantage of this idea to some extent. The phenomena is more general: you can average over many different classification predictors to improve performance. Sources: Zoubin , Caruana Differentiation Different pieces of an average should differentiate to achieve good performance by different methods. This is know as the ‘symmetry breaking’ problem for neural networks, and it’s why weights are initialized randomly. Boosting explicitly attempts to achieve good differentiation by creating new, different, learning problems. Sources: Yann LeCun , Phil Long Deep Representation Ha
3 0.99031663 56 hunch net-2005-04-14-Families of Learning Theory Statements
Introduction: The diagram above shows a very broad viewpoint of learning theory. arrow Typical statement Examples Past->Past Some prediction algorithm A does almost as well as any of a set of algorithms. Weighted Majority Past->Future Assuming independent samples, past performance predicts future performance. PAC analysis, ERM analysis Future->Future Future prediction performance on subproblems implies future prediction performance using algorithm A . ECOC, Probing A basic question is: Are there other varieties of statements of this type? Avrim noted that there are also “arrows between arrows”: generic methods for transforming between Past->Past statements and Past->Future statements. Are there others?
same-blog 4 0.98278415 91 hunch net-2005-07-10-Thinking the Unthought
Introduction: One thing common to much research is that the researcher must be the first person ever to have some thought. How do you think of something that has never been thought of? There seems to be no methodical manner of doing this, but there are some tricks. The easiest method is to just have some connection come to you. There is a trick here however: you should write it down and fill out the idea immediately because it can just as easily go away. A harder method is to set aside a block of time and simply think about an idea. Distraction elimination is essential here because thinking about the unthought is hard work which your mind will avoid. Another common method is in conversation. Sometimes the process of verbalizing implies new ideas come up and sometimes whoever you are talking to replies just the right way. This method is dangerous though—you must speak to someone who helps you think rather than someone who occupies your thoughts. Try to rephrase the problem so the a
5 0.9780696 2 hunch net-2005-01-24-Holy grails of machine learning?
Introduction: Let me kick things off by posing this question to ML researchers: What do you think are some important holy grails of machine learning? For example: – “A classifier with SVM-level performance but much more scalable” – “Practical confidence bounds (or learning bounds) for classification” – “A reinforcement learning algorithm that can handle the ___ problem” – “Understanding theoretically why ___ works so well in practice” etc. I pose this question because I believe that when goals are stated explicitly and well (thus providing clarity as well as opening up the problems to more people), rather than left implicit, they are likely to be achieved much more quickly. I would also like to know more about the internal goals of the various machine learning sub-areas (theory, kernel methods, graphical models, reinforcement learning, etc) as stated by people in these respective areas. This could help people cross sub-areas.
6 0.96756762 367 hunch net-2009-08-16-Centmail comments
7 0.93784857 145 hunch net-2005-12-29-Deadline Season
8 0.92671096 6 hunch net-2005-01-27-Learning Complete Problems
9 0.79275483 21 hunch net-2005-02-17-Learning Research Programs
10 0.69868624 201 hunch net-2006-08-07-The Call of the Deep
11 0.69365102 151 hunch net-2006-01-25-1 year
12 0.68555415 191 hunch net-2006-07-08-MaxEnt contradicts Bayes Rule?
13 0.68260467 60 hunch net-2005-04-23-Advantages and Disadvantages of Bayesian Learning
14 0.68073833 141 hunch net-2005-12-17-Workshops as Franchise Conferences
15 0.64873451 152 hunch net-2006-01-30-Should the Input Representation be a Vector?
16 0.64127034 265 hunch net-2007-10-14-NIPS workshp: Learning Problem Design
17 0.63142538 407 hunch net-2010-08-23-Boosted Decision Trees for Deep Learning
18 0.6211068 25 hunch net-2005-02-20-At One Month
19 0.62083578 416 hunch net-2010-10-29-To Vidoelecture or not
20 0.61067957 438 hunch net-2011-07-11-Interesting Neural Network Papers at ICML 2011