hunch_net hunch_net-2009 hunch_net-2009-358 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: There are many ways that interesting research gets done. For example it’s common at a conference for someone to discuss a problem with a partial solution, and for someone else to know how to solve a piece of it, resulting in a paper. In some sense, these are the easiest results we can achieve, so we should ask: Can all research be this easy? The answer is certainly no for fields where research inherently requires experimentation to discover how the real world works. However, mathematics, including parts of physics, computer science, statistics, etc… which are effectively mathematics don’t require experimentation. In effect, a paper can be simply a pure expression of thinking. Can all mathematical-style research be this easy? What’s going on here is research-by-communication. Someone knows something, someone knows something else, and as soon as someone knows both things, a problem is solved. The interesting thing about research-by-communication is that it is becoming radic
sentIndex sentText sentNum sentScore
1 For example it’s common at a conference for someone to discuss a problem with a partial solution, and for someone else to know how to solve a piece of it, resulting in a paper. [sent-2, score-0.59]
2 The answer is certainly no for fields where research inherently requires experimentation to discover how the real world works. [sent-4, score-0.507]
3 Someone knows something, someone knows something else, and as soon as someone knows both things, a problem is solved. [sent-9, score-0.983]
4 The essential difficulty is that doing good research often requires the simultaneous understanding of several different things—the problem, all the broken approaches to solving some problem, why they break, and some hint about where to look for a solution. [sent-15, score-0.599]
5 Often, a problem is not immediately solved the first time it is thought of, instead a researcher must attack it again and again, until either giving up or finding a solution. [sent-18, score-0.316]
6 A basic parameter in attacking a problem is: How hard is it to resume where you left off? [sent-19, score-0.317]
7 Even if I worked on such a problem yesterday, it might take a half hour or an hour to reach a state where I’m prepared to make progress. [sent-22, score-0.485]
8 Given the difficulty of research, I (and many other people) often struggle in dealing with interruptions. [sent-23, score-0.53]
9 Modern technology has made communication very easy, implying a stream of potential interruptions throughout the day, some of which are undeniably fruitful. [sent-24, score-0.434]
10 And yet, an interruption means the overhead of getting back to thinking must be paid yet again. [sent-25, score-0.284]
11 Trading off properly between the value of avoiding the overhead and the value of dealing with interruptions is a constant struggle which typically did not exist before before modern communication technology made it prevalent. [sent-26, score-1.104]
12 I think it is common to give in to the interrupts, and effectively cease to be able to do research other than research-by-communication. [sent-27, score-0.285]
13 There is the standard problem that people you deal with don’t understand the overhead of switching tasks in research. [sent-30, score-0.392]
14 Coping with the modern world requires that at least some portion of our time be devoted to interrupts, which are almost always easier to deal with than research. [sent-32, score-0.495]
15 Dealing with these interrupts therefore can create a bad habit where you seek interrupts to achieve the (short term, easy) gratification of dealing with them. [sent-33, score-1.021]
16 Thus, multitasking creates an internal expectation of multitasking, which makes multitasking preferred, eliminating the ability to do careful research. [sent-34, score-0.478]
17 I don’t have a good answer to this problem, other than continuing the struggle to preserve substantial contiguous chunks of time for thinking. [sent-36, score-0.432]
18 An alternative sometimes-applicable solution is to reduce the overhead of starting to solve a problem by decomposing the problem into subproblems. [sent-37, score-0.743]
19 Where possible, this is of course valuable, but mathematical research is often almost uniquely undecomposable, because the nature of the problem is that it’s solution is unknown and hence undecomposable. [sent-38, score-0.569]
20 Restated, the real problem is finding a valid decomposition. [sent-39, score-0.245]
wordName wordTfidf (topN-words)
[('interrupts', 0.319), ('multitasking', 0.239), ('struggle', 0.212), ('overhead', 0.212), ('problem', 0.18), ('dealing', 0.18), ('someone', 0.163), ('knows', 0.159), ('communication', 0.147), ('requires', 0.144), ('interruptions', 0.142), ('research', 0.14), ('modern', 0.132), ('hour', 0.118), ('simultaneous', 0.113), ('solution', 0.105), ('mathematics', 0.093), ('easy', 0.085), ('else', 0.084), ('world', 0.079), ('technology', 0.079), ('answer', 0.078), ('however', 0.076), ('effectively', 0.074), ('often', 0.073), ('back', 0.072), ('time', 0.071), ('chunks', 0.071), ('cheaper', 0.071), ('habit', 0.071), ('resume', 0.071), ('voice', 0.071), ('cease', 0.071), ('insidious', 0.071), ('sped', 0.071), ('uniquely', 0.071), ('yesterday', 0.071), ('worked', 0.069), ('easier', 0.069), ('achieve', 0.066), ('coping', 0.066), ('attacking', 0.066), ('periods', 0.066), ('decomposing', 0.066), ('experimentation', 0.066), ('undeniably', 0.066), ('seek', 0.066), ('finding', 0.065), ('difficulty', 0.065), ('essential', 0.064)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999988 358 hunch net-2009-06-01-Multitask Poisoning
Introduction: There are many ways that interesting research gets done. For example it’s common at a conference for someone to discuss a problem with a partial solution, and for someone else to know how to solve a piece of it, resulting in a paper. In some sense, these are the easiest results we can achieve, so we should ask: Can all research be this easy? The answer is certainly no for fields where research inherently requires experimentation to discover how the real world works. However, mathematics, including parts of physics, computer science, statistics, etc… which are effectively mathematics don’t require experimentation. In effect, a paper can be simply a pure expression of thinking. Can all mathematical-style research be this easy? What’s going on here is research-by-communication. Someone knows something, someone knows something else, and as soon as someone knows both things, a problem is solved. The interesting thing about research-by-communication is that it is becoming radic
2 0.16904825 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
Introduction: How do you create an optimal environment for research? Here are some essential ingredients that I see. Stability . University-based research is relatively good at this. On any particular day, researchers face choices in what they will work on. A very common tradeoff is between: easy small difficult big For researchers without stability, the ‘easy small’ option wins. This is often “ok”—a series of incremental improvements on the state of the art can add up to something very beneficial. However, it misses one of the big potentials of research: finding entirely new and better ways of doing things. Stability comes in many forms. The prototypical example is tenure at a university—a tenured professor is almost imposssible to fire which means that the professor has the freedom to consider far horizon activities. An iron-clad guarantee of a paycheck is not necessary—industrial research labs have succeeded well with research positions of indefinite duration. Atnt rese
3 0.13780873 22 hunch net-2005-02-18-What it means to do research.
Introduction: I want to try to describe what doing research means, especially from the point of view of an undergraduate. The shift from a class-taking mentality to a research mentality is very significant and not easy. Problem Posing Posing the right problem is often as important as solving them. Many people can get by in research by solving problems others have posed, but that’s not sufficient for really inspiring research. For learning in particular, there is a strong feeling that we just haven’t figured out which questions are the right ones to ask. You can see this, because the answers we have do not seem convincing. Gambling your life When you do research, you think very hard about new ways of solving problems, new problems, and new solutions. Many conversations are of the form “I wonder what would happen if…” These processes can be short (days or weeks) or years-long endeavours. The worst part is that you’ll only know if you were succesful at the end of the process (and some
4 0.13292401 344 hunch net-2009-02-22-Effective Research Funding
Introduction: With a worldwide recession on, my impression is that the carnage in research has not been as severe as might be feared, at least in the United States. I know of two notable negative impacts: It’s quite difficult to get a job this year, as many companies and universities simply aren’t hiring. This is particularly tough on graduating students. Perhaps 10% of IBM research was fired. In contrast, around the time of the dot com bust, ATnT Research and Lucent had one or several 50% size firings wiping out much of the remainder of Bell Labs , triggering a notable diaspora for the respected machine learning group there. As the recession progresses, we may easily see more firings as companies in particular reach a point where they can no longer support research. There are a couple positives to the recession as well. Both the implosion of Wall Street (which siphoned off smart people) and the general difficulty of getting a job coming out of an undergraduate education s
5 0.12705375 91 hunch net-2005-07-10-Thinking the Unthought
Introduction: One thing common to much research is that the researcher must be the first person ever to have some thought. How do you think of something that has never been thought of? There seems to be no methodical manner of doing this, but there are some tricks. The easiest method is to just have some connection come to you. There is a trick here however: you should write it down and fill out the idea immediately because it can just as easily go away. A harder method is to set aside a block of time and simply think about an idea. Distraction elimination is essential here because thinking about the unthought is hard work which your mind will avoid. Another common method is in conversation. Sometimes the process of verbalizing implies new ideas come up and sometimes whoever you are talking to replies just the right way. This method is dangerous though—you must speak to someone who helps you think rather than someone who occupies your thoughts. Try to rephrase the problem so the a
6 0.1206811 435 hunch net-2011-05-16-Research Directions for Machine Learning and Algorithms
7 0.11645415 313 hunch net-2008-08-18-Radford Neal starts a blog
8 0.11341558 36 hunch net-2005-03-05-Funding Research
9 0.11118221 282 hunch net-2008-01-06-Research Political Issues
10 0.10861105 51 hunch net-2005-04-01-The Producer-Consumer Model of Research
11 0.10850404 67 hunch net-2005-05-06-Don’t mix the solution into the problem
12 0.1077806 454 hunch net-2012-01-30-ICML Posters and Scope
13 0.10664552 151 hunch net-2006-01-25-1 year
14 0.10368761 370 hunch net-2009-09-18-Necessary and Sufficient Research
15 0.1028309 296 hunch net-2008-04-21-The Science 2.0 article
16 0.10196783 332 hunch net-2008-12-23-Use of Learning Theory
17 0.099688545 149 hunch net-2006-01-18-Is Multitask Learning Black-Boxable?
18 0.098544911 42 hunch net-2005-03-17-Going all the Way, Sometimes
19 0.096079789 352 hunch net-2009-05-06-Machine Learning to AI
20 0.095981322 90 hunch net-2005-07-07-The Limits of Learning Theory
topicId topicWeight
[(0, 0.258), (1, -0.003), (2, -0.077), (3, 0.148), (4, -0.097), (5, -0.03), (6, 0.071), (7, 0.041), (8, 0.002), (9, 0.044), (10, -0.027), (11, -0.026), (12, 0.0), (13, 0.062), (14, 0.007), (15, 0.009), (16, 0.019), (17, 0.014), (18, 0.032), (19, 0.051), (20, -0.075), (21, 0.053), (22, -0.014), (23, 0.055), (24, 0.009), (25, -0.025), (26, 0.002), (27, 0.02), (28, -0.051), (29, -0.01), (30, -0.015), (31, 0.001), (32, 0.01), (33, 0.043), (34, -0.058), (35, 0.045), (36, -0.003), (37, 0.02), (38, -0.054), (39, 0.032), (40, 0.044), (41, 0.071), (42, -0.051), (43, 0.013), (44, 0.06), (45, -0.078), (46, -0.012), (47, -0.061), (48, -0.035), (49, -0.014)]
simIndex simValue blogId blogTitle
same-blog 1 0.9713825 358 hunch net-2009-06-01-Multitask Poisoning
Introduction: There are many ways that interesting research gets done. For example it’s common at a conference for someone to discuss a problem with a partial solution, and for someone else to know how to solve a piece of it, resulting in a paper. In some sense, these are the easiest results we can achieve, so we should ask: Can all research be this easy? The answer is certainly no for fields where research inherently requires experimentation to discover how the real world works. However, mathematics, including parts of physics, computer science, statistics, etc… which are effectively mathematics don’t require experimentation. In effect, a paper can be simply a pure expression of thinking. Can all mathematical-style research be this easy? What’s going on here is research-by-communication. Someone knows something, someone knows something else, and as soon as someone knows both things, a problem is solved. The interesting thing about research-by-communication is that it is becoming radic
2 0.76547021 370 hunch net-2009-09-18-Necessary and Sufficient Research
Introduction: Researchers are typically confronted with big problems that they have no idea how to solve. In trying to come up with a solution, a natural approach is to decompose the big problem into a set of subproblems whose solution yields a solution to the larger problem. This approach can go wrong in several ways. Decomposition failure . The solution to the decomposition does not in fact yield a solution to the overall problem. Artificial hardness . The subproblems created are sufficient if solved to solve the overall problem, but they are harder than necessary. As you can see, computational complexity forms a relatively new (in research-history) razor by which to judge an approach sufficient but not necessary. In my experience, the artificial hardness problem is very common. Many researchers abdicate the responsibility of choosing a problem to work on to other people. This process starts very naturally as a graduate student, when an incoming student might have relatively l
3 0.75973529 22 hunch net-2005-02-18-What it means to do research.
Introduction: I want to try to describe what doing research means, especially from the point of view of an undergraduate. The shift from a class-taking mentality to a research mentality is very significant and not easy. Problem Posing Posing the right problem is often as important as solving them. Many people can get by in research by solving problems others have posed, but that’s not sufficient for really inspiring research. For learning in particular, there is a strong feeling that we just haven’t figured out which questions are the right ones to ask. You can see this, because the answers we have do not seem convincing. Gambling your life When you do research, you think very hard about new ways of solving problems, new problems, and new solutions. Many conversations are of the form “I wonder what would happen if…” These processes can be short (days or weeks) or years-long endeavours. The worst part is that you’ll only know if you were succesful at the end of the process (and some
4 0.75604033 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
Introduction: How do you create an optimal environment for research? Here are some essential ingredients that I see. Stability . University-based research is relatively good at this. On any particular day, researchers face choices in what they will work on. A very common tradeoff is between: easy small difficult big For researchers without stability, the ‘easy small’ option wins. This is often “ok”—a series of incremental improvements on the state of the art can add up to something very beneficial. However, it misses one of the big potentials of research: finding entirely new and better ways of doing things. Stability comes in many forms. The prototypical example is tenure at a university—a tenured professor is almost imposssible to fire which means that the professor has the freedom to consider far horizon activities. An iron-clad guarantee of a paycheck is not necessary—industrial research labs have succeeded well with research positions of indefinite duration. Atnt rese
5 0.73840255 76 hunch net-2005-05-29-Bad ideas
Introduction: I found these two essays on bad ideas interesting. Neither of these is written from the viewpoint of research, but they are both highly relevant. Why smart people have bad ideas by Paul Graham Why smart people defend bad ideas by Scott Berkun (which appeared on slashdot ) In my experience, bad ideas are common and over confidence in ideas is common. This overconfidence can take either the form of excessive condemnation or excessive praise. Some of this is necessary to the process of research. For example, some overconfidence in the value of your own research is expected and probably necessary to motivate your own investigation. Since research is a rather risky business, much of it does not pan out. Learning to accept when something does not pan out is a critical skill which is sometimes never acquired. Excessive condemnation can be a real ill when it’s encountered. This has two effects: When the penalty for being wrong is too large, it means people have a
6 0.7335965 91 hunch net-2005-07-10-Thinking the Unthought
7 0.72820818 257 hunch net-2007-07-28-Asking questions
8 0.72033495 147 hunch net-2006-01-08-Debugging Your Brain
9 0.71639526 282 hunch net-2008-01-06-Research Political Issues
10 0.70916224 295 hunch net-2008-04-12-It Doesn’t Stop
11 0.69641572 231 hunch net-2007-02-10-Best Practices for Collaboration
12 0.68627793 73 hunch net-2005-05-17-A Short Guide to PhD Graduate Study
13 0.68440282 106 hunch net-2005-09-04-Science in the Government
14 0.67400587 344 hunch net-2009-02-22-Effective Research Funding
15 0.67286325 296 hunch net-2008-04-21-The Science 2.0 article
16 0.67112452 256 hunch net-2007-07-20-Motivation should be the Responsibility of the Reviewer
17 0.66131914 222 hunch net-2006-12-05-Recruitment Conferences
18 0.65305102 435 hunch net-2011-05-16-Research Directions for Machine Learning and Algorithms
19 0.65120411 162 hunch net-2006-03-09-Use of Notation
20 0.64847249 333 hunch net-2008-12-27-Adversarial Academia
topicId topicWeight
[(0, 0.014), (3, 0.027), (22, 0.226), (27, 0.228), (38, 0.045), (53, 0.087), (55, 0.096), (64, 0.015), (68, 0.02), (71, 0.028), (94, 0.101), (95, 0.049)]
simIndex simValue blogId blogTitle
1 0.9413178 114 hunch net-2005-09-20-Workshop Proposal: Atomic Learning
Introduction: This is a proposal for a workshop. It may or may not happen depending on the level of interest. If you are interested, feel free to indicate so (by email or comments). Description: Assume(*) that any system for solving large difficult learning problems must decompose into repeated use of basic elements (i.e. atoms). There are many basic questions which remain: What are the viable basic elements? What makes a basic element viable? What are the viable principles for the composition of these basic elements? What are the viable principles for learning in such systems? What problems can this approach handle? Hal Daume adds: Can composition of atoms be (semi-) automatically constructed[?] When atoms are constructed through reductions, is there some notion of the “naturalness” of the created leaning problems? Other than Markov fields/graphical models/Bayes nets, is there a good language for representing atoms and their compositions? The answer to these a
2 0.90405464 410 hunch net-2010-09-17-New York Area Machine Learning Events
Introduction: On Sept 21, there is another machine learning meetup where I’ll be speaking. Although the topic is contextual bandits, I think of it as “the future of machine learning”. In particular, it’s all about how to learn in an interactive environment, such as for ad display, trading, news recommendation, etc… On Sept 24, abstracts for the New York Machine Learning Symposium are due. This is the largest Machine Learning event in the area, so it’s a great way to have a conversation with other people. On Oct 22, the NY ML Symposium actually happens. This year, we are expanding the spotlights, and trying to have more time for posters. In addition, we have a strong set of invited speakers: David Blei , Sanjoy Dasgupta , Tommi Jaakkola , and Yann LeCun . After the meeting, a late hackNY related event is planned where students and startups can meet. I’d also like to point out the related CS/Econ symposium as I have interests there as well.
same-blog 3 0.90218776 358 hunch net-2009-06-01-Multitask Poisoning
Introduction: There are many ways that interesting research gets done. For example it’s common at a conference for someone to discuss a problem with a partial solution, and for someone else to know how to solve a piece of it, resulting in a paper. In some sense, these are the easiest results we can achieve, so we should ask: Can all research be this easy? The answer is certainly no for fields where research inherently requires experimentation to discover how the real world works. However, mathematics, including parts of physics, computer science, statistics, etc… which are effectively mathematics don’t require experimentation. In effect, a paper can be simply a pure expression of thinking. Can all mathematical-style research be this easy? What’s going on here is research-by-communication. Someone knows something, someone knows something else, and as soon as someone knows both things, a problem is solved. The interesting thing about research-by-communication is that it is becoming radic
4 0.89374995 113 hunch net-2005-09-19-NIPS Workshops
Introduction: Attendance at the NIPS workshops is highly recommended for both research and learning. Unfortunately, there does not yet appear to be a public list of workshops. However, I found the following workshop webpages of interest: Machine Learning in Finance Learning to Rank Foundations of Active Learning Machine Learning Based Robotics in Unstructured Environments There are many more workshops. In fact, there are so many that it is not plausible anyone can attend every workshop they are interested in. Maybe in future years the organizers can spread them out over more days to reduce overlap. Many of these workshops are accepting presentation proposals (due mid-October).
5 0.83045763 79 hunch net-2005-06-08-Question: “When is the right time to insert the loss function?”
Introduction: Hal asks a very good question: “When is the right time to insert the loss function?” In particular, should it be used at testing time or at training time? When the world imposes a loss on us, the standard Bayesian recipe is to predict the (conditional) probability of each possibility and then choose the possibility which minimizes the expected loss. In contrast, as the confusion over “loss = money lost” or “loss = the thing you optimize” might indicate, many people ignore the Bayesian approach and simply optimize their loss (or a close proxy for their loss) over the representation on the training set. The best answer I can give is “it’s unclear, but I prefer optimizing the loss at training time”. My experience is that optimizing the loss in the most direct manner possible typically yields best performance. This question is related to a basic principle which both Yann LeCun (applied) and Vladimir Vapnik (theoretical) advocate: “solve the simplest prediction problem that s
6 0.76990783 286 hunch net-2008-01-25-Turing’s Club for Machine Learning
7 0.76803589 435 hunch net-2011-05-16-Research Directions for Machine Learning and Algorithms
8 0.76744765 95 hunch net-2005-07-14-What Learning Theory might do
9 0.76550895 351 hunch net-2009-05-02-Wielding a New Abstraction
10 0.7638393 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
11 0.76267701 370 hunch net-2009-09-18-Necessary and Sufficient Research
12 0.76227528 359 hunch net-2009-06-03-Functionally defined Nonlinear Dynamic Models
13 0.76163882 343 hunch net-2009-02-18-Decision by Vetocracy
14 0.76019377 360 hunch net-2009-06-15-In Active Learning, the question changes
15 0.75990933 371 hunch net-2009-09-21-Netflix finishes (and starts)
16 0.75904649 347 hunch net-2009-03-26-Machine Learning is too easy
17 0.75848806 235 hunch net-2007-03-03-All Models of Learning have Flaws
18 0.75829929 450 hunch net-2011-12-02-Hadoop AllReduce and Terascale Learning
19 0.75804341 194 hunch net-2006-07-11-New Models
20 0.7565679 134 hunch net-2005-12-01-The Webscience Future