hunch_net hunch_net-2005 hunch_net-2005-36 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: The funding of research (and machine learning research) is an issue which seems to have become more significant in the United States over the last decade. The word “research” is applied broadly here to science, mathematics, and engineering. There are two essential difficulties with funding research: Longshot Paying a researcher is often a big gamble. Most research projects don’t pan out, but a few big payoffs can make it all worthwhile. Information Only Much of research is about finding the right way to think about or do something. The Longshot difficulty means that there is high variance in payoffs. This can be compensated for by funding many different research projects, reducing variance. The Information-Only difficulty means that it’s hard to extract a profit directly from many types of research, so companies have difficulty justifying basic research. (Patents are a mechanism for doing this. They are often extraordinarily clumsy or simply not applicable.) T
sentIndex sentText sentNum sentScore
1 The funding of research (and machine learning research) is an issue which seems to have become more significant in the United States over the last decade. [sent-1, score-0.814]
2 There are two essential difficulties with funding research: Longshot Paying a researcher is often a big gamble. [sent-3, score-0.61]
3 Most research projects don’t pan out, but a few big payoffs can make it all worthwhile. [sent-4, score-0.566]
4 This can be compensated for by funding many different research projects, reducing variance. [sent-7, score-0.728]
5 The Information-Only difficulty means that it’s hard to extract a profit directly from many types of research, so companies have difficulty justifying basic research. [sent-8, score-0.399]
6 ) These two difficulties together imply that research is often chronically underfunded compared to what would be optimal for any particular nation. [sent-11, score-0.532]
7 They also imply that funding for research makes more sense for larger nations and makes sense for government (rather than private) investment. [sent-12, score-1.303]
8 It made great sense for them because research was a place to stash money (and evade regulators) that might have some return. [sent-15, score-0.582]
9 IBM and HP , who have been historically strong funders of computer-related research have been forced to shift towards more direct research. [sent-19, score-0.5]
10 (Some great research still happens at these places, but the overall trend seems clear. [sent-20, score-0.4]
11 Many companies are funding directed research (with short term expected payoffs). [sent-24, score-1.136]
12 Many other branches of the government fund directed research of one sort or another. [sent-26, score-0.966]
13 From the perspective of a researcher, this isn’t as good as NSF because it is “money with strings attached”, including specific topics, demos, etc… Much of the funding available falls into two or three categories: directed into academia, very directed, or both. [sent-27, score-0.808]
14 Into Academia The difficulty with funding directed into academia is that the professors who it is directed at are incredibly busy with nonresearch. [sent-29, score-1.297]
15 It takes an extraordinary individual to manage all of this and get research done. [sent-31, score-0.404]
16 ) From the perspective of funding research, this is problematic, because the people being funded are forced to devote much time to nonresearch. [sent-33, score-0.742]
17 Very directed It’s a basic fact of research that it is difficult to devote careful and deep attention to something that does not interest you. [sent-38, score-0.877]
18 With the rise of the EU more funding for research makes sense because the benefit applies to a much larger pool of people. [sent-47, score-0.93]
19 On the engineering side, centers like the Mozilla Foundation and OSDL (which are funded by corporate contributions) provide some funding for open source programmers. [sent-49, score-0.546]
20 3% of the Federal government budget so the impact of more funding for basic research is relatively trivial in the big picture. [sent-52, score-1.093]
wordName wordTfidf (topN-words)
[('funding', 0.393), ('directed', 0.341), ('research', 0.335), ('fund', 0.175), ('nsf', 0.153), ('monopoly', 0.131), ('sense', 0.127), ('academia', 0.126), ('devote', 0.118), ('longshot', 0.118), ('government', 0.115), ('budget', 0.109), ('phone', 0.105), ('payoffs', 0.097), ('difficulty', 0.096), ('historically', 0.092), ('difficulties', 0.088), ('starting', 0.088), ('bell', 0.088), ('group', 0.087), ('significant', 0.086), ('funded', 0.084), ('basic', 0.083), ('united', 0.081), ('labs', 0.081), ('projects', 0.076), ('makes', 0.075), ('perspective', 0.074), ('forced', 0.073), ('researcher', 0.071), ('engineering', 0.069), ('ibm', 0.069), ('manage', 0.069), ('companies', 0.067), ('still', 0.065), ('money', 0.064), ('states', 0.063), ('michael', 0.06), ('news', 0.06), ('big', 0.058), ('means', 0.057), ('imply', 0.056), ('made', 0.056), ('amazingly', 0.053), ('breakup', 0.053), ('chronically', 0.053), ('collins', 0.053), ('ethernet', 0.053), ('forgetting', 0.053), ('invented', 0.053)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000001 36 hunch net-2005-03-05-Funding Research
Introduction: The funding of research (and machine learning research) is an issue which seems to have become more significant in the United States over the last decade. The word “research” is applied broadly here to science, mathematics, and engineering. There are two essential difficulties with funding research: Longshot Paying a researcher is often a big gamble. Most research projects don’t pan out, but a few big payoffs can make it all worthwhile. Information Only Much of research is about finding the right way to think about or do something. The Longshot difficulty means that there is high variance in payoffs. This can be compensated for by funding many different research projects, reducing variance. The Information-Only difficulty means that it’s hard to extract a profit directly from many types of research, so companies have difficulty justifying basic research. (Patents are a mechanism for doing this. They are often extraordinarily clumsy or simply not applicable.) T
2 0.34427729 344 hunch net-2009-02-22-Effective Research Funding
Introduction: With a worldwide recession on, my impression is that the carnage in research has not been as severe as might be feared, at least in the United States. I know of two notable negative impacts: It’s quite difficult to get a job this year, as many companies and universities simply aren’t hiring. This is particularly tough on graduating students. Perhaps 10% of IBM research was fired. In contrast, around the time of the dot com bust, ATnT Research and Lucent had one or several 50% size firings wiping out much of the remainder of Bell Labs , triggering a notable diaspora for the respected machine learning group there. As the recession progresses, we may easily see more firings as companies in particular reach a point where they can no longer support research. There are a couple positives to the recession as well. Both the implosion of Wall Street (which siphoned off smart people) and the general difficulty of getting a job coming out of an undergraduate education s
3 0.33234528 50 hunch net-2005-04-01-Basic computer science research takes a hit
Introduction: The New York Times has an interesting article about how DARPA has dropped funding for computer science to universities by about a factor of 2 over the last 5 years and become less directed towards basic research. Partially in response, the number of grant submissions to NSF has grown by a factor of 3 (with the NSF budget staying approximately constant in the interim). This is the sort of policy decision which may make sense for the defense department, but which means a large hit for basic research on information technology development in the US. For example “darpa funded the invention of the internet” is reasonably correct. This policy decision is particularly painful in the context of NSF budget cuts and the end of extensive phone monopoly funded research at Bell labs. The good news from a learning perspective is that (based on anecdotal evidence) much of the remaining funding is aimed at learning and learning-related fields. Methods of making good automated predictions obv
4 0.28134012 154 hunch net-2006-02-04-Research Budget Changes
Introduction: The announcement of an increase in funding for basic research in the US is encouraging. There is some discussion of this at the Computing Research Policy blog. One part of this discussion has a graph of NSF funding over time, presumably in dollar budgets. I don’t believe that dollar budgets are the right way to judge the impact of funding changes on researchers. A better way to judge seems to be in terms of dollar budget divided by GDP which provides a measure of the relative emphasis on research. This graph was assembled by dividing the NSF budget by the US GDP . For 2005 GDP, I used the current estimate and for 2006 and 2007 assumed an increase by a factor of 1.04 per year. The 2007 number also uses the requested 2007 budget which is certain to change. This graph makes it clear why researchers were upset: research funding emphasis has fallen for 3 years in a row. The reality has been significantly more severe due to DARPA decreasing funding and industrial
5 0.25794673 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
Introduction: How do you create an optimal environment for research? Here are some essential ingredients that I see. Stability . University-based research is relatively good at this. On any particular day, researchers face choices in what they will work on. A very common tradeoff is between: easy small difficult big For researchers without stability, the ‘easy small’ option wins. This is often “ok”—a series of incremental improvements on the state of the art can add up to something very beneficial. However, it misses one of the big potentials of research: finding entirely new and better ways of doing things. Stability comes in many forms. The prototypical example is tenure at a university—a tenured professor is almost imposssible to fire which means that the professor has the freedom to consider far horizon activities. An iron-clad guarantee of a paycheck is not necessary—industrial research labs have succeeded well with research positions of indefinite duration. Atnt rese
6 0.21163583 105 hunch net-2005-08-23-(Dis)similarities between academia and open source programmers
7 0.17695837 51 hunch net-2005-04-01-The Producer-Consumer Model of Research
8 0.17295082 121 hunch net-2005-10-12-The unrealized potential of the research lab
9 0.16483687 48 hunch net-2005-03-29-Academic Mechanism Design
10 0.15228739 449 hunch net-2011-11-26-Giving Thanks
11 0.14774047 142 hunch net-2005-12-22-Yes , I am applying
12 0.14191124 425 hunch net-2011-02-25-Yahoo! Machine Learning grant due March 11
13 0.13118468 22 hunch net-2005-02-18-What it means to do research.
14 0.12613472 464 hunch net-2012-05-03-Microsoft Research, New York City
15 0.11584216 73 hunch net-2005-05-17-A Short Guide to PhD Graduate Study
16 0.11341558 358 hunch net-2009-06-01-Multitask Poisoning
17 0.11223065 282 hunch net-2008-01-06-Research Political Issues
18 0.10425753 355 hunch net-2009-05-19-CI Fellows
19 0.10163535 175 hunch net-2006-04-30-John Langford –> Yahoo Research, NY
20 0.095262185 44 hunch net-2005-03-21-Research Styles in Machine Learning
topicId topicWeight
[(0, 0.218), (1, -0.06), (2, -0.18), (3, 0.204), (4, -0.241), (5, -0.062), (6, 0.053), (7, 0.124), (8, -0.071), (9, 0.135), (10, 0.142), (11, -0.016), (12, -0.118), (13, -0.179), (14, -0.023), (15, 0.046), (16, -0.071), (17, 0.043), (18, -0.163), (19, -0.113), (20, -0.079), (21, -0.071), (22, 0.006), (23, 0.044), (24, -0.027), (25, -0.042), (26, -0.13), (27, 0.115), (28, -0.006), (29, -0.045), (30, 0.092), (31, -0.096), (32, 0.089), (33, -0.003), (34, -0.005), (35, -0.012), (36, -0.004), (37, 0.039), (38, 0.075), (39, -0.063), (40, -0.013), (41, -0.082), (42, -0.011), (43, 0.06), (44, 0.009), (45, -0.017), (46, -0.09), (47, -0.029), (48, -0.009), (49, -0.039)]
simIndex simValue blogId blogTitle
same-blog 1 0.96373767 36 hunch net-2005-03-05-Funding Research
Introduction: The funding of research (and machine learning research) is an issue which seems to have become more significant in the United States over the last decade. The word “research” is applied broadly here to science, mathematics, and engineering. There are two essential difficulties with funding research: Longshot Paying a researcher is often a big gamble. Most research projects don’t pan out, but a few big payoffs can make it all worthwhile. Information Only Much of research is about finding the right way to think about or do something. The Longshot difficulty means that there is high variance in payoffs. This can be compensated for by funding many different research projects, reducing variance. The Information-Only difficulty means that it’s hard to extract a profit directly from many types of research, so companies have difficulty justifying basic research. (Patents are a mechanism for doing this. They are often extraordinarily clumsy or simply not applicable.) T
2 0.96357495 154 hunch net-2006-02-04-Research Budget Changes
Introduction: The announcement of an increase in funding for basic research in the US is encouraging. There is some discussion of this at the Computing Research Policy blog. One part of this discussion has a graph of NSF funding over time, presumably in dollar budgets. I don’t believe that dollar budgets are the right way to judge the impact of funding changes on researchers. A better way to judge seems to be in terms of dollar budget divided by GDP which provides a measure of the relative emphasis on research. This graph was assembled by dividing the NSF budget by the US GDP . For 2005 GDP, I used the current estimate and for 2006 and 2007 assumed an increase by a factor of 1.04 per year. The 2007 number also uses the requested 2007 budget which is certain to change. This graph makes it clear why researchers were upset: research funding emphasis has fallen for 3 years in a row. The reality has been significantly more severe due to DARPA decreasing funding and industrial
3 0.89391685 344 hunch net-2009-02-22-Effective Research Funding
Introduction: With a worldwide recession on, my impression is that the carnage in research has not been as severe as might be feared, at least in the United States. I know of two notable negative impacts: It’s quite difficult to get a job this year, as many companies and universities simply aren’t hiring. This is particularly tough on graduating students. Perhaps 10% of IBM research was fired. In contrast, around the time of the dot com bust, ATnT Research and Lucent had one or several 50% size firings wiping out much of the remainder of Bell Labs , triggering a notable diaspora for the respected machine learning group there. As the recession progresses, we may easily see more firings as companies in particular reach a point where they can no longer support research. There are a couple positives to the recession as well. Both the implosion of Wall Street (which siphoned off smart people) and the general difficulty of getting a job coming out of an undergraduate education s
4 0.86185193 50 hunch net-2005-04-01-Basic computer science research takes a hit
Introduction: The New York Times has an interesting article about how DARPA has dropped funding for computer science to universities by about a factor of 2 over the last 5 years and become less directed towards basic research. Partially in response, the number of grant submissions to NSF has grown by a factor of 3 (with the NSF budget staying approximately constant in the interim). This is the sort of policy decision which may make sense for the defense department, but which means a large hit for basic research on information technology development in the US. For example “darpa funded the invention of the internet” is reasonably correct. This policy decision is particularly painful in the context of NSF budget cuts and the end of extensive phone monopoly funded research at Bell labs. The good news from a learning perspective is that (based on anecdotal evidence) much of the remaining funding is aimed at learning and learning-related fields. Methods of making good automated predictions obv
5 0.7671355 121 hunch net-2005-10-12-The unrealized potential of the research lab
Introduction: I attended the IBM research 60th anniversary . IBM research is, by any reasonable account, the industrial research lab which has managed to bring the most value to it’s parent company over the long term. This can be seen by simply counting the survivors: IBM research is the only older research lab which has not gone through a period of massive firing. (Note that there are also new research labs .) Despite this impressive record, IBM research has failed, by far, to achieve it’s potential. Examples which came up in this meeting include: It took about a decade to produce DRAM after it was invented in the lab. (In fact, Intel produced it first.) Relational databases and SQL were invented and then languished. It was only under external competition that IBM released it’s own relational database. Why didn’t IBM grow an Oracle division ? An early lead in IP networking hardware did not result in IBM growing a Cisco division . Why not? And remember … IBM research is a s
6 0.76176667 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
7 0.71318066 48 hunch net-2005-03-29-Academic Mechanism Design
8 0.6972701 51 hunch net-2005-04-01-The Producer-Consumer Model of Research
9 0.67046756 449 hunch net-2011-11-26-Giving Thanks
10 0.65922111 255 hunch net-2007-07-13-The View From China
11 0.61214364 105 hunch net-2005-08-23-(Dis)similarities between academia and open source programmers
12 0.59589934 142 hunch net-2005-12-22-Yes , I am applying
13 0.59262812 73 hunch net-2005-05-17-A Short Guide to PhD Graduate Study
14 0.57455623 110 hunch net-2005-09-10-“Failure” is an option
15 0.57143044 106 hunch net-2005-09-04-Science in the Government
16 0.56333429 76 hunch net-2005-05-29-Bad ideas
17 0.54965019 464 hunch net-2012-05-03-Microsoft Research, New York City
18 0.54947948 296 hunch net-2008-04-21-The Science 2.0 article
19 0.52497977 241 hunch net-2007-04-28-The Coming Patent Apocalypse
20 0.52245229 282 hunch net-2008-01-06-Research Political Issues
topicId topicWeight
[(3, 0.026), (9, 0.012), (10, 0.035), (27, 0.146), (34, 0.012), (37, 0.013), (38, 0.059), (42, 0.015), (47, 0.187), (53, 0.046), (55, 0.099), (80, 0.014), (84, 0.023), (89, 0.063), (94, 0.061), (95, 0.108)]
simIndex simValue blogId blogTitle
1 0.85480636 329 hunch net-2008-11-28-A Bumper Crop of Machine Learning Graduates
Introduction: My impression is that this is a particularly strong year for machine learning graduates. Here’s my short list of the strong graduates I know. Analpha (for perversity’s sake) by last name: Jenn Wortmann . When Jenn visited us for the summer, she had one , two , three , four papers. That is typical—she’s smart, capable, and follows up many directions of research. I believe approximately all of her many papers are on different subjects. Ruslan Salakhutdinov . A Science paper on bijective dimensionality reduction , mastered and improved on deep belief nets which seems like an important flavor of nonlinear learning, and in my experience he’s very fast, capable and creative at problem solving. Marc’Aurelio Ranzato . I haven’t spoken with Marc very much, but he had a great visit at Yahoo! this summer, and has an impressive portfolio of applications and improvements on convolutional neural networks and other deep learning algorithms. Lihong Li . Lihong developed the
same-blog 2 0.84143353 36 hunch net-2005-03-05-Funding Research
Introduction: The funding of research (and machine learning research) is an issue which seems to have become more significant in the United States over the last decade. The word “research” is applied broadly here to science, mathematics, and engineering. There are two essential difficulties with funding research: Longshot Paying a researcher is often a big gamble. Most research projects don’t pan out, but a few big payoffs can make it all worthwhile. Information Only Much of research is about finding the right way to think about or do something. The Longshot difficulty means that there is high variance in payoffs. This can be compensated for by funding many different research projects, reducing variance. The Information-Only difficulty means that it’s hard to extract a profit directly from many types of research, so companies have difficulty justifying basic research. (Patents are a mechanism for doing this. They are often extraordinarily clumsy or simply not applicable.) T
3 0.78094071 320 hunch net-2008-10-14-Who is Responsible for a Bad Review?
Introduction: Although I’m greatly interested in machine learning, I think it must be admitted that there is a large amount of low quality logic being used in reviews. The problem is bad enough that sometimes I wonder if the Byzantine generals limit has been exceeded. For example, I’ve seen recent reviews where the given reasons for rejecting are: [ NIPS ] Theorem A is uninteresting because Theorem B is uninteresting. [ UAI ] When you learn by memorization, the problem addressed is trivial. [NIPS] The proof is in the appendix. [NIPS] This has been done before. (… but not giving any relevant citations) Just for the record I want to point out what’s wrong with these reviews. A future world in which such reasons never come up again would be great, but I’m sure these errors will be committed many times more in the future. This is nonsense. A theorem should be evaluated based on it’s merits, rather than the merits of another theorem. Learning by memorization requires an expon
4 0.72868258 372 hunch net-2009-09-29-Machine Learning Protests at the G20
Introduction: The machine learning department at CMU turned out en masse to protest the G20 summit in Pittsburgh. Arthur Gretton uploaded some great photos covering the event
5 0.72216117 464 hunch net-2012-05-03-Microsoft Research, New York City
Introduction: Yahoo! laid off people . Unlike every previous time there have been layoffs, this is serious for Yahoo! Research . We had advanced warning from Prabhakar through the simple act of leaving . Yahoo! Research was a world class organization that Prabhakar recruited much of personally, so it is deeply implausible that he would spontaneously decide to leave. My first thought when I saw the news was “Uhoh, Rob said that he knew it was serious when the head of ATnT Research left.” In this case it was even more significant, because Prabhakar recruited me on the premise that Y!R was an experiment in how research should be done: via a combination of high quality people and high engagement with the company. Prabhakar’s departure is a clear end to that experiment. The result is ambiguous from a business perspective. Y!R clearly was not capable of saving the company from its illnesses. I’m not privy to the internal accounting of impact and this is the kind of subject where there c
6 0.71811205 105 hunch net-2005-08-23-(Dis)similarities between academia and open source programmers
7 0.71697378 344 hunch net-2009-02-22-Effective Research Funding
8 0.70452255 466 hunch net-2012-06-05-ICML acceptance statistics
9 0.69890374 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
10 0.69331753 343 hunch net-2009-02-18-Decision by Vetocracy
11 0.69235289 373 hunch net-2009-10-03-Static vs. Dynamic multiclass prediction
12 0.68972409 437 hunch net-2011-07-10-ICML 2011 and the future
13 0.68632197 194 hunch net-2006-07-11-New Models
14 0.68592083 84 hunch net-2005-06-22-Languages of Learning
15 0.68589622 454 hunch net-2012-01-30-ICML Posters and Scope
16 0.68429482 406 hunch net-2010-08-22-KDD 2010
17 0.6842435 50 hunch net-2005-04-01-Basic computer science research takes a hit
18 0.68403918 204 hunch net-2006-08-28-Learning Theory standards for NIPS 2006
19 0.68398523 423 hunch net-2011-02-02-User preferences for search engines
20 0.68282974 360 hunch net-2009-06-15-In Active Learning, the question changes