hunch_net hunch_net-2011 hunch_net-2011-424 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: Watson convincingly beat the best champion Jeopardy! players. The apparent significance of this varies hugely, depending on your background knowledge about the related machine learning, NLP, and search technology. For a random person, this might seem evidence of serious machine intelligence, while for people working on the system itself, it probably seems like a reasonably good assemblage of existing technologies with several twists to make the entire system work. Above all, I think we should congratulate the people who managed to put together and execute this project—many years of effort by a diverse set of highly skilled people were needed to make this happen. In academia, it’s pretty difficult for one professor to assemble that quantity of talent, and in industry it’s rarely the case that such a capable group has both a worthwhile project and the support needed to pursue something like this for several years before success. Alina invited me to the Jeopardy watching party
sentIndex sentText sentNum sentScore
1 For a random person, this might seem evidence of serious machine intelligence, while for people working on the system itself, it probably seems like a reasonably good assemblage of existing technologies with several twists to make the entire system work. [sent-4, score-0.346]
2 Above all, I think we should congratulate the people who managed to put together and execute this project—many years of effort by a diverse set of highly skilled people were needed to make this happen. [sent-5, score-0.323]
3 In academia, it’s pretty difficult for one professor to assemble that quantity of talent, and in industry it’s rarely the case that such a capable group has both a worthwhile project and the support needed to pursue something like this for several years before success. [sent-6, score-0.466]
4 Partly, this is because I already knew that computers could answer trivia questions moderately well(*), so the question was just how far this could be improved. [sent-10, score-0.238]
5 Gerry tells me that although Watson’s error rate is still significant, one key element is the ability to estimate with high accuracy when they can answer with high accuracy. [sent-11, score-0.372]
6 The first is that there is clearly very substantial room for improvement, and the second is that having a natural language question/answering device that can quickly search and respond from large sets of text is obviously valuable. [sent-15, score-0.202]
7 The history of textual entailment challenges is another less centralized effort in about the same direction. [sent-17, score-0.198]
8 In the immediate future (next few years), applications in semi-open domains may become viable, particularly when a question/answer device knows when to answer “I don’t know”. [sent-18, score-0.204]
9 Fully conversational speech recognition working in an open domain should take somewhat longer, because speech recognition software has additional error points, conversational systems aren’t so easy to come by, and in a fully open domain the error rates will be higher. [sent-19, score-1.632]
10 Getting the error rate on questions down to the level that a human with access to the internet has difficulty beating is the tricky challenge which has not yet been addressed. [sent-20, score-0.375]
11 Many people believe in human exceptionalism, so when seeing a computer beat Jeopardy, they are surprised that humans aren’t exceptional there. [sent-22, score-0.443]
12 We should understand that this has happened many times before, with chess and mathematical calculation being two areas where computers now dominate, but which were once thought to be the essence of intelligence by some. [sent-23, score-0.268]
13 Similarly, it is not difficult to imagine automated driving (after all, animals can do it), gross object recognition, etc… To avert surprise in the future, human exceptionalists should understand what the really hard things for an AI to do are. [sent-24, score-0.547]
14 The ability to understand your place in the world, navigate the world, and accomplish something. [sent-27, score-0.263]
15 This level implies that routine tasks can be automated. [sent-29, score-0.219]
16 The ability to mimic a typical human well-enough to fool a typical human in open conversation. [sent-31, score-0.822]
17 Watson doesn’t achieve this, but the thrust of the research is in this direction as open domain question answering is probably necessary for this. [sent-32, score-0.495]
18 The ability to efficiently self-program in an open domain so as to continuously improve. [sent-36, score-0.43]
19 At this level human exceptionalism fails, and it is difficult to predict what happens next. [sent-37, score-0.545]
20 (*) About 10 years ago, I had a friend 2 on WWTBAM who called the friend for help on a question, who typed the question and multiple choice answers into CMU ‘s Zephyr system, where a bot I made queried (question,answer) pairs on Google to discover which had the most web pages. [sent-39, score-0.538]
wordName wordTfidf (topN-words)
[('watson', 0.371), ('gerry', 0.222), ('jeopardy', 0.198), ('human', 0.19), ('domain', 0.182), ('conversational', 0.148), ('exceptionalism', 0.148), ('open', 0.139), ('tasks', 0.136), ('device', 0.132), ('recognition', 0.125), ('years', 0.12), ('automated', 0.117), ('project', 0.112), ('ability', 0.109), ('beat', 0.105), ('friend', 0.102), ('error', 0.102), ('intelligence', 0.096), ('probably', 0.092), ('pretty', 0.09), ('driving', 0.089), ('tells', 0.089), ('understand', 0.088), ('surprised', 0.087), ('computers', 0.084), ('speech', 0.084), ('level', 0.083), ('question', 0.082), ('needed', 0.081), ('partly', 0.077), ('fully', 0.072), ('answer', 0.072), ('search', 0.07), ('typed', 0.066), ('entailment', 0.066), ('textual', 0.066), ('worthy', 0.066), ('publicity', 0.066), ('fool', 0.066), ('centralized', 0.066), ('queried', 0.066), ('dominate', 0.066), ('navigate', 0.066), ('serious', 0.065), ('typical', 0.064), ('system', 0.064), ('difficult', 0.063), ('people', 0.061), ('happens', 0.061)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000002 424 hunch net-2011-02-17-What does Watson mean?
Introduction: Watson convincingly beat the best champion Jeopardy! players. The apparent significance of this varies hugely, depending on your background knowledge about the related machine learning, NLP, and search technology. For a random person, this might seem evidence of serious machine intelligence, while for people working on the system itself, it probably seems like a reasonably good assemblage of existing technologies with several twists to make the entire system work. Above all, I think we should congratulate the people who managed to put together and execute this project—many years of effort by a diverse set of highly skilled people were needed to make this happen. In academia, it’s pretty difficult for one professor to assemble that quantity of talent, and in industry it’s rarely the case that such a capable group has both a worthwhile project and the support needed to pursue something like this for several years before success. Alina invited me to the Jeopardy watching party
2 0.14900093 352 hunch net-2009-05-06-Machine Learning to AI
Introduction: I recently had fun discussions with both Vikash Mansinghka and Thomas Breuel about approaching AI with machine learning. The general interest in taking a crack at AI with machine learning seems to be rising on many fronts including DARPA . As a matter of history, there was a great deal of interest in AI which died down before I began research. There remain many projects and conferences spawned in this earlier AI wave, as well as a good bit of experience about what did not work, or at least did not work yet. Here are a few examples of failure modes that people seem to run into: Supply/Product confusion . Sometimes we think “Intelligences use X, so I’ll create X and have an Intelligence.” An example of this is the Cyc Project which inspires some people as “intelligences use ontologies, so I’ll create an ontology and a system using it to have an Intelligence.” The flaw here is that Intelligences create ontologies, which they use, and without the ability to create ont
3 0.14219536 120 hunch net-2005-10-10-Predictive Search is Coming
Introduction: “Search” is the other branch of AI research which has been succesful. Concrete examples include Deep Blue which beat the world chess champion and Chinook the champion checkers program. A set of core search techniques exist including A * , alpha-beta pruning, and others that can be applied to any of many different search problems. Given this, it may be surprising to learn that there has been relatively little succesful work on combining prediction and search. Given also that humans typically solve search problems using a number of predictive heuristics to narrow in on a solution, we might be surprised again. However, the big successful search-based systems have typically not used “smart” search algorithms. Insteady they have optimized for very fast search. This is not for lack of trying… many people have tried to synthesize search and prediction to various degrees of success. For example, Knightcap achieves good-but-not-stellar chess playing performance, and TD-gammon
4 0.14144756 3 hunch net-2005-01-24-The Humanloop Spectrum of Machine Learning
Introduction: All branches of machine learning seem to be united in the idea of using data to make predictions. However, people disagree to some extent about what this means. One way to categorize these different goals is on an axis, where one extreme is “tools to aid a human in using data to do prediction” and the other extreme is “tools to do prediction with no human intervention”. Here is my estimate of where various elements of machine learning fall on this spectrum. Human Necessary Human partially necessary Human unnecessary Clustering, data visualization Bayesian Learning, Probabilistic Models, Graphical Models Kernel Learning (SVM’s, etc..) Decision Trees? Reinforcement Learning The exact position of each element is of course debatable. My reasoning is that clustering and data visualization are nearly useless for prediction without a human in the loop. Bayesian/probabilistic models/graphical models generally require a human to sit and think about what
5 0.11187029 378 hunch net-2009-11-15-The Other Online Learning
Introduction: If you search for “online learning” with any major search engine , it’s interesting to note that zero of the results are for online machine learning. This may not be a mistake if you are committed to a global ordering. In other words, the number of people specifically interested in the least interesting top-10 online human learning result might exceed the number of people interested in online machine learning, even given the presence of the other 9 results. The essential observation here is that the process of human learning is a big business (around 5% of GDP) effecting virtually everyone. The internet is changing this dramatically, by altering the economics of teaching. Consider two possibilities: The classroom-style teaching environment continues as is, with many teachers for the same subject. All the teachers for one subject get together, along with perhaps a factor of 2 more people who are experts in online delivery. They spend a factor of 4 more time designing
6 0.10959809 276 hunch net-2007-12-10-Learning Track of International Planning Competition
7 0.10715757 105 hunch net-2005-08-23-(Dis)similarities between academia and open source programmers
8 0.10390373 295 hunch net-2008-04-12-It Doesn’t Stop
9 0.1024079 297 hunch net-2008-04-22-Taking the next step
10 0.099001467 477 hunch net-2013-01-01-Deep Learning 2012
11 0.096667267 237 hunch net-2007-04-02-Contextual Scaling
12 0.096031085 464 hunch net-2012-05-03-Microsoft Research, New York City
13 0.096006408 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
14 0.093933381 257 hunch net-2007-07-28-Asking questions
15 0.093312964 454 hunch net-2012-01-30-ICML Posters and Scope
16 0.092684783 95 hunch net-2005-07-14-What Learning Theory might do
17 0.092253327 287 hunch net-2008-01-28-Sufficient Computation
18 0.0918134 475 hunch net-2012-10-26-ML Symposium and Strata-Hadoop World
19 0.087107718 380 hunch net-2009-11-29-AI Safety
20 0.08703766 437 hunch net-2011-07-10-ICML 2011 and the future
topicId topicWeight
[(0, 0.246), (1, 0.007), (2, -0.097), (3, 0.096), (4, -0.035), (5, -0.04), (6, 0.024), (7, 0.043), (8, -0.034), (9, -0.052), (10, -0.091), (11, -0.002), (12, -0.058), (13, 0.012), (14, -0.035), (15, 0.064), (16, 0.033), (17, 0.01), (18, 0.062), (19, 0.015), (20, 0.021), (21, 0.094), (22, -0.039), (23, 0.049), (24, 0.065), (25, 0.028), (26, 0.059), (27, -0.064), (28, 0.089), (29, 0.042), (30, 0.14), (31, 0.077), (32, 0.001), (33, -0.033), (34, 0.029), (35, -0.085), (36, 0.053), (37, -0.1), (38, 0.007), (39, -0.06), (40, -0.005), (41, -0.032), (42, 0.025), (43, -0.079), (44, 0.034), (45, 0.042), (46, 0.131), (47, 0.042), (48, 0.088), (49, -0.029)]
simIndex simValue blogId blogTitle
same-blog 1 0.96728855 424 hunch net-2011-02-17-What does Watson mean?
Introduction: Watson convincingly beat the best champion Jeopardy! players. The apparent significance of this varies hugely, depending on your background knowledge about the related machine learning, NLP, and search technology. For a random person, this might seem evidence of serious machine intelligence, while for people working on the system itself, it probably seems like a reasonably good assemblage of existing technologies with several twists to make the entire system work. Above all, I think we should congratulate the people who managed to put together and execute this project—many years of effort by a diverse set of highly skilled people were needed to make this happen. In academia, it’s pretty difficult for one professor to assemble that quantity of talent, and in industry it’s rarely the case that such a capable group has both a worthwhile project and the support needed to pursue something like this for several years before success. Alina invited me to the Jeopardy watching party
2 0.65886539 352 hunch net-2009-05-06-Machine Learning to AI
Introduction: I recently had fun discussions with both Vikash Mansinghka and Thomas Breuel about approaching AI with machine learning. The general interest in taking a crack at AI with machine learning seems to be rising on many fronts including DARPA . As a matter of history, there was a great deal of interest in AI which died down before I began research. There remain many projects and conferences spawned in this earlier AI wave, as well as a good bit of experience about what did not work, or at least did not work yet. Here are a few examples of failure modes that people seem to run into: Supply/Product confusion . Sometimes we think “Intelligences use X, so I’ll create X and have an Intelligence.” An example of this is the Cyc Project which inspires some people as “intelligences use ontologies, so I’ll create an ontology and a system using it to have an Intelligence.” The flaw here is that Intelligences create ontologies, which they use, and without the ability to create ont
3 0.64292777 120 hunch net-2005-10-10-Predictive Search is Coming
Introduction: “Search” is the other branch of AI research which has been succesful. Concrete examples include Deep Blue which beat the world chess champion and Chinook the champion checkers program. A set of core search techniques exist including A * , alpha-beta pruning, and others that can be applied to any of many different search problems. Given this, it may be surprising to learn that there has been relatively little succesful work on combining prediction and search. Given also that humans typically solve search problems using a number of predictive heuristics to narrow in on a solution, we might be surprised again. However, the big successful search-based systems have typically not used “smart” search algorithms. Insteady they have optimized for very fast search. This is not for lack of trying… many people have tried to synthesize search and prediction to various degrees of success. For example, Knightcap achieves good-but-not-stellar chess playing performance, and TD-gammon
4 0.62535024 353 hunch net-2009-05-08-Computability in Artificial Intelligence
Introduction: Normally I do not blog, but John kindly invited me to do so. Since computability issues play a major role in Artificial Intelligence and Machine Learning, I would like to take the opportunity to comment on that and raise some questions. The general attitude is that AI is about finding efficient smart algorithms. For large parts of machine learning, the same attitude is not too dangerous. If you want to concentrate on conceptual problems, simply become a statistician. There is no analogous escape for modern research on AI (as opposed to GOFAI rooted in logic). Let me show by analogy why limiting research to computational questions is bad for any field. Except in computer science, computational aspects play little role in the development of fundamental theories: Consider e.g. set theory with axiom of choice, foundations of logic, exact/full minimax for zero-sum games, quantum (field) theory, string theory, … Indeed, at least in physics, every new fundamental theory seems to
5 0.62429428 153 hunch net-2006-02-02-Introspectionism as a Disease
Introduction: In the AI-related parts of machine learning, it is often tempting to examine how you do things in order to imagine how a machine should do things. This is introspection, and it can easily go awry. I will call introspection gone awry introspectionism. Introspectionism is almost unique to AI (and the AI-related parts of machine learning) and it can lead to huge wasted effort in research. It’s easiest to show how introspectionism arises by an example. Suppose we want to solve the problem of navigating a robot from point A to point B given a camera. Then, the following research action plan might seem natural when you examine your own capabilities: Build an edge detector for still images. Build an object recognition system given the edge detector. Build a system to predict distance and orientation to objects given the object recognition system. Build a system to plan a path through the scene you construct from {object identification, distance, orientation} predictions.
6 0.6199469 287 hunch net-2008-01-28-Sufficient Computation
7 0.61967981 493 hunch net-2014-02-16-Metacademy: a package manager for knowledge
8 0.59829682 295 hunch net-2008-04-12-It Doesn’t Stop
9 0.59607327 3 hunch net-2005-01-24-The Humanloop Spectrum of Machine Learning
10 0.58628416 171 hunch net-2006-04-09-Progress in Machine Translation
11 0.56830144 380 hunch net-2009-11-29-AI Safety
12 0.53825045 477 hunch net-2013-01-01-Deep Learning 2012
13 0.5360986 128 hunch net-2005-11-05-The design of a computing cluster
14 0.52774549 276 hunch net-2007-12-10-Learning Track of International Planning Competition
15 0.51580626 168 hunch net-2006-04-02-Mad (Neuro)science
16 0.5035041 414 hunch net-2010-10-17-Partha Niyogi has died
17 0.49499276 95 hunch net-2005-07-14-What Learning Theory might do
18 0.49028054 297 hunch net-2008-04-22-Taking the next step
19 0.48844153 440 hunch net-2011-08-06-Interesting thing at UAI 2011
20 0.48432648 49 hunch net-2005-03-30-What can Type Theory teach us about Machine Learning?
topicId topicWeight
[(10, 0.026), (14, 0.014), (27, 0.216), (34, 0.021), (37, 0.02), (38, 0.046), (53, 0.038), (55, 0.118), (60, 0.219), (64, 0.03), (94, 0.117), (95, 0.048)]
simIndex simValue blogId blogTitle
same-blog 1 0.91379064 424 hunch net-2011-02-17-What does Watson mean?
Introduction: Watson convincingly beat the best champion Jeopardy! players. The apparent significance of this varies hugely, depending on your background knowledge about the related machine learning, NLP, and search technology. For a random person, this might seem evidence of serious machine intelligence, while for people working on the system itself, it probably seems like a reasonably good assemblage of existing technologies with several twists to make the entire system work. Above all, I think we should congratulate the people who managed to put together and execute this project—many years of effort by a diverse set of highly skilled people were needed to make this happen. In academia, it’s pretty difficult for one professor to assemble that quantity of talent, and in industry it’s rarely the case that such a capable group has both a worthwhile project and the support needed to pursue something like this for several years before success. Alina invited me to the Jeopardy watching party
2 0.88162917 160 hunch net-2006-03-02-Why do people count for learning?
Introduction: This post is about a confusion of mine with respect to many commonly used machine learning algorithms. A simple example where this comes up is Bayes net prediction. A Bayes net where a directed acyclic graph over a set of nodes where each node is associated with a variable and the edges indicate dependence. The joint probability distribution over the variables is given by a set of conditional probabilities. For example, a very simple Bayes net might express: P(A,B,C) = P(A | B,C)P(B)P(C) What I don’t understand is the mechanism commonly used to estimate P(A | B, C) . If we let N(A,B,C) be the number of instances of A,B,C then people sometimes form an estimate according to: P’(A | B,C) = N(A,B,C) / N /[N(B)/N * N(C)/N] = N(A,B,C) N /[N(B) N(C)] … in other words, people just estimate P’(A | B,C) according to observed relative frequencies. This is a reasonable technique when you have a large number of samples compared to the size space A x B x C , but it (nat
3 0.83784634 198 hunch net-2006-07-25-Upcoming conference
Introduction: The Workshop for Women in Machine Learning will be held in San Diego on October 4, 2006. For details see the workshop website: http://www.seas.upenn.edu/~wiml/
4 0.81011623 379 hunch net-2009-11-23-ICML 2009 Workshops (and Tutorials)
Introduction: I’m the workshops chair for ICML this year. As such, I would like to personally encourage people to consider running a workshop. My general view of workshops is that they are excellent as opportunities to discuss and develop research directions—some of my best work has come from collaborations at workshops and several workshops have substantially altered my thinking about various problems. My experience running workshops is that setting them up and making them fly often appears much harder than it actually is, and the workshops often come off much better than expected in the end. Submissions are due January 18, two weeks before papers. Similarly, Ben Taskar is looking for good tutorials , which is complementary. Workshops are about exploring a subject, while a tutorial is about distilling it down into an easily taught essence, a vital part of the research process. Tutorials are due February 13, two weeks after papers.
5 0.78489578 159 hunch net-2006-02-27-The Peekaboom Dataset
Introduction: Luis von Ahn ‘s Peekaboom project has yielded data (830MB). Peekaboom is the second attempt (after Espgame ) to produce a dataset which is useful for learning to solve vision problems based on voluntary game play. As a second attempt, it is meant to address all of the shortcomings of the first attempt. In particular: The locations of specific objects are provided by the data. The data collection is far more complete and extensive. The data consists of: The source images. (1 file per image, just short of 60K images.) The in-game events. (1 file per image, in a lispy syntax.) A description of the event language. There is a great deal of very specific and relevant data here so the hope that this will help solve vision problems seems quite reasonable.
6 0.77484745 343 hunch net-2009-02-18-Decision by Vetocracy
7 0.76589203 95 hunch net-2005-07-14-What Learning Theory might do
8 0.76202422 132 hunch net-2005-11-26-The Design of an Optimal Research Environment
9 0.76172543 286 hunch net-2008-01-25-Turing’s Club for Machine Learning
10 0.75823021 51 hunch net-2005-04-01-The Producer-Consumer Model of Research
11 0.75761062 437 hunch net-2011-07-10-ICML 2011 and the future
12 0.75722069 371 hunch net-2009-09-21-Netflix finishes (and starts)
13 0.75716096 360 hunch net-2009-06-15-In Active Learning, the question changes
14 0.75554568 423 hunch net-2011-02-02-User preferences for search engines
15 0.75412047 221 hunch net-2006-12-04-Structural Problems in NIPS Decision Making
16 0.75305521 204 hunch net-2006-08-28-Learning Theory standards for NIPS 2006
17 0.75235844 325 hunch net-2008-11-10-ICML Reviewing Criteria
18 0.75218403 320 hunch net-2008-10-14-Who is Responsible for a Bad Review?
19 0.75205839 194 hunch net-2006-07-11-New Models
20 0.75184774 453 hunch net-2012-01-28-Why COLT?