fast_ml fast_ml-2013 fast_ml-2013-34 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: You’ve heard about running things on a graphics card, but have you tried it? All you need to taste the speed is a Nvidia card and some software. We run experiments using Cudamat and Theano in Python. GPUs differ from CPUs in that they are optimized for throughput instead of latency. Here’s a metaphor: when you play an online game, you want fast response times, or low latency. Otherwise you get lag. However when you’re downloading a movie, you don’t care about response times, you care about bandwidth - that’s throughput. Massive computations are similiar to downloading a movie in this respect. The setup We’ll be testing things on a platform with an Intel Dual Core CPU @3Ghz and either GeForce 9600 GT, an old card, or GeForce GTX 550 Ti, a more recent card. See the appendix for more info on GPU hardware. Software we’ll be using is Python. On CPU it employs one core*. That is OK in everyday use because while one core is working hard, you can comfortably do something else becau
sentIndex sentText sentNum sentScore
1 All you need to taste the speed is a Nvidia card and some software. [sent-2, score-0.116]
2 GPUs differ from CPUs in that they are optimized for throughput instead of latency. [sent-4, score-0.056]
3 However when you’re downloading a movie, you don’t care about response times, you care about bandwidth - that’s throughput. [sent-7, score-0.082]
4 Massive computations are similiar to downloading a movie in this respect. [sent-8, score-0.087]
5 The setup We’ll be testing things on a platform with an Intel Dual Core CPU @3Ghz and either GeForce 9600 GT, an old card, or GeForce GTX 550 Ti, a more recent card. [sent-9, score-0.088]
6 That is OK in everyday use because while one core is working hard, you can comfortably do something else because there’s another core sitting idle. [sent-13, score-0.222]
7 Apparently there was a lot of changes from CUDA 4 to CUDA 5, and some existing software expects CUDA 4, so you might consider installing that older version . [sent-19, score-0.101]
8 Now put the cudamat dir in PYTHONPATH and you’re good to go. [sent-31, score-0.143]
9 022551'] Even on the old card training is roughly five times faster than on a CPU. [sent-86, score-0.322]
10 The more recent card is four times faster than that and 20 times faster than a CPU. [sent-87, score-0.493]
11 Conclusion On our rig, a GPU seems to be 20 times faster than a somewhat older CPU. [sent-90, score-0.211]
12 There are already quite a few CUDA-capable machine learning toolkits, mainly for neural networks and SVM, and we think that more are coming. [sent-91, score-0.099]
13 They have a few product lines: gaming cards, professional cards, mobile cards, high-end cards, etc. [sent-99, score-0.14]
14 And the answer is that GeForce gaming cards offer the best value for money. [sent-101, score-0.241]
15 He’s a professional and he says that they don’t bother with professional or high-end cards, they use GeForce. [sent-103, score-0.168]
16 Generally the feeling is that Fermi is better for computation as Kepler is optimized for gaming. [sent-106, score-0.102]
17 Long runs on GeForce 600 series GPU's, Windows app GPU Computation time (minutes) (min - max) CPU usage (min-max) NVIDIA GeForce GTX 650 1,725. [sent-119, score-0.063]
18 2%) Long runs on GeForce 500 series cards Windows app NVIDIA GeForce GTX 550Ti 1,933. [sent-155, score-0.248]
19 The downside of 500 series compared to 600 is high energy consumption. [sent-187, score-0.063]
20 From 560 upwards the cards have a double power connector. [sent-188, score-0.231]
wordName wordTfidf (topN-words)
[('gtx', 0.501), ('geforce', 0.429), ('nvidia', 0.348), ('cuda', 0.259), ('cards', 0.185), ('meanvisb', 0.162), ('gpu', 0.153), ('cudamat', 0.143), ('valid', 0.143), ('cpu', 0.142), ('card', 0.116), ('core', 0.111), ('times', 0.105), ('theano', 0.102), ('professional', 0.084), ('ti', 0.084), ('pythonpath', 0.07), ('series', 0.063), ('windows', 0.062), ('gt', 0.061), ('faster', 0.06), ('library', 0.059), ('appendix', 0.056), ('fermi', 0.056), ('gaming', 0.056), ('kepler', 0.056), ('optimized', 0.056), ('toolkit', 0.056), ('seconds', 0.055), ('system', 0.055), ('installing', 0.055), ('svm', 0.055), ('running', 0.05), ('networks', 0.05), ('neural', 0.049), ('recent', 0.047), ('movie', 0.046), ('computation', 0.046), ('gpus', 0.046), ('older', 0.046), ('power', 0.046), ('terrible', 0.046), ('test', 0.041), ('downloading', 0.041), ('minutes', 0.041), ('old', 0.041), ('programming', 0.041), ('response', 0.041), ('include', 0.041), ('nn', 0.041)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999994 34 fast ml-2013-07-14-Running things on a GPU
Introduction: You’ve heard about running things on a graphics card, but have you tried it? All you need to taste the speed is a Nvidia card and some software. We run experiments using Cudamat and Theano in Python. GPUs differ from CPUs in that they are optimized for throughput instead of latency. Here’s a metaphor: when you play an online game, you want fast response times, or low latency. Otherwise you get lag. However when you’re downloading a movie, you don’t care about response times, you care about bandwidth - that’s throughput. Massive computations are similiar to downloading a movie in this respect. The setup We’ll be testing things on a platform with an Intel Dual Core CPU @3Ghz and either GeForce 9600 GT, an old card, or GeForce GTX 550 Ti, a more recent card. See the appendix for more info on GPU hardware. Software we’ll be using is Python. On CPU it employs one core*. That is OK in everyday use because while one core is working hard, you can comfortably do something else becau
2 0.47578853 44 fast ml-2013-11-18-CUDA on a Linux laptop
Introduction: After testing CUDA on a desktop , we now switch to a Linux laptop with 64-bit Xubuntu. Getting CUDA to work is harder here. Will the effort be worth the results? If you have a laptop with a Nvidia card, the thing probably uses it for 3D graphics and Intel’s built-in unit for everything else. This technology is known as Optimus and it happens to make things anything but easy for running CUDA on Linux. The problem is with GPU drivers, specifically between Linux being open-source and Nvidia drivers being not. This strained relation at one time prompted Linus Torvalds to give Nvidia a finger with great passion. Installing GPU drivers Here’s a solution for the driver problem. You need a package called bumblebee . It makes a Nvidia card accessible to your apps. To install drivers and bumblebee , try something along these lines: sudo apt-get install nvidia-current-updates sudo apt-get install bumblebee Note that you don’t need drivers that come with a specific CUDA rel
3 0.10195354 40 fast ml-2013-10-06-Pylearn2 in practice
Introduction: What do you get when you mix one part brilliant and one part daft? You get Pylearn2, a cutting edge neural networks library from Montreal that’s rather hard to use. Here we’ll show how to get through the daft part with your mental health relatively intact. Pylearn2 comes from the Lisa Lab in Montreal , led by Yoshua Bengio. Those are pretty smart guys and they concern themselves with deep learning. Recently they published a paper entitled Pylearn2: a machine learning research library [arxiv] . Here’s a quote: Pylearn2 is a machine learning research library - its users are researchers . This means (…) it is acceptable to assume that the user has some technical sophistication and knowledge of machine learning. The word research is possibly the most common word in the paper. There’s a reason for that: the library is certainly not production-ready. OK, it’s not that bad. There are only two difficult things: getting your data in getting predictions out What’
4 0.079324201 45 fast ml-2013-11-27-Object recognition in images with cuda-convnet
Introduction: Object recognition in images is where deep learning, and specifically convolutional neural networks, are often applied and benchmarked these days. To get a piece of the action, we’ll be using Alex Krizhevsky’s cuda-convnet , a shining diamond of machine learning software, in a Kaggle competition. Continuing to run things on a GPU, we turn to applying convolutional neural networks for object recognition. This kind of network was developed by Yann LeCun and it’s powerful, but a bit complicated: Image credit: EBLearn tutorial A typical convolutional network has two parts. The first is responsible for feature extraction and consists of one or more pairs of convolution and subsampling/max-pooling layers, as you can see above. The second part is just a classic fully-connected multilayer perceptron taking extracted features as input. For a detailed explanation of all this see unit 9 in Hugo LaRochelle’s neural networks course . Daniel Nouri has an interesting story about
5 0.060447652 27 fast ml-2013-05-01-Deep learning made easy
Introduction: As usual, there’s an interesting competition at Kaggle: The Black Box. It’s connected to ICML 2013 Workshop on Challenges in Representation Learning, held by the deep learning guys from Montreal. There are a couple benchmarks for this competition and the best one is unusually hard to beat 1 - only less than a fourth of those taking part managed to do so. We’re among them. Here’s how. The key ingredient in our success is a recently developed secret Stanford technology for deep unsupervised learning: sparse filtering by Jiquan Ngiam et al. Actually, it’s not secret. It’s available at Github , and has one or two very appealling properties. Let us explain. The main idea of deep unsupervised learning, as we understand it, is feature extraction. One of the most common applications is in multimedia. The reason for that is that multimedia tasks, for example object recognition, are easy for humans, but difficult for computers 2 . Geoff Hinton from Toronto talks about two ends
6 0.053518511 62 fast ml-2014-05-26-Yann LeCun's answers from the Reddit AMA
7 0.051891316 54 fast ml-2014-03-06-PyBrain - a simple neural networks library in Python
8 0.050767422 57 fast ml-2014-04-01-Exclusive Geoff Hinton interview
9 0.042948652 15 fast ml-2013-01-07-Machine learning courses online
10 0.0414006 3 fast ml-2012-09-01-Running Unix apps on Windows
11 0.040776998 49 fast ml-2014-01-10-Classifying images with a pre-trained deep network
12 0.040674809 46 fast ml-2013-12-07-13 NIPS papers that caught our eye
13 0.039265204 5 fast ml-2012-09-19-Best Buy mobile contest - big data
14 0.037287362 52 fast ml-2014-02-02-Yesterday a kaggler, today a Kaggle master: a wrap-up of the cats and dogs competition
15 0.037034847 22 fast ml-2013-03-07-Choosing a machine learning algorithm
16 0.034980949 12 fast ml-2012-12-21-Tuning hyperparams automatically with Spearmint
17 0.034305476 35 fast ml-2013-08-12-Accelerometer Biometric Competition
18 0.03270255 16 fast ml-2013-01-12-Intro to random forests
19 0.032486524 55 fast ml-2014-03-20-Good representations, distance, metric learning and supervised dimensionality reduction
20 0.031552672 7 fast ml-2012-10-05-Predicting closed questions on Stack Overflow
topicId topicWeight
[(0, 0.169), (1, 0.13), (2, 0.182), (3, -0.065), (4, -0.649), (5, 0.298), (6, 0.302), (7, -0.147), (8, 0.001), (9, -0.041), (10, -0.021), (11, -0.023), (12, -0.02), (13, -0.121), (14, 0.046), (15, -0.008), (16, -0.027), (17, 0.041), (18, -0.024), (19, -0.019), (20, -0.016), (21, -0.015), (22, -0.049), (23, 0.004), (24, 0.021), (25, -0.031), (26, -0.002), (27, -0.025), (28, -0.003), (29, 0.027), (30, 0.022), (31, -0.022), (32, 0.031), (33, -0.005), (34, 0.025), (35, 0.018), (36, 0.018), (37, 0.023), (38, -0.016), (39, 0.027), (40, 0.015), (41, 0.027), (42, -0.002), (43, 0.01), (44, 0.021), (45, 0.014), (46, 0.038), (47, 0.006), (48, -0.018), (49, 0.005)]
simIndex simValue blogId blogTitle
same-blog 1 0.98782837 34 fast ml-2013-07-14-Running things on a GPU
Introduction: You’ve heard about running things on a graphics card, but have you tried it? All you need to taste the speed is a Nvidia card and some software. We run experiments using Cudamat and Theano in Python. GPUs differ from CPUs in that they are optimized for throughput instead of latency. Here’s a metaphor: when you play an online game, you want fast response times, or low latency. Otherwise you get lag. However when you’re downloading a movie, you don’t care about response times, you care about bandwidth - that’s throughput. Massive computations are similiar to downloading a movie in this respect. The setup We’ll be testing things on a platform with an Intel Dual Core CPU @3Ghz and either GeForce 9600 GT, an old card, or GeForce GTX 550 Ti, a more recent card. See the appendix for more info on GPU hardware. Software we’ll be using is Python. On CPU it employs one core*. That is OK in everyday use because while one core is working hard, you can comfortably do something else becau
2 0.97079796 44 fast ml-2013-11-18-CUDA on a Linux laptop
Introduction: After testing CUDA on a desktop , we now switch to a Linux laptop with 64-bit Xubuntu. Getting CUDA to work is harder here. Will the effort be worth the results? If you have a laptop with a Nvidia card, the thing probably uses it for 3D graphics and Intel’s built-in unit for everything else. This technology is known as Optimus and it happens to make things anything but easy for running CUDA on Linux. The problem is with GPU drivers, specifically between Linux being open-source and Nvidia drivers being not. This strained relation at one time prompted Linus Torvalds to give Nvidia a finger with great passion. Installing GPU drivers Here’s a solution for the driver problem. You need a package called bumblebee . It makes a Nvidia card accessible to your apps. To install drivers and bumblebee , try something along these lines: sudo apt-get install nvidia-current-updates sudo apt-get install bumblebee Note that you don’t need drivers that come with a specific CUDA rel
3 0.1675256 40 fast ml-2013-10-06-Pylearn2 in practice
Introduction: What do you get when you mix one part brilliant and one part daft? You get Pylearn2, a cutting edge neural networks library from Montreal that’s rather hard to use. Here we’ll show how to get through the daft part with your mental health relatively intact. Pylearn2 comes from the Lisa Lab in Montreal , led by Yoshua Bengio. Those are pretty smart guys and they concern themselves with deep learning. Recently they published a paper entitled Pylearn2: a machine learning research library [arxiv] . Here’s a quote: Pylearn2 is a machine learning research library - its users are researchers . This means (…) it is acceptable to assume that the user has some technical sophistication and knowledge of machine learning. The word research is possibly the most common word in the paper. There’s a reason for that: the library is certainly not production-ready. OK, it’s not that bad. There are only two difficult things: getting your data in getting predictions out What’
4 0.13356182 45 fast ml-2013-11-27-Object recognition in images with cuda-convnet
Introduction: Object recognition in images is where deep learning, and specifically convolutional neural networks, are often applied and benchmarked these days. To get a piece of the action, we’ll be using Alex Krizhevsky’s cuda-convnet , a shining diamond of machine learning software, in a Kaggle competition. Continuing to run things on a GPU, we turn to applying convolutional neural networks for object recognition. This kind of network was developed by Yann LeCun and it’s powerful, but a bit complicated: Image credit: EBLearn tutorial A typical convolutional network has two parts. The first is responsible for feature extraction and consists of one or more pairs of convolution and subsampling/max-pooling layers, as you can see above. The second part is just a classic fully-connected multilayer perceptron taking extracted features as input. For a detailed explanation of all this see unit 9 in Hugo LaRochelle’s neural networks course . Daniel Nouri has an interesting story about
5 0.11367229 54 fast ml-2014-03-06-PyBrain - a simple neural networks library in Python
Introduction: We have already written a few articles about Pylearn2 . Today we’ll look at PyBrain. It is another Python neural networks library, and this is where similiarites end. They’re like day and night: Pylearn2 - Byzantinely complicated, PyBrain - simple. We attempted to train a regression model and succeeded at first take (more on this below). Try this with Pylearn2. While there are a few machine learning libraries out there, PyBrain aims to be a very easy-to-use modular library that can be used by entry-level students but still offers the flexibility and algorithms for state-of-the-art research. The library features classic perceptron as well as recurrent neural networks and other things, some of which, for example Evolino , would be hard to find elsewhere. On the downside, PyBrain feels unfinished, abandoned. It is no longer actively developed and the documentation is skimpy. There’s no modern gimmicks like dropout and rectified linear units - just good ol’ sigmoid and ta
6 0.10882488 27 fast ml-2013-05-01-Deep learning made easy
7 0.10792731 49 fast ml-2014-01-10-Classifying images with a pre-trained deep network
8 0.10439789 57 fast ml-2014-04-01-Exclusive Geoff Hinton interview
9 0.10100523 62 fast ml-2014-05-26-Yann LeCun's answers from the Reddit AMA
10 0.092157125 5 fast ml-2012-09-19-Best Buy mobile contest - big data
11 0.087951221 46 fast ml-2013-12-07-13 NIPS papers that caught our eye
12 0.082985081 15 fast ml-2013-01-07-Machine learning courses online
13 0.080469139 18 fast ml-2013-01-17-A very fast denoising autoencoder
14 0.079991177 12 fast ml-2012-12-21-Tuning hyperparams automatically with Spearmint
15 0.076906696 22 fast ml-2013-03-07-Choosing a machine learning algorithm
16 0.075828478 24 fast ml-2013-03-25-Dimensionality reduction for sparse binary data - an overview
17 0.074786708 53 fast ml-2014-02-20-Are stocks predictable?
18 0.07425572 38 fast ml-2013-09-09-Predicting solar energy from weather forecasts plus a NetCDF4 tutorial
19 0.072682694 7 fast ml-2012-10-05-Predicting closed questions on Stack Overflow
20 0.069731951 35 fast ml-2013-08-12-Accelerometer Biometric Competition
topicId topicWeight
[(26, 0.037), (30, 0.019), (31, 0.059), (35, 0.024), (51, 0.186), (55, 0.02), (65, 0.285), (69, 0.081), (71, 0.032), (73, 0.015), (78, 0.012), (79, 0.022), (84, 0.019), (86, 0.012), (97, 0.025), (99, 0.044)]
simIndex simValue blogId blogTitle
same-blog 1 0.90279192 34 fast ml-2013-07-14-Running things on a GPU
Introduction: You’ve heard about running things on a graphics card, but have you tried it? All you need to taste the speed is a Nvidia card and some software. We run experiments using Cudamat and Theano in Python. GPUs differ from CPUs in that they are optimized for throughput instead of latency. Here’s a metaphor: when you play an online game, you want fast response times, or low latency. Otherwise you get lag. However when you’re downloading a movie, you don’t care about response times, you care about bandwidth - that’s throughput. Massive computations are similiar to downloading a movie in this respect. The setup We’ll be testing things on a platform with an Intel Dual Core CPU @3Ghz and either GeForce 9600 GT, an old card, or GeForce GTX 550 Ti, a more recent card. See the appendix for more info on GPU hardware. Software we’ll be using is Python. On CPU it employs one core*. That is OK in everyday use because while one core is working hard, you can comfortably do something else becau
2 0.71639812 21 fast ml-2013-02-27-Dimensionality reduction for sparse binary data
Introduction: Much of data in machine learning is sparse, that is mostly zeros, and often binary. The phenomenon may result from converting categorical variables to one-hot vectors, and from converting text to bag-of-words representation. If each feature is binary - either zero or one - then it holds exactly one bit of information. Surely we could somehow compress such data to fewer real numbers. To do this, we turn to topic models, an area of research with roots in natural language processing. In NLP, a training set is called a corpus, and each document is like a row in the set. A document might be three pages of text, or just a few words, as in a tweet. The idea of topic modelling is that you can group words in your corpus into relatively few topics and represent each document as a mixture of these topics. It’s attractive because you can interpret the model by looking at words that form the topics. Sometimes they seem meaningful, sometimes not. A meaningful topic might be, for example: “cric
3 0.65610164 44 fast ml-2013-11-18-CUDA on a Linux laptop
Introduction: After testing CUDA on a desktop , we now switch to a Linux laptop with 64-bit Xubuntu. Getting CUDA to work is harder here. Will the effort be worth the results? If you have a laptop with a Nvidia card, the thing probably uses it for 3D graphics and Intel’s built-in unit for everything else. This technology is known as Optimus and it happens to make things anything but easy for running CUDA on Linux. The problem is with GPU drivers, specifically between Linux being open-source and Nvidia drivers being not. This strained relation at one time prompted Linus Torvalds to give Nvidia a finger with great passion. Installing GPU drivers Here’s a solution for the driver problem. You need a package called bumblebee . It makes a Nvidia card accessible to your apps. To install drivers and bumblebee , try something along these lines: sudo apt-get install nvidia-current-updates sudo apt-get install bumblebee Note that you don’t need drivers that come with a specific CUDA rel
4 0.58912456 2 fast ml-2012-08-27-Kaggle job recommendation challenge
Introduction: This is an introduction to Kaggle job recommendation challenge . It looks a lot like a typical collaborative filtering thing (with a lot of extra information), but not quite. Spot these two big differences: There are no explicit ratings. Instead, there’s info about which jobs user applied to. This is known as one-class collaborative filtering (OCCF), or learning from positive-only feedback. If you want to dig deeper into the subject, there have been already contests with positive feedback only, for example track two of Yahoo KDD Cup or Millions Songs Dataset Challenge at Kaggle (both about songs). The second difference is less apparent. When you look at test users (that is, the users that we are asked to recommend jobs for), only about half of them made at least one application. For the other half, no data and no collaborative filtering. For the users we have applications data for, it’s very sparse, so we would like to use CF, because it does well in similar se
5 0.32012573 9 fast ml-2012-10-25-So you want to work for Facebook
Introduction: Good news, everyone! There’s a new contest on Kaggle - Facebook is looking for talent . They won’t pay, but just might interview. This post is in a way a bonus for active readers because most visitors of fastml.com originally come from Kaggle forums. For this competition the forums are disabled to encourage own work . To honor this, we won’t publish any code. But own work doesn’t mean original work , and we wouldn’t want to reinvent the wheel, would we? The contest differs substantially from a Kaggle stereotype, if there is such a thing, in three major ways: there’s no money prizes, as mentioned above it’s not a real world problem, but rather an assignment to screen job candidates (this has important consequences, described below) it’s not a typical machine learning project, but rather a broader AI exercise You are given a graph of the internet, actually a snapshot of the graph for each of 15 time steps. You are also given a bunch of paths in this graph, which a
6 0.31311306 48 fast ml-2013-12-28-Regularizing neural networks with dropout and with DropConnect
7 0.31111705 45 fast ml-2013-11-27-Object recognition in images with cuda-convnet
8 0.30806142 24 fast ml-2013-03-25-Dimensionality reduction for sparse binary data - an overview
9 0.30642053 40 fast ml-2013-10-06-Pylearn2 in practice
10 0.29379031 49 fast ml-2014-01-10-Classifying images with a pre-trained deep network
11 0.29367873 54 fast ml-2014-03-06-PyBrain - a simple neural networks library in Python
12 0.2924003 12 fast ml-2012-12-21-Tuning hyperparams automatically with Spearmint
13 0.28596818 43 fast ml-2013-11-02-Maxing out the digits
14 0.28395805 19 fast ml-2013-02-07-The secret of the big guys
15 0.27862662 18 fast ml-2013-01-17-A very fast denoising autoencoder
16 0.27814147 16 fast ml-2013-01-12-Intro to random forests
17 0.27735472 27 fast ml-2013-05-01-Deep learning made easy
18 0.27629697 25 fast ml-2013-04-10-Gender discrimination
19 0.27548665 61 fast ml-2014-05-08-Impute missing values with Amelia
20 0.27391812 26 fast ml-2013-04-17-Regression as classification