fast_ml fast_ml-2013 fast_ml-2013-44 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: After testing CUDA on a desktop , we now switch to a Linux laptop with 64-bit Xubuntu. Getting CUDA to work is harder here. Will the effort be worth the results? If you have a laptop with a Nvidia card, the thing probably uses it for 3D graphics and Intel’s built-in unit for everything else. This technology is known as Optimus and it happens to make things anything but easy for running CUDA on Linux. The problem is with GPU drivers, specifically between Linux being open-source and Nvidia drivers being not. This strained relation at one time prompted Linus Torvalds to give Nvidia a finger with great passion. Installing GPU drivers Here’s a solution for the driver problem. You need a package called bumblebee . It makes a Nvidia card accessible to your apps. To install drivers and bumblebee , try something along these lines: sudo apt-get install nvidia-current-updates sudo apt-get install bumblebee Note that you don’t need drivers that come with a specific CUDA rel
sentIndex sentText sentNum sentScore
1 After testing CUDA on a desktop , we now switch to a Linux laptop with 64-bit Xubuntu. [sent-1, score-0.208]
2 If you have a laptop with a Nvidia card, the thing probably uses it for 3D graphics and Intel’s built-in unit for everything else. [sent-4, score-0.263]
3 This technology is known as Optimus and it happens to make things anything but easy for running CUDA on Linux. [sent-5, score-0.096]
4 The problem is with GPU drivers, specifically between Linux being open-source and Nvidia drivers being not. [sent-6, score-0.307]
5 This strained relation at one time prompted Linus Torvalds to give Nvidia a finger with great passion. [sent-7, score-0.12]
6 Installing GPU drivers Here’s a solution for the driver problem. [sent-8, score-0.307]
7 To install drivers and bumblebee , try something along these lines: sudo apt-get install nvidia-current-updates sudo apt-get install bumblebee Note that you don’t need drivers that come with a specific CUDA release, just Nvidia’s proprietary drivers. [sent-11, score-2.077]
8 Now, usually you don’t log in as root, but you need to run bumblebee’s app, optirun , as root: optirun --no-xorgThe easiest way to do this is to execute sudo su to get a superuser’s shell. [sent-12, score-0.766]
9 That’s because (on Ubuntu) sudo has restrictive security measures that complicate the setup. [sent-13, score-0.245]
10 Your CUDA apps will need a proper environment, including PATH, LD_LIBRARY_PATH etc. [sent-14, score-0.09]
11 There’s /etc/environment , but it’s not a real shell script so we don’t like it. [sent-17, score-0.051]
12 To sum up, we set an alias we then use to execute commands via optirun : alias opti='optirun --no-xorg' opti python some_cuda_app. [sent-19, score-0.716]
13 py Timing tests We’ll run the same tests as in Running things on a GPU , to see how timings compare. [sent-20, score-0.204]
14 The processor is much faster and the card bit slower than previously: the test laptop has an Intel i7-3610QM CPU and a Nvidia GeForce GT 650M video card. [sent-21, score-0.494]
15 CPU: 95 seconds per iteration GeForce GT 650M: 14 seconds (nearly seven times faster) Theano / GSN The output snippets for CPU and GPU, respectively: 1 Train : 0. [sent-27, score-0.334]
16 022535'] For some reason, on the GPU the first iteration goes way faster than the next. [sent-69, score-0.163]
17 Those next iterations are only two times faster compared to the CPU. [sent-70, score-0.134]
18 To sum up, it seems that on a laptop, a good strategy would be to employ all CPU cores. [sent-74, score-0.163]
wordName wordTfidf (topN-words)
[('drivers', 0.307), ('meanvisb', 0.306), ('nvidia', 0.306), ('valid', 0.27), ('gpu', 0.262), ('bumblebee', 0.245), ('sudo', 0.245), ('cpu', 0.208), ('cuda', 0.204), ('optirun', 0.184), ('laptop', 0.163), ('card', 0.153), ('geforce', 0.135), ('alias', 0.123), ('opti', 0.123), ('root', 0.122), ('install', 0.112), ('intel', 0.102), ('tests', 0.102), ('execute', 0.102), ('test', 0.09), ('cudamat', 0.09), ('gt', 0.09), ('faster', 0.088), ('linux', 0.081), ('seconds', 0.081), ('iteration', 0.075), ('time', 0.069), ('sum', 0.061), ('train', 0.056), ('easiest', 0.051), ('technology', 0.051), ('proprietary', 0.051), ('employ', 0.051), ('cards', 0.051), ('graphics', 0.051), ('relation', 0.051), ('snippets', 0.051), ('shell', 0.051), ('specific', 0.051), ('strategy', 0.051), ('uses', 0.049), ('times', 0.046), ('respectively', 0.045), ('apps', 0.045), ('desktop', 0.045), ('along', 0.045), ('happens', 0.045), ('proper', 0.045), ('effort', 0.045)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999976 44 fast ml-2013-11-18-CUDA on a Linux laptop
Introduction: After testing CUDA on a desktop , we now switch to a Linux laptop with 64-bit Xubuntu. Getting CUDA to work is harder here. Will the effort be worth the results? If you have a laptop with a Nvidia card, the thing probably uses it for 3D graphics and Intel’s built-in unit for everything else. This technology is known as Optimus and it happens to make things anything but easy for running CUDA on Linux. The problem is with GPU drivers, specifically between Linux being open-source and Nvidia drivers being not. This strained relation at one time prompted Linus Torvalds to give Nvidia a finger with great passion. Installing GPU drivers Here’s a solution for the driver problem. You need a package called bumblebee . It makes a Nvidia card accessible to your apps. To install drivers and bumblebee , try something along these lines: sudo apt-get install nvidia-current-updates sudo apt-get install bumblebee Note that you don’t need drivers that come with a specific CUDA rel
2 0.47578853 34 fast ml-2013-07-14-Running things on a GPU
Introduction: You’ve heard about running things on a graphics card, but have you tried it? All you need to taste the speed is a Nvidia card and some software. We run experiments using Cudamat and Theano in Python. GPUs differ from CPUs in that they are optimized for throughput instead of latency. Here’s a metaphor: when you play an online game, you want fast response times, or low latency. Otherwise you get lag. However when you’re downloading a movie, you don’t care about response times, you care about bandwidth - that’s throughput. Massive computations are similiar to downloading a movie in this respect. The setup We’ll be testing things on a platform with an Intel Dual Core CPU @3Ghz and either GeForce 9600 GT, an old card, or GeForce GTX 550 Ti, a more recent card. See the appendix for more info on GPU hardware. Software we’ll be using is Python. On CPU it employs one core*. That is OK in everyday use because while one core is working hard, you can comfortably do something else becau
3 0.065605067 45 fast ml-2013-11-27-Object recognition in images with cuda-convnet
Introduction: Object recognition in images is where deep learning, and specifically convolutional neural networks, are often applied and benchmarked these days. To get a piece of the action, we’ll be using Alex Krizhevsky’s cuda-convnet , a shining diamond of machine learning software, in a Kaggle competition. Continuing to run things on a GPU, we turn to applying convolutional neural networks for object recognition. This kind of network was developed by Yann LeCun and it’s powerful, but a bit complicated: Image credit: EBLearn tutorial A typical convolutional network has two parts. The first is responsible for feature extraction and consists of one or more pairs of convolution and subsampling/max-pooling layers, as you can see above. The second part is just a classic fully-connected multilayer perceptron taking extracted features as input. For a detailed explanation of all this see unit 9 in Hugo LaRochelle’s neural networks course . Daniel Nouri has an interesting story about
4 0.065045699 3 fast ml-2012-09-01-Running Unix apps on Windows
Introduction: When it comes to machine learning, most software seems to be in either Python, Matlab or R. Plus native apps, that is, compiled C/C++. These are fastest. Most of them is written for Unix environments, for example Linux or MacOS. So how do you run them on your computer if you have Windows installed? Back in the day, you re-partitioned your hard drive and installed Linux alongside Windows. The added thrill was, if something went wrong, your computer wouldn’t boot. Now it’s easier. You just run Linux inside Windows, using what’s called a virtual machine. You need virtualization software and a machine image to do so. Most popular software seems to be VMware. There is also VirtualBox - it is able to run VMware images. We have experience with WMware mostly, so this is what we’ll refer to. VMware player is free to download and use. There are also many images available, of various flavours of Linux and other operating systems. In fact, you can run Windows inside Linux if you wish
5 0.062124342 40 fast ml-2013-10-06-Pylearn2 in practice
Introduction: What do you get when you mix one part brilliant and one part daft? You get Pylearn2, a cutting edge neural networks library from Montreal that’s rather hard to use. Here we’ll show how to get through the daft part with your mental health relatively intact. Pylearn2 comes from the Lisa Lab in Montreal , led by Yoshua Bengio. Those are pretty smart guys and they concern themselves with deep learning. Recently they published a paper entitled Pylearn2: a machine learning research library [arxiv] . Here’s a quote: Pylearn2 is a machine learning research library - its users are researchers . This means (…) it is acceptable to assume that the user has some technical sophistication and knowledge of machine learning. The word research is possibly the most common word in the paper. There’s a reason for that: the library is certainly not production-ready. OK, it’s not that bad. There are only two difficult things: getting your data in getting predictions out What’
6 0.045379922 27 fast ml-2013-05-01-Deep learning made easy
7 0.044442572 51 fast ml-2014-01-25-Why IPy: reasons for using IPython interactively
8 0.038430762 20 fast ml-2013-02-18-Predicting advertised salaries
9 0.037969463 49 fast ml-2014-01-10-Classifying images with a pre-trained deep network
10 0.034685798 62 fast ml-2014-05-26-Yann LeCun's answers from the Reddit AMA
11 0.032133609 13 fast ml-2012-12-27-Spearmint with a random forest
12 0.03195972 43 fast ml-2013-11-02-Maxing out the digits
13 0.031901091 56 fast ml-2014-03-31-If you use R, you may want RStudio
14 0.030936401 61 fast ml-2014-05-08-Impute missing values with Amelia
15 0.03024042 12 fast ml-2012-12-21-Tuning hyperparams automatically with Spearmint
16 0.029338958 25 fast ml-2013-04-10-Gender discrimination
17 0.027952218 14 fast ml-2013-01-04-Madelon: Spearmint's revenge
18 0.026423598 53 fast ml-2014-02-20-Are stocks predictable?
19 0.025474466 33 fast ml-2013-07-09-Introducing phraug
20 0.024896184 55 fast ml-2014-03-20-Good representations, distance, metric learning and supervised dimensionality reduction
topicId topicWeight
[(0, 0.135), (1, 0.108), (2, 0.135), (3, -0.067), (4, -0.673), (5, 0.305), (6, 0.299), (7, -0.153), (8, 0.016), (9, -0.031), (10, 0.042), (11, -0.035), (12, 0.019), (13, -0.096), (14, 0.03), (15, 0.007), (16, -0.021), (17, 0.064), (18, -0.008), (19, 0.016), (20, 0.026), (21, -0.019), (22, -0.025), (23, 0.014), (24, 0.041), (25, -0.018), (26, 0.049), (27, -0.048), (28, 0.002), (29, 0.007), (30, 0.016), (31, -0.009), (32, -0.026), (33, 0.004), (34, 0.026), (35, 0.01), (36, -0.01), (37, 0.009), (38, -0.003), (39, -0.018), (40, 0.0), (41, -0.006), (42, -0.037), (43, 0.011), (44, 0.05), (45, 0.015), (46, 0.021), (47, -0.04), (48, -0.027), (49, 0.008)]
simIndex simValue blogId blogTitle
same-blog 1 0.98867881 44 fast ml-2013-11-18-CUDA on a Linux laptop
Introduction: After testing CUDA on a desktop , we now switch to a Linux laptop with 64-bit Xubuntu. Getting CUDA to work is harder here. Will the effort be worth the results? If you have a laptop with a Nvidia card, the thing probably uses it for 3D graphics and Intel’s built-in unit for everything else. This technology is known as Optimus and it happens to make things anything but easy for running CUDA on Linux. The problem is with GPU drivers, specifically between Linux being open-source and Nvidia drivers being not. This strained relation at one time prompted Linus Torvalds to give Nvidia a finger with great passion. Installing GPU drivers Here’s a solution for the driver problem. You need a package called bumblebee . It makes a Nvidia card accessible to your apps. To install drivers and bumblebee , try something along these lines: sudo apt-get install nvidia-current-updates sudo apt-get install bumblebee Note that you don’t need drivers that come with a specific CUDA rel
2 0.95914525 34 fast ml-2013-07-14-Running things on a GPU
Introduction: You’ve heard about running things on a graphics card, but have you tried it? All you need to taste the speed is a Nvidia card and some software. We run experiments using Cudamat and Theano in Python. GPUs differ from CPUs in that they are optimized for throughput instead of latency. Here’s a metaphor: when you play an online game, you want fast response times, or low latency. Otherwise you get lag. However when you’re downloading a movie, you don’t care about response times, you care about bandwidth - that’s throughput. Massive computations are similiar to downloading a movie in this respect. The setup We’ll be testing things on a platform with an Intel Dual Core CPU @3Ghz and either GeForce 9600 GT, an old card, or GeForce GTX 550 Ti, a more recent card. See the appendix for more info on GPU hardware. Software we’ll be using is Python. On CPU it employs one core*. That is OK in everyday use because while one core is working hard, you can comfortably do something else becau
3 0.10884231 3 fast ml-2012-09-01-Running Unix apps on Windows
Introduction: When it comes to machine learning, most software seems to be in either Python, Matlab or R. Plus native apps, that is, compiled C/C++. These are fastest. Most of them is written for Unix environments, for example Linux or MacOS. So how do you run them on your computer if you have Windows installed? Back in the day, you re-partitioned your hard drive and installed Linux alongside Windows. The added thrill was, if something went wrong, your computer wouldn’t boot. Now it’s easier. You just run Linux inside Windows, using what’s called a virtual machine. You need virtualization software and a machine image to do so. Most popular software seems to be VMware. There is also VirtualBox - it is able to run VMware images. We have experience with WMware mostly, so this is what we’ll refer to. VMware player is free to download and use. There are also many images available, of various flavours of Linux and other operating systems. In fact, you can run Windows inside Linux if you wish
4 0.10633735 45 fast ml-2013-11-27-Object recognition in images with cuda-convnet
Introduction: Object recognition in images is where deep learning, and specifically convolutional neural networks, are often applied and benchmarked these days. To get a piece of the action, we’ll be using Alex Krizhevsky’s cuda-convnet , a shining diamond of machine learning software, in a Kaggle competition. Continuing to run things on a GPU, we turn to applying convolutional neural networks for object recognition. This kind of network was developed by Yann LeCun and it’s powerful, but a bit complicated: Image credit: EBLearn tutorial A typical convolutional network has two parts. The first is responsible for feature extraction and consists of one or more pairs of convolution and subsampling/max-pooling layers, as you can see above. The second part is just a classic fully-connected multilayer perceptron taking extracted features as input. For a detailed explanation of all this see unit 9 in Hugo LaRochelle’s neural networks course . Daniel Nouri has an interesting story about
5 0.086378977 40 fast ml-2013-10-06-Pylearn2 in practice
Introduction: What do you get when you mix one part brilliant and one part daft? You get Pylearn2, a cutting edge neural networks library from Montreal that’s rather hard to use. Here we’ll show how to get through the daft part with your mental health relatively intact. Pylearn2 comes from the Lisa Lab in Montreal , led by Yoshua Bengio. Those are pretty smart guys and they concern themselves with deep learning. Recently they published a paper entitled Pylearn2: a machine learning research library [arxiv] . Here’s a quote: Pylearn2 is a machine learning research library - its users are researchers . This means (…) it is acceptable to assume that the user has some technical sophistication and knowledge of machine learning. The word research is possibly the most common word in the paper. There’s a reason for that: the library is certainly not production-ready. OK, it’s not that bad. There are only two difficult things: getting your data in getting predictions out What’
6 0.085998349 20 fast ml-2013-02-18-Predicting advertised salaries
7 0.082684562 51 fast ml-2014-01-25-Why IPy: reasons for using IPython interactively
8 0.082074545 49 fast ml-2014-01-10-Classifying images with a pre-trained deep network
9 0.080617189 25 fast ml-2013-04-10-Gender discrimination
10 0.078473695 14 fast ml-2013-01-04-Madelon: Spearmint's revenge
11 0.072349675 27 fast ml-2013-05-01-Deep learning made easy
12 0.070894092 61 fast ml-2014-05-08-Impute missing values with Amelia
13 0.069790199 13 fast ml-2012-12-27-Spearmint with a random forest
14 0.068332709 53 fast ml-2014-02-20-Are stocks predictable?
15 0.063551426 43 fast ml-2013-11-02-Maxing out the digits
16 0.062612943 56 fast ml-2014-03-31-If you use R, you may want RStudio
17 0.058307346 26 fast ml-2013-04-17-Regression as classification
18 0.057235949 24 fast ml-2013-03-25-Dimensionality reduction for sparse binary data - an overview
19 0.055143259 9 fast ml-2012-10-25-So you want to work for Facebook
20 0.05415741 54 fast ml-2014-03-06-PyBrain - a simple neural networks library in Python
topicId topicWeight
[(26, 0.023), (31, 0.023), (33, 0.015), (35, 0.023), (51, 0.595), (55, 0.02), (65, 0.051), (69, 0.103), (71, 0.015), (99, 0.021)]
simIndex simValue blogId blogTitle
same-blog 1 0.94179052 44 fast ml-2013-11-18-CUDA on a Linux laptop
Introduction: After testing CUDA on a desktop , we now switch to a Linux laptop with 64-bit Xubuntu. Getting CUDA to work is harder here. Will the effort be worth the results? If you have a laptop with a Nvidia card, the thing probably uses it for 3D graphics and Intel’s built-in unit for everything else. This technology is known as Optimus and it happens to make things anything but easy for running CUDA on Linux. The problem is with GPU drivers, specifically between Linux being open-source and Nvidia drivers being not. This strained relation at one time prompted Linus Torvalds to give Nvidia a finger with great passion. Installing GPU drivers Here’s a solution for the driver problem. You need a package called bumblebee . It makes a Nvidia card accessible to your apps. To install drivers and bumblebee , try something along these lines: sudo apt-get install nvidia-current-updates sudo apt-get install bumblebee Note that you don’t need drivers that come with a specific CUDA rel
2 0.89523655 2 fast ml-2012-08-27-Kaggle job recommendation challenge
Introduction: This is an introduction to Kaggle job recommendation challenge . It looks a lot like a typical collaborative filtering thing (with a lot of extra information), but not quite. Spot these two big differences: There are no explicit ratings. Instead, there’s info about which jobs user applied to. This is known as one-class collaborative filtering (OCCF), or learning from positive-only feedback. If you want to dig deeper into the subject, there have been already contests with positive feedback only, for example track two of Yahoo KDD Cup or Millions Songs Dataset Challenge at Kaggle (both about songs). The second difference is less apparent. When you look at test users (that is, the users that we are asked to recommend jobs for), only about half of them made at least one application. For the other half, no data and no collaborative filtering. For the users we have applications data for, it’s very sparse, so we would like to use CF, because it does well in similar se
3 0.64099145 34 fast ml-2013-07-14-Running things on a GPU
Introduction: You’ve heard about running things on a graphics card, but have you tried it? All you need to taste the speed is a Nvidia card and some software. We run experiments using Cudamat and Theano in Python. GPUs differ from CPUs in that they are optimized for throughput instead of latency. Here’s a metaphor: when you play an online game, you want fast response times, or low latency. Otherwise you get lag. However when you’re downloading a movie, you don’t care about response times, you care about bandwidth - that’s throughput. Massive computations are similiar to downloading a movie in this respect. The setup We’ll be testing things on a platform with an Intel Dual Core CPU @3Ghz and either GeForce 9600 GT, an old card, or GeForce GTX 550 Ti, a more recent card. See the appendix for more info on GPU hardware. Software we’ll be using is Python. On CPU it employs one core*. That is OK in everyday use because while one core is working hard, you can comfortably do something else becau
4 0.24663784 49 fast ml-2014-01-10-Classifying images with a pre-trained deep network
Introduction: Recently at least two research teams made their pre-trained deep convolutional networks available, so you can classify your images right away. We’ll see how to go about it, with data from the Cats & Dogs competition at Kaggle as an example. We’ll be using OverFeat , a classifier and feature extractor from the New York guys lead by Yann LeCun and Rob Fergus. The principal author, Pierre Sermanet, is currently first on the Dogs vs. Cats leaderboard . The other available implementation we know of comes from Berkeley. It’s called Caffe and is a successor to decaf . Yangqing Jia , the main author of these, is also near the top of the leaderboard. Both networks were trained on ImageNet , which is an image database organized according to the WordNet hierarchy . It was the ImageNet Large Scale Visual Recognition Challenge 2012 in which Alex Krizhevsky crushed the competition with his network. His error was 16%, the second best - 26%. Data The Kaggle competition featur
5 0.24459338 9 fast ml-2012-10-25-So you want to work for Facebook
Introduction: Good news, everyone! There’s a new contest on Kaggle - Facebook is looking for talent . They won’t pay, but just might interview. This post is in a way a bonus for active readers because most visitors of fastml.com originally come from Kaggle forums. For this competition the forums are disabled to encourage own work . To honor this, we won’t publish any code. But own work doesn’t mean original work , and we wouldn’t want to reinvent the wheel, would we? The contest differs substantially from a Kaggle stereotype, if there is such a thing, in three major ways: there’s no money prizes, as mentioned above it’s not a real world problem, but rather an assignment to screen job candidates (this has important consequences, described below) it’s not a typical machine learning project, but rather a broader AI exercise You are given a graph of the internet, actually a snapshot of the graph for each of 15 time steps. You are also given a bunch of paths in this graph, which a
6 0.2411866 45 fast ml-2013-11-27-Object recognition in images with cuda-convnet
7 0.22666873 48 fast ml-2013-12-28-Regularizing neural networks with dropout and with DropConnect
8 0.213809 43 fast ml-2013-11-02-Maxing out the digits
9 0.20100059 27 fast ml-2013-05-01-Deep learning made easy
10 0.19740227 4 fast ml-2012-09-17-Best Buy mobile contest
11 0.19681722 1 fast ml-2012-08-09-What you wanted to know about Mean Average Precision
12 0.19117483 20 fast ml-2013-02-18-Predicting advertised salaries
13 0.18638819 21 fast ml-2013-02-27-Dimensionality reduction for sparse binary data
14 0.18396419 25 fast ml-2013-04-10-Gender discrimination
15 0.18196581 24 fast ml-2013-03-25-Dimensionality reduction for sparse binary data - an overview
16 0.17638412 13 fast ml-2012-12-27-Spearmint with a random forest
17 0.17631353 58 fast ml-2014-04-12-Deep learning these days
18 0.17627165 32 fast ml-2013-07-05-Processing large files, line by line
19 0.17361949 14 fast ml-2013-01-04-Madelon: Spearmint's revenge
20 0.17332891 61 fast ml-2014-05-08-Impute missing values with Amelia