fast_ml fast_ml-2013 fast_ml-2013-47 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: A/B testing is a way to optimize a web page. Half of visitors see one version, the other half another, so you can tell which version is more conducive to your goal - for example selling something. Since June 2013 A/B testing can be conveniently done with Google Analytics. Here’s how. This article is not quite about machine learning. If you’re not interested in testing, scroll down to the bayesian bandits section . Google Content Experiments We remember Google Website Optimizer from a few years ago. It wasn’t exactly user friendly or slick, but it felt solid and did the job. Unfortunately, at one point in time Google pulled the plug, leaving Genetify as a sole free (and open source) tool for multivariate testing. Multivariate means testing a few elements on a page simultanously. At that time they launched Content Experiments in Google Analytics, but it was a giant step backward. Content experiments were very primitive and only allowed rudimentary A/B split testing. It i
sentIndex sentText sentNum sentScore
1 Half of visitors see one version, the other half another, so you can tell which version is more conducive to your goal - for example selling something. [sent-2, score-0.367]
2 If you’re not interested in testing, scroll down to the bayesian bandits section . [sent-6, score-0.325]
3 Multivariate means testing a few elements on a page simultanously. [sent-10, score-0.42]
4 Essentially, now you can have a full-fledged multivariate testing tool akin to Website Optimizer. [sent-14, score-0.315]
5 The catch is that you need to implement content variations yourself in JavaScript. [sent-15, score-0.359]
6 This is not as clear-cut goal, but still you can measure average time on page, page views, returning visits and so on. [sent-24, score-0.47]
7 The setup First you setup the experiment using the normal interface in GA, only skipping the part where you put variation URLs - just enter some subpages there. [sent-25, score-0.998]
8 Now you put some Javascript in your page source: load the content experiments script from Google, providing it the experiment ID from the GA visual interface:get a variation number it selects: var variation = cxApi. [sent-28, score-1.726]
9 chooseVariation(); put your normal Analytics snippet after this Now the only thing left is actually showing the selected variation. [sent-29, score-0.391]
10 Showing the selected variation Suppose that you want to vary contents of a DIV . [sent-30, score-0.595]
11 Then if GA decides it wants to show the variation to a given user, you switch the divs: the original becomes hidden and the variation shows. [sent-36, score-1.074]
12 Here’s an example JavaScript snippet to put at the end of HTML BODY:// original is 0 if ( variation == 1 ) { document. [sent-37, score-0.648]
13 Bayesian bandits When a user first visits the page, Analytics decides which variation to show. [sent-44, score-1.051]
14 The selected variation is stored in a cookie so when the user opens a next page or comes back tomorrow he is shown the same version. [sent-45, score-0.909]
15 The difference is that a multi-arm bandit has many arms and you need to decide which one to pull. [sent-49, score-0.306]
16 Here’s a link to the page with an interactive visualization. [sent-52, score-0.291]
17 In case of web testing they are rather unlikely to change during the experiment, so that makes bandits a good solution for selecting variations to show. [sent-55, score-0.716]
18 Note that Google uses bandits to decide which ads to show - the underlying problem is very similiar. [sent-58, score-0.418]
19 You can either use one of the built-in metrics (including page views, bounce rate and time on page) or a custom goal defined in Analytics. [sent-62, score-0.517]
20 This means thay you can optimize some basic metrics with a click of a button, or, with some effort, pretty much whatever you want. [sent-63, score-0.288]
wordName wordTfidf (topN-words)
[('variation', 0.428), ('page', 0.291), ('bandits', 0.257), ('content', 0.205), ('google', 0.185), ('visitors', 0.171), ('decides', 0.154), ('ga', 0.154), ('variations', 0.154), ('experiment', 0.136), ('experiments', 0.136), ('testing', 0.129), ('bandit', 0.128), ('interface', 0.128), ('multivariate', 0.128), ('click', 0.113), ('web', 0.113), ('user', 0.109), ('analytics', 0.109), ('arms', 0.103), ('html', 0.103), ('june', 0.103), ('visits', 0.103), ('put', 0.102), ('optimize', 0.094), ('goal', 0.087), ('want', 0.086), ('ads', 0.086), ('javascript', 0.086), ('views', 0.086), ('showing', 0.086), ('metrics', 0.081), ('selected', 0.081), ('measure', 0.076), ('website', 0.075), ('decide', 0.075), ('setup', 0.068), ('normal', 0.068), ('bayesian', 0.068), ('original', 0.064), ('change', 0.063), ('half', 0.058), ('tool', 0.058), ('finding', 0.058), ('success', 0.058), ('custom', 0.058), ('probabilities', 0.054), ('snippet', 0.054), ('version', 0.051), ('exactly', 0.048)]
simIndex simValue blogId blogTitle
same-blog 1 0.99999982 47 fast ml-2013-12-15-A-B testing with bayesian bandits in Google Analytics
Introduction: A/B testing is a way to optimize a web page. Half of visitors see one version, the other half another, so you can tell which version is more conducive to your goal - for example selling something. Since June 2013 A/B testing can be conveniently done with Google Analytics. Here’s how. This article is not quite about machine learning. If you’re not interested in testing, scroll down to the bayesian bandits section . Google Content Experiments We remember Google Website Optimizer from a few years ago. It wasn’t exactly user friendly or slick, but it felt solid and did the job. Unfortunately, at one point in time Google pulled the plug, leaving Genetify as a sole free (and open source) tool for multivariate testing. Multivariate means testing a few elements on a page simultanously. At that time they launched Content Experiments in Google Analytics, but it was a giant step backward. Content experiments were very primitive and only allowed rudimentary A/B split testing. It i
2 0.093386173 41 fast ml-2013-10-09-Big data made easy
Introduction: An overview of key points about big data. This post was inspired by a very good article about big data by Chris Stucchio (linked below). The article is about hype and technology. We hate the hype. Big data is hype Everybody talks about big data; nobody knows exactly what it is. That’s pretty much the definition of hype. Google Trends suggest that the term took off at the beginning of 2011 (and the searches are coming mainly from Asia, curiously). Now, to put things in context: Big data is right there (or maybe not quite yet?) with other slogans like web 2.0 , cloud computing and social media . In effect, big data is a generic term for: data science machine learning data mining predictive analytics and so on. Don’t believe us? What about James Goodnight, the CEO of SAS : The term big data is being used today because computer analysts and journalists got tired of writing about cloud computing. Before cloud computing it was data warehousing or
3 0.070475429 18 fast ml-2013-01-17-A very fast denoising autoencoder
Introduction: Once upon a time we were browsing machine learning papers and software. We were interested in autoencoders and found a rather unusual one. It was called marginalized Stacked Denoising Autoencoder and the author claimed that it preserves the strong feature learning capacity of Stacked Denoising Autoencoders, but is orders of magnitudes faster. We like all things fast, so we were hooked. About autoencoders Wikipedia says that an autoencoder is an artificial neural network and its aim is to learn a compressed representation for a set of data. This means it is being used for dimensionality reduction . In other words, an autoencoder is a neural network meant to replicate the input. It would be trivial with a big enough number of units in a hidden layer: the network would just find an identity mapping. Hence dimensionality reduction: a hidden layer size is typically smaller than input layer. mSDA is a curious specimen: it is not a neural network and it doesn’t reduce dimension
4 0.064668246 7 fast ml-2012-10-05-Predicting closed questions on Stack Overflow
Introduction: This time we enter the Stack Overflow challenge , which is about predicting a status of a given question on SO. There are five possible statuses, so it’s a multi-class classification problem. We would prefer a tool able to perform multiclass classification by itself. It can be done by hand by constructing five datasets, each with binary labels (one class against all others), and then combining predictions, but it might be a bit tricky to get right - we tried. Fortunately, nice people at Yahoo, excuse us, Microsoft, recently relased a new version of Vowpal Wabbit , and this new version supports multiclass classification. In case you’re wondering, Vowpal Wabbit is a fast linear learner. We like the “fast” part and “linear” is OK for dealing with lots of words, as in this contest. In any case, with more than three million data points it wouldn’t be that easy to train a kernel SVM, a neural net or what have you. VW, being a well-polished tool, has a few very convenient features.
5 0.064413436 12 fast ml-2012-12-21-Tuning hyperparams automatically with Spearmint
Introduction: The promise What’s attractive in machine learning? That a machine is learning, instead of a human. But an operator still has a lot of work to do. First, he has to learn how to teach a machine, in general. Then, when it comes to a concrete task, there are two main areas where a human needs to do the work (and remember, laziness is a virtue, at least for a programmer, so we’d like to minimize amount of work done by a human): data preparation model tuning This story is about model tuning. Typically, to achieve satisfactory results, first we need to convert raw data into format accepted by the model we would like to use, and then tune a few hyperparameters of the model. For example, some hyperparams to tune for a random forest may be a number of trees to grow and a number of candidate features at each split ( mtry in R randomForest). For a neural network, there are quite a lot of hyperparams: number of layers, number of neurons in each layer (specifically, in each hid
6 0.063815914 50 fast ml-2014-01-20-How to get predictions from Pylearn2
7 0.061164211 11 fast ml-2012-12-07-Predicting wine quality
8 0.057437584 46 fast ml-2013-12-07-13 NIPS papers that caught our eye
9 0.05699062 32 fast ml-2013-07-05-Processing large files, line by line
10 0.054580327 27 fast ml-2013-05-01-Deep learning made easy
11 0.053667393 40 fast ml-2013-10-06-Pylearn2 in practice
12 0.052314825 53 fast ml-2014-02-20-Are stocks predictable?
13 0.050540872 37 fast ml-2013-09-03-Our followers and who else they follow
14 0.050494477 54 fast ml-2014-03-06-PyBrain - a simple neural networks library in Python
15 0.050402816 60 fast ml-2014-04-30-Converting categorical data into numbers with Pandas and Scikit-learn
16 0.048809767 14 fast ml-2013-01-04-Madelon: Spearmint's revenge
17 0.04871624 62 fast ml-2014-05-26-Yann LeCun's answers from the Reddit AMA
18 0.048444193 19 fast ml-2013-02-07-The secret of the big guys
19 0.047307357 1 fast ml-2012-08-09-What you wanted to know about Mean Average Precision
20 0.046725038 23 fast ml-2013-03-18-Large scale L1 feature selection with Vowpal Wabbit
topicId topicWeight
[(0, 0.19), (1, 0.024), (2, 0.046), (3, -0.09), (4, 0.027), (5, 0.067), (6, -0.042), (7, 0.084), (8, 0.122), (9, 0.144), (10, 0.21), (11, 0.183), (12, -0.075), (13, -0.192), (14, -0.353), (15, -0.191), (16, -0.111), (17, -0.052), (18, 0.026), (19, 0.166), (20, -0.243), (21, -0.115), (22, -0.143), (23, 0.122), (24, -0.033), (25, -0.073), (26, -0.095), (27, -0.076), (28, -0.13), (29, -0.024), (30, -0.038), (31, 0.171), (32, 0.165), (33, 0.201), (34, 0.233), (35, -0.129), (36, -0.005), (37, -0.094), (38, 0.334), (39, -0.131), (40, 0.083), (41, 0.087), (42, -0.232), (43, -0.112), (44, -0.136), (45, -0.006), (46, -0.101), (47, -0.057), (48, -0.0), (49, 0.043)]
simIndex simValue blogId blogTitle
same-blog 1 0.99215192 47 fast ml-2013-12-15-A-B testing with bayesian bandits in Google Analytics
Introduction: A/B testing is a way to optimize a web page. Half of visitors see one version, the other half another, so you can tell which version is more conducive to your goal - for example selling something. Since June 2013 A/B testing can be conveniently done with Google Analytics. Here’s how. This article is not quite about machine learning. If you’re not interested in testing, scroll down to the bayesian bandits section . Google Content Experiments We remember Google Website Optimizer from a few years ago. It wasn’t exactly user friendly or slick, but it felt solid and did the job. Unfortunately, at one point in time Google pulled the plug, leaving Genetify as a sole free (and open source) tool for multivariate testing. Multivariate means testing a few elements on a page simultanously. At that time they launched Content Experiments in Google Analytics, but it was a giant step backward. Content experiments were very primitive and only allowed rudimentary A/B split testing. It i
2 0.12114407 7 fast ml-2012-10-05-Predicting closed questions on Stack Overflow
Introduction: This time we enter the Stack Overflow challenge , which is about predicting a status of a given question on SO. There are five possible statuses, so it’s a multi-class classification problem. We would prefer a tool able to perform multiclass classification by itself. It can be done by hand by constructing five datasets, each with binary labels (one class against all others), and then combining predictions, but it might be a bit tricky to get right - we tried. Fortunately, nice people at Yahoo, excuse us, Microsoft, recently relased a new version of Vowpal Wabbit , and this new version supports multiclass classification. In case you’re wondering, Vowpal Wabbit is a fast linear learner. We like the “fast” part and “linear” is OK for dealing with lots of words, as in this contest. In any case, with more than three million data points it wouldn’t be that easy to train a kernel SVM, a neural net or what have you. VW, being a well-polished tool, has a few very convenient features.
3 0.11619791 12 fast ml-2012-12-21-Tuning hyperparams automatically with Spearmint
Introduction: The promise What’s attractive in machine learning? That a machine is learning, instead of a human. But an operator still has a lot of work to do. First, he has to learn how to teach a machine, in general. Then, when it comes to a concrete task, there are two main areas where a human needs to do the work (and remember, laziness is a virtue, at least for a programmer, so we’d like to minimize amount of work done by a human): data preparation model tuning This story is about model tuning. Typically, to achieve satisfactory results, first we need to convert raw data into format accepted by the model we would like to use, and then tune a few hyperparameters of the model. For example, some hyperparams to tune for a random forest may be a number of trees to grow and a number of candidate features at each split ( mtry in R randomForest). For a neural network, there are quite a lot of hyperparams: number of layers, number of neurons in each layer (specifically, in each hid
4 0.11462098 41 fast ml-2013-10-09-Big data made easy
Introduction: An overview of key points about big data. This post was inspired by a very good article about big data by Chris Stucchio (linked below). The article is about hype and technology. We hate the hype. Big data is hype Everybody talks about big data; nobody knows exactly what it is. That’s pretty much the definition of hype. Google Trends suggest that the term took off at the beginning of 2011 (and the searches are coming mainly from Asia, curiously). Now, to put things in context: Big data is right there (or maybe not quite yet?) with other slogans like web 2.0 , cloud computing and social media . In effect, big data is a generic term for: data science machine learning data mining predictive analytics and so on. Don’t believe us? What about James Goodnight, the CEO of SAS : The term big data is being used today because computer analysts and journalists got tired of writing about cloud computing. Before cloud computing it was data warehousing or
5 0.11138428 32 fast ml-2013-07-05-Processing large files, line by line
Introduction: Perhaps the most common format of data for machine learning is text files. Often data is too large to fit in memory; this is sometimes referred to as big data. But do you need to load the whole data into memory? Maybe you could at least pre-process it line by line. We show how to do this with Python. Prepare to read and possibly write some code. The most common format for text files is probably CSV. For sparse data, libsvm format is popular. Both can be processed using csv module in Python. import csv i_f = open( input_file, 'r' ) reader = csv.reader( i_f ) For libsvm you just set the delimiter to space: reader = csv.reader( i_f, delimiter = ' ' ) Then you go over the file contents. Each line is a list of strings: for line in reader: # do something with the line, for example: label = float( line[0] ) # .... writer.writerow( line ) If you need to do a second pass, you just rewind the input file: i_f.seek( 0 ) for line in re
6 0.10766127 27 fast ml-2013-05-01-Deep learning made easy
7 0.10290334 11 fast ml-2012-12-07-Predicting wine quality
8 0.096808463 18 fast ml-2013-01-17-A very fast denoising autoencoder
9 0.093795985 24 fast ml-2013-03-25-Dimensionality reduction for sparse binary data - an overview
10 0.093678534 53 fast ml-2014-02-20-Are stocks predictable?
11 0.090401426 54 fast ml-2014-03-06-PyBrain - a simple neural networks library in Python
12 0.088291503 46 fast ml-2013-12-07-13 NIPS papers that caught our eye
13 0.084655046 1 fast ml-2012-08-09-What you wanted to know about Mean Average Precision
14 0.081863299 50 fast ml-2014-01-20-How to get predictions from Pylearn2
15 0.081638053 9 fast ml-2012-10-25-So you want to work for Facebook
16 0.080125391 14 fast ml-2013-01-04-Madelon: Spearmint's revenge
17 0.078071833 23 fast ml-2013-03-18-Large scale L1 feature selection with Vowpal Wabbit
18 0.077801093 15 fast ml-2013-01-07-Machine learning courses online
19 0.077655748 60 fast ml-2014-04-30-Converting categorical data into numbers with Pandas and Scikit-learn
20 0.072311178 25 fast ml-2013-04-10-Gender discrimination
topicId topicWeight
[(4, 0.5), (6, 0.042), (26, 0.023), (31, 0.033), (35, 0.011), (48, 0.026), (51, 0.017), (55, 0.013), (69, 0.13), (71, 0.026), (78, 0.014), (79, 0.027), (84, 0.013), (99, 0.038)]
simIndex simValue blogId blogTitle
same-blog 1 0.920156 47 fast ml-2013-12-15-A-B testing with bayesian bandits in Google Analytics
Introduction: A/B testing is a way to optimize a web page. Half of visitors see one version, the other half another, so you can tell which version is more conducive to your goal - for example selling something. Since June 2013 A/B testing can be conveniently done with Google Analytics. Here’s how. This article is not quite about machine learning. If you’re not interested in testing, scroll down to the bayesian bandits section . Google Content Experiments We remember Google Website Optimizer from a few years ago. It wasn’t exactly user friendly or slick, but it felt solid and did the job. Unfortunately, at one point in time Google pulled the plug, leaving Genetify as a sole free (and open source) tool for multivariate testing. Multivariate means testing a few elements on a page simultanously. At that time they launched Content Experiments in Google Analytics, but it was a giant step backward. Content experiments were very primitive and only allowed rudimentary A/B split testing. It i
2 0.2764242 9 fast ml-2012-10-25-So you want to work for Facebook
Introduction: Good news, everyone! There’s a new contest on Kaggle - Facebook is looking for talent . They won’t pay, but just might interview. This post is in a way a bonus for active readers because most visitors of fastml.com originally come from Kaggle forums. For this competition the forums are disabled to encourage own work . To honor this, we won’t publish any code. But own work doesn’t mean original work , and we wouldn’t want to reinvent the wheel, would we? The contest differs substantially from a Kaggle stereotype, if there is such a thing, in three major ways: there’s no money prizes, as mentioned above it’s not a real world problem, but rather an assignment to screen job candidates (this has important consequences, described below) it’s not a typical machine learning project, but rather a broader AI exercise You are given a graph of the internet, actually a snapshot of the graph for each of 15 time steps. You are also given a bunch of paths in this graph, which a
3 0.27216229 12 fast ml-2012-12-21-Tuning hyperparams automatically with Spearmint
Introduction: The promise What’s attractive in machine learning? That a machine is learning, instead of a human. But an operator still has a lot of work to do. First, he has to learn how to teach a machine, in general. Then, when it comes to a concrete task, there are two main areas where a human needs to do the work (and remember, laziness is a virtue, at least for a programmer, so we’d like to minimize amount of work done by a human): data preparation model tuning This story is about model tuning. Typically, to achieve satisfactory results, first we need to convert raw data into format accepted by the model we would like to use, and then tune a few hyperparameters of the model. For example, some hyperparams to tune for a random forest may be a number of trees to grow and a number of candidate features at each split ( mtry in R randomForest). For a neural network, there are quite a lot of hyperparams: number of layers, number of neurons in each layer (specifically, in each hid
4 0.26882562 27 fast ml-2013-05-01-Deep learning made easy
Introduction: As usual, there’s an interesting competition at Kaggle: The Black Box. It’s connected to ICML 2013 Workshop on Challenges in Representation Learning, held by the deep learning guys from Montreal. There are a couple benchmarks for this competition and the best one is unusually hard to beat 1 - only less than a fourth of those taking part managed to do so. We’re among them. Here’s how. The key ingredient in our success is a recently developed secret Stanford technology for deep unsupervised learning: sparse filtering by Jiquan Ngiam et al. Actually, it’s not secret. It’s available at Github , and has one or two very appealling properties. Let us explain. The main idea of deep unsupervised learning, as we understand it, is feature extraction. One of the most common applications is in multimedia. The reason for that is that multimedia tasks, for example object recognition, are easy for humans, but difficult for computers 2 . Geoff Hinton from Toronto talks about two ends
5 0.26317641 13 fast ml-2012-12-27-Spearmint with a random forest
Introduction: Now that we have Spearmint basics nailed, we’ll try tuning a random forest, and specifically two hyperparams: a number of trees ( ntrees ) and a number of candidate features at each split ( mtry ). Here’s some code . We’re going to use a red wine quality dataset. It has about 1600 examples and our goal will be to predict a rating for a wine given all the other properties. This is a regression* task, as ratings are in (0,10) range. We will split the data 80/10/10 into train, validation and test set, and use the first two to establish optimal hyperparams and then predict on the test set. As an error measure we will use RMSE. At first, we will try ntrees between 10 and 200 and mtry between 3 and 11 (there’s eleven features total, so that’s the upper bound). Here are the results of two Spearmint runs with 71 and 95 tries respectively. Colors denote a validation error value: green : RMSE < 0.57 blue : RMSE < 0.58 black : RMSE >= 0.58 Turns out that some diffe
6 0.25999743 7 fast ml-2012-10-05-Predicting closed questions on Stack Overflow
7 0.25767022 1 fast ml-2012-08-09-What you wanted to know about Mean Average Precision
8 0.25515851 14 fast ml-2013-01-04-Madelon: Spearmint's revenge
9 0.25156808 43 fast ml-2013-11-02-Maxing out the digits
10 0.2499382 18 fast ml-2013-01-17-A very fast denoising autoencoder
11 0.24897932 17 fast ml-2013-01-14-Feature selection in practice
12 0.24803907 48 fast ml-2013-12-28-Regularizing neural networks with dropout and with DropConnect
13 0.2453362 19 fast ml-2013-02-07-The secret of the big guys
14 0.2405536 20 fast ml-2013-02-18-Predicting advertised salaries
15 0.23885602 40 fast ml-2013-10-06-Pylearn2 in practice
16 0.23283298 16 fast ml-2013-01-12-Intro to random forests
17 0.23089628 35 fast ml-2013-08-12-Accelerometer Biometric Competition
18 0.23058546 23 fast ml-2013-03-18-Large scale L1 feature selection with Vowpal Wabbit
19 0.22843575 55 fast ml-2014-03-20-Good representations, distance, metric learning and supervised dimensionality reduction
20 0.22650939 25 fast ml-2013-04-10-Gender discrimination