fast_ml fast_ml-2014 fast_ml-2014-53 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: We’d like to be able to predict stock market. That seems like a nice way of making money. We’ll address the fundamental issue: can stocks be predicted in the short term, that is a few days ahead? There’s a technique that seeks to answer the question of predictability. It’s called Forecastable Component Analysis . Based on a new forecastability measure, ForeCA finds an optimal transformation to separate a multivariate time series into a forecastable and an orthogonal white noise space. The author, Georg M. Goerg*, implemented it in R package ForeCA . It might be useful in two ways: It can tell you how forecastable time series is. Given a multivariate time series, let’s say a portfolio of stocks, it can find forecastable components. The idea in the second point is similiar to PCA - ForeCA is a linear dimensionality reduction technique. The main difference is that the method explicitly addresses forecastability. It does so by considering an interplay between time and
sentIndex sentText sentNum sentScore
1 We’ll address the fundamental issue: can stocks be predicted in the short term, that is a few days ahead? [sent-3, score-0.189]
2 Based on a new forecastability measure, ForeCA finds an optimal transformation to separate a multivariate time series into a forecastable and an orthogonal white noise space. [sent-6, score-0.914]
3 It might be useful in two ways: It can tell you how forecastable time series is. [sent-9, score-0.604]
4 Given a multivariate time series, let’s say a portfolio of stocks, it can find forecastable components. [sent-10, score-0.547]
5 It does so by considering an interplay between time and frequency: Forecasting is inherently tied to the time domain. [sent-13, score-0.142]
6 Yet, since [equations in the paper] provide a one-to-one mapping between the time and frequency domain, we can use frequency domain properties to measure forecastability. [sent-14, score-0.492]
7 Stationary time series only It only makes sense to apply ForeCA to at least weekly stationary time series. [sent-18, score-0.379]
8 Raw stock data doesn’t fit the bill, but daily returns do. [sent-19, score-0.526]
9 A daily return shows how price changed relative to the one day before. [sent-20, score-0.262]
10 For example, you can learn about returns in section 1. [sent-23, score-0.246]
11 Daily stock data: raw (above) and returns (below). [sent-25, score-0.398]
12 2 for an intuition about the connection between time and frequency spectrum. [sent-28, score-0.216]
13 fEcofin, which contains the equity funds data, is no longer available at CRAN, so install it from R-Forge: install. [sent-31, score-0.356]
14 org/") If you’d like to assess forecastability of a particular univariate series, use the Omega function: XX <- ts(diff(log(EuStockMarkets))[-c(1:1000),]) Omega(XX) DAX SMI CAC FTSE 6. [sent-34, score-0.098]
15 0 means not forecastable (white noise); 100 means perfectly forecastable (a sinusoid). [sent-39, score-0.788]
16 In the equity funds example, the most forecastable component scores roughly 2. [sent-45, score-0.746]
17 The European indexes above look more forecastable with Omega around 6%. [sent-47, score-0.394]
18 The author says he probably used different spectrum. [sent-49, score-0.106]
19 method estimation - different spectrum estimators give different Omega estimates ). [sent-50, score-0.129]
20 When dealing with finance it’s not obvious what’s the benchmark or the limit and there’s not much openness in discussing know-how. [sent-56, score-0.098]
wordName wordTfidf (topN-words)
[('foreca', 0.492), ('forecastable', 0.394), ('omega', 0.295), ('daily', 0.197), ('returns', 0.181), ('equity', 0.148), ('fecofin', 0.148), ('funds', 0.148), ('mod', 0.148), ('stock', 0.148), ('stocks', 0.148), ('frequency', 0.145), ('series', 0.139), ('finance', 0.098), ('forecastability', 0.098), ('georg', 0.098), ('goerg', 0.098), ('ret', 0.098), ('stationary', 0.098), ('ts', 0.098), ('xx', 0.098), ('log', 0.084), ('multivariate', 0.082), ('lynx', 0.082), ('domain', 0.082), ('time', 0.071), ('raw', 0.069), ('white', 0.065), ('noise', 0.065), ('section', 0.065), ('return', 0.065), ('author', 0.062), ('install', 0.06), ('component', 0.056), ('page', 0.056), ('plot', 0.053), ('library', 0.052), ('paper', 0.051), ('measure', 0.049), ('different', 0.044), ('address', 0.041), ('bill', 0.041), ('explains', 0.041), ('humans', 0.041), ('recognizing', 0.041), ('spectrum', 0.041), ('repos', 0.041), ('carlo', 0.041), ('monte', 0.041), ('forecasting', 0.041)]
simIndex simValue blogId blogTitle
same-blog 1 0.9999997 53 fast ml-2014-02-20-Are stocks predictable?
Introduction: We’d like to be able to predict stock market. That seems like a nice way of making money. We’ll address the fundamental issue: can stocks be predicted in the short term, that is a few days ahead? There’s a technique that seeks to answer the question of predictability. It’s called Forecastable Component Analysis . Based on a new forecastability measure, ForeCA finds an optimal transformation to separate a multivariate time series into a forecastable and an orthogonal white noise space. The author, Georg M. Goerg*, implemented it in R package ForeCA . It might be useful in two ways: It can tell you how forecastable time series is. Given a multivariate time series, let’s say a portfolio of stocks, it can find forecastable components. The idea in the second point is similiar to PCA - ForeCA is a linear dimensionality reduction technique. The main difference is that the method explicitly addresses forecastability. It does so by considering an interplay between time and
2 0.06451416 18 fast ml-2013-01-17-A very fast denoising autoencoder
Introduction: Once upon a time we were browsing machine learning papers and software. We were interested in autoencoders and found a rather unusual one. It was called marginalized Stacked Denoising Autoencoder and the author claimed that it preserves the strong feature learning capacity of Stacked Denoising Autoencoders, but is orders of magnitudes faster. We like all things fast, so we were hooked. About autoencoders Wikipedia says that an autoencoder is an artificial neural network and its aim is to learn a compressed representation for a set of data. This means it is being used for dimensionality reduction . In other words, an autoencoder is a neural network meant to replicate the input. It would be trivial with a big enough number of units in a hidden layer: the network would just find an identity mapping. Hence dimensionality reduction: a hidden layer size is typically smaller than input layer. mSDA is a curious specimen: it is not a neural network and it doesn’t reduce dimension
3 0.058921184 50 fast ml-2014-01-20-How to get predictions from Pylearn2
Introduction: A while ago we’ve shown how to get predictions from a Pylearn2 model. It is a little tricky, partly because of splitting data into batches. If you’re able to fit your data in memory, you can strip the batch handling code and it becomes easier to see what’s going on. We exercise the concept to distinguish cats from dogs again, with superior results. Step by step You have a pickled model from Pylearn2. Let’s load it: from pylearn2.utils import serial model_path = 'model.pkl' model = serial.load( model_path ) Next, some Theano weirdness. Theano is a compiler for symbolic expressions and with these expressions we deal when predicting. We need to define expressions for X and Y: X = model.get_input_space().make_theano_batch() Y = model.fprop( X ) Mind you, these are not variables, but rather descriptions of how to get variables. Y is easy to understand: just feed the data to the model and forward-propagate. X is more of an idiom, the incantations above make sur
4 0.053050522 27 fast ml-2013-05-01-Deep learning made easy
Introduction: As usual, there’s an interesting competition at Kaggle: The Black Box. It’s connected to ICML 2013 Workshop on Challenges in Representation Learning, held by the deep learning guys from Montreal. There are a couple benchmarks for this competition and the best one is unusually hard to beat 1 - only less than a fourth of those taking part managed to do so. We’re among them. Here’s how. The key ingredient in our success is a recently developed secret Stanford technology for deep unsupervised learning: sparse filtering by Jiquan Ngiam et al. Actually, it’s not secret. It’s available at Github , and has one or two very appealling properties. Let us explain. The main idea of deep unsupervised learning, as we understand it, is feature extraction. One of the most common applications is in multimedia. The reason for that is that multimedia tasks, for example object recognition, are easy for humans, but difficult for computers 2 . Geoff Hinton from Toronto talks about two ends
5 0.052314825 47 fast ml-2013-12-15-A-B testing with bayesian bandits in Google Analytics
Introduction: A/B testing is a way to optimize a web page. Half of visitors see one version, the other half another, so you can tell which version is more conducive to your goal - for example selling something. Since June 2013 A/B testing can be conveniently done with Google Analytics. Here’s how. This article is not quite about machine learning. If you’re not interested in testing, scroll down to the bayesian bandits section . Google Content Experiments We remember Google Website Optimizer from a few years ago. It wasn’t exactly user friendly or slick, but it felt solid and did the job. Unfortunately, at one point in time Google pulled the plug, leaving Genetify as a sole free (and open source) tool for multivariate testing. Multivariate means testing a few elements on a page simultanously. At that time they launched Content Experiments in Google Analytics, but it was a giant step backward. Content experiments were very primitive and only allowed rudimentary A/B split testing. It i
6 0.049545523 55 fast ml-2014-03-20-Good representations, distance, metric learning and supervised dimensionality reduction
7 0.043709844 24 fast ml-2013-03-25-Dimensionality reduction for sparse binary data - an overview
8 0.043418102 62 fast ml-2014-05-26-Yann LeCun's answers from the Reddit AMA
9 0.041268587 7 fast ml-2012-10-05-Predicting closed questions on Stack Overflow
10 0.040443283 46 fast ml-2013-12-07-13 NIPS papers that caught our eye
11 0.039894093 21 fast ml-2013-02-27-Dimensionality reduction for sparse binary data
12 0.0389666 20 fast ml-2013-02-18-Predicting advertised salaries
13 0.03776326 12 fast ml-2012-12-21-Tuning hyperparams automatically with Spearmint
14 0.035871588 26 fast ml-2013-04-17-Regression as classification
15 0.034924306 19 fast ml-2013-02-07-The secret of the big guys
16 0.034651622 58 fast ml-2014-04-12-Deep learning these days
17 0.034333773 25 fast ml-2013-04-10-Gender discrimination
18 0.033406425 45 fast ml-2013-11-27-Object recognition in images with cuda-convnet
19 0.032567672 35 fast ml-2013-08-12-Accelerometer Biometric Competition
20 0.032457974 32 fast ml-2013-07-05-Processing large files, line by line
topicId topicWeight
[(0, 0.143), (1, 0.038), (2, 0.067), (3, -0.006), (4, -0.016), (5, 0.027), (6, -0.046), (7, -0.018), (8, -0.027), (9, 0.056), (10, 0.128), (11, 0.131), (12, -0.076), (13, -0.202), (14, -0.182), (15, -0.317), (16, -0.087), (17, -0.301), (18, 0.086), (19, 0.003), (20, 0.366), (21, -0.043), (22, 0.111), (23, -0.078), (24, 0.131), (25, 0.276), (26, 0.186), (27, 0.329), (28, -0.204), (29, -0.027), (30, -0.03), (31, -0.2), (32, 0.004), (33, 0.02), (34, -0.044), (35, 0.326), (36, -0.172), (37, -0.035), (38, -0.028), (39, 0.04), (40, 0.096), (41, -0.053), (42, 0.05), (43, 0.036), (44, -0.107), (45, -0.002), (46, -0.003), (47, -0.011), (48, 0.047), (49, 0.014)]
simIndex simValue blogId blogTitle
same-blog 1 0.9868086 53 fast ml-2014-02-20-Are stocks predictable?
Introduction: We’d like to be able to predict stock market. That seems like a nice way of making money. We’ll address the fundamental issue: can stocks be predicted in the short term, that is a few days ahead? There’s a technique that seeks to answer the question of predictability. It’s called Forecastable Component Analysis . Based on a new forecastability measure, ForeCA finds an optimal transformation to separate a multivariate time series into a forecastable and an orthogonal white noise space. The author, Georg M. Goerg*, implemented it in R package ForeCA . It might be useful in two ways: It can tell you how forecastable time series is. Given a multivariate time series, let’s say a portfolio of stocks, it can find forecastable components. The idea in the second point is similiar to PCA - ForeCA is a linear dimensionality reduction technique. The main difference is that the method explicitly addresses forecastability. It does so by considering an interplay between time and
2 0.089012869 27 fast ml-2013-05-01-Deep learning made easy
Introduction: As usual, there’s an interesting competition at Kaggle: The Black Box. It’s connected to ICML 2013 Workshop on Challenges in Representation Learning, held by the deep learning guys from Montreal. There are a couple benchmarks for this competition and the best one is unusually hard to beat 1 - only less than a fourth of those taking part managed to do so. We’re among them. Here’s how. The key ingredient in our success is a recently developed secret Stanford technology for deep unsupervised learning: sparse filtering by Jiquan Ngiam et al. Actually, it’s not secret. It’s available at Github , and has one or two very appealling properties. Let us explain. The main idea of deep unsupervised learning, as we understand it, is feature extraction. One of the most common applications is in multimedia. The reason for that is that multimedia tasks, for example object recognition, are easy for humans, but difficult for computers 2 . Geoff Hinton from Toronto talks about two ends
3 0.083101928 18 fast ml-2013-01-17-A very fast denoising autoencoder
Introduction: Once upon a time we were browsing machine learning papers and software. We were interested in autoencoders and found a rather unusual one. It was called marginalized Stacked Denoising Autoencoder and the author claimed that it preserves the strong feature learning capacity of Stacked Denoising Autoencoders, but is orders of magnitudes faster. We like all things fast, so we were hooked. About autoencoders Wikipedia says that an autoencoder is an artificial neural network and its aim is to learn a compressed representation for a set of data. This means it is being used for dimensionality reduction . In other words, an autoencoder is a neural network meant to replicate the input. It would be trivial with a big enough number of units in a hidden layer: the network would just find an identity mapping. Hence dimensionality reduction: a hidden layer size is typically smaller than input layer. mSDA is a curious specimen: it is not a neural network and it doesn’t reduce dimension
4 0.077508464 7 fast ml-2012-10-05-Predicting closed questions on Stack Overflow
Introduction: This time we enter the Stack Overflow challenge , which is about predicting a status of a given question on SO. There are five possible statuses, so it’s a multi-class classification problem. We would prefer a tool able to perform multiclass classification by itself. It can be done by hand by constructing five datasets, each with binary labels (one class against all others), and then combining predictions, but it might be a bit tricky to get right - we tried. Fortunately, nice people at Yahoo, excuse us, Microsoft, recently relased a new version of Vowpal Wabbit , and this new version supports multiclass classification. In case you’re wondering, Vowpal Wabbit is a fast linear learner. We like the “fast” part and “linear” is OK for dealing with lots of words, as in this contest. In any case, with more than three million data points it wouldn’t be that easy to train a kernel SVM, a neural net or what have you. VW, being a well-polished tool, has a few very convenient features.
5 0.076806195 24 fast ml-2013-03-25-Dimensionality reduction for sparse binary data - an overview
Introduction: Last time we explored dimensionality reduction in practice using Gensim’s LSI and LDA. Now, having spent some time researching the subject matter, we will give an overview of other options. UPDATE : We now consider the topic quite irrelevant, because sparse high-dimensional data is precisely where linear models shine. See Amazon aspires to automate access control , Predicting advertised salaries and Predicting closed questions on Stack Overflow . And the few most popular methods are: LSI/LSA - a multinomial PCA LDA - Latent Dirichlet Allocation matrix factorization, in particular non-negative variants: NMF ICA, or Independent Components Analysis mixtures of Bernoullis stacked RBMs correlated topic models, an extension of LDA We tried the first two before. As regards matrix factorization, you do the same stuff as with movie recommendations (think Netflix challenge). The difference is, now all the matrix elements are known and we are only interested in
6 0.076516017 55 fast ml-2014-03-20-Good representations, distance, metric learning and supervised dimensionality reduction
7 0.072948501 21 fast ml-2013-02-27-Dimensionality reduction for sparse binary data
8 0.072787099 47 fast ml-2013-12-15-A-B testing with bayesian bandits in Google Analytics
9 0.071312398 46 fast ml-2013-12-07-13 NIPS papers that caught our eye
10 0.069566213 62 fast ml-2014-05-26-Yann LeCun's answers from the Reddit AMA
11 0.067861505 50 fast ml-2014-01-20-How to get predictions from Pylearn2
12 0.066242158 38 fast ml-2013-09-09-Predicting solar energy from weather forecasts plus a NetCDF4 tutorial
13 0.066234626 19 fast ml-2013-02-07-The secret of the big guys
14 0.065126292 12 fast ml-2012-12-21-Tuning hyperparams automatically with Spearmint
15 0.06468001 49 fast ml-2014-01-10-Classifying images with a pre-trained deep network
16 0.063641347 44 fast ml-2013-11-18-CUDA on a Linux laptop
17 0.062934458 58 fast ml-2014-04-12-Deep learning these days
18 0.06038605 26 fast ml-2013-04-17-Regression as classification
19 0.059556119 11 fast ml-2012-12-07-Predicting wine quality
20 0.058603089 34 fast ml-2013-07-14-Running things on a GPU
topicId topicWeight
[(6, 0.018), (11, 0.516), (26, 0.045), (31, 0.032), (35, 0.031), (37, 0.012), (55, 0.015), (58, 0.016), (69, 0.117), (71, 0.023), (78, 0.021), (84, 0.032), (99, 0.023)]
simIndex simValue blogId blogTitle
same-blog 1 0.91650856 53 fast ml-2014-02-20-Are stocks predictable?
Introduction: We’d like to be able to predict stock market. That seems like a nice way of making money. We’ll address the fundamental issue: can stocks be predicted in the short term, that is a few days ahead? There’s a technique that seeks to answer the question of predictability. It’s called Forecastable Component Analysis . Based on a new forecastability measure, ForeCA finds an optimal transformation to separate a multivariate time series into a forecastable and an orthogonal white noise space. The author, Georg M. Goerg*, implemented it in R package ForeCA . It might be useful in two ways: It can tell you how forecastable time series is. Given a multivariate time series, let’s say a portfolio of stocks, it can find forecastable components. The idea in the second point is similiar to PCA - ForeCA is a linear dimensionality reduction technique. The main difference is that the method explicitly addresses forecastability. It does so by considering an interplay between time and
2 0.23800689 27 fast ml-2013-05-01-Deep learning made easy
Introduction: As usual, there’s an interesting competition at Kaggle: The Black Box. It’s connected to ICML 2013 Workshop on Challenges in Representation Learning, held by the deep learning guys from Montreal. There are a couple benchmarks for this competition and the best one is unusually hard to beat 1 - only less than a fourth of those taking part managed to do so. We’re among them. Here’s how. The key ingredient in our success is a recently developed secret Stanford technology for deep unsupervised learning: sparse filtering by Jiquan Ngiam et al. Actually, it’s not secret. It’s available at Github , and has one or two very appealling properties. Let us explain. The main idea of deep unsupervised learning, as we understand it, is feature extraction. One of the most common applications is in multimedia. The reason for that is that multimedia tasks, for example object recognition, are easy for humans, but difficult for computers 2 . Geoff Hinton from Toronto talks about two ends
3 0.23676945 13 fast ml-2012-12-27-Spearmint with a random forest
Introduction: Now that we have Spearmint basics nailed, we’ll try tuning a random forest, and specifically two hyperparams: a number of trees ( ntrees ) and a number of candidate features at each split ( mtry ). Here’s some code . We’re going to use a red wine quality dataset. It has about 1600 examples and our goal will be to predict a rating for a wine given all the other properties. This is a regression* task, as ratings are in (0,10) range. We will split the data 80/10/10 into train, validation and test set, and use the first two to establish optimal hyperparams and then predict on the test set. As an error measure we will use RMSE. At first, we will try ntrees between 10 and 200 and mtry between 3 and 11 (there’s eleven features total, so that’s the upper bound). Here are the results of two Spearmint runs with 71 and 95 tries respectively. Colors denote a validation error value: green : RMSE < 0.57 blue : RMSE < 0.58 black : RMSE >= 0.58 Turns out that some diffe
4 0.23195238 19 fast ml-2013-02-07-The secret of the big guys
Introduction: Are you interested in linear models, or K-means clustering? Probably not much. These are very basic techniques with fancier alternatives. But here’s the bomb: when you combine those two methods for supervised learning, you can get better results than from a random forest. And maybe even faster. We have already written about Vowpal Wabbit , a fast linear learner from Yahoo/Microsoft. Google’s response (or at least, a Google’s guy response) seems to be Sofia-ML . The software consists of two parts: a linear learner and K-means clustering. We found Sofia a while ago and wondered about K-means: who needs K-means? Here’s a clue: This package can be used for learning cluster centers (…) and for mapping a given data set onto a new feature space based on the learned cluster centers. Our eyes only opened when we read a certain paper, namely An Analysis of Single-Layer Networks in Unsupervised Feature Learning ( PDF ). The paper, by Coates , Lee and Ng, is about object recogni
5 0.22883323 48 fast ml-2013-12-28-Regularizing neural networks with dropout and with DropConnect
Introduction: We continue with CIFAR-10-based competition at Kaggle to get to know DropConnect. It’s supposed to be an improvement over dropout. And dropout is certainly one of the bigger steps forward in neural network development. Is DropConnect really better than dropout? TL;DR DropConnect seems to offer results similiar to dropout. State of the art scores reported in the paper come from model ensembling. Dropout Dropout , by Hinton et al., is perhaps a biggest invention in the field of neural networks in recent years. It adresses the main problem in machine learning, that is overfitting. It does so by “dropping out” some unit activations in a given layer, that is setting them to zero. Thus it prevents co-adaptation of units and can also be seen as a method of ensembling many networks sharing the same weights. For each training example a different set of units to drop is randomly chosen. The idea has a biological inspiration . When a child is conceived, it receives half its genes f
6 0.22873184 9 fast ml-2012-10-25-So you want to work for Facebook
7 0.22707918 18 fast ml-2013-01-17-A very fast denoising autoencoder
8 0.22668849 12 fast ml-2012-12-21-Tuning hyperparams automatically with Spearmint
9 0.22639932 14 fast ml-2013-01-04-Madelon: Spearmint's revenge
10 0.22503479 1 fast ml-2012-08-09-What you wanted to know about Mean Average Precision
11 0.22422011 17 fast ml-2013-01-14-Feature selection in practice
12 0.2227494 43 fast ml-2013-11-02-Maxing out the digits
13 0.2215203 40 fast ml-2013-10-06-Pylearn2 in practice
14 0.21875213 7 fast ml-2012-10-05-Predicting closed questions on Stack Overflow
15 0.21688375 23 fast ml-2013-03-18-Large scale L1 feature selection with Vowpal Wabbit
16 0.21524212 20 fast ml-2013-02-18-Predicting advertised salaries
17 0.21498165 25 fast ml-2013-04-10-Gender discrimination
18 0.21233919 55 fast ml-2014-03-20-Good representations, distance, metric learning and supervised dimensionality reduction
19 0.20861699 45 fast ml-2013-11-27-Object recognition in images with cuda-convnet
20 0.20687962 35 fast ml-2013-08-12-Accelerometer Biometric Competition