jmlr jmlr2011 jmlr2011-62 knowledge-graph by maker-knowledge-mining

62 jmlr-2011-MSVMpack: A Multi-Class Support Vector Machine Package

Source: pdf

Author: Fabien Lauer, Yann Guermeur

Abstract: This paper describes MSVMpack, an open source software package dedicated to our generic model of multi-class support vector machine. All four multi-class support vector machines (M-SVMs) proposed so far in the literature appear as instances of this model. MSVMpack provides for them the ﬁrst uniﬁed implementation and offers a convenient basis to develop other instances. This is also the ﬁrst parallel implementation for M-SVMs. The package consists in a set of command-line tools with a callable library. The documentation includes a tutorial, a user’s guide and a developer’s guide. Keywords: multi-class support vector machines, open source, C

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 FR LORIA – Equipe ABC Campus Scientiﬁque, BP 239 54506 Vandœuvre-l` s-Nancy cedex, France e Editor: Mikio Braun Abstract This paper describes MSVMpack, an open source software package dedicated to our generic model of multi-class support vector machine. [sent-5, score-0.254]

2 All four multi-class support vector machines (M-SVMs) proposed so far in the literature appear as instances of this model. [sent-6, score-0.103]

3 MSVMpack provides for them the ﬁrst uniﬁed implementation and offers a convenient basis to develop other instances. [sent-7, score-0.05]

4 This is also the ﬁrst parallel implementation for M-SVMs. [sent-8, score-0.025]

5 The package consists in a set of command-line tools with a callable library. [sent-9, score-0.157]

6 The documentation includes a tutorial, a user’s guide and a developer’s guide. [sent-10, score-0.123]

7 Keywords: multi-class support vector machines, open source, C 1. [sent-11, score-0.032]

8 Introduction In the framework of polytomy computation, a multi-class support vector machine (M-SVM) is a support vector machine (SVM) dealing with all the categories simultaneously. [sent-12, score-0.091]

9 The proposed software implements them all in a single package named MSVMpack. [sent-15, score-0.184]

10 Its design paves the way for the implementation of our generic model of M-SVM and the integration of additional functionalities such as model selection algorithms. [sent-16, score-0.167]

11 The current version offers a parallel implementation with the possibility to use custom kernels. [sent-17, score-0.081]

12 This software package is available for Linux under the terms of the GPL at http://www. [sent-18, score-0.149]

13 fr/˜lauer/MSVMpack/ and provides two command-line tools with a C application programming interface without dependencies beside a linear programming solver. [sent-20, score-0.189]

14 Multi-Class Support Vector Machines We consider Q-category classiﬁcation problems where X is the description space and the set Y of the categories can be identiﬁed with [[ 1, Q ]. [sent-22, score-0.027]

15 Let κ be a real-valued positive type function (Berlinet ] 2 and let (H , ·, · and Thomas-Agnan, 2004) on X κ Hκ ) be the corresponding reproducing kernel ¯ Hilbert space. [sent-23, score-0.026]

16 H is the class of functions h = (hk )1 k Q from κ ¯ ¯ ¯ ¯ ¯ X to RQ that can be written as h (·) = h (·) + b = hk (·) + bk 1 k Q , where h = hk 1 k Q ∈ H and b = (bk )1 k Q ∈ RQ . [sent-25, score-0.369]

17 A function h assigns the category y to x if and only if y = argmax1 k Q hk (x) ¯ (cases of ex æquo are dealt with by introducing a dummy category). [sent-26, score-0.224]

18 · H given by: ¯ ¯ ¯ ∀h ∈ H , Q ¯ ¯ h H = ∑ ¯ ¯ hk , hk Hκ = ¯ hk Hκ 1 k Q 2 k=1 . [sent-30, score-0.492]

19 With these deﬁnitions at hand, our generic deﬁnition of a Q-category M-SVM is: Deﬁnition 1 (Generic model of M-SVM, Deﬁnition 4 in Guermeur, forthcoming) Let ((xi , yi ))1 i m ∈ (X ×[[ 1, Q ])m and λ ∈ R∗ . [sent-31, score-0.1]

20 Extending to matrices the notation used to designate the components of ξ and using δ to denote the Kronecker symbol, let us deﬁne the general term of M (2) ∈ MQm,Qm (R) as: (2) mik, jl = (1 − δyi ,k ) 1 − δy j ,l δi, j δk,l + √ Q−1 . [sent-36, score-0.035]

21 The potential of the generic model is discussed in Guermeur (forthcoming). [sent-38, score-0.073]

22 The Software Package MSVMpack includes a C application programming interface (API) and two command-line tools: one for training an M-SVM and one for making predictions with a trained M-SVM. [sent-40, score-0.106]

23 The following discusses some algorithmic issues before presenting these tools and the API. [sent-41, score-0.049]

24 1 Training Algorithm As in the bi-class case, an M-SVM is trained by solving the Wolfe dual of its instantiation of the QP problem in Deﬁnition 1. [sent-43, score-0.1]

25 The corresponding dual variables αik are the Lagrange multipliers of the constraints of correct classiﬁcation. [sent-44, score-0.037]

26 The LP solver included in MSVMpack is lp solve (Berkelaar et al. [sent-46, score-0.046]

27 Let Jd be the dual objective function and let α = (αik ) be a (feasible) solution of the dual problem obtained at some point of the training procedure. [sent-49, score-0.074]

28 The quality of α is measured thanks to the computation of an upper bound U(α) on the optimum J (h∗ , ξ∗ ) = Jd (α∗ ) that goes to this optimum. [sent-50, score-0.033]

29 2 Practical Use and Experiments In its most simple form, the command line ’trainmsvm trainingdata -m WW’ is used to train an M-SVM, where the -m ﬂag allows one to choose the type of M-SVM model according to Table 1. [sent-58, score-0.052]

30 The complete list of options and parameters for these command-line tools can be found in the documentation or simply obtained by calling them without argument. [sent-60, score-0.189]

31 Table 2 shows a comparison of MSVMpack with other implementations of M-SVMs on a subset of the USPS database with 500 instances from 10 classes and the whole CB513 data set with 84119 instances from 3 classes. [sent-61, score-0.111]

32 For the latter, the numbers reﬂect the average error and total times over a 5-fold cross validation, and the implementations that failed due to a lack of memory are not included in the Table. [sent-62, score-0.063]

33 We refer the reader to the documentation for the details of the experimental setup and additional comparisons on other data sets. [sent-63, score-0.09]

34 3 Calling the Library from Other Programs The “Developer’s guide” section of the documentation presents the API reference and an example program including MSVMpack functionalities through this API. [sent-65, score-0.186]

35 The library deﬁnes speciﬁc data structures for M-SVM models and data sets. [sent-66, score-0.027]

36 It also provides wrapper functions, which act according to the M-SVM model type, for example, call the corresponding training function. [sent-67, score-0.052]

37 The standard workﬂow for a train-and-test sequence is: call MSVM make model() to initialize the model; call MSVM make dataset() for each data set to load; call MSVM train() to train the model; and call MSVM classify set() to test the trained classiﬁer. [sent-68, score-0.167]

38 Ongoing and Future Developments MSVMpack implements the four M-SVMs proposed in the literature. [sent-70, score-0.035]

39 Current work focuses on the explicit implementation of our generic model of M-SVM, which will make it possible to study new machines thanks to a simple choice of the values of the hyperparameters M, p, and (Kt )1 t 3 . [sent-71, score-0.177]

40 1s 46m 29s 1m 51s 2h 08m 33s 2m 06s 2m 33s 2m 49s Table 2: Relative performance of different M-SVM implementations on two data sets. [sent-97, score-0.034]

41 On the algorithmic implementation of multiclass kernel-based vector machines. [sent-115, score-0.025]

42 A quadratic loss multi-class SVM for which a radius-margin bound applies. [sent-129, score-0.025]

43 Multicategory support vector machines: Theory and application to the classiﬁcation of microarray data and satellite radiance data. [sent-135, score-0.092]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('msvmpack', 0.732), ('guermeur', 0.285), ('msvm', 0.203), ('hk', 0.164), ('fabien', 0.122), ('iqm', 0.122), ('lauer', 0.122), ('llw', 0.122), ('loria', 0.122), ('ww', 0.114), ('ik', 0.109), ('package', 0.108), ('yann', 0.093), ('documentation', 0.09), ('jd', 0.086), ('berkelaar', 0.081), ('forthcoming', 0.081), ('mcsvm', 0.081), ('monfrini', 0.081), ('uermeur', 0.081), ('qp', 0.079), ('generic', 0.073), ('berlinet', 0.069), ('functionalities', 0.069), ('watkins', 0.062), ('developer', 0.062), ('weston', 0.059), ('crammer', 0.059), ('bsvm', 0.057), ('api', 0.057), ('wolfe', 0.053), ('calling', 0.05), ('rq', 0.05), ('tools', 0.049), ('lp', 0.046), ('machines', 0.046), ('auer', 0.045), ('usps', 0.043), ('bk', 0.041), ('software', 0.041), ('lee', 0.04), ('interface', 0.039), ('dual', 0.037), ('implements', 0.035), ('programming', 0.035), ('radiance', 0.035), ('informatica', 0.035), ('holloway', 0.035), ('pack', 0.035), ('hyi', 0.035), ('designate', 0.035), ('mik', 0.035), ('que', 0.035), ('implementations', 0.034), ('guide', 0.033), ('thanks', 0.033), ('trained', 0.032), ('frank', 0.032), ('support', 0.032), ('dummy', 0.031), ('multicategory', 0.031), ('instantiation', 0.031), ('lass', 0.031), ('beside', 0.031), ('logistics', 0.031), ('quarterly', 0.031), ('custom', 0.031), ('gpl', 0.031), ('fr', 0.029), ('category', 0.029), ('campus', 0.029), ('ector', 0.029), ('upport', 0.029), ('naval', 0.029), ('abc', 0.029), ('failed', 0.029), ('yi', 0.027), ('train', 0.027), ('database', 0.027), ('categories', 0.027), ('library', 0.027), ('call', 0.027), ('achine', 0.027), ('bp', 0.027), ('reference', 0.027), ('cs', 0.026), ('reproducing', 0.026), ('implementation', 0.025), ('offers', 0.025), ('singer', 0.025), ('satellite', 0.025), ('load', 0.025), ('cv', 0.025), ('command', 0.025), ('wrapper', 0.025), ('quadratic', 0.025), ('instances', 0.025), ('hilbert', 0.024), ('mikio', 0.024), ('kt', 0.024)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 1.0000001 62 jmlr-2011-MSVMpack: A Multi-Class Support Vector Machine Package

Author: Fabien Lauer, Yann Guermeur

2 0.032865107 63 jmlr-2011-MULAN: A Java Library for Multi-Label Learning

Author: Grigorios Tsoumakas, Eleftherios Spyromitros-Xioufis, Jozef Vilcek, Ioannis Vlahavas

Abstract: M ULAN is a Java library for learning from multi-label data. It offers a variety of classiﬁcation, ranking, thresholding and dimensionality reduction algorithms, as well as algorithms for learning from hierarchically structured labels. In addition, it contains an evaluation framework that calculates a rich variety of performance measures. Keywords: multi-label data, classiﬁcation, ranking, thresholding, dimensionality reduction, hierarchical classiﬁcation, evaluation 1. Multi-Label Learning A multi-label data set consists of training examples that are associated with a subset of a ﬁnite set of labels. Nowadays, multi-label data are becoming ubiquitous. They arise in an increasing number and diversity of applications, such as semantic annotation of images and video, web page categorization, direct marketing, functional genomics and music categorization into genres and emotions. There exist two major multi-label learning tasks (Tsoumakas et al., 2010): multi-label classiﬁcation and label ranking. The former is concerned with learning a model that outputs a bipartition of the set of labels into relevant and irrelevant with respect to a query instance. The latter is concerned with learning a model that outputs a ranking of the labels according to their relevance to a query instance. Some algorithms learn models that serve both tasks. Several algorithms learn models that primarily output a vector of numerical scores, one for each label. This vector is then converted to a ranking after solving ties, or to a bipartition, after thresholding (Ioannou et al., 2010). Multi-label learning methods addressing these tasks can be grouped into two categories (Tsoumakas et al., 2010): problem transformation and algorithm adaptation. The ﬁrst group of methods are algorithm independent. They transform the learning task into one or more singlelabel classiﬁcation tasks, for which a large body of learning algorithms exists. The second group of methods extend speciﬁc learning algorithms in order to handle multi-label data directly. There exist extensions of decision tree learners, nearest neighbor classiﬁers, neural networks, ensemble methods, support vector machines, kernel methods, genetic algorithms and others. Multi-label learning stretches across several other tasks. When labels are structured as a treeshaped hierarchy or a directed acyclic graph, then we have the interesting task of hierarchical multilabel learning. Dimensionality reduction is another important task for multi-label data, as it is for c 2011 Grigorios Tsoumakas, Eleftherios Spyromitros-Xiouﬁs, Jozef Vilcek and Ioannis Vlahavas. T SOUMAKAS , S PYROMITROS -X IOUFIS , V ILCEK AND V LAHAVAS any kind of data. When bags of instances are used to represent a training object, then multi-instance multi-label learning algorithms are required. There also exist semi-supervised learning and active learning algorithms for multi-label data. 2. The M ULAN Library The main goal of M ULAN is to bring the beneﬁts of machine learning open source software (MLOSS) (Sonnenburg et al., 2007) to people working with multi-label data. The availability of MLOSS is especially important in emerging areas like multi-label learning, because it removes the burden of implementing related work and speeds up the scientiﬁc progress. In multi-label learning, an extra burden is implementing appropriate evaluation measures, since these are different compared to traditional supervised learning tasks. Evaluating multi-label algorithms with a variety of measures, is considered important by the community, due to the different types of output (bipartition, ranking) and diverse applications. Towards this goal, M ULAN offers a plethora of state-of-the-art algorithms for multi-label classiﬁcation and label ranking and an evaluation framework that computes a large variety of multi-label evaluation measures through hold-out evaluation and cross-validation. In addition, the library offers a number of thresholding strategies that produce bipartitions from score vectors, simple baseline methods for multi-label dimensionality reduction and support for hierarchical multi-label classiﬁcation, including an implemented algorithm. M ULAN is a library. As such, it offers only programmatic API to the library users. There is no graphical user interface (GUI) available. The possibility to use the library via command line, is also currently not supported. Another drawback of M ULAN is that it runs everything in main memory so there exist limitations with very large data sets. M ULAN is written in Java and is built on top of Weka (Witten and Frank, 2005). This choice was made in order to take advantage of the vast resources of Weka on supervised learning algorithms, since many state-of-the-art multi-label learning algorithms are based on problem transformation. The fact that several machine learning researchers and practitioners are familiar with Weka was another reason for this choice. However, many aspects of the library are independent of Weka and there are interfaces for most of the core classes. M ULAN is an advocate of open science in general. One of the unique features of the library is a recently introduced experiments package, whose goal is to host code that reproduces experimental results reported on published papers on multi-label learning. To the best of our knowledge, most of the general learning platforms, like Weka, don’t support multi-label data. There are currently only a number of implementations of speciﬁc multi-label learning algorithms, but not a general library like M ULAN. 3. Using M ULAN This section presents an example of how to setup an experiment for empirically evaluating two multi-label algorithms on a multi-label data set using cross-validation. We create a new Java class for this experiment, which we call MulanExp1.java. The ﬁrst thing to do is load the multi-label data set that will be used for the empirical evaluation. M ULAN requires two text ﬁles for the speciﬁcation of a data set. The ﬁrst one is in the ARFF format of Weka. The labels should be speciﬁed as nominal attributes with values “0” and “1” indicating 2412 M ULAN : A JAVA L IBRARY FOR M ULTI -L ABEL L EARNING absence and presence of the label respectively. The second ﬁle is in XML format. It speciﬁes the labels and any hierarchical relationships among them. Hierarchies of labels can be expressed in the XML ﬁle by nesting the label tag. In our example, the two ﬁlenames are given to the experiment class through command-line parameters. String arffFile = Utils.getOption(

3 0.030941844 83 jmlr-2011-Scikit-learn: Machine Learning in Python

Author: Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, Édouard Duchesnay

Abstract: Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. This package focuses on bringing machine learning to non-specialists using a general-purpose high-level language. Emphasis is put on ease of use, performance, documentation, and API consistency. It has minimal dependencies and is distributed under the simpliﬁed BSD license, encouraging its use in both academic and commercial settings. Source code, binaries, and documentation can be downloaded from http://scikit-learn.sourceforge.net. Keywords: Python, supervised learning, unsupervised learning, model selection

4 0.024883278 102 jmlr-2011-Waffles: A Machine Learning Toolkit

Author: Michael Gashler

Abstract: We present a breadth-oriented collection of cross-platform command-line tools for researchers in machine learning called Wafﬂes. The Wafﬂes tools are designed to offer a broad spectrum of functionality in a manner that is friendly for scripted automation. All functionality is also available in a C++ class library. Wafﬂes is available under the GNU Lesser General Public License. Keywords: machine learning, toolkits, data mining, C++, open source

5 0.024721127 50 jmlr-2011-LPmade: Link Prediction Made Easy

Author: Ryan N. Lichtenwalter, Nitesh V. Chawla

Abstract: LPmade is a complete cross-platform software solution for multi-core link prediction and related tasks and analysis. Its ﬁrst principal contribution is a scalable network library supporting highperformance implementations of the most commonly employed unsupervised link prediction methods. Link prediction in longitudinal data requires a sophisticated and disciplined procedure for correct results and fair evaluation, so the second principle contribution of LPmade is a sophisticated GNU make architecture that completely automates link prediction, prediction evaluation, and network analysis. Finally, LPmade streamlines and automates the procedure for creating multivariate supervised link prediction models with a version of WEKA modiﬁed to operate effectively on extremely large data sets. With mere minutes of manual work, one may start with a raw stream of records representing a network and progress through hundreds of steps to generate plots, gigabytes or terabytes of output, and actionable or publishable results. Keywords: link prediction, network analysis, multicore, GNU make, PropFlow, HPLP

6 0.024137843 93 jmlr-2011-The arules R-Package Ecosystem: Analyzing Interesting Patterns from Large Transaction Data Sets

7 0.02316362 92 jmlr-2011-The Stationary Subspace Analysis Toolbox

8 0.022919575 66 jmlr-2011-Multiple Kernel Learning Algorithms

9 0.02277077 13 jmlr-2011-Bayesian Generalized Kernel Mixed Models

10 0.020593207 57 jmlr-2011-Learning a Robust Relevance Model for Search Using Kernel Methods

11 0.020341629 67 jmlr-2011-Multitask Sparsity via Maximum Entropy Discrimination

12 0.019806305 15 jmlr-2011-CARP: Software for Fishing Out Good Clustering Algorithms

13 0.019086806 52 jmlr-2011-Large Margin Hierarchical Classification with Mutually Exclusive Class Membership

14 0.018791627 105 jmlr-2011-lp-Norm Multiple Kernel Learning

15 0.017491892 28 jmlr-2011-Double Updating Online Learning

16 0.017085988 48 jmlr-2011-Kernel Analysis of Deep Networks

17 0.016774151 20 jmlr-2011-Convex and Network Flow Optimization for Structured Sparsity

18 0.016522499 79 jmlr-2011-Proximal Methods for Hierarchical Sparse Coding

19 0.015299935 51 jmlr-2011-Laplacian Support Vector Machines Trained in the Primal

20 0.015175248 64 jmlr-2011-Minimum Description Length Penalization for Group and Multi-Task Sparse Learning

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, 0.074), (1, -0.017), (2, 0.024), (3, -0.042), (4, -0.017), (5, 0.007), (6, -0.007), (7, -0.017), (8, -0.018), (9, -0.049), (10, -0.054), (11, 0.001), (12, 0.128), (13, 0.007), (14, -0.16), (15, -0.027), (16, -0.017), (17, 0.029), (18, 0.127), (19, 0.155), (20, 0.006), (21, -0.072), (22, -0.027), (23, -0.012), (24, -0.019), (25, 0.08), (26, -0.089), (27, 0.083), (28, 0.025), (29, -0.012), (30, 0.027), (31, 0.137), (32, -0.047), (33, 0.15), (34, -0.168), (35, 0.186), (36, -0.146), (37, -0.224), (38, 0.045), (39, -0.22), (40, 0.105), (41, 0.192), (42, 0.031), (43, 0.181), (44, -0.154), (45, -0.183), (46, 0.319), (47, -0.032), (48, 0.176), (49, -0.037)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.96485472 62 jmlr-2011-MSVMpack: A Multi-Class Support Vector Machine Package

Author: Fabien Lauer, Yann Guermeur

2 0.28244805 51 jmlr-2011-Laplacian Support Vector Machines Trained in the Primal

Author: Stefano Melacci, Mikhail Belkin

Abstract: In the last few years, due to the growing ubiquity of unlabeled data, much effort has been spent by the machine learning community to develop better understanding and improve the quality of classiﬁers exploiting unlabeled data. Following the manifold regularization approach, Laplacian Support Vector Machines (LapSVMs) have shown the state of the art performance in semi-supervised classiﬁcation. In this paper we present two strategies to solve the primal LapSVM problem, in order to overcome some issues of the original dual formulation. In particular, training a LapSVM in the primal can be efﬁciently performed with preconditioned conjugate gradient. We speed up training by using an early stopping strategy based on the prediction on unlabeled data or, if available, on labeled validation examples. This allows the algorithm to quickly compute approximate solutions with roughly the same classiﬁcation accuracy as the optimal ones, considerably reducing the training time. The computational complexity of the training algorithm is reduced from O(n3 ) to O(kn2 ), where n is the combined number of labeled and unlabeled examples and k is empirically evaluated to be signiﬁcantly smaller than n. Due to its simplicity, training LapSVM in the primal can be the starting point for additional enhancements of the original LapSVM formulation, such as those for dealing with large data sets. We present an extensive experimental evaluation on real world data showing the beneﬁts of the proposed approach. Keywords: Laplacian support vector machines, manifold regularization, semi-supervised learning, classiﬁcation, optimization

3 0.28181615 83 jmlr-2011-Scikit-learn: Machine Learning in Python

4 0.22835073 102 jmlr-2011-Waffles: A Machine Learning Toolkit

Author: Michael Gashler

5 0.22016522 13 jmlr-2011-Bayesian Generalized Kernel Mixed Models

Author: Zhihua Zhang, Guang Dai, Michael I. Jordan

Abstract: We propose a fully Bayesian methodology for generalized kernel mixed models (GKMMs), which are extensions of generalized linear mixed models in the feature space induced by a reproducing kernel. We place a mixture of a point-mass distribution and Silverman’s g-prior on the regression vector of a generalized kernel model (GKM). This mixture prior allows a fraction of the components of the regression vector to be zero. Thus, it serves for sparse modeling and is useful for Bayesian computation. In particular, we exploit data augmentation methodology to develop a Markov chain Monte Carlo (MCMC) algorithm in which the reversible jump method is used for model selection and a Bayesian model averaging method is used for posterior prediction. When the feature basis expansion in the reproducing kernel Hilbert space is treated as a stochastic process, this approach can be related to the Karhunen-Lo` ve expansion of a Gaussian process (GP). Thus, our sparse e modeling framework leads to a ﬂexible approximation method for GPs. Keywords: reproducing kernel Hilbert spaces, generalized kernel models, Silverman’s g-prior, Bayesian model averaging, Gaussian processes

6 0.20524739 49 jmlr-2011-Kernel Regression in the Presence of Correlated Errors

7 0.19911794 93 jmlr-2011-The arules R-Package Ecosystem: Analyzing Interesting Patterns from Large Transaction Data Sets

8 0.19775884 52 jmlr-2011-Large Margin Hierarchical Classification with Mutually Exclusive Class Membership

9 0.19634271 67 jmlr-2011-Multitask Sparsity via Maximum Entropy Discrimination

10 0.1857084 6 jmlr-2011-A Simpler Approach to Matrix Completion

11 0.16108415 15 jmlr-2011-CARP: Software for Fishing Out Good Clustering Algorithms

12 0.16091439 32 jmlr-2011-Exploitation of Machine Learning Techniques in Modelling Phrase Movements for Machine Translation

13 0.14498909 31 jmlr-2011-Efficient and Effective Visual Codebook Generation Using Additive Kernels

14 0.1399671 63 jmlr-2011-MULAN: A Java Library for Multi-Label Learning

15 0.12256113 57 jmlr-2011-Learning a Robust Relevance Model for Search Using Kernel Methods

16 0.12237208 56 jmlr-2011-Learning Transformation Models for Ranking and Survival Analysis

17 0.12044159 28 jmlr-2011-Double Updating Online Learning

18 0.11610377 64 jmlr-2011-Minimum Description Length Penalization for Group and Multi-Task Sparse Learning

19 0.11479601 71 jmlr-2011-On Equivalence Relationships Between Classification and Ranking Algorithms

20 0.11140878 92 jmlr-2011-The Stationary Subspace Analysis Toolbox

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(4, 0.028), (9, 0.051), (10, 0.027), (24, 0.045), (31, 0.057), (32, 0.043), (36, 0.015), (41, 0.015), (55, 0.473), (60, 0.054), (65, 0.014), (66, 0.016), (73, 0.017), (78, 0.032)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.69788629 62 jmlr-2011-MSVMpack: A Multi-Class Support Vector Machine Package

Author: Fabien Lauer, Yann Guermeur

2 0.2248351 42 jmlr-2011-In All Likelihood, Deep Belief Is Not Enough

Author: Lucas Theis, Sebastian Gerwinn, Fabian Sinz, Matthias Bethge

Abstract: Statistical models of natural images provide an important tool for researchers in the ﬁelds of machine learning and computational neuroscience. The canonical measure to quantitatively assess and compare the performance of statistical models is given by the likelihood. One class of statistical models which has recently gained increasing popularity and has been applied to a variety of complex data is formed by deep belief networks. Analyses of these models, however, have often been limited to qualitative analyses based on samples due to the computationally intractable nature of their likelihood. Motivated by these circumstances, the present article introduces a consistent estimator for the likelihood of deep belief networks which is computationally tractable and simple to apply in practice. Using this estimator, we quantitatively investigate a deep belief network for natural image patches and compare its performance to the performance of other models for natural image patches. We ﬁnd that the deep belief network is outperformed with respect to the likelihood even by very simple mixture models. Keywords: deep belief network, restricted Boltzmann machine, likelihood estimation, natural image statistics, potential log-likelihood

3 0.20810229 48 jmlr-2011-Kernel Analysis of Deep Networks

Author: Grégoire Montavon, Mikio L. Braun, Klaus-Robert Müller

Abstract: When training deep networks it is common knowledge that an efﬁcient and well generalizing representation of the problem is formed. In this paper we aim to elucidate what makes the emerging representation successful. We analyze the layer-wise evolution of the representation in a deep network by building a sequence of deeper and deeper kernels that subsume the mapping performed by more and more layers of the deep network and measuring how these increasingly complex kernels ﬁt the learning problem. We observe that deep networks create increasingly better representations of the learning problem and that the structure of the deep network controls how fast the representation of the task is formed layer after layer. Keywords: deep networks, kernel principal component analysis, representations

4 0.20719063 105 jmlr-2011-lp-Norm Multiple Kernel Learning

Author: Marius Kloft, Ulf Brefeld, Sören Sonnenburg, Alexander Zien

Abstract: Learning linear combinations of multiple kernels is an appealing strategy when the right choice of features is unknown. Previous approaches to multiple kernel learning (MKL) promote sparse kernel combinations to support interpretability and scalability. Unfortunately, this ℓ1 -norm MKL is rarely observed to outperform trivial baselines in practical applications. To allow for robust kernel mixtures that generalize well, we extend MKL to arbitrary norms. We devise new insights on the connection between several existing MKL formulations and develop two efﬁcient interleaved optimization strategies for arbitrary norms, that is ℓ p -norms with p ≥ 1. This interleaved optimization is much faster than the commonly used wrapper approaches, as demonstrated on several data sets. A theoretical analysis and an experiment on controlled artiﬁcial data shed light on the appropriateness of sparse, non-sparse and ℓ∞ -norm MKL in various scenarios. Importantly, empirical applications of ℓ p -norm MKL to three real-world problems from computational biology show that non-sparse MKL achieves accuracies that surpass the state-of-the-art. Data sets, source code to reproduce the experiments, implementations of the algorithms, and further information are available at http://doc.ml.tu-berlin.de/nonsparse_mkl/. Keywords: multiple kernel learning, learning kernels, non-sparse, support vector machine, convex conjugate, block coordinate descent, large scale optimization, bioinformatics, generalization bounds, Rademacher complexity ∗. Also at Machine Learning Group, Technische Universit¨ t Berlin, 10587 Berlin, Germany. a †. Parts of this work were done while SS was at the Friedrich Miescher Laboratory, Max Planck Society, 72076 T¨ bingen, Germany. u ‡. Most contributions by AZ were done at the Fraunhofer Institute FIRST, 12489 Berlin, Germany. c 2011 Marius Kloft, Ulf Brefeld, S¨ ren Sonnenburg and Alexander Zien. o K LOFT, B REFELD , S ONNENBURG AND Z IEN

5 0.20674284 43 jmlr-2011-Information, Divergence and Risk for Binary Experiments

Author: Mark D. Reid, Robert C. Williamson

Abstract: We unify f -divergences, Bregman divergences, surrogate regret bounds, proper scoring rules, cost curves, ROC-curves and statistical information. We do this by systematically studying integral and variational representations of these objects and in so doing identify their representation primitives which all are related to cost-sensitive binary classiﬁcation. As well as developing relationships between generative and discriminative views of learning, the new machinery leads to tight and more general surrogate regret bounds and generalised Pinsker inequalities relating f -divergences to variational divergence. The new viewpoint also illuminates existing algorithms: it provides a new derivation of Support Vector Machines in terms of divergences and relates maximum mean discrepancy to Fisher linear discriminants. Keywords: classiﬁcation, loss functions, divergence, statistical information, regret bounds

6 0.2047378 67 jmlr-2011-Multitask Sparsity via Maximum Entropy Discrimination

7 0.19869253 20 jmlr-2011-Convex and Network Flow Optimization for Structured Sparsity

8 0.19822416 64 jmlr-2011-Minimum Description Length Penalization for Group and Multi-Task Sparse Learning

9 0.1976091 25 jmlr-2011-Discriminative Learning of Bayesian Networks via Factorized Conditional Log-Likelihood

10 0.19708836 86 jmlr-2011-Sparse Linear Identifiable Multivariate Modeling

11 0.19396833 77 jmlr-2011-Posterior Sparsity in Unsupervised Dependency Parsing

12 0.19394194 96 jmlr-2011-Two Distributed-State Models For Generating High-Dimensional Time Series

13 0.19326206 66 jmlr-2011-Multiple Kernel Learning Algorithms

14 0.19319502 79 jmlr-2011-Proximal Methods for Hierarchical Sparse Coding

15 0.19306892 91 jmlr-2011-The Sample Complexity of Dictionary Learning

16 0.19188964 88 jmlr-2011-Structured Variable Selection with Sparsity-Inducing Norms

17 0.19017403 74 jmlr-2011-Operator Norm Convergence of Spectral Clustering on Level Sets

18 0.1885834 4 jmlr-2011-A Family of Simple Non-Parametric Kernel Learning Algorithms

19 0.18798475 102 jmlr-2011-Waffles: A Machine Learning Toolkit

20 0.18796767 12 jmlr-2011-Bayesian Co-Training