nips nips2006 nips2006-179 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Ke Huang, Selin Aviyente
Abstract: In this paper, application of sparse representation (factorization) of signals over an overcomplete basis (dictionary) for signal classification is discussed. Searching for the sparse representation of a signal over an overcomplete dictionary is achieved by optimizing an objective function that includes two terms: one that measures the signal reconstruction error and another that measures the sparsity. This objective function works well in applications where signals need to be reconstructed, like coding and denoising. On the other hand, discriminative methods, such as linear discriminative analysis (LDA), are better suited for classification tasks. However, discriminative methods are usually sensitive to corruption in signals due to lacking crucial properties for signal reconstruction. In this paper, we present a theoretical framework for signal classification with sparse representation. The approach combines the discrimination power of the discriminative methods with the reconstruction property and the sparsity of the sparse representation that enables one to deal with signal corruptions: noise, missing data and outliers. The proposed approach is therefore capable of robust classification with a sparse representation of signals. The theoretical results are demonstrated with signal classification tasks, showing that the proposed approach outperforms the standard discriminative methods and the standard sparse representation in the case of corrupted signals. 1
Reference: text
sentIndex sentText sentNum sentScore
1 edu Abstract In this paper, application of sparse representation (factorization) of signals over an overcomplete basis (dictionary) for signal classification is discussed. [sent-3, score-0.907]
2 Searching for the sparse representation of a signal over an overcomplete dictionary is achieved by optimizing an objective function that includes two terms: one that measures the signal reconstruction error and another that measures the sparsity. [sent-4, score-1.339]
3 This objective function works well in applications where signals need to be reconstructed, like coding and denoising. [sent-5, score-0.274]
4 On the other hand, discriminative methods, such as linear discriminative analysis (LDA), are better suited for classification tasks. [sent-6, score-0.318]
5 However, discriminative methods are usually sensitive to corruption in signals due to lacking crucial properties for signal reconstruction. [sent-7, score-0.702]
6 In this paper, we present a theoretical framework for signal classification with sparse representation. [sent-8, score-0.458]
7 The approach combines the discrimination power of the discriminative methods with the reconstruction property and the sparsity of the sparse representation that enables one to deal with signal corruptions: noise, missing data and outliers. [sent-9, score-1.581]
8 The proposed approach is therefore capable of robust classification with a sparse representation of signals. [sent-10, score-0.396]
9 The theoretical results are demonstrated with signal classification tasks, showing that the proposed approach outperforms the standard discriminative methods and the standard sparse representation in the case of corrupted signals. [sent-11, score-0.842]
10 The problem solved by the sparse representation is to search for the most compact representation of a signal in terms of linear combination of atoms in an overcomplete dictionary. [sent-13, score-0.865]
11 Recent developments in multi-scale and multi-orientation representation of signals, such as wavelet, ridgelet, curvelet and contourlet transforms are an important incentive for the research on the sparse representation. [sent-14, score-0.513]
12 Compared to methods based on orthonormal transforms or direct time domain processing, sparse representation usually offers better performance with its capacity for efficient signal modelling. [sent-15, score-0.643]
13 For instance, in [6], sparse representation is used for image separation. [sent-17, score-0.407]
14 The overcomplete dictionary is generated by combining multiple standard transforms, including curvelet transform, ridgelet transform and discrete cosine transform. [sent-18, score-0.378]
15 In [7], application of the sparse representation to blind source separation is discussed and experimental results on EEG data analysis are demonstrated. [sent-19, score-0.41]
16 In [8], a sparse image coding method with the wavelet transform is presented. [sent-20, score-0.365]
17 In [9], sparse representation with an adaptive dictionary is shown to have state-of-the-art performance in image denoising. [sent-21, score-0.552]
18 The widely used shrinkage method for image desnoising is shown to be the first iteration of basis pursuit that solves the sparse representation problem [10]. [sent-22, score-0.818]
19 In the standard framework of sparse representation, the objective is to reduce the signal reconstruction error with as few number of atoms as possible. [sent-23, score-0.767]
20 However, discriminative methods are usually sensitive to corruption in signals due to lacking crucial properties for signal reconstruction. [sent-25, score-0.702]
21 In this paper, we propose the method of sparse representation for signal classification (SRSC), which modifies the standard sparse representation framework for signal classification. [sent-26, score-1.162]
22 We first show that replacing the reconstruction error with discrimination power in the objective function of the sparse representation is more suitable for the tasks of classification. [sent-27, score-1.055]
23 When the signal is corrupted, the discriminative methods may fail because little information is contained in discriminative analysis to successfully deal with noise, missing data and outliers. [sent-28, score-0.672]
24 To address this robustness problem, the proposed approach of SRSC combines discrimination power, signal reconstruction and sparsity in the objective function for classification. [sent-29, score-0.948]
25 With the theoretical framework of SRSC, our objective is to achieve a sparse and robust representation of corrupted signals for effective classification. [sent-30, score-0.723]
26 Section 2 reviews the problem formulation and solution for the standard sparse representation. [sent-32, score-0.254]
27 Section 3 discusses the motivations for proposing SRSC by analyzing the reconstructive methods and discriminative methods for signal classification. [sent-33, score-0.615]
28 2 Sparse Representation of Signal The problem of finding the sparse representation of a signal in a given overcomplete dictionary can be formulated as follows. [sent-36, score-0.862]
29 Given a N × M matrix A containing the elements of an overcomplete dictionary in its columns, with M > N and usually M >> N , and a signal y ∈ RN , the problem of sparse representation is to find an M × 1 coefficient vector x, such that y = Ax and x 0 is minimized, i. [sent-37, score-0.835]
30 Suboptimal solutions to this problem can be found by iterative methods like the matching pursuit and orthogonal matching pursuit. [sent-44, score-0.526]
31 , the solution is sparse enough, the solution of equation (1) is equivalent to the solution of equation (2), which can be efficiently solved by basis pursuit using linear programming. [sent-51, score-0.782]
32 Except for the intuitive interpretation as obtaining a sparse factorization that minimizes signal reconstruction error, the problem formulated in equation (3) has an equivalent interpretation in the framework of Bayesian decision as follows [13]. [sent-54, score-0.717]
33 3 Reconstruction and Discrimination Sparse representation works well in applications where the original signal y needs to be reconstructed as accurately as possible, such as denoising, image inpainting and coding. [sent-59, score-0.464]
34 However, for applications like signal classification, it is more important that the representation is discriminative for the given signal classes than a small reconstruction error. [sent-60, score-0.917]
35 The difference between reconstruction and discrimination has been widely investigated in literature. [sent-61, score-0.508]
36 It is known that typical reconstructive methods, such as principal component analysis (PCA) and independent component analysis (ICA), aim at obtaining a representation that enables sufficient reconstruction, thus are able to deal with signal corruption, i. [sent-62, score-0.56]
37 On the other hand, discriminative methods, such as LDA [14], generate a signal representation that maximizes the separation of distributions of signals from different classes. [sent-65, score-0.717]
38 While both methods have broad applications in classification, the discriminative methods have often outperformed the reconstructive methods for the classification task [15, 16]. [sent-66, score-0.415]
39 When this assumption does not hold, the classification may suffer from the nonrobust nature of the discriminative methods that contains insufficient information to successfully deal with signal corruptions. [sent-70, score-0.444]
40 Specifically, the representation provided by the discriminative methods for optimal classification does not necessarily contain sufficient information for signal reconstruction, which is necessary for removing noise, recovering missing data and detecting outliers. [sent-71, score-0.603]
41 This performance degradation of discriminative methods on corrupted signals is evident in the examples shown in [17]. [sent-72, score-0.438]
42 In [9], the sparse representation is shown to achieve state-of-the-art performance in image denoising. [sent-74, score-0.407]
43 In [18], missing pixels in images are successfully recovered by inpainting method based on sparse representation. [sent-75, score-0.364]
44 All of these examples motivate the design of a new signal representation that combines the advantages of both reconstructive and discriminative methods to address the problem of robust classification when the obtained signals are corrupted. [sent-77, score-0.951]
45 The proposed method should generate a representation that contain discriminative information for classification, crucial information for signal reconstruction and preferably the representation should be sparse. [sent-78, score-0.842]
46 Due to the evident reconstructive properties [9, 18], the available efficient pursuit methods and the sparsity of representation, we choose the sparse representation as the basic framework for the SRSC and incorporate a measure of discrimination power into the objective function. [sent-79, score-1.488]
47 Therefore, the sparse representation obtained by the proposed SRSC contains both crucial information for reconstruction and discriminative information for classification, which enable a reasonable classification performance in the case of corrupted signals. [sent-80, score-0.801]
48 The three objectives: sparsity, reconstruction and discrimination may not always be consistent. [sent-81, score-0.508]
49 It should be noted that the aim of SRSC is not to improve the standard discriminative methods like LDA in the case of ideal signals, but to achieve comparable classification results when the signals are corrupted. [sent-83, score-0.401]
50 4 Sparse Representation for Signal Classification In this section, the SRSC problem is formulated mathematically and a pursuit method is proposed to optimize the objective function. [sent-86, score-0.447]
51 We first replace the term measuring reconstruction error with a term measuring discrimination power to show the different effects of reconstruction and discrimination. [sent-87, score-0.784]
52 Further, we incorporate measure of discrimination power in the framework of standard sparse representation to effectively address the problem of classifying corrupted signals. [sent-88, score-0.849]
53 The Fisher’s discrimination criterion [14] used in the LDA is applied to quantify the discrimination power. [sent-89, score-0.65]
54 The extracted feature should be as discriminative as possible between the different signal classes. [sent-93, score-0.408]
55 Suppose that we have a set of K signals in a signal matrix Y = [y1 , y2 , . [sent-94, score-0.403]
56 , yK ] with the corresponding representation in the overcomplete dictionary as X = [x1 , x2 , . [sent-97, score-0.377]
57 Finally, the Fisher’s discrimination power 2 T Ki (mi − m)(mi − m) C i=1 The difference between the sample means SB = 2 . [sent-102, score-0.418]
58 Fisher’s criterion is motivated by the intuitive idea that the discrimination power is maximized when the spatial distribution of different classes are as far away as possible and the spatial distribution of samples from the same class are as close as possible. [sent-104, score-0.418]
59 Maximizing J2 (X, λ) generates a sparse representation that has a good discrimination power. [sent-106, score-0.68]
60 Maximizing J3 (X, λ1 , λ2 ) ensures that a representation with discrimination power, reconstruction property and sparsity is extracted for robust classification of corrupted signals. [sent-108, score-0.862]
61 In the case that the signals are corrupted, the two terms K i=1 xi K 0 and i=1 yi − Axi 2 2 robustly recover the signal structure, as in [9, 18]. [sent-109, score-0.403]
62 On the other hand, the inclusion of the term F (X) requires that the obtained representation contains discriminative information for classification. [sent-110, score-0.306]
63 2 Problem Solution Both the objective function J2 (X, λ) defined in equation (9) and the objective function J3 (X, λ1 , λ2 ) defined in equation (10) have similar forms to the objective function defined in the standard sparse representation, as J1 (x; λ) in equation (3). [sent-113, score-0.601]
64 Therefore, not all the pursuit methods, such as basis pursuit and LARS/Homotopy methods, that are applicable to the standard sparse representation method can be directly applied to optimize J2 (X, λ) and J3 (X, λ1 , λ2 ). [sent-115, score-1.087]
65 However, the iterative optimization methods employed in the matching pursuit and the orthogonal matching pursuit provide a direct reference to the optimization of J2 (X, λ) and J3 (X, λ1 , λ2 ). [sent-116, score-0.872]
66 In this paper, we propose an algorithm similar to the orthogonal matching pursuit and inspired by the simultaneous sparse approximation algorithm described in [20, 21]. [sent-117, score-0.712]
67 Taking the optimization of J3 (X, λ1 , λ2 ) as example, the pursuit algorithm can be summarized as follows: 1. [sent-118, score-0.346]
68 The pursuit algorithm for optimizing J2 (X, λ) also follows the same steps. [sent-125, score-0.367]
69 Detailed analysis of this pursuit algorithm can be found in [20, 21]. [sent-126, score-0.346]
70 1, synthesized signals are generated to show the difference between the features extracted by J1 (X, λ) and J2 (X, λ), which reflects the properties of reconstruction and discrimination. [sent-129, score-0.383]
71 Random noise and occlusion are added to the original signals to test the robustness of SRSC. [sent-132, score-0.343]
72 The signals are constructed to show the difference between the reconstructive methods and discriminative methods. [sent-135, score-0.54]
73 Therefore, most of the energy of the signal can be described by the sine function and most of the discrimination power is in the cosine function. [sent-139, score-0.669]
74 The signal component with most energy is not necessary the component with the most discrimination power. [sent-140, score-0.551]
75 Construct a dictionary as {sin t, cos t}, optimizing the objective function J1 (X, λ) with the pursuit method described in Section 4. [sent-141, score-0.628]
76 In the simulation, 100 samples are generated for each class and the pursuit algorithm stops at the first run. [sent-144, score-0.346]
77 The projection of the signals from both classes to the first atom selected by J1 (X, λ) and J2 (X, λ) are shown in Fig. [sent-145, score-0.259]
78 In this implementation, the overcomplete dictionary is a combination of Haar wavelet basis and Gabor basis. [sent-153, score-0.352]
79 Haar basis is good at modelling discontinuities in signal and on the other hand, Gabor basis is good at modelling continuous signal components. [sent-154, score-0.532]
80 In this experiment, noise and occlusion are added to the signals to test the robustness of SRSC. [sent-155, score-0.343]
81 Results in Table 1 and Table 2 show that in the case that signals are ideal (without missing data and noiseless) or nearly ideal, J2 (X, λ) is the best criterion for classification. [sent-161, score-0.285]
82 This is consistent with the known conclusion that discriminative methods outperform reconstructive methods in classification. [sent-162, score-0.389]
83 2590 that the signal structures recovered by the standard sparse representation are more robust to noise and occlusion, thus yield less performance degradation. [sent-197, score-0.663]
84 On the other hand, the SRSC demonstrates lower error rate by the combination of the reconstruction property and the discrimination power in the case that signals are noisy or with occlusions. [sent-198, score-0.778]
85 6 Discussions In summary, sparse representation for signal classification(SRSC) is proposed. [sent-199, score-0.581]
86 SRSC is motivated by the ongoing researches in the area of sparse representation in the signal processing area. [sent-200, score-0.581]
87 SRSC incorporates reconstruction properties, discrimination power and sparsity for robust classification. [sent-201, score-0.733]
88 Both SRSC and RVM incorporate sparsity and reconstruction error into consideration. [sent-206, score-0.274]
89 In RVM, the “dictionary” used for signal representation is the collection of values from the “kernel function”. [sent-209, score-0.349]
90 On the other hand, SRSC roots in the standard sparse representation and recent developments of harmonic analysis, such as curvelet, bandlet, contourlet transforms that show excellent properties in signal modelling. [sent-210, score-0.707]
91 Another difference between SRSC and RVM is how the discrimination power is incorporated. [sent-212, score-0.418]
92 For SRSC, the discrimination power is explicitly incorporated by inclusion of a measure based on the Fisher’s discrimination. [sent-215, score-0.442]
93 Bruckstein, “The K-SVD: An algorithm for designing of overcomplete dictionaries for sparse representation,” IEEE Trans. [sent-244, score-0.393]
94 Donoho, “Image decomposition via the combination of sparse representation and a variational approach,” IEEE Trans. [sent-249, score-0.383]
95 Amari, “Analysis of sparse representation and blind source separation,” Neural Computation, vol. [sent-257, score-0.378]
96 Lewicki, “Learning sparse image codes using a wavelet pyramid architecture,” in NIPS, 2001, pp. [sent-264, score-0.342]
97 Aharon, “Image denoising via learned dictionaries and sparse representation,” in CVPR, 2006. [sent-268, score-0.327]
98 Leonardis, “Combining reconstructive and discriminative subspace methods for robust classification and regression by subsampling,” IEEE Trans. [sent-315, score-0.404]
99 part I: Greedy pursuit,” Signal Processing, special issue on Sparse approximations in signal and image processing, vol. [sent-337, score-0.278]
100 part II: Convex relaxation,” Signal Processing, special issue on Sparse approximations in signal and image processing, vol. [sent-345, score-0.278]
wordName wordTfidf (topN-words)
[('srsc', 0.495), ('pursuit', 0.346), ('discrimination', 0.325), ('sparse', 0.232), ('signal', 0.226), ('reconstruction', 0.183), ('reconstructive', 0.178), ('signals', 0.177), ('discriminative', 0.159), ('rvm', 0.157), ('dictionary', 0.145), ('representation', 0.123), ('overcomplete', 0.109), ('classi', 0.107), ('occlusion', 0.097), ('power', 0.093), ('sparsity', 0.091), ('lda', 0.088), ('elad', 0.086), ('corrupted', 0.076), ('objective', 0.074), ('missing', 0.069), ('inpainting', 0.063), ('donoho', 0.063), ('atom', 0.059), ('curvelet', 0.059), ('mi', 0.059), ('wavelet', 0.058), ('matching', 0.055), ('ax', 0.054), ('image', 0.052), ('atoms', 0.052), ('dictionaries', 0.052), ('corruption', 0.052), ('scalar', 0.05), ('weighting', 0.049), ('equation', 0.049), ('ki', 0.047), ('orthogonal', 0.044), ('subsampling', 0.044), ('sin', 0.043), ('denoising', 0.043), ('cation', 0.043), ('cos', 0.042), ('robust', 0.041), ('noise', 0.041), ('basis', 0.04), ('aviyente', 0.04), ('axi', 0.04), ('contourlet', 0.04), ('overlayed', 0.04), ('ridgelet', 0.04), ('starck', 0.04), ('strauss', 0.04), ('ideal', 0.039), ('fisher', 0.039), ('transforms', 0.036), ('simultaneous', 0.035), ('tradeoff', 0.035), ('lacking', 0.034), ('tropp', 0.034), ('gilbert', 0.034), ('leonardis', 0.034), ('deal', 0.033), ('separation', 0.032), ('coefficient', 0.031), ('noiseless', 0.029), ('sb', 0.029), ('haar', 0.029), ('atomic', 0.029), ('aharon', 0.029), ('adjust', 0.029), ('decomposition', 0.028), ('hand', 0.028), ('crucial', 0.028), ('robustness', 0.028), ('harmonic', 0.027), ('ci', 0.027), ('formulated', 0.027), ('pca', 0.027), ('methods', 0.026), ('replacing', 0.025), ('conducted', 0.025), ('cosine', 0.025), ('shrinkage', 0.025), ('icassp', 0.025), ('usps', 0.024), ('sw', 0.024), ('inclusion', 0.024), ('coding', 0.023), ('objectives', 0.023), ('developments', 0.023), ('blind', 0.023), ('projection', 0.023), ('extracted', 0.023), ('gabor', 0.022), ('solution', 0.022), ('combines', 0.021), ('optimizing', 0.021), ('bayesian', 0.021)]
simIndex simValue paperId paperTitle
same-paper 1 0.99999994 179 nips-2006-Sparse Representation for Signal Classification
Author: Ke Huang, Selin Aviyente
Abstract: In this paper, application of sparse representation (factorization) of signals over an overcomplete basis (dictionary) for signal classification is discussed. Searching for the sparse representation of a signal over an overcomplete dictionary is achieved by optimizing an objective function that includes two terms: one that measures the signal reconstruction error and another that measures the sparsity. This objective function works well in applications where signals need to be reconstructed, like coding and denoising. On the other hand, discriminative methods, such as linear discriminative analysis (LDA), are better suited for classification tasks. However, discriminative methods are usually sensitive to corruption in signals due to lacking crucial properties for signal reconstruction. In this paper, we present a theoretical framework for signal classification with sparse representation. The approach combines the discrimination power of the discriminative methods with the reconstruction property and the sparsity of the sparse representation that enables one to deal with signal corruptions: noise, missing data and outliers. The proposed approach is therefore capable of robust classification with a sparse representation of signals. The theoretical results are demonstrated with signal classification tasks, showing that the proposed approach outperforms the standard discriminative methods and the standard sparse representation in the case of corrupted signals. 1
2 0.11233373 46 nips-2006-Blind source separation for over-determined delayed mixtures
Author: Lars Omlor, Martin Giese
Abstract: Blind source separation, i.e. the extraction of unknown sources from a set of given signals, is relevant for many applications. A special case of this problem is dimension reduction, where the goal is to approximate a given set of signals by superpositions of a minimal number of sources. Since in this case the signals outnumber the sources the problem is over-determined. Most popular approaches for addressing this problem are based on purely linear mixing models. However, many applications like the modeling of acoustic signals, EMG signals, or movement trajectories, require temporal shift-invariance of the extracted components. This case has only rarely been treated in the computational literature, and specifically for the case of dimension reduction almost no algorithms have been proposed. We present a new algorithm for the solution of this problem, which is based on a timefrequency transformation (Wigner-Ville distribution) of the generative model. We show that this algorithm outperforms classical source separation algorithms for linear mixtures, and also a related method for mixtures with delays. In addition, applying the new algorithm to trajectories of human gaits, we demonstrate that it is suitable for the extraction of spatio-temporal components that are easier to interpret than components extracted with other classical algorithms. 1
3 0.10678688 75 nips-2006-Efficient sparse coding algorithms
Author: Honglak Lee, Alexis Battle, Rajat Raina, Andrew Y. Ng
Abstract: Sparse coding provides a class of algorithms for finding succinct representations of stimuli; given only unlabeled input data, it discovers basis functions that capture higher-level features in the data. However, finding sparse codes remains a very difficult computational problem. In this paper, we present efficient sparse coding algorithms that are based on iteratively solving two convex optimization problems: an L1 -regularized least squares problem and an L2 -constrained least squares problem. We propose novel algorithms to solve both of these optimization problems. Our algorithms result in a significant speedup for sparse coding, allowing us to learn larger sparse codes than possible with previously described algorithms. We apply these algorithms to natural images and demonstrate that the inferred sparse codes exhibit end-stopping and non-classical receptive field surround suppression and, therefore, may provide a partial explanation for these two phenomena in V1 neurons. 1
4 0.082433946 51 nips-2006-Clustering Under Prior Knowledge with Application to Image Segmentation
Author: Dong S. Cheng, Vittorio Murino, Mário Figueiredo
Abstract: This paper proposes a new approach to model-based clustering under prior knowledge. The proposed formulation can be interpreted from two different angles: as penalized logistic regression, where the class labels are only indirectly observed (via the probability density of each class); as finite mixture learning under a grouping prior. To estimate the parameters of the proposed model, we derive a (generalized) EM algorithm with a closed-form E-step, in contrast with other recent approaches to semi-supervised probabilistic clustering which require Gibbs sampling or suboptimal shortcuts. We show that our approach is ideally suited for image segmentation: it avoids the combinatorial nature Markov random field priors, and opens the door to more sophisticated spatial priors (e.g., wavelet-based) in a simple and computationally efficient way. Finally, we extend our formulation to work in unsupervised, semi-supervised, or discriminative modes. 1
5 0.079118267 72 nips-2006-Efficient Learning of Sparse Representations with an Energy-Based Model
Author: Marc'aurelio Ranzato, Christopher Poultney, Sumit Chopra, Yann L. Cun
Abstract: We describe a novel unsupervised method for learning sparse, overcomplete features. The model uses a linear encoder, and a linear decoder preceded by a sparsifying non-linearity that turns a code vector into a quasi-binary sparse code vector. Given an input, the optimal code minimizes the distance between the output of the decoder and the input patch while being as similar as possible to the encoder output. Learning proceeds in a two-phase EM-like fashion: (1) compute the minimum-energy code vector, (2) adjust the parameters of the encoder and decoder so as to decrease the energy. The model produces “stroke detectors” when trained on handwritten numerals, and Gabor-like filters when trained on natural image patches. Inference and learning are very fast, requiring no preprocessing, and no expensive sampling. Using the proposed unsupervised method to initialize the first layer of a convolutional network, we achieved an error rate slightly lower than the best reported result on the MNIST dataset. Finally, an extension of the method is described to learn topographical filter maps. 1
6 0.07581649 130 nips-2006-Max-margin classification of incomplete data
7 0.073591188 24 nips-2006-Aggregating Classification Accuracy across Time: Application to Single Trial EEG
8 0.072669387 149 nips-2006-Nonnegative Sparse PCA
9 0.071083054 107 nips-2006-Large Margin Multi-channel Analog-to-Digital Conversion with Applications to Neural Prosthesis
10 0.070951968 186 nips-2006-Support Vector Machines on a Budget
11 0.070574172 16 nips-2006-A Theory of Retinal Population Coding
12 0.069478422 66 nips-2006-Detecting Humans via Their Pose
13 0.066402107 185 nips-2006-Subordinate class recognition using relational object models
14 0.063307486 126 nips-2006-Logistic Regression for Single Trial EEG Classification
15 0.062689602 78 nips-2006-Fast Discriminative Visual Codebooks using Randomized Clustering Forests
16 0.059730463 178 nips-2006-Sparse Multinomial Logistic Regression via Bayesian L1 Regularisation
17 0.059262887 199 nips-2006-Unsupervised Learning of a Probabilistic Grammar for Object Detection and Parsing
18 0.053270873 106 nips-2006-Large Margin Hidden Markov Models for Automatic Speech Recognition
19 0.052706711 65 nips-2006-Denoising and Dimension Reduction in Feature Space
20 0.052288301 128 nips-2006-Manifold Denoising
topicId topicWeight
[(0, -0.179), (1, 0.001), (2, 0.096), (3, -0.037), (4, -0.096), (5, -0.004), (6, -0.054), (7, -0.026), (8, -0.003), (9, 0.038), (10, -0.048), (11, -0.037), (12, -0.029), (13, -0.07), (14, 0.135), (15, 0.021), (16, 0.028), (17, -0.001), (18, 0.139), (19, 0.022), (20, -0.004), (21, -0.05), (22, 0.092), (23, -0.13), (24, 0.116), (25, 0.06), (26, -0.094), (27, 0.003), (28, -0.016), (29, 0.006), (30, 0.143), (31, -0.085), (32, -0.001), (33, -0.013), (34, -0.014), (35, 0.109), (36, -0.062), (37, -0.004), (38, 0.025), (39, -0.053), (40, 0.05), (41, 0.04), (42, -0.107), (43, -0.089), (44, -0.092), (45, 0.137), (46, 0.033), (47, 0.094), (48, 0.109), (49, -0.042)]
simIndex simValue paperId paperTitle
same-paper 1 0.95400035 179 nips-2006-Sparse Representation for Signal Classification
Author: Ke Huang, Selin Aviyente
Abstract: In this paper, application of sparse representation (factorization) of signals over an overcomplete basis (dictionary) for signal classification is discussed. Searching for the sparse representation of a signal over an overcomplete dictionary is achieved by optimizing an objective function that includes two terms: one that measures the signal reconstruction error and another that measures the sparsity. This objective function works well in applications where signals need to be reconstructed, like coding and denoising. On the other hand, discriminative methods, such as linear discriminative analysis (LDA), are better suited for classification tasks. However, discriminative methods are usually sensitive to corruption in signals due to lacking crucial properties for signal reconstruction. In this paper, we present a theoretical framework for signal classification with sparse representation. The approach combines the discrimination power of the discriminative methods with the reconstruction property and the sparsity of the sparse representation that enables one to deal with signal corruptions: noise, missing data and outliers. The proposed approach is therefore capable of robust classification with a sparse representation of signals. The theoretical results are demonstrated with signal classification tasks, showing that the proposed approach outperforms the standard discriminative methods and the standard sparse representation in the case of corrupted signals. 1
2 0.67670226 75 nips-2006-Efficient sparse coding algorithms
Author: Honglak Lee, Alexis Battle, Rajat Raina, Andrew Y. Ng
Abstract: Sparse coding provides a class of algorithms for finding succinct representations of stimuli; given only unlabeled input data, it discovers basis functions that capture higher-level features in the data. However, finding sparse codes remains a very difficult computational problem. In this paper, we present efficient sparse coding algorithms that are based on iteratively solving two convex optimization problems: an L1 -regularized least squares problem and an L2 -constrained least squares problem. We propose novel algorithms to solve both of these optimization problems. Our algorithms result in a significant speedup for sparse coding, allowing us to learn larger sparse codes than possible with previously described algorithms. We apply these algorithms to natural images and demonstrate that the inferred sparse codes exhibit end-stopping and non-classical receptive field surround suppression and, therefore, may provide a partial explanation for these two phenomena in V1 neurons. 1
3 0.62036622 107 nips-2006-Large Margin Multi-channel Analog-to-Digital Conversion with Applications to Neural Prosthesis
Author: Amit Gore, Shantanu Chakrabartty
Abstract: A key challenge in designing analog-to-digital converters for cortically implanted prosthesis is to sense and process high-dimensional neural signals recorded by the micro-electrode arrays. In this paper, we describe a novel architecture for analog-to-digital (A/D) conversion that combines Σ∆ conversion with spatial de-correlation within a single module. The architecture called multiple-input multiple-output (MIMO) Σ∆ is based on a min-max gradient descent optimization of a regularized linear cost function that naturally lends to an A/D formulation. Using an online formulation, the architecture can adapt to slow variations in cross-channel correlations, observed due to relative motion of the microelectrodes with respect to the signal sources. Experimental results with real recorded multi-channel neural data demonstrate the effectiveness of the proposed algorithm in alleviating cross-channel redundancy across electrodes and performing data-compression directly at the A/D converter. 1
4 0.51885486 130 nips-2006-Max-margin classification of incomplete data
Author: Gal Chechik, Geremy Heitz, Gal Elidan, Pieter Abbeel, Daphne Koller
Abstract: We consider the problem of learning classifiers for structurally incomplete data, where some objects have a subset of features inherently absent due to complex relationships between the features. The common approach for handling missing features is to begin with a preprocessing phase that completes the missing features, and then use a standard classification procedure. In this paper we show how incomplete data can be classified directly without any completion of the missing features using a max-margin learning framework. We formulate this task using a geometrically-inspired objective function, and discuss two optimization approaches: The linearly separable case is written as a set of convex feasibility problems, and the non-separable case has a non-convex objective that we optimize iteratively. By avoiding the pre-processing phase in which the data is completed, these approaches offer considerable computational savings. More importantly, we show that by elegantly handling complex patterns of missing values, our approach is both competitive with other methods when the values are missing at random and outperforms them when the missing values have non-trivial structure. We demonstrate our results on two real-world problems: edge prediction in metabolic pathways, and automobile detection in natural images. 1
5 0.47257665 72 nips-2006-Efficient Learning of Sparse Representations with an Energy-Based Model
Author: Marc'aurelio Ranzato, Christopher Poultney, Sumit Chopra, Yann L. Cun
Abstract: We describe a novel unsupervised method for learning sparse, overcomplete features. The model uses a linear encoder, and a linear decoder preceded by a sparsifying non-linearity that turns a code vector into a quasi-binary sparse code vector. Given an input, the optimal code minimizes the distance between the output of the decoder and the input patch while being as similar as possible to the encoder output. Learning proceeds in a two-phase EM-like fashion: (1) compute the minimum-energy code vector, (2) adjust the parameters of the encoder and decoder so as to decrease the energy. The model produces “stroke detectors” when trained on handwritten numerals, and Gabor-like filters when trained on natural image patches. Inference and learning are very fast, requiring no preprocessing, and no expensive sampling. Using the proposed unsupervised method to initialize the first layer of a convolutional network, we achieved an error rate slightly lower than the best reported result on the MNIST dataset. Finally, an extension of the method is described to learn topographical filter maps. 1
6 0.4672094 16 nips-2006-A Theory of Retinal Population Coding
7 0.46221143 149 nips-2006-Nonnegative Sparse PCA
8 0.45861164 46 nips-2006-Blind source separation for over-determined delayed mixtures
9 0.4181729 24 nips-2006-Aggregating Classification Accuracy across Time: Application to Single Trial EEG
10 0.40970424 186 nips-2006-Support Vector Machines on a Budget
11 0.40044737 182 nips-2006-Statistical Modeling of Images with Fields of Gaussian Scale Mixtures
12 0.38558903 140 nips-2006-Multiple Instance Learning for Computer Aided Diagnosis
13 0.38295642 27 nips-2006-An EM Algorithm for Localizing Multiple Sound Sources in Reverberant Environments
14 0.36139381 153 nips-2006-Online Clustering of Moving Hyperplanes
15 0.35472202 185 nips-2006-Subordinate class recognition using relational object models
16 0.35469094 106 nips-2006-Large Margin Hidden Markov Models for Automatic Speech Recognition
17 0.35325155 178 nips-2006-Sparse Multinomial Logistic Regression via Bayesian L1 Regularisation
18 0.35268086 174 nips-2006-Similarity by Composition
19 0.34769753 105 nips-2006-Large Margin Component Analysis
20 0.34668177 73 nips-2006-Efficient Methods for Privacy Preserving Face Detection
topicId topicWeight
[(1, 0.147), (3, 0.021), (7, 0.048), (9, 0.076), (20, 0.026), (22, 0.056), (24, 0.263), (44, 0.055), (57, 0.102), (64, 0.017), (65, 0.031), (69, 0.034), (71, 0.02)]
simIndex simValue paperId paperTitle
1 0.92422187 25 nips-2006-An Application of Reinforcement Learning to Aerobatic Helicopter Flight
Author: Pieter Abbeel, Adam Coates, Morgan Quigley, Andrew Y. Ng
Abstract: Autonomous helicopter flight is widely regarded to be a highly challenging control problem. This paper presents the first successful autonomous completion on a real RC helicopter of the following four aerobatic maneuvers: forward flip and sideways roll at low speed, tail-in funnel, and nose-in funnel. Our experimental results significantly extend the state of the art in autonomous helicopter flight. We used the following approach: First we had a pilot fly the helicopter to help us find a helicopter dynamics model and a reward (cost) function. Then we used a reinforcement learning (optimal control) algorithm to find a controller that is optimized for the resulting model and reward function. More specifically, we used differential dynamic programming (DDP), an extension of the linear quadratic regulator (LQR). 1
2 0.87100267 107 nips-2006-Large Margin Multi-channel Analog-to-Digital Conversion with Applications to Neural Prosthesis
Author: Amit Gore, Shantanu Chakrabartty
Abstract: A key challenge in designing analog-to-digital converters for cortically implanted prosthesis is to sense and process high-dimensional neural signals recorded by the micro-electrode arrays. In this paper, we describe a novel architecture for analog-to-digital (A/D) conversion that combines Σ∆ conversion with spatial de-correlation within a single module. The architecture called multiple-input multiple-output (MIMO) Σ∆ is based on a min-max gradient descent optimization of a regularized linear cost function that naturally lends to an A/D formulation. Using an online formulation, the architecture can adapt to slow variations in cross-channel correlations, observed due to relative motion of the microelectrodes with respect to the signal sources. Experimental results with real recorded multi-channel neural data demonstrate the effectiveness of the proposed algorithm in alleviating cross-channel redundancy across electrodes and performing data-compression directly at the A/D converter. 1
same-paper 3 0.78881156 179 nips-2006-Sparse Representation for Signal Classification
Author: Ke Huang, Selin Aviyente
Abstract: In this paper, application of sparse representation (factorization) of signals over an overcomplete basis (dictionary) for signal classification is discussed. Searching for the sparse representation of a signal over an overcomplete dictionary is achieved by optimizing an objective function that includes two terms: one that measures the signal reconstruction error and another that measures the sparsity. This objective function works well in applications where signals need to be reconstructed, like coding and denoising. On the other hand, discriminative methods, such as linear discriminative analysis (LDA), are better suited for classification tasks. However, discriminative methods are usually sensitive to corruption in signals due to lacking crucial properties for signal reconstruction. In this paper, we present a theoretical framework for signal classification with sparse representation. The approach combines the discrimination power of the discriminative methods with the reconstruction property and the sparsity of the sparse representation that enables one to deal with signal corruptions: noise, missing data and outliers. The proposed approach is therefore capable of robust classification with a sparse representation of signals. The theoretical results are demonstrated with signal classification tasks, showing that the proposed approach outperforms the standard discriminative methods and the standard sparse representation in the case of corrupted signals. 1
4 0.62204129 167 nips-2006-Recursive ICA
Author: Honghao Shan, Lingyun Zhang, Garrison W. Cottrell
Abstract: Independent Component Analysis (ICA) is a popular method for extracting independent features from visual data. However, as a fundamentally linear technique, there is always nonlinear residual redundancy that is not captured by ICA. Hence there have been many attempts to try to create a hierarchical version of ICA, but so far none of the approaches have a natural way to apply them more than once. Here we show that there is a relatively simple technique that transforms the absolute values of the outputs of a previous application of ICA into a normal distribution, to which ICA maybe applied again. This results in a recursive ICA algorithm that may be applied any number of times in order to extract higher order structure from previous layers. 1
5 0.61794484 72 nips-2006-Efficient Learning of Sparse Representations with an Energy-Based Model
Author: Marc'aurelio Ranzato, Christopher Poultney, Sumit Chopra, Yann L. Cun
Abstract: We describe a novel unsupervised method for learning sparse, overcomplete features. The model uses a linear encoder, and a linear decoder preceded by a sparsifying non-linearity that turns a code vector into a quasi-binary sparse code vector. Given an input, the optimal code minimizes the distance between the output of the decoder and the input patch while being as similar as possible to the encoder output. Learning proceeds in a two-phase EM-like fashion: (1) compute the minimum-energy code vector, (2) adjust the parameters of the encoder and decoder so as to decrease the energy. The model produces “stroke detectors” when trained on handwritten numerals, and Gabor-like filters when trained on natural image patches. Inference and learning are very fast, requiring no preprocessing, and no expensive sampling. Using the proposed unsupervised method to initialize the first layer of a convolutional network, we achieved an error rate slightly lower than the best reported result on the MNIST dataset. Finally, an extension of the method is described to learn topographical filter maps. 1
6 0.61757058 75 nips-2006-Efficient sparse coding algorithms
7 0.61688107 65 nips-2006-Denoising and Dimension Reduction in Feature Space
8 0.61686414 51 nips-2006-Clustering Under Prior Knowledge with Application to Image Segmentation
9 0.61164165 32 nips-2006-Analysis of Empirical Bayesian Methods for Neuroelectromagnetic Source Localization
10 0.61154515 8 nips-2006-A Nonparametric Approach to Bottom-Up Visual Saliency
11 0.61089575 158 nips-2006-PG-means: learning the number of clusters in data
12 0.61017954 138 nips-2006-Multi-Task Feature Learning
13 0.6077643 83 nips-2006-Generalized Maximum Margin Clustering and Unsupervised Kernel Learning
14 0.6067363 130 nips-2006-Max-margin classification of incomplete data
15 0.6064828 3 nips-2006-A Complexity-Distortion Approach to Joint Pattern Alignment
16 0.60452503 175 nips-2006-Simplifying Mixture Models through Function Approximation
17 0.60196978 76 nips-2006-Emergence of conjunctive visual features by quadratic independent component analysis
18 0.59965807 152 nips-2006-Online Classification for Complex Problems Using Simultaneous Projections
19 0.59911036 178 nips-2006-Sparse Multinomial Logistic Regression via Bayesian L1 Regularisation
20 0.59833974 154 nips-2006-Optimal Change-Detection and Spiking Neurons