nips nips2010 nips2010-246 knowledge-graph by maker-knowledge-mining

246 nips-2010-Sparse Coding for Learning Interpretable Spatio-Temporal Primitives

Source: pdf

Author: Taehwan Kim, Gregory Shakhnarovich, Raquel Urtasun

Abstract: Sparse coding has recently become a popular approach in computer vision to learn dictionaries of natural images. In this paper we extend the sparse coding framework to learn interpretable spatio-temporal primitives. We formulated the problem as a tensor factorization problem with tensor group norm constraints over the primitives, diagonal constraints on the activations that provide interpretability as well as smoothness constraints that are inherent to human motion. We demonstrate the effectiveness of our approach to learn interpretable representations of human motion from motion capture data, and show that our approach outperforms recently developed matching pursuit and sparse coding algorithms. 1

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 edu Abstract Sparse coding has recently become a popular approach in computer vision to learn dictionaries of natural images. [sent-4, score-0.299]

2 In this paper we extend the sparse coding framework to learn interpretable spatio-temporal primitives. [sent-5, score-0.389]

3 We formulated the problem as a tensor factorization problem with tensor group norm constraints over the primitives, diagonal constraints on the activations that provide interpretability as well as smoothness constraints that are inherent to human motion. [sent-6, score-0.737]

4 We demonstrate the effectiveness of our approach to learn interpretable representations of human motion from motion capture data, and show that our approach outperforms recently developed matching pursuit and sparse coding algorithms. [sent-7, score-0.995]

5 1 Introduction In recent years sparse coding has become a popular paradigm to learn dictionaries of natural images [10, 1, 4]. [sent-8, score-0.389]

6 In these approaches, sparse coding was formulated as the sum of a data ﬁtting term, typically the Frobenius norm, and a regularization term that imposes sparsity. [sent-10, score-0.297]

7 However, the sparsity induced by these norms is local; The estimated representations are sparse in that most of the activations are zero, but the sparsity has no structure, i. [sent-12, score-0.399]

8 [9] extend the sparse coding formulation of natural images to impose structure by ﬁrst clustering the set of image patches and then learning a dictionary where members of the same cluster are encouraged to share sparsity patterns. [sent-16, score-0.547]

9 Learning spatiotemporal representations of motion has been addressed in the neuroscience and motor control literature, in the context of motor synergies [13, 5, 14]. [sent-19, score-0.277]

10 [3] where the goal was to recover primitives from time series of EMG signals recorded from a set of frog muscles. [sent-22, score-0.502]

11 Using matching pursuit [11] and an 0 -type regularization as the underlying mechanism to learn primitives, [3] performed matrix factorization of the time series. [sent-23, score-0.377]

12 The recovered factors represent the primitive dictionary and the primitive activations. [sent-24, score-0.44]

13 In this paper we propose to extend the sparse coding framework to learn motion dictionaries. [sent-26, score-0.409]

14 In particular, we cast the problem of learning spatio-temporal primitives as a tensor factorization prob1 lem and introduce tensor group norms over the primitives that encourage sparsity in order to learn the number of elements in the dictionary. [sent-27, score-1.297]

15 The introduction of additional diagonal constraints in the activations, as well as smoothness constraints that are inherent to human motion, will allow us to learn interpretable representations of human motion from motion capture data. [sent-28, score-0.644]

16 As demonstrated in our experiments, our approach outperforms state-of-the-art matching pursuit [3], as well as recently developed sparse coding algorithms [7]. [sent-29, score-0.562]

17 2 Sparse coding for motion dictionary learning In this section we ﬁrst review the framework of sparse coding, and then show how to extend this framework to learn interpretable dictionaries of human motion. [sent-30, score-0.771]

18 1 Traditional sparse coding Let Y = [y1 , · · · , yN ] be the matrix formed by concatenating the set of training examples drawn i. [sent-32, score-0.261]

19 Sparse coding is usually formulated as a matrix factorization problem composed of a data ﬁtting term, typically the Frobenius norm, and a regularizer that encourages sparsity of the activations min ||Y − WH||2 + λψ(H) . [sent-36, score-0.481]

20 [9] exploit group norm sparsity priors to learn dictionaries of natural images by ﬁrst clustering the training image patches, and then learning a dictionary where members of the same cluster are encouraged to share sparsity patterns. [sent-47, score-0.475]

21 However, the structure imposed by these group norms is not sufﬁcient for learning interpretable motion primitives. [sent-50, score-0.293]

22 We now show how in the case of motion, we can consider the activations and the primitives as tensors and impose group norm sparsity on the tensors. [sent-51, score-0.834]

23 Moreover, we impose additional constraints such as continuity and differentiability that are inherent of human motion data, as well as diagonal constraints that ensure interpretability. [sent-52, score-0.389]

24 2 Motion dictionary learning Let Y ∈ D×L be a D dimensional signal of temporal length L. [sent-54, score-0.167]

25 For simplicity in the discussion we assume that the primitives have the same length. [sent-57, score-0.473]

26 This restriction can be easily removed by setting Q to be the maximum length of the primitives and padding the remaining elements to zero. [sent-58, score-0.522]

27 QP ×L are projections of the tensors to be represented When learning dictionaries of human motion, there is additional structure and constraints that one would like the dictionary elements to satisfy. [sent-66, score-0.369]

28 One important property of human motion is that it is smooth. [sent-67, score-0.158]

29 We impose continuity and differentiability constraints by adding a regularization term that P encourages smooth curvature, i. [sent-68, score-0.167]

30 One of the main difﬁculties with learning motion dictionaries is that the dictionary words might have very different temporal lengths. [sent-71, score-0.361]

31 Note that this problem does not arise in traditional dictionary learning of natural images, since the size of the dictionary words is manually speciﬁed [4, 1, 9]. [sent-72, score-0.27]

32 This makes the learning problem more complex since one would like to identify not only the number of elements in the dictionary, but also the size of each dictionary word. [sent-73, score-0.147]

33 We address this problem by adding a regularization term that prefers dictionaries with small number of primitives, as well as primitives of short length. [sent-74, score-0.592]

34 In particular, we extend the group norms over matrices to be group norms over tensors and deﬁne  r/q 1/r  Q P q/p D p  p,q,r (W) =  |Wi,j,k |  i=1 j=1    k=1 where Wi,j,k is the k-th dimension at the j-th time frame of the i-th primitive in W. [sent-75, score-0.384]

35 We will also like to impose additional constraints on the activations H. [sent-76, score-0.265]

36 , H and W can be recovered up to an invertible transformation WH = (WC−1 )(CH), we impose that the elements of the activation tensor should be in the unit interval, i. [sent-80, score-0.191]

37 As in traditional sparse coding, we encourage the activations to be sparse. [sent-83, score-0.279]

38 Finally, to impose interpretability of the results as spatio-temporal primitives, we impose that when a spatio-temporal primitive is active, it should be active across all its time-length with constant activation strength, i. [sent-85, score-0.299]

39 Matching pursuit is able to recover the number of primitives when using refractory period, however the activations and the primitives are not correct. [sent-129, score-1.475]

40 When we do not use the refractory period, the recovered primitives are very noisy. [sent-130, score-0.637]

41 Sparse coding has a low reconstruction error, but neither the number of primitives, nor the primitives and the activations are correctly recovered. [sent-131, score-1.003]

42 3 Experimental Evaluation We compare our algorithm to two state-of-the-art approaches in the task of discovering interpretable primitives from motion capture data, namely, the sparse coding approach of [7] and matching pursuit [3]. [sent-132, score-1.233]

43 We then show that our approach outperforms matching pursuit and sparse coding when learning dictionaries of walking and running motions. [sent-135, score-0.768]

44 The threshold for MP with refractory period is set to 0. [sent-142, score-0.207]

45 For each iteration in the optimization of H, an over-complete dictionary D is created by taking the primitives in W, and generating candidates by shifting each primitive in time. [sent-147, score-0.763]

46 Note that the cardinality of the candidate dictionary is |D| = P (L + Q − 1) if W has P primitives and the data is composed of L frames. [sent-148, score-0.631]

47 Once the dictionary is created, a set of primitives is iteratively selected (one at a time) by choosing at each iteration the primitive with the largest scalar product with respect to the residual signal that cannot be explained with the already selected primitives. [sent-149, score-0.743]

48 Additionally, in the step of choosing elements in the dictionary, [3] introduced the refractory period, which means that when one element in the dictionary is chosen, all overlapping elements are removed from the dictionary. [sent-152, score-0.31]

49 This is done to avoid multiple activations of primitives. [sent-153, score-0.167]

50 In our experiments we compare our approach to matching pursuit with and without refractory period. [sent-154, score-0.419]

51 Following [7], we solve this optimization problem alternating between solving with respect to the primitives ¯ ¯ W and the activations H. [sent-157, score-0.64]

52 1 Estimating the number of primitives In the ﬁrst experiment we demonstrate the ability of our approach to infer the number of primitives as well as the length of the existing primitives. [sent-159, score-0.972]

53 For this purpose we created a simple dataset which is composed of a single sequence of multiple walking cycles performed by the same subject from the CMU mocap dataset 1 . [sent-160, score-0.247]

54 In this case it is easy to see that since the motion is periodic, the signal could be represented by a single 2D primitive whose length is equal to the length of the period. [sent-171, score-0.318]

55 To perform the experiments we initialize our approach and the baselines with a sum of random smooth functions (sinusoids) whose frequencies are different from the principal frequency of the periodic training data, and set the number of primitives to P = 2. [sent-172, score-0.595]

56 One primitive is set to have approximately the same length as a cycle of the periodic motion and the other primitive is set to be 50% larger. [sent-173, score-0.465]

57 Note that a rough estimate of the length of the primitives could be easily obtained by analyzing the principal frequencies of the signal. [sent-174, score-0.568]

58 The ﬁrst two columns depict the two dimensional primitives recovered (W1 and W2). [sent-177, score-0.497]

59 Note that we expect these primitives to be similar to the original signal, i. [sent-180, score-0.473]

60 The third column depicts the activations vec(H) ∈ (Q1 +Q2 )×L recovered. [sent-185, score-0.229]

61 We expect the successful activations to be diagonal, and to appear only once every cycle. [sent-186, score-0.167]

62 Note that our approach is able to recover the number of primitives as well as the primitive themselves and the correct activations. [sent-187, score-0.648]

63 Matching pursuit without refractory period (ﬁrst row) is not able to recover the primitives, their number, or the activations. [sent-188, score-0.429]

64 Matching pursuit with refractory period (second row) is able to recover the number of primitives, however the activations are underestimated and the primitives are not very accurate. [sent-190, score-1.069]

65 Sparse coding has a low reconstruction error, but neither the primitives, their number, nor the activations are correctly recovered. [sent-191, score-0.53]

66 This conﬁrms the inability of traditional sparse coding to recover interpretable primitives, and the importance of having interpretability constraints such as the refractory period of matching pursuit and our diagonal constraints. [sent-192, score-1.004]

67 As expected one primitive is not enough for accurate reconstruction. [sent-199, score-0.146]

68 The primitives are either initialize randomly or to a smooth set of sinusoids of random frequencies. [sent-201, score-0.536]

69 the learned W and the estimated activations Htest 1 epca = ||Vtest − vec(W)vec(Htest )||F D as well as the error in the original 59D space which can be computed by projecting back into the original space using the singular vectors. [sent-202, score-0.236]

70 Note that W is learned at training, and the activations Htest are estimated at inference time. [sent-203, score-0.167]

71 The primitives are initialized to a sum of sinusoids of random frequencies. [sent-210, score-0.513]

72 We created a walking dataset composed of motions performed by the same subject. [sent-211, score-0.239]

73 We also performed reconstruction experiments for running motions and used motions {17, 18, 20, 21, 22, 23, 24, 25} from subject 35. [sent-213, score-0.375]

74 3 depicts reconstruction error in PCA space and in the original space as a function of the noise variance. [sent-216, score-0.323]

75 4 depicts reconstruction error as a function of the dimensionality of the PCA space. [sent-218, score-0.323]

76 Our approach outperforms matching pursuit with and without refractory period in all scenarios. [sent-219, score-0.508]

77 Note that out method outperforms sparse coding when the output is noisy. [sent-220, score-0.283]

78 This is due to the fact that, given a big enough dictionary, sparse coding overﬁts and can perfectly ﬁt the noise. [sent-221, score-0.261]

79 We also performed reconstruction experiments for running motions performed by different subjects. [sent-222, score-0.287]

80 In particular we use motions {03, 04, 05, 06} of subject 9 and motions {21, 23, 24, 25} of subject 35. [sent-223, score-0.208]

81 5 depicts reconstruction error for our approach when using different numbers of primitives. [sent-225, score-0.323]

82 As expected one primitive is not enough for accurate reconstruction. [sent-226, score-0.146]

83 When using two primitives our approach performs comparable to sparse coding and clearly outperforms the other baselines. [sent-227, score-0.756]

84 55 40 50 35 45 30 230 25 error error 40 " error with missing data 35 20 15 30 " test error 25 10 0 2 4 6 8 210 200 190 180 " training error 20 15 Reconstruction error 220 5 10 12 0 170 1 2 3 − log ! [sent-234, score-0.461]

85 P 4 5 7 6 5 4 3 2 1 log Figure 7: Inﬂuence of η and P on the single subject walking dataset as well as using soft constraints instead of hard constraints on the activations. [sent-235, score-0.245]

86 As expected the reconstruction error of the training data decreases when there is less regularization. [sent-237, score-0.278]

87 For missing data, having good primitives is important, and thus regularization is necessary. [sent-239, score-0.539]

88 Our approach is not sensitive to the value of P ; one primitive is enough for accurate reconstruction in this dataset. [sent-242, score-0.358]

89 This is due to the fact that sparse coding does not have structure, while the structure imposed by our equality constraints, i. [sent-249, score-0.261]

90 Note that as our approach, sparse coding is not sensitive to initialization. [sent-255, score-0.281]

91 Towards this end we use the single subject walking dataset, and compute reconstruction error for the training and test data with and without missing data as a function of η. [sent-258, score-0.455]

92 As expected the reconstruction error of the training data decreases when there is less regularization. [sent-261, score-0.278]

93 When dealing with missing data, having good primitives becomes more important. [sent-263, score-0.52]

94 We use the single subject walking dataset and report errors averaged over 10 partitions of the data. [sent-269, score-0.147]

95 7 (middle) our approach is very insensitive to P ; in this example a single primitive is enough for accurate reconstruction. [sent-271, score-0.169]

96 We ﬁnally investigate the inﬂuence of replacing the hard constraints on the activations by soft constraints |Hi,j,k − Hi,j+1,k+1 | ≤ α. [sent-272, score-0.265]

97 4 Conclusion We have proposed a sparse coding approach to learn interpretable spatio-temporal primitives of human motion. [sent-276, score-0.9]

98 We have formulated the problem as a tensor factorization problem with tensor group norm constraints over the primitives, diagonal constraints on the activations, as well as smoothness constraints that are inherent to human motion. [sent-277, score-0.534]

99 Our approach has proven superior to recently developed matching pursuit and sparse coding algorithms in the task of learning interpretable spatiotemporal primitives of human motion from motion capture data. [sent-278, score-1.412]

100 Image denoising via sparse and redundant representations over learned dictionaries. [sent-299, score-0.148]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('mp', 0.476), ('primitives', 0.473), ('rp', 0.306), ('sc', 0.243), ('pursuit', 0.193), ('reconstruction', 0.192), ('coding', 0.171), ('activations', 0.167), ('primitive', 0.146), ('refractory', 0.14), ('vec', 0.125), ('dictionary', 0.124), ('motion', 0.12), ('walking', 0.106), ('interpretable', 0.1), ('dictionaries', 0.1), ('sparse', 0.09), ('wh', 0.087), ('matching', 0.086), ('tensor', 0.076), ('ep', 0.073), ('error', 0.069), ('period', 0.067), ('motions', 0.063), ('depicts', 0.062), ('htest', 0.061), ('vtest', 0.061), ('dimension', 0.057), ('walk', 0.056), ('synergies', 0.049), ('constraints', 0.049), ('impose', 0.049), ('missing', 0.047), ('mairal', 0.043), ('subject', 0.041), ('sparsity', 0.04), ('sinusoids', 0.04), ('ca', 0.04), ('group', 0.038), ('human', 0.038), ('interpretability', 0.036), ('inherent', 0.036), ('factorization', 0.035), ('norms', 0.035), ('tti', 0.035), ('tensors', 0.035), ('members', 0.034), ('composed', 0.034), ('run', 0.033), ('norm', 0.032), ('denoising', 0.031), ('motor', 0.03), ('diavella', 0.03), ('mocap', 0.03), ('taehwan', 0.03), ('principal', 0.029), ('recover', 0.029), ('learn', 0.028), ('frobenius', 0.028), ('periodic', 0.027), ('variance', 0.027), ('representations', 0.027), ('alternate', 0.027), ('differentiability', 0.027), ('length', 0.026), ('uence', 0.026), ('chicago', 0.024), ('baselines', 0.024), ('recovered', 0.024), ('pca', 0.023), ('insensitive', 0.023), ('encouraged', 0.023), ('elements', 0.023), ('smooth', 0.023), ('outperforms', 0.022), ('gregory', 0.022), ('traditional', 0.022), ('diagonal', 0.021), ('urtasun', 0.021), ('spatiotemporal', 0.021), ('leftmost', 0.021), ('rough', 0.021), ('cmu', 0.021), ('non', 0.021), ('initialization', 0.02), ('sapiro', 0.02), ('created', 0.02), ('sensitive', 0.02), ('frequencies', 0.019), ('activation', 0.019), ('regularization', 0.019), ('smoothness', 0.018), ('formulated', 0.017), ('regularizer', 0.017), ('train', 0.017), ('decreases', 0.017), ('horizontal', 0.017), ('temporal', 0.017), ('image', 0.016), ('performed', 0.016)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 1.0000004 246 nips-2010-Sparse Coding for Learning Interpretable Spatio-Temporal Primitives

Author: Taehwan Kim, Gregory Shakhnarovich, Raquel Urtasun

2 0.15388536 59 nips-2010-Deep Coding Network

Author: Yuanqing Lin, Zhang Tong, Shenghuo Zhu, Kai Yu

Abstract: This paper proposes a principled extension of the traditional single-layer ﬂat sparse coding scheme, where a two-layer coding scheme is derived based on theoretical analysis of nonlinear functional approximation that extends recent results for local coordinate coding. The two-layer approach can be easily generalized to deeper structures in a hierarchical multiple-layer manner. Empirically, it is shown that the deep coding approach yields improved performance in benchmark datasets.

3 0.13483585 89 nips-2010-Factorized Latent Spaces with Structured Sparsity

Author: Yangqing Jia, Mathieu Salzmann, Trevor Darrell

Abstract: Recent approaches to multi-view learning have shown that factorizing the information into parts that are shared across all views and parts that are private to each view could effectively account for the dependencies and independencies between the different input modalities. Unfortunately, these approaches involve minimizing non-convex objective functions. In this paper, we propose an approach to learning such factorized representations inspired by sparse coding techniques. In particular, we show that structured sparsity allows us to address the multiview learning problem by alternately solving two convex optimization problems. Furthermore, the resulting factorized latent spaces generalize over existing approaches in that they allow having latent dimensions shared between any subset of the views instead of between all the views only. We show that our approach outperforms state-of-the-art methods on the task of human pose estimation. 1

4 0.10988358 143 nips-2010-Learning Convolutional Feature Hierarchies for Visual Recognition

Author: Koray Kavukcuoglu, Pierre Sermanet, Y-lan Boureau, Karol Gregor, Michael Mathieu, Yann L. Cun

Abstract: We propose an unsupervised method for learning multi-stage hierarchies of sparse convolutional features. While sparse coding has become an increasingly popular method for learning visual features, it is most often trained at the patch level. Applying the resulting ﬁlters convolutionally results in highly redundant codes because overlapping patches are encoded in isolation. By training convolutionally over large image windows, our method reduces the redudancy between feature vectors at neighboring locations and improves the efﬁciency of the overall representation. In addition to a linear decoder that reconstructs the image from sparse features, our method trains an efﬁcient feed-forward encoder that predicts quasisparse features from the input. While patch-based training rarely produces anything but oriented edge detectors, we show that convolutional training produces highly diverse ﬁlters, including center-surround ﬁlters, corner detectors, cross detectors, and oriented grating detectors. We show that using these ﬁlters in multistage convolutional network architecture improves performance on a number of visual recognition and detection tasks. 1

5 0.10512754 219 nips-2010-Random Conic Pursuit for Semidefinite Programming

Author: Ariel Kleiner, Ali Rahimi, Michael I. Jordan

Abstract: We present a novel algorithm, Random Conic Pursuit, that solves semideﬁnite programs (SDPs) via repeated optimization over randomly selected two-dimensional subcones of the PSD cone. This scheme is simple, easily implemented, applicable to very general SDPs, scalable, and theoretically interesting. Its advantages are realized at the expense of an ability to readily compute highly exact solutions, though useful approximate solutions are easily obtained. This property renders Random Conic Pursuit of particular interest for machine learning applications, in which the relevant SDPs are generally based upon random data and so exact minima are often not a priority. Indeed, we present empirical results to this effect for various SDPs encountered in machine learning; these experiments demonstrate the potential practical usefulness of Random Conic Pursuit. We also provide a preliminary analysis that yields insight into the theoretical properties and convergence of the algorithm. 1

6 0.10488939 181 nips-2010-Network Flow Algorithms for Structured Sparsity

7 0.10196188 258 nips-2010-Structured sparsity-inducing norms through submodular functions

8 0.10066294 231 nips-2010-Robust PCA via Outlier Pursuit

9 0.10041041 98 nips-2010-Functional form of motion priors in human motion perception

10 0.10023946 20 nips-2010-A unified model of short-range and long-range motion perception

11 0.085982233 56 nips-2010-Deciphering subsampled data: adaptive compressive sampling as a principle of brain communication

12 0.081640631 109 nips-2010-Group Sparse Coding with a Laplacian Scale Mixture Prior

13 0.076042011 147 nips-2010-Learning Multiple Tasks with a Sparse Matrix-Normal Penalty

14 0.072064213 189 nips-2010-On a Connection between Importance Sampling and the Likelihood Ratio Policy Gradient

15 0.067945316 76 nips-2010-Energy Disaggregation via Discriminative Sparse Coding

16 0.066308118 266 nips-2010-The Maximal Causes of Natural Scenes are Edge Filters

17 0.065530419 41 nips-2010-Block Variable Selection in Multivariate Regression and High-dimensional Causal Inference

18 0.0654466 201 nips-2010-PAC-Bayesian Model Selection for Reinforcement Learning

19 0.060203858 148 nips-2010-Learning Networks of Stochastic Differential Equations

20 0.05880205 200 nips-2010-Over-complete representations on recurrent neural networks can support persistent percepts

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, 0.14), (1, 0.037), (2, -0.059), (3, 0.058), (4, 0.078), (5, -0.154), (6, 0.004), (7, 0.143), (8, -0.089), (9, -0.025), (10, 0.11), (11, -0.059), (12, 0.034), (13, -0.035), (14, -0.05), (15, -0.011), (16, -0.099), (17, 0.023), (18, -0.03), (19, 0.009), (20, 0.028), (21, -0.077), (22, -0.057), (23, -0.107), (24, -0.03), (25, -0.076), (26, -0.057), (27, -0.083), (28, -0.012), (29, 0.09), (30, 0.105), (31, 0.124), (32, -0.173), (33, 0.061), (34, -0.023), (35, 0.013), (36, -0.118), (37, -0.025), (38, -0.104), (39, -0.024), (40, -0.043), (41, 0.04), (42, 0.079), (43, 0.083), (44, -0.046), (45, 0.072), (46, 0.08), (47, 0.008), (48, 0.093), (49, 0.066)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.96798313 246 nips-2010-Sparse Coding for Learning Interpretable Spatio-Temporal Primitives

Author: Taehwan Kim, Gregory Shakhnarovich, Raquel Urtasun

2 0.67488754 76 nips-2010-Energy Disaggregation via Discriminative Sparse Coding

Author: J. Z. Kolter, Siddharth Batra, Andrew Y. Ng

Abstract: Energy disaggregation is the task of taking a whole-home energy signal and separating it into its component appliances. Studies have shown that having devicelevel energy information can cause users to conserve signiﬁcant amounts of energy, but current electricity meters only report whole-home data. Thus, developing algorithmic methods for disaggregation presents a key technical challenge in the effort to maximize energy conservation. In this paper, we examine a large scale energy disaggregation task, and apply a novel extension of sparse coding to this problem. In particular, we develop a method, based upon structured prediction, for discriminatively training sparse coding algorithms speciﬁcally to maximize disaggregation performance. We show that this signiﬁcantly improves the performance of sparse coding algorithms on the energy task and illustrate how these disaggregation results can provide useful information about energy usage. 1

3 0.63406801 109 nips-2010-Group Sparse Coding with a Laplacian Scale Mixture Prior

Author: Pierre Garrigues, Bruno A. Olshausen

Abstract: We propose a class of sparse coding models that utilizes a Laplacian Scale Mixture (LSM) prior to model dependencies among coefﬁcients. Each coefﬁcient is modeled as a Laplacian distribution with a variable scale parameter, with a Gamma distribution prior over the scale parameter. We show that, due to the conjugacy of the Gamma prior, it is possible to derive efﬁcient inference procedures for both the coefﬁcients and the scale parameter. When the scale parameters of a group of coefﬁcients are combined into a single variable, it is possible to describe the dependencies that occur due to common amplitude ﬂuctuations among coefﬁcients, which have been shown to constitute a large fraction of the redundancy in natural images [1]. We show that, as a consequence of this group sparse coding, the resulting inference of the coefﬁcients follows a divisive normalization rule, and that this may be efﬁciently implemented in a network architecture similar to that which has been proposed to occur in primary visual cortex. We also demonstrate improvements in image coding and compressive sensing recovery using the LSM model. 1

4 0.60945582 56 nips-2010-Deciphering subsampled data: adaptive compressive sampling as a principle of brain communication

Author: Guy Isely, Christopher Hillar, Fritz Sommer

Abstract: A new algorithm is proposed for a) unsupervised learning of sparse representations from subsampled measurements and b) estimating the parameters required for linearly reconstructing signals from the sparse codes. We verify that the new algorithm performs efﬁcient data compression on par with the recent method of compressive sampling. Further, we demonstrate that the algorithm performs robustly when stacked in several stages or when applied in undercomplete or overcomplete situations. The new algorithm can explain how neural populations in the brain that receive subsampled input through ﬁber bottlenecks are able to form coherent response properties. 1

5 0.60795635 59 nips-2010-Deep Coding Network

Author: Yuanqing Lin, Zhang Tong, Shenghuo Zhu, Kai Yu

6 0.54711372 89 nips-2010-Factorized Latent Spaces with Structured Sparsity

7 0.51452452 266 nips-2010-The Maximal Causes of Natural Scenes are Edge Filters

8 0.51019078 143 nips-2010-Learning Convolutional Feature Hierarchies for Visual Recognition

9 0.48207623 45 nips-2010-CUR from a Sparse Optimization Viewpoint

10 0.46357039 200 nips-2010-Over-complete representations on recurrent neural networks can support persistent percepts

11 0.43647859 231 nips-2010-Robust PCA via Outlier Pursuit

12 0.43522951 219 nips-2010-Random Conic Pursuit for Semidefinite Programming

13 0.38976109 37 nips-2010-Basis Construction from Power Series Expansions of Value Functions

14 0.37091586 181 nips-2010-Network Flow Algorithms for Structured Sparsity

15 0.35502836 136 nips-2010-Large-Scale Matrix Factorization with Missing Data under Additional Constraints

16 0.34996006 20 nips-2010-A unified model of short-range and long-range motion perception

17 0.3402648 98 nips-2010-Functional form of motion priors in human motion perception

18 0.31609347 41 nips-2010-Block Variable Selection in Multivariate Regression and High-dimensional Causal Inference

19 0.30820167 209 nips-2010-Pose-Sensitive Embedding by Nonlinear NCA Regression

20 0.30671659 248 nips-2010-Sparse Inverse Covariance Selection via Alternating Linearization Methods

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(2, 0.223), (13, 0.107), (17, 0.045), (27, 0.074), (30, 0.065), (35, 0.063), (45, 0.183), (50, 0.034), (52, 0.052), (60, 0.013), (77, 0.021), (78, 0.018), (90, 0.013)]

similar papers list:

simIndex simValue paperId paperTitle

1 0.90244192 125 nips-2010-Inference and communication in the game of Password

Author: Yang Xu, Charles Kemp

Abstract: Communication between a speaker and hearer will be most efﬁcient when both parties make accurate inferences about the other. We study inference and communication in a television game called Password, where speakers must convey secret words to hearers by providing one-word clues. Our working hypothesis is that human communication is relatively efﬁcient, and we use game show data to examine three predictions. First, we predict that speakers and hearers are both considerate, and that both take the other’s perspective into account. Second, we predict that speakers and hearers are calibrated, and that both make accurate assumptions about the strategy used by the other. Finally, we predict that speakers and hearers are collaborative, and that they tend to share the cognitive burden of communication equally. We ﬁnd evidence in support of all three predictions, and demonstrate in addition that efﬁcient communication tends to break down when speakers and hearers are placed under time pressure.

same-paper 2 0.82790339 246 nips-2010-Sparse Coding for Learning Interpretable Spatio-Temporal Primitives

Author: Taehwan Kim, Gregory Shakhnarovich, Raquel Urtasun

3 0.70640343 89 nips-2010-Factorized Latent Spaces with Structured Sparsity

Author: Yangqing Jia, Mathieu Salzmann, Trevor Darrell

4 0.70362574 7 nips-2010-A Family of Penalty Functions for Structured Sparsity

Author: Jean Morales, Charles A. Micchelli, Massimiliano Pontil

Abstract: We study the problem of learning a sparse linear regression vector under additional conditions on the structure of its sparsity pattern. We present a family of convex penalty functions, which encode this prior knowledge by means of a set of constraints on the absolute values of the regression coefﬁcients. This family subsumes the ℓ1 norm and is ﬂexible enough to include different models of sparsity patterns, which are of practical and theoretical importance. We establish some important properties of these functions and discuss some examples where they can be computed explicitly. Moreover, we present a convergent optimization algorithm for solving regularized least squares with these penalty functions. Numerical simulations highlight the beneﬁt of structured sparsity and the advantage offered by our approach over the Lasso and other related methods.

5 0.70325446 109 nips-2010-Group Sparse Coding with a Laplacian Scale Mixture Prior

Author: Pierre Garrigues, Bruno A. Olshausen

6 0.69570953 56 nips-2010-Deciphering subsampled data: adaptive compressive sampling as a principle of brain communication

7 0.69361746 117 nips-2010-Identifying graph-structured activation patterns in networks

8 0.69358456 45 nips-2010-CUR from a Sparse Optimization Viewpoint

9 0.69311118 221 nips-2010-Random Projections for $k$-means Clustering

10 0.68974239 261 nips-2010-Supervised Clustering

11 0.68822134 265 nips-2010-The LASSO risk: asymptotic results and real world examples

12 0.68733728 284 nips-2010-Variational bounds for mixed-data factor analysis

13 0.68732679 258 nips-2010-Structured sparsity-inducing norms through submodular functions

14 0.68688339 262 nips-2010-Switched Latent Force Models for Movement Segmentation

15 0.68625325 260 nips-2010-Sufficient Conditions for Generating Group Level Sparsity in a Robust Minimax Framework

16 0.68439883 195 nips-2010-Online Learning in The Manifold of Low-Rank Matrices

17 0.68354243 275 nips-2010-Transduction with Matrix Completion: Three Birds with One Stone

18 0.68340045 136 nips-2010-Large-Scale Matrix Factorization with Missing Data under Additional Constraints

19 0.6833452 12 nips-2010-A Primal-Dual Algorithm for Group Sparse Regularization with Overlapping Groups

20 0.68240476 210 nips-2010-Practical Large-Scale Optimization for Max-norm Regularization