nips nips2007 nips2007-107 knowledge-graph by maker-knowledge-mining

107 nips-2007-Iterative Non-linear Dimensionality Reduction with Manifold Sculpting

Source: pdf

Author: Michael Gashler, Dan Ventura, Tony Martinez

Abstract: Many algorithms have been recently developed for reducing dimensionality by projecting data onto an intrinsic non-linear manifold. Unfortunately, existing algorithms often lose signiﬁcant precision in this transformation. Manifold Sculpting is a new algorithm that iteratively reduces dimensionality by simulating surface tension in local neighborhoods. We present several experiments that show Manifold Sculpting yields more accurate results than existing algorithms with both generated and natural data-sets. Manifold Sculpting is also able to beneﬁt from both prior dimensionality reduction efforts. 1

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 Iterative Non-linear Dimensionality Reduction by Manifold Sculpting Mike Gashler, Dan Ventura, and Tony Martinez ∗ Brigham Young University Provo, UT 84604 Abstract Many algorithms have been recently developed for reducing dimensionality by projecting data onto an intrinsic non-linear manifold. [sent-1, score-0.196]

2 Manifold Sculpting is a new algorithm that iteratively reduces dimensionality by simulating surface tension in local neighborhoods. [sent-3, score-0.216]

3 We present several experiments that show Manifold Sculpting yields more accurate results than existing algorithms with both generated and natural data-sets. [sent-4, score-0.067]

4 Manifold Sculpting is also able to beneﬁt from both prior dimensionality reduction efforts. [sent-5, score-0.125]

5 1 Introduction Dimensionality reduction is a two-step process: 1) Transform the data so that more information will survive the projection, and 2) project the data into fewer dimensions. [sent-6, score-0.075]

6 The more relationships between data points that the transformation step is required to preserve, the less ﬂexibility it will have to position the points in a manner that will cause information to survive the projection step. [sent-7, score-0.302]

7 Due to this inverse relationship, dimensionality reduction algorithms must seek a balance that preserves information in the transformation without losing it in the projection. [sent-8, score-0.243]

8 Nonlinear dimensionality reduction (NLDR) algorithms seek this balance by assuming that the relationships between neighboring points contain more informational content than the relationships between distant points. [sent-10, score-0.382]

9 Although non-linear transformations have more potential than do linear transformations to lose information in the structure of the data, they also have more potential to position the data to cause more information to survive the projection. [sent-11, score-0.088]

10 In this process, NLDR algorithms expose patterns and structures of lower dimensionality (manifolds) that exist in the original data. [sent-12, score-0.119]

11 NLDR algorithms, or manifold learning algorithms, have potential to make the high-level concepts embedded in multidimensional data accessible to both humans and machines. [sent-13, score-0.61]

12 This paper introduces a new algorithm for manifold learning called Manifold Sculpting, which discovers manifolds through a process of progressive reﬁnement. [sent-14, score-0.65]

13 Experiments show that it yields more accurate results than other algorithms in many cases. [sent-15, score-0.067]

14 Additionally, it can be used as a postprocessing step to enhance the transformation of other manifold learning algorithms. [sent-16, score-0.677]

15 2 Related Work Many algorithms have been developed for performing non-linear dimensionality reduction. [sent-17, score-0.119]

16 Recent works include Isomap [1], which solves for an isometric embedding of data into fewer dimensions with an algebraic technique. [sent-18, score-0.074]

17 edu 1 Figure 1: Comparison of several manifold learners on a Swiss Roll manifold. [sent-24, score-0.61]

18 Color is used to indicate how points in the results correspond to points on the manifold. [sent-25, score-0.07]

19 ) Locally Linear Embedding (LLE) [2] is able to perform a similar computation using a sparse matrix by using a metric that measures only relationships between vectors in local neighborhoods. [sent-31, score-0.104]

20 Hessian LLE preserves the manifold structure better than the other algorithms but is, unfortunately, computationally expensive. [sent-39, score-0.671]

21 This algorithm iteratively transforms data by balancing two opposing heuristics, one that scales information out of unwanted dimensions, and one that preserves local structure in the data. [sent-44, score-0.085]

22 Experimental results show that this technique preserves information into fewer dimensions with more accuracy than existing manifold learning algorithms. [sent-45, score-0.684]

23 Figure 2: δ and θ deﬁne the relationships that Manifold Sculpting attempts to preserve. [sent-49, score-0.074]

24 2 Step 1: Find the k nearest neighbors of each point. [sent-50, score-0.075]

25 For each data point pi in P (where P is the set of all data points represented as vectors in Rn ), ﬁnd the k-nearest neighbors Ni (such that nij ∈ Ni is the j th neighbor of point pi ). [sent-51, score-0.407]

26 For each j (where 0 < j ≤ k) compute the Euclidean distance δij between pi and each nij ∈ Ni . [sent-53, score-0.21]

27 Also compute the angle θij formed by the two line segments (pi to nij ) and (nij to mij ), where mij is the most colinear neighbor of nij with pi . [sent-54, score-0.492]

28 ) The most colinear neighbor is the neighbor point that forms the angle closest to π. [sent-56, score-0.187]

29 The values of δ and θ are the relationships that the algorithm will attempt to preserve during transformation. [sent-57, score-0.074]

30 The global average distance between all the neighbors of all points δave is also computed. [sent-58, score-0.134]

31 The data may optionally be preprocessed with the transformation step of Principle Component Analysis (PCA), or another efﬁcient algorithm. [sent-60, score-0.094]

32 To the extent that there is a linear component in the manifold, PCA will move the information in the data into as few dimensions as possible, thus leaving less work to be done in step 4 (which handles the non-linear component). [sent-62, score-0.072]

33 This step is performed by computing the ﬁrst |Dpres | principle components of the data (where Dpres is the set of dimensions that will be preserved in the projection), and rotating the dimensional axes to align with these principle components. [sent-63, score-0.247]

34 For each pi ∈ P , the values in Dpres are adjusted to recover the relationships that are distorted by scaling. [sent-75, score-0.198]

35 Intuitively, this step simulates tension on the manifold surface. [sent-76, score-0.678]

36 The denominator values were chosen as normalizing factors because the value of the angle term can range from 0 to π, and the value of the distance term will tend to have a mean of about δave with some variance in both directions. [sent-78, score-0.097]

37 The order in which points are adjusted has some impact on the rate of convergence. [sent-80, score-0.084]

38 To further speed convergence, higher weight, wij , is given to the component of the error contributed by neighbors that have already been adjusted in the current iteration. [sent-84, score-0.185]

39 For all of our experiments, we use wij = 1 if ni has not yet been adjusted in this iteration, and wij = 10, if nij has been adjusted in this iteration. [sent-85, score-0.346]

40 Unfortunately the equation for the true gradient of the error surface deﬁned by this heuristic is complex, and is in O(|D|3 ). [sent-86, score-0.063]

41 Since the error surface is not necessarily convex, the algorithm may potentially converge to local minima. [sent-88, score-0.093]

42 Even if a local 3 Figure 3: The mean squared error of four algorithms with a Swiss Roll manifold using a varying number of neighbors k. [sent-90, score-0.839]

43 When k > 57, neighbor paths cut across the manifold. [sent-91, score-0.111]

44 minimum exists so close to the globally optimal state, it may have a sufﬁciently small error as to be acceptable. [sent-94, score-0.071]

45 Even if one point becomes temporarily stuck in a local minimum, its neighbors are likely to pull it out, or change the topology of its error surface when their values are adjusted. [sent-96, score-0.197]

46 Third, by gradually scaling the values in Dscaled (instead of directly setting them to 0), the system always remains in a state very close to the current globally optimal state. [sent-98, score-0.068]

47 As long as it stays close to the current optimal state, it is unlikely for the error surface to change in a manner that permanently separates it from being able to reach the globally optimal state. [sent-99, score-0.11]

48 (This is why all the dimensions need to be preserved in the PCA pre-processing step. [sent-100, score-0.083]

49 4 Empirical Results Figure 1 shows that Manifold Sculpting appears visually to produce results of higher quality than LLE and Isomap with the Swiss Roll manifold, a common visual test for manifold learning algorithms. [sent-105, score-0.655]

50 Since the actual structure of this manifold is known prior to using any manifold learner, we can use this prior information to quantitatively measure the accuracy of each algorithm. [sent-107, score-1.22]

51 We deﬁne a Swiss Roll in 3D space with n points (xi , yi , zi ) for each 0 ≤ i < n, such that xi = t sin(t), yi is a random number −6 ≤ yi < 6, and zi = t cos(t), √ where t = 8i/n + 2. [sent-110, score-0.18]

52 manifold coordinates, the point is (ui , vi ), such that ui = 2 We created a Swiss Roll with 2000 data points and reduced the dimensionality to 2 with each of four algorithms. [sent-112, score-0.846]

53 Next we tested how well these results align with the expected values by measuring the mean squared distance from each point to its expected value. [sent-113, score-0.068]

54 Results are shown with a varying number of neighbors k. [sent-117, score-0.103]

55 4 Figure 4: The mean squared error of points from an S-Curve manifold for four algorithms with a varying number of data points. [sent-120, score-0.769]

56 We deﬁned the S-Curve points in 3D space with n points (xi , yi , zi ) for each 0 ≤ i < n, such that xi = t, yi = sin(t), and zi is a random number 0 ≤ zi < 2, where t = (2. [sent-127, score-0.22]

57 In 2D manifold coordinates, the point is n t cos2 (w) + 1 dw and vi = yi . [sent-130, score-0.667]

58 (ui , vi ), such that ui = 0 Figure 4 shows the mean squared error of the transformed points from their expected values using the same regression technique described for the experiment with the Swiss Roll problem. [sent-131, score-0.142]

59 A trend can be observed in this data that as the number of sample points increases, the quality of results from Manifold Sculpting also increases. [sent-133, score-0.08]

60 This experiment was also performed with 6 neighbors, but Manifold Sculpting did not always converge within a reasonable time when so few neighbors were used. [sent-136, score-0.075]

61 The other three algorithms do not have this limitation, but the quality of their results still tend to be poor when very few neighbors are used. [sent-137, score-0.157]

62 In cases where the manifold has an intrinsic dimensionality of exactly 1, a path from neighbor to neighbor provides an accurate estimate of isolinear distance. [sent-143, score-0.975]

63 Thus an algorithm that seeks to globally optimize isolinear distances will be less susceptible to the noise from cutting across local corners. [sent-144, score-0.15]

64 When the intrinsic dimensionality is higher than 1, however, paths that follow from neighbor to neighbor produce a zig-zag pattern that introduces excessive noise into the isolinear distance measurement. [sent-145, score-0.416]

65 In these cases, preserving local neighborhood relationships with precision yields better overall results than globally optimizing an error-prone metric. [sent-146, score-0.172]

66 Consistent with this intuition, Isomap is the closest competitor to Manifold Sculpting in other experiments that involved a manifold with a single intrinsic dimension, and yields the poorest results of the four algorithms when the intrinsic dimensionality is larger than one. [sent-147, score-0.927]

67 5 Figure 5: Mean squared error for four algorithms with an Entwined Spirals manifold. [sent-148, score-0.096]

68 Unfortunately, the manifold structure represented by most real-world problems is not known a priori. [sent-152, score-0.61]

69 The accuracy of a manifold learner, however, can still be estimated when the problem involves a video sequence by simply counting the percentage of frames that are sorted into the same order as the video sequence. [sent-153, score-0.9]

70 Figure 6 shows several frames from a video sequence of a person turning his head while gradually smiling. [sent-154, score-0.185]

71 ) The one preserved dimension could then characterize each frame according to the high-level concepts that were previously encoded in many dimensions. [sent-158, score-0.079]

72 The dot below each image corresponds to the single-dimensional value in the preserved dimension for that image. [sent-159, score-0.065]

73 In this case, the ordering of every frame was consistent with the video sequence. [sent-160, score-0.16]

74 Figure 7 shows a comparison of results obtained from a manifold generated by translating an image over a background of random noise. [sent-163, score-0.681]

75 Because two variables (horizontal position and vertical position) were used to generate the dataset, this data creates a manifold with an intrinsic dimensionality of two in a space with an extrinsic dimensionality of 2,401 (the total number of pixels in each image). [sent-167, score-0.951]

76 Because the background is random, the average distance between neighboring points in the input space is uniform, so the ideal result is known to be a square. [sent-168, score-0.086]

77 The distortions produced by Manifold Sculpting tend to be local in nature, while the distortions produced by other algorithms tend to be more global. [sent-169, score-0.175]

78 Note that the points are spread nearly uniformly across the manifold in the results from Manifold Sculpting. [sent-170, score-0.645]

79 6 Figure 7: A comparison of results with a manifold generated by translating an image over a background of noise. [sent-174, score-0.681]

80 Manifold Sculpting tends to produce less global distortion, while other algorithms tend to produce less local distortion. [sent-175, score-0.161]

81 Perhaps more signiﬁcantly, it also tends to keep the intrinsic variables in the dataset more linearly separable. [sent-181, score-0.102]

82 This is particularly important when the dimensionality reduction is used as a pre-processing step for a supervised learning algorithm. [sent-182, score-0.159]

83 We created four video sequences designed to show various types of manifold topologies and measured the accuracy of each manifold learning algorithm. [sent-183, score-1.394]

84 Since the background pixels remain nearly constant while the pixels on the rotating object change in value, the manifold corresponding to the vector encoding of this video will contain both smooth and changing areas. [sent-186, score-0.931]

85 The second video was made by moving a camera down a hallway. [sent-187, score-0.126]

86 This produces a manifold with a continuous range of variability, since pixels near the center of the frame change slowly while pixels near the edges change rapidly. [sent-188, score-0.752]

87 Unlike the video of the rotating stuffed animal, there are no background pixels that remain constant. [sent-190, score-0.318]

88 Unlike the ﬁrst video, however, the high-contrast texture of the object used in this video results in a topology with much more variation. [sent-192, score-0.155]

89 As the black spots shift across the pixels, a manifold is created that swings wildly in the respective dimensions. [sent-193, score-0.635]

90 Due to the large hills and valleys in the topology of this manifold, the nearest neighbors of a frame frequently create paths that cut across the manifold. [sent-194, score-0.188]

91 In all four cases, Manifold Sculpting produced results competitive with Isomap, which does particularly well with manifolds that have an intrinsic dimensionality of Figure 8: Four video sequences were created with varying properties in the corresponding manfolds. [sent-195, score-0.413]

92 Dimensionality was reduced to one with each of four manifold learning algorithms. [sent-196, score-0.633]

93 7 one, but Manifold Sculpting is not limited by the intrinsic dimensionality as shown in the previous experiments. [sent-198, score-0.171]

94 5 Discussion The experiments tested in this paper show that Manifold Sculpting yields more accurate results than other well-known manifold learning algorithms. [sent-199, score-0.652]

95 Manifold Sculpting is more accurate than other algorithms when the manifold is sparsely sampled, and the gap is even wider with higher sampling densities. [sent-201, score-0.656]

96 Manifold Sculpting has difﬁculty when the selected number of neighbors is too small but consistently outperforms other algorithms when it is larger. [sent-202, score-0.1]

97 Manifold Sculpting beneﬁts signiﬁcantly when the data is pre-processed with the transformation step of PCA. [sent-206, score-0.067]

98 The transformation step of any algorithm may be used in place of this step. [sent-207, score-0.067]

99 Current research seeks to identify which algorithms work best with Manifold Sculpting to efﬁciently produce high quality results. [sent-208, score-0.092]

100 Laplacian eigenmaps and spectral techniques for embedding and clustering. [sent-231, score-0.07]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('sculpting', 0.612), ('manifold', 0.61), ('isomap', 0.151), ('nij', 0.136), ('video', 0.126), ('dimensionality', 0.094), ('lle', 0.088), ('roll', 0.081), ('intrinsic', 0.077), ('neighbors', 0.075), ('relationships', 0.074), ('dpres', 0.068), ('dscal', 0.068), ('hlle', 0.068), ('nldr', 0.068), ('swiss', 0.065), ('neighbor', 0.061), ('rotating', 0.06), ('pixels', 0.054), ('entwined', 0.051), ('isolinear', 0.051), ('spirals', 0.051), ('stuffed', 0.051), ('pi', 0.05), ('adjusted', 0.049), ('globally', 0.047), ('preserved', 0.045), ('sam', 0.044), ('survive', 0.044), ('ave', 0.041), ('manifolds', 0.04), ('surface', 0.039), ('dimensions', 0.038), ('frames', 0.038), ('ni', 0.038), ('angle', 0.038), ('wij', 0.037), ('preserves', 0.036), ('embedding', 0.036), ('tend', 0.035), ('points', 0.035), ('step', 0.034), ('charting', 0.034), ('tension', 0.034), ('vin', 0.034), ('eigenmaps', 0.034), ('frame', 0.034), ('transformation', 0.033), ('ij', 0.032), ('zi', 0.032), ('pca', 0.031), ('reduction', 0.031), ('vi', 0.03), ('local', 0.03), ('ventura', 0.03), ('martinez', 0.03), ('ui', 0.029), ('topology', 0.029), ('varying', 0.028), ('background', 0.027), ('yi', 0.027), ('optionally', 0.027), ('trouble', 0.027), ('colinear', 0.027), ('unfortunately', 0.027), ('created', 0.025), ('cut', 0.025), ('distortions', 0.025), ('informational', 0.025), ('distorted', 0.025), ('projection', 0.025), ('principle', 0.025), ('algorithms', 0.025), ('paths', 0.025), ('tends', 0.025), ('distance', 0.024), ('balance', 0.024), ('parzen', 0.024), ('yoshua', 0.024), ('translating', 0.024), ('squared', 0.024), ('hessian', 0.024), ('error', 0.024), ('produce', 0.023), ('trend', 0.023), ('locally', 0.023), ('four', 0.023), ('position', 0.022), ('quality', 0.022), ('joshua', 0.022), ('seeks', 0.022), ('silva', 0.022), ('lose', 0.022), ('mij', 0.022), ('gradually', 0.021), ('yields', 0.021), ('accurate', 0.021), ('align', 0.02), ('image', 0.02), ('iteratively', 0.019)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.99999952 107 nips-2007-Iterative Non-linear Dimensionality Reduction with Manifold Sculpting

Author: Michael Gashler, Dan Ventura, Tony Martinez

2 0.41902623 161 nips-2007-Random Projections for Manifold Learning

Author: Chinmay Hegde, Michael Wakin, Richard Baraniuk

Abstract: We propose a novel method for linear dimensionality reduction of manifold modeled data. First, we show that with a small number M of random projections of sample points in RN belonging to an unknown K-dimensional Euclidean manifold, the intrinsic dimension (ID) of the sample set can be estimated to high accuracy. Second, we rigorously prove that using only this set of random projections, we can estimate the structure of the underlying manifold. In both cases, the number of random projections required is linear in K and logarithmic in N , meaning that K < M ≪ N . To handle practical situations, we develop a greedy algorithm to estimate the smallest size of the projection space required to perform manifold learning. Our method is particularly relevant in distributed sensing systems and leads to signiﬁcant potential savings in data acquisition, storage and transmission costs.

3 0.2269952 115 nips-2007-Learning the 2-D Topology of Images

Author: Nicolas L. Roux, Yoshua Bengio, Pascal Lamblin, Marc Joliveau, Balázs Kégl

Abstract: We study the following question: is the two-dimensional structure of images a very strong prior or is it something that can be learned with a few examples of natural images? If someone gave us a learning task involving images for which the two-dimensional topology of pixels was not known, could we discover it automatically and exploit it? For example suppose that the pixels had been permuted in a ﬁxed but unknown way, could we recover the relative two-dimensional location of pixels on images? The surprising result presented here is that not only the answer is yes, but that about as few as a thousand images are enough to approximately recover the relative locations of about a thousand pixels. This is achieved using a manifold learning algorithm applied to pixels associated with a measure of distributional similarity between pixel intensities. We compare different topologyextraction approaches and show how having the two-dimensional topology can be exploited.

4 0.17201467 186 nips-2007-Statistical Analysis of Semi-Supervised Regression

Author: Larry Wasserman, John D. Lafferty

Abstract: Semi-supervised methods use unlabeled data in addition to labeled data to construct predictors. While existing semi-supervised methods have shown some promising empirical performance, their development has been based largely based on heuristics. In this paper we study semi-supervised learning from the viewpoint of minimax theory. Our ﬁrst result shows that some common methods based on regularization using graph Laplacians do not lead to faster minimax rates of convergence. Thus, the estimators that use the unlabeled data do not have smaller risk than the estimators that use only labeled data. We then develop several new approaches that provably lead to improved performance. The statistical tools of minimax analysis are thus used to offer some new perspective on the problem of semi-supervised learning. 1

5 0.11555367 116 nips-2007-Learning the structure of manifolds using random projections

Author: Yoav Freund, Sanjoy Dasgupta, Mayank Kabra, Nakul Verma

Abstract: We present a simple variant of the k-d tree which automatically adapts to intrinsic low dimensional structure in data. 1

6 0.075280517 153 nips-2007-People Tracking with the Laplacian Eigenmaps Latent Variable Model

7 0.06896849 175 nips-2007-Semi-Supervised Multitask Learning

8 0.065569252 18 nips-2007-A probabilistic model for generating realistic lip movements from speech

9 0.060346633 166 nips-2007-Regularized Boost for Semi-Supervised Learning

10 0.044469241 109 nips-2007-Kernels on Attributed Pointsets with Applications

11 0.043296453 162 nips-2007-Random Sampling of States in Dynamic Programming

12 0.041513294 49 nips-2007-Colored Maximum Variance Unfolding

13 0.04056878 8 nips-2007-A New View of Automatic Relevance Determination

14 0.040135235 122 nips-2007-Locality and low-dimensions in the prediction of natural experience from fMRI

15 0.038693864 188 nips-2007-Subspace-Based Face Recognition in Analog VLSI

16 0.036296081 163 nips-2007-Receding Horizon Differential Dynamic Programming

17 0.036057547 61 nips-2007-Convex Clustering with Exemplar-Based Models

18 0.033568643 10 nips-2007-A Randomized Algorithm for Large Scale Support Vector Learning

19 0.032514878 182 nips-2007-Sparse deep belief net model for visual area V2

20 0.029284751 130 nips-2007-Modeling Natural Sounds with Modulation Cascade Processes

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, -0.139), (1, 0.051), (2, -0.065), (3, 0.043), (4, -0.039), (5, 0.112), (6, -0.08), (7, 0.139), (8, 0.012), (9, 0.311), (10, -0.134), (11, 0.42), (12, -0.078), (13, 0.093), (14, 0.171), (15, -0.116), (16, -0.132), (17, 0.048), (18, -0.041), (19, 0.036), (20, -0.038), (21, -0.056), (22, -0.151), (23, -0.015), (24, -0.008), (25, 0.045), (26, 0.077), (27, 0.061), (28, 0.119), (29, 0.049), (30, 0.041), (31, -0.07), (32, -0.037), (33, -0.058), (34, 0.065), (35, -0.033), (36, 0.088), (37, -0.007), (38, 0.062), (39, -0.02), (40, 0.004), (41, 0.044), (42, -0.059), (43, 0.039), (44, -0.045), (45, 0.031), (46, 0.009), (47, -0.016), (48, 0.049), (49, 0.047)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.9743076 107 nips-2007-Iterative Non-linear Dimensionality Reduction with Manifold Sculpting

Author: Michael Gashler, Dan Ventura, Tony Martinez

2 0.89073551 161 nips-2007-Random Projections for Manifold Learning

Author: Chinmay Hegde, Michael Wakin, Richard Baraniuk

3 0.72638845 115 nips-2007-Learning the 2-D Topology of Images

Author: Nicolas L. Roux, Yoshua Bengio, Pascal Lamblin, Marc Joliveau, Balázs Kégl

4 0.42479423 186 nips-2007-Statistical Analysis of Semi-Supervised Regression

Author: Larry Wasserman, John D. Lafferty

5 0.38876167 116 nips-2007-Learning the structure of manifolds using random projections

Author: Yoav Freund, Sanjoy Dasgupta, Mayank Kabra, Nakul Verma

Abstract: We present a simple variant of the k-d tree which automatically adapts to intrinsic low dimensional structure in data. 1

6 0.35522676 49 nips-2007-Colored Maximum Variance Unfolding

7 0.29040596 153 nips-2007-People Tracking with the Laplacian Eigenmaps Latent Variable Model

8 0.26463714 188 nips-2007-Subspace-Based Face Recognition in Analog VLSI

9 0.2480571 13 nips-2007-A Unified Near-Optimal Estimator For Dimension Reduction in $l \alpha$ ($0<\alpha\leq 2$) Using Stable Random Projections

10 0.21567115 16 nips-2007-A learning framework for nearest neighbor search

11 0.20586756 175 nips-2007-Semi-Supervised Multitask Learning

12 0.19641581 166 nips-2007-Regularized Boost for Semi-Supervised Learning

13 0.1855565 10 nips-2007-A Randomized Algorithm for Large Scale Support Vector Learning

14 0.17556219 163 nips-2007-Receding Horizon Differential Dynamic Programming

15 0.16318023 139 nips-2007-Nearest-Neighbor-Based Active Learning for Rare Category Detection

16 0.16043438 4 nips-2007-A Constraint Generation Approach to Learning Stable Linear Dynamical Systems

17 0.1525014 45 nips-2007-Classification via Minimum Incremental Coding Length (MICL)

18 0.14684287 18 nips-2007-A probabilistic model for generating realistic lip movements from speech

19 0.14587329 109 nips-2007-Kernels on Attributed Pointsets with Applications

20 0.14520712 8 nips-2007-A New View of Automatic Relevance Determination

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(5, 0.041), (13, 0.037), (16, 0.02), (18, 0.012), (19, 0.012), (21, 0.049), (31, 0.04), (34, 0.019), (35, 0.018), (36, 0.25), (39, 0.011), (47, 0.093), (81, 0.025), (83, 0.152), (85, 0.025), (87, 0.035), (90, 0.048)]

similar papers list:

simIndex simValue paperId paperTitle

1 0.86196524 142 nips-2007-Non-parametric Modeling of Partially Ranked Data

Author: Guy Lebanon, Yi Mao

Abstract: Statistical models on full and partial rankings of n items are often of limited practical use for large n due to computational consideration. We explore the use of non-parametric models for partially ranked data and derive efﬁcient procedures for their use for large n. The derivations are largely possible through combinatorial and algebraic manipulations based on the lattice of partial rankings. In particular, we demonstrate for the ﬁrst time a non-parametric coherent and consistent model capable of efﬁciently aggregating partially ranked data of different types. 1

2 0.81038958 164 nips-2007-Receptive Fields without Spike-Triggering

Author: Guenther Zeck, Matthias Bethge, Jakob H. Macke

Abstract: S timulus selectivity of sensory neurons is often characterized by estimating their receptive ﬁeld properties such as orientation selectivity. Receptive ﬁelds are usually derived from the mean (or covariance) of the spike-triggered stimulus ensemble. This approach treats each spike as an independent message but does not take into account that information might be conveyed through patterns of neural activity that are distributed across space or time. Can we ﬁnd a concise description for the processing of a whole population of neurons analogous to the receptive ﬁeld for single neurons? Here, we present a generalization of the linear receptive ﬁeld which is not bound to be triggered on individual spikes but can be meaningfully linked to distributed response patterns. More precisely, we seek to identify those stimulus features and the corresponding patterns of neural activity that are most reliably coupled. We use an extension of reverse-correlation methods based on canonical correlation analysis. The resulting population receptive ﬁelds span the subspace of stimuli that is most informative about the population response. We evaluate our approach using both neuronal models and multi-electrode recordings from rabbit retinal ganglion cells. We show how the model can be extended to capture nonlinear stimulus-response relationships using kernel canonical correlation analysis, which makes it possible to test different coding mechanisms. Our technique can also be used to calculate receptive ﬁelds from multi-dimensional neural measurements such as those obtained from dynamic imaging methods. 1

same-paper 3 0.79675519 107 nips-2007-Iterative Non-linear Dimensionality Reduction with Manifold Sculpting

Author: Michael Gashler, Dan Ventura, Tony Martinez

4 0.72822863 177 nips-2007-Simplified Rules and Theoretical Analysis for Information Bottleneck Optimization and PCA with Spiking Neurons

Author: Lars Buesing, Wolfgang Maass

Abstract: We show that under suitable assumptions (primarily linearization) a simple and perspicuous online learning rule for Information Bottleneck optimization with spiking neurons can be derived. This rule performs on common benchmark tasks as well as a rather complex rule that has previously been proposed [1]. Furthermore, the transparency of this new learning rule makes a theoretical analysis of its convergence properties feasible. A variation of this learning rule (with sign changes) provides a theoretically founded method for performing Principal Component Analysis (PCA) with spiking neurons. By applying this rule to an ensemble of neurons, different principal components of the input can be extracted. In addition, it is possible to preferentially extract those principal components from incoming signals X that are related or are not related to some additional target signal YT . In a biological interpretation, this target signal YT (also called relevance variable) could represent proprioceptive feedback, input from other sensory modalities, or top-down signals. 1

5 0.62971675 115 nips-2007-Learning the 2-D Topology of Images

Author: Nicolas L. Roux, Yoshua Bengio, Pascal Lamblin, Marc Joliveau, Balázs Kégl

6 0.62530297 63 nips-2007-Convex Relaxations of Latent Variable Training

7 0.61952692 180 nips-2007-Sparse Feature Learning for Deep Belief Networks

8 0.61734563 94 nips-2007-Gaussian Process Models for Link Analysis and Transfer Learning

9 0.61660886 18 nips-2007-A probabilistic model for generating realistic lip movements from speech

10 0.61489421 161 nips-2007-Random Projections for Manifold Learning

11 0.61436194 138 nips-2007-Near-Maximum Entropy Models for Binary Neural Representations of Natural Images

12 0.61425459 45 nips-2007-Classification via Minimum Incremental Coding Length (MICL)

13 0.61423123 73 nips-2007-Distributed Inference for Latent Dirichlet Allocation

14 0.61219215 34 nips-2007-Bayesian Policy Learning with Trans-Dimensional MCMC

15 0.61174333 65 nips-2007-DIFFRAC: a discriminative and flexible framework for clustering

16 0.61172479 76 nips-2007-Efficient Convex Relaxation for Transductive Support Vector Machine

17 0.61164623 2 nips-2007-A Bayesian LDA-based model for semi-supervised part-of-speech tagging

18 0.61069727 189 nips-2007-Supervised Topic Models

19 0.61044502 86 nips-2007-Exponential Family Predictive Representations of State

20 0.6096887 187 nips-2007-Structured Learning with Approximate Inference