nips nips2007 nips2007-89 knowledge-graph by maker-knowledge-mining

89 nips-2007-Feature Selection Methods for Improving Protein Structure Prediction with Rosetta

Source: pdf

Author: Ben Blum, Rhiju Das, Philip Bradley, David Baker, Michael I. Jordan, David Tax

Abstract: Rosetta is one of the leading algorithms for protein structure prediction today. It is a Monte Carlo energy minimization method requiring many random restarts to ﬁnd structures with low energy. In this paper we present a resampling technique for structure prediction of small alpha/beta proteins using Rosetta. From an initial round of Rosetta sampling, we learn properties of the energy landscape that guide a subsequent round of sampling toward lower-energy structures. Rather than attempt to ﬁt the full energy landscape, we use feature selection methods—both L1-regularized linear regression and decision trees—to identify structural features that give rise to low energy. We then enrich these structural features in the second sampling round. Results are presented across a benchmark set of nine small alpha/beta proteins demonstrating that our methods seldom impair, and frequently improve, Rosetta’s performance. 1

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 edu Abstract Rosetta is one of the leading algorithms for protein structure prediction today. [sent-7, score-0.238]

2 In this paper we present a resampling technique for structure prediction of small alpha/beta proteins using Rosetta. [sent-9, score-0.432]

3 From an initial round of Rosetta sampling, we learn properties of the energy landscape that guide a subsequent round of sampling toward lower-energy structures. [sent-10, score-0.339]

4 Rather than attempt to ﬁt the full energy landscape, we use feature selection methods—both L1-regularized linear regression and decision trees—to identify structural features that give rise to low energy. [sent-11, score-0.424]

5 We then enrich these structural features in the second sampling round. [sent-12, score-0.214]

6 With the wealth of genome data now available, it is of great interest to determine the structures of the proteins that genes encode. [sent-15, score-0.211]

7 The protein structure prediction problem is to predict this conformation (the protein’s tertiary structure) from the amino acid sequence (the protein’s primary structure). [sent-18, score-0.545]

8 The biological function of a protein is dependent on its structure, so structure prediction is an important step towards function prediction. [sent-19, score-0.238]

9 Experimental methods for protein structure determination are costly and time-intensive, and the number of known protein sequences now far outstrips the capacity of experimentalists to determine their structures. [sent-21, score-0.321]

10 Structure prediction methods fall into two broad camps: comparative modeling, in which solved protein structures are known for one or more proteins with sequences similar to the target sequence (“homologs”), and ab initio modeling, in which no homologs are known. [sent-23, score-0.48]

11 Rosetta is one of the leading methods for ab initio protein structure prediction today. [sent-26, score-0.3]

12 Rosetta uses a Monte Carlo search procedure to minimize an energy function that is sufﬁciently accurate that the conformation found in nature (the “native” conformation) is generally the conformation with lowest Rosetta energy. [sent-27, score-0.609]

13 In particular, previous samples from conformation space might suggest regions of uniformly lower energy; these are regions in which Rosetta may wish to concentrate further sampling. [sent-32, score-0.207]

14 Unfortunately, conformation space is very high-dimensional and very irregular, so response surfaces do not generalize well beyond the span of the points to which they are ﬁtted. [sent-34, score-0.221]

15 No single local minimum computed in the ﬁrst round of Rosetta search will have all the native features. [sent-37, score-0.414]

16 However, many native features are present in at least some of the decoys. [sent-38, score-0.397]

17 In the ﬁrst step, we project the initial set of Rosetta models from continuous conformation space into a discrete feature space. [sent-41, score-0.228]

18 The structural features that we have designed characterize signiﬁcant aspects of protein structure and are largely sufﬁcient to determine a unique conformation. [sent-42, score-0.345]

19 In the second step, we use feature selection methods including both decision trees and Least Angle Regression (LARS) [4] to identify structural features that best account for energy variation in the initial set of models. [sent-43, score-0.421]

20 We can then predict that certain of these features (generally, those associated with low energy) are present in the native conformation. [sent-44, score-0.397]

21 This characterizes the way we map points from our discretized feature space back to continuous conformation space. [sent-48, score-0.228]

22 3 Response Surface Methods As an initial attempt at developing resampling methods for protein structure prediction, we investigated a response surface ﬁtting approach. [sent-54, score-0.466]

23 Our goal was to ﬁt a smoothed energy surface to the Rosetta models seen so far and then to minimize this surface to ﬁnd new starting points for local optimization of the Rosetta energy function. [sent-55, score-0.424]

24 Each residue in an amino acid sequence has two primary degrees of freedom: rotation around the Cα –N bond, referred to as the φ torsion angle, and rotation around the Cα –C bond, referred to as the ψ torsion angle. [sent-58, score-0.782]

25 However, it is difﬁcult to ﬁt a response surface in the space of torsion angles because the energy function is highly irregular in this space; a slight change in a single torsion angle typically causes large global structural changes, which in turn cause large energy changes. [sent-59, score-1.071]

26 Instead, we took the three-dimensional coordinates of the backbone atoms as our conformation space, with all models in the set aligned to a reference model. [sent-60, score-0.303]

27 There are four backbone atoms per residue and three coordinates per backbone atom, so an n-residue protein is represented by a 12n-dimensional vector. [sent-61, score-0.397]

28 Even for small proteins of only around 70 residues this space is very high-dimensional, but we found that most of the structural variation in sets of Rosetta models was captured by the ﬁrst 10 principal components. [sent-62, score-0.317]

29 Along certain directions, energy gradients were detectable that pointed toward the native structure. [sent-64, score-0.438]

30 a; in this graph, the native structure is represented as an ensemble of Rosetta-minimized structures that started at the native conformation). [sent-66, score-0.678]

31 A response surface ﬁtted to the Rosetta models shown in these graphs will therefore have high energy in the vicinity of the natives. [sent-69, score-0.236]

32 This motivates a shift in philosophy: rather than predicting energy and minimizing, we wish to predict features of the native structure and then enforce them independently of each other. [sent-71, score-0.574]

33 Two helices are visible behind a beta pleated sheat consisting of four strands, the bottommost three paired in the anti-parallel orientation and the topmost two paired in the parallel orientation. [sent-74, score-0.225]

34 4 Structural features For the purpose of the work described in this paper, we make use of two types of structural features: torsion angle features and beta contact features. [sent-76, score-0.847]

35 1 Torsion angle features The observed values of the φ and ψ angles for a single residue are strongly clustered in the database of solved protein structures (the PDB). [sent-78, score-0.448]

36 In order to discretize the possible torsion angles for each residue, we divide the Ramachandran plot into ﬁve regions, referred to as “A,” “B,” “E,” “G,” and “O,” (Figure 3. [sent-80, score-0.335]

37 A protein with 70 amino acid residues has 70 torsion bin features, each with possible values A, B, E, G, and O. [sent-83, score-0.626]

38 The primary search move in Rosetta is a fragment replacement move: the conformation of a string of three or nine consecutive residues within the target sequence is replaced with the conformation of a similar subsequence from the PDB. [sent-84, score-0.566]

39 A torsion angle feature can be constrained in Rosetta search by limiting the fragments to those which have torsion angles within the given bin at the given residue position. [sent-85, score-0.811]

40 Strings of torsion features are referred to as barcodes in Rosetta, and the apparatus for deﬁning and constraining them was developed in-house by Rosetta developers. [sent-86, score-0.416]

41 2 Beta contact features Proteins exhibit two kinds of secondary structure, characterized by regular hydrogen bond patterns: alpha helices and beta pleated sheets (Figure 3. [sent-88, score-0.588]

42 In beta sheets, however, the bonds can be between residues that are quite distant along the chain. [sent-91, score-0.267]

43 A beta contact feature for residues i and j indicates the presence of two backbone hydrogen bonds between i and j. [sent-92, score-0.556]

44 We use the same deﬁnition of beta pairing as the standard secondary structure assignment algorithm DSSP [5]. [sent-93, score-0.24]

45 A beta pairing feature is deﬁned for every triple (i, j, o) of residue numbers i and j and orientations o ∈ {parallel, antiparallel}. [sent-97, score-0.28]

46 The possible values of a beta pairing feature are X, indicating no pairing, and P1 or P2, indicating pleating of orientation 1 or 2, respectively. [sent-98, score-0.237]

47 Beta contact features are enforced in Rosetta by means of a technique called “jumping. [sent-99, score-0.281]

48 After a Rosetta search trajectory terminates, an attempt is made to close the chainbreak with local search over several torsion angles on either side of it. [sent-104, score-0.437]

49 5 Prediction of native features Let us transform our set of multi-valued features into a set of 0-1 valued features indicating whether or not a particular value for the feature is present. [sent-105, score-0.636]

50 Let us assume that each binary feature f has an independent energetic effect; if present, it brings with it an average energy bonus bf . [sent-106, score-0.228]

51 Under these assumptions, the full energy of a conformation d is modelled as E0 + d f bf + N , f where E0 is a constant offset, df is either 1 if the feature is present in d or 0 if it is absent, and N is Gaussian noise. [sent-107, score-0.415]

52 This model is partially justiﬁed by the fact that the true energy is indeed a sum of energies from local interactions, and our features capture local structural information. [sent-108, score-0.389]

53 Our hypothesis is that native features have lower energy on average even if other native features are not present. [sent-109, score-0.934]

54 In order to identify a small set of potentially native features, we use L1 regularization, or lasso regression [6], to ﬁnd a sparse model. [sent-110, score-0.363]

55 The small set of features that receive non-zero weights are those that best account for energy variations in the population of decoys. [sent-112, score-0.269]

56 Experience with Rosetta has shown that constraining more than ten or ﬁfteen torsion features can hamper search more than it helps; if there are very few fragments available for a given position that satisfy all torsion constraints, the lack of mobility at that position can be harmful. [sent-115, score-0.709]

57 This chart shows, for each protein, the fraction of LARS-selected features correctly labeled as native or non-native by the sign of the LARS weight. [sent-123, score-0.397]

58 a that LARS is informative about native features for most proteins. [sent-127, score-0.397]

59 Our resampling strategy is therefore to ﬂip a coin at the beginning of the Rosetta run to decide whether or not to constrain a particular LARS feature. [sent-130, score-0.212]

60 Resampling improves on unbiased Rosetta sampling if the number of viable runs (runs in which no non-native features are enforced) is sufﬁciently high that the beneﬁts from the enforcement of native features are visible. [sent-132, score-0.596]

61 2 Decision trees for beta contact features Beta contact features are less suited to the lasso regression approach than torsion angle features, because independence assumptions are not as valid. [sent-135, score-0.985]

62 For instance, contact (i, j, parallel) and contact 5 (a) (b) Figure 4: (a) LARS prediction accuracy when ﬁtted to total decoy population and to the three decision-tree leaves with lowest 10th percentile energies, ordered here by average rmsd. [sent-136, score-0.511]

63 (b) Relation of prediction accuracy to resampling improvement in LARS-only runs. [sent-137, score-0.25]

64 (i + 1, j + 1, parallel) are redundant and will usually co-occur, whereas contact (i, j, parallel) and contact (i − 1, j + 1, parallel) are mutually exclusive and will never co-occur. [sent-138, score-0.278]

65 For beta contact features, we employ a decision tree approach to divide the decoy population into non-overlapping clusters each deﬁned by the presence of several beta contacts. [sent-139, score-0.514]

66 Lasso regression is then employed in each cluster separately to determine likely native torsion features. [sent-140, score-0.611]

67 At each node, a beta contact feature is selected to use as a split point and a child node is created for each of the three possible values X, P1, and P2. [sent-142, score-0.302]

68 The beta contact feature is therefore chosen whose mutual information with the other beta contact features is maximized, as approximated by the sum of the marginal mutual informations with each other feature. [sent-144, score-0.662]

69 Since some clusters are sampled more heavily than others, the lowest energy within a cluster is not a fair measure of its quality, even though, in principle, we care only about the lowest achievable energy. [sent-145, score-0.251]

70 Our resampling strategy, given a decision tree, is to sample evenly from each of the top three leaves as ranked by 10th percentile energy. [sent-148, score-0.301]

71 Within the subpopulation of decoys deﬁned by each leaf, we select torsion features using LARS. [sent-149, score-0.528]

72 In our benchmark set, the top three low-energy leaves of the decision tree were generally closer to the native than the population at large. [sent-150, score-0.494]

73 Leaves are sorted by average rmsd, so “low energy leaf 1,” the “best” leaf, consists of decoys which are closest, on average, to the native conformation. [sent-153, score-0.624]

74 The best leaf consisted of only native contacts for all proteins except 1n0u and 1ogw, but in both these cases it contained structures generally lower in rmsd than the population at large and resampling achieved improvements over plain Rosetta sampling. [sent-154, score-1.012]

75 In general, LARS performed better on the leaves that were closer to the native structure, although there were a few notable exceptions. [sent-155, score-0.336]

76 Including more leaves in the resampling round increases the chances of resampling a native leaf but dilutes sampling of the best leaf in the pool. [sent-157, score-0.934]

77 6 Results We tested two Rosetta resampling schemes over a set of 9 alpha/beta proteins of between 59 and 81 residues. [sent-159, score-0.336]

78 In the ﬁrst scheme (referred to henceforth as “LARS-only”), 15 LARS-predicted torsion features were constrained at 30% frequency. [sent-160, score-0.395]

79 Each resampling scheme was compared against a control population generated at the same time. [sent-242, score-0.278]

80 The control and resampled populations for the LARS-only scheme consist of about 200,000 decoys each. [sent-244, score-0.246]

81 Our two primary measures of success for a resampling run are both based on root-mean-square distance to the native structure. [sent-247, score-0.539]

82 Our ﬁrst measure of success is the rmsd between the native structure and the lowest scoring model. [sent-250, score-0.548]

83 Our second measure of success is lowest rmsd among the twenty-ﬁve top-scoring models. [sent-252, score-0.213]

84 This is a smoother measure of the quality of the lowest scoring Rosetta models, and gives some indication of the prediction quality if more sophisticated minima-selection methods are used than Rosetta energy ˚ ranking. [sent-253, score-0.243]

85 Structures at 1A from the native have atomic-level resolution—this is the goal. [sent-254, score-0.298]

86 In proteins the ˚ size of those in our benchmark, structures more than 5A from the native are poor predictions. [sent-256, score-0.488]

87 The decision-tree scheme performed more consistently and achieved larger improvements on average; it improved the low-energy rmsd in 7 of the 9 benchmark proteins, with a signiﬁcant me˚ dian improvement of 1. [sent-259, score-0.235]

88 The LARS-only scheme was successful as well, providing improved ˚ lowest-energy predictions on 6 of the 9 benchmark proteins with a median improvement of 0. [sent-263, score-0.231]

89 97A away from the native ˚ structure, as compared to 2. [sent-266, score-0.298]

90 The two notable exceptions were 2reb, for which plain Rosetta search performs so well that constraints only hurt sampling, and 1n0u, for which plain Rosetta search concen˚ trates almost entirely on a cluster with incorrect topology at 10A. [sent-270, score-0.237]

91 In the case of 1dcj, resampling does yield lower rmsd structures—the top 25 low rms prediction is superior, and the minimum rmsd from the set is 1. [sent-274, score-0.544]

92 This emphasizes the point that resampling cannot hurt us too much. [sent-279, score-0.221]

93 If a plain Rosetta sampling round of n decoys is followed by a resampling round of n decoys, then no matter how poor the resampled decoys are, sampling efﬁciency is decreased by at most a factor of 2 (since we could have generated n plain Rosetta samples in the same time). [sent-280, score-0.835]

94 The danger is that resampling may overconverge to broad, false energy wells, achieving lower energies in the resampling round even though rmsd is higher. [sent-281, score-0.776]

95 This appears to occur with 2tif, in which the LARS-only low-energy prediction has signiﬁcantly lower energy than the control prediction despite being much farther from the native. [sent-282, score-0.287]

96 7 Discussion and Conclusions Our results demonstrate that our resampling techniques improve structure prediction on a majority of the proteins in our benchmark set. [sent-284, score-0.467]

97 Our ﬁrst resampling method signiﬁcantly improves Rosetta predictions in 3 of the 9 test cases, and marginally improves two or three more. [sent-285, score-0.214]

98 Our second resampling method expands the set of proteins on which we achieve improvements, including an additional atomic-level prediction. [sent-286, score-0.336]

99 Furthermore, it doesn’t hurt Rosetta too badly if a resampling scheme performs worse than unbiased sampling on some proteins, since models from the unbiased sampling round that precedes the resampling round can be used as predictions as well. [sent-290, score-0.737]

100 The success of our feature selection techniques suggests that the high dimensionality and multiple minima that make high resolution protein structure prediction difﬁcult to solve using traditional methods provide an excellent application for modern machine learning methods. [sent-297, score-0.343]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('rosetta', 0.653), ('native', 0.298), ('torsion', 0.268), ('lars', 0.197), ('resampling', 0.191), ('conformation', 0.187), ('rmsd', 0.147), ('proteins', 0.145), ('protein', 0.142), ('energy', 0.14), ('contact', 0.139), ('decoys', 0.134), ('beta', 0.122), ('residues', 0.105), ('features', 0.099), ('residue', 0.07), ('backbone', 0.069), ('structural', 0.067), ('round', 0.064), ('surface', 0.062), ('plain', 0.06), ('prediction', 0.059), ('angle', 0.053), ('leaf', 0.052), ('sampling', 0.048), ('amino', 0.047), ('atoms', 0.047), ('bf', 0.047), ('pairing', 0.047), ('resamp', 0.047), ('structures', 0.045), ('acid', 0.045), ('lowest', 0.044), ('enforced', 0.043), ('energies', 0.043), ('feature', 0.041), ('bond', 0.04), ('bonds', 0.04), ('helices', 0.04), ('hydrogen', 0.04), ('initio', 0.04), ('natives', 0.04), ('ramachandran', 0.04), ('angles', 0.039), ('leaves', 0.038), ('tree', 0.037), ('decision', 0.037), ('structure', 0.037), ('parallel', 0.036), ('fteen', 0.035), ('percentile', 0.035), ('benchmark', 0.035), ('secondary', 0.034), ('response', 0.034), ('bradley', 0.032), ('resampled', 0.032), ('search', 0.032), ('population', 0.03), ('hurt', 0.03), ('control', 0.029), ('referred', 0.028), ('primary', 0.028), ('scheme', 0.028), ('antiparallel', 0.027), ('chainbreak', 0.027), ('decoy', 0.027), ('enforcement', 0.027), ('fragment', 0.027), ('homologs', 0.027), ('pleated', 0.027), ('pleating', 0.027), ('rhiju', 0.027), ('sheets', 0.027), ('subpopulation', 0.027), ('subpopulations', 0.027), ('unbiased', 0.025), ('lasso', 0.025), ('improvements', 0.025), ('landscape', 0.023), ('populations', 0.023), ('predictions', 0.023), ('cluster', 0.023), ('success', 0.022), ('ab', 0.022), ('regression', 0.022), ('resolution', 0.022), ('fragments', 0.021), ('genome', 0.021), ('constraining', 0.021), ('strategy', 0.021), ('concentrate', 0.02), ('david', 0.02), ('local', 0.02), ('minima', 0.02), ('alpha', 0.02), ('generally', 0.019), ('bin', 0.019), ('trees', 0.019), ('trajectory', 0.019), ('identify', 0.018)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 1.0 89 nips-2007-Feature Selection Methods for Improving Protein Structure Prediction with Rosetta

Author: Ben Blum, Rhiju Das, Philip Bradley, David Baker, Michael I. Jordan, David Tax

2 0.083516069 75 nips-2007-Efficient Bayesian Inference for Dynamically Changing Graphs

Author: Ozgur Sumer, Umut Acar, Alexander T. Ihler, Ramgopal R. Mettu

Abstract: Motivated by stochastic systems in which observed evidence and conditional dependencies between states of the network change over time, and certain quantities of interest (marginal distributions, likelihood estimates etc.) must be updated, we study the problem of adaptive inference in tree-structured Bayesian networks. We describe an algorithm for adaptive inference that handles a broad range of changes to the network and is able to maintain marginal distributions, MAP estimates, and data likelihoods in all expected logarithmic time. We give an implementation of our algorithm and provide experiments that show that the algorithm can yield up to two orders of magnitude speedups on answering queries and responding to dynamic changes over the sum-product algorithm. 1

3 0.054088239 141 nips-2007-New Outer Bounds on the Marginal Polytope

Author: David Sontag, Tommi S. Jaakkola

Abstract: We give a new class of outer bounds on the marginal polytope, and propose a cutting-plane algorithm for efﬁciently optimizing over these constraints. When combined with a concave upper bound on the entropy, this gives a new variational inference algorithm for probabilistic inference in discrete Markov Random Fields (MRFs). Valid constraints on the marginal polytope are derived through a series of projections onto the cut polytope. As a result, we obtain tighter upper bounds on the log-partition function. We also show empirically that the approximations of the marginals are signiﬁcantly more accurate when using the tighter outer bounds. Finally, we demonstrate the advantage of the new constraints for ﬁnding the MAP assignment in protein structure prediction. 1

4 0.051457543 203 nips-2007-The rat as particle filter

Author: Aaron C. Courville, Nathaniel D. Daw

Abstract: Although theorists have interpreted classical conditioning as a laboratory model of Bayesian belief updating, a recent reanalysis showed that the key features that theoretical models capture about learning are artifacts of averaging over subjects. Rather than learning smoothly to asymptote (reﬂecting, according to Bayesian models, the gradual tradeoff from prior to posterior as data accumulate), subjects learn suddenly and their predictions ﬂuctuate perpetually. We suggest that abrupt and unstable learning can be modeled by assuming subjects are conducting inference using sequential Monte Carlo sampling with a small number of samples — one, in our simulations. Ensemble behavior resembles exact Bayesian models since, as in particle ﬁlters, it averages over many samples. Further, the model is capable of exhibiting sophisticated behaviors like retrospective revaluation at the ensemble level, even given minimally sophisticated individuals that do not track uncertainty in their beliefs over trials. 1

5 0.046750903 168 nips-2007-Reinforcement Learning in Continuous Action Spaces through Sequential Monte Carlo Methods

Author: Alessandro Lazaric, Marcello Restelli, Andrea Bonarini

Abstract: Learning in real-world domains often requires to deal with continuous state and action spaces. Although many solutions have been proposed to apply Reinforcement Learning algorithms to continuous state problems, the same techniques can be hardly extended to continuous action spaces, where, besides the computation of a good approximation of the value function, a fast method for the identiﬁcation of the highest-valued action is needed. In this paper, we propose a novel actor-critic approach in which the policy of the actor is estimated through sequential Monte Carlo methods. The importance sampling step is performed on the basis of the values learned by the critic, while the resampling step modiﬁes the actor’s policy. The proposed approach has been empirically compared to other learning algorithms into several domains; in this paper, we report results obtained in a control problem consisting of steering a boat across a river. 1

6 0.043280158 180 nips-2007-Sparse Feature Learning for Deep Belief Networks

7 0.042403154 145 nips-2007-On Sparsity and Overcompleteness in Image Models

8 0.040077776 182 nips-2007-Sparse deep belief net model for visual area V2

9 0.036781695 116 nips-2007-Learning the structure of manifolds using random projections

10 0.035363965 79 nips-2007-Efficient multiple hyperparameter learning for log-linear models

11 0.034270771 132 nips-2007-Modeling image patches with a directed hierarchy of Markov random fields

12 0.033841562 214 nips-2007-Variational inference for Markov jump processes

13 0.03310281 99 nips-2007-Hierarchical Penalization

14 0.032669988 197 nips-2007-The Infinite Markov Model

15 0.030186849 193 nips-2007-The Distribution Family of Similarity Distances

16 0.029094646 162 nips-2007-Random Sampling of States in Dynamic Programming

17 0.02874653 211 nips-2007-Unsupervised Feature Selection for Accurate Recommendation of High-Dimensional Image Data

18 0.027709384 8 nips-2007-A New View of Automatic Relevance Determination

19 0.027282808 134 nips-2007-Multi-Task Learning via Conic Programming

20 0.02686397 53 nips-2007-Compressed Regression

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, -0.107), (1, 0.005), (2, -0.014), (3, -0.028), (4, -0.009), (5, 0.007), (6, -0.022), (7, 0.047), (8, 0.028), (9, -0.046), (10, -0.018), (11, 0.02), (12, 0.054), (13, -0.043), (14, 0.021), (15, 0.026), (16, 0.007), (17, 0.01), (18, 0.018), (19, -0.044), (20, 0.024), (21, 0.022), (22, -0.017), (23, 0.039), (24, 0.087), (25, 0.013), (26, -0.057), (27, 0.022), (28, -0.082), (29, 0.022), (30, -0.008), (31, -0.055), (32, -0.018), (33, -0.037), (34, 0.016), (35, 0.016), (36, -0.017), (37, 0.006), (38, 0.045), (39, -0.04), (40, -0.045), (41, 0.075), (42, 0.107), (43, -0.08), (44, -0.051), (45, 0.081), (46, -0.041), (47, -0.055), (48, -0.056), (49, -0.024)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.91870743 89 nips-2007-Feature Selection Methods for Improving Protein Structure Prediction with Rosetta

Author: Ben Blum, Rhiju Das, Philip Bradley, David Baker, Michael I. Jordan, David Tax

2 0.53634596 79 nips-2007-Efficient multiple hyperparameter learning for log-linear models

Author: Chuan-sheng Foo, Chuong B. Do, Andrew Y. Ng

Abstract: In problems where input features have varying amounts of noise, using distinct regularization hyperparameters for different features provides an effective means of managing model complexity. While regularizers for neural networks and support vector machines often rely on multiple hyperparameters, regularizers for structured prediction models (used in tasks such as sequence labeling or parsing) typically rely only on a single shared hyperparameter for all features. In this paper, we consider the problem of choosing regularization hyperparameters for log-linear models, a class of structured prediction probabilistic models which includes conditional random ﬁelds (CRFs). Using an implicit differentiation trick, we derive an efﬁcient gradient-based method for learning Gaussian regularization priors with multiple hyperparameters. In both simulations and the real-world task of computational RNA secondary structure prediction, we ﬁnd that multiple hyperparameter learning can provide a signiﬁcant boost in accuracy compared to using only a single regularization hyperparameter. 1

3 0.51681387 27 nips-2007-Anytime Induction of Cost-sensitive Trees

Author: Saher Esmeir, Shaul Markovitch

Abstract: Machine learning techniques are increasingly being used to produce a wide-range of classiﬁers for complex real-world applications that involve nonuniform testing costs and misclassiﬁcation costs. As the complexity of these applications grows, the management of resources during the learning and classiﬁcation processes becomes a challenging task. In this work we introduce ACT (Anytime Cost-sensitive Trees), a novel framework for operating in such environments. ACT is an anytime algorithm that allows trading computation time for lower classiﬁcation costs. It builds a tree top-down and exploits additional time resources to obtain better estimations for the utility of the different candidate splits. Using sampling techniques ACT approximates for each candidate split the cost of the subtree under it and favors the one with a minimal cost. Due to its stochastic nature ACT is expected to be able to escape local minima, into which greedy methods may be trapped. Experiments with a variety of datasets were conducted to compare the performance of ACT to that of the state of the art cost-sensitive tree learners. The results show that for most domains ACT produces trees of signiﬁcantly lower costs. ACT is also shown to exhibit good anytime behavior with diminishing returns.

4 0.50862062 75 nips-2007-Efficient Bayesian Inference for Dynamically Changing Graphs

Author: Ozgur Sumer, Umut Acar, Alexander T. Ihler, Ramgopal R. Mettu

5 0.47495145 78 nips-2007-Efficient Principled Learning of Thin Junction Trees

Author: Anton Chechetka, Carlos Guestrin

Abstract: We present the ﬁrst truly polynomial algorithm for PAC-learning the structure of bounded-treewidth junction trees – an attractive subclass of probabilistic graphical models that permits both the compact representation of probability distributions and efﬁcient exact inference. For a constant treewidth, our algorithm has polynomial time and sample complexity. If a junction tree with sufﬁciently strong intraclique dependencies exists, we provide strong theoretical guarantees in terms of KL divergence of the result from the true distribution. We also present a lazy extension of our approach that leads to very signiﬁcant speed ups in practice, and demonstrate the viability of our method empirically, on several real world datasets. One of our key new theoretical insights is a method for bounding the conditional mutual information of arbitrarily large sets of variables with only polynomially many mutual information computations on ﬁxed-size subsets of variables, if the underlying distribution can be approximated by a bounded-treewidth junction tree. 1

6 0.47239301 31 nips-2007-Bayesian Agglomerative Clustering with Coalescents

7 0.47162801 67 nips-2007-Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation

8 0.44800162 8 nips-2007-A New View of Automatic Relevance Determination

9 0.43706968 87 nips-2007-Fast Variational Inference for Large-scale Internet Diagnosis

10 0.41662136 209 nips-2007-Ultrafast Monte Carlo for Statistical Summations

11 0.41465074 16 nips-2007-A learning framework for nearest neighbor search

12 0.40857714 99 nips-2007-Hierarchical Penalization

13 0.39227495 53 nips-2007-Compressed Regression

14 0.3886658 116 nips-2007-Learning the structure of manifolds using random projections

15 0.3696937 179 nips-2007-SpAM: Sparse Additive Models

16 0.35902166 171 nips-2007-Scan Strategies for Meteorological Radars

17 0.35548776 180 nips-2007-Sparse Feature Learning for Deep Belief Networks

18 0.35064951 167 nips-2007-Regulator Discovery from Gene Expression Time Series of Malaria Parasites: a Hierachical Approach

19 0.35011014 152 nips-2007-Parallelizing Support Vector Machines on Distributed Computers

20 0.34804267 61 nips-2007-Convex Clustering with Exemplar-Based Models

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(4, 0.015), (5, 0.046), (13, 0.022), (16, 0.021), (18, 0.014), (19, 0.012), (21, 0.06), (31, 0.035), (34, 0.012), (35, 0.028), (47, 0.097), (49, 0.01), (65, 0.36), (83, 0.087), (85, 0.017), (90, 0.059)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.74588346 89 nips-2007-Feature Selection Methods for Improving Protein Structure Prediction with Rosetta

Author: Ben Blum, Rhiju Das, Philip Bradley, David Baker, Michael I. Jordan, David Tax

2 0.59784865 153 nips-2007-People Tracking with the Laplacian Eigenmaps Latent Variable Model

Author: Zhengdong Lu, Cristian Sminchisescu, Miguel Á. Carreira-Perpiñán

Abstract: Reliably recovering 3D human pose from monocular video requires models that bias the estimates towards typical human poses and motions. We construct priors for people tracking using the Laplacian Eigenmaps Latent Variable Model (LELVM). LELVM is a recently introduced probabilistic dimensionality reduction model that combines the advantages of latent variable models—a multimodal probability density for latent and observed variables, and globally differentiable nonlinear mappings for reconstruction and dimensionality reduction—with those of spectral manifold learning methods—no local optima, ability to unfold highly nonlinear manifolds, and good practical scaling to latent spaces of high dimension. LELVM is computationally efﬁcient, simple to learn from sparse training data, and compatible with standard probabilistic trackers such as particle ﬁlters. We analyze the performance of a LELVM-based probabilistic sigma point mixture tracker in several real and synthetic human motion sequences and demonstrate that LELVM not only provides sufﬁcient constraints for robust operation in the presence of missing, noisy and ambiguous image measurements, but also compares favorably with alternative trackers based on PCA or GPLVM priors. Recent research in reconstructing articulated human motion has focused on methods that can exploit available prior knowledge on typical human poses or motions in an attempt to build more reliable algorithms. The high-dimensionality of human ambient pose space—between 30-60 joint angles or joint positions depending on the desired accuracy level, makes exhaustive search prohibitively expensive. This has negative impact on existing trackers, which are often not sufﬁciently reliable at reconstructing human-like poses, self-initializing or recovering from failure. Such difﬁculties have stimulated research in algorithms and models that reduce the effective working space, either using generic search focusing methods (annealing, state space decomposition, covariance scaling) or by exploiting speciﬁc problem structure (e.g. kinematic jumps). Experience with these procedures has nevertheless shown that any search strategy, no matter how effective, can be made signiﬁcantly more reliable if restricted to low-dimensional state spaces. This permits a more thorough exploration of the typical solution space, for a given, comparatively similar computational effort as a high-dimensional method. The argument correlates well with the belief that the human pose space, although high-dimensional in its natural ambient parameterization, has a signiﬁcantly lower perceptual (latent or intrinsic) dimensionality, at least in a practical sense—many poses that are possible are so improbable in many real-world situations that it pays off to encode them with low accuracy. A perceptual representation has to be powerful enough to capture the diversity of human poses in a sufﬁciently broad domain of applicability (the task domain), yet compact and analytically tractable for search and optimization. This justiﬁes the use of models that are nonlinear and low-dimensional (able to unfold highly nonlinear manifolds with low distortion), yet probabilistically motivated and globally continuous for efﬁcient optimization. Reducing dimensionality is not the only goal: perceptual representations have to preserve critical properties of the ambient space. Reliable tracking needs locality: nearby regions in ambient space have to be mapped to nearby regions in latent space. If this does not hold, the tracker is forced to make unrealistically large, and difﬁcult to predict jumps in latent space in order to follow smooth trajectories in the joint angle ambient space. 1 In this paper we propose to model priors for articulated motion using a recently introduced probabilistic dimensionality reduction method, the Laplacian Eigenmaps Latent Variable Model (LELVM) [1]. Section 1 discusses the requirements of priors for articulated motion in the context of probabilistic and spectral methods for manifold learning, and section 2 describes LELVM and shows how it combines both types of methods in a principled way. Section 3 describes our tracking framework (using a particle ﬁlter) and section 4 shows experiments with synthetic and real human motion sequences using LELVM priors learned from motion-capture data. Related work: There is signiﬁcant work in human tracking, using both generative and discriminative methods. Due to space limitations, we will focus on the more restricted class of 3D generative algorithms based on learned state priors, and not aim at a full literature review. Deriving compact prior representations for tracking people or other articulated objects is an active research ﬁeld, steadily growing with the increased availability of human motion capture data. Howe et al. and Sidenbladh et al. [2] propose Gaussian mixture representations of short human motion fragments (snippets) and integrate them in a Bayesian MAP estimation framework that uses 2D human joint measurements, independently tracked by scaled prismatic models [3]. Brand [4] models the human pose manifold using a Gaussian mixture and uses an HMM to infer the mixture component index based on a temporal sequence of human silhouettes. Sidenbladh et al. [5] use similar dynamic priors and exploit ideas in texture synthesis—efﬁcient nearest-neighbor search for similar motion fragments at runtime—in order to build a particle-ﬁlter tracker with observation model based on contour and image intensity measurements. Sminchisescu and Jepson [6] propose a low-dimensional probabilistic model based on ﬁtting a parametric reconstruction mapping (sparse radial basis function) and a parametric latent density (Gaussian mixture) to the embedding produced with a spectral method. They track humans walking and involved in conversations using a Bayesian multiple hypotheses framework that fuses contour and intensity measurements. Urtasun et al. [7] use a dynamic MAP estimation framework based on a GPLVM and 2D human joint correspondences obtained from an independent image-based tracker. Li et al. [8] use a coordinated mixture of factor analyzers within a particle ﬁltering framework, in order to reconstruct human motion in multiple views using chamfer matching to score different conﬁguration. Wang et al. [9] learn a latent space with associated dynamics where both the dynamics and observation mapping are Gaussian processes, and Urtasun et al. [10] use it for tracking. Taylor et al. [11] also learn a binary latent space with dynamics (using an energy-based model) but apply it to synthesis, not tracking. Our work learns a static, generative low-dimensional model of poses and integrates it into a particle ﬁlter for tracking. We show its ability to work with real or partially missing data and to track multiple activities. 1 Priors for articulated human pose We consider the problem of learning a probabilistic low-dimensional model of human articulated motion. Call y ∈ RD the representation in ambient space of the articulated pose of a person. In this paper, y contains the 3D locations of anywhere between 10 and 60 markers located on the person’s joints (other representations such as joint angles are also possible). The values of y have been normalised for translation and rotation in order to remove rigid motion and leave only the articulated motion (see section 3 for how we track the rigid motion). While y is high-dimensional, the motion pattern lives in a low-dimensional manifold because most values of y yield poses that violate body constraints or are simply atypical for the motion type considered. Thus we want to model y in terms of a small number of latent variables x given a collection of poses {yn }N (recorded from a human n=1 with motion-capture technology). The model should satisfy the following: (1) It should deﬁne a probability density for x and y, to be able to deal with noise (in the image or marker measurements) and uncertainty (from missing data due to occlusion or markers that drop), and to allow integration in a sequential Bayesian estimation framework. The density model should also be ﬂexible enough to represent multimodal densities. (2) It should deﬁne mappings for dimensionality reduction F : y → x and reconstruction f : x → y that apply to any value of x and y (not just those in the training set); and such mappings should be deﬁned on a global coordinate system, be continuous (to avoid physically impossible discontinuities) and differentiable (to allow efﬁcient optimisation when tracking), yet ﬂexible enough to represent the highly nonlinear manifold of articulated poses. From a statistical machine learning point of view, this is precisely what latent variable models (LVMs) do; for example, factor analysis deﬁnes linear mappings and Gaussian densities, while the generative topographic mapping (GTM; [12]) deﬁnes nonlinear mappings and a Gaussian-mixture density in ambient space. However, factor analysis is too limited to be of practical use, and GTM— 2 while ﬂexible—has two important practical problems: (1) the latent space must be discretised to allow tractable learning and inference, which limits it to very low (2–3) latent dimensions; (2) the parameter estimation is prone to bad local optima that result in highly distorted mappings. Another dimensionality reduction method recently introduced, GPLVM [13], which uses a Gaussian process mapping f (x), partly improves this situation by deﬁning a tunable parameter xn for each data point yn . While still prone to local optima, this allows the use of a better initialisation for {xn }N (obtained from a spectral method, see later). This has prompted the application of n=1 GPLVM for tracking human motion [7]. However, GPLVM has some disadvantages: its training is very costly (each step of the gradient iteration is cubic on the number of training points N , though approximations based on using few points exist); unlike true LVMs, it deﬁnes neither a posterior distribution p(x|y) in latent space nor a dimensionality reduction mapping E {x|y}; and the latent representation it obtains is not ideal. For example, for periodic motions such as running or walking, repeated periods (identical up to small noise) can be mapped apart from each other in latent space because nothing constrains xn and xm to be close even when yn = ym (see ﬁg. 3 and [10]). There exists a different type of dimensionality reduction methods, spectral methods (such as Isomap, LLE or Laplacian eigenmaps [14]), that have advantages and disadvantages complementary to those of LVMs. They deﬁne neither mappings nor densities but just a correspondence (xn , yn ) between points in latent space xn and ambient space yn . However, the training is efﬁcient (a sparse eigenvalue problem) and has no local optima, and often yields a correspondence that successfully models highly nonlinear, convoluted manifolds such as the Swiss roll. While these attractive properties have spurred recent research in spectral methods, their lack of mappings and densities has limited their applicability in people tracking. However, a new model that combines the advantages of LVMs and spectral methods in a principled way has been recently proposed [1], which we brieﬂy describe next. 2 The Laplacian Eigenmaps Latent Variable Model (LELVM) LELVM is based on a natural way of deﬁning an out-of-sample mapping for Laplacian eigenmaps (LE) which, in addition, results in a density model. In LE, typically we ﬁrst deﬁne a k-nearestneighbour graph on the sample data {yn }N and weigh each edge yn ∼ ym by a Gaussian afﬁnity n=1 2 1 function K(yn , ym ) = wnm = exp (− 2 (yn − ym )/σ ). Then the latent points X result from: min tr XLX⊤ s.t. X ∈ RL×N , XDX⊤ = I, XD1 = 0 (1) where we deﬁne the matrix XL×N = (x1 , . . . , xN ), the symmetric afﬁnity matrix WN ×N , the deN gree matrix D = diag ( n=1 wnm ), the graph Laplacian matrix L = D−W, and 1 = (1, . . . , 1)⊤ . The constraints eliminate the two trivial solutions X = 0 (by ﬁxing an arbitrary scale) and x1 = · · · = xN (by removing 1, which is an eigenvector of L associated with a zero eigenvalue). The solution is given by the leading u2 , . . . , uL+1 eigenvectors of the normalised afﬁnity matrix 1 1 1 N = D− 2 WD− 2 , namely X = V⊤ D− 2 with VN ×L = (v2 , . . . , vL+1 ) (an a posteriori translated, rotated or uniformly scaled X is equally valid). Following [1], we now deﬁne an out-of-sample mapping F(y) = x for a new point y as a semisupervised learning problem, by recomputing the embedding as in (1) (i.e., augmenting the graph Laplacian with the new point), but keeping the old embedding ﬁxed: L K(y) X⊤ min tr ( X x ) K(y)⊤ 1⊤ K(y) (2) x⊤ x∈RL 2 where Kn (y) = K(y, yn ) = exp (− 1 (y − yn )/σ ) for n = 1, . . . , N is the kernel induced by 2 the Gaussian afﬁnity (applied only to the k nearest neighbours of y, i.e., Kn (y) = 0 if y ≁ yn ). This is one natural way of adding a new point to the embedding by keeping existing embedded points ﬁxed. We need not use the constraints from (1) because they would trivially determine x, and the uninteresting solutions X = 0 and X = constant were already removed in the old embedding anyway. The solution yields an out-of-sample dimensionality reduction mapping x = F(y): x = F(y) = X K(y) 1⊤ K(y) N K(y,yn ) PN x n=1 K(y,yn′ ) n ′ = (3) n =1 applicable to any point y (new or old). This mapping is formally identical to a Nadaraya-Watson estimator (kernel regression; [15]) using as data {(xn , yn )}N and the kernel K. We can take this n=1 a step further by deﬁning a LVM that has as joint distribution a kernel density estimate (KDE): p(x, y) = 1 N N n=1 Ky (y, yn )Kx (x, xn ) p(y) = 3 1 N N n=1 Ky (y, yn ) p(x) = 1 N N n=1 Kx (x, xn ) where Ky is proportional to K so it integrates to 1, and Kx is a pdf kernel in x–space. Consequently, the marginals in observed and latent space are also KDEs, and the dimensionality reduction and reconstruction mappings are given by kernel regression (the conditional means E {y|x}, E {x|y}): F(y) = N n=1 p(n|y)xn f (x) = N K (x,xn ) PN x y n=1 Kx (x,xn′ ) n ′ = n =1 N n=1 p(n|x)yn . (4) We allow the bandwidths to be different in the latent and ambient spaces: 2 2 1 Kx (x, xn ) ∝ exp (− 1 (x − xn )/σx ) and Ky (y, yn ) ∝ exp (− 2 (y − yn )/σy ). They 2 may be tuned to control the smoothness of the mappings and densities [1]. Thus, LELVM naturally extends a LE embedding (efﬁciently obtained as a sparse eigenvalue problem with a cost O(N 2 )) to global, continuous, differentiable mappings (NW estimators) and potentially multimodal densities having the form of a Gaussian KDE. This allows easy computation of posterior probabilities such as p(x|y) (unlike GPLVM). It can use a continuous latent space of arbitrary dimension L (unlike GTM) by simply choosing L eigenvectors in the LE embedding. It has no local optima since it is based on the LE embedding. LELVM can learn convoluted mappings (e.g. the Swiss roll) and deﬁne maps and densities for them [1]. The only parameters to set are the graph parameters (number of neighbours k, afﬁnity width σ) and the smoothing bandwidths σx , σy . 3 Tracking framework We follow the sequential Bayesian estimation framework, where for state variables s and observation variables z we have the recursive prediction and correction equations: p(st |z0:t−1 ) = p(st |st−1 ) p(st−1 |z0:t−1 ) dst−1 p(st |z0:t ) ∝ p(zt |st ) p(st |z0:t−1 ). (5) L We deﬁne the state variables as s = (x, d) where x ∈ R is the low-dim. latent space (for pose) and d ∈ R3 is the centre-of-mass location of the body (in the experiments our state also includes the orientation of the body, but for simplicity here we describe only the translation). The observed variables z consist of image features or the perspective projection of the markers on the camera plane. The mapping from state to observations is (for the markers’ case, assuming M markers): P f x ∈ RL − − → y ∈ R3M −→ ⊕ − − − z ∈ R2M −− − − −→ d ∈ R3 (6) where f is the LELVM reconstruction mapping (learnt from mocap data); ⊕ shifts each 3D marker by d; and P is the perspective projection (pinhole camera), applied to each 3D point separately. Here we use a simple observation model p(zt |st ): Gaussian with mean given by the transformation (6) and isotropic covariance (set by the user to control the inﬂuence of measurements in the tracking). We assume known correspondences and observations that are obtained either from the 3D markers (for tracking synthetic data) or 2D tracks obtained from a 2D tracker. Our dynamics model is p(st |st−1 ) ∝ pd (dt |dt−1 ) px (xt |xt−1 ) p(xt ) (7) where both dynamics models for d and x are random walks: Gaussians centred at the previous step value dt−1 and xt−1 , respectively, with isotropic covariance (set by the user to control the inﬂuence of dynamics in the tracking); and p(xt ) is the LELVM prior. Thus the overall dynamics predicts states that are both near the previous state and yield feasible poses. Of course, more complex dynamics models could be used if e.g. the speed and direction of movement are known. As tracker we use the Gaussian mixture Sigma-point particle ﬁlter (GMSPPF) [16]. This is a particle ﬁlter that uses a Gaussian mixture representation for the posterior distribution in state space and updates it with a Sigma-point Kalman ﬁlter. This Gaussian mixture will be used as proposal distribution to draw the particles. As in other particle ﬁlter implementations, the prediction step is carried out by approximating the integral (5) with particles and updating the particles’ weights. Then, a new Gaussian mixture is ﬁtted with a weighted EM algorithm to these particles. This replaces the resampling stage needed by many particle ﬁlters and mitigates the problem of sample depletion while also preventing the number of components in the Gaussian mixture from growing over time. The choice of this particular tracker is not critical; we use it to illustrate the fact that LELVM can be introduced in any probabilistic tracker for nonlinear, nongaussian models. Given the corrected distribution p(st |z0:t ), we choose its mean as recovered state (pose and location). It is also possible to choose instead the mode closest to the state at t − 1, which could be found by mean-shift or Newton algorithms [17] since we are using a Gaussian-mixture representation in state space. 4 4 Experiments We demonstrate our low-dimensional tracker on image sequences of people walking and running, both synthetic (ﬁg. 1) and real (ﬁg. 2–3). Fig. 1 shows the model copes well with persistent partial occlusion and severely subsampled training data (A,B), and quantitatively evaluates temporal reconstruction (C). For all our experiments, the LELVM parameters (number of neighbors k, Gaussian afﬁnity σ, and bandwidths σx and σy ) were set manually. We mainly considered 2D latent spaces (for pose, plus 6D for rigid motion), which were expressive enough for our experiments. More complex, higher-dimensional models are straightforward to construct. The initial state distribution p(s0 ) was chosen a broad Gaussian, the dynamics and observation covariance were set manually to control the tracking smoothness, and the GMSPPF tracker used a 5-component Gaussian mixture in latent space (and in the state space of rigid motion) and a small set of 500 particles. The 3D representation we use is a 102-D vector obtained by concatenating the 3D markers coordinates of all the body joints. These would be highly unconstrained if estimated independently, but we only use them as intermediate representation; tracking actually occurs in the latent space, tightly controlled using the LELVM prior. For the synthetic experiments and some of the real experiments (ﬁgs. 2–3) the camera parameters and the body proportions were known (for the latter, we used the 2D outputs of [6]). For the CMU mocap video (ﬁg. 2B) we roughly guessed. We used mocap data from several sources (CMU, OSU). As observations we always use 2D marker positions, which, depending on the analyzed sequence were either known (the synthetic case), or provided by an existing tracker [6] or speciﬁed manually (ﬁg. 2B). Alternatively 2D point trackers similar to the ones of [7] can be used. The forward generative model is obtained by combining the latent to ambient space mapping (this provides the position of the 3D markers) with a perspective projection transformation. The observation model is a product of Gaussians, each measuring the probability of a particular marker position given its corresponding image point track. Experiments with synthetic data: we analyze the performance of our tracker in controlled conditions (noise perturbed synthetically generated image tracks) both under regular circumstances (reasonable sampling of training data) and more severe conditions with subsampled training points and persistent partial occlusion (the man running behind a fence, with many of the 2D marker tracks obstructed). Fig. 1B,C shows both the posterior (ﬁltered) latent space distribution obtained from our tracker, and its mean (we do not show the distribution of the global rigid body motion; in all experiments this is tracked with good accuracy). In the latent space plot shown in ﬁg. 1B, the onset of running (two cycles were used) appears as a separate region external to the main loop. It does not appear in the subsampled training set in ﬁg. 1B, where only one running cycle was used for training and the onset of running was removed. In each case, one can see that the model is able to track quite competently, with a modest decrease in its temporal accuracy, shown in ﬁg. 1C, where the averages are computed per 3D joint (normalised wrt body height). Subsampling causes some ambiguity in the estimate, e.g. see the bimodality in the right plot in ﬁg. 1C. In another set of experiments (not shown) we also tracked using different subsets of 3D markers. The estimates were accurate even when about 30% of the markers were dropped. Experiments with real images: this shows our tracker’s ability to work with real motions of different people, with different body proportions, not in its latent variable model training set (ﬁgs. 2–3). We study walking, running and turns. In all cases, tracking and 3D reconstruction are reasonably accurate. We have also run comparisons against low-dimensional models based on PCA and GPLVM (ﬁg. 3). It is important to note that, for LELVM, errors in the pose estimates are primarily caused by mismatches between the mocap data used to learn the LELVM prior and the body proportions of the person in the video. For example, the body proportions of the OSU motion captured walker are quite different from those of the image in ﬁg. 2–3 (e.g. note how the legs of the stick man are shorter relative to the trunk). Likewise, the style of the runner from the OSU data (e.g. the swinging of the arms) is quite different from that of the video. Finally, the interest points tracked by the 2D tracker do not entirely correspond either in number or location to the motion capture markers, and are noisy and sometimes missing. In future work, we plan to include an optimization step to also estimate the body proportions. This would be complicated for a general, unconstrained model because the dimensions of the body couple with the pose, so either one or the other can be changed to improve the tracking error (the observation likelihood can also become singular). But for dedicated prior pose models like ours these difﬁculties should be signiﬁcantly reduced. The model simply cannot assume highly unlikely stances—these are either not representable at all, or have reduced probability—and thus avoids compensatory, unrealistic body proportion estimates. 5 n = 15 n = 40 n = 65 n = 90 n = 115 n = 140 A 1.5 1.5 1.5 1.5 1 −1 −1 −1 −1 −1 −1.5 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1 −1 −1 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 −1 −1 1 −1 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 −1.5 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0.3 0.2 0.1 −1 0.4 0.3 0.2 −1.5 −1.5 n = 60 0.4 0.3 −2 −1 −1.5 −1 0 −0.5 n = 49 0.4 0.1 −1 −1.5 1 0 −0.5 −2 −1 −1.5 −1 0.5 0 −0.5 −1.5 −1.5 0.5 1 0.5 0 −0.5 −1.5 −1.5 −1.5 1 0.5 0 −0.5 n = 37 0.2 0.1 −1 −1 −1.5 −0.5 −1 −1.5 −2 1 0.5 0 −0.5 −0.5 −1.5 −1.5 0.3 0.2 0.1 0 0.4 0.3 0.2 −0.5 n = 25 0.4 0.3 −1 0 −0.5 −1 −1.5 1 0.5 0 0 −0.5 1.5 1 0.5 0 −0.5 −2 1 0.5 0.5 0 −1.5 −1.5 −1.5 1.5 1 0.5 0 −1 −1.5 −2 1 1 0.5 −0.5 −1 −1.5 −1.5 n = 13 0.4 −1 −1 −1.5 −1.5 −1.5 n=1 B −1 −1 −1.5 −1.5 1.5 1 0.5 −0.5 0 −1 −1.5 −2 −2 1 0 −0.5 −0.5 −0.5 −1 −1.5 −2 0.5 0 0 −0.5 0.5 0 −0.5 −1 −1.5 −2 1 0.5 0.5 0 1 0.5 0 −0.5 −1 −1.5 −1 −1.5 −2 1 1 0.5 1.5 1 0.5 0 −0.5 0 −0.5 −1 −1.5 −2 1 −0.5 1.5 1.5 1 0.5 0.5 0 −0.5 −1 −1.5 −2 1.5 1 1 0.5 0 −0.5 −2 0 −0.5 −0.5 1.5 1 0.5 0 −1 −1.5 −1 −1.5 0.5 0 0 −0.5 −1.5 −1.5 −1.5 1.5 1.5 1 0.5 −0.5 0 −0.5 −2 1 0.5 0.5 0 −0.5 −1 −1.5 −1.5 1.5 1 1 0.5 0 −1 −1.5 1 1 0.5 0 −0.5 −0.5 1.5 1 −0.5 −2 1 0.5 0 0 −0.5 0.5 0 −1 −1.5 −2 1 0.5 0.5 0 −0.5 1.5 1.5 1 −0.5 −2 1 1 0.5 0.5 0 −1 −1.5 −1 −1.5 −2 −2 1 0 −0.5 1.5 1 0.5 −0.5 0 −0.5 −1 −1.5 −2 0.5 0 −0.5 0.5 0 −0.5 −1 −1.5 −2 1 0.5 0 −0.5 0.5 0 −0.5 −1 −1.5 −1 −1.5 −2 1 0.5 0 1 0.5 0 −0.5 0 −0.5 −1 −1.5 −2 1 0.5 1.5 1 0.5 0.5 0 −0.5 −1 −1.5 1 1 1 0.5 0 −0.5 −2 1.5 1.5 1 1 0.5 0 −1 −1.5 1.5 1.5 1 0.5 −0.5 0.2 0.1 0.1 0 0 0 0 0 0 −0.1 −0.1 −0.1 −0.1 −0.1 −0.1 −0.2 −0.2 −0.2 −0.2 −0.2 −0.2 −0.3 −0.3 −0.3 −0.3 −0.3 −0.3 −0.4 0.4 0.6 0.8 1.5 1 1.2 1.4 1.6 1.8 2 2.2 −0.4 0.4 2.4 1.5 0.6 0.8 1.5 1 1.2 1.4 1.6 1.8 2 2.2 −0.4 0.4 2.4 1.5 0.6 0.8 1.5 1.5 1.5 1 1.2 1.4 1.6 1.8 2 2.2 −0.4 0.4 2.4 1.5 0.6 0.8 1.5 1.5 1.5 1 1.2 1.4 1.6 1.8 2 2.2 −0.4 0.4 2.4 1.5 0.6 0.8 1.5 1.5 1.5 1 1.2 1.4 1.6 1.8 2 2.2 −0.4 0.4 2.4 1.5 0.6 0.8 1.5 1.5 1.5 1 1.2 1.4 1.6 1.8 2 2.2 2.4 1.5 1.5 1.5 1.5 1.5 1 1 0.5 0.5 0 0 1 −0.5 −0.5 0 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 0.1 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 0 −0.5 −1.5 0.5 0 0 −0.5 0 −0.5 −0.5 −0.5 −2 −1 −1 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 −2 −1 −1.5 −1 −1.5 1 0.5 0.5 0 −1.5 −1 1 1 0.5 −2 −1 −1.5 −1.5 −0.5 −1 −2 1 −1 0 −1 −1.5 −2 −0.5 −2 −1.5 0.5 −0.5 −1 −1.5 0 −0.5 −1 −1.5 0 −1.5 0.5 0 −0.5 −1 −0.5 −1 −1.5 1 0.5 0 −0.5 −1 −0.5 −1 1 0.5 0 −1.5 1 0.5 0 −0.5 −2 1 0.5 −2 −1 −1.5 −1.5 −0.5 0 −1 1 −1 0 1 −1.5 −2 −0.5 −2 −1.5 1 0.5 0 0.5 −0.5 −1 −1.5 0 −0.5 −1 −1.5 0 −1.5 0.5 0 −0.5 −1 −0.5 −1 −1.5 1 0.5 0 −0.5 −1 −0.5 −1 1 0.5 0 −1.5 1 0.5 1 0.5 0 −0.5 −2 1 0.5 −2 −1 −1.5 −1.5 −0.5 0 −1 1 −1 0 1 −1.5 −2 −0.5 −2 −1.5 1 0.5 0 0.5 −0.5 −1 −1.5 0 −0.5 −1 −1.5 0 −1.5 0.5 0 −0.5 −1 −0.5 −1 −1.5 1 0.5 0 −0.5 −1 −0.5 −1 1 0.5 0 −1.5 1 0.5 1 0.5 0 −0.5 −2 1 0.5 −2 −1 −1.5 −1.5 −0.5 0 −1 1 −1 0 1 −1.5 −2 −0.5 −2 −1 −1.5 −0.5 1 0.5 0 0.5 −0.5 −1 −1.5 0 −0.5 −0.5 −1 −1 −1.5 0.5 0 0 −0.5 −2 −1.5 0 −1.5 1 0.5 0.5 0 −2 −1 −1.5 −1.5 −1 1 1 0.5 −0.5 −1 1 0.5 1 0.5 −0.5 −1 −2 1 0 −0.5 −1 −1.5 −1.5 −1.5 −2 −1.5 0.5 0 −0.5 −1 −0.5 0 −0.5 −1.5 −1 −1.5 1 0.5 0 −0.5 0 1 −1 −0.5 −1 1 0.5 0 1 0.5 0 0.5 −0.5 −1 −0.5 −2 1 0.5 1 0.5 1 0.5 0 −1 1 0 1 −1.5 −2 0.5 0 1 0.5 −0.5 −1 −1.5 1 0.5 1 0.5 −1.5 −1.5 −1.5 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 −1.5 −1 −0.5 0 0.5 1 0.13 0.12 RMSE C RMSE 0.08 0.06 0.04 0.02 0.11 0.1 0.09 0.08 0.07 0.06 0 0 50 100 150 0.05 0 10 time step n 20 30 40 50 60 time step n Figure 1: OSU running man motion capture data. A: we use 217 datapoints for training LELVM (with added noise) and for tracking. Row 1: tracking in the 2D latent space. The contours (very tight in this sequence) are the posterior probability. Row 2: perspective-projection-based observations with occlusions. Row 3: each quadruplet (a, a′ , b, b′ ) show the true pose of the running man from a front and side views (a, b), and the reconstructed pose by tracking with our model (a′ , b′ ). B: we use the ﬁrst running cycle for training LELVM and the second cycle for tracking. C: RMSE errors for each frame, for the tracking of A (left plot) and B (middle plot), normalised so that 1 equals the M 1 ˆ 2 height of the stick man. RMSE(n) = M j=1 ynj − ynj −1/2 for all 3D locations of the M ˆ markers, i.e., comparison of reconstructed stick man yn with ground-truth stick man yn . Right plot: multimodal posterior distribution in pose space for the model of A (frame 42). Comparison with PCA and GPLVM (ﬁg. 3): for these models, the tracker uses the same GMSPPF setting as for LELVM (number of particles, initialisation, random-walk dynamics, etc.) but with the mapping y = f (x) provided by GPLVM or PCA, and with a uniform prior p(x) in latent space (since neither GPLVM nor the non-probabilistic PCA provide one). The LELVM-tracker uses both its f (x) and latent space prior p(x), as discussed. All methods use a 2D latent space. We ensured the best possible training of GPLVM by model selection based on multiple runs. For PCA, the latent space looks deceptively good, showing non-intersecting loops. However, (1) individual loops do not collect together as they should (for LELVM they do); (2) worse still, the mapping from 2D to pose space yields a poor observation model. The reason is that the loop in 102-D pose space is nonlinearly bent and a plane can at best intersect it at a few points, so the tracker often stays put at one of those (typically an “average” standing position), since leaving it would increase the error a lot. Using more latent dimensions would improve this, but as LELVM shows, this is not necessary. For GPLVM, we found high sensitivity to ﬁlter initialisation: the estimates have high variance across runs and are inaccurate ≈ 80% of the time. When it fails, the GPLVM tracker often freezes in latent space, like PCA. When it does succeed, it produces results that are comparable with LELVM, although somewhat less accurate visually. However, even then GPLVM’s latent space consists of continuous chunks spread apart and offset from each other; GPLVM has no incentive to place nearby two xs mapping to the same y. This effect, combined with the lack of a data-sensitive, realistic latent space density p(x), makes GPLVM jump erratically from chunk to chunk, in contrast with LELVM, which smoothly follows the 1D loop. Some GPLVM problems might be alleviated using higher-order dynamics, but our experiments suggest that such modeling sophistication is less 6 0 0.5 1 n=1 n = 15 n = 29 n = 43 n = 55 n = 69 A 100 100 100 100 100 50 50 50 50 50 0 0 0 0 0 0 −50 −50 −50 −50 −50 −50 −100 −100 −100 −100 −100 −100 50 50 40 −40 −20 −50 −30 −10 0 −40 20 −20 40 n=4 −40 10 20 −20 40 50 0 n=9 −40 −20 30 40 50 0 n = 14 −40 20 −20 40 50 30 20 10 0 0 −40 −20 −30 −20 −40 10 20 −30 −10 0 −50 30 50 n = 19 −50 −30 −10 −50 30 −10 −20 −30 −40 10 50 40 30 20 10 0 −50 −30 −10 −50 20 −10 −20 −30 −40 10 50 40 30 20 10 0 −50 −30 −10 −50 30 −10 −20 −30 −40 0 50 50 40 30 20 10 0 −50 −30 −10 −50 30 −10 −20 −30 −40 10 40 30 20 10 0 −10 −20 −30 50 40 30 20 10 0 −10 −50 100 40 −40 10 20 −50 30 50 n = 24 40 50 n = 29 B 20 20 20 20 20 40 40 40 40 40 60 60 60 60 60 80 80 80 80 80 80 100 100 100 100 100 100 120 120 120 120 120 120 140 140 140 140 140 140 160 160 160 160 160 160 180 180 180 180 180 200 200 200 200 200 220 220 220 220 220 50 100 150 200 250 300 350 50 100 150 200 250 300 350 50 100 150 200 250 300 350 50 100 150 200 250 300 350 20 40 60 180 200 220 50 100 150 200 250 300 350 50 100 150 100 100 100 100 100 50 50 50 50 200 250 300 350 100 50 50 0 0 0 0 0 0 −50 −50 −50 −50 −50 −50 −100 −100 −100 −100 −100 −100 50 50 40 −40 −20 −50 −30 −10 0 −40 10 20 −50 30 40 −40 −20 −50 −30 −10 0 −40 10 20 −50 30 40 −40 −20 −50 −30 −10 0 −40 −20 30 40 50 0 −40 10 20 −50 30 40 50 40 30 30 20 20 10 10 0 −50 −30 −10 −50 20 0 −10 −20 −30 −40 10 40 30 20 10 0 −10 −20 −30 50 50 40 30 20 10 0 −10 −20 −30 50 50 40 30 20 10 0 −10 −20 −30 50 40 30 20 10 0 −10 −50 50 −40 −20 −30 −20 −30 −10 0 −40 10 20 −50 30 40 50 −10 −50 −40 −20 −30 −20 −30 −10 0 −40 10 20 −50 30 40 50 Figure 2: A: tracking of a video from [6] (turning & walking). We use 220 datapoints (3 full walking cycles) for training LELVM. Row 1: tracking in the 2D latent space. The contours are the estimated posterior probability. Row 2: tracking based on markers. The red dots are the 2D tracks and the green stick man is the 3D reconstruction obtained using our model. Row 3: our 3D reconstruction from a different viewpoint. B: tracking of a person running straight towards the camera. Notice the scale changes and possible forward-backward ambiguities in the 3D estimates. We train the LELVM using 180 datapoints (2.5 running cycles); 2D tracks were obtained by manually marking the video. In both A–B the mocap training data was for a person different from the video’s (with different body proportions and motions), and no ground-truth estimate was available for favourable initialisation. LELVM GPLVM PCA tracking in latent space tracking in latent space tracking in latent space 2.5 0.02 30 38 2 38 0.99 0.015 20 1.5 38 0.01 1 10 0.005 0.5 0 0 0 −0.005 −0.5 −10 −0.01 −1 −0.015 −1.5 −20 −0.02 −0.025 −0.025 −2 −0.02 −0.015 −0.01 −0.005 0 0.005 0.01 0.015 0.02 0.025 −2.5 −2 −1 0 1 2 3 −30 −80 −60 −40 −20 0 20 40 60 80 Figure 3: Method comparison, frame 38. PCA and GPLVM map consecutive walking cycles to spatially distinct latent space regions. Compounded by a data independent latent prior, the resulting tracker gets easily confused: it jumps across loops and/or remains put, trapped in local optima. In contrast, LELVM is stable and follows tightly a 1D manifold (see videos). crucial if locality constraints are correctly modeled (as in LELVM). We conclude that, compared to LELVM, GPLVM is signiﬁcantly less robust for tracking, has much higher training overhead and lacks some operations (e.g. computing latent conditionals based on partly missing ambient data). 7 5 Conclusion and future work We have proposed the use of priors based on the Laplacian Eigenmaps Latent Variable Model (LELVM) for people tracking. LELVM is a probabilistic dim. red. method that combines the advantages of latent variable models and spectral manifold learning algorithms: a multimodal probability density over latent and ambient variables, globally differentiable nonlinear mappings for reconstruction and dimensionality reduction, no local optima, ability to unfold highly nonlinear manifolds, and good practical scaling to latent spaces of high dimension. LELVM is computationally efﬁcient, simple to learn from sparse training data, and compatible with standard probabilistic trackers such as particle ﬁlters. Our results using a LELVM-based probabilistic sigma point mixture tracker with several real and synthetic human motion sequences show that LELVM provides sufﬁcient constraints for robust operation in the presence of missing, noisy and ambiguous image measurements. Comparisons with PCA and GPLVM show LELVM is superior in terms of accuracy, robustness and computation time. The objective of this paper was to demonstrate the ability of the LELVM prior in a simple setting using 2D tracks obtained automatically or manually, and single-type motions (running, walking). Future work will explore more complex observation models such as silhouettes; the combination of different motion types in the same latent space (whose dimension will exceed 2); and the exploration of multimodal posterior distributions in latent space caused by ambiguities. Acknowledgments This work was partially supported by NSF CAREER award IIS–0546857 (MACP), NSF IIS–0535140 and EC MCEXT–025481 (CS). CMU data: http://mocap.cs.cmu.edu (created with funding from NSF EIA–0196217). OSU data: http://accad.osu.edu/research/mocap/mocap data.htm. References ´ [1] M. A. Carreira-Perpi˜ an and Z. Lu. The Laplacian Eigenmaps Latent Variable Model. In AISTATS, 2007. n´ [2] N. R. Howe, M. E. Leventon, and W. T. Freeman. Bayesian reconstruction of 3D human motion from single-camera video. In NIPS, volume 12, pages 820–826, 2000. [3] T.-J. Cham and J. M. Rehg. A multiple hypothesis approach to ﬁgure tracking. In CVPR, 1999. [4] M. Brand. Shadow puppetry. In ICCV, pages 1237–1244, 1999. [5] H. Sidenbladh, M. J. Black, and L. Sigal. Implicit probabilistic models of human motion for synthesis and tracking. In ECCV, volume 1, pages 784–800, 2002. [6] C. Sminchisescu and A. Jepson. Generative modeling for continuous non-linearly embedded visual inference. In ICML, pages 759–766, 2004. [7] R. Urtasun, D. J. Fleet, A. Hertzmann, and P. Fua. Priors for people tracking from small training sets. In ICCV, pages 403–410, 2005. [8] R. Li, M.-H. Yang, S. Sclaroff, and T.-P. Tian. Monocular tracking of 3D human motion with a coordinated mixture of factor analyzers. In ECCV, volume 2, pages 137–150, 2006. [9] J. M. Wang, D. Fleet, and A. Hertzmann. Gaussian process dynamical models. In NIPS, volume 18, 2006. [10] R. Urtasun, D. J. Fleet, and P. Fua. Gaussian process dynamical models for 3D people tracking. In CVPR, pages 238–245, 2006. [11] G. W. Taylor, G. E. Hinton, and S. Roweis. Modeling human motion using binary latent variables. In NIPS, volume 19, 2007. [12] C. M. Bishop, M. Svens´ n, and C. K. I. Williams. GTM: The generative topographic mapping. Neural e Computation, 10(1):215–234, January 1998. [13] N. Lawrence. Probabilistic non-linear principal component analysis with Gaussian process latent variable models. Journal of Machine Learning Research, 6:1783–1816, November 2005. [14] M. Belkin and P. Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation, 15(6):1373–1396, June 2003. [15] B. W. Silverman. Density Estimation for Statistics and Data Analysis. Chapman & Hall, 1986. [16] R. van der Merwe and E. A. Wan. Gaussian mixture sigma-point particle ﬁlters for sequential probabilistic inference in dynamic state-space models. In ICASSP, volume 6, pages 701–704, 2003. ´ [17] M. A. Carreira-Perpi˜ an. Acceleration strategies for Gaussian mean-shift image segmentation. In CVPR, n´ pages 1160–1167, 2006. 8

3 0.4081215 138 nips-2007-Near-Maximum Entropy Models for Binary Neural Representations of Natural Images

Author: Matthias Bethge, Philipp Berens

Abstract: Maximum entropy analysis of binary variables provides an elegant way for studying the role of pairwise correlations in neural populations. Unfortunately, these approaches suffer from their poor scalability to high dimensions. In sensory coding, however, high-dimensional data is ubiquitous. Here, we introduce a new approach using a near-maximum entropy model, that makes this type of analysis feasible for very high-dimensional data—the model parameters can be derived in closed form and sampling is easy. Therefore, our NearMaxEnt approach can serve as a tool for testing predictions from a pairwise maximum entropy model not only for low-dimensional marginals, but also for high dimensional measurements of more than thousand units. We demonstrate its usefulness by studying natural images with dichotomized pixel intensities. Our results indicate that the statistics of such higher-dimensional measurements exhibit additional structure that are not predicted by pairwise correlations, despite the fact that pairwise correlations explain the lower-dimensional marginal statistics surprisingly well up to the limit of dimensionality where estimation of the full joint distribution is feasible. 1

4 0.40802813 93 nips-2007-GRIFT: A graphical model for inferring visual classification features from human data

Author: Michael Ross, Andrew Cohen

Abstract: This paper describes a new model for human visual classiﬁcation that enables the recovery of image features that explain human subjects’ performance on different visual classiﬁcation tasks. Unlike previous methods, this algorithm does not model their performance with a single linear classiﬁer operating on raw image pixels. Instead, it represents classiﬁcation as the combination of multiple feature detectors. This approach extracts more information about human visual classiﬁcation than previous methods and provides a foundation for further exploration. 1

5 0.4034009 168 nips-2007-Reinforcement Learning in Continuous Action Spaces through Sequential Monte Carlo Methods

Author: Alessandro Lazaric, Marcello Restelli, Andrea Bonarini

6 0.40284035 34 nips-2007-Bayesian Policy Learning with Trans-Dimensional MCMC

7 0.40227351 158 nips-2007-Probabilistic Matrix Factorization

8 0.40134975 179 nips-2007-SpAM: Sparse Additive Models

9 0.40105209 86 nips-2007-Exponential Family Predictive Representations of State

10 0.40020671 122 nips-2007-Locality and low-dimensions in the prediction of natural experience from fMRI

11 0.39984253 63 nips-2007-Convex Relaxations of Latent Variable Training

12 0.39972022 174 nips-2007-Selecting Observations against Adversarial Objectives

13 0.39925233 100 nips-2007-Hippocampal Contributions to Control: The Third Way

14 0.3988564 18 nips-2007-A probabilistic model for generating realistic lip movements from speech

15 0.39564377 94 nips-2007-Gaussian Process Models for Link Analysis and Transfer Learning

16 0.39519668 28 nips-2007-Augmented Functional Time Series Representation and Forecasting with Gaussian Processes

17 0.39503449 156 nips-2007-Predictive Matrix-Variate t Models

18 0.3941232 96 nips-2007-Heterogeneous Component Analysis

19 0.39373696 45 nips-2007-Classification via Minimum Incremental Coding Length (MICL)

20 0.39293277 79 nips-2007-Efficient multiple hyperparameter learning for log-linear models