cvpr cvpr2013 cvpr2013-306 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Lili Tao, Bogdan J. Matuszewski
Abstract: In this paper, a novel approach based on a non-linear manifold learning technique is proposed to recover 3D nonrigid structures from 2D image sequences captured by a single camera. Most ofthe existing approaches assume that 3D shapes can be accurately modelled in a linear subspace. These techniques perform well when the deformations are relatively small or simple, but fail when more complex deformations need to be recovered. The non-linear deformations are often observed in highly flexible objects for which the use of the linear model is impractical. A specific type of shape variations might be governed by only a small number of parameters, therefore can be wellrepresented in a low dimensional manifold. We learn a nonlinear shape prior using diffusion maps method. The key contribution in this paper is the introduction of the shape prior that constrain the reconstructed shapes to lie in the learned manifold. The proposed methodology has been validated quantitatively and qualitatively on 2D points sequences projected from the 3D motion capture data and real 2D video sequences. The comparisons oftheproposed man- ifold based method against several state-of-the-art techniques are shown on different types of deformable objects.
Reference: text
sentIndex sentText sentNum sentScore
1 uk lt Abstract In this paper, a novel approach based on a non-linear manifold learning technique is proposed to recover 3D nonrigid structures from 2D image sequences captured by a single camera. [sent-4, score-0.505]
2 Most ofthe existing approaches assume that 3D shapes can be accurately modelled in a linear subspace. [sent-5, score-0.309]
3 These techniques perform well when the deformations are relatively small or simple, but fail when more complex deformations need to be recovered. [sent-6, score-0.311]
4 We learn a nonlinear shape prior using diffusion maps method. [sent-9, score-0.495]
5 The key contribution in this paper is the introduction of the shape prior that constrain the reconstructed shapes to lie in the learned manifold. [sent-10, score-0.525]
6 The proposed methodology has been validated quantitatively and qualitatively on 2D points sequences projected from the 3D motion capture data and real 2D video sequences. [sent-11, score-0.226]
7 Introduction The objective of the Structure from Motion (SfM) is to jointly reconstruct 3D shapes and estimate corresponding camera motion trajectories based only on a set of observed image sequences. [sent-14, score-0.515]
8 While the reconstruction of rigid objects has been well-established [19] over the past two decades, deformable shape reconstruction is still challenging, mainly because it is a severely under-constrained problem. [sent-15, score-0.463]
9 [5] was the first to adopt the factorisation algorithm to deformable 3D structures by introducing a low rank shape model to represent deformable shapes. [sent-19, score-0.483]
10 As a time-varying object usually cannot arbitrarily deform, the idea of this model is to represent a deformable shape as a linear combination of basis shapes. [sent-20, score-0.499]
11 Due to its simplicity, shape basis model has been widely used to tackle the NRSfM [4, 26, 1]. [sent-21, score-0.345]
12 [25] proposed a closed-form solution and showed that orthonormality constraints is insufficient to provide unique solution to estimate basis shapes. [sent-26, score-0.239]
13 Departing from the shape basis model, a trajectory based algorithm was proposed in [2] by Akhter et al. [sent-31, score-0.529]
14 The main advantage of this representation is that the basis trajectories can be predefined, thus removing a large number of unknowns from the estimation. [sent-33, score-0.235]
15 However, there is no guarantee that the manifold is planar or isometric to a plane. [sent-39, score-0.397]
16 Despite the manifold learning techniques are becoming increasingly popular and have been successfully used in different applications including medical image analysis [24], object classification [14] and segmentation [9], these techniques have not been widely applied in NRSfM problem. [sent-40, score-0.34]
17 The main reason for their failure when recovering objects with large, complex deformations can be attributed to the reliance on a linear shape model. [sent-43, score-0.345]
18 This paper focuses on modelling non-linear deformable objects with large complex deformations, such as deformable cloth or articulated full-body motion. [sent-44, score-0.456]
19 In this case, the existing methods based on linear space manifold are no longer applicable. [sent-45, score-0.372]
20 We argue that the linear models require more parameters than our method based on non-linear manifold learning approach. [sent-46, score-0.372]
21 This paper proposes a novel method for reconstruction of 3D deformable structures exhibiting large and complex × deformations. [sent-47, score-0.257]
22 The proposed method is based on a recently introduced manifold learning technique called Diffusion Maps [6]. [sent-48, score-0.34]
23 This manifold build as a shape prior, with the reconstructed shapes constrained to lie in the manifold. [sent-49, score-0.83]
24 Learning instead a corresponding low dimensional manifold from the training examples. [sent-54, score-0.416]
25 Our approach is to integrate the learned non-linear shape prior manifold into the NRSfM solver. [sent-56, score-0.528]
26 The goal is to recover camera orientations matrix R and the concatenated time-varying shapes matrix S, based only on the 2D measurement in matrix W. [sent-66, score-0.434]
27 It is an under constrained problem since the shape and motion are both changing with time. [sent-67, score-0.284]
28 ytp]T the Low-rank shape model The points in each observed image can be represented as xt = RtSt, where xt represents input points, Rt is a 2×3 projection matrix representing camera orientation and 2S×t 3∈ pRro3×jecPt oisn a 3aDtri shape projected monertoa othriee ntttaht ofrna maned. [sent-70, score-0.552]
29 Describing the deformation using a shape model in a linear subspace is one way of imposing compactness on S to reduce the dimensionality of the problem. [sent-71, score-0.306]
30 A deformable 3D shape can be represented as a linear combination of K unknown but fixed basis shapes Bl : St=? [sent-72, score-0.776]
31 The measurement matrix can be decomposed and represented by pose, basis shapes and time varying coefficients matrices, therefore it can be rearranged as: W=⎣⎢⎡α F1 1. [sent-79, score-0.616]
32 K1− ⎥⎤⎦=MB (3) Since basis shapes B ∈ R3K×P, and M ∈ R2F×3K the Srainnkce eo fb maseisas suhreampeesn Bt Bm ∈atri Rx W is ,3 aKn adt mMos ∈t ∈in R the absence of noise. [sent-84, score-0.469]
33 According dtoe [i2ne5d], tphe to l aim aimtabtiiognu toyf mthaet cixlo Qsed ∈-f Rorm solution in this approach is that the motion matrix is nonlinear, when an inaccurate set of basis shapes have been chosen, it may not be possible to remove the affine ambiguity. [sent-87, score-0.6]
34 The trajectory for each point is approximated by a linear combination of a small number of basis trajectories Al: Tp=? [sent-89, score-0.451]
35 lK=1Alββ βpl (4) where ββpl are 1×3 coefficient vectors for the basis trajectory The basis trajectory can ebnet predefined tihne an object cintoderypendent way using Discrete Cosine Transform (DCT) basis and therefore avoid training process. [sent-90, score-1.012]
36 The model only needs to consider camera parameters and trajectory coefficients, thus requires less parameters than shape basis model (see Table 1). [sent-91, score-0.593]
37 Non-linear manifold model Our model departs from the linear shape model. [sent-92, score-0.525]
38 The shape basis B in the proposed method are selected from the learned shape manifold. [sent-93, score-0.498]
39 Unlike the low rank shape model, where all the reconstructed shapes are represented as a linear combination of unknown but fixed K basis shapes, in the proposed method, the basis shapes may be different at each frame. [sent-94, score-1.215]
40 Although it may seem to increase the number of parameters in the model, it should be recognised that all the basis shapes are selected from the manifold and are not estimated as a part of the optimisation process. [sent-95, score-0.907]
41 The parameters to be estimated in the proposed approach include only the camera motion and shape coefficients, representing the shape in the local linear barycentric coordinates system approximating the manifold at the location corresponding to the current estimate of St. [sent-96, score-0.974]
42 In most cases, K < 10, F, P > 100, the proposed model requires less parameters than low rank shape model and has a similar order of magnitude as trajectory model. [sent-98, score-0.369]
43 NRSfM with Diffusion Maps In this section, an overview of the proposed manifold based NRSfM algorithm is given first, followed by a short description of the diffusion maps including description of out-of-sample and pre-image problems. [sent-102, score-0.647]
44 As known from [25], enforcing only the rotation constraints cannot guarantee the unique solution for the camera motion and the basis shapes. [sent-103, score-0.387]
45 To solved this, the designed shape prior can help to attract a shape towards the manifold and therefore avoid incorrect reconstruction. [sent-104, score-0.681]
46 are estimated by running a few iteration of the optimisation process in batch NRSfM using linear basis shapes model [17]. [sent-108, score-0.599]
47 Intuitively, if the points in reduced space are relatively close, the corresponding shapes in high-dimensional space should represent similar shapes. [sent-110, score-0.375]
48 Based on this observation, the reconstructed shape at each frame can be represented as weighted sum of K+1 basis shapes from the learned manifold. [sent-111, score-0.682]
49 The coefficients of correspond basis shape are calculated based as barycentric coordinates of K+1 closest points in reduced space. [sent-112, score-0.646]
50 Once the basis shapes and their coefficients have been obtained, an optimisation is applied to minimise the image reprojection error with an additional smoothing term and basic rotation constraint over all frames. [sent-113, score-0.703]
51 Updating basis shapes in each iteration can help to circumvent the problem. [sent-115, score-0.469]
52 The basis shapes are being kept updated as long as 2D measurement error rt exceeds the defined threshold rT (10−3 in our case) or the error between two adjacent frames is relatively large which implies that the current results are unlikely to explain the shapes well. [sent-116, score-1.094]
53 While linear manifold method like PCA is straightforward, the recovered input data lies on a linear subspace of high dimensional space. [sent-120, score-0.443]
54 Diffusion maps is a graph based technique with isometric mapping from original shape space to reduced lowdimensional diffusion space. [sent-122, score-0.624]
55 1) Output: 3D deformable shapes S and camera motion R for each frame. [sent-124, score-0.594]
56 3) 6: Non-linear optimisation by minimising 2D measurement error and shape smooth term to obtain updated shapes St and camera motion Rt,t=1 · · · F . [sent-144, score-0.865]
57 G(iXve)n, a s ,eΨt of( shapes eXr1e e·X X· X· X∈M R ∈, MK ,? [sent-152, score-0.277]
58 re M is the manifold embedded in R·N··, Euc∈lid Mean, wdihsetarenc Me fo irs each pair of shapes ? [sent-154, score-0.617]
59 Out of sample extension In general, the diffusion map Ψ is only able to provide an embedding for the data which is given in the training set. [sent-189, score-0.367]
60 However, in the NRSfM problem, it is necessary to calculate embedding for shapes which are not presented in the training set. [sent-190, score-0.389]
61 Assuming we look for a shape St given by witist embedding xt, if this shape St does not exist in the training dataset, the exact pre-image might not be found in that case. [sent-211, score-0.418]
62 Inspired by this, we assume that the pre-image can be represented as a linear combination of its neighbours on the manifold selected from the training samples. [sent-214, score-0.463]
63 Since diffusion maps provides isometric mapping the data must keep the same structure when embedded into the reduced space and therefore the neighbours on the manifold correspond to the closest neighbours in the ? [sent-217, score-0.992]
64 Cost function The cost function to be minimised consists of the reprojection error, shape smoothing terms an rotation constraint. [sent-242, score-0.247]
65 To eliminate the effect, basis shapes are updated until the 2D measurement error is smaller than predefined threshold rT and the error between two adjacent frames is small enough. [sent-278, score-0.728]
66 Experimental Results The proposed methodology has been validated quantitatively and qualitatively on both motion capture and real data for different types of deformable object. [sent-280, score-0.253]
67 Diffusion maps requires training process, so training datasets for two face sequences are taken from the BU-3DFE [27] and for two surface sequences are obtained from [22]. [sent-290, score-0.316]
68 Since no separate training data are available in CMU database, half of each sequence is used for manifold learning and the other half for testing. [sent-291, score-0.377]
69 The influence of embedding dimensionality For the first set of experiments, we start with tests on motion capture data. [sent-296, score-0.261]
70 The accuracy of 3D shape reconstruction is affected by the dimensionality of the manifold representing prior information. [sent-297, score-0.677]
71 To find the relationship between manifold dimensionality and the reconstruction error, experiments have been carried out with all the test sequences and dimensionality changing between 3 and 10. [sent-298, score-0.639]
72 To simplify visualization of results, all the 12 sequences are separated into 3 groups, those are: small deformation sequences (surprise, talking, cardboard), large deformation sequences (cloth, walking, pick-up, yoga, drink, stretch) and all the dance sequences. [sent-299, score-0.538]
73 tF=1(Δtx+Δty+Δtz) (10) where Δtx,Δty,Δtz are the standard deviations of x,y and z coordinates of ground truth shape at tth frame and etp is the Euclidean distance between corresponding point p at frame t in the reconstructed and ground truth shapes. [sent-303, score-0.247]
74 2 shows the means of reconstruction error for each group and the overall average results when different manifold dimensions K are used. [sent-305, score-0.516]
75 As expected, in general, increasing the number of manifold dimensions decreases the error. [sent-306, score-0.34]
76 This is especially true for the group of dance sequences and the group representing large deformations. [sent-307, score-0.282]
77 This does make sense as only a small number of basis shapes is required to describe the data variability containing only relatively small number of degrees of freedom. [sent-310, score-0.501]
78 Bars left to right: Group of small deformation sequences, group of large deformation sequences, group of all dance sequences, all the sequences. [sent-312, score-0.319]
79 Sensitivity to noise In most cases, inaccurate 2D measurement caused by feature tracking/detection error may lead to shape reconstruction failure for most previously proposed approaches, as those are very sensitive to noise. [sent-315, score-0.434]
80 The measurement W was perturbed by Gaussian noise according to the standard deviation of the measurement data with given level of noise. [sent-318, score-0.231]
81 Even with the noise presented in the measurements, reasonably accurate shapes are still obtainable, showing that manifold based method can produce results that are better than those obtained by previously proposed methods. [sent-328, score-0.662]
82 Table 2 summarises the results showing 3D reconstruction errors of each method and each sequence, together with the optimal number of bases for which minimal reconstruction error on the test data is obtained. [sent-332, score-0.237]
83 6543210KPMCDSTFAM2Le4vlof6Ni8se102 (a) (b) Figure 3: Reconstruction error as function of the measurement noise for the walking data. [sent-335, score-0.235]
84 (a) Our method with varying number of manifold dimension K; (b) Our method evaluated against four other methods. [sent-336, score-0.34]
85 The best result for DM method is chosen by changing manifold dimension K from 3 to 10. [sent-338, score-0.34]
86 Considering the ambiguity of estimated camera motion [2], the shapes are aligned using a single global rotation based on Procrustes alignment method. [sent-339, score-0.472]
87 As shown on the Table 2, trajectory based methods PTA, CSF and KSFM are able to provide comparable results, to the proposed method on objects with small deformations (e. [sent-340, score-0.303]
88 But those methods provide relatively large error on highly non-rigid human motion sequences (e. [sent-345, score-0.307]
89 Note that although the initial shapes of our method may not belong to the manifold M, after optimisation process, the results demonstrate good convergence msinisceat tiohen 3prDo errors are relatively somnsaltlr. [sent-350, score-0.747]
90 4 shows a comparison of our reconstructed shapes with the results obtained from MP, PTA, KSFM methods. [sent-360, score-0.337]
91 Comments and Future work The paper presented a new approach to integrate the idea from non-linear manifold learning techniques into the NRSfM framework for the task of reconstructing complex and highly deformable shapes. [sent-362, score-0.503]
92 The diffusion maps have been introduced in order to build non-linear shape prior manifold. [sent-363, score-0.495]
93 This approach significantly improved the reconstruction quality and is well-adapted to large deformation of complex objects, especially for non-rigid articulated body movement, which cannot be accurately represented in a linear subspace. [sent-364, score-0.342]
94 As we only use limited number of shapes in training pro- cess, to overcome this, the future work would include collecting and generating data for building a sufficiently dense representation of the manifold to further improve the performance. [sent-367, score-0.654]
95 As manifold learning has shown to be a very powerful approach for analysis of the shapes, we believe the manifold based method is a suitable groundwork for reconstruction of deformable shapes. [sent-368, score-0.896]
96 A factorization approach to structure from motion with shape priors. [sent-431, score-0.365]
97 Computing smooth timetrajectories for camera and deformable shape in structure from motion with occlusion. [sent-449, score-0.512]
98 Nonrigid structure-from-motion: Estimating shape and motion with hierarchical priors. [sent-509, score-0.284]
99 A closed-form solution to non-rigid shape and motion recovery. [sent-542, score-0.284]
100 A factorization-based approach for articulated nonrigid shape, motion and kinematic chain recovery from video. [sent-547, score-0.31]
wordName wordTfidf (topN-words)
[('manifold', 0.34), ('nrsfm', 0.299), ('shapes', 0.277), ('diffusion', 0.255), ('basis', 0.192), ('trajectory', 0.184), ('shape', 0.153), ('ksfm', 0.152), ('pta', 0.15), ('motion', 0.131), ('deformable', 0.122), ('dance', 0.121), ('deformations', 0.119), ('articulated', 0.109), ('cardboard', 0.108), ('csf', 0.1), ('optimisation', 0.098), ('st', 0.098), ('sequences', 0.095), ('reconstruction', 0.094), ('measurement', 0.093), ('matuszewski', 0.091), ('tlbtl', 0.091), ('xt', 0.091), ('rt', 0.088), ('embedding', 0.075), ('akhter', 0.071), ('nonrigid', 0.07), ('barycentric', 0.067), ('deformation', 0.066), ('reduced', 0.066), ('mp', 0.064), ('camera', 0.064), ('cloth', 0.062), ('minimised', 0.061), ('rtst', 0.061), ('varol', 0.061), ('reconstructed', 0.06), ('tl', 0.059), ('isometric', 0.057), ('xj', 0.057), ('lk', 0.056), ('dm', 0.056), ('dimensionality', 0.055), ('neighbours', 0.054), ('coefficients', 0.054), ('btl', 0.054), ('xtp', 0.054), ('factorisation', 0.054), ('stretch', 0.054), ('bedding', 0.054), ('yoga', 0.054), ('maps', 0.052), ('del', 0.05), ('gotardo', 0.05), ('error', 0.049), ('calculated', 0.049), ('sfm', 0.048), ('walking', 0.048), ('adjacency', 0.047), ('rot', 0.047), ('orthonormality', 0.047), ('noise', 0.045), ('drink', 0.045), ('nystr', 0.045), ('trajectories', 0.043), ('talking', 0.043), ('optimising', 0.043), ('bregler', 0.043), ('initialisation', 0.043), ('coifman', 0.043), ('tz', 0.043), ('arias', 0.043), ('cmu', 0.042), ('structure', 0.042), ('bue', 0.042), ('mapping', 0.041), ('complex', 0.041), ('salzmann', 0.04), ('xi', 0.039), ('factorization', 0.039), ('dimensional', 0.039), ('frames', 0.037), ('training', 0.037), ('surprise', 0.035), ('sensitivity', 0.035), ('dct', 0.035), ('sheikh', 0.035), ('prior', 0.035), ('torresani', 0.034), ('coordinates', 0.034), ('facial', 0.033), ('group', 0.033), ('reprojection', 0.033), ('linear', 0.032), ('rank', 0.032), ('relatively', 0.032), ('predefined', 0.031), ('duality', 0.031), ('closest', 0.031)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000011 306 cvpr-2013-Non-rigid Structure from Motion with Diffusion Maps Prior
Author: Lili Tao, Bogdan J. Matuszewski
Abstract: In this paper, a novel approach based on a non-linear manifold learning technique is proposed to recover 3D nonrigid structures from 2D image sequences captured by a single camera. Most ofthe existing approaches assume that 3D shapes can be accurately modelled in a linear subspace. These techniques perform well when the deformations are relatively small or simple, but fail when more complex deformations need to be recovered. The non-linear deformations are often observed in highly flexible objects for which the use of the linear model is impractical. A specific type of shape variations might be governed by only a small number of parameters, therefore can be wellrepresented in a low dimensional manifold. We learn a nonlinear shape prior using diffusion maps method. The key contribution in this paper is the introduction of the shape prior that constrain the reconstructed shapes to lie in the learned manifold. The proposed methodology has been validated quantitatively and qualitatively on 2D points sequences projected from the 3D motion capture data and real 2D video sequences. The comparisons oftheproposed man- ifold based method against several state-of-the-art techniques are shown on different types of deformable objects.
2 0.38885772 113 cvpr-2013-Dense Variational Reconstruction of Non-rigid Surfaces from Monocular Video
Author: Ravi Garg, Anastasios Roussos, Lourdes Agapito
Abstract: This paper offers the first variational approach to the problem of dense 3D reconstruction of non-rigid surfaces from a monocular video sequence. We formulate nonrigid structure from motion (NRSfM) as a global variational energy minimization problem to estimate dense low-rank smooth 3D shapes for every frame along with the camera motion matrices, given dense 2D correspondences. Unlike traditional factorization based approaches to NRSfM, which model the low-rank non-rigid shape using a fixed number of basis shapes and corresponding coefficients, we minimize the rank of the matrix of time-varying shapes directly via trace norm minimization. In conjunction with this low-rank constraint, we use an edge preserving total-variation regularization term to obtain spatially smooth shapes for every frame. Thanks to proximal splitting techniques the optimization problem can be decomposed into many point-wise sub-problems and simple linear systems which can be easily solved on GPU hardware. We show results on real sequences of different objects (face, torso, beating heart) where, despite challenges in tracking, illumination changes and occlusions, our method reconstructs highly deforming smooth surfaces densely and accurately directly from video, without the need for any prior models or shape templates.
3 0.23609799 341 cvpr-2013-Procrustean Normal Distribution for Non-rigid Structure from Motion
Author: Minsik Lee, Jungchan Cho, Chong-Ho Choi, Songhwai Oh
Abstract: Non-rigid structure from motion is a fundamental problem in computer vision, which is yet to be solved satisfactorily. The main difficulty of the problem lies in choosing the right constraints for the solution. In this paper, we propose new constraints that are more effective for non-rigid shape recovery. Unlike the other proposals which have mainly focused on restricting the deformation space using rank constraints, our proposal constrains the motion parameters so that the 3D shapes are most closely aligned to each other, which makes the rank constraints unnecessary. Based on these constraints, we define a new class ofprobability distribution called the Procrustean normal distribution and propose a new NRSfM algorithm, EM-PND. The experimental results show that the proposed method outperforms the existing methods, and it works well even if there is no temporal dependence between the observed samples.
4 0.22429971 126 cvpr-2013-Diffusion Processes for Retrieval Revisited
Author: Michael Donoser, Horst Bischof
Abstract: In this paper we revisit diffusion processes on affinity graphs for capturing the intrinsic manifold structure defined by pairwise affinity matrices. Such diffusion processes have already proved the ability to significantly improve subsequent applications like retrieval. We give a thorough overview of the state-of-the-art in this field and discuss obvious similarities and differences. Based on our observations, we are then able to derive a generic framework for diffusion processes in the scope of retrieval applications, where the related work represents specific instances of our generic formulation. We evaluate our framework on several retrieval tasks and are able to derive algorithms that e. g. achieve a 100% bullseye score on the popular MPEG7 shape retrieval data set.
5 0.19138904 259 cvpr-2013-Learning a Manifold as an Atlas
Author: Nikolaos Pitelis, Chris Russell, Lourdes Agapito
Abstract: In this work, we return to the underlying mathematical definition of a manifold and directly characterise learning a manifold as finding an atlas, or a set of overlapping charts, that accurately describe local structure. We formulate the problem of learning the manifold as an optimisation that simultaneously refines the continuous parameters defining the charts, and the discrete assignment of points to charts. In contrast to existing methods, this direct formulation of a manifold does not require “unwrapping ” the manifold into a lower dimensional space and allows us to learn closed manifolds of interest to vision, such as those corresponding to gait cycles or camera pose. We report state-ofthe-art results for manifold based nearest neighbour classification on vision datasets, and show how the same techniques can be applied to the 3D reconstruction of human motion from a single image.
6 0.16922268 237 cvpr-2013-Kernel Learning for Extrinsic Classification of Manifold Features
7 0.1687396 405 cvpr-2013-Sparse Subspace Denoising for Image Manifolds
8 0.16289502 170 cvpr-2013-Fast Rigid Motion Segmentation via Incrementally-Complex Local Models
9 0.16282849 203 cvpr-2013-Hierarchical Video Representation with Trajectory Binary Partition Tree
10 0.14365809 33 cvpr-2013-Active Contours with Group Similarity
12 0.14218423 91 cvpr-2013-Consensus of k-NNs for Robust Neighborhood Selection on Graph-Based Manifolds
13 0.13407792 433 cvpr-2013-Top-Down Segmentation of Non-rigid Visual Objects Using Derivative-Based Search on Sparse Manifolds
14 0.13169894 178 cvpr-2013-From Local Similarity to Global Coding: An Application to Image Classification
15 0.13092805 367 cvpr-2013-Rolling Riemannian Manifolds to Solve the Multi-class Classification Problem
16 0.12852713 46 cvpr-2013-Articulated and Restricted Motion Subspaces and Their Signatures
17 0.1268708 276 cvpr-2013-MKPLS: Manifold Kernel Partial Least Squares for Lipreading and Speaker Identification
18 0.12313238 105 cvpr-2013-Deep Learning Shape Priors for Object Segmentation
19 0.11696918 223 cvpr-2013-Inductive Hashing on Manifolds
20 0.11491308 238 cvpr-2013-Kernel Methods on the Riemannian Manifold of Symmetric Positive Definite Matrices
topicId topicWeight
[(0, 0.246), (1, 0.116), (2, -0.052), (3, 0.012), (4, -0.021), (5, -0.071), (6, 0.001), (7, -0.207), (8, 0.001), (9, -0.028), (10, 0.065), (11, 0.1), (12, -0.203), (13, -0.128), (14, 0.087), (15, 0.018), (16, -0.117), (17, 0.058), (18, -0.191), (19, 0.061), (20, 0.047), (21, 0.038), (22, 0.083), (23, 0.04), (24, 0.07), (25, 0.082), (26, 0.05), (27, 0.029), (28, -0.002), (29, -0.022), (30, 0.033), (31, -0.204), (32, -0.176), (33, -0.035), (34, -0.01), (35, 0.025), (36, -0.034), (37, -0.052), (38, -0.033), (39, -0.13), (40, 0.112), (41, 0.061), (42, -0.008), (43, -0.01), (44, -0.059), (45, -0.082), (46, -0.079), (47, 0.178), (48, 0.007), (49, 0.009)]
simIndex simValue paperId paperTitle
same-paper 1 0.95801169 306 cvpr-2013-Non-rigid Structure from Motion with Diffusion Maps Prior
Author: Lili Tao, Bogdan J. Matuszewski
Abstract: In this paper, a novel approach based on a non-linear manifold learning technique is proposed to recover 3D nonrigid structures from 2D image sequences captured by a single camera. Most ofthe existing approaches assume that 3D shapes can be accurately modelled in a linear subspace. These techniques perform well when the deformations are relatively small or simple, but fail when more complex deformations need to be recovered. The non-linear deformations are often observed in highly flexible objects for which the use of the linear model is impractical. A specific type of shape variations might be governed by only a small number of parameters, therefore can be wellrepresented in a low dimensional manifold. We learn a nonlinear shape prior using diffusion maps method. The key contribution in this paper is the introduction of the shape prior that constrain the reconstructed shapes to lie in the learned manifold. The proposed methodology has been validated quantitatively and qualitatively on 2D points sequences projected from the 3D motion capture data and real 2D video sequences. The comparisons oftheproposed man- ifold based method against several state-of-the-art techniques are shown on different types of deformable objects.
2 0.78630513 113 cvpr-2013-Dense Variational Reconstruction of Non-rigid Surfaces from Monocular Video
Author: Ravi Garg, Anastasios Roussos, Lourdes Agapito
Abstract: This paper offers the first variational approach to the problem of dense 3D reconstruction of non-rigid surfaces from a monocular video sequence. We formulate nonrigid structure from motion (NRSfM) as a global variational energy minimization problem to estimate dense low-rank smooth 3D shapes for every frame along with the camera motion matrices, given dense 2D correspondences. Unlike traditional factorization based approaches to NRSfM, which model the low-rank non-rigid shape using a fixed number of basis shapes and corresponding coefficients, we minimize the rank of the matrix of time-varying shapes directly via trace norm minimization. In conjunction with this low-rank constraint, we use an edge preserving total-variation regularization term to obtain spatially smooth shapes for every frame. Thanks to proximal splitting techniques the optimization problem can be decomposed into many point-wise sub-problems and simple linear systems which can be easily solved on GPU hardware. We show results on real sequences of different objects (face, torso, beating heart) where, despite challenges in tracking, illumination changes and occlusions, our method reconstructs highly deforming smooth surfaces densely and accurately directly from video, without the need for any prior models or shape templates.
3 0.77910018 341 cvpr-2013-Procrustean Normal Distribution for Non-rigid Structure from Motion
Author: Minsik Lee, Jungchan Cho, Chong-Ho Choi, Songhwai Oh
Abstract: Non-rigid structure from motion is a fundamental problem in computer vision, which is yet to be solved satisfactorily. The main difficulty of the problem lies in choosing the right constraints for the solution. In this paper, we propose new constraints that are more effective for non-rigid shape recovery. Unlike the other proposals which have mainly focused on restricting the deformation space using rank constraints, our proposal constrains the motion parameters so that the 3D shapes are most closely aligned to each other, which makes the rank constraints unnecessary. Based on these constraints, we define a new class ofprobability distribution called the Procrustean normal distribution and propose a new NRSfM algorithm, EM-PND. The experimental results show that the proposed method outperforms the existing methods, and it works well even if there is no temporal dependence between the observed samples.
4 0.69525236 259 cvpr-2013-Learning a Manifold as an Atlas
Author: Nikolaos Pitelis, Chris Russell, Lourdes Agapito
Abstract: In this work, we return to the underlying mathematical definition of a manifold and directly characterise learning a manifold as finding an atlas, or a set of overlapping charts, that accurately describe local structure. We formulate the problem of learning the manifold as an optimisation that simultaneously refines the continuous parameters defining the charts, and the discrete assignment of points to charts. In contrast to existing methods, this direct formulation of a manifold does not require “unwrapping ” the manifold into a lower dimensional space and allows us to learn closed manifolds of interest to vision, such as those corresponding to gait cycles or camera pose. We report state-ofthe-art results for manifold based nearest neighbour classification on vision datasets, and show how the same techniques can be applied to the 3D reconstruction of human motion from a single image.
5 0.61494589 433 cvpr-2013-Top-Down Segmentation of Non-rigid Visual Objects Using Derivative-Based Search on Sparse Manifolds
Author: Jacinto C. Nascimento, Gustavo Carneiro
Abstract: The solution for the top-down segmentation of non-rigid visual objects using machine learning techniques is generally regarded as too complex to be solved in its full generality given the large dimensionality of the search space of the explicit representation ofthe segmentation contour. In order to reduce this complexity, theproblem is usually divided into two stages: rigid detection and non-rigid segmentation. The rationale is based on the fact that the rigid detection can be run in a lower dimensionality space (i.e., less complex and faster) than the original contour space, and its result is then used to constrain the non-rigid segmentation. In this paper, we propose the use of sparse manifolds to reduce the dimensionality of the rigid detection search space of current stateof-the-art top-down segmentation methodologies. The main goals targeted by this smaller dimensionality search space are the decrease of the search running time complexity and the reduction of the training complexity of the rigid detec- tor. These goals are attainable given that both the search and training complexities are function of the dimensionality of the rigid search space. We test our approach in the segmentation of the left ventricle from ultrasound images and lips from frontal face images. Compared to the performance of state-of-the-art non-rigid segmentation system, our experiments show that the use of sparse manifolds for the rigid detection leads to the two goals mentioned above.
6 0.60956097 276 cvpr-2013-MKPLS: Manifold Kernel Partial Least Squares for Lipreading and Speaker Identification
7 0.59214467 33 cvpr-2013-Active Contours with Group Similarity
8 0.5751788 126 cvpr-2013-Diffusion Processes for Retrieval Revisited
9 0.56757259 105 cvpr-2013-Deep Learning Shape Priors for Object Segmentation
10 0.56436825 170 cvpr-2013-Fast Rigid Motion Segmentation via Incrementally-Complex Local Models
11 0.55227607 367 cvpr-2013-Rolling Riemannian Manifolds to Solve the Multi-class Classification Problem
12 0.55109996 110 cvpr-2013-Dense Object Reconstruction with Semantic Priors
13 0.54137516 46 cvpr-2013-Articulated and Restricted Motion Subspaces and Their Signatures
14 0.53625107 321 cvpr-2013-PDM-ENLOR: Learning Ensemble of Local PDM-Based Regressions
15 0.52702075 358 cvpr-2013-Robust Canonical Time Warping for the Alignment of Grossly Corrupted Sequences
16 0.51582104 91 cvpr-2013-Consensus of k-NNs for Robust Neighborhood Selection on Graph-Based Manifolds
17 0.49384949 191 cvpr-2013-Graph-Laplacian PCA: Closed-Form Solution and Robustness
18 0.48724177 405 cvpr-2013-Sparse Subspace Denoising for Image Manifolds
19 0.47568151 301 cvpr-2013-Multi-target Tracking by Rank-1 Tensor Approximation
20 0.46893239 426 cvpr-2013-Tensor-Based Human Body Modeling
topicId topicWeight
[(10, 0.105), (16, 0.021), (26, 0.044), (33, 0.427), (67, 0.042), (69, 0.04), (87, 0.069), (90, 0.17)]
simIndex simValue paperId paperTitle
1 0.98970985 409 cvpr-2013-Spectral Modeling and Relighting of Reflective-Fluorescent Scenes
Author: Antony Lam, Imari Sato
Abstract: Hyperspectral reflectance data allows for highly accurate spectral relighting under arbitrary illumination, which is invaluable to applications ranging from archiving cultural e-heritage to consumer product design. Past methods for capturing the spectral reflectance of scenes has proven successful in relighting but they all share a common assumption. All the methods do not consider the effects of fluorescence despite fluorescence being found in many everyday objects. In this paper, we describe the very different ways that reflectance and fluorescence interact with illuminants and show the need to explicitly consider fluorescence in the relighting problem. We then propose a robust method based on well established theories of reflectance and fluorescence for imaging each of these components. Finally, we show that we can relight real scenes of reflective-fluorescent surfaces with much higher accuracy in comparison to only considering the reflective component.
2 0.96616548 106 cvpr-2013-Deformable Graph Matching
Author: Feng Zhou, Fernando De_la_Torre
Abstract: Graph matching (GM) is a fundamental problem in computer science, and it has been successfully applied to many problems in computer vision. Although widely used, existing GM algorithms cannot incorporate global consistence among nodes, which is a natural constraint in computer vision problems. This paper proposes deformable graph matching (DGM), an extension of GM for matching graphs subject to global rigid and non-rigid geometric constraints. The key idea of this work is a new factorization of the pair-wise affinity matrix. This factorization decouples the affinity matrix into the local structure of each graph and the pair-wise affinity edges. Besides the ability to incorporate global geometric transformations, this factorization offers three more benefits. First, there is no need to compute the costly (in space and time) pair-wise affinity matrix. Second, it provides a unified view of many GM methods and extends the standard iterative closest point algorithm. Third, it allows to use the path-following optimization algorithm that leads to improved optimization strategies and matching performance. Experimental results on synthetic and real databases illustrate how DGM outperforms state-of-the-art algorithms for GM. The code is available at http : / / human s en s ing . c s . cmu .edu / fgm.
3 0.96543586 191 cvpr-2013-Graph-Laplacian PCA: Closed-Form Solution and Robustness
Author: Bo Jiang, Chris Ding, Bio Luo, Jin Tang
Abstract: Principal Component Analysis (PCA) is a widely used to learn a low-dimensional representation. In many applications, both vector data X and graph data W are available. Laplacian embedding is widely used for embedding graph data. Wepropose a graph-Laplacian PCA (gLPCA) to learn a low dimensional representation of X that incorporates graph structures encoded in W. This model has several advantages: (1) It is a data representation model. (2) It has a compact closed-form solution and can be efficiently computed. (3) It is capable to remove corruptions. Extensive experiments on 8 datasets show promising results on image reconstruction and significant improvement on clustering and classification.
same-paper 4 0.95203614 306 cvpr-2013-Non-rigid Structure from Motion with Diffusion Maps Prior
Author: Lili Tao, Bogdan J. Matuszewski
Abstract: In this paper, a novel approach based on a non-linear manifold learning technique is proposed to recover 3D nonrigid structures from 2D image sequences captured by a single camera. Most ofthe existing approaches assume that 3D shapes can be accurately modelled in a linear subspace. These techniques perform well when the deformations are relatively small or simple, but fail when more complex deformations need to be recovered. The non-linear deformations are often observed in highly flexible objects for which the use of the linear model is impractical. A specific type of shape variations might be governed by only a small number of parameters, therefore can be wellrepresented in a low dimensional manifold. We learn a nonlinear shape prior using diffusion maps method. The key contribution in this paper is the introduction of the shape prior that constrain the reconstructed shapes to lie in the learned manifold. The proposed methodology has been validated quantitatively and qualitatively on 2D points sequences projected from the 3D motion capture data and real 2D video sequences. The comparisons oftheproposed man- ifold based method against several state-of-the-art techniques are shown on different types of deformable objects.
5 0.93390656 250 cvpr-2013-Learning Cross-Domain Information Transfer for Location Recognition and Clustering
Author: Raghuraman Gopalan
Abstract: Estimating geographic location from images is a challenging problem that is receiving recent attention. In contrast to many existing methods that primarily model discriminative information corresponding to different locations, we propose joint learning of information that images across locations share and vary upon. Starting with generative and discriminative subspaces pertaining to domains, which are obtained by a hierarchical grouping of images from adjacent locations, we present a top-down approach that first models cross-domain information transfer by utilizing the geometry ofthese subspaces, and then encodes the model results onto individual images to infer their location. We report competitive results for location recognition and clustering on two public datasets, im2GPS and San Francisco, and empirically validate the utility of various design choices involved in the approach.
6 0.93388408 203 cvpr-2013-Hierarchical Video Representation with Trajectory Binary Partition Tree
7 0.9338398 46 cvpr-2013-Articulated and Restricted Motion Subspaces and Their Signatures
8 0.9331491 370 cvpr-2013-SCALPEL: Segmentation Cascades with Localized Priors and Efficient Learning
9 0.93311375 340 cvpr-2013-Probabilistic Label Trees for Efficient Large Scale Image Classification
10 0.93302089 355 cvpr-2013-Representing Videos Using Mid-level Discriminative Patches
11 0.93295234 296 cvpr-2013-Multi-level Discriminative Dictionary Learning towards Hierarchical Visual Categorization
12 0.9327451 309 cvpr-2013-Nonparametric Scene Parsing with Adaptive Feature Relevance and Semantic Context
13 0.93235958 333 cvpr-2013-Plane-Based Content Preserving Warps for Video Stabilization
14 0.9321838 304 cvpr-2013-Multipath Sparse Coding Using Hierarchical Matching Pursuit
15 0.93211919 284 cvpr-2013-Mesh Based Semantic Modelling for Indoor and Outdoor Scenes
16 0.93209398 20 cvpr-2013-A New Model and Simple Algorithms for Multi-label Mumford-Shah Problems
17 0.93206275 390 cvpr-2013-Semi-supervised Node Splitting for Random Forest Construction
18 0.93202072 450 cvpr-2013-Unsupervised Joint Object Discovery and Segmentation in Internet Images
19 0.93195713 300 cvpr-2013-Multi-target Tracking by Lagrangian Relaxation to Min-cost Network Flow
20 0.93193203 268 cvpr-2013-Leveraging Structure from Motion to Learn Discriminative Codebooks for Scalable Landmark Classification