cvpr cvpr2013 cvpr2013-113 cvpr2013-113-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Ravi Garg, Anastasios Roussos, Lourdes Agapito
Abstract: This paper offers the first variational approach to the problem of dense 3D reconstruction of non-rigid surfaces from a monocular video sequence. We formulate nonrigid structure from motion (NRSfM) as a global variational energy minimization problem to estimate dense low-rank smooth 3D shapes for every frame along with the camera motion matrices, given dense 2D correspondences. Unlike traditional factorization based approaches to NRSfM, which model the low-rank non-rigid shape using a fixed number of basis shapes and corresponding coefficients, we minimize the rank of the matrix of time-varying shapes directly via trace norm minimization. In conjunction with this low-rank constraint, we use an edge preserving total-variation regularization term to obtain spatially smooth shapes for every frame. Thanks to proximal splitting techniques the optimization problem can be decomposed into many point-wise sub-problems and simple linear systems which can be easily solved on GPU hardware. We show results on real sequences of different objects (face, torso, beating heart) where, despite challenges in tracking, illumination changes and occlusions, our method reconstructs highly deforming smooth surfaces densely and accurately directly from video, without the need for any prior models or shape templates.
[1] I. Akhter, Y. Sheikh, and S. Khan. In defense oforthonormality constraints for nonrigid structure from motion. In CVPR, 2009. 1, 2
[2] I. Akhter, Y. Sheikh, S. Khan, and T. Kanade. Trajectory space: A dual representation for nonrigid structure from motion. PAMI, 2011. 2, 5, 6
[3] R. Angst, C. Zach, and M. Pollefeys. The generalized tracenorm and its application to structure-from-motion problems. In ICCV, 2011. 2
[4] A. Bartoli, V. Gay-Bellile, U. Castellani, J. Peyras, S. Olsen, and P. Sayd. Coarse-to-fine low-rank structure-from-motion. In CVPR, 2008. 1, 2
[5] A. Bartoli, Y. Gerard, F. Chadebecq, and T. Collins. On template-based reconstruction from a single view: Analytical solutions and proofs of well-posedness for developable, isometric and conformal surfaces. In CVPR, 2012. 2
[6] M. Brand. A direct method for 3D factorization of nonrigid motion observed in 2D. In CVPR, 2005. 2
[7] C. Bregler, A. Hertzmann, and H. Biermann. Recovering non-rigid 3D shape from image streams. In CVPR, 2000. 1, 2, 4
[8] E. J. Cand e`s. The power of convex relaxation: The surprising stories of matrix completion and compressed sensing. In SODA, 2010. 2 11111222227777768866 (a) (b) Fig(cu)re4.3Dreconstrucionresult forthebacksequnce.(a)In- put images. (b) 3D reconstruction of the deformed surface. (c) Textured rendering of the result (with an additional light source). (a) (b) (Fc)igure5.3Dreconstructionresultsfortheheartsequence.(a)In- put images. (b) Front view of the estimation of the deforming and rotating surface. (c) Textured rendering of the reconstruction from a side view, using in all frames the texture of the reference image.
[9] A. Chambolle and T. Pock. A first-order primal-dual algorithm for convex problems with applications to imaging. JMIV, 2011. 4
[10] T. Collins and A. Bartoli. Locally affine and planar deformable surface reconstruction from video. VMV, 2010. 1, 2
[11] P. Combettes and J. Pesquet. Proximal splitting methods in signal processing. Fixed-Point Algorithms for Inverse Problems in Science and Engineering, 2011. 4
[12] Y. Dai, H. Li, and M. He. A simple prior-free method for non rigid structure from motion factorization. In CVPR, 2012. 1, 2, 4
[13] A. Del Bue. A factorization approach to structure from motion with shape priors. In CVPR, 2008. 2
[14] Y. Furukawa and J. Ponce. Accurate, dense, and robust multiview stereopsis. PAMI, 2010. 1
[15] R. Garg, A. Roussos, and L. Agapito. A variational approach to video registration with subspace constraints. IJCV, 2013. 1, 2, 3, 4, 6
[16] P. Gotardo and A. Martinez. Computing smooth time-
[17]
[18]
[19]
[20]
[21]
[22]
[23]
[24]
[25]
[26] trajectories for camera and deformable shape in structure from motion with occlusion. PAMI, 2011. 2 A. Handa, R. Newcombe, A. Angeli, and A. Davison. Applications of Legendre-Fenchel transformation to computer vision problems. Technical Report DTR1 1-7, Imperial College, 2011. 4 R. Mazumder, T. Hastie, and R. Tibshirani. Spectral regularization algorithms for learning large incomplete matrices. JMLR, 2010. 5 R. Newcombe, S. Lovegrove, and A. Davison. DTAM: Dense tracking and mapping in real-time. In ICCV, 2011. 1, 2, 4 M. Paladini, A. Del Bue, J. Xavier, L. Agapito, M. Stosic, and M. Dodig. Optimal metric projections for deformable and articulated structure-from-motion. IJCV, 2012. 1, 2, 4, 5, 6 H. S. Park, T. Shiratori, I. Matthews, and Y. Sheikh. 3D reconstruction of a moving point from a series of 2D projections. In ECCV, 2010. 2, 6 D. Pizarro and A. Bartoli. Feature-based deformable surface detection with self-occlusion reasoning. In 3DPVT, 2010. 2 S. Ricco and C. Tomasi. Dense lagrangian motion estimation with occlusions. In CVPR, 2012. 2, 3 A. Roussos, C. Russell, R. Garg, and L. Agapito. Dense multibody motion estimation and reconstruction from a handheld camera. In ISMAR, 2012. 1, 2 L. Rudin, S. Osher, and E. Fatemi. Nonlinear total variation based noise removal algorithms. Physica D, 1992. 4 C. Russell, J. Fayad, and L. Agapito. Energy based multiple model fitting for non-rigid structure from motion. In CVPR, 2011. 1, 6
[27] C. Russell, J. Fayad, and L. Agapito. Dense non-rigid structure from motion. In 3DIMPVT, 2012. 2
[28] S. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski. A comparison and evaluation of multi-view stereo reconstruction algorithms. In CVPR, 2006. 1
[29] D. Stoyanov. Stereoscopic scene flow for robotic assisted minimally invasive surgery. In MICCAI, 2012. 6
[30] J. Taylor, A. D. Jepson, and K. N. Kutulakos. Non-rigid structure from locally-rigid motion. In CVPR, 2010. 1 [3 1] C. Tomasi and T. Kanade. Shape and motion from image streams under orthography: A factorization approach. IJCV, 1992. 2, 5
[32] L. Torresani, A. Hertzmann, and C. Bregler. Non-rigid structure-from-motion: Estimating shape and motion with hierarchical priors. PAMI, 2008. 1, 2, 4
[33] D. Vlasic, M. Brand, H. Pfister, and J. Popovi´ c. Face transfer with multilinear models. In SIGGRAPH, 2005. 5
[34] C. Zach, T. Pock, and H. Bischof. A duality based approach for realtime TV-L1 optical flow. In DAGM, 2007. 4 11111222227777779977