iccv iccv2013 iccv2013-444 iccv2013-444-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Tal Hassner
Abstract: We present a data-driven method for estimating the 3D shapes of faces viewed in single, unconstrained photos (aka “in-the-wild”). Our method was designed with an emphasis on robustness and efficiency with the explicit goal of deployment in real-world applications which reconstruct and display faces in 3D. Our key observation is that for many practical applications, warping the shape of a reference face to match the appearance of a query, is enough to produce realistic impressions of the query ’s 3D shape. Doing so, however, requires matching visual features between the (possibly very different) query and reference images, while ensuring that a plausible face shape is produced. To this end, we describe an optimization process which seeks to maximize the similarity of appearances and depths, jointly, to those of a reference model. We describe our system for monocular face shape reconstruction and present both qualitative and quantitative experiments, comparing our method against alternative systems, and demonstrating its capabilities. Finally, as a testament to its suitability for real-world applications, we offer an open, online implementation of our system, providing unique means – of instant 3D viewing of faces appearing in web photos.
[1] J. T. Barron and J. Malik. Shape, albedo, and illumination from a single image of an unknown object. In Proc. Conf. Comput. Vision Pattern Recognition, pages 334–341, 2012.
[2] D. Bertsekas. Nonlinear programming. Athena Scientific, 1999.
[3] V. Blanz, K. Scherbaum, T. Vetter, and H. Seidel. Exchanging faces in images. Comput. Graphics Forum, 23(3):669– 676, 2004.
[4] V. Blanz and T. Vetter. Morphable model for the synthesis of 3D faces. In Proc. ACM SIGGRAPH Conf. Comput. Graphics, pages 187–194, 1999.
[5] T. Brox, A. Bruhn, N. Papenberg, and J. Weickert. High accuracy optical flow estimation based on a theory for warping. European Conf. Comput. Vision, pages 25–36, 2004.
[6] A. Bruhn, J. Weickert, and C. Schn o¨rr. Lucas/kanade meets horn/schunck: Combining local and global optic flow methods. Int. J. Comput. Vision, 61(3):21 1–23 1, 2005.
[7] R. Dovgard and R. Basri. Statistical symmetric shape from shading for 3D structure recovery of faces. European Conf. Comput. Vision, pages 99–1 13, 2004.
[8] R. I. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, ISBN: 0521540518, second edition, 2004.
[9] T. Hassner and R. Basri. Example based 3D reconstruction from single 2D images. In Proc. Conf. Comput. Vision Pattern Recognition, 2006.
[10] T. Hassner and R. Basri. Single view depth estimation from examples. CoRR, abs/1304.3915, 2013.
[11] Y. Hu, D. Jiang, S. Yan, L. Zhangg, and H. zhang. Automatic 3D reconstruction for face recognition. In Int. Conf. on Automatic Face and Gesture Recognition, pages 843–848. IEEE, 2004.
[12] G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. University of Massachusetts, Amherst, TR 07-49, 2007.
[13] K. Karsch, C. Liu, and S. B. Kang. Depth extraction from video using non-parametric sampling. In European Conf. Comput. Vision, 2012.
[14] I. Kemelmacher-Shlizerman and R. Basri. 3D face reconstruction from a single image using a single reference face shape. Trans. Pattern Anal. Mach. Intell., 33(2), 2011.
[15] I. Kemelmacher-Shlizerman and S. Seitz. Face reconstruction in the wild. In Proc. Int. Conf. Comput. Vision, pages 1746–1753. IEEE, 2011.
[16] R. Knothe. A Global-to-local model for the representation of human faces. PhD thesis, University of Basel, 2009.
[17] C. Liu, J. Yuen, and A. Torralba. Sift flow: Dense correspondence across scenes and its applications. Trans. Pattern Anal. Mach. Intell., 33(5):978–994, 2011. Available: people . cs ail .mit .edu / ce l iu/ S IFT flow/ .
[18] C. Liu, J. Yuen, A. Torralba, J. Sivic, and W. Freeman. Sift flow: dense correspondence across different scenes. In European Conf. Comput. Vision, pages 28–42, 2008. Available: people . cs ail .mit .edu / ce l iu/ECCV2 0 0 8 / .
[19] D. Lowe. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision, 60(2):91–1 10, 2004.
[20] Z. Luo and P. Tseng. On the convergence of the coordinate descent method for convex differentiable minimization. J. of Optimization Theory and Applications, 72(1):7–35, 1992.
[21] A. Saxena, M. Sun, and A. Ng. Make3D: Learning 3D scene structure from a single still image. Trans. Pattern Anal. Mach. Intell., 3 1(5):824–840, 2009. Available: http : / /make 3 d . c s . co rne l . edu / . l
[22] Singular Inversions Inc. Facegen modeller manual. www . facegen . com, 2009.
[23] Y. Taigman and L. Wolf. Leveraging billions of faces to overcome performance barriers in unconstrained face recognition. arXiv preprint arXiv:1108.1122, 2011.
[24] H. Tang, Y. Hu, Y. Fu, M. Hasegawa-Johnson, and T. S. Huang. Real-time conversion from a single 2d face image to a 3D text-driven emotive audio-visual avatar. In Int. Conf. on Multimedia and Expo, pages 1205–1208. IEEE, 2008.
[25] USF. DARPA Human-ID 3D Face Database:. Courtesy of Prof. Sudeep Sarkar, University of South Florida, Tampa, FL.
[26] A. Vedaldi and B. Fulkerson. Vlfeat: An open and portable library of computer vision algorithms. In Proc. int. conf. on Multimedia, pages 1469–1472, 2010. Available: www . vlfeat .org/ .
[27] P. Viola and M. Jones. Robust real-time face detection. Int. J. Comput. Vision, 57(2): 137–154, 2004.
[28] Vizago Research GmbH. 3D face reconstruction. www . viz ago . ch.
[29] D. Vlasic, M. Brand, H. Pfister, and J. Popovi´ c. Face transfer with multilinear models. ACM Trans. on Graphics, 24(3):426–433, 2005.
[30] C. Wang, S. Yan, H. Li, H. Zhang, and M. Li. Automatic, effective, and efficient 3D face reconstruction from arbitrary view image. In Pacific Rim Conf. on Multimedia, pages 553– 560. Springer-Verlag, 2004. [3 1] F. Yang, J. Wang, E. Shechtman, L. Bourdev, and D. Metaxas. Expression flow for 3D-aware face component transfer. ACM Trans. on Graphics, 30(4):60, 2011.
[32] M. Zhou, L. Liang, J. Sun, and Y. Wang. AAM based face tracking with temporal matching and face segmentation. In Proc. Conf. Comput. Vision Pattern Recognition, pages 701– 708, 2010.
[33] X. Zhu and D. Ramanan. Face detection, pose estimation, and landmark localization in the wild. In Proc. Conf. Comput. Vision Pattern Recognition, pages 2879–2886, 2012. Available: www . i s .uci . edu / ˜ x zhu / face / . c 33660147