iccv iccv2013 iccv2013-284 iccv2013-284-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Jaesik Park, Sudipta N. Sinha, Yasuyuki Matsushita, Yu-Wing Tai, In So Kweon
Abstract: We propose a method for accurate 3D shape reconstruction using uncalibrated multiview photometric stereo. A coarse mesh reconstructed using multiview stereo is first parameterized using a planar mesh parameterization technique. Subsequently, multiview photometric stereo is performed in the 2D parameter domain of the mesh, where all geometric and photometric cues from multiple images can be treated uniformly. Unlike traditional methods, there is no need for merging view-dependent surface normal maps. Our key contribution is a new photometric stereo based mesh refinement technique that can efficiently reconstruct meshes with extremely fine geometric details by directly estimating a displacement texture map in the 2D parameter domain. We demonstrate that intricate surface geometry can be reconstructed using several challenging datasets containing surfaces with specular reflections, multiple albedos and complex topologies.
[1] Y. Boykov and V. Kolmogorov. An experimental comparison of min-cut/max-flow algorithms for energy minimization in computer vision. IEEE Trans. Pattern Anal. Mach. Intell., 26(9): 1124–1 137, 2004.
[2] A. W. Fitzgibbon, G. Cross, and A. Zisserman. Automatic 3D model construction for turn-table sequences. In 3D Structure from Multiple Images of Large-Scale Environments, pages 155–170. 1998.
[3] U. I. Gupta, D. T. Lee, and J. Y.-T. Leung. Efficient algorithms for interval graphs and circular-arc graphs. Networks, 12(4):459–467, 1982.
[4] H. Hayakawa. Photometric stereo under a light source with arbitrary motion. Journal of the Optical Society of America, 11(11):3079– 3089, 1994.
[5] C. Hern a´ndez, G. Vogiatzis, and R. Cipolla. Probabilistic visibility for multi-view stereo. In Proc. of Computer Vision and Pattern Recognition (CVPR), 2007.
[6] C. Hern a´ndez, G. Vogiatzis, and R. Cipolla. Multi-view photometric stereo. IEEE Trans. Pattern Anal. Mach. Intell., 30(3):548–554, 2008.
[7] H. Hirschm u¨ller. Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. , 30(2):328– 341, 2008.
[8] H. Hoppe. New quadric metric for simplifying meshes with appearance attributes. In Proc. of IEEE Visualization Conference, 1999.
[9] B. K. P. Horn. Shape from shading: A method for obtaining the shape of a smooth opaque object from one view. PhD thesis, MIT, 1970.
[10] S. Ikehata, D. Wipf, Y. Matsushita, and K. Aizawa. Robust photometric stereo using sparse regression. In Proc. of Computer Vision and Pattern Recognition (CVPR), pages 318–325, 2012.
[11] H. P. A. Lensch, J. Kautz, M. Goesele, W. Heidrich, and H. peter Seidel. Image-based reconstruction of spatial appearance and geometric detail. ACM Trans. on Graph., 22:234–257, 2003.
[12] H. C. Longuet-Higgins. A computer algorithm for reconstructing a scene from two projections. Nature, 193: 133–135, 1981 .
[13] W. E. Lorensen and H. E. Cline. Marching cubes: A high resolution 3d surface construction algorithm. In Proc. of SIGGRAPH, pages 163–169, 1987.
[14] D. M. Mount and S. Arya. ANN: A library for approximate nearest neighbor searching. http://www.cs.umd.edu/˜mount/ANN/, 2010.
[15] D. Nehab, S. Rusinkiewicz, J. Davis, and R. Ramamoorthi. Efficiently combining positions and normals for precise 3D geometry. ACM Trans. on Graph., 24(3), 2005.
[16] T. Okatani and K. Deguchi. Optimal integration of photometric and geometric surface measurements using inaccurate reflectance/illumination knowledge. In Proc. of Computer Vision and Pattern Recognition (CVPR), pages 254–261, 2012.
[17] H. Rushmeier and F. Bernardini. Computing consistent normals and colors from photometric data. In 3.
[18] S. M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski. A comparison and evaluation of multi-view stereo reconstruction algorithms. In Proc. of Computer Vision and Pattern Recognition (CVPR), 2006.
[19] A. Sheffer, E. Praun, and K. Rose. Mesh parameterization methods and their applications. Found. Trends. Comput. Graph. Vis., 2(2): 105–171, 2006.
[20] N. Snavely, S. M. Seitz, and R. Szeliski. Photo tourism: Exploring photo collections in 3d. In Proc. of SIGGRAPH, pages 835–846. ACM Press, 2006.
[21] L. Szirmay-Kalos and T. Umenhoffer. Displacement mapping on the GPU - State of the Art. Computer Graphics Forum, 27(1), 2008.
[22] G. Taubin. A signal processing approach to fair surface design. In Proc. of SIGGRAPH, pages 351–358, 1995.
[23] D. Vlasic, P. Peers, I. Baran, P. Debevec, J. Popovi´ c, S. Rusinkiewicz, and W. Matusik. Dynamic shape capture using multi-view photometric stereo. ACM Trans. on Graph., 28(5), 2009.
[24] G. Vogiatzis, C. Hern a´ndez Esteban, P. H. S. Torr, and R. Cipolla. Multiview stereo via volumetric graph-cuts and occlusion robust photo-consistency. IEEE Trans. Pattern Anal. Mach. Intell., 29(12), Dec. 2007.
[25] R. J. Woodham. Photometric method for determining surface orientation from multiple images. Optical Engineering, 19(1), 1980.
[26] C. Wu, Y. Liu, Q. Dai, and B. Wilburn. Fusing multiview and photometric stereo for 3d reconstruction under uncalibrated illumination. IEEE Trans. on Visualization and Computer Graphics, 17(8): 1082– 1095, 2011.
[27] C. Wu, B. Wilburn, Y. Matsushita, and C. Theobalt. High-quality shape from multi-view stereo and shading under general illumination. In Proc. of Computer Vision and Pattern Recognition (CVPR), 2011.
[28] L. Zhang, B. Curless, A. Hertzmann, and S. M. Seitz. Shape and motion under varying illumination: Unifying structure from motion, photometric stereo, and multi-view stereo. In Proc. of Int’l Conf. on Computer Vision (ICCV), pages 618–625, 2003.
[29] Q. Zhang, M. Ye, R. Yang, Y. Matsushita, B. Wilburn, and H. Yu. Edge-preserving photometric stereo via depth fusion. In Proc. of Computer Vision and Pattern Recognition (CVPR), 2012.
[30] K. Zhou, J. Snyder, B. Guo, and H.-Y. Shum. Iso-charts: Stretchdriven mesh parameterization using spectral analysis. In Eurographics Symposium on Geometry Processing, pages 45–54, 2004. 11 116677 one of input images, the base mesh from MVS, and the refined mesh. The corresponding surface normal and displacement maps are shown in the supplementary material. Clear failure cases are highlighted by red rectangles; these occur at textureless dark regions. 11 116688