iccv iccv2013 iccv2013-209 iccv2013-209-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: David Ferstl, Christian Reinbacher, Rene Ranftl, Matthias Ruether, Horst Bischof
Abstract: In this work we present a novel method for the challenging problem of depth image upsampling. Modern depth cameras such as Kinect or Time of Flight cameras deliver dense, high quality depth measurements but are limited in their lateral resolution. To overcome this limitation we formulate a convex optimization problem using higher order regularization for depth image upsampling. In this optimization an anisotropic diffusion tensor, calculated from a high resolution intensity image, is used to guide the upsampling. We derive a numerical algorithm based on a primaldual formulation that is efficiently parallelized and runs at multiple frames per second. We show that this novel upsampling clearly outperforms state of the art approaches in terms of speed and accuracy on the widely used Middlebury 2007 datasets. Furthermore, we introduce novel datasets with highly accurate groundtruth, which, for the first time, enable to benchmark depth upsampling methods using real sensor data.
[1] K. Bredies, K. Kunisch, and T. Pock. Total generalized variation. SIAM Journal on Imaging Sciences, 3(3):492– 526, 2010.
[2] A. Chambolle and T. Pock. A first-order primal-dual algorithm for convex problems with applications to imaging. Journal of Mathematical Imaging and Vision, 40: 120 –145, 2011.
[3] D. Chan, H. Buisman, C. Theobalt, and S. Thrun. A NoiseAware Filter for Real-Time Depth Upsampling. In Proc. ECCV Workshops, 2008.
[4] Y. Cui, S. Schuon, D. Chan, S. Thrun, and C. Theobalt. 3d shape scanning with a time-of-flight camera. In Proc. CVPR, 2010.
[5] J. Diebel and S. Thrun. An application of markov random fields to range sensing. In Proc. NIPS, 2006.
[6] E. Esser, X. Zhang, and T. Chan. A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science. SIAM Journal on Imaging Sciences, 3(4): 1015 –1046, 2010. 999999 (a) Intensity + ToF(b) Groundtruth(c) OURS(d) Error Figure 6. Visual evaluation on the real datasets Books (first row), Shark (second row) and Devil (third row). In column (a) the low resolution ToF image and the high resolution intensity image are shown, whereas column (b) shows the high resolution groundtruth depth. The black areas are not correctly reconstructed due to occlusions in the stereo system and therefore set invalid for the RMSE calculation. In column (c) the upsampling result of our method is shown whereas in column (d) the relative depth error to the known groundtruth is shown.
[7] S. Fuchs and G. Hirzinger. Extrinsic and depth calibration of tof-cameras. In Proc. CVPR, 2008.
[8] S. A. Gudmundsson, H. Aanaes, and R. Larsen. Fusion of stereo vision and time-of-flight imaging for improved 3d estimation. International Journal of Intelligent Systems Technologies and Applications, 5(3/4):425 –433, 2008.
[9] K. He, J. Sun, and X. Tang. Guided image filtering. In Proc. ECCV, 2010.
[10] H. Hirschmuller and D. Scharstein. Evaluation of cost functions for stereo matching. In Proc. CVPR, 2007.
[11] J. Kopf, M. F. Cohen, D. Lischinski, and M. Uyttendaele. Joint bilateral upsampling. ACM Transactions on Graphics, 26(3), 2007.
[12] R. Lange. 3D Time-of-Flight distance measurement with custom solid-state image sensors in CMOS/CCD technology. PhD thesis, Department of Electrical Engineering and
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22]
[23] Computer Science at University of Siegen, 2000. J. Li, G. Zeng, R. Gan, H. Zha, and L. Wang. A bayesian approach to uncertainty-based depth map super resolution. In Proc. ACCV, 2012. R. A. Newcombe, S. Izadi, O. Hilliges, D. Molyneaux, D. Kim, A. J. Davison, P. Kohli, J. Shotton, S. Hodges, and A. Fitzgibbon. Kinectfusion: Real-time dense surface mapping and tracking. In Proc. ISMAR, 2011. J. Park, H. Kim, Y.-W. Tai, M. Brown, and I. Kweon. High quality depth map upsampling for 3d-tof cameras. In Proc. ICCV, 2011. PMD Technologies. Siegen, Germany. Camboard Nano. T. Pock and A. Chambolle. Diagonal preconditioning for first order primal-dual algorithms in convex optimization. In Proc. ICCV, 2011. R. Ranftl, S. Gehrig, T. Pock, and H. Bischof. Pushing the limits of stereo using variational stereo estimation. In IEEE Intelligent Vehicles Symposium, 2012. L. I. Rudin, S. Osher, and E. Fatemi. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena, 60(1-4):259 –268, 1992. D. Scharstein and C. Pal. Learning conditional random fields for stereo. In Proc. CVPR, 2007. M. Schmidt. Analysis, Modeling and Dynamic Optimization of 3D Time-of-Flight Imaging Systems. PhD thesis, Ruperto-Carola University of Heidelberg, Germany, 2011. S. Schuon, C. Theobalt, J. Davis, and S. Thrun. Lidarboost: Depth superresolution for tof 3d shape scanning. In Proc. CVPR, 2009. A. Torralba and W. Freeman. Properties and applications of shape recipes. In Proc. CVPR, 2003.
[24] Q. Yang, R. Yang, J. Davis, and D. Nister. Spatial-depth super resolution for range images. In Proc. CVPR, 2007.
[25] Z. Zhang. A flexible new technique for camera calibration. TPAMI, 22(1 1): 1330 –1334, 2000.
[26] J. Zhu, L. Wang, R. Yang, J. Davis, and Z. Pan. Reliability fusion of time-of-flight depth and stereo geometry for high quality depth maps. TPAMI, 33(7): 1400 –1414, 2011. 11000000