iccv iccv2013 iccv2013-317 iccv2013-317-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Christoph Vogel, Konrad Schindler, Stefan Roth
Abstract: Estimating dense 3D scene flow from stereo sequences remains a challenging task, despite much progress in both classical disparity and 2D optical flow estimation. To overcome the limitations of existing techniques, we introduce a novel model that represents the dynamic 3D scene by a collection of planar, rigidly moving, local segments. Scene flow estimation then amounts to jointly estimating the pixelto-segment assignment, and the 3D position, normal vector, and rigid motion parameters of a plane for each segment. The proposed energy combines an occlusion-sensitive data term with appropriate shape, motion, and segmentation regularizers. Optimization proceeds in two stages: Starting from an initial superpixelization, we estimate the shape and motion parameters of all segments by assigning a proposal from a set of moving planes. Then the pixel-to-segment assignment is updated, while holding the shape and motion parameters of the moving planes fixed. We demonstrate the benefits of our model on different real-world image sets, including the challenging KITTI benchmark. We achieve leading performance levels, exceeding competing 3D scene flow methods, and even yielding better 2D motion estimates than all tested dedicated optical flow techniques.
[1] G. Adiv. Determining three-dimensional motion and structure from optical flow generated by several moving objects. PAMI, 7(4):384–401, 1985.
[2] A. M. Ali, A. A. Farag, and G. L. Gimel’Farb. Optimizing binary MRFs with higher order cliques. ECCV 2008.
[3] T. Basha, Y. Moses, and N. Kiryati. Multi-view scene flow estimation: A view centered variational approach. CVPR ’10.
[4] M. Bleyer, C. Rother, P. Kohli, D. Scharstein, and S. N. Sinha. Object stereo Joint stereo matching and object segmentation. CVPR 2011.
[5] T. Brox, A. Bruhn, N. Papenberg, and J. Weickert. High accuracy optical flow estimation based on a theory for warping. ECCV 2004.
[6] R. L. Carceroni and K. N. Kutulakos. Multi-view scene capture by surfel sampling: From video streams to non-rigid 3D motion, shape and reflectance. IJCV, 49: 175–214, 2002.
[7] J. Cech, J. Sanchez-Riera, and R. P. Horaud. Scene flow estimation by growing correspondence seeds. CVPR 2011.
[8] A. Geiger, P. Lenz, and R. Urtasun. Are we ready for autonomous driving? CVPR 2012.
[9] H. Hirschm u¨ller. Stereo processing by semiglobal matching and mutual information. PAMI, 30(2):328–341, 2008.
[10] F. Huguet and F. Devernay. A variational method for scene flow estimation from stereo sequences. ICCV 2007.
[11] V. Lempitsky, S. Roth, and C. Rother. FusionFlow: Discretecontinuous optimization for optical flow estimation. CVPR 2008. –
[12] T. Nir, A. Bruckstein, and R. Kimmel. Over-parameterized variational optical flow. IJCV, 76(2):205–216, 2008.
[13] C. Rabe, T. M ¨uller, A. Wedel, and U. Franke. Dense, robust, and accurate motion field estimation from stereo image sequences in real-time. ECCV 2010.
[14] R. Ranftl, S. Gehrig, T. Pock, and H. Bischof. Pushing the limits of stereo using variational stereo estimation. IV 2012.
[15] C. Rother, V. Kolmogorov, V. Lempitsky, and M. Szummer. OptimizingbinaryMRFs viaextendedroofduality. CVPR ’07.
[16] D. Sun, E. B. Sudderth, and M. J. Black. Layered image motion with explicit occlusions, temporal consistency, and depth ordering. NIPS 2010.
[17] M. Unger, M. Werlberger, T. Pock, and H. Bischof. Joint motion estimation and segmentation of complex scenes with label costs and occlusion modeling. CVPR 2012.
[18] L. Valgaerts, A. Bruhn, H. Zimmer, J. Weickert, C. Stoll, and C. Theobalt. Joint estimation of motion, structure and geometry from stereo sequences. ECCV 2010.
[19] T. Vaudrey, C. Rabe, R. Klette, and J. Milburn. Differences between stereo and motion behaviour on synthetic and realworld stereo sequences. IVCNZ 2008.
[20] S. Vedula, S. Baker, R. Collins, T. Kanade, and P. Rander. Three-dimensional scene flow. CVPR 1999.
[21] O. Veksler, Y. Boykov, and P. Mehrani. Superpixels and supervoxels in an energy optimization framework. ECCV 2010.
[22] C. Vogel, K. Schindler, and S. Roth. 3D scene flow estimation with a rigid motion prior. ICCV 2011.
[23] J. Wang and E. Adelson. Representing moving images with layers. IEEE TIP, 3:625–638, 1994.
[24] A. Wedel, C. Rabe, T. Vaudrey, T. Brox, U. Franke, and D. Cremers. Efficient dense scene flow from sparse or dense
[25]
[26]
[27]
[28] stereo data. ECCV 2008. M. Werlberger. Convex Approaches for High Performance Video Processing. PhD thesis, TU Graz, 2012. K. Yamaguchi, T. Hazan, D. McAllester, and R. Urtasun. Continuous Markov random fields for robust stereo estimation. ECCV 2012. K. Yamaguchi, D. McAllester, and R. Urtasun. Robust monocular epipolar flow estimation. CVPR 2013. R. Zabih and J. Woodfill. Non-parametric local transforms for computing visual correspondence. ECCV 1994. 1384